Mono Test Coverage

I'm in the process of collecting profile data from the various unit tests on Mono code.
That way I can pipe the profile data through monocov, and then spit out the coverage data and associate it with frequency information collected from the field.

The current coverage tool, Monocov, isn't written/distributed in a very Windows friendly manager. In addition, I have a feeling that Mono could use a good streamline process for their unit tests. It appears they used to run a webserver that was updated with test suite status about 2 years ago, but since have stopped.

Prototype Started...

Since I did all I could with my ASE submission, I decided to take my previous hacks/scripts, and make an initial stab at a prototype.

The protoype loads an assembly(s), and performs a scan to count references to system type method calls. This scan is from a professionally developed C# application. In this example, String.Concat is statically referenced 1148 times. This started me thinking what other information could you use with a method usage profile? Classification, bad call smells?

My next step will be to associate this information with the unit test coverage tool. Then, the prototype would have completed the full process. However, there are several problems with the basic frequency metrics and straightforward tool. (1) The calling context is not taken into account (which is related to dynamic usage is not static usage). (2) The nature of the coverage data and Mono code structure is not accounted for. (3) The presentation of information can use a more advanced task-oriented interface, report, and/or visualization options.

Status #2

I have been spending this week working on a conference paper deadline. I had did some work up front before Google Summer of Code started in anticipation of this paper deadline.

This coming week, I want to have a prototype that runs through the entire process of the tool. The prototype will start with a simple user interface that allows selection and parameterization of the deployed executable. Then it will run an method/type usage report and present the results.

Status

I have used a more robust way of identifying to which assembly a call belongs. Rather than just the namespace, the signed key for the assembly can be accessed through the Context property.

I imagine on the front end that one way a user would want to make a query on the types usage is something like: System.Text.* or System.*.

I have talked with Alex Orso who has pointed out some related work in residual coverage testing.

Furthermore, I have began setting up my testbed. I've been trying to gather various Mono applications in the wild. It is not as straight-forward as it sounds considering how diverse each project configuration is. So any donations for Mono executables would be accepted :)