The Tarantula team took the initiative, downloaded our open-source platform, and started running their own tests.  They are coming in next week (I think) to share their findings and get our feedback.  Kudos to the Tarantula team for thinking out of the box, and finding a way to get their product out there in the hands of users. There is most certainly a lesson there for other entrepreneurs – never take no for an answer – be creative – adapt, improvise, and overcome.  Be a real partner!

I am excited about the meeting, and I’ll post a followup here with the results of their findings.

If you aren’t familiar with Tarantula, here is a quick rundown:

Tarantula is a technique and tool for helping developers find bugs in source code.  The tool takes source coverage information from a set of test cases and produces a color-coded visual representation of the code. The color coding identifies areas of the code which are most likely to be buggy so that the developer may focus his/her attention in suspicious areas first.

For each statement in the source code, a suspiciousness and confidence value is computed. The suspiciousness is based on the relative numbers of passing and failing test cases that execute a statement. The confidence is based on the percentage of the total passing and failing test case sets that execute a statement. In the visualization, suspiciousness is represented by hue ranging from red to green; more suspicious statements appear more red and less suspicious statements appear more green. Confidence is represented by the brightness of the color; less confident statements appear dimmer and more confident statements appear brighter.

How does it work?

The Tarantula tool requires the source code, a set of test cases and per-test coverage information. While it has an independent data format that will support other tools in the future, the current implementation requires test cases written in JUnit and the use of Clover for coverage instrumentation. To use Tarantula, one instruments the source code with Clover and then runs a set of JUnit test cases. Assuming there are failing test cases to address, the data is imported from Clover reports into Tarantula. Then, the project is opened in Tarantula and the code is reviewed using the information it provides. The import functionality is available both as a wizard and a custom Ant task.

Cheers.