Difference between revisions of "MutationTesting"
m (Typo in link) |
|||
| (6 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | Mutation testing tools: Jester, Jumble & Test Police | |
| − | + | Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Martijn, Mark, David, Ivan, Kirsten | |
| − | |||
| − | |||
| − | Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, | ||
Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. | Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. | ||
| − | + | Problems: it is hard to integrate it with build scripts, and with multiple tests. | |
Promise of a new release! | Promise of a new release! | ||
It is too slow to run on an entire project. | It is too slow to run on an entire project. | ||
Jumble: similar, but changes bytecode, not source. | Jumble: similar, but changes bytecode, not source. | ||
| − | * | + | * Problems: difficult to use, not clear how well maintained. Also, Class.getPackage() doesn't return the package name you expect. |
some metrics: | some metrics: | ||
* 5 seconds per compilation | * 5 seconds per compilation | ||
| Line 27: | Line 24: | ||
David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests. | David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests. | ||
| − | + | μJava: a nasty competitor to Jester! does mutation of classes | |
Iven: there is probably a tool integrated with Maven. Must find it again. | Iven: there is probably a tool integrated with Maven. Must find it again. | ||
Must also look at Hansel (& Gretel) | Must also look at Hansel (& Gretel) | ||
| + | |||
| + | Tuomas: I'd like to look into infinitest, and see if that could be used to show which lines are exercised by which tests: http://code.google.com/p/infinitest/ | ||
| + | also, I'm interested in seeing how mutation testing can properly identify which abstract superclasses should be mutated. (I wrote Test Police) | ||
| + | |||
== Links to tools == | == Links to tools == | ||
* Jumble: [http://jumble.sourceforge.net/] | * Jumble: [http://jumble.sourceforge.net/] | ||
* Jester: [http://jester.sourceforge.net/] | * Jester: [http://jester.sourceforge.net/] | ||
| − | * MuJava: [http:// | + | * Test Police: [http://www.laughingpanda.org/mediawiki/index.php/Test_Police] |
| + | * MuJava: [http://cs.gmu.edu/~offutt/mujava/] | ||
| + | * More links on [http://ivan.truemesh.com/archives/000720.html] | ||
Latest revision as of 00:12, 25 October 2010
Mutation testing tools: Jester, Jumble & Test Police
Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Martijn, Mark, David, Ivan, Kirsten
Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. Problems: it is hard to integrate it with build scripts, and with multiple tests. Promise of a new release! It is too slow to run on an entire project.
Jumble: similar, but changes bytecode, not source.
- Problems: difficult to use, not clear how well maintained. Also, Class.getPackage() doesn't return the package name you expect.
some metrics:
- 5 seconds per compilation
- 20 mutations per class, each requiring compilation
- 1200 classes
- 120000 seconds to run = 33.3 hours
Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage.
Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant.
Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code.
David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.
μJava: a nasty competitor to Jester! does mutation of classes
Iven: there is probably a tool integrated with Maven. Must find it again. Must also look at Hansel (& Gretel)
Tuomas: I'd like to look into infinitest, and see if that could be used to show which lines are exercised by which tests: http://code.google.com/p/infinitest/ also, I'm interested in seeing how mutation testing can properly identify which abstract superclasses should be mutated. (I wrote Test Police)
