Difference between revisions of "MutationTesting"
(Add links to tools - squirrel) |
|||
| Line 1: | Line 1: | ||
Notes to follow | Notes to follow | ||
| + | |||
| + | Mutation testing tools: Jester & Jumble | ||
| + | |||
| + | Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Marteijn, Mark, David, Ivan, Kirsten | ||
| + | |||
| + | Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. | ||
| + | Pb: it is hard to integrate it with build scripts, and with multiple tests. | ||
| + | Promise of a new release! | ||
| + | It is too slow to run on an entire project. | ||
| + | |||
| + | Jumble: similar, but changes bytecode, not source. | ||
| + | * Pb: difficult to use, not really maintained anymore. Also, has a tendency to change the package names in literal. | ||
| + | some metrics: | ||
| + | * 5 seconds per compilation | ||
| + | * 20 mutations per class, each requiring compilation | ||
| + | * 1200 classes | ||
| + | * 120000 seconds to run = 33.3 hours | ||
| + | |||
| + | Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage. | ||
| + | |||
| + | Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant. | ||
| + | |||
| + | Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code. | ||
| + | |||
| + | David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests. | ||
| + | |||
| + | muJava (uJava? cannot get the mu Greek letter on my keyboard): a nasty competitor to Jester! does mutation of classes | ||
| + | |||
| + | Iven: there is probably a tool integrated with Maven. Must find it again. | ||
| + | Must also look at Hansel (& Gretel) | ||
== Links to tools == | == Links to tools == | ||
| − | Jumble: [http://jumble.sourceforge.net/] | + | * Jumble: [http://jumble.sourceforge.net/] |
| − | Jester: [http://jester.sourceforge.net/] | + | * Jester: [http://jester.sourceforge.net/] |
| − | MuJava: [http://ise.gmu.edu/~ofut/mujava/] | + | * MuJava: [http://ise.gmu.edu/~ofut/mujava/] |
Revision as of 12:20, 22 October 2007
Notes to follow
Mutation testing tools: Jester & Jumble
Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Marteijn, Mark, David, Ivan, Kirsten
Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. Pb: it is hard to integrate it with build scripts, and with multiple tests. Promise of a new release! It is too slow to run on an entire project.
Jumble: similar, but changes bytecode, not source.
- Pb: difficult to use, not really maintained anymore. Also, has a tendency to change the package names in literal.
some metrics:
- 5 seconds per compilation
- 20 mutations per class, each requiring compilation
- 1200 classes
- 120000 seconds to run = 33.3 hours
Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage.
Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant.
Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code.
David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.
muJava (uJava? cannot get the mu Greek letter on my keyboard): a nasty competitor to Jester! does mutation of classes
Iven: there is probably a tool integrated with Maven. Must find it again. Must also look at Hansel (& Gretel)
