MutationTesting

From CitconWiki
Jump to navigationJump to search

Notes to follow


Mutation testing tools: Jester & Jumble

Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Marteijn, Mark, David, Ivan, Kirsten

Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. Pb: it is hard to integrate it with build scripts, and with multiple tests. Promise of a new release! It is too slow to run on an entire project.

Jumble: similar, but changes bytecode, not source.

  • Pb: difficult to use, not really maintained anymore. Also, has a tendency to change the package names in literal.

some metrics:

  • 5 seconds per compilation
  • 20 mutations per class, each requiring compilation
  • 1200 classes
  • 120000 seconds to run = 33.3 hours

Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage.

Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant.

Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code.

David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.

muJava (uJava? cannot get the mu Greek letter on my keyboard): a nasty competitor to Jester! does mutation of classes

Iven: there is probably a tool integrated with Maven. Must find it again. Must also look at Hansel (& Gretel)

Links to tools