Difference between revisions of "MutationTesting"

From CitconWiki
Jump to navigationJump to search
m (Typo in link)
 
(3 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
Mutation testing tools: Jester, Jumble & Test Police
 
Mutation testing tools: Jester, Jumble & Test Police
  
Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Marteijn, Mark, David, Ivan, Kirsten
+
Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Martijn, Mark, David, Ivan, Kirsten
  
 
Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing.
 
Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing.
Line 24: Line 24:
 
David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.
 
David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.
  
muJava (uJava? cannot get the mu Greek letter on my keyboard): a nasty competitor to Jester! does mutation of classes
+
μJava: a nasty competitor to Jester! does mutation of classes
  
 
Iven: there is probably a tool integrated with Maven. Must find it again.
 
Iven: there is probably a tool integrated with Maven. Must find it again.
 
Must also look at Hansel (& Gretel)
 
Must also look at Hansel (& Gretel)
 +
 +
Tuomas: I'd like to look into infinitest, and see if that could be used to show which lines are exercised by which tests: http://code.google.com/p/infinitest/
 +
also, I'm interested in seeing how mutation testing can properly identify which abstract superclasses should be mutated.  (I wrote Test Police)
 +
  
 
== Links to tools ==
 
== Links to tools ==
Line 33: Line 37:
 
* Jester: [http://jester.sourceforge.net/]
 
* Jester: [http://jester.sourceforge.net/]
 
* Test Police: [http://www.laughingpanda.org/mediawiki/index.php/Test_Police]
 
* Test Police: [http://www.laughingpanda.org/mediawiki/index.php/Test_Police]
* MuJava: [http://ise.gmu.edu/~ofut/mujava/]
+
* MuJava: [http://cs.gmu.edu/~offutt/mujava/]
 
* More links on [http://ivan.truemesh.com/archives/000720.html]
 
* More links on [http://ivan.truemesh.com/archives/000720.html]

Latest revision as of 00:12, 25 October 2010

Mutation testing tools: Jester, Jumble & Test Police

Squirrel, Wolf, Eric, Ulrich, Erik, Mieke, Robert, Martijn, Mark, David, Ivan, Kirsten

Jester is a mutation test tool that change the code. If the tests still pass, then it means that tests are missing. Problems: it is hard to integrate it with build scripts, and with multiple tests. Promise of a new release! It is too slow to run on an entire project.

Jumble: similar, but changes bytecode, not source.

  • Problems: difficult to use, not clear how well maintained. Also, Class.getPackage() doesn't return the package name you expect.

some metrics:

  • 5 seconds per compilation
  • 20 mutations per class, each requiring compilation
  • 1200 classes
  • 120000 seconds to run = 33.3 hours

Ajester: similar to Jester, but with bytecode. The one interesting thing is that it is running on the classes that already have a good code coverage.

Martijn: it'd be nice if you could detect tests that always find out the same changes in the code, as tey would be redundant.

Ivan: an "MD5 test" is when a test is almost a rehash of the code, and it makes it harder to change the code.

David: it'd be nice if we could combine Jumble and Emma, and get reports that tell which lines are exercised by which tests.

μJava: a nasty competitor to Jester! does mutation of classes

Iven: there is probably a tool integrated with Maven. Must find it again. Must also look at Hansel (& Gretel)

Tuomas: I'd like to look into infinitest, and see if that could be used to show which lines are exercised by which tests: http://code.google.com/p/infinitest/ also, I'm interested in seeing how mutation testing can properly identify which abstract superclasses should be mutated. (I wrote Test Police)


Links to tools