Difference between revisions of "Does high Code Coverage mean high test quality?"
From CitconWiki
Jump to navigationJump to search| Line 1: | Line 1: | ||
| − | It's possible to write a test which achieves extremely high code coverage for a system yet | + | It's possible to write a test which achieves extremely high code coverage for a system yet doesn't test the system at all. |
We looked at ways of preventing this, and what other measures one should use to ensure the effectiveness of your test suite? | We looked at ways of preventing this, and what other measures one should use to ensure the effectiveness of your test suite? | ||
| Line 5: | Line 5: | ||
* Code Review should be used to ensure that tests, actually test and are not just propping up the coverage metrics | * Code Review should be used to ensure that tests, actually test and are not just propping up the coverage metrics | ||
* Mutation testing, using a tool such as Jester, will also uncover tests which are "weak" | * Mutation testing, using a tool such as Jester, will also uncover tests which are "weak" | ||
| − | * [http://atlassian.com/clover Clover2 ] | + | * [http://atlassian.com/clover Clover2 ] exposes the list of classes each test covered ("per-test coverage") which can uncover power tests that possibly cover a lot of code yet have little value since they are so ufocused |
| + | * Discussion of whether the number of assert statements a test has could be an effective measure of test quality? | ||
| − | [[ | + | ---- |
| + | References: | ||
| + | * [http://jester.sourceforge.net/ Jester] | ||
| + | * [http://www.atlassian.com/software/crucible/ Crucible, Code Review] | ||
| + | * [http://atlassian.com/clover Clover2] | ||
Revision as of 02:52, 29 June 2008
It's possible to write a test which achieves extremely high code coverage for a system yet doesn't test the system at all.
We looked at ways of preventing this, and what other measures one should use to ensure the effectiveness of your test suite?
- Code Review should be used to ensure that tests, actually test and are not just propping up the coverage metrics
- Mutation testing, using a tool such as Jester, will also uncover tests which are "weak"
- Clover2 exposes the list of classes each test covered ("per-test coverage") which can uncover power tests that possibly cover a lot of code yet have little value since they are so ufocused
- Discussion of whether the number of assert statements a test has could be an effective measure of test quality?
References:
* Jester * Crucible, Code Review * Clover2
