What Mistakes Can You Make in Designing Your Automated Tests
We subtitled this session "Mistaeks I Hav Made" in Nat Pryce's honour. Facilitated by Paul O'Keeffe.
The Laundry List
Here's the list of gotchas we discussed:
- Too much mocking.
- Unit tests too tight.
- Not refactoring test code as much as production code!
- Copying and pasting test code.
- Commenting out broken tests.
- Testing at the wrong level.
- Too many tests - take too long to run.
- No cleanup after tests.
- Not starting with clean test data setup.
- Tests that don't respect the domain.
- Only having big integration tests.
- Not having team ownership of tests.
- No test independence.
- Tests that work by coincidence.
- Time/timing dependent tests.
- Not starting with a broken test.
- Testing things that don't matter (for example, minor details of HTML).
- Not testing exception cases.
- Tests that spit out noise.
- Tests that are just plain wrong.
- Tests that don't check anything.
- Not using data driven tests.