Fragile Test
From CitconWiki
Jump to navigationJump to searchThe printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
Less Fragile Tests
Tests
- Asserts
- Component
- Integration
- Functional
- UAT
- Non-Functional
- Perf
- Security
- Usability
- See ISO 9126
Good Tests
- Pass Repeatedly
- Timing (e.g. Selenium)
- External Dependencies
- External Systems
- Reliability
- Availability
- Data
- Set Values
- Incorrect State
- External Systems
- Heavy Mocking
- Utility to build mocks
- Easy switching mock to real
- use same data, settings
Bad Tests
- Hidden cause of failure
- use screen capture logger, e.g. TimeSnapper
- Visible Replay
- Dump state data at failure
Tool advice
- Time Snapper
- EasyMock
- Pretend Classes
- DBUnit gets prod snapshot regularly (security issues: scrubbing) Table dependancies makes it a significant task
Data
- Domain (name, addr, etc class pulled from DB
- (abstracted data from test)
- Data Driven Approach
- Fitness Integration Testing
Strong correlation between importance of tests depending on the types of testing and what kind of thing you are doing (Product type development = high focus on test vs bespoke = low focus
Suggestion
- Use a virtual machine, setup - refresh test machine using VMWARe to simulate MSG based, 20 mins business transactions
Test Data Repository
- Used keyword substitution to create the real data
Typical Data
- Edge / Border cases, Exception testing
Broken windows
- Continually failing, low priority tests/defects cause a 'Broken windows' effect which degrades the value of CI.