Fragile Test

From CitconWiki
Revision as of 01:01, 28 July 2007 by 220.236.1.25 (talk)
Jump to navigationJump to search

Less Fragile Tests Tests Asserts Component Integration Functional UAT Non-Functional Perf Security Usability See ISO 9126 Good Tests Pass Repeatedly Timing (e.g. Selenium) External Dependencies External Systems Reliability Availability Data Set Values Incorrect State Heavy Mocking Utility to build mocks Easy switching mock to real Bad Test Hidden cause of failure Visible Replay using TimeSnapper Visible Replay Dump state data at failure Tool advice Time Snapper EasyMock Pretend Classes DBUnit gets prod snapshot regularly (security issues: scrubbing) Table dependancies makes it a significant task Data Domain (name, addr, etc class pulled from DB (abstracted data from test) Data Driven Approach Fitness Integration Testing Strong correlation between importance of tests depending on the types of testing and what kind of thing you are doing (Product type development = high focus on test vs bespoke = low focus Suggestion Use a virtual machine, setup - refresh test machine using VMWARe to simulate MSG based, 20 mins business transactions Test Data Repository Used keyword substitution to create the real data Typical Data Edge / Border cases, Exception testing Broken windows Continually failing, low priority tests/defects cause a 'Broken windows' effect which degrades the value of CI.