Fragile Test

From CitconWiki
Jump to navigationJump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Less Fragile Tests

Tests

  • Asserts
    • Component
    • Integration
  • Functional
    • UAT
  • Non-Functional
    • Perf
    • Security
    • Usability
  • See ISO 9126

Good Tests

  • Pass Repeatedly
    • Timing (e.g. Selenium)
    • External Dependencies
      • External Systems
        • Reliability
        • Availability
      • Data
        • Set Values
        • Incorrect State
    • Heavy Mocking
      • Utility to build mocks
      • Easy switching mock to real
        • use same data, settings

Bad Tests

  • Hidden cause of failure
    • use screen capture logger, e.g. TimeSnapper
  • Visible Replay
  • Dump state data at failure

Tool advice

  • Time Snapper
  • EasyMock
  • Pretend Classes
  • DBUnit gets prod snapshot regularly (security issues: scrubbing) Table dependancies makes it a significant task

Data

  • Domain (name, addr, etc class pulled from DB
  • (abstracted data from test)
  • Data Driven Approach
  • Fitness Integration Testing

Strong correlation between importance of tests depending on the types of testing and what kind of thing you are doing (Product type development = high focus on test vs bespoke = low focus


Suggestion

  • Use a virtual machine, setup - refresh test machine using VMWARe to simulate MSG based, 20 mins business transactions

Test Data Repository

  • Used keyword substitution to create the real data

Typical Data

  • Edge / Border cases, Exception testing

Broken windows

  • Continually failing, low priority tests/defects cause a 'Broken windows' effect which degrades the value of CI.