View source for Functional tests take a long time

Long Running Functional Tests

These are the few notes I took from the "long running functional tests" discussions.

'''Problems'''

* Complete features take 1 day
* Functional test takes 15 hours

'''Alternatives, Risks, and Trade-offs'''

* Parallel testing vs. pipelined testing
** Compile
** Fast unit tests
** Slow unit tests
** Functional tests

* Incremental feedback during test runs
** Show failures sooner, but
** Does not typically lead to stopping the tests because we want to know all the results from that set of code


Alternate set of incomplete notes:

What do I do if my functional tests take 5 hours?

4 hour cycle vs. 5 minutes

(ci-machismo:  "mine is smaller than yours")

some tests we run all the time and some that we don't

order of magnitude & build pipeline tradeoff:
	5 minutes vs. 50
	20 minutes vs. 30

long running test suite but reporting of status after each suite  (separate emails?)

re-order decomposable steps

what tools are people using to do their functional tests?

new process: added qa, added release management.  developers aren't responsive to breaks in the QA builds. often there is disagreement about what the customer really wants, what the problems are. mini-waterfall labeled as Agile. requirements sprint, design sprint, coding sprint, testing sprint.