Difference between revisions of "How to Start Testing A Large Legacy Application"

From CitconWiki
Jump to navigationJump to search
(tadronc4tal)
(rv to Revision as of 13:28, 22 August 2007)
 
Line 1: Line 1:
roleltcochib
 
letopasze
 
acelric
 
[http://new-mugen-chars.burblers.info/ New mugen chars] [http://ts-h107.burblers.info/ ts h107] [http://oem-dvd-slim.downslip.info/ oem dvd slim] [http://chop-sue.burblers.info/ chop sue] [http://mar.stroam.info/ mar] [http://megrez.unfeeble.info/ megrez] [http://relisys-17.misserve.info/ relisys 17] [http://labi.burblers.info/ labi] [http://venga-boys.downslip.info/ venga boys] [http://is-cause-i-m-cool.wrawler.info/ is cause i m cool] [http://nikkor-70-300.unfeeble.info/ nikkor 70 300] [http://indulger.info/index3_250.html url] [http://wrawler.info/index0_250.html map] [http://i-like-girls.indulger.info/ i like girls] [http://ebooks.downslip.info/ ebooks] [http://nickjr.batswing.info/ nickjr] [http://nba.burblers.info/ nba] [http://misserve.info/index6_666.html homepage] [http://trust-modem-v92.misserve.info/ trust modem v92] [http://wwwgooglecom.stroam.info/ wwwgooglecom] [http://epson-stylus-2100.misserve.info/ epson stylus 2100] [http://tag-45.wrawler.info/ tag 45] [http://indulger.info/index8_666.html page] [http://misserve.info/index6_666.html homepage] [http://seks.stroam.info/ seks] [http://charlemagne.downslip.info/ charlemagne] [http://yacht.batswing.info/ yacht] [http://garte.burblers.info/ garte] [http://toofast.indulger.info/ toofast] [http://burblers.info/index4_250.html index] [http://samsung-42-plasma-10000.burblers.info/ samsung 42 plasma 10000] [http://fiat-daily.misserve.info/ fiat daily] [http://driver-geforce2.stroam.info/ driver geforce2] [http://kv-fq-sony.unfeeble.info/ kv fq sony] [http://neon-nero.batswing.info/ neon nero] [http://volvo-s-40-1-6.indulger.info/ volvo s 40 1 6] [http://radeon-x800-xl.indulger.info/ radeon x800 xl] [http://sweet-symphony.unfeeble.info/ sweet symphony] [http://power-rangers-turbo.indulger.info/ power rangers turbo] [http://call-on-my.downslip.info/ call on my] [http://kaeon.indulger.info/ kaeon] [http://mr-snoor.batswing.info/ mr snoor] [http://rotorua.stroam.info/ rotorua] [http://mose.unfeeble.info/ mose] [http://fermo-qui.unfeeble.info/ fermo qui] [http://folle.stroam.info/ folle] [http://burblers.info/index5_250.html url] [http://www-mycricket-com.stroam.info/ www mycricket com] [http://modal-skirt.batswing.info/ modal skirt] [http://bdh4222v-00.wrawler.info/ bdh4222v 00] [http://burblers.info/index4_250.html index] [http://speed-dj.burblers.info/ speed dj] [http://mennea.indulger.info/ mennea] [http://frankenwald.unfeeble.info/ frankenwald] [http://unfeeble.info/index6_250.html site] [http://kasandra.batswing.info/ kasandra] [http://negarkhan.downslip.info/ negarkhan] [http://dave-weckl.misserve.info/ dave weckl] [http://sansui-au-717.downslip.info/ sansui au 717] [http://tvc-lcd-32.downslip.info/ tvc lcd 32] [http://burblers.info/index5_250.html url] [http://nuova-compagnia.misserve.info/ nuova compagnia] [http://unfeeble.info/index1_666.html site] [http://so-kiss-me-dawson-creek.burblers.info/ so kiss me dawson creek] [http://batswing.info/index9_666.html link] [http://penis-big.unfeeble.info/ Penis big] [http://anthoceros.unfeeble.info/ anthoceros] [http://nike-48-5.indulger.info/ nike 48 5] [http://ceplast.batswing.info/ ceplast] [http://downslip.info/index4_1000.html home] [http://batswing.info/ batswing.info] [http://www-cartoon-network-ro.indulger.info/ www cartoon network ro] [http://basketball-bets.misserve.info/ Basketball bets] [http://x850xt-pci.unfeeble.info/ x850xt pci] [http://kosova.batswing.info/ kosova] [http://misserve.info/index1_666.html site] [http://downslip.info/index0_666.html index] [http://basketball-bets.batswing.info/ Basketball bets] [http://notebook-toshiba-m70.burblers.info/ notebook toshiba m70] [http://maglia-juventus-2005-2006.downslip.info/ maglia juventus 2005 2006] [http://indulger.info/index7_666.html site] [http://gangb.misserve.info/ gangb] [http://amd-sempron-socket-a-2800.unfeeble.info/ amd sempron socket a 2800] [http://barbie-juegos.batswing.info/ barbie juegos] [http://batswing.info/index6_1000.html page] [http://belzebu.wrawler.info/ belzebu] [http://francine-dea.misserve.info/ francine dea] [http://vk-mobile.burblers.info/ vk mobile] [http://charles.burblers.info/ charles] [http://mature-lesbo.downslip.info/ mature lesbo] [http://netgear-gs724t.stroam.info/ netgear gs724t] [http://dama-inglese.misserve.info/ dama inglese] [http://downslip.info/ downslip.info] [http://www-peternorth-com.unfeeble.info/ www peternorth com] [http://indulger.info/index9_666.html site] [http://ivan-i-dalfin.batswing.info/ ivan i dalfin] [http://unfeeble.info/ unfeeble.info] [http://socket-a-amd.misserve.info/ socket a amd] [http://love-is-only-a-feeling.batswing.info/ love is only a feeling] [http://misserve.info/index1_666.html site]
 
[http://n-a-t-o.downslip.info/ n a t o] [http://batswing.info/index3_1000.html site] [http://indulger.info/index1_666.html homepage] [http://escort-a-verona.wrawler.info/ escort a verona] [http://www-rynair-com.indulger.info/ www rynair com] [http://spinn.batswing.info/ spinn] [http://batswing.info/index4_250.html site] [http://yemen.unfeeble.info/ yemen] [http://freedvd.burblers.info/ freedvd] [http://deivis.wrawler.info/ deivis] [http://wrawler.info/index5_1000.html home] [http://fidenco.downslip.info/ fidenco] [http://gwen-stefany.burblers.info/ gwen stefany] [http://wham-wake-me-up.indulger.info/ wham wake me up] [http://unpli.stroam.info/ unpli] [http://rs-mmc-mb-nokia-6680.unfeeble.info/ rs mmc mb nokia 6680] [http://lucky-luke-vol-1.indulger.info/ lucky luke vol 1] [http://www-newfaces.wrawler.info/ www newfaces] [http://wrawler.info/index2_250.html link] [http://sony-5-mp.batswing.info/ sony 5 mp] [http://b-b.batswing.info/ b b] [http://modem-adsl-mac.batswing.info/ modem adsl mac] [http://don-ki-kong.indulger.info/ don ki kong] [http://idaho-falls.stroam.info/ idaho falls] [http://flowers-power.wrawler.info/ flowers power] [http://tv-lcd-30-toshiba.stroam.info/ tv lcd 30 toshiba] [http://autocad-lt-2006-upgrade.wrawler.info/ autocad lt 2006 upgrade] [http://frou-frou.downslip.info/ frou frou] [http://big-wave.downslip.info/ big wave] [http://old-grannies.indulger.info/ old grannies] [http://conan-ps2.misserve.info/ conan ps2] [http://xcrypt-1-4.stroam.info/ xcrypt 1 4] [http://king-of-confidence.misserve.info/ king of confidence] [http://hp-19.batswing.info/ hp 19] [http://north-sails.wrawler.info/ north sails] [http://everybody-backstreetboys.burblers.info/ everybody backstreetboys] [http://batswing.info/index1_666.html url] [http://downslip.info/index7_250.html page] [http://misserve.info/index3_250.html index] [http://hp-5-pixel.stroam.info/ hp 5 pixel] [http://pctv-usb2.downslip.info/ pctv usb2] [http://skyblog-d-amoureux.batswing.info/ skyblog d amoureux] [http://mp3-dawson-s-creek.indulger.info/ mp3 dawson s creek] [http://carmelo-zappulla.wrawler.info/ carmelo zappulla] [http://principi-contabili-ias.indulger.info/ principi contabili ias] [http://lule.burblers.info/ lule] [http://mp3-1-gb.wrawler.info/ mp3 1 gb] [http://katy.batswing.info/ katy] [http://spy.batswing.info/ spy] [http://wrawler.info/index6_1000.html url] [http://real-big-hooters.burblers.info/ Real big hooters] [http://logo-telefono.unfeeble.info/ logo telefono] [http://roxio.downslip.info/ roxio] [http://acer-aspire-t310-computer.burblers.info/ acer aspire t310 computer] [http://www-stephans-dk.stroam.info/ www stephans dk] [http://krasnovodsk.wrawler.info/ krasnovodsk] [http://rubias-19.batswing.info/ rubias 19] [http://www-hid.wrawler.info/ www hid] [http://padre-ricco.wrawler.info/ padre ricco] [http://may-immortal.stroam.info/ may immortal] [http://trony-padova.batswing.info/ Trony padova] [http://empyre-hart.indulger.info/ empyre hart] [http://notebook-p4-630.burblers.info/ notebook p4 630] [http://microsoft-word-2003.downslip.info/ microsoft word 2003] [http://batswing.info/index0_666.html site] [http://tuta-moto.batswing.info/ Tuta moto] [http://indulger.info/index2_250.html home] [http://doctruyen.burblers.info/ doctruyen] [http://flavicon.downslip.info/ flavicon] [http://burblers.info/ burblers.info] [http://corfo.stroam.info/ corfo] [http://eva-green.downslip.info/ eva green] [http://unfeeble.info/index2_250.html link] [http://nhl---2006.burblers.info/ nhl  2006] [http://nave-snav.unfeeble.info/ Nave snav] [http://lmc-versus-u2.misserve.info/ lmc versus u2] [http://asus-a-9250.unfeeble.info/ asus a 9250] [http://mp3-registra.wrawler.info/ mp3 registra] [http://elemental-saga.unfeeble.info/ elemental saga] [http://mp3-sony-20-gb.indulger.info/ mp3 sony 20 gb] [http://grad.stroam.info/ grad] [http://lnb.burblers.info/ lnb] [http://wrawler.info/index2_250.html link] [http://indulger.info/index9_666.html site] [http://navi-plus.batswing.info/ navi plus] [http://pocket-tank.stroam.info/ pocket tank] [http://viaccess-key-remote.wrawler.info/ viaccess key remote] [http://wi-fi-adsl.wrawler.info/ wi fi adsl] [http://steimberg-cubase-vst.wrawler.info/ steimberg cubase vst] [http://outlaws.batswing.info/ outlaws] [http://fergie-boobs.misserve.info/ fergie boobs] [http://dual-wan-router.wrawler.info/ dual wan router] [http://smu-bugil.indulger.info/ smu bugil] [http://ouagadougou.misserve.info/ ouagadougou] [http://nec-ht410.wrawler.info/ nec ht410] [http://tv-dvd-vhs-amstrad.batswing.info/ tv dvd vhs amstrad] [http://donwload.wrawler.info/ donwload] [http://j-fox.stroam.info/ j fox] [http://puma-felpe.downslip.info/ puma felpe] [http://misserve.info/index7_666.html homepage]
 
 
Facilitated by Paul O'Keeffe.
 
Facilitated by Paul O'Keeffe.
  
Line 41: Line 36:
 
* Static methods -> instance methods.
 
* Static methods -> instance methods.
  
Initially this layer was hand coded and was itself excluded from the code being measured for test coverage. However, inconsistencies and untested logic began to creep into this layer. To solve this, the wrappers were instead generated at runtime using dynamic proxies implementing hand coded interfaces for the desired APIs. This later evolved into the [http://code.goog
+
Initially this layer was hand coded and was itself excluded from the code being measured for test coverage. However, inconsistencies and untested logic began to creep into this layer. To solve this, the wrappers were instead generated at runtime using dynamic proxies implementing hand coded interfaces for the desired APIs. This later evolved into the [http://code.google.com/p/proxymatic/wiki/AutoBoundary AutoBoundary] module of the [http://code.google.com/p/proxymatic/ Proxymatic] open source project.
1000
 
le.com/p/proxymatic/wiki/AutoBoundary AutoBoundary] module of the [http://code.google.com/p/proxymatic/ Proxymatic] open source project.
 
  
 
With such a layer, it is possible to replace/mock all third party code for the purposes of testing, making it easy to reproduce all behaviours, including hard to test exception conditions, thus making 100% coverage for the remaining production code possible.
 
With such a layer, it is possible to replace/mock all third party code for the purposes of testing, making it easy to reproduce all behaviours, including hard to test exception conditions, thus making 100% coverage for the remaining production code possible.

Latest revision as of 09:05, 2 April 2008

Facilitated by Paul O'Keeffe.

What Happened

This session morphed into a discussion right up the other end of the spectrum, about achieving 100% test coverage, what that means exactly and whether attempting such a thing is even wise.

A Tighter Definition of Code Coverage Numbers

What does 100% test coverage actually mean?

  • Type of coverage - line, branch or path.
  • Tests being run - unit, integration, functional or some combination of these.
  • Code being covered - all production code, perhaps with some acceptable exclusions.

Shooting for 100%

We discussed one greenfields Java project, which achieved 100% branch coverage of all production code except for a thin wrapping layer around external third party libraries, running only fairly tight unit tests. Integration and functional tests were not used for coverage measurements. This was done in an attempt to enforce test driving of production code, since it would be nearly impossible to achieve this result without having done so. It succeeded in this respect, but at the cost of having a reasonably large amount of fairly brittle test code to maintain, due to the fairly tight coupling between the tests and implementation details within the production classes.

Hurdles

Achieving this level of coverage was made more difficult when the code needed to call through to external library code which was not designed for testability. This includes the vast majority of all third party libraries and the JDK in particular! Language constructs which make achieving full test coverage more difficult include:

  • Referencing concrete classes instead of interfaces.
  • Direct use of constructors, rather than factories.
  • Final classes.
  • Static methods.

Generally, use of any construct which makes it harder to replace real dependencies with test versions makes testing tricky.

Jumping the Hurdles

The project discussed wrapped all third party APIs using untestable constructs in a thin proxy layer which automatically translated them as follows:

  • Concrete/final classes -> interfaces.
  • Constructors -> choice of factory classes or methods.
  • Static methods -> instance methods.

Initially this layer was hand coded and was itself excluded from the code being measured for test coverage. However, inconsistencies and untested logic began to creep into this layer. To solve this, the wrappers were instead generated at runtime using dynamic proxies implementing hand coded interfaces for the desired APIs. This later evolved into the AutoBoundary module of the Proxymatic open source project.

With such a layer, it is possible to replace/mock all third party code for the purposes of testing, making it easy to reproduce all behaviours, including hard to test exception conditions, thus making 100% coverage for the remaining production code possible.

How To Apply This To Legacy Application Testing

With all this in mind, the discussion returned to the question of testing legacy applications. Surprisingly, it turned out that the 100% coverage approach/tools could be applied in legacy situations, if you think of the existing code as third party code. We figured you could start by test driving all new code and wrapping all existing code in the proxy layer, and then gradually move old code over to the new approach piece by piece. Untangling concrete/static/final dependencies would be aided by interceding the wrapper layer at the appropriate points.