Automated tests - facts and myths


Published on

The discussion on automated tests is hot topic. The approach has same number of advocates and skeptics. More and more tools eases testing, but also introduces a fundamental question: what, when and how to test? Practise and experience let's answer those questions or guide in the right direction.

In this talk usage examples of unit, functional and behavioral tests will be shown. Importance of properly handling dependencies and mocking them will be discussed as well. But most of important part will be hints on how to write code, that could be tested automaticaly.

Slides are available in interactive mode here:

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Automated tests - facts and myths

  1. 1. Automated tests Facts and myths Wojciech Sznapka / @sznapka
  2. 2. Cześć
  3. 3. An introduction I work in software industry for about 9 years Care a lot about robust and testable architectures Loves software craftsmanship, sophisticated architectures, Big Data and ice hockey
  4. 4. Test Driven Development What is this?
  5. 5. Add a test
  6. 6. Write an implementation
  7. 7. Refactor code
  8. 8. Repeat!
  9. 9. What's important here?
  10. 10. ... to have tests
  11. 11. You won’t go to hell if you’ll write a test after declaring an interface or prototyping a class ...
  12. 12. ... but you’ll surely end up in hell, if there won’t be a test coverage
  13. 13. Myths about automated tests
  14. 14. Boss doesn't pay for automated tests
  15. 15. It's a myth! A 2005 study found that using TDD meant writing more tests and, in turn, programmers who wrote more tests tended to be more productive by American Scientists
  16. 16. But this ain't that hard ... casRmnovre ls oaCnetr { poetd$ovrin =[ rtce cnesos 10 = '' 90= 'M,50= '' 40= 'D,10= '' 00 > M, 0 > C' 0 > D, 0 > C' 0 > C, 9 = 'C,5 = '' 4 = 'L,1 = '' 0 > X' 0 > L, 0 > X' 0 > X, 9= 'X,5= '' 4= 'V,1= ''; > I' > V, > I' > I] pbi fnto cnet$nrbc ulc ucin ovr(iAai) { i (i_uei(iAai) { f !snmrc$nrbc) trwnwnaiAgmnEcpin' cnetnmrc'; ho e Ivldruetxeto(I ovr ueis) } i (iAai < 0 { f $nrbc = ) rtr '; eun ' } ls (aai,$oa)=$hs>ovrinatro(iAai) it $rbc rmn ti-cnesoFcoFr$nrbc; } rtr $oa .$hs>ovr(iAai -$rbc; eun rmn ti-cnet$nrbc aai) poetdfnto cnesoFcoFr$nrbc rtce ucin ovrinatro(iAai) { frah(ti-cnesosa $rbc= $oa){ oec $hs>ovrin s aai > rmn i (aai < $nrbc { f $rbc = iAai) rtr [aai,$oa] eun $rbc rmn; } } } }
  17. 17. ... start from obvious things casRmnovreTs etnsPPntFaeokTsCs ls oaCnetret xed HUi_rmwr_etae { pbi fnto tsEpy) ulc ucin etmt( { $hs>setqas',(e Rmnovre)>ovr(') ti-asrEul(' nw oaCnetr-cnet'); } }
  18. 18. ... provide some meat ... casRmnovreTs etnsPPntFaeokTsCs ls oaCnetret xed HUi_rmwr_etae { /*@aarvdrpoieetaa* * dtPoie rvdTsDt / pbi fnto tsCnesos$rbc $oa) ulc ucin etovrin(aai, rmn { $ovre =nwRmnovre(; cnetr e oaCnetr) $hs>setqas$oa,$ovre-cnet$rbc) ti-asrEul(rmn cnetr>ovr(aai); } } pbi fnto poieetaa) ulc ucin rvdTsDt( { rtr ary eun ra[ [47 'MCXVI] 39, MMDCI', [,'',[,'I] [,'I] [,'X] [0 'L] 1 I] 2 I', 6 V', 9 I', 4, X', [5 'L',[0 'C] [0,'',[0,'D] 4, XV] 9, X', 10 C] 40 C' ] ; }
  19. 19. ... test edge cases casRmnovreTs etnsPPntFaeokTsCs ls oaCnetret xed HUi_rmwr_etae { /*@xetdxeto naiAgmnEcpin* * epceEcpin Ivldruetxeto / pbi fnto tsEpy) ulc ucin etmt( { (e Rmnovre)>ovr(wfIa psighr') nw oaCnetr-cnet't m asn ee); } }
  20. 20. Help yourself with tests generation ppntsegn-ts Rmnovre hui-kle -et oaCnetr
  21. 21. ... or using Symfony's XSolveUnitSkelgenBundle .apcnoexov:kle:etAm/xmlBnl/evc/ /p/osl slesegnts ceEapeudeSrie. .apcnoexov:kle:etAm//otolr* /p/osl slesegnts ce*Cnrle/
  22. 22. We are building too complicated system for an automated tests
  23. 23. TDD forces you to write cleaner code
  24. 24. It's easier to test smaller units of code
  25. 25. so you write smaller classes which are less coupled and that makes your system more stable and open for an extension in the future
  26. 26. Your Object Oriented design becomes S.O.L.I.D. compilant and this is awesome!
  27. 27. You will of course tackle costly dependencies
  28. 28. Use PHPUnit's mock framework or Mockery
  29. 29. Fake it till you make it casMceTs etnsPPntFaeokTsCs ls okdet xed HUi_rmwr_etae { pbi fnto tsNnxsigauOjcs) ulc ucin etoEitnVlebet( { $ofgrto =okr:mc(ofgrtoVlebet,[ cniuain Mcey:ok'CniuainauOjc' 'eUl = 'tp/tdsnpap' gtr' > ht:/d.zak.l, 'eFra'= 'm') gtomt > xl]; $hs>setqas'm' $ofgrto-gtomt);/ O ti-asrEul(xl, cniuain>eFra() / K } }
  30. 30. Mock things that can't be tested quickly or nonreproducable casMceTs etnsPPntFaeokTsCs ls okdet xed HUi_rmwr_etae { pbi fnto tsAial( ulc ucin etpCls) { $uzok okr:mc(uzBosr) bzMc= Mcey:ok'Bzrwe'; $uzok>hudeev(gt) bzMc-solRcie'e' -adeun'rsos>mo>wsmt<mo>/epne'; >nRtr(<epne<odAeoiy/od<rsos>) / ti as i ws slto,t wiexlfxue i fl / hs lo s ie ouin o rt m itrs n ie / $uzok>hudeev(gt)>ne) / bzMc-solRcie'e'-oc( / / -adeunfl_e_otns_DR_.'fxue.m') >nRtr(iegtcnet(_I_ /itrsxl); } } $p =nwpCnue(bzMc) ai e Aiosmr$uzok; $p-asrEul(Aeoiy,$p-gturnMo() / O ai>setqas'wsmt' ai>eCretod); / K
  31. 31. Expect declared behaviors casMceTs etnsPPntFaeokTsCs ls okdet xed HUi_rmwr_etae { pbi fnto tsEpcainDcaain( ulc ucin etxettoselrtos) { $uzok okr:mc(uzBosr) bzMc= Mcey:ok'Bzrwe'; $uzok>hudeev(gt) bzMc-solRcie'e' -adeun'rsos>mo>wsmt<mo>/epne'; >nRtr(<epne<odAeoiy/od<rsos>) $ogrok=okr:mc(oooogr) lgeMc Mcey:ok'MnlgLge'; / w js wn t b sr ta Lge:if wscle ol oc / e ut at o e ue ht ogr:no a ald ny ne $ogrok>hudeev(if'-oc(; lgeMc-solRcie'no)>ne) } } $p =nwpCnue(bzMc,$ogrok; ai e Aiosmr$uzok lgeMc) $p-asrEul(Aeoiy,$p-gturnMo() / O ai>setqas'wsmt' ai>eCretod); / K
  32. 32. Be prepared for failures and check if you prepared for unexpected situations casMceTs etnsPPntFaeokTsCs ls okdet xed HUi_rmwr_etae { /*@xetdxeto yxetoWapr* * epceEcpin MEcpinrpe / pbi fnto tsFieCneto( ulc ucin etaldoncin) { $uzok okr:mc(uzBosr) bzMc= Mcey:ok'Bzrwe'; $uzok>hudeev(gt) bzMc-solRcie'e' -adho(uzEcpinCinEcpin) >nTrw'Bzxetoletxeto'; $ogrok=okr:mc(oooogr) lgeMc Mcey:ok'MnlgLge'; $ogrok>hudeev(if'-nvr) lgeMc-solRcie'no)>ee(; $ogrok>hudeev(er)>ne) lgeMc-solRcie'r'-oc(; } } (e pCnue(bzMc,$ogrok)>eCretod) nw Aiosmr$uzok lgeMc)-gturnMo(;
  33. 33. We providing API for external consumers it can't be tested...
  34. 34. Use Symfony's WebTestCase to test your API I call it integration tests
  35. 35. Call your API and check if it returns prepared data casEpniueotolretetnsWbetae ls xedtrCnrleTs xed eTsCs { ueIoaeTssri;/ i rst ts evrnet s sltdetTat / t ees et niomn pbi fnto tsGtitnsn) ulc ucin eteLsIJo( { $let=sai:cetCin(; cin ttc:raelet) $let>eus(GT,' cin-rqet'E' /xedtrsjo'; $sn=jo_eoe$let>eRsos(-gtotn() jo sndcd(cin-gtepne)>eCnet); } } $hs>setre$let>eRsos(-iScesu() ti-asrTu(cin-gtepne)>sucsfl); $hs>seton(0 $sn; ti-asrCut8, jo) $hs>setraeTaOEul ti-asrGetrhnrqa( nwaeie$sn7]>rae_t, e DtTm(jo[9-cetda) nwaeie$sn0-cetda); e DtTm(jo[]>rae_t)
  36. 36. Use fixtures and reset environment IsolatedTestsTrait should do the trick
  37. 37. Steps required to effectively run in isolation 1. 2. 3. 4. 5. 6. 7. configure PDO SQLite in file create database drop schema load fixtres copy database as a backup copy database from backup for every test delete database backup after test suite
  38. 38. Requirements changes frequently we can't keep up with unit tests
  39. 39. Use behavioral approach Behat in PHP
  40. 40. Describe features as scenarios It will be readable for: business, developers and machines
  41. 41. Perfectly fits into Agile process
  42. 42. Simple example Faue lo frajb etr: ok o o I odrt fn co jb n re o id ol o A a aprn porme s n siig rgamr Ine t b al t ls jbofr ed o e be o it o fes Seai:ls ofr frP vrin cnro it fes o L eso GvnIa o "" ie m n / Te Isol se"oązd tau hn hud e Dłc o em" Adcik"oązd tau n lc Dłc o em" Te Isol b o "krea hn hud e n /air" AdIsol se"H Sno Dvlpr(lwc) n hud e PP eir eeoe Giie" Seai:n ofr frE st cnro o fes o N ie GvnIa o "e" ie m n /n Te Isol ntse"oązd tau hn hud o e Dłc o em" AdIsol ntsea "ji-s eeet n hud o e n #onu" lmn
  43. 43. Our tests are slow!
  44. 44. This could be true ...
  45. 45. Always use in-memory sqlite database Or create clean sqlite database and copy it for every test
  46. 46. Group your tests Use PHPUnit's @group or Behat's @tag
  47. 47. Create "smoke tests" groups Those should be fast test, which ensures your system is most likely stable
  48. 48. Slower tests should run during night
  49. 49. Facts
  50. 50. It gives you confidence about changes and your code
  51. 51. Team is able to rapidly experiment with code
  52. 52. TDD enforces better Object Oriented design Smaller units of code and lower coupling always leads to better understanding of the codebase by future devs
  53. 53. End user experiences better quality lower 500 error ratio in the production
  54. 54. Happier users ↓ more $$$ in future
  55. 55. Conclusion
  56. 56. Setting up an working environment for automated tests is timely costly at the begining, but it pays off in the future
  57. 57. System without any kind of automated tests has big potential to be a big ball of mud
  58. 58. Good coverage can be easily achieved with mix of unit, functional and behavioral tests
  59. 59. You need to build and cultivate TDD culture in your surrounding
  60. 60. Thank you so much for attending! Feedback is much appreciated Twitter: @sznapka GitHub: @wowo
  61. 61. Join XSolve team :-)