Testing an Erlang Backend

1. IntroductionWhy should I test?Types of testsGood Practices Testing an Erlang BackendTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance Tests Enrique PazMaking ithappen Senior Backend Developer @ Team ServicesTDD & BDDAutomatingeverythingQ&A EUG-NL / 12-07-2012 1/40

2. IntroductionWhy should I test?Types of tests IntroductionGood PracticesTestingTechniquesUnit tests Testing TechniquesIntegration testsAcceptance TestsPerformance TestsMaking it Making it happenhappenTDD & BDDAutomatingeverythingQ&A Q&A 2/40

3. Reasons for testingIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • It increases your conﬁdence in the code you writeIntegration testsAcceptance TestsPerformance Tests • Tests themselves are useful documentationMaking ithappen • It makes refactoring really an optionTDD & BDDAutomating • It helps ﬁnding bugs earlier, so the impact is much lowereverythingQ&A 3/40

4. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”TestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 4/40

5. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 4/40

6. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 4/40

7. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 4/40

8. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance Tests Was your original code working or you spent the time ﬁxing it?Making ithappenTDD & BDDAutomatingeverythingQ&A 4/40

9. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance Tests Was your original code working or you spent the time ﬁxing it?Making ithappen Have you consider refactoring?TDD & BDDAutomatingeverythingQ&A 4/40

10. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance Tests Was your original code working or you spent the time ﬁxing it?Making ithappen Have you consider refactoring?TDD & BDDAutomating • “Setting up the scenario for testing my code is too complex”everythingQ&A 4/40

11. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance Tests Was your original code working or you spent the time ﬁxing it?Making ithappen Have you consider refactoring?TDD & BDDAutomating • “Setting up the scenario for testing my code is too complex”everythingQ&A So. . . you do it manually everytime? 4/40

12. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance Tests Was your original code working or you spent the time ﬁxing it?Making ithappen Have you consider refactoring?TDD & BDDAutomating • “Setting up the scenario for testing my code is too complex”everythingQ&A So. . . you do it manually everytime? Automating that scenario for testing will be expensive. . . but only once! 4/40

13. Yeah, but...IntroductionWhy should I test?Types of testsGood Practices • “My code is so clear/simple it doesn’t need tests”Testing simplicity and clarity are not universal wordsTechniquesUnit tests The simpler your code is, the easier for you to test itIntegration testsAcceptance Tests • “It takes me much longer testing the code than writing it”Performance Tests Was your original code working or you spent the time ﬁxing it?Making ithappen Have you consider refactoring?TDD & BDDAutomating • “Setting up the scenario for testing my code is too complex”everythingQ&A So. . . you do it manually everytime? Automating that scenario for testing will be expensive. . . but only once! • In short, stop arguing and start testing! 4/40

14. Types of testsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • Unit tests What you have written is exactly what you were trying to writeIntegration testsAcceptance Tests • Integration tests The API of your component can interact with other APIsPerformance TestsMaking it • Acceptance tests The API you have implemented ﬁts the bussinesshappenTDD & BDD requirementsAutomatingeverything • Performance tests Your solution is suitable for the expected scenarioQ&A 5/40

15. General Good PracticesIntroductionWhy should I test?Types of testsGood PracticesTesting • Tests should be repeatable (setup and cleanup)TechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 6/40

16. General Good PracticesIntroductionWhy should I test?Types of testsGood PracticesTesting • Tests should be repeatable (setup and cleanup)TechniquesUnit tests • Tests should be independent from each other (use sequences whenIntegration testsAcceptance Tests necesary)Performance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 6/40

17. General Good PracticesIntroductionWhy should I test?Types of testsGood PracticesTesting • Tests should be repeatable (setup and cleanup)TechniquesUnit tests • Tests should be independent from each other (use sequences whenIntegration testsAcceptance Tests necesary)Performance TestsMaking it • Tests should be isolated from each other (leave the environment as youhappenTDD & BDD found it)AutomatingeverythingQ&A 6/40

18. General Good PracticesIntroductionWhy should I test?Types of testsGood PracticesTesting • Tests should be repeatable (setup and cleanup)TechniquesUnit tests • Tests should be independent from each other (use sequences whenIntegration testsAcceptance Tests necesary)Performance TestsMaking it • Tests should be isolated from each other (leave the environment as youhappenTDD & BDD found it)Automatingeverything • Tests should test specs and not depend on the implementation detailsQ&A 6/40

19. General Good PracticesIntroductionWhy should I test?Types of testsGood PracticesTesting • Tests should be repeatable (setup and cleanup)TechniquesUnit tests • Tests should be independent from each other (use sequences whenIntegration testsAcceptance Tests necesary)Performance TestsMaking it • Tests should be isolated from each other (leave the environment as youhappenTDD & BDD found it)Automatingeverything • Tests should test specs and not depend on the implementation detailsQ&A • Separate the tests from the code. If not possible, refactor! 6/40

21. Main toolsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 8/40

22. Main toolsIntroductionWhy should I test?Types of testsGood Practices • EunitTestingTechniques The classic *unit approachUnit testsIntegration tests Starting to use it is very easyAcceptance TestsPerformance Tests Out of the box integration with rebarMaking it But. . . Might be annoying to debug ithappenTDD & BDDAutomatingeverythingQ&A 8/40

23. Main toolsIntroductionWhy should I test?Types of testsGood Practices • EunitTestingTechniques The classic *unit approachUnit testsIntegration tests Starting to use it is very easyAcceptance TestsPerformance Tests Out of the box integration with rebarMaking it But. . . Might be annoying to debug ithappenTDD & BDD • Common TestAutomatingeverything Much more powerfulQ&A Setting it up can be painful It shines in complex scenarios, so usually is better for integration tests 8/40

24. A different approach: property testingIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests Finding good testcases might be hard:Integration testsAcceptance TestsPerformance Tests • There might be lots of themMaking it • They might be just hard to predicthappenTDD & BDDAutomating • They might be expensive / hard to generateeverythingQ&A 9/40

25. Property testing toolsIntroductionWhy should I test?Types of testsGood Practices PropEr / Quickcheck Let’s use some math!TestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 10/40

26. Property testing toolsIntroductionWhy should I test?Types of testsGood Practices PropEr / Quickcheck Let’s use some math!TestingTechniquesUnit tests • You fed them with properties your code should satisfyIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 10/40

27. Property testing toolsIntroductionWhy should I test?Types of testsGood Practices PropEr / Quickcheck Let’s use some math!TestingTechniquesUnit tests • You fed them with properties your code should satisfyIntegration testsAcceptance Tests • You specify how the input should look likePerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 10/40

28. Property testing toolsIntroductionWhy should I test?Types of testsGood Practices PropEr / Quickcheck Let’s use some math!TestingTechniquesUnit tests • You fed them with properties your code should satisfyIntegration testsAcceptance Tests • You specify how the input should look likePerformance TestsMaking it • They smartly generate input and try to verify whether the properties arehappenTDD & BDD satisﬁedAutomatingeverythingQ&A 10/40

29. Property testing toolsIntroductionWhy should I test?Types of testsGood Practices PropEr / Quickcheck Let’s use some math!TestingTechniquesUnit tests • You fed them with properties your code should satisfyIntegration testsAcceptance Tests • You specify how the input should look likePerformance TestsMaking it • They smartly generate input and try to verify whether the properties arehappenTDD & BDD satisﬁedAutomatingeverything • If they ﬁnd a counterexample, they simplify it for you!Q&A 10/40

30. Property testing toolsIntroductionWhy should I test?Types of testsGood Practices PropEr / Quickcheck Let’s use some math!TestingTechniquesUnit tests • You fed them with properties your code should satisfyIntegration testsAcceptance Tests • You specify how the input should look likePerformance TestsMaking it • They smartly generate input and try to verify whether the properties arehappenTDD & BDD satisﬁedAutomatingeverything • If they ﬁnd a counterexample, they simplify it for you!Q&A • They can also verify the transitions in your gen_servers or gen_fsms 10/40

31. Unit Tests Examples To be testedIntroductionWhy should I test? −spec re mo ve _du pl ic at es ( [ i n t e g e r ( ) ] ) −> [ i n t e g e r ( ) ] .Types of testsGood Practices r e mo ve _d upl ic at es ( [ ] ) −>Testing [];Techniques r e mo ve _d upl ic at es ( L ) when i s _ l i s t ( L ) −>Unit tests l i s t s : usort (L) .Integration testsAcceptance Tests r e m o v e _ d u p l i c a t e s _ f r o m _ l i s t s _ f i l e ( F i l e ) −>Performance Tests case l o a d _ l i s t s _ f r o m _ f i l e ( F i l e ) o f { e r r o r , Reason } −>Making it { e r r o r , Reason } ;happen { ok , L i s t s } −>TDD & BDD { ok , [ rem ov e_ du pli ca te s ( L ) | | L <− L i s t s ] }Automating end .everythingQ&A % Internal functions % l o a d _ l i s t s _ f r o m _ f i l e ( F i l e ) −> case f i l e : c o n s u l t ( F i l e ) o f { e r r o r , Reason } −> { e r r o r , Reason } ; { ok , Terms } −> { ok , [ L | | L <− Terms , i s _ l i s t ( L ) ] } end . 11/40

32. Unit Tests Examples EunitIntroductionWhy should I test? s i m p l e _ c a s e s _ t e s t _ ( ) −>Types of tests [ ? _ a s s e r t E q u a l ( [ ] , t s _ l i s t s : re mov e_ du pl ica te s ( [ ] ) ) ,Good Practices % Bad t e s t c a s e ( dependent / f o r c i n g on t h e i m p l e m e n t a t i o n ! ! )Testing ? _ a s s e r t E q u a l ( [ 1 , 2 , 3 , 4 , 5 ] , t s _ l i s t s : re mo ve_ du pl ic at es ( [ 1 , 2 , 2 , 3 , 4 , 4 , 5 ] ) ) ] .Techniques l o a d _ l i s t s _ f r o m _ f i l e _ t e s t _ ( ) −>Unit tests { setup ,Integration tests % SETUP c r e a t e s a v a l i d f i l e w i t h 2 l i s t s and ensures I n v a l i d F i l e does n o t e x i s tAcceptance Tests f u n ( ) −> s e t u p _ f i l e s ( ) end ,Performance Tests f u n ( { V a l i d F i l e , _ I n v a l i d F i l e } ) −> ok= f i l e : d e l e t e ( V a l i d F i l e ) end , f u n ( { V a l i d F i l e , I n v a l i d F i l e } ) −>Making it [ ? _ a s s e r t E q u a l ( { ok , [ [ 1 , 2 , −2, −2, −1] , [ −2, 2 , 2 , 1 ] ] } , t s _ l i s t s : l o a d _ l i s t s _ f r o m _ f i l e (happen ValidFile ) ) ,TDD & BDD ? _ a s s e r t E q u a l ( { e r r o r , enoent } , t s _ l i s t s : l o a d _ l i s t s _ f r o m _ f i l e ( I n v a l i d F i l e ) ) ]Automating endeverything }.Q&A r e m o v e _ d u p l i c a t e s _ f r o m _ l i s t s _ f i l e _ t e s t _ ( ) −> { setup , f u n ( ) −> s e t u p _ f i l e s ( ) end , f u n ( { V a l i d F i l e , _ I n v a l i d F i l e } ) −> ok= f i l e : d e l e t e ( V a l i d F i l e ) end , f u n ( { V a l i d F i l e , I n v a l i d F i l e } ) −> [ ? _ a s s e r t E q u a l ( { e r r o r , enoent } , t s _ l i s t s : r e m o v e _ d u p l i c a t e s _ f r o m _ l i s t s _ f i l e ( I n v a l i d F i l e ) ) , ? _assertMatch ( { ok , [ L1 , L2 ] } when i s _ l i s t ( L1 ) and i s _ l i s t ( L2 ) , t s _ l i s t s : remove_duplicates_from_lists_file ( ValidFile ) ) ] end } . 12/40

33. Unit Tests Examples ProperIntroductionWhy should I test? p r o p _ r e m o v e _ d u p l i c a t e s _ k e e p s _ a l l _ e l e m e n t s ( ) −>Types of tests numtests (1000 , ?FORALL( L , n e _ l i s t ( i n t e g e r ( ) ) ,Good Practices beginTesting Processed= t s _ l i s t s : re mo ve _d upl ic at es ( L ) ,Techniques ?WHENFAIL(Unit tests i o : f o r m a t ( " L=~p , Processed=~p~n " , [ L , Processed ] ) ,Integration tests l i s t s : a l l ( f u n (N) −> l i s t s : member (N, Processed ) end , L )Acceptance Tests )Performance Tests end )).Making it p r o p _ r e m o v e _ d u p l i c a t e s _ c o n t a i n s _ n o _ d u p l i c a t e s ( ) −>happen numtests (1000 , ?FORALL( O r i g i n a l , n e _ l i s t ( i n t e g e r ( ) ) ,TDD & BDD beginAutomating L= t s _ l i s t s : r em ov e_ du pli ca te s ( O r i g i n a l ) ,everything Seq= l i s t s : seq ( 1 , l e n g t h ( L ) ) ,Q&A ?WHENFAIL( i o : f o r m a t ( " O r i g i n a l =~p , Processed=~p~n " , [ O r i g i n a l , L ] ) , l i s t s : a l l ( f u n (N) −> Elem= l i s t s : n t h (N, L ) , { _Before , [ Elem | A f t e r ] } = l i s t s : s p l i t w i t h ( f u n (X) −> X/ = Elem end , L ) , n o t l i s t s : member ( Elem , A f t e r ) end , Seq ) ) end )). 13/40

34. Integration TestsIntroductionWhy should I test?Types of testsGood PracticesTesting Common Test A ﬂexible tool for complex scenariosTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 14/40

35. Integration TestsIntroductionWhy should I test?Types of testsGood PracticesTesting Common Test A ﬂexible tool for complex scenariosTechniquesUnit tests • per_suite and per_testcase setup and teardownIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 14/40

36. Integration TestsIntroductionWhy should I test?Types of testsGood PracticesTesting Common Test A ﬂexible tool for complex scenariosTechniquesUnit tests • per_suite and per_testcase setup and teardownIntegration testsAcceptance TestsPerformance Tests • Easy to automatically avoid certain testcases under certain conditionsMaking it ({skip, Reason})happenTDD & BDDAutomatingeverythingQ&A 14/40

37. Integration TestsIntroductionWhy should I test?Types of testsGood PracticesTesting Common Test A ﬂexible tool for complex scenariosTechniquesUnit tests • per_suite and per_testcase setup and teardownIntegration testsAcceptance TestsPerformance Tests • Easy to automatically avoid certain testcases under certain conditionsMaking it ({skip, Reason})happenTDD & BDD • Also possible to run the test suites against a conﬁgurable set of runningAutomatingeverything nodesQ&A 14/40

38. Integration TestsIntroductionWhy should I test?Types of testsGood PracticesTesting Common Test A ﬂexible tool for complex scenariosTechniquesUnit tests • per_suite and per_testcase setup and teardownIntegration testsAcceptance TestsPerformance Tests • Easy to automatically avoid certain testcases under certain conditionsMaking it ({skip, Reason})happenTDD & BDD • Also possible to run the test suites against a conﬁgurable set of runningAutomatingeverything nodesQ&A • ... 14/40

39. Common Test The way we use itIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 15/40

40. Common Test The way we use itIntroductionWhy should I test?Types of testsGood Practices 1. We use the SUITE setup to prepare the pair of projects we want toTestingTechniques integrateUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 15/40

41. Common Test The way we use itIntroductionWhy should I test?Types of testsGood Practices 1. We use the SUITE setup to prepare the pair of projects we want toTestingTechniques integrateUnit testsIntegration testsAcceptance Tests 2. Still in the SUITE setup, we use ct_slave to start the VMs for those projectsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 15/40

42. Common Test The way we use itIntroductionWhy should I test?Types of testsGood Practices 1. We use the SUITE setup to prepare the pair of projects we want toTestingTechniques integrateUnit testsIntegration testsAcceptance Tests 2. Still in the SUITE setup, we use ct_slave to start the VMs for those projectsPerformance Tests 3. We use per testcase setup for creating speciﬁc conditions in each nodeMaking ithappen via RPCTDD & BDDAutomatingeverythingQ&A 15/40

43. Common Test The way we use itIntroductionWhy should I test?Types of testsGood Practices 1. We use the SUITE setup to prepare the pair of projects we want toTestingTechniques integrateUnit testsIntegration testsAcceptance Tests 2. Still in the SUITE setup, we use ct_slave to start the VMs for those projectsPerformance Tests 3. We use per testcase setup for creating speciﬁc conditions in each nodeMaking ithappen via RPCTDD & BDDAutomatingeverything 4. Being A and B the nodes we want to integrate, testcases executeQ&A something in A that requires it to interact with B 15/40

44. Common Test The way we use itIntroductionWhy should I test?Types of testsGood Practices 1. We use the SUITE setup to prepare the pair of projects we want toTestingTechniques integrateUnit testsIntegration testsAcceptance Tests 2. Still in the SUITE setup, we use ct_slave to start the VMs for those projectsPerformance Tests 3. We use per testcase setup for creating speciﬁc conditions in each nodeMaking ithappen via RPCTDD & BDDAutomatingeverything 4. Being A and B the nodes we want to integrate, testcases executeQ&A something in A that requires it to interact with B 5. Testcases assert the obtained response matches the expected one given the testcase preparation 15/40

45. General ConsiderationsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 16/40

46. General ConsiderationsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • These tests should check the whole stack, top to bottomIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 16/40

47. General ConsiderationsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • These tests should check the whole stack, top to bottomIntegration testsAcceptance Tests • People without technical knowledge about the project should be able toPerformance Tests write themMaking ithappenTDD & BDDAutomatingeverythingQ&A 16/40

48. General ConsiderationsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • These tests should check the whole stack, top to bottomIntegration testsAcceptance Tests • People without technical knowledge about the project should be able toPerformance Tests write themMaking ithappenTDD & BDD • They can be written in any languageAutomatingeverythingQ&A 16/40

49. General ConsiderationsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • These tests should check the whole stack, top to bottomIntegration testsAcceptance Tests • People without technical knowledge about the project should be able toPerformance Tests write themMaking ithappenTDD & BDD • They can be written in any languageAutomatingeverything • They don’t look for bugs, they validate the requirementsQ&A 16/40

50. What we use at Team ServicesIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • Since we are developing an HTTP stack, we use JMeter for validationIntegration testsAcceptance Tests • We could use any of the already described erlang tools, but. . .Performance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 17/40

51. What we use at Team ServicesIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • Since we are developing an HTTP stack, we use JMeter for validationIntegration testsAcceptance Tests • We could use any of the already described erlang tools, but. . .Performance TestsMaking it We want QA people from other teams to be able to black-box test our codehappenTDD & BDDAutomatingeverythingQ&A 17/40

52. What we use at Team ServicesIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • Since we are developing an HTTP stack, we use JMeter for validationIntegration testsAcceptance Tests • We could use any of the already described erlang tools, but. . .Performance TestsMaking it We want QA people from other teams to be able to black-box test our codehappen We want to avoid corrupting the acceptance tests with our knowledge of theTDD & BDDAutomating system internalseverythingQ&A 17/40

53. What we use at Team Services JMeterIntroductionWhy should I test?Types of tests • Simple assertion-based mechanismGood PracticesTesting • Nice management interfaceTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 18/40

54. Some other tools used inside Spil Games CucumberIntroductionWhy should I test?Types of testsGood Practices • Scenario descriptions in Gherkin (close to natural language)TestingTechniques • Multi-language support (Ruby, Php, Python. . . )Unit testsIntegration tests # language : enAcceptance Tests Feature : Payment Method CountPerformance Tests As a user , I want t o see a c e r t a i n number o f payment methods f o r my c o u n t r y .Making ithappen Scenario O u t l i n e : Payment Method ChoiceTDD & BDD Given I am a user i n " <Country > "Automatingeverything And I open t h e PSS Then I should see " <PaymentMethodCount> " payment methodsQ&A Examples : | Country | PaymentMethodCount | | Poland | 6 | | France | 5 | | Brazil | 6 | 19/40

55. Some other tools used inside Spil Games CucumberIntroductionWhy should I test?Types of tests • Scenario implementationGood Practices /∗∗ @Given / ^ I am a user i n " ( [ ^ " ] ∗) " $ / ∗/Testing p u b l i c f u n c t i o n iAmAUserIn ( $ c o u n t r y )Techniques {Unit tests $country = s t r t o l o w e r ( $country ) ;Integration tests $ t h i s−>s e s s i o n−>setRequestHeader ( ’ SPILTESTIPADDRESS ’ , S p i l T e s t I P : : addressFor ( $ c o u n t r y ) ) ;Acceptance Tests }Performance Tests /∗∗ @Given / ^ I open t h e PSS$ / ∗/ p u b l i c f u n c t i o n iOpenThePss ( )Making it {happen r e t u r n $ t h i s−>iOpenThePaymentSelectionScreen ( ) ;TDD & BDD }Automatingeverything /∗∗ @Then / ^ I should see " ( [ ^ " ] ∗) " payment methods$ / ∗/ p u b l i c f u n c t i o n iShouldSeePaymentMethods ( $nummethods )Q&A { $page = $ t h i s−>s e s s i o n−>getPage ( ) ; $ e l = $ t h i s−>getDomNodeByXpath ( $page , ’ / / ∗ [ @id=" p s s _ m e t h o d _ c o l l e c t i o n " ] ’ ) ; $foundmethods = count ( $ e l−> f i n d A l l ( ’ css ’ , ’ d i v . pss_method ’ ) ) ; i f ( $nummethods ! = $foundmethods ) { throw new Ex c e p t i o n ( " Found payment methods does n o t equal $nummethods " ) ; } } 20/40

56. Basic BenchmarkingIntroductionWhy should I test?Types of testsGood PracticesTestingTechniques • Testing a function doesn’t have to be painfulUnit testsIntegration testsAcceptance Tests • The sooner you realize is bad, the less impactPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 21/40

57. Basic BenchmarkingIntroductionWhy should I test?Types of testsGood PracticesTestingTechniques • Testing a function doesn’t have to be painfulUnit testsIntegration testsAcceptance Tests • The sooner you realize is bad, the less impactPerformance Tests −spec t e s t _ f u n ( any ( ) , f u n ( ( ) −> ok ) , i n t e g e r ( ) , i n t e g e r ( ) ) −> { f l o a t ( ) , f l o a t ( ) } .Making ithappen t e s t _ f u n (Name, TestFun , ParCount , SeqCount ) −>TDD & BDD RS = t e s t _ f u n _ x _ t i m e s (Name, TestFun , SeqCount ) ,Automating RP = t e s t _ f u n _ x _ t i m e s _ p a r a l l e l (Name, TestFun , ParCount , SeqCount ) ,everything {RS, RP} .Q&A 21/40

58. Basic Benchmarking Sync ExecutionIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests t e s t _ f u n _ x _ t i m e s (Name, Fun , SeqCount ) −>Integration tests T i c k = now ( ) ,Acceptance Tests l i s t s : f o r e a c h ( f u n ( _ ) −>Performance Tests Fun ( ) end , l i s t s : seq ( 1 , SeqCount ) ) ,Making it Tock = now ( ) ,happen AvgTime = t i m e r : n o w _ d i f f ( Tock , T i c k ) / 1 0 0 0 ,TDD & BDD Speed = SeqCount ∗1000/AvgTime ,Automating i o : f o r m a t ( " ~40p req / sec ~.1 f ~n " , [ Name, Speed ] ) ,everything Speed .Q&A 22/40

59. Basic Benchmarking Async ExecutionIntroductionWhy should I test?Types of testsGood PracticesTestingTechniques t e s t _ f u n _ x _ t i m e s _ p a r a l l e l (Name, Fun , ParCount , SeqCount ) −>Unit tests T i c k = now ( ) ,Integration tests pmap ( f u n ( _ ) −>Acceptance Tests l i s t s : f o r e a c h ( f u n ( _ ) −>Performance Tests Fun ( ) end , l i s t s : seq ( 1 , SeqCount ) )Making it end , l i s t s : seq ( 1 , ParCount ) , 250000) , %t i m e o u thappen Tock = now ( ) ,TDD & BDD AvgTime = t i m e r : n o w _ d i f f ( Tock , T i c k ) / 1 0 0 0 ,Automatingeverything Speed = ParCount∗SeqCount ∗1000/AvgTime , i o : f o r m a t ( " ~40p req / sec ~10.1 f ~n " , [ Name, Speed ] ) ,Q&A Speed . 23/40

60. TsungIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration tests • It can be used to generate waves of load according to different parametersAcceptance TestsPerformance Tests • It generates graphics showing how the target system reacted to the loadMaking ithappen • Uses XML especiﬁcationsTDD & BDDAutomatingeverythingQ&A 24/40

61. TsungIntroductionWhy should I test? • getToken waves of 100 and 500 req/s during 600 secsTypes of tests <?xml v e r s i o n = " 1 . 0 " ?>Good Practices < !DOCTYPE tsung SYSTEM " / u s r / share / tsung / tsung −1.0. d t d " [ ] > <tsung l o g l e v e l = " debug " d u m p t r a f f i c = " p r o t o c o l " >Testing <clients>Techniques < c l i e n t h o s t = " l o c a l h o s t " u s e _ c o n t r o l l e r _ v m = " t r u e " >< / c l i e n t >Unit tests </ clients>Integration tests <servers>Acceptance Tests < s e r v e r h o s t = " a p i . spilgames . com" p o r t = " 443 " t y p e = " s s l " >< / s e r v e r >Performance Tests < / servers>Making it <load>happen < a r r i v a l p h a s e phase= " 1 " d u r a t i o n = " 600 " u n i t = " second " >TDD & BDD <users a r r i v a l r a t e = " 100 " u n i t = " second " >< / users >Automating </ arrivalphase>everything < a r r i v a l p h a s e phase= " 2 " d u r a t i o n = " 600 " u n i t = " second " > <users a r r i v a l r a t e = " 500 " u n i t = " second " >< / users >Q&A < / load> <sessions> < s e s s i o n name= " a p i " p r o b a b i l i t y = " 100 " t y p e = " t s _ h t t p " > <request> < d y n _ v a r i a b l e name= " g u es t t ok e n " j s o n p a t h = " auth . token " / > < h t t p u r l = ’ / account / getToken / ’ v e r s i o n = ’ 1 . 1 ’ c o n t e n t s = ’ {&quot ; t y p e&quot ; : &quot ; sample&quot ; } ’ c o n t e n t _ t y p e = ’ t e x t / j s o n ’ method= ’POST ’ > </ http> < / request> < / session> < / sessions> < / tsung > 25/40

63. The purist approachIntroductionWhy should I test?Types of testsGood Practices 1. A developer receives some requirements from his manager, who ideallyTestingTechniques has writting some acceptance testsUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 27/40

64. The purist approachIntroductionWhy should I test?Types of testsGood Practices 1. A developer receives some requirements from his manager, who ideallyTestingTechniques has writting some acceptance testsUnit testsIntegration tests 2. He designs the simplest (ﬂexible) solution ﬁtting those requirementsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 27/40

65. The purist approachIntroductionWhy should I test?Types of testsGood Practices 1. A developer receives some requirements from his manager, who ideallyTestingTechniques has writting some acceptance testsUnit testsIntegration tests 2. He designs the simplest (ﬂexible) solution ﬁtting those requirementsAcceptance TestsPerformance Tests 3. He documents and specs the API, and implements only the exportedMaking ithappen functions headers all returning unimplementedTDD & BDDAutomatingeverythingQ&A 27/40

66. The purist approachIntroductionWhy should I test?Types of testsGood Practices 1. A developer receives some requirements from his manager, who ideallyTestingTechniques has writting some acceptance testsUnit testsIntegration tests 2. He designs the simplest (ﬂexible) solution ﬁtting those requirementsAcceptance TestsPerformance Tests 3. He documents and specs the API, and implements only the exportedMaking ithappen functions headers all returning unimplementedTDD & BDDAutomatingeverything 4. He writes as many unit and integration tests as necesary for that APIQ&A 27/40

67. The purist approachIntroductionWhy should I test?Types of testsGood Practices 1. A developer receives some requirements from his manager, who ideallyTestingTechniques has writting some acceptance testsUnit testsIntegration tests 2. He designs the simplest (flexible) solution fitting those requirementsAcceptance TestsPerformance Tests 3. He documents and specs the API, and implements only the exportedMaking ithappen functions headers all returning unimplementedTDD & BDDAutomatingeverything 4. He writes as many unit and integration tests as necesary for that APIQ&A 5. He hands over his work to another developer, who after agreeing with the design fills the implementation until all the tests pass 27/40

68. A more practical approachIntroductionWhy should I test?Types of tests When writting new codeGood PracticesTesting • Write the tests ASAP, if possible before writing the code itselfTechniquesUnit tests • While implementing, keep in mind everything you write you have to test itIntegration testsAcceptance TestsPerformance Tests • Start executing your tests ASAP, specially to validate those small simpleMaking it functions you use all around (yes, those that can never fail)happenTDD & BDDAutomatingeverythingQ&A 28/40

69. A more practical approachIntroductionWhy should I test?Types of tests When writting new codeGood PracticesTesting • Write the tests ASAP, if possible before writing the code itselfTechniquesUnit tests • While implementing, keep in mind everything you write you have to test itIntegration testsAcceptance TestsPerformance Tests • Start executing your tests ASAP, specially to validate those small simpleMaking it functions you use all around (yes, those that can never fail)happenTDD & BDD When changing existing codeAutomatingeverything • Even if the existing coverage is poor, write tests for the changes youQ&A introduce • Make sure all the tests are running before touching a single line! 28/40

70. TDD refactor Original TestsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance Tests • One of the following tests crashed on June 23Performance Tests g e t _ a g e _ t e s t _ ( ) −>Making it [happen ? _ a s s e r t E q u a l ( 4 7 , su_user : get_age ( # date { year =1965 , month =2 , day=1 } ) ) ,TDD & BDD ? _ a s s e r t E q u a l ( 2 7 , su_user : get_age ( # date { year =1984 , month =6 , day=27 } ) )Automating ].everythingQ&A 29/40

71. TDD refactor Original CodeIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit tests • The implementation had an obvious ﬂawIntegration testsAcceptance Tests −spec get_age ( # date { } ) −> non_neg_integer ( ) .Performance Tests get_age ( # date { year =0 , month =0 , day=0 } ) −>Making it 0;happen get_age ( # date { year=Y , month=M, day=D} ) −>TDD & BDD B i r t h G r e g o r i a n D a y s = c a l e n d a r : d a t e _ t o _ g r e g o r i a n _ d a y s ( { Y , M, D} ) ,Automating { NowDate , _ } = c a l e n d a r : now_to_datetime ( os : timestamp ( ) ) ,everything CurrentGregorianDays = c a l e n d a r : d a t e _ t o _ g r e g o r i a n _ d a y s ( NowDate ) ,Q&A ( CurrentGregorianDays−B i r t h G r e g o r i a n D a y s ) d i v 365. 30/40

72. TDD refactor Updated TestsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniques • We update the tests with the requirements for a human-like approachUnit testsIntegration tests g e t _ a g e _ t e s t _ ( ) −>Acceptance Tests [Performance Tests ? _ a s s e r t E q u a l ( 4 7 , su_user : get_age ( # date { year =1965 , month =2 , day=1 } ) ) ].Making it i s _ b i r t h d a y _ g o n e _ t e s t _ ( ) −>happen Dob=# date { year =1972 , month =4 , day=21 } ,TDD & BDD [Automating ? _ a s s e r t E q u a l ( f a l s e , su_user : i s _ b i r t h d a y _ g o n e ( Dob , 4 , 20) )everything ? _ a s s e r t E q u a l ( t r u e , su_user : i s _ b i r t h d a y _ g o n e ( Dob , 4 , 22) ) ,Q&A ? _ a s s e r t E q u a l ( t r u e , su_user : i s _ b i r t h d a y _ g o n e ( Dob , 4 , 21) ) , ]. 31/40

73. TDD refactor Updated CodeIntroductionWhy should I test?Types of testsGood PracticesTesting • The resultant code is still very clean. And now it works!TechniquesUnit tests −spec get_age ( # date { } ) −> non_neg_integer ( ) .Integration tests get_age ( # date { year =0 , month=_ , day=_ } ) −>Acceptance Tests 0;Performance Tests get_age ( # date { year=Y , month=_ , day=_ } =Dob ) −> { {NY, NM, ND} , _ } = c a l e n d a r : now_to_datetime ( os : timestamp ( ) ) ,Making it case i s _ b i r t h d a y _ g o n e ( Dob , NM, ND) o fhappen t r u e −> NY − Y ;TDD & BDD f a l s e −> NY − Y −1Automatingeverything end .Q&A −spec i s _ b i r t h d a y _ g o n e ( # date { } , p o s _ i n t e g e r ( ) , p o s _ i n t e g e r ( ) ) −> boolean ( ) . i s _ b i r t h d a y _ g o n e ( # date { month=M, day=D} , CurrentMonth , CurrentDay ) −> (M∗100+D) =< ( CurrentMonth ∗100+CurrentDay ) . 32/40

74. CI EnvironmentIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 33/40

75. CI EnvironmentIntroductionWhy should I test?Types of testsGood Practices • Builds and tests a release your projects after every commit (Unit,TestingTechniques Integration and Acceptance)Unit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 33/40

76. CI EnvironmentIntroductionWhy should I test?Types of testsGood Practices • Builds and tests a release your projects after every commit (Unit,TestingTechniques Integration and Acceptance)Unit testsIntegration tests • This automatic V&V process should never take longer than 5 minutesAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 33/40

77. CI EnvironmentIntroductionWhy should I test?Types of testsGood Practices • Builds and tests a release your projects after every commit (Unit,TestingTechniques Integration and Acceptance)Unit testsIntegration tests • This automatic V&V process should never take longer than 5 minutesAcceptance TestsPerformance Tests • If the process fails, immediatly blames the originator of the changeMaking ithappenTDD & BDDAutomatingeverythingQ&A 33/40

78. CI EnvironmentIntroductionWhy should I test?Types of testsGood Practices • Builds and tests a release your projects after every commit (Unit,TestingTechniques Integration and Acceptance)Unit testsIntegration tests • This automatic V&V process should never take longer than 5 minutesAcceptance TestsPerformance Tests • If the process fails, immediatly blames the originator of the changeMaking ithappen • Long duration test suites can be put together in a regression taskTDD & BDDAutomatingeverything performed during the nightQ&A 33/40

79. CI EnvironmentIntroductionWhy should I test?Types of testsGood Practices • Builds and tests a release your projects after every commit (Unit,TestingTechniques Integration and Acceptance)Unit testsIntegration tests • This automatic V&V process should never take longer than 5 minutesAcceptance TestsPerformance Tests • If the process fails, immediatly blames the originator of the changeMaking ithappen • Long duration test suites can be put together in a regression taskTDD & BDDAutomatingeverything performed during the nightQ&A CI + AutomaticDeployment = ContinuousDelivery 33/40

80. CI Environment JenkinsIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 34/40

81. Keeping an eye on qualityIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration tests • Most CI systems have desktop or browser pluginsAcceptance TestsPerformance Tests • Screens on the wall can be an option as wellMaking ithappen • The quality of the ﬁnal product starts with quality of your codeTDD & BDDAutomatingeverythingQ&A 35/40

82. Measuring code qualityIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 36/40

83. Measuring code qualityIntroductionWhy should I test?Types of testsGood Practices • Cover can be integrated with eunit and Common Test to show you the testTestingTechniques coverageUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 36/40

84. Measuring code qualityIntroductionWhy should I test?Types of testsGood Practices • Cover can be integrated with eunit and Common Test to show you the testTestingTechniques coverageUnit testsIntegration testsAcceptance Tests • Dialyzer performs static type analysis and clear specs help a lot whenPerformance Tests testingMaking ithappenTDD & BDDAutomatingeverythingQ&A 36/40

85. Measuring code qualityIntroductionWhy should I test?Types of testsGood Practices • Cover can be integrated with eunit and Common Test to show you the testTestingTechniques coverageUnit testsIntegration testsAcceptance Tests • Dialyzer performs static type analysis and clear specs help a lot whenPerformance Tests testingMaking ithappen • Sonar analyses your code looking for risksTDD & BDDAutomatingeverythingQ&A 36/40

86. Measuring code qualityIntroductionWhy should I test?Types of testsGood Practices • Cover can be integrated with eunit and Common Test to show you the testTestingTechniques coverageUnit testsIntegration testsAcceptance Tests • Dialyzer performs static type analysis and clear specs help a lot whenPerformance Tests testingMaking ithappen • Sonar analyses your code looking for risksTDD & BDDAutomating Acts as an interface for cover and dialyzer resultseverythingQ&A 36/40

87. Measuring code qualityIntroductionWhy should I test?Types of testsGood Practices • Cover can be integrated with eunit and Common Test to show you the testTestingTechniques coverageUnit testsIntegration testsAcceptance Tests • Dialyzer performs static type analysis and clear specs help a lot whenPerformance Tests testingMaking ithappen • Sonar analyses your code looking for risksTDD & BDDAutomating Acts as an interface for cover and dialyzer resultseverything Points you to the weakest parts of your sourceQ&A 36/40

88. Measuring code qualityIntroductionWhy should I test?Types of testsGood Practices • Cover can be integrated with eunit and Common Test to show you the testTestingTechniques coverageUnit testsIntegration testsAcceptance Tests • Dialyzer performs static type analysis and clear specs help a lot whenPerformance Tests testingMaking ithappen • Sonar analyses your code looking for risksTDD & BDDAutomating Acts as an interface for cover and dialyzer resultseverything Points you to the weakest parts of your sourceQ&A Shows nice graphics about test coverage, unreachable code, nested code. . . over the time 36/40

89. Quality control with SonarIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 37/40

90. Quality control with SonarIntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration testsAcceptance TestsPerformance TestsMaking ithappenTDD & BDDAutomatingeverythingQ&A 38/40

92. Thanks Everyone!IntroductionWhy should I test?Types of testsGood PracticesTestingTechniquesUnit testsIntegration tests • Questions?Acceptance TestsPerformance Tests • Comments?Making ithappen • Suggestions?TDD & BDDAutomatingeverythingQ&A 40/40

Testing an Erlang Backend

enriquepazperez

Testing an Erlang Backend