Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018)

461 views

Published on

Abstract: This session is about my experience with BDD in a Continuous Delivery model. Many teams use Behavior Driven Development (BDD) for automated UI testing. Although UI testing might be an important element of your verification pipeline, it is slow, brittle and costly. But in the era of modern software development we would like to have fast feedback and quick reactions. A continuous delivery / deployment (CD) model can support this well. But what does it look like with BDD then? Can BDD help implementing CD?
This session is an attempt to define the role of BDD in CD. What works, what does not? Continuous delivery should mean continuous quality for the delivery team. What kind of quality criteria do we want to target with CD and how is it supported by BDD?

Published in: Software
  • Be the first to comment

  • Be the first to like this

Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018)

  1. 1. Continuous Behavior BDD in Continuous Delivery CoDers Who Test 15th May 2018 Gáspár Nagy coach • trainer • bdd addict • creator of specflow “The BDD Books: Discovery” • http://bddbooks.com @gasparnagy • gaspar@specsolutions.eu
  2. 2. Copyright © Gaspar NagyCopyright © Gaspar Nagy bdd addict given.when.then CAUTION! on the stage Gáspár Nagy coach, trainer and bdd addict creator of SpecFlow gaspar@specsolutions.eu https://specsolutions.eu @gasparnagy
  3. 3. Copyright © Gaspar NagyCopyright © Gaspar Nagy What makes you nervous on the day of the release?
  4. 4. My first app…
  5. 5. A long time ago* in a galaxy far, far away**… * 1995 ** Veszprém, Hungary
  6. 6. Copyright © Gaspar NagyCopyright © Gaspar Nagy My dream-projects in 1995 • Working software that was in use • Exciting technical solutions • Continuous learning • And no conflicts in the team… …because I was alone
  7. 7. Copyright © Gaspar NagyCopyright © Gaspar Nagy My first “real” project…
  8. 8. Copyright © Gaspar NagyCopyright © Gaspar Nagy Practicing team work
  9. 9. Copyright © Gaspar NagyCopyright © Gaspar Nagy Practicing team work And by a modem for “A”
  10. 10. Copyright © Gaspar NagyCopyright © Gaspar Nagy Integration is more than merging… changed foo.cpp changed bar.cpp ? void ProcessOrder() { … + account->Withdraw(sum); … } void FinalizeOrder() { … + account->Withdraw(sum); … }
  11. 11. Copyright © Gaspar NagyCopyright © Gaspar Nagy What is integration, then? Ensuring that the improvements developed independently still form a working unit Resolving file- level conflicts Can compile, can build Can be deployed There is no regression [Enter your specific step here]
  12. 12. Copyright © Gaspar NagyCopyright © Gaspar Nagy Proper integration is difficult…
  13. 13. Copyright © Gaspar NagyCopyright © Gaspar Nagy Frequency reduces difficulty (Fowler) • „if it hurts, do it more often”
  14. 14. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous Integration, CI Ensures that the improvements developed independently still form a working unit • Resolving file-level conflicts • Compilation, build • Unit tests • ...
  15. 15. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous integration revealed the next problems… • Manual regression testing does not scale… • Manual deployments are error-prone…
  16. 16. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous Delivery, CD Ensures that the application is always in a potentially deployable state (whether we deploy is a manual decision) • CI + • Deployable • Automated regression tests (Acceptance Tests) • ...
  17. 17. Copyright © Gaspar NagyCopyright © Gaspar Nagy CD vs. CD Continuous Delivery vs. Continuous Deployment
  18. 18. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous Delivery, Continuous Deployment
  19. 19. Copyright © Gaspar NagyCopyright © Gaspar Nagy “Continues delivery can be achieved by anyone, Continues deployment is only for some specific products…” /some people/
  20. 20. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous Deployment, CD Ensures that the application is always actual • CD + • Fully* automated verification • Automated non-functional tests • Appropriate context (eg. SaaS) • ... • ??? „live”
  21. 21. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous Deployment is the key to understand the “CD” mentality… …so let’s try!
  22. 22. Copyright © Gaspar NagyCopyright © Gaspar Nagy What are your fears? • I am afraid of pushing the “release” button! • The feedback I get from my tests are not useful!
  23. 23. Copyright © Gaspar NagyCopyright © Gaspar Nagy More testing and more automation! Source: World Quality Report 2015/16 increased by 35% increased by 60% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Amount of IT budget allocated to QA Ratio of test case automation 2014 2015
  24. 24. Small tests = small problems; Big tests = big problems • They are SLOOOOOOW! • They are BRITTLE! • They need awful lot of MAINTENANCE! • They FAIL!!!
  25. 25. Copyright © Gaspar NagyCopyright © Gaspar Nagy Our typical automated tests are not good enough for CD…
  26. 26. Copyright © Gaspar NagyCopyright © Gaspar Nagy What’s wrong with our tests?
  27. 27. Episode #1 PO: We have far too many prod issues. We have to improve our CD pipeline… Write more tests! Team: How much more?
  28. 28. Episode #2 PO: Are your tests balanced according to the test automation pyramid? Team: … not fully…, more like an ice-cream cone PO: Then write more unit tests!
  29. 29. Episode #3 PO: You should increase your test coverage! Team: You mean line coverage? How much? PO: Well… what is exactly a code line doing?
  30. 30. Episode #4 PO: You should cover the risks better! Team: How do we know if we have covered them?
  31. 31. Copyright © Gaspar NagyCopyright © Gaspar Nagy PO controlling testing activities… Problem Domain Solution Domain Lightbulb by Unknown Author is licensed under CC BY-SA PO REQ APP TESTS
  32. 32. Copyright © Gaspar NagyCopyright © Gaspar Nagy You cannot control something that is outside of your control…
  33. 33. Copyright © Gaspar NagyCopyright © Gaspar Nagy Tests should be part of shared ownership… Problem Domain Solution Domain Lightbulb by Unknown Author is licensed under CC BY-SA REQ APP TESTS
  34. 34. Copyright © Gaspar NagyCopyright © Gaspar Nagy How could the entire team own the tests?
  35. 35. Copyright © Gaspar NagyCopyright © Gaspar Nagy Tests should be part of shared ownership… Problem Domain Solution Domain Lightbulb by Unknown Author is licensed under CC BY-SA The old PIN should be correct public void VerifyOldPin(string enteredOldPin) { var db = new CardRespoitory(); var oldPin = db.LoadOldPin(CurrentCard); if (oldPin != enteredOldPin) { throw new InvalidPinException(); } } Implements [TestMethod] public void ShuldFailWhenOldPinIncorrect() { var cardStoreMock = new CardStoreMock(); cardStoreMock.SetCard("1234678", "1234"); var sut = new CardService(); var result = sut.ChangePin("1234678", "1111", "1212", "1212"); Assert.IsFalse(result); }
  36. 36. Copyright © Gaspar NagyCopyright © Gaspar Nagy Specification by example Problem Domain Solution Domain Lightbulb by Unknown Author is licensed under CC BY-SA The old PIN should be correct public void VerifyOldPin(string enteredOldPin) { var db = new CardRespoitory(); var oldPin = db.LoadOldPin(CurrentCard); if (oldPin != enteredOldPin) { throw new InvalidPinException(); } } Implements [TestMethod] public void ShuldFailWhenOldPinIncorrect() { var cardStoreMock = new CardStoreMock(); cardStoreMock.SetCard("1234678", "1234"); var sut = new CardService(); var result = sut.ChangePin("1234678", "1111", "1212", "1212"); Assert.IsFalse(result); } Old PIN is incorrect • PIN is 1234 • Enter old PIN: 1111 • Change PIN ⇒ PIN change rejected Illustrates Automates
  37. 37. Copyright © Gaspar NagyCopyright © Gaspar Nagy Codified tests are hard to maintain • What was the example? • Why were these numbers used? • What did we want to verify? [TestMethod] public void ShuldFailWhenOldPinIncorrect() { var cardStoreMock = new CardStoreMock(); cardStoreMock.SetCard("1234678", "1234"); var sut = new CardService(); var result = sut.ChangePin("1234678", "1111", "1212", "1212"); Assert.IsFalse(result); }
  38. 38. Copyright © Gaspar NagyCopyright © Gaspar Nagy BDD is here to help?
  39. 39. Copyright © Gaspar NagyCopyright © Gaspar Nagy Formulated BDD scenarios Problem Domain Solution Domain The old PIN should be correct public void VerifyOldPin(string enteredOldPin) { var db = new CardRespoitory(); var oldPin = db.LoadOldPin(CurrentCard); if (oldPin != enteredOldPin) { throw new InvalidPinException(); } } Implements Old PIN is incorrect • PIN is 1234 • Enter old PIN: 1111 • Change PIN ⇒ PIN change rejected Illustrates Scenario: Old PIN is incorrect Given my old pin is 1234 And I enter 1111 as old PIN When I try to change my PIN Then the PIN change should be rejected [Then] public void I_enter_PIN_as_old_pin(string pin) { changePinContext.SetOldPin(pin); } [When] public void I_enter_PIN_as_old_pin(string pin) { changePinContext.SetOldPin(pin); } [Given] public void I_enter_PIN_as_old_pin(string pin) { changePinContext.SetOldPin(pin); }
  40. 40. Copyright © Gaspar NagyCopyright © Gaspar Nagy Behavior Driven Development Problem Domain Solution Domain Lightbulb by Unknown Author is licensed under CC BY-SA REQ APP Given When Then SPECS Validates Verifies
  41. 41. Copyright © Gaspar NagyCopyright © Gaspar Nagy Behavior Driven Development is about understanding & validating business requirements through illustrative examples
  42. 42. Copyright © Gaspar NagyCopyright © Gaspar Nagy BDD is not about tests!
  43. 43. Copyright © Gaspar NagyCopyright © Gaspar Nagy Functional quality is not only about whether IT WORKS as expected… but also about whether the EXPECTATIONS ARE GOOD and DOCUMENTED… Functional quality
  44. 44. Episode in a later season… PO: We have got a strange prod issue. Could we look into that? Team: Sure. Let’s have a look at the related BDD scenarios… PO: I see. I think there is an important case we did not detail enough…
  45. 45. Copyright © Gaspar NagyCopyright © Gaspar Nagy Happy ending
  46. 46. Copyright © Gaspar NagyCopyright © Gaspar Nagy Finally I’ve got a dream-project again • It was a complex service used by hundreds of apps and through them millions of users all over the world • Processed several thousands of requests per second • Was critical in the client’s service chain • We rolled it out without any severe issues… • And we knew that there will be no problems… • Because we had shared control over the specs (BDD scenarios) • And this gave us confidence to push the release button
  47. 47. Copyright © Gaspar NagyCopyright © Gaspar Nagy Continuous deployment is about CONFIDENCE …and you need to keep the link to the problem domain to gain that
  48. 48. Why was my first project working? • Short cycles • Instant release • Close to “business” but also close to users
  49. 49. Copyright © Gaspar NagyCopyright © Gaspar Nagy Think about YOUR tests… Are they also linking the solution with the problem?
  50. 50. Source: https://shashinki.com/
  51. 51. Thank you! Gáspár Nagy coach • trainer • bdd addict • creator of specflow “The BDD Books: Discovery” • http://bddbooks.com @gasparnagy • gaspar@specsolutions.eu
  52. 52. Copyright © Gaspar NagyCopyright © Gaspar Nagy

×