Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Aistė Stikliutė - Testing in continuous delivery


Published on

Continuous delivery makes it possible to develop and deploy software within hours instead of months or weeks or days. But what about testing? Testing is not fast. Which means you will have to cut corners, but add additional measures to compensate. Many companies market their continuous delivery practices, so others can see how cool they are, but also learn from their experiences. In this talk I will go through some highlights and things that work well and not so well in these example continuous delivery implementations, and then will concentrate on testing patterns, approaches and little hints that I have learnt both from these examples and from my own experience.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Aistė Stikliutė - Testing in continuous delivery

  1. 1. Testing in Continuous Delivery Aistė Stikliūtė, Agile turas 2015
  2. 2. Agenda What is Continuous Delivery and why it’s (slowly) going mainstream Examples Challenge - testing • Problems • Solutions
  3. 3. What is Continuous Delivery C.D. Build Deploy Test Release
  4. 4. Do you do Continuous Delivery? How often do you / can you deploy to Prod? What if you’ve got to do a big change? Who decides when to deploy / release?
  5. 5. Some examples
  6. 6. … examples
  7. 7. The testing challenge Months → Weeks → Days → Hours → Minutes
  8. 8. Have the cake and eat it too? ● do thorough regression testing and do it often? ● add changes fast and keep up with automated tests? ● ensure good quality without (much) manual testing?
  9. 9. Cycle time: idea  production Design Develop Test Merge to trunk Deploy to test environment Test Deploy to stage Test Deploy to production
  10. 10. Cycle time to production: testing Manual tests before merge Automated tests after merge Staging tests Analyse results Update tests Bugfix Clarify open questions Bugfix Analyse results Bugfix
  11. 11. Solve the testing challenge Patterns and ideas
  12. 12. [obvious] agile testing Testers involved in team: grooming, planning, development “Show me” testing before it’s finished, input for automated tests When it’s finished, tester must be ready to test
  13. 13. [obvious] developer testing Develop Test
  14. 14. developer responsibility You are responsible for your code • Testing • Deploying • Supporting • Defined code ownership • Better automated tests • Monitoring. Faster corrections • Faster & more successful deployments
  15. 15. branching patterns Centralized Finds and fixes problems ASAP Distributed Develop every change in its own branch and test Temporary branches Effective, combines centralized and distributed Matrix of services Use centralized CI for a massively scalable system
  16. 16. test automation Fast tests Reliable tests
  17. 17. test automation Test optimization Well-thought infrastructure Run in parallel Test categories
  18. 18. test automation Test coverage Test environment Keep tests green “Pass” ratio
  19. 19. testing in production A/B testing feature toggling monitoring
  20. 20. feature toggling ● Needs to be implemented ● Reduces need for lab-testing, but it’s still needed ● Needs to be tested itself ● Reduces risk ● Enables A/B testing on off staff user_list opt-in 0-100%
  21. 21. A/B testing
  22. 22. monitoring Errors Performance User behavior User community / support automatic notifications
  23. 23. The future