Continuous Feedback from Testing - Continuous Delivery meetup - Amsterdam, 14-7-2016
•Download as PPTX, PDF•
3 likes•417 views
This document discusses feedback in the continuous delivery process. It notes that feedback is information about actions that is returned to the source of those actions. In continuous delivery, there are several types of testing that provide important feedback, including unit, integration, contract, end-to-end, performance, exploratory, and resilience testing. Tests should be created by both developers and testers to catch bugs and ensure stability as code is delivered frequently. Continuous feedback allows for continuous improvement of developing and releasing high quality software.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Continuous Feedback from Testing - Continuous Delivery meetup - Amsterdam, 14-7-2016
Similar to Continuous Feedback from Testing - Continuous Delivery meetup - Amsterdam, 14-7-2016 (20)
More from Pavel Chunyayev
More from Pavel Chunyayev (7)
Recently uploaded
Recently uploaded (20)
Continuous Feedback from Testing - Continuous Delivery meetup - Amsterdam, 14-7-2016
- 1. Feedback in Continuous Delivery Feedback from Testing in Continuous Delivery Process by Pavel Chunyayev, 14-7-2016 Amsterdam, ABN AMRO Continuous Delivery meetup
- 2. @PavelChunyayev Amsterdam Levi9 HQ Amsterdam – 2005 25 people Novi Sad Serbia Novi Sad – 2005 350+ people Zrenjanin Serbia Zrenjanin– 2014 50+ people Iasi Romania Iasi – 2007 100+ people Kyiv Ukraine Kyiv – 2008 160+ people Lviv Belgrade Ukraine Lviv– 2016 20+ people
- 3. @PavelChunyayev E-Retail Digital Marketing Traffic and Transport FinTech Software Services
- 4. @PavelChunyayev Customer satisfaction 2015 • 4th year great results in Outsourcing Performance study • 2015: 100% customer recommendation score Excellent trust score
- 5. @PavelChunyayev About me • 12 years of IT experience • Lived and worked in Ukraine and Estonia • Two years ago moved to the Netherlands • Love cycling • Love Dutch language • Love software development processes • Love working with people
- 7. @PavelChunyayev Feedback is information about actions returned to the source of the actions. — Wikipedia
- 10. @PavelChunyayev Continuous Delivery Plan Code Build Test Release OperateEnvision
- 12. @PavelChunyayev Keep the product releasable Build quality in
- 13. @PavelChunyayev Continuous Delivery Plan Code Build Test Release OperateEnvision
- 14. @PavelChunyayev Continuous Delivery Plan Code Build Test Release OperateEnvision
- 15. @PavelChunyayev
- 16. @PavelChunyayev
- 17. @PavelChunyayev
- 18. @PavelChunyayev
- 19. @PavelChunyayev
- 20. @PavelChunyayev
- 22. @PavelChunyayev What kind of feedback is possible in CD? Plan Code Build Test Release OperateEnvision
- 23. @PavelChunyayev What kind of feedback does development need? Plan Code Build Test Release OperateEnvision
- 24. @PavelChunyayev Plan Code Build Test Release OperateEnvision
- 25. @PavelChunyayev What kind of feedback does business need? Plan Code Build Test Release OperateEnvision
- 26. @PavelChunyayev Plan Code Build Test Release OperateEnvision
- 27. @PavelChunyayev Plan Code Build Test Release OperateEnvision
- 28. @PavelChunyayev
- 30. @PavelChunyayev Feedback from unit testing
- 31. @PavelChunyayev Feedback from integration testing
- 32. @PavelChunyayev Feedback from contract testing
- 33. @PavelChunyayev Feedback from end-to-end testing
- 34. @PavelChunyayev
- 41. @PavelChunyayev Don’t break the contract!
- 45. @PavelChunyayev Contracts testing Performance testing Exploratory testing Resilience testing
- 46. @PavelChunyayev
- 47. @PavelChunyayev
- 48. @PavelChunyayev
- 49. @PavelChunyayev
- 50. @PavelChunyayev
- 51. @PavelChunyayev
- 53. @PavelChunyayev Who creates tests? Instability and slowliness
- 54. @PavelChunyayev Who creates tests? Instability and slowliness Release vs deployment
- 56. @PavelChunyayev Continuous Delivery Keep the product releasable Build quality in Frequent, incremental releases Continuous improvement Plan Code Build Test Release OperateEnvision
- 57. @PavelChunyayev Thank you :) Keep the product releasable Build quality in Frequent, incremental releases Continuous feedback +31 6 2117 5817 pavel@levi9.com Plan Code Build Test Release OperateEnvision
Editor's Notes
- Questions are welcome during the talk. Disagreement is welcome, but after the talk. Yes, but; what if.
- Over 700 employees now Picture of Belgrade is pending
- .we develop core software for (and with) these companies
- .
- .
- .
- Engineering Biology Economics Acoustics Software
- Adaptive systems Positive feedback loops Reinforce, amplify Generate growth Facilitate desired behavior Negative feedback loops Stop undesired behavior Degrade, reduce
- .
- Not every feature is needed. Shorter and shorter release cycles => Release more software in less time.
- It’s all about quality. Increasing speed by ensuring quality - positive feedback loop
- .
- Build – Test – Release
- First and the most important
- Reproducible and repeatable process (including testing). Keeping the product releasable
- .
- Run over and over again.
- .
- No DTAP. Immutable (testing framework?) TTL Docker Framework to allow restart - select arbitrary test
- BTR on every level Separation of testing. Move tests left.
- Feedback is in the foundation for Agile / Lean processes Frequently checking where you are Are you on track? Is everything safe? Do I need to correct anything? Any adjustments needed? Comes in various forms When we measure - we want to produce feedback.
- Business and development
- .
- Very important to get information from production to development. No FL - no visibility => silos Build – everything’s fine with code? Test – quality okay? Where is the problem? Release - integration? Operation - performance
- .
- Plan – when Code – happiness, velocity, etc. Build – green / red Test – are features there? Working as intended? Release - Operation – money, validation
- FL to Dev is important! Why? Vision, roadmap.
- .
- Building the thing right Building the right thing
- FL = Know exactly what is broken. 100% coverage? Test your unit tests.
- FL = how units are integrated? Connect units together How parts work together Communication
- FL = Black box testing. How service works as a whole. Different integrations – pub/sub, request/reply.
- FL = integration Proof we are still on track Not only selenium Focus on: business flows personas, user journeys BDD + DSL? Data independency
- Unit + Integration – level of microservices. Contract + E2E – level of system Different kind of feedback on every level
- User vs producer testing? Versioning? Ideally – not change API ever Once published, customer start using, impact on customer is huge. Better to do it right from the first try. * Semantic or not semantic * API versioning - public or also internal? * Transition between versions – testing different versions * Running several versions together
- .
- .
- .
- .
- .
- .
- How fast does the page load? API Response How many users can one service handle? And in degraged mode? Can we autoscale under load? Bursts of traffic?
- Manual regression / exploratory tests are okay They train people to think structurally into how the application is tested But don’t make them part of the process. Exploratory testing is very important! Good tester (security tester) will consider tests that can’t be automated!
- Manual regression / exploratory tests are okay They train people to think structurally into how the application is tested But don’t make them part of the process. Exploratory testing is very important! Good tester (security tester) will consider tests that can’t be automated!
- .
- .
- .
- .
- .
- .
- .
- Ideally – developer However average developer is bad at writing tests Same with security testing Role of tester is changing - facilitator We need to arrange learning for developers Look through the eyes of the user Try to break things
- Don’t accept flaky tests Remove them Update them Fight the slowness Parallelization TTL
- Deployment is an act of putting code (binary) to the server Release - the new code starts working - users can experience it
- Adaptive systems. Create feedback loops that will facilitate the behavor we want.
- Feedback is very important.