Continuous Delivery without Test Automation @STPCon, San Diego
•Download as PPTX, PDF•
1 like•1,490 views
Talk 1.4.2015 at Software Testing Professionals Conference on Continuous Delivery without Automation.
More Related Content
What's hot
What's hot (20)
Similar to Continuous Delivery without Test Automation @STPCon, San Diego
Similar to Continuous Delivery without Test Automation @STPCon, San Diego (20)
More from Maaret Pyhäjärvi
More from Maaret Pyhäjärvi (20)
Recently uploaded
Recently uploaded (20)
Continuous Delivery without Test Automation @STPCon, San Diego
- 1. Continuous Delivery without Test Automation Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
- 2. A Bit of Background • 3 years into a web based (.NET/C#) product • 1:8 tester:developer ratio • Missing agile technical practices: test automation non-existent • Scrum-like monthly releases for first 2 years – Releases late – Releases not tested well enough • Negative cycle of failing with estimates eating team morale • Jira in a central role with estimates and burndown
- 3. Where does this say fast? Where does this say automation ?
- 5. The Main Driver for Change: Testing Scheduled release Feature 1 Feature 2 Feature 3 Feature 4 Testing Feature 1 Feature 2 Feature 3 Feature 4 R1 R2 R3 Done includes Tested Schedule over Quality SCRUM + SCHEDULED DELIVERY with continuous integration KANBAN + CONTINUOUS DELIVERY
- 6. Enablers that Made Timing Just Right • Henri Karhatsu and #NoEstimates – experience report • Availability of Git in Visual Studio without plugins • Problems with schedules in Scrum • Lean and value delivery focus in Product Management
- 7. Changing How We Managed Our Work: Setting up Jira Kanban Removed hour & story point estimates Agreed in WIP for each phase Agreed on swarming Updated issue types
- 8. Branches and Test Environments Feature Test Environment Feature branches on-demand builds “Developers think this can be tested” Development Test Environment Integration branch build-on-checkin “Developers think this can be released” Acceptance Test Environment Master branch with fixes and merges from integration on-demand builds “Release next morning”
- 9. Continuous Deployment but Significant Lead Times Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri Change to Master Integration to master Feature to integration Developers drive the decision on what they want feedback on
- 10. Things Can Look Different from Different Perspectives
- 11. Exploratory Testing: Better tests, better testers! • An approach to software testing – Emphasized freedom and responsibility of an individual in a process where continuous optimization of value of information is important • “Any testing process that involves simultaneous (mutually supporting) learning, test design, and execution.” – James Bach & Cem Kaner • Disciplined, planned and controlled testing that emphasizes continuous learning • Research in Finland, Itkonen et al. 2007 – No significant difference in results for preplanned test cases and exploratory testing – More false alarms with test cases – Comparing overall effort: significantly more in test-case based testing Unknown territory Test-related learning Design of new tests Test execution Result interpretation 11
- 12. Exploratory Testing Enables Continuous Deployment • Every Jira task gets planned for – Sometimes we go with developer testing only – Sometimes we test extensively • Exploratory Planning – No set test cases – Talking to developers and reflecting to a model of of use created through earlier explorations – Actionable information first –principle – Production monitoring is an option for getting information
- 13. Exploratory Testing: Learning & Modeling ”A day’s work” Vision (“Sandbox”) Current Charter Other Charters Details Bug Reports Perception of quality and coverage Quality ReportDebriefing Tester Test Manager Past Results Obstacles Outlook Feelings ? # xCharter backlog of the future testing Out of budget Next in importance! #, ?, x, + 20:20:60 Session sheets of the past testing Idea of exploration Metrics summary Coachin g 13 Playbooks Coverage outlines
- 14. Changes • Active discussion about schedules and merging, and needs of testing in the branches • More pairing for testing the features • More group work on defining the features • Introducing Flowdock due to increased need to collaborate; integrating logs, emails • Focus on getting better (scope of test automation; refactoring; pairing and group work; individuals’ skills)
- 15. To End This With: Key Observations on Ideas that Guide Our Test Design
- 16. Testers don’t break your code, they break your illusions about the code. -- adapted from James Bach
Editor's Notes
- Mention background, Altom and seeking work in San Diego area for love.
- Fast as in few hours. Fast is embedded in responding to change: some change comes only from testing the value in real production environment. Just saying we can take best of the context we have and start living to the values.
- Value in production
- With 9 developers, it’s possible to end up having a feature per day ready for production…
- We want it to work in production. And with developer-tester perspectives, we’ve managed to make that true. We had recruited a tester because we knew (the devs) that they couldn’t see things from all the necessary angles. So they had asked for the feedback.
- Mention: exploratory regression testing, performance testing and test automation.
- Happy agile team with satisfied customers with collaboration and smart exploratory testing, not automation
- Remember to talk about tacit knowledge and the fact that there’s skills and heuristics that are teachable that testers tend to specialize on…
- Automation: db tests, selenium, unit tests
- The way we actually test is still the same: there’s no agile testing, there’s just testing in agile context.
- Happy agile team with satisfied customers with collaboration and smart exploratory testing, not automation