Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Testing as Exploration
aka. Continuous Delivery without
Automation
Maaret Pyhäjärvi
Email: <maaret@iki.fi> | Twitter: maar...
1. LET’S TALK ABOUT
EXPLORATORY TESTING
?
Things Can Look Different from
Different Perspectives
Realizations about Nature of
Testing
20
16
1639
5±2
4
Exploratory Testing:
Better tests, better testers!
• An approach, not a technique
• Find unknown unknowns
• Disciplined
• ...
Exploratory Testing:
Learning & Modeling
”A day’s work”
Vision (“Sandbox”) Current Charter
Other Charters Details
Bug
Repo...
2. CONTINUOUS DELIVERY
WITHOUT AUTOMATION
Where
does this
say fast?
Where does
this say
automation
?
A Bit of Background
• 3 years into a web based (.NET/C#)
product
• 1:8 tester:developer ratio
• Missing agile technical pr...
Going for Continuous Delivery
The Main Driver for Change:
Testing
Scheduled release
Feature 1
Feature 2
Feature 3
Feature 4
Testing
Feature 1
Feature 2
...
Enablers that Made Timing Just
Right
• Henri Karhatsu and #NoEstimates –
experience report
• Availability of Git in Visual...
Changing How We Managed Our
Work: Setting up Jira Kanban
Removed
hour &
story point
estimates
Agreed in
WIP for
each
phase...
Branches and Test
Environments
Feature Test
Environment
Feature branches
on-demand builds
“Developers think
this can be te...
Continuous Deployment but
Significant Lead Times
Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri
Change to
Master
Integration
to m...
Exploratory Testing Enables
Continuous Deployment
• Every Jira task gets planned for
– Sometimes we go with developer
test...
Changes
• Active discussion about schedules and
merging, and needs of testing in the
branches
• More pairing for testing t...
Testers don’t break
your code, they
break your illusions
about the code.
-- adapted from James Bach
Upcoming SlideShare
Loading in …5
×

Agile San Diego: Testing as Exploration (Continuous Delivery w/o Automation)

755 views

Published on

Talk 2.4.2015 at San Diego Agile User Group.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Agile San Diego: Testing as Exploration (Continuous Delivery w/o Automation)

  1. 1. Testing as Exploration aka. Continuous Delivery without Automation Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
  2. 2. 1. LET’S TALK ABOUT EXPLORATORY TESTING ?
  3. 3. Things Can Look Different from Different Perspectives
  4. 4. Realizations about Nature of Testing 20 16 1639 5±2 4
  5. 5. Exploratory Testing: Better tests, better testers! • An approach, not a technique • Find unknown unknowns • Disciplined • Test is a performance, not artifact – Artifacts support human memory – Many forms: e.g. checklists and automation • Exploratory performance testing, Exploratory test automation, Exploratory regression testing Test-related learning Design of new tests Test execution Result interpretation 5
  6. 6. Exploratory Testing: Learning & Modeling ”A day’s work” Vision (“Sandbox”) Current Charter Other Charters Details Bug Reports Perception of quality and coverage Quality ReportDebriefing Tester Test Manager Past Results Obstacles Outlook Feelings ? # xCharter backlog of the future testing Out of budget Next in importance! #, ?, x, + 20:20:60 Session sheets of the past testing Idea of exploration Metrics summary Coachin g 6 Playbooks Coverage outlines
  7. 7. 2. CONTINUOUS DELIVERY WITHOUT AUTOMATION
  8. 8. Where does this say fast? Where does this say automation ?
  9. 9. A Bit of Background • 3 years into a web based (.NET/C#) product • 1:8 tester:developer ratio • Missing agile technical practices: test automation non-existent • Scrum-like monthly releases for first 2 years – Releases late – Releases not tested well enough • Negative cycle of failing with estimates eating team morale • Jira in a central role with estimates and burndown
  10. 10. Going for Continuous Delivery
  11. 11. The Main Driver for Change: Testing Scheduled release Feature 1 Feature 2 Feature 3 Feature 4 Testing Feature 1 Feature 2 Feature 3 Feature 4 R1 R2 R3 Done includes Tested Schedule over Quality SCRUM + SCHEDULED DELIVERY with continuous integration KANBAN + CONTINUOUS DELIVERY
  12. 12. Enablers that Made Timing Just Right • Henri Karhatsu and #NoEstimates – experience report • Availability of Git in Visual Studio without plugins • Problems with schedules in Scrum • Lean and value delivery focus in Product Management
  13. 13. Changing How We Managed Our Work: Setting up Jira Kanban Removed hour & story point estimates Agreed in WIP for each phase Agreed on swarming Updated issue types
  14. 14. Branches and Test Environments Feature Test Environment Feature branches on-demand builds “Developers think this can be tested” Development Test Environment Integration branch build-on-checkin “Developers think this can be released” Acceptance Test Environment Master branch with fixes and merges from integration on-demand builds “Release next morning”
  15. 15. Continuous Deployment but Significant Lead Times Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri Change to Master Integration to master Feature to integration Developers drive the decision on what they want feedback on
  16. 16. Exploratory Testing Enables Continuous Deployment • Every Jira task gets planned for – Sometimes we go with developer testing only – Sometimes we test extensively • Exploratory Planning – No set test cases – Talking to developers and reflecting to a model of of use created through earlier explorations – Actionable information first –principle – Production monitoring is an option for getting information
  17. 17. Changes • Active discussion about schedules and merging, and needs of testing in the branches • More pairing for testing the features • More group work on defining the features • Introducing Flowdock due to increased need to collaborate; integrating logs, emails • Focus on getting better (scope of test automation; refactoring; pairing and group work; individuals’ skills)
  18. 18. Testers don’t break your code, they break your illusions about the code. -- adapted from James Bach

×