Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Author
Father
Agile & Lean coach
www.crisp.se
Consultant
Henrik Kniberg
henrik.kniberg@crisp.se
@HenrikKniberg
What is an ...
00:22
Agile tester mindset
Henrik Kniberg
Henrik Kniberg
Henrik Kniberg
Henrik Kniberg
Case study: Game development company
Concept
pres.
Resource
planning
Graphics
design
Sound
design
Dev
Test &
deploy
1m
4h
...
Before
Concept
pres.
Resource
planning
Graphics
design
Sound
design
Dev
Integr. &
deploy
1m
4h
6m
8
Game backlog
1w 6m 6m
...
Cross-functional teams
Henrik Kniberg
9
Dave
Joe
 Lisa
Dave
Joe
Lisa
January
 February
 March
 April
 May
 June
 July
6 mo...
Henrik Kniberg
Test phase
 Test team
Tester role
Henrik Kniberg
Req
 Code
 Test
Test phase
Henrik Kniberg
Test team
Programmers
 Testers
We own
quality
Dev team
Tester
Henrik Kniberg
Tester role
I own
quality
Henrik Kniberg
Cross-functional development team
QA
 = Quality Assurance
= Quality Assistance
Role 
 Competency
We own
qua...
Roles
Henrik Kniberg
Backend
 Test
DB
GUI
Design
Competencies
Henrik Kniberg
Design
Backend
DB
GUI
Test
Cross functional team
Doesn’t mean everyone has to know everything
Henrik Kniberg
DB
 Test
Web
Java
 Domain
 CM
Lisa
Joe
F...
Deliver, measure, adjust
continuously
How do you know that your product works?
Henrik Kniberg
ensure
1. Understand the
pro...
00:22
Typical activities
Henrik Kniberg
Make sure backlog items are testable & valuable
Henrik Kniberg
As a buyer
I want to save my shopping cart
so that I can co...
Find defects early!
Henrik Kniberg
Age of defect
$
Cost of
defect
Henrik Kniberg
Developers
Testers
Developers & Testers
Meeting room
Sit with the developers
Shorten the feedback loop
Henrik Kniberg
# of
handoffs
0
1
2
3
4
5
Length of feedback cycle
minutes
 hours
 days
 weeks
 m...
Push for Continuous Delivery
Henrik Kniberg
Test &
integrate
Deploy to
staging
Deploy
to prod
Manual
test
Manual
 Code &
c...
Create a shared vocabulary
Henrik Kniberg
Unit
Test?
Quality?
Integration
test?
Acceptance
test?
 Technical
debt?
Set working agreements for test automation
Each user story has at least
one black-box acceptance test
Almost all code is
c...
Do & teach exploratory testing
Henrik Kniberg
!
?
Done includes ”no added technical debt”
Henrik Kniberg
Backlog
 Ready
for dev
Ready for
production
- No added tech debt
In...
Example: Test automation backlog
•  Change skin 
•  Security alert
•  Transaction history 
•  Block account 
•  Add new us...
Step 2: Classify each test
Henrik Kniberg
Test case
Change skin
Security alert
Transaction
history
Block account
Add new u...
Step 3: Sort the list
Henrik Kniberg
Test case Risk
Manual Test
Cost
Automation
Cost
Block
account high 5 hrs low
Validate...
Example: Tech backlog
Henrik Kniberg
Product backlog
Tech backlog
Reserve X% of team capacity for the tech backlog
Henrik Kniberg
Tech backlog
Product backlog
Sprint
80%
20%
00:22
Example:
Spotify
Henrik Kniberg
Henrik Kniberg
Play Everywhere!
Like a magical music player in which
you’ve bought every song in the world!
>400 people in tech
Henrik Kniberg
36
Stockholm

Gothenburg

New York

San Francisco
> 50 squads
Henrik Kniberg
Henrik Kniberg
Autonomous Squad
Cross-functional, co-located, self-organizing team
39
Squads are grouped into Tribes
Henrik Kniberg
Tribe
 Tribe
 Tribe
Tribe
Tribe
 Tribe
Tribe
 Tribe
Chapter
Chapter
Chapter
Chapter
Guild
Each Tribe is a lightweight matrix focused on delivery
Vertical = Deliv...
Reality is messy
Henrik Kniberg
Decoupling to enable
frequent releases
Henrik Kniberg
Feature squads
Client App squad
!#?
Self-service model
Henrik Kniberg
Client App squads
IOS
 Android
 Desktop
 Web
Feature squads
Infrastructure squads
Enable...
Henrik Kniberg
Release trains & Feature toggles
Failure Recovery is more important
than Failure Avoidance
Henrik Kniberg
Failure Recovery
Failure Avoidance
“Limited Blast Radius” via decoupled architecture
Henrik Kniberg
”Limited Blast Radius” via gradual rollout
Henrik Kniberg
Trust > Control
100% control = 0% motion
Henrik Kniberg
If everything’s under control,
you’re going too slow!
- Mario Andr...
Henrik Kniberg
Analyze data
Narrative &
Prototypes &
Metrics
Build MVP
Deploy
Tweak 
“Radio you
can save!”
A/B stats
Idea/...
00:22
Example:
Big Government Project
Henrik Kniberg
Team structure - before
Henrik Kniberg
52
3
Development
teams
Test
team
Requirements
analyst
team
?
%#
!
!
?
%#
!
!
Henrik Kniberg
53
Next 10
features
Ideas
 Features
 Development
 System
test
User
acceptance
test
Production
Henrik Kniberg
54
Next 10
features
Ready for
sys test
Dev
in progress
Sys test
progress
Team
swimlanes
Henrik Kniberg
 55
Dev Team 1
 Dev Team 2
 Dev Team 3
Next 10
features
Dev
in progress
Ready for
sys test
Team
swimlanes
Henrik Kniberg
56
”Daily cocktail party” 9:15 – 10:15
Henrik Kniberg
57
Feature team 1
 Feature team 2
 Feature team 3
9:30 – 9:45
 9:15 – 9:30
9:30 – 9:45
9:45 – 10:00
Test
sy...
Henrik Kniberg
58
Count
cards
Velocity per week
Example: Measuring velocity by counting cards
Henrik Kniberg
60
60
Total
# of 
delivered
features
Week
Example: Release planning using a burnup chart
All of these
will ...
Henrik Kniberg
61
”Oh no, bottleneck
in System Test!
FLOW
Tech stories
Henrik Kniberg
62
Next 5 tech
stories
Next 10
features
Henrik Kniberg
63
Bottleneck
”Let’s stop
building new
features”
”... and focus on
test automation!”
Henrik Kniberg
64
Everyone
doing tech
stories
Henrik Kniberg
65
Top 3 recurring bugs
Henrik Kniberg
66
Definition of
”ready for
development”
Definition of
”ready for
system test”
Henrik Kniberg
67Henrik Kniberg 67
Henrik Kniberg
68
Test Fix %&@#!
Release
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Release
Week 1 Week 2 Wee...
Bug fixing process
Henrik Kniberg
69
Bug found!
More important
than any of the
current top 30?
No
Write sticky-note,
find ...
Three input queues
Henrik Kniberg
70
Next 5 tech
stories
Next 10
features
Next 5 lower
priority bugs
00:22
Wrapup
Henrik Kniberg
What is an Agile Tester?
Henrik Kniberg
•  An agile team member with testing expertise
•  Helps the team become quality-aw...
Mindset
Henrik Kniberg
Quality
Assurance
Quality
Assistance
Manual test
Functional
test
Requirements
Automatic
test
Explor...
Agile is a direction, not a place
Henrik Kniberg
The important thing isn’t
how you work.
The important thing is
how you im...
Upcoming SlideShare
Loading in …5
×

'What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

2,841 views

Published on

Crisp Lean and Agile Coach henrik Kniberg's presentation delivered at the Agile QA technical track at the Colombo Agile Conference 2014

Published in: Software

'What is an Agile tester': Henrik Kniberg @ Colombo Agile Conference 2014

  1. 1. Author Father Agile & Lean coach www.crisp.se Consultant Henrik Kniberg henrik.kniberg@crisp.se @HenrikKniberg What is an Agile Tester? Colombo Agile Conf, June 2014
  2. 2. 00:22 Agile tester mindset Henrik Kniberg
  3. 3. Henrik Kniberg
  4. 4. Henrik Kniberg
  5. 5. Henrik Kniberg
  6. 6. Case study: Game development company Concept pres. Resource planning Graphics design Sound design Dev Test & deploy 1m 4h 6m 8 Game backlog 1w 6m 6m 15 Design-ready games 12 Production-ready games 1m 3w 3m 3w1d (1m+2m) Actual work: 3 months Time to market: 25 months
  7. 7. Before Concept pres. Resource planning Graphics design Sound design Dev Integr. & deploy 1m 4h 6m 8 Game backlog 1w 6m 6m 15 Design-ready games 12 Production-ready games 1m 3w 3m (1m+2m) 3w1d Cross-functional game teams Game team (graphics, sound, dev, test, deploy, etc) Time to market: 3-4 months After Actual work: 3 months Time to market: 25 months 7 times faster Better games Less Planning More fun
  8. 8. Cross-functional teams Henrik Kniberg 9 Dave Joe Lisa Dave Joe Lisa January February March April May June July 6 months 3 months Release Release We’re alot faster! I’m a bit slower We’re slow! I’m fast!
  9. 9. Henrik Kniberg Test phase Test team Tester role
  10. 10. Henrik Kniberg Req Code Test Test phase
  11. 11. Henrik Kniberg Test team Programmers Testers We own quality
  12. 12. Dev team Tester Henrik Kniberg Tester role I own quality
  13. 13. Henrik Kniberg Cross-functional development team QA = Quality Assurance = Quality Assistance Role Competency We own quality
  14. 14. Roles Henrik Kniberg Backend Test DB GUI Design
  15. 15. Competencies Henrik Kniberg Design Backend DB GUI Test
  16. 16. Cross functional team Doesn’t mean everyone has to know everything Henrik Kniberg DB Test Web Java Domain CM Lisa Joe Fred Jenny David Erik Product Backlog Skills Needed to implement Top X backlog items I’m good at Test! I can do Java, but I’m not so good at it. I don’t know CM at all. But I’m willing to learn! I won’t even go near a database! Knows a bit about many things Knows a lot about one thing
  17. 17. Deliver, measure, adjust continuously How do you know that your product works? Henrik Kniberg ensure 1. Understand the problem 2. Iterate until you’ve solved it Who are the stakeholders? What need do they have, that we want to solve? How will we know when we’ve solved it? How will we know if we’re moving in the right direction? Minimize the distance to MVP
  18. 18. 00:22 Typical activities Henrik Kniberg
  19. 19. Make sure backlog items are testable & valuable Henrik Kniberg As a buyer I want to save my shopping cart so that I can continue shopping later How to demo: 1)  Enter store 2)  Put a book in shopping cart 3)  Press ”save cart” 4)  Leave store, and enter it again 5)  Check that the book is in my cart
  20. 20. Find defects early! Henrik Kniberg Age of defect $ Cost of defect
  21. 21. Henrik Kniberg Developers Testers Developers & Testers Meeting room Sit with the developers
  22. 22. Shorten the feedback loop Henrik Kniberg # of handoffs 0 1 2 3 4 5 Length of feedback cycle minutes hours days weeks months years Maker User 1 2 3 People (# of handoffs) Time (Feedback delay)
  23. 23. Push for Continuous Delivery Henrik Kniberg Test & integrate Deploy to staging Deploy to prod Manual test Manual Code & commit Build Automatic
  24. 24. Create a shared vocabulary Henrik Kniberg Unit Test? Quality? Integration test? Acceptance test? Technical debt?
  25. 25. Set working agreements for test automation Each user story has at least one black-box acceptance test Almost all code is covered by SOME kind of test (AT or UT) We know which code isn’t covered and have a good reason for it Henrik Kniberg AT and UT complement each other •  Acceptance tests: coupled to UI, decoupled from internal design •  Unit tests: coupled to internal design, decoupled from UI Complex code has unit tests
  26. 26. Do & teach exploratory testing Henrik Kniberg ! ?
  27. 27. Done includes ”no added technical debt” Henrik Kniberg Backlog Ready for dev Ready for production - No added tech debt In progress
  28. 28. Example: Test automation backlog •  Change skin •  Security alert •  Transaction history  •  Block account  •  Add new user  •  Sort query results   •  Deposit cash   •  Validate transfer   Henrik Kniberg Step 1: Decide what needs to be tested
  29. 29. Step 2: Classify each test Henrik Kniberg Test case Change skin Security alert Transaction history Block account Add new user Sort query results Deposit cash Validate transfer Risk Manual Test Cost Automation Cost low 0.5 hrs high high 1 hrs high med 3 hrs low high 5 hrs low low 0.5 hrs low med 2 hrs medium high 1.5 hrs low high 3 hrs medium Pay every time Pay once
  30. 30. Step 3: Sort the list Henrik Kniberg Test case Risk Manual Test Cost Automation Cost Block account high 5 hrs low Validate transfer high 3 hrs medium Transaction history med 3 hrs low Sort query results med 2 hrs medium Deposit cash high 1.5 hrs low Security alert high 1 hr high Add new user low 0.5 hrs low Change skin low 0.5 hrs high Automate first! Automate later Don’t bother automating
  31. 31. Example: Tech backlog Henrik Kniberg Product backlog Tech backlog
  32. 32. Reserve X% of team capacity for the tech backlog Henrik Kniberg Tech backlog Product backlog Sprint 80% 20%
  33. 33. 00:22 Example: Spotify Henrik Kniberg
  34. 34. Henrik Kniberg Play Everywhere! Like a magical music player in which you’ve bought every song in the world!
  35. 35. >400 people in tech Henrik Kniberg 36 Stockholm Gothenburg New York San Francisco
  36. 36. > 50 squads Henrik Kniberg
  37. 37. Henrik Kniberg Autonomous Squad Cross-functional, co-located, self-organizing team
  38. 38. 39
  39. 39. Squads are grouped into Tribes Henrik Kniberg Tribe Tribe Tribe Tribe Tribe Tribe
  40. 40. Tribe Tribe Chapter Chapter Chapter Chapter Guild Each Tribe is a lightweight matrix focused on delivery Vertical = Delivery. Horizontal = knowledge sharing & personal development
  41. 41. Reality is messy Henrik Kniberg
  42. 42. Decoupling to enable frequent releases Henrik Kniberg Feature squads Client App squad !#?
  43. 43. Self-service model Henrik Kniberg Client App squads IOS Android Desktop Web Feature squads Infrastructure squads Enable & support Enable & support Enable & support
  44. 44. Henrik Kniberg Release trains & Feature toggles
  45. 45. Failure Recovery is more important than Failure Avoidance Henrik Kniberg Failure Recovery Failure Avoidance
  46. 46. “Limited Blast Radius” via decoupled architecture Henrik Kniberg
  47. 47. ”Limited Blast Radius” via gradual rollout Henrik Kniberg
  48. 48. Trust > Control 100% control = 0% motion Henrik Kniberg If everything’s under control, you’re going too slow! - Mario Andretti
  49. 49. Henrik Kniberg Analyze data Narrative & Prototypes & Metrics Build MVP Deploy Tweak “Radio you can save!” A/B stats Idea/Problem “Follow your favorite artist”
  50. 50. 00:22 Example: Big Government Project Henrik Kniberg
  51. 51. Team structure - before Henrik Kniberg 52 3 Development teams Test team Requirements analyst team ? %# ! ! ? %# ! !
  52. 52. Henrik Kniberg 53 Next 10 features Ideas Features Development System test User acceptance test Production
  53. 53. Henrik Kniberg 54 Next 10 features Ready for sys test Dev in progress Sys test progress Team swimlanes
  54. 54. Henrik Kniberg 55 Dev Team 1 Dev Team 2 Dev Team 3 Next 10 features Dev in progress Ready for sys test Team swimlanes
  55. 55. Henrik Kniberg 56 ”Daily cocktail party” 9:15 – 10:15
  56. 56. Henrik Kniberg 57 Feature team 1 Feature team 2 Feature team 3 9:30 – 9:45 9:15 – 9:30 9:30 – 9:45 9:45 – 10:00 Test sync Requirements sync Dev sync 9:45 – 10:00 Project sync 10:00 – 10:15
  57. 57. Henrik Kniberg 58 Count cards Velocity per week Example: Measuring velocity by counting cards
  58. 58. Henrik Kniberg 60 60 Total # of delivered features Week Example: Release planning using a burnup chart All of these will be done Some of these will be done, but not all None of these will be done
  59. 59. Henrik Kniberg 61 ”Oh no, bottleneck in System Test! FLOW
  60. 60. Tech stories Henrik Kniberg 62 Next 5 tech stories Next 10 features
  61. 61. Henrik Kniberg 63 Bottleneck ”Let’s stop building new features” ”... and focus on test automation!”
  62. 62. Henrik Kniberg 64 Everyone doing tech stories
  63. 63. Henrik Kniberg 65 Top 3 recurring bugs
  64. 64. Henrik Kniberg 66 Definition of ”ready for development” Definition of ”ready for system test”
  65. 65. Henrik Kniberg 67Henrik Kniberg 67
  66. 66. Henrik Kniberg 68 Test Fix %&@#! Release Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Release Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Test at end: Test %&@#! Fix Fix Time saved! Test Test continuously: Before: Test at end Now: Test continuously
  67. 67. Bug fixing process Henrik Kniberg 69 Bug found! More important than any of the current top 30? No Write sticky-note, find developer, fix now! Yes Replace one of the other top 30 bugs with this one Yes Ignore it No Don’t log it. Fix it NOW! Blocker?
  68. 68. Three input queues Henrik Kniberg 70 Next 5 tech stories Next 10 features Next 5 lower priority bugs
  69. 69. 00:22 Wrapup Henrik Kniberg
  70. 70. What is an Agile Tester? Henrik Kniberg •  An agile team member with testing expertise •  Helps the team become quality-aware •  ... and learn how to deliver better stuff
  71. 71. Mindset Henrik Kniberg Quality Assurance Quality Assistance Manual test Functional test Requirements Automatic test Exploratory test Customer needs Late involvement Early involvement Long feedback loop Short feedback loop Find defects Prevent defects
  72. 72. Agile is a direction, not a place Henrik Kniberg The important thing isn’t how you work. The important thing is how you improve the way you work!

×