SlideShare a Scribd company logo
MI
AM Tutorial
10/13/2014 8:30:00 AM
"A Dozen Keys to Agile Testing
Maturity"
Presented by:
Bob Galen, Velocity Partners
Mary Thorn, ChannelAdvisor
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Bob Galen
Velocity Partners
An agile methodologist, practitioner, and coach based in Cary, NC, Bob Galen helps guide
companies in their adoption of Scrum and other agile methodologies and practices. Bob is a
principal agile evangelist at Velocity Partners, a leading agile nearshore development partner;
president of RGCG; and frequent speaker on software development, project management,
software testing, and team leadership at conferences and professional groups. He is a Certified
Scrum Coach, Certified Scrum Product Owner, and an active member of the Agile and Scrum
Alliances. In 2013 Bob published Scrum Product Ownership–Balancing Value from the Inside
Out. Reach him at bob@rgalen.com.
Mary Thorn
ChannelAdvisor
A QA director at ChannelAdvisor in Morrisville, NC, Mary Thorn has a broad testing
background that spans automation, data warehouses, and web-based systems in a wide variety
of technologies and testing techniques. During her more than seventeen years of experience in
healthcare, HR, financial, and SaaS-based products, Mary has held manager and contributor
level positions in software development organizations. She is a strong leader in agile testing
methodologies and has direct experience leading teams through agile adoption and beyond.
1
Keys for Transitioning to Agile
Testing
Myths & Realities from the Trenches
Bob Galen
bob@rgalen.com
Mary Thorn
marythorn@gmail.com
Copyright © 2014 RGCG, LLC 2
Introduction
Bob Galen
Independent Agile Coach (CSC) at RGCG, LLC
Principle Agile Evangelist at Velocity Partners
Somewhere ‘north’ of 30 years overall experience ☺
Wide variety of technical stacks and business domains
Developer first, then Project Management / Leadership, then
Testing
Senior/Executive software development leadership for 20 years
Practicing formal agility since 2000
XP, Lean, Scrum, and Kanban experience
From Cary, North Carolina
Connect w/ me via LinkedIn and Twitter @bobgalen
Bias Disclaimer:
Agile is THE BEST Methodology
for Software Development
However, NOT a Silver Bullet!
2
Copyright © 2014 RGCG, LLC 3
Introduction
Mary Thorn
Mary Thorn is the Director of QA and Agile Coach at
ChannelAdvisor in Morrisville, North Carolina,
Mary has a broad testing background that spans
automation, data warehouses, and web-based systems
in a wide variety of technologies and testing
techniques.
During her more than fifteen years of experience in
healthcare, HR, agriculture, and SaaS-based products,
Mary has held manager and contributor level positions
in software development organizations.
She has a strong interest in agile testing methodologies
and direct experience leading agile teams through
Scrum adoption & beyond.
Copyright © 2014 RGCG, LLC 4
3
Copyright © 2014 RGCG, LLC 5
Outline – Myths & Realities
Introduction
1. Transforming your
Team
2. Automation
3. Developers &
Automation
4. Developers Testing
5. Test Planning &
Scripts
6. Testing within the
Sprint
7. Exploratory Testing
8. Role of Testers
9. Developer to Tester
Workflow
10. Managing Agile Testers
11. Test Metrics
12. Retrospectives – The
Secret Sauce
13. Continuous Improvement
14. The Customer
15. Agile Requirements – The
Product Backlog
3-Pillars of Agile
Testing & Quality
Copyright © 2014 RGCG, LLC 6
#1, Transforming your team
Myth: You need all programmers or highly technical
testers when you move to agile
Reality: A mix is best –
Manual, domain-centric and technical skills
Some programming / scripting skills
Soft / collaborative skills
Reality: And throw out all of that Developer-to-Tester
ratio ‘stuff’.
4
Copyright © 2014 RGCG, LLC 7
#2, Automation
Myth: You need 100% automation to start
agile testing.
Reality: You simply need to have a strategy AND
doggedly pursue automation where it makes sense
Make it part of the Backlog and work it every sprint
Reality: There are some excellent Open Source tools
that supplement agile automation development
Reality: The Agile Test Automation Pyramid is the right
overall strategy
Agile Test Automation Pyramid
Mike Cohn; Lisa Crispin & Janet Gregory
http://behaviordrivendevelopment.wikispaces.com/Testing
Copyright © 2014 RGCG, LLC 8
5
Brainstorm…
Agile, Multi-tiered Automation
Get together in small groups of 4-6 to discuss
Take a few minutes and think about your current
automation approaches:
Tooling, approaches & strategies, strengths, weaknesses,
opportunities, maintenance challenges, future technology, etc.
What sorts of adjustments would you need to make to
take this approach?
What would be the largest challenges in taking this
approach? How might you overcome them?
Do you “buy” the whole-team view to automation?
Copyright © 2014 RGCG, LLC 9
Copyright © 2014 RGCG, LLC 10
#3, Developers & Automation
Myth: QA designs, writes & runs all of the test
automation
Reality: Everyone should be responsible for automation
Developers need to minimally attend to Unit Level
Participate in any framework or re-use development
Writing ‘glue’ code – fixtures, step files, etc.
Reality: It also extends into your Build & Continuous
Integration systems
All automation should be ‘wired’ into CI
Dashboards, trending, lava lamps, etc. for all to see
6
#4, Developers Testing
Myth: Developers can’t test their own code—they’re not
independent enough nor skilled enough to do it properly.
Reality: We need to stop stereotyping team members,
their strengths and their abilities.
Developers can absolutely test their own code.
Some are better at it than others
Pair with them to help test appropriately
Copyright © 2014 RGCG, LLC 11
Copyright © 2014 RGCG, LLC 12
#5, Test Planning & Scripts
Myth: You don’t need to plan
(it just happens )
and you don’t need functional test cases
(automation takes care of everything )
Reality: Plans help the team focus on the risk-based
testing required within an iteration AND across a release
Reality: Scripts (test cases) help formalize and drive
your testing;
Absolutely required in regulatory environments
Reality: You’ll never actually automate every test
7
Brainstorm…
Agile Planning & Execution
Get together in small groups of 4-6
Take a few minutes discuss your current planning and
test process mechanisms.
What would an Agile Test Plan “look like” in your organization?
What would Test Cases “look like”? What about progress
measures? And traceability?
Can you move from the “individual” to the “team”?
Be prepared to share
Copyright © 2014 RGCG, LLC 13
Copyright © 2014 RGCG, LLC 14
#6, Testing within the Sprint
Myth: You simply need to run 100%
of the tests within the constraints of the Sprint that’s
“Agile”
Reality: Rarely possible in most contexts.
You first need a high-degree of automation and business support
(for example: equipment costs)
Very mature test automation and CI / CD environments
Reality: Most agile teams adopt some sort of risk-based
testing approach for within the sprints
Dealing with Technical Test Debt
Then leverage Hardening / Stabilization pre-release sprints
8
Copyright © 2014 RGCG, LLC
The Agile Release Train
Synchronized
Iterate
Iterate
Team 1
Team 2
Team 3
Team 4
Iterate
Iterate
Harden Iterate Iterate Iterate
X-team
Harden
Harden
Harden
Harden
Iterate Iterate
Iterate Iterate
Iterate Iterate
Iterate Iterate Iterate Iterate
Iterate
Iterate
Iterate
Internal
Release
External
Release
Docs,
Training,
Support,
UAT,
Comp.
Team n
Continuous Integration
Continuous Integration
Continuous Integration
Continuous Integration
15
Copyright © 2014 RGCG, LLC
The Agile Release Train
Example: eCommerce / SaaS Model
Iterate
Iterate
Team 1
Team 2
Team 3
Team 4
Iterate
Iterate
Harden
X-team
Harden
Docs,
Training
Harden
Harden
Harden
Iterate Iterate
Iterate Iterate
External
Release
Team 8
Continuous Integration
Continuous Integration
Continuous Integration
Continuous Integration
10 days 10 days 5 + 2 days
Rinse
&
Repeat
Environment
Evolution Dev + QA Dev + QA QA -> Staging Production
16
9
Copyright © 2014 RGCG, LLC 17
The Agile Release Train
Example: iContact / SaaS Model
Team 1
Team 2
Team 3
Team 4
Iterate
Iterate
Harden
X-team
Harden
Docs,
Training
Harden
Harden
Harden
Iterate
Iterate
Production
Release
Team 10
Continuous Integration
Continuous Integration
Continuous Integration
Continuous Integration
3 weeks / 15 days 4-5 days
Rinse
&
Repeat
Environment
Evolution Dev + QA QA -> Staging Production
SBET, Exploratory –
Regression Testing
Brainstorm…
“Your” Agile Release Train
Get together in small groups of 4-6 to discuss
Take a few minutes and think about your current release
constraints
timing, customers, domain, competition, # of teams, technology,
etc.
Design a release train model for your organization
Overlay it with testing activities, plans, and milestones
Present it to your larger table/group; gain feedback &
adjust
Be prepared to share
Copyright © 2014 RGCG, LLC 18
10
#7, Exploratory Testing
Myth: There is no place for Session Based Exploratory
Testing in agile contexts.
Reality: ET and SBET are a beautiful complement to
agile testing.
Helping nurture pairing & collaboration across teams and
functions
Defining new (more valuable) test cases
Quickly gaining quality & usability feedback
Let’s explore the details of SBET
Copyright © 2014 RGCG, LLC 19
Copyright © 2014 RGCG, LLC 20
#8, Role of Testers
Myth: That the testers alone own quality & testing
practices within each team and sprint
Reality: The testers foster a “Whole Team” view
towards quality—focusing less on “Testing” and more on
“Quality Practices & the Customer”
Serving as guides for the team; Testing the “hard bits”
Facilitating exploratory testing sessions—finding more
interesting / valuable tests
Working with the Product Owners—are we solving the
customers problems?
11
Copyright © 2014 RGCG, LLC 21
#9, Developer to Tester Workflow
Myth: There is always a hand-off from developers to
testers; usually quite late in the sprint. That’s simply the
“way of things” in software development.
Reality: Scrummer-fall is alive and well but, Wrong!
Teams need to swarm on their work, as flow &
throughput matter the most.
WIP limits and close proximity / collaboration help establish a
healthy tempo of developer & tester pairing
Micro-handoffs – testing as development progresses!
Do you log bugs? Or do you fix bugs?
Copyright © 2014 RGCG, LLC 22
#10, Managing Agile Testers
Myth: The functional test manager is
in charge of deciding how, who, when , etc. for the test
team.
Reality: You still absolutely need functional leadership
within agile teams;
However, it’s focused towards quality practices, strategy &
coaching, and handling impediments / escalations
Encouraging transparency, transforming metrics & reporting
Supporting & protecting the teams
Encouraging risk-taking, innovation & creativity (Slack Time)
12
Levels of Done-Ness Criteria
Activity Criteria Example
Basic Team
Work Products
Done’ness criteria Pairing or pair inspections of code prior to check-in; or
development, execution and passing of unit tests.
User Story or
Theme Level
Acceptance Tests
Development of FitNesse based acceptance tests with the
customer AND their successful execution and passing.
Developed toward individual stories and/or themes for sets
of stories.
Sprint or
Iteration Level
Done’ness criteria Defining a Sprint Goal that clarifies the feature
development and all external dependencies associcated with
a sprint.
Release Level Release criteria
Defining a broad set of conditions (artifacts, testing
activities or coverage levels, results/metrics, collaboration
with other groups, meeting compliance levels, etc.) that IF
MET would mean the release could occur.
23Copyright © 2014 RGCG, LLC
Brainstorm…
“Your” Definition of Done
Get together in small groups of 4-6 to discuss
Using the 4-tier approach referenced start filling in the 4
levels as a group.
Consider any criteria you are currently using at your
companies?
Also consider current issues or challenge you might
have where “done-ness” would help?
And what about Ready-ness?
Be prepared to share
Copyright © 2014 RGCG, LLC 24
13
Copyright © 2014 RGCG, LLC 25
#11, Test Metrics
Myth: You can and should move
forward reporting everything exactly as
you have before.
Including any ‘dysfunctional’ metrics that your process and/or
PMO dictates.
Reality: The metrics should change immediately.
From QA and Test centric towards Team-Centric metrics (Value,
Throughput, Quality, Team)
Stop reporting out on “Testing”; it’s irrelevant!
This effects planning as well—estimation, progress, risk, etc.
Contribute quality-centric Information radiators to the mix
Copyright © 2014 RGCG, LLC 26
Brainstorm…
Morphing your Metrics
Get together in small groups of 4-6 to discuss
What are you measuring today? Why?
How are they driving your success and behaviors?
As you move to agile, what can/should you be
measuring in the 4 key areas:
Value, Quality, Throughput & Predictability, and Team?
How will you change your existing metrics? What
behaviors are you trying to inspire?
Be prepared to share
14
#12, Retrospectives:
The Secret Sauce
Myth: Testers are “Second Class” citizens who don’t
play an active part in the project & team
Reality: There are many places to “make a difference”
Getting the 800 lb. Gorillas out on the table; Showing courage;
telling truth
Fostering continuous improvement within the team
Setting the example; showing vulnerability—admitting you’re
wrong
Team listening; active planning; dependencies; pairing
Risk-taking; Failure!
Copyright © 2014 RGCG, LLC 27
#13, Continuous Improvement
Myth: We’re generally ‘stuck’ in our
approaches so just accept them and do
the “best you can”.
Reality: Continuous improvement is everyone’s
responsibility—to engage, suggest, take ownership of
current results, explore root causes, etc.
Active participation in your teams Retrospectives is a key way to
guide quality, testing, and customer-centric improvements.
Courage!
Copyright © 2014 RGCG, LLC 28
15
#14, The Customer
Myth: Business Analysts capture
customer requirements and testers test them for
completeness.
Reality: You need to begin to partner with the Customer
– Stakeholders – Product Owners to produce software
that solves the their problems.
Move to the “front” and help define & refine User Stories with your
Product Owner
Actively participate in Sprint Reviews
Show value for automation; placing test investments in the
Backlog
Copyright © 2014 RGCG, LLC 29
Copyright © 2014 RGCG, LLC 30
#15, Agile Requirements –
The Product Backlog
Myth: We can’t start testing until the requirements are
finished or stable; no matter how ‘agile’ we are.
Reality: Hogwash! Get over it
Ambiguity and incompleteness need to become your friend and
ally.
As does working with your Product Owners and Customers to
help define the requirements
Realizing that the requirements (User Stories) are only complete
at the end of each sprint.
16
Copyright © 2014 RGCG, LLC 31
Brainstorm…
Agile Requirements
Get together in small groups of 4-6 to discuss
Are iterative, are intentionally incomplete
The “older” the are, the larger and less defined they are
Enter the sprint at 70%, exit at 100%
Drive questions, dialogue, discussion, and collaboration;
think 3 Amigos or the Triad
So, WHY? And how will you make this work as a tester?
Be prepared to share
Agile Test Transformation Strategy:
3 Pillars of Agile Quality
Copyright © 2014 RGCG, LLC 32
17
3 Pillars of Agile Quality
Copyright © 2014 RGCG, LLC
33
Development & Test
Automation
• Pyramid-based Strategy:
(Unit + Cucumber +
Selenium)
• Continuous Integration
• Attack technical
infrastructure in the Backlog
• Visual Feedback –
Dashboards
• Actively practice ATDD and
BDD
Software Testing
• Risk-based testing:
Functional & Non-Functional
• Test planning @ Release &
Sprint levels
• Exploratory Testing
• Standards – checklists,
templates, repositories
• Balance across manual,
exploratory & automation
Cross-Functional Team
Practices
• Team-based Pairing
• Stop-the-Line Mindset
• Code Reviews & Standards
• Active Done-Ness
• Aggressive Refactoring of
Technical Debt
• User Stories, “3 Amigo”
based Conversations
• Whole Team Ownership of “Quality”
• Building it ‘Right’; Building the ‘Right’ Thing
• Healthy – Agile Centric Metrics
• Center of Excellence or Community of Practice
• Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement
Foundation of the 3 Pillars
Copyright © 2014 RGCG, LLC
34
• Whole Team Ownership of
“Quality”
• Building it ‘Right’; Building
the ‘Right’ Thing
• Healthy – Agile Centric
Metrics
• Center of Excellence or
Community of Practice
• Strategic balance across 3
Pillars; Assessment,
Recalibration, and
Continuous Improvement
• Whole team view includes building it right,
everyone tests,
• Focus on features/stories, confirmation,
conversation, and getting them staged
properly OVER testing
• 4-tier metrics: Quality, Value, Prediction, Team
• Agile strategies need light-handed “steering”;
establish a CoE (heavier weight) or a CoP
(lightweight)
• Consider finding an assessment framework
and then tying it to your strategy
measurement, recalibration, and continuous
improvement.
• Make the Foundation visible thru Information
Radiators and metrics
18
3 Pillars of Agile Quality
Copyright © 2014 RGCG, LLC
35
Development &
Test Automation
• Pyramid-based
Strategy: (Unit +
Cucumber + Selenium)
• Continuous Integration
• Attack technical
infrastructure in the
Backlog
• Visual Feedback –
Dashboards
• Actively practice ATDD
and BDD
A central part of agile adoption is focusing on CI, 3-
tiered Automation development, and Dashboards to
begin incrementally building coverage for faster
feedback on changes.
In the interim, Hardening or Stabilization Sprints and
having a risk-based Release Train concept help
It’s important that Test or QA not ‘own’ the tooling or
all of the automation efforts. The strategy can come
from Test, but the tactical automation development is
best left to the team.
Mature teams invest in automation as part of Done-
ness and continually on their backlogs
3 Pillars of Agile Quality
Copyright © 2014 RGCG, LLC
36
Software Testing
• Risk-based testing:
Functional & Non-
Functional
• Test planning @
Release & Sprint levels
• Exploratory Testing
• Standards – checklists,
templates, repositories
• Balance across
manual, exploratory &
automation
Exploratory Testing (Charter / Session based and
paired) can be an incredibly effective way to
establish a whole-team, collaborative view towards
quality and testing. It also emerges new tests.
Leverage ‘plans’ as a whole-team collaboration
mechanism and do plan.
Do not measure testing or tester progress; instead,
measure throughput, output, sprint outcomes, and
done-ness escapes at a team level.
You need a balanced test team; not everyone needs
to be able to program. But everyone needs to be
skilled testers.
Agile testing is a Risk-Based play in every Sprint and
across a release sequence. Don’t forget your
techniques!
19
3 Pillars of Agile Quality
Copyright © 2014 RGCG, LLC
37
Cross-Functional
Team Practices
• Team-based Pairing
• Stop-the-Line Mindset
• Code Reviews &
Standards
• Active Done-Ness
• Aggressive Refactoring
of Technical Debt
• User Stories – 3 Amigo
based Conversations
One of the hardest areas to get ‘right’ culturally. It
needs leadership alignment from Quality/Testing to
Product to Development and a consistent voice of
whole-team approaches.
This is where LEAN lives, where whole-team
collaboration happens, where professionalism and
craftsmanship are held dear.
I like the view of testers becoming the VOC,
champions of quality, and consistent questioners of
what is being build. Are we solving the right
problems as simply as possible. Notions of Minimal
Viable Product / Feature help with focus.
And yes Virginia, there ARE standards, templates,
and a focus on consistency!
Copyright © 2014 RGCG, LLC 3838
Software Testing
Strategies
It ALL starts with empowering testers AND creating a
Whole-Team view towards Quality
Critical Early Steps:
Creating a sense of empowered Functional Team
Applying Testing Standards across all teams
Deploying Exploratory Testing across all teams
Defining a core set of Agile KPI / metrics
ACTIVE participants in Sprint Planning
20
Copyright © 2014 RGCG, LLC 3939
Cross-Functional Team Practices
Strategies
Training
Agile / Lean in general, Story writing, Acceptance, Unit testing,
etc.
Teaming – for example: feedback or 5 Dysfunctions / Trust
Critical Early Steps:
Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP
Limits across teams
Define prescriptive and aggressive Done-Ness for ALL teams
Implement coding standards & Crucible / code reviews across the
center (appropriate for technology stacks)
Release Planning BEFORE allowing a team to start Sprint #1
Backlogs have Bug + Refactoring + Automation targets (20%)?
Copyright © 2014 RGCG, LLC 4040
Organizational Quality
Strategies
Continuously communicate your unified Vision
Your strategy must be aligned/shared across:
Development, Quality/Testing, and Product
Keep working your strategy across the pillars
Don’t get stuck with too narrow a focus (easy road)
Make your strategy visible (Information Radiators)
Show progress (Ex: burn up of test automation coverage across tiers)
Visualize organizational impediments to your Agile Quality
strategies
Attack them!
Quarterly read-outs on progress, plans and adjustments
Listen to your teams; Celebrate successes!
21
What will be (your) agile strategy when you get
back home?
Either in groups or individually
Consider the 3-Pillars discussion
Consider your current team / organization agile ‘state’
Define a broad, 3-pillar view towards some immediate
focus points when you get back into the office
What will you focus on? Why?
How will you communicate the need for change?
How will you measure results?
What will come immediately afterwards?
Copyright © 2014 RGCG, LLC 41
Wrapping up…
Agile is the best thing that’s
happened to testers since
The Great Depression
Whole Team view
Testing, Metrics, Automation
Planning, Reporting, Quality
Facilitate feedback
Multi-tiered automation
Just-in-Time, risk-based testing
Continuous improvement
Trust the Team
Retrospective
Copyright © 2014 RGCG, LLC 42
22
Contact Info
Bob Galen
Principal Consultant,
RGalen Consulting Group, L.L.C.
Experience-driven agile focused training,
coaching & consulting
Cell: (919) 272-0719
bob@rgalen.com www.rgalen.com
bgalen@velocitypartners.net www.velocitypartners.net
Blogs
Project Times - http://www.projecttimes.com/robert-galen/
BA Times - http://www.batimes.com/robert-galen/
Podcast on all things ‘agile’ - http://www.meta-cast.com/
43Copyright © 2014 RGCG, LLC 43
Additional Topics
Copyright © 2014 RGCG, LLC 44
23
Two Pillars of Lean ‘Thinking’
Respect for
People
Customer, Employees,
Vendors
Develop your teams
Trust & coach
No wasteful work
Continuous
Improvement
Embrace change,
challenge everything
Kaizen – small, incremental
change
Kaikaku – larger scale,
fundamental
45
From http://www.leanprimer.com
Copyright © 2014 RGCG, LLC
Agile Testing Quadrants
Brian Marick; Lisa Crispin & Janet Gregory
Copyright © 2014 RGCG, LLC 46
Exploratory testing
Scenarios
Usability testing
UAT
Alpha / Beta
Unit tests
Component tests
API tests
Functional tests
Story tests
Examples
Prototypes
Simulations
Performance testing
Load testing
Security testing
Non-functional requirements
Q1
Q2 Q3
Q4
Automated &
Manual
Automated &
Manual
Manual
Automation,
Tools, and
Manual
Business Facing
Technology Facing
SupportingtheTeam
CritiquetheProduct
24
Copyright © 2014 RGCG, LLC 47
10 Tenets of Agile Testing Jean Tabaka, Rally Software
1. The system always runs
2. Stop the line, vs. logging
defects
3. If it’s not tested, it’s not
“Done”
4. Testing comes first, not last
5. Finding defects after
Development is “Done” is too
late
Continuous Integration
Lean – fix it now!
Early feedback; Earned
Value
Collaborative testing, focus
on building in quality
Early feedback; fix it now!
Copyright © 2014 RGCG, LLC 48
10 Tenets of Agile Testing Jean Tabaka, Rally Software
6. “Development Complete” is
meaningless
7. Use testing, not analysis, to
explore requirements
8. Automation is “how” not a
“whether” or “when”
9. Tests are your second most
detailed specification
10. Testers are Customer-
Developer liaisons
Whole Team complete view
– no “partial credit”
Executable requirements
Automate all testing;
feedback
Code is first; later is
traditional specifications
VOC; guide effective team
collaboration; ask the right
questions
25
Copyright © 2014 RGCG, LLC 49
10 Commitments of Agile Testing Jean Tabaka,
Rally Software
1. We commit to not moving forward if a hole is found
through root cause analysis without first writing a test
2. We commit to not relying solely on just automated
testing or just manual testing
3. We commit to not sitting behind a QA wall (no
boundaries!)
4. We commit to not allowing a code complete without
test code harness complete
5. We commit to not waiting for a test phase but rather
working in smaller and smaller pieces, sooner and
sooner
Copyright © 2014 RGCG, LLC 50
10 Commitments of Agile Testing Jean Tabaka,
Rally Software
6. We commit to not testing one iteration after
development is “Done”
7. We commit to not allowing surprises to accumulate for
large end-to-end testing (“mock it now”)
8. We commit to not leaving the riskiest tests to the end
9. We commit to being an equal participant with the
customer and the developer in defining “Doneness”
10.We commit to not taking this oath lightly

More Related Content

What's hot

2012 04 18 Knoxville Agile Adoption&Adaptation
2012 04 18 Knoxville Agile Adoption&Adaptation2012 04 18 Knoxville Agile Adoption&Adaptation
2012 04 18 Knoxville Agile Adoption&Adaptation
alstonhodge
 
Agile DevOps: The Long, Ugly Story of How We Got Better
Agile DevOps: The Long, Ugly Story of How We Got BetterAgile DevOps: The Long, Ugly Story of How We Got Better
Agile DevOps: The Long, Ugly Story of How We Got Better
Tommy Norman
 
ScrumOps - Scrum + Practical DevOps
ScrumOps - Scrum + Practical DevOpsScrumOps - Scrum + Practical DevOps
ScrumOps - Scrum + Practical DevOps
Naveen Kumar Singh
 
Agile introduction for the American Chamber of Commerce members
Agile introduction for the American Chamber of Commerce membersAgile introduction for the American Chamber of Commerce members
Agile introduction for the American Chamber of Commerce membersAndy Brandt
 
QA team transition to agile testing at Alcatel Lucent
QA team transition to agile testing at Alcatel LucentQA team transition to agile testing at Alcatel Lucent
QA team transition to agile testing at Alcatel Lucent
AgileSparks
 
DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015Yuval Yeret
 
Agile Program and Portfolio Management
Agile Program and Portfolio ManagementAgile Program and Portfolio Management
Agile Program and Portfolio Management
Mike Cottmeyer
 
Introduction to DevOps and Kanban
Introduction to DevOps and KanbanIntroduction to DevOps and Kanban
Introduction to DevOps and Kanban
Yuval Yeret
 
Agile lean workshop for managers & exec leadership
Agile lean workshop for managers & exec leadershipAgile lean workshop for managers & exec leadership
Agile lean workshop for managers & exec leadership
Ravi Tadwalkar
 
2015 06-03 ti4 agile presented at ncs
2015 06-03 ti4 agile presented at ncs2015 06-03 ti4 agile presented at ncs
2015 06-03 ti4 agile presented at ncs
Ruud Teunissen
 
Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015
Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015
Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015
Yuval Yeret
 
LKIN2018: leveraging Lean and Kanban to implement continuous improvement
LKIN2018: leveraging Lean and Kanban to implement continuous improvementLKIN2018: leveraging Lean and Kanban to implement continuous improvement
LKIN2018: leveraging Lean and Kanban to implement continuous improvement
Ravi Tadwalkar
 
Agile Odyssey: Case Study of Agile Adoption within A Health Insurance Company
Agile Odyssey: Case Study of Agile Adoption within A Health Insurance CompanyAgile Odyssey: Case Study of Agile Adoption within A Health Insurance Company
Agile Odyssey: Case Study of Agile Adoption within A Health Insurance Company
alstonehodge
 
Test management in scrum
Test management in scrumTest management in scrum
Test management in scrumQA Club Kiev
 
Refactoring workshop
Refactoring workshop Refactoring workshop
Refactoring workshop
Itzik Saban
 
Practical DevOps
Practical DevOpsPractical DevOps
Practical DevOps
Naveen Kumar Singh
 
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex EnvironmentsDr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex EnvironmentsAgileSparks
 
DevOps or Devops - living in silos or living as a team
DevOps or Devops -  living in silos or living as a teamDevOps or Devops -  living in silos or living as a team
DevOps or Devops - living in silos or living as a team
Vinay Krishna
 
Agile Framework
Agile FrameworkAgile Framework
Agile FrameworkSubbuiyer
 
A confused tester in agile world finalversion
A confused tester in agile world finalversionA confused tester in agile world finalversion
A confused tester in agile world finalversion
Ashish Kumar
 

What's hot (20)

2012 04 18 Knoxville Agile Adoption&Adaptation
2012 04 18 Knoxville Agile Adoption&Adaptation2012 04 18 Knoxville Agile Adoption&Adaptation
2012 04 18 Knoxville Agile Adoption&Adaptation
 
Agile DevOps: The Long, Ugly Story of How We Got Better
Agile DevOps: The Long, Ugly Story of How We Got BetterAgile DevOps: The Long, Ugly Story of How We Got Better
Agile DevOps: The Long, Ugly Story of How We Got Better
 
ScrumOps - Scrum + Practical DevOps
ScrumOps - Scrum + Practical DevOpsScrumOps - Scrum + Practical DevOps
ScrumOps - Scrum + Practical DevOps
 
Agile introduction for the American Chamber of Commerce members
Agile introduction for the American Chamber of Commerce membersAgile introduction for the American Chamber of Commerce members
Agile introduction for the American Chamber of Commerce members
 
QA team transition to agile testing at Alcatel Lucent
QA team transition to agile testing at Alcatel LucentQA team transition to agile testing at Alcatel Lucent
QA team transition to agile testing at Alcatel Lucent
 
DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015
 
Agile Program and Portfolio Management
Agile Program and Portfolio ManagementAgile Program and Portfolio Management
Agile Program and Portfolio Management
 
Introduction to DevOps and Kanban
Introduction to DevOps and KanbanIntroduction to DevOps and Kanban
Introduction to DevOps and Kanban
 
Agile lean workshop for managers & exec leadership
Agile lean workshop for managers & exec leadershipAgile lean workshop for managers & exec leadership
Agile lean workshop for managers & exec leadership
 
2015 06-03 ti4 agile presented at ncs
2015 06-03 ti4 agile presented at ncs2015 06-03 ti4 agile presented at ncs
2015 06-03 ti4 agile presented at ncs
 
Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015
Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015
Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015
 
LKIN2018: leveraging Lean and Kanban to implement continuous improvement
LKIN2018: leveraging Lean and Kanban to implement continuous improvementLKIN2018: leveraging Lean and Kanban to implement continuous improvement
LKIN2018: leveraging Lean and Kanban to implement continuous improvement
 
Agile Odyssey: Case Study of Agile Adoption within A Health Insurance Company
Agile Odyssey: Case Study of Agile Adoption within A Health Insurance CompanyAgile Odyssey: Case Study of Agile Adoption within A Health Insurance Company
Agile Odyssey: Case Study of Agile Adoption within A Health Insurance Company
 
Test management in scrum
Test management in scrumTest management in scrum
Test management in scrum
 
Refactoring workshop
Refactoring workshop Refactoring workshop
Refactoring workshop
 
Practical DevOps
Practical DevOpsPractical DevOps
Practical DevOps
 
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex EnvironmentsDr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
 
DevOps or Devops - living in silos or living as a team
DevOps or Devops -  living in silos or living as a teamDevOps or Devops -  living in silos or living as a team
DevOps or Devops - living in silos or living as a team
 
Agile Framework
Agile FrameworkAgile Framework
Agile Framework
 
A confused tester in agile world finalversion
A confused tester in agile world finalversionA confused tester in agile world finalversion
A confused tester in agile world finalversion
 

Similar to A Dozen Keys to Agile Testing Maturity

Seven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing TransitionSeven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing Transition
TechWell
 
Seven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing TransitionSeven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing Transition
TechWell
 
The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test StrategyThe Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy
TechWell
 
The 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary Thorn
The 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary ThornThe 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary Thorn
The 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary Thorn
TEST Huddle
 
Test Automation Strategies for the Agile World
Test Automation Strategies for the Agile WorldTest Automation Strategies for the Agile World
Test Automation Strategies for the Agile World
TechWell
 
The Tester's Role in Agile Planning
The Tester's Role in Agile PlanningThe Tester's Role in Agile Planning
The Tester's Role in Agile Planning
TechWell
 
The Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing StrategyThe Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing Strategy
TechWell
 
The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy
Brian Estep
 
Agile foundation online - slide share
Agile  foundation   online - slide shareAgile  foundation   online - slide share
Agile foundation online - slide share
Raghavendra (Raghav) Mithare
 
Yin and Yang: Metrics within Agile and Traditional Lifecycles
Yin and Yang: Metrics within Agile and Traditional LifecyclesYin and Yang: Metrics within Agile and Traditional Lifecycles
Yin and Yang: Metrics within Agile and Traditional Lifecycles
TechWell
 
Where Did My Testers Go? Test Management on Agile Projects
Where Did My Testers Go? Test Management on Agile ProjectsWhere Did My Testers Go? Test Management on Agile Projects
Where Did My Testers Go? Test Management on Agile Projects
TechWell
 
The Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing StrategyThe Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing Strategy
TechWell
 
A Tester’s Guide to Collaborating with Product Owners
A Tester’s Guide to Collaborating with Product OwnersA Tester’s Guide to Collaborating with Product Owners
A Tester’s Guide to Collaborating with Product Owners
TechWell
 
Agile Testing at Scale
Agile Testing at ScaleAgile Testing at Scale
Agile Testing at Scale
TechWell
 
Agile Testing at Scale
Agile Testing at ScaleAgile Testing at Scale
Agile Testing at Scale
TechWell
 
Essential Patterns of Mature Agile Teams
Essential Patterns of Mature Agile TeamsEssential Patterns of Mature Agile Teams
Essential Patterns of Mature Agile Teams
TechWell
 
Practical Software Quality and Testing
Practical Software Quality and TestingPractical Software Quality and Testing
Practical Software Quality and Testingjerrykprague
 
Methodologies 3: Using Spira for Waterfall
Methodologies 3: Using Spira for WaterfallMethodologies 3: Using Spira for Waterfall
Methodologies 3: Using Spira for Waterfall
Inflectra
 
The agile way: the complete guide to understanding agile methodologies
The agile way:  the complete guide to understanding agile methodologiesThe agile way:  the complete guide to understanding agile methodologies
The agile way: the complete guide to understanding agile methodologies
QASymphony
 
Agile testing guide_2021
Agile testing guide_2021Agile testing guide_2021
Agile testing guide_2021
QMetry
 

Similar to A Dozen Keys to Agile Testing Maturity (20)

Seven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing TransitionSeven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing Transition
 
Seven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing TransitionSeven Keys to Navigating Your Agile Testing Transition
Seven Keys to Navigating Your Agile Testing Transition
 
The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test StrategyThe Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy
 
The 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary Thorn
The 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary ThornThe 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary Thorn
The 3 Pillars Approach to Agile Testing Strategy with Bob Galen & Mary Thorn
 
Test Automation Strategies for the Agile World
Test Automation Strategies for the Agile WorldTest Automation Strategies for the Agile World
Test Automation Strategies for the Agile World
 
The Tester's Role in Agile Planning
The Tester's Role in Agile PlanningThe Tester's Role in Agile Planning
The Tester's Role in Agile Planning
 
The Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing StrategyThe Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing Strategy
 
The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy
 
Agile foundation online - slide share
Agile  foundation   online - slide shareAgile  foundation   online - slide share
Agile foundation online - slide share
 
Yin and Yang: Metrics within Agile and Traditional Lifecycles
Yin and Yang: Metrics within Agile and Traditional LifecyclesYin and Yang: Metrics within Agile and Traditional Lifecycles
Yin and Yang: Metrics within Agile and Traditional Lifecycles
 
Where Did My Testers Go? Test Management on Agile Projects
Where Did My Testers Go? Test Management on Agile ProjectsWhere Did My Testers Go? Test Management on Agile Projects
Where Did My Testers Go? Test Management on Agile Projects
 
The Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing StrategyThe Three Pillars Approach to an Agile Testing Strategy
The Three Pillars Approach to an Agile Testing Strategy
 
A Tester’s Guide to Collaborating with Product Owners
A Tester’s Guide to Collaborating with Product OwnersA Tester’s Guide to Collaborating with Product Owners
A Tester’s Guide to Collaborating with Product Owners
 
Agile Testing at Scale
Agile Testing at ScaleAgile Testing at Scale
Agile Testing at Scale
 
Agile Testing at Scale
Agile Testing at ScaleAgile Testing at Scale
Agile Testing at Scale
 
Essential Patterns of Mature Agile Teams
Essential Patterns of Mature Agile TeamsEssential Patterns of Mature Agile Teams
Essential Patterns of Mature Agile Teams
 
Practical Software Quality and Testing
Practical Software Quality and TestingPractical Software Quality and Testing
Practical Software Quality and Testing
 
Methodologies 3: Using Spira for Waterfall
Methodologies 3: Using Spira for WaterfallMethodologies 3: Using Spira for Waterfall
Methodologies 3: Using Spira for Waterfall
 
The agile way: the complete guide to understanding agile methodologies
The agile way:  the complete guide to understanding agile methodologiesThe agile way:  the complete guide to understanding agile methodologies
The agile way: the complete guide to understanding agile methodologies
 
Agile testing guide_2021
Agile testing guide_2021Agile testing guide_2021
Agile testing guide_2021
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
TechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
TechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
TechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
TechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
TechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
TechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
TechWell
 
Ma 15
Ma 15Ma 15
Ma 15
TechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
TechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
TechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
TechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
TechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
TechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
TechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
TechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
TechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
TechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
TechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
TechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
CatarinaPereira64715
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 

Recently uploaded (20)

Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 

A Dozen Keys to Agile Testing Maturity

  • 1. MI AM Tutorial 10/13/2014 8:30:00 AM "A Dozen Keys to Agile Testing Maturity" Presented by: Bob Galen, Velocity Partners Mary Thorn, ChannelAdvisor Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Bob Galen Velocity Partners An agile methodologist, practitioner, and coach based in Cary, NC, Bob Galen helps guide companies in their adoption of Scrum and other agile methodologies and practices. Bob is a principal agile evangelist at Velocity Partners, a leading agile nearshore development partner; president of RGCG; and frequent speaker on software development, project management, software testing, and team leadership at conferences and professional groups. He is a Certified Scrum Coach, Certified Scrum Product Owner, and an active member of the Agile and Scrum Alliances. In 2013 Bob published Scrum Product Ownership–Balancing Value from the Inside Out. Reach him at bob@rgalen.com. Mary Thorn ChannelAdvisor A QA director at ChannelAdvisor in Morrisville, NC, Mary Thorn has a broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques. During her more than seventeen years of experience in healthcare, HR, financial, and SaaS-based products, Mary has held manager and contributor level positions in software development organizations. She is a strong leader in agile testing methodologies and has direct experience leading teams through agile adoption and beyond.
  • 3. 1 Keys for Transitioning to Agile Testing Myths & Realities from the Trenches Bob Galen bob@rgalen.com Mary Thorn marythorn@gmail.com Copyright © 2014 RGCG, LLC 2 Introduction Bob Galen Independent Agile Coach (CSC) at RGCG, LLC Principle Agile Evangelist at Velocity Partners Somewhere ‘north’ of 30 years overall experience ☺ Wide variety of technical stacks and business domains Developer first, then Project Management / Leadership, then Testing Senior/Executive software development leadership for 20 years Practicing formal agility since 2000 XP, Lean, Scrum, and Kanban experience From Cary, North Carolina Connect w/ me via LinkedIn and Twitter @bobgalen Bias Disclaimer: Agile is THE BEST Methodology for Software Development However, NOT a Silver Bullet!
  • 4. 2 Copyright © 2014 RGCG, LLC 3 Introduction Mary Thorn Mary Thorn is the Director of QA and Agile Coach at ChannelAdvisor in Morrisville, North Carolina, Mary has a broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques. During her more than fifteen years of experience in healthcare, HR, agriculture, and SaaS-based products, Mary has held manager and contributor level positions in software development organizations. She has a strong interest in agile testing methodologies and direct experience leading agile teams through Scrum adoption & beyond. Copyright © 2014 RGCG, LLC 4
  • 5. 3 Copyright © 2014 RGCG, LLC 5 Outline – Myths & Realities Introduction 1. Transforming your Team 2. Automation 3. Developers & Automation 4. Developers Testing 5. Test Planning & Scripts 6. Testing within the Sprint 7. Exploratory Testing 8. Role of Testers 9. Developer to Tester Workflow 10. Managing Agile Testers 11. Test Metrics 12. Retrospectives – The Secret Sauce 13. Continuous Improvement 14. The Customer 15. Agile Requirements – The Product Backlog 3-Pillars of Agile Testing & Quality Copyright © 2014 RGCG, LLC 6 #1, Transforming your team Myth: You need all programmers or highly technical testers when you move to agile Reality: A mix is best – Manual, domain-centric and technical skills Some programming / scripting skills Soft / collaborative skills Reality: And throw out all of that Developer-to-Tester ratio ‘stuff’.
  • 6. 4 Copyright © 2014 RGCG, LLC 7 #2, Automation Myth: You need 100% automation to start agile testing. Reality: You simply need to have a strategy AND doggedly pursue automation where it makes sense Make it part of the Backlog and work it every sprint Reality: There are some excellent Open Source tools that supplement agile automation development Reality: The Agile Test Automation Pyramid is the right overall strategy Agile Test Automation Pyramid Mike Cohn; Lisa Crispin & Janet Gregory http://behaviordrivendevelopment.wikispaces.com/Testing Copyright © 2014 RGCG, LLC 8
  • 7. 5 Brainstorm… Agile, Multi-tiered Automation Get together in small groups of 4-6 to discuss Take a few minutes and think about your current automation approaches: Tooling, approaches & strategies, strengths, weaknesses, opportunities, maintenance challenges, future technology, etc. What sorts of adjustments would you need to make to take this approach? What would be the largest challenges in taking this approach? How might you overcome them? Do you “buy” the whole-team view to automation? Copyright © 2014 RGCG, LLC 9 Copyright © 2014 RGCG, LLC 10 #3, Developers & Automation Myth: QA designs, writes & runs all of the test automation Reality: Everyone should be responsible for automation Developers need to minimally attend to Unit Level Participate in any framework or re-use development Writing ‘glue’ code – fixtures, step files, etc. Reality: It also extends into your Build & Continuous Integration systems All automation should be ‘wired’ into CI Dashboards, trending, lava lamps, etc. for all to see
  • 8. 6 #4, Developers Testing Myth: Developers can’t test their own code—they’re not independent enough nor skilled enough to do it properly. Reality: We need to stop stereotyping team members, their strengths and their abilities. Developers can absolutely test their own code. Some are better at it than others Pair with them to help test appropriately Copyright © 2014 RGCG, LLC 11 Copyright © 2014 RGCG, LLC 12 #5, Test Planning & Scripts Myth: You don’t need to plan (it just happens ) and you don’t need functional test cases (automation takes care of everything ) Reality: Plans help the team focus on the risk-based testing required within an iteration AND across a release Reality: Scripts (test cases) help formalize and drive your testing; Absolutely required in regulatory environments Reality: You’ll never actually automate every test
  • 9. 7 Brainstorm… Agile Planning & Execution Get together in small groups of 4-6 Take a few minutes discuss your current planning and test process mechanisms. What would an Agile Test Plan “look like” in your organization? What would Test Cases “look like”? What about progress measures? And traceability? Can you move from the “individual” to the “team”? Be prepared to share Copyright © 2014 RGCG, LLC 13 Copyright © 2014 RGCG, LLC 14 #6, Testing within the Sprint Myth: You simply need to run 100% of the tests within the constraints of the Sprint that’s “Agile” Reality: Rarely possible in most contexts. You first need a high-degree of automation and business support (for example: equipment costs) Very mature test automation and CI / CD environments Reality: Most agile teams adopt some sort of risk-based testing approach for within the sprints Dealing with Technical Test Debt Then leverage Hardening / Stabilization pre-release sprints
  • 10. 8 Copyright © 2014 RGCG, LLC The Agile Release Train Synchronized Iterate Iterate Team 1 Team 2 Team 3 Team 4 Iterate Iterate Harden Iterate Iterate Iterate X-team Harden Harden Harden Harden Iterate Iterate Iterate Iterate Iterate Iterate Iterate Iterate Iterate Iterate Iterate Iterate Iterate Internal Release External Release Docs, Training, Support, UAT, Comp. Team n Continuous Integration Continuous Integration Continuous Integration Continuous Integration 15 Copyright © 2014 RGCG, LLC The Agile Release Train Example: eCommerce / SaaS Model Iterate Iterate Team 1 Team 2 Team 3 Team 4 Iterate Iterate Harden X-team Harden Docs, Training Harden Harden Harden Iterate Iterate Iterate Iterate External Release Team 8 Continuous Integration Continuous Integration Continuous Integration Continuous Integration 10 days 10 days 5 + 2 days Rinse & Repeat Environment Evolution Dev + QA Dev + QA QA -> Staging Production 16
  • 11. 9 Copyright © 2014 RGCG, LLC 17 The Agile Release Train Example: iContact / SaaS Model Team 1 Team 2 Team 3 Team 4 Iterate Iterate Harden X-team Harden Docs, Training Harden Harden Harden Iterate Iterate Production Release Team 10 Continuous Integration Continuous Integration Continuous Integration Continuous Integration 3 weeks / 15 days 4-5 days Rinse & Repeat Environment Evolution Dev + QA QA -> Staging Production SBET, Exploratory – Regression Testing Brainstorm… “Your” Agile Release Train Get together in small groups of 4-6 to discuss Take a few minutes and think about your current release constraints timing, customers, domain, competition, # of teams, technology, etc. Design a release train model for your organization Overlay it with testing activities, plans, and milestones Present it to your larger table/group; gain feedback & adjust Be prepared to share Copyright © 2014 RGCG, LLC 18
  • 12. 10 #7, Exploratory Testing Myth: There is no place for Session Based Exploratory Testing in agile contexts. Reality: ET and SBET are a beautiful complement to agile testing. Helping nurture pairing & collaboration across teams and functions Defining new (more valuable) test cases Quickly gaining quality & usability feedback Let’s explore the details of SBET Copyright © 2014 RGCG, LLC 19 Copyright © 2014 RGCG, LLC 20 #8, Role of Testers Myth: That the testers alone own quality & testing practices within each team and sprint Reality: The testers foster a “Whole Team” view towards quality—focusing less on “Testing” and more on “Quality Practices & the Customer” Serving as guides for the team; Testing the “hard bits” Facilitating exploratory testing sessions—finding more interesting / valuable tests Working with the Product Owners—are we solving the customers problems?
  • 13. 11 Copyright © 2014 RGCG, LLC 21 #9, Developer to Tester Workflow Myth: There is always a hand-off from developers to testers; usually quite late in the sprint. That’s simply the “way of things” in software development. Reality: Scrummer-fall is alive and well but, Wrong! Teams need to swarm on their work, as flow & throughput matter the most. WIP limits and close proximity / collaboration help establish a healthy tempo of developer & tester pairing Micro-handoffs – testing as development progresses! Do you log bugs? Or do you fix bugs? Copyright © 2014 RGCG, LLC 22 #10, Managing Agile Testers Myth: The functional test manager is in charge of deciding how, who, when , etc. for the test team. Reality: You still absolutely need functional leadership within agile teams; However, it’s focused towards quality practices, strategy & coaching, and handling impediments / escalations Encouraging transparency, transforming metrics & reporting Supporting & protecting the teams Encouraging risk-taking, innovation & creativity (Slack Time)
  • 14. 12 Levels of Done-Ness Criteria Activity Criteria Example Basic Team Work Products Done’ness criteria Pairing or pair inspections of code prior to check-in; or development, execution and passing of unit tests. User Story or Theme Level Acceptance Tests Development of FitNesse based acceptance tests with the customer AND their successful execution and passing. Developed toward individual stories and/or themes for sets of stories. Sprint or Iteration Level Done’ness criteria Defining a Sprint Goal that clarifies the feature development and all external dependencies associcated with a sprint. Release Level Release criteria Defining a broad set of conditions (artifacts, testing activities or coverage levels, results/metrics, collaboration with other groups, meeting compliance levels, etc.) that IF MET would mean the release could occur. 23Copyright © 2014 RGCG, LLC Brainstorm… “Your” Definition of Done Get together in small groups of 4-6 to discuss Using the 4-tier approach referenced start filling in the 4 levels as a group. Consider any criteria you are currently using at your companies? Also consider current issues or challenge you might have where “done-ness” would help? And what about Ready-ness? Be prepared to share Copyright © 2014 RGCG, LLC 24
  • 15. 13 Copyright © 2014 RGCG, LLC 25 #11, Test Metrics Myth: You can and should move forward reporting everything exactly as you have before. Including any ‘dysfunctional’ metrics that your process and/or PMO dictates. Reality: The metrics should change immediately. From QA and Test centric towards Team-Centric metrics (Value, Throughput, Quality, Team) Stop reporting out on “Testing”; it’s irrelevant! This effects planning as well—estimation, progress, risk, etc. Contribute quality-centric Information radiators to the mix Copyright © 2014 RGCG, LLC 26 Brainstorm… Morphing your Metrics Get together in small groups of 4-6 to discuss What are you measuring today? Why? How are they driving your success and behaviors? As you move to agile, what can/should you be measuring in the 4 key areas: Value, Quality, Throughput & Predictability, and Team? How will you change your existing metrics? What behaviors are you trying to inspire? Be prepared to share
  • 16. 14 #12, Retrospectives: The Secret Sauce Myth: Testers are “Second Class” citizens who don’t play an active part in the project & team Reality: There are many places to “make a difference” Getting the 800 lb. Gorillas out on the table; Showing courage; telling truth Fostering continuous improvement within the team Setting the example; showing vulnerability—admitting you’re wrong Team listening; active planning; dependencies; pairing Risk-taking; Failure! Copyright © 2014 RGCG, LLC 27 #13, Continuous Improvement Myth: We’re generally ‘stuck’ in our approaches so just accept them and do the “best you can”. Reality: Continuous improvement is everyone’s responsibility—to engage, suggest, take ownership of current results, explore root causes, etc. Active participation in your teams Retrospectives is a key way to guide quality, testing, and customer-centric improvements. Courage! Copyright © 2014 RGCG, LLC 28
  • 17. 15 #14, The Customer Myth: Business Analysts capture customer requirements and testers test them for completeness. Reality: You need to begin to partner with the Customer – Stakeholders – Product Owners to produce software that solves the their problems. Move to the “front” and help define & refine User Stories with your Product Owner Actively participate in Sprint Reviews Show value for automation; placing test investments in the Backlog Copyright © 2014 RGCG, LLC 29 Copyright © 2014 RGCG, LLC 30 #15, Agile Requirements – The Product Backlog Myth: We can’t start testing until the requirements are finished or stable; no matter how ‘agile’ we are. Reality: Hogwash! Get over it Ambiguity and incompleteness need to become your friend and ally. As does working with your Product Owners and Customers to help define the requirements Realizing that the requirements (User Stories) are only complete at the end of each sprint.
  • 18. 16 Copyright © 2014 RGCG, LLC 31 Brainstorm… Agile Requirements Get together in small groups of 4-6 to discuss Are iterative, are intentionally incomplete The “older” the are, the larger and less defined they are Enter the sprint at 70%, exit at 100% Drive questions, dialogue, discussion, and collaboration; think 3 Amigos or the Triad So, WHY? And how will you make this work as a tester? Be prepared to share Agile Test Transformation Strategy: 3 Pillars of Agile Quality Copyright © 2014 RGCG, LLC 32
  • 19. 17 3 Pillars of Agile Quality Copyright © 2014 RGCG, LLC 33 Development & Test Automation • Pyramid-based Strategy: (Unit + Cucumber + Selenium) • Continuous Integration • Attack technical infrastructure in the Backlog • Visual Feedback – Dashboards • Actively practice ATDD and BDD Software Testing • Risk-based testing: Functional & Non-Functional • Test planning @ Release & Sprint levels • Exploratory Testing • Standards – checklists, templates, repositories • Balance across manual, exploratory & automation Cross-Functional Team Practices • Team-based Pairing • Stop-the-Line Mindset • Code Reviews & Standards • Active Done-Ness • Aggressive Refactoring of Technical Debt • User Stories, “3 Amigo” based Conversations • Whole Team Ownership of “Quality” • Building it ‘Right’; Building the ‘Right’ Thing • Healthy – Agile Centric Metrics • Center of Excellence or Community of Practice • Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement Foundation of the 3 Pillars Copyright © 2014 RGCG, LLC 34 • Whole Team Ownership of “Quality” • Building it ‘Right’; Building the ‘Right’ Thing • Healthy – Agile Centric Metrics • Center of Excellence or Community of Practice • Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement • Whole team view includes building it right, everyone tests, • Focus on features/stories, confirmation, conversation, and getting them staged properly OVER testing • 4-tier metrics: Quality, Value, Prediction, Team • Agile strategies need light-handed “steering”; establish a CoE (heavier weight) or a CoP (lightweight) • Consider finding an assessment framework and then tying it to your strategy measurement, recalibration, and continuous improvement. • Make the Foundation visible thru Information Radiators and metrics
  • 20. 18 3 Pillars of Agile Quality Copyright © 2014 RGCG, LLC 35 Development & Test Automation • Pyramid-based Strategy: (Unit + Cucumber + Selenium) • Continuous Integration • Attack technical infrastructure in the Backlog • Visual Feedback – Dashboards • Actively practice ATDD and BDD A central part of agile adoption is focusing on CI, 3- tiered Automation development, and Dashboards to begin incrementally building coverage for faster feedback on changes. In the interim, Hardening or Stabilization Sprints and having a risk-based Release Train concept help It’s important that Test or QA not ‘own’ the tooling or all of the automation efforts. The strategy can come from Test, but the tactical automation development is best left to the team. Mature teams invest in automation as part of Done- ness and continually on their backlogs 3 Pillars of Agile Quality Copyright © 2014 RGCG, LLC 36 Software Testing • Risk-based testing: Functional & Non- Functional • Test planning @ Release & Sprint levels • Exploratory Testing • Standards – checklists, templates, repositories • Balance across manual, exploratory & automation Exploratory Testing (Charter / Session based and paired) can be an incredibly effective way to establish a whole-team, collaborative view towards quality and testing. It also emerges new tests. Leverage ‘plans’ as a whole-team collaboration mechanism and do plan. Do not measure testing or tester progress; instead, measure throughput, output, sprint outcomes, and done-ness escapes at a team level. You need a balanced test team; not everyone needs to be able to program. But everyone needs to be skilled testers. Agile testing is a Risk-Based play in every Sprint and across a release sequence. Don’t forget your techniques!
  • 21. 19 3 Pillars of Agile Quality Copyright © 2014 RGCG, LLC 37 Cross-Functional Team Practices • Team-based Pairing • Stop-the-Line Mindset • Code Reviews & Standards • Active Done-Ness • Aggressive Refactoring of Technical Debt • User Stories – 3 Amigo based Conversations One of the hardest areas to get ‘right’ culturally. It needs leadership alignment from Quality/Testing to Product to Development and a consistent voice of whole-team approaches. This is where LEAN lives, where whole-team collaboration happens, where professionalism and craftsmanship are held dear. I like the view of testers becoming the VOC, champions of quality, and consistent questioners of what is being build. Are we solving the right problems as simply as possible. Notions of Minimal Viable Product / Feature help with focus. And yes Virginia, there ARE standards, templates, and a focus on consistency! Copyright © 2014 RGCG, LLC 3838 Software Testing Strategies It ALL starts with empowering testers AND creating a Whole-Team view towards Quality Critical Early Steps: Creating a sense of empowered Functional Team Applying Testing Standards across all teams Deploying Exploratory Testing across all teams Defining a core set of Agile KPI / metrics ACTIVE participants in Sprint Planning
  • 22. 20 Copyright © 2014 RGCG, LLC 3939 Cross-Functional Team Practices Strategies Training Agile / Lean in general, Story writing, Acceptance, Unit testing, etc. Teaming – for example: feedback or 5 Dysfunctions / Trust Critical Early Steps: Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP Limits across teams Define prescriptive and aggressive Done-Ness for ALL teams Implement coding standards & Crucible / code reviews across the center (appropriate for technology stacks) Release Planning BEFORE allowing a team to start Sprint #1 Backlogs have Bug + Refactoring + Automation targets (20%)? Copyright © 2014 RGCG, LLC 4040 Organizational Quality Strategies Continuously communicate your unified Vision Your strategy must be aligned/shared across: Development, Quality/Testing, and Product Keep working your strategy across the pillars Don’t get stuck with too narrow a focus (easy road) Make your strategy visible (Information Radiators) Show progress (Ex: burn up of test automation coverage across tiers) Visualize organizational impediments to your Agile Quality strategies Attack them! Quarterly read-outs on progress, plans and adjustments Listen to your teams; Celebrate successes!
  • 23. 21 What will be (your) agile strategy when you get back home? Either in groups or individually Consider the 3-Pillars discussion Consider your current team / organization agile ‘state’ Define a broad, 3-pillar view towards some immediate focus points when you get back into the office What will you focus on? Why? How will you communicate the need for change? How will you measure results? What will come immediately afterwards? Copyright © 2014 RGCG, LLC 41 Wrapping up… Agile is the best thing that’s happened to testers since The Great Depression Whole Team view Testing, Metrics, Automation Planning, Reporting, Quality Facilitate feedback Multi-tiered automation Just-in-Time, risk-based testing Continuous improvement Trust the Team Retrospective Copyright © 2014 RGCG, LLC 42
  • 24. 22 Contact Info Bob Galen Principal Consultant, RGalen Consulting Group, L.L.C. Experience-driven agile focused training, coaching & consulting Cell: (919) 272-0719 bob@rgalen.com www.rgalen.com bgalen@velocitypartners.net www.velocitypartners.net Blogs Project Times - http://www.projecttimes.com/robert-galen/ BA Times - http://www.batimes.com/robert-galen/ Podcast on all things ‘agile’ - http://www.meta-cast.com/ 43Copyright © 2014 RGCG, LLC 43 Additional Topics Copyright © 2014 RGCG, LLC 44
  • 25. 23 Two Pillars of Lean ‘Thinking’ Respect for People Customer, Employees, Vendors Develop your teams Trust & coach No wasteful work Continuous Improvement Embrace change, challenge everything Kaizen – small, incremental change Kaikaku – larger scale, fundamental 45 From http://www.leanprimer.com Copyright © 2014 RGCG, LLC Agile Testing Quadrants Brian Marick; Lisa Crispin & Janet Gregory Copyright © 2014 RGCG, LLC 46 Exploratory testing Scenarios Usability testing UAT Alpha / Beta Unit tests Component tests API tests Functional tests Story tests Examples Prototypes Simulations Performance testing Load testing Security testing Non-functional requirements Q1 Q2 Q3 Q4 Automated & Manual Automated & Manual Manual Automation, Tools, and Manual Business Facing Technology Facing SupportingtheTeam CritiquetheProduct
  • 26. 24 Copyright © 2014 RGCG, LLC 47 10 Tenets of Agile Testing Jean Tabaka, Rally Software 1. The system always runs 2. Stop the line, vs. logging defects 3. If it’s not tested, it’s not “Done” 4. Testing comes first, not last 5. Finding defects after Development is “Done” is too late Continuous Integration Lean – fix it now! Early feedback; Earned Value Collaborative testing, focus on building in quality Early feedback; fix it now! Copyright © 2014 RGCG, LLC 48 10 Tenets of Agile Testing Jean Tabaka, Rally Software 6. “Development Complete” is meaningless 7. Use testing, not analysis, to explore requirements 8. Automation is “how” not a “whether” or “when” 9. Tests are your second most detailed specification 10. Testers are Customer- Developer liaisons Whole Team complete view – no “partial credit” Executable requirements Automate all testing; feedback Code is first; later is traditional specifications VOC; guide effective team collaboration; ask the right questions
  • 27. 25 Copyright © 2014 RGCG, LLC 49 10 Commitments of Agile Testing Jean Tabaka, Rally Software 1. We commit to not moving forward if a hole is found through root cause analysis without first writing a test 2. We commit to not relying solely on just automated testing or just manual testing 3. We commit to not sitting behind a QA wall (no boundaries!) 4. We commit to not allowing a code complete without test code harness complete 5. We commit to not waiting for a test phase but rather working in smaller and smaller pieces, sooner and sooner Copyright © 2014 RGCG, LLC 50 10 Commitments of Agile Testing Jean Tabaka, Rally Software 6. We commit to not testing one iteration after development is “Done” 7. We commit to not allowing surprises to accumulate for large end-to-end testing (“mock it now”) 8. We commit to not leaving the riskiest tests to the end 9. We commit to being an equal participant with the customer and the developer in defining “Doneness” 10.We commit to not taking this oath lightly