This document discusses the role of quality assurance (QA) professionals in agile teams. It describes the author's experience transitioning to agile in 2006 when there was little guidance available. The document proposes a model called TOAST for Quality to help organize the QA process within agile projects. TOAST stands for Types, Organization, Added value, Skills, and Tracking. The document provides examples and best practices for each element of the TOAST framework to help define the QA role in an agile context.

1. We did it!!?
There is place for QAs in Agile!?
Michał Kujałowicz
michal.kujalowicz@spartez.com

2. • Partner
• Involved in development of core Atlassian products
• Commercial, Open-Source add-ons,
customizations
• Developers, Product Managers, Designers, QAs,
Agile Team Leads
• Me: JIRA QA Team Lead

3. QA vs Tester
NOT TODAY,
THANKS!

4. My story
2006
!
• My first real
Agile Project
• About it later
• Heard about Agile in 2006 on
ISTQB straining
• First project - 2006
• No one knew what to do
• Testers in Agile? No
experiences available
• Short story
2012
2013
!
!
• Quality
Assistance

5. The fear
2009
Popularity
Developer
Testing Interdisciplinary Technical Skills

6. How Agile is your team?
“There are now many teams that label themselves as
‘agile’, but some agile teams are more agile than
others”!
! ! ! ! ! ! ! ! ! ! ! Stuart Reid “Are all pigs equal?” Testing Experience 2009

7. How Agile is your team?
My take:
• Team empowered to improve their process
• Fast feedback and adaptation
• Production ready increment as result of iteration
• Everyone is responsible for the team’s output

8. My first “real” Agile project
• Test Manager
• Around 10 developers at the beginning, 3 testers
• From time perspective - not good
• No shared quality responsibility, bad fitting,
technical debt
• No experience, no enough thinking, no guidance

9. TOAST FOR QUALITY

10. TOAST for Quality
• A model helping organise
QA process inside Agile
project
• Refers to whole quality
process - not only QAs
T - Types and Techniques
O - Organisation!
A - Added value
S - Skills !
T - Tracking

11. Types and Techniques -
O - A - S - T
• Which test types do you need to execute within
your Agile process?
• Which software characteristics you need to cover
(performance, usability)?
• What techniques will you use for those tests?

12. Types and Techniques -
• Types:
O - A - S - T
• Functional tests, Performance, Load, Stress, Usability, …
• Techniques:
• Scripted testing, Automated tests, Exploratory testing
• White-box, Grey-box, Black-box

13. Feature
Story
Story
Feature
…
Task
Release
… Task
…
TO PRODUCTION
but also Timing

14. Feature
Story
but also Timing
RELEASE TESTING
PRODUCT !
REGRESSION
Story
Feature
…
Task
Release
… Task
…
STORY TESTING
FEATURE !
REGRESSION
PRODUCT !
REGRESSION
FEATURE TESTING
PRODUCT !
REGRESSION

15. T - Organisation - A - S - T
• How do you plan to organise testing activities?
• Quality is the responsibility of the complete team!
• What is the team setup (How many QAs? How
many Developers)?
• What is the QA role in testing activities?

16. T - O - Added Value - S - T
• Where do we add value beyond testing?
• Automation, Business Analysis, Acceptance
Criteria, Customer Support, Customer Contact,
Test Data, Test Environments?

17. T - O - A - Skills - T
• What skills do we need in the team?
• What do we need to do to fulfil needs?
• Where can QAs specialise?
• Are QAs technical enough?

18. T - O - A - S - Tracking
• How do we estimate QA activities and verify those?
• How do we measure the process?
• How do we measure product quality?
• How do we get inputs for retrospective?
• …

19. PRACTICES

20. Types
Types, Techniques,
Timing
• Which software characteristics are crucial for you?
• Functionality, Usability, Scalability, …ility
• Depends on your context
KEEP!
CALM!
ITS
NOT IN!
SCOPE

21. Types, Techniques,
Timing
Agile Testing Quadrants
“Agile Testing: A Practical Guide for Testers and Agile Teams” Lisa Crispin and Janet Gregory

22. Techniques
Types, Techniques,
Timing
• Scripted, Exploratory, Ad-hoc, Automated
• White-box, Grey-box, Black-box
!
• Agile moves onto Exploratory, White-Box and
Automated

23. Exploratory
Types, Techniques,
Timing
“Telling your exploratory story” Jon Bach, Agile 2010

24. SBTM
Types, Techniques,
Timing
• Session Based Test Management
• Charters
• Sessions
• Notes
• Debriefing (PROOF)

25. SBTM
Types, Techniques,
Timing
Organisation
“Adding value in an agile context” !
by Henrik Andersson

26. SBTM
Types, Techniques,
Timing
Organisation
• Not only exp. testing technique but a whole
organisational approach!
• Every Testing activity is a Session (~90-minutes)
• Estimate number of possible sessions for every
iteration (3-4 sessions a day)
• Fight on the Sprint planning to prioritise
• Take metrics e.g. Sessions burndown chart

27. Types, Techniques,
Timing
A word about exploratory
• We do not know how to describe the exploratory
thinking
• Considered as ‘playing around’ without any test
documentation
• Fight that!
• Think about it before encouraging your organisation

28. Automation
Types, Techniques,
Timing
UI
Continuous
Services
Integration Unit Tests

29. Feature
Story
RELEASE TESTING
PRODUCT !
REGRESSION
Story
Feature
…
Task
Release
… Task
…
STORY TESTING
FEATURE !
REGRESSION
PRODUCT !
REGRESSION
FEATURE TESTING
PRODUCT !
REGRESSION
Types, Techniques,
Timing
Timing

30. Types, Techniques,
Timing
Timing
• Think what you have and
what you do not have!
• The testing scope activities
represent risks - not all risks
must be covered with testing
• … but they must be
addressed!

31. Release every 2 weeks
FEATURES STORIES
Testing
Release 2-3 times a year
FEATURES STORIES
Types, Techniques,
Timing
NO
(Optional) Blitz Test
STORY TESTING
Autom. (CI)
Autom. (CI)
FEATURE !
REGRESSION
PRODUCT !
REGRESSION
FEATURE TESTING
PRODUCT !
REGRESSION
RELEASE TESTING
PRODUCT !
REGRESSION
Release
Feature
Story
Autom. (CI) Autom. (CI)
Manual exploratory tests!
+ Automated tests

32. Teams
Organisation
• QA is part of the team (3-4 developers)
• Main responsibility - testing (moving towards exp.
testing)
!
• Other responsibilities = Added value
• Everyone has ‘Developer’ role

33. • Tool usage: sub-task of every story, separate
testing tasks
• Part of the definition of done
• SBTM
Tasks
Organisation

34. !
Organisation
Quality is everyones responsibility!
• No satisfying answer
• Sharing partly testing with developers (checking
vs. testing)
• Unit tests, TDD, Automated tests
• My answer: Quality Assistance

35. Quality Assistance
Organisation
• 1 QA per 10 developers
• Testing in hands of developers (both manual and
automatic)
• Teaches, mentors what and how to test, helps
identifying most risk areas for testing
• QA does not test - has time for process
improvement, experimenting, bugs analysis, test
tools

36. Quality Assistance
Organisation
• Quality is indeed everyones responsibility
• Automated tests from scratch
• Tech camp presentation:
• http://www.slideshare.net/mkujalowicz/1x10-the-role-
of-qa-engineer-in-jira
• http://vimeo.com/86691004

37. Keys to succeed
Added Value
• The team must feel QA is mandatory element of the team
and its every activity
• Get the unique strengths and use them
• support Product Owner to transfer customer needs onto
backlog and acceptance criteria / automated tests
• technical debt
• test tooling and test coverage
• …

38. Added Value
• Visibility of QAs work!
• “Yesterday testing, today
testing”
• Daily stand-ups
• Interesting idea:
• ‘Report on your
product’s health’ by
Henrik Andersson
(http://tvfortesters.com/video/adding-value-
in-an-agile-context-by-henrik-andersson/)

39. Added Value
• TDD
• ATDD
• BDD
• Specification by
example
• Not my cartoon :P -
maybe yours?
http://www.fanpop.com/clubs/dexters-laboratory/images/13130720/title/deedee-photo

40. Added Value
• Teaching / mentoring
• Test tooling, test coverage
• Process improvement, bug analysis
• Customer view
• We are “linkers” and “sharers”

41. Skills

42. Technical skills
Skills
• Basics:
• IDE, SCM situation, Reviews
• White-box, Grey-box skills
• Technologies
• Further:
• Depends on your project

43. Skills
Technical skills
• Comes out of your added values analysis
• Test techniques, Data gathering, Test Tools
• Sorry - no recipe here
!
• Go beyond what is on paper

44. Tracking
What should we track?
• QA Tasks tracking
• Internal and External product quality
• Dev speed, QA speed
• Product health
• Product usage

45. Tracking
Do we do that?
• Not much people sharing it
• Create QA activities backlog and monitor its burn-down
- SBTM and Test Sessions
• Bugs and quality - we should have it covered - do
we? In most projects - not trivial

46. Tracking
• Story rejection rate
• OD Incidents, Stable regressions
• Rework time
• SaaS delivery index = JBS * 0.5 + PBS * 0.5 +
C2M * 20 + C2DEV * 10 + D8W * -10 + NI * 100

47. BUT…

48. … our projects are different
• Products in production for several years - not build
from scratch
• Almost full manual regression when switching to
Agile
• No automation possible/no automation in place
• Test documentation requirements
• Dislocated teams

49. No automation
Does not Scale

50. No automation
“Agile Testing” Tito Linz, Testwarez 2014

51. No automation
“Telling your exploratory story” Jon Bach, Agile 2010
But we require test documentation!

52. Test documentation
necessary
• What is actually needed?
What does one wants to
achieve?
• Does anyone read it at all?
• Good exploratory testing
charters
• Find your own way!

53. No automation
• You won’t win without experimenting and taking
risks!
• Application specialist - interesting idea
• Going further: White-box QA

54. White-box QA
• Pick the most technical person
• Let her/him analyse the dependencies, how it
works
• Manual regression tests recommendations
• Teach others how to do it

55. No automation != no work
on automation

56. • Web Applications
automation - easy
• Desktop - mature
• Mobile - growing but hard
• Embedded - self-made
!
• Start with quick wins - e.g.
Visual Regression

57. Multiple locations
• Is it still Agile?
• No unicorns here
• Use the available tools, look for tools
• Martin Fowler: “Using an Agile Software Process with
Offshore Development”

58. WHAT DO WE MISS

59. What does it mean to be
Agile QA
Process & Product

60. Unexplainable expectation
• QA must limit the
speed, QA must hold
development plans,
QA must make sure
we do not take risks

61. Product risks
• What does it mean to leave this area untested?
• What does it mean to leave this bug in release?
• Gather data regarding customer usage

62. Process risks
• Experiment around your process
• Take a risk to have time on process improvement
• Failure is also a success
• Make sure you know what success means

63. Our story
• QA Testing
• Testing notes
• DoTing
• QA Demo
• Dev as a QA

64. The train is leaving
• Development specificity
changes
• QA specificity changes:
• Analytics, Experiments,
A/B Testing, Continuous
Deployment
• Explore new paths!

65. Summary
• Think if you merit a TOAST? Not everything is
required but it must be a conscious decision
• QA work visibility
• Think through your Regression test situation
• Quality responsibility of the complete team - Quality
Assistance
• Take risks and experiment

66. Satisfaction!
The fear
2009
Popularity
Developer
Testing
Interdisciplinary Technical Skills

67. Thank you!

68. Images - credits
• Żuraw in Gdańsk - by JM_GD - CC BY 2.0
• 3D Growth Projections - by Chris Potter - CC BY 2.0
• Architetto -- Donna casinista - by francesco_rollandin - CC BY 2.0
• Toast - by Stiefen Schlingen - CC BY 2.0
• Toasting Champagne - Waldo Jaquith - CC BY-SA 2.0
• Thinking - by Wade M - CC BY-SA 2.0
• Stadion Narodowy w Warszawie - by Artur Malinowski - CC BY 2.0
• Thumbup - Pratheepps - CC0 1.0
• Documentation - by Marta Pucciarelli - CC-BY-SA-3.0
• Difficult - Sea Turtle - CC BY-NC-ND 2.0
• Chemia Laboratorium Experyment - pixabay.com - CC0 1.0
• Comic Daredevil Bello Nock - Chris Phutully - CC BY 2.0
• Surprise major - by Achim Hering - CC-BY-SA-3.0
• Train Left - SElefant - CC-BY-SA-3.0