Your SlideShare is downloading. ×
0
Agile Testing and the Role of the
           Agile Tester

                               Declan Whelan
                  ...
Agenda
•   What is Agility?
•   Agile Testing Success Factors
•   Agile/Scrum Overview
•   What is Quality?
    • Context-...
Notes on this Presentation
This presentation was originally delivered at the QUEST
Toronto 2008 conference. Prior to the c...
Agile Testing Success Factors
Testers are part        • Be cathedral builders not stone cutters
  of the team           • ...
Agile Testing Success Factors
                       • Session-based testing,
 Foundation of           agile test environm...
Traditional Approach - Waterfall


         Requirements     Analysis &         Code                Test                 D...
Agile Approach
          Iteration 1         Iteration 2          Iteration 3         Iteration 4
          Requirements  ...
What is Agility?
“Values, principles and practices that
  foster team communication and
    feedback to regularly deliver
...
Agile Values

Individuals & Interactions                             Working Software
                                    ...
Source: Brian Marick
Quarry
Lean
Engineering
XP – Customer Bill of Rights
• You have the right to an overall plan, to know what can be
  accomplished, when, and at wha...
XP – Tester Bill of Rights
• You have the right to bring up issues related to quality and process
  at any time.
• You hav...
Scrum Overview




 http://www.mountaingoatsoftware.com/scrum_figures
Story

                Card




Confirmation           Conversation
What is a Story?
• Functionality that      Independent
  provides value
                          Negotiable
• Written in ...
Story Card
• The “headline” for some new functionality.
• It is an “invitation to a conversation”.
• Written in business (...
Story Conversation
• Exchange of thoughts, opinions, and feelings.
• Takes place over time, particularly when the
  story ...
Story Confirmation
• Story tests are acceptance tests for the
  product owner
• Product owner specifies the story tests bu...
Task
• Some atomic unit of work that can be “done”.
• Development estimate includes design,
  development, unit test, refa...
Scrum Overview




 http://www.mountaingoatsoftware.com/scrum_figures
Scrum Team
    Product
                     ScrumMaster             Team
    Owner
• Feature           • Represents       ...
Product Backlog
• Master list of all
  “features”
• High priority features
  are split into “stories”
  achievable within ...
Sprint Planning Meeting
• Highest priority
  stories are
  reviewed.
• Team selects stories
• Team breaks stories
  down i...
Sprint Backlog
Daily Scrum
• Each team member
  describes:
  – What they did
  – What they plan to do
  – Obstacles
• ScrumMaster tracks
...
Sprint Demo
• Team demonstrates
  working software to
  product owner
• Product owner accepts
  or rejects completed
  wor...
Sprint Retrospective
• Team meets to review:
   – What is working?
   – What is not working?
• Team adds tasks for
  immed...
What is Quality?


Jerry Weinberg:

   “Value to some person(s).”
Testing Schools of Thought
            • How much has                     • Controlled process                    • Focus ...
Context-Driven Principles
1. The value of any practice depends on its context.
2. There are good practices in context, but...
Session Based Testing
A method specifically designed
to make exploratory testing
auditable and measurable on a
wider scale...
Session Based Structure

                 Charter


  Parsing
                               Session
  Results




       ...
Testing Dashboard
Area         Effort Assessment
                 Coverage Comments




       Source: http://www.satisfic...
Agile Quality – A Team Deliverable
 Agile Practice   Benefits
 Whole Team       • Quality is not just a tester
           ...
Agile Testing Challenges
• Team may not value testers
• Testers may not value team
• Unclear role of testers on team
• Tes...
Agile Testing Approach
• Testers are first class citizens on agile teams and part
  of the “whole team” supporting custome...
Brian Marick’s Agile Testing Matrix
  Automate                               Customer Facing                     Manual

 ...
Tester Activities
         Product Owner                                                 Customer
                        ...
Agile Testing Iterations
     Q3, Q4:              Q1:                 Q2:
     Product           Testing &           Plan...
Why Automate Tests?
•   Provides safety net
•   Supports rapid iterations
•   Provides footholds to keep notching upward
•...
Types of Automated Tests
                           TestComplete
 UI Tests    Through UI      SilkTest
                   ...
Agile Testing Success Factors
Testers are part        • Be cathedral builders not stone cutters
  of the team           • ...
Agile Testing Success Factors
                       • Session-based testing,
 Foundation of           agile test environm...
Agile Testing Reading List
• Agile Project Management
  Jim Highsmith; 2004

• Lean Software Development
  Mary & Tom Popp...
Web Sites
•   http://www.exampler.com (Brian Marick)
•   http://www.developsense.com (Michael Bolton)
•   http://www.satis...
Session Based Testing Extras
The following additional information on
session based testing was primarily obtained
or deriv...
Session Based Structure

                 Charter


  Parsing
                               Session
  Results




       ...
Goals
Find bugs fast

Fix bugs fast – through rapid testing feedback

Visibility

Transparency

Auditability

Maximize tes...
Charter
A goal or agenda for a test session in 1
to 3 sentences.

Created by the test team prior to the
start of testing, ...
Session
An uninterrupted period of time spent testing,
ideally lasting one to two hours.

Each session is focused on a cha...
Report
• Charter                               • Any files used
• Areas Tested                          • % of time spent ...
Debrief
A debrief is a short discussion
between the manager and tester
(or testers) about the session and
the session repo...
Parsing Results
With a standardized Session
Report, software tools can be used to parse
and store the results as aggregate...
Low Tech Testing Dashboard




     Source: http://www.satisfice.com/rst.pdf
The Test Status Problem
  “What’s the status of testing?”

   “What are you doing today?”

   “When will you be finished?”...
Testing Reporting Challenges
• Management has little patience for detailed test status
  reports.

• Management doesn’t un...
A Low Tech Dashboard Solution
 Report test cycle progress in a simple, structured way...


         ...that shows progress...
Testing Dashboard
Area         Effort Assessment
                 Coverage Comments




       Source: http://www.satisfic...
The Dashboard Concept




   Source: http://www.satisfice.com/rst.pdf
Product Area




Source: http://www.satisfice.com/rst.pdf
Test Effort




Source: http://www.satisfice.com/rst.pdf
Test Coverage




Source: http://www.satisfice.com/rst.pdf
Quality Assessment




 Source: http://www.satisfice.com/rst.pdf
Comments




Source: http://www.satisfice.com/rst.pdf
Upcoming SlideShare
Loading in...5
×

Agile Testing: The Role Of The Agile Tester

41,773

Published on

This presentation provides an overview of the role of testers on agile teams.

In essence, the differences between testers and developers should blur so that focus is the whole team completing stories and delivering value.

Testers can add more value on agile teams by contributing earlier and moving from defect detection to defect prevention.

Published in: Technology
2 Comments
44 Likes
Statistics
Notes
No Downloads
Views
Total Views
41,773
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
2,340
Comments
2
Likes
44
Embeds 0
No embeds

No notes for slide

Transcript of "Agile Testing: The Role Of The Agile Tester"

  1. 1. Agile Testing and the Role of the Agile Tester Declan Whelan dwhelan@dpwhelan.com This presentation is licensed under a Creative Commons Attribution 2.5 License, which means you can share and adapt it, including commercial and derivative works, as long as you include attribution to Declan Whelan.
  2. 2. Agenda • What is Agility? • Agile Testing Success Factors • Agile/Scrum Overview • What is Quality? • Context-Driven School of Testing • Session-Based Testing & Low-Tech Dashboards • Agile Quality – Agile Testing Matrix – Agile Tester Activities – Test Automation
  3. 3. Notes on this Presentation This presentation was originally delivered at the QUEST Toronto 2008 conference. Prior to the conference it was discovered that information on session based testing was not properly attributed to James Bach (http://www.satisfice.com). Unfortunately an updated presentation was not available in time to be included on the USB sticks provided. This updated presentation correctly attributes this material to James and also includes some minor formatting changes. Declan wishes to apologize to James for the omission and to thank him both for his understanding and his contributions to exploratory testing.
  4. 4. Agile Testing Success Factors Testers are part • Be cathedral builders not stone cutters of the team • Collective ownership Agile testing • Drop the “Quality Police” mindset mindset • Focus on team goals & customer value • Automate tests wherever practical Automate tests • Need rapid feedback Look at the big • Balance against developer focus on technical implementation picture • Use agile test matrix as guide Source: http://www.agiletester.ca/
  5. 5. Agile Testing Success Factors • Session-based testing, Foundation of agile test environments critical practices • Informative workspace • Collaborate with customers Collaborate • Collaborate within team • Team retrospectives Continually • Personal training: reading, blogs, improve QAI, local QA groups Source: http://www.agiletester.ca/
  6. 6. Traditional Approach - Waterfall Requirements Analysis & Code Test Deploy Cost • Business Design • C#, C, C++ etc. • Validation Tests Of Requirements • System • Big-Bang • Verification • Technical Specifications Integration Tests Change Requirements • Component Specifications Time
  7. 7. Agile Approach Iteration 1 Iteration 2 Iteration 3 Iteration 4 Requirements Requirements Requirements Requirements Analysis & Design Analysis & Design Analysis & Design Analysis & Design Cost of Code Code Code Code Change Test Test Test Test Deploy Time
  8. 8. What is Agility? “Values, principles and practices that foster team communication and feedback to regularly deliver customer value through working software.”
  9. 9. Agile Values Individuals & Interactions Working Software Comprehensive Processes & Tools Documentation Customer Collaboration Responding to Change Contract Negotiation Following a Plan http://agilemanifesto.org/
  10. 10. Source: Brian Marick
  11. 11. Quarry
  12. 12. Lean
  13. 13. Engineering
  14. 14. XP – Customer Bill of Rights • You have the right to an overall plan, to know what can be accomplished, when, and at what cost. • You have the right to see progress in a running system, proven to work by passing repeatable tests that you specify. • You have the right to change your mind, to substitute functionality, and to change priorities. • You have the right to be informed of schedule changes, in time to choose how to reduce scope to restore the original date. You can even cancel at any time and be left with a useful working system reflecting investment to date.
  15. 15. XP – Tester Bill of Rights • You have the right to bring up issues related to quality and process at any time. • You have the right to ask questions of customers and programmers and receive timely answers. • You have the right to ask for and receive help from anyone on the project team, including programmers, managers and customers. • You have the right to make and update your own estimates for your own tasks and have these included in estimates for stories. • You have the right to the tools you need to do your job in a timely manner. • You have the right to expect your project team, not just yourself, to be responsible for quality. Source: http://home.att.net/~lisa.crispin/XPTesterBOR.htm
  16. 16. Scrum Overview http://www.mountaingoatsoftware.com/scrum_figures
  17. 17. Story Card Confirmation Conversation
  18. 18. What is a Story? • Functionality that Independent provides value Negotiable • Written in business (domain) language Valuable • Ideally < 2 days work Estimable • “As a <role>, I want to <feature> so that Small <benefit>” Testable
  19. 19. Story Card • The “headline” for some new functionality. • It is an “invitation to a conversation”. • Written in business (domain) language. • It is not a specification … specifications are documented using story tests. • Stories can be used to drive additional work such as building test scripts/frameworks
  20. 20. Story Conversation • Exchange of thoughts, opinions, and feelings. • Takes place over time, particularly when the story is estimated. • Conversations also occur at the iteration planning meeting when the story is scheduled for implementation. • Supplement with documents as needed.
  21. 21. Story Confirmation • Story tests are acceptance tests for the product owner • Product owner specifies the story tests but will collaborate with team to create them • Team can add additional tests • Most tests can and should be automated (e.g. FIT, xBehave, xSpec, stress tests, load tests)
  22. 22. Task • Some atomic unit of work that can be “done”. • Development estimate includes design, development, unit test, refactoring, check in • Usually maintained on a whiteboard or bulletin board for wide visibility • Add testing tasks for non-functional “ility” testing • Typically 1-4 hours effort
  23. 23. Scrum Overview http://www.mountaingoatsoftware.com/scrum_figures
  24. 24. Scrum Team Product ScrumMaster Team Owner • Feature • Represents • Self organizing definition management • Cross- • Release dates • Removes functional • Single decision obstacles • Estimates point • Ensures Scrum • Tracks • Accepts or process • Gets ‘er done rejects work • Servant leader • ROI
  25. 25. Product Backlog • Master list of all “features” • High priority features are split into “stories” achievable within an iteration. • Each “story” is prioritized and scoped.
  26. 26. Sprint Planning Meeting • Highest priority stories are reviewed. • Team selects stories • Team breaks stories down into tasks & re-estimates. • Team commits to next iteration’s deliverables.
  27. 27. Sprint Backlog
  28. 28. Daily Scrum • Each team member describes: – What they did – What they plan to do – Obstacles • ScrumMaster tracks and resolves obstacles • 10 – 15 minutes
  29. 29. Sprint Demo • Team demonstrates working software to product owner • Product owner accepts or rejects completed work • Result should be potentially shippable
  30. 30. Sprint Retrospective • Team meets to review: – What is working? – What is not working? • Team adds tasks for immediate actions for working better
  31. 31. What is Quality? Jerry Weinberg: “Value to some person(s).”
  32. 32. Testing Schools of Thought • How much has • Controlled process • Focus on QA vs. been tested? • Formal rules testing • Objectivity • Standards & • Process • Branch of certification watchdog CS/Math • Conformance focus • Risk driven • QA signoff Analytic Factory Quality • Skilled mental activity • People set context Agile Helps • Testing provides information Set Context • Value driven Context Driven Source: http://www.qaiasia.com/bpw/innerpages/documents/Process_Improvement/Model_and_Framework/PI_007-shrinivas_kulkarni.pdf
  33. 33. Context-Driven Principles 1. The value of any practice depends on its context. 2. There are good practices in context, but there are no best practices. 3. People, working together, are the most important part of any project's context. 4. Projects unfold over time in ways that are often not predictable. 5. The product is a solution. If the problem isn't solved, the product doesn't work. 6. Good software testing is a challenging intellectual process. 7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products. Source: http://www.context-driven-testing.com
  34. 34. Session Based Testing A method specifically designed to make exploratory testing auditable and measurable on a wider scale. Source: http://en.wikipedia.org/wiki/Exploratory_testing
  35. 35. Session Based Structure Charter Parsing Session Results Debrief Report
  36. 36. Testing Dashboard Area Effort Assessment Coverage Comments Source: http://www.satisfice.com/rst.pdf
  37. 37. Agile Quality – A Team Deliverable Agile Practice Benefits Whole Team • Quality is not just a tester responsibility • Quality is more than just testing • Testing role shifts to quality infusion throughout project life cycle Continuous • Developers cannot check in code Integration with failing tests Continuous • Avoids long delays with “big-bang” Testing testing after the “final build” • Bugs found closer to when they are introduced making them easier to fix
  38. 38. Agile Testing Challenges • Team may not value testers • Testers may not value team • Unclear role of testers on team • Testing is often squeezed as deadlines approach • Developers and testers are often in different operational silos • Team may not have the skills or domain expertise to develop/test effectively
  39. 39. Agile Testing Approach • Testers are first class citizens on agile teams and part of the “whole team” supporting customers, business stakeholders, developers and other team members • Testers support quality infusion through entire team and product cycle • Test tasks and stories are planned and executed like development tasks and stories • Automate where possible and use session-based testing for exploratory testing • Communicate through information radiators
  40. 40. Brian Marick’s Agile Testing Matrix Automate Customer Facing Manual Functional Tests User Acceptance Tests Customer Tests Exploratory Tests Supports Development Critiques Product Story Tests/Examples Usability Tests Q2 Q3 Q1 Q4 Unit Tests Performance Tests Integration Tests Load Tests Automate Technology Facing Tools
  41. 41. Tester Activities Product Owner Customer Customer Facing Collaboration Collaboration Product Specifications UAT Design Supports Development Test Ideas Exploratory Testing Critiques Product Testing Q2 Q3 Usability Testing Test Ideas Q1 Q4 Test Scripts Test Development Testing Testing Test Analysis Developer IT Technology Facing Collaboration Collaboration
  42. 42. Agile Testing Iterations Q3, Q4: Q1: Q2: Product Testing & Planning & Testing Collaboration Test Ideas Working Working Working Stories Stories Stories Product Product Product Previous Current Next Iteration Iteration Iteration
  43. 43. Why Automate Tests? • Provides safety net • Supports rapid iterations • Provides footholds to keep notching upward • Provides rapid feedback • Focuses effort on what is valuable • Frees people to do their best work Need to balance automation costs against delivered value
  44. 44. Types of Automated Tests TestComplete UI Tests Through UI SilkTest TestDirector c Functional In business Fit/FitNesse o language Tests s Unit Tests In same language as system xUnit t
  45. 45. Agile Testing Success Factors Testers are part • Be cathedral builders not stone cutters of the team • Collective ownership Agile testing • Drop the “Quality Police” mindset mindset • Focus on team goals & customer value • Automate tests wherever practical Automate tests • Need rapid feedback Look at the big • Balance against developer focus on technical implementation picture • Use agile test matrix as guide Source: http://www.agiletester.ca/
  46. 46. Agile Testing Success Factors • Session-based testing, Foundation of agile test environments critical practices • Informative workspace • Collaborate with customers Collaborate • Collaborate within team • Team retrospectives Continually • Personal training: improve reading, blogs, QAI, local QA groups Source: http://www.agiletester.ca/
  47. 47. Agile Testing Reading List • Agile Project Management Jim Highsmith; 2004 • Lean Software Development Mary & Tom Poppendieck; 2003 • Extreme Programming Explained 2nd Edition Kent Beck, Cynthia Andres; 2004 • Testing Extreme Programming Lisa Crispin; 2002 • Agile Testing: The Tester Role on an Agile Project Lisa Crispin, Janet Gregory; Can pre-order on Amazon
  48. 48. Web Sites • http://www.exampler.com (Brian Marick) • http://www.developsense.com (Michael Bolton) • http://www.satisfice.com (James Bach) • http://www.context-driven-testing.com • http://www.kohl.ca/blog (Jonathon Kohl) • http://tech.groups.yahoo.com/group/agile-testing • http://www.agiletester.ca/ (Lisa Crispin, Janet Gregory) • www.agilemanifesto.com • www.agilealliance.org
  49. 49. Session Based Testing Extras The following additional information on session based testing was primarily obtained or derived from a set of slides called “Rapid Software Testing” by James Bach. Source: http://www.satisfice.com/rst.pdf
  50. 50. Session Based Structure Charter Parsing Session Results Debrief Report
  51. 51. Goals Find bugs fast Fix bugs fast – through rapid testing feedback Visibility Transparency Auditability Maximize testing value
  52. 52. Charter A goal or agenda for a test session in 1 to 3 sentences. Created by the test team prior to the start of testing, but may be added or changed at any time. Often created from a specification, test plan, or by examining results from previous test sessions. Source: http://en.wikipedia.org/wiki/Session-based_testing#Charter
  53. 53. Session An uninterrupted period of time spent testing, ideally lasting one to two hours. Each session is focused on a charter, but testers can also explore new opportunities or issues during this time. The tester creates and executes test cases on-the-fly and records their progress. Source: http://en.wikipedia.org/wiki/Session-based_testing#Session
  54. 54. Report • Charter • Any files used • Areas Tested • % of time spent on • Detailed notes on how charter vs. other: testing was conducted • Session start time and • A list of any bugs found duration • A list of issues: • Tester name – open questions – product concerns – process concerns Source: http://en.wikipedia.org/wiki/Session-based_testing#Session_Report
  55. 55. Debrief A debrief is a short discussion between the manager and tester (or testers) about the session and the session report. Source: http://en.wikipedia.org/wiki/Session-based_testing#Session_Report
  56. 56. Parsing Results With a standardized Session Report, software tools can be used to parse and store the results as aggregate data for reporting and metrics. This allows reporting on the number of sessions per area or a breakdown of time spent on testing, bug investigation, and setup / other activities. Source: http://en.wikipedia.org/wiki/Session-based_testing#Parsing_Results
  57. 57. Low Tech Testing Dashboard Source: http://www.satisfice.com/rst.pdf
  58. 58. The Test Status Problem “What’s the status of testing?” “What are you doing today?” “When will you be finished?” “Why is it taking so long?” “Have you tested _______, yet?” Source: http://www.satisfice.com/rst.pdf
  59. 59. Testing Reporting Challenges • Management has little patience for detailed test status reports. • Management doesn’t understand testing: • Testing is confused with improving. • Testing is considered a linear, independent task. • Testing is assumed to be exhaustive. • Testing is assumed to be continuous. • Test results are assumed to stay valid over time. • Impact of regression testing is not appreciated. • Test metrics are hard to interpret. Source: http://www.satisfice.com/rst.pdf
  60. 60. A Low Tech Dashboard Solution Report test cycle progress in a simple, structured way... ...that shows progress toward a goal... ... manages expectations... ...and inspires support... ...for an effective test process. Source: http://www.satisfice.com/rst.pdf
  61. 61. Testing Dashboard Area Effort Assessment Coverage Comments Source: http://www.satisfice.com/rst.pdf
  62. 62. The Dashboard Concept Source: http://www.satisfice.com/rst.pdf
  63. 63. Product Area Source: http://www.satisfice.com/rst.pdf
  64. 64. Test Effort Source: http://www.satisfice.com/rst.pdf
  65. 65. Test Coverage Source: http://www.satisfice.com/rst.pdf
  66. 66. Quality Assessment Source: http://www.satisfice.com/rst.pdf
  67. 67. Comments Source: http://www.satisfice.com/rst.pdf
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×