More Related Content
What's hot
What's hot (20)
Similar to Risk-Based Testing for Agile Projects
Similar to Risk-Based Testing for Agile Projects (20)
More from TechWell
More from TechWell (20)
Recently uploaded
Recently uploaded (20)
Risk-Based Testing for Agile Projects
- 1. W10 Agile Testing 5/7/2014 1:45:00 PM Risk-Based Testing for Agile Projects Presented by: Erik van Veenendaal Improve Quality IT Services BV Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Erik van Veenendaal Improve Quality IT Services BV A leading international consultant, trainer, and recognized expert in software testing, Erik van Veenendaal (erikvanveenendaal.nl) is the founder of Improve Quality Services BV, a company that specializes in testing, requirements engineering, and quality management. Erik is the author of a number of books and papers, a core developer of the TMap testing methodology and the TMMi improvement model, a participant in the International Requirements Engineering Board working parties, currently on the TMMi Foundation board, and a frequent speaker at international testing conferences. For his major contribution to the field of testing, Erik received the 2007 European Testing Excellence Award.
- 3. © 2011 Improve Quality Services BV PRISMA tutorial- 1 1 Practical Risk-Based Testing also for Agile projects PRISMA® Erik van Veenendaal www.erikvanveendaal.nl Improve Quality Services BV 2 Erik van VeenendaalErik van Veenendaal Founder and major shareholder ImproveQS In testing since 1989 working for many different clients and in many different roles Author “TMap”, “TMMi model” and many other books and papers Former Vice-President International Software Testing Qualifications Board (ISTQB) Supporting member IREB board Keynote speaker, e.g. EuroSTAR, STAR Winner of the European Testing Excellence Award www. erikvanveenendaal.nl
- 4. © 2011 Improve Quality Services BV PRISMA tutorial- 2 Improve Quality Services B.V. 3 Testing ChallengesTesting Challenges Increasing importance and size of software in society as a whole − e.g., amount of software in consumer product doubles every 24 months, − number of defects is almost constant High Competition & Outsourcing − Time-To-Market, Product Quality, Price Levels are essential for business success Testing often takes 30 - 40% of project costs Risk based testing deals with both effectiveness and efficiency Improve Quality Services B.V. 4 What is Risk?What is Risk? “A factor that could result in a future negative consequence; usually expressed as impact and likelihood” (ISTQB Glossary) Testers ‘only’ have the responsibility to identify the risks and provide information on their status “to dare to undertake” − management attitude and style ..
- 5. © 2011 Improve Quality Services BV PRISMA tutorial- 3 Testing = Risk ManagementTesting = Risk Management Objective: most feasible coverage − effective usage of limited resources − optimize test effort Limited resources staffing infrastructure time ! .. The right level and type of coverage on the right parts at the right time Always applicable, whether it is V-model, Agile or .... Improve Quality Services B.V. 5 Our Challenge .Our Challenge . Improve Quality Services B.V. 6 ifif only weonly we knew !!knew !!
- 6. © 2011 Improve Quality Services BV PRISMA tutorial- 4 RiskRisk--Based TestingBased Testing Risk identification looks at ways of establishing what the risks are and where they are Risk analysis looks into the critical, complex and potential error prone areas Then we build tests to mitigate the risk Subsequently we monitor and report regarding the risks Improve Quality Services B.V. 7 Based on Practical ExperiencesBased on Practical Experiences Improve Quality Services B.V. 8 8
- 7. © 2011 Improve Quality Services BV PRISMA tutorial- 5 Risk IdentificationRisk Identification Split up test basis in functional or technical items V-model: based on requirements Agile: backlog items / user stories for next sprint Can also be based on a brainstorm session Maximum number of appr. 35 risk items Improve Quality Services B.V. 9 Risk item 1 Functionality Risk item 2 Security Risk item 3 Functionality Risk item 4 Interoperability RiskRisk AnalysisAnalysis Risk = impact x likelihood − What is the impact for the business ? − What is the likelihood that there are defects ? Determine factors based on previous projects, e.g., defect patterns Improve Quality Services B.V. 10 Impact – business risk Likelihood technicalrisk You already know this ! Exercise: Risk Factors
- 8. © 2011 Improve Quality Services BV PRISMA tutorial- 6 Improve Quality Services B.V. 11 Factors From PracticeFactors From Practice Likelihood − complexity − new development (level of re-uses) − interfacing − size − technology − geographical spread − inexperience (of development team) Impact − business importance (“selling item”) − financial (or other) damage (e.g. safety) − usage intensity − external visibility − cost of rework Customization required defect patterns / history Risk Assessment in a NutshellRisk Assessment in a Nutshell 1) Stakeholder analysis: Assigning Factors 2) Individual scoring 3) Consensus meeting 4) Position the risk items in a Product Risk Matrix 5) Define a differentiated risk-based test approach Improve Quality Services B.V. 12 Business importance Usage intensity Safety Item 1 Item 2 Item 3 Item 4 5 5 4 5 5 4 3 2 they shall make choices
- 9. © 2011 Improve Quality Services BV PRISMA tutorial- 7 Product Risk MatrixProduct Risk Matrix Improve Quality Services B.V. 13 IIII IVIV II IIIIII L H M L HM x x x x x Likelihood Impact Must Test Should Test Could Test “Won’t Test” focus of unit testing focus of acceptance testing MoSCoW priorities AgileAgile -- Play the Card Game:Play the Card Game: RiskRisk--PokerPoker Poker Planning / Wide Band Delphi based Improve Quality Services BV 14
- 10. © 2011 Improve Quality Services BV PRISMA tutorial- 8 Improve Quality Services B.V. 15 Wide Band Delphi / Poker Planning (1)Wide Band Delphi / Poker Planning (1) Bottom-up estimation techniques - wide variance in estimates - commitment of people involved - accurate estimation we use the collective wisdom of the team Each estimator has cards with valid estimates A user story (product backlog item) is discussed Each estimator selects an estimate Cards are turned over Discuss differences (especially outliers) Repeat until consensus Improve Quality Services B.V. 16 User Story Explain Estimates Collect Share info OK? Final estimate Moderator Estimator OK? Wide Band Delphi / Poker Planning (2)Wide Band Delphi / Poker Planning (2)
- 11. © 2011 Improve Quality Services BV PRISMA tutorial- 9 Improve Quality Services B.V. 17 Risk Poker the same, butRisk Poker the same, but Scoring separately for likelihood and impact Discussion shall be factor based Limited number of cards for each color (value) Agreement, Product owner (impact) and lead- developer (likelihood) act as tie breaker Improve Quality Services B.V. 18 Example User Story As as bank account holder, I want a yearly financial overview of my accounts, so that I can submit my taxes Impact ? Likelihood ? Let’s PlayLet’s Play Impact: Likelihood:
- 12. © 2011 Improve Quality Services BV PRISMA tutorial- 10 Improve Quality Services B.V. 19 AgileAgile PRISMAPRISMA RiskRisk MatrixMatrix See any “problems” with this? Example User Story Improve Quality Services BV 20 Differentiated Test Approach !!Differentiated Test Approach !! Test design techniques Reviews & inspection Test design start-up meetings Reviews of test design Level of detail of test cases Exit criteria Level of independence More time & effort Most experienced person Priority setting Regression testing Re-testing without this risk assessment doesn’t make much sense !!
- 13. © 2011 Improve Quality Services BV PRISMA tutorial- 11 Improve Quality Services BV 21 RealReal--LifeLife AgileAgile ExampleExample IIII IVIV II IIIIII LL HH MM LL HMM x x x x x Likelihood Impact Must Test Should Test Could Test “Won’t Test” Test design techniques Review design Support module tests Review module tests Test design techniques Review design Exploratory Testing Support module tests Review module tests Exploratory Testing AgileAgile ““OneOne Page Test Plan”Page Test Plan” Improve Quality Services B.V. 22 Must Test .. Test Approach .. Should test Test Approach .. Could Test .. Test Approach .. Would Test .. Test Approach .. Shall include a definition of “done” A 1-2 hour session to achieve a one page Sprint Test Plan
- 14. © 2011 Improve Quality Services BV PRISMA tutorial- 12 CommunicationCommunication Improve Quality Services B.V. 23 Product Risk MatrixProduct Risk Matrix –– InitialInitial US 58 US 31US 49 US 27 Impact - Business risk Likelihood-TechnicalRisk IVIII III
- 15. © 2011 Improve Quality Services BV PRISMA tutorial- 13 Product Risk MatrixProduct Risk Matrix –– ProgressProgress US 58 US 31US 49 US 27 Impact - Business risk Likelihood-TechnicalRisk IVIII III Improve Quality Services BV 26 Key learning pointsKey learning points A structured andA structured and practical approachpractical approach forfor riskrisk--basedbased testing istesting is availableavailable Make itMake it AgileAgile, but use the formal rules, but use the formal rules Efficient, Effective and AttractiveEfficient, Effective and Attractive DefineDefine aa riskrisk--basedbased differentiated testdifferentiated test approachapproach ProvideProvide riskrisk--basedbased trackingtracking && reportingreporting
- 16. © 2011 Improve Quality Services BV PRISMA tutorial- 14 DownloadsDownloads On www.erikvanveenendaal.nl A full PRISMA white paper (e-book) Paper: 10 Recommendations on Risk-Based Testing Paper: Product Risk Assessments in Agile Projects (Risk-Poker) To acquire the PRISMA freeware tool sent a request to info@improveqs.nl Book “Practical Risk-Based Testing; The PRISMA Approach”, ISBN9-78490-986070 (www.utn.nl) Improve Quality Services B.V. 27 Improve Quality Services B.V. 28