The Future Tester at Suncorp - A Journey of Building Quality In Through Agile


Published on

Presentation delivered by Craig Smith, Adrian Smith with Dallas Thorneycroft at STANZ 2011 in Melbourne, Australia.

When Suncorp started down the path of rolling out its agile program over four years ago, it was viewed by many internally and the industry with much scepticism and angst, yet now it is approaching mainstream adoption in the industry. One of the key challenges of becoming agile was improving our approach to testing and quality. In this talk we will talk about why we had to change, why we had to improve the “speed to cool” in relation to testing, our challenges and approach and our blueprint for the “future tester” at Suncorp. Like our agile journey, our vision for testing has been regarded an ambitious, so join us to hear why we believe raising the profile, empowerment and skillset of testing is critical to our (and your) future success.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Future Tester at Suncorp - A Journey of Building Quality In Through Agile

  1. 1. The FutureTester atSuncorp: Craig Smith Adrian SmithA Journey of withBuilding Quality Dallas ThorneycroftIn Through Agile
  2. 2. Welcome…Image: © Walt Disney Pictures / Pixar C A
  3. 3. Welcome… Not Related…Image: © Universal Pictures C
  4. 4. First, let’s go back in timeImage: © Universal Pictures C
  5. 5. Waterfall …1970Image: C
  6. 6. Read past page 2!Image: C
  7. 7. 1980’s… 1990’s…Image: C
  8. 8. One TPS report too many…Image: C
  9. 9. 2001XP C
  10. 10. XP: Extreme ProgrammingImage: Beck, Kent – Extreme Programming Explained: Embrace Change C
  11. 11. Scrum XP: Extreme ProgrammingImage: A
  12. 12. ScrumImage: A
  13. 13. Agile: An Umbrella TermImage: A
  14. 14. Then… in Suncorp landImage: David Jackmanson on Flickr A
  15. 15. A leader with a visionImage: © SlatteryIT C
  16. 16. Speed to coolImage: Pineapples101 (Flickr) C
  17. 17. Thumbs up… Cool!Image: C
  18. 18. Choke...Image: C
  19. 19. CSI: SoftwareImage: © Jerry Bruckheimer Television C
  20. 20. Agile & TestingImage: C
  21. 21. To Do Analysis Develop Test Done We reflect work like this... C
  22. 22. To Do Analysis Develop Test Done Which is really workinglike this... C
  23. 23. Remind you of something?Image: Craig Smith C
  24. 24. Testers in an Analyst / agile team Customer Project Elaboration / Specification Acceptance Criteria Manager High Performing AGILE Developer Automation Tester C
  25. 25. Testers in an agile team Analysts need more testing involvement AnalystTesters need moretechnical involvement Tester Developer Developers need more testing involvementMore Less Requirements Functional / ExploratoryTechnical Technical Automated Customer Specialist Unit Test Manual Design Collab. Code Tests Test Test C
  26. 26. Everyone likes to be seen to value qualityImage: © Reveille Productions C
  27. 27. Quality is everyone’s responsibilityImage: © AP C
  28. 28. The JourneyImage: C
  29. 29. Number 1 ProblemImage: C
  30. 30. Slow ExpensiveImage:
  31. 31. Fighting Fires CImage:
  32. 32. Problem &Proposed Solution A
  33. 33. Building Quality InImage: A
  34. 34. Training & Processes & Roles & Tools & Comms & Coaching Practices Responsibility Infrastructure ChangeAgile Create central Define minimum Preferred tools Testerassessment tool repository for skill set vs technology Surveyfor software QA artefacts matrixquality and Create software Invigoratetesting maturity Create central / QA hiring guide Investigate testing agreed QA opportunities to communityUp-skill existing strategy Define skills for simplify testAgile coaches in Agile team data and Internal /testing practices Create standard members in environments External templates relation to QA CommsUpdate courses Create ready-to- Strategyand training Create glossary Review position go test toolguides descriptions and packages Create BVC career roadmap template for testing rolesCreate Agile Create workingquality/testing example ofchampions Create Agile Define skills for acceptance software QA speciality testing driven one-pagers rolesAssist in-flight developmentprojects Streams... C A
  35. 35. Processes & PracticesImage: A
  36. 36. Success sliders Cost Scope Quality Time Fixed 1 X 2 X 3 XFlexible 4 X A
  37. 37. Cost Scope Quality Time Your Fixed 1 X Turn! 2 X 3 X Flexible 4 XImage: A
  38. 38. Quality almost always at number 3 Cost Scope Quality Time Fixed 1 X 2 X 3 XFlexible 4 X A
  39. 39. What does quality meanDefining 1 Quality Advocates to the different roles in the team?Quality 2 Quality Taxonomy What are quality attributes? Quality How do we know what 3 Prioritisation quality attributes to include? What are risks with the 4 Quality Tradeoff Risks quality attributes we are trading off? 5 Quality Measurement How do we test and measure quality? 6 Success Sliders How does quality relate to the sliders? 7 Quality Definition What does quality mean? 8 Next Steps How do we apply quality to our work C
  40. 40. QualityDefinition C
  41. 41. Shared understanding of qualityImage: © 2008 Melissa McKenney C
  42. 42. Simple QA TestStrategy& Plans A
  43. 43. Reporting PROJECT TESTING 9 25 7 5 8 New Risks & Issues Raised Number of Risks & Issues 6 7 20 Number of Defects 4 Number of Tests 5 6 15 3 5 4 4 3 10 2 3 2 2 5 1 1 1 0 0 0 0 1 2 3 4 5 6 7 8 HEALTH 1 2 3 4 5 6 7 8 TEST Iteration Iteration COVERAGE OVERALL DEVELOPMENT USER 5700 1800 9 $30 5600 1600 8 5500 $25 1400 7 Number of Features Lines of Test Code 5400 Business Value Lines of Code 1200 6 $20 5300 1000 5 5200 $15 800 4 5100 600 3 $10 5000 4900 400 2 $5 4800 200 1 4700 0 0 $-MAINTAINABILITY 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 PERFORMANCE Iteration Iteration A
  44. 44. Tools & InfrastructureImage: A
  45. 45. ATDD /Specification ATDD Lifecycle by example Feature Discuss Discuss feature requirements Identify SME BA Test Dev Identify new features BA Test Dev or requirements Refactor Distill Distill into stories Stories TDD with acceptance Cycle criteria Demo Green Red Demonstrate feature and verify using exploratory testing Software Develop SME BA Test Dev Develop the code and instrument tests Test Dev A
  46. 46. ATDD /Specification Living Documentation by example C
  47. 47. Preferred Tools Best (Specification by Example) Better (ATDD / BDD) Good (Automation) Traditional / Isolated (Manual Tester Focussed) C
  48. 48. Training & CoachingImage: C
  49. 49. Training C
  50. 50. Testing ChampionsImage: C
  51. 51. In-flight Project AssistanceImage: © Little Britain Productions / BBC Productions C
  52. 52. Quality MaturityImage: © Gracie Films / 20th Century Fox Television A
  53. 53. Agile QualityPractices A
  54. 54. Team Self-Assessment A
  55. 55. QualityRequirements A
  56. 56. QualityDevelopment A
  57. 57. QualityTesting A
  58. 58. Quality TeamImage: A
  59. 59. Your Turn!Image: C A
  60. 60.  Test artifacts are treated with the same importance as code and are4 properly maintained and refactored  The relevant ____ility and security tests form part of the test strategy  Non-functional requirements are agreed and included in the test strategy3  Test planning is based on risk (technical complexity and business impact)  Test strategy is agreed and understood by the team and includes2 targets (coverage, defects, …)  Test strategy is documented and visible as a BVC used by the team  An agreed approach to testing that is aligned to a definition of quality1  Tests are considered a valuable asset that can be reused as features are progressively implemented  No agreed approach to testing Test-1  Tests are not considered a valuable asset Planning C A
  62. 62. Collaboration: Estimating: Recruitment & Development: I work with analysts and SMEs to I am involved in developing I am attracted to the organisation ensure that testable acceptance estimates for projects at a story because testing is a cool career path criteria are created for all stories level, for tools and infrastructure that offers heaps of opportunities and a and also for deployment/release way of continually developing my skills activitiesAutomation:I work with developers toautomate tests that provide Reporting:business value and identify I provide metrics that givesystem risks insight into project health and system qualityStrategy & Planning:I am involved in the project at all Agile:stages to ensure that testing I understand Agile project deliveryprovides the greatest value and and the differences between testingquality objectives are achieved in the different phases of an phase Qualifications & Training:Tools & Techniques: I have recognised qualifications inI use both manual and testing and continually update andautomated techniques using the maintain my skillspreferred testing tools based onthe situation Architecture: I have an understanding of the system architecture and can create tests that Community & Teams: verify individual components and the I am part of a community of Development: system as a whole testers that are embedded I take an interest in within teams but share development practices and common values monitor code quality metrics Future tester C A
  63. 63. Transition Plan New Graduates,Recruits, Consultants & PartnersCurrent Test Analysts New Software Test Assessment Engineer Role Capable + Current Skills + Desire Up Capable + Desire Skill Not Capable + Desire BA Not Capable + No Desire SME Other **Assessment = Technical Test + Interview** C A
  64. 64. Communications & ChangeImage: C
  65. 65. Internal / External Communications StrategyImage: C
  66. 66. Invigorate Testing CommunityImage: C
  67. 67. Case Study
  68. 68. RegroupImage:
  69. 69. MAF
  70. 70. Story + Acceptance Test Criteria Scenarios Test Case Capture MAF for Selenium MainFrame for WebAutomation Executable ContinuousProcess - TAF Specification Integration
  71. 71. Wrap UpImage: © Chuck Lorre Productions C
  72. 72. Agile testing is a journeyImage: A
  73. 73. Measure, Improve and keep improving your approachImage: © Talent Associates C
  74. 74. Testing skills are still really, really, important CImage: © Jerry Bruckheimer Television
  75. 75. Are you hunting down best practices?Image: © King Features Syndicate / Lee Falk A
  76. 76. Testers need to be viewed as super heroesImage: © DC Comics A
  77. 77. Don’t forget to use(or improve) our quality templates C
  78. 78. Testing needs to be coolImage: © Miller-Milkis Productions C
  79. 79. Craig Smith Questions ??? Dr. Adrian Smith @adrianlsmith
  80. 80. Suncorp is one of Australia and New Zealands largest diversified financial servicesproviders, supplying banking, insurance and wealth management products to around9 million customers through well-established and recognised brands such as AAMI,Australian Pensioners Insurance Agency, Shannons, Vero, Asteron and Tyndall, aswell as Suncorp and GIO. Today, Suncorp is Australias fifth largest bank and secondlargest domestic general insurance group, with over 16,000 staff. Suncorp hasrepresentation in 450 offices, branches and agencies throughout Australia and NewZealand. ( is an technology company that has focused on providing the engineering andconstruction industries with innovative products and services that improve projectexecution and operational efficiency. Ennova base their the project delivery approachon combination of Lean and Agile principles and practices. This provides clients witha high level of transparency and real value for money. Consulting engagements areundertaken using a pragmatic and common sense approach that is aimed atincreasing the clients internal capability and building business agility.( Agile Academy is the next wave in Agile learning and collaboration. Designed forIT professionals, the Agile Academy is a knowledge hub that promotes Agile relatedlearning, knowledge sharing and capability development across the IT industry. Weoffer an integrated curriculum of leading edge courses across the entire solutiondevelopment life cycle, covering all roles at varying levels.(