Creating a high performing QA function… SimplesBy Carl Bruiners
Presumptions• Continuous Integration (CI) environmental at an embryotic stage• No Continuous Delivery (CD) model• No or li...
Note: This should form part of the teams definition of doneTeam QA Vision
What is high performing QA• Testing an integral part of the team• Automated testing• Continuous integration / delivery• Re...
Commercial Benefits
Customer• Increased frequency of new feature deliveries• Improved customer experience by reducing likelihood of encounteri...
Product• Reduced time spent on fixing defects = More time spent working onproduct features• Teams work on small incrementa...
Financials• Average daily cost of £58 p/p• Average effort time per defect 2 days• Average weekly incoming defect rate 10We...
Agile Development
Agile Application Lifecycle Management
Agile Delivery Process
• QA integrated within the team• Refactoring as the team goes along• Test Driven Development (TDD)• Pair Programming – to ...
Defects•Testers review incoming defects•Ensuring defects have enough information supplied•Identifying priority / severity ...
Continuous IntegrationAutomated testing•Automated unit testing•Automated smoke testingEnvironment•Integration server•SCM –...
Continuous Delivery – Post CI buildAutomated testing•GUI testing•Functional testing•Security testing•Performance testing•R...
Continuous delivery workflowDev test suite Test suiteBuild PerformanceUnit test SecurityCode scan (U)ATCompliancyRegressio...
The Plan
Deliverable goals – Year 1• Reduce escaping defects through implementing QC gates• Reduce incoming defects• Implement / im...
Short Term (Q1 / Q2 2012)* Note: Subject to change
Medium Term (Q3 / Q4 2012)
Long Term 2013+
My role• Reshape teams to be more test driven / orientated – Testers embedded in the teams• Ensure features being delivere...
Questions
Handouts
Fig. 1 – Cone of uncertainty
Fig. 2 – Agile test curve
Fig. 3 – CI dashboard
Fig. 4 – Prototyping
Fig. 5 – Scrum Ceremonies
Upcoming SlideShare
Loading in...5
×

Agile QA presentation

5,895

Published on

Agile High Performing QA presentation.

Sharing as there is a lot of useful reusable content.

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,895
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
115
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Work with HR on SM roles – Not having a independent dedicated Scrum Master in the team is like having a football match without a referee, the players know the rules, but we all know that they are unlikely to adhere to them. Back to basics – Some of teams have over complicated their Agile model and lost sight of some of the core values of some of our Scrum ceremonies. This includes all area’s of our model, writing INVEST’ed / SMART User Stories, creating backlog that can be grouped under a release as feature. Improve our training / coaching capabilities – Working with Linda to create a program suite of training development; from the introduction through to advanced Agile. This will include CSM’s, PO training, User Story creation and identifying suitable candidates for training, and also highlighting SPOF removal through training. My view is that all RM’s and SM’s attend the Cohn CSM course and PO’s the CPO course, only after doing the external training then do they sit the GE Agile course (use note about Epic management being incorrect). Improve our CI environments – Get everyone using the same tool Improve our consistency – Create a baseline Agile model and then allow teams to ‘sensibly evolve’, CI will be supported, change for the sake of change won’t be. Understanding a need or identifying a fault will be needed to help justify change Measuring our CI is vital, at present changes are made without any evidence of either need or their success Changing the negative culture concerning Agile – Improve our reporting capabilities – Anyone should be able to generate reports as and when they need without the dependency of another individual. On the fly, real-time reporting.
  • Decreased delivery cycles – Our delivery cycles are no longer set over years, instead we release incremental value feature releases Deliver meaningful releases aligned to features – Instead of having a fixed release pattern (i.e. every 6 weeks) we shall have variable releases against a features CD including all tests - Our CD environment includes all automated tests, this will also require all manual tests to be replaced by automated testing Fully TDD – Each team becomes test orientated, increasing our QA and reducing the number of escaping defects Improve our consistency using our Tools – Consolidating our tool base, improving integration between tools, etc…
  • Questions
  • Agile QA presentation

    1. 1. Creating a high performing QA function… SimplesBy Carl Bruiners
    2. 2. Presumptions• Continuous Integration (CI) environmental at an embryotic stage• No Continuous Delivery (CD) model• No or limited reporting around QA• Certain aspects around Development / Agile has been omitted to keep thefocus on QA
    3. 3. Note: This should form part of the teams definition of doneTeam QA Vision
    4. 4. What is high performing QA• Testing an integral part of the team• Automated testing• Continuous integration / delivery• Reducing escaping defects• Eliminating debt• Reduce uncertainty• Compliance – Accessibility, W3C, etc…• Coding standards, practices and patterns• Improving code base through refactoring• Features / functions delivered are aligned to commercial requirements• Hi-visibility of failing tests – i.e. CI / defect information radiators• Team take shared responsibility for failed tests• Guaranteed environment up time – Managed through SLA’s
    5. 5. Commercial Benefits
    6. 6. Customer• Increased frequency of new feature deliveries• Improved customer experience by reducing likelihood of encounteringa defect• Improved ability to test granular level functions without impacting onthe product (in combination with A/B or multi-variant testing).• Improved flexibility to react to competitors
    7. 7. Product• Reduced time spent on fixing defects = More time spent working onproduct features• Teams work on small incremental feature / function deliveries so thatnew features ready for production are not held back by otherunfinished features• Base product improvement will help extensibility and scalability• Feature areas are modular, encapsulated independent features whichlessens any escaping defects impacting on other features / areas ofthe product• A/B / Multi-variant testing feedback helps commercial make customerdriven decisions
    8. 8. Financials• Average daily cost of £58 p/p• Average effort time per defect 2 days• Average weekly incoming defect rate 10Weekly Monthly Annual Target Cost in daysBase £1,160 £4,640 £55,680 n/a 9601stYear £696 £2,784 £33,408 40% 5762ndYear £11.60 £46.40 £556.80 99% 9.5
    9. 9. Agile Development
    10. 10. Agile Application Lifecycle Management
    11. 11. Agile Delivery Process
    12. 12. • QA integrated within the team• Refactoring as the team goes along• Test Driven Development (TDD)• Pair Programming – to reduce SPOF’s• GUI / Functional automated testing• Set a sensible BAU percentage – 20%-30%• Development practices / patterns – i.e. OO, MVC, patterns, etc..• Defects reviewed on a daily basis by the PO and team• Testers validate defects – ensure reproducible steps• Defining a commercial focused MMFS (Minimal Marketable Feature Set)• Quality gates in swim lane – Peer review and ready for acceptance (on stage)Development
    13. 13. Defects•Testers review incoming defects•Ensuring defects have enough information supplied•Identifying priority / severity of defect/s based on commercial value•Frequently review defects•Minimise escalations•PM / POs to be inclusive of prioritisation of defectsTesting•Define smoke testing areas on the product•Adopt Test Driven Development (TDD)•Introducing Quality Control Gates (QC) into SprintsManaging Risk
    14. 14. Continuous IntegrationAutomated testing•Automated unit testing•Automated smoke testingEnvironment•Integration server•SCM – All project deliverables stored under source control i.e. images,SQL scripts, documentation, etc…•Builds should happen as frequently as possible•Continuous visible feedback on testing•Configuration management
    15. 15. Continuous Delivery – Post CI buildAutomated testing•GUI testing•Functional testing•Security testing•Performance testing•Regression testing•Smoke testingAutomated deployment•Stage•Production
    16. 16. Continuous delivery workflowDev test suite Test suiteBuild PerformanceUnit test SecurityCode scan (U)ATCompliancyRegressionGUIFunctional
    17. 17. The Plan
    18. 18. Deliverable goals – Year 1• Reduce escaping defects through implementing QC gates• Reduce incoming defects• Implement / improve CI environment• Implement CD (inc. fully automated testing)• Ensure features / functions delivered are aligned to commercialdrivers• Introduce Quality Controls into Agile process• Work with teams to ensure correct defects are targeted• Work with teams on correct best practice development practices• Measure success rates of defects and improve when needed• Promote X-team development through sharing each others journeys• Measure escaping defects
    19. 19. Short Term (Q1 / Q2 2012)* Note: Subject to change
    20. 20. Medium Term (Q3 / Q4 2012)
    21. 21. Long Term 2013+
    22. 22. My role• Reshape teams to be more test driven / orientated – Testers embedded in the teams• Ensure features being delivered on are returning health ROI• Ensuring commercial alignment;• Create Scrum of Scrums not only amongst the teams – I attend• Daily Scrum of Scrums between myself and other Senior Management – Prioritised defect list shared• Protect the teams from external influences• Reviewing defect backlogs to ensure that defects being worked on are aligned to commercial alignment• Stopping enhancements by reviewing the defect backlogs• Spot review stories to ensure stories are using I.N.V.E.S.T• Measure escaping defects• Mitigation of SPOFs• Ensuring Operations / IT have environments up against a ILSA• Bring in ISLA’s• Manage ISLA’s• Define, manage and negotiate rollback plans• Define, manage and negotiate dependency reductions – self sufficient teams / department• Managing other senior managers expectations – transparency / honesty• Protect teams from external factors that are out of scope• Manage scope creep• Resource management – Alignment of staff to peek periods or demands required to meet commercial goals.• Building strongest equal teams• Carry out an annual Agile assessment (incorporates QA)• Organising the introduction of hi-visibility information radiators• Reduce aged defects over x days old• CI our QA strategy• Reduce cost of QA through improved process and productivity improvements
    23. 23. Questions
    24. 24. Handouts
    25. 25. Fig. 1 – Cone of uncertainty
    26. 26. Fig. 2 – Agile test curve
    27. 27. Fig. 3 – CI dashboard
    28. 28. Fig. 4 – Prototyping
    29. 29. Fig. 5 – Scrum Ceremonies
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×