0
IS THE QA DEPARTMENT DEAD
IN AN AGILE WORLD
Delivering better software quality on Agile projects
9/26/13
2©Alliance Global Services 2013
Today’s Presenter
As Vice President, Solution Engineering, Sreekanth has more
than 12 year...
3©Alliance Global Services 2013
AGENDA
4©Alliance Global Services 2013
Agenda
• The case for QA in an Agile World
• Changing role of QA in an Agile World
• Role ...
5©Alliance Global Services 2013
The Case for QA in an Agile World
6©Alliance Global Services 2013
The Promise of Agile
Develop iteratively as business evolves
Deliver working software rapi...
7©Alliance Global Services 2013
Agile Challenges
Functionality/defect treadmill
High regression defect seepage
“Epic” fail...
8©Alliance Global Services 2013
What Agile Assumes
Communication
Collaboration
Access to
Stakeholders
Cross Functional
Tea...
9©Alliance Global Services 2013
Is QA Dead in this Agile World?
Some Agile proponents did not see (or argued against)
the ...
10©Alliance Global Services 2013
Agile Layers of Testing
Prep
Development
Start Wrapper Sprint2 or 4 Week Sprints
JumpStar...
11©Alliance Global Services 2013
Specialization Has Benefits
12©Alliance Global Services 2013
Role of QA in an Agile World
13©Alliance Global Services 2013
Organization empowers Health plans to transform the performance of their provider network...
14©Alliance Global Services 2013
Agile Layers of Testing
Prep
Development
Start Wrapper Sprint2 or 4 Week Sprints
JumpStar...
15©Alliance Global Services 2013
Testing Activities
Activity Developers Testers
Requirements Testing
 
Acceptance 
Unit...
16©Alliance Global Services 2013
Quality Assurance
Activity Developers Testers
Test Strategy
 
Test Planning  
Process...
17©Alliance Global Services 2013
Profile of Testers & Best Practices
18©Alliance Global Services 2013
Technology Transformation - 2002
19©Alliance Global Services 2013
Technology Transformation - 2008
20©Alliance Global Services 2013
Technology Transformation - 2013
21©Alliance Global Services 2013
Distributed Teams
22©Alliance Global Services 2013
Communication Realities
23©Alliance Global Services 2013
Skillsets for an Agile Tester
Risk Assessment
Test Strategy &
Planning
Test Design &
Exec...
24©Alliance Global Services 2013
Automation Best Practices
25©Alliance Global Services 2013
Automation Objectives
26©Alliance Global Services 2013
Layers of Automation
Layered Automation Testing Strategy for end to end coverage, automat...
27©Alliance Global Services 2013
Automation - Best Practices
Automation is
development
Automation
Architecture
Lagging spr...
28©Alliance Global Services 2013
Environments
Test Management
CI Integration
Reporting
Infrastructure
29©Alliance Global Services 2013
• Re-usable automated test
suites with minimal
maintenance effort
• Cross application E2E...
30©Alliance Global Services 2013
Metrics
31©Alliance Global Services 2013
Metrics
32©Alliance Global Services 2013
Measuring Effectiveness
Automation initiatives need metrics reporting and analysis as an ...
33©Alliance Global Services 2013
Key Metrics for Agile Projects
•Use Validation points to evaluate application’s map with
...
34©Alliance Global Services 2013
Metrics & Effectiveness Measurements
Detailed execution metrics will be captured to glean...
35©Alliance Global Services 2013
36©Alliance Global Services 2013
Thank You
Sreekanth Singaraju
VP of Solution Engineering
ssingaraju@allianceglobalservice...
37©Alliance Global Services 2013
Coverage Metrics
Principle
Communicate to all stakeholders the coverage provided by forma...
38©Alliance Global Services 2013
Defect Identification Efficiency
Principle
Identify how defects are identified and create...
39©Alliance Global Services 2013
Technical Debt
Principle
Provide a financial impact of technical choices on the overall p...
40©Alliance Global Services 2013
Confidence Level
Principle
Demonstrate the confidence of the testing team in the testing ...
Upcoming SlideShare
Loading in...5
×

Is the QA Department Dead in an Agile World?

875

Published on

Lots of Agile pundits proclaim that the role of QA Department in an Agile world is going by the way of the stenographer pool – soon to be extinct overtaken by new ideas and technologies.

WE AGREE…if by QA Department they mean the traditional QA departments using approaches like a Double-V model of verification and validation. These QA departments are already extinct in most modern Enterprise Development teams which are using Agile approaches to develop multi-platform applications, deployed on cloud, delivered through mobile, web and native, by teams located all around the world.

However, there is a smarter, nimbler and multifaceted QA organization taking their place and delivering substantial value in these organizations. In this webinar, I will present some of the key facets of these successful organizations including real life examples from our varied experiences.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
875
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
40
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Is the QA Department Dead in an Agile World?"

  1. 1. IS THE QA DEPARTMENT DEAD IN AN AGILE WORLD Delivering better software quality on Agile projects 9/26/13
  2. 2. 2©Alliance Global Services 2013 Today’s Presenter As Vice President, Solution Engineering, Sreekanth has more than 12 years of senior technology leadership experience and leads Alliance’s Solution Engineering organization in developing cutting edge solutions. He is a respected technology thought leader with a focus in testing Agile projects, test process automation and metrics-based development approaches and utilizing lean techniques in software development. Today’s Host As Director of Marketing, Sharon Lee heads the marketing strategy and brand messaging for Alliance. With over 12 years of experience in both digital and traditional marketing, she is focused on the effective use of media for compelling brand messaging and creating successful marketing programs with measurable results that impact revenue. Sharon holds a B.A. from the University of Pennsylvania.
  3. 3. 3©Alliance Global Services 2013 AGENDA
  4. 4. 4©Alliance Global Services 2013 Agenda • The case for QA in an Agile World • Changing role of QA in an Agile World • Role of QA in Agile organizations • Profile of Testers and QA Analysts for these organizations • Automation best practices • Metrics and measurements
  5. 5. 5©Alliance Global Services 2013 The Case for QA in an Agile World
  6. 6. 6©Alliance Global Services 2013 The Promise of Agile Develop iteratively as business evolves Deliver working software rapidly Flexibility to change with business needs Incorporate end-user feedback Highly productive technical teams
  7. 7. 7©Alliance Global Services 2013 Agile Challenges Functionality/defect treadmill High regression defect seepage “Epic” failures Non-functional requirements fail Low effective team throughput
  8. 8. 8©Alliance Global Services 2013 What Agile Assumes Communication Collaboration Access to Stakeholders Cross Functional Teams
  9. 9. 9©Alliance Global Services 2013 Is QA Dead in this Agile World? Some Agile proponents did not see (or argued against) the need for QA specialists on an assembled team. If you use TDD, Unit Testing & “Everyone Tests their own Stuff” do we need a dedicated QA team?
  10. 10. 10©Alliance Global Services 2013 Agile Layers of Testing Prep Development Start Wrapper Sprint2 or 4 Week Sprints JumpStart 24h 24h Harden, Docs, Release Vision ProductBacklog Sprint Backlog Tested, Working Software Increment Product X Sprint ReviewSprint Planning Release Planning Sprint 1 Sprint 2 Sprint n 3 - 5 Sprints Prototyping Reqs Testing Unit Testing Regression Integration Compatibility Performance Security Quality Assurance Acceptance Functional Deployment
  11. 11. 11©Alliance Global Services 2013 Specialization Has Benefits
  12. 12. 12©Alliance Global Services 2013 Role of QA in an Agile World
  13. 13. 13©Alliance Global Services 2013 Organization empowers Health plans to transform the performance of their provider networks, while significantly reducing medical, administrative and IT costs and simplifies the design, maintenance, reimbursement and performance management of provider networks, while facilitating provider-patient collaboration. Development Objectives: • Rapid release of new solutions and rich new features of their core Platform into market to take advantage of increased health care IT spending in the payer market due to health insurance reform (PPACA) • Ensure high level of customer satisfaction dependent on user experience, maintainability, scalability and platform performance • Ensure the highest levels of security and confidential data access meeting all regulatory needs and customer specific needs Development Organization: • TDD approach for Developers engaged in 2-week sprint cycles • Enterprise development technologies • Interfaces with several in-house and 3rd party development teams and applications • SCRUM, Kanban and Lean techniques Organization Profile
  14. 14. 14©Alliance Global Services 2013 Agile Layers of Testing Prep Development Start Wrapper Sprint2 or 4 Week Sprints JumpStart 24h 24h Harden, Docs, Release Vision ProductBacklog Sprint Backlog Tested, Working Software Increment Product X Sprint ReviewSprint Planning Release Planning Sprint 1 Sprint 2 Sprint n 3 - 5 Sprints Prototyping Reqs Testing Unit Testing Regression Integration Compatibility Performance Security Quality Assurance Functional Deployment Acceptance
  15. 15. 15©Alliance Global Services 2013 Testing Activities Activity Developers Testers Requirements Testing   Acceptance  Unit Testing  Functional/Exploratory Testing   Regression Testing  Compatibility Testing  Performance Testing  Security Testing  Deployment Testing 
  16. 16. 16©Alliance Global Services 2013 Quality Assurance Activity Developers Testers Test Strategy   Test Planning   Process Compliance  Root Cause Analysis  Metrics & Reporting  
  17. 17. 17©Alliance Global Services 2013 Profile of Testers & Best Practices
  18. 18. 18©Alliance Global Services 2013 Technology Transformation - 2002
  19. 19. 19©Alliance Global Services 2013 Technology Transformation - 2008
  20. 20. 20©Alliance Global Services 2013 Technology Transformation - 2013
  21. 21. 21©Alliance Global Services 2013 Distributed Teams
  22. 22. 22©Alliance Global Services 2013 Communication Realities
  23. 23. 23©Alliance Global Services 2013 Skillsets for an Agile Tester Risk Assessment Test Strategy & Planning Test Design & Execution Requirements & Acceptance Metrics Data Deployment CI Cloud UX
  24. 24. 24©Alliance Global Services 2013 Automation Best Practices
  25. 25. 25©Alliance Global Services 2013 Automation Objectives
  26. 26. 26©Alliance Global Services 2013 Layers of Automation Layered Automation Testing Strategy for end to end coverage, automated solutions for each tier and testing on actual devices UX Testing Services & API Testing Unit Testing
  27. 27. 27©Alliance Global Services 2013 Automation - Best Practices Automation is development Automation Architecture Lagging sprints Non-functional Integration with CI
  28. 28. 28©Alliance Global Services 2013 Environments Test Management CI Integration Reporting Infrastructure
  29. 29. 29©Alliance Global Services 2013 • Re-usable automated test suites with minimal maintenance effort • Cross application E2E automated testing • Excel based test suite • Detailed test summary reports along with step wise description and screen shots • Test scripts, test reports integrated with QMS • Automated 55% of regression scenarios which includes 70% of business critical scenarios. Test Automation Framework
  30. 30. 30©Alliance Global Services 2013 Metrics
  31. 31. 31©Alliance Global Services 2013 Metrics
  32. 32. 32©Alliance Global Services 2013 Measuring Effectiveness Automation initiatives need metrics reporting and analysis as an integral part of the initiative to ensure success of the initiative. • Metrics should capture both business level and execution level values to ensure efficiency is accurately captured and reported • Metrics should not be left for the end of the project to implement • Metrics measurements allow implementation level course corrections or expectations management easier • Business Objectives outlined should be directly tied to the metrics program and measured through out the initiative • Assumptions and approximations can be made at the beginning of the initiative
  33. 33. 33©Alliance Global Services 2013 Key Metrics for Agile Projects •Use Validation points to evaluate application’s map with requirements •Communicate to all stakeholders the coverage provided by formal testing Coverage Metrics •Number of defects identified in different stages of software lifecycle •Number of defects created in different stages of software lifecycle Defect Identification Efficiency •Quality of software code •Impact of technical or architectural choicesTechnical Debt •Demonstrate the confidence of the testing team in the testing efforts executed on a release with the resources provided for testingConfidence Level
  34. 34. 34©Alliance Global Services 2013 Metrics & Effectiveness Measurements Detailed execution metrics will be captured to glean insights about test process, execution and SDLC process.
  35. 35. 35©Alliance Global Services 2013
  36. 36. 36©Alliance Global Services 2013 Thank You Sreekanth Singaraju VP of Solution Engineering ssingaraju@allianceglobalservices.com www.allianceglobalservices.com
  37. 37. 37©Alliance Global Services 2013 Coverage Metrics Principle Communicate to all stakeholders the coverage provided by formal testing Dimensions • Use Validation points to evaluate application’s compliance with requirements • Weight them by business criticality Definition Designed Coverage = Designed Validation Points / All Validation Points (Estimated) Executed Coverage = Executed Validation Points / All Validation Points (Estimated) Best Practices • Keep coverage metrics simple • Introduce granularity to address business needs • Trend to show progress (or declines – great case for additional resources)
  38. 38. 38©Alliance Global Services 2013 Defect Identification Efficiency Principle Identify how defects are identified and created. Evaluate compliance of Agile principles. Dimensions • Number of defects identified in different stages of software lifecycle • Number of defects created in different stages of software lifecycle • Use key Software stages to draw valuable inferences Definition Proportion of defects identified by each stage of a lifecycle – Ex – Testing – 56%, UAT – 18%, Production - 4%, other - 22% Best Practices • Start with defects identified first – then introduce defects created • Use “Other” to catch exceptions and introduce granularity incrementally • Factor in defect severity if that information is readily available
  39. 39. 39©Alliance Global Services 2013 Technical Debt Principle Provide a financial impact of technical choices on the overall product resulting from technology, process and implementation choices made during development Dimensions • Quality of software code • Unresolved defects that are business or quality critical • Impact of technical or architectural choices Definition Cost estimates for redressing most critical (Business perspective) defects or technical choices Best Practices • Explosive- Handle and communicate with care • Be conservative – use a very strong criteria to evaluate “criticality” • Use several available tools to evaluate software code quality
  40. 40. 40©Alliance Global Services 2013 Confidence Level Principle Demonstrate the confidence of the testing team in the testing efforts executed on a release with the resources provided for testing. Informational • Test Execution Coverage • Expertise of resources to test the application • Quantitative results of defects including Defect identification efficiency Definition Based on the evaluation of the 3 dimensions above rank each of them on a scale of 0(Poor) – 1(High). Overall confidence = (3.Coverage.Resources.Results)/(Coverage+Resources+Results) Best Practices • Be prepared with supporting facts • Define clearly laid out criteria for ranking
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×