Optimizing Your Agile Testing Processes September 23, 2008 www.uTest.com
Agile Manifesto Individuals and interactions  over  processes and tools Working software  over  comprehensive documentation Customer collaboration  over  contract negotiation Responding to change  over  following a plan
Standard S/W Theory Release! Requirements Analysis System Design Coding QA & Bug Fixes
The Reality… Release! Requirements Analysis System Design Coding QA New  Requirements Redesign Coding Boss’s Pet Feature Redesign Coding
QA is a  defensive stance… “Protect the user” mentality
Agile Development Short  Development Cycles (aka Iteration, Sprint, etc.) Don’t over design:  Only what you need for this sprint Testing integral to process Focus on software, not documentation Design / Test / Code  as a team Business User is part of cycle Iterative Releases 2-4 weeks apart Unique value in each release
Is Agile New?
Move the  deadline up! Kill all documentation! Keep coding till the  day of release! This is  NOT Agile!
Agile is VERY orderly
Test  Driven  Development Write Unit Tests Run Tests Write Some Code Run Tests Fail Fail Yes Pass More  Features? No Pass Check-in
Sample JUnit Test Case Method in JUnit that: Checks if the two arguments are equal Displays message if assertion fails The functionality we are about to develop, which needs to be tested The JUnit Test Case that we are building @Test  public void testAddition() {  assertEquals( 12 , myMathClass.add(7,5) ); }
Sample Unit Test Code @Test  public void testAddition() {  assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals(  3 , myMathClass.add(3,0) ); }
Sample Unit Test Code @Test  public void testAddition() {  assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals(  3 , myMathClass.add(3,0) ); assertEquals(  2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); }
Sample Unit Test Code @Test  public void testAddition() {  assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals(  3 , myMathClass.add(3,0) ); assertEquals(  2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); assertEquals( 133332 , myMathClass.add(55555,77777) ); }
Sample Unit Test Code @Test  public void testAddition() {  assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals(  3 , myMathClass.add(3,0) ); assertEquals(  2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); assertEquals( 133332 , myMathClass.add(55555,77777) ); assertEquals(  6 , myMathClass.add(1,2,3) ); }
Don’t worry  about that Failure… The Feature Is Not For This Release Don’t worry about that Failure… It’s a  Known Bug
Testing Phases
Compliance Matrix Complexity Testing Compliance Matrix keeps getting bigger For web apps, who is the customer in the team?  1997 1999 2001 2003 2005 2007 2009 Medium Complexity:  OS,  Hardware,  Device Drivers High Complexity: OS, Browser,  Browser Settings/Plug-ins, Firewall / Anti-Virus / Filtering, Connection Speed, Language/Region Low Complexity: Browser Version Native-Code  HTML Server-side Dynamic Web Apps
The Tester’s Role Unit Tests = Specs QA and PM meet early No more SRS    TP Alchemy Testing is part of  development process Team development calls for testing frame of mind  In a programming team / pair, not everyone is homogenous  Acceptance Testing by Customer Coordinating (and participating in)  customer testing / community testing
Case Study: Testing in Short Iterations
2 week development cycle Mon - Fri Development Develop features for  current Sprint cycle Fri 5PM Release To Testing Post the Sprint version and test plans to Testing arena Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing  by uTest Community Mon 9AM Complete Bug Report R&D Reviews Bugs Identified in Sprint version Mon - Fri Bug Fixing Stabilization and Bug Fixing. No new development Fri 5PM Release To Testing Post the Release Candidate to Testing arena Fri  5PM - Mon 9AM Testing Simultaneous worldwide testing  by uTest Community Mon 9AM Version Release Showstopper fixes,  release decision and  post to  production site
Scoping each Sprint Fri - Mon  Testing Mon - Fri Bug Fixing Fri - Mon  Testing Mon 3PM Business Priority Meeting Sales, Mktg, Prod Mgmt  choose se 6-8 candidate features for upcoming cycle Tues 11AM Sprint Planning Meeting Prod Mgmt presents candidates to  R&D  Mgmt Fri 10 AM R&D Response to  Biz Team R&D analysis is presented to business team Fri 3PM Sprint Decision Closure List of candidate features for  upcoming Sprint cycle is finalized Wed-Thurs Dev Speccing R&D Team Leaders analyze candidates for code  complexity, cost of development and risk factors Mon - Fri Development
Testing and the End-User
Thank you! Q&A For more info: http://www.utest.com/solutions_agile_testing.htm Download the “Agile Testing with uTest” Case Study: http://www.utest.com/downloadcs.php

Optimizing Your Agile Testing Processes

  • 1.
    Optimizing Your AgileTesting Processes September 23, 2008 www.uTest.com
  • 2.
    Agile Manifesto Individualsand interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
  • 3.
    Standard S/W TheoryRelease! Requirements Analysis System Design Coding QA & Bug Fixes
  • 4.
    The Reality… Release!Requirements Analysis System Design Coding QA New Requirements Redesign Coding Boss’s Pet Feature Redesign Coding
  • 5.
    QA is a defensive stance… “Protect the user” mentality
  • 6.
    Agile Development Short Development Cycles (aka Iteration, Sprint, etc.) Don’t over design: Only what you need for this sprint Testing integral to process Focus on software, not documentation Design / Test / Code as a team Business User is part of cycle Iterative Releases 2-4 weeks apart Unique value in each release
  • 7.
  • 8.
    Move the deadline up! Kill all documentation! Keep coding till the day of release! This is NOT Agile!
  • 9.
  • 10.
    Test Driven Development Write Unit Tests Run Tests Write Some Code Run Tests Fail Fail Yes Pass More Features? No Pass Check-in
  • 11.
    Sample JUnit TestCase Method in JUnit that: Checks if the two arguments are equal Displays message if assertion fails The functionality we are about to develop, which needs to be tested The JUnit Test Case that we are building @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); }
  • 12.
    Sample Unit TestCode @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); }
  • 13.
    Sample Unit TestCode @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); }
  • 14.
    Sample Unit TestCode @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); assertEquals( 133332 , myMathClass.add(55555,77777) ); }
  • 15.
    Sample Unit TestCode @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); assertEquals( 133332 , myMathClass.add(55555,77777) ); assertEquals( 6 , myMathClass.add(1,2,3) ); }
  • 16.
    Don’t worry about that Failure… The Feature Is Not For This Release Don’t worry about that Failure… It’s a Known Bug
  • 17.
  • 18.
    Compliance Matrix ComplexityTesting Compliance Matrix keeps getting bigger For web apps, who is the customer in the team? 1997 1999 2001 2003 2005 2007 2009 Medium Complexity: OS, Hardware, Device Drivers High Complexity: OS, Browser, Browser Settings/Plug-ins, Firewall / Anti-Virus / Filtering, Connection Speed, Language/Region Low Complexity: Browser Version Native-Code HTML Server-side Dynamic Web Apps
  • 19.
    The Tester’s RoleUnit Tests = Specs QA and PM meet early No more SRS  TP Alchemy Testing is part of development process Team development calls for testing frame of mind In a programming team / pair, not everyone is homogenous Acceptance Testing by Customer Coordinating (and participating in) customer testing / community testing
  • 20.
    Case Study: Testingin Short Iterations
  • 21.
    2 week developmentcycle Mon - Fri Development Develop features for current Sprint cycle Fri 5PM Release To Testing Post the Sprint version and test plans to Testing arena Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing by uTest Community Mon 9AM Complete Bug Report R&D Reviews Bugs Identified in Sprint version Mon - Fri Bug Fixing Stabilization and Bug Fixing. No new development Fri 5PM Release To Testing Post the Release Candidate to Testing arena Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing by uTest Community Mon 9AM Version Release Showstopper fixes, release decision and post to production site
  • 22.
    Scoping each SprintFri - Mon Testing Mon - Fri Bug Fixing Fri - Mon Testing Mon 3PM Business Priority Meeting Sales, Mktg, Prod Mgmt choose se 6-8 candidate features for upcoming cycle Tues 11AM Sprint Planning Meeting Prod Mgmt presents candidates to R&D Mgmt Fri 10 AM R&D Response to Biz Team R&D analysis is presented to business team Fri 3PM Sprint Decision Closure List of candidate features for upcoming Sprint cycle is finalized Wed-Thurs Dev Speccing R&D Team Leaders analyze candidates for code complexity, cost of development and risk factors Mon - Fri Development
  • 23.
  • 24.
    Thank you! Q&AFor more info: http://www.utest.com/solutions_agile_testing.htm Download the “Agile Testing with uTest” Case Study: http://www.utest.com/downloadcs.php

Editor's Notes

  • #2 Welcome everyone. Today, we’re going to discuss Quality testing in an Agile Development environment. My name is Brad Young, I’m a community evangelist here at uTest, and our experiences with Agile come both from our in house development of our own product platform, as well as working with many customers who also develop using various Agile methodologies.