Optimizing Your Agile Testing Processes September 23, 2008 www.uTest.com

Agile Manifesto Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

Standard S/W Theory Release! Requirements Analysis System Design Coding QA & Bug Fixes

The Reality… Release! Requirements Analysis System Design Coding QA New Requirements Redesign Coding Boss’s Pet Feature Redesign Coding

QA is a defensive stance… “Protect the user” mentality

Agile Development Short Development Cycles (aka Iteration, Sprint, etc.) Don’t over design: Only what you need for this sprint Testing integral to process Focus on software, not documentation Design / Test / Code as a team Business User is part of cycle Iterative Releases 2-4 weeks apart Unique value in each release

Short Development Cycles

(aka Iteration, Sprint, etc.)

Don’t over design: Only what you need for this sprint

Testing integral to process

Focus on software, not documentation

Design / Test / Code as a team

Business User is part of cycle

Iterative Releases

2-4 weeks apart

Unique value in each release

Is Agile New?

Move the deadline up! Kill all documentation! Keep coding till the day of release! This is NOT Agile!

Agile is VERY orderly

Test Driven Development Write Unit Tests Run Tests Write Some Code Run Tests Fail Fail Yes Pass More Features? No Pass Check-in

Sample JUnit Test Case Method in JUnit that: Checks if the two arguments are equal Displays message if assertion fails The functionality we are about to develop, which needs to be tested The JUnit Test Case that we are building @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); }

Method in JUnit that:

Checks if the two arguments are equal

Displays message if assertion fails

Sample Unit Test Code @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); }

Sample Unit Test Code @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); }

Sample Unit Test Code @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); assertEquals( 133332 , myMathClass.add(55555,77777) ); }

Sample Unit Test Code @Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); assertEquals( 133332 , myMathClass.add(55555,77777) ); assertEquals( 6 , myMathClass.add(1,2,3) ); }

Don’t worry about that Failure… The Feature Is Not For This Release Don’t worry about that Failure… It’s a Known Bug

Testing Phases

Compliance Matrix Complexity Testing Compliance Matrix keeps getting bigger For web apps, who is the customer in the team? 1997 1999 2001 2003 2005 2007 2009 Medium Complexity: OS, Hardware, Device Drivers High Complexity: OS, Browser, Browser Settings/Plug-ins, Firewall / Anti-Virus / Filtering, Connection Speed, Language/Region Low Complexity: Browser Version Native-Code HTML Server-side Dynamic Web Apps

Testing Compliance Matrix keeps getting bigger

For web apps, who is the customer in the team?

The Tester’s Role Unit Tests = Specs QA and PM meet early No more SRS  TP Alchemy Testing is part of development process Team development calls for testing frame of mind In a programming team / pair, not everyone is homogenous Acceptance Testing by Customer Coordinating (and participating in) customer testing / community testing

Unit Tests = Specs

QA and PM meet early

No more SRS  TP Alchemy

Testing is part of development process

Team development calls for testing frame of mind

In a programming team / pair, not everyone is homogenous

Acceptance Testing by Customer

Coordinating (and participating in) customer testing / community testing

Case Study: Testing in Short Iterations

2 week development cycle Mon - Fri Development Develop features for current Sprint cycle Fri 5PM Release To Testing Post the Sprint version and test plans to Testing arena Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing by uTest Community Mon 9AM Complete Bug Report R&D Reviews Bugs Identified in Sprint version Mon - Fri Bug Fixing Stabilization and Bug Fixing. No new development Fri 5PM Release To Testing Post the Release Candidate to Testing arena Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing by uTest Community Mon 9AM Version Release Showstopper fixes, release decision and post to production site

Scoping each Sprint Fri - Mon Testing Mon - Fri Bug Fixing Fri - Mon Testing Mon 3PM Business Priority Meeting Sales, Mktg, Prod Mgmt choose se 6-8 candidate features for upcoming cycle Tues 11AM Sprint Planning Meeting Prod Mgmt presents candidates to R&D Mgmt Fri 10 AM R&D Response to Biz Team R&D analysis is presented to business team Fri 3PM Sprint Decision Closure List of candidate features for upcoming Sprint cycle is finalized Wed-Thurs Dev Speccing R&D Team Leaders analyze candidates for code complexity, cost of development and risk factors Mon - Fri Development

Testing and the End-User

Thank you! Q&A For more info: http://www.utest.com/solutions_agile_testing.htm Download the “Agile Testing with uTest” Case Study: http://www.utest.com/downloadcs.php