Agile Development Case Study and Workshop
Background Craig Parsons Revolution IT, Melbourne Dev projects - 9 years Agile dev projects  - 3 years QA and Release Manager, Rightmove.co.uk  Agile since 2005 Lots of concurrent projects Adopting Agile changed everything! Cohesive and productive teams  
This Session…. What is Agile? Becoming Agile Dealing with change Team structure Streaming iteration output – uninterrupted dev and test cycles Making it work Procedures and policies Quality checkpoints Communication and collaboration  Managing releases Lessons learned and observations
What is Agile? agile   Adjective 1 . quick in movement; nimble  2 . mentally quick or acute [Latin  agilis ]  Iterative development Analysis -> design -> code -> test -> release small components (stories) Relies heavily on collaboration and effective communication Cross functional teams A team is not  doing  agile, they  are  agile.
Manifesto for Agile Software Development Individuals and interactions  over processes and tools Working software  over comprehensive documentation Customer collaboration  over contract negotiation  Responding to change  over following a plan  http://agilemanifesto.org/
The Agile World Planning  Requirements and stories Estimating Test cases Technical design Meetings and communicating Iteration kickoff Stand-ups Walkthroughs Showcases Build meetings Pre-release meetings Retrospectives Managing iterations Pre-test entry criteria Pre-release exit criteria Continuous integration / build systems Dealing with issues/bug fixes
Cross Functional Teams Before Agile QA / Release Java Oracle BA / PM SYS
Cross Functional Teams Agile Team Structure QA / Release Agile 2 TL BA J DB D J QA Agile 1 Agile 2 Agile 4 Support SYS
Becoming Agile Strange at first Benefits for test quickly evident Transparency took getting used to Not instantly popular Reorganise teams to become Agile Much easier to allocate resources Responsibilities clear Formalise Agile team QA remain objective, whilst contributing to Agile teams
Working Guidelines for Agile Stories Estimating Walkthroughs Quality checkpoints Iteration kickoff Daily stand-ups Showcases Retrospectives
Story Guidelines Should be clear and specific  Enough basis to develop without questions Enough basis to write detailed tests Point of reference when there are disagreements If there is ambiguity, ask the BA to update the story! Everything that happens within a project should be in a story Example summaries: As a  public user I’d like to have my saved search results emailed to me As an admin user I’d like to manage customer contact details. Migrate data to new USER table on QA
Example Story ID: RM_209 Summary:  As a user I want to register a new login Requirements: Registration screen must comply with page template (refer story RM_205) Username must be no less than seven characters Username should be unique so that no two registered users have the same username Username must have alpha numeric characters Password must be more than six characters When user enters password text it should appear marked as ‘*’ s Meaningful confirmation message must return once registration is successful Passwords must be hashed in the database User info must be written to new table RMUSER with columns USERID, UNAME, PWORD (refer DB update RM_227)
Story Lifecycle (1) Analysis  Write story  Story review Dev and Test estimates Compile iteration based on estimates Iteration kickoff meeting  Iteration Dev phase (Dev) Develop story /unit tests/code review  (QA) Write test cases
Story Lifecycle (2) Dev walkthrough story with QA and BA (BA) Review work for story relevance (Dev) Review test cases (QA) Ensure unit tests/code reviews are done Iteration QA phase (test ->  fix -> test ….) Showcase / signoff Release story (iteration) Retrospective Stand ups monitor progress throughout!
Story Lifecycle Analysis Write Review Estimate Allocate Iteration KO Develop Prep Tests Walkthrough QA Showcase Release Retrospective
Estimating Estimates occur before a story is allocated to an iteration Estimates should be measured in story points Most effort = 5 points Least effort = 0.5 points Iterations have max capacity (e.g. 30 points) Present range of stories to align effort and capacity Make estimates fun  5 pts =  0.5 points =  Dev and test estimate separately  A dev  (small library change) may be a tester’s  (requiring lots of regression)
Estimate board – example Iteration 8 estimates Story Dev Test RM_234 Update all pages with new logo and disclaimer .05 RM_246 As a user I want to be able to register a new login  2 RM_289 Re-factor batch libraries  4 4 RM_267 Compose and distribute useability test invitations 0 RM_765 Add opt in / opt out config in Site Admin 2 RM_256 Migrate logins to new RMUSER table on QA Total points 27.5 13.5 14
Walkthroughs Dev, QA and BA present Dev walks through changes in dev environment Handover point from dev to test Gives BA chance to review work for adherence to story Dev and BA review test cases QA reviews entry criteria Code reviews (did it pass, any issues?) Unit tests
Entry Criteria Helps keep the  train moving Developer cannot pass the story to QA until: Unit tests have been written/updated to accommodate change and passed (Cruise) Peer code review has passed CVS / Subversion logs for changed added to story Ensures quality of work before test/release effort Encourages developer collaboration Traceability Reduce risk Unit tests always up to date
Exit Criteria Assessed by the entire team before release BA, QA/Release, Dev, SYS Story should not be released with iteration until: Walkthrough done and stakeholders signed off Testing completed (verification, regression, performance) Outstanding defects reviewed Outstanding entry criteria satisfied Risk analysis OK
Iteration Kickoff Brief Occurs after previous retrospective Overhang from last iteration? Run through all stories allocated to the iteration  Review estimates Make sure everyone is ready Risks “ Go team!”
Daily Stand Ups Every morning (without fail!) Run by project team leader No longer than 15 minutes All project team must discuss: What I did yesterday What I’m doing today Any issues: Test environment System issues Release to environments Dependencies Entry criteria not satisfied
Stand-ups - considerations No excuses for suffering in silence   Everyone should leave clear on what they should be doing and  what everyone else is doing! Learned to: Manage dominators Encourage quieter team mates
Showcase Occurs prior to release and after QA complete Showcase new and changed stories for iteration All dev/test/business stakeholders present Platform for official release signoff  Transparency of work Stakeholders feel involved What it took to get to this point Marks team achievement!
Retrospectives Lead by team leader or BA Occur after iteration is released / completed! All stakeholders should be  present Minutes recorded on whiteboard Discuss: What was released? What didn’t make it? Outstanding defects What went well? (around the room) What didn’t go well? (around the room)  How will we prevent these points in the future? Agree a way forward – everyone!! Distribute minutes, and refer back to them!
Agile Iteration and Release Management Release cycles Communication! Release meetings Release  tags Cruise Control + Selenium Roles and responsibilities
Release Management - Agile Release to Production every two weeks Four week iterations (2 dev + 2 QA) QA team responsible for testing and managing releases Release management roster Testers are gatekeepers throughout process Work with tags, so dev can commit during regression Integrated release system – Cruise Control + Selenium Environments Dev  (to create) Test (to verify) QA/Pre-Prod (for regression, performance, session replication, fail over, etc)  Prod  (the real world)
Iteration Release Cycle (1) Story allocated to iteration  and  release number in JIRA. Build manager exports list of changes for build and books build meeting All developers, QA and system architects present Build manager runs through list to ensure change passed in Cruise and QA have started testing Discuss high risk changes in detail Rollback procedure BM pushes Cruise build to test environment at request  Sending notification each time
Iteration Release Cycle (2) At end of verification phase, BM creates tags and deploys to QA env... Regression test phase Pre-release meeting Review exit criteria for release Compose release notes to distribute company wide All repositories to be released CVS tags Database changes
Iteration / Release Cycle Phase 1 Phase 2 Weds  Thurs - Tues Weds Thurs - Fri Mon Tues Prod  Release Stand-up Stand-up Stand-up Stand-up Stand-up Stand-up Send release notes Deploy to test Performance tests Merge tags Verification tests Verification Tests Regression tests Regression Tests QA signoff  Deploy trunk to test env. Fix defects Dev next iteration Dev next iteration Dev next iteration Dev next iteration Retrospectives Dev next iteration Build meeting Showcase Prepare release  notes Iteration kickoff Create tags Story reviews Pre release meeting Walkthroughs Deploy to QA env. Agile Team Developers Testers Build Manager Systems
Communicating Builds Release Manager responsible for communicating build information Vital for project team to know when changes have been deployed (to any environment) Any delays could delay project! Any build related questions, ask the build manager Email group,  Release  setup for build communication. (incl. QA, Dev, BA’s, SYS, anyone else who wants to know) Notification sent to  Release  for every deployment to test and QA environments
Build Emails - Sample To:  Release Group Subject:  New build released to QA Good morning, A new build has successfully been deployed to <environment> The updated modules are: publicsite/trunk batch/trunk admin/trunk They include all changes passed through Cruise as at <time> Regards, Your friendly build manager
Build Email - Sample To:   Release Group Subject:  Deployment issues Good morning, We are currently experiencing issues with our deployment  system and cannot release any builds until they have been  resolved. Resolving this is our highest priority. I apologise for the delay, and will keep you informed.  Regards, Your friendly build manager.
Build Preview Meeting Occurs mid iteration Just before release Tag is created  Ensure everything is ready for the build Helps gather information about release items Identify risks introduced by new code (what does this impact?)  Provides visibility to the build Platform to discuss/enforce entry criteria Opportunity for Systems team to assess risk – especially DB changes
Pre-Release Meeting Official handover from QA to SYS Platform to discuss release with all stakeholders. QA / Release Manager runs through Release Notes Highlight issues found during testing Identify risks associated with release Agree a release plan DB changes first Order of modules Updates to cron jobs
Config Management Before Agile – we had commit freeze during testing Needed to keep the train moving Lots of CVS project modules Lots of testing and developing at the same time Developers should be able to commit at any time regardless of cycle No measure of quality before QA starts Solution  Create live tags during regression phase for release Implement and maintain Selenium tests for all modules to run at build time
Releasing Tags Trunk Tag/snapshot Iteration 1 Dev/Verify Iteration 1 Fix/Regression Iteration 2 Dev Release Merge Tag
Builds – Cruise Control CVS Module1 Module2 Module3 Module4 Cruise Control Selenium Tests Selenium Tests Selenium Tests Selenium Tests Alert! Build Failed! M1Build_145 Commit  new  change M2Build_097 M3Build_225 M4Build_002
Deploying Cruise builds M1Build_145 Deploy  Test Pre-Production Production
Integrating Changes Change is not ready to test until it is part of Cruise build  Compiled, integrated and Selenium passed  Developer’s responsibility to monitor their change building after committing in CVS Immediate action required if build fails Module is locked out until last commit passes Cruise
When it can go wrong…. Cut corners Ineffective communication Sticking to process when it doesn’t work Not learning from retros Preparing for meetings Do things without telling anyone Dev changes System changes Data refresh…
Observations - Agile Responding to change doesn’t create hassle Little to no rework after release Much better relationship between IT and business More team bonding within IT Cross functional teams promoted multi-discipline learning –  and appreciation! Better working relationships Faster throughput of work Issues resolved promptly! Consider dependencies!!
Review Agile overview Agile team structures Story guidelines Story lifecycle Dev and Test estimates Walkthroughs / quality checks Stand-ups Showcases Retrospectives Managing iterations Config management and build systems Release management and communication
Questions? Comments? Thoughts?

Agile Testing and Release Management

  • 1.
    Agile Development CaseStudy and Workshop
  • 2.
    Background Craig ParsonsRevolution IT, Melbourne Dev projects - 9 years Agile dev projects - 3 years QA and Release Manager, Rightmove.co.uk Agile since 2005 Lots of concurrent projects Adopting Agile changed everything! Cohesive and productive teams 
  • 3.
    This Session…. Whatis Agile? Becoming Agile Dealing with change Team structure Streaming iteration output – uninterrupted dev and test cycles Making it work Procedures and policies Quality checkpoints Communication and collaboration Managing releases Lessons learned and observations
  • 4.
    What is Agile?agile Adjective 1 . quick in movement; nimble 2 . mentally quick or acute [Latin agilis ] Iterative development Analysis -> design -> code -> test -> release small components (stories) Relies heavily on collaboration and effective communication Cross functional teams A team is not doing agile, they are agile.
  • 5.
    Manifesto for AgileSoftware Development Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan http://agilemanifesto.org/
  • 6.
    The Agile WorldPlanning Requirements and stories Estimating Test cases Technical design Meetings and communicating Iteration kickoff Stand-ups Walkthroughs Showcases Build meetings Pre-release meetings Retrospectives Managing iterations Pre-test entry criteria Pre-release exit criteria Continuous integration / build systems Dealing with issues/bug fixes
  • 7.
    Cross Functional TeamsBefore Agile QA / Release Java Oracle BA / PM SYS
  • 8.
    Cross Functional TeamsAgile Team Structure QA / Release Agile 2 TL BA J DB D J QA Agile 1 Agile 2 Agile 4 Support SYS
  • 9.
    Becoming Agile Strangeat first Benefits for test quickly evident Transparency took getting used to Not instantly popular Reorganise teams to become Agile Much easier to allocate resources Responsibilities clear Formalise Agile team QA remain objective, whilst contributing to Agile teams
  • 10.
    Working Guidelines forAgile Stories Estimating Walkthroughs Quality checkpoints Iteration kickoff Daily stand-ups Showcases Retrospectives
  • 11.
    Story Guidelines Shouldbe clear and specific Enough basis to develop without questions Enough basis to write detailed tests Point of reference when there are disagreements If there is ambiguity, ask the BA to update the story! Everything that happens within a project should be in a story Example summaries: As a public user I’d like to have my saved search results emailed to me As an admin user I’d like to manage customer contact details. Migrate data to new USER table on QA
  • 12.
    Example Story ID:RM_209 Summary: As a user I want to register a new login Requirements: Registration screen must comply with page template (refer story RM_205) Username must be no less than seven characters Username should be unique so that no two registered users have the same username Username must have alpha numeric characters Password must be more than six characters When user enters password text it should appear marked as ‘*’ s Meaningful confirmation message must return once registration is successful Passwords must be hashed in the database User info must be written to new table RMUSER with columns USERID, UNAME, PWORD (refer DB update RM_227)
  • 13.
    Story Lifecycle (1)Analysis Write story Story review Dev and Test estimates Compile iteration based on estimates Iteration kickoff meeting Iteration Dev phase (Dev) Develop story /unit tests/code review (QA) Write test cases
  • 14.
    Story Lifecycle (2)Dev walkthrough story with QA and BA (BA) Review work for story relevance (Dev) Review test cases (QA) Ensure unit tests/code reviews are done Iteration QA phase (test -> fix -> test ….) Showcase / signoff Release story (iteration) Retrospective Stand ups monitor progress throughout!
  • 15.
    Story Lifecycle AnalysisWrite Review Estimate Allocate Iteration KO Develop Prep Tests Walkthrough QA Showcase Release Retrospective
  • 16.
    Estimating Estimates occurbefore a story is allocated to an iteration Estimates should be measured in story points Most effort = 5 points Least effort = 0.5 points Iterations have max capacity (e.g. 30 points) Present range of stories to align effort and capacity Make estimates fun 5 pts = 0.5 points = Dev and test estimate separately A dev (small library change) may be a tester’s (requiring lots of regression)
  • 17.
    Estimate board –example Iteration 8 estimates Story Dev Test RM_234 Update all pages with new logo and disclaimer .05 RM_246 As a user I want to be able to register a new login 2 RM_289 Re-factor batch libraries 4 4 RM_267 Compose and distribute useability test invitations 0 RM_765 Add opt in / opt out config in Site Admin 2 RM_256 Migrate logins to new RMUSER table on QA Total points 27.5 13.5 14
  • 18.
    Walkthroughs Dev, QAand BA present Dev walks through changes in dev environment Handover point from dev to test Gives BA chance to review work for adherence to story Dev and BA review test cases QA reviews entry criteria Code reviews (did it pass, any issues?) Unit tests
  • 19.
    Entry Criteria Helpskeep the train moving Developer cannot pass the story to QA until: Unit tests have been written/updated to accommodate change and passed (Cruise) Peer code review has passed CVS / Subversion logs for changed added to story Ensures quality of work before test/release effort Encourages developer collaboration Traceability Reduce risk Unit tests always up to date
  • 20.
    Exit Criteria Assessedby the entire team before release BA, QA/Release, Dev, SYS Story should not be released with iteration until: Walkthrough done and stakeholders signed off Testing completed (verification, regression, performance) Outstanding defects reviewed Outstanding entry criteria satisfied Risk analysis OK
  • 21.
    Iteration Kickoff BriefOccurs after previous retrospective Overhang from last iteration? Run through all stories allocated to the iteration Review estimates Make sure everyone is ready Risks “ Go team!”
  • 22.
    Daily Stand UpsEvery morning (without fail!) Run by project team leader No longer than 15 minutes All project team must discuss: What I did yesterday What I’m doing today Any issues: Test environment System issues Release to environments Dependencies Entry criteria not satisfied
  • 23.
    Stand-ups - considerationsNo excuses for suffering in silence  Everyone should leave clear on what they should be doing and what everyone else is doing! Learned to: Manage dominators Encourage quieter team mates
  • 24.
    Showcase Occurs priorto release and after QA complete Showcase new and changed stories for iteration All dev/test/business stakeholders present Platform for official release signoff Transparency of work Stakeholders feel involved What it took to get to this point Marks team achievement!
  • 25.
    Retrospectives Lead byteam leader or BA Occur after iteration is released / completed! All stakeholders should be present Minutes recorded on whiteboard Discuss: What was released? What didn’t make it? Outstanding defects What went well? (around the room) What didn’t go well? (around the room) How will we prevent these points in the future? Agree a way forward – everyone!! Distribute minutes, and refer back to them!
  • 26.
    Agile Iteration andRelease Management Release cycles Communication! Release meetings Release tags Cruise Control + Selenium Roles and responsibilities
  • 27.
    Release Management -Agile Release to Production every two weeks Four week iterations (2 dev + 2 QA) QA team responsible for testing and managing releases Release management roster Testers are gatekeepers throughout process Work with tags, so dev can commit during regression Integrated release system – Cruise Control + Selenium Environments Dev (to create) Test (to verify) QA/Pre-Prod (for regression, performance, session replication, fail over, etc) Prod (the real world)
  • 28.
    Iteration Release Cycle(1) Story allocated to iteration and release number in JIRA. Build manager exports list of changes for build and books build meeting All developers, QA and system architects present Build manager runs through list to ensure change passed in Cruise and QA have started testing Discuss high risk changes in detail Rollback procedure BM pushes Cruise build to test environment at request Sending notification each time
  • 29.
    Iteration Release Cycle(2) At end of verification phase, BM creates tags and deploys to QA env... Regression test phase Pre-release meeting Review exit criteria for release Compose release notes to distribute company wide All repositories to be released CVS tags Database changes
  • 30.
    Iteration / ReleaseCycle Phase 1 Phase 2 Weds Thurs - Tues Weds Thurs - Fri Mon Tues Prod Release Stand-up Stand-up Stand-up Stand-up Stand-up Stand-up Send release notes Deploy to test Performance tests Merge tags Verification tests Verification Tests Regression tests Regression Tests QA signoff Deploy trunk to test env. Fix defects Dev next iteration Dev next iteration Dev next iteration Dev next iteration Retrospectives Dev next iteration Build meeting Showcase Prepare release notes Iteration kickoff Create tags Story reviews Pre release meeting Walkthroughs Deploy to QA env. Agile Team Developers Testers Build Manager Systems
  • 31.
    Communicating Builds ReleaseManager responsible for communicating build information Vital for project team to know when changes have been deployed (to any environment) Any delays could delay project! Any build related questions, ask the build manager Email group, Release setup for build communication. (incl. QA, Dev, BA’s, SYS, anyone else who wants to know) Notification sent to Release for every deployment to test and QA environments
  • 32.
    Build Emails -Sample To: Release Group Subject: New build released to QA Good morning, A new build has successfully been deployed to <environment> The updated modules are: publicsite/trunk batch/trunk admin/trunk They include all changes passed through Cruise as at <time> Regards, Your friendly build manager
  • 33.
    Build Email -Sample To: Release Group Subject: Deployment issues Good morning, We are currently experiencing issues with our deployment system and cannot release any builds until they have been resolved. Resolving this is our highest priority. I apologise for the delay, and will keep you informed. Regards, Your friendly build manager.
  • 34.
    Build Preview MeetingOccurs mid iteration Just before release Tag is created Ensure everything is ready for the build Helps gather information about release items Identify risks introduced by new code (what does this impact?) Provides visibility to the build Platform to discuss/enforce entry criteria Opportunity for Systems team to assess risk – especially DB changes
  • 35.
    Pre-Release Meeting Officialhandover from QA to SYS Platform to discuss release with all stakeholders. QA / Release Manager runs through Release Notes Highlight issues found during testing Identify risks associated with release Agree a release plan DB changes first Order of modules Updates to cron jobs
  • 36.
    Config Management BeforeAgile – we had commit freeze during testing Needed to keep the train moving Lots of CVS project modules Lots of testing and developing at the same time Developers should be able to commit at any time regardless of cycle No measure of quality before QA starts Solution Create live tags during regression phase for release Implement and maintain Selenium tests for all modules to run at build time
  • 37.
    Releasing Tags TrunkTag/snapshot Iteration 1 Dev/Verify Iteration 1 Fix/Regression Iteration 2 Dev Release Merge Tag
  • 38.
    Builds – CruiseControl CVS Module1 Module2 Module3 Module4 Cruise Control Selenium Tests Selenium Tests Selenium Tests Selenium Tests Alert! Build Failed! M1Build_145 Commit new change M2Build_097 M3Build_225 M4Build_002
  • 39.
    Deploying Cruise buildsM1Build_145 Deploy Test Pre-Production Production
  • 40.
    Integrating Changes Changeis not ready to test until it is part of Cruise build Compiled, integrated and Selenium passed Developer’s responsibility to monitor their change building after committing in CVS Immediate action required if build fails Module is locked out until last commit passes Cruise
  • 41.
    When it cango wrong…. Cut corners Ineffective communication Sticking to process when it doesn’t work Not learning from retros Preparing for meetings Do things without telling anyone Dev changes System changes Data refresh…
  • 42.
    Observations - AgileResponding to change doesn’t create hassle Little to no rework after release Much better relationship between IT and business More team bonding within IT Cross functional teams promoted multi-discipline learning – and appreciation! Better working relationships Faster throughput of work Issues resolved promptly! Consider dependencies!!
  • 43.
    Review Agile overviewAgile team structures Story guidelines Story lifecycle Dev and Test estimates Walkthroughs / quality checks Stand-ups Showcases Retrospectives Managing iterations Config management and build systems Release management and communication
  • 44.

Editor's Notes