Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Agile Testing and/or Agility in Testing? Juha Itkonen SoberIT Teknillinen Korkeakoulu Test Summit, 29.7.2003, Savonlinna
  2. 2. Contents <ul><li>Overview of agility </li></ul><ul><li>Agile principles and testing </li></ul><ul><li>Two views of agile testing: eXtreme and Exploratory </li></ul><ul><li>Third view and challenges of agile testing </li></ul><ul><li>Agile development and testing services </li></ul>
  3. 3. Agile Manifesto <ul><li>Individuals and interactions over processes and tools </li></ul><ul><li>Working software over comprehensive documentation </li></ul><ul><li>Customer collaboration over contract negotiation </li></ul><ul><li>Responding to change over following a plan </li></ul>How do these values affect testing?
  4. 4. Some agile principles <ul><li>Satisfy the customer through early and continuous delivery of valuable software. </li></ul><ul><li>Working software is the primary measure of progress. </li></ul><ul><li>Deliver working software frequently, from a couple of weeks to a couple of months . </li></ul><ul><li>Welcome changing requirements , even late in development. </li></ul><ul><li>The most efficient and effective method of conveying information to and within a development team is face-to-face conversation . </li></ul><ul><li>Business people and developers must work together daily throughout the project. </li></ul><ul><li>Simplicity --the art of maximizing the amount of work not done--is essential. </li></ul>
  5. 5. Agile methodologies and testing <ul><li>eXtreme programming </li></ul><ul><li>Scrum </li></ul><ul><li>DSDM </li></ul><ul><li>FDD </li></ul><ul><li>LD </li></ul><ul><li>ASD </li></ul><ul><li>MS Synch-and-Stabilize </li></ul><ul><li>Crystal family </li></ul><ul><li>Some define strict disciplined testing practices </li></ul><ul><li>Some do not say much about testing approach </li></ul><ul><ul><li>E.g. FDD: “… processes used for testing are not the main process issues with which the organisations are struggling … and most organizations already have reasonable testing processes in place” </li></ul></ul>“ Agile Testing” is a buzzword that is not yet defined
  6. 6. Two views of agile testing <ul><li>eXtreme Testing </li></ul><ul><li>Automated unit testing </li></ul><ul><ul><li>Developers write tests </li></ul></ul><ul><ul><li>Test first development </li></ul></ul><ul><ul><li>Daily builds with unit tests always 100% pass </li></ul></ul><ul><li>Functional testing </li></ul><ul><ul><li>Customer-owned </li></ul></ul><ul><ul><li>Comprehensive </li></ul></ul><ul><ul><li>Repeatable </li></ul></ul><ul><ul><li>Automatic </li></ul></ul><ul><ul><li>Timely </li></ul></ul><ul><ul><li>Public </li></ul></ul><ul><li>Exploratory Testing </li></ul><ul><li>Manual testing by professional skilled testers </li></ul><ul><li>Freedom, flexibility and fun for testers </li></ul><ul><li>Controllability, reliability and high quality for managers </li></ul><ul><li>Optimized to find bugs </li></ul><ul><li>Continually adjusting plans, re-focusing on the most promising risk areas </li></ul><ul><li>Following hunches </li></ul><ul><li>Minimizing time spent on documentation </li></ul>Focus on automated verification – enabling agile software development Focus on manual validation – making testing activities agile
  7. 7. Definitions of Exploratory Testing <ul><li>In scripted testing, tests are first designed and recorded. Then they may be executed at some later time or by a different tester. </li></ul><ul><li>In exploratory testing, tests are designed and executed at the same time, and they often are not recorded. </li></ul><ul><li>You build a mental model of the product while you test it. This model includes what the product is and how it behaves, and how it’s supposed to behave. </li></ul>James Bach, Rapid Software Testing, 2002 <ul><li>To the extent that the next test we do is influenced by the result of the last test we did, we are doing exploratory testing. James Bach, 2001 </li></ul><ul><li>Exploratory testing involves simultaneously learning, planning, running tests, troubleshooting and reporting results. Cem Kaner, 2001 </li></ul>
  8. 8. Scripted vs. Exploratory Tests mines fixes
  9. 9. Value of testing in agile development <ul><li>Testing in agile software development should provide the information that stakeholders need to make decisions and steer the development into the right direction </li></ul><ul><li>This information must be provided promptly </li></ul><ul><li>Testing provides data on the status of the deliverables and generates project intelligence </li></ul><ul><li>Project intelligence is knowledge of risks and benefits </li></ul><ul><li>Knowledge of risks, benefits, test records and results are more valuable than test documentation and infrastructure </li></ul><ul><li>We can increase the value of testing most by </li></ul><ul><ul><li>Improved intelligence </li></ul></ul><ul><ul><li>Providing intelligence earlier </li></ul></ul>
  10. 10. Third view of agile testing <ul><li>Embracing the change </li></ul><ul><li>Testing practices must not be fixed </li></ul><ul><ul><li>Test procedures must be clearly defined and described as atomic modules </li></ul></ul><ul><ul><li>Test procedures must be easily selected to suit the changing situations and requirements </li></ul></ul><ul><li>Development and testing must collaborate and agree on the test procedures and quality goals for each increment </li></ul><ul><ul><li>Not only functionality but also non-functional features </li></ul></ul><ul><ul><li>Test requirements </li></ul></ul><ul><ul><li>Expected results </li></ul></ul><ul><li>Business must set the upper-level quality requirements </li></ul>
  11. 11. Challenges for testing in agile software development <ul><li>What information is the testing based on? </li></ul><ul><ul><li>What to test and what are the expected results? </li></ul></ul><ul><ul><li>How to make testing, development and business collaborate? </li></ul></ul><ul><ul><li>How to involve customer and business people in testing? </li></ul></ul><ul><li>How do we know where we are? </li></ul><ul><ul><li>Working software is the primary measure of progress </li></ul></ul><ul><ul><li>Testing should provide the data </li></ul></ul><ul><ul><li>If it’s not tested it doesn’t exist </li></ul></ul><ul><li>How to keep up with the pace of the development? </li></ul><ul><ul><li>How to produce and communicate relevant information promptly? </li></ul></ul><ul><ul><li>How to test early but not do anticipatory test design? </li></ul></ul>
  12. 12. Agile development and testing services <ul><li>It is important to understand the challenges of agile practices </li></ul><ul><ul><li>Hard to solve even within one company </li></ul></ul><ul><li>If an agile company wishes to use testing services (e.g. for reliability, load, stress, compatibility or functional testing) it might be confronted with big problems </li></ul><ul><ul><li>Arranging collaboration and communication </li></ul></ul><ul><ul><li>Service provider’s ability to respond to change </li></ul></ul><ul><ul><li>Service provider’s ability to adopt the pacing </li></ul></ul><ul><li> The challenges of testing in agile development have to be solved! </li></ul>
  13. 13. Further reading <ul><li>Beck, K., Test-Driven Development by Example , Addison-Wesley, 2003. </li></ul><ul><li>Crispin, L., T. House, Testing Extreme Programming , Addison-Wesley, 2003. </li></ul><ul><li>Kaner, C., J. Bach, B. Pettichord, Lessons Learned in Software Testing , John Wiley & Sons, 2002. </li></ul><ul><li>www. agilemanifesto . org / </li></ul><ul><li>www. xprogramming . com / </li></ul><ul><li>www. satisfice . com / articles /et- article . pdf </li></ul><ul><li>www. testingeducation . org / coursenotes / </li></ul><ul><li>www. testinglessons . com / </li></ul><ul><li>www. agilealliance . com / articles / </li></ul><ul><li>www. context - driven - testing . com / </li></ul>