Agile Methods and Software Testing

693 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
693
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Agile Methods and Software Testing

  1. 1. Agile Methods and Software Testing Paul Gerrard Technical Director, Systeme Evolutif Systeme Evolutif Limited 9 Cavendish Place London W1G 0QD, UK Tel: +44 (0)20 7636 6060 email: paulg@evolutif.co.uk http://www.evolutif.co.uk/ Version 1.0 ©2002 Systeme Evolutif Ltd Slide 1 Paul Gerrard Paul is the Technical Director and principal consultant for Systeme Evolutif. He has conducted consultancy and training assignments in all aspects of Software Testing and Quality Assurance. Previously, he has worked as a developer, designer, project manager and consultant for small and large developments using 3 and 4GLs. Paul has engineering degrees from the Universities of Oxford and London, is Co-Programme Chair for the BCS SIG in Software Testing, was a member of the BCS Software Component Test Standard Committee and Former Chair of the IS Examination Board (ISEB) Certification Board for a Tester Qualification whose aim is to establish a certification scheme for testing professionals and training organisations. He is a regular speaker at seminars and conferences in Europe and the US, and won the ‘Best Presentation’ award at EuroSTAR ’95. His first book, “Risk-Based E-Business Testing” will be published in July 2002. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 2 © 2002 Systeme Evolutif Limited Page 1 Version 1.0
  2. 2. Agenda l Collaborative game of invention and communication l Agile Manifesto: www.agilealliance.org/ l Some of the increasing number of agile test methods (for more info: www.testing.com/agile/) – eXtreme Programming (and testing) – Exploratory Testing – “Good-Enough” Testing – Bug Advocacy – Risk Based Testing (I’m working on this one) l Watch this space… Version 1.0 ©2002 Systeme Evolutif Ltd Slide 3 The struggle for productivity l We despair that software development productivity only advances a few percent each year l We don’t worry about – Time it takes to make new laws – Time it takes to write a book – They take as long as they take l Just like software development? l Software development is a “resource-limited collaborative game of invention and communication” l Maybe that’s why we struggle to improve. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 4 © 2002 Systeme Evolutif Limited Page 2 Version 1.0
  3. 3. “A Collaborative Game of Invention and Communication” Version 1.0 ©2002 Systeme Evolutif Ltd Slide 5 Types of game l Zero-sum – I win, you lose e.g. tennis, draughts, darts, football l Non-zero-sum – multiple winners and losers e.g. poker, marathon, hide and seek l Positional games, where you plan and can trace every move e.g. chess, go, othello l Non-positional games e.g. snooker, bridge, rock- climbing, football l These games are all competitive. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 6 © 2002 Systeme Evolutif Limited Page 3 Version 1.0
  4. 4. Which game is software development? l Positional? – Every move planned, executed, documented in triplicate – Documentation reflects both current state and history l Zero-sum? – Developers deliver late, testers are squeezed? l Competitive? – We deliver before the customer changes their mind (regardless of whether it ‘works’) l Software development is a collaborative, non-zero- sum, non-positional game. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 7 Software development as rock- climbing l Collaborative l Tools l Goal seeking l Resource-limited l No single route to l Planned success l Stressful l Team based l Improvised l Individuals with talent l Challenging l Skill-sensitive l Dangerous l Training required l Fun (?) Version 1.0 ©2002 Systeme Evolutif Ltd Slide 8 © 2002 Systeme Evolutif Limited Page 4 Version 1.0
  5. 5. Software development is invention and communication l Requirements are a fuzzy, incomplete, ambiguous set of intents, ambitions and needs l Deliverable is an executable language, difficult to express and unforgiving of error l Development is a complex translation process l Invent props and devices to understand and express the problem to be solved (at all levels) l Need trails of markers for ourselves and others who come later. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 9 Communication l Poor communication is the main reason big projects need many more people and fail – It’ll take 10 good developers six months – It’ll take 200 people two years l Excessive documentation in abstract models that are over complex, never finished and out of date anyway are barriers to progress l Surely, it’s better just to talk face to face? Version 1.0 ©2002 Systeme Evolutif Ltd Slide 10 © 2002 Systeme Evolutif Limited Page 5 Version 1.0
  6. 6. Cost of poor communication l Pair programming (pair testing) demonstrably more effective l Proximity has a huge influence on cost – Next desk – I see/hear everything – Next room – I see/hear nothing, it costs me five minutes to ask/answer a question – Next building – I see/hear nothing, I don’t bother to talk anymore, I write documents. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 11 Diminishing returns l Time, resource and money are sparse – don’t waste it perfecting products l Products are ready when they allow you to make the next move l When documentation becomes drudgery, rather than useful l When test planning doesn’t generate useful tests anymore l The “Good Enough” approach. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 12 © 2002 Systeme Evolutif Limited Page 6 Version 1.0
  7. 7. Levels of maturity l Level 1 - following – people need written instructions to do everything – it’s novel, they are inexperienced l Level 2 – detaching – know the process, knows where it breaks down, knows of alternatives, can adapt them l Level 3 - fluent – process is irrelevant, knowledge is integrated with the task at hand and action required to perform it. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 13 A touch of Zen? l Kent Beck, when asked about XP and the Capability Maturity Model (CMM): 1. Do everything as written 2. Then, experiment with variations in the rules 3. Eventually, you don’t care if you are doing XP or not l A higher level of consciousness? l Pretentious? Dangerous? Or what? Version 1.0 ©2002 Systeme Evolutif Ltd Slide 14 © 2002 Systeme Evolutif Limited Page 7 Version 1.0
  8. 8. Agile Manifesto www.agilealliance.com www.agilealliance.org Version 1.0 ©2002 Systeme Evolutif Ltd Slide 15 Agile Alliance Manifesto l Individuals and l Processes and tools interactions l Working software l Comprehensive documentation l Customer l Contract negotiation collaboration l Responding to change l Following a plan XP, Adaptive Software Development, Crystal, DSDM, Feature Driven Development, Scrum, XBreed See www.testing.com, www.agilealliance.orgvalue We value We these MORE these Version 1.0 ©2002 Systeme Evolutif Ltd Slide 16 © 2002 Systeme Evolutif Limited Page 8 Version 1.0
  9. 9. Agile Alliance Principles l Highest priority is to satisfy the customer through early and continuous delivery of valuable software l Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage l Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale l Business people and developers must work together daily throughout the project. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 17 Agile Alliance Principles 2 l Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done l The most efficient and effective method of conveying information to and within a development team is face-to-face conversation l Working software is the primary measure of progress l Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 18 © 2002 Systeme Evolutif Limited Page 9 Version 1.0
  10. 10. Agile Alliance Principles 3 l Continuous attention to technical excellence and good design enhances agility l Simplicity - the art of maximizing the amount of work not done - is essential l The best architectures, requirements, and designs emerge from self-organizing teams l At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 19 Extreme Programming and Testing Version 1.0 ©2002 Systeme Evolutif Ltd Slide 20 © 2002 Systeme Evolutif Limited Page 10 Version 1.0
  11. 11. Key attributes of XP l Code reviews are good, so we’ll review the code all the time (pair programming) l Testing is good, so we all test all the time (developers unit test, customers system test) l Integration testing is good, so we’ll integrate and test several times a day (continuous integration) l Design is good, so it’s part of everyone's business l Simplicity is good, so we’ll always strive to use the simplest design possible l Etc. etc. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 21 Developer testing l Define and build tests from stated requirements - before coding – if the requirement (or the solution) is unclear write some tests - this will flush issues out – if there is a situation which the code must deal with, write a test to cover it – if you find a problem later, write another test to cover it – if you do any redesign (refactoring), write tests to cover the changed functionality. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 22 © 2002 Systeme Evolutif Limited Page 11 Version 1.0
  12. 12. Developer testing 2 l All code must have tests l All tests are automated and maintained forever l After integration, all tests must run 100% successfully before release l If a bug is found, more tests are written l If test stops ‘working’, getting the test to work again is the top priority. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 23 Exploratory Testing Version 1.0 ©2002 Systeme Evolutif Ltd Slide 24 © 2002 Systeme Evolutif Limited Page 12 Version 1.0
  13. 13. What is Exploratory Testing (ET)? l Concurrent – Product exploration – Test design and – Test execution l But that’s just error-guessing isn’t it? Version 1.0 ©2002 Systeme Evolutif Ltd Slide 25 ET process l Briefing - one page manifesto to identify hotspots l Get familiar with product & target hotspots l Try extremes, anything you like to break product l When you find a bug, explore around it – Look for patterns, clues for other bugs – Log an incident – Move onto next interesting area l When to use? – Always? - probably not – When normal testing is suspended or you believe more focused tests in an area will be productive. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 26 © 2002 Systeme Evolutif Limited Page 13 Version 1.0
  14. 14. Good Enough Testing Version 1.0 ©2002 Systeme Evolutif Ltd Slide 27 “Good Enough” l James Bach* is main advocate of the ‘Good Enough’ view (also Yourdon and others) l A reaction to compulsive formalism – if you aim at perfection, you’ll never succeed – your users/customers/businesses live in the real world, why don’t you? – compromise is inevitable, don’t kid yourself – guilt and fear should not be part of the process. * James Bach, “Good Enough Quality: Beyond the Buzzword”, Computer, August 1997 also STAR ’97 presentation, “Good Enough Testing”. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 28 © 2002 Systeme Evolutif Limited Page 14 Version 1.0
  15. 15. “Good Enough” means: 1. X has sufficient benefits 2. X has no critical problems 3. Benefits of X sufficiently outweigh problems 4. In the present situation, and all things considered, improving X would cause more harm than good. All the above must apply Version 1.0 ©2002 Systeme Evolutif Ltd Slide 29 Contribution of testing to the release decision l Have sufficient benefits been delivered? – tests must at least demonstrate that features providing benefits are delivered completely l Are there any critical problems? – test records must show that any critical problems have been corrected, re-tested, regression tested l Is our testing Good Enough? – have we provided sufficient evidence to be confident in our assessment? Version 1.0 ©2002 Systeme Evolutif Ltd Slide 30 © 2002 Systeme Evolutif Limited Page 15 Version 1.0
  16. 16. Bug Advocacy Version 1.0 ©2002 Systeme Evolutif Ltd Slide 31 Bug Advocacy l Bug (incident) reports are your main work product l The best tester gets most bugs fixed (not just found) l A bug report is a tool for getting bugs fixed l Sales techniques – Motivate the buyer (get the developer to fix) – Overcome objections (argue against objections). Version 1.0 ©2002 Systeme Evolutif Ltd Slide 32 © 2002 Systeme Evolutif Limited Page 16 Version 1.0
  17. 17. Bug advocacy 2 l Motivate the developer – It’s a dangerous, expensive, embarrassing, expensive bug that is easy to fix! l Overcome objections – Make it easy to reproduce - research it further – It’s a bug, not a feature – Make the bug report perfectly clear – And so on. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 33 Risk-Based Testing (I’m working on this one…) Version 1.0 ©2002 Systeme Evolutif Ltd Slide 34 © 2002 Systeme Evolutif Limited Page 17 Version 1.0
  18. 18. Proposed test objectives l To provide evidence (and confidence) that – The benefits delivered are acceptably high – The risk of failure is acceptably low l To report progress by identifying: – Risks addressed – Benefits delivered l Stakeholders want to know about benefits delivered (and the benefits under threat) l Project management want to know about the residual risks that block the benefits. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 35 Key steps to RBT Test Risk objectives Assessment Techniques/ Master Test coverage Plan Test Test Specification Planning Test Test Scripts Preparation Test Incidents, Test results, logs. Execution Release/ Acceptance Technical involvement (developers, designers, business analysts) Stakeholder involvement (management, customers etc.) Version 1.0 ©2002 Systeme Evolutif Ltd Slide 36 © 2002 Systeme Evolutif Limited Page 18 Version 1.0
  19. 19. Risk-based test reporting Planned start today end Residual Risks all risks ‘open’ at the start residual risks of releasing TODAY Progress through the test plan Version 1.0 ©2002 Systeme Evolutif Ltd Slide 37 Benefit & objectives based test reporting Objective Objective Objective Objective Objective Benefit Benefit Benefit Benefit Benefit Benefit Open Risks (or tests) Closed Open Closed Closed Open Closed Open Version 1.0 ©2002 Systeme Evolutif Ltd Benefits available for release Slide 38 © 2002 Systeme Evolutif Limited Page 19 Version 1.0
  20. 20. Watch This Space Version 1.0 ©2002 Systeme Evolutif Ltd Slide 39 Status of agile (testing) methods l Multiple lightweight development methods – Trend from predictive to adaptive – No overarching theory just a set of principles – All promoted by influential individuals not corporations l Same goes for agile test methods – Collection of disintegrated techniques – Few (if any) links to the development methods – Little guidance on when to use, how to combine/choose between them, how to fit into real projects l Increasing attention being paid to them… Watch this space. Version 1.0 ©2002 Systeme Evolutif Ltd Slide 40 © 2002 Systeme Evolutif Limited Page 20 Version 1.0
  21. 21. Agile Methods and Software Testing Paul Gerrard paulg@evolutif.co.uk www.evolutif.co.uk www.riskbasedtesting.com Version 1.0 ©2002 Systeme Evolutif Ltd Slide 41 © 2002 Systeme Evolutif Limited Page 21 Version 1.0

×