• Save
Testing: Chances and Challenges in an agile World
Upcoming SlideShare
Loading in...5
×
 

Testing: Chances and Challenges in an agile World

on

  • 2,308 views

New Chances and challenges for testing in an agile world.

New Chances and challenges for testing in an agile world.
Introduction to Behavior Driven Development and Acceptance Test Driven Development.

Statistics

Views

Total Views
2,308
Views on SlideShare
1,884
Embed Views
424

Actions

Likes
2
Downloads
0
Comments
0

8 Embeds 424

http://blog.jonasbandi.net 299
http://nextloop.blogspot.com 95
http://nextloop.blogspot.ch 20
http://mrtn.ws 4
http://www.slideshare.net 2
http://www.docshut.com 2
http://closed-loop.blogspot.com 1
http://posterous.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Testing: Chances and Challenges in an agile World Presentation Transcript

  • 1. Testing im Wandel Neue Herausforderungen und Chancen in einer agilen Welt Jonas Bandi Software Architect TechTalk jonas.bandi@techtalk.ch blog.jonasbandi.net www.techtalk.ch twitter.com/jbandi Thursday, November 5, 2009
  • 2. "We're still figuring this stuff out. All of us." - Jay Fields Thoughts on developer Testing: http://blog.jayfields.com/2009/02/thoughts-on- developer-testing.html Thursday, November 5, 2009
  • 3. My point of view • Software Architect at TechTalk • TechTalk is a software development and consulting company with ~60 people located in Vienna, Budapest and Zürich. • The main business is building custom IT solutions for clients in Austria and Switzerland. • Projects of TechTalk involve development efforts of up to several thousand person days and team sizes of up to ten persons. Thursday, November 5, 2009
  • 4. Once upon a time... ... in the golden Age of the Waterfall. Thursday, November 5, 2009
  • 5. ... and they lived happily ever after! Thursday, November 5, 2009
  • 6. Welcome to the real world! Thursday, November 5, 2009
  • 7. ... the golden age is over! Thursday, November 5, 2009
  • 8. What did really change? 49% 24% 23% 44% 32% 28% sucessful challenged failed 2000 2008 Project success Chaos Report, Standish Group Thursday, November 5, 2009
  • 9. What is the problem? Thursday, November 5, 2009
  • 10. There are many! Thursday, November 5, 2009
  • 11. Focus: Requirements Thursday, November 5, 2009
  • 12. What defines success? Software development is about delivering business value! Thursday, November 5, 2009
  • 13. What is being used? 19% 16% Never Seldom 13% Occasionally 45% 7% Often Always CHAOS Report 2000, Standish Group Thursday, November 5, 2009
  • 14. Providing Business Value? • Only 20% of all delivered features are actually used? • Something is wrong! • We can profit tremendously when we improve in requirements gathering and in transporting them successfully to the implementation Thursday, November 5, 2009
  • 15. Where Errors are introduced Requirements 27% Design 56% Code 7% Other 10% Source: James Martin, An Information Systems Manifesto Thursday, November 5, 2009
  • 16. Fighting Errors? • Traditional Developer Testing focuses on code and maybe design • Testing requirements is hardly ever a topic • We can profit tremendously when we improve in requirements gathering. Thursday, November 5, 2009
  • 17. Detailgrad Vision Verständnisgrenze Actor/Stakeholder Lastenheft Pflichtenheft Designdokument Kein Feedback von Stakeholdern möglich! Code Manual Test Automated „Usage“ Test Zeit Waterfall Project Thursday, November 5, 2009
  • 18. Costs to fix defects rise exponentially over time The longer a defect stays in the development process, the more expensive it is to fix Thursday, November 5, 2009
  • 19. Agile to the rescue! Thursday, November 5, 2009
  • 20. Detailgrad Verständnisgrenze Actor/Stakeholder Actor/ Business Vision Stakeholder Projektstart goals goals Product Epics Stories Backlog Sprint Planning Preparation Sprint Acceptance Sprint 1-n Stories Sprint Planning Backlog Criteria Sprint Implementation Acceptance Kurze Feedbackzyklen Test Definition mit Stakeholdern Code Manual Test/ Automated Demo Test Zeit Iterative Project Thursday, November 5, 2009
  • 21. The benefits • Constant feedback and priorization by the Stakeholder 19%16% • Build those features that are 45% 7% 13% needed and useful • Features have a much shorter 27% through-put time 7% • Allows validation of requirements 56% 10% Thursday, November 5, 2009
  • 22. Wait... ... isn’t this presentation supposed to be about testing? Thursday, November 5, 2009
  • 23. "I believe that the hardest part of software projects, the most common source of project failure, is communication with the customers and users of that software." - Martin Fowler, DSL Book Thursday, November 5, 2009
  • 24. It is all about communication! Thursday, November 5, 2009
  • 25. Verständnisgrenze Actor/Stakeholder Actor/ Business Vision Stakeholder Projektstart goals goals Product Epics Stories Backlog Sprint Planning Preparation Sprint Acceptance Sprint 1-n Stories Sprint Planning Backlog Criteria Sprint Implementation Acceptance Kurze Feedbackzyklen Test „Vertrauenswürdige Definition mit Stakeholdern Code Kommunikation“ Manual Test/ Automated Demo Test Zeit Enable communication Thursday, November 5, 2009
  • 26. How? Thursday, November 5, 2009
  • 27. Catch bugs in requirements! Thursday, November 5, 2009
  • 28. Thursday, November 5, 2009 Cost RE Analysis & Design Requirements Engineering Implementation Testing Deployment Analysis & Design Implementation Time Testing RE Analysis & Design Implementation Deployment Testing Deployment
  • 29. How? Thursday, November 5, 2009
  • 30. Now... ... finally testing! Thursday, November 5, 2009
  • 31. Requirements == Tests “As formality increases, tests and requirements become indistinguishable. At the limit, tests and requirements are equivalent.” - Equivalence hypothesis, Robert C. Martin • Traditionally this is not recognized because RE and Testing are set up as different disciplines • But they are about the same: how the system is used • Wouldn’t it make sense to leverage some synergies? Thursday, November 5, 2009
  • 32. Ideally Tests ... • … describe what the system should do • … are business readable • … establish a common vocabulary • … capture a shared set of examples • … enable communication • … are bound to the implementation • … are executable ... express shared understanding Thursday, November 5, 2009
  • 33. Building a shared understanding - Scripts - Test Data Mapping - Acceptance Criteria Business Business - Specification - Scripts Tester Tester - Use Cases - Test Data - User Stories - Acceptance Criteria Shared ? - Specification Understanding - Use Cases Mapping - User Stories - Code Mapping - Code - Unit Tests - Unit Tests - Test Data - Test Data Developer Developer Thursday, November 5, 2009
  • 34. Finding a common abstraction Shared Business Developer Tester Understanding Business Tester Developer Thursday, November 5, 2009
  • 35. Shared Understanding • Terminology and abstractions of business • Concrete enough for developers • Business readable • Bound to implementation • Consistent Thursday, November 5, 2009
  • 36. Specification by example • Abstract requirements and specifications are not a good tool for communication. Real-life examples are much better. • We mostly communicate by examples • Usually examples are not formalized and not shared Thursday, November 5, 2009
  • 37. Examples • Enable developers and testers to communicate efficiently with business people • Provide enough information for developers to implement and for testers to verify • Realistic examples contain precise information and ask for precise answers • Are a way to test specifications! Thursday, November 5, 2009
  • 38. A new tool! Thursday, November 5, 2009
  • 39. Agile Acceptance Testing Acceptance Test Driven Development Behavior Driven Development Thursday, November 5, 2009
  • 40. Demo Thursday, November 5, 2009
  • 41. Agile: new challenges! Thursday, November 5, 2009
  • 42. Challenges • Effort • Regression • Definition of Done • Streaming Requirements • Agile Planing • Authoritative Source of Information Thursday, November 5, 2009
  • 43. Effort Thursday, November 5, 2009
  • 44. Effort • Potentially shippable software each iteration • Testing everything each iteration • A lot of iterations -> a lot to test! • Consequence: Automation is a must! Thursday, November 5, 2009
  • 45. Regression Thursday, November 5, 2009
  • 46. Regression • Potentially shippable software each iteration • High velocity, changes are embraced • How to guarantee quality? • Consequence: Automation is a must! Thursday, November 5, 2009
  • 47. Definition of Done Thursday, November 5, 2009
  • 48. Definition of Done • When are we finished? • Testing is an integral part of the development process and not a separate discipline • Implementation and testing have to work together on a fine grained level • There has to be an efficient communication between implementation and testing • Consequence: Relation between implementation and testing has to be redefined! Thursday, November 5, 2009
  • 49. Streaming Requirements Thursday, November 5, 2009
  • 50. Streaming Requirements • Not all of the requirements are delivered upfront • Requirements are delivered just in time • Incrementally reveal details just-in-time • Pull instead of push • Embrace Change • Consequence: Requirements providers (stakeholders) must find a new way to deliver requirements Thursday, November 5, 2009
  • 51. Agile Planning Thursday, November 5, 2009
  • 52. Agile Planning • Goal-driven instead of scope-driven • How to plan something when the scope is uncertain? • Managing changes and uncertainty • You don’t know what can be tested when... • Consequence: Testers and developers have to interact on a fine grained and self organized level Thursday, November 5, 2009
  • 53. Authoritative Source of Information Thursday, November 5, 2009
  • 54. Authoritative Source of Information • Avoid excessive, disconnected artifacts. The source code is the artifact that ultimately matters. • No specification, no documentation doesn’t work either! • Communication is more important than documentation • But people as the only source of information is not an option • Consequence: Find a way for efficient specifications and documentation that is connected to the source code. Thursday, November 5, 2009
  • 55. "We're still figuring this stuff out. All of us." - Jay Fields Thoughts on developer Testing: http://blog.jayfields.com/2009/02/thoughts-on- developer-testing.html Thursday, November 5, 2009
  • 56. Müssen Entwickler Braucht es noch Tester? jetzt Testen? JA! Thursday, November 5, 2009
  • 57. Entwickler... • Bindet Beispiele and die Implementierung • Worauf gebunden wird ist flexibel! Thursday, November 5, 2009
  • 58. Tester ... • Bringen Beispiele in eine testbare Form • Kollaboration mit Stakeholders • Mindestens ein Beispiel pro Feature • Economy of Scale • Entwickeln die Tessprache • Nutzung auch für manuelle Tests Thursday, November 5, 2009
  • 59. Manuelle Tests ... • Zur Validierung von Main-Flows und Integration • Demo an Stakeholder • Geringere Anzahl • Unterstützt durch „Testsprache“ • Automatisierung aufwändiger! Thursday, November 5, 2009
  • 60. Aufwand? • Ca. 15% des Gesamtaufwands für Testspezifikation -> 1 Tester versorgt 6-7 Entwickler • Entwickler: 10-25% Aufwand für Binding -> Synergien Thursday, November 5, 2009
  • 61. Wartbarkeit? • Richtiger Abstraktionsgrad • Wiederverwendbarkeit von Bindings • Ebene der Bindings Thursday, November 5, 2009
  • 62. Why are tests important? "Nothing makes a system more flexible than a comprehensive suite of tests! Far above good architecture and good design!" - Robert C. Martin, Oredev, 2008 http://archive.oredev.org/topmenu/video/keynotebobmartin.4.5a2d30d411ee6ffd28880002007.html Thursday, November 5, 2009
  • 63. Testing is more than QA! • Regression Tests enable change • TDD: Drive the design through tests • Tests are examples that enable customer interaction • Tests are examples that can be the specification • Tests are examples that document the system • ATDD: Drive the specification through acceptance tests • Tests can be the definition of ‘done’ Thursday, November 5, 2009
  • 64. The Topic is Hot! www.cukes.info www.concordion.net www.fitnesse.org www.specflow.org JBehave / NBehave http://studios.thoughtworks.com/agile-test-automation Thursday, November 5, 2009
  • 65. Resources The RSpec Agile Testing Book Bridging the communication gap Thursday, November 5, 2009
  • 66. BDD is a Mindset not a Tool! Thursday, November 5, 2009
  • 67. Viel Erfolg beim Einsatz von Behavior Driven Development und Agilen Akzeptanztests! Jonas Bandi Software Architect TechTalk jonas.bandi@techtalk.ch blog.jonasbandi.net www.techtalk.ch twitter.com/jbandi Thursday, November 5, 2009