Testing: Chances and Challenges in an agile World

  • 1,523 views
Uploaded on

New Chances and challenges for testing in an agile world. …

New Chances and challenges for testing in an agile world.
Introduction to Behavior Driven Development and Acceptance Test Driven Development.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,523
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Testing im Wandel Neue Herausforderungen und Chancen in einer agilen Welt Jonas Bandi Software Architect TechTalk jonas.bandi@techtalk.ch blog.jonasbandi.net www.techtalk.ch twitter.com/jbandi Thursday, November 5, 2009
  • 2. "We're still figuring this stuff out. All of us." - Jay Fields Thoughts on developer Testing: http://blog.jayfields.com/2009/02/thoughts-on- developer-testing.html Thursday, November 5, 2009
  • 3. My point of view • Software Architect at TechTalk • TechTalk is a software development and consulting company with ~60 people located in Vienna, Budapest and Zürich. • The main business is building custom IT solutions for clients in Austria and Switzerland. • Projects of TechTalk involve development efforts of up to several thousand person days and team sizes of up to ten persons. Thursday, November 5, 2009
  • 4. Once upon a time... ... in the golden Age of the Waterfall. Thursday, November 5, 2009
  • 5. ... and they lived happily ever after! Thursday, November 5, 2009
  • 6. Welcome to the real world! Thursday, November 5, 2009
  • 7. ... the golden age is over! Thursday, November 5, 2009
  • 8. What did really change? 49% 24% 23% 44% 32% 28% sucessful challenged failed 2000 2008 Project success Chaos Report, Standish Group Thursday, November 5, 2009
  • 9. What is the problem? Thursday, November 5, 2009
  • 10. There are many! Thursday, November 5, 2009
  • 11. Focus: Requirements Thursday, November 5, 2009
  • 12. What defines success? Software development is about delivering business value! Thursday, November 5, 2009
  • 13. What is being used? 19% 16% Never Seldom 13% Occasionally 45% 7% Often Always CHAOS Report 2000, Standish Group Thursday, November 5, 2009
  • 14. Providing Business Value? • Only 20% of all delivered features are actually used? • Something is wrong! • We can profit tremendously when we improve in requirements gathering and in transporting them successfully to the implementation Thursday, November 5, 2009
  • 15. Where Errors are introduced Requirements 27% Design 56% Code 7% Other 10% Source: James Martin, An Information Systems Manifesto Thursday, November 5, 2009
  • 16. Fighting Errors? • Traditional Developer Testing focuses on code and maybe design • Testing requirements is hardly ever a topic • We can profit tremendously when we improve in requirements gathering. Thursday, November 5, 2009
  • 17. Detailgrad Vision Verständnisgrenze Actor/Stakeholder Lastenheft Pflichtenheft Designdokument Kein Feedback von Stakeholdern möglich! Code Manual Test Automated „Usage“ Test Zeit Waterfall Project Thursday, November 5, 2009
  • 18. Costs to fix defects rise exponentially over time The longer a defect stays in the development process, the more expensive it is to fix Thursday, November 5, 2009
  • 19. Agile to the rescue! Thursday, November 5, 2009
  • 20. Detailgrad Verständnisgrenze Actor/Stakeholder Actor/ Business Vision Stakeholder Projektstart goals goals Product Epics Stories Backlog Sprint Planning Preparation Sprint Acceptance Sprint 1-n Stories Sprint Planning Backlog Criteria Sprint Implementation Acceptance Kurze Feedbackzyklen Test Definition mit Stakeholdern Code Manual Test/ Automated Demo Test Zeit Iterative Project Thursday, November 5, 2009
  • 21. The benefits • Constant feedback and priorization by the Stakeholder 19%16% • Build those features that are 45% 7% 13% needed and useful • Features have a much shorter 27% through-put time 7% • Allows validation of requirements 56% 10% Thursday, November 5, 2009
  • 22. Wait... ... isn’t this presentation supposed to be about testing? Thursday, November 5, 2009
  • 23. "I believe that the hardest part of software projects, the most common source of project failure, is communication with the customers and users of that software." - Martin Fowler, DSL Book Thursday, November 5, 2009
  • 24. It is all about communication! Thursday, November 5, 2009
  • 25. Verständnisgrenze Actor/Stakeholder Actor/ Business Vision Stakeholder Projektstart goals goals Product Epics Stories Backlog Sprint Planning Preparation Sprint Acceptance Sprint 1-n Stories Sprint Planning Backlog Criteria Sprint Implementation Acceptance Kurze Feedbackzyklen Test „Vertrauenswürdige Definition mit Stakeholdern Code Kommunikation“ Manual Test/ Automated Demo Test Zeit Enable communication Thursday, November 5, 2009
  • 26. How? Thursday, November 5, 2009
  • 27. Catch bugs in requirements! Thursday, November 5, 2009
  • 28. Thursday, November 5, 2009 Cost RE Analysis & Design Requirements Engineering Implementation Testing Deployment Analysis & Design Implementation Time Testing RE Analysis & Design Implementation Deployment Testing Deployment
  • 29. How? Thursday, November 5, 2009
  • 30. Now... ... finally testing! Thursday, November 5, 2009
  • 31. Requirements == Tests “As formality increases, tests and requirements become indistinguishable. At the limit, tests and requirements are equivalent.” - Equivalence hypothesis, Robert C. Martin • Traditionally this is not recognized because RE and Testing are set up as different disciplines • But they are about the same: how the system is used • Wouldn’t it make sense to leverage some synergies? Thursday, November 5, 2009
  • 32. Ideally Tests ... • … describe what the system should do • … are business readable • … establish a common vocabulary • … capture a shared set of examples • … enable communication • … are bound to the implementation • … are executable ... express shared understanding Thursday, November 5, 2009
  • 33. Building a shared understanding - Scripts - Test Data Mapping - Acceptance Criteria Business Business - Specification - Scripts Tester Tester - Use Cases - Test Data - User Stories - Acceptance Criteria Shared ? - Specification Understanding - Use Cases Mapping - User Stories - Code Mapping - Code - Unit Tests - Unit Tests - Test Data - Test Data Developer Developer Thursday, November 5, 2009
  • 34. Finding a common abstraction Shared Business Developer Tester Understanding Business Tester Developer Thursday, November 5, 2009
  • 35. Shared Understanding • Terminology and abstractions of business • Concrete enough for developers • Business readable • Bound to implementation • Consistent Thursday, November 5, 2009
  • 36. Specification by example • Abstract requirements and specifications are not a good tool for communication. Real-life examples are much better. • We mostly communicate by examples • Usually examples are not formalized and not shared Thursday, November 5, 2009
  • 37. Examples • Enable developers and testers to communicate efficiently with business people • Provide enough information for developers to implement and for testers to verify • Realistic examples contain precise information and ask for precise answers • Are a way to test specifications! Thursday, November 5, 2009
  • 38. A new tool! Thursday, November 5, 2009
  • 39. Agile Acceptance Testing Acceptance Test Driven Development Behavior Driven Development Thursday, November 5, 2009
  • 40. Demo Thursday, November 5, 2009
  • 41. Agile: new challenges! Thursday, November 5, 2009
  • 42. Challenges • Effort • Regression • Definition of Done • Streaming Requirements • Agile Planing • Authoritative Source of Information Thursday, November 5, 2009
  • 43. Effort Thursday, November 5, 2009
  • 44. Effort • Potentially shippable software each iteration • Testing everything each iteration • A lot of iterations -> a lot to test! • Consequence: Automation is a must! Thursday, November 5, 2009
  • 45. Regression Thursday, November 5, 2009
  • 46. Regression • Potentially shippable software each iteration • High velocity, changes are embraced • How to guarantee quality? • Consequence: Automation is a must! Thursday, November 5, 2009
  • 47. Definition of Done Thursday, November 5, 2009
  • 48. Definition of Done • When are we finished? • Testing is an integral part of the development process and not a separate discipline • Implementation and testing have to work together on a fine grained level • There has to be an efficient communication between implementation and testing • Consequence: Relation between implementation and testing has to be redefined! Thursday, November 5, 2009
  • 49. Streaming Requirements Thursday, November 5, 2009
  • 50. Streaming Requirements • Not all of the requirements are delivered upfront • Requirements are delivered just in time • Incrementally reveal details just-in-time • Pull instead of push • Embrace Change • Consequence: Requirements providers (stakeholders) must find a new way to deliver requirements Thursday, November 5, 2009
  • 51. Agile Planning Thursday, November 5, 2009
  • 52. Agile Planning • Goal-driven instead of scope-driven • How to plan something when the scope is uncertain? • Managing changes and uncertainty • You don’t know what can be tested when... • Consequence: Testers and developers have to interact on a fine grained and self organized level Thursday, November 5, 2009
  • 53. Authoritative Source of Information Thursday, November 5, 2009
  • 54. Authoritative Source of Information • Avoid excessive, disconnected artifacts. The source code is the artifact that ultimately matters. • No specification, no documentation doesn’t work either! • Communication is more important than documentation • But people as the only source of information is not an option • Consequence: Find a way for efficient specifications and documentation that is connected to the source code. Thursday, November 5, 2009
  • 55. "We're still figuring this stuff out. All of us." - Jay Fields Thoughts on developer Testing: http://blog.jayfields.com/2009/02/thoughts-on- developer-testing.html Thursday, November 5, 2009
  • 56. Müssen Entwickler Braucht es noch Tester? jetzt Testen? JA! Thursday, November 5, 2009
  • 57. Entwickler... • Bindet Beispiele and die Implementierung • Worauf gebunden wird ist flexibel! Thursday, November 5, 2009
  • 58. Tester ... • Bringen Beispiele in eine testbare Form • Kollaboration mit Stakeholders • Mindestens ein Beispiel pro Feature • Economy of Scale • Entwickeln die Tessprache • Nutzung auch für manuelle Tests Thursday, November 5, 2009
  • 59. Manuelle Tests ... • Zur Validierung von Main-Flows und Integration • Demo an Stakeholder • Geringere Anzahl • Unterstützt durch „Testsprache“ • Automatisierung aufwändiger! Thursday, November 5, 2009
  • 60. Aufwand? • Ca. 15% des Gesamtaufwands für Testspezifikation -> 1 Tester versorgt 6-7 Entwickler • Entwickler: 10-25% Aufwand für Binding -> Synergien Thursday, November 5, 2009
  • 61. Wartbarkeit? • Richtiger Abstraktionsgrad • Wiederverwendbarkeit von Bindings • Ebene der Bindings Thursday, November 5, 2009
  • 62. Why are tests important? "Nothing makes a system more flexible than a comprehensive suite of tests! Far above good architecture and good design!" - Robert C. Martin, Oredev, 2008 http://archive.oredev.org/topmenu/video/keynotebobmartin.4.5a2d30d411ee6ffd28880002007.html Thursday, November 5, 2009
  • 63. Testing is more than QA! • Regression Tests enable change • TDD: Drive the design through tests • Tests are examples that enable customer interaction • Tests are examples that can be the specification • Tests are examples that document the system • ATDD: Drive the specification through acceptance tests • Tests can be the definition of ‘done’ Thursday, November 5, 2009
  • 64. The Topic is Hot! www.cukes.info www.concordion.net www.fitnesse.org www.specflow.org JBehave / NBehave http://studios.thoughtworks.com/agile-test-automation Thursday, November 5, 2009
  • 65. Resources The RSpec Agile Testing Book Bridging the communication gap Thursday, November 5, 2009
  • 66. BDD is a Mindset not a Tool! Thursday, November 5, 2009
  • 67. Viel Erfolg beim Einsatz von Behavior Driven Development und Agilen Akzeptanztests! Jonas Bandi Software Architect TechTalk jonas.bandi@techtalk.ch blog.jonasbandi.net www.techtalk.ch twitter.com/jbandi Thursday, November 5, 2009