Synchronizing Software Testing with Agile Requirements Practic


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Ask who has done agile testing, who’s heard of it, who hasn’t
  • Who we are
  • what we will be covering today
  • Since many are not familiar with agile testing, I’d like to set the context – what is Agile and how is it different ( I think better). We’ll go over what I have seen on many teams that are starting to embrace agile – these are both technical challenges – how to accomplish agile testing, as well as organizational challenges – team and people issues. I’ll then outline what I have observed as keys to successfully moving the team to agile testing and becoming efficient and effective at short iterations and rapid releases
  • There are many different but similar agile methods. Some of them you might have heard of or about. They all focus on …
  • Underlines are what we’ll focus on today
  • Do Features- generalized – could be Stories, Use Cases, Other Requirements, Defects. We have the concept of Story Cards – the specification of the work to do, that is further elaborated by a UC, Req, etc Shows how features get done through multiple iterations leading to a release. At Rally, we have settled into 4 dev iterations, a 1 week hardening and Releases every 2 months or so. This also illustrates the Define, plan, iteration and accept rhythm that we get into doing agile development.
  • Within an iteration you definitely want to get into a good rhythm. There needs to be just enough elaboration ahead of time so that the team can accurately estimate dev, doc and testing tasks. An iteration planning meeting is held at the beginning to go through what is planned. The team then tasks out to get a detailed level of estimation. This allows the team to know what can fit into the iteration. Based on previous velocity, you can determine what fits. A rank order for the story cards gives the order in which a feature needs to be developed. The goal is to get the most you can but it is better to get some all over the goal line as opposed to working on all of them at once. In this illustration you would get Feat one developed, tested and accepted before moving onto the next feature. At the end you can go over the features and demo the results. You can also hold a retrospective to talk about things to change – to start doing and to stop doing
  • Let the fun begin!
  • Good to have mock-ups, Use Cases with main success scenario, a few Extensions, some other requirements that help to explain the functionality of the feature. Reality with 2 week iterations, there is minimal time to have the team review before the start. Further elaboration continues in the Iteration as Dev, Test and Doc start to examine what needs to be done. Further elaboration continues in the Iteration as Dev, Test and Doc start to examine what needs to be done.
  • Testing Architecture to Avoid at all costs
  • Evil rabbits Good News: The whole team became highly motivated to do it differently, and to find a better way
  • FitNesse it is the great bridge between Dev and Test, GUI tests and Unit Tests
  • Team needs to figure out what they want to try, what they want to fix, what they want to stay the same Don’t feel like you have to everything all at once. Can change some things, and not others. Just keep learning and adapting – hold retrospectives at least every Release to keep moving the bar Don’t feel like you have to everything all at once. Can change some things, and not others. Keep learning and adapting – hold retrospectives at least every Release to keep moving the bar
  • Synchronizing Software Testing with Agile Requirements Practic

    1. 1. Synchronizing Software Testing with Agile Requirements Practices Jean McAuliffe, Dean Leffingwell May 2005
    2. 2. Background on Speakers <ul><li>Jean McAuliffe </li></ul><ul><ul><li>Product Manager, Rally Software Development </li></ul></ul><ul><ul><li>2 years Agile Development, Certified Scrum Master </li></ul></ul><ul><ul><li>Former Senior QA Manager for Rational RequisitePro. </li></ul></ul><ul><ul><li>Over 15 years experience in all aspects of software development (defining, developing, testing, training and supporting) for Software Development, Bio-Engineering and Aerospace companies </li></ul></ul><ul><li>Dean Leffingwell </li></ul><ul><ul><li>Advisor and coach to a number of developmental-stage software businesses. </li></ul></ul><ul><ul><li>Former Senior VP of Rational Software Corporation where he was responsible for the commercial introduction of the Rational Unified Process. </li></ul></ul><ul><ul><li>Lead author of the text Managing Software Requirements: Second Edition: A Use Case Approach, Addison Wesley, 2003 </li></ul></ul>
    3. 3. Abstract <ul><li>Agile requirements practices generally defer commitment to artifacts such as requirements until the “last responsible moment.” This challenges the test team to stay continuously synchronized with the product owners and developers as they have little lead time, if any, from the time requirements are available until the time the test artifacts must be in place. </li></ul><ul><li>This presentation describes the organizational and technical challenges associated with just-in-time agile requirements practices and techniques test teams can use to address them. </li></ul>
    4. 4. Agenda <ul><li>Context for Agile Testing </li></ul><ul><li>Technical Challenges </li></ul><ul><li>Organizational Challenges </li></ul><ul><li>Keys for Success </li></ul>
    5. 5. Popular Agile Methods Agile Rational Unified Process (RUP) Scrum (Ken Schwaber) Feature Driven Development (Jeff DeLuca) Crystal (Alistair Cockburn) Lean Software Development (Mary Poppendieck) Adaptive Software Development (Jim Highsmith) XP (Kent Beck) Dynamic System Development Method (Dane Faulkner)
    6. 6. Excerpts from the Agile Manifesto <ul><li>Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. </li></ul><ul><li>Welcome changing requirements , even late in development. Agile processes harness change for the customer's competitive advantage. </li></ul><ul><li>Working software is the primary measure of progress . </li></ul><ul><li>Deliver working software frequently , from a couple of weeks to a couple of months, with a preference to the shorter timescale. </li></ul><ul><li>Business people and developers must work together daily throughout the project. </li></ul><ul><li>Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done . </li></ul><ul><li>http:// </li></ul>
    7. 7. A Generalized Agile Release Process Release Iteration 1 Iteration 2 Iteration 3 Iteration … <ul><li>Do Feature 1 </li></ul><ul><li>Do Feature 2 </li></ul><ul><li>Do Feature 3a </li></ul><ul><li>Do Feature 3b </li></ul><ul><li>Do Feature 4a </li></ul><ul><li>Do Feature 4b </li></ul><ul><li>Do Feature 5 </li></ul><ul><li>Do Feature 4c </li></ul><ul><li>Do Feature 6 </li></ul><ul><li>Do Feature 7 </li></ul>Backlog <ul><li>Feature 8 </li></ul><ul><li>Feature 9 </li></ul><ul><li>Feature 10 </li></ul><ul><li>… . </li></ul>Backlog <ul><li>Feature 1 </li></ul><ul><li>Feature 2 </li></ul><ul><li>Feature 3 </li></ul><ul><li>… . </li></ul><ul><li>Feature 9 </li></ul>
    8. 8. Agile Iteration Cadence Demo & Retro Iteration N Iteration N+1 Iteration N-1 Detailed Iteration Planning & Design Dev Feature Priority 1 Auto. Tests Feature 1 Dev Feature Priority 2 Dev Feature Priority 3 Auto. Tests Feature 3 Dev Feature Priority 4 Auto. Tests Feature 4 Dev Feature Priority 5 Auto. Tests Feature 5 Accept Accept Accept Auto. Tests Feature 2 Accept Accept Requirements Are Refined Initial Elaboration Requirements With Tests
    9. 9. What’s Different about Testing in Agile? <ul><li>Just-In Time Requirements Elaboration </li></ul><ul><ul><li>No SRS-level waterfall documents to drive testing plan </li></ul></ul><ul><ul><li>Requirements and Test Cases developed in parallel or test first strategy </li></ul></ul><ul><li>More Frequent Iterations, More Frequent Releases </li></ul><ul><ul><li>Testing needs to happen Early and Often </li></ul></ul><ul><ul><li>Frequent to continuous regression testing </li></ul></ul><ul><ul><li>High need to automate nearly everything </li></ul></ul><ul><ul><li>Everyone needs to Test </li></ul></ul><ul><li>Two Levels of Testing </li></ul><ul><ul><li>Iteration Vs. Release testing patterns </li></ul></ul>
    10. 10. Technical Challenges Requirements are changing fast. How does test keep up? Test early and often. How exactly do we move testing forward? Need to move off manual testing and more into automation. How does this happen? Different kinds of testing need to happen at different times. How do these get managed?
    11. 11. Requirements are Changing Code & Deliver Generate TCs UC/SR Update TCs Run TCs Fail TCs Code & Deliver Pass & Accept Code & Deliver Updates Updates
    12. 12. Requirements Changing is a Good Thing? <ul><li>Probably the hardest agile principle for the team to embrace. </li></ul><ul><ul><li>Need to elaborate the feature ahead of time </li></ul></ul><ul><ul><li>There is minimal time to have the team review before the start. </li></ul></ul><ul><ul><li>Sometimes you have to rewrite </li></ul></ul><ul><li>Bottom-line: everyone collaborates to make the feature as useful for the customer as possible. </li></ul>
    13. 13. Requirements to Test Cases <ul><li>Use Case Scenario Tests are perfect Acceptance Tests </li></ul><ul><li>Use Case A </li></ul><ul><ul><li>Scenario 1 Test Case 1 </li></ul></ul><ul><ul><li>Scenario 2 Test Case 2 </li></ul></ul><ul><li>Declarative Requirements that further refine the Use Case may be better suited to going directly to automation </li></ul><ul><ul><li>Have one Test Case be the container for all of the automation results. </li></ul></ul><ul><ul><li>All automated tests have to pass before the Test Case passes. </li></ul></ul>
    14. 14. Need to Test early and often <ul><li>Need to test early in the Iteration – do not want mini-waterfalls </li></ul><ul><li>Need to test on check-in – Don’t break the build </li></ul><ul><li>Need to test nightly – Don’t wait for a Regression Iteration </li></ul>
    15. 15. Mike Cohn’s Testing Pyramid <ul><li>Small number </li></ul><ul><li>Automate many </li></ul><ul><li>Find the right ones </li></ul><ul><li>Largest numbers </li></ul><ul><li>Foster Test Driven Design </li></ul>Start Stop ? Start Stop Look GUI Acceptance Tests FitNesse Unit Tests
    16. 16. Break the Manual Testing Paradigm Manual GUI Acceptance Tests Unit Tests Automated GUI Tests <ul><li>Easy to Create </li></ul><ul><li>Very familiar – what we always do </li></ul><ul><li>Typically tedious </li></ul><ul><li>How do we know coverage? </li></ul><ul><li>Need Automation specialists </li></ul><ul><li>Automation good for performance </li></ul><ul><li>Seems like we always rewrite </li></ul><ul><li>Sometimes fragile </li></ul><ul><li>What is Dev testing? </li></ul><ul><li>How do we know what these are? </li></ul><ul><li>How do we know when they fail? </li></ul>Start Stop ? Start Stop Look
    17. 17. Manual Testing Conundrum <ul><li>“You can never have too many manual acceptance tests” </li></ul><ul><ul><li>Manual tests are cute little bunnies, before you know it you have hundreds or thousands in your regression suite </li></ul></ul><ul><ul><li>You inadvertently dig a hole you can never get out of </li></ul></ul><ul><ul><li>Whole team had to help run regression suite </li></ul></ul><ul><li>Defect count typically is high </li></ul><ul><ul><li>Most defects were found as manual tests were elaborated </li></ul></ul><ul><ul><li>Regression tests typically didn’t find many defects </li></ul></ul><ul><ul><li>Commonly found defects – things we didn’t think of </li></ul></ul>
    18. 18. Better, But Not Perfect Testing Architecture <ul><li>Still too many here </li></ul><ul><li>Add FitNesse </li></ul><ul><li>Increase Coverage </li></ul><ul><li>Increase Capability </li></ul>Start Stop Look Unit Tests Manual GUI Acceptance Tests Automated GUI Tests & FitNesse
    19. 19. Testing Types and Scheduling <ul><li>Manual Group Explore </li></ul><ul><li>Roles and Personas </li></ul><ul><li>Acceptance and Functional tests from previous Iterations </li></ul><ul><li>Profiling and Simulation Automation </li></ul><ul><li>Combination of Unit tests, FitNesse </li></ul><ul><li>Minimize Manual </li></ul><ul><li>Generate off of Use Cases to get scenario tests. </li></ul><ul><li>Before Releasing </li></ul>Exploratory <ul><li>Run Nightly </li></ul>Regression <ul><li>Do it periodically </li></ul><ul><li>Don’t wait till the end of the Release cycle </li></ul>Load & Performance <ul><li>Build Verification and Run Nightly </li></ul>Acceptance -Functional <ul><li>During the Iteration </li></ul>Acceptance – GUI?
    20. 20. Keys to Overcome the Technical Challenges <ul><li>Continuous Builds </li></ul><ul><li>Nightly Regression testing </li></ul><ul><li>Find a way to increase FitNesse testing at the application layer http:// </li></ul><ul><li>Make Unit Testing a priority </li></ul><ul><li>From found defects – create automated tests that go into Regression </li></ul>
    21. 21. Organizational Challenges <ul><li>Dev as Testers and Testers as Dev – how does that happen? </li></ul><ul><li>Resistance to Change – how do we get the team to welcome and embrace changes and not feel threatened? </li></ul><ul><li>Testers are an integral part of the team- do we need to re-organize to make this happen? </li></ul>
    22. 22. I’m a Developer, Not a Tester <ul><li>Pretty typical to hear push back from developers that they </li></ul><ul><ul><li>Don’t have time to do all of this testing </li></ul></ul><ul><ul><li>Number of features delivered will go down </li></ul></ul><ul><ul><li>Don’t really want to do all this testing </li></ul></ul><ul><li>Testers can help </li></ul><ul><ul><li>Provide guidance on how to break software, art of creative destruction </li></ul></ul><ul><ul><li>Pair testing with developers works well </li></ul></ul><ul><li>Have developers help out with manual regression testing. </li></ul><ul><ul><li>“Can’t I write a test for this instead of running it manually?” </li></ul></ul>
    23. 23. I’m a Tester Not a Developer <ul><li>Pretty typical to hear from testers </li></ul><ul><ul><li>That they don’t feel comfortable or knowledgeable about coding </li></ul></ul><ul><ul><li>That they maybe won’t be needed anymore </li></ul></ul><ul><li>Developers can help </li></ul><ul><ul><li>Developers can create the fixtures (code running the test) needed to make FitNesse testing work </li></ul></ul><ul><ul><li>To make it easier to auto test the code at the GUI level </li></ul></ul>
    24. 24. Resisting Change <ul><li>Resistance is common </li></ul><ul><ul><li>It is easier to do what is familiar, than risk something new </li></ul></ul><ul><ul><li>Time-challenges may keep you doing the old way </li></ul></ul><ul><ul><li>Fear of failing keeps you in the status quo </li></ul></ul><ul><li>Get the whole team involved in trying to change </li></ul><ul><ul><li>Team needs to figure what works best </li></ul></ul><ul><ul><li>Don’t feel like you have to do everything all at once </li></ul></ul><ul><ul><li>Keep learning and adapting </li></ul></ul>
    25. 25. Testers on the Team <ul><li>Your organization may have testing as a separate group – look for ways to integrate them into the team </li></ul><ul><ul><li>Creating feature or component teams comprised of all disciplines is one way </li></ul></ul><ul><li>Co-location is a great way to hear and share information </li></ul><ul><li>Daily stand-ups with the whole team keeps the information current </li></ul>
    26. 26. Keys to Overcome the Organizational Challenges <ul><li>Have Dev help run manual Regression tests </li></ul><ul><li>Pair Dev and Test on Unit and FitNesse Testing </li></ul><ul><li>Co-location of all the team </li></ul><ul><li>Daily Standups </li></ul><ul><li>Do Retrospectives </li></ul>
    27. 27. Summary <ul><li>Agile Pulls Testing Forward </li></ul><ul><ul><li>You need to change your tools and approaches to move it forward </li></ul></ul><ul><ul><li>You might need to change the model/structure of your team </li></ul></ul><ul><li>With Agile, you will create faster Release cycles, shorter Iterations, more satisfied customers, and team members that enjoy what they are doing </li></ul>
    28. 28. Useful References <ul><li>Beck, Kent, Test-Driven Development By Example, Addison Wesley, 2003 </li></ul><ul><li>Cohn, Mike, User Stories Applied For Agile Software Development , Addison Wesley, 2003 </li></ul><ul><li>Crispin, Lisa, House, Tip, Testing Extreme Programming , Addison Wesley, 2003 </li></ul><ul><li>Leffingwell, Dean, Widrig, Don, Managing Software Requirements: Second Edition: A Use Case Approach, Addison Wesley, 2003 </li></ul>
    29. 29. Thank You Contact Info Jean: [email_address] Dean: [email_address] Questions? Copyright 2003-2005, Rally Software Development Corp