Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Effective Testing Practices in an Agile Environment


Published on

This is a practitioner’s view of testing and testing practices within an iterative development environment. We will explore the challenges of testing within such an environment and ways to better integrate the QA professional into what is inherently a developer-centric methodology. If quality is paramount, then we ought to move testing to the front of the line and test early and often. Automation lies at the heart of agility and we will look at how test automation techniques and test-first design philosophy might be applied at multiple-levels to drive quality.

Published in: Technology

Effective Testing Practices in an Agile Environment

  1. 1. Effective Testing Practices in an Agile Environment Raj Indugula  
  2. 2. ON-­‐DECK   AT-­‐BAT   DONE   What  is  the   problem  context?   Can  I  see  it  in   ac8on?   Concluding   thoughts.    Any   ques8ons?   What  are  the  guiding   principles  of  Agile   Tes8ng  ?   What  are  the   enabling  prac8ces?  
  3. 3. Agile Manifesto Individuals and interactions Processes and toolsover Working software Comprehensive documentation over Customer collaboration Contract negotiationover Responding to change Following a planover That is, while there is value in the items on the right, we value the items on the left more. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: 3
  4. 4. Shared Context drives Quality Individuals and interactions Working software Customer collaboration Responding to change 4
  5. 5. Quality is a priority Agile Approach Maximize value and quality within specified project constraints. 5   Quality 5
  6. 6. Integrated Teams & Iterative Delivery Challenges 6 How do we bridge the communication gap between Bus./QA/Dev? How do we ensure that the evolving software does not regress?
  7. 7. ON-­‐DECK   AT-­‐BAT   DONE   What  is  the   problem  context?   Can  I  see  it  in   ac8on?   Concluding   thoughts.    Any   ques8ons?   What  are  the  guiding   principles  of  Agile   Tes8ng  ?   What  are  the   enabling  prac8ces?  
  8. 8. Move Quality Upstream •  Testing not a trailing activity •  QA leads the Sprint by providing guidance and feedback on the business needs of the emerging product 8
  9. 9. Testing is Continuous, NOT a Phase 9
  10. 10. Testing is Collaborative •  Quality is everyone’s problem, not just of the testers •  Testing is the responsibility of the whole team 10
  11. 11. Tests represent Customer Expectations •  Shared understanding of what it means for a story to be done 11
  12. 12. Quick Feedback •  Faster feedback loops increase Agility – the ability to respond to change •  Test automation provides rapid feedback on how the software is behaving 12
  13. 13. “Leave No Broken Windows” •  Fix bugs as they are found •  The sooner you find a defect, the cheaper it is to fix 13
  14. 14. It isn't Done until it’s… “Done Done” Coded Tested Done 14
  15. 15. Test-Driven Defining tests with the requirements guides development 15
  16. 16. ON-­‐DECK   AT-­‐BAT   DONE   What  is  the   problem  context?   Can  I  see  it  in   ac8on?   Concluding   thoughts.    Any   ques8ons?   What  are  the  guiding   principles  of  Agile   Tes8ng  ?   What  are  the   enabling  prac8ces?   ATDD   TDD   Other   prac8ces  
  17. 17. Agile Engineering Practices 17 Agile Engineering practices allow teams to move fast, be flexible and deliver high quality software Bill Wake,
  18. 18. What are acceptance tests? •  Tests that demonstrate business intent of system from end user’s point of view •  Black-box testing 18
  19. 19. What is Acceptance Test Driven Development (ATDD)? A practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. - Elisabeth Hendrickson 19
  20. 20. Key Characteristics 1.  Elicit details from the business stakeholder(s) about their expectations 2.  Distill acceptance criteria into automatable tests expressed in a natural language 3.  Wire the tests to SUT with “glue” code (as part of implementation) 20
  21. 21. 21
  22. 22. When to Use •  During Requirements Elaboration “First Cut” once the story is written Card level acceptance criteria •  During a Three Amigos session (Business, QA, and Dev) Expand the story card acceptance criteria Complete set of test scenarios 22
  23. 23. User Story As a <type of user>, I can <immediate goal> so that <business outcome>. User Role (Who?) Desired Function (What?) End Result (Why?) 23 3 C’s – CARD, CONVERSATION, CONFIRMATION
  24. 24. Modern Agile Acceptance Model Conditions of Satisfaction – Broad Terms Acceptance Criteria – Further Refined Examples – Actual scenarios or data Executable Examples – Ready to automate 24
  25. 25. Acceptance – Conditions of Satisfaction Testing a Registration Function •  What constraints should we impose? •  Business stakeholders and PO agree that passwords should not be easy to crack. Please fill in to register. Email Address Password Register 25
  26. 26. Acceptance – Acceptance Criteria Acceptance Criteria, the Details The PO works with testers and developers from the team, business stakeholders, users or SMEs to come up with this definition of “not easy to crack”: •  Must be at least 8 characters and no more than 12 •  Must contain only alphanumerics and the period •  Must contain at least one digit •  Must contain at least one alpha character Please fill in to register. Email Address Password ???????????? 26
  27. 27. Acceptance – Examples Written •  Examples, Making it Real •  Actual examples are the ultimate way to communicate requirements Password Expected Expected Message Comment abc123 Invalid You password must be at least 8 characters long, and no more than 12 characters long. abcdefghi Invalid Your password must contain at least one character and one number. 1aaaaaaaaa Valid Why valid? ajx972dab Valid 27
  28. 28. Acceptance – Testable Examples in Gherkin •  Executable Example, Making it Repeatable •  Examples that can be executed are the final step Given the “Unregistered User” user has navigated to the “register” page When entering “newuser” in the “Username” field And entering “abc123” in the “Password” field And entering “abc123” in the “Confirm Password” field And pressing the “Register” button Then the text “Thank you for Registering” should appear on page And the URL should end with “use/accountPage” 28
  29. 29. Gherkin – Another Example •  Plain English with a specific format §  Feature §  Scenario §  Given When Then And But •  Can be used for both manual testing and automation 29
  30. 30. Automated Acceptance Tests Tool (Cucumber, SpecFlow, FitNesse) Specification expressed in common language “Glue” code that ties specification to SUT 30
  31. 31. Business Analyst Develop usage scenarios Developer Create and maintain acceptance test “glue” code that ties test specification to system under test Tester Create and execute tests to simulate usage scenarios; Automate regression testing •  Should be an integral part of every iteration/sprint, not just a trailing activity •  Vital not only to prove completeness of a feature in repeatable fashion, but also to prove that software does not regress as it evolves •  Success depends on cooperation & collaboration Testing Collaboratively 31
  32. 32. Benefits of ATDD •  Improved requirements elicitation •  Consensus between BA/QA/Dev on the story helps prevent bugs & gives clear target for development •  Reuse of Acceptance Scenarios for all phases of testing •  Creates clear examples that everyone understands; discovers some problems before any development 32
  33. 33. ON-­‐DECK   AT-­‐BAT   DONE   What  is  the   problem  context?   Can  I  see  it  in   ac8on?   Concluding   thoughts.    Any   ques8ons?   What  are  the  guiding   principles  of  Agile   Tes8ng  ?   What  are  the   enabling  prac8ces?   ATDD   TDD   Other   prac8ces  
  34. 34. What are unit tests? •  Developer tests that determine whether the smallest piece of testable software in an application is behaving as expected •  White-box testing 34
  35. 35. Structure of a Unit Test @Test! public void itAddsOnePlayerToTheGame()! {! !Game game = new Game();! ! !Player al = game.AddPlayer("Al");! ! !Assert.assertEquals(1, game.numPlayers());! !Assert.assertTrue(game.isPlaying(al));! !Assert.assertFalse(game.isPlayable());! }! Assert Arrange Act 35   …all related to same behavior …title is a behavior
  36. 36. Tests should… •  Not depend on one another •  Not affect one another •  Leave the system under test unaltered •  Be fast executing •  Be executable in any order, number or combination of one another 36
  37. 37. What is Test Driven Development (TDD)? 1.  Write a unit test 1.  Use unit testing and mocking framework (JUnit, Mockito etc.) 2.  May need to write some skeleton code to make it compile 2.  Run it and watch it fail (no code yet) 3.  Write code until it passes 4.  Repeat 37 Write a new test Red (Failing Test) Write Code Green (Test Passes) The tests help determine what code you write (“Incremental Design”)
  38. 38.     Write a new test Test fails Integrate Write code Refactor code, make sure tests pass Test passes Developer heartbeat Red, Green, Refactor Refactoring is supported by the tests and good unit test coverage makes refactoring safe to do 38
  39. 39. TDD – Benefits •  Better design – less coupling •  Supports change •  Prevents gold-plating •  Prevent bugs •  Discover bugs 39
  40. 40. Test Levels Isolation Tests •  True unit tests, focusing on behaviors •  Many, and fast Collaboration Tests •  “End-to-end” tests •  Find bugs in conversations between collaborating modules 40
  41. 41. Test Doubles (Mocks) Used to isolate unit tests form external dependencies Why use them? •  Ease of setup •  Fast executing •  To work with a subsystem that doesn’t exist •  Simulate various execution paths of external system 41
  42. 42. Test Automation Pyramid 42 Unit Tests/ Component Tests Cucumber, FitNesse, SpecFlow xUnit, Mocks Selenium Developer-centric (Are we building the code right?) Business-centric (Are we building the right code?) Adaptation of Mike Cohn's test automation pyramid Exploratory Testing GUI Tests Acceptance Tests
  43. 43. ATDD/TDD Cycle 43
  44. 44. TDD vs. ATDD Historically ATDD did not come about till well after TDD ATTD §  is a specification and testing practice §  Determines what “Done” means for a story §  Creates a common understanding and shared language to discuss features (especially with Gherkin) §  Cycle times on the order of hours to days TDD §  is a coding and design practice §  Provides a mechanism for exploring the solution space §  Creates base of tests to support refactoring §  Cycle times on the order of seconds to minutes 44
  45. 45. ON-­‐DECK   AT-­‐BAT   DONE   What  is  the   problem  context?   Can  I  see  it  in   ac8on?   Concluding   thoughts.    Any   ques8ons?   What  are  the  guiding   principles  of  Agile   Tes8ng  ?   What  are  the   enabling  prac8ces?   ATDD   TDD   Other   prac8ces  
  46. 46. Version control Tests with Code Source Code Unit/Component Tests Acceptance Tests Source Control Repository (Git, Subversion…) 46
  47. 47. Requirements gathering Application Development Source Control Continuous Integration (CI) Server Continuously commit changes and merge changes from others Pull changes, Build, run tests Deploy to higher environment Poll for changes Development tasks from requirements test scenarios from business requirements for acceptance criteria Acceptance Test Environment - Test harness - Tests (automated + manual) Execute acceptance tests against the deployed application QA Environment Deployable artifacts Build Automation & Continuous Integration 47
  48. 48. ON-­‐DECK   AT-­‐BAT   DONE   What  is  the   problem  context?   Can  I  see  it  in   ac8on?   Concluding   thoughts.    Any   ques8ons?   What  are  the  guiding   principles  of  Agile   Tes8ng  ?   What  are  the   enabling  prac8ces?  
  49. 49. Barriers to Adoption Developer : “I’m a developer, not a tester” Management: “Why does Dev help with testing, when we have QA?” •  Necessitates behavioral change •  Quality is not free •  Simple, but not easy •  Requires discipline •  Needs generalizing specialists 49
  50. 50. Bridging the Divide SharedUnderstanding Collaborative Testing Quick Feedback 50
  51. 51. “Before beginning a hunt, it is wise to ask someone what you are looking for before you begin looking for it.” —Winnie the Pooh 51
  52. 52. Thanks! 52
  53. 53. Resources 53
  54. 54. Some Tools (open source) •  Unit testing: JUnit, Mockito, EasyMock •  Acceptance testing: FitNesse Cucumber •  GUI testing: Selenium, Sahi •  Javascript testing: Jasmine, Qunit •  Testing the database: DBUnit, SQLUnit, DBFit •  Performance testing: JUnitPerf, JMeter •  Usability testing: Manual unfortunately 54
  55. 55. Some Tools (open source) •  Unit testing: MSTest, NUnit, Moq •  Acceptance testing: SpecFlow •  GUI testing: Selenium •  Javascript testing: Chutzpa •  Testing the database: Tsqlt, SSDT SQL Test •  Performance testing: JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest •  Usability testing: Manual unfortunately 55
  56. 56. Agile Software Development •  Extreme Programming Explained - Kent Beck •  TDD: A Practical Guide – Dave Astels •  Refactoring – Martin Fowler •  Agile Estimating and Planning – Mike Cohn •  Pragmatic Project Automation – Pragmatic Prog. 56
  57. 57. Acceptance Testing •  Bridging the Communications Gap – Gojko Adzic •  Agile Testing – Lisa Crispin and Janet Gregory •  The Cucumber Book - Wynne M. and Hellesoy A. Gherkin Cucumber 57
  58. 58. TDD/Refactoring •  TDD: By Example - Kent Beck •  TDD: A Practical Guide – Dave Astels §  The Art of Agile Development – James Shore •  Refactoring – Martin Fowler •  Working Effectively with Legacy Code – Mike Feathers •  Refactoring to Patterns – Joshua Kerievsky •,, 58