Testing & Scrum
•Download as PPTX, PDF•
1 like•1,127 views
The document discusses organizing testing within the Scrum framework. It provides key messages: 1) everyone is a tester, 2) testing is infused into everything, not isolated, 3) some testing can be placed outside the Scrum team if multiple teams exist, and 4) complexity determines who performs different types of testing. It also discusses test ownership, automation, regression testing, and defines types of testing like isolated, contract, and integration testing. The overall message is that testing is integral to Scrum and should involve the whole team, while allowing for specialized testing roles.
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Testing & Scrum
Similar to Testing & Scrum (20)
More from Johan Hoberg
More from Johan Hoberg (20)
Recently uploaded
Recently uploaded (20)
Testing & Scrum
- 1. TESTING & SCRUM Experiences of organizing and structuring testing within the Scrum Framework
- 2. Introduction - Me • Johan Hoberg • 10 years at Sony Mobile and 1 year at King • Tester, Test Team Leader, Test Leader, Test Architect/Strategist • Passion for testing and computer games
- 3. Introduction – This presentation • My experiences from working with Scrum, and how I apply that into organizing and structuring test within the Scrum Framework • Not a best practice – just my thoughts applied to my specific context • Hopefully it will give you some ideas on how to do something similar in your context
- 5. Generalizing Specialists (or Specializing Generalists) [12]
- 6. KEY MESSAGE #1 Everyone is a Tester
- 7. Definition of Quality “Quality is value to some person”
- 8. Agile Test Quadrants [7]
- 9. Acceptance Criteria Given / When / Then
- 10. Acceptance Criteria & Testers Writing good Acceptance Criteria requires a testing skillset
- 11. Testability [13] The practical testability of a product is how easy it is to test* by a particular tester and test process, in a given con- text†. Practical testability is a function of five other “testabilities:” project-related testability, value-related testability, subjective testability, intrinsic testability, and epistemic testability (also known as the “risk gap”). Just as in the case for quality in general, testability is a plastic and multi-dimensional concept that cannot be usefully expressed in any single metric. But we can identify testability problems and heuristics for improving testability in general. Interesting Testability Dynamics
- 12. KEY MESSAGE #2 Testing is infused into everything
- 13. Test Ownership Scrum Team Outside of Scrum Team Isolated Tests Contract/Collaboration Tests Integration Tests System Tests Equipment & Competence Specific Tests • Clear ownership important • What ownership structure you use is less important • This structure works in my context
- 14. Definitions: Testing in the Scrum Team • Isolated Tests • Contract Tests • Collaboration Tests • Integration Tests
- 15. Definitions: Testing outside the Scrum Team • System Tests • Equipment Specific Tests • Competence Specific Tests
- 16. KEY MESSAGE #3 You can place some testing outside of the Scrum Team if you have multiple teams
- 17. Who tests what? (Simplification) Anyone DeveloperTester Tester
- 18. KEY MESSAGE #4 Complexity is the main separator for who does what
- 19. Test Automation The Scrum Team should make the decision what they want to automate and what they want to test manually
- 20. Regression Testing Everyone is responsible for covering their own test areas during regression test activities
- 21. Testing & Checking “Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.” [5]
- 22. (Exploratory) Testing [6] • “Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.” • All testing is exploratory, even scripted testing, if you are doing it responsibly
- 23. Conclusion • Testing is an integral part of the Scrum Framework • Everyone should contribute • But there is still a place for a strong testing skillset, driven by the complexity of the product • Test Ownership should be clear and it is possible to place some testing outside of the Scrum Team
- 24. References [1] Definition of Quality Weinberg, Gerald M. (1992), Quality Software Management: Volume 1, Systems Thinking, New York, NY: Dorset House Publishing, p. 7 [2]Agile Manifesto Principles http://agilemanifesto.org/principles.html [3] The Scrum Guide http://www.scrumguides.org/docs/scrumguide/v1/scrum-guide-us.pdf [4] Acceptance Criteria http://www.leadingagile.com/2014/09/acceptance-criteria/ [5] Testing and Checking http://www.satisfice.com/blog/archives/856 [6] Exploratory Testing 3.0 http://www.satisfice.com/blog/archives/1509 [7] Agile Testing Quadrants http://www.developsense.com/presentations/2014-06-Dublin-RSTAgileTesting.pdf [8]Integration Tests are a Scam https://vimeo.com/80533536 [9]Cynefin http://en.wikipedia.org/wiki/Cynefin [10] Heuristic Risk-Based Testing http://www.satisfice.com/articles/hrbt.pdf [11]Contract Tests: An Example http://blog.thecodewhisperer.com/2011/07/07/contract-tests-an-example/ [12]To combine … or not http://angryweasel.com/blog/to-combine-or-not/ [13] Heuristics of SoftwareTestability http://www.satisfice.com/tools/testable.pdf
Editor's Notes
- There are meetings and artifacts described in the Scrum Framework These are not the end goal – these are a way to reach the goal Which is self organizing teams Once a team is self organizing, they themselves can choose how they want to work
- “A lot of people seem to think that discipline-free software teams, everyone can do everything – which is, of course, flat out wrong. Instead, it’s critical that a good software team has (generalizing) specialists who can look critically at quality areas that span the product.” “There also will/must be folks who live entirely in the outer ring, and there will be people like me who typically live in the outer ring, but dive into product code as needed to address code problems or feature gaps related to the activities in the outer loop. Leaders need to support (and encourage – and celebrate) this behavior…but with this much interaction between the outer loop of testing and investigation, and the inner loop of creating quality features, it’s more efficient to have everyone on one team.”
- Everyone in the Scrum Team must contribute to the testing effort Each person has a different skillset that must be used efficiently in the testing effort So everyone in the team is a tester – but certain people have a skillset that is optimal for finding those complex problems that other people might miss
- “Quality is value to some person”[1] Working software is primary measure of progress [2] The Scrum Team owns the quality of the product [3] The Definition of Done should define a level of quality of the output that the Development Team delivers
- James Bach Build something As we do so we – build cleanly and simply So that we can – build something with change in mind As we do so we – foster testability So that we can – study what we have built As we do so we – experiment imaginatively and suspiciously So that we can – discover something worth building As we do so we – develop the design So that we can – build some of it
- “Acceptance Criteria are the conditions that a software product must satisfy to be accepted by a user, customer, or in the case of system level functionality, the consuming system.” [4] “Acceptance Criteria are a set of statements, each with a clear pass/fail result, that specify both functional and non-functional requirements, and are applicable at the Epic, Feature, and Story Level. Acceptance criteria constitute our “Definition of Done”, and by done I mean well done.” [4] The Given/When/Then format is helpful way to specify acceptance criteria: Given some precondition When I do some action Then I expect some result
- Writing Acceptance Criteria has a lot in common with testing It requires much of the same skill set to be able to write good Acceptance Criteria as it does to perform testing Even if the Product Owner is responsible for the Acceptance Criteria, a tester can add value by contributing with his/her expertise
- James Bach Working to improve testability is also a key part of a testers job Make sure the right skills and tools are available Highlight the need of designing a product that is testable Make sure the right communication channels are in place Make sure test oracles are in place And so on …
- Scrum is founded on Empirical Process Control [3] Empiricism asserts that knowledge comes from experience and making decisions based on what is known [3] Testing is not something you just do at the end of a sprint – it is infused into basically every activity
- We need to identify who is responsible for performing what types of tests There are only two important parties for the Scrum Team The Team Outside of the Team Clear ownership is Key This is just my way of grouping different types of tests, it is not the only way – find a way that works for you
- Isolated Tests State-based tests In other words, we are trying to answer this question: If everyone around the object-under-test works perfectly, does that object work perfectly "...assume perfect collaborators in order to establish basic correctness, but remembering to qualify the assumption made about those collaborators by closing contracts and collaboration checks…” Contract / Collaboration / Integration tests Integrated Tests are a Scam by J.B. Rainsberger [8] Contract Test: http://thecodewhisperer.tumblr.com/post/1325859246/in-brief-contract-tests “Contract Tests explain how a class should extend a superclass or implement an interface, so that I don’t have to read a bunch of prose to figure out how to do that.” “…a test class that can run the same set of tests for two different implementations of the same interface.”[11] Collaboration Tests: Also known as interaction tests, as opposed to state-based tests “Does the client talk to the next layer correctly?” “Testing interactions means you're verifying that the code under test calls certain methods properly.” http://googletesting.blogspot.se/2013/03/testing-on-toilet-testing-state-vs.html Integration tests: http://en.wikipedia.org/wiki/Integration_testing “An integrated test is any (low level) test where when it fails you cannot pinpoint exactly what went wrong.“ Individual software modules are combined and tested as a group Integration tests give you feedback about whether your implementation works : Isolated tests give you feedback about whether your design works (because they manage complication) Integration tests give you feedback about whether there are threading problems, performance problems, etc : Isolated tests give you feedback about whether the contracts between internal actors are binding and valid, regardless of the nature of the data passed through the system (because contracts and collaborations are sought after). Integration tests give you feedback about whether the system's external actors are live/reachable/unreachable/unavailable : Isolated tests give you feedback about the level of difficulty involved in switching one existing implementation for another (because interface knowledge per se is accrued and managed)
- System Test Tests that span across features and teams due to the complexity of the product If product and/or feature complexity is very low, then integration tests may be enough to sufficiently cover the test space Equipment Specific Tests Tests that require specific hardware or software that is not available to the development teams If all tools are available to the Scrum Team(s), then this category disappears Competence Specific Tests Tests that require specific competence that is not available in the development teams Examples: Localization, Tracking, Cross promotions, Cross missions, tests that require understanding of multiple King games, etc. If all competences are available to the Scrum Team(s), then this category disappears
- There is a point in placing some testing outside of the Scrum Team System test that spans over multiple features and teams Tests that require equipment not readily available to all teams Tests that require competence not readily available to all teams However these test teams must work in parallel with all the Scrum Teams, and this creates some interesting coordination problems
- Using the Cynefin (Kih-neh-vihn) framework [9] Simple tests can be done by anyone (unless you want to automate it, in which case you need to know how to do that obviously) Sense – Categorize – Respond Simple = easily knowable. Complicated tests are well suited for someone with a good understanding of the system Sense – Analyze – Respond Complicated = not simple, but still knowable. Complex tests are well suited for someone with a good testing skillset and a good understanding of the system Probe – Sense – Respond Complex = not fully knowable, but reasonably predictable. Chaotic tests are … difficult? Act – Sense – Respond Chaotic = neither knowable nor predictable.
- There is a place for someone with a strong testing skillset both in the Scrum Team, and outside of the Scrum Team
- This way of working obviously favors automated checks, but in no way mandates it The Scrum Team should make the decision what they want to automate and what they want to test manually Even if you have automated checks that cover isolated components, communication between components, and how multiple components work together in a group, you will still need to perform manual testing if you have a user interface
- This motivates programmers and testers to design the system properly with testability in mind, and to automate efficiently During all test activities you must always have a risk-based approach [10] You cannot cover the entire test space every time and still be efficient This drives the need to reduce system complexity and design the system with this in mind
- Preferably all manual testing should be actual testing and not checking Checking detailed Acceptance Criteria that are hard to automate could be an exception Designing and prototyping automated checks is a test activity Running automated regression checks is a check activity