Downloaded 27 times
More Related Content
Similar to The Tester's Role in Agile Planning
![[QE 2018] Paul Gerrard – Automating Assurance: Tools, Collaboration and DevOps](https://cdn.slidesharecdn.com/ss_thumbnails/automatingassurancegerrard-180629130855-thumbnail.jpg?width=640&height=640&fit=bounds)
The Tester's Role in Agile Planning
- 1. Transitioning to AgileTesting Lessons from the Trenches Bob Galen President & Principal Consultant RGCG, LLC bob@rgalen.com
- 2. Introduction Bob Galen Somewhere ‗north‘of 30 years experience Various lifecycles – Waterfall variants, RUP, Agile, Chaos… Various domains – SaaS, Medical, Financial Services, Computer & Storage Systems, eCommerce, and Telecommunications Developer first, then Project Management / Leadership, then Testing Leveraged ‗pieces‘ of Scrum in late 90‘s; before ‗agile‘ was ‗Agile‘ Agility @ Lucent in 2000 – 2001 using Extreme Programming Formally using Scrum since 2000 Currently an independent Agile Coach (CSC – Certified Scrum Coach, one of 50 world-wide; 20+ in North America) at RGCG, LLC and Director of Agile Solutions at Zenergy Technologies From Cary, North Carolina Connect w/ me via LinkedIn and Twitter if you wish… Bias Disclaimer: Agile is THE BEST Methodology for Software Development… However, NOT a Silver Bullet! Copyright © 2013 RGCG, LLC 3
- 3. Outline – Myths& Realities Track Talk Introduction 1. Transforming your Team 2. Automation 3. Developers & Automation 4. Developers Testing 5. Test Planning & Scripts 6. Testing within the Sprint 7. Exploratory Testing 8. Role of Testers Copyright © 2013 RGCG, LLC 9. Developer to Tester Workflow 10. Managing Agile Testers 11. Test Metrics 12. Retrospectives – The Secret Sauce 13. Continuous Improvement 14. The Customer 15. Agile Requirements – The Product Backlog 4
- 4. #1, Transforming yourteam Myth: You need all programmers or highly technical testers when you move to agile Reality: A mix is best – Manual, domain-centric and technical skills Some programming / scripting skills Soft / collaborative skills Reality: And throw out all of that Developer-to-Tester ratio ‗stuff‘. Copyright © 2013 RGCG, LLC 5
- 5. #2, Automation Myth: Youneed 100% automation to start agile testing. Reality: You simply need to have a strategy AND doggedly pursue automation where it makes sense Make it part of the Backlog and work it every sprint Reality: There are some excellent Open Source tools that supplement agile automation development Reality: The Agile Test Automation Pyramid is the right overall strategy Copyright © 2013 RGCG, LLC 6
- 6. Agile Test AutomationPyramid Mike Cohn; Lisa Crispin & Janet Gregory http://behaviordrivendevelopment.wikispaces.com/Testing Copyright © 2013 RGCG, LLC 7
- 7. #3, Developers &Automation Myth: QA designs, writes & runs all of the test automation Reality: Everyone should be responsible for automation Developers need to minimally attend to Unit Level Participate in any framework or re-use development Writing ‗glue‘ code – fixtures, step files, etc. Reality: It also extends into your Build & Continuous Integration systems All automation should be ‗wired‘ into CI Dashboards, trending, lava lamps, etc. for all to see… Copyright © 2013 RGCG, LLC 8
- 8. #4, Developers Testing Myth:Developers can‘t test their own code—they‘re not independent enough nor skilled enough to do it properly. Reality: We need to stop stereotyping team members, their strengths and their abilities. Developers can absolutely test their own code. Some are better at it than others Pair with them to help test appropriately Copyright © 2013 RGCG, LLC 9
- 9. #5, Test Planning& Scripts Myth: You don‘t need to plan and you don‘t need functional test cases (automation takes care of everything…) Reality: Plans help the team focus on the risk-based testing required within an iteration AND across a release Reality: Scripts (test cases) help formalize and drive your testing; (it just happens…) Absolutely required in regulatory environments Reality: You‘ll never actually automate every test Copyright © 2013 RGCG, LLC 10
- 10. #6, Testing withinthe Sprint Myth: You simply need to run 100% of the tests within the constraints of the Sprint…that‘s ―Agile‖ Reality: Rarely possible in most contexts. You first need a high-degree of automation and business support (for example: equipment costs) Very mature test automation and CI / CD environments Reality: Most agile teams adopt some sort of risk-based testing approach for within the sprints Dealing with Technical Test Debt Then leverage Hardening / Stabilization pre-release sprints Copyright © 2013 RGCG, LLC 11
- 11. The Agile ReleaseTrain Example: eCommerce / SaaS Model 10 days Team 1 10 days Iterate Iterate 5 + 2 days External Release Harden Continuous Integration Team 2 Iterate Iterate Harden X-team Harden Rinse Docs, Training Repeat Continuous Integration Team 3 Iterate Iterate Harden & Continuous Integration … Team 8 Team 4 Iterate Iterate Harden Continuous Integration Environment Evolution Dev + QA Copyright © 2013 RGCG, LLC Dev + QA QA -> Staging Production 12
- 12. The Agile ReleaseTrain Example: iContact / SaaS Model 3 weeks / 15 days SBET, Exploratory – Regression Testing 4-5 days Production Release Team 1 Iterate Harden Continuous Integration Team 2 Iterate Harden X-team Harden Harden Docs, Training Continuous Integration Team 3 Iterate Rinse & Repeat Continuous Integration …Team 10 Team 4 Iterate Harden Continuous Integration Environment Evolution Copyright © 2013 RGCG, LLC Dev + QA QA -> Staging Production 13
- 13. #7, Exploratory Testing Myth:There is no place for Session Based Exploratory Testing in agile contexts. Reality: ET and SBET are a beautiful complement to agile testing. Helping nurture pairing & collaboration across teams and functions Defining new (more valuable) test cases Quickly gaining quality & usability feedback Copyright © 2013 RGCG, LLC 14
- 14. #8, Role ofTesters Myth: That the testers alone own quality & testing practices within each team and sprint Reality: The testers foster a ―Whole Team‖ view towards quality—focusing less on ―Testing‖ and more on ―Quality Practices & the Customer‖ Serving as guides for the team; Testing the ―hard bits‖ Facilitating exploratory testing sessions—finding more interesting / valuable tests Working with the Product Owners—are we solving the customers problems? Copyright © 2013 RGCG, LLC 15
- 15. #9, Developer toTester Workflow Myth: There is always a hand-off from developers to testers; usually quite late in the sprint. That‘s simply the ―way of things‖ in software development. Reality: Scrummer-fall is alive and well…but, Wrong! Teams need to swarm on their work, as flow & throughput matter the most. WIP limits and close proximity / collaboration help establish a healthy tempo of developer & tester pairing Micro-handoffs – testing as development progresses! Do you log bugs? Or do you fix bugs? Copyright © 2013 RGCG, LLC 16
- 16. #10, Managing AgileTesters Myth: The functional test manager is in charge of deciding how, who, when , etc. for the test team. Reality: You still absolutely need functional leadership within agile teams; However, it‘s focused towards quality practices, strategy & coaching, and handling impediments / escalations Encouraging transparency, transforming metrics & reporting Supporting & protecting the teams Encouraging risk-taking, innovation & creativity (Slack Time) Copyright © 2013 RGCG, LLC 17
- 17. Levels of Done-NessCriteria Activity Criteria Basic Team Work Products Done’ness criteria User Story or Theme Level Acceptance Tests Sprint or Iteration Level Done’ness criteria Release Level Release criteria Copyright © 2013 RGCG, LLC Example Pairing or pair inspections of code prior to check-in; or development, execution and passing of unit tests. Development of FitNesse based acceptance tests with the customer AND their successful execution and passing. Developed toward individual stories and/or themes for sets of stories. Defining a Sprint Goal that clarifies the feature development and all external dependencies associcated with a sprint. Defining a broad set of conditions (artifacts, testing activities or coverage levels, results/metrics, collaboration with other groups, meeting compliance levels, etc.) that IF MET would mean the release could occur. 18
- 18. #11, Test Metrics Myth:You can and should move forward reporting everything exactly as you have before. Including any ‗dysfunctional‘ metrics that your process and/or PMO dictates. Reality: The metrics should change immediately. From QA and Test centric towards Team-Centric metrics (Value, Throughput, Quality, Team) Stop reporting out on ―Testing‖; it‘s irrelevant! This effects planning as well—estimation, progress, risk, etc. Contribute quality-centric Information radiators to the mix Copyright © 2013 RGCG, LLC 19
- 19. #12, Retrospectives: The SecretSauce Myth: Testers are ―Second Class‖ citizens who don‘t play an active part in the project & team Reality: There are many places to ―make a difference‖ Getting the 800 lb. Gorillas out on the table; Showing courage; telling truth Fostering continuous improvement within the team Setting the example; showing vulnerability—admitting you‘re wrong Team listening; active planning; dependencies; pairing Risk-taking; Failure! Copyright © 2013 RGCG, LLC 20
- 20. #13, Continuous Improvement Myth:We‘re generally ‗stuck‘ in our approaches so just accept them and do the ―best you can‖. Reality: Continuous improvement is everyone‘s responsibility—to engage, suggest, take ownership of current results, explore root causes, etc. Active participation in your teams Retrospectives is a key way to guide quality, testing, and customer-centric improvements. Courage! Copyright © 2013 RGCG, LLC 21
- 21. #14, The Customer Myth:Business Analysts capture customer requirements and testers test them for completeness. Reality: You need to begin to partner with the Customer – Stakeholders – Product Owners to produce software that solves the their problems. Move to the ―front‖ and help define & refine User Stories with your Product Owner Actively participate in Sprint Reviews Show value for automation; placing test investments in the Backlog Copyright © 2013 RGCG, LLC 22
- 22. #15, Agile Requirements– The Product Backlog Myth: We can‘t start testing until the requirements are finished or stable; no matter how ‗agile‘ we are. Reality: Hogwash! Get over it… Ambiguity and incompleteness need to become your friend and ally. As does working with your Product Owners and Customers to help define the requirements Realizing that the requirements (User Stories) are only complete at the end of each sprint. Copyright © 2013 RGCG, LLC 23
- 23. Wrapping up… Agile isthe best thing that’s happened to testers since… The Great Depression Whole Team view Testing, Metrics, Automation Planning, Reporting, Quality Facilitate feedback Multi-tiered automation Just-in-Time, risk-based testing Continuous improvement Trust the Team Retrospective Copyright © 2013 RGCG, LLC 24
- 24. Contact Info Bob Galen PrincipalConsultant, RGalen Consulting Group, L.L.C. Director of Agile Solutions, Zenergy Technologies, Blogs Project Times http://www.projecttimes.com/robert-galen/ Business Analyst – BA Times http://www.batimes.com/robert-galen/ My Podcast on all things ‗agile‘ http://www.meta-cast.com/ Experience-driven agile focused training, coaching & consulting Contact: (919) 272-0719 bob@rgalen.com bob.galen@zenergytechnologies.com www.rgalen.com Copyright © 2013 RGCG, LLC 25
Editor's Notes
- #2 The first ‘run’ of this at StarWest 2011 went quite well. There were lots of questions around how automation fit into this model—in fact all types of testing. So it might be a useful extension for future versions?The other thing that came out were questions around setting up charters/sessions – so how to “begin”. In a workshop variant of this I’d like to decompose a real app and do some iterative, simulated session execution.And also what the ‘”gap” between sessions “looks like” – so, debrief, note reviews, bug triage, re-planning charters, setting up the next session strategies, etc. That would be another focus point.
- #3 The first ‘run’ of this at StarWest 2011 went quite well. There were lots of questions around how automation fit into this model—in fact all types of testing. So it might be a useful extension for future versions?The other thing that came out were questions around setting up charters/sessions – so how to “begin”. In a workshop variant of this I’d like to decompose a real app and do some iterative, simulated session execution.And also what the ‘”gap” between sessions “looks like” – so, debrief, note reviews, bug triage, re-planning charters, setting up the next session strategies, etc. That would be another focus point.
- #6 Bob: I get asked all the time about ratio’s. They no longer matter. If you have zero testers, you still need to test. But now you don’t have any expertise for that role.MT: (Challenge: you should set a baseline of expectations, the best scenario I have seen work is 1 manual and 1 automator per 3-4 developers). Only once you get a ton of you automation completed can someone play both roles. Currently fighting the old stereo type for waterfall 80/20 rule for testers on a team…does NOT work. End the end it’s is the whole team responsible for the completion of the story/sprint so if all your testers are on vacation devs have to pick up the tasks.Bob: Might want to tell the Sharda story here? Solid manual tester who made a huge difference.Bob: Might want to explore whether “soft skills” are more important that “technical skills”. That being said, I would want some SDET-like folks.MT: Agree, Saradha turned into a bad egg and that will kill a SCRUM team.
- #7 Bob: You hear this a lot, always from the agile pundits, purists, and most coaches. Many of these folks have only done “agile coaching” for too many years OR they’re working in small start-up teams or greenfield projects.Bob: Talk about first miss start. The reality is that it takes TIME to build up automation in a legacy based system and you need a consistent strategy. Might share the story of iContact where we had 2 miss-starts on automation before we really clicked on Mary: Talk about second restart. Dedicated focus and solid skill of Mike was a huge help! Did spoil the automation team, only wanted to build frameworks after.Bob: And making the “Business Care” is always a challenge. I like the Salesforce example of an organization-wide commitment to coverage!Mary: iContact story on all the “promotion” I had to do to make people care. But it worked.Bob/Mary: Explore the anti-pattern of focusing on ATDD – BDD as a test automation activity first.
- #8 QA Manager should partner with the Dev Manager to explore unit testing standards and make sure everyonen is on the same page
- #9 Bob: One of the challenges here is skill. In many organizations I’ve seen, there isn’t a strong level of experience in writing solid unit tests. Or there are reactions (negative) to retro-fitting legacy systems with unit tests.Mary: Total believer of writing the right test for the right reason. Not to have 100% code coverage. QA can code review unit tests, Dev’s can code review integration/regression tests. So training and mentoring is a strong part of your improvement play.TRUST also comes into play. I usually ask “testers” if they trust developer-based tests & testing (whether manual or automation). Nearly 90% of the respondents (not a scientific poll ;-) say no…and will create replicated tests to cover the same things.Might also mention the connect this has to Done-Ness criteria; for automation incremental completeness
- #10 In the shorter “versions”, this slide is “hidden”. In the ½ day workshop, we’ll cover it…Mary: agreed, comments from slide 3A could go here as well.Bob: Mary, be prepared to comment on this one as an “extension” to Slide #3.
- #11 Mary: Discuss Test Ideas and Mind Maps from iContact. Here the story of our journey at iContact would be helpful. Explaining our use of a “Test Plan” for each Sprint to drive team-based conversations around how they’ll attack testing (quality) within each sprint for the work they’ve signed up to do. Notice I didn’t say for each story…but for the “body of work” in support of their Sprint Goal.And if you can’t (doesn’t make sense to) automate everything first, you do need a record of test cases. But “agilify” them. What do I mean by that? Perhaps put that question out to attendees…
- #15 In the shorter “versions”, this slide is “hidden”. In the ½ day workshop, we’ll cover it…Bob: I’m not sure if Mary saw it this way… But I’ve always felt that ET was a key to our whole-team attitudes towards quality and testing. Yes, we had to coach it into people by setting the standard. But, getting folks to pair / collaborate and test together was key. The dynamic pairing, the inclusion of other functions (customer support), using it on release nights as our “regression mechanism”, and having the whole team take responsibility for product quality.
- #16 Bob: I always remember a discussion that Lisa Crispin and I had at a conference. She was away from her team, and if you know Lisa’s context, she’s a tester on a very small XP/Agile team. My questions was – what happens while she’s “away”. And her answer was around her whole team tester and was responsible for quality. The burden didn’t just fall to her. So she felt “comfortable” traveling, do what she could to test remotely, but that her team understood their role.The other point here is that Testing doesn’t mean Quality. Let’s broaden that view in this discussion—particularly speaking to “Finding the Customer”.Mary: Talk about the QA manager and Director at iContact leading/championing this, even to the point of salary equity. Might want to share how our testers at iContact take a very vital and interactive role in our sprint reviews. Not only at a feature level…but in making our testing & quality efforts visible.
- #17 Bob:This is the place where I want to speak to HEALTHY, day-by-day dynamics between testers and developers within highly productive agile teams. Topics: Mary: Pairing, Code reviews, Paired demo’s with the POMary:Micro-handoffsMary:Bug reporting ;-) Report OR Fix? DO NOT WRITE UP EVERY BUGBob:Power of WIP, Power of Collaboration – sitting togetherBob:New hires…learning curve
- #18 Bob:First point of emphasis is the balancing act of supporting Self-Directed teams. So, introduce the notion of self-direction.Bob:Perhaps tell the Teradata story where the Managers went quiet as “Chickens”Mary:Emphasize the importance of Done-Ness in guiding agile teams; also the notion of Guard-Rails. When I got to iContact the QA team could not recite what the Done-ness criteria was.Mary: Servant leadership….Information Radiators, Transparency, and Agile-centric metric would be good to discuss. From QA and Test centric towards Team-Centric metrics (Value, Throughput, Quality, Team)
- #19 These are all “exit criteria”.What might be interesting here is to introduce the discussion around “Readiness Criteria” – particularly for User Stories. What would be some of the aspects of that?And what behaviors or changes in sprint execution would it influence?
- #20 In the shorter “versions”, this slide is “hidden”. In the ½ day workshop, we’ll cover it…Mary: It would be interesting if Mary could tell a story around our KPI attempts at iContact. Some of the things we tried and the variations. Also, recounting our struggles to get the metrics, no matter what we measured, to drive action and improvement actions across our teams…Retrospect more than 2 Sev 1’s per release.
- #21 Bob: Perhaps share my ChannelAdvisor retrospective tale. Not only talking about trivial changes vs. impactful changes…but it leads into:CourageTeam trustWIP and co-locationCollaborationWhole teamMary: Quarterly QA only retrospective.Bob: Then perhaps discuss my blog post regarding failure…chat with the iContact Scrum Masters regarding not failing enough ;-)
- #22 In the shorter “versions”, this slide is “hidden”. In the ½ day workshop, we’ll cover it…Bob: Lots of agile teams “go flat” over time. Under the banner of quality, I usually ask testers to influence the continual improvement practices within their teams. Not that it’s solely their job, but I’d like them to consider the opportunity to “lead” in this area.The stage for this is a couple-fold:How the team is executing and delivering valueHow the team is increasing productivityHow the team is holding to their quality agreements (Done-ness, testing as much as possible, no escapes, etc.)How the team is challenging themselves in the retrospectives… Mary: I don’t have any direct stories to tell here. I wonder if Mary does?
- #23 Mary: A place to start here is the “partnership” I expect between testers and their Product Owners. Helping to make the Backlogs clearer…more testable. The KEY of Acceptance Tests in providing clarity.Mary: Talking about BDD again if we have not already done so much.Bob: Clarity leading towards better solutions, better designs, simpler solutionsTesters “taking the stage” as part of Sprint Reviews; showing the product, but also showing “Quality” in action!
- #24 In the shorter “versions”, this slide is “hidden”. In the ½ day workshop, we’ll cover it…Mary: Talk about Leadership meetings that I hold with Dev/QA Managers and SM/POBob: We should look at this in other contexts with attendees, for example, in regulated environments where requirements need to be verified/traced…what do you do? I think the key thing is to not look at requirements completeness as an “entry” vehicle any longer. They are now complete upon exit of working code from an iteration (or better yet) in the customers hands via a release. AND the customer determines completeness by “usage”.
- #25 Bob: Let’s try to gather some feedback here. What did we do well as a group? What could we get better at? And what did we miss entirely?