This is the story of how I let go of being the lone tester and became the testing teacher and coach for my team. I have always thought of myself as an agile tester. However, over the past couple of years, I have transformed from a pseudo-agile tester to a true agile tester. To improve collaboration with the rest of my product delivery team, I have been gradually increasing the visibility of testing activities through exploratory test charter management, defect backlog organization, and paired exploratory testing with both testers and non-testers. The feedback loops have shortened and the abbreviated time between activities necessitated adjusting how I provide information.
Now that the audience for my testing comprises a mix of disciplines and the work environment has shifted from a heavier process to transparent, quick information access, I have been experimenting with different ways to execute testing and to represent the outcomes of that testing activity so that the information consumers understand it in ways that best suit each of their perspectives.
We will examine distinct agile team member personas and their implications for presenting and maintaining testing information as well as the inherent tensions between their distinct and various needs. I will trace my learning curve of adjusting to their needs through the various experiments I have completed in this context, though these lessons extend beyond a purely cross-functional agile product development team.
Attendees will come away with a fresh perspective on viewing their product team members and focus on the value testing artifacts provide to a software development team as well as answers to these questions
- What value can testing bring to an agile software product team?
- How can we focus on outcomes over output, both as an agile software product team and as tester contributors?
- How do you keep the work sustainable?
- How do we maintain effective communication?
- What does it mean to test in agile environments?
17. Ben the Business Advocate
@aclairefication #BVT #Agile2013
How
quickly can
we provide
value?
ROI Of
Testing
Can We
Ship?
Subject
Matter
Expert
Impact
On The
Bottom
Line?
Average
Computer
User
Ease Of
Learning Severity
21. Bug Board – Site Map
@aclairefication #BVT #Agile2013
22. Bug Board – Site Map
@aclairefication #BVT #Agile2013
23. @aclairefication #BVT #Agile2013
Todd the Technology Advocate
Does
It
Work?
Principle
Doer
How
Bad Is
It?
Translate
To Action
What’s
Next?
Expert
Computer
User
Can We
Sustain
It?
Accuracy
24. Sprint Board – Bug Lane
@aclairefication #BVT #Agile2013
25. Sprint Board – Bugs in Stories
@aclairefication #BVT #Agile2013
27. Special thanks to:
- my UX folks Will & Astrid & Emily
who taught me all about personas
- my employer Oracle for sending me
to Agile 2013
- my friends whose photos I used for
personas (“whatever those are” )
Find Claire Moss online:
Twitter: @aclairefication
claire@aclairefication.com
@aclairefication #BVT #Agile2013
Editor's Notes
Hi, I'm Claire Moss and I'm here to talk to you about Big Visible Testing. For those of you who like to tweet, you can see my Twitter handle @aclairefication here in the header and my talk's hash tag #BVT along with the conference hash tag #Agile2013.I have always thought of myself as an agile tester - hence my talk's riff on the agile Big Visible Charts concept. However, over the course of a year, I have been transforming into a truly agile tester working as part of a Scrum team to build a nearly green field product.
The other members of my product delivery team are from a mix of disciplines but do not have a background in software testing as I know it. This audience for my testing needs transparent, quick information access in a form that best suits their distinct perspectives. It has been a learning curve for all of us and I hope that you benefit from our experience. I hope you will leave this presentation with a fresh perspective on your product team members so that you can focus on the value of the testing artifacts you produce.
One of the Scrum artifacts that we produced for our product team is the Definition of Done. As you can see, we define the results we want for each story with an emphasis on quality, so the testing information needs to support this value. You may notice that the product team isn't interested in the overall picture of the testing process in the way my testing manager is. They want more short-term feedback from testing so that they can do something about it.
My first attempt to educate my teammates about testing was to create a low-tech defect dashboard, or as I like to call it the bug board. I started with defects because my teammates already had some understanding of the relationship between software development and that form of testing feedback. I wanted to meet them where they were and to collaborate on the testing reporting they needed to schedule, design, and implement stories.
Imagine my delight the morning I arrived at work to find my bug board rearranged: it had been the focus of fruitful discussion among the members of my Product Owner team, that is the folks making the decisions about what to do.
As our pilot release progressed, we had deeper discussions about how testing serves the product team and I gained insight into their various disciplines, ways of thinking, and preferred ways to communicate. After we reflected on what we learned, I resolved to apply this to future experiments.
We iterated and I came to see my teammates as quite distinct customers of my work. I began to think of them in the way they think of their customers, applying the lessons they taught me to them(selves?). And so I developed user personas corresponding to what value each of them emphasized. Since these personas are quite detailed, I will only be able to hit the highlights in this presentation but will be glad to discuss more afterward.I began by focusing on the goals and greatest concerns for each of these roles. Since user personas can be proposed ad hoc and evaluated through research, I began with a guess based on my past experience and tested that against my current context.
[READ ABOUT BEN THE BUSINESS ADVOCATE]
For Ben, I chunked bugs by Severity and discussed that with him, aiming for ease of reference.
We revised the representation so that he could efficiently gather information and at a glance know when to follow up with questions to elicit well-rounded information for making business decisions. He wanted to know whether to address defects and when we needed to complete them. When our critical and highest severity buckets were empty, he thinks, "Let's get a return on our investment. Ship it!“Although Ben is the CEO of the product, his is not the only perspective to consider. There is an inherent tension between this business motivation and what drives my second persona Ursula.
[SHOW URSULA]She balances the team by bringing a sharp focus on the users, whom she knows deeply through broad customer contact, whether through usability testing or contextual inquiry. She sees the system through their eyes - and frequently prefers a visual representation of the impact on the user.[READ ABOUT URSULA THE USER ADVOCATE]
For Ursula, I organized bugs around a site map I built for our web application. Ursula is a "doer" since she designs the user experience. In Scrum terms, she is a pig. In UX terms, she is a primary persona: I want to optimize my quality information for her use in problem resolution.Since Ursula is also a judicious perfectionist with a focus on consistency, she and I have a great relationship and collaborate closely. I use her designs and user research to feed my test design and she uses my granular test output in the larger context of the product.
Ursula consumes defects more in clusters than singly, so we revised the bug board accordingly. I can enhance the bug board even more by focusing on efficient translation to her action. Thus describing problems as a scenario emphasizes when and how bugs impact a user's day-to-day life with the software so she can iterate on the user experience, which encompasses more than just graphic design. In turn, I can provide feedback and critique mockups or prototypes, asking helpful questions.
While everyone on the team is conversant with software from our own usage, no one has as deep an understanding as Todd the Technologist - sometimes he has to explain his recursion jokes for the rest of us.[READ ABOUT TODD THE TECHNOLOGIST]As the principle "doer" on the team, he is my highest priority customer. Although Ursula has the chops to get into the user interface code, Todd is the one to keep things running smoothly behind the scenes. Test-driven development and automated checks are his first line of defense again bugs, so he may respond with puzzlement to the presence of a bug backlog. Let's translate it for him.
First, we tried scheduling a time box for bug fixing, our bug bucket, to attack our most urgent concerns since Todd cares about what is the highest priority work so he can attack it. However, Todd is not interested only in an expedient solution. He wants to keep the product sustainable, so he may report issues for the backlog that incorporate technical debt, which may relate to clustering of defects.
Ursula isn't the only one who likes a good user scenario, so we shifted to relating bugs to stories scheduled for sprints. After all, Todd needs to know the whole picture so he can incorporate refactoring and bug fixing in the estimate along with the feature enhancement that together produce value for the user. He also likes preventative measures such as a scripted smoke test that covers a story's acceptance criteria - just another way for him to know that a story is done.
That brings us full circle to Ben who makes the ultimate judgment on whether the story is done when he runs the application against the acceptance criteria. Although Ben may be a secondary persona for me - one I want to accommodate but not as directly as my primaries - he benefits from all the testing information along the way since he has confidence when he accepts a story.Our current sprint board informs Ben about the status of our sprint's story progress, including defect discovery.Now that the whole team understands defects better, we also have begun to incorporate exploratory testing charters in our stories so that everyone can see the progress of testing over time and what charters uncover the defects. I have been able to involve everyone on the team in exploratory testing to some degree and our product's users will reap the benefit of the common understanding. After all, while I may be in the information business as a tester, it is all ultimately in service of our users.
I want to thank Will & Astrid for teaching me all about user personas and the role that UX plays in software development, but I especially want to thank my employer Oracle for sending me to Agile2013 so we can confer today. And thank you my audience for your time.Any questions from those present or anyone following along at home?