Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Experimenting our Way to Team-based Testing (Soft test 2019 final version)
- 1. Experimenting our Way to Team-based Testing Rob Meaney & Conor Fitzgerald @RobMeaney @ConorFi #SoftTest
- 2. Are You A Lone Tester? @ConorFi
- 3. @RobMeaney@RobMeaney Quality is a Battle @ConorFi
- 4. @RobMeaney@RobMeaney@ConorFi Weight On Your Shoulders @ConorFi
- 6. @RobMeaney@RobMeaney Last Line Of Defence @RobMeaney
- 10. @RobMeaney Whole Team Quality Focus @RobMeaney
- 13. Poppulo - Great Place to Work @ConorFi
- 15. Open to New Ideas @ConorFi
- 16. Coaching - Big Bang @ConorFi
- 17. 3 Years Of Experiments @ConorFi
- 20. @RobMeaney@RobMeaney We Lost 3 Testers @RobMeaney
- 27. Our Approach: 1. Visualise the Work 2. Socialise Problems 3. Establish a Quality Culture @RobMeaney
- 29. Visualising the Work: 1. Quality Indicator Board 2. Department SoS Board 3. Team Kanban Board @RobMeaney
- 41. @ConorFiThe Impact of Kanban @ConorFi
- 45. 1. Visualise the Workflow @ConorFi
- 46. 2. Limit Work In Progress (WIP) @ConorFi
- 48. @ConorFi “Trust [the] quality of what you know, not [the] quantity.” Miyagi Kanban - Quality Focus @ConorFi
- 49. @ConorFiBreaking things to smaller parts @ConorFi @ConorFi
- 51. Lone Tester & High WIP @ConorFi
- 52. Team Based Testing & Reduced WIP @ConorFi
- 54. Our Approach 1. Daily Stand-ups 2. Learning Reviews 3. Department Retros 4. Team Retros @ConorFi
- 60. @RobMeaney Discuss In Small Groups @RobMeaney
- 61. @RobMeaney@RobMeaney Present, Vote & Act @RobMeaney
- 62. @ConorFiRetro - Team Level @ConorFi@ConorFi @ConorFi
- 64. @ConorFiEveryone has a Voice @ConorFi
- 66. @ConorFi @ConorFi@ConorFi@ConorFi 3. Establishing a Quality Culture @ConorFi
- 67. Our Approach 1. Exploratory Testing 2. Pairing & Mobbing 3. Bug Bashes 4. Fresh Perspectives 5. Mentoring & Coaching @ConorFi
- 69. @ConorFi Not just Checking, Learning @ConorFi
- 76. @ConorFiFind Big Problems Sooner @ConorFi @ConorFi
- 90. @RobMeaney@RobMeaney@RobMeaney Sharing & Discussing Models @RobMeaney
- 92. Our Approach 1. Visualise the Work 2. Socialise Problems 3. Establishing a Quality Culture @ConorFi
- 94. At Team Level: ● Quality Indicator Boards ● Kanban Board ● Retros & Learning Reviews ● Pairing/Mobbing ● Bug Bashes ● Revisit Coaching @ConorFi
- 95. At Department Level: 1. Be The Example 2. Pain Is A Teacher 3. Make Problems Visible 4. Give People a Voice @RobMeaney
Editor's Notes
- Are you a lone tester?
- Does every quality discussion feel like a battle
- Do you feel like the weight of the world is on your shoulders?
- Are you overwhelmed with the burden of testing?
- We’ve all worked on teams as the last line of defense, furiously trying to figure out all the weird and wacky ways the software can break.
- Scrambling to try to get adequate test coverage before the impending release, hacking together automation, logging bugs, retesting bugs
- Before you the tester alone has to sign off on the release as “Assured the Quality”??
- I’d seen enough in my career to suggest that there was a different way a better way.
- where quality was a whole team responsibility
- I dared to dream of a new world where testers and developers worked together towards a common goal
- Having worked together in the past we both Conor & I decided we wanted to break the cycle of the "lone tester"
- Poppulo focuses on employee engagement. Rob and I both joined poppulo 3 years ago, having worked together in the past. It felt that Poppulo could provide the platform for us to try to break the cycle of the lone tester and focus on team based testing. Fantastic Workplace with highly engaged colleagues, passionate about their work Culture of innovation Regularly features in the great place to work awards
- Fast Growing Company using leading tech
- BUT Most Importantly it felt that everyone seemed to be open to new ideas, change and expermation
- We initially went for a Big Bang approach and our initial plan was to implement a Coaching Model. Whereby every tester would become a quality coach. We quickly learnt that sometimes Smaller is better
- Instead we began to try short experiments .. So 3 years later and lots of experiments, we want to share our story Successes, failures, and if time permits plans for the future. We’ve all worked on teams as the last line of defense, furiously trying to figure out all the weird and wacky ways the software can break. Scrambling to try to get adequate test coverage before the impending release, hacking together automation, logging bugs, retesting bugs, fixing the automation before signing off on the release because you’ve “Assured the Quality” of the product, you’ve squeezed every last bit of testing juice out of your brain, you’ve done all you can do with the limited time available but by the time i’d joined Poppulo I’d seen enough elsewhere to suggest that there was a different way, a better way. I was lured to Poppulo by the open minded learning culture and opportunity to work with great people. So over the last 3 years I’ve taken the opportunity to explore better ways of testing software that has resulted in dozens of experiments, successes, failures and lots of learning.
- We faced some Challenges
- Rob faced challenges at a strategic department level working within the Engineering management team How can we change mindsets and influence teams to focus on quality? Conor faced challenges working across two delivery teams including one team that were building a entirely new mobile product from scratch in the Cloud!!! PLUS
- Went from 8 to 5 testers
- Inconsistent process across teams Inconsistent tech stack Inconsistent testing approach
- Teams were moving from Monolith to Microservice so there were lots of new challenges & risks - deal with the worst of both worlds
- Testers spending lots of time writing, maintaining & debugging automation Testers only ones that cared about acceptance queue & automation failures
- So the testers weren’t analysing risk with the team to uncover issues early and mitigate them together as a team
- Because of the overwhelming workload around test automation and the acceptance queue testers had almost no time to explore the product and some testers had very little exploratory testing experience.
- Biggest challenge! - Quality Police Testers owning the Acceptance Queues Signing off on releases
- Over the last 3 years we’ve seen a pattern emerge that has worked for us Often all 3 of these can overlap
- Make all the work done in the team visible - often testing tasks aren’t tracked - eg. Investigating Automation failures etc. Also lets us see bottlenecks and things that impede our work this allows the whole team to see what's really going on
- I’ll run through 3 examples of how we’ve visualised the work
- I wanted the teams as a whole(not just the testers) to become more conscious of the reliability of their feedback loops To understand some of the impediments to moving to continuous delivery But I wanted to do it in a lean, low cost way with minimum impact to the teams
- Every morning wheeled around my board to each team & asked 4 simple questions Any alerts in last 24 hours Any build deploy issues any e2e test failures Any rework
- Huge benefit of not automating it! Started to have really interesting conversations and gaining an insight into the common problems that the teams were experiencing as well as problems that were specific to individual teams. Allowed me to gain a deeper understanding of each team
- You can change a culture by changing people habits By walking to the teams everyday they developed a habit of checking the indicators every morning before I came around, a habit which even became part of stand up for some of the teams
- After only a couple days teams began to look at other teams quality indicators and compare to theirs, some teams didn’t like what they were seeing
- So the teams started to make small improvements and changes without any prompting Lesson: When you make problems visible people seem to act to so long standing problems like, noisey alerts, flaky e-2-e tests were fixed :)
- All team members started to get more engaged & interested in the feedback loops. It all culminated one day during our Team leads metrcis meeting, with 15 mins left the team leads actively asked for the quality indicator board unprompted so that they could discuss the findings as a team :)
- So we have a department meeting everyday, all 65 people in the engineering department attend and we discuss our high-level goals together - progress, impediments etc.
- Changed our SoS board so that all customer impacting issues HAVE to be placed on the SosS board First question every morning was if there were any new customer impacting issues & updates on existing ones Highlights that customer impacting issues are more important than standard work!!!!
- We added an incident reviews lane at the very top so that every customer impacting incident needed to have a learning review with associated actions
- Actions from Reviews were visualised and added to the board so that learning & improvement is part of how we work Example security training and monthly security bug bash
- At a team level we had quality indicator boards and each day gave our updates at the scrum of scrums We also use kanban and I believe it complemented the use of quality boards and scrum of scrums We started kanban by having a book club, read the book kanban in action and began to use it. As we reading a chapter in each week, we were learning from our kanban board.
- Kan meaning visual and ban board Literally sign board /billboard Kanban Visualises both the process(the workflow) and the actual work passing through it.
- Kanban is essentially a Process Improvement Tool Which means it can be used to improve scrum Can be used to improved your scrum process
- 1. Kanban board Radiates information Makes hidden work apparent
- 2. Number of items you have going on High WIP vs Low WIP Value of reduced context switching and decomposing work in to small tasks
- 3. Monitor, measure and report All about eliminating bottlenecks Highlights the “Testing Bottleneck” (Theory of Constraints)
- We learnt that when toyota was using kanban They focused on quality not quantity
- Smaller tasks/ Reduced WIP Reduced Complexity Easier to Test Reduced Risk and Analysis Easier to Convince other to test :) Less intimidating to test
- In poppulo in the first team i had fallen in to a gatekeeper role Kanban helped me move from being a gatekeeper to focsu on more team based testing Where we began to colloboate more and more
- If we look at the way our kanban board was initially. The developers focused on developing and reviewing, and i was tested their outputs. We had too much work in progress and little was getting done
- We made to simple changes We reduced our work and began to work together to get things done. Developers would take items to test from the acceptance queue I would review unit tests with developers It began to lead the groundwork for team based testing
- We had 3 approachs now for visualising work and most importantly quality problems. We now needed to solve those problems with others What we call this socialising the problems, rather than solving the problme alone, solve it with others -- Actions from Reviews were viualised and added to the board so that improvement visualised and part of the way we work Example security training and monthly security bug bash
- The daily stand provides an opportunity to share problems with others Highlight problems from the quality board Make Changes to the kanban board Raise issues at the scrum of scrums
- Distill as much learning as possible from each customer impacting issue
- Everyone involved in an incident can safely talk about their experience at each stage of the incident Reinforce good behaviours Identify areas for improvement
- Are a regular meeting that allows a Team to reflect and create a plan for improvement
- Every Quarter have a Department Retro opportunity for everyone to bubble up issues Facilitated it for 65 people
- @RobMeaney Split in small, mixed groups - EVERYONE Gets a voice Identified, discussed & voted for their top Liked/Lacked/Learned
- Spokesperson for the team presents their top choices And Discuss as a department Whole department dot vote to identify priority Engineering management team focuses on the top liked/lacked/learned
- Retrospectives provide a great opportunity to learn and grow as a team. We have learnt 3 important lessons on running retros Fun Retos Blameless Have a voice
- Focus on blameless retros both for those in and outside the team Blameless Have a voice
- Like rob spoke about earlier, use post-its to ensure every has a change to get their ideas discussed Blameless Have a voice
- We use funrerospectives.com to try different formats for each retro. Blameless Have a voice
- Once we have visuaised the problems and socialised them. Then it was time to figure out how to establish a culture of quality
- We wanted to introduce more ET, as rob alluded to earlier ET was not part of the culture
- We want to take the focus away from just checking and more to exploring, learning about the software. Asking open ended questions
- Training sessions didn’t seem to be the way forward, so we focused more on bring people together
- But by bringing people together
- P
- On of the teams we had issues around the quality of the Apis. BY pairing on the testing togther we created quality APIS Questions to ask .. when Pair testing with Dev What problem are you solving What is your solution What testing have you performed Pause . . Rubber Duck Effect :) Then Suggest other Tests
- Mobbing can be Daunting As a tester it can be hard to be writing code with really telented developers But when it comes to testing, you can see they find it difficult, so there is empathy Safe Environment is important And it all Ties back to Kanban Pairing and Kanban reduce WIP
- Definition taken from here: https://usersnap.com/blog/bug-bash-guide/ Putting People at the heart of Quality A bug bash is a company or team-wide event held to discover a large number of bugs within a short time frame. Prior to Release or Tight Testing Timelines Help spread some of the exploratory testing concepts
- Efficient at big problems sooner
- Not just bugs but also the feedback around the user experince Sales and support begin to spread the word about new features
- A whole company can provide so much diversity that a lone tester or evena team in both terms of cultural and deiversity of discpline
- Make them fun, bring food, give a prize for the best bug
- A key part of changing our culture was introducing fresh ideas & perspectives into the Org
- As we were at the start of building out our new mobile platform Talked about mobile testing and specifically designing and testing for performance
- In one of our learning reviews we identified that we had a lack of security testing knowledge in house so I set 2 a 2 day hands-on training session with Dan Billing where we had people from across the organisation learn about and do security testing of our App. I set up a monthly Security bug bash too.
- We brought in Gwen to talk about her experiences in Monzo(Engineering Manager) and run an exploratory testing workshop with participants from UI/UX, Dev and Test
- So we do weekly show and tells and we took this as an opportunity to talk about great testing & quality work In this particular picture we have one of our most senior Development Engineers talking through some really great performance testing
- Learning from community - going to conferences - networking & learning Least diverse team ever, but we’ve changed that since :)
- So I have used coaching when working with teams to influence quality using workshops and modeling, it’s a hugely valuable tool in your toolkit
- Run a workshop with teams called the “Team Testing Experience” to talk through the 10 factors that influence Testability so that we can reflect & improve continuously
- I work with teams to Identify various forms of Risk and how best to mitigate them
- I do workshops focusing on the architectural design where I ask questions about Risk Testability Observability Operability
- Having conversations with the whole team, share models and discuss appropriate approaches in their context
- Key things that Conor thinked worked .. Visualise the problem Kanban Board Quality Board Creating a quality culture Pairing and Mobbing Bug Bash I would like to Revisit Coaching
- Rob - Show people problems don't give them solutions Get people having good conversations Pain is a great teacher, use incidents to drive change Be the example rather than tell people Give people the opportunity to talk about problems and solve them together There's lots of tools & techniques out there but the most important thing is safe, open, honest conversation When you make problems visible people seem to act
- Key things that Conor thinked worked .. Visualise the problem Kanban Board Quality Board Creating a quality culture Pairing and Mobbing Bug Bash I would like to Revisit Coaching
- Think big, start small, learn fast
- Think about Quality in a whole team holistic sense
- Start with small, Lean safe to fail experiments
- Reflect on those small experiments together and iterate on your learnings :)