Session Based Testing Made Fun
•Download as PPTX, PDF•
0 likes•66 views
Less than a year ago, we introduced time-boxed (2-hour) session based tests at ESO Solutions to reduce the number of defects escaping into production, to cross-train the quality assurance engineers as well as provide our support team (frontline support, sustaining engineering) a way to test new features prior to rollout to production. The format and execution of these sessions have evolved over time based on feedback from participants. They have become tremendously popular in the organization. This talk will go into details about the goals, motivation of these events - called bug bashes -, explain the lessons learned and both the expected and unexpected benefits. The presentation will describe how the bug bashes were setup, the specific roles, the rules that the participants followed as well as how the event was gamified. In addition to the positive impact on product quality, the talk will go into detail on non-technical gains as well such as bringing down organizational silos and creating an atmosphere that software test and development engineers and non-engineering staff enjoy and look forward to.
Recommended
More Related Content
Similar to Session Based Testing Made Fun
Similar to Session Based Testing Made Fun (20)
More from Bertold Kolics
More from Bertold Kolics (9)
Recently uploaded
Recently uploaded (20)
Session Based Testing Made Fun
- 1. A P R E S E N T A T I O N B Y E S O Session-Based Testing Made Fun A P R E S E N T A T I O N B Y E S O
- 2. T H E E S O M I S S I O N Make a Difference Improve Community Health and Safety Through the Power of Data
- 3. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, ESO Engineering – A Matrix Organization • SaaS software suite with multiple software modules • Each Agile team owns a module • Team consists of developers, testers, business analysts • Developers report to development manager (typically scrum master) • Test engineers report to Director of QA • Challenges & opportunities: • Silos especially for test engineers working in matrix • Silos in a quickly growing organization • More and more millennials
- 4. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Session-based Tests • The definition • Exploratory testing • Time-boxed • Follows a mission
- 5. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Session-based Tests at ESO • Goals • Improve quality in production • Cross-pollinate product knowledge inside and outside QA team • Introduce new features to field prior to rollout to production • Build camaraderie inside organization • Have fun
- 6. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Application • Web-based app • Accessible from desktop and mobile devices • Large browser support matrix • Hard to automate areas
- 7. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, The Implementation • Invitation • Consider day of the week – mid- week works better • SELL IT to the organization • Time-boxed & catered • 2 hours – running from 11:30AM to 1:30PM • Never underestimate the power of food • And the cost compared to the benefits
- 8. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, The Implementation – cont’d • Venue • Not at your desk – we want quality face to face time • Large conference room • Communication • Slack • Dedicated channel for all participants • Quick information sharing • Conference call – helps remote employees / offices to feel better involved • Roles • Triage team: QA engineer of tested product, development manager, business analyst • Testers: QA engineers, developers, front-line support, sustaining, professional services, product managers, business analysts
- 9. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, The Gamification • Prize! • Triage team picks winner • Winner gets • The bug bash gavel • Gift card or book • Public recognition on #eso-kudos
- 10. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Timeline - before • Announcement (>1 week before event) • Invitation • In-person as well as e-mail • Invite participants to join Slack channel. • Articulate the need and the general scope • Goal is to maximize participation • Mission Document • Detailed scope and goals • Include information about • Test environment covering SUT, clients / devices to use • Credentials • User personas • Reporting issues
- 11. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Timeline - before • Setup environment • Setup credentials / users • Avoid participants to step on each other • Setup devices if needed • Consider splitting up testing for smaller missions • Post mission, templates, README-s to dedicated Slack channel • Order food!
- 12. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Timeline – During • Start on time • Development manager or business analyst to review the mission • Questions • Sent to Slack channel, or asked in the room • Triage team responds • Confirmed issues entered into issue tracking system • Eat the pizza
- 13. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Timeline – During • Start on time • Development manager or business analyst to review the mission • Questions • Sent to Slack channel, or asked in the room • Triage team responds • Confirmed issues entered into issue tracking system • Eat the pizza
- 14. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Timeline – After • Triage team reviews filed defects • Update them if needed • Collect more information if necessary • Done within 1-2 days • List of defects is captured / archived • # / severity are good indicators of product maturity • Triage team determines winner • Winner is announced to the entire company
- 15. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Timeline – After • Retrospective • With triage team & development team first and foremost • With other participants • Goal is continuous improvement • Determine if repeat is necessary in a few weeks
- 16. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Lessons Learned • Different teams / products have different needs – cater to them • Set session date to maximize participation • Arrange a visit during the event by an executive or subject matter expert • Express appreciation – create a positive experience • To the participants • To the triage team • Be present • Schedule events on a regular basis
- 17. Test Leadership Congress June 4th - 6th 2018, New York City, Test Leadership Congress June 4th – 6th 2018, New York City, Benefits • Visibility • Creating a mission across the organization • Personal connections • Positive reinforcement • Accountability