Selenium Conf: Patterns to Whole Team Test Automation Transformation

•

0 likes•44 views

The document discusses test automation transformation and lessons learned from a team's experience improving their test automation process. Some key points: - The team was able to improve test automation maturity and increase speed of releases, decrease time to release, and improve productivity, despite not having an explicit test automation strategy or careful tool selection. - Perceived success factors included whole team effort, expert team members, self-motivated team members, allowing time for learning, and choosing the right test tools and infrastructure. - An incremental approach, observing and optimizing processes, and an internal open-source community mindset also contributed to the team's success in improving their test automation capabilities.

Software

@maaretp
Patterns to Whole Team
Test Automation Transformation
by Maaret Pyhäjärvi

@maaretp
‘plexus’
particularly python library to drive hardware interfaces in embedded systems
js/ts

@maaretp
A published story from…
2019
https://www.academia.edu/49495368/Test_Automation_Process_Improvement_in_a_DevOps_Team_Experience_Report

@maaretp
We did particularly
well with our Test
Automation (TA)
WHY?

@maaretp
WB
(fail)
DMU
(success)
AMS
(success)
AVMT
(inconclusive)
SND
(fail)
EPT
(inconclusive)
NM
(inconclusive)
PSU
(fail)
FIR
(fail)
Success /
Failure is a
Snapshot
VSP
(success)

@maaretp
Conflict with previous research!
• No explicit test automation strategy
• No careful tool selection
• No measuring quality and performance of TA
• No explicit guidelines
“…relaxed, verbally communicated strategy
without strict rules and processes relying on
developers’ voluntary participation.We
consider this an unusual success with TAPI …

@maaretp
High satisfaction and maturity (self-reported 2019) (2021)
- Increasing speed to release
(from 2-4 releases to 9 releases) (from 1 release to 9 releases)
- Decreasing time to making release
(from 5 days to 4 hours) (from 34 days to 2 days)
- Improving team productivity
(11 team members, 2917 code changes out of 6583 for 3 teams) (significant investment)
- Shared platform work efficiency
- Sustainable test automation maintenance effort
- Finding relevant issues
(new product in days, MANY timing issues and regressions) (new product in days, MANY new issues)
- High satisfaction of customers
(167 support issues, 1 escalation for millions of customers) (new product, few customers)
- High test efficiency
(213 708 tests in single working day) (~100 tests each night, green within day)
- Reasonable investment in test automation process improvement (ehh, none) (significant)

@maaretp
Perceived Success Factors
Dimension Factor
Human Whole team effort
Expert team members
Self-motivated team members
Organizing Allow time for learning curve
Internal open-source community mindset
Technical Test tool choice and architecture
Testlab infrastructure
Product testability
Telemetry
Process Incremental approach
Process observation and optimization

@maaretp
Moving from specialist
language to generalist
language

@maaretp
Visualizing testing debt
and coverage

@maaretp
Showcase progress made
of continuous flow of small
changes

@maaretp
Choose to release based on
automation no matter how
little test automation there is
* Explore by following change

@maaretp
Growing individual
competencies by sharing
YOUR screen when working
together

@maaretp
Ensemble
Testing
at
Selenium
Conference
2022

@maaretp
Semantic selectors over
position-based, but I just
wanted to try them out
The vscode coloring bug
bothers me but not enough

@maaretp
Contemporary
Exploratory
Testing
Improved Understanding of Success
No manual and automated testing

@maaretp
2020 2016
MIATPP
Most Influential Agile Testing
Professional Person
#PayToSpeak #TechVoices
#EnsembleTesting #EnsembleProgramming #StrongStylePairing
#ExploratoryTesting #TestAutomation
#ModernAgile
#AwesomeTesters
Maaret Pyhäjärvi (from Finland)
Email: maaret@iki.fi
Twitter: @maaretp
Web: maaretp.com
Blog: visible-quality.blogspot.fi
(please connect with me through
Twitter or LinkedIn)
2019, 2020, 2021
https://exploratorytestingacademy.com
Ohjelmistotestaus ry
https://techvoices.org

A new generation of testing is emerging, led by testers who believe test automation is an inseparable part of how we do exploratory testing. Yes, you read that right: exploratory testing with test automation, and on unit, api, ui and e2e levels all at our reach with team collaboration. We work on acceptance criteria and examples in our short increments, yet people are at their most creative on what could go wrong when they use the product or a piece of the product as their external imagination. When we’re creative, we need to do things we can’t do by hand, using programming to extend our reach. When we’re learning, we need to take notes now and to keep for later, again using programming to replenish information we deem worthwhile. We want to work on exploring a category of bugs before implementing, and a category of bugs before releasing, and a category of bugs after releasing. Last year I worked with a team that was transforming from a traditional manual test case + test automation -based approach to contemporary exploratory testing. This year I work with another team that takes this style of testing further than we’ve taken it before with whole-team testing. Let’s learn from a practitioner’s examples of ideas leading into this, changes enabling this, and results we are seeing with contemporary exploratory testing.

Testing Voices '21 keynote: Testing Becoming Harder To BeValuable

Maaret Pyhäjärvi

ICST/NEXTA2020: Experience Report on Test Automation Process Improvement

Maaret Pyhäjärvi

Qtp - Introduction values

Vibrant Technologies & Computers

Measuring Coverage From E2E Tests

Anand Bagmar

It is easy to measure code coverage when running unit tests. However, very frequently the following questions come up: - How can we measure API test coverage and e2e / UI test coverage? - Does e2e / UI test coverage add value? - If not, what other data can we look at to know if the e2e tests have good coverage? This session is about understanding the above questions, and finding solutions for the same.

Exploratory testing using heuristics

Michelle Lagare, CSM

rTest, a Testing Tool for FME Workspaces

Safe Software

The process of checking if a FME workspace produces a right outcome is heavy and tedious, that can generally be made manually. It is likewise difficult to find the regressions when you wish to optimize a FME workspace or upgrade it to a new environment. This obstacle often leads to freeze workspaces after a manual test stage. In production, some key workspaces, designed with old versions of FME and unable to upgrade, can often be found. In order to solve this problem, we have developed rTest, a free tool allowing you to identify the malfunctions and regressions in your FME workspaces. Associated with an adapted development methodology, rTest enables you to increase the deliverable quality and to decrease the maintenance costs.

Parameter tuning or default values

Vivek Nair

Neotys organized its first Performance Advisory Council in Scotland, the 14th & 15th of November. With 15 Load Testing experts from several countries (UK, France, New-Zeland, Germany, USA, Australia, India…) we explored several theme around Load Testing such as DevOps, Shift Right, AI etc. By discussing around their experience, the methods they used, their data analysis and their interpretation, we created a lot of high-value added content that you can use to discover what will be the future of Load Testing. You want to know more about this event ? https://www.neotys.com/performance-advisory-council

Bridging the communication gap

Guillagui San

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

Maaret Pyhäjärvi

SAP Testing Servicesr_shanki

AgileTD Scandia: Mob Testing

Maaret Pyhäjärvi

MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing

Maaret Pyhäjärvi

MoT Sarajevo: How to get better at Exploratory testing

Maaret Pyhäjärvi

The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing

Perfecto by Perforce

The widespread adoption of test automation has created many challenges — for everything from development lifecycle integration to scripting strategy. One pitfall of automation is that teams often rush to automate everything they can. This is the automation firehose. However, just because a scenario CAN be automated does not mean it SHOULD be automated. For scenarios that should be automated, teams must adopt implementation plans to ensure tests are reliable and deriving value. Join this webinar led by Perfecto’s Chief Evangelist, Eran Kinsbruner, along with Thomas Haver, Manager of Automation & Delivery. In this session, the audience will: -Understand which test scenarios to automate. -Learn how to maximize the benefits of automation. -Receive a checklist to determine automation feasibility and ROI.

Can we induce change with what we measure?

Michaela Greiler

Tom DeMarco states that “You can’t control what you can’t measure”, but how much can we change and control (with) what we measure? This talk investigates the opportunities and limits of data-driven software engineering, shows which opportunities lie ahead of us when we engage in mining and analyzing software engineering process data, but also highlights important factors that influence the success and adaptability of data-based improvement approaches.

DesiradhaRam Gadde - Testers & Testing in ChatGPT-AI world.pptx

Joseph Beale

Gone are the days where you can simply learn QA and QC concepts to get into software testing job. In the next generations “QA Role” would be evolved into automation first mindset. Focused where you need to be good with automation concepts even if you are tasked with manual testing on day-to-day basis. The automation technologies and testing methodologies has evolved the software testing and some of them even redefined how testing can be done. The current buzz word in the industry is “AI”, we believe that AI will also have the similar impact on software testing in the future. In this interactive session, we would like to discuss about our ideas on where AI & ML can benefit testing and testers. What do we need to be prepared for next “Decade” to stay relevant in the market. How testers need to evolve in the world of “ChatGPT”.

DesiradhaRam Gadde - Testers & Testing in ChatGPT-AI world.pptx

QA or the Highway

DesiradhaRam Gadde - Testers _ Testing in ChatGPT-AI world.pptx

QA or the Highway

Improving Translator Productivity with MT: A Patent Translation Case Study

Iconic Translation Machines

First steps in testing analytics: Does test code quality matter?

Andy Zaidman

Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)

Neotys_Partner

Since its beginning, the Performance Advisory Council aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing. During this event, 12 participants convened in Chamonix (France) exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.

7 Reasons Why Value Stream Integration Improves Software Quality assurance

Tasktop

Key takeaways from this webinar include: - How to elevate the “QA Brand” in your organization with Value Stream Integration - How to improve your Defect Detection Effectiveness (DDE) by understanding 2 key principles - Techniques to drive effective change impact analysis, control, and management of changes - How to improve automated test coverage with real-time feedback loops - Why “swimming upstream” creates high quality software - How to eliminate the “Ping Pong Effect” and achieve lasting developer & tester alignment - Debunking the “One-Size-Fits-All” trend to platforms, governance, and Testing Centers of Excellence All the buzz today is on DevOps, accelerating commit to release time, and scaling Agile. Sure, that’s important stuff – but where does Quality Assurance (QA) fit into this picture? An award-winning recipe contains a specific set of ingredients, otherwise the final product just doesn’t taste right. QA Leaders are now recognizing the vital importance of Value Stream Integration (VSI) as a key ingredient in their cake batter. In this presentation, we break down 7 key reasons why VSI will inherently improve the effectiveness of your QA and Testing practice. Inspired by Matt Angerer’s trials and tribulations as a Program Testing Consultant for over 8 vertical markets and 10 end-to-end SAP implementations, this presentation is a no-nonsense action-packed roadmap to understanding VSI. You’ll learn how to immediately apply this knowledge to champion cross-functional initiatives that revolutionize how you deliver and protect software quality for lasting results.

Agile Mumbai 2020 Conference | How to get the best ROI on Your Test Automati...

AgileNetwork

$C:\documents and settings\selvam.mc\my documents\automation testing process$ $C:\documents and settings\selvam.mc\my documents\automation testing process$

C:\documents and settings\selvam.mc\my documents\automation testing process

selvam mc

Thomas Kauders - Agile Test Design And Automation of a Life-Critical Medical ...

TEST Huddle

Why Test Automation Fails

Ranorex

ATAGTR '20 Keynote: Building status

Maaret Pyhäjärvi

MoT Cork: Exploring Realities of Testing - From Theory to Practice

Maaret Pyhäjärvi

Similar to Selenium Conf: Patterns to Whole Team Test Automation Transformation

Neotys PAC - Stijn Schepers

Neotys_Partner

Bridging the communication gap

Guillagui San

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

Maaret Pyhäjärvi

SAP Testing Servicesr_shanki

AgileTD Scandia: Mob Testing

Maaret Pyhäjärvi

MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing

Maaret Pyhäjärvi

MoT Sarajevo: How to get better at Exploratory testing

Maaret Pyhäjärvi

The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing

Perfecto by Perforce

Can we induce change with what we measure?

Michaela Greiler

DesiradhaRam Gadde - Testers & Testing in ChatGPT-AI world.pptx

Joseph Beale

DesiradhaRam Gadde - Testers & Testing in ChatGPT-AI world.pptx

QA or the Highway

DesiradhaRam Gadde - Testers _ Testing in ChatGPT-AI world.pptx

QA or the Highway

Improving Translator Productivity with MT: A Patent Translation Case Study

Iconic Translation Machines

First steps in testing analytics: Does test code quality matter?

Andy Zaidman

Jonathon Wright - Intelligent Performance Cognitive Learning (AIOps)

Neotys_Partner

7 Reasons Why Value Stream Integration Improves Software Quality assurance

Tasktop

Agile Mumbai 2020 Conference | How to get the best ROI on Your Test Automati...

AgileNetwork

$C:\documents and settings\selvam.mc\my documents\automation testing process$ $C:\documents and settings\selvam.mc\my documents\automation testing process$

C:\documents and settings\selvam.mc\my documents\automation testing process

selvam mc

Thomas Kauders - Agile Test Design And Automation of a Life-Critical Medical ...

TEST Huddle

Why Test Automation Fails

Ranorex

Similar to Selenium Conf: Patterns to Whole Team Test Automation Transformation (20)

Neotys PAC - Stijn Schepers

Bridging the communication gap

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

SAP Testing Services