Business demands quicker and cleaner SDLC’s, best streamlined by DevOps. DevOps is changing the face of QA, and QA empowers DevOps. Join QualiTest and Zeenyx for a webinar that will address these changes and present a path for testing success as part of a DevOps program. Find out more by visiting www.QualiTestGroup.com

1. DevOps challenges facing QA

2. Webinar Presenters:
Garry Heon Brian Le Suer

3. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

4. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

5. | World’s 2nd Largest Independent Testing Company
|QualiTest recognized for the past 3 years as the only
visionary pure play QA and Testing Services Company
in the “ ” Magic Quadrant”About
QualiTest
Group

6. About
Zeenyx
| Creators of AscentialTest, the award-winning
testing solution for Windows, Web, dotNet,
Java, PowerBuilder, Terminal and PDF
| Committed to reducing the cost of software
testing by making testers more productive

7. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

8. What is DevOps?
| Born from Agile software development and the need to keep up with the new software
velocity
| It’s combined with the understanding of value of collaboration between organizations
throughout all stages of development
| The increased importance of operations in the SAAS world has driven the need for
innovation on the systems side
| As its name suggests, DevOps is a combination of software development, QA and
operations disciplines
| DevOps recognizes the interdependence of the groups

9. What DevOps is not?
| DevOps is not Development taking over Operations
| DevOps is not just culture
| DevOps is not just tools
| DevOps is not just Development and Operations

10. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

11. Challenges
| Alignment of development, QA and operations perspectives
| Learning new skills
| Frequent code changes must be tested quickly
| Automation suites must be resilient to application changes
| Having the ability to automate in sprint
| Non-automatable tests must be integrated into the test cycle (exploratory, Ux, usability
testing)
| Maintaining a developer to QA ratio that delivers consistently

12. Addressing the QA Challenges
| User stories are available and clearly defined before the sprint open/close meeting
| Mock-ups/HTML of the UI design are made available to Dev and QA team at the same time
| Testers must design their automation test case using mockups (pseudo code and/or the
layout of the keyword)
| When Development completes UI integration, DOM elements are available for QA to
complete the test case

13. Addressing the QA Challenges
| Testers work closely with Development
| When the Xpath or the identifier of a page object is available, QA can make changes immediately
| Page object identifiers/Xpaths must be implemented so that all page objects are defined uniquely
| An appropriate automation test suite should be identified and plugged into the continuous
integration pipeline

14. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

15. Resources
| Need to maintain a good developer to QA ratio
| “QualiTesters” are finding the most success
| Test resources that are well rounded who can analyze requirements, automate test cases and execute
them manually when needed
| SDETs find the transition easiest
| QA engineers are well-versed with technology, tools and process
| CI/CD, Agile
| Bamboo, Jenkins, Git/TFS, Automation frameworks, AWS
| Keep up-to-date on trending technologies related to DevOps

16. Resources
| QA engineers need to be empowered
| QA engineers need to communicate clearly during scrum meetings
| QA engineers need to hold other team members (Dev, Ops) accountable to commitments
| QA engineers need to facilitate collaboration within the teams
| QA engineers must address issues immediately

17. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

18. Requirements for
an automated
testing solution in
an Agile/DevOps
environment
|Frequent code/component changes means that:
| Tests must be executed frequently so that defects can be traced
directly to most recent changes
| Tests must be resilient to application changes
| Tests must be built using tools and approaches that makes it
possible to update tests quickly and easily

19. AscentialTest
Integration with
Version Control
|AscentialTest provides integration with version
control to facilitate collaboration:
| Teams of people modify integrated test components
| Test projects must compile and be ready to execute at all times
|AscentialTest version control integration:
| Check-out/check-in
| Update project
| Merge changes
| Version history

20. AscentialTest
Integration with
Build Systems
|AscentialTest provides integration with build systems so
that tests can be executed automatically on demand:
| AscentialTest’s command line provides a mechanism to checkout the
latest version of tests to target test machines
| AscentialTest’s execution engine is accessible from the command line
to allow tests to be run directly by the build system
| AscentialTest report generator is accessible from the command line
so that reports can be generated at the conclusion of a test run
| Zeenyx Portal makes test results and reports available to the entire
organization from any web browser

21. Rapid Test
Development and
Maintenance
|AscentialTest ‘AppObjects’ insulate tests from changes
in the application’s GUI
| AppObject definitions provide a single point of change
| If an application object is modified, only the AppObject definition
needs to be updated. No changes to tests are required.
|AscentialTest’s Step-based Tests
| Tests are built with reusable components called ‘steps’ that speed
up the test creation process and reduce the maintenance effort
when the target application changes.
| If application logic changes, only impacted ‘steps’ need to be
modified. Tests that implement those ‘steps’ do not require any
maintenance.

26. Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

27. DevOps Metrics
Metric Description
Change Volume How many user stories/fixes are being deployed.
Change Lead Time When commitment is made to business -> production.
QA Lead Time The time from when the code is committed to when it’s deployed
Change Failure Rate How many defects does a change introduce into the system
Production Defects Number of defects found in production
Hotfix/Rollback Production Defects Number of defects found in production that require a hotfix
MTTR (Rollbacks) When a failure occurs how much time does it take to roll back
MTTR When a failure occurs how much time does it take to recover (Hotfix)
Number of Hotfixes/Rollbacks Number of hotfixes deployed over time
Deployment Frequency How often is new code being deployed
Time to deploy Time to deploy new code
Percentage of Failed Deployments Percentage of deployments had issues
Availability Overall uptime of the system

28. gheon@qualitestgroup.com brian@zeenyx.com

29. www.QualiTestGroup.com
Thank You!