This document discusses DevOps and continuous testing. It begins with defining DevOps as a process that increases communication between development and operations teams to automate and speed up software delivery. It then covers the benefits of DevOps like faster release cycles and time to market. Several case studies are presented showing how companies used DevOps and continuous testing to reduce testing time, increase coverage, and lower costs. The document concludes with a demo and opportunities for questions.

1. Guruswamy B M, Partner Enablement
Mgr.
Email:
Shorten Business Life
Cycle Using DevOps

2. Ask us questions in the Q & A Panel
We will send you the recording and slides
HOUSEKEEPING

3. Agenda
3
• What is DevOps?
• Benefits and Importance of DevOps
• Life Cycle of DevOps
• Actors and Their Roles
• Few Challenges in DevOps
• Approaches to DevOps Testing
• Continuous Testing for DevOps with Perfecto
• Demo
• Case Studies
• Q & A

4. What is DevOps?
DevOps can be defined as a culture or
process or practice within an
organization that increases the
communication, collaboration and
integration of the Development (which
includes the QA team) and the
Operations (IT Operations) teams. The
aim is to automate and speed up the
software delivery process much more
frequently and reliably

5. Business Pain
1. Release Cycle
2. Faster TTM
3. Product Stabilization
4. Feedback Mechanism
5. Supported Tools

6. Benefits And Importance of
DevOps

7. Life Cycle of DevOps

8. Actors and Their Roles
• As a Ops engineer I want to monitor
production data via Splunk,
Crashlytics
• As a Ops I should be able to do CI
and kick automation scripts
• As a Ops I should be able to auto
deploy various platforms (CD) and
Kick automation scripts
• As a Ops I should be able to trigger
execution status email to teams for
further analysis
• As Scrum master, I should have
Reports dashboard on daily
standups to prioritize the tasks
• As a Product team I should receive
build status and automation results
email daily basis
• As a manual tester I would be able
test feature in real device
• As a automation tester I need build
and devices to write scripts
• Faster feedback available for quick
script fixes and test stabilization
• As a Automation tester I should be
able to write scripts involving real
end scenarios and Performance
oriented
•
• As a developer I want to run my
unit tests, Espresso, XCUItests on
real device
• As a developer, I need a quick
feedback on the tests executed
• As a developer I should be able
initiate CI, CD jobs if required.
• As a developer I should be able to
access latest OS and latest devices
released to market
DEV QA
OPSPRODUCT

9. Few Challenges in DevOps
How to increase release frequency while retaining high
product quality
Frequent changes in the market – new OSes and
devices make a direct impact on app quality
It’s hard to embed stable and reliable automation that
simulates real user experience as part of the
development process.

10. Continuous Testing at Scale
Always Green, Everything
Every Day
Goal
Optimize the DevOps Pipeline
- through -
Continuous Testing
- by -
Building stable, scalable and
maintainable test automation
that you TRUST
Challenge
Test automation projects
deliver limited value
• Flaky tests
• Untrusted results
• Noise
• Hard to decipher results
Solution
Perfecto’s Continuous
Testing Blueprint
Best practices and support
focused on customers achieving
positive business outcomes tied
to the value we sold them
Green means Green
Go Faster Confidently
Frustration
Achieving Value
Systematically

11. 1
2
3 4
Approaches to DevOps Testing
Coverage
DailySprint
Smoke
Full
Coverage
before
Robustness
Robustness
Before
Coverage
Most of the market
• Sprint level execution
• Half-way to coverage
1. Create some scripts
2. Create more scripts
3. Even more scripts
4. (Try to) run them daily
1
2
3
4
1. Create few robust scripts
2. Make sure they run daily
3. Slowly add more scripts
1. Maintain stability & visibility
4. Add more scripts
• Too many failures
• Too much time spent on analysis

12. Continuous Testing for DevOps with Perfecto
DevOPs cycle
DevOps cycle for quick development
and deployment
Increase coverage to automate
across required platforms, environment
conditions and digital interfaces
Max. digital
coverage
Faster
Feedbac
k
Perfecto has in-built reporting methodology, gives
historical data, slide/dice results. We can get to know
what are the gaps in execution even before we actually
do a deep-dive leading to very quick feedback
High degree
of automation
Scale robust testing to cover all relevant use cases,
timed to daily executions across multiple teams
Usability Testing Test little, test often

13. DEMO

14. Bank of America – Reduce Release Cycle – Time to Market
© 2017, Perfecto Mobile Ltd. All Rights Reserved.
• Challenge
- Test Cycle taking too long – 2 weeks
- Unable to run automation at scale
• Goals
- Streamline QA process to get faster
feedback to Dev
- Decrease regression testing from 2
weeks to 48 hrs
- Reduce escape defects with improved
test coverage
• Key Metrics
- Reduced Test Cycle by 85%
- Coverage increased by 500%
Test Cycle
Before: 2 weeks
After: 7 hours

15. Citizens Bank – Award Winning App – Shift Quality Left
© 2017, Perfecto Mobile Ltd. All Rights Reserved.
• Challenge
- Manual risked based approach to
testing
 10% market coverage
- Defects found late in the release
 100+ escape defects
• Goals
- Automate manual testing
- Reduce escape defects
- Increase device coverage
 5 devices vs. 32 devices
• Key Metrics
- Automated 80% of manual testing
Before:
2 Star App
After:
Award Winning App
*Financial

16. CIBC – Reduce release cycle time – Time to market
© 2017, Perfecto Mobile Ltd. All Rights Reserved.
• Challenge
- Demand for high-quality, useful features
delivered quickly growing
- Low funding
- Release faster, with better quality,
without extra resources
• Goals
- Achieve 80%+ automation for regression
- Give 50%+ of regression time back to
partners for alternate use
- Improve mobile coverage by 100%
• Key Metrics
- 600% improvement in coverage
Before: 15-day day regression
6 devices
Test cases distributed across devices
Manually tested
After: 5-day regression
15 devices
All regression test cases run on all devices
85% Automation (cheque capture, finger-print ID)

17. Tangerine – Manual to automated– Reduce the
cost of quality
• Challenge
- 2 frameworks for mobile and web
- Attended technical expertise required for auto execution
- High first-run fail rate = lack of trust In
• Goals
- Unify 2 frameworks into one framework
- Lower skillset required to operate automation program
- Unattended automation to allow for increased execution
• Key Metrics
- 9-month payback on Perfecto ACV + FTE + Black Belt
- 70%+ first-run pass rate
- 400 % improvement in coverage
- Eliminate UFT license costs
Mobile regression
Before: 15 hours attended
After: 6 hours unattended

18. Lloyds – High Cost Testing Approach – Reduce Cost
© 2017, Perfecto Mobile Ltd. All Rights Reserved.
• Challenge
- Increasing manual costs
- High risk of escape defects
• Goals
- Automate manual effort
 Days Saved 317 (12 Months)
 428 Scripts across 34 platforms
 Regression: 40 hours to >2 hours
– Manual –UFT-Appium
- Increase Velocity
 Bi Weekly (4x coverage)
 4.5 days more development per cycle
• Key Metrics
- 63% Increased Platform Coverage
- 60% Automation
- 53% Labour Costs Saving through Offshoring
- 4.6* App
Regression Time
Before: 40 Hours
After: >2 hours
95% Improvement

19. Q & A

20. Thank You
Guruswamy B M
Partner Enablement Manager
Email:
guruswamy.bm@perfectomobile.com