Automation is a key requirement in DevOps. Yet, the 2017 World Quality Report has shown that only 16% of tests are automated. Whether you're in dev, test or operations, this webinar will help you solve the 5 key challenges teams face in scaling up their test automation. You will learn how to: * Overcome tight release schedule challenges Stabilize your test automation practices to achieve continuous testing *Enable the DevOps pipeline activities by optimizing test execution management *Maintain test sets that maximize productivity Get your tool stack ready and in sync with organizational capabilities

1. Joint Webinar
Eran Kinsbruner | Director, Lead Technical Evangelist, Perfecto
Manish Mathuria | CTO, Founder, Infostretch
January 30, 2018
Top 5 Test Automation Challenges
& How to Solve Them

2. Housekeeping
• Please participate in our live poll and end-of-webinar survey
• Ask us questions in the QA panel
• We will send you the recording and slides
1/30/2018 2© 2018, Perfecto Mobile Ltd. All Rights Reserved.

3. Agenda
• Digital Market Trends
• Key challenges in Test Automation and Continuous Testing
• Recommended Practices
• Q & A
1/30/2018 3© 2018, Perfecto Mobile Ltd. All Rights Reserved.

4. There’s a Paradigm Shift in Software
Development
+
1/30/2018 4© 2018, Perfecto Mobile Ltd. All Rights Reserved.

5. Quality Is Everyone's ‘Problem’
1/30/2018 5© 2018, Perfecto Mobile Ltd. All Rights Reserved.

6. 1
2
3 4
Approaches to DevOps Testing
Coverage
DailySprint
Smoke
Full
Coverage
before
Robustness
Robustness
Before
Coverage
Most of the market
• Sprint level execution
• Half-way to coverage
1. Create some scripts
2. Create more scripts
3. Even more scripts
4. (Try to) run them daily
1
2
3
4
1. Create few robust scripts
2. Make sure they run daily
3. Slowly add more scripts
1. Maintain stability & visibility
4. Add more scripts
• Too many failures
• Too much time spent on analysis
1/30/2018 6© 2018, Perfecto Mobile Ltd. All Rights Reserved.

7. Coverage Planning Tools
1/30/2018 7© 2018, Perfecto Mobile Ltd. All Rights Reserved.

8. Coverage Planning Tools
1/30/2018 81/30/2018 8© 2018, Perfecto Mobile Ltd. All Rights Reserved.

9. Poll Question
Top Automation Challenges
© 2018, Perfecto Mobile Ltd. All Rights Reserved.

10. Top 5 Automation Challenges
Tighter release schedule
leaves less time to
automate and test
(lack of) Test
automation stability
and reliability block
continuous testing
Test execution
management isn’t
intelligent and
optimized enough to
guide teams through
the DevOps pipeline
activities
Evolving and
maintaining test sets
and maximizing
productivity isn’t an
ongoing practice
Overall test
automation tool stack
isn’t in sync with
organizational
capabilities
1/30/2018 10© 2018, Perfecto Mobile Ltd. All Rights Reserved.

11. Tighter release schedule leaves less time
to automate and test
 Consider ATDD/BDD software development practices
 Build a consistent and unified CI process between
Dev-QA-OPS and eliminate ‘shadow CI’
 Consider adjusting your test pyramid and focus on the
highly valuable and fastest test suites
 Leverage faster test frameworks (XCUITest,
Espresso, headless – Puppeteer etc.)
1/30/2018 11© 2018, Perfecto Mobile Ltd. All Rights Reserved.

12. (lack of) Test automation stability
and reliability block continuous testing
 Test authoring based on industry standards (POM,
automate what’s write)
 Continuously maintain your test as you maintain code (SCM)
 Consider breaking your tests into sub test suites based on context
 Gain control over your test suites through
measurements and agreed upon KPI’s
 Stabilize your lab and testing environment
 Make sure your target platforms are in “ready state”
mode
1/30/2018 12© 2018, Perfecto Mobile Ltd. All Rights Reserved.

13. Test execution management isn’t intelligent
and optimized enough
 Start exploring innovative machine learning tools that can optimize your
test code
 Re-factor test code and include tagging, analytics hooks and other post-
execution debugging capabilities
 Periodic test code review throughout the pipeline
1/30/2018 13© 2018, Perfecto Mobile Ltd. All Rights Reserved.

14. Evolving and maintaining test sets and
maximizing productivity
 Open-source technologies struggle to keep up with innovative features
(Face ID etc.)
 A mix of commercial and open-source tools seems like a winning strategy
for many organizations
1/30/2018 14© 2018, Perfecto Mobile Ltd. All Rights Reserved.

15. BDD method has crossed the chasm
Source: InfoQ
1/30/2018 15© 2018, Perfecto Mobile Ltd. All Rights Reserved.

16. Architecture
Perfecto Report
Object Spy
Repository
file
Live dashboard
Cucumber BDD test
Test Java code
testNG
Perfecto
utils
Pre define
steps
User
advance
code
Script creation Execute Report
Access to devices
Remote debugging
Real devices and browsers
Manual services
1/30/2018 16© 2018, Perfecto Mobile Ltd. All Rights Reserved.

17. BDD – Cucumber
Test -Feature files
Pre define steps
by Perfecto
Logical steps
created by
customers
1/30/2018 17© 2018, Perfecto Mobile Ltd. All Rights Reserved.

18. BDD – Steps Definition
Test -Feature files
Pre define steps
by Perfecto
Logical steps
created by
customers
1/30/2018 18© 2018, Perfecto Mobile Ltd. All Rights Reserved.

19. BDD – Object Repository
objectID = locatorID = locator
1/30/2018 19© 2018, Perfecto Mobile Ltd. All Rights Reserved.

20. BDD – Components
Test -Feature files
Pre define steps
by Perfecto
(Low Level)
Logical steps
created by
customers
• Standard BDD – cucumber
• Auto complete by standard Cucumber
plugin
Requirements from user:
• Identified elements and add to repository
• Build flow by using low level and logical
functions
1/30/2018 20© 2018, Perfecto Mobile Ltd. All Rights Reserved.

21. Code – Components
Test -Feature files
Pre define steps
by Perfecto
Logical steps
created by
customers
• Customer code
• Standard java code
• Focused only on the test actions and validations
• QAF manage
• Driver init
• getDriver() – provide the exiting driver
• Object repository
• Predefine functions (sendKey, assertText,…)
• Integration with Reportium execution and steps (no
code need to added)
• Perfecto
• Utils code with specific commands like (WT, swipe)
1/30/2018 21© 2018, Perfecto Mobile Ltd. All Rights Reserved.

22. Project Structure
BDD
Logical commands
Java tests for java coder testers
Perfecto extensions
TestNG execution
Data repository
Cloud / app configuration file
1/30/2018 22© 2018, Perfecto Mobile Ltd. All Rights Reserved.

23. Project Config File – Driver
Appium
Selenium
1/30/2018 23© 2018, Perfecto Mobile Ltd. All Rights Reserved.

24. Execution – Threads and Devices
Each test = new thread with the requested device
Thread 1
iOS
Thread 2
Android
1/30/2018 24© 2018, Perfecto Mobile Ltd. All Rights Reserved.

25. Quantum
Area Components Quantum
Management
Reporting
Execution Manager
Test
Development
Test Authoring
(IDE+Language)
Object Repository
Object Locator/Spy
Appium/Selenium
✓OSS
=
✓OSS
✓OSS
http://projectQuantum.io
1/30/2018 25© 2018, Perfecto Mobile Ltd. All Rights Reserved.

26. Demo

27. Available Resources
Download the free
eBook here -
http://bit.ly/2E0ZfzS
1/30/2018 27© 2018, Perfecto Mobile Ltd. All Rights Reserved.
Download the free
Whitepaper here -
http://bit.ly/qewhitepaper
Read Chapter 15:
Master Your “Shift Left”
Moves -
http://bit.ly/2GrXkpp
Manish Mathuria
Chief Technology
Officer & Founder
Infostretch
Get a Free QE Maturity
Assessment - See how your
organization measures up
against industry best
practices -
http://bit.ly/qeassessment

28. Thank You!
© 2018, Perfecto Mobile Ltd. All Rights Reserved.