When making the transformation to DevOps and agile, many organizations find it hard to meet tough product release schedules and to cope with large data sets. Triaging failures across multiple platforms has become tedious and time-consuming. Reporting test-driven development (RTDD) is an innovative approach to agile testing that helps you write and implement tests with the end in mind (i.e., the test reports). RTDD puts structure, governance, and advanced capabilities into your test automation strategy. RTDD enables the entire product team to collaborate and make data-driven decisions in real time, resulting in test scenarios and test suites that are structured and easy to manage.

1. Drive Faster Quality
Insights through
Customized Test
Automation

2. Presenters
Tzvika Shahaf
Director Product
Management
Perfecto
Eran Kinsbruner
Dir. Technical Evangelist
Perfecto

3. Fast Feedback and Quality Analysis Drives Better Release Velocity
#Of Tests
per cycle
#Number of
Executions
#Release
Frequency

4. Current Mobile Market State

5. Versions: N, N-1, Beta
Windows OS: 7, 8.1, 10
Mac OS X: Sierra, El-Capitan
15 Combinations
Versions: N, N-1, Beta
Windows OS: 7, 8.1, 10
9 Combinations
Windows 7, 8.1, 10: IE 11
Windows 7: IE 8, 9
5 Combinations
Versions: 10.1, 9.1.3
Mac OS X Sierra: 10.1
Mac OS X El-Capitan, Yosemite: 9.1.3
3 Combinations
Windows 10: Edge 141 Combinations
33 Combinations For
Max Market Coverage
* Validate against your
web traffic analytics
Current Web Market State

6. Did You Ever Ran Into Such Conversation?
4/24/2017 6© 2015, Perfecto Mobile Ltd. All Rights Reserved.

7. Customer Research Shows
How Much Time Is Spent On Test Automation Regression Analysis Post
Execution?
• 1 Hour – 10%
• 2-4 Hours – 14%
• 3-5 Hours – 18%
• 1 Business Day – 20%
• I Don’t Know – 30%
4/24/2017 7© 2015, Perfecto Mobile Ltd. All Rights Reserved.

8. Quality Analysis Key Challenges
4/24/2017 8© 2017, Perfecto Mobile Ltd. All Rights Reserved.
Test Data Analysis Time
Is too long
Fit Analysis Into Existing
Processes and Tools
For quality decision making
Limited Quality Health visibility
across platforms
1 2 3

9. Desired State of Quality Feedback Across the SDLC
On-Demand access to digital
quality dashboard and
custom reports
Ability to drive fast Go/No Go
decision based on quality
trends
Bridge gaps and drive
ongoing communication
across Dev, QA, and Biz
Focus more on feature
development and validation,
less on analysis
Empower continuous quality
while eliminating flaky tests
for faster cycles
ENHANCED QUALITY
VISIBILITY
THROUGHOUT THE
PIPELINE
MEET THE FEEDBACK
WINDOW FOR RELEASE
(VELOCITY GOALS)
CAN I RELEASE A NEW
PATCH TO PROD IN LIKE 3
HOURS?
Cross Team Tools
Alignment For Zero
Blind Spots
Optimized quality strategy
across the entire organization
Test Data Analysis Time
Is too long
Fit Analysis Into Existing
Processes and Tools
For quality decision making
Limited Quality Health visibility
across platforms
1 2 3

10. DigitalZoom™ Solution

11. Digital Zoom™ - Using Reports Across The SDLC
4/24/2017 11© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Developer QA Manager/ExecutivesDev & Test Engineer Agile/Feature Teams
• Show me my build status
• Point me to the exact breaking
point
• I’d like to get data using Jira,
Jenkins and Slack
• Cross feature team
communication through Slack,
Jira and custom reports
• DevOps quality visibility per
feature team and post
integration phase
• Have the quality visibility to support
decisions during the SDLC
• Understand where problems are in order
to distribute across right people/teams
• Understand trends and challenges over
time
• Help me optimize quality using
tags and custom fields
• Maximize context during test
authoring
• Work within my IDE/FW

12. Introducing RTDD (Reporting Test Driven Design)
RTDD Is a method that enables teams to design quality with the debugging and insight in mind (context based).
Among the key benefits of RTDD are
1) Structured quality dashboard that can be sliced and diced by various persona’s
2) Fast MTTR of defects unrelated to the test suite size
3) Reliable and much more stable test suite (less flaky tests)
4) Low risk Go/No-Go decision due to a data-driven dashboard and quality artifacts
4/24/2017 12© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• Structured test authoring via SDK
• testStart()
• testStep()
• testStop()
• Context based Tagging (“regression”)

13. Benefits of Tagging Strategy
4/24/2017 13© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Logical Steps (incl. BDD support) Tags
Heat Map Based on Tags
Fast Quality Analysis/On-Demand Status
Context and Tags Drive Distribution for Analysis
For quality drill down and triaging process

14. Drilling Down Capabilities
• Quality dashboard in the digital era means:
• Cross platform visibility vs. single
platform reports
• Customizable report with API’s vs. locked
with off the shelf solutions
• Any dev language
• Any IDE
• Any test framework
• Data driven risk management based on
digital platform data
• 3 layer reporting
• Single test report (STR)
• Suite level report (Grid)
• Complete Dashboard View (By
category, By platform, By tags)
4/24/2017 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.

15. Defect Triaging – DigitalZoom™ Artifacts For RCA
• Digital reporting artifacts supports fast MTTR
• Logs
• Videos
• Screenshots
• Stack Trace
• Vitals
• Network PCAP files
• PDF report
• Export report Using Public API (Json) for further processing
4/24/2017 15© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Video Stack Trace Screenshots (incl. expected vs. Actual)

16. CI Dashboard Enables Build Trending And Overall Quality History Visibility
4/24/2017 16© 2015, Perfecto Mobile Ltd. All Rights Reserved.

17. CI Dashboard Enables Build Trending And Overall Quality History Visibility
4/24/2017 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.

18. Communication & Alignment Empowers Quality Collaboration
4/24/2017 18© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Seamless Integrations

19. LIVE DEMO

20. Thank You