This document summarizes key topics from the GTAC 2015 conference on test automation and innovative development. It discusses trends in test automation like component testing, mock testing, and production monitoring. The top five talks are summarized, focusing on continuous deployment, transforming monolithic code to microservices, and using production traffic for regression testing. Statistical sampling techniques are discussed for reducing large production datasets. Measuring efficiency, success versus failure, and challenges of startups versus corporations are also addressed.

1. GTAC 2015 and
Innovative Development
Igor Goulko,
Independent Consultant
e-mail: goulko@gmail.com
17-Mar-2016 (ANZTB)20

2. Agenda
1. Innovation inTest Automation
2. TOP 5Talks
3. Efficiency and measurements
4. Experiments and Learning
5. Success and Failure
6. Start Up vs Corporation

3. Innovation inTest Automations
General Direction and Best Practice
◦ ComponentTesting
◦ Mock everything (LinkedIn, Google and others)
◦ Production DataTesting
◦ Production Monitoring
◦ More advanced testing tools
◦ Custom tools

4. Top 5 talks
 Testing Tube (Jurgen Allgayer)
◦ Unified dev/test environment
◦ Everyone writes tests
◦ Dev=integration
◦ No surprises = Borg
◦ Daily pushes
◦ 200 services
 Large-Scale AutomatedVisual Testing
(Adam Carmi, Applitools)
◦ Very sophisticated image comparison
◦ Cross browsers and device testing
◦ Monitoring dynamic applications
◦ Automate a maintenance

5. Hands Off RegressionTesting
 Continuous deploy
 Transformation from Huge Monolithic
code to micro services
 Unit tests and Component tests
 5 Code Paths for 6 Methods ~ 15KTC
 Diffy uses production traffic for
regression tests
 Candidate (new), Primary and Secondary
deployments

6. DiffyTopology
diffy
secondary
candidate
primary
raw differences
non-deterministic noise
filtered
differences
sampled
producti
on traffic

7. Statistical Data Sampling
Celal Ziftci and Ben Greenberg
 Smart Production Sample Data (1.6B->78K)
 14 hours -> 2 mins and it runs for EVERY single submit
Data
Sampler
Data
Sampler
Data
Sampler
Data
Sampler
Complete Production Data
...
Data
Sampler
Final Sample
Sample Sample Sample Sample

8. YourTests Aren't Flaky
(Alister Scott,WorldPress)
“

9. Success and Failure
 Failure is part of our process
 Success is our Goal
 Ratio
 Risk vs Opportunities

10. Efficiency and measurements
 Time, quality, price
 Productivity, efficiency and product price

11. Efficiency and measurements
 Team’s Open space vs Offshore
 Informal meetings and discussions

12. Efficiency and measurements
 Greenfield project vsTransformation
 Transformation has to became BAU

13. Start Up vs Corporation
 Legacy code and Monsters feeding
 Try, learn, implement, start over
 Knowledge, Skills, People, Culture
 Processes,Technologies, Innovations

14. Q&A
e-mail: goulko@gmail.com