Watch on-demand: http://info.rainforestqa.com/webinar-continuous-testing-for-ctos Slides from the Continuous Testing for CTOs webinar with Laks Srini (CTO, Zenefits) and Russell Smith (CTO, Rainforest QA). QA testing is often a bottleneck to true continuous deployment, and traditional processes are ill-suited for CI/CD environments. As your company grows, the cost of scaling traditional QA practices to meet the needs of a larger audience with more complex products can overextend bandwidth. With continuous testing, you can keep your growing organization's QA costs down and empower your team to test more effectively. - How Zenefits uses continuous testing to speed up their QA cycle to match their CD environment - How continuous testing minimizes the cost of scaling your QA team - How to plug continuous testing into your existing Agile or CD workflow for minimal disruption - How to speed up manual QA activities to ensure fast but comprehensive testing

1. Continuous Testing for CTOs:
How Zenefits Deploys 3x Day
Laks Srini
CTO & Co-Founder
Russ Smith
CTO & Co-Founder

2. #continuoustesting
Agenda
● Why do Continuous testing?
● Types of testing
● How do you integrate testing into dev flow?
● How did Zenefits approach this need for QA?

3. #continuoustesting
Challenges with Testing Today
● Manual testing is an important aspect of comprehensive testing, but it is
usually done in huge chunks and takes forever
● Automated testing is brittle, not comprehensive
● Most in-house QA teams have very limited bandwidth
https://www.capgemini.com/events/world-quality-report-2015-16-pre-order-form

4. #continuoustesting
Test Continuously (Shift Left)
Test as early & often as possible for Faster Shipping, Better Quality
● Unit tests
● UI tests (automated vs. manual)
Production monitoring + Rollbacks

5. #continuoustesting
How does continuous testing fit into the
continuous delivery pipeline?
Code
Development
Continuous
Integration
Staging Production
Continuous
Deployment

6. #continuoustesting
How does continuous testing fit into the
continuous delivery pipeline?
Code
Development
Continuous
Integration
Staging Production
Continuous
Deployment

7. #continuoustesting
Test Strategies in Development
Code
Development
Staging Production
Usability Testing
Functional Testing
Regression Testing
GUI Testing
Cross Browser Testing

8. #continuoustesting
Test Strategies in Staging
Code
Development
Staging Production
Exploratory
End to end
Usability
Functional
Regression
GUI
Cross Browser Testing
Database Testing

9. #continuoustesting
Test Strategies in Production
Code
Development
Staging Production
Sanity /
Smoke Tests
Automated Testing

10. #continuoustesting
How do you begin testing continuously?
● Identify how comprehensive are your
current testing processes?
● Incorporate smaller tests, throughout your
development pipeline
● Continuous Integration (CI) - automate
regression suites
● Automation alone can be brittle and flaky
Zenefits was able to scale their
customer base to 10,000+
without an in-house QA team
by supplementing their testing
automation suites with manual
testing.

11. #continuoustesting
Challenge:
Maintain product quality while growing exponentially.
There was no room for failure.

12. #continuoustesting
Why did Zenefits approach Continuous Testing?

13. #continuoustesting
Why did you choose Rainforest?

14. #continuoustesting
What is the Hub & Spoke model?
Unique challenges with this

15. #continuoustesting
Challenges in SOA?

16. #continuoustesting
Scalability

17. #continuoustesting
Focus on Product

18. #continuoustesting
Ah-ha! moments
How has continuous testing has improved Zenefits?
● Automate the execution of manual testing
● Firing off manual tests is just as easy as automated tests, part of the same
process
● Now we can confirm results of automated tests and test things in bulk
with a human.
● Overcome brittle/flaky automated tests with safety net
● Deployment happened when it was ready

19. Questions?
Laks Srini
CTO & Co-Founder
Russ Smith
CTO & Co-Founder

20. Thank you!