Learn how we automated the prep, execution and report generation of our preprod performance testing for TurboTax Online.

1. Revised: May 20, 2016
Automated Performance
Testing

2. The Challenge
• Fast pace environment producing frequent application changes
• Application changes requires new scripts
• Manual effort is inefficient (takes 2-5 hours to create test scripts manually)
• Human error leads to inconsistent test results
• Spending multiple hours everyday creating test scripts is BORING
• Complex systems being developed requiring performance engineers expertise

3. Manual Workflow
Manually record new
perf scripts
Import scripts into
Tank
Massage scripts
manually
Kickoff performance
test
Create test events in
snoop
Collect performance
data
Create test reports
Email reports for
analysis
Manual TANK SNOOP
Deploy App

4. Here’s how we automated our testing
1) Automated the test scripting
– QE’s developed Page Objects framework for functional automation based on Selenium web driver.
– We reused QE’s automation framework (test cases)
– Created Jenkins plugin to record perf scripts when automation runs
2) Automated the load test setup
– We use Intuit Tank as our performance testing tool which is open source
– Added the ability to automatically massage scripts
3) Automated the load test execution
– Created REST API’s in Tank to consume project information and execute a load test
4) Automated report generation
– Using our in-house report generation tool we enhanced it to automatically pull all key metrics from the system under
test and dynamically create reports to help the performance engineers quickly analyze and compare the results to
prior tests

5. Automated Workflow
Deploy App
Kickoff Jenkins job to
record script
Start proxy & run
Selenium Script
Send recorded script
to TANK
Massage scripts Run performance test
Test information sent
to Snoop
Collect performance
data
Create test reports
Email reports for
analysis
Automation Workflows TANK SNOOPManual

6. What we learned
• This will take an investment in tooling
• Must partner with Quality counterparts to be successful
• Great forcing mechanism to help drive performance testing earlier
into the SDLC
• Lab environment stability is key; treat preprod like prod

7. From To
Manually recording scripts Automated recording of scripts
Frequently finding functional issues that
block perf testing
Identifying blocking issues prior to
promoting to perf environment
Lengthy testing cycle Comprehensive automated process for
running perf test
Limited time allocated to driving
performance improvements
Partnering with development teams to
tune and optimize applications
Summary
7

8. Thank you