1
10x Test Coverage, Less Drama: Shift Left
Functional & Performance Testing
Dave Karow, Developer Advocate – BlazeMeter
Leo Laskin, Senior Solutions Architect – Sauce Labs
2
SHIFTING LEFT TO GET COVERAGE @ SPEED
Dave Karow
{ Developer Advocate }
3
Poll Questions Coming Up – Let’s Define Shift Left First
Shifting left means starting testing earlier in the development cycle and writing those early tests
in the same small team that is developing the code.
Put another way, shifting left means not waiting for a “hand-off” of specifications or code to a
separate group or waiting until developers think they are “done” with a sprint or release.
Shift left eliminates surprises where fundamental problems are found late in the process.
Shift left reinforces the definition of “done” being when a story has been delivered as properly
functioning code that performs to expectations.
4
Test Coverage @ Speed
+20%
Better
Tools
Increased
Efficiency
To achieve SUPERIOR TEST COVERAGE
we must TEST MORE
HOW?
+1000%
Shift
Left
Superior Test
Coverage
5
“Shift Left” Vision
As code is committed
tests specific to that code run automatically
and developers get instant feedback,
so the entire team always knows the state of the code,
and bugs get fixed before anyone declares “done”
6
Shift Left Means Start At The Very Beginning…
SHIFT LEFT SHIFT RIGHT
DELIVERY
CONTINUOUS TESTING
CODE BUILD PRODUCTION
7
Before Shifting Left
Feature #1
Development time
Feature #2
Development Time
Feature #3
Development Time
Feature #1
Functional & Regression Testing
Feature #2
Functional & Regression
Testing
Feature #1
Perf Test
Automation
Development
Iteration 3Iteration 2Iteration 1
Iteration 1
“ready” for release
8
Issues Always Pile Up…
 Technical debt accumulates
 “Release vs quality” tension a constant
 Context switch = inefficient thrashing
 Automation is an after thought & often abandoned
9
Let’s Try Another Way
 Create simple text-based files to define small tests
 Store the tests with your code
 Run the code and the test on every commit
10
Demo: Build an API Endpoint Test In 3 Minutes
1. Create a template via the Chrome Extension
2. Add a variable, assertion and performance threshold
3. Test it on our desktop
4. Commit, Clone and Repeat
11
After Shifting Left
Feature #1
Development time
Feature #2
Development Time
Feature #3
Development Time
Feature #1
Test Development &
Execution
Feature #2
Test Development &
Execution
Feature #3
Test Development &
Execution
Integration Tests = New Feature + Regression + Performance
Shippable Feature !
12
After Shifting Left
 Technical debt doesn’t grow
 Quality is embedded in every step of the SDLC
 Immediate developer feedback
 More tests are written as part of the SDLC
 Faster ”Shippable product” with higher quality

10x Test Coverage, Less Drama: Shift Left Functional & Performance Testing

  • 1.
    1 10x Test Coverage,Less Drama: Shift Left Functional & Performance Testing Dave Karow, Developer Advocate – BlazeMeter Leo Laskin, Senior Solutions Architect – Sauce Labs
  • 2.
    2 SHIFTING LEFT TOGET COVERAGE @ SPEED Dave Karow { Developer Advocate }
  • 3.
    3 Poll Questions ComingUp – Let’s Define Shift Left First Shifting left means starting testing earlier in the development cycle and writing those early tests in the same small team that is developing the code. Put another way, shifting left means not waiting for a “hand-off” of specifications or code to a separate group or waiting until developers think they are “done” with a sprint or release. Shift left eliminates surprises where fundamental problems are found late in the process. Shift left reinforces the definition of “done” being when a story has been delivered as properly functioning code that performs to expectations.
  • 4.
    4 Test Coverage @Speed +20% Better Tools Increased Efficiency To achieve SUPERIOR TEST COVERAGE we must TEST MORE HOW? +1000% Shift Left Superior Test Coverage
  • 5.
    5 “Shift Left” Vision Ascode is committed tests specific to that code run automatically and developers get instant feedback, so the entire team always knows the state of the code, and bugs get fixed before anyone declares “done”
  • 6.
    6 Shift Left MeansStart At The Very Beginning… SHIFT LEFT SHIFT RIGHT DELIVERY CONTINUOUS TESTING CODE BUILD PRODUCTION
  • 7.
    7 Before Shifting Left Feature#1 Development time Feature #2 Development Time Feature #3 Development Time Feature #1 Functional & Regression Testing Feature #2 Functional & Regression Testing Feature #1 Perf Test Automation Development Iteration 3Iteration 2Iteration 1 Iteration 1 “ready” for release
  • 8.
    8 Issues Always PileUp…  Technical debt accumulates  “Release vs quality” tension a constant  Context switch = inefficient thrashing  Automation is an after thought & often abandoned
  • 9.
    9 Let’s Try AnotherWay  Create simple text-based files to define small tests  Store the tests with your code  Run the code and the test on every commit
  • 10.
    10 Demo: Build anAPI Endpoint Test In 3 Minutes 1. Create a template via the Chrome Extension 2. Add a variable, assertion and performance threshold 3. Test it on our desktop 4. Commit, Clone and Repeat
  • 11.
    11 After Shifting Left Feature#1 Development time Feature #2 Development Time Feature #3 Development Time Feature #1 Test Development & Execution Feature #2 Test Development & Execution Feature #3 Test Development & Execution Integration Tests = New Feature + Regression + Performance Shippable Feature !
  • 12.
    12 After Shifting Left Technical debt doesn’t grow  Quality is embedded in every step of the SDLC  Immediate developer feedback  More tests are written as part of the SDLC  Faster ”Shippable product” with higher quality

Editor's Notes

  • #5 Test coverage @ speed But can your test coverage keep the same pace as your software without compromising the time to release? How do you achieve superior test coverage at speed? We can always use better tools. Tools that were built recently. However, simply using better tools will probably be more efficient, but definitely not significant. To see a significant impact, we have to practice the “Shift Left” paradigm. Shifting left will not add 20% efficiency, it will drive your test coverage to be 10 times as wide (or probably a 100 times).
  • #7 Change the definition of done – code + working tests
  • #11 Use recorder to capture view product call Delete everything except the API call Leave Headers there Save as Taurus Show YML file Delete all headers except content = JSON Switch to enhanced YML file Show variable for ID, assertion, threshold Run it in Taurus
  • #12 Squashing bugs mid sprint – not after handing to qa, no burn down charts Swarm to resolve
  • #13 Squashing bugs mid sprint – not after handing to qa, no burn down charts Swarm to resolve aim for technical debt to be zero