Downloaded 11 times
Uploaded byOptimizely
Fail and Win: Why a Failed Test Isn’t a Bad Thing
Caleb Whitmore discusses the significance of testing in improving outcomes, emphasizing that both successful and failed tests provide valuable insights. He advocates for a 360-degree view of testing impacts across user interactions and highlights the importance of measuring diverse metrics. Ultimately, he suggests that both failings and victories in testing should be viewed as part of a successful process when aligned with clear objectives.
More Related Content
![[Elite Camp 2016] Peep Laja - Fresh Out Of the Oven](https://cdn.slidesharecdn.com/ss_thumbnails/08elitecamp-peeplaja-freshoutoftheoven-160708115231-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Elite Camp 2016] Craig Sullivan - Elite Camp Summary Session](https://cdn.slidesharecdn.com/ss_thumbnails/18elitecamp-craigsullivan-roundup-160708121125-thumbnail.jpg?width=640&height=640&fit=bounds)
![[CXL Live 16] You Can’t Make This Stuff Up by Alex Harris](https://cdn.slidesharecdn.com/ss_thumbnails/r4uxo85stoa2ezcmn5xo-signature-42db88fde2139044dd6963018b134da7e709739587431e34ce8d4285185c8547-poli-160403200010-thumbnail.jpg?width=640&height=640&fit=bounds)
![[CXL Live 16] What to Test Next - Prioritizing Your Tests by Pauline Marol](https://cdn.slidesharecdn.com/ss_thumbnails/9lbrkitrioxqyhzpxuwd-signature-42db88fde2139044dd6963018b134da7e709739587431e34ce8d4285185c8547-poli-160403200010-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Fail and Win: Why a Failed Test Isn’t a Bad Thing
Fail and Win: Why a Failed Test Isn’t a Bad Thing
- 1. Fail and Win: Why a Failed Test Isn’ Caleb Whitmore Founder & CEO Analy<cs Pros @CalebWhitmore
- 2.
- 3. I climbed a mountain… once.
- 4.
- 5. And get to work with some awesome clients!
- 6.
- 7. Why Test? TESTING… • Should focus on improving an outcome • Is the shortest distance between you and improvements • Is done best with rapid itera<on and lots of data
- 8. Some things may seem obvious to test…
- 9.
- 10.
- 11. But you should s=ll test them!
- 12. Tes=ng depends on data
- 13.
- 14.
- 15. Not all tests are created equally
- 16. Some tests will win
- 17. Some tests will fail
- 18. Is winning a success?
- 19. Is failing a loss?
- 20. Tes=ng is a Success When: • A process is followed • An outcome is measured • A complete picture is seen
- 21. When Things Shake, What Moves?
- 22. When Things Shake, What Moves?
- 23. Crea=ng a 360 Degree View • A test will impact more than what you intended • You must measure the impact across your spectrum of user interac<on • Measure your test with your exis<ng infrastructure
- 24. Crea=ng a 360 Degree View What happens to… • The “key” conversion point • Engagement on the test page • Length of the en<re visit • Total pageviews/interac<ons in the visit • Secondary conversion goals – view product, view content, start checkout, download a file, etc…
- 25.
- 26. Example Experiment ASSUMPTIONS • Auto-‐play repels visitors – Audio without warning • Thus, bounce rates are high and more people leave • Turn off auto play & more people will stay and engage
- 27. Op=mizely Shows a Winner
- 28.
- 29.
- 30.
- 31. Setup a 360 Degree View of Tests • Tip: your Digital Analy<cs plaXorm has this view already!
- 32.
- 33. Pipes data straight into GA
- 34. Is this a good outcome?
- 35.
- 36. See the full picture
- 37. Capture Experiment Data • Tie experiment data into your Tag Management and Digital Data frameworks
- 38. Leverage Built-‐in Repor=ng • Create lots of Goals in Op<mizely – pages, clicks, revenue, etc…
- 39. QUESTIONS? @CalebWhitmore of @Analy<csPros
- 40. Fail and Win: Why a Failed Test Isn’t a Bad Thing Caleb Whitmore Founder & CEO Analy<cs Pros @CalebWhitmore