This document outlines the data-driven process for conducting conversion optimization experiments on a website. It discusses determining key performance indicators, gathering insights from data, developing testable hypotheses about ways to improve conversions, prioritizing hypotheses, designing and testing experiments, and learning from the results. The goal of conversion optimization is to better understand customers and provide a better experience to simplify completing desired actions like signups, purchases or downloads. It emphasizes establishing a process, using data to inform hypotheses rather than assumptions, and continuously testing and improving based on results.

1. The data driven process
to identify, prioritize,
launch and monitor your
CRO experiments

2. What is conversion optimization?
Conversions – Actions you want your visitors to take on your site
◦ Signs up to your newsletter
◦ Buys a product
◦ Downloads a file
◦ Etc.
Conversion optimization – understand your customers, provide them with a better experience
on your site and simplify everything so they can convert more.

3. The Big Picture
Conversion optimization is about doing better
marketing
Assuming Knowing
Data

4. The Big Picture
“Data are just summaries of thousands of stories – tell a few of those stories to
help make the data meaningful.”
– Chip & Dan Heath
Data Information Insight

5. Don’t just assume
It makes an ass
out of u
and me

6. Mindset of the Optimizer
Rule #1: There is no one size fits all

7. Mindset of the Optimizer
There is no “this always works”
Ego and Bias – not your friends when it comes to optimization
We need to know, not assume

8. Typical CRO Process
If you can’t describe what you are doing
as a process, you don’t know what you’re
doing.
– W. Edwards Deming

9. Determine Business
Objective – KPIs,
Metrics to increase
Setup Data
Gathering
Insight Phase –
Sources of traffic,
patterns, personas
Define
Hypotheses
DesignTechnical IntegrationTesting
Learning and
Improving Customer
Theory
Back to Step 2
Prioritize
Hypotheses –
Testing Plan

10. Conversion Research 101
Experience based assessment
◦ Relevance
◦ Clarity
◦ Value
◦ Friction
◦ Distraction

11. Conversion Research 101
-Online surveys
-Web traffic surveys
-Live chat transcripts
-Customer support insights
-User testing
Qualitative Research

12. -Web analytics research
-Mouse tracking analytics
Quantitative Research

13. Developing a hypothesis
With each hypothesis we want to understand:
◦Which problem we’re solving
◦What’s the solution
◦Which metric we are trying to improve with this

14. Hypothesis Formula
By changing [problem] into [solution] we will increase [metric ]
All hypotheses should come as a result of conversion research: KNOW, don’t GUESS

15. Prioritizing Experiments
Optimizing the optimization process is often just as important as the tests themselves
WiderFunnel uses this PIE Framework:
PIE Framework:
Potential
Importance
Ease

16. T.I.R. Method
◦ Time 1 – longest time 5 – shortest time
◦ Impact 1 – lowest impact 5 – highest impact
◦ Resources 1 – most resources 5 – least resources
Multiply these scores together and start working on the projects that have the highest scores
as those will have the most lift with the least amount of time and the least amount of
resources.

17. Test It Right
Wait until statistical significance
Make sure your sample size is big enough
Sample size calculator: http://www.evanmiller.org/ab-testing/sample-size.html
Don’t call tests too early

18. Validity Threats
History effect
Instrumentation effect
Selection effect
Compatibility issues – broken code

19. Questions?
andra@marketizator.com