This document summarizes tips and best practices for conversion rate optimization (CRO) testing based on a book review. It recommends following a structured process that involves analyzing site usage data to identify problem areas, hypothesizing solutions, testing solutions, and learning from results. Key steps include using heat maps and surveys to understand user behavior, testing specific hypotheses with a sufficient sample size, prioritizing opportunities based on potential impact and ease of implementation, and viewing CRO as an ongoing, data-driven process rather than a one-time fix. The overall message is that an effective, professional approach to CRO requires understanding user needs, having a methodical testing methodology, and being willing to learn from tests that do not produce desired results

1. Book Review #3
Ryan Dekmar
Mktg Analytics: 491
Spring 2015

2. Amateurs
Frameworks & processes
...you don't know what
you're doing.”
“If you can't describe what
you are doing as a process...

3. “You start by figuring out which pages cause the biggest drop-
offs - where the flow is stuck. Once you understand WHERE the
problem is, you proceed to identifying WHAT the problem is.”
“You seek to understand your customers better - their
needs, sources of hesitation, conversations going on inside
their minds.”

4. HOW?
Let go of your ego and
understand...
Your opinion doesn’tYour opinion doesn’t
mattermatter
You don’t know whatYou don’t know what
will workwill work
There are no magicThere are no magic
templatestemplates
Instead...

5. ResearchXL
Technical analysis: Utilize services to find bottlenecks.
Heuristic analysis: Experience for yourself what works.
•Relevancy
•Clarity
•Value
•Limited Friction
•Limited Distractions
The site should have:

6. And also when
you use...
Analyze: get a better picture of who, what, and how
much.

7. Heat Maps
Click Maps
Attention Maps
If you discover something (image, sentence etc) that people
want to click on, but isn't a link, then:
A) make it into a link
B) don't make it look like a link.
Mouse Tracking

8. Exit surveys: hit them with a popup when they're about to
leave your site.
On-page surveys:
ask them to fill out
a survey as they're
on a specific page.
*Looking for actionable and sufficient data*

9. Observe actual people use and interact with your website.
• A specific task
• A broad task
• Funnel completion
Include 3 types of tasks in
your test protocol:

10. Allocate the
findings from
steps
1 through 6
Test Instrument
Hypothesize
Just Do It
Investigate
Need more info
Easy fix
No clear problem
No clear solution
Still need initial testing

11. “We can’t do everything at once and hence need to prioritize.”
Opportunity score
(subjective opinion on
how big of a lift you
might get).
Ease of implementation
(time/ complexity/ risk).

12. “You need to make
sure your sample
size is big enough.”
“Most researchers use the 95% confidence level before
making any conclusions.”
Understand statistical confidence!
“Don't confuse
statistical significance
with validity.”

13. “The odds of you solving all problems with the first test are slim!”
Always test a
specific hypothesis!
In short: Learn from the failure.
“If you genuinely
have a test
which failed to
move any
segments, it’s a
crap test.”

14. Conversion optimization is a process. “Anyone who is not
able to describe their CRO work as a systematic,
repeatable process is a complete amateur.”