1.
How to Get Started With
Conversion Rate Optimization on
Your Website
P A M E L A V A U G H A N
@ p a m e l u m p

2.
Who here has ever heard of
imposter syndrome?

3.
the feeling that you are doing something you’re entirely
unqualified or untrained for, and any second everyone
around you is going to realize and rat you out
IMPOSTER
SYNDROME

4.
He’s an imposter!
#yousitonathroneoflies

5.
That was me a
couple of years ago.
@pamelump

6.
Company Blog
Speaker
Management
Award Submissions
PUBLIC RELATIONS
A LITTLE ABOUT ME
A N D M Y 1 0 - Y E A R T E N U R E A T H U B S P O T
Social Media Strategy
Monitoring
Content Creation
CONTENT + SOCIAL MEDIA
Strategy & Analytics
Editing
Content Creation
BLOGGING
Analytics
SEO
CRO
Historical Optimization
BLOG OPTIMIZATION
Copywriting
Testing & Analytics
CRO Strategy
WEBSITE OPTIMIZATION
2 0 0 8 - 2 0 1 1 2 0 0 9 - 2 0 1 1 2 0 1 1 - 2 0 1 3 2 0 1 3 - 2 0 1 5 2 0 1 5 - T O D A Y
@pamelump

7.
Company Blog
Speaker
Management
Award Submissions
PUBLIC RELATIONS
A LITTLE ABOUT ME
A N D M Y 1 0 - Y E A R T E N U R E A T H U B S P O T
CONTENT + SOCIAL MEDIA BLOGGING
Analytics
SEO
CRO
Historical Optimization
BLOG OPTIMIZATION WEBSITE OPTIMIZATION
2016 was the height
of my imposter
syndrome
2 0 0 8 - 2 0 1 1 2 0 0 9 - 2 0 1 1 2 0 1 1 - 2 0 1 3 2 0 1 3 - 2 0 1 5 2 0 1 5 - T O D A Y
@pamelump
Strategy & Analytics
Editing
Content Creation
Social Media Strategy
Monitoring
Content Creation
Copywriting
Testing & Analytics
CRO Strategy

8.
CEO & Co-Founder,
SEOmoz
RAND FISHKIN
Founder,
CXL
PEEP LAJA
Co-Founder,
Unbounce
OLI GARDNER
CEO, MobileMonkey
Founder, WordStream
LARRY KIM
CRO & Copywriting,
HubSpot
PAMELA VAUGHAN

9.
Why am I telling you this?

10.
@pamelump

11.
70%of people have experienced imposter
syndrome at some point in their lives.
I N T E R N A T I O N A L J O U R N A L O F B E H A V I O R A L S C I E N C E S , 2 0 1 1
@pamelump

12.
12
“I am not a writer. I’ve
been fooling myself
and other people.”
— J O H N S T E I N B E C K ,
A W A R D - W I N N I N G A U T H O R
E V E N T H I S G U Y !
@pamelump

13.
CRO is still a very
young industry.

14.
58%of CRO professionals have been in the
field for 2 years or fewer.
C X L 2 0 1 7 S T A T E O F T H E I N D U S T R Y R E P O R T
@pamelump

15.
Testing /
Statistics
User
ExperienceCopywritingAnalytics
Qualitative
Research
Web
Development
Visual Design
THE CRO UMBRELLA
C R O I S A M U L T I D I S C I P L I N A R Y F I E L D
Psychology
@pamelump

16.
In the past couple years, I’ve gotten over
my imposter syndrome.

17.
19% Increase in
Free Sign-Up CVR
OPTIMIZATION RESULTS THAT
PROVE I'M NOT A TOTAL HACK
2 0 1 7 - 2 0 1 8
74% Increase in
Chats Passed to
Sales Team
109% Increase in Free
Sign-Up CVR +
49% Increase in Hand-
Raiser CVR
15% Increase in Free
Sign-Up Conversion
Rate (CVR )
CRM Product Page
(CTA Copy Test)
Pricing Page
(Redesign)
Marketing Hub Product
Page (Chatbot Test)
Sales Hub Product Page
(Hero Copy Test)
@pamelump

18.
CRO IS STILL A VERY
YOUNG INDUSTRY
Don’t let imposter syndrome
hold you back.

19.
Ready to
get started with CRO?

20.
BUILD YOUR
DREAM TEAM
@pamelump

21.
Testing /
Statistics
User
ExperienceCopywritingAnalytics
Qualitative
Research
Web
Development
Visual Design
THE CRO UMBRELLA
C R O I S A M U L T I D I S C I P L I N A R Y F I E L D
Psychology
@pamelump

22.
22
CORE TEAM
Pamela Vaughan
CREATIVE PARTNERS
CRO/Copywriting
Rebecca Hinton
CRO/Copywriting
Hristo Kanchev
UX Design
Johannes Mehlem
Analytics
Anjana Aggarwal
Analytics
Kiona Highbridge
Visual Design
Kara Ebrahim
Web Development
Nhori Lopchan Tamang
Web Development
Marissa Okoli
Web Development
David Guerrera
Video
Vishu Singh
Project Management
MEET HUBSPOT’S WEB STRATEGY TEAM

23.
Testing
Copywriting
Qualitative Research
User Experience
Analytics
THE CRO TEAM OF THREE
UX Design
Visual Design
Web Development
R E S P O N S I B I L I T I E S
@pamelump

24.
Visual Design
THE CRO TEAM OF ONE
Testing Analytics
User Experience Web Development
Qualitative Research Copywriting
R E S P O N S I B I L I T I E S
@pamelump

25.
Multidisciplinary knowledge is
helpful even on larger teams where
roles are more specialized.

26.
CHOOSE YOUR
TOOL SET
@pamelump

27.
A/B TESTING TOOLS
• Part of HubSpot’s all-in-one
marketing software
• Ability to A/B test headlines, copy,
images, calls-to-action, and more
• Includes built-in analytics
@pamelump

28.
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

29.
• Integration / implementation with current ecosystem
(including analytics)
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

30.
• Integration / implementation with current ecosystem
(including analytics)
• Ease of use/setup
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

31.
• Integration / implementation with current ecosystem
(including analytics)
• Ease of use/setup
• Price
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

32.
• Integration / implementation with current ecosystem
(including analytics)
• Ease of use/setup
• Price
• Personalization capabilities
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

33.
• Integration / implementation with current ecosystem
(including analytics)
• Ease of use/setup
• Price
• Personalization capabilities
• Advanced features
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

34.
• Integration / implementation with current ecosystem
(including analytics)
• Ease of use/setup
• Price
• Personalization capabilities
• Advanced features
• Community and support access
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL

35.
• Integration / implementation with current ecosystem
(including analytics)
• Ease of use/setup
• Price
• Personalization capabilities
• Advanced features
• Community and support access
• Analytics/statistics engine
FACTORS TO
CONSIDER
WHEN
EVALUATING A
TESTING TOOL
@pamelump

36.
HUBSPOT’S TOOLSET
T E S T I N G + U X R E S E A R C H + A N A L Y T I C S
+ +
@pamelump

37.
IDENTIFY YOUR
OPTIMIZATION
OPPORTUNITIES
@pamelump

38.
Take “best practices”
with a grain of salt.

39.
What may work for one company …
May not work for another.

40.
Real-life example: the one- versus two-column form test.

41.
CONTROL VARIATION
@pamelump

42.
And the winner is …

43.
And the winner is …
The two-column form, which
converted 22% better than the
one-column form!

44.
45
“Two-column forms also
don’t typically perform as
well as a single column
form.”
“Test two-column forms with
single-column forms.”
— P E E P L A J A , F O U N D E R , C X L
E X P E R T S A G R E E

45.
USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE

46.
• Speed: How fast does your site load?
USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE

47.
• Speed: How fast does your site load?
• Focus: Are there elements distracting users away from
the desired goal (i.e. converting into a free user)?USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE

48.
• Speed: How fast does your site load?
• Focus: Are there elements distracting users away from
the desired goal (i.e. converting into a free user)?
• Clarity: Is it clear to users what you’re selling?
USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE

49.
• Speed: How fast does your site load?
• Focus: Are there elements distracting users away from
the desired goal (i.e. converting into a free user)?
• Clarity: Is it clear to users what you’re selling?
• Motivation: Are you appealing to the motivations of the
users who arrived on your website?
USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE

50.
• Speed: How fast does your site load?
• Focus: Are there elements distracting users away from
the desired goal (i.e. converting into a free user)?
• Clarity: Is it clear to users what you’re selling?
• Motivation: Are you appealing to the motivations of the
users who arrived on your website?
• Trust / Proof: How much do people trust your website /
company and what you’re offering?
USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE

51.
• Speed: How fast does your site load?
• Focus: Are there elements distracting users away from
the desired goal (i.e. converting into a free user)?
• Clarity: Is it clear to users what you’re selling?
• Motivation: Are you appealing to the motivations of the
users who arrived on your website?
• Trust / Proof: How much do people trust your website /
company and what you’re offering?
• Usability: How easy is it for people to use your website
and find / get what they’re looking for?
USE HEURISTICS
TO SPOT
OPPOURTUNITIES
IN YOUR SITE’S
PERFORMANCE
@pamelump

52.
Heuristics can help you identify areas where
your site is breaking down.

53.
CONDUCT A WEB
& CONVERSION
ANALYTICS
AUDIT

54.
• How much traffic are you getting to core site pages?
CONDUCT A WEB
& CONVERSION
ANALYTICS
AUDIT

55.
• How much traffic are you getting to core site pages?
• What is the average CTA clickthrough rate on your
core pages?
CONDUCT A WEB
& CONVERSION
ANALYTICS
AUDIT

56.
• How much traffic are you getting to core site pages?
• What is the average CTA clickthrough rate on your
core pages?
• What are users clicking on? How far do users scroll?
CONDUCT A WEB
& CONVERSION
ANALYTICS
AUDIT

57.
• How much traffic are you getting to core site pages?
• What is the average CTA clickthrough rate on your
core pages?
• What are users clicking on? How far do users scroll?
• Are certain pages converting better than others?
CONDUCT A WEB
& CONVERSION
ANALYTICS
AUDIT

58.
• How much traffic are you getting to core site pages?
• What is the average CTA clickthrough rate on your
core pages?
• What are users clicking on? How far do users scroll?
• Are certain pages converting better than others?
• Are certain sources of traffic or device types
converting better than others?
CONDUCT A WEB
& CONVERSION
ANALYTICS
AUDIT
@pamelump

59.
Heuristics and web analytics can
help you identify the “what?”
“where?” and “how much?”

60.
Qualitative user research can
help you understand the “why?”
@pamelump

61.
QUALITATIVE USER RESEARCH
M E T H O D S + T O O L S
USER TESTING
(MODERATED OR UNMODERATED)
Helps you explore “why” users are
doing what they’re doing on a given
page.
Example: “I see that users are clicking
on that box, but it doesn’t do anything.
Why are they clicking on it? What do
they expect it to do?”
CLICK TESTS, FLOW TESTS, PREFERENCE
TESTS, 5-SECOND TESTS
For the mix of “what” and “why”
questions, yielding a quantitative and
qualitative set of information. Doesn’t
require a working prototype or product
— you can run tests with a mere image
of a design.
Get a sense for where users will click,
how well they’ll convert, how usable the
design is, or even how aesthetically
appealing it is, all with a simple
wireframe or mockup.
USER SURVEYS
For widespread questions.
Our product marketing team regularly
uses this tool to construct messaging
strategies and determine ideal names
for product features
Can also be used to determine what
users think of your product, what
issues they encounter, etc.
@pamelump

62.
ANOTHER GREAT TOOL: CHAT TRANSCRIPTS

63.
“The CRM is free?
What’s the catch?”

64.
CRM CTA COPY TEST
“Get started free” vs. “Get free CRM”
CONTROL VARIATION
@pamelump

65.
VARIATION
15%INCREASE IN
CONVERSION
RATE
@pamelump

66.
GET YOUR
PRIORITIES
STRAIGHT
@pamelump

67.
PIE Framework for Prioritizing Tests
From WiderFunnel

68.
PIE Framework for Prioritizing Tests
From WiderFunnel
Potential
How much total impact can this test
have?

69.
PIE Framework for Prioritizing Tests
From WiderFunnel
Potential
How much total impact can this test
have?
Importance
How valuable will this improvement be?

70.
PIE Framework for Prioritizing Tests
From WiderFunnel
Potential
How much total impact can this test
have?
Importance
How valuable will this improvement be?
Ease
How difficult will it be to run this test?

71.
WHAT MAKES PIE
SO GREAT?

72.
• It’s delicious.
WHAT MAKES PIE
SO GREAT?

73.
• It’s delicious.
• It makes you take a step back and evaluate whether a
test you’re considering is actually worth running.
WHAT MAKES PIE
SO GREAT?

74.
• It’s delicious.
• It makes you take a step back and evaluate whether a
test you’re considering is actually worth running.
• It prevents you from wasting time on tests that won’t
move the needle.
WHAT MAKES PIE
SO GREAT?

75.
• It’s delicious.
• It makes you take a step back and evaluate whether a
test you’re considering is actually worth running.
• It prevents you from wasting time on tests that won’t
move the needle.
• It helps you prioritize the order in which you should
tackle experiments.
WHAT MAKES PIE
SO GREAT?

76.
• It’s delicious.
• It makes you take a step back and evaluate whether a
test you’re considering is actually worth running.
• It prevents you from wasting time on tests that won’t
move the needle.
• It helps you prioritize the order in which you should
tackle experiments.
• It forces you to use data in your decision-making so
you’re not arbitrarily assigning value to tests.
WHAT MAKES PIE
SO GREAT?
@pamelump

77.
It’s not perfect …
(Sometimes it’s really hard to
predict the potential lift of a test.)

78.
It’s not perfect …
(Sometimes it’s really hard to
predict the potential lift of a test.)
… but just going through the PIE scoring
exercise does serve as a good gut check.

79.
Build testing roadmaps.
(Good for planning a series of tests on the same page)
@pamelump

80.
Build testing roadmaps.
(Good for planning a series of tests on the same page)
Hero Copy
Test
CTA
Copy Test
Video
Test
Chatbot
Test
Hero
Design
Test
@pamelump

81.
But be agile.

82.
DESIGN YOUR
EXPERIMENTS

83.
Form your hypothesis.

84.
A SOLID
HYPOTHESIS
I S M A D E U P O F T W O T H I N G S …
Credit: Growth Team at Patreon

85.
AN INSIGHT
What you believe to be true based on
research, results from previous tests, etc.
Should include “we believe … because …”
A SOLID
HYPOTHESIS
I S M A D E U P O F T W O T H I N G S …
Credit: Growth Team at Patreon

86.
AN INSIGHT
What you believe to be true based on
research, results from previous tests, etc.
Should include “we believe … because …”
A PREDICTION
What you believe will happen if that
insight holds true.
A SOLID
HYPOTHESIS
I S M A D E U P O F T W O T H I N G S …
Credit: Growth Team at Patreon

87.
AN INSIGHT
What you believe to be true based on
research, results from previous tests, etc.
Should include “we believe … because …”
A PREDICTION
What you believe will happen if that
insight holds true.
A SOLID
HYPOTHESIS
I S M A D E U P O F T W O T H I N G S …
EXAMPLE
We believe people are skeptical our CRM
is actually free because it has been
mentioned repeatedly in live chat.
Credit: Growth Team at Patreon

88.
AN INSIGHT
What you believe to be true based on
research, results from previous tests, etc.
Should include “we believe … because …”
A PREDICTION
What you believe will happen if that
insight holds true.
A SOLID
HYPOTHESIS
I S M A D E U P O F T W O T H I N G S …
EXAMPLE
We believe people are skeptical our CRM
is actually free because it has been
mentioned repeatedly in live chat.
EXAMPLE
If we modify our CTA copy to minimize
that skepticism, we can generate more
free CRM users.
Credit: Growth Team at Patreon

89.
Weigh your testing options.

90.
SETTING UP YOUR TEST
M O S T C O M M O N T Y P E S …
@pamelump

91.
SETTING UP YOUR TEST
A/B/n Test
Also known as a “split test.” Used for
testing two or more variations of the
same web page.
The more variations you have, the
more traffic you’ll need to reach
statistically significance.
M O S T C O M M O N T Y P E S …
@pamelump

92.
SETTING UP YOUR TEST
A/B/n Test
Also known as a “split test.” Used for
testing two or more variations of the
same web page.
The more variations you have, the
more traffic you’ll need to reach
statistically significance.
Multivariate Test
Used for testing a hypothesis in which
multiple page elements are modified.
The goal of multivariate testing is to
determine which combination of
elements performs the best out of all of
the possible combinations.
Again, the more variations you have,
the more traffic you’ll need to reach
statistically significance.
M O S T C O M M O N T Y P E S …
@pamelump

93.
SETTING UP YOUR TEST
A/B/n Test
Also known as a “split test.” Used for
testing two or more variations of the
same web page.
The more variations you have, the
more traffic you’ll need to reach
statistically significance.
Multivariate Test
Used for testing a hypothesis in which
multiple page elements are modified.
The goal of multivariate testing is to
determine which combination of
elements performs the best out of all of
the possible combinations.
Again, the more variations you have,
the more traffic you’ll need to reach
statistically significance.
Redirect Test
Also known as a “split URL test.” Used
for testing separate web pages against
each other. Variations are identified by
URL or path instead of an element(s)
on the page.
Useful when you want to test two very
different landing pages, or a complete
redesign of a page.
M O S T C O M M O N T Y P E S …
@pamelump

94.
Understand the interaction between
sample size, test duration, and statistical significance.

95.
EXPERIMENT VALIDITY
I N T E R D E P E N D E N T F A C T O R S
@pamelump

96.
EXPERIMENT VALIDITY
Statistical Significance
The likelihood that the difference in
conversion rates between a given
variation and the baseline/control isn’t
due to random chance. Reflects your
risk tolerance and confidence level.
The accepted standard is 95%, which
means you can be 95% confident that
the results are real and not an error
caused by randomness.
I N T E R D E P E N D E N T F A C T O R S
@pamelump

97.
EXPERIMENT VALIDITY
Statistical Significance
The likelihood that the difference in
conversion rates between a given
variation and the baseline/control isn’t
due to random chance. Reflects your
risk tolerance and confidence level.
The accepted standard is 95%, which
means you can be 95% confident that
the results are real and not an error
caused by randomness.
Test Duration
How long you need to run the test in
order to generate valid results. You
should never base test validity on
statistical significance alone.
Rule of thumb: Run a test for no fewer
than 2 weeks and no longer than 4
weeks.
I N T E R D E P E N D E N T F A C T O R S
@pamelump

98.
EXPERIMENT VALIDITY
Statistical Significance
The likelihood that the difference in
conversion rates between a given
variation and the baseline/control isn’t
due to random chance. Reflects your
risk tolerance and confidence level.
The accepted standard is 95%, which
means you can be 95% confident that
the results are real and not an error
caused by randomness.
Test Duration
How long you need to run the test in
order to generate valid results. You
should never base test validity on
statistical significance alone.
Rule of thumb: Run a test for no fewer
than 2 weeks and no longer than 4
weeks.
Sample Size
The amount of visitors needed for a
test. More specifically, the amount of
visitors needed for each variation in the
test. Sample size calculators can help
you determine how much traffic you’ll
need depending on the expected
conversion lift and your statistical
significance threshold.
Optimizely has a simple one.
I N T E R D E P E N D E N T F A C T O R S
@pamelump

99.
“But … what if I don’t get
enough traffic?”

100.
OPTIMIZATION
TIPS FOR
LOW-TRAFFIC
SITES

101.
• Keep variants to a minimum.
OPTIMIZATION
TIPS FOR
LOW-TRAFFIC
SITES
@pamelump

102.
• Keep variants to a minimum.
• Test high-impact changes. The higher the impact, the
less traffic you need.
OPTIMIZATION
TIPS FOR
LOW-TRAFFIC
SITES
@pamelump

103.
• Keep variants to a minimum.
• Test high-impact changes. The higher the impact, the
less traffic you need.
• Switch your focus to micro conversions (e.g. measure
clickthrough rate instead of conversion rate).
OPTIMIZATION
TIPS FOR
LOW-TRAFFIC
SITES
@pamelump

104.
• Keep variants to a minimum.
• Test high-impact changes. The higher the impact, the
less traffic you need.
• Switch your focus to micro conversions (e.g. measure
clickthrough rate instead of conversion rate).
• Revisit your heuristics. You can optimize without testing.
OPTIMIZATION
TIPS FOR
LOW-TRAFFIC
SITES
@pamelump

105.
• Keep variants to a minimum.
• Test high-impact changes. The higher the impact, the
less traffic you need.
• Switch your focus to micro conversions (e.g. measure
clickthrough rate instead of conversion rate).
• Revisit your heuristics. You can optimize without testing.
• Consult “best practices.” They are based on some truth.
OPTIMIZATION
TIPS FOR
LOW-TRAFFIC
SITES
@pamelump

106.
Your test is done. Now what?

107.
ANALYZE
AND
ITERATE

108.
• Analyze results using a statistical significance calculator
to determine the winner. (VWO has a simple one.)
ANALYZE
AND
ITERATE

109.
• Analyze results using a statistical significance calculator
to determine the winner. (VWO has a simple one.)
• Really scrutinize the data: Question why the winning
variant won.
ANALYZE
AND
ITERATE

110.
• Analyze results using a statistical significance calculator
to determine the winner. (VWO has a simple one.)
• Really scrutinize the data: Question why the winning
variant won.
• Implement the winning variant on your site.
ANALYZE
AND
ITERATE

111.
• Analyze results using a statistical significance calculator
to determine the winner. (VWO has a simple one.)
• Really scrutinize the data: Question why the winning
variant won.
• Implement the winning variant on your site.
• Plan iterative tests based on insights learned.
ANALYZE
AND
ITERATE
@pamelump

112.
Don’t be discouraged by inconclusive results
(no statistical significance) …
… or failed tests (the control wins).

113.
Don’t be discouraged by inconclusive results
(no statistical significance) …
… or failed tests (the control wins).
Learn from them.

114.
LEVERAGE
INSIGHTS FROM
PREVIOUS
TESTS
@pamelump

115.
The more tests you run …

116.
The more tests you run …
The more insights you’ll collect …

117.
The more tests you run …
The more insights you’ll collect …
The more winning results you’ll get.

118.
Be diligent about
documentation.

119.
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

120.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

121.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
• Test Design: Was it an A/B/n test? Multivariate? What
variable(s) were manipulated?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

122.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
• Test Design: Was it an A/B/n test? Multivariate? What
variable(s) were manipulated?
• Test Dates & Duration: How long did you run the test
for? On what specific dates?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

123.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
• Test Design: Was it an A/B/n test? Multivariate? What
variable(s) were manipulated?
• Test Dates & Duration: How long did you run the test
for? On what specific dates?
• Test Page(s): What was the URL of the page or pages
where you ran your test?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

124.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
• Test Design: Was it an A/B/n test? Multivariate? What
variable(s) were manipulated?
• Test Dates & Duration: How long did you run the test
for? On what specific dates?
• Test Page(s): What was the URL of the page or pages
where you ran your test?
• Goal Metric(s): What KPI(s) did you measure your
results against?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

125.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
• Test Design: Was it an A/B/n test? Multivariate? What
variable(s) were manipulated?
• Test Dates & Duration: How long did you run the test
for? On what specific dates?
• Test Page(s): What was the URL of the page or pages
where you ran your test?
• Goal Metric(s): What KPI(s) did you measure your
results against?
• Screenshots: What did the control and each variation
look like?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …

126.
• Hypothesis: What research/insight motivated you to run
this test? What did you predict the results will be?
• Test Design: Was it an A/B/N test? Multivariate? What
variable(s) were manipulated?
• Test Dates & Duration: How long did you run the test
for? On what specific dates?
• Test Page(s): What was the URL of the page or pages
where you ran your test?
• Goal Metric(s): What KPI(s) did you measure your
results against?
• Screenshots: What did the control and each variation
look like?
• Results & Insights: How did each variation perform
against your goal metrics? Were the results statistically
significant? What did you learn?
EXPERIMENT
DOCUMENTATION
SHOULD
INCLUDE …
@pamelump

127.
EXAMPLE
CONTROL VARIATION
Remember this test?
@pamelump

128.
VARIATION
15%INCREASE IN
CONVERSION
RATE
@pamelump

129.
WE APPLIED THIS INSIGHT TO THE HOMEPAGE …

130.
WE APPLIED THIS INSIGHT TO THE HOMEPAGE …
9% increase in conversion rate

131.
TO
REVIEW
@pamelump

132.
• Pie is delicious.
TO
REVIEW
@pamelump

133.
• Pie is delicious.
• Testing is just one piece of the pie. CRO is a whole,
multidisciplinary process.
TO
REVIEW
@pamelump

134.
• Pie is delicious.
• Testing is just one piece of the pie. CRO is a whole,
multidisciplinary process.
• Base your testing on real insights — not whims.TO
REVIEW
@pamelump

135.
• Pie is delicious.
• Testing is just one piece of the pie. CRO is a whole,
multidisciplinary process.
• Base your testing on real insights — not whims.
• Analytics and user research are gold mines for
discovering optimization opportunities.
TO
REVIEW
@pamelump

136.
• Pie is delicious.
• Testing is just one piece of the pie. CRO is a whole,
multidisciplinary process.
• Base your testing on real insights — not whims.
• Analytics and user research are gold mines for
discovering optimization opportunities.
• Be thoughtful about test setup and validity.
TO
REVIEW
@pamelump

137.
• Pie is delicious.
• Testing is just one piece of the pie. CRO is a whole,
multidisciplinary process.
• Base your testing on real insights — not whims.
• Analytics and user research are gold mines for
discovering optimization opportunities.
• Be thoughtful about test setup and validity.
• Be diligent about documentation.
TO
REVIEW
@pamelump

138.
• Pie is delicious.
• Testing is just one piece of the pie. CRO is a whole,
multidisciplinary process.
• Base your testing on real insights — not whims.
• Analytics and user research are gold mines for
discovering optimization opportunities.
• Be thoughtful about test setup and validity.
• Be diligent about documentation.
• Imposter syndrome is real … but get over it.
TO
REVIEW
@pamelump

139.
If you were feeling like this before …

140.
I hope you’re feeling
more like this now.

141.
@pamelump