View the full replay here: http://www.marketingexperiments.com/site-optimization/negative-lifts.html

1. Negative Lifts:
How a 24% loss produced a 141% increase in conversion
Educational Funding Provided by:

2. Join the conversation on Twitter
#webclinic
Funding provided by:
#webclinic

3. Today’s team
Jon Powell
Dr. Flint McGlaughlin
Senior Research
Managing Director
Manager
Funding provided by:
#webclinic

4. Background and test design
 Experiment ID: RegOnline Homepage Test
Location: MarketingExperiments Research Library
Test Protocol Number: TP1427
Research Notes:
Background: RegOnline is event management software that lets users create
online registration forms and event websites to manage their events.
Goal: To increase number of completed leads on home page.
Primary research question: Which page will achieve the greatest addressable
lead rate?
Approach: A/B multifactor split test
Funding provided by:
#webclinic

5. Experiment: Control Homepage Step 1
• Testimonials
• Lead form
• Screenshots
5
Funding provided by:
#webclinic

6. Experiment: Control
Homepage Step 2
6
Funding provided by:
#webclinic

7. Experiment: Treatment Homepage Step 1
• Clarified the Value Proposition
of the offer to emphasize Free
Access.
• Reduced form fields in first step
to capture more partial leads.
• Reduced friction by matching
flow to thought sequence.
7
Funding provided by:
#webclinic

8. Experiment: Treatment
Homepage Step 2
8
Funding provided by:
#webclinic

9. Experiment: Treatment
Homepage Step 3
9
Funding provided by:
#webclinic

10. Experiment: Side-by-side
Control Treatment
10
Funding provided by:
#webclinic

11. Experiment: Results
24.5% Decrease in Conversion
The Treatment generated 24.5% less completed leads
Conversion
Versions Rel. diff
Rate
Control – Two-step home page 2.3% -
Treatment – Three-step home page 1.7% -24.5%
 What you needformunderstand: In spite of the control stillvalue and reducing
the amount of
to
fields in the first-step
having clearer
out-performed
the treatment.
11
Funding provided by:
#webclinic

12. Experiment 2: Background and test design
 Experiment ID: RegOnline Homepage Test
Location: MarketingExperiments Research Library
Test Protocol Number: TP1428
Research Notes:
Background: RegOnline is event management software that lets users create
online registration forms and event websites to manage their events.
Goal: To increase number of completed leads on home page.
Primary research question: Which process will yield a higher conversion rate?
Approach: A/B multifactor split test
12
Funding provided by:
#webclinic

13. Experiment 2: Side-by-side
Control Treatment
13
Funding provided by:
#webclinic

14. Experiment 2: Results
89.8% Increase in Conversion
The Treatment generated 89.8% more completed leads
Versions Conversion Rate Rel. diff
Control 0.3% -
Treatment 0.5% 89.8%
 What you need toofunderstand: Using what we learned in previous tests, we
directed the flow the visitor’s thought sequence through the value
before revealing the price. This is what gave us our lift.
Note: After further testing, we were finally able to increase conversion on
the homepage by 141%
14
Funding provided by:
#webclinic

15. How did we turn this loss into a lift?
F Key Principles
A. The goal of a test is not to get a lift, but rather to get a learning
B. To achieve the maximum amount of learning, your tests should be designed
around two key elements:
1. A research question
15
Funding provided by:
#webclinic

16. Part 1: The research question
Test Protocol Section 1
1. Question u = 2q + t + m+ 2v + i ©
1. < Final research question… >
16
Funding provided by:
#webclinic

17. Definition: The research question
The Research Question:
A research question is a behavioral question expressed in a factorial
split so it must begin with “which.”
Funding provided by:
#webclinic

18. Examples: The research question
Example #1:
Not this..
What is the best price for product X?
But this…
Which of these three price points is best for product X?
Example #2:
Not this..
Why am I losing customers in the last step of my checkout process?
But this…
Eliminating which form element best reduces customer drop-off?
18
Funding provided by:
#webclinic

19. The research question
? How would you refine the following research questions?
1. What is the best headline for my landing page?
2. How many objectives should I have on my homepage?
19
Funding provided by:
#webclinic

20. What we discovered
F Key Principles
A. The goal of a test is not to get a lift, but rather to get a learning
B. To achieve the maximum amount of learning, your tests should be designed
around two key elements:
1. A research question
2. A theory question
20
Funding provided by:
#webclinic

21. Definition: The theory question
The Theory Question:
The theory question is a broader question about your customer that is
derived from asking “WHY” of the research question.
The answer to the theory question should contribute to a wider
understanding of your customer.
Our researchers call this the Customer Theory.
Funding provided by:
#webclinic

22. Chart: Marketers don’t validate results
22
Funding provided by:
#webclinic

23. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
23
Funding provided by:
#webclinic

24. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you
rather than your competitors?
24
Funding provided by:
#webclinic

25. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you Ideal Customer
rather than your competitors?
25
Funding provided by:
#webclinic

26. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you Ideal Customer
rather than your competitors?
26
Funding provided by:
#webclinic

27. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you Ideal Customer
rather than your competitors?
27
Funding provided by:
#webclinic

28. Example: The theory question
Research Question Theory Question
Which headline will generate a What does my ideal customer
higher response? prefer?
Why?
A. Save $100 on product X A. Cost savings
B. Product X comes with a B. Product quality
lifetime guarantee
28
Funding provided by:
#webclinic

29. Example: The theory question
Research Question Theory Question
Which call to action will Where is my ideal customer in
generate a higher response? the thought sequence?
Why?
A. Get your product X now A. Ready to buy?
B. Learn more about product X B. Ready to learn more?
29
Funding provided by:
#webclinic

30. Theory Question: How we turned our loss into a win
24%
Homepage
30
Funding provided by:
#webclinic

31. Theory Question: How we turned our loss into a win
Why?
24%
Homepage
31
Funding provided by:
#webclinic

32. Theory Question: How we turned our loss into a win
Why?
24%
Homepage
SEO Landing Page
548%
32
Funding provided by:
#webclinic

33. Theory Question: How we turned our loss into a win
Why?
24%
Homepage
SEO Landing Page
548%
Why?
33
Funding provided by:
#webclinic

34. Theory Question: How we turned our loss into a win
Why?
24% 90%
Homepage Homepage
SEO Landing Page
548%
Why?
34
Funding provided by:
#webclinic

35. Theory Question: How we turned our loss into a win
Why? Why?
24% 90%
Homepage Homepage
SEO Landing Page
548%
Why?
35
Funding provided by:
#webclinic

36. Theory Question: How we turned our loss into a win
Why? Why?
24% 90% 27%
Homepage Homepage
SEO Landing Page
548%
Why?
36
Funding provided by:
#webclinic

37. Theory Question: How we turned our loss into a win
Why? Why?
24% 90% 27%
141%
Homepage Homepage
Cumulative Lift
SEO Landing Page
548%
Why?
37
Funding provided by:
#webclinic

38. The theory question applied
201%
Test results are interpreted and a
second test was created based on
the analyst’s observations
38
Funding provided by:
#webclinic

39. The theory question applied
Again, test results are
interpreted and the next
round of testing is started
for this page
201% 2%
Test results are interpreted and a
second test was created based on
the analyst’s observations
39
Funding provided by:
#webclinic

40. The theory question applied
Again, test results are
interpreted and the next
round of testing is started
for this page
201% 2% 29%
Test results are interpreted and a Test is again interpreted and
second test was created based on transferrable principles are
the analyst’s observations applied to other offer pages
40
Funding provided by:
#webclinic

41. The theory question applied
• The discoveries and insights
about customer motivation 451%
from the three prior tests were
applied to other landing pages
and used to optimize PPC
campaigns.
• The purposeful effort to 302%
identify and selectively apply
these transferrable insights led
to widespread optimization
gains .
257% 28% 603%
41
Funding provided by:
#webclinic

42. Summary: Putting it all together
F Key Principles
A. The goal of a test is not to get a lift, but rather to get a learning
B. To achieve the maximum amount of learning, your tests should be designed
around two key elements:
1. A research question
2. A theory question
42
Funding provided by:
#webclinic

43. Get a Free website grade from HubSpot
Get a FREE detailed report of how your site is
doing compared to your peers and competitors in:
-Generating Traffic
-Search Engine Optimization
-Social Engagement
Visit the link below for more…
Check Out HubSpot’s
Website Grader
bit.ly/wsgrader
43
Funding provided by:
#webclinic

44. Testing Strategy Sessions
44
Funding provided by:
#webclinic

45. Strategy Session #1: QuoteScout.com
45
Funding provided by:
#webclinic

46. Strategy Session #1: QuoteScout.com
46
Funding provided by:
#webclinic

47. Strategy Session #1: QuoteScout.com
47
Funding provided by:
#webclinic

48. Strategy Session #2: Warmup.com
48
Funding provided by:
#webclinic

49. Strategy Session #2: Warmup.com
49
Funding provided by:
#webclinic

50. Strategy Session #2: Warmup.com
50
Funding provided by:
#webclinic

51. Strategy Session #3: Clinics-trials-events.com
Funding provided by:
#webclinic

52. Strategy Session #3: Clinics-trials-events.com
Funding provided by:
#webclinic

53. Strategy Session #3: Clinics-trials-events.com
Funding provided by:
#webclinic

54. Strategy Session #3: Clinics-trials-events.com
54
Funding provided by:
#webclinic

55. Strategy Session #4: Cooksdirect.com
55
Funding provided by:
#webclinic

56. View the original presentation free
You can view the full audio
and visual presentation at
MarketingExperiments.com
View the original presentation >>
Funding provided by:
#webclinic