Negative Lifts:How a 24% loss produced a 141% increase in conversionEducational Funding Provided by:
Join the conversation on Twitter                       #webclinicFunding provided by:                        #webclinic
Today’s team                                               Jon Powell                       Dr. Flint McGlaughlin         ...
Background and test design                Experiment ID: RegOnline Homepage Test                 Location: MarketingExper...
Experiment: Control              Homepage Step 1      • Testimonials      • Lead form      • Screenshots                  ...
Experiment: Control                          Homepage Step 2                                            6Funding provided ...
Experiment: Treatment                       Homepage Step 1   • Clarified the Value Proposition     of the offer to emphas...
Experiment: Treatment                         Homepage Step 2                                           8Funding provided ...
Experiment: Treatment                         Homepage Step 3                                           9Funding provided ...
Experiment: Side-by-side                Control                Treatment                                                  ...
Experiment: Results                       24.5% Decrease in Conversion                       The Treatment generated 24.5%...
Experiment 2: Background and test design                Experiment ID: RegOnline Homepage Test                 Location: ...
Experiment 2: Side-by-side                Control                Treatment                                                ...
Experiment 2: Results                       89.8% Increase in Conversion                       The Treatment generated 89....
How did we turn this loss into a lift?    F       Key Principles         A. The goal of a test is not to get a lift, but r...
Part 1: The research question           Test Protocol Section 1             1. Question                                   ...
Definition: The research question              The Research Question:              A research question is a behavioral que...
Examples: The research question        Example #1:              Not this..                   What is the best price for pr...
The research question       ?       How would you refine the following research questions?                 1. What is the ...
What we discovered    F       Key Principles         A. The goal of a test is not to get a lift, but rather to get a learn...
Definition: The theory question              The Theory Question:              The theory question is a broader question a...
Chart: Marketers don’t validate results                                             22Funding provided by:                ...
Theory Question: Behavior reflects identity       Which?          Why?             What?  Research Question               ...
Theory Question: Behavior reflects identity       Which?                   Why?                  What?  Research Question ...
Theory Question: Behavior reflects identity       Which?                   Why?                  What?  Research Question ...
Theory Question: Behavior reflects identity       Which?                   Why?                  What?  Research Question ...
Theory Question: Behavior reflects identity       Which?                   Why?                  What?  Research Question ...
Example: The theory question              Research Question                               Theory Question     Which headli...
Example: The theory question              Research Question                              Theory Question     Which call to...
Theory Question: How we turned our loss into a win                       24%            Homepage                          ...
Theory Question: How we turned our loss into a win                        Why?                       24%            Homepa...
Theory Question: How we turned our loss into a win                        Why?                       24%            Homepa...
Theory Question: How we turned our loss into a win                        Why?                       24%            Homepa...
Theory Question: How we turned our loss into a win                        Why?                       24%                  ...
Theory Question: How we turned our loss into a win                        Why?                       Why?                 ...
Theory Question: How we turned our loss into a win                        Why?                       Why?                 ...
Theory Question: How we turned our loss into a win                        Why?                       Why?                 ...
The theory question applied                                201%             Test results are interpreted and a            ...
The theory question applied                                                     Again, test results are                   ...
The theory question applied                                                     Again, test results are                   ...
The theory question applied    • The discoveries and insights      about customer motivation                             4...
Summary: Putting it all together    F       Key Principles         A. The goal of a test is not to get a lift, but rather ...
Get a Free website grade from HubSpot       Get a FREE detailed report of how your site is       doing compared to your pe...
Testing Strategy Sessions                                     44Funding provided by:                        #webclinic
Strategy Session #1: QuoteScout.com                                         45Funding provided by:                       #...
Strategy Session #1: QuoteScout.com                                         46Funding provided by:                       #...
Strategy Session #1: QuoteScout.com                                         47Funding provided by:                       #...
Strategy Session #2: Warmup.com                                     48Funding provided by:                       #webclinic
Strategy Session #2: Warmup.com                                     49Funding provided by:                       #webclinic
Strategy Session #2: Warmup.com                                     50Funding provided by:                       #webclinic
Strategy Session #3: Clinics-trials-events.comFunding provided by:                       #webclinic
Strategy Session #3: Clinics-trials-events.comFunding provided by:                       #webclinic
Strategy Session #3: Clinics-trials-events.comFunding provided by:                       #webclinic
Strategy Session #3: Clinics-trials-events.com                                                    54Funding provided by:  ...
Strategy Session #4: Cooksdirect.com                                          55Funding provided by:                      ...
View the original presentation free  You can view the full audio  and visual presentation at  MarketingExperiments.com    ...
Upcoming SlideShare
Loading in...5
×

Negative Lifts: How a 24% loss produced a 141% increase in conversion

752

Published on

View the full replay here: http://www.marketingexperiments.com/site-optimization/negative-lifts.html

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
752
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
51
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 2 questions in animation.
  • Submitted by: JohnPrevious Testing: nonePrimary Objective: To drive traffic into our forms and convert leadsTraffic Source: Organic SEOIntended Audience: Consumers nationwide interested in multiple auto, home and life insurance quotes.
  • Submitted by: JohnPrevious Testing: nonePrimary Objective: To drive traffic into our forms and convert leadsTraffic Source: Organic SEOIntended Audience: Consumers nationwide interested in multiple auto, home and life insurance quotes.
  • Submitted by: JohnPrevious Testing: nonePrimary Objective: To drive traffic into our forms and convert leadsTraffic Source: Organic SEOIntended Audience: Consumers nationwide interested in multiple auto, home and life insurance quotes.
  • Submitted by: MaryPrevious Testing: nonePrimary Objective: To gain people's interestTraffic Source: ad wordsIntended Audience: end users; trades people and retailers
  • Submitted by: MaryPrevious Testing: nonePrimary Objective: To gain people's interestTraffic Source: ad wordsIntended Audience: end users; trades people and retailers
  • Submitted by: MaryPrevious Testing: nonePrimary Objective: To gain people's interestTraffic Source: ad wordsIntended Audience: end users; trades people and retailers
  • Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
  • Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
  • Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
  • Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
  • Submitted by: CandacePrevious Testing: none at this timePrimary Objective: To generate purchaseTraffic Source: most from organic searchIntended Audience: Professional foodservice buyers (i.e. restaurateurs, cafeteria mangers, etc)
  • Transcript of "Negative Lifts: How a 24% loss produced a 141% increase in conversion"

    1. 1. Negative Lifts:How a 24% loss produced a 141% increase in conversionEducational Funding Provided by:
    2. 2. Join the conversation on Twitter #webclinicFunding provided by: #webclinic
    3. 3. Today’s team Jon Powell Dr. Flint McGlaughlin Senior Research Managing Director ManagerFunding provided by: #webclinic
    4. 4. Background and test design  Experiment ID: RegOnline Homepage Test Location: MarketingExperiments Research Library Test Protocol Number: TP1427 Research Notes: Background: RegOnline is event management software that lets users create online registration forms and event websites to manage their events. Goal: To increase number of completed leads on home page. Primary research question: Which page will achieve the greatest addressable lead rate? Approach: A/B multifactor split testFunding provided by: #webclinic
    5. 5. Experiment: Control Homepage Step 1 • Testimonials • Lead form • Screenshots 5Funding provided by: #webclinic
    6. 6. Experiment: Control Homepage Step 2 6Funding provided by: #webclinic
    7. 7. Experiment: Treatment Homepage Step 1 • Clarified the Value Proposition of the offer to emphasize Free Access. • Reduced form fields in first step to capture more partial leads. • Reduced friction by matching flow to thought sequence. 7Funding provided by: #webclinic
    8. 8. Experiment: Treatment Homepage Step 2 8Funding provided by: #webclinic
    9. 9. Experiment: Treatment Homepage Step 3 9Funding provided by: #webclinic
    10. 10. Experiment: Side-by-side Control Treatment 10Funding provided by: #webclinic
    11. 11. Experiment: Results 24.5% Decrease in Conversion The Treatment generated 24.5% less completed leads Conversion Versions Rel. diff Rate Control – Two-step home page 2.3% - Treatment – Three-step home page 1.7% -24.5%  What you needformunderstand: In spite of the control stillvalue and reducing the amount of to fields in the first-step having clearer out-performed the treatment. 11Funding provided by: #webclinic
    12. 12. Experiment 2: Background and test design  Experiment ID: RegOnline Homepage Test Location: MarketingExperiments Research Library Test Protocol Number: TP1428 Research Notes: Background: RegOnline is event management software that lets users create online registration forms and event websites to manage their events. Goal: To increase number of completed leads on home page. Primary research question: Which process will yield a higher conversion rate? Approach: A/B multifactor split test 12Funding provided by: #webclinic
    13. 13. Experiment 2: Side-by-side Control Treatment 13Funding provided by: #webclinic
    14. 14. Experiment 2: Results 89.8% Increase in Conversion The Treatment generated 89.8% more completed leads Versions Conversion Rate Rel. diff Control 0.3% - Treatment 0.5% 89.8%  What you need toofunderstand: Using what we learned in previous tests, we directed the flow the visitor’s thought sequence through the value before revealing the price. This is what gave us our lift. Note: After further testing, we were finally able to increase conversion on the homepage by 141% 14Funding provided by: #webclinic
    15. 15. How did we turn this loss into a lift? F Key Principles A. The goal of a test is not to get a lift, but rather to get a learning B. To achieve the maximum amount of learning, your tests should be designed around two key elements: 1. A research question 15Funding provided by: #webclinic
    16. 16. Part 1: The research question Test Protocol Section 1 1. Question u = 2q + t + m+ 2v + i © 1. < Final research question… > 16Funding provided by: #webclinic
    17. 17. Definition: The research question The Research Question: A research question is a behavioral question expressed in a factorial split so it must begin with “which.”Funding provided by: #webclinic
    18. 18. Examples: The research question Example #1: Not this.. What is the best price for product X? But this… Which of these three price points is best for product X? Example #2: Not this.. Why am I losing customers in the last step of my checkout process? But this… Eliminating which form element best reduces customer drop-off? 18Funding provided by: #webclinic
    19. 19. The research question ? How would you refine the following research questions? 1. What is the best headline for my landing page? 2. How many objectives should I have on my homepage? 19Funding provided by: #webclinic
    20. 20. What we discovered F Key Principles A. The goal of a test is not to get a lift, but rather to get a learning B. To achieve the maximum amount of learning, your tests should be designed around two key elements: 1. A research question 2. A theory question 20Funding provided by: #webclinic
    21. 21. Definition: The theory question The Theory Question: The theory question is a broader question about your customer that is derived from asking “WHY” of the research question. The answer to the theory question should contribute to a wider understanding of your customer. Our researchers call this the Customer Theory.Funding provided by: #webclinic
    22. 22. Chart: Marketers don’t validate results 22Funding provided by: #webclinic
    23. 23. Theory Question: Behavior reflects identity Which? Why? What? Research Question Theory Question 23Funding provided by: #webclinic
    24. 24. Theory Question: Behavior reflects identity Which? Why? What? Research Question Theory Question Value Proposition Question: If I am your ideal customer, why should I buy from you rather than your competitors? 24Funding provided by: #webclinic
    25. 25. Theory Question: Behavior reflects identity Which? Why? What? Research Question Theory Question Value Proposition Question: If I am your ideal customer, why should I buy from you Ideal Customer rather than your competitors? 25Funding provided by: #webclinic
    26. 26. Theory Question: Behavior reflects identity Which? Why? What? Research Question Theory Question Value Proposition Question: If I am your ideal customer, why should I buy from you Ideal Customer rather than your competitors? 26Funding provided by: #webclinic
    27. 27. Theory Question: Behavior reflects identity Which? Why? What? Research Question Theory Question Value Proposition Question: If I am your ideal customer, why should I buy from you Ideal Customer rather than your competitors? 27Funding provided by: #webclinic
    28. 28. Example: The theory question Research Question Theory Question Which headline will generate a What does my ideal customer higher response? prefer? Why? A. Save $100 on product X A. Cost savings B. Product X comes with a B. Product quality lifetime guarantee 28Funding provided by: #webclinic
    29. 29. Example: The theory question Research Question Theory Question Which call to action will Where is my ideal customer in generate a higher response? the thought sequence? Why? A. Get your product X now A. Ready to buy? B. Learn more about product X B. Ready to learn more? 29Funding provided by: #webclinic
    30. 30. Theory Question: How we turned our loss into a win 24% Homepage 30Funding provided by: #webclinic
    31. 31. Theory Question: How we turned our loss into a win Why? 24% Homepage 31Funding provided by: #webclinic
    32. 32. Theory Question: How we turned our loss into a win Why? 24% Homepage SEO Landing Page 548% 32Funding provided by: #webclinic
    33. 33. Theory Question: How we turned our loss into a win Why? 24% Homepage SEO Landing Page 548% Why? 33Funding provided by: #webclinic
    34. 34. Theory Question: How we turned our loss into a win Why? 24% 90% Homepage Homepage SEO Landing Page 548% Why? 34Funding provided by: #webclinic
    35. 35. Theory Question: How we turned our loss into a win Why? Why? 24% 90% Homepage Homepage SEO Landing Page 548% Why? 35Funding provided by: #webclinic
    36. 36. Theory Question: How we turned our loss into a win Why? Why? 24% 90% 27% Homepage Homepage SEO Landing Page 548% Why? 36Funding provided by: #webclinic
    37. 37. Theory Question: How we turned our loss into a win Why? Why? 24% 90% 27% 141% Homepage Homepage Cumulative Lift SEO Landing Page 548% Why? 37Funding provided by: #webclinic
    38. 38. The theory question applied 201% Test results are interpreted and a second test was created based on the analyst’s observations 38Funding provided by: #webclinic
    39. 39. The theory question applied Again, test results are interpreted and the next round of testing is started for this page 201% 2% Test results are interpreted and a second test was created based on the analyst’s observations 39Funding provided by: #webclinic
    40. 40. The theory question applied Again, test results are interpreted and the next round of testing is started for this page 201% 2% 29% Test results are interpreted and a Test is again interpreted and second test was created based on transferrable principles are the analyst’s observations applied to other offer pages 40Funding provided by: #webclinic
    41. 41. The theory question applied • The discoveries and insights about customer motivation 451% from the three prior tests were applied to other landing pages and used to optimize PPC campaigns. • The purposeful effort to 302% identify and selectively apply these transferrable insights led to widespread optimization gains . 257% 28% 603% 41Funding provided by: #webclinic
    42. 42. Summary: Putting it all together F Key Principles A. The goal of a test is not to get a lift, but rather to get a learning B. To achieve the maximum amount of learning, your tests should be designed around two key elements: 1. A research question 2. A theory question 42Funding provided by: #webclinic
    43. 43. Get a Free website grade from HubSpot Get a FREE detailed report of how your site is doing compared to your peers and competitors in: -Generating Traffic -Search Engine Optimization -Social Engagement Visit the link below for more… Check Out HubSpot’s Website Grader bit.ly/wsgrader 43Funding provided by: #webclinic
    44. 44. Testing Strategy Sessions 44Funding provided by: #webclinic
    45. 45. Strategy Session #1: QuoteScout.com 45Funding provided by: #webclinic
    46. 46. Strategy Session #1: QuoteScout.com 46Funding provided by: #webclinic
    47. 47. Strategy Session #1: QuoteScout.com 47Funding provided by: #webclinic
    48. 48. Strategy Session #2: Warmup.com 48Funding provided by: #webclinic
    49. 49. Strategy Session #2: Warmup.com 49Funding provided by: #webclinic
    50. 50. Strategy Session #2: Warmup.com 50Funding provided by: #webclinic
    51. 51. Strategy Session #3: Clinics-trials-events.comFunding provided by: #webclinic
    52. 52. Strategy Session #3: Clinics-trials-events.comFunding provided by: #webclinic
    53. 53. Strategy Session #3: Clinics-trials-events.comFunding provided by: #webclinic
    54. 54. Strategy Session #3: Clinics-trials-events.com 54Funding provided by: #webclinic
    55. 55. Strategy Session #4: Cooksdirect.com 55Funding provided by: #webclinic
    56. 56. View the original presentation free You can view the full audio and visual presentation at MarketingExperiments.com View the original presentation >>Funding provided by: #webclinic
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×