Your SlideShare is downloading. ×
0
Larry Kim and Neil Patel
The 10 Weirdest A/B Tests
Guaranteed to Double Your
Business Growth
March 2014
@larrykim@neilpatel
#KISSwebinar
Join us on Twi er
Neil Patel
KISSmetrics


@neilpatel
Larry Kim
WordStream


@larrykim
Larry Kim is the founder and CTO at WordStream
where ...
1 Larry’s Weirdest A/B Tests
2 Neil’s Weirdest A/B Tests
Table of Contents
WATCH THIS WEBINAR RECORDING NOW
Larry’s Weirdest A/B Test
1
The A/B Landing Page Optimization Gurus Have it Wrong
Warning: Lots of Data Ahead!
Weirdest A/B Test #1:

The Classic A/B

Conversion Rate
Optimization Test Is
Often Quite Silly …
We changed the:
• Font Type
• Spacing
• Button Color
• Image
• Etc.
We got a 5% Increase in conversions!!
The Great Landin...
• The Early Lead Disappears!
• We want to believe our hard work paid off
but we are often deluding ourselves…
Typical Conv...
• Small Changes Result in Small Changes
• The x% Increase you think you got often isn’t
significant!
Conventional Landing ...
Re-Arrange Deck Chairs On the Titanic and ignore sinking
Weirdest Test #2:

So … What’s a Good
Conversion Rate?
What’s a Good Conversion Rate?
What’s a Good Conversion Rate?
Survey Methodology
Even the “Winner” is a Loser!
CONFIDENTIAL – DO NOT DISTRIBUTE
16
Trees vs. Forest
Distribution
Point
Conversion Rate Vs Average Comments
Average 2.35% Unremarkable
Top 25% 5.31% 2x Awesome
Top 10% 11.45% ...
Distribution
Point
All accounts Ecommerce Legal B2B Finance
Median CVR 2.35% 1.84% 2.07% 2.23% 5.01%
Top 25% CVR 5.31% 3.7...
Aim For 2-4x Increases Not 2-4%!
Landing Page Unicorns?!
(+3x Avg. Conversion Rate!)
Weirdest A/B Testing
Epiphany #3: 

What Do Landing Page
Unicorns 

Look Like?
@larrykim, @perrymarshall
Hunting Landing Page Unicorns
1. Changing The Offer
The New Offer:
CONFIDENTIAL – DO NOT DISTRIBUTE
24
The New Offer
CONFIDENTIAL – DO NOT DISTRIBUTE
25
• Average or Below Average Conversion Rates
• Ask Your Customers!
How to Know if your ...
2. Change The Flow (Reverse Funnel)
2. New Flow: Registration At End
2. New Flow: Quality vs. Quantity
New Flow: Quality vs. Quantity
Another Example of Changing The Flow
CONFIDENTIAL – DO NOT DISTRIBUTE
30
Changing the Flow: Let Them Choose
of people who visit a website
leave without completing 

the actions marketers want
them to take
96%
Your Ad
Your%Site
X
-...
X
-
Users who have 

visited your web site
Bring them back to
your website
Opportunity to …
• Turn abandoners into leads
•...
Weirdest A/B Testing
Epiphany #4

You Need To Run 10 A/B
Tests To Find 1 Unicorn
Name Percentile Relative
Abundance
Vs. Expected
Conversion
Rate
Awesome
Landing Pages
Top 25% 1:4 2x Higher
Unicorns Top 1...
But I Have 10 Landing Pages!
Larry Kim (@larrykim)
This is the same landing page with different spacing…
Weirdest A/B Testing
Epiphany #5

It’s Not as Hard As You
Think
80% of Traffic Goes to 10% of Landing Pages
The Bar for Landing Page Excellence is (Very) Low
You Don’t Need Dozens of Landing Pages…
But You Don’t Need Dozens of Landing Pages…
1. Most Landing Page Optimization =
Rearranging Deck Chairs, Small %
Changes (Often Insignificant)
2. Insane Landing Page ...
LOG IN WITH GOOGLE
Start Your Free KISSmetrics Trial
Neil’s Weirdest A/B Test
2
6. Use GEO based data throughout your funnel
!
Test Six
Don’t forget about these steps
!
Test Six
Results
!
!
!
!
20% increase in leads and 68% improvement
in qualified leads as people felt the solution
was more customize...
!
!
!
We found that targeting to a specific city
decreased conversions by 19% as people felt
creeped out.
Section OneWhat t...
7. Try using authentication
Test Seven
!
!
!
By pulling log in details from Google, we
increased signups by 59.4%.
Section OnePeople are lazy
Test sign in versus sign up
Test Seven
!
!
!
Difference between sign up and sign in was.
22% Total conversion increase was a 94%.
Section OneWords ma er
Focus on what’s new
Test Seven
•Using authentication at the end of your funnel
isn’t as effective.
•You have to analyze your drop off rates per step
to figu...
8. Quizzes are more effective for lead generation
!
Test Eight
Use interactive quizzes
!
!
Test Eight
Difference was a 281% increase in leads
Section OnePeople love games
Make sure you use images
!
!
!
Test Eight
•Make your quiz 5 questions
•Use images when possible
•Limit the choices per question to a
maximum of 4 options
Section On...
9. Free trials can increase cash flow
!
!
!
Test Nine
•Increase cash flow by 60%
•Decreased churn by 41%
Section OneThe data
Section OneWatch out for processing issues
By refunding anyone who complains and by
charging a dollar for the trial (then ...
10. Double your revenue through 3 upsells
!
!
!
!
Test Ten
Make your first product affordable
Test Ten
Section OneTips
•Make you first upsell 3x in price
•Make the second upsell 1.5x in price
•Make the 3rd upsell 2x in price
Section OneDon’t make people re-enter their CC details
By asking for CC details on each step you will
see a decrease in re...
Questions?
Shoot and we will do our best
to answer
THANK YOU
Larry Kim and Neil Patel
@larrykim @neilpatel
Upcoming SlideShare
Loading in...5
×

The 10 Weirdest A/B Tests Guaranteed to Double Your Business Growth

11,339

Published on

Neil Patel is the co-founder of KISSmetrics, Crazy Egg and Hello Bar. He helps companies like Amazon, NBC, GM, HP, and Viacom grow their revenue. The Wall Street Journal calls him a top influencer on the web and Entrepreneur Magazine says he created one of the 100 most brilliant companies in the world. Larry Kim is the founder and CTO at WordStream where he is contributor to both the product team and marketing teams. Larry is committed to helping small businesses learn how to grow their businesses through search marketing.
The A/B Landing Page Optimization Gurus Have it Wrong. The Classic A/B
 Conversion Rate Optimization Test Is Often Quite Silly. We changed the: • Font Type • Spacing • Button Color • Image • Etc. We got a 5% Increase in conversions!! The Great Landing Page Optimization Fairy Tale • We want to believe our hard work paid off but we are often deluding ourselves… Typical Conversion Rate Optimization Test
• Small Changes Result in Small Changes • The X% Increase you think you got often isn’t significant! Conventional Landing Page Optimization is Over-Rated
So … What’s a Good Conversion Rate?
Survey Methodology. Distribution Point Conversion Rate Vs. Average Comments Average 2.35% Unremarkable Top 25% 5.31% 2x Awesome Top 10% 11.45% 5x Unicorns Unremarkable vs. Unicorns. Distribution Point All accounts Ecommerce Legal B2B Finance Median CVR 2.35% “Top 10% of accounts have a CVR 3x average” rule
Aim For 2-4x Increases Not 2-4%!
• Average or Below Average Conversion Rates • Ask Your Customers! How to Know if your Offer Stinks?
Change The Flow (Reverse Funnel)
New Flow: Registration At End. Changing the Flow: Let Them Choose. No. of people who visit a website leave without completing. X - Users who have visited your web site Bring them back to your website Opportunity to. Why I View Remarketing as CRO*
You Need To Run 10 A/B Tests To Find 1 Unicorn. Percentile Relative Abundance Vs. Expected Conversion Rate. This is the same landing page with different spacing. 80% of Traffic Goes to 10% of Landing Pages
The Bar for Landing Page Excellence is (Very) Low
Use GEO based data throughout your funnel!
Results! 20% increase in leads and 68% improvement in qualified leads as people felt the solution was more customized to them.
We found that targeting to a specific city decreased conversions by 19% as people felt creeped out. What to avoid
Try using authentication. Pulling log in details from Google. Difference between sign up and sign in was. 22% Total conversion increase was a 94%.
Quizzes are more effective for lead generation! Difference was a 281% increase in leads. People love games
Free trials can increase cash flow! Increase cash flow by 60% •Decreased churn by 41%. Watch out for processing issues
Double your revenue through 3 upsells! Don’t make people re-enter their CC details by asking for CC details on each step.
Join us on Twitter @larrykim @neilpatel #KISSwebinar

Published in: Marketing, Technology
1 Comment
47 Likes
Statistics
Notes
No Downloads
Views
Total Views
11,339
On Slideshare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
140
Comments
1
Likes
47
Embeds 0
No embeds

No notes for slide

Transcript of "The 10 Weirdest A/B Tests Guaranteed to Double Your Business Growth"

  1. 1. Larry Kim and Neil Patel The 10 Weirdest A/B Tests Guaranteed to Double Your Business Growth March 2014
  2. 2. @larrykim@neilpatel #KISSwebinar Join us on Twi er
  3. 3. Neil Patel KISSmetrics 
 @neilpatel Larry Kim WordStream 
 @larrykim Larry Kim is the founder and CTO at WordStream where he is contributor to both the product team and marketing teams. Larry is commi ed to helping small businesses learn how to grow their businesses through search marketing. Neil Patel is the co-founder of KISSmetrics, Crazy Egg and Hello Bar. He helps companies like Amazon, NBC, GM, HP, and Viacom grow their revenue. The Wall Street Journal calls him a top influencer on the web and Entrepreneur Magazine says he created one of the 100 most brilliant companies in the world.
  4. 4. 1 Larry’s Weirdest A/B Tests 2 Neil’s Weirdest A/B Tests Table of Contents
  5. 5. WATCH THIS WEBINAR RECORDING NOW
  6. 6. Larry’s Weirdest A/B Test 1
  7. 7. The A/B Landing Page Optimization Gurus Have it Wrong
  8. 8. Warning: Lots of Data Ahead!
  9. 9. Weirdest A/B Test #1:
 The Classic A/B
 Conversion Rate Optimization Test Is Often Quite Silly …
  10. 10. We changed the: • Font Type • Spacing • Button Color • Image • Etc. We got a 5% Increase in conversions!! The Great Landing Page Optimization Fairy Tale
  11. 11. • The Early Lead Disappears! • We want to believe our hard work paid off but we are often deluding ourselves… Typical Conversion Rate Optimization Test
  12. 12. • Small Changes Result in Small Changes • The x% Increase you think you got often isn’t significant! Conventional Landing Page Optimization is Over-Rated
  13. 13. Re-Arrange Deck Chairs On the Titanic and ignore sinking
  14. 14. Weirdest Test #2:
 So … What’s a Good Conversion Rate?
  15. 15. What’s a Good Conversion Rate? What’s a Good Conversion Rate?
  16. 16. Survey Methodology
  17. 17. Even the “Winner” is a Loser! CONFIDENTIAL – DO NOT DISTRIBUTE 16 Trees vs. Forest
  18. 18. Distribution Point Conversion Rate Vs Average Comments Average 2.35% Unremarkable Top 25% 5.31% 2x Awesome Top 10% 11.45% 5x Unicorns Unremarkable vs. Unicorns
  19. 19. Distribution Point All accounts Ecommerce Legal B2B Finance Median CVR 2.35% 1.84% 2.07% 2.23% 5.01% Top 25% CVR 5.31% 3.71% 4.12% 4.31% 11.19% Top 10% CVR 11.45% 6.25% 6.46% 11.70% 24.48% “Top 10% of accounts have a CVR 3x average” rule
  20. 20. Aim For 2-4x Increases Not 2-4%!
  21. 21. Landing Page Unicorns?! (+3x Avg. Conversion Rate!)
  22. 22. Weirdest A/B Testing Epiphany #3: 
 What Do Landing Page Unicorns 
 Look Like?
  23. 23. @larrykim, @perrymarshall Hunting Landing Page Unicorns
  24. 24. 1. Changing The Offer
  25. 25. The New Offer: CONFIDENTIAL – DO NOT DISTRIBUTE 24 The New Offer
  26. 26. CONFIDENTIAL – DO NOT DISTRIBUTE 25 • Average or Below Average Conversion Rates • Ask Your Customers! How to Know if your Offer Stinks?
  27. 27. 2. Change The Flow (Reverse Funnel)
  28. 28. 2. New Flow: Registration At End
  29. 29. 2. New Flow: Quality vs. Quantity New Flow: Quality vs. Quantity
  30. 30. Another Example of Changing The Flow
  31. 31. CONFIDENTIAL – DO NOT DISTRIBUTE 30 Changing the Flow: Let Them Choose
  32. 32. of people who visit a website leave without completing 
 the actions marketers want them to take 96% Your Ad Your%Site X - of people abandon their shopping cart without completing a purchase 70% 3. Use Remarketing as a CRO Tool
  33. 33. X - Users who have 
 visited your web site Bring them back to your website Opportunity to … • Turn abandoners into leads • Increase Branded Searches • Increase repeat visitor rates & engagement • Increase effectiveness of Landing Pages 3. Why I View Remarketing as CRO*
  34. 34. Weirdest A/B Testing Epiphany #4
 You Need To Run 10 A/B Tests To Find 1 Unicorn
  35. 35. Name Percentile Relative Abundance Vs. Expected Conversion Rate Awesome Landing Pages Top 25% 1:4 2x Higher Unicorns Top 10% 1:10 5x Higher!! Larry Kim (@larrykim) Relative Abundance
  36. 36. But I Have 10 Landing Pages!
  37. 37. Larry Kim (@larrykim) This is the same landing page with different spacing…
  38. 38. Weirdest A/B Testing Epiphany #5
 It’s Not as Hard As You Think
  39. 39. 80% of Traffic Goes to 10% of Landing Pages
  40. 40. The Bar for Landing Page Excellence is (Very) Low
  41. 41. You Don’t Need Dozens of Landing Pages…
  42. 42. But You Don’t Need Dozens of Landing Pages…
  43. 43. 1. Most Landing Page Optimization = Rearranging Deck Chairs, Small % Changes (Often Insignificant) 2. Insane Landing Page Optimization = 2-5x Changes AND improve lead quality, too. A Quick Recap
  44. 44. LOG IN WITH GOOGLE Start Your Free KISSmetrics Trial
  45. 45. Neil’s Weirdest A/B Test 2
  46. 46. 6. Use GEO based data throughout your funnel ! Test Six
  47. 47. Don’t forget about these steps ! Test Six
  48. 48. Results ! ! ! ! 20% increase in leads and 68% improvement in qualified leads as people felt the solution was more customized to them.
  49. 49. ! ! ! We found that targeting to a specific city decreased conversions by 19% as people felt creeped out. Section OneWhat to avoid
  50. 50. 7. Try using authentication Test Seven
  51. 51. ! ! ! By pulling log in details from Google, we increased signups by 59.4%. Section OnePeople are lazy
  52. 52. Test sign in versus sign up Test Seven
  53. 53. ! ! ! Difference between sign up and sign in was. 22% Total conversion increase was a 94%. Section OneWords ma er
  54. 54. Focus on what’s new Test Seven
  55. 55. •Using authentication at the end of your funnel isn’t as effective. •You have to analyze your drop off rates per step to figure out where to use it. Section OneWhat to avoid
  56. 56. 8. Quizzes are more effective for lead generation ! Test Eight
  57. 57. Use interactive quizzes ! ! Test Eight
  58. 58. Difference was a 281% increase in leads Section OnePeople love games
  59. 59. Make sure you use images ! ! ! Test Eight
  60. 60. •Make your quiz 5 questions •Use images when possible •Limit the choices per question to a maximum of 4 options Section OneTips
  61. 61. 9. Free trials can increase cash flow ! ! ! Test Nine
  62. 62. •Increase cash flow by 60% •Decreased churn by 41% Section OneThe data
  63. 63. Section OneWatch out for processing issues By refunding anyone who complains and by charging a dollar for the trial (then refund it), you’re conversions will go up.
  64. 64. 10. Double your revenue through 3 upsells ! ! ! ! Test Ten
  65. 65. Make your first product affordable Test Ten
  66. 66. Section OneTips •Make you first upsell 3x in price •Make the second upsell 1.5x in price •Make the 3rd upsell 2x in price
  67. 67. Section OneDon’t make people re-enter their CC details By asking for CC details on each step you will see a decrease in revenue by by 29%.
  68. 68. Questions? Shoot and we will do our best to answer
  69. 69. THANK YOU Larry Kim and Neil Patel @larrykim @neilpatel
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×