0
AB
Testing
AB Testing?

+17.1%
$ per visitor
2008
Lesson #1

Define Quantifiable Success
Metrics
visitors
per day

Visit

sign up
rate

$ per recipient

Email

$
Optimizely

visitors
per day

Visit

sign up
rate

$ per recipient

Email

$
Media

Button
Button

Media

1. Sign up

1. Get Involved Image

2. Learn More

2. Family Image

3. Join Us Now

3. Change Image

4. Sign...
Button: “Sign Up”
Button: “Learn More”
Button: “Join Us Now”
Button: “Sign Up Now”
Button

Media

1. Sign up

1. Get Involved Image

2. Learn More

2. Family Image

3. Join Us Now

3. Change Image

4. Sign...
Media: “Get Involved”
Media: “Fam ily”
Media: “Change”
Media: “Change”

Media: “Barack’s Video”
Media: “Change”

Media: “Springfield Video”
Media: “Change”

Media: “Sam’s Video”
And the Winner Was…
Button

Media

1. Sign up

1. Get Involved Image

2. Learn More

2. Family Image

3. Join Us Now
4. Si...
And the Winner Was…
Button

Media

1. Sign up

1. Get Involved Image

2. Learn More

2. Family Image

3. Join Us Now

3. C...
And the Winner Was…
What Did That Mean?

Original

Email
Subscription
s
7,120,000

+40.6%

+2,880,000

288,000

+$57,000,000

New

10,000,000
...
Lesson #1

Define Quantifiable Success
Metrics
Lesson #2

Less Is More
Vs

+8% Engagement
+15% Conversion Rate
+10% $ per visitor
+16.8%
$ per visitor
Lesson #2

Less Is More
Lesson #3

Words Matter
Focus on Your Call to Action
Vs

+15.75%
$ per visitor
Never Signed
Signed Up,
Up
Never Donated
0.0%
0.0%

Previously
Donated
0.0%

+2.3%

+27.8%

+16.3%

-27.8%

N/A

N/A

+15....
Vs
+14.6%
Clicks
Lesson #3

Words Matter
Focus on Your Call to Action
Lesson #4

Fail Fast
Vs
-92.3%
Video
Clicks
Vs

-10% Conversion Value
Lesson #4

Fail Fast
Lesson #5

Seek the Global Maximum
+600%
Engagement
“This new interface is just awful”

+0.5% Retention
+1.4% Engagement
Vs

+49% Conversion Rate
from 240 A/B Tests
Lesson #5

Seek the Global Maximum
Lesson #6

Start Today
Optimizely for Magento
Lessons Learned
1.
2.
3.
4.
5.
6.

Define Quantifiable Success Metrics
Less is More
Words Matter
Fail Fast
Seek The Global...
Happy
Testing!
Optimizely: A/B testing
Optimizely: A/B testing
Optimizely: A/B testing
Upcoming SlideShare
Loading in...5
×

Optimizely: A/B testing

386

Published on

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
386
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Optimizely: A/B testing"

  1. 1. AB Testing
  2. 2. AB Testing? +17.1% $ per visitor
  3. 3. 2008
  4. 4. Lesson #1 Define Quantifiable Success Metrics
  5. 5. visitors per day Visit sign up rate $ per recipient Email $
  6. 6. Optimizely visitors per day Visit sign up rate $ per recipient Email $
  7. 7. Media Button
  8. 8. Button Media 1. Sign up 1. Get Involved Image 2. Learn More 2. Family Image 3. Join Us Now 3. Change Image 4. Sign Up Now 4. Barack’s Video 5. Springfield Video 6. Sam’s Video
  9. 9. Button: “Sign Up”
  10. 10. Button: “Learn More”
  11. 11. Button: “Join Us Now”
  12. 12. Button: “Sign Up Now”
  13. 13. Button Media 1. Sign up 1. Get Involved Image 2. Learn More 2. Family Image 3. Join Us Now 3. Change Image 4. Sign Up Now 4. Barack’s Video 5. Springfield Video 6. Sam’s Video
  14. 14. Media: “Get Involved”
  15. 15. Media: “Fam ily”
  16. 16. Media: “Change”
  17. 17. Media: “Change” Media: “Barack’s Video”
  18. 18. Media: “Change” Media: “Springfield Video”
  19. 19. Media: “Change” Media: “Sam’s Video”
  20. 20. And the Winner Was… Button Media 1. Sign up 1. Get Involved Image 2. Learn More 2. Family Image 3. Join Us Now 4. Sign Up Now ?????? ????? 3. Change Image 4. Barack’s Video 5. Springfield Video 6. Sam’s Video ?????? ?????
  21. 21. And the Winner Was… Button Media 1. Sign up 1. Get Involved Image 2. Learn More 2. Family Image 3. Join Us Now 3. Change Image 4. Sign Up Now 4. Barack’s Video 5. Springfield Video 6. Sam’s Video
  22. 22. And the Winner Was…
  23. 23. What Did That Mean? Original Email Subscription s 7,120,000 +40.6% +2,880,000 288,000 +$57,000,000 New 10,000,000 10,000 $200,000,000 Volunteers Amount Raised 712,000 $143,000,000
  24. 24. Lesson #1 Define Quantifiable Success Metrics
  25. 25. Lesson #2 Less Is More
  26. 26. Vs +8% Engagement
  27. 27. +15% Conversion Rate +10% $ per visitor
  28. 28. +16.8% $ per visitor
  29. 29. Lesson #2 Less Is More
  30. 30. Lesson #3 Words Matter Focus on Your Call to Action
  31. 31. Vs +15.75% $ per visitor
  32. 32. Never Signed Signed Up, Up Never Donated 0.0% 0.0% Previously Donated 0.0% +2.3% +27.8% +16.3% -27.8% N/A N/A +15.2% -24.6% +11.9% +8.5% +2.9% +18.4%
  33. 33. Vs +14.6% Clicks
  34. 34. Lesson #3 Words Matter Focus on Your Call to Action
  35. 35. Lesson #4 Fail Fast
  36. 36. Vs -92.3% Video Clicks
  37. 37. Vs -10% Conversion Value
  38. 38. Lesson #4 Fail Fast
  39. 39. Lesson #5 Seek the Global Maximum
  40. 40. +600% Engagement
  41. 41. “This new interface is just awful” +0.5% Retention +1.4% Engagement
  42. 42. Vs +49% Conversion Rate from 240 A/B Tests
  43. 43. Lesson #5 Seek the Global Maximum
  44. 44. Lesson #6 Start Today
  45. 45. Optimizely for Magento
  46. 46. Lessons Learned 1. 2. 3. 4. 5. 6. Define Quantifiable Success Metrics Less is More Words Matter Fail Fast Seek The Global Maximum Start Today
  47. 47. Happy Testing!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×