Upcoming SlideShare
×

# Best Practices & Lessons Learned from 30,000 A/B and Multivariate Tests

6,207 views

Published on

Most businesses know they should be A/B testing their website but few actually do it on a regular basis. One of their biggest challenges is knowing what to test. In this talk Dan will share the best practices & lessons learned from working with over 10,000 users who have created more than 30,000 experiments using a dead-simple A/B testing tool called Optimizely. Dan will share key insights from these experiments and leave you inspired to start testing today.

1 Comment
25 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Great presentation Dan!

Are you sure you want to  Yes  No
Views
Total views
6,207
On SlideShare
0
From Embeds
0
Number of Embeds
1,854
Actions
Shares
0
178
1
Likes
25
Embeds 0
No embeds

No notes for slide
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• \n
• ### Best Practices & Lessons Learned from 30,000 A/B and Multivariate Tests

1. 1. Best Practices & LessonsLearned from 30,000 A/B and Multivariate Tests Dan Siroker Co-Founder & CEO
2. 2. Lesson #1:Nothing is sacred. Question assumptions.
3. 3. MediaButton
10. 10. Media: “Get Involved”
11. 11. Media: “Family”
12. 12. Media: “Change”
13. 13. Media: “Barack’s Video”
14. 14. Media: “Springﬁeld Video”
15. 15. Media: “Sam’s Video”
19. 19. Splash page experiment results
20. 20. Splash page experiment results
21. 21. Splash page experiment results
22. 22. Splash page experiment results Email Volunteers Amount Raised SubscriptionsOriginal: 7,120,000 712,000 \$143,000,000+40.6% +2,880,000 +288,000 +\$57,000,000 New: 10,000,000 1,000,000 \$200,000,000
23. 23. Splash page experiment results Email Volunteers Amount Raised SubscriptionsOriginal: 7,120,000 712,000 \$143,000,000+40.6% +2,880,000 +288,000 +\$57,000,000 New: 10,000,000 1,000,000 \$200,000,000
24. 24. Splash page experiment results Email Volunteers Amount Raised SubscriptionsOriginal: 7,120,000 712,000 \$143,000,000+40.6% +2,880,000 +288,000 +\$57,000,000 New: 10,000,000 1,000,000 \$200,000,000
25. 25. vs.
26. 26. vs. +600% engagement
27. 27. Lesson #1:Nothing is sacred. Question assumptions.
28. 28. Lesson #2:Less is more. Reduce choices.
29. 29. vs.
30. 30. vs.+8.4%signups
31. 31. vs.
32. 32. vs. +10.97% \$ per visitor
33. 33. vs.
34. 34. vs. +8% engagement
35. 35. Lesson #2:Less is more. Reduce choices.
36. 36. Lesson #3:Words matter. Focus on your call to action.
37. 37. Donation button experimentVariations
38. 38. Donation button experiment results Never Signed Up, PreviouslyVariations Signed Up Never Donated Donated 0.0% 0.0% 0.0% +2.3% +27.8% +16.3% -27.8% N/A N/A +15.2% -24.6% +11.9% +8.5% +2.9% +18.4%
39. 39. vs.
40. 40. vs.+10%clicks
41. 41. vs.
42. 42. vs. +14.6% clicks
43. 43. vs.
44. 44. vs. +15.75% \$ per visitor
45. 45. Lesson #3:Words matter. Focus on your call to action.
46. 46. Lesson #4:Fail fast.
47. 47. vs.
48. 48. vs. -92.3% video clicks
49. 49. Lesson #4:Fail fast.
50. 50. Lesson #5:Start today.
51. 51. Happy CustomersFeatured In
52. 52. Lesson #5:Start today.
53. 53. Lessons Learned1. Nothing is sacred. Question assumptions.2. Less is more. Reduce choices.3. Words matter. Focus on your call to action.4. Fail fast.5. Start today.
54. 54. Email me: Follow me:dan@optimizely.com @dsiroker