Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Growing Your Business Through Experimentation

31,941 views

Published on

A talk given to the BayCHI group about A/B testing.

Published in: Business, Data & Analytics

Growing Your Business Through Experimentation

  1. Growing Your Business Through Experimentation Hiten Shah hnshah@gmail.com
  2. Product, Design, Engineering, Marketing, Growth, Sales What do you do?  
  3. How do you make all the things better?! Figuring this out is your job.
  4. A/b testing
  5. Only  1  out  of  5  tests  win. 4  out  of  5  tests  cost   you  money.
  6. So, why experiment if so many tests fail?
  7. Go Where Customers Are! Long, slow, SaaS ramp of death from 2006…
  8. Go Where Customers Are! Now…
  9. Go Where Customers Are! Growth can be sped up!
  10. Increase Hello Bar’s installation rate
  11. Improve the KISSmetrics features page
  12. Increase signups from Crazy Egg’s homepage
  13. Grow Crazy Egg’s revenue
  14. Results from 3 months of focused growth effort
  15. Results: KISSmetrics signups over time
  16. Simple 4-step experimentation process Data Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide what’s next
  17. Data Figure out what to improve
  18. Data doesn’t make decisions for you.
  19. Data informs your decision making.
  20. Data helps you identify the most painful problems people have. (not the ones that bother you the most)
  21. Find problems worth solving by starting with quantitative data.
  22. When in doubt, look at your top pages. Usually the homepage.
  23. A simple framework for choosing what to improve
  24. Find pages with high traffic and high bounce rate HTTP://GOOGLE.COM/ANALYTICS
  25. Find pages with high traffic and low conversion rate HTTP://UNBOUNCE.COM
  26. Find sources with high traffic and low conversion rate HTTP://HUBSPOT.COM
  27. Find opportunities based on user paths HTTP://KISS.LY/KMPATH
  28. Increase Conversions By A/B Testing Do a Happy Dance But Watch it Closely Get More Traffic So You Can Start A/B Testing Get More Traffic So You Can Do a Happy Dance High Traffic Low Traffic High ConversionsLow Conversions How To Figure Out What To Improve
  29. Where do you have the most visitors, users, customers, emails, or notifications? More volume helps you test faster.
  30. ๏ Website Traffic / App Installs ๏ Welcome Emails ๏ Web / Mobile Onboarding ๏ Email Digests ๏ Triggered Notifications ๏ Cart / Funnel Abandonment ๏ Dormant / Churned users Common areas with large volume and high impact potential
  31. What percentage of your customers or users have adopted each feature? Intercom on Product Management When you’ve already got usage, analyze it.
  32. Find out why by getting qualitative data.
  33. A few qualitative tools HTTP://USERTESTING.COM HTTP://SURVEYMONKEY.COM
  34. A few more qualitative tools HTTP://QUALAROO.COM HTTP://OLARK.COM
  35. Use exit surveys to learn why people bounce HTTP://QUALAROO.COM
  36. Use post-conversion surveys to understand why HTTP://QUALAROO.COM
  37. Discover if your web pages are effective at their goal HTTP://QUALAROO.COM
  38. What frustrates people? EMAIL
  39. Find people who don’t do things and email them EMAIL + SURVEYMONKEY
  40. User Research gives you a headstart
  41. Hypothesis Make an educated guess
  42. Ideas Designs Solutions Assumptions (A guess about the best way to solve a problem)}
  43. Everything can start with a testable hypothesis.
  44. Before you create anything, make an educated guess.
  45. Use simple equations to assess potential impact of each idea DAN MCKINLEY HTTP://KISS.LY/DANETSY visits  x  conversion  rate  x  order  value  x  li4  =  ? users  x  order  value  x  resurrec7on  rate  =  ? ESTIMATED ESTIMATED MODELED MODELED
  46. Assess the potential impact, difficulty and priority of each test idea you have. Prioritize your tests
  47. Experiment Test your guess
  48. Test your hypothesis by starting an experiment.
  49. Not button colors…
  50. Don’t copy “best practices” blindly
  51. Test specific button copy
  52. Sensitivity tests help you learn what your visitors care about and what they don’t. Run sensitivity tests on your web pages
  53. Remove graphical elements and see what happens
  54. Does social proof and logos matter for your visitors?
  55. Remove social proof and learn what happens
  56. Act Decide what’s next
  57. You’ve probably seen an A/B testing report before
  58. 1. Your starting hypothesis. 2. Dates for when you ran the test. 3. Screenshots of control and all variations used during the test. 4. Expected change in conversion. 5. Probability of change. 6. Screenshot of the raw data used in your analysis. 7. A screenshot to the A/B Test Report in KISSmetrics. 8. The decision you've made from the test. 9. What you learned from the test. Document every A/B test and create a playbook
  59. What learnings do we want to carry over from the past? Helps you avoid unknowingly decreasing your conversions.
  60. ✓ Keep the footer links ✓ Autofocus on the first form field ✓ Do not include social proof ✓ Link to the features page ✓ Use “Log in” copy on the button Create experiment checklists based on learnings
  61. If you’re not testing, you’re not learning. ☀
  62. Always have tests running, no matter how small. ⌾
  63. If you don't have a process, you won't continuously improve. ∞
  64. Work on things that matter. Make those things better. Be happier ☺
  65. Thank Y0u =) Got questions? Need help? Contact me any time! Hiten Shah hnshah@gmail.com

×