Lean Analytics Cycle

202,981 views

Published on

Published in: Data & Analytics, Technology
2 Comments
332 Likes
Statistics
Notes
  • Thats incredible
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • hi hiten shah ur presentation is really v good
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
202,981
On SlideShare
0
From Embeds
0
Number of Embeds
43,599
Actions
Shares
0
Downloads
1,843
Comments
2
Likes
332
Embeds 0
No embeds

No notes for slide

Lean Analytics Cycle

  1. Lean Analytics Lean Product & Lean UX Silicon Valley - July 2014 Hiten Shah hnshah@gmail.com Download the slides: h p://kiss.ly/leanprod2014
  2. Continuous Learning & Improvement
  3. Don’t know where to start?
  4. Wondering if you’re working on the right things?
  5. Or, you just want to hear me babble!
  6. HTTP://KISS.LY/LEANLYTICSBOOK
  7. Lean Analytics helps you Optimize for Learning
  8. Lean Analytics Cycle AVINASH KAUSHIK HTTP://KISS.LY/LEANAC
  9. Metrics  →  Hypothesis  →  Experiment  →  Act
  10. HTTP://HELLOBAR.COM
  11. What is Hello Bar?
  12.  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  13. Our funnel showed a low installation rate Hello Bar Funnel Metrics Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 15068 1868 716 688 398 % prev step 12.40% 38.33% 96.09% 57.85% % drop off -87.60% -61.67% -3.91% -42.15% % of visitors 100% 12.40% 4.75% 4.57% 2.64%
  14. We  achieved  a  40%  increase  for  our   installa2on  rate  with  just  this  one   first  experiment. The Lean Analytics Cycle At Work
  15. Original Hello Bar installation screen
  16.  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  17. Find out why people don’t install Hello Bar
  18. Didn’t Meet My Expectations
  19. I Had Trouble Installing It
  20. I Don’t Have A Website
  21. Not Ready To Install
  22. Other
  23. Hello Bar Experiment Hypothesis If  more  installa2on  op2ons  are  given,   then  more  people  will  add  Hello  Bar   to  their  website  due  to  the  need  for  a   WordPress  plugin  and  ability  to  email   the  code  to  someone  else.
  24.  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  25. Created a new flow to test
  26. I can install code myself.
  27. 41.67% responded with “WordPress Plugin”
  28. 25% responded with “email my developer / web designer”
  29. I need help.
  30.  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  31. Full funnel metrics for the experiment Variation Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7456 906 342 329 276 % prev step 12.15% 37.75% 96.20% 83.89% % drop off -87.85% -62.25% -3.80% -16.11% % of visitors 100% 12.15% 4.59% 4.41% 3.70% Control Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7469 925 353 341 197 % prev step 12.38% 38.16% 96.60% 57.77% % drop off -87.62% -61.84% -3.40% -42.23% % of visitors 100% 12.38% 4.73% 4.57% 2.64%
  32. Resulted in a 30% installation rate increase HTTP://ISVALID.ORG Experiment resulted in a 40% improvement
  33. ACer  a  dozen  more  experiments,  our   installa2on  rate  is  now  hovering   between  5%  and  7%   That’s  an  addi;onal  89%  increase!!! The Lean Analytics Cycle At Work
  34. Metrics  →  Hypothesis  →  Experiment  →  Act How You Can Put It Into Practice
  35. Metrics   Figure  out  what  to  improve
  36. “What gets measured, gets managed.” Peter Drucker
  37. Baselines DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA
  38. Funnels KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL
  39. SaaS Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL
  40. eCommerce Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL
  41. Revenue BUFFER BAREMETRICS HTTP://KISS.LY/BUFFERBARE
  42. Spreadsheets and Calculations HELLOBAR HTTP://HELLOBAR.COM Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Created Second Bar Totals 15068 1868 716 688 398 193 % prev step 12.40% 38.33% 96.09% 57.85% 48.49% % drop off -87.60% -61.67% -3.91% -42.15% -51.51% % of visitors 100% 12.40% 4.75% 4.57% 2.64% 1.28%
  43. Hypothesis   Make  an  educated  guess
  44. “A solid test hypothesis is an informed solution to a real problem – not an arbitrary guess. The more research and data you have to base your hypothesis on, the be er it will be.”
 — Michael Aagaard, h p://kiss.ly/abthypo
  45. Research   Tools  &  Tac;cs
  46. Customer Development HTTP://WWW.CUSTDEV.COM
  47. Jobs To Be Done #JTBD HTTP://JOBSTOBEDONE.ORG
  48. Usability Testing HTTP://USERTESTING.COM
  49. Live Chat HTTP://OLARK.COM
  50. Mouse Tracking HTTP://CRAZYEGG.COM
  51. Long Form Surveying HTTP://SURVEYMONKEY.COM
  52. On-Page Surveying HTTP://QUALAROO.COM
  53. Personalized Messaging HTTP://INTERCOM.IO
  54. Email a survey to people HTTP://SURVEYMONKEY.COM
  55. Ask people why they canceled HTTP://SURVEYMONKEY.COM
  56. Get feedback right when they cancel
  57. Exit Surveys HTTP://QUALAROO.COM
  58. Discover what’s missing HTTP://QUALAROO.COM
  59. Find out what words people use to describe your product HTTP://QUALAROO.COM
  60. Learn about people’s expectations HTTP://INTERCOM.IO
  61. Learn what frustrates people HTTP://INTERCOM.IO
  62. More actionable reading… QUALITATIVE DATA HTTP://KISS.LY/QUALABT
  63. Experiment Hypothesis Formula DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA
  64. Experiment   Test  your  guess
  65. Only  1  out  of  5  tests  win. 4  out  of  5  tests   cost  you  money. $
  66. 1)  Which  top  metric  are  we  trying  to  improve?   2)  What  can  you  anchor  your  metric  against?   3)  What  is  your  hypothesis  about  why  this  metric  is  at  x%?   4)  What  do  we  expect  from  this  experiment?   5)  What  was  the  result  and  what  did  we  learn  from  it? HTTP://KISS.LY/6QSAB Answer key questions for each experiment
  67. Prioritize your experiments HTTP://KISS.LY/ABPIE
  68. Don’t copy “best practices” blindly
  69. Learn what works best for you
  70. Act   Decide  what’s  next
  71. Explore  the  results  and  make   data  informed  decisions. THE ANALYTICS THAT MATTER TO FACEBOOK HTTP://KISS.LY/FBINFORM
  72. KISSmetrics A/B Testing Report KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  73. We know that people love Google Analytics
  74. KISSmetrics Experiment Hypothesis If  people  authen2cate  with  Google,   then  more  people  will  sign  up  due  to   their  exis2ng  familiarity  with  Google   Analy2cs  and  alignment  with  our   winning  headline.
  75. KISSmetrics Homepage Test Control
  76. KISSmetrics Homepage Test Variation
  77. KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  78. KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  79. KISSmetrics Homepage New Control
  80. Inspiration
  81. KISSmetrics Homepage New Test Variation
  82. KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  83. KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  84. Growth! HTTP://KISSMETRICS.COM
  85.  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  86. Hiten Shah hnshah@gmail.com @hnshah You will get all you want in life if you help enough other people get what they want. Zig Ziglar Download the slides: h p://kiss.ly/leanprod2014

×