Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building a Data Driven Business


Published on

The intro lecture for a course on Data Driven Marketing

Published in: Business
  • @Alex Kosykh thanks!
    Are you sure you want to  Yes  No
    Your message goes here
  • Great post emphasizing the role of the data in the process of learning the customer base. Provided solid data-driven framework is focusing on the front-office of an enterprise. Improving bottom-line can be done on both ends of the business and it would interesting to see how data-driven approach can improve IT Operations. Data is abundant in that domain, but diversity of sources is a big challenge.
    Are you sure you want to  Yes  No
    Your message goes here

Building a Data Driven Business

  1. 1. Building a data driven business Konstantin Savenkov, CEO, Intento
  2. 2. Konstantin Savenkov, PhD Chief Research Officer, Zvooq COO, Bookmate and Dream Industries now Founder & CEO
  3. 3. Turned a number of companies data-driven • B(2B)2C content services • edutainment • ad tech • commercial space management • consulting: online2offline, UGC, biotech Bookmate, Zvooq Theory&Practice, Exchanges Unisound DI Telegraph @ksavenkov
  4. 4. Without data, people take decisions based on biases and beliefs @ksavenkov
  5. 5. …data replaces beliefs and provides a competitive advantage @ksavenkov
  6. 6. If you’re competitor-focused, you have to wait until there’s a competitor doing something. Being customer-focused allows you to be more pioneering. Jeff Bezos Amazon@ksavenkov
  7. 7. …data helps to be customer-centric @ksavenkov
  8. 8. Information is the oil of the 21st century, and analytics is the combustion engine. Peter Sondegaard Gartner@ksavenkov
  9. 9. …check the gas quality and do the maintenance @ksavenkov
  10. 10. Half of the money spent on advertising is wasted; the trouble is I don’t know which half. John Wanamaker or William Lever @ksavenkov
  11. 11. …channel attribution and ROI will tell you @ksavenkov
  12. 12. The average conversion rate in the United States whether you are selling elephants or iPods, will be 2% (с) Авинаш Кошик @ksavenkov
  13. 13. …data get you above the average @ksavenkov
  14. 14. Mastery Business Metamorphosis Data Aware Data Monetisation Data-driven Business Optimisation Analytics Business Insights Data-Driven Maturity Index Collect Data Business Monitoring @ksavenkov
  15. 15. @ksavenkov
  16. 16. To be data driven, it’s not enough to collect and mine data. You need the culture. @ksavenkov
  17. 17. Don’t optimise for data! Optimise for learning! @ksavenkov
  18. 18. All initiatives must improve KPI Formulate hypotheses Run experiments @ksavenkov
  19. 19. Data validate hypotheses behind your own decisions. Data does not make decisions for you! @ksavenkov
  20. 20. DATA CulturePeople Process Technology Analytics Business model PlanningIterations KPI Hypotheses Experiments Changes @ksavenkov
  21. 21. I Setting Goals and KPI @ksavenkov
  22. 22. Looks simple MRR MarginMAU but there are issues @ksavenkov
  23. 23. IT’S UNCLEAR who’s responsible? how to imrove? how does that relate with resources spent? (besides CEO) (besides “work better”) (when most of the business processes are automatic) @ksavenkov
  24. 24. KPI Trees MAU New Loyal Returned Traffic Conversion Retention Reactivation @ksavenkov
  25. 25. KPI Trees MRR MRR stable net new MRR list MRR MRR upgrades MRR downgrades New clients MRR expansion new MRR Reactivation direct sales @ksavenkov
  26. 26. KPI Trees Margin LTV CAC COGSARPU Lifetime Commissions @ksavenkov
  27. 27. ADVANCED MODE Ratio Analysis Cohort Analysis @ksavenkov
  28. 28. HELPS TO Identify bottlenecks Define roles and responsibilities Generate ideas Measure impact of initiatives @ksavenkov
  29. 29. II Measure Impact of Initiatives @ksavenkov
  30. 30. For planning, groups tasks in Epics that matter Track results of launched Epics A culture of data success learning @ksavenkov
  31. 31. Case study Marketing project were tracked until launched Study of a half-year long cross-promo campaign discovered a significant loss via COGS. We’e identified key mechanics that led to the loss. We’e adjusted the mechanics, re-negotiated with partners on the ongoing campaigns. PROFIT @ksavenkov
  32. 32. III Hypotheses and experiments @ksavenkov
  33. 33. It doesn’t work this way: marketing campaign @ksavenkov impact!
  34. 34. product feature launched billing failure pupils back from holidays AppStore featuring people back from vacations @ksavenkov It doesn’t work this way: marketing campaign impact!
  35. 35. product feature launched billing failure pupils back from holidays AppStore featuring people back from vacations @ksavenkov It doesn’t work this way: marketing campaign impact! DIFFERENT CHANNELS CONVERT WITH A DIFFERENCE IN ORDER OF MAGNITUDE
  36. 36. ALSO: A lot of Epics that shouldn’t have been launched at all A culture of praising a random success and explaining failures by external causes Impossibility to learn by mistakes @ksavenkov
  37. 37. formulate measurable hypotheses carefully plan experiments define a condition of success/failure prior to implementation data collection and attribution, split- testing, controlled variables, statistical significance demonstrate explicit risks use models built on past data prioritisation aid and how the failure affects the roadmap ability to prove multiple hypotheses simultaneously @ksavenkov
  38. 38. Case Studies Estimating mechanics of marketing projects before they launched Product improvement that results in 2.5x conversion, 2x lifetime Split testing of targeting and creative materials for ad campaigns, resulting in conversion 2-3 times higher than organic Immediate increase of “conversion” from all initiatives to successful ones This approach is behind all conclusions in these slides lots of failed experiments success despite of all external factors NO OTHER WAY@ksavenkov
  39. 39. Case Study: Recommender system for Conversion • Hypotheses to prove: 1. There’re enough users who will use RS output 2. Their conversion will be above average • A/B testing is the only way: – different channels convert with up to 20x difference – current traffic mix is unpredictable and hard to control in case of app installs • Do pilots: – Run with limited resources, then extrapolate and decide if run full-scale
  40. 40. Case Study: Recommender system for Conversion
  41. 41. Let’s look at the economics • In case of using a third-party RS on a CPO basis, in this case the CPO is limited by $0.14 (actually, much less) • In case of a flat fee of $1000**/month, this is feasible starting from 7143 new subscribers/month, or $35K of marketing budget. * CAC and marketing budget are model data ** some arbitrary number
  42. 42. IV Leading indicators @ksavenkov
  43. 43. Make decisions on monthly KPI or finished projects is like going backwards @ksavenkov
  44. 44. Look ahead or at least watch your step, not backwards Make the data work for you @ksavenkov
  45. 45. Daily indicators Incremental indicators Leading indicators Predictive models spot problems and anomalies just in time example: a perfect input for inbound marketing accurate goals and perfect financial planning @ksavenkov
  46. 46. Case Study Some loyal subscribers churned away for 2-3 months, to come back later. It was historically attributed to holidays and other external factors. The daily indicators have shown that all churned users subscribed on weekdays. What a riddle! It turned out that on weekdays the code base is frequently deployed to the production server, flushing the message queue of subscription renewals. The fix increased a lifetime of paying users by 20% @ksavenkov
  47. 47. Case Study Predicting a lifetime for users that registered right now (10% accuracy) Accurate unit economics for contracts with B2B2C partners, content providers, pricing @ksavenkov
  48. 48. Case Study A probabilistic model for segmenting users Input data for chained communication (inbound marketing) @ksavenkov
  49. 49. Case Study Accurate goals loyal new churned guaranteed growth? inevitable stagnation!* *unless KPI are increased@ksavenkov
  50. 50. Case Study Accurate financial planning Operational model Marketing plan-fact Financial plan-fact Marketing budget CAC, conversion organic forecast deal terms unit economics deals forecast revenue forecasts @ksavenkov
  51. 51. V Business process automatisation @ksavenkov
  52. 52. Hire another employee or train another model? @ksavenkov
  53. 53. Affects all business processes that scale linearly with a headcount: customer support editorial office content management marketing @ksavenkov
  54. 54. Case Studies User base grows from 1M to 2M, doubling the headcount in customer support? Implemented auto-reply using our knowledge base and smart templates for support engineers A number of markets increased, adding more editors? Created an algorithm to provide a short-lists based on a user behavior An amount of UGC explodes, more content-managers? Improving reduplication and computer aid based on the collected data @ksavenkov
  55. 55. VI Improving the Company Itself @ksavenkov
  56. 56. Operational analytics - improve the process of KPI improvement (productivity) @ksavenkov
  57. 57. ITERATION Case Study Improving the Agile process the expectations: @ksavenkov ROADMAPS DESIGN DEVELOPMENT QA SHIPPED PLANNING
  59. 59. Data as an Asset @ksavenkov
  60. 60. Case Stuies • Compare B2B2C deals through unit economics • Estimate traffic quality for partner ad networks • Data partnerships • Targeted user communication • Personalisation and recommender systems (the next slide) • Bonus track: Investigating a large number of purchase returns for an internet retailer @ksavenkov
  61. 61. Using Recommender Systems and personalisation CAC LTV Content Costs Marketing Expenses New Customers ARPU Lifetime Consumed Content Mix Conversion Retention Reactivation Exposed Content Mix ÷ × * the recommendation fairy *
  62. 62. Innovative business experiments, as there’s no recipes To pioneer, you need to iterate quicker than competitors @ksavenkov
  63. 63. Data, models built over the data and experimental results is the main asset created and exploited by the innovative business @ksavenkov “THE UNFAIR ADVANTAGE”
  64. 64. Q&A Konstantin Savenkov