Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Machine Learning In Real Life: Gather. Unite. Predict

318 views

Published on

By Alexandra Kulachikova – Yandex

Machine learning is not a magic tool; with using it one can get real results in the present. In this session, Alexandra will show a few cases of how real businesses use machine learning opportunities to find new clients across devices on each stage of their paths and predict their conversions. During the presentation she will provide a step-by-step guide, so that everyone can try to build their own prediction models.

Published in: Marketing
  • Be the first to comment

Machine Learning In Real Life: Gather. Unite. Predict

  1. 1. Machine learning in the real life: gather, unite, predict Alexandra Kulachikova, Head of Yandex.Metrica promotion
  2. 2. ▌ Leading web analytics product › Millions of clients › #2 analytics tool by domain share –w3techs.com, datanyze.com › International product 3 Top Countries Russia United States Turkey Ukraine Germany India Brazil
  3. 3. How I tried to stay in Paris for a weekend
  4. 4. Cross-device journey
  5. 5. 1 real person = 3 cookies = 3 users
  6. 6. 1 real person = 3 cookies = 3 users Ad Search Type-in
  7. 7. How to calculate real users? Deterministic tracking › Log-ins › UserIDs Probabilistic tracking › Machine learning
  8. 8. How to calculate real users? Deterministic tracking › Log-ins › UserIDs Probabilistic tracking › Machine learning
  9. 9. See how mobile affects your desktop 10
  10. 10. Your loyal audience comes twice ▌ Cross-device performance › Mobile-then-desktop › User-centric traffic source › Real picture of core audience 11
  11. 11. 12
  12. 12. 13
  13. 13. Audience intersection 14 46.11% Airports Audience visited Paris 19.95% Paris Audience visited Airports 60.34% Galeries Lafayette’s Audience visited Airports 1.98% Airports Audience visited Galeries Lafayette
  14. 14. ▌ ML based advertising › Geo-targeting › Remarketing › Look-alike Pari s Galeries Lafayett e Airport
  15. 15. Predicting the future 16
  16. 16. ML based predictions for DIY online store 17 220 Volt – one of top online stores in Russia › predict the probability of conversion knowing all the customer’s history › use predictions for analysis and advertisement
  17. 17. Say ”no” to attribution modelling 18 › Prediction works on the user level regardless of attribution modelling
  18. 18. Working process 19 Collect all data Build a model Set up tracking and ads
  19. 19. Tools 20 › extract raw-data from Yandex.Metrica with Logs API › use ClickHouse to calculate customers features › machine learning algorythms to built a model
  20. 20. More than 60 characteristics 21 User characteristics: device, browser, region Behavioral: traffic sources, revenue, last visit date etc. designer loves heavy metal 28 years old iPhone likes coffee Yandex.Browse r heavy site visitor wants to buy a drill woman
  21. 21. What characteristics are most important?
  22. 22. Everything matters 23 Characteristic Importance Days since last visit 0.1445 Days since first visit 0.1041 Number of items viewed 0.0777 Avg. time on site 0.0771 Avg. depth on site 0.0701 Revenue 0.0470 Number of non-bounce visits 0.0395 Region 0.0392 Number of visits 0.0340 Number of visits from advertising 0.0259
  23. 23. Never be in a hurry when you training a model 24 › Best algorythm – XGBoost › Quality metric ROC AUC ~0.9 › Old data is a new data › Triple-check before start
  24. 24. Bitter truth 25 More than 80% visitors have probability to purchase less than 5% Probability to purchase %ofusers
  25. 25. Divide visitors by probability 26 › Excellent: probability >= 50% › Good: 15% <= probability < 50% › Normal: 5% <= probability < 15% › Bad: probability < 5%
  26. 26. Daily update 27 › Recalculate and predict probability for each user every day › Use user parameters data upload to keep data updated
  27. 27. Excellent conversion of excellent visitors 28 CR of group Excellent is about 5 times higher than average CR
  28. 28. Let’s make an experiment 29 › Remarketing › Bid corrections › A/B tests
  29. 29. Results: A/B tests 30 Campaign Clicks CTR (%) CPC (rub.) Orders Revenue (rub.) Conversion (%) without corrections 1 364 0,19 20,22 66 158 412 4,84 with corrections 1 283 0,16 27,16 78 310 938 6.08
  30. 30. Results: rise of revenue Retargeting › Revenue grew by 96% › Conversion grew by 25% › Costs grew by 26% 31 Advertising network › Revenue grew by 31% › Conversion hasn’t changed › Costs haven’t changed
  31. 31. Where to use 32 › Analysis: traffic sources performance and users behavior › Advertisement: bids, remarketing, look-alike › Direct-marketing
  32. 32. The recipe of magic ML potion 33 Big Data + Machine learning › free tools only: YM Logs API + ClickHouse + Python + Pandas + XGBoost Bright mind and straight hands :)
  33. 33. Machine learning is our ”today”, not a future
  34. 34. www.linkedin.com/in/kulachikova/ www.yandex.com/support/metrika/troubleshooting. xml Thank you! Alexandra Kulachikova 35

×