Cohort Analysis at Scale
•Download as PPTX, PDF•
6 likes•5,819 views
Presented at Strata San Jose 2018. Shares how Netflix enables business teams to perform cohort analysis on very large, high dimensional data by using Big Data and web application technologies such as Spark, Druid, Node, React, and D3
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Cohort Analysis at Scale
Similar to Cohort Analysis at Scale (20)
Recently uploaded
Recently uploaded (20)
Cohort Analysis at Scale
- 1. Cohort Analysis at Scale BLAKE IRVINE STRATA SAN JOSE 2019.03.06
- 7. Partners help us Grow
- 8. Partners are companies that make it easier for people to sign up and engage with our service and help us retain members. BLAKE IRVINE | STRATA SAN JOSE 2018
- 9. BLAKE IRVINE | STRATA SAN JOSE 2018
- 10. Let’s take a (virtual) trip!
- 11. Trip to Strata San Jose
- 12. Trip to Strata San Jose
- 13. Trip to Strata San Jose
- 14. Multiple Partner Associations Trip to Strata San Jose
- 15. ● Are partners helping us acquire members? ● Through which channels? ● How does a partner impact the regional market? ● Do members use our service differently on partner devices? ● How do partners compare to each other? Evaluating Partners BLAKE IRVINE | STRATA SAN JOSE 2018
- 16. Cohorts are the collections of members associated with a partner that are relevant to the business question. BLAKE IRVINE | STRATA SAN JOSE 2018
- 17. ● Our trip example showed how one member can be associated with many partners. ● Business teams want to explore the nuance of cohorts... Cohorts can be complex BLAKE IRVINE | STRATA SAN JOSE 2018
- 18. … for example BLAKE IRVINE | STRATA SAN JOSE 2018
- 19. ● 100+ million members ● Dozens of partners ● Many combinations of members and partners ● Leads to… ○ High dimensionality, high cardinality datasets ○ Very large datasets of member-level time-series activity Evaluating Cohorts BLAKE IRVINE | STRATA SAN JOSE 2018
- 20. Cohort Analysis at Scale
- 21. ● Data platform ● Data construction ● Data product for cohort analysis Cohort Analysis at Scale BLAKE IRVINE | STRATA SAN JOSE 2018
- 22. Data Platform Simplified Overview Big Data Portal
- 23. Data construction: Data Model member device partner playback events isp billing events billing processor BLAKE IRVINE | STRATA SAN JOSE 2018
- 24. Data construction: Cohort Dataset signup_events cohort playback_events billing_events x_events BLAKE IRVINE | STRATA SAN JOSE 2018
- 25. Data construction: Flat Tables playback_f cohort_playback_s device_d isp_d geo_d partner_d cohort_d BLAKE IRVINE | STRATA SAN JOSE 2018
- 26. Data for consumption: Flat Table key memb er_id device _id device_name device_categor y partner_name country region data_payload 1 1213 674 Amazon Fire TV Set Top Box Amazon US Americas [{"id":5025945823792539,"sequen ce":41,"time":1491962092955}, {"id":5025947899236389,"sequen ce":95,"time":1491962104824}] 2 7623 1172 Chromecast Streaming Stick Google DE EMEA … 3 4291 129 PS3 Game Console Sony ES EMEA … 4 9013 447 iPad 4 Tablet Apple CA Americas … BLAKE IRVINE | STRATA SAN JOSE 2018
- 27. Data construction: Copy forward BLAKE IRVINE | STRATA SAN JOSE 2018 cohort_playback_s Big Data Portal
- 28. ● Goals ○ Serve dozens of users ○ Provide interactive / low-latency tool ○ Provide many different perspectives ● Challenges ○ Manage high dimensionality ○ Very large time-series datasets Data Product for Cohort Analysis BLAKE IRVINE | STRATA SAN JOSE 2018
- 29. Analytic Tool Choices Choice 1 Choice 2 Choice 3 Analytic Tool Tool 1 Tool 2 Tool 3 Data Engine MPP Cloud In memory Data Size 1B rows 10B rows 100M rows Performance (SWAG) Up to many minutes Many minutes Several Seconds BLAKE IRVINE | STRATA SAN JOSE 2018
- 30. ● Data stored in Druid ● Custom app built with Javascript Choice 4... BLAKE IRVINE | STRATA SAN JOSE 2018
- 31. ● An open source data store for analytic applications ● Distributed, column-oriented, indexed architecture ● Well suited to serve our “flat” tables Druid white paper: http://static.druid.io/docs/druid.pdf BLAKE IRVINE | STRATA SAN JOSE 2018
- 32. ● Built with Express, React, Redux, D3 ● Custom UX / UI to manage views and dimensionality ● Enabled access to data served by Druid ● Enabled management of query execution and caching BLAKE IRVINE | STRATA SAN JOSE 2018
- 33. ● Video demo with simulated data BLAKE IRVINE | STRATA SAN JOSE 2018
- 34. ● PED DEMO BLAKE IRVINE | STRATA SAN JOSE 2018
- 35. Challenges
- 36. ● Dimensions (aka slice-n-dice) ○ More is always better ○ Changes require restatement ● “Typical” use cases must be met also ○ Not a solution for every data question ○ Analysts and other tools are still needed Challenges BLAKE IRVINE | STRATA SAN JOSE 2018
- 37. ● Data volume always increases… ○ More members, more partners, more devices, more metrics ● Custom app development time is longer, and ongoing ○ But for the right use cases, worthwhile Challenges BLAKE IRVINE | STRATA SAN JOSE 2018
- 38. Partners help Netflix grow. We measure partner value through cohorts. Big data tools enable efficient analysis. BLAKE IRVINE | STRATA SAN JOSE 2018 Thank you!
- 39. Blake Irvine - Growth Data Products birvine@netflix.com @blakeirvine linkedin.com/in/blakeirvine/
Editor's Notes
- In 2016, Netflix became available globally in almost all countries!