Data-Driven @ Netflix
•
26 likes•8,059 views
Michelle Ufford is a Principal Architect at Netflix who leads their Data Engineering and Analytics team. Netflix has over 86 million members who watch over 125 million hours of content daily on over 1000 supported devices. The data team manages a 40 petabyte data warehouse with 4 petabyte daily reads and 300 terabyte daily writes, processing over 700 billion events. The data is used to predict content value, optimize the user experience, analyze news and PR, monitor global service delivery, and power experimentation.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Data-Driven @ Netflix
Similar to Data-Driven @ Netflix (20)
Recently uploaded
Recently uploaded (20)
Data-Driven @ Netflix
- 1. Data-Driven @ Netflix Michelle Ufford Principal Architect Data Engineering & Analytics
- 2. Michelle Ufford Highlights ● Principal Architect at Netflix Data Engineering & Analytics ● Prev. Engineering Manager at GoDaddy Data Platform ● Microsoft Data Platform MVP ● 10+ years building web-scale analytics & data engineering infrastructure ● advises on Big Data topics Microsoft, Hortonworks, Teradata, etc. Gratuitous picture of my kids
- 4. The business numbers. 86.7 million members 1000+ devices supported 125+ million hours watched launched 19 years ago every. day.
- 5. Any device. Anywhere.* * Well, almost anywhere.
- 6. The data numbers. 4 petabyte DW reads 300 terabyte DW writes 40 petabyte data warehouse 700+ billion events written
- 8. Content. What should we license?
- 9. Predicting Value for Licensed Content. Feature Engineering Predictive Models License Terms Content Efficiency
- 10. Predicting Value for Licensed Content. ● past performance of similar content on Netflix ● broadcast & Box Office performance ● talent (writers, actors, directors, etc.) ● critic & user reviews ● awards & accolades Feature Engineering Predictive Models License Terms Content Efficiency
- 11. Predicting Value for Licensed Content. Feature Engineering Predictive Models License Terms Content Efficiency
- 12. Predicting Value for Licensed Content. ● terms (length, exclusivity, etc.) ● bid amount ● negotiations Feature Engineering Predictive Models License Terms Content Efficiency
- 13. Predicting Value for Licensed Content. ● value / cost ● if efficient, license Feature Engineering Predictive Models License Terms Content Efficiency
- 14. “ last year our original content overall was some of our most efficient content. ”
- 15. “ We are building a studio in the cloud and pioneering new approaches to movie production, optimizing pitches, production schedules, subtitling, and digital asset management for our Original content. ”
- 17. What should we license?create?
- 18. Product UX. Data. Driven. Experience.
- 24. There are 86 million different versions of Netflix.
- 25. billboard rows, row order titles, title order title artwork
- 26. Public Relations. Analytics of news.Analytics is news.
- 29. Content Delivery. Monitoring a global service.
- 31. YouTube video of Vizceral demo https://youtu.be/JctsPpgEsVs
- 33. data access AWS S3 Big Data Platform Amazon Redshift data processing fast storage data viz METACA T data services events data operational data elastic storage Apache Pig
- 34. Philosophy.
- 41. Highly Aligned & Loosely Coupled.
- 43. Data Engineering & Analytics MarketingProduct PlaybackContent Finance 105 talented engineers & analysts data viz engineers analytics engineers data engineers Big Data Platform analysts
- 48. Experimentation Platform Batch & Ad Hoc Analysis
- 49. Questions?
- 50. Thank you for attending! Michelle Ufford linkedin.com/in/mufford @sqlfool Data @ Netflix @NetflixData hadoopsie.com techblog.netflix.com tinyurl.com/NetflixData
Editor's Notes
- Abstract: Netflix is the quintessential data-driven company. It’s 83 million members stream more than 125 million hours in over 190 countries every day and generate more than 700 billion events in the process. In this session, we’ll share how data is used to make informed decisions across the entire business — from content acquisition to content delivery, and everything in between. We’ll look at how Netflix successfully employs a scalable cloud-based data platform to support a constant deluge of data and a small army of data analysts, engineers, and scientists. We’ll discuss the advanced analytical capabilities that are enabled through modern data technologies. Lastly, we’ll explore some of the architectural & operational principals that enable Netflix to so effectively make use of its data.
- Obligatory “why should you listen to me talk?” slide
- Numbers as of Q3 2016
- During CES 2016 this January, ‘flipped the switch’ making Netflix available in 130+ new countries. Netflix is presently available in over 190 countries worldwide.
- What content should we license? How much should we bid? How should we value exclusivity? How should we measure content performance?
- Originals content: 2015 - 450 hours 2016 - 600 hours 2017 - 1000 hours
- Netflix website: circa 2012
- Netflix website: circa 2013
- Netflix website: circa 2014
- Netflix website: circa 2015
- Netflix website: circa 2016
- Vizceral Open-Source Project: https://github.com/netflix/vizceral http://techblog.netflix.com/2016/08/vizceral-open-source.html http://techblog.netflix.com/2015/10/flux-new-approach-to-system-intuition.html
- Genie – federated job execution engine Metacat – federated metadata service Kragle – python APIs
- 15m views on SlideShare
- Minimize rules Make smart choices Take ownership
- Avoid prescriptive requirements Give visibility
- Set context (strategy, goals) Communicate only as much as needed
- At Netflix, we use the scientific method We’re often right at predicting behavior – for people exactly like us Most people aren’t like us