Recommended
More Related Content
Similar to Playback Data Systems @ Netflix
Similar to Playback Data Systems @ Netflix (20)
Recently uploaded
Recently uploaded (20)
Playback Data Systems @ Netflix
- 1. Distributed Systems Engineer Playback Data Systems Ketan Duvedi
- 2. Netflix The world’s leading Internet TV network 190+ countries137+ million members 1000+ devices supported
- 3. Big Global Opportunity Global broadband households continue to grow
- 4. Innovative Culture Values are what we value High Performance Freedom & Responsibility Context, not Control Highly Aligned, Loosely Coupled Pay top of market Promotions & Development https://jobs.netflix.com/culture
- 5. Team Focus Build and Operate server side infrastructure to collect, process, and provide high quality data about Netflix playback experiences.
- 6. Data Collection Viewing Data Playback events Player QoE Errors Ingest > 2.5M events/sec
- 7. Data Collection Device Attributes Firmware, SDK, UI State changes Ingest ~ 135K events/sec
- 9. Data Processing - High Volume @ Low Latency PDS owns almost all of top 5 Cassandra Clusters at Netflix based on data size and IOPs
- 10. Data Providing What you’ve watched Where you’re at For streaming and downloaded content Real time Operational Insights
- 11. Data Providing Data feeds ● Events and summaries ● Large number of consumers Personalized Recommendations
- 12. DevOps Culture Insights into our systems enables Operational Excellence You Build It, You Run It
- 13. Technology Stack
- 15. Global Scale Improve the microservices architecture to ensure linear scalability at sublinear cost.
- 16. Flexible Processing Build platform to enable partners to quickly explore new data transformations and views.
- 17. Data quality Improve system resiliency and deliver high quality data despite diversity of quality in input events.
- 18. Evolution to Stream Processing Simplify the microservices architecture by evolving to stateful stream processing applications
- 19. For more info... Details of the team’s work and approaches: ● Scaling Time Series Data - Part I and II ● How Netflix knows where you are in a Show My personal experience working at Netflix ● Inspiring Culture ● Imposter Syndrome ● Interviewing @ Netflix ● Stand out in the crowd
Editor's Notes
- Background image from https://www.flickr.com/photos/theholyllama/4143228359/, no changes made, https://creativecommons.org/licenses/by-sa/2.0/
- Image from https://www.flickr.com/photos/ericsbardo/3757385525/, no changes made, https://creativecommons.org/licenses/by-sa/2.0/
- The team acts as the “data edge” for playback-oriented interactions.
- How much was watched What was the quality What’s changing on the device over time
- How much was watched What was the quality What’s changing on the device over time
- Sessionize Sanitize / clean Synthesize Summarize Aggregate
- Exploring sharding, compression, moving to pure stream processing architecture
- Through approaches like containerized stream processing rather than existing solutions that are skewed towards batch processing.
- Apply at https://jobs.netflix.com/jobs/866030 or contact us at kduvedi@netflix.com or kmcgurk@netflix.com