Making it easy to work with data
•
2 likes•707 views
This document summarizes the history and evolution of Netflix's data platform architecture from 2010 to 2018. It describes Netflix's transition from an on-premise datacenter to using AWS for cloud and streaming in 2009-2011. It then outlines the key services and tools Netflix developed over time like Genie, Metacat, Kragle to handle data storage, processing, and analytics as their data and user base grew significantly from 2012-2018. The document emphasizes building for flexibility and change while avoiding duplicative work.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Making it easy to work with data
Similar to Making it easy to work with data (20)
Recently uploaded
Recently uploaded (20)
Making it easy to work with data
- 1. Making it easy To work with data Charles Smith Data Platform Architecture
- 2. How do you decide what to build?
- 4. Good: Bad: Users These are the people that will use the data to move the platform. They really don’t know what they need technically.
- 5. Good: Bad: Vision Build the platform that works for you. Miss the real needs or build in a bubble.
- 6. Good: Bad: Peers Take the best ideas and even better tools. Missing context. They aren’t solving your problems.
- 9. History of Netflix and big data 2010 Netflix expands to Canada . 2009 Netflix starts moving to AWS 2007 Netflix begins Streaming 2011 AWS Cloud/Streaming only
- 10. Build fast, but expect change
- 11. Avoid undifferentiated heavy lifting
- 12. Self-service
- 16. What will we need in the future?
- 17. 2012 - 2016 2012 Netflix: Originals and Expansion 2016 Netflix streams globally
- 19. Genie
- 20. Genie
- 21. Metacat
- 22. Metacat
- 24. 2012 Datacenter -> AWS 2013 What’s happening?
- 25. Lipstick
- 27. Sting
- 28. Sting
- 30. Pensive
- 32. Charlotte
- 34. 2012 Datacenter -> AWS 2013 What’s happening? 2014 Better Patterns .
- 38. Kragle
- 39. Kragle
- 42. Big Data Portal
- 43. Big Data Portal
- 45. 2012 Datacenter -> AWS 2013 What’s happening? 2014 Better Patterns . 2015 Prepare for Scale
- 50. Kragle Genie Metacat Redshift Presto HiveSpark S3 Pig Big Data Portal Docker
- 53. Spinnaker
- 54. Commuter
- 55. Commuter
- 57. Microbots
- 58. Insight
- 59. Insight
- 60. What will we need in the future?
- 62. Data Platform ML Analysis Visualization
- 64. We currently have 100s of TB of usage data.
- 65. Thanks