Uploaded byDaniel Jacobson
History and Future of the Netflix API - Mashery Evolution of Distribution
The document discusses the evolution and significance of the Netflix API, which was initially launched to engage the public developer community and has grown to support over 30 billion requests monthly. It highlights the importance of building a scalable data distribution pipeline to enhance user experience across various devices and platforms. Key lessons learned include understanding target audiences, prioritizing mobile and international needs, and integrating the API into Netflix's engineering culture.
More Related Content
Similar to History and Future of the Netflix API - Mashery Evolution of Distribution
History and Future of the Netflix API - Mashery Evolution of Distribution
- 1. The Netflix APIHowNetflix Launched an API and Evolved it to Serve Millions on Hundreds of DevicesBy Daniel Jacobson
- 2. Who Am I?Directorof Engineering for Netflix API since October 2010Tenure at NPR from 1999 to 2010Built the custom CMS in 2002Extended system for RSS and PodcastingLaunched the NPR API in 2008Launched NPR redesign in 2009
- 3. Netflix OverviewNetflix offerssubscriptions to unlimited streaming movies and TV shows for a very low priceAbout 700 operational employees, 300 engineersMore than 23 million subscribers in US and CanadaMarket capitalization is about $12BResponsible for about 30% of US bandwidth during peak hours (by some accounts)
- 4. How Does theAPI Fit in at Netflix?
- 5. Original Charter forthe Netflix APIExpose Netflix metadata and services to the public developer community to “let 1,000 flowers bloom”. That community will build rich and exciting new tools and services to improve the value of Netflix to our customers.
- 6. Netflix APIThere arecurrently over18,000 flowers
- 10. Eat Your OwnDog Food
- 11. New Target AudiencesInternalEngineering TeamsNetflix product ownersNetflix developersPartner RelationshipsExternal device manufacturersPublic Developer Community1,000 flowers
- 12.
- 13. Some of themany Netflix-ready devices
- 15.
- 16. Netflix API Requestsby Audience
- 17. New Charter forthe Netflix APIBuild and maintain an infinitely scalable data distribution pipeline for getting metadata and services from internal Netflix systems to streaming client apps on all platforms in the format and/or delivery method that is most optimal for each app and platform.
- 18. APIPersonalization EngineUser InfoMovieMetadataMovie RatingsSimilar MoviesReviewsA/B Test Engine
- 19. What is thefuture of the API?
- 20.
- 21. Growth of theNetflix APIAbout 30 Billion requests per month
- 24. <catalog_titles> <number_of_results>1140</number_of_results> <start_index>0</start_index> <results_per_page>10</results_per_page> <catalog_title> <id>http://api.netflix.com/catalog/titles/movies/60021896</id><title short="Star" regular="Star"></title> <box_art small="http://alien2.netflix.com/us/boxshots/tiny/60021896.jpg" medium="http://alien2.netflix.com/us/boxshots/small/60021896.jpg" large="http://alien2.netflix.com/us/boxshots/large/60021896.jpg"></box_art> <link href="http://api.netflix.com/catalog/titles/movies/60021896/synopsis" rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis"></link> <release_year>2001</release_year> <category scheme="http://api.netflix.com/catalog/titles/mpaa_ratings" label="NR"></category> <category scheme="http://api.netflix.com/categories/genres" label="Foreign"></category> <link href="http://api.netflix.com/catalog/titles/movies/60021896/cast" rel="http://schemas.netflix.com/catalog/people.cast" title="cast"></link><link href="http://api.netflix.com/catalog/titles/movies/60021896/screen_formats" rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screen formats"></link <link href="http://api.netflix.com/catalog/titles/movies/60021896/languages_and_audio" rel="http://schemas.netflix.com/catalog/titles/languages_and_audio" title="languages and audio"></link> <average_rating>1.9</average_rating> <link href="http://api.netflix.com/catalog/titles/movies/60021896/similars" rel="http://schemas.netflix.com/catalog/titles.similars" title="similars"></link> <link href="http://www.netflix.com/Movie/Star/60021896" rel="alternate" title="webpage"></link> </catalog_title> <catalog_title> <id>http://api.netflix.com/catalog/titles/movies/17985448</id><title short="Lone Star" regular="Lone Star"></title> <box_art small="http://alien2.netflix.com/us/boxshots/tiny/17985448.jpg" medium="http://alien2.netflix.com/us/boxshots/small/17985448.jpg" large=""></box_art> <link href="http://api.netflix.com/catalog/titles/movies/17985448/synopsis" rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis"></link> <release_year>1996</release_year> <category scheme="http://api.netflix.com/catalog/titles/mpaa_ratings" label="R"></category> <category scheme="http://api.netflix.com/categories/genres" label="Drama"></category><link href="http://api.netflix.com/catalog/titles/movies/17985448/awards" rel="http://schemas.netflix.com/catalog/titles/awards" title="awards"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/format_availability" rel="http://schemas.netflix.com/catalog/titles/format_availability" title="formats"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/screen_formats" rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screen formats"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/languages_and_audio" rel="http://schemas.netflix.com/catalog/titles/languages_and_audio" title="languages and audio"></link> <average_rating>3.7</average_rating> <link href="http://api.netflix.com/catalog/titles/movies/17985448/previews" rel="http://schemas.netflix.com/catalog/titles/previews" title="previews"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/similars" rel="http://schemas.netflix.com/catalog/titles.similars" title="similars"></link> <link href="http://www.netflix.com/Movie/Lone_Star/17985448" rel="alternate" title="webpage"></link> </catalog_title></catalog_titles>
- 25. Improve Efficiency ofAPI RequestsCould it have been 5 billion requests per month?(Assuming everything else remained the same)
- 26. Design for ourKey AudiencesSome of the many Netflix-ready devices
- 27. Future architecture needsto support key audience first with a trickle down of features to the other audiencesNetflix API
- 28. Design for TheAudience We WantIf our current audience is this sizeWe will designour system for an audience that is this size
- 29.
- 30. Key Lessons fromNetflixUnderstand target audiencesThink bigThink mobileThink internationalDesign API for critical audiences firstInternalize the API as part of your engineering DNAIf building a public API, help them bloom
- 31. The Netflix APITeam is Hiring!
Editor's Notes
- #7 In the beginning, the Netflix API only supported the 1,000 flowers. Now there are over 18,000 of them.
- #8 Here are some examples of companies or individual developers that have built apps using the Netflix API.
- #9 Along the way, Netflix launched another developer community-driven program – The Netflix Prize. This was a program that offered $1,000,000 to the first team that could improve the recommendation algorithm by 10%. It took 2.5 years for the prize to be awarded.
- #10 Then streaming started taking off for Netflix and the API became a viable option for getting Netflix streaming onto devices.
- #11 This is what is known as the “eat your own dogfood” model.If you have to eat the dog food, you are more likely to make it taste good. For us, if we build an API for ourselves, others can be assured that it will taste good because we will be eating it too.
- #12 The shift due to streaming results in a redefinition of target audiences for the API.
- #13 So, the API interaction model started to look different with the addition of all of these devices.
- #14 Today, we have hundreds of devices being run off the API, driving tremendous business growth.
- #15 And a few weeks ago, we added another set of devices – select Android phones.
- #16 As a result of the device implementations against the API, the Netflix API traffic has gone through the roofAveraging (today) nearly 30B requests per monthWith peak traffic at about 20,000 requests per second
- #17 And the proportion of API requests coming to the Netflix API through the public has gone way down!The 1,000 flowers represent less than ½ a percent of the total API traffic
- #18 Now the Netflix API has a new charter…
- #19 Netflix engineering teams are set up in support of this charter.Internal engineering teams produce or manage content and algorithmic output (the bottom row).Different engineering and product teams are responsible for presentation layers on devices (top row).The API is responsible for delivering the content from the internal engineering teams to the device presentation layers. The API is also responsible for scaling horizontally to handle the growing load.
- #21 Again, the Netflix API traffic has gone through the roof in this model.
- #22 Metrics like 30B requests per month sound great, don’t they? The reality is that this number is concerning…
- #23 In the web world, increasing request numbers mean increasing opportunity of ad impressions, which means increasing opportunity for generating revenue. And when you hit certain thresholds in impressions, CPMs start to rise, which means even more money.
- #24 Some companies go further to generate more page views by adding things like pagination on article pages. These additional page views translate into additional ad impressions.
- #25 But for systems that yield output that looks like this, such as APIs, ad impressions are not part of the game. As a result, the increase in requests don’t translate into more revenue. In fact, they translate into more expenses. That is, to handle more requests requires more servers, more systems-admins, a potentially different application architecture, etc.
- #26 We are challenging ourselves to redesign the API to see if those same 30 billion requests could have been 5 billion or perhaps even less, assuming everything else remained the same. Through more targeted API designs based on what we have learned through our metrics, we will be able to reduce our API traffic as Netflix’ overall traffic grows.
- #27 As we embark on the redesign of the API, we will plan to design it for our key audiences… the devices.
- #28 And the redesigned API for the devices will be trickled down to other parties.
- #29 Along the way, we plan to design the API for the audience that we want, not the audience we have. That doesn’t necessarily mean that we will implement the system for the dream audience. It just means that our designs should allow for the system to scale to the dream audience.
- #30 For Netflix, we will be designing our server architecture, using AWS, to be highly scalable across many different areas. The API redesign will help us scale the software better to handle the scaling service more effectively and efficiently.