• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
History and Future of the Netflix API - Mashery Evolution of Distribution
 

History and Future of the Netflix API - Mashery Evolution of Distribution

on

  • 7,389 views

Presentation on the history and future of the Netflix API. This presentation walks through how the API was formed, why it needs a redesign and some of the principles that will be applied in the ...

Presentation on the history and future of the Netflix API. This presentation walks through how the API was formed, why it needs a redesign and some of the principles that will be applied in the redesign effort.

This presentation was given at the Mashery Evolution of Distribution session in San Francisco on June 2, 2011.

Statistics

Views

Total Views
7,389
Views on SlideShare
7,361
Embed Views
28

Actions

Likes
6
Downloads
3
Comments
0

6 Embeds 28

http://paper.li 18
http://blog.programmableweb.com 5
http://infinitefire.com 2
http://www.slideshare.net 1
http://webcache.googleusercontent.com 1
http://feeds.feedburner.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In the beginning, the Netflix API only supported the 1,000 flowers. Now there are over 18,000 of them.
  • Here are some examples of companies or individual developers that have built apps using the Netflix API.
  • Along the way, Netflix launched another developer community-driven program – The Netflix Prize. This was a program that offered $1,000,000 to the first team that could improve the recommendation algorithm by 10%. It took 2.5 years for the prize to be awarded.
  • Then streaming started taking off for Netflix and the API became a viable option for getting Netflix streaming onto devices.
  • This is what is known as the “eat your own dogfood” model.If you have to eat the dog food, you are more likely to make it taste good. For us, if we build an API for ourselves, others can be assured that it will taste good because we will be eating it too.
  • The shift due to streaming results in a redefinition of target audiences for the API.
  • So, the API interaction model started to look different with the addition of all of these devices.
  • Today, we have hundreds of devices being run off the API, driving tremendous business growth.
  • And a few weeks ago, we added another set of devices – select Android phones.
  • As a result of the device implementations against the API, the Netflix API traffic has gone through the roofAveraging (today) nearly 30B requests per monthWith peak traffic at about 20,000 requests per second
  • And the proportion of API requests coming to the Netflix API through the public has gone way down!The 1,000 flowers represent less than ½ a percent of the total API traffic
  • Now the Netflix API has a new charter…
  • Netflix engineering teams are set up in support of this charter.Internal engineering teams produce or manage content and algorithmic output (the bottom row).Different engineering and product teams are responsible for presentation layers on devices (top row).The API is responsible for delivering the content from the internal engineering teams to the device presentation layers. The API is also responsible for scaling horizontally to handle the growing load.
  • Again, the Netflix API traffic has gone through the roof in this model.
  • Metrics like 30B requests per month sound great, don’t they? The reality is that this number is concerning…
  • In the web world, increasing request numbers mean increasing opportunity of ad impressions, which means increasing opportunity for generating revenue. And when you hit certain thresholds in impressions, CPMs start to rise, which means even more money.
  • Some companies go further to generate more page views by adding things like pagination on article pages. These additional page views translate into additional ad impressions.
  • But for systems that yield output that looks like this, such as APIs, ad impressions are not part of the game. As a result, the increase in requests don’t translate into more revenue. In fact, they translate into more expenses. That is, to handle more requests requires more servers, more systems-admins, a potentially different application architecture, etc.
  • We are challenging ourselves to redesign the API to see if those same 30 billion requests could have been 5 billion or perhaps even less, assuming everything else remained the same. Through more targeted API designs based on what we have learned through our metrics, we will be able to reduce our API traffic as Netflix’ overall traffic grows.
  • As we embark on the redesign of the API, we will plan to design it for our key audiences… the devices.
  • And the redesigned API for the devices will be trickled down to other parties.
  • Along the way, we plan to design the API for the audience that we want, not the audience we have. That doesn’t necessarily mean that we will implement the system for the dream audience. It just means that our designs should allow for the system to scale to the dream audience.
  • For Netflix, we will be designing our server architecture, using AWS, to be highly scalable across many different areas. The API redesign will help us scale the software better to handle the scaling service more effectively and efficiently.

History and Future of the Netflix API - Mashery Evolution of Distribution History and Future of the Netflix API - Mashery Evolution of Distribution Presentation Transcript

  • The Netflix API
    How Netflix Launched an API and Evolved it to Serve Millions on Hundreds of Devices
    By Daniel Jacobson
  • Who Am I?
    Director of Engineering for Netflix API since October 2010
    Tenure at NPR from 1999 to 2010
    Built the custom CMS in 2002
    Extended system for RSS and Podcasting
    Launched the NPR API in 2008
    Launched NPR redesign in 2009
  • Netflix Overview
    Netflix offers subscriptions to unlimited streaming movies and TV shows for a very low price
    About 700 operational employees, 300 engineers
    More than 23 million subscribers in US and Canada
    Market capitalization is about $12B
    Responsible for about 30% of US bandwidth during peak hours (by some accounts)
  • How Does the API Fit in at Netflix?
  • Original Charter for the Netflix API
    Expose Netflix metadata and services to the public developer community to “let 1,000 flowers bloom”. That community will build rich and exciting new tools and services to improve the value of Netflix to our customers.
  • Netflix
    API
    There are currently over
    18,000 flowers
  • Eat Your Own Dog Food
  • New Target Audiences
    Internal Engineering Teams
    Netflix product owners
    Netflix developers
    Partner Relationships
    External device manufacturers
    Public Developer Community
    1,000 flowers
  • Netflix
    API
  • Some of the many
    Netflix-ready
    devices
  • Growth of Netflix API
  • Netflix API Requests by Audience
  • New Charter for the Netflix API
    Build and maintain an infinitely scalable data distribution pipeline for getting metadata and services from internal Netflix systems to streaming client apps on all platforms in the format and/or delivery method that is most optimal for each app and platform.
  • API
    Personalization Engine
    User Info
    Movie Metadata
    Movie Ratings
    Similar Movies
    Reviews
    A/B Test Engine
  • What is the future of the API?
  • Growth of Netflix API
  • Growth of the Netflix API
    About 30 Billion requests per month
  • <catalog_titles>
    <number_of_results>1140</number_of_results>
    <start_index>0</start_index>
    <results_per_page>10</results_per_page>
    <catalog_title>
    <id>http://api.netflix.com/catalog/titles/movies/60021896</id><title short="Star" regular="Star"></title>
    <box_art small="http://alien2.netflix.com/us/boxshots/tiny/60021896.jpg"
    medium="http://alien2.netflix.com/us/boxshots/small/60021896.jpg"
    large="http://alien2.netflix.com/us/boxshots/large/60021896.jpg"></box_art>
    <link href="http://api.netflix.com/catalog/titles/movies/60021896/synopsis"
    rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis"></link>
    <release_year>2001</release_year>
    <category scheme="http://api.netflix.com/catalog/titles/mpaa_ratings" label="NR"></category>
    <category scheme="http://api.netflix.com/categories/genres" label="Foreign"></category>
    <link href="http://api.netflix.com/catalog/titles/movies/60021896/cast"
    rel="http://schemas.netflix.com/catalog/people.cast" title="cast"></link>
    <link href="http://api.netflix.com/catalog/titles/movies/60021896/screen_formats" rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screen formats"></link
    <link href="http://api.netflix.com/catalog/titles/movies/60021896/languages_and_audio" rel="http://schemas.netflix.com/catalog/titles/languages_and_audio" title="languages and audio"></link>
    <average_rating>1.9</average_rating>
    <link href="http://api.netflix.com/catalog/titles/movies/60021896/similars" rel="http://schemas.netflix.com/catalog/titles.similars" title="similars"></link>
    <link href="http://www.netflix.com/Movie/Star/60021896" rel="alternate" title="webpage"></link>
    </catalog_title>
    <catalog_title>
    <id>http://api.netflix.com/catalog/titles/movies/17985448</id><title short="Lone Star" regular="Lone Star"></title>
    <box_art small="http://alien2.netflix.com/us/boxshots/tiny/17985448.jpg" medium="http://alien2.netflix.com/us/boxshots/small/17985448.jpg" large=""></box_art>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/synopsis" rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis"></link>
    <release_year>1996</release_year>
    <category scheme="http://api.netflix.com/catalog/titles/mpaa_ratings" label="R"></category>
    <category scheme="http://api.netflix.com/categories/genres" label="Drama"></category>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/awards" rel="http://schemas.netflix.com/catalog/titles/awards" title="awards"></link>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/format_availability" rel="http://schemas.netflix.com/catalog/titles/format_availability" title="formats"></link>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/screen_formats" rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screen formats"></link>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/languages_and_audio" rel="http://schemas.netflix.com/catalog/titles/languages_and_audio" title="languages and audio"></link>
    <average_rating>3.7</average_rating>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/previews" rel="http://schemas.netflix.com/catalog/titles/previews" title="previews"></link>
    <link href="http://api.netflix.com/catalog/titles/movies/17985448/similars" rel="http://schemas.netflix.com/catalog/titles.similars" title="similars"></link>
    <link href="http://www.netflix.com/Movie/Lone_Star/17985448" rel="alternate" title="webpage"></link>
    </catalog_title>
    </catalog_titles>
  • Improve Efficiency of API Requests
    Could it have been 5 billion requests per month?
    (Assuming everything else remained the same)
  • Design for our Key Audiences
    Some of the many
    Netflix-ready
    devices
  • Future architecture needs to support key audience first with a trickle down of features to the other audiences
    Netflix
    API
  • Design for The Audience We Want
    If our current
    audience is this size
    We will designour system for
    an audience that is this size
  • AWS Cloud
  • Key Lessons from Netflix
    Understand target audiences
    Think big
    Think mobile
    Think international
    Design API for critical audiences first
    Internalize the API as part of your engineering DNA
    If building a public API, help them bloom
  • The Netflix API Team is Hiring!