Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Like this? Share it with your network

Share

API Strategy Evolution at Netflix

on

  • 157,906 views

Developing an API strategy should be considered a journey, not a project with a predetermined outcome. This presentation describes Netflix's journey to discover a winning API strategy as well as ...

Developing an API strategy should be considered a journey, not a project with a predetermined outcome. This presentation describes Netflix's journey to discover a winning API strategy as well as future directions for the API.

Statistics

Views

Total Views
157,906
Views on SlideShare
13,061
Embed Views
144,845

Actions

Likes
22
Downloads
539
Comments
2

20 Embeds 144,845

http://techblog.netflix.com 142063
http://www.hackingnetflix.com 1550
http://elasticenterprise.wordpress.com 782
http://theelasticenterprise.com 407
http://static.slidesharecdn.com 10
http://www.twylah.com 8
http://translate.googleusercontent.com 7
http://dashboard.bloglines.com 2
https://p.yammer.com 2
http://paper.li 2
http://xss.yandex.net 2
http://webcache.googleusercontent.com 2
http://www.mashup.or.kr 1
http://rahasailio 1
https://mdc-web-tomcat17.ubisoft.org 1
http://runninginflipflops.com 1
http://107.22.196.6 1
http://rdbcci:8100 1
http://www.hanrss.com 1
http://www.newsblur.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • I love these slides! Did you know we’re running a competition on SlideShare to win a 3M PocketProjector MP180? To enter, simply tag your presentation with ‘3MInform’. Head over to our page for more details... and don’t forget to follow us to find out if you get shortlisted!
    Are you sure you want to
    Your message goes here
    Processing…
  • Michael,

    Love the service but let's hope my mom, who loves it too, doesn't see your slide 17 or for that matter half of the US population as it will be pretty embarrassing for you folks (Reed too) to defend the use of an Evolution reference in a tech preso.

    ~Clint
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • The first point I’d like to make is that API strategy, like any strategy, is only valid in a particular context, the opportunities and constraints within which each business operatesYour mileage may vary.
  • The initial API strategy could be called A Thousand Flowers, as in “let a thousand flowers bloom”
  • The inspiration for the API program came from the same inspiration as the Netflix prize- an attempt to outsource some aspects of innovation.The Netflix Prize: thousands of teams spent almost3 years trying to come up with a recommendations engine that could beat our own by 10% and win $1M.A testament to not only our ability outsource some aspects of innovation, but also to our own ability to innovate: it took three years for thousands of teams to beat the recommendations engine we developed by just 10%.Outsourcing some innovation is just one of the ways Netflix tries to stay small, a “big startup”.
  • This is a slide I showed at the Business of APIs conference two years ago showing a range of potential partners from large to small.Could we recreate the same innovation dynamic around an API as we did with the Netflix Prize, enabling external innovation around our API to create user experiences that would improve movie discovery, especially by enabling the smaller developers down the tail to have an equal shot of innovating with our service as larger potential partners?We win if one developer comes up with a very big idea or a bunch of developers develop apps that addressed specific niches that, in total, would be significant.@Netflix: the simplest measure of “significant” for any initiative means delivering some tangible value to at least 20% of our subs in any given month. That criteria helps us maintain focus and stay small and nimble by not chasing small opportunities. What is a small opportunity for Netflix might be a relatively larger opportunity for others business. Again, YMMV.
  • As you can see in this distribution of API-enabled application usage, the long tail didn’t really materialize.Furthermore, the total value delivered didn’t come close to our 20% bar.Most popular apps weren’t all that innovative in terms of scenarios, they just delivered our existing scenarios to form factors it wasn’t efficient for us to reach. Mobile apps were the most successful, but they were just delivering existing Netflix website scenarios like search and recommendations. Why? With varied implementation technologies (C, C#, Objective C), and unclear sub reach, it would have been costly in both real and opportunity costs for Netflix to experiment with developing apps for these platforms, and external developers filled that void.Even with that bright spot, overall sub usage wasn’t high enough to justify the API program alone.
  • The something interesting happened: Xbox approached us for ideas for the v2 version of the Netflix applicationWe were already seeing signs that streaming device reach was big driver for the business.Up until this point, our streaming device apps didn’t support discovery of titles (we weren’t sure we could do this well).We also delivered our streaming app to partners via a C-based Linux application supported by a client SDK in turn supported by private APIs separate from our public API. A partner app couldn’t be built directly against this API and minor extensions to the API required a whole new API version. Porting the app and SDK to a non-Linux platform like Xbox also took extra time.It took at least a quarter to roll out new scenarios through this implementation stack.Our private API plus client SDK didn’t have anything new to offer over the v1 application.Did we just learn something from the 1000 flowers experiment? The REST-based open API could be revved more quickly and was easier to work with cross-platform given it was designed without any client requirement It happened to have a new recommendation API, internally called LOLOMO for “Lists of Lists of Movies”, which was our first foray into enabling subscribers to discover movies and tv shows on a device.LOLOMO was integrated into Xbox using the open API and streaming usage went up noticeably after launch. It was now clear that we could deliver new, Netflix-originated scenarios quickly to devices via the open API.
  • Exciting, but that put a big “?” on the 1000 flowers strategy.Rights restrictions on streaming titles demanded that they be delivered only within Netflix-branded applicationsNetflix branded applications demanded an application certification process to ensure a quality experience for our subscribers. Unlike third-party applications, our name was on the app which meant we couldn’t simply disable it if it was a disappointing experience for our subscribers, used the API inefficiently, etc.However IW device reach had proven to be a major driver for our business, and the user attach rate per Netflix app was much higher than for third-party appsCouple that with the possibility of increased viewing per device, and it became possible to move metrics beyond sub usage to subscriber acquisition, retention and lower distribution costs due to streams substituting for disc shipments
  • Achieved fantastic success accelerating the rollout of proven app scenarios quickly across branded Netflix applicationsAPI gave us flexibility to deliver to quickly to C/Linux apps, Flash apps, and moreAlso variety of development approaches, Netflix, vendor or partner. Equal access to our service functionality for all.That flexibility in both client platforms and development approaches helped us deliver these new scenarios quickly to all three major game consoles and the top 10 brands in TVs and Blu-ray playersThis picture is from May, we’re now on over 200 devices at retailAt this point, our investment in the API had clearly paid off. Over 60% of subs streamed last quarter and we’re also realizing significant per sub streaming increases, albeit some percentage are PC and pre API devices.But having achieved this kind of TV device reach, what would we do next?
  • Mobile was next frontier for us, having reached this critical mass of TV connected devices.At this time, we were also investigating the promise of using HTML 5 andJavascript stack for our device applications. HTML 5 offered a number of potential benefits.Deliver compelling device experiences with visual effects that didn’t make you feel like you were on a PC website.User expectations on TV devices: EPG, performance you get with native apps and local data. Game consoles bar is higher. On mobile, most native apps trump web apps with similar functionality.Despite the outsourcing of some innovation through programs like the API and the Netflix Prize, we do also do a fairly good job of innovating at Netflix. Innovation through iteration on www through two week site updates.Web delivered UI gave us the power to completely control our experience on partner devices. We don’t have to incur the release delays associated with iPhone app store or other device partner application approval or certification processes, since updates to HTML content are very low risk compared to native code updates.Our iPhone app shown here is basically a browser shell for an HTML UI with extensions for DRM’d video playback. If you play this out to a logical conclusion, it could mean the end of the API usage for delivering Netflix device experiences at some point in the future. Server-generated HTML web pages wouldn’t necessarily need a publicly accessible API.Server-generated web pages couldn’t achieve performance goals without leveraging AJAX heavily. Consider a two-dimensional grid of movie recommendations. Only part of the grid is visible at any time, and offscreen titles can be loaded incrementally. API responses are also smaller network payloads than downloaded HTML.The API gave us exactly what we needed to build great AJAX-powered experiences. And, guess what? Most compelling mobile platforms like iPhone and Android lead in HTML 5 browser penetration. In fact, unlike on a PC, it’s only HTML 5 browsers on those platforms. No cross-browser compatibility concerns on a device.We now could leverage the API effectively for the aforementioned device applications plus a Netflix-delivered, web-based user on HTML-5-enabled mobile devices. But, we didn’t stop at mobile.
  • What if instead of implementing our UI natively on a TV device, we brought WebKit to the party instead?
  • Not only do we update service features rapidly on two week release cycles, all new features or even incremental enhancements to existing features also undergo rigorous A/B testing, involving different user experiences being tested in parallel with distinct groups of users. This is the only practical way to ensure that new experiences are indeed better than their predecessors, and not simply outperforming them due to other external factors like seasonality, new content license deals, etc. We’ve been employing these techniques to improve our website experience for years.When we re-launched our PS3 app a few weeks back we began testing of 4 different app experiences concurrently.Once an API-supported scenario has been rapidly developed and tested on one or more HTML 5 applications, it can be finalized and then be leveraged in other less updateable application.
  • Once your API starts supporting rapid product innovation, you have to ensure that rapidly developed APIs supporting test experiences for thousands of users don’t affect the stability of APIs supporting millions of users. How do you balance these competing goals? You actually don’t have to choose between the two.Provisional APIs are one tactic for addressing this tension. Since new features are tested on platforms with easily updated client code, provisional APIs can be developed for a test scenario and then deprecated or modified before creating stable APIs that can be used by a wider range of clients as well as external developers. The test client can be updated to manage these changes as necessary.These provisional APIs can also be deployed and operated on different server clusters than the stable APis, minimizing the chance of less tested, more rapidly developed code being tested with thousands of users from potentially destabilizing services being used by millions of other users. Apigee’s Gateway allow you to easily route incoming API traffic to enable this partitioning.
  • Call this the third evolution of Netflix API Strategy: Tons of Tests. While any potential competitor will have to work hard to get our device reach, we’ll be accelerating our user experience innovation on top of our API.
  • Despite these major changes, the API has still been able to achieve the goals of the company to the point that it is now serving roughly ? billion requests per month.
  • Over the last two years, the API has seen a ton of changes. Some tactical, some fundamental.
  • All under great time constraints and pressure to propel the business forward.

API Strategy Evolution at Netflix Presentation Transcript

  • 1. API Strategy Evolution at
    Michael Hart (@michaelhart)
    Director of Engineering, Social Systems
    (Former Director of Engineering, API)
    Daniel Jacobson (@daniel_jacobson)
    Director of Engineering, API
  • 2. Presentation is designed for presenting vs. reading.
    Please review speaker notes if reading online.
  • 3. (disclaimer)
    YMMV
  • 4. A Thousand Flowers
  • 5.
  • 6. Reach and Usage
    Sites, Applications, Devices
    Netflix Freak
  • 7.
  • 8.
  • 9. A Thousand Flowers?
  • 10. Hundreds of Devices
  • 11.
  • 12.
  • 13.
  • 14. Agility vs. Stability
  • 15. Tons of Tests
  • 16. Recap:
    A Thousand Flowers
    Hundreds of Devices
    Tons of Tests
    More to come…
    ?


  • 17. Developing an effective API strategy is a journey- not just a single project
    Be prepared to evolve
  • 18. The Future of the Netflix API
    To Infinity and Beyond…
  • 19. Growth of the Netflix API
    Over 10 BILLION requests in November
    (Peaks at about 10,000 requests per second)
  • 20.
  • 21.
  • 22.
  • 23. It is time to get back to thinking about the future…
  • 24. First Step: Back to the Drawing Board…
  • 25. Some of our Key Goals
    Resiliency, Reliability and Stability
    Maximize uptime
    Scalability
    System can grow without needing frequent refactoring
    Simplicity
    Easier integration for API consumers
    Agility
    Architecture will support rapid development without compromising the core of the system
  • 26.
  • 27. Wii
    iPad App
    PS3
    iPhone
    App
    Netflix
    API
    Apple TV
    Google TV
    TiVo
    Roku
    XBox
  • 28. Future Architecture needs to support key audience first with a trickle down of features to the public audience
    Wii
    Roku
    PS3
    iPhone
    App
    iPad App
    Google TV
    Netflix
    API
    Apple TV
    XBox
    TiVo
  • 29. Hide Our Internal Business Dependencies from API Consumers
  • 30. (Some) API Interfaces and Their Dependencies
    Auto-Complete
    Rental History
    Agg
    Lists
    Similars
    LIsts
    Queues
    SEARCH
    SIMILARS SYSTEM
    RECOMMENDATION ENGINE
    QUEUE
    RENTAL HISTORY
    Title Details
    Previews
    Ratings
    Recommendations
    Title States
    CINEMATCH
    CONTENT MANAGEMENT SYSTEM
    TITLE STATES
    REST Interface
    API Dependency
  • 31. INTERNET REQUEST
    UNIFIED LIST/TITLE RESPONSE
    WITH PARTIAL RESPONSE BUILT IN
    SIMILARS
    LIST
    SEARCH
    LIST
    RENTAL HISTORY
    LIST
    TITLE LIST
    QUEUE
    LIST
    AUTO
    COMPLETE
    LIST
    ETC…
    TITLE INFORMATION
    TITLE
    STATE
    RATING
    RECOMMEND
    ATION
    TITLE DETAILS
  • 32. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 33. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    THESE ARE SOME OF THE DATA SOURCES THAT PROVIDE CONTENT TO THE API
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 34. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    THESE ARE SOME OF THE SERVICES PROVIDED BY SOME OF THE API DEPENDENCIES THROUGH WHICH CONTENT IS MADE AVAILABLE TO THE API
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 35. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    THIS IS THE LAYER WHERE THE API CAN NORMALIZE THE DATA TO PREPARE IT FOR CONSUMPTION BY API CLIENTS. IT COULD ALSO BE WHERE THE API CAN APPLY RESILIENCY MEASURES TO PROTECT AGAINST FAILURES FROM THE API DEPENDENCIES.
    ALTHOUGH THE DIAGRAM REPRESENTS THIS AS A DISCRETE API LAYER, IT COULD JUST AS EASILY BE A DISTRIBUTED MODEL ACROSS THE DEPENDENCIES.
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 36. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    MAKING PURE APIs WILL IMPROVE THE CONSISTENCY OF THE API CODE AND THOSE THAT CONSUME IT. THIS WOULD HELP PRODUCT MANAGERS AS WELL SINCE THE CODE FOR THEIR PRODUCTS CAN ALSO BECOME MORE UNIFORM.
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 37. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    THE SHARED API INTERFACE LAYER ALLOWS THE VARIOUS CLIENTS TO MERGE RESULTS FROM THE PURE APIs AND PREPARE THEM IN WAYS THAT ARE CONDUCIVE TO THE GOALS OF THEIR APP. MEANWHILE, ANY OF THESE INTERFACES CAN BE SHARED ACROSS THE CLIENTS SINCE MANY OF THESE INTERFACES WILL BE COMMON ACROSS THE PLATFORMS.
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 38. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    THE WRAPPER LAYER IS DESIGNED TO CONSUME THE PURE APIS AND/OR THE SHARED INTERFACES WHILE ALLOWING DEVICE/APP-SPECIFIC IMPLEMENTATIONS. SOME OF THESE WRAPPERS ARE THEORETICAL WHILE OTHERS MAY ACTUALLY TAKE ADVANTAGE OF THIS TIER.
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 39. Recommend-ation
    Recommend-ation Service
    iPhone Wrapper
    Xbox Wrapper
    Ratings Service
    SHARED
    API
    INTER-FACES
    LIST API
    Ratings
    Wii Wrapper
    THESE ARE SOME OF THE ACTUAL INTERNAL API CUSTOMERS TODAY. THIS LIST WILL GROW OVER TIME, BUT THIS ARCHITECTURE WILL SCALE ACCORDINGLY.
    PS3 Wrapper
    TITLE API
    Rental History
    Rental History Service
    Roku Wrapper
    USER API
    User Account
    iPad Wrapper
    Normal-ization
    And Resiiency
    AppleTV Wrapper
    Queue
    AUTH
    API
    PC / Mac Wrapper
    Title Catalog
    TiVo Wrapper
    Source Data
    Layer
    API Repository
    Layer
    Web Service
    Layer
    Wrapper
    Layer
    Shared
    Layer
    App Layer
    API Layer
  • 40. Flexible API RequestsAs flexible as SQL
    SELECT person_first_name, person_last_name, dept_name
    FROM person, department
    WHERE UPPER(dept_name) LIKE ‘%Engineering%’
    AND person_added_date BETWEEN ‘2002-01-01’ AND sysdate
    ORDER BY person_last_name ASC
    Manages Partial Response
    Determines the Selected API
    Restricts Output
    Sorts Results
    GOAL: API should be able to handle any request that you can do in SQL
  • 41. The Problem with Versioning
    1.0
    1.5
    2.0
    3.0?
    4.0?
    5.0?
    Today
  • 42. Maybe we don’t need versioning?
    1.0
    1.5
    2.0
    Future APIs
    Today
  • 43. Branching can always be done later…
    1.0
    1.5
    2.0
    New API
    Today
  • 44. XML vs. JSON?Or both? Or other?
    xml
  • 45. REST…as long as it makes sense for the business
  • 46. Questions?
    API Management
    Engineering Manager – Web API Systems: http://bit.ly/9O3Hic
    API Integration Systems Analyst: http://bit.ly/hyPxCh
    API Engineering
    Senior Software Engineer/Architect – Web API Platform: http://bit.ly/e0AKw4
    API Test Engineering
    Senior Software Engineer in Test – API: http://bit.ly/9zHPtJ
    Senior Software Engineer in Test – API Platform: http://bit.ly/eD3tLn
    Lead Software Engineer in Test – API: http://bit.ly/fqxBYC
    Social Engineering
    Social Systems – Engineer Architect: http://bit.ly/dYq6C8