API World 2013 - Transforming the Netflix API
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

API World 2013 - Transforming the Netflix API

on

  • 4,265 views

The Netflix API has undergone a transformation since its inception in 2008. It has transitioned from being a public API with a generic RESTful interface to a platform for creating highly optimized, ...

The Netflix API has undergone a transformation since its inception in 2008. It has transitioned from being a public API with a generic RESTful interface to a platform for creating highly optimized, device-centric APIs that are critical to delivering the Netflix streaming experience on over 1000 different device types.

This talk covers the design principles that shaped the transformation of the API as well as the technology that powers it, enabling rapid user experience iteration and bringing Netflix streaming to almost 38 million subscribers around the world.

Statistics

Views

Total Views
4,265
Views on SlideShare
2,765
Embed Views
1,500

Actions

Likes
8
Downloads
41
Comments
0

5 Embeds 1,500

http://www.scoop.it 1375
https://twitter.com 120
http://moderation.local 2
https://www.google.com 2
http://www.google.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

API World 2013 - Transforming the Netflix API Presentation Transcript

  • 1. Transforming the Netflix API Ben Schmaus, Netflix October 2013, API World bschmaus@netflix.com || @schmaus
  • 2. Streaming TV Shows & Movies Globally
  • 3. > 1000 Devices
  • 4. 1/3 of Internet at peak
  • 5. Exclusive Content
  • 6. Almost 38 million subscribers in over 40 countries
  • 7. Personalization Engine User Info Movie Metadata Ratings Similar Movies Instant Queue A/B Test Engine API
  • 8. Personalization Engine User Info Movie Metadata Ratings Similar Movies Instant Queue A/B Test Engine API Enable UX Innovation Insulate from Failure
  • 9. It wasn’t always this way
  • 10. Foster Innovation from Outside
  • 11. REST for Easy Interop, Integration RESTful API
  • 12. REST for Easy Interop, Integration RESTful API Model resources like users, movies, series, ratings, etc
  • 13. REST for Easy Interop, Integration RESTful API /catalog/titles /catalog/titles/movies /catalog/titles/series /catalog/titles/series/episodes
  • 14. REST for Easy Interop, Integration RESTful API /users/lists /users/title_states /users/ratings
  • 15. Enter Streaming Devices
  • 16. Lots of devices, lots of variety across platforms
  • 17. External apps
  • 18. 1 /apps/{app_id}/config 1 /users/{user_id} 1 /users/{user_id}/queues/instant/available 1 /users/{user_id}/lists 26 /catalog/titles?...
  • 19. Doesn’t include JS, CSS, images, etc. 1 /apps/{app_id}/config 1 /users/{user_id} 1 /users/{user_id}/queues/instant/available 1 /users/{user_id}/lists 26 /catalog/titles?...
  • 20. Device variance: interaction models
  • 21. Device variance: interaction models & form factors
  • 22. A/B Tests Add to the Challenge
  • 23. “Bolt on” functionality expand=@queue_item@title,@box_art_exists, @short_synopsis,@directors,@cast,@episodes,@formats, @format_availability,@subtitle_languages, @languages_and_audio,@maturity_ratings
  • 24. Resource-based API wasn’t scaling
  • 25. Model Resources /catalog, /users Model Experiences /tv/home
  • 26. Reduce network chattiness Support device optimizations Enable fast dev iterations
  • 27. GET /users/{user_id}/lists apiGateway.getLists(userId)
  • 28. Discrete HTTP requests pay WAN tax repeatedly
  • 29. Single, optimized request; pay WAN tax once
  • 30. Single, optimized request; pay WAN tax once Client data assembly logic pushed to server
  • 31. Add server-side scripting capability Enable independent dev and device optimization
  • 32. API Internet /tv/home
  • 33. API /tv/home Client Server Internet
  • 34. API Application Client Server Internet /tv/home
  • 35. API Client Server Internet UI Teams Application /tv/home
  • 36. API API Team Client Server Internet UI Teams Application /tv/home
  • 37. API UI Teams API Team Mid-tier Services Client Server Internet Application /tv/home
  • 38. RxJava Hystrix API Team JavaServiceLayer Mid-tier Services UI Teams Client Server Internet Application /tv/home
  • 39. ELB Zuul Mid-tier Services Scriptable Backend Scriptable Backend + API Layer
  • 40. https://github.com/Netflix/Hystrix resilience patterns for distributed systems https://github.com/Netflix/RxJava async, event-based programming https://github.com/Netflix/zuul dynamic request routing, realtime monitoring, resiliency
  • 41. Need well-appointed toolkit
  • 42. Cassandra upload script.groovy for /script
  • 43. Cassandra Cassandra activate /script upload script.groovy for /script
  • 44. Cassandra Cassandra activate /script Cassandra Scriptable Backend load /script upload script.groovy for /script
  • 45. Cassandra Cassandra activate /script Cassandra Scriptable Backend load /script upload script.groovy for /script Scriptable Backend Device Client GET /script
  • 46. Deployment & Ops
  • 47. Historical & realtime metrics, sort realtime by error/request rate
  • 48. Distributed grep + tail 2013-05-09.20:38:54 MX 200 us-east-1c i-1824cb73 i-1c61b77f prod NFPS3-001-8G50FJCX... 288404769389848058 90ms api-global.netflix.com GET /tvui/release/470/plus/pathEvaluator - amazon.ami-id: ami-502eb039 amazon.availability-zone: us-east-1c amazon.instance-id: i-1824cb73 amazon.instance-type: m2.2xlarge amazon.local-ipv4: 10.6.213.112 amazon.public-hostname: ec2-54-243-4-69.compute-1.amazonaws.com amazon.public-ipv4: 54.243.4.69 cookie_esn: NFPS3-001-8G50FJCX... country: MX currentTime: 1368131934468 duration-millis: 90 esn: NFPS3-001-8G50FJCX... geo.city: CIUDADOBREGON ... $ ./simple_stream.py -f -q 'e["country"]=="MX" && e["esn"] ==~/NFPS3.*/' -r us
  • 49. Go for haystack handing you the needle
  • 50. Or at least be able to make smaller haystacks
  • 51. Run 1% of your traffic on the new code and see how it does
  • 52. API ami-123 API ami-456 2xx 4xx 5xx latency busy threads load ...
  • 53. Confidence score for each AMI based on comparison of 1000+ metrics
  • 54. Scannable visualization of metric space More important Less important
  • 55. Your basic red/black push
  • 56. Doing red/black by hand for multiple clusters across multiple regions is not fun
  • 57. Automate multi-cluster/region pushes
  • 58. Automate multi-cluster/region pushes Don't forget to automate rollbacks, too!
  • 59. $Who, $What, $Where, $When e.g., "bschmaus, ami-123, Sandbox Canary, 2013-05-06 19:05" Latest prod change in chat topic
  • 60. Quickly see status of all clusters in a region
  • 61. API for APIs Control network interactions Device optimization Development agility Automation & insights
  • 62. Continuously experiment to make hard things easier
  • 63. bschmaus@netflix.com @schmaus Great Engineers Wanted! jobs.netflix.com