Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scaling Patterns for Netflix's Edge

47 views

Published on

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2Ezs08q.

Justin Ryan talks about Netflix’ scalability issues and some of the ways they addressed it. He shares successes they’ve had from unintuitively partitioning computation into multiple services to get better runtime characteristics. He introduces us to useful probabilistic data structures, innovative bi-directional data passing, open-source projects available from Netflix that make this all possible. Filmed at qconsf.com.

Justin Ryan is Playback Edge Engineering at Netflix. He works on some of the most critical services at Netflix, specifically focusing on user and device authentication. Years of building developer tools has also given him a healthy set of opinions on developer productivity.

Published in: Technology
  • Be the first to comment

Scaling Patterns for Netflix's Edge

  1. 1. Justin Ryan, Senior Software Engineer @ Netflix Twitter: quidryan, LinkedIn: www.linkedin.com/in/justin-charles-ryan Scaling Patterns for Netflix’s Edge November 2019
  2. 2. InfoQ.com: News & Community Site • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ netflix-edge-scalability-patterns/
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  4. 4. Scaling Patterns for Netflix’s Edge Justin Ryan Laundry
  5. 5. Scaling Patterns for Netflix’s Edge Justin Ryan Laundry
  6. 6. Scaling Patterns for Netflix’s Edge Justin Ryan Laundry
  7. 7. Scaling Patterns for Netflix’s Edge Justin Ryan Laundry
  8. 8. Scaling Patterns for Netflix’s Edge Justin Ryan Space-Time Compromise
  9. 9. Scaling Patterns for Netflix’s Edge Justin Ryan Laundry
  10. 10. Scaling Patterns for Netflix’s Edge Justin Ryan Scaling Q32011 Q32019
  11. 11. Scaling Patterns for Netflix’s Edge Justin Ryan Scaling Edge
  12. 12. Scaling Patterns for Netflix’s Edge Justin Ryan Scaling Problems — Debuggability — Infrastructure — Managing
  13. 13. Scaling Patterns for Netflix’s Edge Justin Ryan Scalability
  14. 14. Scaling Patterns for Netflix’s Edge Justin Ryan Trade Offs
  15. 15. Scaling Patterns for Netflix’s Edge Justin Ryan Policies
  16. 16. Scaling Patterns for Netflix’s Edge Justin Ryan Accessible
  17. 17. Scaling Patterns for Netflix’s Edge Justin Ryan Five Use Cases — Melnitz — Mantis — Passport — Device Types — Sharding
  18. 18. Scaling Patterns for Netflix’s Edge Justin Ryan Logged In
  19. 19. Scaling Patterns for Netflix’s Edge Justin Ryan Cookies
  20. 20. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz Trust Cookie Cookie Expiration
  21. 21. Scaling Patterns for Netflix’s Edge Justin Ryan Cookies L7 Proxy Account Service API Expired Yes No
  22. 22. Scaling Patterns for Netflix’s Edge Justin Ryan Boom
  23. 23. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz Copyright Columbia Pictures
  24. 24. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz Set<CustomerId> Cookie Expiration
  25. 25. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz Bloom Filter: Compact Set which can answer the question is an object “Possibly in Set”?
  26. 26. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz
  27. 27. Scaling Patterns for Netflix’s Edge Justin Ryan Side Note: Bloom Filter — Bad Interview Question? Yes — Used in complex databases? Yes — Guava Library? Yes — Usable by you in your day-to-day jobs? Yes
  28. 28. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz L7 Proxy Account Service API Bloom Filter Expired Yes No Maybe No
  29. 29. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz Account Service Auth Event Melnitz A DB Melnitz B 1m L7 Proxy
  30. 30. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz
  31. 31. Scaling Patterns for Netflix’s Edge Justin Ryan Melnitz Trade Offs — Probabilistic Data Structure — Off the Shelf Components
  32. 32. Scaling Patterns for Netflix’s Edge Justin Ryan Side Note: Bloom Filter
  33. 33. Scaling Patterns for Netflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis — Passport — Device Types — Sharding
  34. 34. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis
  35. 35. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis 2 million request per second * 3 kilobytes per request * 10 micro-services = 57 GB/s =~ 4.6 PB / day
  36. 36. Scaling Patterns for Netflix’s Edge Justin Ryan Naive Approach Service Elastic Search Kibana
  37. 37. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis
  38. 38. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis Master Mantis Service Mantis Agent Mantis Source Job Elastic Search Kibana Kafka
  39. 39. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis Master Service Mantis Mantis Source Job Elastic Search Browser Mantis Agent Kafka
  40. 40. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis Master Mantis Service Mantis Agent Mantis Source Job Elastic Search Browser Metrics Javascript Transform
  41. 41. Scaling Patterns for Netflix’s Edge Justin Ryan Avoid Work
  42. 42. Scaling Patterns for Netflix’s Edge Justin Ryan Mantis Trade Offs — Mantis Query Language — Scale Solution to Zero
  43. 43. Scaling Patterns for Netflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport — Device Types — Sharding
  44. 44. Scaling Patterns for Netflix’s Edge Justin Ryan Passport
  45. 45. Scaling Patterns for Netflix’s Edge Justin Ryan Service Plan API L7Proxy Mid Tier A Mid Tier B Mid Tier C Passport
  46. 46. Scaling Patterns for Netflix’s Edge Justin Ryan Passport Service Plan API L7Proxy Mid Tier A Mid Tier B Mid Tier C
  47. 47. Scaling Patterns for Netflix’s Edge Justin Ryan Cache Service Plan API L7Proxy Mid Tier A Mid Tier B Mid Tier C
  48. 48. Scaling Patterns for Netflix’s Edge Justin Ryan Cache API L7Proxy Mid Tier A Mid Tier B Mid Tier C Service Plan
  49. 49. Scaling Patterns for Netflix’s Edge Justin Ryan Cache API L7Proxy Mid Tier A Mid Tier B Mid Tier C Service Plan
  50. 50. Scaling Patterns for Netflix’s Edge Justin Ryan Passport
  51. 51. Scaling Patterns for Netflix’s Edge Justin Ryan Passport Trade Offs — Data Passing — Heavy Data Structures
  52. 52. Scaling Patterns for Netflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport - Heavy Data Structure — Device Types — Sharding
  53. 53. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types
  54. 54. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types NFAPPL-01-IPAD7=5-09BEC57F64CA5F837C75F924B94D NFAPPL-01-IPAD3=4-15B2D9B33A994CC373017A9C6586 NFAPPL-02-IOS2=4-FT7WHG8H2UR2MCJ4YKR3P0DFH6 NFANDROID1-PRV-P-GOOGLPIXEL=2-7351-0F441DF98B8D99074 NFANDROID1-PRV-4339-0E50FC7582396F5C4496ED935F2F48ED 1395 1007 964
  55. 55. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types
  56. 56. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types
  57. 57. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types Mid-Tier F Mid-Tier E Mid-Tier D Mid-Tier C Mid-Tier BMid-Tier A DTS
  58. 58. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types Mid-Tier CMid-Tier BMid-Tier A DTS PubSub
  59. 59. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types Mid-Tier CMid-Tier BMid-Tier A DTS PubSub Device Types Device TypesDevice Types
  60. 60. Scaling Patterns for Netflix’s Edge Justin Ryan Device Types
  61. 61. Scaling Patterns for Netflix’s Edge Justin Ryan Device Type Trade Offs — Business Tuned Fallbacks — Leverage Existing Infrastructure
  62. 62. Scaling Patterns for Netflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport - Heavy Data Structure — Device Types - Device Grouping — Sharding
  63. 63. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding
  64. 64. Scaling Patterns for Netflix’s Edge Justin Ryan MSL — Message Security Layer == MSL — Secure Messaging Framework
  65. 65. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding MSL L7Proxy APIAPIAPIAPIAPIAPI
  66. 66. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding MSL L7Proxy MSL APIAPIAPI
  67. 67. Scaling Patterns for Netflix’s Edge Justin Ryan CPU Cost Per Request - CPU to RPS ratio - Load Average 31% decrease
  68. 68. Scaling Patterns for Netflix’s Edge Justin Ryan Latency 30% decrease 20% p99 decrease
  69. 69. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding MSL
  70. 70. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding MSL - % of Wall Clock in GC - GC Pause Time
  71. 71. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding MSL - Average Latency - 99% Latency
  72. 72. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding 3 = 1 + 1
  73. 73. Scaling Patterns for Netflix’s Edge Justin Ryan Sharding Trade Offs — Operational Overhead
  74. 74. Scaling Patterns for Netflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport - Heavy Data Structure — Device Types - Device Grouping — Sharding - Runtime Refactoring
  75. 75. Scaling Patterns for Netflix’s Edge Justin Ryan Trough
  76. 76. Scaling Patterns for Netflix’s Edge Justin Ryan Trough - Used Instances - Reserved Instances
  77. 77. Scaling Patterns for Netflix’s Edge Justin Ryan Pattern #6 Trough — Leverage Unused Resources
  78. 78. Scaling Patterns for Netflix’s Edge Justin Ryan Trade Odds ● Probabilistic Data Structure ● Off the Shelf Components ● Avoid Work ● Scale Solution to Zero ● Data Passing ● Heavy Data Structures ● Leverage Existing Infrastructure ● Business Tuned Fallbacks ● Runtime Refactoring Melnitz Melnitz Mantis Mantis Passport Passport DTS DTS Sharding
  79. 79. Scaling Patterns for Netflix’s Edge Justin Ryan When to Use Debuggability ✗ ✓ ✓ ✓ ✓ ✗ Infrastructure ✓ ✗ ✗ ✓ ✗ ✗ Managing ✗ ✗ ✗ ✓ ✗ ✗ Costs ✓ ✓ ✗ ✗ ✓ ✓ Melnitz Mantis Passport DeviceTypesSharding Trough
  80. 80. Scaling Patterns for Netflix’s Edge Justin Ryan Bonus Trade Offs — Do the laundry
  81. 81. Justin Ryan jryan@netflix.com Twitter: @quidryan Thank You.
  82. 82. Scaling Patterns for Netflix’s Edge Justin Ryan References — github.com/Netflix ◆ eureka ◆ zuul ◆ mantis ◆ msl — Spinnaker — Apache Pulsar — Google Guava (Bloom Filter)
  83. 83. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ netflix-edge-scalability-patterns/

×