MED202 Netflix’s Transcoding Transformation - AWS re: Invent 2012

  • 7,809 views
Uploaded on

Netflix designed a massive scale cloud based media transcoding system from scratch for processing professionally produced studio content(to meet the unique scale and time constraints of our business). …

Netflix designed a massive scale cloud based media transcoding system from scratch for processing professionally produced studio content(to meet the unique scale and time constraints of our business). We bucked the common industry trend of vertical scaling and, instead, designed a horizontally scaled elastic system using AWS to meet the unique scale and time constraints of our business. Come hear how we designed this system, how it continues to get less expensive for Netflix, and how AWS represents a transformative opportunity in the wider media owning industry.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,809
On Slideshare
0
From Embeds
0
Number of Embeds
30

Actions

Shares
Downloads
0
Comments
0
Likes
11

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. tweet #reinvent
  • 2. Netflix Media In AWS  Matrix : The Netflix media pipeline  MAPLE : Next generation media pipeline  Digital vaulting opportunity  Concluding thoughts 2
  • 3. Netflix Media Pipeline Open Connect FTP S3 S3 EC2 EC2 EC2 3
  • 4. Driving To Hollywood Game 4
  • 5. Rules of the game 200 MPH!  85 MPH Purchase only  Lease, cancel anytime Quantities limited  Unlimited quantity It breaks, you fix it  It breaks, replace it, no charge Pay for parking  No parking, just walk away Obsolete in 1 year  Brand new each year 5
  • 6. Industry heritage : optimize for latency  Interactive editing  Master creation  DVD/Blu-ray authoring  Edits for television 6
  • 7. Netflix 2008  Custom data center  Custom GPU Encoders  Fixed size  New format needed  PC, Mac, Xbox  Content library doubled  Frequent HW failures  Fail! Catalog incomplete 7
  • 8. Fall 2009 – Launch Netflix PS3 Player  First 100% AWS transcode  New format, unique to Netflix PS3 player  Encode recipe nailed down late  3 weeks, transcode entire catalog 8
  • 9. Netflix 2009 to present  US East AWS  Variable sized EC2 farm  S3 for storage  Optimized for throughput, not latency  No more missed deadlines  Devices, catalogs, countries 9
  • 10. Spring 2010 – Launch Netflix IPad Player  Launch April 10th  Apple approached us in mid February  Grew EC2 farm to 4,000 instances  Entire library transcoded in 2 weeks  New format ready for launch 10
  • 11. Netflix Media Pipeline Open Connect FTP S3 S3 EC2 EC2 EC2 11
  • 12. For Netflix, throughput trumps latency Think horizontal, not vertical Priuses move more people than Ferraris Frequent re-encodes of growing libraries Netflix is nimble because of AWS 12
  • 13. More proof that horizontal wins New countries, new content Codec innovation 13
  • 14. AWS handles Netflix scale 5 regional catalogs 4 formats supported today  1 VC-1, 3 H.264  Multiple bit rates per format 10’s of 1000’s of hours of content Several petabytes of S3 storage 14
  • 15. Netflix Media In AWS  Matrix : The Netflix media pipeline  MAPLE : Next generation media pipeline  Digital vaulting opportunity  Concluding thoughts 15
  • 16. Next Generation : Address faults and latency  2 to 3 day HD transcode  Fault intolerant  Maintenance is challenging 80 – 100 Mbps  2 to 3 days is sometimes too EC2: C1 Medium slow 4 Mbps  Day after broadcast  Redelivery of damaged content S3 16
  • 17. MAPLE : Massively Parallel Encoding  5 minute chunks  Close to ‘real time’  Fault tolerant  Easy maintenanceEC2  Address low latency use cases  Day after broadcast  Redelivery of damaged content S3  In development now 17
  • 18. Netflix Media In AWS  Matrix : The Netflix media pipeline  MAPLE : Next generation media pipeline  Digital vaulting opportunity  Concluding thoughts 18
  • 19. Versionitis Problem In Hollywood Dr. Zhivago  1965  Cinematography Oscar 19
  • 20. Dr. Zhivago On Netflix Streaming 2.20 : 1 Original Aspect Ratio 4:3 Television Aspect Ratio 20
  • 21. Original Video DNA Original production master Native frame rate Native aspect ratio Whatever came out of the production cameras 21
  • 22. DNA Based Digital Vault English English Mandarin Spanish 22
  • 23. Opportunity for content owners  On time delivery  Low error rates  More $$ licensing opportunities  October 15 Nordic launch 23
  • 24. AWS Glacier For Digital Vault Storage  Glacier  $.01 per GB per month  Bitrate 80 Mbps  1 hour = 35GB  $0.35 per hour per month  $4.22 per hour per year 24
  • 25. Netflix Media In AWS  Matrix : The Netflix media pipeline  MAPLE : Next generation media pipeline  Digital vaulting opportunity  Concluding thoughts 25
  • 26. We would do it all over again  Don’t be fooled by IT cost comparisons  We don’t administer the gear  6,000 EC2 instances  Petabytes of storage  High network traffic  Storage is durable  It is a moving target  You cannot put a price on nimble 26
  • 27. • Wed (1:00PM-1:50PM) MED203: Scalable Media Processing with AWS• kmcentee@netflix.com• Later today at the Netflix booth 27
  • 28. We are sincerely eager to tweet #reinvent hear your feedback on thispresentation and on re:Invent. Please fill out an evaluation form when you have a chance.