Netflix in the Cloud

Ne#lix in the Cloud Oct 7, 2010 Adrian Cockcro: @adrianco h<p://www.linkedin.com/in/adriancockcro:

With more than 15 million members, Ne#lix, Inc. is the world’s largest subscripJon service streaming movies and TV episodes over the Internet and sending DVDs by mail.

Why Use AWS? •  Stop building our own datacenters –  Capacity growth rate is acceleraJng, unpredictable –  Product launch spikes -‐ iPhone, Wii, PS3, XBox –  DC is large inﬂexible capital commitment •  Leverage AWS Scale – the biggest –  AWS investment in tooling and automaJon –  AWS zones for high availability, scalability –  AWS skills are common on resumes… •  Leverage AWS Feature Set – most advanced –  EC2, S3, SDB, SQS, EBS, EMR, ELB, ASG, RDB..

Ne#lix Deployed on AWS •  “batch” Movie Encoding farm (2009) –  Thousands of EC2 instances, Petabytes of S3 –  Movie ﬁles staged out to CDNs for delivery •  Hadoop -‐ ElasJc Map-‐Reduce (2009) –  Large scale log processing and analyJcs •  Streaming Service Back-‐end (early 2010) –  Highly available and scalable “play bu<on” •  Web site, a page at a Jme (through 2010) –  Ramp from 25% of views to 80% in the coming weeks •  API for TV devices and iPhone etc. (2010) –  Personalized movie choosing algorithm back-‐end

Learnings… •  Datacenter oriented tools don’t work –  Ephemeral instances, high rate of change •  Cloud Tools Don’t Scale for Enterprise –  Built our own tools, drove vendors hard •  It’s faster in the end to re-‐code than Jnker –  “fork-‐li:ed” apps don’t work well –  Re-‐architected and re-‐wrote much of our code base using ﬁne grain web services, leveraging open source in Java, highly instrumented. “NoSQL” SimpleDB backend. In the datacenter, robust code is best prac2ce. In the cloud, it’s essen2al

Next Few Years… •  “System of Record” moves to Cloud –  Cut the datacenter to cloud replicaJon link •  InternaJonal Expansion – Global Clouds –  Rapid deployments to new markets •  GPU Clouds opJmized for video encoding •  MulJple Vendor Deployments –  Ne#lix already uses several streaming CDN vendors –  Large scale compeJtors will oﬀer AWS clones –  Higher resilience, higher scalability –  CompeJJon drives cost down We would prefer to be an insigniﬁcant customer in a giant cloud

Takeaway Ne#lix is path-‐ﬁnding the use of public AWS cloud to replace in-‐house IT for non-‐trivial applicaJons with hundreds of developers and thousands of systems. h<p://www.linkedin.com/in/adriancockcro: @adrianco

http://ws-security.blogspot.com	1099
http://www.hackingnetflix.com	500
http://www.1nima.com	137
http://softwarestrategiesblog.com	87
http://ws-security.blogspot.in	79
http://ws-security.blogspot.ca	15
http://ws-security.blogspot.co.uk	11
http://ws-security.blogspot.com.au	8
http://ws-security.blogspot.se	8
http://translate.googleusercontent.com	6
http://www.ws-security.blogspot.com	6
http://ws-security.blogspot.de	5
http://unlimited-data.blogspot.com	5
http://ws-security.blogspot.fr	5
http://feeds.feedburner.com	4
http://www.ws-security.blogspot.co.uk	3
http://static.slidesharecdn.com	3
http://webcache.googleusercontent.com	3
http://ws-security.blogspot.jp	2
http://www.ws-security.blogspot.com.au	2
http://ws-security.blogspot.com.es	2
http://irr.posterous.com	2
http://itnewscast.com	2
http://ws-security.blogspot.co.nz	1
http://runninginflipflops.com	1
http://rdbcci:8100	1
http://ws-security.blogspot.it	1

Netflix in the Cloud

by Adrian Cockcroft on Oct 07, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

27 Embeds 1,998

Statistics

Netflix in the Cloud — Presentation Transcript