Living on the Edge
Adrian Cole
@adrianfcole #netflixoss @denominatorOSS
http://www.linkedin.com/in/adrianforrestcole
How Netflix Streaming Works
Denominator
Cloud Prize
Netflix Member Web Site Home Page
Personalization Driven – What goes on to make this?
How Netflix Streaming Works
Customer Device
(PC, PS3, TV…)
Web Site or
Discovery API
User Data
Personalization
Streaming A...
Content Delivery Service
Open Source Hardware Design + FreeBSD, bird, nginx
November 2012 Traffic
Real Web Server Dependencies Flow
(Netflix Home page business transaction as seen by AppDynamics)
Start Here
memcached
Cas...
Asgard
http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html
Portable DNS Control
Cassandra Replicas
Zone A
Cassandra Replicas
Zone B
Cassandra Replicas
Zone C
Regional Load Balancers...
PORTABLE CONTROL OF DNS CLOUDS
Feature Set
• Do stuff in batches
• Cleanly handle advanced records
• Play nice with persistence
• Don’t do too much
• … U...
A Cloud Native Open Source Platform
Hello Denominator
get denominator from bintray or homebrew
create config.yml
name: dynect
provider: dynect
credentials:
cu...
Basic list
$ denominator -n dynect-test -C ~/.denominator/config.yml zone
[Session#login] ---> POST https://api2.dynect.ne...
Add Record
$ denominator -n dynect-test -C ~/.denominator/config.yml record -z denominator.io.
replace -n www.denominator....
From Java
mgr = Denominator.create(”dynect”,
(customer, username, password))
for (Zone zone : mgr.api(). ()) {
processZone...
Availability by reshaping traffic
Netflix Cloud Prize
Boosting the @netflixOSS
Ecosystem
Entrants
Netflix
Engineering
Judges Winners
Nominations
Conforms to
Rules
Working
Code
Community
Traction
Categories
Regis...
Functionality and scale now, portability coming
Moving from parts to a platform in 2013
Netflix is fostering an ecosystem
...
Takeaway
Denominator is a multi-cloud DNS abstraction built as a library and a cli.
Fixing DNS APIs, one vendor at a time
...
Adrian Cole of Netflix: Living On The Edge from Geek Summer Camp
Adrian Cole of Netflix: Living On The Edge from Geek Summer Camp
Adrian Cole of Netflix: Living On The Edge from Geek Summer Camp
Upcoming SlideShare
Loading in …5
×

Adrian Cole of Netflix: Living On The Edge from Geek Summer Camp

1,623 views

Published on

If you haven't heard of Netflix, welcome back from outer space! We were lucky to bring Adrian Cole to Dyn Geek Summer Camp to give a presentation on his day-to-day role as their Cloud Guy, how the streaming service really works, web server dependencies and tons more.

Enjoy his slides and watch his presentation here:
http://dyn.wistia.com/medias/a2ie7htu35

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Adrian Cole of Netflix: Living On The Edge from Geek Summer Camp

  1. 1. Living on the Edge Adrian Cole @adrianfcole #netflixoss @denominatorOSS http://www.linkedin.com/in/adrianforrestcole
  2. 2. How Netflix Streaming Works Denominator Cloud Prize
  3. 3. Netflix Member Web Site Home Page Personalization Driven – What goes on to make this?
  4. 4. How Netflix Streaming Works Customer Device (PC, PS3, TV…) Web Site or Discovery API User Data Personalization Streaming API DRM QoS Logging OpenConnect CDN Boxes CDN Management and Steering Content Encoding Consumer Electronics AWS Cloud Services CDN Edge Locations
  5. 5. Content Delivery Service Open Source Hardware Design + FreeBSD, bird, nginx
  6. 6. November 2012 Traffic
  7. 7. Real Web Server Dependencies Flow (Netflix Home page business transaction as seen by AppDynamics) Start Here memcached Cassandra Web service S3 bucket Three Personalization movie group choosers (for US, Canada and Latam) Each icon is three to a few hundred instances across three AWS zones
  8. 8. Asgard http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html
  9. 9. Portable DNS Control Cassandra Replicas Zone A Cassandra Replicas Zone B Cassandra Replicas Zone C Regional Load Balancers Cassandra Replicas Zone A Cassandra Replicas Zone B Cassandra Replicas Zone C Regional Load Balancers UltraDNS AWS Route53 A portable way to manage multiple DNS providers from Java DenominatorDynECT
  10. 10. PORTABLE CONTROL OF DNS CLOUDS
  11. 11. Feature Set • Do stuff in batches • Cleanly handle advanced records • Play nice with persistence • Don’t do too much • … Use cool things
  12. 12. A Cloud Native Open Source Platform
  13. 13. Hello Denominator get denominator from bintray or homebrew create config.yml name: dynect provider: dynect credentials: customer: your_customer username: your_user password: your_password denominator -n dynect –C config.yml zone list
  14. 14. Basic list $ denominator -n dynect-test -C ~/.denominator/config.yml zone [Session#login] ---> POST https://api2.dynect.net/REST/Session HTTP/1.1 [Session#login] <--- HTTP/1.1 200 OK (948ms) [DynECT#zones] ---> GET https://api2.dynect.net/REST/Zone HTTP/1.1 [DynECT#zones] <--- HTTP/1.1 200 OK (514ms)
  15. 15. Add Record $ denominator -n dynect-test -C ~/.denominator/config.yml record -z denominator.io. replace -n www.denominator.io. -t A -d 192.0.2.1 [Session#login] ---> POST https://api2.dynect.net/REST/Session HTTP/1.1 [Session#login] <--- HTTP/1.1 200 OK (914ms) [DynECT#recordsInZoneByNameAndType] ---> GET https://api2.dynect.net/REST/ARecord/denominator.io./www.denominator.io.?detail=Y HTTP/1.1 [DynECT#recordsInZoneByNameAndType] <--- HTTP/1.1 200 OK (504ms) [DynECT#scheduleDeleteRecord] ---> DELETE https://api2.dynect.net/REST/ARecord/denominator.io./www.denominator.io/75626502 HTTP/1.1 [DynECT#scheduleDeleteRecord] <--- HTTP/1.1 200 OK (157ms) [DynECT#scheduleCreateRecord] ---> POST https://api2.dynect.net/REST/ARecord/denominator.io./www.denominator.io. HTTP/1.1 [DynECT#scheduleCreateRecord] <--- HTTP/1.1 200 OK (157ms) [DynECT#publish] ---> PUT https://api2.dynect.net/REST/Zone/denominator.io. HTTP/1.1 [DynECT#publish] <--- HTTP/1.1 200 OK (209ms)
  16. 16. From Java mgr = Denominator.create(”dynect”, (customer, username, password)) for (Zone zone : mgr.api(). ()) { processZone(zone); } mgr. (“denominator.io.”) . (a("www.denominator.io.", 300, "192.0.2.1"));
  17. 17. Availability by reshaping traffic
  18. 18. Netflix Cloud Prize Boosting the @netflixOSS Ecosystem
  19. 19. Entrants Netflix Engineering Judges Winners Nominations Conforms to Rules Working Code Community Traction Categories Registration Opened March 13 Github Apache Licensed Contributions Github Close Entries September 15 Github Award Ceremony Dinner November AWS Re:Invent Ten Prize Categories $10K cash $5K AWS AWS Re:Invent Tickets Trophy
  20. 20. Functionality and scale now, portability coming Moving from parts to a platform in 2013 Netflix is fostering an ecosystem Rapid Evolution - Low MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)
  21. 21. Takeaway Denominator is a multi-cloud DNS abstraction built as a library and a cli. Fixing DNS APIs, one vendor at a time https://github.com/Netflix/denominator https://groups.google.com/forum/#!forum/denominator-user http://www.linkedin.com/in/adrianforrestcole @adrianfcole #netflixoss @denominatorOSS

×