Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Assembling the Puzzle Ruslan Meshenberg  Adrian Cockcroft
Open House timeline• 7:00pm – 7:30pm – NetflixOSS overview, glimpse  into 2013 roadmap, announcements• 7:30pm – 8:00pm - L...
A bit of history…
@NetflixOSS progression in 2012                                                                                           ...
Our Current Catalog of ReleasesFree code available at http://netflix.github.com
2013 Roadmap - highlights                   • BakeriesBuild and Deploy   • Workflow orchestration                   • Push...
Recipes & Launcher
Whole is greater than sum of the parts
Three Questions Why is Netflix doing this?How does it all fit together?   What is coming next?
Netflix Deconstructed             Content as a Service on a PlatformLong term strategic       Easy to usebarriers to compe...
Platform InvestmentCloud platform                         Netflix                  By leveraging                          ...
Platform Evolution  2009-2010                2011-2012                      2013-2014Bleeding Edge              Common    ...
Grasping an Opportunity  One of many              Commondiverse platforms          platform                         Netfli...
Making it easy to followExploring the wild west each time   vs. laying down a shared route
Establish our            Hire, Retain and  solutions as Best            Engage TopPractices / Standards           Engineer...
Progress during 2012From pushing the platform uphill   to runaway success
How does it all fit together?
Open Source Projects         Legend Github / Techblog                 Priam                           Exhibitor           ...
NetflixOSS Continuous Build and Deployment  Github           Maven            AWS NetflixOSS        Central        Base AM...
NetflixOSS Services ScopeAWS AccountAsgard ConsoleArchaius Config                  Multiple AWS Regions    Service Cross r...
NetflixOSS Instance Libraries                 • Baked AMI – Tomcat, Apache, your codeInitialization   • Governator – Guice...
NetflixOSS Testing and Automation Test Tools    • CassJmeter – load testing for C*               • Circus Monkey – test re...
What’s Coming Next?           Better portability           Higher availability MoreFeatures   Easier to deploy           C...
SPoF - Single Point of Failure     Failure to be Scalable?     Failure to be Portable?    Failure to be Functional?
Optimizing for SPoF              Cloud platforms, Eucalyptus, Cloudstack, OpenStack etc.              Playing feature catc...
Managing Multi-Region Availability                                         AWS                                            ...
Denominator           “The next version is more portable…” for DNS                                                    Edda...
Announcements• Denominator  – New! Discuss it with Adrian Cole here today  – Techblog and github coming soon• Next Netflix...
Functionality and scale now, portability coming   Moving from parts to a platform in 2013       Netflix is fostering an ec...
Next - Lightning TalksPlease save your questions for the in    person sessions that follow
NetflixOSS Meetup
Upcoming SlideShare
Loading in …5
×

NetflixOSS Meetup

157,113 views

Published on

Introduction to the Netflix Open Source Software project, explains why Netflix is doing this, how all the parts fit together and what is planned to come next. Presented at the inaugural NetflixOSS Meetup February 6th 2013 at Netflix headquarters in Los Gatos.

Published in: Technology
  • Be the first to comment

NetflixOSS Meetup

  1. 1. Assembling the Puzzle Ruslan Meshenberg Adrian Cockcroft
  2. 2. Open House timeline• 7:00pm – 7:30pm – NetflixOSS overview, glimpse into 2013 roadmap, announcements• 7:30pm – 8:00pm - Lightning Talks about individual NetflixOSS projects• 8:00 – 8:30 – Asgard Deep Dive (Theater)• 8:00pm - 9:00pm - Lab/Demo Stations
  3. 3. A bit of history…
  4. 4. @NetflixOSS progression in 2012 RxJava Ribbon Janitor Turbine Monkey Hystrix Blitz4J Govern Eureka ator Chaos Asgard Monkey Archaius Exhibito Jmeter r Priam for C* Servo Astyanax AutoCurato Scalingr
  5. 5. Our Current Catalog of ReleasesFree code available at http://netflix.github.com
  6. 6. 2013 Roadmap - highlights • BakeriesBuild and Deploy • Workflow orchestration • Push-button Launcher • Sample applications Recipes • Karyon - Base server • More Monkeys, Error and Latency Injection Availability • Denominator (more later…) • Atlas - monitoring • Genie – Hadoop Paas Analytics • Explorers / visualization • EvCache – persister and memcached service Persistence • More Astyanax Recipes
  7. 7. Recipes & Launcher
  8. 8. Whole is greater than sum of the parts
  9. 9. Three Questions Why is Netflix doing this?How does it all fit together? What is coming next?
  10. 10. Netflix Deconstructed Content as a Service on a PlatformLong term strategic Easy to usebarriers to competition Personalized Service Exclusive Agile, Reliable and Scalable, Secure Extensive Low cost, Global Content Platform Enables the business, but doesn’t Netflix differentiate against large competitors
  11. 11. Platform InvestmentCloud platform Netflix By leveraging AWS we canadapts as new concentrateAWS services investment inappear Cloud Platform Netflix unique services Light Lifting Amazon Web Services Undifferentiated Heavy Lifting
  12. 12. Platform Evolution 2009-2010 2011-2012 2013-2014Bleeding Edge Common Shared Innovation Pattern Pattern Netflix ended up several years ahead of the industry, but it’s not a sustainable position
  13. 13. Grasping an Opportunity One of many Commondiverse platforms platform NetflixOSS couldIf Netflix makes the become aNetflixOSS platform significant platform easier to adopt… ecosystem
  14. 14. Making it easy to followExploring the wild west each time vs. laying down a shared route
  15. 15. Establish our Hire, Retain and solutions as Best Engage TopPractices / Standards Engineers Goals Build up Netflix Benefit from a Technology Brand shared ecosystem
  16. 16. Progress during 2012From pushing the platform uphill to runaway success
  17. 17. How does it all fit together?
  18. 18. Open Source Projects Legend Github / Techblog Priam Exhibitor Servo and Autoscaling ScriptsApache Contributions Cassandra as a Service Zookeeper as a Service Astyanax Curator Genie Techblog Post Cassandra client for Java Zookeeper Patterns Hadoop PaaS Coming Soon CassJMeter EVCache Hystrix Cassandra test suite Memcached as a Service Robust service pattern Cassandra Eureka / Discovery Multi-region EC2 datastore RxJava Reactive Patterns support Service Directory Asgard Aegisthus Archaius AutoScaleGroup based AWS Hadoop ETL for Cassandra Dynamics Properties Service console Edda Chaos Monkey Explorers Config state with history Robustness verification Governator Denominator Library lifecycle and dependency Latency Monkey injection (Announce today) Odin Ribbon Janitor Monkey Orchestration for Asgard REST Client + mid-tier LB Karyon Blitz4j Async logging Bakeries and AMI Instrumented REST Base Server
  19. 19. NetflixOSS Continuous Build and Deployment Github Maven AWS NetflixOSS Central Base AMI Source Dynaslave Jenkins AWS AWS Build Bakery Baked AMIs Slaves Odin Asgard AWS Orchestration (+ Frigga) Account API Console
  20. 20. NetflixOSS Services ScopeAWS AccountAsgard ConsoleArchaius Config Multiple AWS Regions Service Cross region Priam C* Eureka Registry Explorers Dashboards Exhibitor ZK 3 AWS Zones Application Priam Evcache Atlas Edda History Clusters Cassandra Memcached Monitoring Autoscale Groups Persistent Storage Ephemeral Storage Instances Simian ArmyGenie Hadoop Services
  21. 21. NetflixOSS Instance Libraries • Baked AMI – Tomcat, Apache, your codeInitialization • Governator – Guice based dependency injection • Archaius – dynamic configuration properties client • Eureka - service registration client Service • Karyon - Base Server for inbound requests • RxJava – Reactive pattern • Hystrix/Turbine – dependencies and real-time status Requests • Ribbon - REST Client for outbound calls • Astyanax – Cassandra client and pattern libraryData Access • Evcache – Zone aware Memcached client • Curator – Zookeeper patterns • Blitz4j – non-blocking logging Logging • Servo – metrics export for autoscaling • Atlas – high volume instrumentation
  22. 22. NetflixOSS Testing and Automation Test Tools • CassJmeter – load testing for C* • Circus Monkey – test rebalancing • Janitor MonkeyMaintenance • Efficiency Monkey • Doctor Monkey • Howler Monkey • Chaos Monkey - InstancesAvailability • Chaos Gorilla – Availability Zones • Chaos Kong - Regions • Latency Monkey – latency and error injection Security • Security Monkey • Conformity Monkey
  23. 23. What’s Coming Next? Better portability Higher availability MoreFeatures Easier to deploy Contributions from end users Contributions from vendors More Use Cases
  24. 24. SPoF - Single Point of Failure Failure to be Scalable? Failure to be Portable? Failure to be Functional?
  25. 25. Optimizing for SPoF Cloud platforms, Eucalyptus, Cloudstack, OpenStack etc. Playing feature catch-up with AWS, “Next version has X…” Eventually Functional Scalable Portable or Functional Eventually Eventually Portable ScalableNetflixOSS CloudFoundryAlready scalable and functional MongoDB etc.“Next version is more portable…” “Next version scales…”
  26. 26. Managing Multi-Region Availability AWS DynECT Route53 UltraDNS DNS Regional Load Balancers Regional Load Balancers Zone A Zone B Zone C Zone A Zone B Zone CCassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas What we need is a portable way to manage multiple DNS providers….
  27. 27. Denominator “The next version is more portable…” for DNS Edda, Multi- Use Cases Region Failover Common Model DenominatorDNS Vendor Plug-in AWS Route53 DynECT UltraDNS Etc…API Models (varied IAM Key Auth User/pwd User/pwdand mostly broken) REST REST SOAP Currently being built by Adrian Cole (the jClouds guy, he works for Netflix now…)
  28. 28. Announcements• Denominator – New! Discuss it with Adrian Cole here today – Techblog and github coming soon• Next NetflixOSS Meetup – Five weeks from today, March 13th, at Netflix – More & bigger announcements, don’t miss it…
  29. 29. Functionality and scale now, portability coming Moving from parts to a platform in 2013 Netflix is fostering an ecosystem Rapid Evolution - Low MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)
  30. 30. Next - Lightning TalksPlease save your questions for the in person sessions that follow

×