NetflixOSS meetup lightning talks and roadmap

141,928 views

Published on

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
141,928
On SlideShare
0
From Embeds
0
Number of Embeds
138,622
Actions
Shares
0
Downloads
56
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

NetflixOSS meetup lightning talks and roadmap

  1. 1. Lightning talks Season 1, Episode 2 March 13, 2013
  2. 2. Overview● Lightning talks: 7:00 - 7:20● NeflixOSS Roadmap: 7:20 - 7:30● Announcement: 7:30 - 7:45● Demo Stations and Q+A: 8:00 - 9:30
  3. 3. Karyon https://github.com/Netflix/karyon● The nucleus of a Composable Web Service.● Container Agnostic with extensions for frameworks/containers.
  4. 4. Karyon https://github.com/Netflix/karyon● Bootstrapping ○ Dependency & Lifecycle management via Governator. ○ Service registry as Eureka. ○ Property management via Archaius.
  5. 5. Karyon https://github.com/Netflix/karyon● Administration (Auto-Discovered Embedded console) ○ Environment ○ Eureka ○ JMX
  6. 6. Karyon https://github.com/Netflix/karyonRuntime (Coming Soon)● SLA● Throttling● Request tracing and more ....
  7. 7. Karyon https://github.com/Netflix/karyon● Sample Service using Karyon available as "Hello-netflix-oss" on github● Recipes ...
  8. 8. Netflix OSS Recipe: RSS ReaderFirst sampleapplication that Karyon Turbine Ribbondemonstrates how Servoto build scalableedge and middle Hystrix Archaius Eurekatier applicationusing the NetflixOSS components.Accelerate yourlearning curve !
  9. 9. Netflix OSS Recipe: RSS ReaderFirst sampleapplication thatdemonstrates howto build scalableedge and middletier applicationusing the NetflixOSS components.Accelerate yourlearning curve ! https://github.com/Netflix/recipes-rss
  10. 10. Netflix OSS Recipe: RSS Reader
  11. 11. ● Ephemeral Volatile Cache ○ Based on memcached/spymemcached● Highly Scalable ○ Over 2 MM RPS at peak within Netflix ○ Over 100 K RPS / instance● Fast ○ Average latency ~ 1 ms● Elastic ○ Dynamically add/remove clusters, zones and/or instances● Heavily Used @Netflix ○ Fronts services & stores (Cassandra, S3, SimpleDB, RDS)
  12. 12. ● AWS Zone Aware ○ Clusters by zone ○ Replication by zones ○ Eliminate cross zone traffic● Connection Pool ○ Separate pools for Read and Write Operations● Zone fall through● Resilient to zone outage● Upcoming OSS ○ EVCache Server, Admin and Monitoring https://github.com/Netflix/EVCache
  13. 13. Denominator● Portable Control of DNS clouds ○ DynECT, Route53, UltraDNS and mock providers● Java Api and CLI ○ idempotent commands against record sets● GSLB Functionality in progress ○ Load Balanced and Directional (GEO) record sets github.com/Netflix/denominator
  14. 14. Aminator
  15. 15. Aminator● What ● Evolved from Bakery● Why● How ● AMI creation tool● When ○ CLI● Who ○ EBS AMIs ○ Redhat/Debian OSs ○ Python 2.6 & 2.7 ○ boto
  16. 16. Aminator● What ● Experiment with other● Why distros at scale, like● How Ubuntu● When● Who ● Remove embedded customizations ● You asked for it!
  17. 17. Aminator● What ● Customize existing AMI● Why ○ Create volume from base AMI● How snapshot ○ Attach volume● When ○ provision package● Who ○ snapshot volume ○ register snapshot
  18. 18. Aminator● What ● Coming soon, really.● Why● How● When● Who github.com/Netflix/aminator
  19. 19. Aminator● What ● Brian Moyles● Why ○ @bmoyles● How● When ● Karate Vick● Who ○ @kvick ● Michael Tripoli ○ @mtripoli
  20. 20. NetflixGraph● Compact in-memory data structure used to represent generic data sets.● Reduce the size of your applications memory footprint ○ Potentially by an order of magnitude● Remove I/O bottlenecks by holding data in RAM● Simple, easy to use API● Built-in serialization helps to transmit your data sets across your infrastructure
  21. 21. NetflixGraph - Memory Reduction● Drastically reduce the size of your data sets in memory ○ Eliminate Access Latency - Pull more into RAM ○ Reduce Cost - Use Less Hardware ○ Stabilize your application
  22. 22. NetflixGraph - Memory/CPU tradeoff● Easily select and test your memory-CPU tradeoff for different parts of your data sets.● Choose between tighter memory or faster access times on a per-type basis. ○ Mix and match, or just use the defaults!
  23. 23. NetflixGraph - Memory/CPU tradeoff● Tighter memory requirements? Use compact sets. ○ Variable-byte integers encode connections between nodes Wikipedia: Variable-Length Quantity
  24. 24. NetflixGraph - Memory/CPU tradeoff● Faster access times required? Use hashed sets! ○ Still RAM-efficient, provides O(1) access for contains() operations.
  25. 25. NetflixGraph - Built-in Serialization● Transmit data sets across your infrastructure● Build the data set on one server, then distribute. http://netflix.github.com/netflix-graph
  26. 26. Netflix OSS Continuous Integration● Make current state of code visible to all● Build & test on public Jenkins, hosted by our friends at CloudBees● Each push or merge to master is built and status shown on the GitHub project page
  27. 27. Netflix OSS Continuous Integration
  28. 28. Netflix OSS Continuous Integration● Build & test all pull requests● Isolated environment to protect against malicious code● Jenkins adds a comment to the pull request
  29. 29. Netflix OSS Continuous Integration● One-click creation of build jobs for new projects https://netflixoss.ci.cloudbees.com http://bit.ly/jobdsl
  30. 30.

×