9. Conformity Monkey
● Mark and Notify approach
● Customize rules
● Customize rule set
● Add new rules
● Customize execution time and frequency
● Can migrate to other cloud platforms
10. Sample Conformity Rules
● Security Groups
● Instance Configuration In Eureka
● ELB and ASG mismatch
● Cluster Naming Conventions
12. Zuul
● Netflix's edge tier
○ In place of Nginx, Apache Http or Traffic Server
● Dynamic filters that act on HTTP requests
● Filters written in Groovy / Dynamically
updated
○ Any JVM language Supported
19. Background
● There is increased resource usage in the cloud
● That is very decentralized
● But lacks broad and deep insights about resource usage
20. ICE Design Approach
● Cloud resource usage has many dimensions: account,
region, zone, product, usage type, user-defined tags
● ICE writer tags each line item in billing file with different
tags
● ICE UI provides a way for users to query cloud resource
usage freely in arbitrary dimensions
21.
22. ICE Usage
● ICE provides a birds-eye view of large and complex
cloud landscape from a usage and cost perspective.
● ICE UI is easy and intended to be used by:
● Managers
● Reservation Planners
● Engineers
● ICE OSS at https://github.com/Netflix/ice
23. ICE Next Steps
● Accessing billing files though IAM role and allow
multiple payer accounts
● Create Wiki page and Google Group
● Configurable cost alerts
25. Genie - Configuration Service
● Abstraction of physical details of
back-end Hadoop clusters in the
cloud
○ Status: up, out of service, or
terminated
○ Cluster naming/tagging for job
submissions
○ Site-specific Hadoop, Hive and
Pig configurations
26. Genie - Job Execution Service
● Simple API to run Hadoop, Hive and
Pig jobs
○ Auto-magic submission of jobs to
the right Hadoop cluster
○ Abstracting away cluster details
from clients
27. Genie - Use Cases Enabled at Netflix
● Running 100% of all SLA jobs, and a high
percentage of ad-hoc jobs
● Running nightly short-lived “bonus” clusters to
augment ETL processing
● Re-routing traffic between clusters
● “Red/black” pushes for clusters
● Attaching stand-alone gateways to clusters
29. Lipstick
Visualization of Pig workflows at a logical level.
● quickly spot common errors / inefficiencies
● visually monitor execution
● empowers users to support themselves
● facilitates communication between
infrastructure team and users
41. Acme Air OSS Sample and Benchmark
● Fictitious airline sample application
● Mobile app (iOS/Android) and desktop browser versions
● NodeJS and Java versions with multiple NoSQL data tier impls
● Benchmarking framework that allows “Web Scale” runs
● Previously run at 4.3 Billion customer facing REST calls per
second (50,000 req/sec) using IBM middleware and cloud
● Re-implemented using Netflix OSS
● Microservices architecture
● Eureka, Hystrix, Karyon, Ribbon
● Asgard
● Run at (peak) ~ 13,000 req/sec – 1.1 Billion per day
● Replicate this yourself in ~ 4 hours with the OSS
Learn more at http://bit.ly/acmeairblog and from @aspyker
42. Eucalyptus, AWS, and NetflixOSS
Asgard manages autoscaling across both clouds.
Eucalyptus/AWS compatibility makes it possible!
Some
NetflixOSS
services
can run in
either cloud
Simian Army
Edda
Aminator
Priam
Servo
30+ NetflixOSS services
...and more on the way!
43. Confidential and Proprietary
● Multi Data Center Support
● Single Dashboard to manage OpenStack & AWS
● Access multiple Tenants in Single Login
● One-Stop-Shop to manage
● Compute, Storage, Network
● Security
● Cloud Templates
● Asgard Framework and Architecture
● Why Asgard?
● Janitor Monkey for OpenStack
● Future Plans
● Demo
HIGHLIGHTS
44. Riot Games rCloud
●Aminator +chef-solo provisioner
●Asgard
●Edda +Security Audits
●Simian Army
●Ice
●Eureka
●Archaius + Configuration Service
●Ribbon, Karyon, Hystrix
●And more
46. State of the Cloud Prize
Forks of github.com/netflix/cloud-prize ~700
Mailchimp signup form complete ~120
Public active submissions ~20
Two more months for procrastination!
Stickers in return for Mailchimp signup...