SlideShare a Scribd company logo
Netflix Architecture and
Open Source
Andrew Spyker
Senior Software Engineer, Netflix
About Netflix
69M members
2000+ employees (1400 tech)
80+ countries
> 100M hours watch per day
> ⅓ NA internet download traffic
500+ Microservices
Many 10’s of thousands VM’s
3 regions across the world
About the Speaker
Cloud platform technologies
Distributed configuration, service discovery, RPC, application
frameworks, non-Java sidecar
Container cloud
Resource management and scheduling, making Docker containers
operational in Amazon EC2/ECS
Open Source
Organize @NetflixOSS meetups & internal group
Performance
Assist across Netflix, but focused mainly on cloud platform perf
With Netflix for ~ 1 year. Previously at IBM here in Raleigh/Durham
(RTP)
@aspyker
ispyker.blog
spot.com
Agenda
NetflixOSS
Netflix Cloud Architecture
Getting started
Why does Netflix open source?
Allows engineers to gather feedback
Openly talk, through code, on our approach
Collaboration on key projects with the world
Happily use proven outside open source
And improve it for Netflix scale and availability
Netflix culture of freedom and responsibility
Want to open source?
Go for it, be responsible!
Recruiting and Retention
Candidates know exactly what they can work on
NetflixOSS engineers choose to stay at Netflix
NetflixOSS is widely used
The architecture has shaped public cloud usage
Immutability, Red/Black Deploys, Chaos,
Regional and worldwide high availability
Offerings
Pivotal Spring Cloud
Large usage
IBM Watson as a Service (on IBM Cloud)
Nike Digital is hiring NetflixOSS experts
Interesting usage
“To help locate new troves of data claiming to be the files stolen from
AshleyMadison, the company’s forensics team has been using a tool
that Netflix released last year called Scumblr”
NetflixOSS Website Relaunch
http://netflix.github.io
Key aspects of NetflixOSS website
Show how the pieces fit together
Projects now discussed with each other in context
OSS categories mirror internal teams
No artificial categories, focal points for each area
Focus on projects that are core to Netflix
Projects mentioned are core and strategic
Agenda
NetflixOSS
Netflix Cloud Architecture
Getting Started
Elastic, Web and Hyper Scale
Doing this
Not doing that
Elastic, Web and Hyper Scale
Front end
API
Another
Microservice
Temporal
caching
Durable
Storage
Load
Balancers
Strategy Benefit
Make deployments automated Without automation impossible
Expose well designed API to users Offloads presentation complexity to clients
Remove state for mid tier services Allows easy elastic scale out
Push temporal state to client and caching tier Leverage clients, avoids data tier overload
Use partitioned data storage Data design and storage scales with HA
Recommendation
Microservice
HA and Automatic Recovery
Feeling This
Not Feeling That
Micro service
Implementation
Call microservice #2
Highly Available Service Runtime Recipe
Ribbon REST client
with Eureka
Microservice #1
(REST services)
App Service
Microservice #2
Execute
call
Hystrix
Eureka
Server(s)
Eureka
Server(s)
Eureka
Server(s)
Karyon
Fallback
Implementation
Implementation Detail Benefits
Decompose into micro services
• Key user path always available
• Failure does not propagate across service boundaries
Karyon /w automatic Eureka registration
• New instances are quickly found
• Failing individual instances disappear
Ribbon client with Eureka awareness
• Load balances & retries across instances with “smarts”
• Handles temporal instance failure
Hystrix as dependency circuit breaker
• Allows for fast failure
• Provides graceful cross service degradation/recovery
IaaS High Availability
Region (us-east-1)
us-east-1e
us-east-1c
Eureka
Web App Service1 Service2
Cluster Auto Recovery and Scaling Services (Auto Scaling Groups)
ELB’s
Rule Why?
Always > 2 of everything 1 is SPOF, 2 doesn’t web scale and slow DR recovery
Including IaaS and cloud services You’re only as strong as your weakest dependency
Use auto scaler/recovery monitoring Clusters guarantee availability and service latency
Use application level health checks Instance on the network != healthy
Worldwide availability Data replication, global front-end routing, cross region traffic
us-east-1d
A truly global service
Replicate data across
regions
Be able to redirect traffic
from region to region
Be able to migrate regional
traffic to other regions
Have automated control
across regions
Flux Demo
Testing is only way to prove HA
Chaos Monkey
Kill instances in production - runs regularly
Chaos Gorilla
Kills availability zones (single datacenter)
Also testing for split brain important
Chaos Kong
Kill entire region and shift traffic globally
Run frequently but with prior scheduling
Continuous Delivery
Reading This
Not This
v
Continuous Delivery
Cluster v1 Canary v2 Cluster V2
Step Technology
Developers test locally Unit test frameworks
Continuous build Continuous build server based on gradle builds
Build “bakes” full instance image Aminator and deployment pipeline bake images from build artifacts
Developer work across dev and test Archaius allows for environment based context
Developers do canary tests, red/black
deployments in prod
Asgard console provides app cluster common devops approach,
security patterns, and visibility
Continuous
Build Server
Baked to images
(AMI’s)
From Asgard to Spinnaker
Spinnaker is our CI/CD solution
CI/CD solution including baking and Jenkins integration
Workflow engine for the continuous delivery
Pipeline based deployment including baking
Global visibility across all of our AWS regions
Provides an API first design
A microservices runtime HA architecture
More flexible cloud model so the community can contribute back
improvements not related to AWS
Asgard continues to work side-by-side
Spinnaker is this new end to end CI/CD tool
Spinnaker Examples
Works at
Netflix
scale
Views of
global
pipelines
From simple Asgard
like deployment to
advanced CI/CD
pipelines
Operational Visibility
If you can’t see it, you can’t improve it
Operational Visibility
Microservice #1 Microservice #2
Visibility Point Technology
Basic IaaS instance monitoring Not enough (not scalable, not app specific)
User like external monitoring SaaS offerings or OSS like Uptime
Targeted performance, sampling Vector performance and app level metrics
Service to service interconnects Hystrix streams ➔Turbine aggregation ➔Hystrix dashboard
Application centric metrics Servo/Spectator gauges, counters, timers sent to metrics store like Atlas
Remote logging Logstash/Kibana or similar log aggregation and analysis frameworks
Threshold monitoring and alerts Services like Atlas and PagerDuty for incident management
Servo/
Spectator
Hystrix/Turbine
External
Uptime
Monitoring Metric/Event
Repositories
LogStash/Elastic
Search/Kibana
Incidents
Atlas
Vector
Security
Dynamic
Security
Done in new ways
NOT
Dynamic, Web Scale & Simpler Security
Security Monkey
Monitors security policies, tracks changes, alerts on situations
Scumblr
Searches internet for security “nuggets” (credentials, hacking discussions)
Sketchy
A safe way to collect text and screenshots from websites
FIDO
Automated event detection, analysis, enrichment & and enforcement
Sleepy Puppy
Delayed cross site scripting propagation testing framework
Lemur
x.509 certificate orchestration framework
What did we not cover?
Over 50 github projects
NetflixOSS is “Technical indigestion as a service”
Big Data, Data Persistence and UI Engineering
Big Data tools used well beyond Netflix
Ephemeral, semi and fully persistent data systems
Recent addition of UI OSS and Falcor
Agenda
NetflixOSS
Netflix Cloud Architecture
Getting Started
How do I get started?
All of the previous slides shows NetflixOSS components
Code: http://netflix.github.io
Announcements: http://techblog.netflix.com/
Want to get running a bit faster?
ZeroToCloud
Workshop for getting started with build/bake/deploy in Amazon EC2
ZeroToDocker
Docker images that containing running Netflix technologies (not production
ready, but easy to understand)
ZeroToDocker Demo
Mac OS X
Virtual Box
Ubuntu 14.04
single kernel
Container#1
Filesystem+
process
Eureka
Container
ZuulContainer
Another
Container
...
Docker running instances
Single kernel
Contained processes
Zookeeper and Exhibitor
A Microservices app and
surrounding NetflixOSS
services (Zuul to Karyon
with Eureka)
Questions
?

More Related Content

What's hot

Dapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any LanguageDapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any Language
Bilgin Ibryam
 
Micro-services architecture
Micro-services architectureMicro-services architecture
Micro-services architecture
Farwa Ansari
 
Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...
Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...
Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...
Codemotion
 
Querying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and CouchbaseQuerying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and Couchbase
Brant Burnett
 
MySQL DBaaS with OpenStack Trove
MySQL DBaaS with OpenStack TroveMySQL DBaaS with OpenStack Trove
MySQL DBaaS with OpenStack Trove
Matt Lord
 
Tuning TCP and NGINX on EC2
Tuning TCP and NGINX on EC2Tuning TCP and NGINX on EC2
Tuning TCP and NGINX on EC2
Chartbeat
 
Infrastructure as Code with Terraform and Ansible
Infrastructure as Code with Terraform and AnsibleInfrastructure as Code with Terraform and Ansible
Infrastructure as Code with Terraform and Ansible
DevOps Meetup Bern
 
Apache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesApache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice Architectures
Kai Wähner
 
HA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybridHA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybrid
James Serra
 
VMware Horizon (view) 7 Lab Manual
VMware Horizon (view) 7 Lab Manual VMware Horizon (view) 7 Lab Manual
VMware Horizon (view) 7 Lab Manual
Sanjeev Kumar
 
A Brief History of Stream Processing
A Brief History of Stream ProcessingA Brief History of Stream Processing
A Brief History of Stream Processing
Aleksandr Kuboskin, CFA
 
03. it정보화전략-솔루션 도입
03. it정보화전략-솔루션 도입03. it정보화전략-솔루션 도입
03. it정보화전략-솔루션 도입
InGuen Hwang
 
Disaster Recovery in the Cloud
Disaster Recovery in the CloudDisaster Recovery in the Cloud
Disaster Recovery in the Cloud
Amazon Web Services
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red Hat
Amazon Web Services
 
RabbitMQ & Kafka
RabbitMQ & KafkaRabbitMQ & Kafka
RabbitMQ & Kafka
VMware Tanzu
 
Disaster Recovery Plans for Apache Kafka
Disaster Recovery Plans for Apache KafkaDisaster Recovery Plans for Apache Kafka
Disaster Recovery Plans for Apache Kafka
confluent
 
가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714
가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714
가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714
경오 이
 
An Introduction to Kafka Cruise Control with Viktor Somogyi-Vass
An Introduction to Kafka Cruise Control with Viktor Somogyi-VassAn Introduction to Kafka Cruise Control with Viktor Somogyi-Vass
An Introduction to Kafka Cruise Control with Viktor Somogyi-Vass
HostedbyConfluent
 
Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019
Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019 Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019
Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019
Amazon Web Services
 
Dns presentation
Dns presentationDns presentation
Dns presentation
Anurag Pandey
 

What's hot (20)

Dapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any LanguageDapr - A 10x Developer Framework for Any Language
Dapr - A 10x Developer Framework for Any Language
 
Micro-services architecture
Micro-services architectureMicro-services architecture
Micro-services architecture
 
Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...
Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...
Andrzej Ludwikowski - Event Sourcing - what could possibly go wrong? - Codemo...
 
Querying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and CouchbaseQuerying Nested JSON Data Using N1QL and Couchbase
Querying Nested JSON Data Using N1QL and Couchbase
 
MySQL DBaaS with OpenStack Trove
MySQL DBaaS with OpenStack TroveMySQL DBaaS with OpenStack Trove
MySQL DBaaS with OpenStack Trove
 
Tuning TCP and NGINX on EC2
Tuning TCP and NGINX on EC2Tuning TCP and NGINX on EC2
Tuning TCP and NGINX on EC2
 
Infrastructure as Code with Terraform and Ansible
Infrastructure as Code with Terraform and AnsibleInfrastructure as Code with Terraform and Ansible
Infrastructure as Code with Terraform and Ansible
 
Apache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesApache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice Architectures
 
HA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybridHA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybrid
 
VMware Horizon (view) 7 Lab Manual
VMware Horizon (view) 7 Lab Manual VMware Horizon (view) 7 Lab Manual
VMware Horizon (view) 7 Lab Manual
 
A Brief History of Stream Processing
A Brief History of Stream ProcessingA Brief History of Stream Processing
A Brief History of Stream Processing
 
03. it정보화전략-솔루션 도입
03. it정보화전략-솔루션 도입03. it정보화전략-솔루션 도입
03. it정보화전략-솔루션 도입
 
Disaster Recovery in the Cloud
Disaster Recovery in the CloudDisaster Recovery in the Cloud
Disaster Recovery in the Cloud
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red Hat
 
RabbitMQ & Kafka
RabbitMQ & KafkaRabbitMQ & Kafka
RabbitMQ & Kafka
 
Disaster Recovery Plans for Apache Kafka
Disaster Recovery Plans for Apache KafkaDisaster Recovery Plans for Apache Kafka
Disaster Recovery Plans for Apache Kafka
 
가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714
가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714
가상머신 환경에서 리눅스 및 큐브리드 설치 가이드 20150714
 
An Introduction to Kafka Cruise Control with Viktor Somogyi-Vass
An Introduction to Kafka Cruise Control with Viktor Somogyi-VassAn Introduction to Kafka Cruise Control with Viktor Somogyi-Vass
An Introduction to Kafka Cruise Control with Viktor Somogyi-Vass
 
Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019
Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019 Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019
Security benefits of the Nitro architecture - SEP401-R - AWS re:Inforce 2019
 
Dns presentation
Dns presentationDns presentation
Dns presentation
 

Similar to Netflix Cloud Architecture and Open Source

Netflix Architecture and Open Source
Netflix Architecture and Open SourceNetflix Architecture and Open Source
Netflix Architecture and Open Source
All Things Open
 
Netflix Cloud Platform and Open Source
Netflix Cloud Platform and Open SourceNetflix Cloud Platform and Open Source
Netflix Cloud Platform and Open Source
aspyker
 
Triangle Devops Meetup 10/2015
Triangle Devops Meetup 10/2015Triangle Devops Meetup 10/2015
Triangle Devops Meetup 10/2015
aspyker
 
Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...
Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...
Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...
DevOps.com
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
Adrian Cockcroft
 
Fast, Secure Deployments with Docker on AWS
Fast, Secure Deployments with Docker on AWSFast, Secure Deployments with Docker on AWS
Fast, Secure Deployments with Docker on AWS
Amazon Web Services
 
Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023
Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023
Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023
VMware Tanzu
 
Modernizing Testing as Apps Re-Architect
Modernizing Testing as Apps Re-ArchitectModernizing Testing as Apps Re-Architect
Modernizing Testing as Apps Re-Architect
DevOps.com
 
Gluecon Monitoring Microservices and Containers: A Challenge
Gluecon Monitoring Microservices and Containers: A ChallengeGluecon Monitoring Microservices and Containers: A Challenge
Gluecon Monitoring Microservices and Containers: A Challenge
Adrian Cockcroft
 
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
Ludovic Piot
 
AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)
AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)
AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)
Amazon Web Services
 
CSC AWS re:Invent Enterprise DevOps session
CSC AWS re:Invent Enterprise DevOps sessionCSC AWS re:Invent Enterprise DevOps session
CSC AWS re:Invent Enterprise DevOps session
Tom Laszewski
 
Business and IT agility through DevOps and microservice architecture powered ...
Business and IT agility through DevOps and microservice architecture powered ...Business and IT agility through DevOps and microservice architecture powered ...
Business and IT agility through DevOps and microservice architecture powered ...
Lucas Jellema
 
The Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian CockcroftThe Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian Cockcroft
Dun & Bradstreet Cloud Innovation Center
 
Stay productive while slicing up the monolith
Stay productive while slicing up the monolithStay productive while slicing up the monolith
Stay productive while slicing up the monolith
Markus Eisele
 
Microservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration PatternsMicroservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration Patterns
Araf Karsh Hamid
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices
Hendri Karisma
 
12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클
12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클
12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클
Oracle Korea
 
The elegant way of implementing microservices with istio
The elegant way of implementing microservices with istioThe elegant way of implementing microservices with istio
The elegant way of implementing microservices with istio
Inho Kang
 
Data Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDBData Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDB
confluent
 

Similar to Netflix Cloud Architecture and Open Source (20)

Netflix Architecture and Open Source
Netflix Architecture and Open SourceNetflix Architecture and Open Source
Netflix Architecture and Open Source
 
Netflix Cloud Platform and Open Source
Netflix Cloud Platform and Open SourceNetflix Cloud Platform and Open Source
Netflix Cloud Platform and Open Source
 
Triangle Devops Meetup 10/2015
Triangle Devops Meetup 10/2015Triangle Devops Meetup 10/2015
Triangle Devops Meetup 10/2015
 
Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...
Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...
Zero to 1000+ Applications - Large Scale CD Adoption at Cisco with Spinnaker ...
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
 
Fast, Secure Deployments with Docker on AWS
Fast, Secure Deployments with Docker on AWSFast, Secure Deployments with Docker on AWS
Fast, Secure Deployments with Docker on AWS
 
Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023
Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023
Simplify and Scale Enterprise Spring Apps in the Cloud | March 23, 2023
 
Modernizing Testing as Apps Re-Architect
Modernizing Testing as Apps Re-ArchitectModernizing Testing as Apps Re-Architect
Modernizing Testing as Apps Re-Architect
 
Gluecon Monitoring Microservices and Containers: A Challenge
Gluecon Monitoring Microservices and Containers: A ChallengeGluecon Monitoring Microservices and Containers: A Challenge
Gluecon Monitoring Microservices and Containers: A Challenge
 
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
 
AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)
AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)
AWS re:Invent 2016: Develop Your Migration Toolkit (ENT312)
 
CSC AWS re:Invent Enterprise DevOps session
CSC AWS re:Invent Enterprise DevOps sessionCSC AWS re:Invent Enterprise DevOps session
CSC AWS re:Invent Enterprise DevOps session
 
Business and IT agility through DevOps and microservice architecture powered ...
Business and IT agility through DevOps and microservice architecture powered ...Business and IT agility through DevOps and microservice architecture powered ...
Business and IT agility through DevOps and microservice architecture powered ...
 
The Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian CockcroftThe Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian Cockcroft
 
Stay productive while slicing up the monolith
Stay productive while slicing up the monolithStay productive while slicing up the monolith
Stay productive while slicing up the monolith
 
Microservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration PatternsMicroservices Architecture, Monolith Migration Patterns
Microservices Architecture, Monolith Migration Patterns
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices
 
12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클
12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클
12월 16일 Meetup [Deep Dive] Microservice 트래픽 관리를 위한 Istio 알아보기 | 강인호 컨설턴트, 오라클
 
The elegant way of implementing microservices with istio
The elegant way of implementing microservices with istioThe elegant way of implementing microservices with istio
The elegant way of implementing microservices with istio
 
Data Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDBData Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDB
 

More from aspyker

Herding Kats - Netflix’s Journey to Kubernetes Public
Herding Kats - Netflix’s Journey to Kubernetes PublicHerding Kats - Netflix’s Journey to Kubernetes Public
Herding Kats - Netflix’s Journey to Kubernetes Public
aspyker
 
Season 7 Episode 1 - Tools for Data Scientists
Season 7 Episode 1 - Tools for Data ScientistsSeason 7 Episode 1 - Tools for Data Scientists
Season 7 Episode 1 - Tools for Data Scientists
aspyker
 
CMP376 - Another Week, Another Million Containers on Amazon EC2
CMP376 - Another Week, Another Million Containers on Amazon EC2CMP376 - Another Week, Another Million Containers on Amazon EC2
CMP376 - Another Week, Another Million Containers on Amazon EC2
aspyker
 
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and DaemonsQConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
aspyker
 
NetflixOSS Meetup S6E2 - Spinnaker, Kayenta
NetflixOSS Meetup S6E2 - Spinnaker, KayentaNetflixOSS Meetup S6E2 - Spinnaker, Kayenta
NetflixOSS Meetup S6E2 - Spinnaker, Kayenta
aspyker
 
NetflixOSS Meetup S6E1 - Titus & Containers
NetflixOSS Meetup S6E1 - Titus & ContainersNetflixOSS Meetup S6E1 - Titus & Containers
NetflixOSS Meetup S6E1 - Titus & Containers
aspyker
 
SRECon Lightning Talk
SRECon Lightning TalkSRECon Lightning Talk
SRECon Lightning Talk
aspyker
 
Container World 2018
Container World 2018Container World 2018
Container World 2018
aspyker
 
Netflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open SourceNetflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open Source
aspyker
 
Netflix OSS Meetup Season 5 Episode 1
Netflix OSS Meetup Season 5 Episode 1Netflix OSS Meetup Season 5 Episode 1
Netflix OSS Meetup Season 5 Episode 1
aspyker
 
Series of Unfortunate Netflix Container Events - QConNYC17
Series of Unfortunate Netflix Container Events - QConNYC17Series of Unfortunate Netflix Container Events - QConNYC17
Series of Unfortunate Netflix Container Events - QConNYC17
aspyker
 
Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4
aspyker
 
Re:invent 2016 Container Scheduling, Execution and AWS Integration
Re:invent 2016 Container Scheduling, Execution and AWS IntegrationRe:invent 2016 Container Scheduling, Execution and AWS Integration
Re:invent 2016 Container Scheduling, Execution and AWS Integration
aspyker
 
Netflix and Containers: Not A Stranger Thing
Netflix and Containers:  Not A Stranger ThingNetflix and Containers:  Not A Stranger Thing
Netflix and Containers: Not A Stranger Thing
aspyker
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
aspyker
 
Velocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ NetflixVelocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ Netflix
aspyker
 
Netflix Open Source Meetup Season 4 Episode 3
Netflix Open Source Meetup Season 4 Episode 3Netflix Open Source Meetup Season 4 Episode 3
Netflix Open Source Meetup Season 4 Episode 3
aspyker
 
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016
aspyker
 
Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2
aspyker
 
Netflix Container Runtime - Titus - for Container Camp 2016
Netflix Container Runtime - Titus - for Container Camp 2016Netflix Container Runtime - Titus - for Container Camp 2016
Netflix Container Runtime - Titus - for Container Camp 2016
aspyker
 

More from aspyker (20)

Herding Kats - Netflix’s Journey to Kubernetes Public
Herding Kats - Netflix’s Journey to Kubernetes PublicHerding Kats - Netflix’s Journey to Kubernetes Public
Herding Kats - Netflix’s Journey to Kubernetes Public
 
Season 7 Episode 1 - Tools for Data Scientists
Season 7 Episode 1 - Tools for Data ScientistsSeason 7 Episode 1 - Tools for Data Scientists
Season 7 Episode 1 - Tools for Data Scientists
 
CMP376 - Another Week, Another Million Containers on Amazon EC2
CMP376 - Another Week, Another Million Containers on Amazon EC2CMP376 - Another Week, Another Million Containers on Amazon EC2
CMP376 - Another Week, Another Million Containers on Amazon EC2
 
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and DaemonsQConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemons
 
NetflixOSS Meetup S6E2 - Spinnaker, Kayenta
NetflixOSS Meetup S6E2 - Spinnaker, KayentaNetflixOSS Meetup S6E2 - Spinnaker, Kayenta
NetflixOSS Meetup S6E2 - Spinnaker, Kayenta
 
NetflixOSS Meetup S6E1 - Titus & Containers
NetflixOSS Meetup S6E1 - Titus & ContainersNetflixOSS Meetup S6E1 - Titus & Containers
NetflixOSS Meetup S6E1 - Titus & Containers
 
SRECon Lightning Talk
SRECon Lightning TalkSRECon Lightning Talk
SRECon Lightning Talk
 
Container World 2018
Container World 2018Container World 2018
Container World 2018
 
Netflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open SourceNetflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open Source
 
Netflix OSS Meetup Season 5 Episode 1
Netflix OSS Meetup Season 5 Episode 1Netflix OSS Meetup Season 5 Episode 1
Netflix OSS Meetup Season 5 Episode 1
 
Series of Unfortunate Netflix Container Events - QConNYC17
Series of Unfortunate Netflix Container Events - QConNYC17Series of Unfortunate Netflix Container Events - QConNYC17
Series of Unfortunate Netflix Container Events - QConNYC17
 
Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4
 
Re:invent 2016 Container Scheduling, Execution and AWS Integration
Re:invent 2016 Container Scheduling, Execution and AWS IntegrationRe:invent 2016 Container Scheduling, Execution and AWS Integration
Re:invent 2016 Container Scheduling, Execution and AWS Integration
 
Netflix and Containers: Not A Stranger Thing
Netflix and Containers:  Not A Stranger ThingNetflix and Containers:  Not A Stranger Thing
Netflix and Containers: Not A Stranger Thing
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
 
Velocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ NetflixVelocity NYC 2016 - Containers @ Netflix
Velocity NYC 2016 - Containers @ Netflix
 
Netflix Open Source Meetup Season 4 Episode 3
Netflix Open Source Meetup Season 4 Episode 3Netflix Open Source Meetup Season 4 Episode 3
Netflix Open Source Meetup Season 4 Episode 3
 
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016
 
Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2
 
Netflix Container Runtime - Titus - for Container Camp 2016
Netflix Container Runtime - Titus - for Container Camp 2016Netflix Container Runtime - Titus - for Container Camp 2016
Netflix Container Runtime - Titus - for Container Camp 2016
 

Recently uploaded

Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsGetting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
ScyllaDB
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
FilipTomaszewski5
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...
AlexanderRichford
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
ScyllaDB
 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
UiPathCommunity
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 

Recently uploaded (20)

Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsGetting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 

Netflix Cloud Architecture and Open Source

  • 1. Netflix Architecture and Open Source Andrew Spyker Senior Software Engineer, Netflix
  • 2. About Netflix 69M members 2000+ employees (1400 tech) 80+ countries > 100M hours watch per day > ⅓ NA internet download traffic 500+ Microservices Many 10’s of thousands VM’s 3 regions across the world
  • 3. About the Speaker Cloud platform technologies Distributed configuration, service discovery, RPC, application frameworks, non-Java sidecar Container cloud Resource management and scheduling, making Docker containers operational in Amazon EC2/ECS Open Source Organize @NetflixOSS meetups & internal group Performance Assist across Netflix, but focused mainly on cloud platform perf With Netflix for ~ 1 year. Previously at IBM here in Raleigh/Durham (RTP) @aspyker ispyker.blog spot.com
  • 5. Why does Netflix open source? Allows engineers to gather feedback Openly talk, through code, on our approach Collaboration on key projects with the world Happily use proven outside open source And improve it for Netflix scale and availability Netflix culture of freedom and responsibility Want to open source? Go for it, be responsible! Recruiting and Retention Candidates know exactly what they can work on NetflixOSS engineers choose to stay at Netflix
  • 6. NetflixOSS is widely used The architecture has shaped public cloud usage Immutability, Red/Black Deploys, Chaos, Regional and worldwide high availability Offerings Pivotal Spring Cloud Large usage IBM Watson as a Service (on IBM Cloud) Nike Digital is hiring NetflixOSS experts Interesting usage “To help locate new troves of data claiming to be the files stolen from AshleyMadison, the company’s forensics team has been using a tool that Netflix released last year called Scumblr”
  • 8. Key aspects of NetflixOSS website Show how the pieces fit together Projects now discussed with each other in context OSS categories mirror internal teams No artificial categories, focal points for each area Focus on projects that are core to Netflix Projects mentioned are core and strategic
  • 10. Elastic, Web and Hyper Scale Doing this Not doing that
  • 11. Elastic, Web and Hyper Scale Front end API Another Microservice Temporal caching Durable Storage Load Balancers Strategy Benefit Make deployments automated Without automation impossible Expose well designed API to users Offloads presentation complexity to clients Remove state for mid tier services Allows easy elastic scale out Push temporal state to client and caching tier Leverage clients, avoids data tier overload Use partitioned data storage Data design and storage scales with HA Recommendation Microservice
  • 12. HA and Automatic Recovery Feeling This Not Feeling That
  • 13. Micro service Implementation Call microservice #2 Highly Available Service Runtime Recipe Ribbon REST client with Eureka Microservice #1 (REST services) App Service Microservice #2 Execute call Hystrix Eureka Server(s) Eureka Server(s) Eureka Server(s) Karyon Fallback Implementation Implementation Detail Benefits Decompose into micro services • Key user path always available • Failure does not propagate across service boundaries Karyon /w automatic Eureka registration • New instances are quickly found • Failing individual instances disappear Ribbon client with Eureka awareness • Load balances & retries across instances with “smarts” • Handles temporal instance failure Hystrix as dependency circuit breaker • Allows for fast failure • Provides graceful cross service degradation/recovery
  • 14. IaaS High Availability Region (us-east-1) us-east-1e us-east-1c Eureka Web App Service1 Service2 Cluster Auto Recovery and Scaling Services (Auto Scaling Groups) ELB’s Rule Why? Always > 2 of everything 1 is SPOF, 2 doesn’t web scale and slow DR recovery Including IaaS and cloud services You’re only as strong as your weakest dependency Use auto scaler/recovery monitoring Clusters guarantee availability and service latency Use application level health checks Instance on the network != healthy Worldwide availability Data replication, global front-end routing, cross region traffic us-east-1d
  • 15. A truly global service Replicate data across regions Be able to redirect traffic from region to region Be able to migrate regional traffic to other regions Have automated control across regions Flux Demo
  • 16. Testing is only way to prove HA Chaos Monkey Kill instances in production - runs regularly Chaos Gorilla Kills availability zones (single datacenter) Also testing for split brain important Chaos Kong Kill entire region and shift traffic globally Run frequently but with prior scheduling
  • 18. v Continuous Delivery Cluster v1 Canary v2 Cluster V2 Step Technology Developers test locally Unit test frameworks Continuous build Continuous build server based on gradle builds Build “bakes” full instance image Aminator and deployment pipeline bake images from build artifacts Developer work across dev and test Archaius allows for environment based context Developers do canary tests, red/black deployments in prod Asgard console provides app cluster common devops approach, security patterns, and visibility Continuous Build Server Baked to images (AMI’s)
  • 19. From Asgard to Spinnaker Spinnaker is our CI/CD solution CI/CD solution including baking and Jenkins integration Workflow engine for the continuous delivery Pipeline based deployment including baking Global visibility across all of our AWS regions Provides an API first design A microservices runtime HA architecture More flexible cloud model so the community can contribute back improvements not related to AWS Asgard continues to work side-by-side Spinnaker is this new end to end CI/CD tool
  • 20. Spinnaker Examples Works at Netflix scale Views of global pipelines From simple Asgard like deployment to advanced CI/CD pipelines
  • 21. Operational Visibility If you can’t see it, you can’t improve it
  • 22. Operational Visibility Microservice #1 Microservice #2 Visibility Point Technology Basic IaaS instance monitoring Not enough (not scalable, not app specific) User like external monitoring SaaS offerings or OSS like Uptime Targeted performance, sampling Vector performance and app level metrics Service to service interconnects Hystrix streams ➔Turbine aggregation ➔Hystrix dashboard Application centric metrics Servo/Spectator gauges, counters, timers sent to metrics store like Atlas Remote logging Logstash/Kibana or similar log aggregation and analysis frameworks Threshold monitoring and alerts Services like Atlas and PagerDuty for incident management Servo/ Spectator Hystrix/Turbine External Uptime Monitoring Metric/Event Repositories LogStash/Elastic Search/Kibana Incidents Atlas Vector
  • 24. Dynamic, Web Scale & Simpler Security Security Monkey Monitors security policies, tracks changes, alerts on situations Scumblr Searches internet for security “nuggets” (credentials, hacking discussions) Sketchy A safe way to collect text and screenshots from websites FIDO Automated event detection, analysis, enrichment & and enforcement Sleepy Puppy Delayed cross site scripting propagation testing framework Lemur x.509 certificate orchestration framework
  • 25. What did we not cover? Over 50 github projects NetflixOSS is “Technical indigestion as a service” Big Data, Data Persistence and UI Engineering Big Data tools used well beyond Netflix Ephemeral, semi and fully persistent data systems Recent addition of UI OSS and Falcor
  • 27. How do I get started? All of the previous slides shows NetflixOSS components Code: http://netflix.github.io Announcements: http://techblog.netflix.com/ Want to get running a bit faster? ZeroToCloud Workshop for getting started with build/bake/deploy in Amazon EC2 ZeroToDocker Docker images that containing running Netflix technologies (not production ready, but easy to understand)
  • 28. ZeroToDocker Demo Mac OS X Virtual Box Ubuntu 14.04 single kernel Container#1 Filesystem+ process Eureka Container ZuulContainer Another Container ... Docker running instances Single kernel Contained processes Zookeeper and Exhibitor A Microservices app and surrounding NetflixOSS services (Zuul to Karyon with Eureka)