SlideShare a Scribd company logo
Netflix and Open Source

            March 2013
          Adrian Cockcroft
    @adrianco #netflixcloud @NetflixOSS
 http://www.linkedin.com/in/adriancockcroft
Cloud Native

NetflixOSS – Cloud Native On-Ramp

Netflix Open Source Cloud Prize
Netflix Member Web Site Home Page
    Personalization Driven – How Does It Work?
How Netflix Streaming Works
Consumer
Electronics                                           User Data
                                    Web Site or
AWS Cloud
                                   Discovery API
 Services
                                                    Personalization
CDN Edge
Locations
                                                         DRM
                 Customer Device
                                   Streaming API
                  (PC, PS3, TV…)
                                                     QoS Logging


                                                        CDN
                                                   Management and
                                                      Steering
                                   OpenConnect
                                    CDN Boxes
                                                   Content Encoding
Content Delivery Service
Open Source Hardware Design + FreeBSD, bird, nginx
November 2012 Traffic
Real Web Server Dependencies Flow
         (Netflix Home page business transaction as seen by AppDynamics)

Each icon is
three to a few
hundred
instances
across three                                        Cassandra
AWS zones
                                                                memcached
                                                            Web service
         Start Here
                                                                S3 bucket




Three Personalization movie group
choosers (for US, Canada and Latam)
Cloud Native Architecture

     Clients                      Things




 Autoscaled Micro      JVM         JVM         JVM
     Services


 Autoscaled Micro      JVM         JVM       Memcached
     Services


Distributed Quorum   Cassandra   Cassandra   Cassandra
 NoSQL Datastores

                     Zone A       Zone B       Zone C
Non-Native Cloud Architecture

Agile Mobile       iOS/Android
 Mammals

  Cloudy
                   App Servers
  Buffer

Datacenter
               MySQL       Legacy Apps
Dinosaurs
New Anti-Fragile Patterns

          Micro-services
          Chaos engines
Highly available systems composed
   from ephemeral components
Stateless Micro-Service Architecture

Linux Base AMI (CentOS or Ubuntu)
  Optional
   Apache
  frontend,
                Java (JDK 6 or 7)
memcached,
non-java apps
                AppDynamics


 Monitoring
                  appagent
                 monitoring
                                Tomcat
 Log rotation                   Application war file, base     Healthcheck, status
    to S3       GC and thread    servlet, platform, client   servlets, JMX interface,
AppDynamics     dump logging     interface jars, Astyanax        Servo autoscale
machineagent
  Epic/Atlas
Cassandra Instance Architecture

Linux Base AMI (CentOS or Ubuntu)

 Tomcat and
Priam on JDK   Java (JDK 7)
Healthcheck,
   Status
               AppDynamics
                 appagent
                monitoring
                               Cassandra Server
 Monitoring
AppDynamics                    Local Ephemeral Disk Space – 2TB of SSD or 1.6TB disk
               GC and thread             holding Commit log and SSTables
machineagent   dump logging
 Epic/Atlas
Configuration State Management

      Datacenter CMDB’s woeful
      Cloud native is the solution
         Dependably complete
Edda – Configuration History
 http://techblog.netflix.com/2012/11/edda-learn-stories-of-your-cloud.html


                    Eureka
                   Services
                   metadata
   AWS
                                     AppDynamics
Instances,
                                     Request flow
ASGs, etc.



                   Edda                  Monkeys
Edda Query Examples
Find any instances that have ever had a specific public IP address
$ curl "http://edda/api/v2/view/instances;publicIpAddress=1.2.3.4;_since=0"
 ["i-0123456789","i-012345678a","i-012345678b”]

Show the most recent change to a security group
$ curl "http://edda/api/v2/aws/securityGroups/sg-0123456789;_diff;_all;_limit=2"
--- /api/v2/aws.securityGroups/sg-0123456789;_pp;_at=1351040779810
+++ /api/v2/aws.securityGroups/sg-0123456789;_pp;_at=1351044093504
@@ -1,33 +1,33 @@
 {
…
      "ipRanges" : [
        "10.10.1.1/32",
        "10.10.1.2/32",
+        "10.10.1.3/32",
-       "10.10.1.4/32"
…
 }
Cloud Native

Master copies of data are cloud resident
 Everything is dynamically provisioned
      All services are ephemeral
Scalability Demands
Asgard
http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html
Cloud Deployment Scalability
       New Autoscaled AMI – zero to 500 instances from 21:38:52 - 21:46:32, 7m40s
Scaled up and down over a few days, total 2176 instance launches, m2.2xlarge (4 core 34GB)

                        Min. 1st Qu. Median Mean 3rd Qu. Max.
                         41.0 104.2 149.0 171.8 215.8 562.0
Ephemeral Instances
  • Largest services are autoscaled
  • Average lifetime of an instance is 36 hours
                                                  P
                                                  u
                                                  s
                                                  h



Autoscale Up
                Autoscale Down
Leveraging Public Scale


           1,000 Instances   100,000 Instances

                        Grey
Public                                    Private
                        Area

Startups                     Netflix             Google
How big is Public?
    AWS Maximum Possible Instance Count 3.7 Million
      Growth >10x in Three Years, >2x Per Annum




AWS upper bound estimate based on the number of public IP Addresses
       Every provisioned instance gets a public IP by default
Availability

        Is it running yet?
How many places is it running in?
 How far apart are those places?
Antifragile API Patterns
Functional Reactive with Circuit Breakers and Bulkheads
Outages
• Running very fast with scissors
  – Mostly self inflicted – bugs, mistakes
  – Some caused by AWS bugs and mistakes


• Next step is multi-region
  – Investigating and building in stages during 2013
  – Could have prevented some of our 2012 outages
Managing Multi-Region Availability

                                         AWS                                                           DynECT
                                        Route53                        UltraDNS                         DNS


                Regional Load Balancers                                                           Regional Load Balancers




     Zone A                  Zone B                    Zone C                          Zone A                    Zone B               Zone C

Cassandra Replicas      Cassandra Replicas        Cassandra Replicas              Cassandra Replicas        Cassandra Replicas   Cassandra Replicas




        What we need is a portable way to manage multiple DNS providers….
Denominator
                     Software Defined DNS for Java

                                                    Edda, Multi-
    Use Cases                                         Region
                                                      Failover




 Common Model                                   Denominator




DNS Vendor Plug-in      AWS Route53        DynECT              UltraDNS     Etc…




API Models (varied      IAM Key Auth      User/pwd             User/pwd
and mostly broken)         REST             REST                 SOAP


    Currently being built by Adrian Cole (the jClouds guy, he works for Netflix now…)
A Cloud Native Open Source Platform
Inspiration
Three Questions


 Why is Netflix doing this?

How does it all fit together?

   What is coming next?
Beware of Geeks Bearing Gifts: Strategies for an
         Increasingly Open Economy
      Simon Wardley - Researcher at the Leading Edge Forum
How did Netflix get ahead?
Netflix Business + Developer Org   Traditional IT Operations
•   Doing it right now             • Taking their time
•   SaaS Applications              • Pilot private cloud projects
•   PaaS for agility               • Beta quality installations
•   Public IaaS for AWS features   • Small scale
•   Big data in the cloud          • Integrating several vendors
•   Integrating many APIs          • Paying big $ for software
•   FOSS from github               • Paying big $ for consulting
•   Renting hardware for 1hr       • Buying hardware for 3yrs
•   Coding in Java/Groovy/Scala    • Hacking at scripts
Netflix Platform Evolution


  2009-2010                2011-2012                      2013-2014


Bleeding Edge              Common                         Shared
  Innovation                Pattern                       Pattern

          Netflix ended up several years ahead of the
          industry, but it’s not a sustainable position
Making it easy to follow
Exploring the wild west each time   vs. laying down a shared route
Establish our            Hire, Retain and
  solutions as Best            Engage Top
Practices / Standards           Engineers


                     Goals


  Build up Netflix             Benefit from a
 Technology Brand            shared ecosystem
How does it all fit together?
NetflixOSS Continuous Build and Deployment

  Github           Maven            AWS
 NetflixOSS        Central        Base AMI
  Source




 Cloudbees
                  Dynaslave
   Jenkins                           AWS
                  AWS Build
 Aminator                         Baked AMIs
                   Slaves
   Bakery




    Odin           Asgard           AWS
 Orchestration    (+ Frigga)       Account
     API           Console
NetflixOSS Services Scope


AWS Account
Asgard Console


Archaius Config
                  Multiple AWS Regions
    Service


 Cross region
  Priam C*        Eureka Registry


  Explorers
 Dashboards
                   Exhibitor ZK
                                    3 AWS Zones
                                      Application
                                                             Priam              Evcache
     Atlas         Edda History        Clusters
                                                          Cassandra           Memcached
  Monitoring                        Autoscale Groups
                                                       Persistent Storage   Ephemeral Storage
                                       Instances
                   Simian Army
Genie Hadoop
  Services
NetflixOSS Instance Libraries

                 • Baked AMI – Tomcat, Apache, your code

Initialization   • Governator – Guice based dependency injection
                 • Archaius – dynamic configuration properties client
                 • Eureka - service registration client




  Service        • Karyon - Base Server for inbound requests
                 • RxJava – Reactive pattern
                 • Hystrix/Turbine – dependencies and real-time status
 Requests        • Ribbon - REST Client for outbound calls



                 • Astyanax – Cassandra client and pattern library

Data Access      • Evcache – Zone aware Memcached client
                 • Curator – Zookeeper patterns
                 • Denominator – DNS routing abstraction



                 • Blitz4j – non-blocking logging
  Logging        • Servo – metrics export for autoscaling
                 • Atlas – high volume instrumentation
NetflixOSS Testing and Automation

               • CassJmeter – Load testing for Cassandra
 Test Tools    • Circus Monkey – Test account reservation rebalancing




               • Janitor Monkey – Cleans up unused resources
               • Efficiency Monkey
Maintenance    • Doctor Monkey
               • Howler Monkey – Complains about expiring certs


               • Chaos Monkey – Kills Instances
               • Chaos Gorilla – Kills Availability Zones
Availability   • Chaos Kong – Kills Regions
               • Latency Monkey – Latency and error injection




               • Security Monkey
  Security     • Conformity Monkey
Example Application – RSS Reader
What’s Coming Next?

           Better portability

           Higher availability
 More
Features   Easier to deploy

           Contributions from end users

           Contributions from vendors

                     More Use Cases
Vendor Driven Portability
     Interest in using NetflixOSS for Enterprise Private Clouds

                                            “It’s done when it runs Asgard”
                                            Functionally complete
                                            Demonstrated March
                                            Release 3.3 in 2Q13


                                            Some vendor interest
Some vendor interest                        Many missing features
Needs AWS compatible Autoscaler             Bait and switch AWS API strategy
AWS 2009 vs. ???




 Eucalyptus 3.3
Netflix Cloud Prize

Boosting the @NetflixOSS Ecosystem
In 2012 Netflix Engineering won this..
We’d like to give out prizes too
            But what for?
     Contributions to NetflixOSS!
     Shared under Apache license
          Located on github
How long do you have?

   Entries open March 13th
 Entries close September 15th
         Six months…
Who can win?

   Almost anyone, anywhere…
Except current or former Netflix or
         AWS employees
Who decides who wins?

   Nominating Committee
      Panel of Judges
Judges

         Aino Corry
                                                                        Martin Fowler
Program Chair for Qcon/GOTO          Simon Wardley              Chief Scientist Thoughtworks
                                        Strategist




       Werner Vogels                                                Yury Izrailevsky
       CTO Amazon                       Joe Weinman                 VP Cloud Netflix
                              SVP Telx, Author “Cloudonomics”
What are Judges Looking For?
   Eligible, Apache 2.0 licensed
                              Original and useful contribution to NetflixOSS

       Code that successfully builds and passes a test suite

                 A large number of watchers, stars and forks on github

   NetflixOSS project pull requests
                                                  Good code quality and structure

               Documentation on how to build and run it

Evidence that code is in use by other projects, or is running in production
What do you win?
One winner in each of the 10 categories
  Ticket and expenses to attend AWS
      Re:Invent 2013 in Las Vegas
               A Trophy
How do you enter?
    Get a (free) github account
Fork github.com/netflix/cloud-prize
    Send us your email address
   Describe and build your entry

           Twitter #cloudprize
Award
                                           Apache
         Registration                                                  Close Entries          AWS       Ceremony
Github   Opens Today
                               Github     Licensed           Github   September 15                       Dinner
                                        Contributions                                       Re:Invent
                                                                                                        November




                                                  Judges                                          Winners
            $10K cash
            $5K AWS


                                                   Netflix
                                                                                       Nominations          Categories
          Ten Prize                           Engineering
         Categories
                          AWS
Trophy                  Re:Invent                                 Conforms to           Working             Community
                         Tickets                 Entrants            Rules               Code                Traction
Functionality and scale now, portability coming

   Moving from parts to a platform in 2013

       Netflix is fostering an ecosystem

      Rapid Evolution - Low MTBIAMSH
      (Mean Time Between Idea And Making Stuff Happen)
Takeaway

Netflix is making it easy for everyone to adopt Cloud Native patterns.

     Open Source is not just the default, it’s a strategic weapon.

                        http://netflix.github.com
                       http://techblog.netflix.com
                       http://slideshare.net/Netflix

                http://www.linkedin.com/in/adriancockcroft

                   @adrianco #netflixcloud @NetflixOSS

More Related Content

What's hot

Azure kubernetes service (aks)
Azure kubernetes service (aks)Azure kubernetes service (aks)
Azure kubernetes service (aks)
Akash Agrawal
 
Continuous Deployment Practices, with Production, Test and Development Enviro...
Continuous Deployment Practices, with Production, Test and Development Enviro...Continuous Deployment Practices, with Production, Test and Development Enviro...
Continuous Deployment Practices, with Production, Test and Development Enviro...
Amazon Web Services
 
Containers on AWS: An Introduction
Containers on AWS: An IntroductionContainers on AWS: An Introduction
Containers on AWS: An Introduction
Amazon Web Services
 
Aws overview (Amazon Web Services)
Aws overview (Amazon Web Services)Aws overview (Amazon Web Services)
Aws overview (Amazon Web Services)
Jatinder Randhawa
 
Basics AWS Presentation
Basics AWS PresentationBasics AWS Presentation
Basics AWS Presentation
Shyam Kumar
 
9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)
9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)
9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)
Amazon Web Services Korea
 
Azure Application Modernization
Azure Application ModernizationAzure Application Modernization
Azure Application Modernization
Karina Matos
 
Serverless Architecture and Best Practices
Serverless Architecture and Best PracticesServerless Architecture and Best Practices
Serverless Architecture and Best Practices
Amazon Web Services
 
AWS
AWSAWS
AWS
ARYA TM
 
Continuous Delivery with AWS Lambda - AWS April 2016 Webinar Series
Continuous Delivery with AWS Lambda - AWS April 2016 Webinar SeriesContinuous Delivery with AWS Lambda - AWS April 2016 Webinar Series
Continuous Delivery with AWS Lambda - AWS April 2016 Webinar Series
Amazon Web Services
 
Introduction to CloudFront
Introduction to CloudFrontIntroduction to CloudFront
Introduction to CloudFront
Amazon Web Services
 
Architectures for High Availability - QConSF
Architectures for High Availability - QConSFArchitectures for High Availability - QConSF
Architectures for High Availability - QConSF
Adrian Cockcroft
 
Introduction to Cloud Computing with AWS (Thai Session)
Introduction to Cloud Computing with AWS (Thai Session)Introduction to Cloud Computing with AWS (Thai Session)
Introduction to Cloud Computing with AWS (Thai Session)
Amazon Web Services
 
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Amazon Web Services
 
AWS Cloud Adoption Framework
AWS Cloud Adoption Framework AWS Cloud Adoption Framework
AWS Cloud Adoption Framework
Amazon Web Services
 
Microsoft azure
Microsoft azureMicrosoft azure
Microsoft azure
Mohammad Ilyas Malik
 
AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...
AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...
AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...
Amazon Web Services Korea
 
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesYow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Adrian Cockcroft
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
Karthik Gaekwad
 
Introduction To AWS & AWS Lambda
Introduction To AWS & AWS LambdaIntroduction To AWS & AWS Lambda
Introduction To AWS & AWS Lambda
An Nguyen
 

What's hot (20)

Azure kubernetes service (aks)
Azure kubernetes service (aks)Azure kubernetes service (aks)
Azure kubernetes service (aks)
 
Continuous Deployment Practices, with Production, Test and Development Enviro...
Continuous Deployment Practices, with Production, Test and Development Enviro...Continuous Deployment Practices, with Production, Test and Development Enviro...
Continuous Deployment Practices, with Production, Test and Development Enviro...
 
Containers on AWS: An Introduction
Containers on AWS: An IntroductionContainers on AWS: An Introduction
Containers on AWS: An Introduction
 
Aws overview (Amazon Web Services)
Aws overview (Amazon Web Services)Aws overview (Amazon Web Services)
Aws overview (Amazon Web Services)
 
Basics AWS Presentation
Basics AWS PresentationBasics AWS Presentation
Basics AWS Presentation
 
9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)
9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)
9월 웨비나 - AWS 클라우드 보안의 이해 (양승도 솔루션즈 아키텍트)
 
Azure Application Modernization
Azure Application ModernizationAzure Application Modernization
Azure Application Modernization
 
Serverless Architecture and Best Practices
Serverless Architecture and Best PracticesServerless Architecture and Best Practices
Serverless Architecture and Best Practices
 
AWS
AWSAWS
AWS
 
Continuous Delivery with AWS Lambda - AWS April 2016 Webinar Series
Continuous Delivery with AWS Lambda - AWS April 2016 Webinar SeriesContinuous Delivery with AWS Lambda - AWS April 2016 Webinar Series
Continuous Delivery with AWS Lambda - AWS April 2016 Webinar Series
 
Introduction to CloudFront
Introduction to CloudFrontIntroduction to CloudFront
Introduction to CloudFront
 
Architectures for High Availability - QConSF
Architectures for High Availability - QConSFArchitectures for High Availability - QConSF
Architectures for High Availability - QConSF
 
Introduction to Cloud Computing with AWS (Thai Session)
Introduction to Cloud Computing with AWS (Thai Session)Introduction to Cloud Computing with AWS (Thai Session)
Introduction to Cloud Computing with AWS (Thai Session)
 
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
 
AWS Cloud Adoption Framework
AWS Cloud Adoption Framework AWS Cloud Adoption Framework
AWS Cloud Adoption Framework
 
Microsoft azure
Microsoft azureMicrosoft azure
Microsoft azure
 
AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...
AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...
AWS Transit Gateway를 통한 Multi-VPC 아키텍처 패턴 - 강동환 솔루션즈 아키텍트, AWS :: AWS Summit ...
 
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesYow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
 
Introduction To AWS & AWS Lambda
Introduction To AWS & AWS LambdaIntroduction To AWS & AWS Lambda
Introduction To AWS & AWS Lambda
 

Viewers also liked

Netflix CDN and Open Source
Netflix CDN and Open SourceNetflix CDN and Open Source
Netflix CDN and Open Source
Gleb Smirnoff
 
Netflix Global Cloud Architecture
Netflix Global Cloud ArchitectureNetflix Global Cloud Architecture
Netflix Global Cloud Architecture
Adrian Cockcroft
 
NetflixOSS Meetup
NetflixOSS MeetupNetflixOSS Meetup
NetflixOSS Meetup
Adrian Cockcroft
 
Dystopia as a Service
Dystopia as a ServiceDystopia as a Service
Dystopia as a Service
Adrian Cockcroft
 
Netflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at GlueconNetflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at Gluecon
Adrian Cockcroft
 
Gluecon keynote
Gluecon keynoteGluecon keynote
Gluecon keynote
Adrian Cockcroft
 
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial IntroductionGluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Adrian Cockcroft
 
Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013
Adrian Cockcroft
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
Adrian Cockcroft
 
Cassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWSCassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWS
Adrian Cockcroft
 
AWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at NetflixAWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at Netflix
Adrian Cockcroft
 
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Adrian Cockcroft
 
SV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformSV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source Platform
Adrian Cockcroft
 
MicroServices at Netflix - challenges of scale
MicroServices at Netflix - challenges of scaleMicroServices at Netflix - challenges of scale
MicroServices at Netflix - challenges of scale
Sudhir Tonse
 
Ejercicios del tema4
Ejercicios del tema4Ejercicios del tema4
Ejercicios del tema4
Luis Maria Morales Alonso
 
Netflix Culture: Freedom & Responsibility 넷플릭스 문화: 자유와 책임 (한국어 번역)
Netflix Culture: Freedom & Responsibility      넷플릭스 문화: 자유와 책임 (한국어 번역)Netflix Culture: Freedom & Responsibility      넷플릭스 문화: 자유와 책임 (한국어 번역)
Netflix Culture: Freedom & Responsibility 넷플릭스 문화: 자유와 책임 (한국어 번역)
Hong Nam Yang
 
ITI-Presentation-netflix
ITI-Presentation-netflixITI-Presentation-netflix
ITI-Presentation-netflix
Angela Chen
 
NetflixOSS Open House Lightning talks
NetflixOSS Open House Lightning talksNetflixOSS Open House Lightning talks
NetflixOSS Open House Lightning talks
Ruslan Meshenberg
 
Netflix Aws Startup Tour 090617134938 Phpapp02
Netflix Aws Startup Tour 090617134938 Phpapp02Netflix Aws Startup Tour 090617134938 Phpapp02
Netflix Aws Startup Tour 090617134938 Phpapp02
GovCloud Network
 
Netflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search RoadshowNetflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search Roadshow
Adrian Cockcroft
 

Viewers also liked (20)

Netflix CDN and Open Source
Netflix CDN and Open SourceNetflix CDN and Open Source
Netflix CDN and Open Source
 
Netflix Global Cloud Architecture
Netflix Global Cloud ArchitectureNetflix Global Cloud Architecture
Netflix Global Cloud Architecture
 
NetflixOSS Meetup
NetflixOSS MeetupNetflixOSS Meetup
NetflixOSS Meetup
 
Dystopia as a Service
Dystopia as a ServiceDystopia as a Service
Dystopia as a Service
 
Netflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at GlueconNetflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at Gluecon
 
Gluecon keynote
Gluecon keynoteGluecon keynote
Gluecon keynote
 
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial IntroductionGluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
 
Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
 
Cassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWSCassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWS
 
AWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at NetflixAWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at Netflix
 
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
 
SV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformSV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source Platform
 
MicroServices at Netflix - challenges of scale
MicroServices at Netflix - challenges of scaleMicroServices at Netflix - challenges of scale
MicroServices at Netflix - challenges of scale
 
Ejercicios del tema4
Ejercicios del tema4Ejercicios del tema4
Ejercicios del tema4
 
Netflix Culture: Freedom & Responsibility 넷플릭스 문화: 자유와 책임 (한국어 번역)
Netflix Culture: Freedom & Responsibility      넷플릭스 문화: 자유와 책임 (한국어 번역)Netflix Culture: Freedom & Responsibility      넷플릭스 문화: 자유와 책임 (한국어 번역)
Netflix Culture: Freedom & Responsibility 넷플릭스 문화: 자유와 책임 (한국어 번역)
 
ITI-Presentation-netflix
ITI-Presentation-netflixITI-Presentation-netflix
ITI-Presentation-netflix
 
NetflixOSS Open House Lightning talks
NetflixOSS Open House Lightning talksNetflixOSS Open House Lightning talks
NetflixOSS Open House Lightning talks
 
Netflix Aws Startup Tour 090617134938 Phpapp02
Netflix Aws Startup Tour 090617134938 Phpapp02Netflix Aws Startup Tour 090617134938 Phpapp02
Netflix Aws Startup Tour 090617134938 Phpapp02
 
Netflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search RoadshowNetflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search Roadshow
 

Similar to Netflix and Open Source

게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
Amazon Web Services Korea
 
Microservices reativos usando a stack do Netflix na AWS
Microservices reativos usando a stack do Netflix na AWSMicroservices reativos usando a stack do Netflix na AWS
Microservices reativos usando a stack do Netflix na AWS
Diego Pacheco
 
Web Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud PlatformWeb Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud Platform
Sudhir Tonse
 
Introduction to AWS tools
Introduction to AWS toolsIntroduction to AWS tools
Introduction to AWS tools
Amazon Web Services
 
Netflix presents at MassTLC Cloud Summit 2013
Netflix presents at MassTLC Cloud Summit 2013Netflix presents at MassTLC Cloud Summit 2013
Netflix presents at MassTLC Cloud Summit 2013
MassTLC
 
AWS Summit 2011: Application Security Best Practices
AWS Summit 2011: Application Security Best PracticesAWS Summit 2011: Application Security Best Practices
AWS Summit 2011: Application Security Best Practices
Amazon Web Services
 
Netflix on Cloud - combined slides for Dev and Ops
Netflix on Cloud - combined slides for Dev and OpsNetflix on Cloud - combined slides for Dev and Ops
Netflix on Cloud - combined slides for Dev and Ops
Adrian Cockcroft
 
Svc 202-netflix-open-source
Svc 202-netflix-open-sourceSvc 202-netflix-open-source
Svc 202-netflix-open-source
Ruslan Meshenberg
 
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...
Amazon Web Services
 
AWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:CapAWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:Cap
Ian Massingham
 
AWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:CapAWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:Cap
Adrian Hornsby
 
Usman Shakeel - Cloud Rendering at Scale :: AWS Rendering Seminar
Usman Shakeel - Cloud Rendering at Scale :: AWS Rendering SeminarUsman Shakeel - Cloud Rendering at Scale :: AWS Rendering Seminar
Usman Shakeel - Cloud Rendering at Scale :: AWS Rendering Seminar
Amazon Web Services Korea
 
Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012
CLOUDIAN KK
 
AWS re:Invent 2016 recap (part 1)
AWS re:Invent 2016 recap (part 1)AWS re:Invent 2016 recap (part 1)
AWS re:Invent 2016 recap (part 1)
Julien SIMON
 
Ram chinta hug-20120922-v1
Ram chinta hug-20120922-v1Ram chinta hug-20120922-v1
Ram chinta hug-20120922-v1
Ram Chinta
 
4K Media Workflows on AWS
4K Media Workflows on AWS4K Media Workflows on AWS
4K Media Workflows on AWS
Amazon Web Services
 
Docker based Hadoop provisioning - Hadoop Summit 2014
Docker based Hadoop provisioning - Hadoop Summit 2014 Docker based Hadoop provisioning - Hadoop Summit 2014
Docker based Hadoop provisioning - Hadoop Summit 2014
Janos Matyas
 
Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk
Eran Gampel
 
Aws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaAws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon Elisha
Helen Rogers
 
AWS Cloud School | London - Part 1
AWS Cloud School | London - Part 1AWS Cloud School | London - Part 1
AWS Cloud School | London - Part 1
Amazon Web Services
 

Similar to Netflix and Open Source (20)

게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
게임을 위한 Cloud Native on AWS (김일호 솔루션즈 아키텍트, AWS) :: Gaming on AWS 2018
 
Microservices reativos usando a stack do Netflix na AWS
Microservices reativos usando a stack do Netflix na AWSMicroservices reativos usando a stack do Netflix na AWS
Microservices reativos usando a stack do Netflix na AWS
 
Web Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud PlatformWeb Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud Platform
 
Introduction to AWS tools
Introduction to AWS toolsIntroduction to AWS tools
Introduction to AWS tools
 
Netflix presents at MassTLC Cloud Summit 2013
Netflix presents at MassTLC Cloud Summit 2013Netflix presents at MassTLC Cloud Summit 2013
Netflix presents at MassTLC Cloud Summit 2013
 
AWS Summit 2011: Application Security Best Practices
AWS Summit 2011: Application Security Best PracticesAWS Summit 2011: Application Security Best Practices
AWS Summit 2011: Application Security Best Practices
 
Netflix on Cloud - combined slides for Dev and Ops
Netflix on Cloud - combined slides for Dev and OpsNetflix on Cloud - combined slides for Dev and Ops
Netflix on Cloud - combined slides for Dev and Ops
 
Svc 202-netflix-open-source
Svc 202-netflix-open-sourceSvc 202-netflix-open-source
Svc 202-netflix-open-source
 
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...
 
AWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:CapAWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:Cap
 
AWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:CapAWS re:Invent 2016 Day 1 Keynote re:Cap
AWS re:Invent 2016 Day 1 Keynote re:Cap
 
Usman Shakeel - Cloud Rendering at Scale :: AWS Rendering Seminar
Usman Shakeel - Cloud Rendering at Scale :: AWS Rendering SeminarUsman Shakeel - Cloud Rendering at Scale :: AWS Rendering Seminar
Usman Shakeel - Cloud Rendering at Scale :: AWS Rendering Seminar
 
Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012
 
AWS re:Invent 2016 recap (part 1)
AWS re:Invent 2016 recap (part 1)AWS re:Invent 2016 recap (part 1)
AWS re:Invent 2016 recap (part 1)
 
Ram chinta hug-20120922-v1
Ram chinta hug-20120922-v1Ram chinta hug-20120922-v1
Ram chinta hug-20120922-v1
 
4K Media Workflows on AWS
4K Media Workflows on AWS4K Media Workflows on AWS
4K Media Workflows on AWS
 
Docker based Hadoop provisioning - Hadoop Summit 2014
Docker based Hadoop provisioning - Hadoop Summit 2014 Docker based Hadoop provisioning - Hadoop Summit 2014
Docker based Hadoop provisioning - Hadoop Summit 2014
 
Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk
 
Aws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaAws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon Elisha
 
AWS Cloud School | London - Part 1
AWS Cloud School | London - Part 1AWS Cloud School | London - Part 1
AWS Cloud School | London - Part 1
 

More from Adrian Cockcroft

Netflix in the Cloud at SV Forum
Netflix in the Cloud at SV ForumNetflix in the Cloud at SV Forum
Netflix in the Cloud at SV Forum
Adrian Cockcroft
 
Cloud Architecture Tutorial - Why and What (1of 3)
Cloud Architecture Tutorial - Why and What (1of 3) Cloud Architecture Tutorial - Why and What (1of 3)
Cloud Architecture Tutorial - Why and What (1of 3)
Adrian Cockcroft
 
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Adrian Cockcroft
 
Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)
Adrian Cockcroft
 
Global Netflix Platform
Global Netflix PlatformGlobal Netflix Platform
Global Netflix Platform
Adrian Cockcroft
 
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Adrian Cockcroft
 
Migrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraMigrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global Cassandra
Adrian Cockcroft
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011
Adrian Cockcroft
 
Migrating to Public Cloud
Migrating to Public CloudMigrating to Public Cloud
Migrating to Public Cloud
Adrian Cockcroft
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connect
Adrian Cockcroft
 
Netflix in the cloud 2011
Netflix in the cloud 2011Netflix in the cloud 2011
Netflix in the cloud 2011
Adrian Cockcroft
 
Cmg06 utilization is useless
Cmg06 utilization is uselessCmg06 utilization is useless
Cmg06 utilization is useless
Adrian Cockcroft
 
NoSQL for Netflix
NoSQL for NetflixNoSQL for Netflix
NoSQL for Netflix
Adrian Cockcroft
 

More from Adrian Cockcroft (13)

Netflix in the Cloud at SV Forum
Netflix in the Cloud at SV ForumNetflix in the Cloud at SV Forum
Netflix in the Cloud at SV Forum
 
Cloud Architecture Tutorial - Why and What (1of 3)
Cloud Architecture Tutorial - Why and What (1of 3) Cloud Architecture Tutorial - Why and What (1of 3)
Cloud Architecture Tutorial - Why and What (1of 3)
 
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
 
Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)
 
Global Netflix Platform
Global Netflix PlatformGlobal Netflix Platform
Global Netflix Platform
 
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
 
Migrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraMigrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global Cassandra
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011
 
Migrating to Public Cloud
Migrating to Public CloudMigrating to Public Cloud
Migrating to Public Cloud
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connect
 
Netflix in the cloud 2011
Netflix in the cloud 2011Netflix in the cloud 2011
Netflix in the cloud 2011
 
Cmg06 utilization is useless
Cmg06 utilization is uselessCmg06 utilization is useless
Cmg06 utilization is useless
 
NoSQL for Netflix
NoSQL for NetflixNoSQL for Netflix
NoSQL for Netflix
 

Recently uploaded

Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 

Recently uploaded (20)

Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 

Netflix and Open Source

  • 1. Netflix and Open Source March 2013 Adrian Cockcroft @adrianco #netflixcloud @NetflixOSS http://www.linkedin.com/in/adriancockcroft
  • 2. Cloud Native NetflixOSS – Cloud Native On-Ramp Netflix Open Source Cloud Prize
  • 3. Netflix Member Web Site Home Page Personalization Driven – How Does It Work?
  • 4. How Netflix Streaming Works Consumer Electronics User Data Web Site or AWS Cloud Discovery API Services Personalization CDN Edge Locations DRM Customer Device Streaming API (PC, PS3, TV…) QoS Logging CDN Management and Steering OpenConnect CDN Boxes Content Encoding
  • 5. Content Delivery Service Open Source Hardware Design + FreeBSD, bird, nginx
  • 7. Real Web Server Dependencies Flow (Netflix Home page business transaction as seen by AppDynamics) Each icon is three to a few hundred instances across three Cassandra AWS zones memcached Web service Start Here S3 bucket Three Personalization movie group choosers (for US, Canada and Latam)
  • 8. Cloud Native Architecture Clients Things Autoscaled Micro JVM JVM JVM Services Autoscaled Micro JVM JVM Memcached Services Distributed Quorum Cassandra Cassandra Cassandra NoSQL Datastores Zone A Zone B Zone C
  • 9. Non-Native Cloud Architecture Agile Mobile iOS/Android Mammals Cloudy App Servers Buffer Datacenter MySQL Legacy Apps Dinosaurs
  • 10. New Anti-Fragile Patterns Micro-services Chaos engines Highly available systems composed from ephemeral components
  • 11. Stateless Micro-Service Architecture Linux Base AMI (CentOS or Ubuntu) Optional Apache frontend, Java (JDK 6 or 7) memcached, non-java apps AppDynamics Monitoring appagent monitoring Tomcat Log rotation Application war file, base Healthcheck, status to S3 GC and thread servlet, platform, client servlets, JMX interface, AppDynamics dump logging interface jars, Astyanax Servo autoscale machineagent Epic/Atlas
  • 12. Cassandra Instance Architecture Linux Base AMI (CentOS or Ubuntu) Tomcat and Priam on JDK Java (JDK 7) Healthcheck, Status AppDynamics appagent monitoring Cassandra Server Monitoring AppDynamics Local Ephemeral Disk Space – 2TB of SSD or 1.6TB disk GC and thread holding Commit log and SSTables machineagent dump logging Epic/Atlas
  • 13. Configuration State Management Datacenter CMDB’s woeful Cloud native is the solution Dependably complete
  • 14. Edda – Configuration History http://techblog.netflix.com/2012/11/edda-learn-stories-of-your-cloud.html Eureka Services metadata AWS AppDynamics Instances, Request flow ASGs, etc. Edda Monkeys
  • 15. Edda Query Examples Find any instances that have ever had a specific public IP address $ curl "http://edda/api/v2/view/instances;publicIpAddress=1.2.3.4;_since=0" ["i-0123456789","i-012345678a","i-012345678b”] Show the most recent change to a security group $ curl "http://edda/api/v2/aws/securityGroups/sg-0123456789;_diff;_all;_limit=2" --- /api/v2/aws.securityGroups/sg-0123456789;_pp;_at=1351040779810 +++ /api/v2/aws.securityGroups/sg-0123456789;_pp;_at=1351044093504 @@ -1,33 +1,33 @@ { … "ipRanges" : [ "10.10.1.1/32", "10.10.1.2/32", + "10.10.1.3/32", - "10.10.1.4/32" … }
  • 16. Cloud Native Master copies of data are cloud resident Everything is dynamically provisioned All services are ephemeral
  • 19. Cloud Deployment Scalability New Autoscaled AMI – zero to 500 instances from 21:38:52 - 21:46:32, 7m40s Scaled up and down over a few days, total 2176 instance launches, m2.2xlarge (4 core 34GB) Min. 1st Qu. Median Mean 3rd Qu. Max. 41.0 104.2 149.0 171.8 215.8 562.0
  • 20. Ephemeral Instances • Largest services are autoscaled • Average lifetime of an instance is 36 hours P u s h Autoscale Up Autoscale Down
  • 21. Leveraging Public Scale 1,000 Instances 100,000 Instances Grey Public Private Area Startups Netflix Google
  • 22. How big is Public? AWS Maximum Possible Instance Count 3.7 Million Growth >10x in Three Years, >2x Per Annum AWS upper bound estimate based on the number of public IP Addresses Every provisioned instance gets a public IP by default
  • 23. Availability Is it running yet? How many places is it running in? How far apart are those places?
  • 24. Antifragile API Patterns Functional Reactive with Circuit Breakers and Bulkheads
  • 25.
  • 26. Outages • Running very fast with scissors – Mostly self inflicted – bugs, mistakes – Some caused by AWS bugs and mistakes • Next step is multi-region – Investigating and building in stages during 2013 – Could have prevented some of our 2012 outages
  • 27. Managing Multi-Region Availability AWS DynECT Route53 UltraDNS DNS Regional Load Balancers Regional Load Balancers Zone A Zone B Zone C Zone A Zone B Zone C Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas What we need is a portable way to manage multiple DNS providers….
  • 28. Denominator Software Defined DNS for Java Edda, Multi- Use Cases Region Failover Common Model Denominator DNS Vendor Plug-in AWS Route53 DynECT UltraDNS Etc… API Models (varied IAM Key Auth User/pwd User/pwd and mostly broken) REST REST SOAP Currently being built by Adrian Cole (the jClouds guy, he works for Netflix now…)
  • 29. A Cloud Native Open Source Platform
  • 31. Three Questions Why is Netflix doing this? How does it all fit together? What is coming next?
  • 32. Beware of Geeks Bearing Gifts: Strategies for an Increasingly Open Economy Simon Wardley - Researcher at the Leading Edge Forum
  • 33. How did Netflix get ahead? Netflix Business + Developer Org Traditional IT Operations • Doing it right now • Taking their time • SaaS Applications • Pilot private cloud projects • PaaS for agility • Beta quality installations • Public IaaS for AWS features • Small scale • Big data in the cloud • Integrating several vendors • Integrating many APIs • Paying big $ for software • FOSS from github • Paying big $ for consulting • Renting hardware for 1hr • Buying hardware for 3yrs • Coding in Java/Groovy/Scala • Hacking at scripts
  • 34. Netflix Platform Evolution 2009-2010 2011-2012 2013-2014 Bleeding Edge Common Shared Innovation Pattern Pattern Netflix ended up several years ahead of the industry, but it’s not a sustainable position
  • 35. Making it easy to follow Exploring the wild west each time vs. laying down a shared route
  • 36. Establish our Hire, Retain and solutions as Best Engage Top Practices / Standards Engineers Goals Build up Netflix Benefit from a Technology Brand shared ecosystem
  • 37. How does it all fit together?
  • 38. NetflixOSS Continuous Build and Deployment Github Maven AWS NetflixOSS Central Base AMI Source Cloudbees Dynaslave Jenkins AWS AWS Build Aminator Baked AMIs Slaves Bakery Odin Asgard AWS Orchestration (+ Frigga) Account API Console
  • 39. NetflixOSS Services Scope AWS Account Asgard Console Archaius Config Multiple AWS Regions Service Cross region Priam C* Eureka Registry Explorers Dashboards Exhibitor ZK 3 AWS Zones Application Priam Evcache Atlas Edda History Clusters Cassandra Memcached Monitoring Autoscale Groups Persistent Storage Ephemeral Storage Instances Simian Army Genie Hadoop Services
  • 40. NetflixOSS Instance Libraries • Baked AMI – Tomcat, Apache, your code Initialization • Governator – Guice based dependency injection • Archaius – dynamic configuration properties client • Eureka - service registration client Service • Karyon - Base Server for inbound requests • RxJava – Reactive pattern • Hystrix/Turbine – dependencies and real-time status Requests • Ribbon - REST Client for outbound calls • Astyanax – Cassandra client and pattern library Data Access • Evcache – Zone aware Memcached client • Curator – Zookeeper patterns • Denominator – DNS routing abstraction • Blitz4j – non-blocking logging Logging • Servo – metrics export for autoscaling • Atlas – high volume instrumentation
  • 41. NetflixOSS Testing and Automation • CassJmeter – Load testing for Cassandra Test Tools • Circus Monkey – Test account reservation rebalancing • Janitor Monkey – Cleans up unused resources • Efficiency Monkey Maintenance • Doctor Monkey • Howler Monkey – Complains about expiring certs • Chaos Monkey – Kills Instances • Chaos Gorilla – Kills Availability Zones Availability • Chaos Kong – Kills Regions • Latency Monkey – Latency and error injection • Security Monkey Security • Conformity Monkey
  • 43. What’s Coming Next? Better portability Higher availability More Features Easier to deploy Contributions from end users Contributions from vendors More Use Cases
  • 44. Vendor Driven Portability Interest in using NetflixOSS for Enterprise Private Clouds “It’s done when it runs Asgard” Functionally complete Demonstrated March Release 3.3 in 2Q13 Some vendor interest Some vendor interest Many missing features Needs AWS compatible Autoscaler Bait and switch AWS API strategy
  • 45. AWS 2009 vs. ??? Eucalyptus 3.3
  • 46. Netflix Cloud Prize Boosting the @NetflixOSS Ecosystem
  • 47. In 2012 Netflix Engineering won this..
  • 48. We’d like to give out prizes too But what for? Contributions to NetflixOSS! Shared under Apache license Located on github
  • 49.
  • 50. How long do you have? Entries open March 13th Entries close September 15th Six months…
  • 51. Who can win? Almost anyone, anywhere… Except current or former Netflix or AWS employees
  • 52. Who decides who wins? Nominating Committee Panel of Judges
  • 53. Judges Aino Corry Martin Fowler Program Chair for Qcon/GOTO Simon Wardley Chief Scientist Thoughtworks Strategist Werner Vogels Yury Izrailevsky CTO Amazon Joe Weinman VP Cloud Netflix SVP Telx, Author “Cloudonomics”
  • 54. What are Judges Looking For? Eligible, Apache 2.0 licensed Original and useful contribution to NetflixOSS Code that successfully builds and passes a test suite A large number of watchers, stars and forks on github NetflixOSS project pull requests Good code quality and structure Documentation on how to build and run it Evidence that code is in use by other projects, or is running in production
  • 55. What do you win? One winner in each of the 10 categories Ticket and expenses to attend AWS Re:Invent 2013 in Las Vegas A Trophy
  • 56. How do you enter? Get a (free) github account Fork github.com/netflix/cloud-prize Send us your email address Describe and build your entry Twitter #cloudprize
  • 57. Award Apache Registration Close Entries AWS Ceremony Github Opens Today Github Licensed Github September 15 Dinner Contributions Re:Invent November Judges Winners $10K cash $5K AWS Netflix Nominations Categories Ten Prize Engineering Categories AWS Trophy Re:Invent Conforms to Working Community Tickets Entrants Rules Code Traction
  • 58. Functionality and scale now, portability coming Moving from parts to a platform in 2013 Netflix is fostering an ecosystem Rapid Evolution - Low MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)
  • 59. Takeaway Netflix is making it easy for everyone to adopt Cloud Native patterns. Open Source is not just the default, it’s a strategic weapon. http://netflix.github.com http://techblog.netflix.com http://slideshare.net/Netflix http://www.linkedin.com/in/adriancockcroft @adrianco #netflixcloud @NetflixOSS

Editor's Notes

  1. When Netflix first moved to cloud it was bleeding edge innovation, we figured stuff out and made stuff up from first principles. Over the last two years more large companies have moved to cloud, and the principles, practices and patterns have become better understood and adopted. At this point there is intense interest in how Netflix runs in the cloud, and several forward looking organizations adopting our architectures and starting to use some of the code we have shared. Over the coming years, we want to make it easier for people to share the patterns we use.
  2. The railroad made it possible for California to be developed quickly, by creating an easy to follow path we can create a much bigger ecosystem around the Netflix platform