Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Engineering Velocity: Shifting the Curve at Netflix


Published on

QCon New York keynote

Published in: Engineering, Technology
  • I love you Ben you’ve literally saved my life! Lets just say I was in a lot of trouble financially until I found you.Now I’m in complete AWE every time I open my betting account it’s like the numbers aren’t real, that’s why I’m constantly withdrawing the cash lol.God bless you Ben! and thank you so much for allowing me access to this amazing service. ➤➤
    Are you sure you want to  Yes  No
    Your message goes here
  • Could you use an extra $1750 a week? I'm guessing you could right? If you would like to see how you could make this type of money, right from the comfort of your own home, you absolutely need to check out this short free video. ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • Get Paid To Write Articles? YES! View 1000s of companies hiring online writers now! ●●●
    Are you sure you want to  Yes  No
    Your message goes here
  • You can get paid up to $25 per hour to be on Facebook? ◆◆◆
    Are you sure you want to  Yes  No
    Your message goes here
  • HOW TO UNLOCK HER LEGS! (SNEAK PEAK), learn more... ▲▲▲
    Are you sure you want to  Yes  No
    Your message goes here

Engineering Velocity: Shifting the Curve at Netflix

  1. Engineering Velocity: Shifting the Curve at Netflix Dianne Marsh (@dmarsh) QConNY 2014
  2. en-gi-neer-ing + ve-loc-i-ty ! applying science and technology to designing and building speed into a system
  3. Availability vs. Rate of Change Availablity(in9’s) 0 1 2 3 4 5 6 Rate of Change 0 10 100 1000
  4. Shift the Curve Availablity(in9’s) 0 1 2 3 4 5 6 Rate of Change 0 10 100 1000 10000
  5. Free the People. Optimize the Tools.
  6. Culture Freedom and Responsibility
  7. With Freedom comes Responsibility
  8. Managers’ Role Context, not Control Loosely coupled, Tightly aligned Attract and retain great talent!
  9. Get out of the Way Freedom to Innovate
  10. Support Experimentation ! How We Built a Predictive Autoscaling Engine
  11. Support Independent Paths of Exploration
  12. Build a Blameless Culture
  13. Developers deploy their own code Rapid Innovation Detection Response
  14. Optimize your Tools
  15. Netflix Build Language • Based on Gradle • Internal and Open Source • Gradle Summit talk:
  16. Jenkins Job DSL Configuration as Code Groovy Script Scripts go in Version Control
  17. Aminator Create AMI from Base AMI Image contains service and everything needed to run it Builds Unit of Deployment for Test and Prod Abstracts Cloud Details
  18. Asgard Deploys Netflix to the Cloud Red/Black push Developed to address delays in rollback
  19. Red/Black Push • Scale up new instances while running the old version • Cloud Native • Turn on traffic to new ASG • Canary Analysis • Turn off traffic to old ASG • Wait … Analyze … Roll Back?
  20. Canary Analysis ! • Production Deployment Pattern • Compare Metrics vs. Baseline Version • “Canary Analyze All The Things: How we learned to Keep Calm and Release Often”, Roy Rapoport
  21. Continuous Delivery Workflow Support the Journey Judges between Stages Represent Best Practices
  22. One Click Deployment?
  23. Regional Isolation Limit Impact of Human Error • Stagger Deployments? • Canary Testing per Region? Know your Service!
  24. Multi-Region Consistency Build Tooling to: • Schedule Deployments • Prefer Off-Peak • Choose Next Available Region • Provide Visibility by Region
  26. Chaos Monkey Kills Running Instances • Simulates failures inherent to running in the cloud • In Production
  27. Latency Monkey Introduces Latency between services
  28. Conformity Monkey Have Deployments Diverged? • Balance Regional Consistency with Regional Isolation • Build Best Practices into Tooling and Reporting
  29. Janitor Monkey Reduce Cognitive Load and Cost • Remove unused instances • Uniform way to clean up
  30. Shifting the Curve with Tools at Netflix • Value Self-Service • Test Everywhere • Awareness of Multiple Regions • Best Practices Represented in Tooling • Recover Quickly and Easily • Be Cloud Native • Respect the Journey
  31. Shifting the Curve with Culture at Netflix • Free the People! • Context not Control • Freedom to Experiment • Blameless Culture
  32. ArsTechnica, November 2012 “As the number of applications and the scale of the campaign's AWS infrastructure use climbed, the DevOps team shifted to using Asgard—an open-source tool developed by Netflix to manage cloud deployments.”
  33. Thanks! Dianne Marsh (@dmarsh)