Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Engineering Tools at Netflix: Enabling Continuous Delivery

4,715 views

Published on

An overview of the Netflix build, bake and deploy infrastructure, and the tools used to enable continuous delivery.

Published in: Technology
  • Be the first to comment

Engineering Tools at Netflix: Enabling Continuous Delivery

  1. 1. Engineering Tools at Netflix Enabling Continuous Delivery Mike McGarr mmcgarr@netflix.com @SonOfGarr http://www.MikeMcGarr.com
  2. 2. about me • J. Michael (Mike) McGarr • 13 years as a software engineer • Manager of Build Tools, Netflix • Founder, DC Continuous Delivery Meetup • Former Director of EngOps, Blackboard Inc. 10/10/14 @SonOfGarr 2
  3. 3. 10/10/14 @SonOfGarr 3
  4. 4. freedom and responsibility 10/10/14 @SonOfGarr 4
  5. 5. cloud native 10/10/14 @SonOfGarr 5
  6. 6. microservices (SOA) 10/10/14 @SonOfGarr 6
  7. 7. highly aligned, loosely coupled 10/10/14 @SonOfGarr http://bit.ly/1uM62Wi 7
  8. 8. open source 10/10/14 @SonOfGarr 8
  9. 9. continuous delivery “Continuous delivery is a set of practices and principles aimed at, building, testing and releasing software faster and more frequently.” 10/10/14 @SonOfGarr 9
  10. 10. confidence 10/10/14 @SonOfGarr http://bit.ly/1nV4szx 10
  11. 11. teams should focus on delivering business value 10/10/14 @SonOfGarr http://bit.ly/1sb829y 11
  12. 12. Engineering Tools • Simplify build automation • Codify proven practices • Enable self-service • Automate deployments Not an Ops team! 10/10/14 @SonOfGarr 12
  13. 13. Paved Road 10/10/14 @SonOfGarr http://bit.ly/1oIqYGA 13
  14. 14. Paved Road 10/10/14 @SonOfGarr 14
  15. 15. BUILD 10/10/14 http://bit.ly/1vMALQ8 @SonOfGarr 15
  16. 16. Netflix Build Language (NeBuLa) • Gradle plugins • Netflix OSS http://github.com/nebula-plugins 10/10/14 @SonOfGarr 16
  17. 17. build script 10/10/14 @SonOfGarr 17
  18. 18. building OS packages? 10/10/14 @SonOfGarr 18
  19. 19. locking dependencies? 10/10/14 @SonOfGarr 19
  20. 20. generate lock file build.gradle dependencies.lock 10/10/14 @SonOfGarr 20
  21. 21. Jenkins • Continuous integration • ‘nuff said 10/10/14 @SonOfGarr 21
  22. 22. multiple masters Billing Merch OpsEng PPD ProdEng MCE Memb Platform 10/10/14 @SonOfGarr 22
  23. 23. Jenkins Job DSL • Groovy DSL • Define job config in source https://github.com/jenkinsci/job-dsl-plugin 10/10/14 @SonOfGarr 23
  24. 24. jobs.dsl.groovy 10/10/14 @SonOfGarr 24
  25. 25. BAKE 10/10/14 http://bit.ly/1CTzPNO @SonOfGarr 25
  26. 26. immutable servers • Avoid configuration drift • Change a line of code = build a new OS 10/10/14 @SonOfGarr 26
  27. 27. Baking Process 10/10/14 @SonOfGarr 27
  28. 28. Aminator • Python library • Build application AMI • NetflixOSS https://github.com/Netflix/aminator 10/10/14 @SonOfGarr 28
  29. 29. Bakery • REST API’s for Aminator • Single API for all regions • Not OSS 10/10/14 @SonOfGarr 29
  30. 30. DEPLOY 10/10/14 @SonOfGarr http://1.usa.gov/1uBl4wF 30
  31. 31. AWS “applications” 10/10/14 @SonOfGarr 31
  32. 32. Asgard • Netflix AWS console • Provides “app” abstraction • Hides AWS keys • NetflixOSS https://github.com/Netflix/asgard 10/10/14 @SonOfGarr 32
  33. 33. deployments 10/10/14 @SonOfGarr 33
  34. 34. deployments 10/10/14 @SonOfGarr 34
  35. 35. red/black deployment 10/10/14 @SonOfGarr 35
  36. 36. inventing clusters @SonOfGarr 36 10/10/14
  37. 37. canary releases 10/10/14 @SonOfGarr 37
  38. 38. Mimir • Deployment workflows • Integrates Jenkins/Bakery/Asg ard • Prototype, not OSS 10/10/14 @SonOfGarr 38
  39. 39. we are evolving • Unifies Asgard & Mimir • Netflix needs have evolved • Highly available 10/10/14 @SonOfGarr 39
  40. 40. Simian Army • Kill/inspect running instances • Chaos Monkey • Janitor Monkey • Security Monkey • Conformity Monkey • Chaos Gorilla* • Chaos Kong* https://github.com/Netflix/SimianArmy https://github.com/Netflix/security_monkey 10/10/14 @SonOfGarr 40
  41. 41. what I didn’t cover • Ribbon • EvCache • Hystrix • Governator • Karyon • Edda • Archaius • Eureka • Zuul • Data/Big Data tools 10/10/14 @SonOfGarr 41
  42. 42. Zero to the Cloud • Stand-up your own NetflixOSS cloud • Aminator • BaseAMI • Asgard • Edda • Eureka https://github.com/Netflix-Skunkworks/zerotocloud 10/10/14 @SonOfGarr 42
  43. 43. Thank you! • Mike McGarr • mmcgarr@netflix.com • @jMichaelMcGarr • http://www.mikemcgarr.com We are Hiring! netflix.com/jobs 10/10/14 @SonOfGarr 43

×