Mitchell Hashimoto, HashiCorp

10,300 views

Published on

HighLoad++ 2013

Published in: Technology

Mitchell Hashimoto, HashiCorp

  1. 1. Towards FutureOps: Stable, repeatable, environments from dev to prod.
  2. 2. I’m Mitchell Hashimoto Also known as @mitchellh
  3. 3. I build tools http://hashicorp.com
  4. 4. Vagrant http://www.vagrantup.com Packer http://www.packer.io SERF http://www.serfdom.io
  5. 5. History of Ops Or, the history of pretending that anything works.
  6. 6. Single Dedicated Server
  7. 7. Single Dedicated Server • Manually setup
  8. 8. Single Dedicated Server • • Manually setup Expensive to replace (time and money)
  9. 9. Single Dedicated Server • • • Manually setup Expensive to replace (time and money) Professionals, usually
  10. 10. What is hard/slow/expensive? • • • • Acquiring hardware Manual setup Expertise Downtime, Failures
  11. 11. Multiple Dedicated Servers
  12. 12. Multiple Dedicated Servers • Golden images
  13. 13. Multiple Dedicated Servers • • Golden images Manually made
  14. 14. Multiple Dedicated Servers • • • Golden images Manually made Slow to change, expensive to grow
  15. 15. Multiple Dedicated Servers • Eventually: config management (CFEngine, Puppet, Chef)
  16. 16. What is hard/slow/expensive? • • • • Acquiring hardware Making images Expertise Downtime, Failures
  17. 17. The Low-Cost VPS Server
  18. 18. The VPS Server • Manually made (cargo cult, usually)
  19. 19. The VPS Server • • Manually made (cargo cult, usually) Cheaper to replace
  20. 20. The VPS Server • • • Manually made (cargo cult, usually) Cheaper to replace Hobbyist, startup
  21. 21. What is hard/slow/expensive? • • • Expertise Downtime, Failures Rebuilding it
  22. 22. The Cloud
  23. 23. The Cloud • Many cheap servers
  24. 24. The Cloud • • Many cheap servers Seconds to create
  25. 25. The Cloud • • • Many cheap servers Seconds to create Configuration management
  26. 26. What is hard/slow/expensive? • • • • Configuration mgmt Portability Repeatability Can be expensive
  27. 27. The Ops of Tomorrow Better, faster, stronger
  28. 28. Immutable Infrastructure
  29. 29. Immutable Infrastructure Instead of modifying or maintaining a server, replace it with a new one.
  30. 30. Immutable Infrastructure Servers don’t change anymore, they’re pre-built, static images that are tested/verified.
  31. 31. Immutable Wins • • • Super fast server deployment Failure expected, not a big deal Testable, more stable
  32. 32. Automation Critical • • • • Machine Images Software install/configure Failure detection, recovery etc…
  33. 33. Machine Images Easily build machine images for multiple platforms with Packer.
  34. 34. Machine image: A single deployable unit that contains a pre-configured OS and software.
  35. 35. Ew... Images. The anti-image stigma
  36. 36. DevOps has historically been against machine images.
  37. 37. Important to understand why DevOps has historically been against machine images.
  38. 38. Golden images used to be the way.
  39. 39. Quarterly, unchanged, blessed image.
  40. 40. Getting any changes (ops or dev) in was slow and frustrating.
  41. 41. Necessary evil while tooling wasn’t as good as it is today.
  42. 42. With modern config management, easy to simply ignore images.
  43. 43. Machine images also have a ton of benefits.
  44. 44. Super fast infrastructure deployment.
  45. 45. Multi-cloud portability
  46. 46. Stability and Testability
  47. 47. Analogy: Ops without machine images is like applications without binaries
  48. 48. Put it another way: There is no promise of reproducible builds in ops, and no binary-like snapshots
  49. 49. Source Code Binary
  50. 50. Source Code libA 1.0 Binary libB 1.0 libC 1.0
  51. 51. Source Code libA 2.0 Binary libB 1.0 libC 1.0
  52. 52. Source Code libA 2.0 COMPILE FAILED libB 1.0 libC 1.0
  53. 53. Chef, Puppet, Scripts, etc. New Server Ready Server
  54. 54. Chef, Puppet, Scripts, etc. New Server Packages Ready Server Network Chef/Puppet Changes
  55. 55. Chef, Puppet, Scripts, etc. SERVER SETUP FAILED New Server Package Changes Network Chef/Puppet Changes
  56. 56. Chef, Puppet, Scripts, etc. SERVER SETUP FAILED New Server Packages Network Unreliable Chef/Puppet Changes
  57. 57. Chef, Puppet, Scripts, etc. SERVER SETUP FAILED New Server Packages Network Chef/Puppet Changed
  58. 58. Machine Image New Server Ready Server
  59. 59. Machine Image New Server Ready Server
  60. 60. Packer Embraces modern best practices and automates building images
  61. 61. Takes template input and repeatably produces image output.
  62. 62. $ packer build ! -var ‘aws_access_key=YOUR KEY’ ! -var ‘aws_secret_key=YOUR SECRET’ ! template.json! ...
  63. 63. Automated, repeatable, fast, uses Chef/Puppet/ Shell/etc. Gets rid of old downsides.
  64. 64. Can also produce Vagrant boxes for development, at the same time.
  65. 65. Workflows What does this look like in practice?
  66. 66. Ops Ops Push Packer Build Machine Image(s)
  67. 67. Ops Ops Push Packer Build Machine Image(s) Setup users, install software, configure software, configure services. No orchestration.
  68. 68. Server Deploy Request Server Running Server
  69. 69. Server Deploy • Server started with Packerbuilt image • Running server has all preinstalled software • Service orchestration runs now Request Server Running Server
  70. 70. Development “vagrant up” Ready to work
  71. 71. Development • Vagrant downloads the latest machine image created by Packer • “vagrant up” takes seconds to minutes and runs, nearly identical to production “vagrant up” Ready to work
  72. 72. Packer Wins • • • • Automated, repeatable images Fast deployment from dev to prod Testable, very stable Embraces configuration mgmt
  73. 73. SERF Service orchestration, discovery, failure detection.
  74. 74. Remember, Chef/Puppet/etc. aren’t doing service orchestration for the image.
  75. 75. Service Discovery: Web servers in a load balancer, Memcache in a cluster, MySQL master/slave, etc.
  76. 76. Service Orchestration: Add this node to your cluster, deploy this application, restart, etc.
  77. 77. Serf is a decentralized, highly available, fault tolerant, and lightweight solution to service orchestration and discovery.
  78. 78. Example Web servers in a load balancer
  79. 79. Run the Agent $ serf agent ! ! -event-handler=“handle.sh”! …
  80. 80. Join a Cluster $ serf join 10.0.1.5! …
  81. 81. Serf Events Serf Agent Handle.sh Add or Remove Node from LB
  82. 82. Serf Events Serf Agent handle.sh Add or Remove Node from LB Serf invokes event handlers for any event: member-join, member-leave, member-failed, user.
  83. 83. How? How does Serf do what it does?
  84. 84. GOSSIP-BASED MEMBERSHIP
  85. 85. FAILURE DETECTION
  86. 86. CUSTOM EVENTS
  87. 87. Config Management Chef/Puppet Run Configure Serf Agent Services Don’t start Serf. Serf starts on boot.
  88. 88. Config Management Chef/Puppet Run Configure Serf agent init script Don’t start Serf. Serf starts on boot. Chef and Puppet just configure the Serf init script to start the agent.
  89. 89. Serf Wins • • • No orchestration in image build Fast membership detection, don’t wait for Chef/Puppet Easily and infinitely extensible
  90. 90. The Future is Immutable Big wins, little downsides.
  91. 91. As servers become cheaper, more ondemand, and infrastructures become more distributed, immutable infrastructure will win.
  92. 92. Important to remember: You don’t have to go all in on immutable. You can start with certain servers. Example: databases are hard, just ignore that for awhile.
  93. 93. Mostly Immutable There are still big benefits for going mostly immutable. Things like quick config changes and deploys can still be mutable. Serf is good for this.
  94. 94. The Wins • • • • • Super fast deployment Repeatability High failure tolerance Improved stability and testability Versioning
  95. 95. The Downsides • • • Requires proper mindset Tooling is new Deploys, simple config changes
  96. 96. Thank you!
  97. 97. Vagrant http://www.vagrantup.com Packer http://www.packer.io SERF http://www.serfdom.io

×