Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Root conf2017 Configuration Management Patterns

WIP

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Root conf2017 Configuration Management Patterns

  1. 1. CONFIGURATION MANAGEMENT: PATTERNS, PRACTICES & SOLUTIONS A technology neutral view of Ansible/Salt/Chef/Puppet RootConf 2017 Bangalore
  2. 2. TABLE OF CONFIGURATIONS (PUN INTENDED).... • Evolution of Configuration Management • Roles • Classification mechanism • Infrastructure Data • Data which tells a little bit about infrastructure • Configuration Data • Data which we build and use to • Templates • Everything scaled with templating • How do I deploy and test infrastructure code? • That was easy with application code.. • Multi Data Center problems • Dreams of when I grow web scale • Secrets, events and execution patterns • Passwords & failure events
  3. 3. VISHAL BIYANI CTO & Founder at infraCloud technologies (www.infracloud.io ) 2004 Java, PLM, JSP, Servlets, Java Swing, 2004-2009 eMatrix, PLM, J2EE, Database, architecture, Shell, Windchill, jUnit, Oracle, 2010 - 2013: Spring, Maven, Jenkins, ElasticSearch, CloudFoundry, Google App Engine, APIs, CI 2013: Puppet, Chef, Ansible, CD/CI, DevOps Coach, Docker, API Mgmt, Microservices, Infra as code, Artifactory, Nexuus, Integration as a Service Now: Containers, Kubernetes, Mesos, Salt, Scale, Distributed, Serverless https://twitter.com/vishal_biyani https://www.vishalbiyani.com
  4. 4. WE USED TO MANAGE THIS
  5. 5. AND NOW IT IS … … WEB SCALE
  6. 6. SO WERE BORN THESE
  7. 7. A WHOLE NEW VOCABULARY Term Puppet Chef Ansible Salt The smallest packaging unit Module Cookbook Playbook Formula Node side data Facts Attributes (From Ohai) Facts Grains Configuration data Configuratiom data databag Host vars and Group Vars Pillar Role Class Run Lists Role Grain: Roles The master node Puppet Master Chef Server Control Server Salt Master Agent name Puppet Agent Chef Client NA (SSH based) Salt Minion Built in modules Resources Module State/Module Templating Engine Ruby like ERB (Embedded Ruby) Jinja Jinja Secrets Using Databag Databag Vault Orchestration Orchestrate Events Report Processor Beacons, Reactor Community Modules Puppet Forge Chef Supermarket Galaxy Github
  8. 8. AND SIMILAR PROBLEMS, IN ALL STACKS • How do I best split roles? • How to organize, version control & secure the configuration data? • How can I test changes in infra code before deploying to prod? • Can I re-create the whole infrastructure stack for testing? • Should I execute my playbooks every hour?
  9. 9. ROLES - INTRODUCTION • Roles are “labels” we give to nodes - and they can be multiple. Primarily a classification mechanism • They are composed of multiple playbooks/modules/formulas • Think of role as responsibilities on that server - a “web server” role, a “firewall rules” role • Enable you to query infrastructure: Give me all web servers which are running Ubuntu 14.04 or CentOS 6
  10. 10. ROLES - BEST PRACTICES • Keep them small & modular. You can always compose smaller once to build larger functionality • They should enable you to slice & dice your infrastructure easily • Some roles will be used by clustered apps (Kafka, Elastic) - design for that Operating System Ntp role Firewall role Mesos Agent Role Operating System Base Role (Firewall + NTP) Mesos Agent Role
  11. 11. INFRA DATA - WHAT IS IT? • Data which the CM systems can derive from machine by themselves • Might contain some attributed assigned by humans - like role • Is mostly system data - also helpful for classification • For ex. You can get info of OS, Kernel version, Region, services list etc.
  12. 12. INFRA DATA - BEST PRACTICES • As much you can - use infra data instead of configuration data in code for classification • Updating of infra data can be optimized based on churn rate of servers
  13. 13. CONFIG DATA • Data which infrastructure engineers define and use in code but stored separate from data • Typically stored in a hierarchy/tree structure • Can use infra data, roles etc. as variabled to build config data dynamically
  14. 14. CONFIG DATA - BEST PRACTICES • Version control the configuration data - it can grow real fast. • Secrets in configuration data need to be handled separately (We will cover shortly) • Structure configuration data to tame complexity over time
  15. 15. INFRA & CONFIG DATA - SAMPLES • TBD
  16. 16. TEMPLATES - THE SCALING FACTOR • Everything - from code to config data will use templates. • Templates enable programming constructs - loops, decisions, variables etc. • Puppet & Chef are primarily ruby based, Ansible & Salt use Jinja/Python heavily • Templates/code allow querying infra, config data and infra data - which makes them super powerful.
  17. 17. TESTING • Unit test: Simulate with Vagrant/Containers • How to test Docker/Mesos playbooks? • Integration test: How to get as close to real infrastructure as you can? • Can you really test all practical scenarios? • Test frameworks: TestInfra, ServerSpec, Test Kitchen etc
  18. 18. DEPLOYMENT • Deployment is easy - but execution of playbooks needs watching • Usual tools can be used for deploy (Jenkins/Bamboo etc.) • Should deploy many changes at once, or smaller once at a time? •
  19. 19. SECRETS • Some CM systems offer in built secrets management (Ansible has Vault) • For others best to use something like Hashicorp Vault • Secrets are refered in Config data but actual details reside in external vault
  20. 20. MULTI DC • One master per DC or one master for multiple DCs? • Latency between DCs for execution • Code & Config data - central or per DC? Pull from Git or Rsync from super master?
  21. 21. EVENTS & ORCHESTRATION • Events need action/reporting etc. • Events can be too verbose depending on configuration • Orchestration in infrastructure is harder to build & debug
  22. 22. ধন্যবাদ! Dank je! Kiitos! આભાર! धन्यवाद! Grazie! Je vous remercie! ありがとうございました! ਤੁਹਾਡਾ ਧੰਨਵਾਦ! நன்றி! ధన్యవాదాలు! നന്ദി! THANK YOU! Thanks to HasGeek for doing all they they do for us! Thanks a lot to you the audience who made this possible

×