Real-time Cloud Management with SaltStack

23,830 views
23,436 views

Published on

Seth House, SaltStack senior engineer, presented at the first Rackspace Unlocked.io event in New York City the week of Cloud Expo. His presentation titled, "Real-time cloud management with SaltStack" is provided here.

Published in: Technology, Business

Real-time Cloud Management with SaltStack

  1. 1. Real-time cloud management with Salt Unlocked.io Seth House <shouse@saltstack.com> 2013-06-14 1 / 47
  2. 2. Salt philosophy 1 Salt philosophy Building blocks What are our building blocks? Building blocks and complexity Success comes from simplicity Salt is aptly named Flow + state = EVERYTHING Building on state and flow Building blocks for infra Building remote ex Building states Building cloud Building monitoring Conclusion 2 Salt internals (briefly) 2 / 47
  3. 3. Salt philosophy Building blocks Building blocks 3 / 47
  4. 4. Salt philosophy Building blocks Building blocks Everything we have is built on earlier generations Building blocks of history Agriculture Cities Governments Mathematics Manufacturing Language 4 / 47
  5. 5. Salt philosophy What are our building blocks? What are our building blocks? Computers Systems Data centers are the modern platform of invention Large scale computing drives virtually all high level human interactions Data centers themselves are becoming the building blocks 5 / 47
  6. 6. Salt philosophy Building blocks and complexity Building blocks and complexity As systems grow they become more complex Increased complexity makes using larger building blocks impossible Building blocks used today are simple Easy to use Easy to understand Yet powerful 6 / 47
  7. 7. Salt philosophy Success comes from simplicity Success comes from simplicity Viable building blocks are fundamentally simple Things that are in themselves simple to understand and use Concepts that can cross barriers and cleanly apply to many higher level uses YAML is a great example Unix is a great example 7 / 47
  8. 8. Salt philosophy Salt is aptly named Salt is aptly named 8 / 47
  9. 9. Salt philosophy Salt is aptly named Salt is aptly named The name is the vision Salt is in nature, a building block Salt is in cooking, a building block Salt is in life, a building block Yes, this is where the logo came from Yes, this is why it is Salt STACK 9 / 47
  10. 10. Salt philosophy Flow + state = EVERYTHING Flow + state = EVERYTHING Core idea behind Salt Flow = Access and control State = The functional state of the system Between them All things are controlled 10 / 47
  11. 11. Salt philosophy Building on state and flow Building on state and flow Basics Remote execution built on Flow Config Management built on State Ramping Up Monitoring build on Flow + State Cloud management is Flow + State Auto healing is Flow + Flow Orchestration is Flow + State Distributed Deployment is Flow + State 11 / 47
  12. 12. Salt philosophy Building blocks for infra Building blocks for infra Salt has “Configuration Management” Salt has “Remote Execution” 12 / 47
  13. 13. Salt philosophy Building remote ex Building remote ex Async Message Bus Micro publish commands Minion Side Logic Return to arbitrary location 13 / 47
  14. 14. Salt philosophy Building states Building states All about data Low level data structure calls state functions High level data structure defines state calls Languages are now arbitrary 14 / 47
  15. 15. Salt philosophy Building cloud Building cloud Remote Ex controls hypers Remote ex commands hypers All runs on Live data 15 / 47
  16. 16. Salt philosophy Building monitoring Building monitoring Remote Ex build on system libs Data gathering is already there! States used to detect the state of a system States can declare that the state of a system is out of bounds Fires back to the event bus 16 / 47
  17. 17. Salt philosophy Conclusion Conclusion 17 / 47
  18. 18. Salt philosophy Conclusion Conclusion Salt is comprised of many building blocks The underlying components are available, but they do not need to be interfaced with Salt is made to morph into the needs of the infra Made to make small infrastructures easy, and large infrastructures powerful 18 / 47
  19. 19. Salt internals (briefly) 1 Salt philosophy 2 Salt internals (briefly) Master Minions Execution modules Execution example State modules State module example 3 Salt speed 4 Minion data 5 Events 19 / 47
  20. 20. Salt internals (briefly) Master Master salt-master -d Open two ports (pub/sub & reply channel) 20 / 47
  21. 21. Salt internals (briefly) Minions Minions salt-minion -d Connect to the master No open ports required Listens for pubs from the master /etc/salt/minion: #master: salt 21 / 47
  22. 22. Salt internals (briefly) Execution modules Execution modules Contain all the functionality 22 / 47
  23. 23. Salt internals (briefly) Execution example Execution example salt ’web-*’ network.interfaces 23 / 47
  24. 24. Salt internals (briefly) State modules State modules Wrap execution modules Before-check test=true Call out to execution modules After-check 24 / 47
  25. 25. Salt internals (briefly) State module example State module example top.sls: base: ’web-*’: - httpd httpd.sls: httpd: pkg: - installed 25 / 47
  26. 26. Salt speed 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed Communication pub/sub 4 Minion data 5 Events 6 Schedules 7 What’s coming 26 / 47
  27. 27. Salt speed Communication Communication ZeroMQ msgpack 27 / 47
  28. 28. Salt speed pub/sub pub/sub Asynchronous Minions determine targeting match Minions do all the work 28 / 47
  29. 29. Minion data 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data Peer Peer example Salt Mine Salt Mine example Returners Returner full-circle example 5 Events 29 / 47
  30. 30. Minion data Live data; peer interface Recent data; Salt mine Historical data; returners 30 / 47
  31. 31. Minion data Peer Peer /etc/salt/master: peer: lb-.*: - network.interfaces Be mindful of data security Communication still goes through the master 31 / 47
  32. 32. Minion data Peer example Peer example Configuring haproxy.cfg: {% for server,ip in salt[’publish.publish’]( ’web*’, ’network.interfaces’, [’eth0’]).items() %} server {{ server }} {{ ip[0] }}:80 check {% endfor %} 32 / 47
  33. 33. Minion data Salt Mine Salt Mine /etc/salt/{master,minion}: mine_functions: network.interfaces: [eth0] mine_interval: 60 New in Salt v0.15 Either master or minion config Be mindful of data security 33 / 47
  34. 34. Minion data Salt Mine example Salt Mine example Configuring haproxy.cfg: {% for server,ip in salt[’mine.get’]( ’web-*’, ’network.interfaces’, [’eth0’]).items() %} server {{ server }} {{ ip[0] }}:80 check {% endfor %} 34 / 47
  35. 35. Minion data Returners Returners /etc/salt/{master,minion}: redis.db: 0 redis.host: myredis redis.port: 6379 Minions write directly Can be read into Pillar via ext_pillar 35 / 47
  36. 36. Minion data Returner full-circle example Returner full-circle example Collect the data: salt ’web-*’ network.interfaces eth0 --return redis_return Fetch the data via a custom ext_pillar module. Use the data: {% for server,ip in salt[’pillar.get’](’web.ip_addrs’, {}).items() server {{ server }} {{ ip[0] }}:80 check {% endfor %} 36 / 47
  37. 37. Events 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data 5 Events Fire events Watch for events (manually) Reactor (react to events) 6 Schedules 7 What’s coming 37 / 47
  38. 38. Events Fire events Fire events salt ’lb-*’ event.fire_master refresh_pool loadbalancer 38 / 47
  39. 39. Events Watch for events (manually) Watch for events (manually) Some assembly required salt/tests/eventlisten.py Coming soon to salt-api 39 / 47
  40. 40. Events Reactor (react to events) Reactor (react to events) /etc/salt/master: reactor: - loadbalancer: - /src/reactor/refresh_pool.sls /src/reactor/refresh_pool.sls: {% if data[’type’] == ’refresh_pool’ %} highstate_run: cmd.state.highstate: - tgt: lb-* {% endif %} 40 / 47
  41. 41. Schedules 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data 5 Events 6 Schedules Add events Stats gathering 7 What’s coming 41 / 47
  42. 42. Schedules Add events Add events /etc/salt/{master,minion} (or pillar): schedule: highstate: function: state.highstate minutes: 60 42 / 47
  43. 43. Schedules Stats gathering Stats gathering schedule: uptime: function: status.uptime seconds: 60 returner: redis meminfo: function: status.meminfo minutes: 5 returner: redis 43 / 47
  44. 44. What’s coming 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data 5 Events 6 Schedules 7 What’s coming Monitoring states Data resolution 44 / 47
  45. 45. What’s coming Salt v.0.next 45 / 47
  46. 46. What’s coming Monitoring states Monitoring states Configure inline with existing states Individual components are in place Needed: glue Needed: alerting 46 / 47
  47. 47. What’s coming Data resolution Data resolution Time-series data Thin and/or summarize older and older data Free with some returners 47 / 47

×