Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Real-time cloud management with Salt
Unlocked.io Seth House <shouse@saltstack.com>
2013-06-14
1 / 47
Salt philosophy
1 Salt philosophy
Building blocks
What are our building blocks?
Building blocks and complexity
Success com...
Salt philosophy Building blocks
Building blocks
3 / 47
Salt philosophy Building blocks
Building blocks
Everything we have is built on earlier generations
Building blocks of hist...
Salt philosophy What are our building blocks?
What are our building blocks?
Computers Systems
Data centers are the modern ...
Salt philosophy Building blocks and complexity
Building blocks and complexity
As systems grow they become more complex
Inc...
Salt philosophy Success comes from simplicity
Success comes from simplicity
Viable building blocks are fundamentally simpl...
Salt philosophy Salt is aptly named
Salt is aptly named
8 / 47
Salt philosophy Salt is aptly named
Salt is aptly named
The name is the vision
Salt is in nature, a building block
Salt is...
Salt philosophy Flow + state = EVERYTHING
Flow + state = EVERYTHING
Core idea behind Salt
Flow = Access and control
State ...
Salt philosophy Building on state and flow
Building on state and flow
Basics
Remote execution built on Flow
Config Management...
Salt philosophy Building blocks for infra
Building blocks for infra
Salt has “Configuration Management”
Salt has “Remote Ex...
Salt philosophy Building remote ex
Building remote ex
Async Message Bus
Micro publish commands
Minion Side Logic
Return to...
Salt philosophy Building states
Building states
All about data
Low level data structure calls state functions
High level d...
Salt philosophy Building cloud
Building cloud
Remote Ex controls hypers
Remote ex commands hypers
All runs on Live data
15...
Salt philosophy Building monitoring
Building monitoring
Remote Ex build on system libs
Data gathering is already there!
St...
Salt philosophy Conclusion
Conclusion
17 / 47
Salt philosophy Conclusion
Conclusion
Salt is comprised of many building blocks
The underlying components are available, b...
Salt internals (briefly)
1 Salt philosophy
2 Salt internals (briefly)
Master
Minions
Execution modules
Execution example
Sta...
Salt internals (briefly) Master
Master
salt-master -d
Open two ports (pub/sub & reply channel)
20 / 47
Salt internals (briefly) Minions
Minions
salt-minion -d
Connect to the master
No open ports required
Listens for pubs from ...
Salt internals (briefly) Execution modules
Execution modules
Contain all the functionality
22 / 47
Salt internals (briefly) Execution example
Execution example
salt ’web-*’ network.interfaces
23 / 47
Salt internals (briefly) State modules
State modules
Wrap execution modules
Before-check
test=true
Call out to execution mo...
Salt internals (briefly) State module example
State module example
top.sls:
base:
’web-*’:
- httpd
httpd.sls:
httpd:
pkg:
-...
Salt speed
1 Salt philosophy
2 Salt internals (briefly)
3 Salt speed
Communication
pub/sub
4 Minion data
5 Events
6 Schedul...
Salt speed Communication
Communication
ZeroMQ
msgpack
27 / 47
Salt speed pub/sub
pub/sub
Asynchronous
Minions determine targeting match
Minions do all the work
28 / 47
Minion data
1 Salt philosophy
2 Salt internals (briefly)
3 Salt speed
4 Minion data
Peer
Peer example
Salt Mine
Salt Mine e...
Minion data
Live data; peer interface
Recent data; Salt mine
Historical data; returners
30 / 47
Minion data Peer
Peer
/etc/salt/master:
peer:
lb-.*:
- network.interfaces
Be mindful of data security
Communication still ...
Minion data Peer example
Peer example
Configuring haproxy.cfg:
{% for server,ip in
salt[’publish.publish’](
’web*’,
’networ...
Minion data Salt Mine
Salt Mine
/etc/salt/{master,minion}:
mine_functions:
network.interfaces: [eth0]
mine_interval: 60
Ne...
Minion data Salt Mine example
Salt Mine example
Configuring haproxy.cfg:
{% for server,ip in
salt[’mine.get’](
’web-*’,
’ne...
Minion data Returners
Returners
/etc/salt/{master,minion}:
redis.db: 0
redis.host: myredis
redis.port: 6379
Minions write ...
Minion data Returner full-circle example
Returner full-circle example
Collect the data:
salt ’web-*’ network.interfaces et...
Events
1 Salt philosophy
2 Salt internals (briefly)
3 Salt speed
4 Minion data
5 Events
Fire events
Watch for events (manua...
Events Fire events
Fire events
salt ’lb-*’ event.fire_master 
refresh_pool loadbalancer
38 / 47
Events Watch for events (manually)
Watch for events (manually)
Some assembly required
salt/tests/eventlisten.py
Coming soo...
Events Reactor (react to events)
Reactor (react to events)
/etc/salt/master:
reactor:
- loadbalancer:
- /src/reactor/refre...
Schedules
1 Salt philosophy
2 Salt internals (briefly)
3 Salt speed
4 Minion data
5 Events
6 Schedules
Add events
Stats gat...
Schedules Add events
Add events
/etc/salt/{master,minion} (or pillar):
schedule:
highstate:
function: state.highstate
minu...
Schedules Stats gathering
Stats gathering
schedule:
uptime:
function: status.uptime
seconds: 60
returner: redis
meminfo:
f...
What’s coming
1 Salt philosophy
2 Salt internals (briefly)
3 Salt speed
4 Minion data
5 Events
6 Schedules
7 What’s coming
...
What’s coming
Salt v.0.next
45 / 47
What’s coming Monitoring states
Monitoring states
Configure inline with existing states
Individual components are in place
...
What’s coming Data resolution
Data resolution
Time-series data
Thin and/or summarize older and older data
Free with some r...
Upcoming SlideShare
Loading in …5
×

Real-time Cloud Management with SaltStack

24,615 views

Published on

Seth House, SaltStack senior engineer, presented at the first Rackspace Unlocked.io event in New York City the week of Cloud Expo. His presentation titled, "Real-time cloud management with SaltStack" is provided here.

Published in: Technology, Business

Real-time Cloud Management with SaltStack

  1. 1. Real-time cloud management with Salt Unlocked.io Seth House <shouse@saltstack.com> 2013-06-14 1 / 47
  2. 2. Salt philosophy 1 Salt philosophy Building blocks What are our building blocks? Building blocks and complexity Success comes from simplicity Salt is aptly named Flow + state = EVERYTHING Building on state and flow Building blocks for infra Building remote ex Building states Building cloud Building monitoring Conclusion 2 Salt internals (briefly) 2 / 47
  3. 3. Salt philosophy Building blocks Building blocks 3 / 47
  4. 4. Salt philosophy Building blocks Building blocks Everything we have is built on earlier generations Building blocks of history Agriculture Cities Governments Mathematics Manufacturing Language 4 / 47
  5. 5. Salt philosophy What are our building blocks? What are our building blocks? Computers Systems Data centers are the modern platform of invention Large scale computing drives virtually all high level human interactions Data centers themselves are becoming the building blocks 5 / 47
  6. 6. Salt philosophy Building blocks and complexity Building blocks and complexity As systems grow they become more complex Increased complexity makes using larger building blocks impossible Building blocks used today are simple Easy to use Easy to understand Yet powerful 6 / 47
  7. 7. Salt philosophy Success comes from simplicity Success comes from simplicity Viable building blocks are fundamentally simple Things that are in themselves simple to understand and use Concepts that can cross barriers and cleanly apply to many higher level uses YAML is a great example Unix is a great example 7 / 47
  8. 8. Salt philosophy Salt is aptly named Salt is aptly named 8 / 47
  9. 9. Salt philosophy Salt is aptly named Salt is aptly named The name is the vision Salt is in nature, a building block Salt is in cooking, a building block Salt is in life, a building block Yes, this is where the logo came from Yes, this is why it is Salt STACK 9 / 47
  10. 10. Salt philosophy Flow + state = EVERYTHING Flow + state = EVERYTHING Core idea behind Salt Flow = Access and control State = The functional state of the system Between them All things are controlled 10 / 47
  11. 11. Salt philosophy Building on state and flow Building on state and flow Basics Remote execution built on Flow Config Management built on State Ramping Up Monitoring build on Flow + State Cloud management is Flow + State Auto healing is Flow + Flow Orchestration is Flow + State Distributed Deployment is Flow + State 11 / 47
  12. 12. Salt philosophy Building blocks for infra Building blocks for infra Salt has “Configuration Management” Salt has “Remote Execution” 12 / 47
  13. 13. Salt philosophy Building remote ex Building remote ex Async Message Bus Micro publish commands Minion Side Logic Return to arbitrary location 13 / 47
  14. 14. Salt philosophy Building states Building states All about data Low level data structure calls state functions High level data structure defines state calls Languages are now arbitrary 14 / 47
  15. 15. Salt philosophy Building cloud Building cloud Remote Ex controls hypers Remote ex commands hypers All runs on Live data 15 / 47
  16. 16. Salt philosophy Building monitoring Building monitoring Remote Ex build on system libs Data gathering is already there! States used to detect the state of a system States can declare that the state of a system is out of bounds Fires back to the event bus 16 / 47
  17. 17. Salt philosophy Conclusion Conclusion 17 / 47
  18. 18. Salt philosophy Conclusion Conclusion Salt is comprised of many building blocks The underlying components are available, but they do not need to be interfaced with Salt is made to morph into the needs of the infra Made to make small infrastructures easy, and large infrastructures powerful 18 / 47
  19. 19. Salt internals (briefly) 1 Salt philosophy 2 Salt internals (briefly) Master Minions Execution modules Execution example State modules State module example 3 Salt speed 4 Minion data 5 Events 19 / 47
  20. 20. Salt internals (briefly) Master Master salt-master -d Open two ports (pub/sub & reply channel) 20 / 47
  21. 21. Salt internals (briefly) Minions Minions salt-minion -d Connect to the master No open ports required Listens for pubs from the master /etc/salt/minion: #master: salt 21 / 47
  22. 22. Salt internals (briefly) Execution modules Execution modules Contain all the functionality 22 / 47
  23. 23. Salt internals (briefly) Execution example Execution example salt ’web-*’ network.interfaces 23 / 47
  24. 24. Salt internals (briefly) State modules State modules Wrap execution modules Before-check test=true Call out to execution modules After-check 24 / 47
  25. 25. Salt internals (briefly) State module example State module example top.sls: base: ’web-*’: - httpd httpd.sls: httpd: pkg: - installed 25 / 47
  26. 26. Salt speed 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed Communication pub/sub 4 Minion data 5 Events 6 Schedules 7 What’s coming 26 / 47
  27. 27. Salt speed Communication Communication ZeroMQ msgpack 27 / 47
  28. 28. Salt speed pub/sub pub/sub Asynchronous Minions determine targeting match Minions do all the work 28 / 47
  29. 29. Minion data 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data Peer Peer example Salt Mine Salt Mine example Returners Returner full-circle example 5 Events 29 / 47
  30. 30. Minion data Live data; peer interface Recent data; Salt mine Historical data; returners 30 / 47
  31. 31. Minion data Peer Peer /etc/salt/master: peer: lb-.*: - network.interfaces Be mindful of data security Communication still goes through the master 31 / 47
  32. 32. Minion data Peer example Peer example Configuring haproxy.cfg: {% for server,ip in salt[’publish.publish’]( ’web*’, ’network.interfaces’, [’eth0’]).items() %} server {{ server }} {{ ip[0] }}:80 check {% endfor %} 32 / 47
  33. 33. Minion data Salt Mine Salt Mine /etc/salt/{master,minion}: mine_functions: network.interfaces: [eth0] mine_interval: 60 New in Salt v0.15 Either master or minion config Be mindful of data security 33 / 47
  34. 34. Minion data Salt Mine example Salt Mine example Configuring haproxy.cfg: {% for server,ip in salt[’mine.get’]( ’web-*’, ’network.interfaces’, [’eth0’]).items() %} server {{ server }} {{ ip[0] }}:80 check {% endfor %} 34 / 47
  35. 35. Minion data Returners Returners /etc/salt/{master,minion}: redis.db: 0 redis.host: myredis redis.port: 6379 Minions write directly Can be read into Pillar via ext_pillar 35 / 47
  36. 36. Minion data Returner full-circle example Returner full-circle example Collect the data: salt ’web-*’ network.interfaces eth0 --return redis_return Fetch the data via a custom ext_pillar module. Use the data: {% for server,ip in salt[’pillar.get’](’web.ip_addrs’, {}).items() server {{ server }} {{ ip[0] }}:80 check {% endfor %} 36 / 47
  37. 37. Events 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data 5 Events Fire events Watch for events (manually) Reactor (react to events) 6 Schedules 7 What’s coming 37 / 47
  38. 38. Events Fire events Fire events salt ’lb-*’ event.fire_master refresh_pool loadbalancer 38 / 47
  39. 39. Events Watch for events (manually) Watch for events (manually) Some assembly required salt/tests/eventlisten.py Coming soon to salt-api 39 / 47
  40. 40. Events Reactor (react to events) Reactor (react to events) /etc/salt/master: reactor: - loadbalancer: - /src/reactor/refresh_pool.sls /src/reactor/refresh_pool.sls: {% if data[’type’] == ’refresh_pool’ %} highstate_run: cmd.state.highstate: - tgt: lb-* {% endif %} 40 / 47
  41. 41. Schedules 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data 5 Events 6 Schedules Add events Stats gathering 7 What’s coming 41 / 47
  42. 42. Schedules Add events Add events /etc/salt/{master,minion} (or pillar): schedule: highstate: function: state.highstate minutes: 60 42 / 47
  43. 43. Schedules Stats gathering Stats gathering schedule: uptime: function: status.uptime seconds: 60 returner: redis meminfo: function: status.meminfo minutes: 5 returner: redis 43 / 47
  44. 44. What’s coming 1 Salt philosophy 2 Salt internals (briefly) 3 Salt speed 4 Minion data 5 Events 6 Schedules 7 What’s coming Monitoring states Data resolution 44 / 47
  45. 45. What’s coming Salt v.0.next 45 / 47
  46. 46. What’s coming Monitoring states Monitoring states Configure inline with existing states Individual components are in place Needed: glue Needed: alerting 46 / 47
  47. 47. What’s coming Data resolution Data resolution Time-series data Thin and/or summarize older and older data Free with some returners 47 / 47

×