Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intelligent infrastructure with SaltStack

934 views

Published on

Event > React > Orchestrate

Published in: Technology
  • Be the first to comment

Intelligent infrastructure with SaltStack

  1. 1. Intelligent Infrastructure with SaltStack Love Billingskog Nyberg Infrastructure and Opera=ons Team Lead Event > React > Orchestrate
  2. 2. Hi! Love @jacksoncage
  3. 3. bloglovin tens of millions of monthly unique visitors hundred of millions requests per day couple of million stories parsed per day hybrid bare-metal servers/cloud vm’s 4 salt masters ~90 salt minions
  4. 4. salt @ bloglovin Parts we use! States/Pillars/Grains Pillarstack* Orchestrate Reactor Beacons Engines
  5. 5. Parallel execu=on to run all the resources concurrently Event driven to monitor and react dynamically only to changes Next genera=on
  6. 6. Next genera=on
  7. 7. Parallel execu=on root@salt-dev:~# salt '*' test.ping web101: True db101: True ship101: True master: True Easy to target remote execu5on against mul5ple targets
  8. 8. Parallel execu=on Run commands via execu5on modules root@salt-dev:~# salt web* service.stop nginx web101: True
  9. 9. Tradi=onal configura=on Se;ng state on interval or on run
  10. 10. Event driven configura=on Se;ng state when needed or change has happened
  11. 11. Goals Monitoring - How can salt react to monitoring events and adjust system without any manual interference
 Minions - When a new device is connected get it’s data and no=fy me, chat Vw!
 Containers - When a new container starts, update load balancer for that cluster and start sending traffic. Also when a container stops, get logs and restart and send logs to team
  12. 12. Events salt/job/20160413070418586332/new { "_stamp": "2016-04-13T11:04:18.587129", "arg": [], "fun": "test.ping", "jid": "20160413070418586332", "minions": [ "web101" ], "tgt": "web101", "tgt_type": "glob", "user": "sudo_love" } salt-run state.event pretty=True
  13. 13. Events salt/job/20160413070418586332/ret/web101 { "_stamp": "2016-04-13T11:04:18.789831", "cmd": "_return", "fun": "test.ping", "fun_args": [], "id": "web101", "jid": "20160413070418586332", "retcode": 0, "return": true, "success": true }
  14. 14. State events Enabling state events causes the Salt minion to send progress events as each func5on in a state run completes execu5on. state_events: True minion
  15. 15. State events Which can become an over load, let’s fire only on specific states installed/vim: pkg.installed: - name: vim - fire_event: installed/vim minion
  16. 16. Custom event Enabling state events causes the Salt minion to send progress events as each func5on in a state run completes execu5on. salt-call event.send /deploy/test/case '{"data": "Ready"}' /deploy/test/case { "_stamp": "2016-04-13T11:01:24.553140", "cmd": "_minion_event", "data": { "__pub_fun": "event.send", "__pub_jid": "20160413070124500641", "__pub_pid": 17046, "__pub_tgt": "salt-call", "data": "Ready" … minion
  17. 17. Beacons beacons: service: nginx: onchangeonly: True Hook into system processes. Translate external events into the salt event bus minion
  18. 18. Engines engines: docker_events: docker_url: unix://var/run/docker.sock Salt Engines are long-running, external system processes that leverage Salt. minion
  19. 19. Great! Events for everything!
  20. 20. Reactor Salt’s Reactor system gives you the ability to trigger ac5ons in response to any event. reactor: # Master config section "reactor" - 'salt/minion/*/start': # Match tag "salt/minion/*/start" - /srv/reactor/start.sls # Things to do when a minion starts - 'deploy/containers': # Match tag "deploy/containers" - /srv/reactor/deploy-containers.sls # Things to do when a container is deployed - 'salt/netapi/hook/api/devlovin/create': # Listing on salt api for /devlovin/create - /srv/reactor/devlovin-create.sls # Things to do when a create request is sent master
  21. 21. Reactor Salt’s Reactor system gives you the ability to trigger ac5ons in response to any event. Remote execu=on: run an execu=on module on the targeted minions. Runners: These are tasks you would start using salt-run. Wheel: Wheel commands manage your Salt environment. master
  22. 22. Reactor Salt’s Reactor system gives you the ability to trigger ac5ons in response to any event. <section id>: local.<function>: - tgt: <target> - arg: <arguments> master
  23. 23. # This script is to be called from the reactor system {%- set application = data['data']['application'] %} {%- set container = data['data']['container'] %} {%- set target = data['data']['target'] %} {%- set tag = data['data']['tag'] %} sync-states: local.saltutil.sync_states: - tgt: {{ target }} deploy-containers: local.state.sls: - tgt: {{ target }} - arg: - containers.{{ container }} - kwarg: pillar: containers: imagetag: {{ application }}: {{ tag }} master
  24. 24. highstate_run: local.state.highstate: - tgt: db101 call_some_orchestrate_file: runner.state.orchestrate: - mods: some_orchestrate_file - pillar: stuff: things Highstate run Star5ng a orchestrator runner master
  25. 25. Orchestrate The orchestrate runner generalizes the Salt state system to a Salt master context. master
  26. 26. # Create AWS EC2 instance create-instance: salt.state: - tgt: {{ master }} - sls: devlovin.cloud.create - concurrent: True - pillar: devlovinreact: instancename: {{ basename }}{{ username }} # Run highstate highstate-run: salt.state: - tgt: {{ basename }}{{ username }} - highstate: True - pillar: devlovinreact: username: {{ username }} - require: - salt: create-instance master
  27. 27. Demo Monitoring - How can salt react to monitoring events and adjust system without any manual interference
  28. 28. Demo Minions - When a new device is connected get it’s data and no=fy me, chat Vw!
  29. 29. Demo Containers - When a new container starts, update load balancer for that cluster and start sending traffic. Also when a container stops, get logs and restart and send logs to team
  30. 30. Ques=ons? Love Billingskog Nyberg @jacksoncage love@bloglovin.com
  31. 31. Thanks! Love Billingskog Nyberg @jacksoncage love@bloglovin.com Use the SaltConf16 event app to provide feedback

×