Real-time Infrastructure Management with SaltStack - OpenWest 2013

5,268 views

Published on

SaltStack is fast, among other things. At it's core, SaltStack is a real-time infrastructure management tool that utilizes a high-speed communication channel for remote execution and configuration management. Proper systems administration is not possible without both. This presentation will demonstrate how to create real-time interaction between your code and your infrastructure using SaltStack.

Published in: Technology

Real-time Infrastructure Management with SaltStack - OpenWest 2013

  1. 1. Real-time infrastructure management with SaltSeth House <seth@eseth.com>OpenWest Conference 20132013-05-03Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 1 / 34
  2. 2. Salt internals (briefly)Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 2 / 34
  3. 3. Salt internals (briefly)Outline1 Salt internals (briefly)2 Salt speed3 Minion data4 Events5 Schedules6 What’s comingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 2 / 34
  4. 4. Salt internals (briefly)Salt internalsSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 3 / 34
  5. 5. Salt internals (briefly)Mastersalt-master -dOpen two ports (pub/sub & reply channel)Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 4 / 34
  6. 6. Salt internals (briefly)Minionssalt-minion -dConnect to the masterNo open ports requiredListens for pubs from the master/etc/salt/minion:#master: saltSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 5 / 34
  7. 7. Salt internals (briefly)Execution modulesContain all the functionalitySeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 6 / 34
  8. 8. Salt internals (briefly)Execution examplesalt ’web-*’ network.interfacesSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 7 / 34
  9. 9. Salt internals (briefly)State modulesWrap execution modulesBefore-checktest=trueCall out to execution modulesAfter-checkSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 8 / 34
  10. 10. Salt internals (briefly)State module exampletop.sls:base:’web-*’:- httpdhttpd.sls:httpd:pkg:- installedSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 9 / 34
  11. 11. Salt speedOutline1 Salt internals (briefly)2 Salt speed3 Minion data4 Events5 Schedules6 What’s comingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 10 / 34
  12. 12. Salt speedWhy is Salt fast?Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 11 / 34
  13. 13. Salt speedCommunicationZeroMQmsgpackSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 12 / 34
  14. 14. Salt speedpub/subAsynchronousMinions determine targeting matchMinions do all the workSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 13 / 34
  15. 15. Minion dataOutline1 Salt internals (briefly)2 Salt speed3 Minion data4 Events5 Schedules6 What’s comingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 14 / 34
  16. 16. Minion dataSharing minion data<-- Live -- Recent -- Historic -->peer interface Salt Mine ReturnersSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 15 / 34
  17. 17. Minion dataPeer/etc/salt/master:peer:lb-.*:- network.interfacesBe mindful of data securityCommunication still goes through the masterSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 16 / 34
  18. 18. Minion dataPeer exampleConfiguring haproxy.cfg:{% for server,ip insalt[’publish.publish’](’web*’,’network.interfaces’,[’eth0’]).items() %}server {{ server }} {{ ip[0] }}:80 check{% endfor %}Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 17 / 34
  19. 19. Minion dataSalt Mine/etc/salt/{master,minion}:mine_functions:network.interfaces: [eth0]mine_interval: 60New in Salt v0.15Either master or minion configBe mindful of data securitySeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 18 / 34
  20. 20. Minion dataSalt Mine exampleConfiguring haproxy.cfg:{% for server,ip insalt[’mine.get’](’web-*’,’network.interfaces’,[’eth0’]).items() %}server {{ server }} {{ ip[0] }}:80 check{% endfor %}Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 19 / 34
  21. 21. Minion dataReturners/etc/salt/{master,minion}:redis.db: 0redis.host: myredisredis.port: 6379Minions write directlyCan be read into Pillar via ext_pillarSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 20 / 34
  22. 22. Minion dataReturner full-circle exampleCollect the data:salt ’web-*’ network.interfaces eth0 --return redis_returnFetch the data via a custom ext_pillar module.Use the data:{% for server,ip insalt[’pillar.get’](’web.ip_addrs’, {}).items() %}server {{ server }} {{ ip[0] }}:80 check{% endfor %}Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 21 / 34
  23. 23. EventsOutline1 Salt internals (briefly)2 Salt speed3 Minion data4 Events5 Schedules6 What’s comingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 22 / 34
  24. 24. EventsEventsSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 23 / 34
  25. 25. EventsFire eventssalt ’lb-*’ event.fire_master refresh_pool loadbalancerSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 24 / 34
  26. 26. EventsWatch for events (manually)Some assembly requiredsalt/tests/eventlisten.pyComing soon to salt-apiSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 25 / 34
  27. 27. EventsReactor (react to events)/etc/salt/master:reactor:- loadbalancer:- /src/reactor/refresh_pool.sls/src/reactor/refresh_pool.sls:{% if data[’type’] == ’refresh_pool’ %}highstate_run:cmd.state.highstate:- tgt: lb-*{% endif %}Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 26 / 34
  28. 28. SchedulesOutline1 Salt internals (briefly)2 Salt speed3 Minion data4 Events5 Schedules6 What’s comingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 27 / 34
  29. 29. SchedulesSchedulesSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 28 / 34
  30. 30. SchedulesAdd events/etc/salt/{master,minion} (or pillar):schedule:highstate:function: state.highstateminutes: 60Seth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 29 / 34
  31. 31. SchedulesStats gatheringschedule:uptime:function: status.uptimeseconds: 60returner: redismeminfo:function: status.meminfominutes: 5returner: redisSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 30 / 34
  32. 32. What’s comingOutline1 Salt internals (briefly)2 Salt speed3 Minion data4 Events5 Schedules6 What’s comingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 31 / 34
  33. 33. What’s comingWhat’s comingSalt v.0.nextSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 32 / 34
  34. 34. What’s comingMonitoring statesConfigure inline with existing statesIndividual components are in placeNeeded: glueNeeded: alertingSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 33 / 34
  35. 35. What’s comingData resolutionTime-series dataThin and/or summarize older and older dataFree with some returnersSeth House <seth@eseth.com> (OpenWest Conference 2013)Real-time infrastructure management with Salt 2013-05-03 34 / 34

×