Salt stack introduction

1,098 views
999 views

Published on

salt stack is distributed remote execution and configuration management tool build on python.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,098
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • “Salt is:

    a configuration management system, capable of maintaining remote nodes in defined states (for example, ensuring that specific packages are installed and specific services are running)
    a distributed remote execution system used to execute commands and query data on remote nodes, either individually or by arbitrary selection criteria”


    All the backend communcation is run on zero_mq
  • “Simplicity

    Providing versatility between massive scale deployments and smaller systems may seem daunting, but Salt is very simple to set up and maintain, regardless of the size of the project. The architecture of Salt is designed to work with any number of servers, from a handful of local network systems to international deployments across different datacenters. The topology is a simple server/client model with the needed functionality built into a single set of daemons. While the default configuration will work with little to no modification, Salt can be fine tuned to meet specific needs.

    “Parallel execution

    The core functions of Salt:

    enable commands to remote systems to be called in parallel rather than serially
    use a secure and encrypted protocol
    use the smallest and fastest network payloads possible
    provide a simple programming interface
    Salt also introduces more granular controls to the realm of remote execution, allowing systems to be targeted not just by hostname, but also by system properties.”


    “Building on proven technology

    Salt takes advantage of a number of technologies and techniques. The networking layer is built with the excellent ZeroMQ [http://zeromq.org/] networking library, so the Salt daemon includes a viable and transparent AMQ broker. Salt uses public keys for authentication with the master daemon, then uses faster AES [https://en.wikipedia.org/wiki/Advanced_Encryption_Standard] encryption for payload communication; authentication and encryption are integral to Salt. Salt takes advantage of communication via msgpack [http://msgpack.org/], enabling fast and light network traffic.

    Python client interface
    “In order to allow for simple expansion, Salt execution routines can be written as plain Python modules. The data collected from Salt executions can be sent back to the master server, or to any arbitrary program. Salt can be called from a simple Python API, or from the command line, so that Salt can be used to execute one-off commands as well as operate as an integral part of a larger application.”

    Fast, flexible, scalable
    “The result is a system that can execute commands at high speed on target server groups ranging from one to very many servers. Salt is very fast, easy to set up, amazingly malleable and provides a single remote execution architecture that can manage the diverse requirements of any number of servers. The Salt infrastructure brings together the best of the remote execution world, amplifies its capabilities and expands its range, resulting in a system that is as versatile as it is practical, suitable for any network.”


    “Open

    Salt is developed under the Apache 2.0 license [http://www.apache.org/licenses/LICENSE-2.0.html], and can be used for open and proprietary projects. Please submit your expansions back to the Salt project so that we can all benefit together as Salt grows. Please feel free to sprinkle Salt around your systems and let the deliciousness come forth.”

  • Each physical data center
    multiple “fabrics” (logical grouping of hosts)
    single salt master (largest set of minions = 8+k)
    warm backup (same private key)
    minions configured with CNAME to master
    Files stored in subversion
    states, grains, modules
    Runners
    Reactor


    master The Salt master is the central server that all minions connect to. Commands are run on the minions through the master, and minions send data back to the master (unless otherwise redirected with a returner). It is started with the salt-master program.
    minion Salt minions are the potentially hundreds or thousands of servers that may be queried and controlled from the master.
  • All cmd are send the parallel (completely async)

    Ssh mode, you can have any system without any agent on them to run commands, anti topology , slow and scale issue

    Returners –

    Syndec –

    Peer interface – the allows minions to send/control other minions

    Reactor – seats on the mater and listens for events from minions, minions can send events back to master which can has logic to handle those events. Ex build system
  • • use the reactor system to send metrics
    •metrics gathering is all home grown
    •trying to open source it



    file updates (every 5 mins)
    •modules, states, grains
  • Syncing only every hour by cfengine
  • Salt stack introduction

    1. 1. ©2014 LinkedIn Corporation. All Rights Reserved. Who am I? Muzammil Rehman SRE for Linkedin Co-founder Tom Hatch.
    2. 2. ©2014 LinkedIn Corporation. All Rights Reserved. What is LinkedIn?  Social media company connecting the world’s professionals  5000+ employees  Offices throughout the world  HQ Mountain View, CA
    3. 3. ©2014 LinkedIn Corporation. All Rights Reserved. How Big Is lnkedin.com?  Several data centers – Customer facing apps (aka “production”) – Staging for production apps – Internal only apps  Several Hundred Apps  30+K Hosts – 90+% Linux – Mac and Linux Desktops
    4. 4. ©2014 LinkedIn Corporation. All Rights Reserved. What is salt?  Configuration management  Remote command execution framework
    5. 5. ©2014 LinkedIn Corporation. All Rights Reserved. Why salt?  Simplicity  Parallel execution  Building on proven technology  Python client interface  Fast, flexible, scalable  Open
    6. 6. ©2014 LinkedIn Corporation. All Rights Reserved. Salt Used  Using salt since 0.8.9, now 2014.14  Installation of new apps  Config management  Automation
    7. 7. ©2014 LinkedIn Corporation. All Rights Reserved. Salt Architecture  master The Salt master is the central server that all minions connect to.  minion Salt minions are the potentially hundreds or thousands of servers that may be queried and controlled from the master.
    8. 8. ©2014 LinkedIn Corporation. All Rights Reserved. Master minion minion Master minion minion db Masterdb git
    9. 9. ©2014 LinkedIn Corporation. All Rights Reserved. Installing Salt  cfengine will push new salt releases and restart minions – cfengine also manages minion install and configs  salt master is wrapped in a “runit” script – salt API – use the reactor system to send metrics
    10. 10. ©2014 LinkedIn Corporation. All Rights Reserved. Where can salt help  Running multiple commands  Repeating the same on n # servers  Automated installation  Deployment system  etc
    11. 11. ©2014 LinkedIn Corporation. All Rights Reserved. Salt modules  Check for available modules – Group: https://groups.google.com/forum/#!forum/salt-users – #salt freenode – https://github.com/saltstack/salt – http://www.saltstack.com/blog/
    12. 12. ©2014 LinkedIn Corporation. All Rights Reserved. Salt modules cont..  Small oversight last year caused massive issues  Developed process to “promote” modules  Salt environments: – dev -> vm -> test -> stage -> prod  minions are configured to look at certain environments
    13. 13. ©2014 LinkedIn Corporation. All Rights Reserved. Salt module example def mkdir(dir, user='root', group='root', mode=0755): ''' Make a directory salt '*' li.mkdir /path/to/dir [user] [group] [mode] Returns True or error string ''’ if os.path.isdir(dir): return True elif os.path.exists(dir): return False else: try: os.makedirs(dir, mode) uid = pwd.getpwnam(user)[2] gid = grp.getgrnam(group)[2] os.chown(dir, uid, gid) except OSError, e: return e except KeyError, e: return e return True
    14. 14. ©2014 LinkedIn Corporation. All Rights Reserved. Salt stale (sls) aka config managment  require, watch and prereq. httpd: pkg: - installed file.managed: - name: /etc/httpd/conf/httpd.conf - source: salt://httpd/httpd.conf - require: - pkg: httpd redis: pkg: - latest file.managed: - source: salt://redis/redis.conf - name: /etc/redis.conf - require: - pkg: redis service.running: - enable: True - watch: - file: /etc/redis.conf - pkg: redis
    15. 15. ©2014 LinkedIn Corporation. All Rights Reserved.  Order  Prereq  Many more vim: pkg.installed: - order: 1 graceful-down: cmd.run: - name: service apache graceful - prereq: - file: site-code site-code: file.recurse: - name: /opt/site_code - source: salt://site/code
    16. 16. ©2014 LinkedIn Corporation. All Rights Reserved. Access control  Grant access to non-administrative users  Only sudo and block modules client_acl: fred: - web*: - pkg.list_pkgs - test.* - apache.* client_acl_blacklist: users: - root - '^(?!sudo_).*$' # all non sudo users modules: - cmd
    17. 17. ©2014 LinkedIn Corporation. All Rights Reserved. Troubleshooting  Run master in foreground – salt-master -l debug – salt-minion -l debug  Port 4505, 4506  Open file, salt needs at least minions x 2
    18. 18. ©2014 LinkedIn Corporation. All Rights Reserved. Problem with Salt  Education – Most salt customizations by small group a users – Few power users  Corrupted keys  Module sync  No syncing on Solaris  No high state enforcement
    19. 19. ©2014 LinkedIn Corporation. All Rights Reserved. Demo…
    20. 20. ©2014 LinkedIn Corporation. All Rights Reserved. Q n A

    ×