How to write custom execution modules for Saltstack and put credentials and other secrets into your Python code. The presentation reviews the use of Salt Pillars as well as the Salt API

1. PASSING SECRETS THROUGH
CUSTOM EXECUTION
MODULES WITH SALTSTACK
Julian Pacheco
jpacheco@isystematics.com

2. WHAT IS SALTSTACK
• System for remote execution of commands and configuration management
• Server/client model
• Built for parallel execution, scalability and security
• Written in Python
• Open source https://github.com/saltstack/salt

3. SALT EXECUTION MODULES
• Python functions called by the Salt Master to be executed on targeted minions
• Salt is prepackaged with modules to
• Restart services
• Install packages
• Transfer files
• Etc
• https://docs.saltstack.com/en/latest/ref/modules/all/index.html#all-salt-modules

4. EXAMPLE #1

5. EXAMPLE #1
• Make the module available to execute on the desired minion or minions
• salt ‘<minion id(s)>' saltutil.sync_modules
• Execute the module
• salt ‘<minion id(s)>’ salt_presentation.first_test

6. EXAMPLE RECAP
• Python file in /srv/salt/_modules on the Salt Master
• Defined function that handles module logic or lack thereof
• Plaintext credentials in code in two locations!!
• Salt master in /srv/salt/_modules/salt_presentation.py
• Salt minion in /var/cache/salt/minion/extmods/modules/salt_presentation.py

7. WAYS TO PASS SECRETS IN PYTHON
• Use separate settings.py or similar with credentials and import them
• Use environment variables on the host and os.environ.get()
• Create a hidden file on the host and use a ConfigParser
• Use argparse and command line arguments to parameterize credentials
• Accept secrets from user input each time code is run

8. WHERE THESE METHODS FALL SHORT
• Not designed for remote execution
• Add little in the way of security if an attacker gains access to the host executing the
code
• May not be easily repeatable (ie; requiring user input every run through)

9. SALT PILLARS
• Designed for data that needs to be securely sent only to targeted minions
• Stored in memory on the targeted minions
• Can be encrypted on the Salt master for extra security
• Provides code portability since they’re great for variable or arbitrary data
• Just a Python dictionary

10. EXAMPLE #2

11. EXAMPLE #2

12. EXAMPLE #2
• Target the pillar to the desired minion within top.sls in /srv/pillar

13. EXAMPLE #2
• Ensure the minion has the new pillar data and module

14. EXAMPLE #2

15. EXAMPLE RECAP
• Removed hard coded credentials from source code and from host running our code
• Credentials are passed only to targeted minion(s)
• Only have credentials on the Salt master

16. OTHER WAYS SALT LETS US PASS SECRETS
• Salt-API
• Run on the Salt-master
• API calls to run Salt modules from any other host on the network
• Pass pillar kwargs to state.apply module
• External Pillars
• Pull Pillar data from sources such as Postgres, HTTP API requests, and more
• https://docs.saltstack.com/en/latest/topics/development/external_pillars.html

17. EXAMPLE #3
• Setup and start the salt-api on the Salt Master
• https://docs.saltstack.com/en/latest/ref/netapi/all/salt.netapi.rest_cherrypy.html#a-rest-
api-for-salt
• Create a Salt state on the master to run our execution module

18. EXAMPLE #3
• Choose a host that has network access to the Salt master
• Call the Salt Api to execute the state against the targeted minion(s) from this host
• Curl
• Python with Requests
• etc

19. EXAMPLE #3

20. EXAMPLE #3
• Executing the previous python code results in the execution module running against
the ‘ubuntu’ minion

21. EXAMPLE RECAP
• Provided further segregation of secrets from source code
• Added flexibility to run Salt commands from other applications and hosts
• Can allow other users to run Salt commands without having root access to the Salt
master

22. SALT IN THE REAL WORLD
• Pull Nessus scans from multiple network environments into a centralized location
for auditing
• Gather data from hosts on multiple networks and store it in a database
• Configure Splunk communications to use custom SSL certificates and certificate
verification
• Create self-healing infrastructure