A3Sec Advanced
Deployment System
Managing all your Alienvault's
Infrastructure with Salt
About the Author
●
●
●
●
●
●
●

David Gil <dgil@a3sec.com>
Developer, sysadmin, project manager
Really believes in Open So...
Summary
●
●
●
●
●
●
●
●
●

Distribute configuration management
Remote execution platform
Salt Stack
Configuration states
S...
Distributed Configuration
Management - Why?
● Control of all the infrastructure configuration
● Ensure configurations are ...
Distributed Configuration
Management - Why?
● Control of all the infrastructure configuration
● Ensure configurations are ...
Remote Execution Platform
Why?
Examples:
● I want to restart apache2 on all my
frameworks
● I don't want to have to shell ...
Remote Execution Platform
Why?
Examples:
● I want to restart apache2 on all my
frameworks
● I don't want to have to shell ...
What is Salt?
● FOSS project for distributed configuration
and remote execution
● Still young (2011) but active developed
...
Salt Keys - Just Easy
● Simple Architecture
○ Fits fine with Alienvault's Architecture

● Easy Installation
○ Squeeze debi...
Salt Keys - Scalable
● Able to manage tens of thousands of servers
● ZeroMQ based for messaging
● Persistent connections /...
Salt Keys - Secure
Secure and encrypted communication transport
layer:
● Public Key for Master Authentication
● 256 bit AE...
Configuration State example
● You have 20 servers with exim4 as default
mail transport
● You want to get rid of exim4 and ...
Configuration State example
# apt-get install postfix (x20)
Configuration State example
# apt-get install postfix

postfix:
pkg:
- installed
Configuration State example
# apt-get install postfix
# vim /etc/postfix/master.cf (x20)

postfix:
pkg:
- installed
Configuration State example
# apt-get install postfix
# vim /etc/postfix/master.cf

postfix:
pkg:
- installed

/etc/postfi...
Configuration State example
# apt-get install postfix
# vim /etc/postfix/master.cf
# /etc/init.d/postfix restart (x20)
pos...
Configuration State example
# apt-get install postfix
# vim /etc/postfix/master.cf
# /etc/init.d/postfix restart
postfix:
...
Configuration State example
# apt-get install postfix
# vim /etc/postfix/master.cf
# /etc/init.d/postfix restart
postfix:
...
Security Deployment approach
● Always well-known behaviors
● Avoid accidental mistakes
● All configuration changes stored ...
Security Deployment approach
A few common and repetitive distribution tasks,
for every single server on your infrastructur...
Security Deployment approach
Hostname resolution
# address and name management in /etc/hosts file
mcafee-database:
host.pr...
Security Deployment approach
Code fixes and improvements distribution:
# patch --forward Detector.py agent_hot_fix.patch
/...
Salt Architecture
Master (server)
● Approved minion keys for communication
● Send commands to minions
● Store configuratio...
Salt Architecture
Client/Server topology
Master

Minion #1

Minion #2

states
modules
etc.

Minion #n
Salt Architecture
Alienvault Deployment Topology
AV SIEM

AV LOGGER
Minion

AV SENSOR #1
Minion

AV SENSOR #2
Minion

Mast...
Salt Architecture
Alienvault Deployment Topology
AV SIEM
Minion

AV SENSOR #1
Minion

AV LOGGER
Master

AV SENSOR #2
Minio...
Salt Architecture
Master

Services Model

AV SIEM

AV LOGGER
Minion

AV SENSOR #1
Minion

AV SENSOR #2
Minion

Syndic

AV ...
Built-in State Modules (partial list)
●
●
●
●
●
●
●
●
●
●

cmd
cron
file
git
group
host
locale
mount
mongo*
mysql*

●
●
●
...
Built-in Exec Modules (partial list)
●
●
●
●
●
●
●
●
●
●
●

apache
apt
archive
at
cassandra
cluster
cron
disk
file
gem
git...
Targeting
● Match minions for cli commands and state
files
● Force different states for every type of server
● Alienvault ...
Targeting
base:
'*':
- core.tools
- core.users
- core.hosts
- core.munin
- core.nagios
'av(siem|logger)*':
- alienvault.se...
Targeting
base:
'*':
- core.tools
- core.users
- core.hosts
- core.munin
- core.nagios
'av(siem|logger)*':
- alienvault.se...
Targeting
● By Hostname: Minion_id
'*', 'avsensor.*', 'av(siem|logger)', 'web1,web2,web3'

● By system information: Grains...
Targeting - Grains
Static bits of information that a minion collects
about the system (inventory!)
cpu_model
cpuarch
domai...
Targeting - Grains
● Using grains to define states
● Jinja2 template markup
apache:
pkg.installed:
{% if grains['os'] == '...
Extending - Custom grains
You can write your own custom grains:
# Return the host profile for Alienvault minions
def av_pr...
Extending - Custom grains
And use them:
# salt -G 'os:Debian' test.ping
avsiem.client01.com: True
avlogger.client01.com: T...
Extending - Custom modules
Build and use your custom exec modules:
# salt '*.alienvault' alienvault.profile
'server01.alie...
Extending - Custom modules
Example: Nagios configuration management
# salt ‘nagios01’ nagios.add_host 
“salt-test-host” “1...
Extending - Custom states
Custom states for
availability monitoring,
using exec modules

# add new host to nagios
ZZZ-salt...
Dynamic Module Distribution
● Build your custom
grains, exec
modules and states
● Automatically
distributed to the
minions...
User interface
Aggregation&Management UI
Build services on top of Salt architecture:
● MSSPs and SOCs
● Monitoring, Deployment and Manage...
Aggregation&Management UI
Aggregation&Management UI
A3Sec
http://www.a3sec.com
@a3sec
Spain Head Office
C/ Aravaca, 6 Piso 2 Derecha
28040 Madrid
Tlf. +34 915 330 978
México ...
Upcoming SlideShare
Loading in …5
×

A3Sec Advanced Deployment System

1,046 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,046
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A3Sec Advanced Deployment System

  1. 1. A3Sec Advanced Deployment System Managing all your Alienvault's Infrastructure with Salt
  2. 2. About the Author ● ● ● ● ● ● ● David Gil <dgil@a3sec.com> Developer, sysadmin, project manager Really believes in Open Source model Programming since he was 9 years old Ossim developer at its early stage Python lover :-) Debian package maintainer (a long, long time ago) ● Sci-Fi books reader and mountain bike rider
  3. 3. Summary ● ● ● ● ● ● ● ● ● Distribute configuration management Remote execution platform Salt Stack Configuration states Security deployment approach Salt architecture Builtin modules Targeting Services
  4. 4. Distributed Configuration Management - Why? ● Control of all the infrastructure configuration ● Ensure configurations are the same on all servers (or on a type of servers) ● Auto-restart services on configuration changes ● Auto-Install required packages
  5. 5. Distributed Configuration Management - Why? ● Control of all the infrastructure configuration ● Ensure configurations are the same on all servers (or on a type of servers) ● Auto-restart services on configuration changes ● Auto-Install required packages Don't sysadmin, develop configuration states. Make your life easier!
  6. 6. Remote Execution Platform Why? Examples: ● I want to restart apache2 on all my frameworks ● I don't want to have to shell into every server just to run an apt-get upgrade command ● I have {n} servers that I want to check what version of {package} is installed
  7. 7. Remote Execution Platform Why? Examples: ● I want to restart apache2 on all my frameworks ● I don't want to have to shell into every server just to run an apt-get upgrade command ● I have {n} servers that I want to check what version of {package} is installed "ssh in a for loop is not a solution" Luka Kanies, Puppet developer
  8. 8. What is Salt? ● FOSS project for distributed configuration and remote execution ● Still young (2011) but active developed ● Other tools: ○ puppet (ruby): fedora, mozilla, sans ○ chef (ruby): openstack, cloudfoundry ● Salt is easier (IMHO) than chef and puppet (steeper learning curve) ● Configure states by writing simple lists of items (yaml), more readable for sysadmins than vanilla python or ruby
  9. 9. Salt Keys - Just Easy ● Simple Architecture ○ Fits fine with Alienvault's Architecture ● Easy Installation ○ Squeeze debian packages, pip, bootstrap git ● Easy Configuration ○ No need to learn a new programming language ● Extensible ○ Develop Alienvault specific modules is quite easy!
  10. 10. Salt Keys - Scalable ● Able to manage tens of thousands of servers ● ZeroMQ based for messaging ● Persistent connections / Parallel execution ● MessagePack: fast and small message format (fluentd, redis, etc.)
  11. 11. Salt Keys - Secure Secure and encrypted communication transport layer: ● Public Key for Master Authentication ● 256 bit AES for payload high speed encryption So, VPN configuration is not needed
  12. 12. Configuration State example ● You have 20 servers with exim4 as default mail transport ● You want to get rid of exim4 and install postfix instead ● In your 20 servers? One by one?
  13. 13. Configuration State example # apt-get install postfix (x20)
  14. 14. Configuration State example # apt-get install postfix postfix: pkg: - installed
  15. 15. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf (x20) postfix: pkg: - installed
  16. 16. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf postfix: pkg: - installed /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  17. 17. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart (x20) postfix: pkg: - installed /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  18. 18. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart postfix: pkg: - installed service: - running /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  19. 19. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart postfix: pkg: - installed service: - running - watch: - file: /etc/postfix/master.cf /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  20. 20. Security Deployment approach ● Always well-known behaviors ● Avoid accidental mistakes ● All configuration changes stored in a single and unique place (master filesever) ● Private Git repository for Knowledge Database (configuration states developed at customers) ● Reusable configurations for other deployments! ● Just code one time, test it and apply where you need
  21. 21. Security Deployment approach A few common and repetitive distribution tasks, for every single server on your infrastructure: ● Hostname resolution ● Custom plugins distribution ● Remote code execution ● Snort threshold and rules ● Logrotate files ● Rsyslog filters ● Firewall rules ● etc.
  22. 22. Security Deployment approach Hostname resolution # address and name management in /etc/hosts file mcafee-database: host.present: - ip: 192.168.0.42 Cron # Clean user crontab management ntpdate-debian > /var/log/syslog: cron.present: - user: root - minute: 7 - hour: 2
  23. 23. Security Deployment approach Code fixes and improvements distribution: # patch --forward Detector.py agent_hot_fix.patch /usr/share/ossim-agent/ossim_agent/Detector.py: file.patch: - source: salt://agent_hot_fix.patch - hash: md5=e138491e9d5b97023cea823fe17bac22 # Ensure root login is disabled /etc/ssh/sshd_config: file.sed: - before: 'PermitRootLogin no' - after: 'PermitRootLogin yes'
  24. 24. Salt Architecture Master (server) ● Approved minion keys for communication ● Send commands to minions ● Store configurations, files and resources for minions Minon (client) ● Connects to Master ● Listens for commands ● Downloads configurations from Master and apply them (update config states)
  25. 25. Salt Architecture Client/Server topology Master Minion #1 Minion #2 states modules etc. Minion #n
  26. 26. Salt Architecture Alienvault Deployment Topology AV SIEM AV LOGGER Minion AV SENSOR #1 Minion AV SENSOR #2 Minion Master Minion AV SENSOR #N Minion
  27. 27. Salt Architecture Alienvault Deployment Topology AV SIEM Minion AV SENSOR #1 Minion AV LOGGER Master AV SENSOR #2 Minion Minion AV SENSOR #N Minion
  28. 28. Salt Architecture Master Services Model AV SIEM AV LOGGER Minion AV SENSOR #1 Minion AV SENSOR #2 Minion Syndic AV SENSOR #N Minion
  29. 29. Built-in State Modules (partial list) ● ● ● ● ● ● ● ● ● ● cmd cron file git group host locale mount mongo* mysql* ● ● ● ● ● ● ● pkg postgres* rabbitmq* service ssh* timezone user Custom modules can be written in Python!
  30. 30. Built-in Exec Modules (partial list) ● ● ● ● ● ● ● ● ● ● ● apache apt archive at cassandra cluster cron disk file gem git ● hosts ● iptables ● keyboard ● ldap ● locale ● network All supported: salt 'node' sys.doc Custom modules can be written in Python!
  31. 31. Targeting ● Match minions for cli commands and state files ● Force different states for every type of server ● Alienvault profiles: ○ ○ ○ ○ ○ SIEM Logger Framework Sensor Database
  32. 32. Targeting base: '*': - core.tools - core.users - core.hosts - core.munin - core.nagios 'av(siem|logger)*': - alienvault.server - alienvault.database 'av(sensor|agent)*': - alienvault.agent - alienvault.plugins
  33. 33. Targeting base: '*': - core.tools - core.users - core.hosts - core.munin - core.nagios 'av(siem|logger)*': - alienvault.server - alienvault.database 'av(sensor|agent)*': - alienvault.agent - alienvault.plugins $ tree /srv/salt/ /srv/salt/ |-- alienvault | |-- agent.sls | |-- database.sls | |-- plugins.sls | `-- server.sls |-- core | |-- hosts.sls | |-- munin.sls | |-- tools.sls | `-- users.sls `-- top.sls
  34. 34. Targeting ● By Hostname: Minion_id '*', 'avsensor.*', 'av(siem|logger)', 'web1,web2,web3' ● By system information: Grains grains['os'] == 'Debian' and 'Server' in grains ['av_profile'] ● By defined groups: Node Groups nodegroups: 'sensors': avsensor*.domain.com, foo.bar 'servers': av(siem|logger).domain.com, bar.foo ● Mix combination: Compound 'webserv* and G@os:Debian or E@web-dc1-srv.*'
  35. 35. Targeting - Grains Static bits of information that a minion collects about the system (inventory!) cpu_model cpuarch domain fqdn gpus host kernel kernelrelease lsb_codename lsb_description lsb_os lsb_release manufacturer mem_total nodename num_cpus os oscodename osfullname osrelease path productname server_id shell virtual etc.
  36. 36. Targeting - Grains ● Using grains to define states ● Jinja2 template markup apache: pkg.installed: {% if grains['os'] == 'RedHat' %} - name: httpd {% elif grains['os'] == 'Ubuntu' %} - name: apache2 {% endif %}
  37. 37. Extending - Custom grains You can write your own custom grains: # Return the host profile for Alienvault minions def av_profile(): alienvault_config = '/etc/ossim/ossim_setup.conf' if not os.path.isfile(alienvault_config): return {} config = ConfigParser.RawConfigParser() config.read(alienvault_config) profile = config.get('root', 'profile') return {'alienvault_profile': profile}
  38. 38. Extending - Custom grains And use them: # salt -G 'os:Debian' test.ping avsiem.client01.com: True avlogger.client01.com: True avsensor01.client01.com: True avsensor02.client01.com: True # salt -G 'av_profile:Server' cmd.run 'uname -r' avsiem.client01.com: 2.6.32-5-amd64 avlogger.client01.com: 2.6.31.6
  39. 39. Extending - Custom modules Build and use your custom exec modules: # salt '*.alienvault' alienvault.profile 'server01.alienvault': 'Server,Database' 'logger01.alienvault': 'Server,Database' # salt '*.alienvault' alienvault.snort_dbsize 'server01.alienvault': 457Mb 'logger01.alienvault': 0Mb # salt '*.alienvault' alienvault.numalarms 'server01.alienvault': 393 'logger01.alienvault': 0
  40. 40. Extending - Custom modules Example: Nagios configuration management # salt ‘nagios01’ nagios.add_host “salt-test-host” “127.0.0.1” nagios01: [nagios] Added new host: salt-test-host # salt ‘nagios01’ nagios.add_service “salt-test-host” “HTTP” “check_http” nagios01: [nagios] Added new service: HTTP for host salttest-host
  41. 41. Extending - Custom states Custom states for availability monitoring, using exec modules # add new host to nagios ZZZ-salt-test-host: monitoring.add_host: - address: 127.0.0.1 # add new services to nagios HTTP: monitoring.add_service: - checkname: check_http - hostnames: - ZZZ-salt-test-host - YYY-salt-test-host
  42. 42. Dynamic Module Distribution ● Build your custom grains, exec modules and states ● Automatically distributed to the minions with the salt-master file server ● Directories are prepended with an underscore /srv/salt/ |-- alienvault | |-- agent.sls | |-- database.sls | |-- plugins.sls | `-- server.sls |-- core | |-- hosts.sls | |-- munin.sls | |-- tools.sls | `-- users.sls |-- _grains |-- _modules |-- _states `-- top.sls
  43. 43. User interface
  44. 44. Aggregation&Management UI Build services on top of Salt architecture: ● MSSPs and SOCs ● Monitoring, Deployment and Management off all the infrastructure in a single dashboard ● Configurations backup on demand ● Patches and fixes distribution ● Full Inventory ● etc...
  45. 45. Aggregation&Management UI
  46. 46. Aggregation&Management UI
  47. 47. A3Sec http://www.a3sec.com @a3sec Spain Head Office C/ Aravaca, 6 Piso 2 Derecha 28040 Madrid Tlf. +34 915 330 978 México Head Office Avda. Paseo de la Reforma, 389 Piso 10 México DF Tlf. +52 55 5980 3547

×