A3Sec Advanced Deployment System
Upcoming SlideShare
Loading in...5
×
 

A3Sec Advanced Deployment System

on

  • 274 views

 

Statistics

Views

Total Views
274
Views on SlideShare
274
Embed Views
0

Actions

Likes
0
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

A3Sec Advanced Deployment System A3Sec Advanced Deployment System Presentation Transcript

  • A3Sec Advanced Deployment System Managing all your Alienvault's Infrastructure with Salt
  • About the Author ● ● ● ● ● ● ● David Gil <dgil@a3sec.com> Developer, sysadmin, project manager Really believes in Open Source model Programming since he was 9 years old Ossim developer at its early stage Python lover :-) Debian package maintainer (a long, long time ago) ● Sci-Fi books reader and mountain bike rider
  • Summary ● ● ● ● ● ● ● ● ● Distribute configuration management Remote execution platform Salt Stack Configuration states Security deployment approach Salt architecture Builtin modules Targeting Services View slide
  • Distributed Configuration Management - Why? ● Control of all the infrastructure configuration ● Ensure configurations are the same on all servers (or on a type of servers) ● Auto-restart services on configuration changes ● Auto-Install required packages View slide
  • Distributed Configuration Management - Why? ● Control of all the infrastructure configuration ● Ensure configurations are the same on all servers (or on a type of servers) ● Auto-restart services on configuration changes ● Auto-Install required packages Don't sysadmin, develop configuration states. Make your life easier!
  • Remote Execution Platform Why? Examples: ● I want to restart apache2 on all my frameworks ● I don't want to have to shell into every server just to run an apt-get upgrade command ● I have {n} servers that I want to check what version of {package} is installed
  • Remote Execution Platform Why? Examples: ● I want to restart apache2 on all my frameworks ● I don't want to have to shell into every server just to run an apt-get upgrade command ● I have {n} servers that I want to check what version of {package} is installed "ssh in a for loop is not a solution" Luka Kanies, Puppet developer
  • What is Salt? ● FOSS project for distributed configuration and remote execution ● Still young (2011) but active developed ● Other tools: ○ puppet (ruby): fedora, mozilla, sans ○ chef (ruby): openstack, cloudfoundry ● Salt is easier (IMHO) than chef and puppet (steeper learning curve) ● Configure states by writing simple lists of items (yaml), more readable for sysadmins than vanilla python or ruby
  • Salt Keys - Just Easy ● Simple Architecture ○ Fits fine with Alienvault's Architecture ● Easy Installation ○ Squeeze debian packages, pip, bootstrap git ● Easy Configuration ○ No need to learn a new programming language ● Extensible ○ Develop Alienvault specific modules is quite easy!
  • Salt Keys - Scalable ● Able to manage tens of thousands of servers ● ZeroMQ based for messaging ● Persistent connections / Parallel execution ● MessagePack: fast and small message format (fluentd, redis, etc.)
  • Salt Keys - Secure Secure and encrypted communication transport layer: ● Public Key for Master Authentication ● 256 bit AES for payload high speed encryption So, VPN configuration is not needed
  • Configuration State example ● You have 20 servers with exim4 as default mail transport ● You want to get rid of exim4 and install postfix instead ● In your 20 servers? One by one?
  • Configuration State example # apt-get install postfix (x20)
  • Configuration State example # apt-get install postfix postfix: pkg: - installed
  • Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf (x20) postfix: pkg: - installed
  • Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf postfix: pkg: - installed /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  • Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart (x20) postfix: pkg: - installed /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  • Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart postfix: pkg: - installed service: - running /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  • Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart postfix: pkg: - installed service: - running - watch: - file: /etc/postfix/master.cf /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
  • Security Deployment approach ● Always well-known behaviors ● Avoid accidental mistakes ● All configuration changes stored in a single and unique place (master filesever) ● Private Git repository for Knowledge Database (configuration states developed at customers) ● Reusable configurations for other deployments! ● Just code one time, test it and apply where you need
  • Security Deployment approach A few common and repetitive distribution tasks, for every single server on your infrastructure: ● Hostname resolution ● Custom plugins distribution ● Remote code execution ● Snort threshold and rules ● Logrotate files ● Rsyslog filters ● Firewall rules ● etc.
  • Security Deployment approach Hostname resolution # address and name management in /etc/hosts file mcafee-database: host.present: - ip: 192.168.0.42 Cron # Clean user crontab management ntpdate-debian > /var/log/syslog: cron.present: - user: root - minute: 7 - hour: 2
  • Security Deployment approach Code fixes and improvements distribution: # patch --forward Detector.py agent_hot_fix.patch /usr/share/ossim-agent/ossim_agent/Detector.py: file.patch: - source: salt://agent_hot_fix.patch - hash: md5=e138491e9d5b97023cea823fe17bac22 # Ensure root login is disabled /etc/ssh/sshd_config: file.sed: - before: 'PermitRootLogin no' - after: 'PermitRootLogin yes'
  • Salt Architecture Master (server) ● Approved minion keys for communication ● Send commands to minions ● Store configurations, files and resources for minions Minon (client) ● Connects to Master ● Listens for commands ● Downloads configurations from Master and apply them (update config states)
  • Salt Architecture Client/Server topology Master Minion #1 Minion #2 states modules etc. Minion #n
  • Salt Architecture Alienvault Deployment Topology AV SIEM AV LOGGER Minion AV SENSOR #1 Minion AV SENSOR #2 Minion Master Minion AV SENSOR #N Minion
  • Salt Architecture Alienvault Deployment Topology AV SIEM Minion AV SENSOR #1 Minion AV LOGGER Master AV SENSOR #2 Minion Minion AV SENSOR #N Minion
  • Salt Architecture Master Services Model AV SIEM AV LOGGER Minion AV SENSOR #1 Minion AV SENSOR #2 Minion Syndic AV SENSOR #N Minion
  • Built-in State Modules (partial list) ● ● ● ● ● ● ● ● ● ● cmd cron file git group host locale mount mongo* mysql* ● ● ● ● ● ● ● pkg postgres* rabbitmq* service ssh* timezone user Custom modules can be written in Python!
  • Built-in Exec Modules (partial list) ● ● ● ● ● ● ● ● ● ● ● apache apt archive at cassandra cluster cron disk file gem git ● hosts ● iptables ● keyboard ● ldap ● locale ● network All supported: salt 'node' sys.doc Custom modules can be written in Python!
  • Targeting ● Match minions for cli commands and state files ● Force different states for every type of server ● Alienvault profiles: ○ ○ ○ ○ ○ SIEM Logger Framework Sensor Database
  • Targeting base: '*': - core.tools - core.users - core.hosts - core.munin - core.nagios 'av(siem|logger)*': - alienvault.server - alienvault.database 'av(sensor|agent)*': - alienvault.agent - alienvault.plugins
  • Targeting base: '*': - core.tools - core.users - core.hosts - core.munin - core.nagios 'av(siem|logger)*': - alienvault.server - alienvault.database 'av(sensor|agent)*': - alienvault.agent - alienvault.plugins $ tree /srv/salt/ /srv/salt/ |-- alienvault | |-- agent.sls | |-- database.sls | |-- plugins.sls | `-- server.sls |-- core | |-- hosts.sls | |-- munin.sls | |-- tools.sls | `-- users.sls `-- top.sls
  • Targeting ● By Hostname: Minion_id '*', 'avsensor.*', 'av(siem|logger)', 'web1,web2,web3' ● By system information: Grains grains['os'] == 'Debian' and 'Server' in grains ['av_profile'] ● By defined groups: Node Groups nodegroups: 'sensors': avsensor*.domain.com, foo.bar 'servers': av(siem|logger).domain.com, bar.foo ● Mix combination: Compound 'webserv* and G@os:Debian or E@web-dc1-srv.*'
  • Targeting - Grains Static bits of information that a minion collects about the system (inventory!) cpu_model cpuarch domain fqdn gpus host kernel kernelrelease lsb_codename lsb_description lsb_os lsb_release manufacturer mem_total nodename num_cpus os oscodename osfullname osrelease path productname server_id shell virtual etc.
  • Targeting - Grains ● Using grains to define states ● Jinja2 template markup apache: pkg.installed: {% if grains['os'] == 'RedHat' %} - name: httpd {% elif grains['os'] == 'Ubuntu' %} - name: apache2 {% endif %}
  • Extending - Custom grains You can write your own custom grains: # Return the host profile for Alienvault minions def av_profile(): alienvault_config = '/etc/ossim/ossim_setup.conf' if not os.path.isfile(alienvault_config): return {} config = ConfigParser.RawConfigParser() config.read(alienvault_config) profile = config.get('root', 'profile') return {'alienvault_profile': profile}
  • Extending - Custom grains And use them: # salt -G 'os:Debian' test.ping avsiem.client01.com: True avlogger.client01.com: True avsensor01.client01.com: True avsensor02.client01.com: True # salt -G 'av_profile:Server' cmd.run 'uname -r' avsiem.client01.com: 2.6.32-5-amd64 avlogger.client01.com: 2.6.31.6
  • Extending - Custom modules Build and use your custom exec modules: # salt '*.alienvault' alienvault.profile 'server01.alienvault': 'Server,Database' 'logger01.alienvault': 'Server,Database' # salt '*.alienvault' alienvault.snort_dbsize 'server01.alienvault': 457Mb 'logger01.alienvault': 0Mb # salt '*.alienvault' alienvault.numalarms 'server01.alienvault': 393 'logger01.alienvault': 0
  • Extending - Custom modules Example: Nagios configuration management # salt ‘nagios01’ nagios.add_host “salt-test-host” “127.0.0.1” nagios01: [nagios] Added new host: salt-test-host # salt ‘nagios01’ nagios.add_service “salt-test-host” “HTTP” “check_http” nagios01: [nagios] Added new service: HTTP for host salttest-host
  • Extending - Custom states Custom states for availability monitoring, using exec modules # add new host to nagios ZZZ-salt-test-host: monitoring.add_host: - address: 127.0.0.1 # add new services to nagios HTTP: monitoring.add_service: - checkname: check_http - hostnames: - ZZZ-salt-test-host - YYY-salt-test-host
  • Dynamic Module Distribution ● Build your custom grains, exec modules and states ● Automatically distributed to the minions with the salt-master file server ● Directories are prepended with an underscore /srv/salt/ |-- alienvault | |-- agent.sls | |-- database.sls | |-- plugins.sls | `-- server.sls |-- core | |-- hosts.sls | |-- munin.sls | |-- tools.sls | `-- users.sls |-- _grains |-- _modules |-- _states `-- top.sls
  • User interface
  • Aggregation&Management UI Build services on top of Salt architecture: ● MSSPs and SOCs ● Monitoring, Deployment and Management off all the infrastructure in a single dashboard ● Configurations backup on demand ● Patches and fixes distribution ● Full Inventory ● etc...
  • Aggregation&Management UI
  • Aggregation&Management UI
  • A3Sec http://www.a3sec.com @a3sec Spain Head Office C/ Aravaca, 6 Piso 2 Derecha 28040 Madrid Tlf. +34 915 330 978 México Head Office Avda. Paseo de la Reforma, 389 Piso 10 México DF Tlf. +52 55 5980 3547