OpenNebula
and
SaltStack
Valentin Bud
CEO
databus.pro | valentin@databus.pro | @databuspro
OpenNebulaConf
Berlin, Germany, September 25th
#OpenNebulaConf
SaltStack
About
• The name is the vision
• A different approach to infrastructure
management
• Transparent control system
• A simple viable building block
#OpenNebulaConfValentin Bud | @valentinbudm
SaltStack
Standing on the Shoulders of the Giants
• Python
• 0MQ
• MessagePack
• M2Crypto
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
Architecture
...
0MQ
MASTER(S)
minion
minion
minion
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The Remote Execution Engine
root@salt ~ # salt ‘*’ test.ping
nfs:
True
salt:
True
node01:
True
salt nfs
test.ping
response
0MQ
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
Meet your minions - Grains
root@salt ~ # salt ‘nfs’ grains.items
nfs:
...
kernel: Linux
kernelrelease: 2.6.32-358.18.1.el6.x86_64
localhost: nfs master: salt
mem_total: 1877
nodename: nfs
num_cpus: 2
num_gpus: 1
os: CentOS
os_family: RedHat
oscodename: Final
osfullname: CentOS
...
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
Grains - Tag your minions
root@salt ~ # salt ‘salt’ grains.setval role opennebula-frontend
salt:
role: opennebula-frontend
root@salt ~ # salt ‘salt’ grains.item role
salt:
role:
opennebula-frontend
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
Target your Minions
GLOBBING
root@salt ~ # salt -G 'web*' test.ping
REGULAR EXPRESSIONS
root@salt ~ # salt -E 'web1-(prod|devel)' test.ping
LISTS
root@salt ~ # salt -L 'web1,web2,web3' test.ping
GRAINS
root@salt ~ # salt -G '@os:CentOS' test.ping
#OpenNebulaConfValentin Bud | @valentinbud
OpenNebula and SaltStack
OpenNebulaConf Demo Cloud
salt
frontend
node01
salt
nfs
Services Network
private
virtual
machine
network
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The State System
• The Configuration Management
component
• Already available with a basic setup
• Powerful, fast, lightweight system
#OpenNebulaConfValentin Bud | @valentinbud
opennebula-server State
Install the OpenNebula Server
opennebula-server/init.sls:
opennebula-server:
pkg:
- installed
service:
- running
- require:
- pkg: opennebula-server
#OpenNebulaConfValentin Bud | @valentinbud
The Anatomy of a State
IT IS ALL JUST DATA!
opennebula-server/init.sls:
opennebula-server:
pkg:
- installed
service:
- running
- require:
- pkg: opennebula-server
ID DECLARATION
STATE DECLARATION
STATE ARGUMENTS
#OpenNebulaConfValentin Bud | @valentinbud
opennebula-sunstone State
Beyond a single State File
opennebula-server/init.sls:
opennebula-server:
pkg:
- installed
service:
- running
- require:
- pkg: opennebula-server
opennebula-sunstone/init.sls:
include:
- opennebula-server
opennebula-sunstone:
pkg:
- installed
service:
- running
- require:
- service: opennebula-server
#OpenNebulaConfValentin Bud | @valentinbud
State Execution
Behind the scenes
salt minion
state.sls opennebula-server
response
root@salt ~ # salt ‘minion’ state.sls opennebula-server
1
parse
and
execute
2
3
0MQ
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The Highstate
root@salt ~ # salt ‘*’ state.highstate
states/top.sls
base:
‘frontend’:
- opennebula-server
- opennebula-frontend
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The Render System
• State data is just that - data
• It does not need to be represented in
YAML
• State files can be rendered from any
medium
• Modular system
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
Be a Ninja with Jinja2
libvirt/init.sls:
libvirt:
pkg:
- installed
{% if grains[‘os’] == ‘Debian’ %}
- name: libvirt-bin
{% elif grains[‘os’] == ‘CentOS’ %}
- name: libvirt
{% endif %}
#OpenNebulaConfValentin Bud | @valentinbud
State Execution
Behind the scenes with Jinja2
salt minion
state.sls libvirt
response
root@salt ~ # salt ‘minion’ state.sls opennebula-server
1
parse
execute
2
4
render
3
0MQ
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The Pillar Interface
• Generate/store data for specific minions
• Highly sensitive data
• Minion configuration
• Variables
• Arbitrary data
#OpenNebulaConfValentin Bud | @valentinbud
OpenNebula oneadmin Password
Stored in Pillar
pillar/opennebula.sls:
opennebula:
oneadmin:
password: SecurePassword
state/oneadmin.sls:
oneadmin_password:
cmd:
- run
- name: oneuser passwd 0 {{ pillar[‘opennebula’][‘oneadmin’]
[‘password’] }}
#OpenNebulaConfValentin Bud | @valentinbud
OpenNebula Node Communication
TOP SECRET
frontend node
START VM
VM STARTED
NEEDS:
SSH password-less communication
Distribute Public Key to Nodes
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The Mine
node
mine
mine.send
oneadmin_public_key.put
1
mine.get
2
/var/lib/one/.ssh/authorized_keys
3
frontend:
oneadmin_public_key: ssh-rsa ...
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
The Mine in States
state/oneadmin_ssh_auth.sls:
oneadmin_ssh_auth:
ssh_auth:
- present
- user: oneadmin
- name: {{ salt['mine.get']('frontend', 'oneadmin_public_key.get') }}
#OpenNebulaConfValentin Bud | @valentinbud
SaltStack
Peer Communication
• Allow minion to “talk” to each other
• Chatting done via Master
• Must be enabled on the Master
• Live data from other minions!
#OpenNebulaConfValentin Bud | @valentinbud
OpenNebula Node Up
Create it on the Frontend
salt node
state.sls opennebula-node-kvm
communicate frontend to
create node
1
configure
node
2
4
3
frontend
createnode
#OpenNebulaConfValentin Bud | @valentinbud
Bootstrap OpenNebula Demo
Cloud
The SaltStack Overstate
overstate
all
network
nfs-server
storage
frontend
frontend
nodes
nodes
1
2
3
4
5
6
7
8
#OpenNebulaConfValentin Bud | @valentinbud
Conclusions
• Two simple tools together = POWER
• Salt is fast
• Both are easy to use
• Salt can morph and grow together with
your needs
2
#OpenNebulaConfValentin Bud | @valentinbud
databus.pro
@

OpenNebula and SaltStack - OpenNebulaConf 2013

  • 1.
    OpenNebula and SaltStack Valentin Bud CEO databus.pro |valentin@databus.pro | @databuspro OpenNebulaConf Berlin, Germany, September 25th #OpenNebulaConf
  • 2.
    SaltStack About • The nameis the vision • A different approach to infrastructure management • Transparent control system • A simple viable building block #OpenNebulaConfValentin Bud | @valentinbudm
  • 3.
    SaltStack Standing on theShoulders of the Giants • Python • 0MQ • MessagePack • M2Crypto #OpenNebulaConfValentin Bud | @valentinbud
  • 4.
  • 5.
    SaltStack The Remote ExecutionEngine root@salt ~ # salt ‘*’ test.ping nfs: True salt: True node01: True salt nfs test.ping response 0MQ #OpenNebulaConfValentin Bud | @valentinbud
  • 6.
    SaltStack Meet your minions- Grains root@salt ~ # salt ‘nfs’ grains.items nfs: ... kernel: Linux kernelrelease: 2.6.32-358.18.1.el6.x86_64 localhost: nfs master: salt mem_total: 1877 nodename: nfs num_cpus: 2 num_gpus: 1 os: CentOS os_family: RedHat oscodename: Final osfullname: CentOS ... #OpenNebulaConfValentin Bud | @valentinbud
  • 7.
    SaltStack Grains - Tagyour minions root@salt ~ # salt ‘salt’ grains.setval role opennebula-frontend salt: role: opennebula-frontend root@salt ~ # salt ‘salt’ grains.item role salt: role: opennebula-frontend #OpenNebulaConfValentin Bud | @valentinbud
  • 8.
    SaltStack Target your Minions GLOBBING root@salt~ # salt -G 'web*' test.ping REGULAR EXPRESSIONS root@salt ~ # salt -E 'web1-(prod|devel)' test.ping LISTS root@salt ~ # salt -L 'web1,web2,web3' test.ping GRAINS root@salt ~ # salt -G '@os:CentOS' test.ping #OpenNebulaConfValentin Bud | @valentinbud
  • 9.
    OpenNebula and SaltStack OpenNebulaConfDemo Cloud salt frontend node01 salt nfs Services Network private virtual machine network #OpenNebulaConfValentin Bud | @valentinbud
  • 10.
    SaltStack The State System •The Configuration Management component • Already available with a basic setup • Powerful, fast, lightweight system #OpenNebulaConfValentin Bud | @valentinbud
  • 11.
    opennebula-server State Install theOpenNebula Server opennebula-server/init.sls: opennebula-server: pkg: - installed service: - running - require: - pkg: opennebula-server #OpenNebulaConfValentin Bud | @valentinbud
  • 12.
    The Anatomy ofa State IT IS ALL JUST DATA! opennebula-server/init.sls: opennebula-server: pkg: - installed service: - running - require: - pkg: opennebula-server ID DECLARATION STATE DECLARATION STATE ARGUMENTS #OpenNebulaConfValentin Bud | @valentinbud
  • 13.
    opennebula-sunstone State Beyond asingle State File opennebula-server/init.sls: opennebula-server: pkg: - installed service: - running - require: - pkg: opennebula-server opennebula-sunstone/init.sls: include: - opennebula-server opennebula-sunstone: pkg: - installed service: - running - require: - service: opennebula-server #OpenNebulaConfValentin Bud | @valentinbud
  • 14.
    State Execution Behind thescenes salt minion state.sls opennebula-server response root@salt ~ # salt ‘minion’ state.sls opennebula-server 1 parse and execute 2 3 0MQ #OpenNebulaConfValentin Bud | @valentinbud
  • 15.
    SaltStack The Highstate root@salt ~# salt ‘*’ state.highstate states/top.sls base: ‘frontend’: - opennebula-server - opennebula-frontend #OpenNebulaConfValentin Bud | @valentinbud
  • 16.
    SaltStack The Render System •State data is just that - data • It does not need to be represented in YAML • State files can be rendered from any medium • Modular system #OpenNebulaConfValentin Bud | @valentinbud
  • 17.
    SaltStack Be a Ninjawith Jinja2 libvirt/init.sls: libvirt: pkg: - installed {% if grains[‘os’] == ‘Debian’ %} - name: libvirt-bin {% elif grains[‘os’] == ‘CentOS’ %} - name: libvirt {% endif %} #OpenNebulaConfValentin Bud | @valentinbud
  • 18.
    State Execution Behind thescenes with Jinja2 salt minion state.sls libvirt response root@salt ~ # salt ‘minion’ state.sls opennebula-server 1 parse execute 2 4 render 3 0MQ #OpenNebulaConfValentin Bud | @valentinbud
  • 19.
    SaltStack The Pillar Interface •Generate/store data for specific minions • Highly sensitive data • Minion configuration • Variables • Arbitrary data #OpenNebulaConfValentin Bud | @valentinbud
  • 20.
    OpenNebula oneadmin Password Storedin Pillar pillar/opennebula.sls: opennebula: oneadmin: password: SecurePassword state/oneadmin.sls: oneadmin_password: cmd: - run - name: oneuser passwd 0 {{ pillar[‘opennebula’][‘oneadmin’] [‘password’] }} #OpenNebulaConfValentin Bud | @valentinbud
  • 21.
    OpenNebula Node Communication TOPSECRET frontend node START VM VM STARTED NEEDS: SSH password-less communication Distribute Public Key to Nodes #OpenNebulaConfValentin Bud | @valentinbud
  • 22.
  • 23.
    SaltStack The Mine inStates state/oneadmin_ssh_auth.sls: oneadmin_ssh_auth: ssh_auth: - present - user: oneadmin - name: {{ salt['mine.get']('frontend', 'oneadmin_public_key.get') }} #OpenNebulaConfValentin Bud | @valentinbud
  • 24.
    SaltStack Peer Communication • Allowminion to “talk” to each other • Chatting done via Master • Must be enabled on the Master • Live data from other minions! #OpenNebulaConfValentin Bud | @valentinbud
  • 25.
    OpenNebula Node Up Createit on the Frontend salt node state.sls opennebula-node-kvm communicate frontend to create node 1 configure node 2 4 3 frontend createnode #OpenNebulaConfValentin Bud | @valentinbud
  • 26.
    Bootstrap OpenNebula Demo Cloud TheSaltStack Overstate overstate all network nfs-server storage frontend frontend nodes nodes 1 2 3 4 5 6 7 8 #OpenNebulaConfValentin Bud | @valentinbud
  • 27.
    Conclusions • Two simpletools together = POWER • Salt is fast • Both are easy to use • Salt can morph and grow together with your needs 2 #OpenNebulaConfValentin Bud | @valentinbud
  • 28.