How we migrated the labs of two Italian schools from Windows XP to Linux.

1. Salt at school
by Flavio Castelli & Silvio Moioli

2. Agenda
• Motivation
• Requirements & current status
• Deployment process
• Salt primer
• Benefits
• Salt’s Good, Bad and Ugly

3. Motivation
• Windows XP EOL ⇒ machine EOL
• Lots of machines need replacement
• Public school has budget problems!
• Linux is the (only) way out

4. Requirements
• Edubuntu
• Additional educational open source software
• Must run on available hardware

5. Requirements
• Two schools’ labs:
• Torre Boldone, Flavio’s hometown, 30+ PCs
• Mozzo, Silvio’s hometown, 15+ PCs
• Scarce manpower: Flavio, Silvio and two others
(unexperienced and in Torre Boldone only)

6. Current status
• Mozzo: 13 clients and 1 server migrated, in use
• Torre Boldone: 15 clients migrated, 15 to come

7. Deployment process
• Server: copy of a VM on a host
• Clients:
• Stage 1: PXE boot and base installation
• Stage 2: proper configuration with Salt

8. Stage 1 (installer)
• Setup storage: disk wiping and partitioning
• Creation of unique and persistent hostname
• Minimal package installation: text only, sshd,
salt-minion

9. Stage 2 (Salt)
• Turn Ubuntu into Edubuntu
• Install additional software
• Apply ad hoc configurations:
• reconfigure repo mirror (to local server)
• use lightdm/GNOME 2 as default WM
• user account creation, automatic login
• ntp client

10. DHCP TFTP HTTP ØMQ BIOS
DISCOVER
DHCP server
dnsmasq
BIOS
OFFER (IP, DNS, TFTP server
name)
DHCP server
dnsmasq
BIOS
RRQ
TFTP server
dnsmasq
BIOS
DATA (image ⊃ kernel options ⊃
kickstart and preseed URL)
TFTP server
dnsmasq
salt-minion daemon
hostname, salt key
salt-master deamon
salt-minion daemon
salt commands
salt-master deamon
salt-minion daemon
salt grains
salt-master deamon
Installer
kickstart, preseed, package
requests/responses
HTTP server
Apache httpd
kickstart post-install
script
HTTP request (I am be:ef:ba:be:
00:01)
mac2address
Go app
kickstart post-install
script
HTTP response (I baptize you
lab12)
mac2address
Go app

11. DHCP TFTP BIOS
DISCOVER
DHCP server
dnsmasq
BIOS
OFFER (IP, DNS, TFTP server
name)
DHCP server
dnsmasq
BIOS
RRQ
TFTP server
dnsmasq
BIOS
DATA (image ⊃ kernel options ⊃
kickstart and preseed URL)
TFTP server
dnsmasq

12. TFTP HTTP BIOS
DATA (image ⊃ kernel options ⊃
kickstart and preseed URL)
TFTP server
dnsmasq
salt-minion daemon
hostname, salt key
salt-master deamon
Installer
kickstart, preseed, package
requests/responses
HTTP server
Apache httpd
kickstart post-install
script
HTTP request (I am be:ef:ba:be:
00:01)
mac2address
Go app
kickstart post-install
script
HTTP response (I baptize you
lab12)
mac2address
Go app

13. HTTP ØMQ
salt-minion daemon
hostname, salt key
salt-master deamon
salt-minion daemon
salt commands
salt-master deamon
salt-minion daemon
salt grains
salt-master deamon
HTTP request (I am be:ef:ba:be:
00:01)
Go app
kickstart post-install
script
HTTP response (I baptize you
lab12)
mac2address
Go app

14. Server: nontrivial!
• dnsmasq, tftpd, ntpd, sshd, httpd,
mac2hostname!
• APT package mirror
• Salt master
• …also self-managed with Salt!

15. Salt primer
RAET
(master) (message bus) (minions)

16. Salt primer
• salt-master and salt-minon are daemons
written in Python
• ØMQ is written in C++ with bindings
• Salt implements strong crypto and
authentication on top of ØMQ

17. Salt State (SLS) Modules
• Represent a state in which a system should be in
• Composed by State Declarations
• Text files ending with sls extension
• YAML files
• Templates (default Jinja2, others available)
• Pure Python code

18. State Declarations
• Define of “how an aspect of a minion should be”
• Implemented as calls to State Functions
• Every Declaration has an ID

19. State Functions
• Code that can bring a minion to a specific state
• Examples: pkg.installed,
service.running, file.managed…
• Grouped into modules
• A library of modules is available

20. File example
lightdm_custom_conf_file: # ID!
file: # State Module name!
- managed # State Function name!
- source: salt://lightdm/lightdm.conf!
- name: /etc/lightdm/lightdm.conf.d/ic_torre_boldone.conf!
- user: root!
- group: root!
- mode: 644!
- require:!
- file: lightdm_custom_conf_dir!

21. Service example
lightdm:!
pkg:!
- installed!
service:!
- running!
- enable: True!
- watch:!
- file: lightdm_custom_conf_file!
- require:!
- pkg: lightdm!

22. top.sls
• special State Module that assigns other State Modules
to minions
• can be used to define environments
• Minions can be matched using:
• Regular expressions
• Compound matches: grains, subnet/IP, range cluster
• Boolean operators available

23. top.sls
base:!
'lab*':!
- lightdm!
- software!
- ntpdate!
- users!
'school-server*':!
- apache!
- apt-mirror!
- dnsmasq!
…

24. High State
• special State compiled by Salt by applying all
relevant State Modules
• Force minions to high state:
sudo salt state.highstate lab*

25. Data in Salt
• Salt Grains: information from minions
• Salt Pillars: user-defined data
• can be YAML or templates
• has ACLs, eg. for credentials
• Plain file serving

26. Templated definition with
Pillar Example
{% for symlink_id in pillar.get('symlinks', {}).keys() %}!
{{symlink_id}}_apache_link:!
file.symlink:!
- name: {{pillar.get('symlinks')[symlink_id]['name']}}!
- target: {{pillar.get('symlinks')[symlink_id]['target']}}!
- force: True!
- require:!
- pkg: apache!
{% endfor %}

27. Templated definition with
Pillar Example
extra_apache_link:!
file.symlink:!
- name: /var/www/html/extras!
- target: /var/spool/…/ubuntu/!
- force: True!
- require:!
- pkg: apache

28. Accessing ØMQ directly
• We want to power off machines at the end of
Stage 2
• Not easy to express declaratively
• More of a “one time command”

29. Accessing ØMQ directly
• Solution:
• Subscribe to ØMQ
• Look for "highstate successfully completed"
announcements
• Send a "shutdown yourself" message to the
publisher

30. Accessing ØMQ directly
• Easy to implement:
• Official Salt Python module has full access to
ØMQ
• Salt messages are easy to understand
• 76 LOC Python tool (with comments and
formatting)

31. Benefits (schools)
• No more licensing issues
• Free updates for the next 4 years
• No need for hardware changes

32. Benefits (admins)
• Easy to replicate changes across PCs
• Easy to enforce a desired state
• Easy to reinstall a PC from scratch
• GitHub-based configuration!

33. Future work
• Automatically accept all minion keys
• Automatically force the High State on new
minions
• Look into testing frameworks
• …solve “production” issues!

34. The Good
• Simple architecture: Python almost everywhere
• Easy to setup both on the master and on the
minion
• Can trigger execution of system commands on
the minion
• ØMQ can be used to extend it
• Good docs and source code

35. The Bad
• Still in its early days
• Limited amount of existing modules
• Limited feedback while executing states

36. The Ugly
• No Ugly so far!

37. Resources
• SaltStack project: http://www.saltstack.com/
• Official documentation: http://docs.saltstack.com/
• Our Salt files:https://github.com/ic-torre-boldone/salt

38. Questions?

39. Thanks for your attention!