Brian Brazil, founder of Ansible, discusses the use of Ansible for configuration management and his contributions to open source projects. He shares insights from his experience at the FOSDEM conference, emphasizing the importance of proper infrastructure setup, automating processes, and the need for a well-architected role layout in Ansible. Brazil also highlights key considerations such as module bugs, the quality of Galaxy roles, and ensuring idempotency in handlers.

1. Brian Brazil
Founder
Ansible at FOSDEM

2. Who am I?
Engineer passionate about running software reliably in production.
● CS degree from TCD
● Contributor to many open source projects, including Prometheus, Python,
Aurora and Zookeeper.
● Google SRE for 7 years, working on reliable systems and monitoring
● Founder of Robust Perception, providing commercial Prometheus support
and scaling consulting

3. The Ansible connection
Needed a configuration management system for Boxever.
Evaluated Puppet, Chef, Salt and Ansible - Ansible was clear winner.
Almost everything converted to ansible.
I use ansible at home.
Several contributions: Worked on “always” tag, monit module and wrote
dpkg_selections module.

4. FOSDEM
Open Source conference in Brussels.
Started in 2000, free to attend.
Thousands of attendees, live streaming, 10k+ devices on wifi.
Using a mix of existing Brussels university infrastructure and it’s own
In 2015 network was rushed, wanted to avoid that this year

5. 2016 Plan
● Get infrastructure setup well in advance
● Automate it and check it in
● Produce a “conference in a box”
RichiH was hanging around #prometheus, and recruited me and some others to
help setup both monitoring and infrastructure.

6. Starting off
Two weeks in advance we started hacking.
Had a basic working setup two days beforehand, including DNS, Prometheus,
SNMP monitoring, SSL, Docker etc.
Collaboration via Github.

7. At the event
Continued improving the system.
Adding more machines and services was pretty easy, rapid iteration.
Ran into a few issues.
Adding a 2nd completely different type of machine (video boxes, ARM) trickier.

8. Things to keep in mind: #1 Module bugs
If you run into a bug in a module, check the ansible issue tracker.
There’s probably already a patch there that you can grab.
You can put the fixed version in the ‘library’ directory while waiting for it to get into
a proper ansible release.
We had a problem with ‘unarchive’ and URLs.

9. Things to keep in mind: #2 Galaxy roles
The quality of Ansible Galaxy roles can vary substantially.
Can also complicate what would otherwise be 3-4 modules when done by hand.
Choose your Galaxy roles carefully.
I’d advise using them for reference (oh, that’s what that setting is called) and
maintaining your own consistent set of roles; rather than a mish-mash of Galaxy
roles you have to reverse engineer to understand.

10. Things to keep in mind: #3 Bootstrapping and Order
Make sure when you add new modules and roles that you have dependencies
before other modules that need them.
E.g. adding APT repos that require HTTPS before the relevant package has been
installed.
Make sure that your roles always work when applied to a fresh machine.
Have a “base” role that does the setup that’s the same on every machine.

11. Things to keep in mind: #4 Spurious handlers
Idempotency is a key value of ansible, if things are already correct don’t change
them.
Handlers that fire on every run are annoying, and slow you down a lot as you want
for things to restart/reload.
Also make you anxious - did you just push out some other change?
Avoid things like timestamps in files when using the ‘template’ module, so
handlers only fire if there’s real changes.

12. Things to keep in mind: #5 Missing notify
Make sure you have a notify on everything that changes how your services run.
Init files, systemd unit files, configuration files, upgraded deb/rpms, new binary
versions.
Having to restart things by hand is annoying.
Use flush_handlers if a subsequent module needs the change.

13. Summary
Ansible is great for managing machines and collaborating, and also very easy to
learn.
A well architected role, handler and variable layout is essential.
Consistency keeps you sane. Configuration is code.

14. Resources
FOSDEM: https://fosdem.org
Github: https://github.com/FOSDEM/infrastructure/tree/master/ansible
Talk: https://fosdem.org/2016/schedule/event/fosdem_infrastructure_review/
Robust Perception Website: www.robustperception.io
Ansible example: https://github.com/RobustPerception/demo_prometheus_ansible