Source DevOps stack
Or, how to quit managing the infrastructure to
manage your actual infrastructure
CTO at OlinData since 2008
Certified Puppet Instructor since pre 1.0 days
Background: MySQL DBA, Delphi programmer, FOSS Lover
Almost 15 years of professional IT experience
● With the arrival of a new stack of tools comes a ton of time that needs to be
invested in managing a stack of tools to manage our infrastructure.
○ When can we start doing what we set out to do, manage those web/database/whatever
servers the business needs?
● A lot of infrastructure management is (should) be at the very least similar
across organisations given that best practices have been adopted.
○ Why should your OS monitoring be different from mine?
○ We all want centralised logging/monitoring/logrotation/etc.
A tool is good. Two tools is (probably) better. Two well integrated tools are best.
OpsTheater brings together some fundamental pillars that should be present in
every operations environment:
● Config management (puppet)
● Version control & CI/CD (GitLab / GitLab CI)
● Server Monitoring (Icinga)
● Centralised logging (ELK stack)
● ChatOps (mattermost)
We are all experts (?) in this field, why not share our knowledge?
Log Storage / Analysis
Provision Instances / Deploy Virtual Machines
Triggers automated builds in
Retrieves source code
Physical InfraPublic CloudPrivate Cloud Virtual Infra
● A combination of publically acknowledged best practices open source
software glued together using puppet.
○ Built by a community of infra engineers with a common interest
● An abstraction layer that implements all things that are non-specific but
essential to an organisation’s infrastructure
○ Eg. when new puppet code is pushed to GitLab it needs to be tested by GitLab CI, a
notification should go to a chat room and the code needs to be deployed if all lights are green
○ Eg. a grafana dashboard should show all puppet runs and mark all failed ones
○ Eg. when diskspace on any server falls below 5% it needs to alert
○ Eg. when a server is managed through puppet it needs to have filebeat and a monitoring
Who is this for? And who is it not for?
● Infrastructures with 10+ servers
● Environments ideally already using one or
more of these tools
● New environments
● Ops teams with skills in one or more of the
● Non-puppet environments (Puppet is the
glue between the components)
● Pure-sang cloud environments like AWS
(anywhere where puppet is not ideal)
● Very small infras (the overhead of 5 nodes
OpsTheater becomes too much)
● Fully implemented and automated
environments (retrofitting is a b*tch)
Current state of OpsTheater
Just released version 2.0.0 this week (hooray!)
● Easier to get started (`git clone && vagrant up`)
● Updated all tools and puppet modules to latest stable
○ ELK stack 5, Grafana 4.1, Puppet 4.9, Foreman 1.13, Icinga 2.6, GitLab 8.16
● Foreman Smart Parameters instead of hiera
● SMTP, LDAP working
In short, it’s the first version with basic integrations of all tools that makes it easy
for people to explore and contribute
The road ahead
● Deeper integration between tools
○ Automatic mattermost notifications for certain actions
○ ChatBots to interact with different parts of OpsTheater
○ Pre-compiled grafana dashboards
○ Automatically properly monitor servers
● Package management
○ Self-hosting versions of puppet modules
○ Pinning package versions
● Easier deployment
● Security scans
● Automatic backup integration?
How can you help?
● Try it out
○ If you see problems, things that are unclear -> submit issues/PR’s in
● Add functionality
○ Complete terraform/packer stuff
○ Make switching of eg. monitoring tool or centralised logging possible
● Spread the word (@opstheater)!
● Build us a website :)