Spotify has over 20 million monthly active users and launched in 17 countries with over 20 million songs and more than 1 billion playlists. Spotify uses Puppet to manage roughly 3500 nodes across 3 Puppet installations, each with their own PuppetDB and certificate authority. Puppet has been used at Spotify for 2.5 years, with more than 300 changes per month contributed by 137 committers to their Puppet repository, which includes 240 Puppet modules. The infrastructure uses Git branches and Puppet environments to manage changes and will switch from various node classification methods to using Hiera.

1. Puppet at Spotify
Erik Dalén
February 1, 2013

2. Spotify
• Over 20 million monthly active users
• Launched in 17 countries
• Over 20 million songs
• More than 1 billion playlists
Growing quickly

3. Puppet Infrastructure
• Rougly 3500 nodes
• 3 different Puppet installations, each with
their own PuppetDB and CA
• One puppetmaster per data centre

4. Puppet Infrastructure
Puppet users since 2.5 years
• More than 300 changes per month
• 137 committers to our Puppet repository
• 240 puppet modules
Code review by operations using Gerrit

5. Puppet Infrastructure
Git branch
==
Puppet environment

6. Node Classification
At the moment using different ENCs in different
Puppet installations
Will be switching to using hiera to classify
nodes

7. Node Classification
Hostname standard:
sto1-puppet-a1
<site>-<service>-<pool><poolid>
Mainly service fact used to determine the
classes a machine should have

8. Service discovery
Using SRV records to discover services

9. Service discovery
wasd: github.com/spotify/wasd
Ingests Cassandra rings and DNS records across
multiple sites at a regular interval, then exposes
information about hosts and services.
One can ask what a host does, optionally for which sites;
which hosts in a site offer a service; which hosts offer
a service for a site.
Using a REST API.

10. Questions?