1. When your configurationWhen your configuration
management is too slow tomanagement is too slow to
reconfigure your infrareconfigure your infra
Kris Buytaert
@krisbuytaert

2. KrisKris BuytaertBuytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source
Consultant @inuits.euConsultant @inuits.eu
● Everything is an effing DNS ProblemEverything is an effing DNS Problem
● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore
● Organising too many confs , #devopsdays,Organising too many confs , #devopsdays,
#loadays, #cfgmgmtcamp ...#loadays, #cfgmgmtcamp ...
● Evangelizing devopsEvangelizing devops

3. #devops=~C(L)AMS#devops=~C(L)AMS
● CultureCulture
● (Lean)(Lean)
● AutomationAutomation
● Monitoring and MeasurementMonitoring and Measurement
● SharingSharing
Damon Edwards and John WillisDamon Edwards and John Willis
Gene KimGene Kim

4. Infrastructure as CodeInfrastructure as Code

5. Infrastructure as CodeInfrastructure as Code
● Treat configuration automation as codeTreat configuration automation as code
● Development best practicesDevelopment best practices
•
Model your infrastructureModel your infrastructure
•
Version your cookbooks / manifestsVersion your cookbooks / manifests
•
Test your cookbooks/ manifestsTest your cookbooks/ manifests
•
Dev/ test /uat / prod for your infraDev/ test /uat / prod for your infra
● Model your infrastructureModel your infrastructure
● A working service = automated ( Application Code + Infrastructure Code + Security +A working service = automated ( Application Code + Infrastructure Code + Security +
Monitoring )Monitoring )
● IAC -ne scriptingIAC -ne scripting

6. 3 types of files3 types of files
● Part of a packagePart of a package
•
rpm -qfrpm -qf
● Managed by configuration management toolsManaged by configuration management tools
•
““### This file is managed by ..”### This file is managed by ..”
● ““User” generated dataUser” generated data
•
Actual user dataActual user data
•
Logs ..Logs ..

7. Idempodent InfrastructureIdempodent Infrastructure
● Configuration management ensures stateConfiguration management ensures state
● Periodically,Periodically,
● Repeatedly,Repeatedly,
● Manual changes are revertedManual changes are reverted

8. Puppet for CfgMgmtPuppet for CfgMgmt
● Puppet is opensource,Puppet is opensource,
● Puppet makes your infra idempodent,Puppet makes your infra idempodent,
● Puppet scales,Puppet scales,
● Puppet runs every X minutes, orPuppet runs every X minutes, or
● Puppet is triggered on demandPuppet is triggered on demand
● A puppetrun takes between 3s and 10minA puppetrun takes between 3s and 10min

9. Typical ArchTypical Arch
● Puppet masterPuppet master
● Puppet agentsPuppet agents
● PuppetdbPuppetdb

10. Stored Configs andStored Configs and
Exported resourcesExported resources

11. Sometimes,Sometimes,
● That's not fast enough.That's not fast enough.
•
Think adding new nodes to a loadbalancedThink adding new nodes to a loadbalanced
setupsetup
•
Removing nodes from a load balanced setupRemoving nodes from a load balanced setup
•
Exported Resource + Collected resourceExported Resource + Collected resource
roundtrip is to slowroundtrip is to slow

12. Sometimes,Sometimes,
● That's not enough.That's not enough.
•
Containrz spinning up faster than nodesContainrz spinning up faster than nodes
•
AWS, when you don't want AWS LockInAWS, when you don't want AWS LockIn

13. CattleCattle

14. ““In a not so distant future”In a not so distant future”
● People will build small minimal functionPeople will build small minimal function
applications that constantly talk to eachotherapplications that constantly talk to eachother
● They will want to know where the otherThey will want to know where the other
applications liveapplications live
● They will want to register their applications andThey will want to register their applications and
broadcast that to their peersbroadcast that to their peers

15. So you want service registration of your healthySo you want service registration of your healthy
(containerized) applications ?(containerized) applications ?

16. Consul.ioConsul.io

17. Consul.ioConsul.io
●
““Service discovery”Service discovery”
●
Failure detectionFailure detection
●
Using Gossip build on top of SerfUsing Gossip build on top of Serf
●
Random node 2 node communicationRandom node 2 node communication
●
A HashiCorp projectA HashiCorp project

18. Setting upSetting up

19. List membersList members

20. Defining a ServiceDefining a Service

21. Query ServicesQuery Services

22. YamaYama
● Yet another Mcollective AlternativeYet another Mcollective Alternative

23. git@github.com:solarkennedy/puppet-consul.gitgit@github.com:solarkennedy/puppet-consul.git
Puppet-consulPuppet-consul

24. Consul_templateConsul_template
● Dynamically refils your templates based onDynamically refils your templates based on
changes in consulchanges in consul
● Performs actions you definePerforms actions you define
● Go Template formatGo Template format

25. Puppet-consul_templatePuppet-consul_template
● git@github.com:KrisBuytaert/puppet-git@github.com:KrisBuytaert/puppet-
consul_template.gitconsul_template.git
● Might be merged into the puppet-consulMight be merged into the puppet-consul
modulemodule

26. PuppetizePuppetize

27. Live demoLive demo

28. Live DemoLive Demo
● 3 nodes3 nodes
● 1x Nginx template rewritten by consul1x Nginx template rewritten by consul
● 2x apache2x apache
● Monitoring_plugins based checkMonitoring_plugins based check
● Vagrantproject :Vagrantproject :

29. Other use casesOther use cases
● Monitoring + AlertingMonitoring + Alerting
● Service registrationService registration

30. Other approachesOther approaches
● HaProxy ?HaProxy ?
● LinuxHA / Corosync / .... ?LinuxHA / Corosync / .... ?
● ZookeeperZookeeper
● EtcdEtcd
●

31. Remember idempotenceRemember idempotence
● Consul seemed instableConsul seemed instable
● Puppet using old ruby json libPuppet using old ruby json lib
● Writing of json config was not idempodentWriting of json config was not idempodent
●
● Reconfiguring consul every single timeReconfiguring consul every single time
● Fixed now.Fixed now.

32. ContactContact
Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be
Further ReadingFurther Reading
@krisbuytaert@krisbuytaert
http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/
http://www.inuits.be/http://www.inuits.be/
InuitsInuits
Duboistraat 50Duboistraat 50
2060 Antwerpen2060 Antwerpen
BelgiumBelgium
891.514.231891.514.231
+32 475 961221+32 475 961221