Downloaded 21 times
Uploaded byPuppet
Puppet Camp Sydney 2015: Sneak peek into the Puppet environment at Fairfax Media
This document summarizes the puppet environment at Fairfax Media/Domain Group. It describes their history with puppet and lessons learned, including not making direct changes to the puppet master and defining a standard workflow. It outlines their current architecture which uses tools like Packer, Puppetlabs R10k, and a custom Ruby service called Captain Hook to deploy changes across a high availability puppet master infrastructure. It also describes their process for standardizing modules and using a Git feature branch workflow and CloudFormation for infrastructure automation.
More Related Content
Similar to Puppet Camp Sydney 2015: Sneak peek into the Puppet environment at Fairfax Media
Puppet Camp Sydney 2015: Sneak peek into the Puppet environment at Fairfax Media
- 1. SNEAK PEEK The PuppetEnvironment at Fairfax Media / Domain Group Dejan Golja // Gonzalo Servat
- 2. Introduction Dejan Golja Senior SystemsEngineer // Fairfax Media Twitter @dejangolja
- 3. Introduction Gonzalo Servat Senior DevOpsEngineer // Domain Group Twitter @gservat
- 4.
- 5. Our Puppet History ☁Early Puppet adopters … since version 0.25.X ☁ Large scale environment with distributed team ☁ We made every possible mistake We’re on our 3rd major Puppet implementation!
- 6. Lessons Learned ☁ DoNOT make changes directly on the Puppet Master ☁ DO your testing - lint, code validation, etc. ☁ Do NOT pollute your downloaded official modules ☁ DO define and document the standard workflow ☁ Do NOT store sensitive data into modules
- 7. Example of ugliness(old site.pp)
- 8. Standardised process for deploying changes Easily repeatable formultiple environments Puppet “V3” Requirements High Availability Bring order to our modules
- 9. Ingredients ☁ Packer ☁ PuppetlabsR10k ☁ Custom Ruby Service - “Captain Hook” ☁ Notification Systems - Slack/Flowdock EC2 ASG ELB SNS SQS RDS CloudFormation
- 10. ☁ Module preferences ☁Puppetlabs supported ☁ Vendor supported ☁ Puppet approved ☁ Custom modules (role/profile) ☁ Create module guidelines else it gets very messy! ☁ Each module lives in a separate Git repository Puppet Modules
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18. $ aws cloudformationcreate-stack --stack-name puppetmaster-staging --templatebody file://infrastructure-puppetmaster.template --parameters ParameterKey=Environment,ParameterValue=staging ParameterKey=InstanceType,ParameterValue=m3.large ParameterKey=MinScalingGroupSize,ParameterValue=2 ParameterKey=MaxScalingGroupSize,ParameterValue=4 CloudFormation Example
- 19. Awesome, so wehave a neat-looking HA/FT set-up… …but how do we manage pushing changes to Puppet Masters?!
- 20. ☁ Puppet masterscan “come and go” randomly ☁ Keep the modules up to date per environment ☁ K.I.S.S. - Tame the learning curve for the team ☁ Avoid reinventing the wheel Challenges
- 21.
- 22.
- 23. 1. Make changesand commit/push to git server 2. Git server triggers post-commit hook (POST) to Captain Hook server 3. Captain Hook server validates the payload & creates new message: ☁ Full refresh ➩ r10k deploy environment -p ☁ Light refresh ➩ r10k deploy environment ☁ Module refresh ➩ r10k deploy module <name> 4. Captain hook server pushes new message to SNS 5. Captain hook client polls & reads messages ( ➩ r10k) 6. … and we get notified in slack/flowdock/dashboard Typical Puppet Workflow
- 24.
- 25. ☁ Fairfax isa large scale complex environment ☁ Many systems engineers are constantly pushing changes ☁ Puppet architecture designed for HA and fault tolerance ☁ Puppet workflow helps us deploy changes to multiple masters safely and easily Summary
- 27. Our Blogs http://engineering.fairfaxmedia.com.au/ http://tech.domain.com.au/ Our Code http://github.com/fairfaxmedia YourFeedback or Questions You can contact us on twitter: @dejangolja @gservat Before you go ...