SlideShare a Scribd company logo
1 of 24
Download to read offline
EMERGING CHEF
PATTERNS AND
PRACTICES
Owain Perry
Owain Perry
• Software architect at the trainline.com
• perry@peek.org.uk
• github: perryofpeek.
• Focus at the moment is on build, deployment and the
software delivery machine.
TheTrainline
• £1.2+ Billion turnover.
• 2 transactions a second average
• ~1000 Physical and virtual servers
• Multiple development sites
• 25+ build and test (non production) environments
Using chef?
Using chef
• Using chef in various ways for the last 12 months
• Worked with a 3rd Party supplier who implemented some
aspects
• Multiple teams with various knowledge levels.
• These patterns / anti-patterns have emerged over the last
year in our code bases.
Anti-pattern
• Download and copy community cookbooks into one git
repository.
• Just change that bit… and diverge from the community…
• How do you update them?
• Promotes unintentional coupling between cookbooks.
• Hard to test
Antidote:
• use lots of small cookbooks.
Anti-pattern
• My chef code is the centre of the universe
• I am going to delete all the databags
• I am going to delete all the roles
• I am going to delete all the environments
• I am going to do this just before you showcase a whole
bunch of work to a team of people…
• Antidote:
• Hey you , you are not that important! Let’s live together in peace
and harmony
Anti-pattern
• Manual uploads to the chef server
• I am going to overwrite you!
• Antidote
Automate this….
Pattern
• Small cookbooks with one purpose
• Easy to manage
• Easy to test
• Easy to build.
• Easy to document
• Easy to understand
• This fits very well with git as a technology
Pattern
• Continuous integration builds of cookbooks
• Build script
• Knife cookbook test
• Food critic
• Chef-spec
• Mini test
• Vagrant test
• (why-run) manual testing.
• Knife cookbook upload
Pattern
• Roles in chef are cookbooks
• Supports versioning
• Easy to test
• Use different versions side by side
Pattern
• Cascading cookbooks
• Library cookbooks – do stuff with sane defaults
• Generalisation cookbooks
• Specialisation cookbooks.
• Example
go_server [sane defaults 1GB memory]
general_go_server [license key]  use this for templates
team_a_go_server [4GB memory, define site url, … ]
Pattern
• Try to make library cookbooks open source
• Forces abstraction of business implementation details
• Improves code quality
• Supporting reuse
• Sane defaults
• Might get some free testing and development
Pattern
• Continuous integration builds of the chef server changes.
• Avoiding using knife to change server state
• Databag uploads
• Roles uploads
• Nodes uploads
• Download latest versions of community cookbooks using librarian
and upload.
Pattern
• Use attributes as much as possible.
• Make cookbooks reusable
• Supports overriding attributes.
Pattern
• Make things idempotent, (but also fast!)
• Remote_file is idempotent BUT!
• It downloads the 100MB file and compares every 10 mins.
Pattern
• Develop Light Weight Resource Providers
• Code getting to long?
• Code with too much logic?
• Make it simple and develop a LWRP
• Abstract the complexity
• Easier to test the code using standard ruby frameworks
• Simpler to understand the cookbook consumer
Pattern
• Use vagrant
• Vagrant is just cool.
• Test on one or more platforms
• Fast feedback cycle while developing
• Repeatable
Pattern
• Using chef solo for automated builds of templates
• Pull down cookbooks
• Run chef solo
• Create a virtual machine using CI
• Running the generalised cookbooks
• Test template
• Convert machine to Vmware ESX template
• Upload template
• Provision template
• Run specific cookbook / role to give specific purpose
Anti-pattern
• Git submodules for cookbooks
• Some people don’t get git submodules
• Easy to break in CI
• Antidote
• Use librarian or berkshef to resolve dependencies
Anti-pattern
• DIY package management on windows implemented in
chef
• It’s not as simple as you think?
• Is chef really the right tool for this?
• How do you do upgrades?
• What versions are installed now?
• Antidote
• Choose an operating system with package management
• Failing that use chocolatey or similar on windows.
Anti pattern
• Hack community cookbooks if you don’t intend a pull
request
• How do we mange changes
• How do we merge changes in 6 months , 2 years from now….
• Antidote
• Wrap the cookbook and extend the wrapper.
Anti-Patten
• Testing, what testing….
• Antidote
• Write some tests….
• Practice Test Driven Development
• Make the feedback loop as fast as possible.
Questions?

More Related Content

What's hot

Ice breaker with dev ops
Ice breaker with dev opsIce breaker with dev ops
Ice breaker with dev opsMukta Aphale
 
Codecoon - A technical Case Study
Codecoon - A technical Case StudyCodecoon - A technical Case Study
Codecoon - A technical Case StudyMichael Lihs
 
Using Docker for Testing
Using Docker for TestingUsing Docker for Testing
Using Docker for TestingMukta Aphale
 
Infrastructure Automation with Chef
Infrastructure Automation with Chef Infrastructure Automation with Chef
Infrastructure Automation with Chef REAN Cloud
 
Chef onlinuxonpower
Chef onlinuxonpowerChef onlinuxonpower
Chef onlinuxonpowerMoya Brannan
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsMahesh Arali
 
Jenkins scaling best practices
Jenkins scaling best practicesJenkins scaling best practices
Jenkins scaling best practicesHenry McBride
 
Azure DevOps Overview [Arabic]
Azure DevOps Overview [Arabic]Azure DevOps Overview [Arabic]
Azure DevOps Overview [Arabic]ahmadezzeir
 
Building and Releasing a Golang CLI Tool
Building and Releasing a Golang CLI ToolBuilding and Releasing a Golang CLI Tool
Building and Releasing a Golang CLI ToolBradley Cypert
 
Vagrant for Effective DevOps Culture
Vagrant for Effective DevOps CultureVagrant for Effective DevOps Culture
Vagrant for Effective DevOps CultureVaidik Kapoor
 
Automated Infrastructure and Application Management
Automated Infrastructure and Application ManagementAutomated Infrastructure and Application Management
Automated Infrastructure and Application ManagementClark Everetts
 
Continuous Delivery and Infrastructure as Code
Continuous Delivery and Infrastructure as CodeContinuous Delivery and Infrastructure as Code
Continuous Delivery and Infrastructure as CodeSascha Möllering
 
Chef Delivery
Chef DeliveryChef Delivery
Chef DeliveryChef
 
Ceylon From Here to Infinity: The Big Picture and What's Coming
Ceylon From Here to Infinity: The Big Picture and What's Coming Ceylon From Here to Infinity: The Big Picture and What's Coming
Ceylon From Here to Infinity: The Big Picture and What's Coming Virtual JBoss User Group
 
Саша Белецкий "Continuous Delivery в продуктовой разработке"
Саша Белецкий "Continuous Delivery в продуктовой разработке"Саша Белецкий "Continuous Delivery в продуктовой разработке"
Саша Белецкий "Continuous Delivery в продуктовой разработке"Agile Base Camp
 
Migrating to git
Migrating to gitMigrating to git
Migrating to gitXpand IT
 

What's hot (20)

Ice breaker with dev ops
Ice breaker with dev opsIce breaker with dev ops
Ice breaker with dev ops
 
Codecoon - A technical Case Study
Codecoon - A technical Case StudyCodecoon - A technical Case Study
Codecoon - A technical Case Study
 
Using Docker for Testing
Using Docker for TestingUsing Docker for Testing
Using Docker for Testing
 
Infrastructure as Code
Infrastructure as CodeInfrastructure as Code
Infrastructure as Code
 
Infrastructure Automation with Chef
Infrastructure Automation with Chef Infrastructure Automation with Chef
Infrastructure Automation with Chef
 
Chef onlinuxonpower
Chef onlinuxonpowerChef onlinuxonpower
Chef onlinuxonpower
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teams
 
Jenkins scaling best practices
Jenkins scaling best practicesJenkins scaling best practices
Jenkins scaling best practices
 
Introduction to chef
Introduction to chefIntroduction to chef
Introduction to chef
 
Azure DevOps Overview [Arabic]
Azure DevOps Overview [Arabic]Azure DevOps Overview [Arabic]
Azure DevOps Overview [Arabic]
 
Building and Releasing a Golang CLI Tool
Building and Releasing a Golang CLI ToolBuilding and Releasing a Golang CLI Tool
Building and Releasing a Golang CLI Tool
 
Vagrant for Effective DevOps Culture
Vagrant for Effective DevOps CultureVagrant for Effective DevOps Culture
Vagrant for Effective DevOps Culture
 
CI for Chefs
CI for ChefsCI for Chefs
CI for Chefs
 
Automated Infrastructure and Application Management
Automated Infrastructure and Application ManagementAutomated Infrastructure and Application Management
Automated Infrastructure and Application Management
 
The Three Musketeers
The Three MusketeersThe Three Musketeers
The Three Musketeers
 
Continuous Delivery and Infrastructure as Code
Continuous Delivery and Infrastructure as CodeContinuous Delivery and Infrastructure as Code
Continuous Delivery and Infrastructure as Code
 
Chef Delivery
Chef DeliveryChef Delivery
Chef Delivery
 
Ceylon From Here to Infinity: The Big Picture and What's Coming
Ceylon From Here to Infinity: The Big Picture and What's Coming Ceylon From Here to Infinity: The Big Picture and What's Coming
Ceylon From Here to Infinity: The Big Picture and What's Coming
 
Саша Белецкий "Continuous Delivery в продуктовой разработке"
Саша Белецкий "Continuous Delivery в продуктовой разработке"Саша Белецкий "Continuous Delivery в продуктовой разработке"
Саша Белецкий "Continuous Delivery в продуктовой разработке"
 
Migrating to git
Migrating to gitMigrating to git
Migrating to git
 

Similar to Emerging chef patterns and practices

Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Controlindiver
 
Continuous Integration as a Way of Life
Continuous Integration as a Way of LifeContinuous Integration as a Way of Life
Continuous Integration as a Way of LifeMelissa Benua
 
The Art and Zen of Managing Nagios With Puppet
The Art and Zen of Managing Nagios With PuppetThe Art and Zen of Managing Nagios With Puppet
The Art and Zen of Managing Nagios With PuppetMike Merideth
 
Continuous Integration at Mollie
Continuous Integration at MollieContinuous Integration at Mollie
Continuous Integration at Molliewillemstuursma
 
MyHeritage - QA Automations in a Continuous Deployment environment
MyHeritage -  QA Automations in a Continuous Deployment environmentMyHeritage -  QA Automations in a Continuous Deployment environment
MyHeritage - QA Automations in a Continuous Deployment environmentMatanGoren
 
Arch9 - A cloud based continuous delivery implementation
Arch9 - A cloud based continuous delivery implementationArch9 - A cloud based continuous delivery implementation
Arch9 - A cloud based continuous delivery implementationPavel Chunyayev
 
Introduction to Cooking with Chef
Introduction to Cooking with ChefIntroduction to Cooking with Chef
Introduction to Cooking with ChefJohn Osborne
 
The Art & Zen of Managing Nagios with Puppet
The Art & Zen of Managing Nagios with PuppetThe Art & Zen of Managing Nagios with Puppet
The Art & Zen of Managing Nagios with PuppetVictorOps
 
OSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy HawkinsOSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy HawkinsNETWAYS
 
Why your company loves to welcome change but sucks at accommodating it
Why your company loves to welcome change but sucks at accommodating itWhy your company loves to welcome change but sucks at accommodating it
Why your company loves to welcome change but sucks at accommodating itFarooq Ali
 
Codifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared LibraryCodifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared LibraryAlvin Huang
 
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchAugust Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchHoward Greenberg
 
Compliance Automation with InSpec
Compliance Automation with InSpecCompliance Automation with InSpec
Compliance Automation with InSpec Nathen Harvey
 
Opscode Chef for Dummies
Opscode Chef for DummiesOpscode Chef for Dummies
Opscode Chef for Dummiesdilippanwar
 
Bringing DevOps to the Database
Bringing DevOps to the DatabaseBringing DevOps to the Database
Bringing DevOps to the DatabaseMichaela Murray
 
Development environments in seconds using Bitnami containers
Development environments in seconds using Bitnami containersDevelopment environments in seconds using Bitnami containers
Development environments in seconds using Bitnami containersAdnan Abdulhussein
 
Continuous Delivery with Chef and GoCD
Continuous Delivery with Chef and GoCDContinuous Delivery with Chef and GoCD
Continuous Delivery with Chef and GoCDAroj George
 
Chef Fundamentals Training Series Module 1: Overview of Chef
Chef Fundamentals Training Series Module 1: Overview of ChefChef Fundamentals Training Series Module 1: Overview of Chef
Chef Fundamentals Training Series Module 1: Overview of ChefChef Software, Inc.
 

Similar to Emerging chef patterns and practices (20)

Chef Cookbook Workflow
Chef Cookbook WorkflowChef Cookbook Workflow
Chef Cookbook Workflow
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Control
 
Continuous Integration as a Way of Life
Continuous Integration as a Way of LifeContinuous Integration as a Way of Life
Continuous Integration as a Way of Life
 
The Art and Zen of Managing Nagios With Puppet
The Art and Zen of Managing Nagios With PuppetThe Art and Zen of Managing Nagios With Puppet
The Art and Zen of Managing Nagios With Puppet
 
Continuous Integration at Mollie
Continuous Integration at MollieContinuous Integration at Mollie
Continuous Integration at Mollie
 
MyHeritage - QA Automations in a Continuous Deployment environment
MyHeritage -  QA Automations in a Continuous Deployment environmentMyHeritage -  QA Automations in a Continuous Deployment environment
MyHeritage - QA Automations in a Continuous Deployment environment
 
Arch9 - A cloud based continuous delivery implementation
Arch9 - A cloud based continuous delivery implementationArch9 - A cloud based continuous delivery implementation
Arch9 - A cloud based continuous delivery implementation
 
Introduction to Cooking with Chef
Introduction to Cooking with ChefIntroduction to Cooking with Chef
Introduction to Cooking with Chef
 
The Art & Zen of Managing Nagios with Puppet
The Art & Zen of Managing Nagios with PuppetThe Art & Zen of Managing Nagios with Puppet
The Art & Zen of Managing Nagios with Puppet
 
OSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy HawkinsOSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy Hawkins
 
Why your company loves to welcome change but sucks at accommodating it
Why your company loves to welcome change but sucks at accommodating itWhy your company loves to welcome change but sucks at accommodating it
Why your company loves to welcome change but sucks at accommodating it
 
Codifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared LibraryCodifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared Library
 
Hacking the way you work
Hacking the way you workHacking the way you work
Hacking the way you work
 
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchAugust Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
 
Compliance Automation with InSpec
Compliance Automation with InSpecCompliance Automation with InSpec
Compliance Automation with InSpec
 
Opscode Chef for Dummies
Opscode Chef for DummiesOpscode Chef for Dummies
Opscode Chef for Dummies
 
Bringing DevOps to the Database
Bringing DevOps to the DatabaseBringing DevOps to the Database
Bringing DevOps to the Database
 
Development environments in seconds using Bitnami containers
Development environments in seconds using Bitnami containersDevelopment environments in seconds using Bitnami containers
Development environments in seconds using Bitnami containers
 
Continuous Delivery with Chef and GoCD
Continuous Delivery with Chef and GoCDContinuous Delivery with Chef and GoCD
Continuous Delivery with Chef and GoCD
 
Chef Fundamentals Training Series Module 1: Overview of Chef
Chef Fundamentals Training Series Module 1: Overview of ChefChef Fundamentals Training Series Module 1: Overview of Chef
Chef Fundamentals Training Series Module 1: Overview of Chef
 

Emerging chef patterns and practices

  • 2. Owain Perry • Software architect at the trainline.com • perry@peek.org.uk • github: perryofpeek. • Focus at the moment is on build, deployment and the software delivery machine.
  • 3. TheTrainline • £1.2+ Billion turnover. • 2 transactions a second average • ~1000 Physical and virtual servers • Multiple development sites • 25+ build and test (non production) environments
  • 5. Using chef • Using chef in various ways for the last 12 months • Worked with a 3rd Party supplier who implemented some aspects • Multiple teams with various knowledge levels. • These patterns / anti-patterns have emerged over the last year in our code bases.
  • 6. Anti-pattern • Download and copy community cookbooks into one git repository. • Just change that bit… and diverge from the community… • How do you update them? • Promotes unintentional coupling between cookbooks. • Hard to test Antidote: • use lots of small cookbooks.
  • 7. Anti-pattern • My chef code is the centre of the universe • I am going to delete all the databags • I am going to delete all the roles • I am going to delete all the environments • I am going to do this just before you showcase a whole bunch of work to a team of people… • Antidote: • Hey you , you are not that important! Let’s live together in peace and harmony
  • 8. Anti-pattern • Manual uploads to the chef server • I am going to overwrite you! • Antidote Automate this….
  • 9. Pattern • Small cookbooks with one purpose • Easy to manage • Easy to test • Easy to build. • Easy to document • Easy to understand • This fits very well with git as a technology
  • 10. Pattern • Continuous integration builds of cookbooks • Build script • Knife cookbook test • Food critic • Chef-spec • Mini test • Vagrant test • (why-run) manual testing. • Knife cookbook upload
  • 11. Pattern • Roles in chef are cookbooks • Supports versioning • Easy to test • Use different versions side by side
  • 12. Pattern • Cascading cookbooks • Library cookbooks – do stuff with sane defaults • Generalisation cookbooks • Specialisation cookbooks. • Example go_server [sane defaults 1GB memory] general_go_server [license key]  use this for templates team_a_go_server [4GB memory, define site url, … ]
  • 13. Pattern • Try to make library cookbooks open source • Forces abstraction of business implementation details • Improves code quality • Supporting reuse • Sane defaults • Might get some free testing and development
  • 14. Pattern • Continuous integration builds of the chef server changes. • Avoiding using knife to change server state • Databag uploads • Roles uploads • Nodes uploads • Download latest versions of community cookbooks using librarian and upload.
  • 15. Pattern • Use attributes as much as possible. • Make cookbooks reusable • Supports overriding attributes.
  • 16. Pattern • Make things idempotent, (but also fast!) • Remote_file is idempotent BUT! • It downloads the 100MB file and compares every 10 mins.
  • 17. Pattern • Develop Light Weight Resource Providers • Code getting to long? • Code with too much logic? • Make it simple and develop a LWRP • Abstract the complexity • Easier to test the code using standard ruby frameworks • Simpler to understand the cookbook consumer
  • 18. Pattern • Use vagrant • Vagrant is just cool. • Test on one or more platforms • Fast feedback cycle while developing • Repeatable
  • 19. Pattern • Using chef solo for automated builds of templates • Pull down cookbooks • Run chef solo • Create a virtual machine using CI • Running the generalised cookbooks • Test template • Convert machine to Vmware ESX template • Upload template • Provision template • Run specific cookbook / role to give specific purpose
  • 20. Anti-pattern • Git submodules for cookbooks • Some people don’t get git submodules • Easy to break in CI • Antidote • Use librarian or berkshef to resolve dependencies
  • 21. Anti-pattern • DIY package management on windows implemented in chef • It’s not as simple as you think? • Is chef really the right tool for this? • How do you do upgrades? • What versions are installed now? • Antidote • Choose an operating system with package management • Failing that use chocolatey or similar on windows.
  • 22. Anti pattern • Hack community cookbooks if you don’t intend a pull request • How do we mange changes • How do we merge changes in 6 months , 2 years from now…. • Antidote • Wrap the cookbook and extend the wrapper.
  • 23. Anti-Patten • Testing, what testing…. • Antidote • Write some tests…. • Practice Test Driven Development • Make the feedback loop as fast as possible.

Editor's Notes

  1. Roles are nasty global
  2. Early on , tried using snapshots on VMWare, took ages to roll back the snapshot.Some bright spark deleted all the snapshots.
  3. Chocolatey Chocolatey provider – supporting why run.