Growing Pains with Chef – a Tale of DevOps in a Large Organization
Growing Pains with Chef a tale of DevOps in a large organiza7on
Disclaimer• All opinions stated are those of the presenter and do not necessarily reﬂect those of Turner or any of its aﬃliates or partners.
About @bakins • Horrible public speaker• Born and raised in Alabama• Husband and father of four• Senior Principal Architect, Turner Broadcas7ng• “Large” web site and systems opera7ons• C and Lua hacker• Learning Go, Erlang, and MIG welding
Thank you• Thanks to the Chef community• Mailing list, IRC, github, etc• You make my life easier4
About Turner MPTO• Media PlaRorm Technology & Opera7ons• Turner’s “web people”– CNN, NCAA, NBA, Adultswim, Cartoon Network, Money, iReport, etc• Five engineering groups• One opera7ons group
Open source chef to private chefMigra7onEarly Private Chef Customer
"One" Datacenter to many• mul7ple Chef servers– One cluster per datacenter• keeping cookbooks, etc, in sync• Search wrapper• East/West Coast datacenters plus public cloud• Applica7on resiliency• S7ll a work in progress– How to properly model in Chef?
One Big Chef Repo• "chef-‐main”• Evolu7on of cookbook organiza7on– knife cookbook site vendor – braid– Chef-‐Librarian– Berkshelf• Cookbook Chef Organiza7on
Heterogeneous Business Units• “chef-‐main” again• Vastly diﬀerent business units and dev teams• Chef Organiza7on per business unit and/or “large” site– “limit the blast radius”– Support mul7ple works ﬂows• Work ﬂow issues– Varying level of comfort with toolset– number of moving parts
Cookbook development workﬂow• Was: hack, upload, chef-‐client, repeat• Berkshelf+Vagrant and local development• Cookbook Chef Organiza7on • Build server integra7on for mul7ple orgs– foodcri7c– Unit/Integra7on tests
Everyone should write cookbooks• Well, maybe not everyone• Can be a distrac7on from wri7ng actual code• Not everyone wants to write cookbooks• Liale consistency in style, quality, interoperability, etc• Rapidly changing stack: Berkshelf, Vagrant, EC2, etc• Who “owns” a cookbook? Who should?11
Roles and environments• Confusing work ﬂows• Lack of versions• Everything in a cookbook– wrapper cookbooks– One item per node run list• Build server integra7on
I can’t be alone• How are other large “enterprise” companies doing this?• Community seems oriented toward small and/or homogeneous users• We’re hiring. Work remote.13