Shomi's Journey to Chef: Lessons Learned on Implementing Chef
1. shomi's journey to chef: lessons
learned on implementing chef
Augusto Rosa
Head of Video & Network Infrastructure
Pierre Durrete
devops Engineer (aka chef master)
2. 2
• who is shomi_?
• shomi by numbers
• where we started
• evolution
• lessons learned
agenda
3. 3who is shomi_
shomi™ is a new kind of
instant streaming video
subscription service created
for entertainment lovers by
entertainment lovers that
makes discovering
entertainment nearly as
enjoyable as watching it.
5. 5
- Node.js, C# back end application, Angular front end.
- SOA archicture to microservice. 25 M.S. and counting
in 9 months.
- 29 billions requests in Q1 2016
- 400+ servers
- GB/s like traffic at peaks
- Petabytes of storage required
shomi behind the scene
6. 6
• Manual layout infrastructure at AWS and Physical (no automation)
• Servers are treated like pets, not cattle
• Implemented chef hosted on windows and linux
• No local testing
• No version locking
• Cookbooks not environment agnostic
• Attributes reside in multiple places (cookbooks, environments, roles)
• No naming conventions
• Forking of community cookbooks blocking us from upstream fixes
where we started
7. 7demo
• Environments dev, stage and production layout the same way. Same automation.
• Moved to berkshef cookbooks
• Run testkitchen locally
8. 8evolution
• Environments dev, stage and production layout the same way. Same automation.
• Use the Berkshelf Way, Environment Cookbook Pattern:
– Environment cookbook
– Application cookbook
– Wrapper cookbook
– Community cookbook
– shomi-base
• Environments per application (Environment Cookbook Pattern), including one application
cookbook per software repo
• Implement testkitchen. Run test locally first.
9. 9
Follow best practices
Environment Cookbook Pattern
Chef in a Continuous integration system
Version control and version locking
Limit powers of who can create environments, build, deploy
Chef hosted can be a challenge as compared to locally.
Feature comparison not the same
Not highly available
lessons learned
10. 10
Do you have questions?
We are hiring at the moment …
Developers (node.js, front end, C++, .net)
Devops Enginners(chef, terraform,)
Site Reliability Enginners (sumologic, Newrelic,
Datadog, trend analysis)
Video Processing Engineers (workflow developers)
open for questions