SlideShare a Scribd company logo
1 of 10
shomi's journey to chef: lessons
learned on implementing chef
Augusto Rosa
Head of Video & Network Infrastructure
Pierre Durrete
devops Engineer (aka chef master)
2
• who is shomi_?
• shomi by numbers
• where we started
• evolution
• lessons learned
agenda
3who is shomi_
shomi™ is a new kind of
instant streaming video
subscription service created
for entertainment lovers by
entertainment lovers that
makes discovering
entertainment nearly as
enjoyable as watching it.
4who is shomi_
5
- Node.js, C# back end application, Angular front end.
- SOA archicture to microservice. 25 M.S. and counting
in 9 months.
- 29 billions requests in Q1 2016
- 400+ servers
- GB/s like traffic at peaks
- Petabytes of storage required
shomi behind the scene
6
• Manual layout infrastructure at AWS and Physical (no automation)
• Servers are treated like pets, not cattle
• Implemented chef hosted on windows and linux
• No local testing
• No version locking
• Cookbooks not environment agnostic
• Attributes reside in multiple places (cookbooks, environments, roles)
• No naming conventions
• Forking of community cookbooks blocking us from upstream fixes
where we started
7demo
• Environments dev, stage and production layout the same way. Same automation.
• Moved to berkshef cookbooks
• Run testkitchen locally
8evolution
• Environments dev, stage and production layout the same way. Same automation.
• Use the Berkshelf Way, Environment Cookbook Pattern:
– Environment cookbook
– Application cookbook
– Wrapper cookbook
– Community cookbook
– shomi-base
• Environments per application (Environment Cookbook Pattern), including one application
cookbook per software repo
• Implement testkitchen. Run test locally first.
9
 Follow best practices
Environment Cookbook Pattern
Chef in a Continuous integration system
Version control and version locking
 Limit powers of who can create environments, build, deploy
 Chef hosted can be a challenge as compared to locally.
Feature comparison not the same
Not highly available
lessons learned
10
Do you have questions?
We are hiring at the moment …
Developers (node.js, front end, C++, .net)
Devops Enginners(chef, terraform,)
Site Reliability Enginners (sumologic, Newrelic,
Datadog, trend analysis)
Video Processing Engineers (workflow developers)
open for questions

More Related Content

Similar to Shomi's Journey to Chef: Lessons Learned on Implementing Chef

Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationNick Josevski
 
Automated Visual Regression Testing by Dave Sadlon
Automated Visual Regression Testing by Dave SadlonAutomated Visual Regression Testing by Dave Sadlon
Automated Visual Regression Testing by Dave SadlonQA or the Highway
 
DockerCon 15 Keynote - Day 2
DockerCon 15 Keynote - Day 2DockerCon 15 Keynote - Day 2
DockerCon 15 Keynote - Day 2Docker, Inc.
 
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchAugust Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchHoward Greenberg
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsMahesh Arali
 
Using Nagios with Chef
Using Nagios with ChefUsing Nagios with Chef
Using Nagios with ChefBryan McLellan
 
Chef onlinuxonpower
Chef onlinuxonpowerChef onlinuxonpower
Chef onlinuxonpowerMoya Brannan
 
goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020
goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020
goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020AWSKRUG - AWS한국사용자모임
 
Dev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDavide Benvegnù
 
What's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to NowWhat's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to NowSauce Labs
 
Building a PaaS using Chef
Building a PaaS using ChefBuilding a PaaS using Chef
Building a PaaS using ChefShaun Domingo
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityGeoff Harcourt
 
How bigtop leveraged docker for build automation and one click hadoop provis...
How bigtop leveraged docker for build automation and  one click hadoop provis...How bigtop leveraged docker for build automation and  one click hadoop provis...
How bigtop leveraged docker for build automation and one click hadoop provis...Evans Ye
 
Immutable Infrastructure: Rise of the Machine Images
Immutable Infrastructure: Rise of the Machine ImagesImmutable Infrastructure: Rise of the Machine Images
Immutable Infrastructure: Rise of the Machine ImagesC4Media
 
Atmosphere 2014: Really large scale systems configuration - Phil Dibowitz
Atmosphere 2014: Really large scale systems configuration - Phil DibowitzAtmosphere 2014: Really large scale systems configuration - Phil Dibowitz
Atmosphere 2014: Really large scale systems configuration - Phil DibowitzPROIDEA
 
habitat at docker bud
habitat at docker budhabitat at docker bud
habitat at docker budMandi Walls
 
Sonian, Open Source and Sensu
Sonian, Open Source and SensuSonian, Open Source and Sensu
Sonian, Open Source and SensuPete Cheslock
 
Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)sKaushikNarayanan
 
Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)MvkZ
 
Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)MvkZ
 

Similar to Shomi's Journey to Chef: Lessons Learned on Implementing Chef (20)

Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable application
 
Automated Visual Regression Testing by Dave Sadlon
Automated Visual Regression Testing by Dave SadlonAutomated Visual Regression Testing by Dave Sadlon
Automated Visual Regression Testing by Dave Sadlon
 
DockerCon 15 Keynote - Day 2
DockerCon 15 Keynote - Day 2DockerCon 15 Keynote - Day 2
DockerCon 15 Keynote - Day 2
 
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's WorkbenchAugust Webinar - Water Cooler Talks: A Look into a Developer's Workbench
August Webinar - Water Cooler Talks: A Look into a Developer's Workbench
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teams
 
Using Nagios with Chef
Using Nagios with ChefUsing Nagios with Chef
Using Nagios with Chef
 
Chef onlinuxonpower
Chef onlinuxonpowerChef onlinuxonpower
Chef onlinuxonpower
 
goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020
goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020
goployer, 코드 기반의 배포 도구 - 송주영 (beNX) :: AWS Community Day 2020
 
Dev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps world
 
What's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to NowWhat's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to Now
 
Building a PaaS using Chef
Building a PaaS using ChefBuilding a PaaS using Chef
Building a PaaS using Chef
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production Parity
 
How bigtop leveraged docker for build automation and one click hadoop provis...
How bigtop leveraged docker for build automation and  one click hadoop provis...How bigtop leveraged docker for build automation and  one click hadoop provis...
How bigtop leveraged docker for build automation and one click hadoop provis...
 
Immutable Infrastructure: Rise of the Machine Images
Immutable Infrastructure: Rise of the Machine ImagesImmutable Infrastructure: Rise of the Machine Images
Immutable Infrastructure: Rise of the Machine Images
 
Atmosphere 2014: Really large scale systems configuration - Phil Dibowitz
Atmosphere 2014: Really large scale systems configuration - Phil DibowitzAtmosphere 2014: Really large scale systems configuration - Phil Dibowitz
Atmosphere 2014: Really large scale systems configuration - Phil Dibowitz
 
habitat at docker bud
habitat at docker budhabitat at docker bud
habitat at docker bud
 
Sonian, Open Source and Sensu
Sonian, Open Source and SensuSonian, Open Source and Sensu
Sonian, Open Source and Sensu
 
Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)
 
Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)
 
Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)Big datatraining.in devops-part2 (1)
Big datatraining.in devops-part2 (1)
 

Recently uploaded

Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - InfographicHr365.us smith
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataBradBedford3
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based projectAnoyGreter
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
Cloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEECloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEEVICTOR MAESTRE RAMIREZ
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...Christina Lin
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEOrtus Solutions, Corp
 
Folding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesFolding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesPhilip Schwarz
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsAhmed Mohamed
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样umasea
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
Intelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalmIntelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalmSujith Sukumaran
 

Recently uploaded (20)

Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - Infographic
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based project
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
Cloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEECloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEE
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
 
Folding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesFolding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a series
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML Diagrams
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
Intelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalmIntelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalm
 

Shomi's Journey to Chef: Lessons Learned on Implementing Chef

  • 1. shomi's journey to chef: lessons learned on implementing chef Augusto Rosa Head of Video & Network Infrastructure Pierre Durrete devops Engineer (aka chef master)
  • 2. 2 • who is shomi_? • shomi by numbers • where we started • evolution • lessons learned agenda
  • 3. 3who is shomi_ shomi™ is a new kind of instant streaming video subscription service created for entertainment lovers by entertainment lovers that makes discovering entertainment nearly as enjoyable as watching it.
  • 5. 5 - Node.js, C# back end application, Angular front end. - SOA archicture to microservice. 25 M.S. and counting in 9 months. - 29 billions requests in Q1 2016 - 400+ servers - GB/s like traffic at peaks - Petabytes of storage required shomi behind the scene
  • 6. 6 • Manual layout infrastructure at AWS and Physical (no automation) • Servers are treated like pets, not cattle • Implemented chef hosted on windows and linux • No local testing • No version locking • Cookbooks not environment agnostic • Attributes reside in multiple places (cookbooks, environments, roles) • No naming conventions • Forking of community cookbooks blocking us from upstream fixes where we started
  • 7. 7demo • Environments dev, stage and production layout the same way. Same automation. • Moved to berkshef cookbooks • Run testkitchen locally
  • 8. 8evolution • Environments dev, stage and production layout the same way. Same automation. • Use the Berkshelf Way, Environment Cookbook Pattern: – Environment cookbook – Application cookbook – Wrapper cookbook – Community cookbook – shomi-base • Environments per application (Environment Cookbook Pattern), including one application cookbook per software repo • Implement testkitchen. Run test locally first.
  • 9. 9  Follow best practices Environment Cookbook Pattern Chef in a Continuous integration system Version control and version locking  Limit powers of who can create environments, build, deploy  Chef hosted can be a challenge as compared to locally. Feature comparison not the same Not highly available lessons learned
  • 10. 10 Do you have questions? We are hiring at the moment … Developers (node.js, front end, C++, .net) Devops Enginners(chef, terraform,) Site Reliability Enginners (sumologic, Newrelic, Datadog, trend analysis) Video Processing Engineers (workflow developers) open for questions