SlideShare a Scribd company logo

DevOps & SRE at Google Scale

Kaushik Bhattacharya
Kaushik Bhattacharya
Kaushik BhattacharyaCustomer Engineer at Google

How Google works and how can you benefit from it? Test drive now a complete Microservices application with Istio, gRPC, Redis, BigQuery, Spring Boot, Spring Cloud and Stackdriver on Google Cloud Platform: https://git.io/fhzCx

DevOps & SRE at Google Scale

1 of 48
Download to read offline
Google Cloud Platform 1
By Kaushik Bhattacharya, Customer Engineer
Google Cloud, the Netherlands
kbhattacTweets
DevOps & SRE at Google Scale
How Google does it & How can you benefit from this
2
1.How the
engineering
processes at
Google works
Engineering at Google
3.From open
source to Google
Cloud for
enterprises
2. Our learnings,
how we contribute
back to open
source
Google Cloud Platform 3
Building software at Google
Google Cloud Platform 4
From product to idea 10x
Product idea
X 10
Google confidential | Do not distribute
Moonshot thinking: Solving for X.
6
“To organize the world’s information and make it
universally accessible and useful.”
- Google

Recommended

SRE-iously! Defining the Principles, Habits, and Practices of Site Reliabilit...
SRE-iously! Defining the Principles, Habits, and Practices of Site Reliabilit...SRE-iously! Defining the Principles, Habits, and Practices of Site Reliabilit...
SRE-iously! Defining the Principles, Habits, and Practices of Site Reliabilit...Tori Wieldt
 
Getting started with Site Reliability Engineering (SRE)
Getting started with Site Reliability Engineering (SRE)Getting started with Site Reliability Engineering (SRE)
Getting started with Site Reliability Engineering (SRE)Abeer R
 
SRE 101 (Site Reliability Engineering)
SRE 101 (Site Reliability Engineering)SRE 101 (Site Reliability Engineering)
SRE 101 (Site Reliability Engineering)Hussain Mansoor
 
Site (Service) Reliability Engineering
Site (Service) Reliability EngineeringSite (Service) Reliability Engineering
Site (Service) Reliability EngineeringMark Underwood
 
DevOps Vs SRE Major Differences That You Need To Know - Hidden Brains Infotech
DevOps Vs SRE Major Differences That You Need To Know - Hidden Brains InfotechDevOps Vs SRE Major Differences That You Need To Know - Hidden Brains Infotech
DevOps Vs SRE Major Differences That You Need To Know - Hidden Brains InfotechRosalie Lauren
 
Overview of Site Reliability Engineering (SRE) & best practices
Overview of Site Reliability Engineering (SRE) & best practicesOverview of Site Reliability Engineering (SRE) & best practices
Overview of Site Reliability Engineering (SRE) & best practicesAshutosh Agarwal
 
About DevOps in simple steps
About DevOps in simple stepsAbout DevOps in simple steps
About DevOps in simple stepsIhor Odynets
 

More Related Content

What's hot

Site reliability engineering
Site reliability engineeringSite reliability engineering
Site reliability engineeringJason Loeffler
 
How Small Team Get Ready for SRE (public version)
How Small Team Get Ready for SRE (public version)How Small Team Get Ready for SRE (public version)
How Small Team Get Ready for SRE (public version)Setyo Legowo
 
DevOps a pratical approach
DevOps a pratical approachDevOps a pratical approach
DevOps a pratical approachSiderlan Santos
 
Site Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet SugathadasaSite Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet SugathadasaKeet Sugathadasa
 
Site reliability engineering - Lightning Talk
Site reliability engineering - Lightning TalkSite reliability engineering - Lightning Talk
Site reliability engineering - Lightning TalkMichae Blakeney
 
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of KubernetesDevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of KubernetesDevOps.com
 
SRE-iously! Reliability!
SRE-iously! Reliability!SRE-iously! Reliability!
SRE-iously! Reliability!New Relic
 
A Crash Course in Building Site Reliability
A Crash Course in Building Site ReliabilityA Crash Course in Building Site Reliability
A Crash Course in Building Site ReliabilityAcquia
 
SRE (service reliability engineer) on big DevOps platform running on the clou...
SRE (service reliability engineer) on big DevOps platform running on the clou...SRE (service reliability engineer) on big DevOps platform running on the clou...
SRE (service reliability engineer) on big DevOps platform running on the clou...DevClub_lv
 
Bjorn Rabenstein. SRE, DevOps, Google, and you
Bjorn Rabenstein. SRE, DevOps, Google, and youBjorn Rabenstein. SRE, DevOps, Google, and you
Bjorn Rabenstein. SRE, DevOps, Google, and youIT Arena
 
CI/CD Best Practices for Your DevOps Journey
CI/CD Best  Practices for Your DevOps JourneyCI/CD Best  Practices for Your DevOps Journey
CI/CD Best Practices for Your DevOps JourneyDevOps.com
 
DevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesDevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesSlideTeam
 
DevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the AutomationDevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the AutomationKeith Pleas
 
What is Site Reliability Engineering (SRE)
What is Site Reliability Engineering (SRE)What is Site Reliability Engineering (SRE)
What is Site Reliability Engineering (SRE)jeetendra mandal
 
SRE Demystified - 05 - Toil Elimination
SRE Demystified - 05 - Toil EliminationSRE Demystified - 05 - Toil Elimination
SRE Demystified - 05 - Toil EliminationDr Ganesh Iyer
 

What's hot (20)

Sre summary
Sre summarySre summary
Sre summary
 
Site reliability engineering
Site reliability engineeringSite reliability engineering
Site reliability engineering
 
How Small Team Get Ready for SRE (public version)
How Small Team Get Ready for SRE (public version)How Small Team Get Ready for SRE (public version)
How Small Team Get Ready for SRE (public version)
 
devops
devops devops
devops
 
DevOps a pratical approach
DevOps a pratical approachDevOps a pratical approach
DevOps a pratical approach
 
Site Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet SugathadasaSite Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
Site Reliability Engineering (SRE) - Tech Talk by Keet Sugathadasa
 
DevOps
DevOpsDevOps
DevOps
 
SRE in Startup
SRE in StartupSRE in Startup
SRE in Startup
 
Site reliability engineering - Lightning Talk
Site reliability engineering - Lightning TalkSite reliability engineering - Lightning Talk
Site reliability engineering - Lightning Talk
 
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of KubernetesDevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
 
SRE-iously! Reliability!
SRE-iously! Reliability!SRE-iously! Reliability!
SRE-iously! Reliability!
 
A Crash Course in Building Site Reliability
A Crash Course in Building Site ReliabilityA Crash Course in Building Site Reliability
A Crash Course in Building Site Reliability
 
SRE (service reliability engineer) on big DevOps platform running on the clou...
SRE (service reliability engineer) on big DevOps platform running on the clou...SRE (service reliability engineer) on big DevOps platform running on the clou...
SRE (service reliability engineer) on big DevOps platform running on the clou...
 
Bjorn Rabenstein. SRE, DevOps, Google, and you
Bjorn Rabenstein. SRE, DevOps, Google, and youBjorn Rabenstein. SRE, DevOps, Google, and you
Bjorn Rabenstein. SRE, DevOps, Google, and you
 
CI/CD Best Practices for Your DevOps Journey
CI/CD Best  Practices for Your DevOps JourneyCI/CD Best  Practices for Your DevOps Journey
CI/CD Best Practices for Your DevOps Journey
 
DevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesDevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation Slides
 
DevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the AutomationDevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the Automation
 
What is Site Reliability Engineering (SRE)
What is Site Reliability Engineering (SRE)What is Site Reliability Engineering (SRE)
What is Site Reliability Engineering (SRE)
 
Introduction to DevOps
Introduction to DevOpsIntroduction to DevOps
Introduction to DevOps
 
SRE Demystified - 05 - Toil Elimination
SRE Demystified - 05 - Toil EliminationSRE Demystified - 05 - Toil Elimination
SRE Demystified - 05 - Toil Elimination
 

Similar to DevOps & SRE at Google Scale

Efficient platform engineering with Microk8s & gopaddle.pdf
Efficient platform engineering  with  Microk8s & gopaddle.pdfEfficient platform engineering  with  Microk8s & gopaddle.pdf
Efficient platform engineering with Microk8s & gopaddle.pdfVinothini Raju
 
Google Cloud Next 2021 Recap
 Google Cloud Next 2021 Recap Google Cloud Next 2021 Recap
Google Cloud Next 2021 RecapErvin Weber
 
How google cloud platform can benefit devops?
How google cloud platform can benefit devops?How google cloud platform can benefit devops?
How google cloud platform can benefit devops?VishnuAnji
 
OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...
OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...
OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...OW2
 
Getting started with GCP ( Google Cloud Platform)
Getting started with GCP ( Google  Cloud Platform)Getting started with GCP ( Google  Cloud Platform)
Getting started with GCP ( Google Cloud Platform)bigdata trunk
 
CI/CD on Google Cloud Platform
CI/CD on Google Cloud PlatformCI/CD on Google Cloud Platform
CI/CD on Google Cloud PlatformDevOps Indonesia
 
Google Cloud Platform - Introduction & Certification Path 2018
Google Cloud Platform - Introduction & Certification Path 2018Google Cloud Platform - Introduction & Certification Path 2018
Google Cloud Platform - Introduction & Certification Path 2018Pavan Dikondkar
 
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2GDSCBVCOENM
 
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)Ido Green
 
Anthos - Oxford - AI - Cloud and edge implementations.pdf
Anthos - Oxford - AI - Cloud and edge implementations.pdfAnthos - Oxford - AI - Cloud and edge implementations.pdf
Anthos - Oxford - AI - Cloud and edge implementations.pdfAntonioGulli2
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixAll Things Open
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Programaspyker
 
Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...
Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...
Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...GRUC
 
Introduction to GCP
Introduction to GCPIntroduction to GCP
Introduction to GCPKnoldus Inc.
 
Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...
Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...
Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...DataStax
 
Google cloud Study Jam 2023.pptx
Google cloud Study Jam 2023.pptxGoogle cloud Study Jam 2023.pptx
Google cloud Study Jam 2023.pptxGDSCNiT
 
Powerup & GCP | Workshop on Google Kubernetes Engine
Powerup & GCP | Workshop on Google Kubernetes EnginePowerup & GCP | Workshop on Google Kubernetes Engine
Powerup & GCP | Workshop on Google Kubernetes EnginePowerup
 
Cloud Run and Containers
Cloud Run and ContainersCloud Run and Containers
Cloud Run and ContainersOmar Fathy
 

Similar to DevOps & SRE at Google Scale (20)

Efficient platform engineering with Microk8s & gopaddle.pdf
Efficient platform engineering  with  Microk8s & gopaddle.pdfEfficient platform engineering  with  Microk8s & gopaddle.pdf
Efficient platform engineering with Microk8s & gopaddle.pdf
 
Google Cloud Next 2021 Recap
 Google Cloud Next 2021 Recap Google Cloud Next 2021 Recap
Google Cloud Next 2021 Recap
 
How google cloud platform can benefit devops?
How google cloud platform can benefit devops?How google cloud platform can benefit devops?
How google cloud platform can benefit devops?
 
OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...
OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...
OW2con'16 Keynote address: Kubernetes, the rising tide of systems administrat...
 
Gdsc muk - innocent
Gdsc   muk - innocentGdsc   muk - innocent
Gdsc muk - innocent
 
Google Cloud Platform
Google Cloud PlatformGoogle Cloud Platform
Google Cloud Platform
 
Getting started with GCP ( Google Cloud Platform)
Getting started with GCP ( Google  Cloud Platform)Getting started with GCP ( Google  Cloud Platform)
Getting started with GCP ( Google Cloud Platform)
 
CI/CD on Google Cloud Platform
CI/CD on Google Cloud PlatformCI/CD on Google Cloud Platform
CI/CD on Google Cloud Platform
 
Google Cloud Platform - Introduction & Certification Path 2018
Google Cloud Platform - Introduction & Certification Path 2018Google Cloud Platform - Introduction & Certification Path 2018
Google Cloud Platform - Introduction & Certification Path 2018
 
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
GDSC BVCOENM - Google Cloud Study Jam October 2021 | Day 1 + Day 2
 
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
Scale with a smile with Google Cloud Platform At DevConTLV (June 2014)
 
Anthos - Oxford - AI - Cloud and edge implementations.pdf
Anthos - Oxford - AI - Cloud and edge implementations.pdfAnthos - Oxford - AI - Cloud and edge implementations.pdf
Anthos - Oxford - AI - Cloud and edge implementations.pdf
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at Netflix
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
 
Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...
Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...
Webcast Presentation: Be lean. Be agile. Work together with DevOps Services (...
 
Introduction to GCP
Introduction to GCPIntroduction to GCP
Introduction to GCP
 
Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...
Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...
Cassandra on Google Cloud Platform (Ravi Madasu, Google / Ben Lackey, DataSta...
 
Google cloud Study Jam 2023.pptx
Google cloud Study Jam 2023.pptxGoogle cloud Study Jam 2023.pptx
Google cloud Study Jam 2023.pptx
 
Powerup & GCP | Workshop on Google Kubernetes Engine
Powerup & GCP | Workshop on Google Kubernetes EnginePowerup & GCP | Workshop on Google Kubernetes Engine
Powerup & GCP | Workshop on Google Kubernetes Engine
 
Cloud Run and Containers
Cloud Run and ContainersCloud Run and Containers
Cloud Run and Containers
 

Recently uploaded

Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Adrian Sanabria
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfDomotica daVinci
 
Heltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdfHeltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdfMarielaL5
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfIsidro Navarro
 
Manual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveManual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveDomotica daVinci
 
Power of 2024 - WITforce Odyssey.pptx.pdf
Power of 2024 - WITforce Odyssey.pptx.pdfPower of 2024 - WITforce Odyssey.pptx.pdf
Power of 2024 - WITforce Odyssey.pptx.pdfkatalinjordans1
 
Manual sensor Zigbee 3.0 MOES ZSS-X-PIRL-C
Manual  sensor Zigbee 3.0 MOES ZSS-X-PIRL-CManual  sensor Zigbee 3.0 MOES ZSS-X-PIRL-C
Manual sensor Zigbee 3.0 MOES ZSS-X-PIRL-CDomotica daVinci
 
Introduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptxIntroduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptxBrandon Minnick, MBA
 
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxEvolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxKyle Willson
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolProduct School
 
Dynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringDynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringMassimo Talia
 
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfThomas Poetter
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfDomotica daVinci
 
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEDNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEandreiandasan
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfDomotica daVinci
 
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)Memory Fabric Forum
 
2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch TuesdayIvanti
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVARobert McDermott
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17Ana-Maria Mihalceanu
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS Chicago
 

Recently uploaded (20)

Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdf
 
Heltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdfHeltun_HE-RS01_User_Manual_B9AH.pdf
Heltun_HE-RS01_User_Manual_B9AH.pdf
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdf
 
Manual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveManual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-Wave
 
Power of 2024 - WITforce Odyssey.pptx.pdf
Power of 2024 - WITforce Odyssey.pptx.pdfPower of 2024 - WITforce Odyssey.pptx.pdf
Power of 2024 - WITforce Odyssey.pptx.pdf
 
Manual sensor Zigbee 3.0 MOES ZSS-X-PIRL-C
Manual  sensor Zigbee 3.0 MOES ZSS-X-PIRL-CManual  sensor Zigbee 3.0 MOES ZSS-X-PIRL-C
Manual sensor Zigbee 3.0 MOES ZSS-X-PIRL-C
 
Introduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptxIntroduction to Serverless with AWS Lambda in C#.pptx
Introduction to Serverless with AWS Lambda in C#.pptx
 
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptxEvolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
Evolution of Chatbots: From Custom AI Chatbots and AI Chatbots for Websites.pptx
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product School
 
Dynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringDynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineering
 
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdfLLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
LLMs, LMMs, their Improvement Suggestions and the Path towards AGI.pdf
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
 
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEDNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
 
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
Q1 Memory Fabric Forum: Intel Enabling Compute Express Link (CXL)
 
2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch Tuesday
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVA
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user group
 

DevOps & SRE at Google Scale

  • 1. Google Cloud Platform 1 By Kaushik Bhattacharya, Customer Engineer Google Cloud, the Netherlands kbhattacTweets DevOps & SRE at Google Scale How Google does it & How can you benefit from this
  • 2. 2 1.How the engineering processes at Google works Engineering at Google 3.From open source to Google Cloud for enterprises 2. Our learnings, how we contribute back to open source
  • 3. Google Cloud Platform 3 Building software at Google
  • 4. Google Cloud Platform 4 From product to idea 10x Product idea X 10
  • 5. Google confidential | Do not distribute Moonshot thinking: Solving for X.
  • 6. 6 “To organize the world’s information and make it universally accessible and useful.” - Google
  • 7. Project Loon: Balloon powered internet for everyone!
  • 10. Prototyping: First version of Google Glass was created in 90 min!
  • 12. Google Cloud Platform 12 Code Development Product idea Writing code public class foo {}
  • 13. Google Cloud Platform 13 What it takes to be a Google engineer Working on problems with SPEED AND SCALE is a challenge. Engineers keep raising the bar on the tools and infrastructure. Google Culture: • Collaboration and co-development • Sharing between products and teams (tools, libraries, services) • Engineers have autonomy. • Agile/Scrum, daily stand-up meetings
  • 14. Google’s entire codebase is a giant single repository of more than 2 billion lines of code
  • 15. Google Repository statistics As of Jan 2015 Total number of files 1+ billion Number of source files 9 million Lines of code 2+ billion Depth of history 35 million commits Size of content 86 terabytes
  • 17. Google Cloud Platform 17 Advantages of monolithic repo ● Unified versioning - One source of truth ● Extensive code sharing and reuse ● Collaboration across teams ● Simplified dependency management ● Large scale refactoring ● Flexible team boundaries & code ownership ● Code visibility
  • 18. Google Cloud Platform 18 Automated Test / Analysis Google uses its own version control system called: Piper Sync workspace Write code Code Review Commit Read/Write Access per folder Code Quality & Syntax Check (by humans and by tooling) Create personal copy Auto Rollback if needed MANDATORY A single code tree, with fast access to the code through tooling. All types of code languages. Everyone, works in Trunk. - Branches are for releases.
  • 19. Google Cloud Platform 19 Software testing Product idea Writing code Testing
  • 20. Google Cloud Platform 20 Testing at Google ● Developing & Testing go hand in hand ● 3 million test a day ● 20+ OS and Browser combos
  • 21. Google Cloud Platform 21 Build processes Product idea Writing code Testing Building
  • 22. Google Cloud Platform 22 Build systems Why do we need build systems? Well code has a lot of dependencies and you don’t want to compile and link these all manually. The steps of a general build system: 1. Loading 2. Analysis 3. Execution by build system
  • 23. Google Cloud Platform 23 Google’s continuous build and test system Google has its own continuous build & test system. Remember, at Google we develop everything at HEAD in the repo. Endless CPU, Cross User Caching, because of Cloud Computing.
  • 24. Google Cloud Platform 24 Devops at Google Product idea Writing code Testing Building Deploying
  • 25. Each week Google launches over 4 billion containers. Google is using container technology for more than 10 years.
  • 26. Enter the container Virtual machine OS Dependencies Application Code Hardware Bare-metal server OS Dependencies Application Code Hardware Container OS Dependencies Application Code Hardware
  • 27. Google Cloud Platform 27 So, you mean Docker? 2004 2016 ● Docker is a popular software container platform. ● Containers are a way to package software in a format that can run isolated on a shared operating system.
  • 28. Enter the container… and new challenges ● Scheduling, scaling across clusters of servers ● Networking and connectivity ● Security and Access control ● Logging, Monitoring, and Debugging ● Health checks and uptime preservation ● ...
  • 29. Google Cloud Platform 29 Large-scale cluster management at Google with Borg 2004 2016 ● It’s software that manages all production machines at Google and runs jobs (binaries) that engineers give it on them. ● Borg ran pretty much everything inside the company, including Google Search, Gmail, Google Maps, Google Docs... ● These binaries are run in a container environment. ● When tasks die, they are automatically started up again, and they may run on a different machine.
  • 30. Google Cloud Platform 30 Site Reliability Engineering Product idea Writing code Testing Building Deploying SRE
  • 31. “Hope is not a strategy. Engineering solutions to design, build, and run large-scale systems scalably, reliably and efficiently is a strategy, and a good one.”
  • 32. 32 Site Reliability Engineering ● Site Reliability Engineering is a specialized job function that focuses on the reliability and maintainability of large systems. ● SRE is also a mindset, and a set of engineering approaches to running better production systems ● Google has SRE teams of site reliability engineers responsible for a service globally available. https://landing.google.com/sre/book.html
  • 33. Google Cloud Platform 33 Open Source Googlers contribute back to the community.
  • 34. 34 Google is leader in Open Source src: Stackalytics
  • 35. 35 Popular Google open source projects https://opensource.google.com
  • 36. 36 Contributions to other popular open source projects and standards by Google
  • 37. 37 https://research.google.com/ Google wrote lots of white papers which inspires the big data community. ● Bigtable ● GFS ● Mapreduce ● Chubby ● Sawzall ● Dapper ● Dremel ● Borg
  • 38. Google Cloud Platform 38 From Google to OSS 2004 2016 Internal Google ● Borg Container Orchestration ● Machine Learning ● Go Lang ● Google Chrome ● Stubby ● Dapper ● GFS/BigTable Open Source ● Kubernetes ● Tensorflow ● Go Lang ● Chromium ● gRPC ● Zipkin ● HDFS/HBase
  • 39. 39 Tensorflow Tensorflow is what we use for our own internal machine learning projects, and now it’s available to you! Google made it open source. More than 480 contributions 10,000 commits in a year 53k star rating Tutorials to get started at https://www.tensorflow.org
  • 40. 40 Kubernetes abstracts away the hardware infrastructure and exposes your whole data center as a single enormous computing resource. ● Multiple container engines (Docker, rkt, Windows) ● Cloud and bare-metal environments ● Container Engine = Managed Kubernetes in Google Cloud Kubernetes https://kubernetes.io
  • 41. 41 ● A complete framework for connecting, securing, managing and monitoring services ● Secure and monitor traffic for microservices and legacy services without requiring any changes to application code ● An open platform with key contributions from Google, IBM, Lyft and others ● Allows developers to authenticate and secure the communications between different applications using a TLS connection ● Multi-environment and multi-platform, but Kubernetes first Istio (A Service Mesh)
  • 42. Google Cloud Google infrastructure for your company. Open Source
  • 43. Google Cloud Platform 43 From OSS to Google Cloud 2004 2016 Open Source ● Kubernetes ● Istio ● Tensorflow ● MySQL / Postgresql ● Spark / Hadoop ● Apache Beam ● Spinnaker Google Cloud ● Google Kubernetes Engine ● ML Engine/Auto ML ● Cloud SQL ● Dataproc ● Dataflow
  • 44. CONTAINERIZATION ORCHESTRATION CI / CD SERVICE MESH Package applications Run applications Manage applications Connect and secure applications DevOps on Google Cloud
  • 45. CI/CD on Google Cloud Build/ Test Artifact storage Deploy Cloud Build Container Registry Cloud Storage Source Source Repository GitHub Bitbucket Jenkins Circle CI quay Docker Hub jenkins Codefresh
  • 46. DOCKER KUBERNETES SPINNAKER ISTIO Package applications Run applications Manage applications Connect and secure applications DevOps on Google Cloud
  • 47. 47 Google has two decades of experience with building secure software on large scale. Conclusion Your company can make use of the same infrastructure like Google does. Scalable, Secure and Open. The learnings are shared through whitepapers and contributed back through open source.