SlideShare a Scribd company logo
1 of 22
Download to read offline
#ossummit
Open Source Cloud Costs
OpenCost's Impact on Enterprise Cloud Cost Management
Don O'Neill
Matt Ray
@handle
Who are we?
Don O’Neill
Program Lead - Standardize & Enforce, MuleSoft Governance
Co-leader of Seattle CoffeeOps - a DevOps/SRE meetup
Matt Ray
OpenCost Community Manager
Open Source veteran, Software Defined Talk podcast co-host
The complexity of operating
Kubernetes efficiently is real
3
Technical
COMPLEXITY
Metal
VMs
K8s
Higher-level abstractions, more shared
resources, and increasingly dynamic
Behavioral
COMPLEXITY
Decentralized releases means any
engineer can increase spend quickly
Abstraction
Any engineer Any org Any time
$
$
$
$
$
$
What is OpenCost?
Open Source Kubernetes and Cloud Costs Monitoring
Specification and Implementation
Cloud Native Computing Foundation Sandbox Project
FinOps Certified Solution
• https://opencost.io
• https://github.com/opencost
• https://www.cncf.io/projects/opencost/
OpenCost Specification
https://www.opencost.io/docs/specification
Kubernetes workloads are often transient and vary in the resources they consume.
How do we measure who is responsible for what and how much?
• Management fees
• Expenses from nodes
• Persistent volumes
• Attached disks
• Load balancers
• Network ingress/egress
Kubernetes Costs Specification
What is Measured
• CPU
• Memory
• GPU
• Storage Volume
• Load Balancer
Aggregations
• Cluster
• Node
• Container
• Annotation
• Controller Kind
• Controller Name
• DaemonSet
• Deployment
• Job
• Label
• Namespace
• Pod
• Service
• StatefulSet
What is Monitored
Kubernetes
• Amazon Web Services
• Google Cloud Platform
• Microsoft Azure
• Oracle Cloud Infrastructure
• On-premises
Cloud Costs
• Billing data
• Cost and usage reports
Carbon Costs (1.110)
• Carbon footprint of workloads
OpenCost Plugins (1.110)
• New Plugin architecture
• Datadog first example
OpenCost
an end user’s perspective
Kubernetes - the great mystery!
Let's dig into some terminology
AWS CUR - Cost and Usage Report
EKS - Elastic Kubernetes Service
EKS costs in the CUR
Your bill is $10,000 dollars for EKS
Enter OpenCost
How do we help teams understand their costs, efficiency and usage?
OpenCost architecture (at MuleSoft)
OpenCost architecture (cont.)
● deployed in its own namespace “oversight”
● deployed in all clusters, in all flavors of Kubernetes
● writes data daily as a .csv file to an S3 bucket
● uses a seven day rolling data window
● data lake consumes the .csv file, refreshing the
reports
OpenCost lessons learned
● Be sure you have a Prometheus endpoint available
● Bringing your own Prometheus? tune its memory and cpu
requests/limits!
● Pre existing data lake? Parquet exporter
● You can use your own rate sheet
Running OpenCost
https://www.opencost.io/docs/install
Prometheus
• Datastore for metrics
Helm Chart
• Configurable settings (Prometheus, namespaces, etc.)
Docker
• Run without Kubernetes, Cloud Costs and other data sources
Prometheus OpenCost Exporter
• Prometheus Community Helm chart
OpenCost UI
Accessing OpenCost Data
● API
● Backstage
● CSV export
● Grafana Dashboards
● kubectl cost
● OpenCost Parquet Exporter
● Prometheus directly
● Prometheus OpenCost Exporter
● Web UI
OpenCost on GitHub
https://github.com/opencost
● https://github.com/opencost/opencost
● https://github.com/opencost/opencost-grafana-dashboards
● https://github.com/opencost/opencost-helm-chart/
● https://github.com/opencost/opencost-parquet-exporter
● https://github.com/opencost/opencost-plugins
● https://github.com/opencost/opencost-website/
Get Involved with OpenCost!
https://opencost.io
Slack
• https://slack.cncf.io/ #opencost
OpenCost Community Meetings
• https://bit.ly/opencost-calendar
• https://bit.ly/opencost-community-meeting-cncf
Social Media
• https://www.linkedin.com/showcase/opencost/ (LinkedIn)
• https://bsky.app (Bluesky)
• https://hachyderm.io/@opencost (Mastodon)
• https://twitter.com/open_cost (Twitter)
Thank you
Any questions?
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on Enterprise Cloud Cost Management

More Related Content

Similar to Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on Enterprise Cloud Cost Management

Choosing PaaS: Cisco and Open Source Options: an overview
Choosing PaaS:  Cisco and Open Source Options: an overviewChoosing PaaS:  Cisco and Open Source Options: an overview
Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet
 
Oscon 2017: Build your own container-based system with the Moby project
Oscon 2017: Build your own container-based system with the Moby projectOscon 2017: Build your own container-based system with the Moby project
Oscon 2017: Build your own container-based system with the Moby projectPatrick Chanezon
 
Containerized architectures for deep learning
Containerized architectures for deep learningContainerized architectures for deep learning
Containerized architectures for deep learningAntje Barth
 
Monitoring kubernetes across data center and cloud
Monitoring kubernetes across data center and cloudMonitoring kubernetes across data center and cloud
Monitoring kubernetes across data center and cloudDatadog
 
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
Develop and deploy Kubernetes  applications with Docker - IBM Index 2018Develop and deploy Kubernetes  applications with Docker - IBM Index 2018
Develop and deploy Kubernetes applications with Docker - IBM Index 2018Patrick Chanezon
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to KubernetesVishal Biyani
 
Application Centric Microservices Architecture
Application Centric Microservices ArchitectureApplication Centric Microservices Architecture
Application Centric Microservices ArchitectureKen Owens
 
Resilient microservices with Kubernetes - Mete Atamel
Resilient microservices with Kubernetes - Mete AtamelResilient microservices with Kubernetes - Mete Atamel
Resilient microservices with Kubernetes - Mete AtamelITCamp
 
IBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec KubernetesIBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec KubernetesIBM France Lab
 
TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...
TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...
TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...tdc-globalcode
 
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMwareVMUG IT
 
Net core, mssql, container und kubernetes
Net core, mssql, container und kubernetesNet core, mssql, container und kubernetes
Net core, mssql, container und kubernetesThomas Fricke
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatAmazon Web Services
 
Docker Application to Scientific Computing
Docker Application to Scientific ComputingDocker Application to Scientific Computing
Docker Application to Scientific ComputingPeter Bryzgalov
 

Similar to Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on Enterprise Cloud Cost Management (20)

Choosing PaaS: Cisco and Open Source Options: an overview
Choosing PaaS:  Cisco and Open Source Options: an overviewChoosing PaaS:  Cisco and Open Source Options: an overview
Choosing PaaS: Cisco and Open Source Options: an overview
 
56k.cloud training
56k.cloud training56k.cloud training
56k.cloud training
 
Oscon 2017: Build your own container-based system with the Moby project
Oscon 2017: Build your own container-based system with the Moby projectOscon 2017: Build your own container-based system with the Moby project
Oscon 2017: Build your own container-based system with the Moby project
 
Containerized architectures for deep learning
Containerized architectures for deep learningContainerized architectures for deep learning
Containerized architectures for deep learning
 
Monitoring kubernetes across data center and cloud
Monitoring kubernetes across data center and cloudMonitoring kubernetes across data center and cloud
Monitoring kubernetes across data center and cloud
 
Kubernetes intro
Kubernetes introKubernetes intro
Kubernetes intro
 
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
Develop and deploy Kubernetes  applications with Docker - IBM Index 2018Develop and deploy Kubernetes  applications with Docker - IBM Index 2018
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
OpenShift Evolution
OpenShift EvolutionOpenShift Evolution
OpenShift Evolution
 
Cont0519
Cont0519Cont0519
Cont0519
 
Application Centric Microservices Architecture
Application Centric Microservices ArchitectureApplication Centric Microservices Architecture
Application Centric Microservices Architecture
 
Openshift Workshop
Openshift Workshop Openshift Workshop
Openshift Workshop
 
Resilient microservices with Kubernetes - Mete Atamel
Resilient microservices with Kubernetes - Mete AtamelResilient microservices with Kubernetes - Mete Atamel
Resilient microservices with Kubernetes - Mete Atamel
 
IBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec KubernetesIBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Orchestrer Docker avec Kubernetes
 
TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...
TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...
TDC2017 | São Paulo - Trilha Cloud Computing How we figured out we had a SRE ...
 
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
 
NextGenML
NextGenML NextGenML
NextGenML
 
Net core, mssql, container und kubernetes
Net core, mssql, container und kubernetesNet core, mssql, container und kubernetes
Net core, mssql, container und kubernetes
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red Hat
 
Docker Application to Scientific Computing
Docker Application to Scientific ComputingDocker Application to Scientific Computing
Docker Application to Scientific Computing
 

More from Matt Ray

KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...
SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...
SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...Matt Ray
 
HashiTalks 2020 - Chef Tools & Terraform: Better Together
HashiTalks 2020 - Chef Tools & Terraform: Better TogetherHashiTalks 2020 - Chef Tools & Terraform: Better Together
HashiTalks 2020 - Chef Tools & Terraform: Better TogetherMatt Ray
 
EmacsConf 2019: Interactive Remote Debugging and Development with TRAMP Mode
EmacsConf 2019: Interactive Remote Debugging and Development with TRAMP ModeEmacsConf 2019: Interactive Remote Debugging and Development with TRAMP Mode
EmacsConf 2019: Interactive Remote Debugging and Development with TRAMP ModeMatt Ray
 
Wellington DevOps: Bringing Your Applications into the Future with Habitat
Wellington DevOps: Bringing Your Applications into the Future with HabitatWellington DevOps: Bringing Your Applications into the Future with Habitat
Wellington DevOps: Bringing Your Applications into the Future with HabitatMatt Ray
 
DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...
DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...
DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...Matt Ray
 
Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...
Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...
Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...Matt Ray
 
Compliance as Code Everywhere
Compliance as Code EverywhereCompliance as Code Everywhere
Compliance as Code EverywhereMatt Ray
 
DevOpsDays Jakarta: State of DevOps 2018
DevOpsDays Jakarta: State of DevOps 2018DevOpsDays Jakarta: State of DevOps 2018
DevOpsDays Jakarta: State of DevOps 2018Matt Ray
 
DevOps Talks Melbourne 2018: Whales, Cats and Kubernetes
DevOps Talks Melbourne 2018: Whales, Cats and KubernetesDevOps Talks Melbourne 2018: Whales, Cats and Kubernetes
DevOps Talks Melbourne 2018: Whales, Cats and KubernetesMatt Ray
 
Infrastructure and Compliance Delight with Chef Automate
Infrastructure and Compliance Delight with Chef AutomateInfrastructure and Compliance Delight with Chef Automate
Infrastructure and Compliance Delight with Chef AutomateMatt Ray
 
Cooking Up Windows with Chef Automate
Cooking Up Windows with Chef AutomateCooking Up Windows with Chef Automate
Cooking Up Windows with Chef AutomateMatt Ray
 
DevOpsDays Singapore - Continuous Auditing with Compliance as Code
DevOpsDays Singapore - Continuous Auditing with Compliance as CodeDevOpsDays Singapore - Continuous Auditing with Compliance as Code
DevOpsDays Singapore - Continuous Auditing with Compliance as CodeMatt Ray
 
DevOpsDays Singapore Habitat Ignite
DevOpsDays Singapore Habitat IgniteDevOpsDays Singapore Habitat Ignite
DevOpsDays Singapore Habitat IgniteMatt Ray
 
Chef Automate - Azure Sydney User Group
Chef Automate - Azure Sydney User GroupChef Automate - Azure Sydney User Group
Chef Automate - Azure Sydney User GroupMatt Ray
 
Automating Compliance with InSpec - AWS North Sydney
Automating Compliance with InSpec - AWS North SydneyAutomating Compliance with InSpec - AWS North Sydney
Automating Compliance with InSpec - AWS North SydneyMatt Ray
 
Automating Applications with Habitat - Sydney Cloud Native Meetup
Automating Applications with Habitat - Sydney Cloud Native MeetupAutomating Applications with Habitat - Sydney Cloud Native Meetup
Automating Applications with Habitat - Sydney Cloud Native MeetupMatt Ray
 
Automating AWS Compliance with InSpec
Automating AWS Compliance with InSpec Automating AWS Compliance with InSpec
Automating AWS Compliance with InSpec Matt Ray
 
Chef Automate - Infracoders Canberra August 8, 2017
Chef Automate - Infracoders Canberra August 8, 2017Chef Automate - Infracoders Canberra August 8, 2017
Chef Automate - Infracoders Canberra August 8, 2017Matt Ray
 
OpsWorks for Chef Automate - Auckland AWS
OpsWorks for Chef Automate - Auckland AWS OpsWorks for Chef Automate - Auckland AWS
OpsWorks for Chef Automate - Auckland AWS Matt Ray
 

More from Matt Ray (20)

KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...
SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...
SCaLE 20X: Kubernetes Cloud Cost Monitoring with OpenCost & Optimization Stra...
 
HashiTalks 2020 - Chef Tools & Terraform: Better Together
HashiTalks 2020 - Chef Tools & Terraform: Better TogetherHashiTalks 2020 - Chef Tools & Terraform: Better Together
HashiTalks 2020 - Chef Tools & Terraform: Better Together
 
EmacsConf 2019: Interactive Remote Debugging and Development with TRAMP Mode
EmacsConf 2019: Interactive Remote Debugging and Development with TRAMP ModeEmacsConf 2019: Interactive Remote Debugging and Development with TRAMP Mode
EmacsConf 2019: Interactive Remote Debugging and Development with TRAMP Mode
 
Wellington DevOps: Bringing Your Applications into the Future with Habitat
Wellington DevOps: Bringing Your Applications into the Future with HabitatWellington DevOps: Bringing Your Applications into the Future with Habitat
Wellington DevOps: Bringing Your Applications into the Future with Habitat
 
DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...
DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...
DevOps Days Singapore 2018 Ignite - Bringing Your Applications into the Futur...
 
Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...
Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...
Cloud Expo Asia 20181010 - Bringing Your Applications into the Future with Ha...
 
Compliance as Code Everywhere
Compliance as Code EverywhereCompliance as Code Everywhere
Compliance as Code Everywhere
 
DevOpsDays Jakarta: State of DevOps 2018
DevOpsDays Jakarta: State of DevOps 2018DevOpsDays Jakarta: State of DevOps 2018
DevOpsDays Jakarta: State of DevOps 2018
 
DevOps Talks Melbourne 2018: Whales, Cats and Kubernetes
DevOps Talks Melbourne 2018: Whales, Cats and KubernetesDevOps Talks Melbourne 2018: Whales, Cats and Kubernetes
DevOps Talks Melbourne 2018: Whales, Cats and Kubernetes
 
Infrastructure and Compliance Delight with Chef Automate
Infrastructure and Compliance Delight with Chef AutomateInfrastructure and Compliance Delight with Chef Automate
Infrastructure and Compliance Delight with Chef Automate
 
Cooking Up Windows with Chef Automate
Cooking Up Windows with Chef AutomateCooking Up Windows with Chef Automate
Cooking Up Windows with Chef Automate
 
DevOpsDays Singapore - Continuous Auditing with Compliance as Code
DevOpsDays Singapore - Continuous Auditing with Compliance as CodeDevOpsDays Singapore - Continuous Auditing with Compliance as Code
DevOpsDays Singapore - Continuous Auditing with Compliance as Code
 
DevOpsDays Singapore Habitat Ignite
DevOpsDays Singapore Habitat IgniteDevOpsDays Singapore Habitat Ignite
DevOpsDays Singapore Habitat Ignite
 
Chef Automate - Azure Sydney User Group
Chef Automate - Azure Sydney User GroupChef Automate - Azure Sydney User Group
Chef Automate - Azure Sydney User Group
 
Automating Compliance with InSpec - AWS North Sydney
Automating Compliance with InSpec - AWS North SydneyAutomating Compliance with InSpec - AWS North Sydney
Automating Compliance with InSpec - AWS North Sydney
 
Automating Applications with Habitat - Sydney Cloud Native Meetup
Automating Applications with Habitat - Sydney Cloud Native MeetupAutomating Applications with Habitat - Sydney Cloud Native Meetup
Automating Applications with Habitat - Sydney Cloud Native Meetup
 
Automating AWS Compliance with InSpec
Automating AWS Compliance with InSpec Automating AWS Compliance with InSpec
Automating AWS Compliance with InSpec
 
Chef Automate - Infracoders Canberra August 8, 2017
Chef Automate - Infracoders Canberra August 8, 2017Chef Automate - Infracoders Canberra August 8, 2017
Chef Automate - Infracoders Canberra August 8, 2017
 
OpsWorks for Chef Automate - Auckland AWS
OpsWorks for Chef Automate - Auckland AWS OpsWorks for Chef Automate - Auckland AWS
OpsWorks for Chef Automate - Auckland AWS
 

Recently uploaded

Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - InfographicHr365.us smith
 
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfkalichargn70th171
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样umasea
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningVitsRangannavar
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsMehedi Hasan Shohan
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataBradBedford3
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfPower Karaoke
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...Christina Lin
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...OnePlan Solutions
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
buds n tech IT solutions
buds n  tech IT                solutionsbuds n  tech IT                solutions
buds n tech IT solutionsmonugehlot87
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 

Recently uploaded (20)

Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - Infographic
 
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learning
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software Solutions
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdf
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
buds n tech IT solutions
buds n  tech IT                solutionsbuds n  tech IT                solutions
buds n tech IT solutions
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 

Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on Enterprise Cloud Cost Management

  • 1. #ossummit Open Source Cloud Costs OpenCost's Impact on Enterprise Cloud Cost Management Don O'Neill Matt Ray @handle
  • 2. Who are we? Don O’Neill Program Lead - Standardize & Enforce, MuleSoft Governance Co-leader of Seattle CoffeeOps - a DevOps/SRE meetup Matt Ray OpenCost Community Manager Open Source veteran, Software Defined Talk podcast co-host
  • 3. The complexity of operating Kubernetes efficiently is real 3 Technical COMPLEXITY Metal VMs K8s Higher-level abstractions, more shared resources, and increasingly dynamic Behavioral COMPLEXITY Decentralized releases means any engineer can increase spend quickly Abstraction Any engineer Any org Any time $ $ $ $ $ $
  • 4. What is OpenCost? Open Source Kubernetes and Cloud Costs Monitoring Specification and Implementation Cloud Native Computing Foundation Sandbox Project FinOps Certified Solution • https://opencost.io • https://github.com/opencost • https://www.cncf.io/projects/opencost/
  • 5. OpenCost Specification https://www.opencost.io/docs/specification Kubernetes workloads are often transient and vary in the resources they consume. How do we measure who is responsible for what and how much? • Management fees • Expenses from nodes • Persistent volumes • Attached disks • Load balancers • Network ingress/egress
  • 6. Kubernetes Costs Specification What is Measured • CPU • Memory • GPU • Storage Volume • Load Balancer Aggregations • Cluster • Node • Container • Annotation • Controller Kind • Controller Name • DaemonSet • Deployment • Job • Label • Namespace • Pod • Service • StatefulSet
  • 7. What is Monitored Kubernetes • Amazon Web Services • Google Cloud Platform • Microsoft Azure • Oracle Cloud Infrastructure • On-premises Cloud Costs • Billing data • Cost and usage reports Carbon Costs (1.110) • Carbon footprint of workloads OpenCost Plugins (1.110) • New Plugin architecture • Datadog first example
  • 9. Kubernetes - the great mystery!
  • 10. Let's dig into some terminology AWS CUR - Cost and Usage Report EKS - Elastic Kubernetes Service
  • 11. EKS costs in the CUR Your bill is $10,000 dollars for EKS
  • 12. Enter OpenCost How do we help teams understand their costs, efficiency and usage?
  • 14. OpenCost architecture (cont.) ● deployed in its own namespace “oversight” ● deployed in all clusters, in all flavors of Kubernetes ● writes data daily as a .csv file to an S3 bucket ● uses a seven day rolling data window ● data lake consumes the .csv file, refreshing the reports
  • 15. OpenCost lessons learned ● Be sure you have a Prometheus endpoint available ● Bringing your own Prometheus? tune its memory and cpu requests/limits! ● Pre existing data lake? Parquet exporter ● You can use your own rate sheet
  • 16. Running OpenCost https://www.opencost.io/docs/install Prometheus • Datastore for metrics Helm Chart • Configurable settings (Prometheus, namespaces, etc.) Docker • Run without Kubernetes, Cloud Costs and other data sources Prometheus OpenCost Exporter • Prometheus Community Helm chart
  • 18. Accessing OpenCost Data ● API ● Backstage ● CSV export ● Grafana Dashboards ● kubectl cost ● OpenCost Parquet Exporter ● Prometheus directly ● Prometheus OpenCost Exporter ● Web UI
  • 19. OpenCost on GitHub https://github.com/opencost ● https://github.com/opencost/opencost ● https://github.com/opencost/opencost-grafana-dashboards ● https://github.com/opencost/opencost-helm-chart/ ● https://github.com/opencost/opencost-parquet-exporter ● https://github.com/opencost/opencost-plugins ● https://github.com/opencost/opencost-website/
  • 20. Get Involved with OpenCost! https://opencost.io Slack • https://slack.cncf.io/ #opencost OpenCost Community Meetings • https://bit.ly/opencost-calendar • https://bit.ly/opencost-community-meeting-cncf Social Media • https://www.linkedin.com/showcase/opencost/ (LinkedIn) • https://bsky.app (Bluesky) • https://hachyderm.io/@opencost (Mastodon) • https://twitter.com/open_cost (Twitter)