SlideShare a Scribd company logo
OSMC
Kris Buytaert

@krisbuytaert
Pipeline all the Dashboards as Code ?
November , 2021
• I used to be a developer
• Then I became an Ops person
• Chief Trolling/Travel/Technical Officer @ Inuits.eu
• Everything is a Freaking DNS Problem
• DNS : devops needs sushi
• Buiding Clouds since before the Bookstore
• Organiser of #devopsdays, #cfgmgmtcamp, #loadays, ...
Who am I ?
@krisbuytaert
• Operational Metrics
• Application Metrics
• Business Metrics
• Travel time to home during Traffic Jams
• Time between last conference
• Legobricks ordered
• Sandwiches ordered by the team
Dashboard all the things
@krisbuytaert
• People edit a Dashboard
• People clone a Dashboard
• People create a new Dashboard
• People "improve" other people's Dashboards
• People delete other people's Dashboards
class {'chaos':

ensure => present

}

Dashboards Sprawl
@krisbuytaert
• No more hacking in production
No more manual changes?
@krisbuytaert
What do we want
@krisbuytaert
Jsonnet is a DSL to build JSON Data,

JSON On steroids
Jsonnet is dead
Jsonnet is alive
Grafonet

GrafanaLib
Dashboard as Code
@krisbuytaert
Look a Squirrel !
@krisbuytaert
We are all JSON engineers

We are all YAML engineers
The average end user who wants to create his own dashboard

is not a devoops YAML Engineer
This is just too complex
Problems with Dashboard as Code
@krisbuytaert
• A dev Grafana
• Dashboard as Code
• Dashboard for Dirty Clickers
• A prod Grafana (read only)
• A way to promote from dev to prod
What we really need
@krisbuytaert
https://github.com/inuits/dashboard-manager
A command line utility to manage and promote dashboards between Grafana
instances, coupled with gitlab-ci.
Dashboard Manager for Grafana
@krisbuytaert
grafana_instances_output:

- api_key_file: production-secret

url: http://127.0.0.1:3000

name: prod

grafana_instances_input:

- api_key_file: dev-secret

url: https://remote-dev.example.com/

name: dev

http_client:

# Configured like alertmanager http client:

# https://prometheus.io/docs/alerting/latest/configuration/#http_conf
tls_config:

insecure_skip_verify: true

Example Configuration
@krisbuytaert
fetch --output-directory=OUTPUT-DIRECTORY

Fetch dashboards from input grafana.



compare --dashboards-directory=DASHBOARDS-DIRECTORY --results=RESULTS

Compare dashboards.



upload --dashboards-directory=DASHBOARDS-DIRECTORY --input-instance=INPUT-I
--dashboards=DASHBOARDS

Upload dashboards.



Usage
@krisbuytaert
Step 1 : User creates Dashboard in dev
@krisbuytaert
Step 2 : Trigger the pipeline
@krisbuytaert
Step 3 : Check downstream Created Jobs
@krisbuytaert
Step 4 : Promote the (individual) Dashboard
@krisbuytaert
Step 5 : New dashboard is in prod
@krisbuytaert
• Usable for End Users
• Version controlled
• Reproducable
• Preventing dashboard Sprawl
• Can still be used for dashboards from "real" code. to do the promotions
Conclusion
@krisbuytaert
Kris Buytaert
@krisbuytaert
kris@inuits.eu
Essensteenweg 31
2930 Brasschaat
Belgium
Contact:
info@inuits.eu
+32-3-8082105

More Related Content

What's hot

PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...
PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...
PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...
Puppet
 
Docker Best Practices Workshop
Docker Best Practices WorkshopDocker Best Practices Workshop
Docker Best Practices Workshop
Ahmed AbouZaid
 
Git and GitHub for Documentation
Git and GitHub for DocumentationGit and GitHub for Documentation
Git and GitHub for Documentation
Anne Gentle
 
JHipster overview and roadmap (August 2017)
JHipster overview and roadmap (August 2017)JHipster overview and roadmap (August 2017)
JHipster overview and roadmap (August 2017)
Julien Dubois
 
ContainerCon - Test Driven Infrastructure
ContainerCon - Test Driven InfrastructureContainerCon - Test Driven Infrastructure
ContainerCon - Test Driven Infrastructure
Yury Tsarev
 
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostustaVincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
VincitOy
 
Building A Distributed Build System at Google Scale (StrangeLoop 2016)
Building A Distributed Build System at Google Scale (StrangeLoop 2016)Building A Distributed Build System at Google Scale (StrangeLoop 2016)
Building A Distributed Build System at Google Scale (StrangeLoop 2016)
Aysylu Greenberg
 
Code review vs pull request
Code review vs pull requestCode review vs pull request
Code review vs pull request
Bryan Liu
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Control
indiver
 
An intro to git
An intro to gitAn intro to git
An intro to git
Dan Shrader
 
calmio-cicd-containers
calmio-cicd-containerscalmio-cicd-containers
calmio-cicd-containers
Balaji Janakiram
 
Ice breaker with dev ops
Ice breaker with dev opsIce breaker with dev ops
Ice breaker with dev ops
Mukta Aphale
 
CI is dead, long live CI
CI is dead, long live CICI is dead, long live CI
CI is dead, long live CI
Frédéric Lepied
 
Gitlab ci-cd
Gitlab ci-cdGitlab ci-cd
Gitlab ci-cd
Dan MAGIER
 
Distributed systems in practice, in theory (JAX London)
Distributed systems in practice, in theory (JAX London)Distributed systems in practice, in theory (JAX London)
Distributed systems in practice, in theory (JAX London)
Aysylu Greenberg
 
Jenkins vs GitLab CI
Jenkins vs GitLab CIJenkins vs GitLab CI
Jenkins vs GitLab CI
CEE-SEC(R)
 
Using Docker for Testing
Using Docker for TestingUsing Docker for Testing
Using Docker for Testing
Mukta Aphale
 
GitLab for CI/CD process
GitLab for CI/CD processGitLab for CI/CD process
GitLab for CI/CD process
HYS Enterprise
 
Hadoop Summit 2013 : Continuous Integration on top of hadoop
Hadoop Summit 2013 : Continuous Integration on top of hadoopHadoop Summit 2013 : Continuous Integration on top of hadoop
Hadoop Summit 2013 : Continuous Integration on top of hadoop
Wisely chen
 
JHipster Conf 2019 English keynote
JHipster Conf 2019 English keynoteJHipster Conf 2019 English keynote
JHipster Conf 2019 English keynote
Julien Dubois
 

What's hot (20)

PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...
PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...
PuppetConf 2016: A Tale of Two Hierarchies: Group Policy & Puppet – Matt Ston...
 
Docker Best Practices Workshop
Docker Best Practices WorkshopDocker Best Practices Workshop
Docker Best Practices Workshop
 
Git and GitHub for Documentation
Git and GitHub for DocumentationGit and GitHub for Documentation
Git and GitHub for Documentation
 
JHipster overview and roadmap (August 2017)
JHipster overview and roadmap (August 2017)JHipster overview and roadmap (August 2017)
JHipster overview and roadmap (August 2017)
 
ContainerCon - Test Driven Infrastructure
ContainerCon - Test Driven InfrastructureContainerCon - Test Driven Infrastructure
ContainerCon - Test Driven Infrastructure
 
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostustaVincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
Vincit Teatime 2015.2 - Niko Kurtti: SaaSiin pa(i)nostusta
 
Building A Distributed Build System at Google Scale (StrangeLoop 2016)
Building A Distributed Build System at Google Scale (StrangeLoop 2016)Building A Distributed Build System at Google Scale (StrangeLoop 2016)
Building A Distributed Build System at Google Scale (StrangeLoop 2016)
 
Code review vs pull request
Code review vs pull requestCode review vs pull request
Code review vs pull request
 
Make It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version ControlMake It Cooler: Using Decentralized Version Control
Make It Cooler: Using Decentralized Version Control
 
An intro to git
An intro to gitAn intro to git
An intro to git
 
calmio-cicd-containers
calmio-cicd-containerscalmio-cicd-containers
calmio-cicd-containers
 
Ice breaker with dev ops
Ice breaker with dev opsIce breaker with dev ops
Ice breaker with dev ops
 
CI is dead, long live CI
CI is dead, long live CICI is dead, long live CI
CI is dead, long live CI
 
Gitlab ci-cd
Gitlab ci-cdGitlab ci-cd
Gitlab ci-cd
 
Distributed systems in practice, in theory (JAX London)
Distributed systems in practice, in theory (JAX London)Distributed systems in practice, in theory (JAX London)
Distributed systems in practice, in theory (JAX London)
 
Jenkins vs GitLab CI
Jenkins vs GitLab CIJenkins vs GitLab CI
Jenkins vs GitLab CI
 
Using Docker for Testing
Using Docker for TestingUsing Docker for Testing
Using Docker for Testing
 
GitLab for CI/CD process
GitLab for CI/CD processGitLab for CI/CD process
GitLab for CI/CD process
 
Hadoop Summit 2013 : Continuous Integration on top of hadoop
Hadoop Summit 2013 : Continuous Integration on top of hadoopHadoop Summit 2013 : Continuous Integration on top of hadoop
Hadoop Summit 2013 : Continuous Integration on top of hadoop
 
JHipster Conf 2019 English keynote
JHipster Conf 2019 English keynoteJHipster Conf 2019 English keynote
JHipster Conf 2019 English keynote
 

Similar to Pipeline all the Dashboards as Code

stackconf 2022: Infrastructure Automation (anti) patterns
stackconf 2022: Infrastructure Automation (anti) patternsstackconf 2022: Infrastructure Automation (anti) patterns
stackconf 2022: Infrastructure Automation (anti) patterns
NETWAYS
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code Patterns
Kris Buytaert
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
panagenda
 
4D Pubs - Distributed Dynamic Document Dsplay
4D Pubs - Distributed Dynamic Document Dsplay4D Pubs - Distributed Dynamic Document Dsplay
4D Pubs - Distributed Dynamic Document Dsplay
Chris Despopoulos
 
Freelancer Weapons of mass productivity
Freelancer Weapons of mass productivityFreelancer Weapons of mass productivity
Freelancer Weapons of mass productivity
Gregg Coppen
 
DevOpsCon 2015 - DevOps in Mobile Games
DevOpsCon 2015 - DevOps in Mobile GamesDevOpsCon 2015 - DevOps in Mobile Games
DevOpsCon 2015 - DevOps in Mobile Games
Andreas Katzig
 
Serverless and you - where do i run my stateless code
Serverless and you  - where do i run my stateless codeServerless and you  - where do i run my stateless code
Serverless and you - where do i run my stateless code
Gabriela Ferrara
 
Stash – Taking Expedia to New Heights - David Williams and Christopher Pepe
Stash – Taking Expedia to New Heights - David Williams and Christopher PepeStash – Taking Expedia to New Heights - David Williams and Christopher Pepe
Stash – Taking Expedia to New Heights - David Williams and Christopher Pepe
Atlassian
 
Web and App Performance: Top Problems to avoid to keep you out of the News
Web and App Performance: Top Problems to avoid to keep you out of the NewsWeb and App Performance: Top Problems to avoid to keep you out of the News
Web and App Performance: Top Problems to avoid to keep you out of the News
Andreas Grabner
 
Serverless and you @ Women Who Code London 2020
Serverless and you  @ Women Who Code London 2020Serverless and you  @ Women Who Code London 2020
Serverless and you @ Women Who Code London 2020
Gabriela Ferrara
 
Devops For Drupal
Devops  For DrupalDevops  For Drupal
Devops For Drupal
Kris Buytaert
 
Moving to Databricks & Delta
Moving to Databricks & DeltaMoving to Databricks & Delta
Moving to Databricks & Delta
Databricks
 
Tips and Tricks for Automating Windows with Chef
Tips and Tricks for Automating Windows with ChefTips and Tricks for Automating Windows with Chef
Tips and Tricks for Automating Windows with Chef
Chef Software, Inc.
 
Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...
Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...
Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...
Andrea Fontana
 
Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...
Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...
Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...
Daniel Bryant
 
GIB2021 - Dan Probert - BizTalk Migrator Deep Dive
GIB2021 - Dan Probert - BizTalk Migrator Deep DiveGIB2021 - Dan Probert - BizTalk Migrator Deep Dive
GIB2021 - Dan Probert - BizTalk Migrator Deep Dive
probertdaniel
 
Habitat at LinuxLab IT
Habitat at LinuxLab ITHabitat at LinuxLab IT
Habitat at LinuxLab IT
Mandi Walls
 
Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++
Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++
Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++
Icinga
 
CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...
CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...
CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...
Kevin Reynen
 
qooxdoo 3.5
qooxdoo 3.5qooxdoo 3.5
qooxdoo 3.5
Martin Wittemann
 

Similar to Pipeline all the Dashboards as Code (20)

stackconf 2022: Infrastructure Automation (anti) patterns
stackconf 2022: Infrastructure Automation (anti) patternsstackconf 2022: Infrastructure Automation (anti) patterns
stackconf 2022: Infrastructure Automation (anti) patterns
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code Patterns
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
 
4D Pubs - Distributed Dynamic Document Dsplay
4D Pubs - Distributed Dynamic Document Dsplay4D Pubs - Distributed Dynamic Document Dsplay
4D Pubs - Distributed Dynamic Document Dsplay
 
Freelancer Weapons of mass productivity
Freelancer Weapons of mass productivityFreelancer Weapons of mass productivity
Freelancer Weapons of mass productivity
 
DevOpsCon 2015 - DevOps in Mobile Games
DevOpsCon 2015 - DevOps in Mobile GamesDevOpsCon 2015 - DevOps in Mobile Games
DevOpsCon 2015 - DevOps in Mobile Games
 
Serverless and you - where do i run my stateless code
Serverless and you  - where do i run my stateless codeServerless and you  - where do i run my stateless code
Serverless and you - where do i run my stateless code
 
Stash – Taking Expedia to New Heights - David Williams and Christopher Pepe
Stash – Taking Expedia to New Heights - David Williams and Christopher PepeStash – Taking Expedia to New Heights - David Williams and Christopher Pepe
Stash – Taking Expedia to New Heights - David Williams and Christopher Pepe
 
Web and App Performance: Top Problems to avoid to keep you out of the News
Web and App Performance: Top Problems to avoid to keep you out of the NewsWeb and App Performance: Top Problems to avoid to keep you out of the News
Web and App Performance: Top Problems to avoid to keep you out of the News
 
Serverless and you @ Women Who Code London 2020
Serverless and you  @ Women Who Code London 2020Serverless and you  @ Women Who Code London 2020
Serverless and you @ Women Who Code London 2020
 
Devops For Drupal
Devops  For DrupalDevops  For Drupal
Devops For Drupal
 
Moving to Databricks & Delta
Moving to Databricks & DeltaMoving to Databricks & Delta
Moving to Databricks & Delta
 
Tips and Tricks for Automating Windows with Chef
Tips and Tricks for Automating Windows with ChefTips and Tricks for Automating Windows with Chef
Tips and Tricks for Automating Windows with Chef
 
Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...
Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...
Solutions to reduce Total Cost of Setup (TCS) and simplify your life! - #iJac...
 
Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...
Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...
Devoxx UK 22: Debugging Java Microservices "Remocally" in Kubernetes with Tel...
 
GIB2021 - Dan Probert - BizTalk Migrator Deep Dive
GIB2021 - Dan Probert - BizTalk Migrator Deep DiveGIB2021 - Dan Probert - BizTalk Migrator Deep Dive
GIB2021 - Dan Probert - BizTalk Migrator Deep Dive
 
Habitat at LinuxLab IT
Habitat at LinuxLab ITHabitat at LinuxLab IT
Habitat at LinuxLab IT
 
Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++
Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++
Icinga Camp Berlin 2018 - Dev and Ops Stories - Integrations++
 
CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...
CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...
CiviCRM for Drupal Developers, Designers, and Site Builders - DrupalCamp Colo...
 
qooxdoo 3.5
qooxdoo 3.5qooxdoo 3.5
qooxdoo 3.5
 

More from Kris Buytaert

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoops
Kris Buytaert
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,Ignite
Kris Buytaert
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learning
Kris Buytaert
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live Devops
Kris Buytaert
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ?
Kris Buytaert
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
Kris Buytaert
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
Kris Buytaert
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness
Kris Buytaert
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
Kris Buytaert
 
Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?
Kris Buytaert
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
Kris Buytaert
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
Kris Buytaert
 
Open Source Monitoring in 2019
Open Source Monitoring in 2019 Open Source Monitoring in 2019
Open Source Monitoring in 2019
Kris Buytaert
 
Migrating to Puppet 5
Migrating to Puppet 5Migrating to Puppet 5
Migrating to Puppet 5
Kris Buytaert
 
Repositories as Code
Repositories as CodeRepositories as Code
Repositories as Code
Kris Buytaert
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
Kris Buytaert
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?
Kris Buytaert
 
Deploying your SaaS stack OnPrem
Deploying your SaaS stack OnPremDeploying your SaaS stack OnPrem
Deploying your SaaS stack OnPrem
Kris Buytaert
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcamp
Kris Buytaert
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as Code
Kris Buytaert
 

More from Kris Buytaert (20)

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoops
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,Ignite
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learning
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live Devops
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ?
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
 
Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
Open Source Monitoring in 2019
Open Source Monitoring in 2019 Open Source Monitoring in 2019
Open Source Monitoring in 2019
 
Migrating to Puppet 5
Migrating to Puppet 5Migrating to Puppet 5
Migrating to Puppet 5
 
Repositories as Code
Repositories as CodeRepositories as Code
Repositories as Code
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?
 
Deploying your SaaS stack OnPrem
Deploying your SaaS stack OnPremDeploying your SaaS stack OnPrem
Deploying your SaaS stack OnPrem
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcamp
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as Code
 

Recently uploaded

Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 

Recently uploaded (20)

Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 

Pipeline all the Dashboards as Code

  • 1. OSMC Kris Buytaert @krisbuytaert Pipeline all the Dashboards as Code ? November , 2021
  • 2. • I used to be a developer • Then I became an Ops person • Chief Trolling/Travel/Technical Officer @ Inuits.eu • Everything is a Freaking DNS Problem • DNS : devops needs sushi • Buiding Clouds since before the Bookstore • Organiser of #devopsdays, #cfgmgmtcamp, #loadays, ... Who am I ? @krisbuytaert
  • 3. • Operational Metrics • Application Metrics • Business Metrics • Travel time to home during Traffic Jams • Time between last conference • Legobricks ordered • Sandwiches ordered by the team Dashboard all the things @krisbuytaert
  • 4. • People edit a Dashboard • People clone a Dashboard • People create a new Dashboard • People "improve" other people's Dashboards • People delete other people's Dashboards class {'chaos': ensure => present } Dashboards Sprawl @krisbuytaert
  • 5. • No more hacking in production No more manual changes? @krisbuytaert
  • 6. What do we want @krisbuytaert
  • 7. Jsonnet is a DSL to build JSON Data, JSON On steroids Jsonnet is dead Jsonnet is alive Grafonet GrafanaLib Dashboard as Code @krisbuytaert
  • 8. Look a Squirrel ! @krisbuytaert
  • 9. We are all JSON engineers We are all YAML engineers The average end user who wants to create his own dashboard is not a devoops YAML Engineer This is just too complex Problems with Dashboard as Code @krisbuytaert
  • 10. • A dev Grafana • Dashboard as Code • Dashboard for Dirty Clickers • A prod Grafana (read only) • A way to promote from dev to prod What we really need @krisbuytaert
  • 11. https://github.com/inuits/dashboard-manager A command line utility to manage and promote dashboards between Grafana instances, coupled with gitlab-ci. Dashboard Manager for Grafana @krisbuytaert
  • 12. grafana_instances_output: - api_key_file: production-secret url: http://127.0.0.1:3000 name: prod grafana_instances_input: - api_key_file: dev-secret url: https://remote-dev.example.com/ name: dev http_client: # Configured like alertmanager http client: # https://prometheus.io/docs/alerting/latest/configuration/#http_conf tls_config: insecure_skip_verify: true Example Configuration @krisbuytaert
  • 13. fetch --output-directory=OUTPUT-DIRECTORY Fetch dashboards from input grafana. compare --dashboards-directory=DASHBOARDS-DIRECTORY --results=RESULTS Compare dashboards. upload --dashboards-directory=DASHBOARDS-DIRECTORY --input-instance=INPUT-I --dashboards=DASHBOARDS Upload dashboards. Usage @krisbuytaert
  • 14. Step 1 : User creates Dashboard in dev @krisbuytaert
  • 15. Step 2 : Trigger the pipeline @krisbuytaert
  • 16. Step 3 : Check downstream Created Jobs @krisbuytaert
  • 17. Step 4 : Promote the (individual) Dashboard @krisbuytaert
  • 18. Step 5 : New dashboard is in prod @krisbuytaert
  • 19. • Usable for End Users • Version controlled • Reproducable • Preventing dashboard Sprawl • Can still be used for dashboards from "real" code. to do the promotions Conclusion @krisbuytaert
  • 20. Kris Buytaert @krisbuytaert kris@inuits.eu Essensteenweg 31 2930 Brasschaat Belgium Contact: info@inuits.eu +32-3-8082105