Nicolas Charles, CfgMgmtCamp 2019.
More and more services expose their state, internal details and metrics to be observable, and improve overall quality of service.
But what about observing the infrastructure they are deployed, configured and maintained on?
What can we learn from that, and what do we need from configuration management to get these features and metrics?
Logs from installation is a good start, but they need centralization, aggregation and especially knowledge derivation from these - but also we need to observe these features over time, to trace changes, and correlate them with monitoring.
Rudder was built around the predicate that all actions of the configuration agent need to be traced, centralized and exposed in a meaningful way - with agents ensuring the continuous configuration of systems, and this talk will show the rationale behind this predicate, how we implemented this solution, and the benefits of this approach for the modern IT world.
How can we be sure of the continuous configuration management proper operation? How to expose factual topic-related reports to dev, sec, managers, customers...?
We believe that, in order to deliver the full business and collaboration value of continuous configuration management, the solution needs to go further than simply applying policies - it must ensure configuration reliability; prove historized application and status; share it to other teams; notify of any drift with a relevant context.
This talk will present why and how we should be concerned about transmitting factual measures on infrastructure management to all parties involved. We will also guide you through the journey to include a full-fledged reporting feature in a configuration management solution.
OSIS 2019 - Qu’apporte l’observabilité à la gestion de configuration ?RUDDER
On parle d’observabilité des services lorsque ceux-ci exposent des états et métriques internes pour améliorer la disponibilité globale.
Qu’en est-il de l’observabilité des infrastructures sur lesquelles ils sont déployés, configurés et maintenus ?
Les différents logs (centralisés, agrégés) permettent un bon début d’analyse mais il faut aussi observer les systèmes au fil de l’eau pour tracer chaque changement et les corréler avec le monitoring. Aujourd’hui, ces étapes de configuration IT devraient être prises en charge par les outils de gestion de configuration, qui deviennent la passerelle vers l’observabilité des opérations.
Nous montrerons l'intérêt de cette approche pour la gestion IT moderne avec un retour d’expérience sur les challenges de leur mise en place dans Rudder, notre solution libre d’audit et de gestion de configuration en continu.
On parle d’observabilité des services lorsque ceux-ci exposent des états et métriques internes pour améliorer la disponibilité globale.
Qu’en est-il de l’observabilité des infrastructures sur lesquelles ils sont déployés, configurés et maintenus ?
Les différents logs (centralisés, agrégés) permettent un bon début d’analyse mais il faut aussi observer les systèmes au fil de l’eau pour tracer chaque changement et les corréler avec le monitoring. Aujourd’hui, ces étapes de configuration IT devraient être prises en charge par les outils de gestion de configuration, qui deviennent la passerelle vers l’observabilité des opérations.
Nous montrerons l'intérêt de cette approche pour la gestion IT moderne avec un retour d’expérience sur les challenges de leur mise en place dans Rudder, notre solution libre d’audit et de gestion de configuration en continu.
Rudder recently got new features allowing to integrate data from various sources into the configuration policies. This talk will cover the data management workflow in Rudder, including the improvements in 4.0 and 4.1, focusing on real practical usecases.
In particular, we will go through the possible data flows: the data sources, that can be local to the server, the node or fetched from a remote API or another node, the data manipulation tools, in the server or in the policies, and finally the ways to use this data in the policies (as directive parameters, templating data, etc.)
Splunk, SIEMs, and Big Data - The Undercroft - November 2019Jonathan Singer
Guild members join us on Thursday November 14th at 6pm for our class on Splunk. Our Analyze Guild Master Jonathan Singer will be hitting on Centralized Logging, SEIM, Big Data, and much more.
Cloud computing transforms the way we can store, process and share our data. New applications and workloads are growing rapidly, which brings every day more sensitive data into the conversation about risk and what constitutes natural targets for bad actors. This presentation reflects on current best practices to address the most significant security concerns for sensitive data in the cloud, and offers participants a list of steps to achieve enterprise-grade safety with MongoDB deployments among the expanding service provider options.
How can we be sure of the continuous configuration management proper operation? How to expose factual topic-related reports to dev, sec, managers, customers...?
We believe that, in order to deliver the full business and collaboration value of continuous configuration management, the solution needs to go further than simply applying policies - it must ensure configuration reliability; prove historized application and status; share it to other teams; notify of any drift with a relevant context.
This talk will present why and how we should be concerned about transmitting factual measures on infrastructure management to all parties involved. We will also guide you through the journey to include a full-fledged reporting feature in a configuration management solution.
OSIS 2019 - Qu’apporte l’observabilité à la gestion de configuration ?RUDDER
On parle d’observabilité des services lorsque ceux-ci exposent des états et métriques internes pour améliorer la disponibilité globale.
Qu’en est-il de l’observabilité des infrastructures sur lesquelles ils sont déployés, configurés et maintenus ?
Les différents logs (centralisés, agrégés) permettent un bon début d’analyse mais il faut aussi observer les systèmes au fil de l’eau pour tracer chaque changement et les corréler avec le monitoring. Aujourd’hui, ces étapes de configuration IT devraient être prises en charge par les outils de gestion de configuration, qui deviennent la passerelle vers l’observabilité des opérations.
Nous montrerons l'intérêt de cette approche pour la gestion IT moderne avec un retour d’expérience sur les challenges de leur mise en place dans Rudder, notre solution libre d’audit et de gestion de configuration en continu.
On parle d’observabilité des services lorsque ceux-ci exposent des états et métriques internes pour améliorer la disponibilité globale.
Qu’en est-il de l’observabilité des infrastructures sur lesquelles ils sont déployés, configurés et maintenus ?
Les différents logs (centralisés, agrégés) permettent un bon début d’analyse mais il faut aussi observer les systèmes au fil de l’eau pour tracer chaque changement et les corréler avec le monitoring. Aujourd’hui, ces étapes de configuration IT devraient être prises en charge par les outils de gestion de configuration, qui deviennent la passerelle vers l’observabilité des opérations.
Nous montrerons l'intérêt de cette approche pour la gestion IT moderne avec un retour d’expérience sur les challenges de leur mise en place dans Rudder, notre solution libre d’audit et de gestion de configuration en continu.
Rudder recently got new features allowing to integrate data from various sources into the configuration policies. This talk will cover the data management workflow in Rudder, including the improvements in 4.0 and 4.1, focusing on real practical usecases.
In particular, we will go through the possible data flows: the data sources, that can be local to the server, the node or fetched from a remote API or another node, the data manipulation tools, in the server or in the policies, and finally the ways to use this data in the policies (as directive parameters, templating data, etc.)
Splunk, SIEMs, and Big Data - The Undercroft - November 2019Jonathan Singer
Guild members join us on Thursday November 14th at 6pm for our class on Splunk. Our Analyze Guild Master Jonathan Singer will be hitting on Centralized Logging, SEIM, Big Data, and much more.
Cloud computing transforms the way we can store, process and share our data. New applications and workloads are growing rapidly, which brings every day more sensitive data into the conversation about risk and what constitutes natural targets for bad actors. This presentation reflects on current best practices to address the most significant security concerns for sensitive data in the cloud, and offers participants a list of steps to achieve enterprise-grade safety with MongoDB deployments among the expanding service provider options.
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...Guglielmo Iozzia
Slides from my talk at the Hadoop User Group Ireland meetup on June 13th 2016: building a data pipeline to ingest data from sources of different nature into Hadoop in minutes (and no coding at all) using the Open Source Streamsets Data Collector tool.
Securing Your Enterprise Web Apps with MongoDB Enterprise MongoDB
Speaker: Jay Runkel, Principal Solution Architect, MongoDB
Level: 200 (Intermediate)
Track: Operations
When architecting a MongoDB application, one of the most difficult questions to answer is how much hardware (number of shards, number of replicas, and server specifications) am I going to need for an application. Similarly, when deploying in the cloud, how do you estimate your monthly AWS, Azure, or GCP costs given a description of a new application? While there isn’t a precise formula for mapping application features (e.g., document structure, schema, query volumes) into servers, there are various strategies you can use to estimate the MongoDB cluster sizing. This presentation will cover the questions you need to ask and describe how to use this information to estimate the required cluster size or cloud deployment cost.
What You Will Learn:
- How to architect a sharded cluster that provides the required computing resources while minimizing hardware or cloud computing costs
- How to use this information to estimate the overall cluster requirements for IOPS, RAM, cores, disk space, etc.
- What you need to know about the application to estimate a cluster size
Container monitoring for resource and application metrics with cAdvisor. Shipping monitoring information with the container so it is monitored irrespective of the host it runs on.
Intro to monitoring in distributed systems, cAdvisor, heapster, kubedash, kubernetes
“Lights Out”Configuration using Tivoli Netcool AutoDiscovery ToolsAntonio Rolle
Review why a CMDB is essential to and is the foundation of your BSM strategy
Outline the known challenges that require planning at the outset of a CMDB initiative
Drill down into the approach and lessons learned in the initial stages of a CMDB rollout for one of the largest financial institutions in North America
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseVictoriaMetrics
Monitoring is the key to successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast growing, high performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Altinity Ltd
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHouse Webinar Slides
Monitoring is the key to the successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open-source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast-growing, high-performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Presented by:
Roman Khavronenko, Co-Founder at VictoriaMetrics
Robert Hodges, CEO at Altinity
Deploy 22 microservices from scratch in 30 mins with GitOpsOpsta
- What do you need to deploy microservices?
- What is Docker, Kubernetes, Infrastructure, and GitOps?
- Why can GitOps help us to improve the DevOps process?
- Demo GitOps
Jirayut Nimsaeng
Founder & CEO
Opsta (Thailand) Co., Ltd.
Google DevFest 2022
Oracle Exadata 12c: Learn Online Oracle Exadata Training with Certification Material PDF, Enroll for Oracle Exadata Certification Training Course, Tutorial Videos, Attend free Demo & you will find Spiritsofts is best online training institute within reasonable fee. Exadata Database Machine: 12c Administration Workshop Ed 2
Spiritsofts is the best Training Institutes to expand your skills and knowledge. We Provides the best learning Environment. Obtain all the training by our expert professionals which is having working experience from Top IT companies.
The Training in is every thing we explained based on real time scenarios, it works which we do in companies.
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsZohar Elkayam
Oracle Week 2017 slides.
Agenda:
Basics: How and What To Tune?
Using the Automatic Workload Repository (AWR)
Using AWR-Based Tools: ASH, ADDM
Real-Time Database Operation Monitoring (12c)
Identifying Problem SQL Statements
Using SQL Performance Analyzer
Tuning Memory (SGA and PGA)
Parallel Execution and Compression
Oracle Database 12c Performance New Features
What if configuration management didn't need to be lvl60 in dev?RUDDER
Slides from Alexandre BRIANCEAU's talk at #OSSPARIS19 (Open Source Summit.
Server infrastructure automation is not simple. Several solutions have existed for several years and most of them rely on infra-as-code to achieve their mission. By the way, why infra-as-code?
And unfortunately, these solutions require strong development skills. So how can we do this when the infrastructure team does not have sufficient and, above all, homogeneous expertise? Because otherwise, beware of the "Guru Team" effect, or how the infrastructure automation to save time ends up with a huge SPOF because only one person in the team knows how it works....
I would like to discuss this together and introduce you to RUDDER briefly. RUDDER is a configuration management solution, and therefore infra-as-code, that allows you to automate your systems by relying entirely on a graphical interface to manage your configurations. Because the infrastructure is complex enough to add a layer!
Slides from Alexandre BRIANCEAU's talk at #OSSPARIS19 (Open Source Summit Paris 2019).
Security is everyone's business, an exploited breach is enough. Teams are aware of this and yet it is still as difficult as ever to be able to ensure, be confident, and reassure others (prove) that at least one party is under control.
And when it comes to server infrastructure, especially at the OS / middleware level, everything gets complicated. Even with an operational security team, it is difficult to ensure that the Information System Security Policy and security recommendations are properly implemented on all servers.
How can we be sure that our security policies are properly applied on all our servers other than through a massive and costly audit? Even if they were when they were created, how do you know if they remain perfectly compliant after a few days / weeks / months?
Let's discover together RUDDER, an open-source solution for continuous compliance based on configuration management to automatically audit and/or correct our systems.
More Related Content
Similar to What uses for observing operations of Configuration Management?
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...Guglielmo Iozzia
Slides from my talk at the Hadoop User Group Ireland meetup on June 13th 2016: building a data pipeline to ingest data from sources of different nature into Hadoop in minutes (and no coding at all) using the Open Source Streamsets Data Collector tool.
Securing Your Enterprise Web Apps with MongoDB Enterprise MongoDB
Speaker: Jay Runkel, Principal Solution Architect, MongoDB
Level: 200 (Intermediate)
Track: Operations
When architecting a MongoDB application, one of the most difficult questions to answer is how much hardware (number of shards, number of replicas, and server specifications) am I going to need for an application. Similarly, when deploying in the cloud, how do you estimate your monthly AWS, Azure, or GCP costs given a description of a new application? While there isn’t a precise formula for mapping application features (e.g., document structure, schema, query volumes) into servers, there are various strategies you can use to estimate the MongoDB cluster sizing. This presentation will cover the questions you need to ask and describe how to use this information to estimate the required cluster size or cloud deployment cost.
What You Will Learn:
- How to architect a sharded cluster that provides the required computing resources while minimizing hardware or cloud computing costs
- How to use this information to estimate the overall cluster requirements for IOPS, RAM, cores, disk space, etc.
- What you need to know about the application to estimate a cluster size
Container monitoring for resource and application metrics with cAdvisor. Shipping monitoring information with the container so it is monitored irrespective of the host it runs on.
Intro to monitoring in distributed systems, cAdvisor, heapster, kubedash, kubernetes
“Lights Out”Configuration using Tivoli Netcool AutoDiscovery ToolsAntonio Rolle
Review why a CMDB is essential to and is the foundation of your BSM strategy
Outline the known challenges that require planning at the outset of a CMDB initiative
Drill down into the approach and lessons learned in the initial stages of a CMDB rollout for one of the largest financial institutions in North America
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseVictoriaMetrics
Monitoring is the key to successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast growing, high performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Altinity Ltd
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHouse Webinar Slides
Monitoring is the key to the successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open-source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast-growing, high-performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Presented by:
Roman Khavronenko, Co-Founder at VictoriaMetrics
Robert Hodges, CEO at Altinity
Deploy 22 microservices from scratch in 30 mins with GitOpsOpsta
- What do you need to deploy microservices?
- What is Docker, Kubernetes, Infrastructure, and GitOps?
- Why can GitOps help us to improve the DevOps process?
- Demo GitOps
Jirayut Nimsaeng
Founder & CEO
Opsta (Thailand) Co., Ltd.
Google DevFest 2022
Oracle Exadata 12c: Learn Online Oracle Exadata Training with Certification Material PDF, Enroll for Oracle Exadata Certification Training Course, Tutorial Videos, Attend free Demo & you will find Spiritsofts is best online training institute within reasonable fee. Exadata Database Machine: 12c Administration Workshop Ed 2
Spiritsofts is the best Training Institutes to expand your skills and knowledge. We Provides the best learning Environment. Obtain all the training by our expert professionals which is having working experience from Top IT companies.
The Training in is every thing we explained based on real time scenarios, it works which we do in companies.
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsZohar Elkayam
Oracle Week 2017 slides.
Agenda:
Basics: How and What To Tune?
Using the Automatic Workload Repository (AWR)
Using AWR-Based Tools: ASH, ADDM
Real-Time Database Operation Monitoring (12c)
Identifying Problem SQL Statements
Using SQL Performance Analyzer
Tuning Memory (SGA and PGA)
Parallel Execution and Compression
Oracle Database 12c Performance New Features
Similar to What uses for observing operations of Configuration Management? (20)
What if configuration management didn't need to be lvl60 in dev?RUDDER
Slides from Alexandre BRIANCEAU's talk at #OSSPARIS19 (Open Source Summit.
Server infrastructure automation is not simple. Several solutions have existed for several years and most of them rely on infra-as-code to achieve their mission. By the way, why infra-as-code?
And unfortunately, these solutions require strong development skills. So how can we do this when the infrastructure team does not have sufficient and, above all, homogeneous expertise? Because otherwise, beware of the "Guru Team" effect, or how the infrastructure automation to save time ends up with a huge SPOF because only one person in the team knows how it works....
I would like to discuss this together and introduce you to RUDDER briefly. RUDDER is a configuration management solution, and therefore infra-as-code, that allows you to automate your systems by relying entirely on a graphical interface to manage your configurations. Because the infrastructure is complex enough to add a layer!
Slides from Alexandre BRIANCEAU's talk at #OSSPARIS19 (Open Source Summit Paris 2019).
Security is everyone's business, an exploited breach is enough. Teams are aware of this and yet it is still as difficult as ever to be able to ensure, be confident, and reassure others (prove) that at least one party is under control.
And when it comes to server infrastructure, especially at the OS / middleware level, everything gets complicated. Even with an operational security team, it is difficult to ensure that the Information System Security Policy and security recommendations are properly implemented on all servers.
How can we be sure that our security policies are properly applied on all our servers other than through a massive and costly audit? Even if they were when they were created, how do you know if they remain perfectly compliant after a few days / weeks / months?
Let's discover together RUDDER, an open-source solution for continuous compliance based on configuration management to automatically audit and/or correct our systems.
OW2Con - Configurations, do you prove yours?RUDDER
How can we be sure of the continuous configuration management proper operation? How to expose factual topic-related reports to dev, sec, managers, customers...?
We believe that, in order to deliver the full business and collaboration value of continuous configuration management, the solution needs to go further than simply applying policies - it must ensure configuration reliability; prove historized application and status; share it to other teams; notify of any drift with a relevant context.
This talk will present why and how we should be concerned about transmitting factual measures on infrastructure management to all parties involved. We will also guide you through the journey to include a full-fledged reporting feature in a configuration management solution.
The latest major version of the solution has brought a major new feature to the Rudder solution: a plugin ecosystem.
The Rudder software architect will present the reasons for this new feature, how it works, and what are the different plugins available.
Benoit Peccatte, CfgMgmtCamp 2019.
Benoit Peccatte started out as a developer for air traffic control systems but quickly became more interested in writing code generators to automate his job.
After meeting some smart sysadmins on the beach, he switched jobs and has been automating servers for the past decade.
He stumbled across open source in engineering school, and quickly became convinced that free software is the only way to keep software maintainable whatever happens in the future.
Benoit is now trying to automate his job on Rudder, developing features in Rudder to continuously configure and audit more and more servers.
UX challenges of a UI-centric config management toolRUDDER
Raphaël Gauthier, CfgMgmtCamp 2019.
One of Rudder’s main focuses is its comprehensive graphical user interface, which allows users to view and manage its configurations without writing a line of code.
The user experience and interface considerations for a tool as technical and complex, and with such potential to break things as a configuration management tool are certainly a challenge, and in some ways in unchartered territory. Rudder’s frontend developer will present an analysis of the situation, the issues encountered and the approach adopted for the improvement of UX and UI planned for 2019.
What happened in RUDDER in 2018 and what’s next?RUDDER
Alexis Mousset, CfgMgmtCamp 2019.
Let’s take a look at Rudder’s new features from 2018, both in terms of the features of versions 4.3 and 5.0 as well as the new documentation and our platform for building and distributing binaries.
We will then present the provisional roadmap for 2019: let’s go to Rudder 5.1 and 5.2!
Alexandre Brianceau, CfgMgmtCamp 2019.
Rudder is an open source configuration management tool that includes continuous auditing (with or without remediation), compliance info and graphs and the possibility to configure everything in the UI and/or APIs.
It has been around for more than six years and has users large (think 10 000 nodes) and small around the world.
Let’s take a moment to look at the vision that lead us here, how Rudder is different from similar tools, and what users find invaluable, nice (or annoying - I’ll be honest!).
If you’re not familiar with Rudder this is a great talk to attend to get the basics covered.
L'audit en continu : clé de la conformité démontrable (#POSS 2018)RUDDER
Présentation issue du talk pour le Paris Open Source Summit 2018 par Alexandre Brianceau dans le track Cybersécurité.
Les politiques de sécurité sont de plus en plus complexes et exigeantes à mettre en oeuvre pour les équipes opérationnelles. Comment pouvons-nous être certains que nos politiques de sécurité soient bien appliquées sur tous sos serveurs autrement qu’à travers un audit massif et coûteux ? Quand bien même le seraient-elles lors de leur création, comment savoir si elles restent parfaitement conformes après quelques jours / semaines / mois ?
Nous montrerons comment définir des règles techniques d'une politique de sécurité dans RUDDER, une solution d'automatisation de conformité informatique open source issue du monde devops où la gestion automatique de la configuration est déjà la norme. ensuite toutes les 5 minutes sur chacun des serveurs afin de remonter un résumé global permettant alors d’inspecter les problèmes qui doivent être corrigés.
Nous expliquerons également comment une politique d’audit déployée avec succès peut être imposée sur tous les systèmes avec le même outil, en passant de l’audit automatique à la remédiation automatique.
Fiabilité et conformité continues en production avec Rudder (#BBOOST 2018)RUDDER
Présentation issue du talk pour le BBOOST 2018 par Alexandre Brianceau.
Une infrastructure dont les configurations ne sont pas homogènes, surveillées et maintenues en conformité en continu finit inévitablement par dériver, entraînant failles de sécurité et incidents de production.
Alors que la fiabilité de l’IT est devenue critique, la méthode traditionnelle consistant à mener des audits tous les X mois montre ses limites : une dérive entre deux audits peut passer inaperçue et causer un incident.
RUDDER est une solution qui garantit la conformité des configurations en permanence.
Stay up - voyage d'un éditeur de logiciels libresRUDDER
Voici le retour d'expérience d'un des fondateurs Rudder sur ce que c'est qu'être entrepreneur dans les logiciels libres et les 10 ans de voyage écoulés à travers 4 étapes clés:
- la constitution de l'équipe,
- le passage par un incubateur,
- la levée de fond (ou pas),
- et la recherche d'un business model soutenable.
How we scaled Rudder to 10k, and the road to 50kRUDDER
Management graphical interface, real-time compliance and ease of use are some of Rudder core principles. When Rudder was created in 2010, hundreds of servers were considered a large installation, and the constraints and limits to manage systems were totally different than nowadays, as IT speaks in terms of thousands of nodes. I’ll present how we scaled Rudder from hundreds to 10k nodes, on each different aspect of the product: changing the way nodes talk with the Rudder server, rewriting the data model, evolving the UI, how we detected new limits - further away - and how we removed them; and made sure these limits don’t come back through tooling and testing. Finally, I’ll present the planned evolutions in upcoming releases to reach 50k managed nodes.
Rudder 4.1 was released in March 2017 with:
- an advanced feature to query external APIs and pull in node properties dynamically
the ability to add "key=value" tags to all Rules and Directives in order to categorize them
- a new API on relay servers to enable node-to-node file sharing and remote run in firewalled environments performance improvements
- a new plugin package format
Rudder 4.2 was released in September 2017 and includes the support for a new plugin that adds support for a new Windows DSC-based agent. Rudder 4.3 will include:
- Parameters for Technique Editor techniques
- ACLs on the API accounts
- Many architecture improvements
In parallel, new plugins are being developed:
- A plugin to integrate data from external APIs
- Monitoring integration with Centreon
- CMDB integration with iTop
- A reporting plugin for historized compliance
This talk will introduce these new features and show how to use them, hopefully getting you as excited as we are! Then, we will move on to explain about longer-term feature ideas we have for Rudder, and the general vision linked to future developments.
About Nicolas Charles
Nicolas is a tinkerer who likes when things just work, and tries his best to reach this goal. He started as a developer 15 years ago, and often had to reach out of this role to solve issues.
In 2010, he co-founded Normation, and he still enjoys fixing things in Rudder and at its users.
DevOps D-Day 2017 - Gestion des configurations et mise en conformité chez un ...RUDDER
En tant qu’hébergeur et infogérant, Jaguar Network est confronté à une double évolution :
Le marché attend de la part d’un Service Provider de prendre en charge une part toujours plus importante de la gestion du système d’information.
La croissance de l’entreprise entraîne une pression plus importante quantitativement (scalabilité) et qualitativement (garantir la fiabilité et la sécurité sur l’ensemble du parc géré).
Ainsi, Jaguar Network a dû trouver une solution capable de résoudre cette double problématique à laquelle de plus en plus de sociétés sont confrontées : assurer la croissance rapide du parc tout en améliorant et en garantissant la fiabilité.
Grâce à RUDDER, solution open-source française de Continuous Configuration dédiée aux contraintes de la production, l’atteinte de cet objectif a été grandement facilité. En duo avec l’éditeur de RUDDER, Jaguar Network racontera le déroulement de ce projet, de la mise en place de l’outil aux résultats constatés, en passant par l’intégration avec les autres technologies du SI.
Un retour d’expérience concret et complet sur le concept de Continuous Configuration et son implémentation avec RUDDER.
RUDDER is an easy to use, web-driven, role-based solution for IT Infrastructure Automation and Compliance. With a focus on continuously checking configurations and centralising real-time status data, RUDDER can show a high-level summary (“ISO 27001 rules are at 100%!”) and break down noncompliance issues to a deep technical level (“Host prod-web-03: SSH server configuration allows root logins”).
A few things that make RUDDER stand out:
- A simple framework allows you to extend the built-in rules to implement specific low-level configuration patterns, however complex they may be, using simple building blocks (“ensure package installed in version X,” “ensure file content,” “ensure line in file,” etc.). A graphical builder lowers the technical level required to use this.
- Each policy can be independently set to be automatically checked or enforced on a policy or host level. In Enforce mode, each remediation action is recorded, showing the value of these invisible fixes.
- RUDDER works on almost every kind of device, so you’ll be managing physical and virtual servers in the data center, cloud instances, and embedded IoT devices in the same way.
- RUDDER is designed for critical environments where a security breach can mean more than a blip in the sales stats. Built-in features include change requests, audit logs, and strong authentication.
- RUDDER relies on an agent that needs to be installed on all hosts to audit. The agent is very lightweight (10 to 20 MB of RAM at peak) and blazingly fast (it’s written in C and takes less than 10 seconds to verify 100 rules). Installation is self-contained, via a single package, and can auto-update to limit agent management burden.
- RUDDER is a true and professional open source solution—the team behind RUDDER doesn’t believe in the dual-speed licensing approach that makes you reinstall everything and promotes open source as little more than a “demo version.”
RUDDER is an established project with several 10000s of node managed, in companies from small to biggest-in-their-field. Typical deployments manage 100s to 1000s of nodes. The biggest known deployment in 2016 is about 7000 nodes.
Rudder is an easy to use, web-driven, role-based solution for IT Infrastructure Automation and Compliance. With a focus on continuously checking configurations and centralising real-time status data, Rudder can show a high-level summary (“ISO 27001 rules are at 100%!”) and break down noncompliance issues to a deep technical level (“Host prod-web-03: SSH server configuration allows root logins”).
A few things that make Rudder stand out:
- A simple framework allows you to extend the built-in rules to implement specific low-level configuration patterns, however complex they may be, using simple building blocks (“ensure package installed in version X,” “ensure file content,” “ensure line in file,” etc.). A graphical builder lowers the technical level required to use this.
- Each policy can be independently set to be automatically checked or enforced on a policy or host level. In Enforce mode, each remediation action is recorded, showing the value of these invisible fixes.
- Rudder works on almost every kind of device, so you’ll be managing physical and virtual servers in the data center, cloud instances, and embedded IoT devices in the same way.
- Rudder is designed for critical environments where a security breach can mean more than a blip in the sales stats. Built-in features include change requests, audit logs, and strong authentication.
- Rudder relies on an agent that needs to be installed on all hosts to audit. The agent is very lightweight (10 to 20 MB of RAM at peak) and blazingly fast (it’s written in C and takes less than 10 seconds to verify 100 rules). Installation is self-contained, via a single package, and can auto-update to limit agent management burden.
- Rudder is a true and professional open source solution—the team behind Rudder doesn’t believe in the dual-speed licensing approach that makes you reinstall everything and promotes open source as little more than a “demo version.”
Rudder is an established project with several 10000s of node managed, in companies from small to biggest-in-their-field. Typical deployments manage 100s to 1000s of nodes. The biggest known deployment in 2016 is about 7000 nodes.
"D'une programmation objet classique au free monad."
Cette présentation explique le cheminement suivi par le code #Scala au sein de @RudderProject, en partant d'un erzatz de Java vers plus de programmation fonctionnelle et de Développement Dirigé par les Types.
Automating the manual - feedback on including existing systems in configurati...RUDDER
When designing a new infrastructure, weaving configuration management within it is a natural solution nowadays.
However, there are many systems in the wild that are still manually managed, if managed at all; mission critical servers that can’t be shut down, systems that runs proprietary software which depend on out-of-date databases, … They may even have been configured using forgotten conventions (that can be different on different iteration of systems).
Using configuration automation tools on these system can seem like an impossible task, but it is not, and the efforts are really worth the benefits.
This talk will present feedback from a couple of projects I’ve worked on, describing how to manage these “existing, manual and critical” systems automatically, most specifically the reverse engineering of existing systems (compiling all documents, inventorying systems, devising the rules, auditing deviations), and the steps to managing them automatically.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
What uses for observing operations of Configuration Management?
1. rudder.io
What uses for observing operations of
Configuration Management?
Nicolas CHARLES
nicolas@rudder.io - @nico_charles 1
2. Are we really looking at logs?
2
I’m sure everyone here does, but...
3. No error nor change in logs means success?
3
Aren’t we missing something?
4. Getting and understanding the info is complex
4
Operators, Managers, Experts, APIs have differents needs
Frustration if we need a third party to get data
We mistrust what we don’t understand
5. Getting and understanding the info is complex
Putting errors into perspective
Errors can be expected
Errors in production can have catastrophic consequences
Errors in a Vagrant VM is much less critical
10. These concepts are core to Rudder
Everyone/thing can be an actor of configuration management
11. These concepts are core to Rudder
Technique
A set of operations & configurations to reach a state
With variables for configuration
Created by experts
13. These concepts are core to Rudder
Directive
Technique + Parameters
Defines how services must be managed
Driven by business needs, managed by admins or APIs
14. These concepts are core to Rudder
Rule
The application of Directive(s) to Group(s)
Defines the targets of the Directive(s)
Higher approach of services, managed by admins or APIs
15. Each can focus on what is relevant
15
Operators
Security Experts
16. Each can focus on what is relevant
16
Managers
APIs
"rules": [
{
"id": "32377fd7-02fd-43d0-aab7-28460a91347b",
"name": "Security rules - baseline",
"compliance": 100,
"mode": "full-compliance",
"complianceDetails": {
"successAlreadyOK": 87.47,
"successNotApplicable": 12.53
},
"directives": [
{
"id": "c16e3a90-b9d7-427d-83c1-d80e33124e4c",
"name": "CIS Benchmark 2.1.6 - rsh",
"compliance": 100.0,
"complianceDetails": {
"successAlreadyOK": 100.00
}
17. What is this compliance?
PARAM
RULE
● Id
DIRECTIVE
● Id
● (Components)
GROUP
● Id
RUDDER config
(global)
● Policy Mode
● Schedule
NODE
● Properties
● Policy Mode
● Schedule
Environmental context
● Id : . . .
● Generated : . . .
Files
Node configuration
Change request
Historisation
Historization
Event logs
18. What is this compliance?
RUDDER config
(global)
● Policy Mode
● Schedule
NODE
● Properties
● Policy Mode
● Schedule
Environmental context
● Id : . . .
● Generated : . . .
Files
Node configuration
Change request
Historisation
Event logs
PARAM
RULE
● Id
● Groups + Directives
DIRECTIVE
● Id
● Components
GROUP
● Id
Historization
19. What is this compliance?
PARAM
RULE
● Id
DIRECTIVE
● Id
● (Components)
GROUP
● Id
RUDDER config
(global)
● Policy Mode
● Schedule
NODE
● Properties
● Policy Mode
● Schedule
Environmental context
● Id : . . .
● Generated : . . .
Files
Node configuration
Change request
Historisation
Historization Event logs
20. What is this compliance?
PARAM
RULE
● Id
DIRECTIVE
● Id
● (Components)
GROUP
● Id
RUDDER config
(global)
● Policy Mode
● Schedule
NODE
● Properties
● Policy Mode
● Schedule
Environmental context
● Id : . . .
● Generated : . . .
Files
Node configuration
Change request
Historisation
Historization
Event logs
21. What is this compliance?
PARAM
RULE
● Id
DIRECTIVE
● Id
● (Components)
GROUP
● Id
RUDDER config
(global)
● Policy Mode
● Schedule
NODE
● Properties
● Policy Mode
● Schedule
Environmental context
● Id : . . .
● Generated : . . .
Files
Node configuration
Change request
Historisation
Historization
Event logs
22. What is this compliance?
22
● Id : . . .
● Generated : . . .
Files
Node configuration
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
● Signature
Get Policy
Send configuration
reports
Expected reports
(node id, config id,
timestamp)
Run reports
Historization
Compliance
historized
Store expected reports
Metadata
● Integrity
● Signature
Config
● Id
● For Rule R,
Directive D1,
Component C
23. What is this compliance?
23
● Id : . . .
● Generated : . . .
Files
Node configuration
Run reports
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
● Signature
Get Policy
Send configuration
reports
Expected reports
node id
config id
timestamp
end of validity
Historization
Compliance
historized
Store expected reports
Metadata
● Integrity
● Signature
Config
● Id
● For Rule R,
Directive D1,
Component C
24. What is this compliance?
24
● Id : . . .
● Generated : . . .
Files
Node configuration
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
● Signature
Get Policy
Send configuration
reports
Expected reports
(node id, config id,
timestamp)
Run reports
Historization
Compliance
historized
Store expected reports
Metadata
● Integrity
● Signature
Config
● Id
● For Rule R,
Directive D1,
Component C
25. What is this compliance?
25
● Id : . . .
● Generated : . . .
Files
Node configuration
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
● Signature
Get Policy
Send configuration
reports
Expected reports
(node id, config id,
timestamp)
Run reports
Historization
Compliance
historized
Store expected reports
Metadata
● Integrity
● Signature
Config
● Id
● For Rule R,
Directive D1,
Component C
26. What is this compliance?
26
● Id : . . .
● Generated : . . .
Files
Node configuration
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
● Signature
Get Policy
Send configuration
reports
Expected reports
(node id, config id,
timestamp)
Run reports
Historization
Compliance
historized
Store expected reports
Metadata
● Integrity
● Signature
Config
● Id
● For Rule R,
Directive D1,
Component C
27. Make information available
27
A lot information from inside Rudder, usable in Rudder context
Details of each run (timestamped info)
Policy generation details
Serialization of configurations
Inventories
...
30. Causality and dependencies of events
30
Diagnostic on infrastructures is hard
● Many systems
● Dependencies across systems
● Many actors involved
An issue on one component can impact hundred systems
We need to separate the causes from the symptoms
31. Causality and dependencies of events
31
Monitoring can only correlate
Causes and precedences help root cause analysis
33. Event sourcing & Tracing
33
Events happen on the whole infrastructure
Describe and analyze over systems
Order events
Contextualize
34. Event sourcing & Tracing
34
Terminology (Dapper & OpenTracing)
Trace: Description of a “transaction” as it moves through systems
Span: Named and timed operation, piece of workflow (+ tags and logs)
Span context: Trace information that accompanies the transaction
35. Event sourcing & Tracing
35
What’s in a span?
Operation name
Start & end timestamps
Tags: Set of key:value
Logs: Set of key:value
SpanContext
36. Event sourcing & Tracing
36
Temporal relationships between Spans in a single Trace
https://www.jaegertracing.io/docs/1.9/architecture/
37. Event sourcing & Tracing
37
What would be the traces?
Defining the infrastructure state is a trace
Each changes before validation is a span
Validating results in a change request closes the trace
Computing the nodes configurations is a trace
Computing targets, overrides and generating files are spans
Closes with the serialization of the nodes configurations in database
Each run on an node is a trace
Each configuration check is a span
38. Event sourcing & Tracing
38
RULE
● Id
DIRECTIVE
● Id
GROUP
● Id
Environmental
context
● Id : . . .
● Generated : . .
● Commit id.
Files
Node configuration
Change request
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
RUN
● Reports
● Reports
● ...
● ...
METADATA
● node id
● config id
● run timestamp
● Signature
Get config
Send configuration
reports
Expected reports
(node id, config id,
timestamp)
Run reports
Historisation
Store expected reports
Metadata
● Integrity
● CommitId
● Signature
Config
● For Rule R,
Directive D1,
Component C
Events
Commit Id
Defining state
Trace + Spans
Trace
Run: Trace
Each step: span
Message bus
39. Event sourcing & Tracing
39
● Id : . . .
● Generated : . .
● Commit id.
Files
Node configuration
METADATA
● node id
● config id
● run timestamp
RUN
METADATA
Signature
Get config
Send configuration
reports
Expected reports
(node id, config id,
timestamp)
Run reports
Store expected reports
Metadata
● Integrity
● CommitId
● Signature
Config
● For Rule R,
Directive D1,
Component C
Trace
Message bus
Run: Trace
Each step: span
Compliance
CMDB Hooks
Monitoring
40. Event sourcing & Tracing
40
Store Traces & Events:
● Integrate with systems in place
● Many tools are compatible with OpenTracing
Correlate with non-observable systems
44. Closing thoughts
44
What can we do of these billions events?
Reactive approach
Query, search and analyze traces in case of problems
45. Closing thoughts
45
What can we do of these billions events?
Proactive approach
Process mining: Machine Learning on these events
Detect unusual behaviours
Outliers
Inconsistencies across systems
48. Security?
48
Events, trace and logs hold critical data
Within a unique system, security can be built-in
AuthN/AuthZ
For distributed system, it’s much harder
Who can see what?
Who defines and enforces the authorizations?
Tags on events for authorizations
50. rudder.io
What uses for observing operations of
Configuration Management?
Nicolas CHARLES
nicolas@rudder.io - @nico_charles 50
51. Event sourcing & Tracing
51
Temporal relationships between Spans in a single Trace
––|–––––––|–––––––|–––––––|–––––––|–––––––|–––––––|–––––––|–> time
[Span A···················································]
[Span B··············································]
[Span D··········································]
[Span C········································]
[Span E·······] [Span F··] [Span G··] [Span H··]
https://opentracing.io/specification/
52. Event sourcing & Tracing
52
Every components need to know the context
● Carry the Span Context along each events
Add some information for each events
● Save on logging thanks to context
Send these traces on message bus