This talk looks at the evolution of monitoring over time, the ways in which you can approach monitoring, where Prometheus fit into all this, and how Prometheus itself has grown over time.
Prometheus: From Berlin to Bonanza (Keynote CloudNativeCon+Kubecon Europe 2017)Brian Brazil
From its humble beginnings right here in Berlin in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will go over the core ideas behind Prometheus, give a brief tour of its end-to-end feature set and show how these combine with other CNCF projects to allow you to scale your systems and culture in a dynamic cloud native world.
If you're looking for help with Prometheus, contact us at prometheus@robustperception.io
Prometheus is a next-generation monitoring system. It lets you see you not just what your systems look like from the outside, but also gives visibility into the internals and business aspects of your systems. This allows everyone to benefit, including both operations and developers. This talk will look at the concepts behind monitoring with Prometheus, how it's designed, why it's suitable for Cloud Native environments and how you can get involved.
What does "monitoring" mean? (FOSDEM 2017)Brian Brazil
Monitoring can mean very different things to different people, and this often leads to confusion and misunderstandings. There are many offerings both free software and commercials, and it's not always clear where each fits in the bigger picture. This talk will look a bit at the history of monitoring, and then into the general categories of Metrics, Logs, Profiling and Distributed tracing and how each of these is important in Cloud-based environment.
Video: https://www.youtube.com/watch?v=hCBGyLRJ1qo
Prometheus for Monitoring Metrics (Percona Live Europe 2017)Brian Brazil
From its humble beginnings in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will provide an overview of the core ideas behind Prometheus and its feature set.
Evolving Prometheus for the Cloud Native World (FOSDEM 2018)Brian Brazil
As the industry moves towards more cloud based and containerised solutions such as Kubernetes, monitoring tools have to keep up. These new environments are far more dynamic than the hand-maintained machines of old, requiring more sophisticated and scalable approaches. This talk will look at how Prometheus has evolved over the past 5 years to be better able to cope with these challenges, including the 2.0 release and practices that we encourage in a cloud native world.
Evolution of the Prometheus TSDB (Percona Live Europe 2017)Brian Brazil
Prometheus is a monitoring system with a custom time series database at its core. Prometheus 2.0 features the 3rd major iteration of this database. This talk will look at how it has evolved, and how it fits into the goal of doing metrics-based monitoring.
Microservices and Prometheus (Microservices NYC 2016)Brian Brazil
If you'd like to learn more about Prometheus, contact us at prometheus@robustperception.io or follow us on twitter at https://twitter.com/RobustPerceiver
Prometheus is a next-generation monitoring system designed for microservices. This talk will look at what's the best way to monitor your microservices, which metrics you should care about, how to have useful alerts and how Prometheus empowers you to do things the right way.
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Brian Brazil
Often what you monitor and get alerted on is defined by your tools, rather than what makes the most sense to you and your organisation. Alerts on metrics such as CPU usage which are noisy and rarely spot real problems, while outages go undetected. Monitoring systems can also be challenging to maintain, and overall provide a poor return on investment.
In the past few years several new monitoring systems have appeared with more powerful semantics and which are easier to run, which offer a way to vastly improve how your organisation operates Prometheus is one such system. This talk will look at the monitoring ideal and how whitebox monitoring with a time series database, multi-dimensional labels and a powerful querying/alerting language can free you from midnight pages.
Prometheus: From Berlin to Bonanza (Keynote CloudNativeCon+Kubecon Europe 2017)Brian Brazil
From its humble beginnings right here in Berlin in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will go over the core ideas behind Prometheus, give a brief tour of its end-to-end feature set and show how these combine with other CNCF projects to allow you to scale your systems and culture in a dynamic cloud native world.
If you're looking for help with Prometheus, contact us at prometheus@robustperception.io
Prometheus is a next-generation monitoring system. It lets you see you not just what your systems look like from the outside, but also gives visibility into the internals and business aspects of your systems. This allows everyone to benefit, including both operations and developers. This talk will look at the concepts behind monitoring with Prometheus, how it's designed, why it's suitable for Cloud Native environments and how you can get involved.
What does "monitoring" mean? (FOSDEM 2017)Brian Brazil
Monitoring can mean very different things to different people, and this often leads to confusion and misunderstandings. There are many offerings both free software and commercials, and it's not always clear where each fits in the bigger picture. This talk will look a bit at the history of monitoring, and then into the general categories of Metrics, Logs, Profiling and Distributed tracing and how each of these is important in Cloud-based environment.
Video: https://www.youtube.com/watch?v=hCBGyLRJ1qo
Prometheus for Monitoring Metrics (Percona Live Europe 2017)Brian Brazil
From its humble beginnings in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will provide an overview of the core ideas behind Prometheus and its feature set.
Evolving Prometheus for the Cloud Native World (FOSDEM 2018)Brian Brazil
As the industry moves towards more cloud based and containerised solutions such as Kubernetes, monitoring tools have to keep up. These new environments are far more dynamic than the hand-maintained machines of old, requiring more sophisticated and scalable approaches. This talk will look at how Prometheus has evolved over the past 5 years to be better able to cope with these challenges, including the 2.0 release and practices that we encourage in a cloud native world.
Evolution of the Prometheus TSDB (Percona Live Europe 2017)Brian Brazil
Prometheus is a monitoring system with a custom time series database at its core. Prometheus 2.0 features the 3rd major iteration of this database. This talk will look at how it has evolved, and how it fits into the goal of doing metrics-based monitoring.
Microservices and Prometheus (Microservices NYC 2016)Brian Brazil
If you'd like to learn more about Prometheus, contact us at prometheus@robustperception.io or follow us on twitter at https://twitter.com/RobustPerceiver
Prometheus is a next-generation monitoring system designed for microservices. This talk will look at what's the best way to monitor your microservices, which metrics you should care about, how to have useful alerts and how Prometheus empowers you to do things the right way.
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Brian Brazil
Often what you monitor and get alerted on is defined by your tools, rather than what makes the most sense to you and your organisation. Alerts on metrics such as CPU usage which are noisy and rarely spot real problems, while outages go undetected. Monitoring systems can also be challenging to maintain, and overall provide a poor return on investment.
In the past few years several new monitoring systems have appeared with more powerful semantics and which are easier to run, which offer a way to vastly improve how your organisation operates Prometheus is one such system. This talk will look at the monitoring ideal and how whitebox monitoring with a time series database, multi-dimensional labels and a powerful querying/alerting language can free you from midnight pages.
Anatomy of a Prometheus Client Library (PromCon 2018)Brian Brazil
Prometheus client libraries are notably different from most other options in the space. In order to get the best insights into your applications it helps to know how they are designed, and why they are designed that way. This talk will look at how client libraries are structured, how that makes them easy to use, some tips for instrumentation, and why you should use them even if you aren't using Prometheus.
Prometheus: A Next Generation Monitoring System (FOSDEM 2016)Brian Brazil
A look at how Prometheus's instrumentation, data model, query language, manageability and reliability make it a next generation solution.
Video: https://www.youtube.com/watch?v=cwRmXqXKGtk
Contact us: prometheus@robustperception.io
Prometheus for Monitoring Metrics (Fermilab 2018)Brian Brazil
From its humble beginnings in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will give an overview of the core ideas behind Prometheus, its feature set and how it has grown to met the challenges of modern cloud-based systems.
An Introduction to Prometheus (GrafanaCon 2016)Brian Brazil
Often what you monitor and get alerted on is defined by your tools, rather than what makes the most sense to you and your organisation. Alerts on metrics such as CPU usage which are noisy and rarely spot real problems, while outages go undetected. Monitoring systems can also be challenging to maintain, and overall provide a poor return on investment.
In the past few years several new monitoring systems have appeared with more powerful semantics and which are easier to run, which offer a way to vastly improve how your organisation operates and prepare you for a Cloud Native environment. Prometheus is one such system. This talk will look at the monitoring ideal and how whitebox monitoring with a time series database, multi-dimensional labels and a powerful querying/alerting language can free you from midnight pages.
Cloud Native Night August 2016, Munich: Talk by Julius Volz (@juliusvolz, Co-founder at Prometheus).
Join our Meetup: www.meetup.com/cloud-native-muc
Abstract: This talk is on monitoring dynamic cloud environments with Prometheus.
Prometheus Design and Philosophy by Julius Volz at Docker Distributed System Summit
Prometheus - https://github.com/Prometheus
Liveblogging: http://canopy.mirage.io/Liveblog/MonitoringDDS2016
Ansible at FOSDEM (Ansible Dublin, 2016)Brian Brazil
At FOSDEM 2016 we used Ansible for the first time to manage the infrastructure. This talk looks at how we did that, and tips for getting the most out of your Ansible setup.
OpenMetrics: What Does It Mean for You (PromCon 2019, Munich)Brian Brazil
The OpenMetrics format intends to standardise metric exposition, making it easy for both those developing and operating systems to monitor them. It is however a new format. Will it be supported by your monitoring system? Will you need to rewrite your existing instrumentation? What's needed to transition? What about 3rd party systems you don't control? How does this differ and expand, and improve on the existing Prometheus format? This session will cover all of these questions.
Prometheus is a open-source time series database with a powerful query language designed for operational monitoring.
Contact us at prometheus@robustperception.io
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)Brian Brazil
Prometheus is a next-generation monitoring system with a time series database at it's core. Once you have a time series database, what do you do with it though? This talk will look at getting data in, and more importantly how to use the data you collect productively.
Contact us at prometheus@robustperception.io
Container environments make it easy to deploy hundreds of microservices in today’s infrastructures. Monitoring thousands of metrics efficiently introduces new challenges to not lose insight, avoid alert fatigue and maintain a high development velocity. In this talk I’ll present an overview of important metrics including the 4 golden signals, discuss strategies to organize alerting efficiently, give insight into SoundCloud’s monitoring history and highlight a few success and failure stories.
Monitoring Cloud Native Applications with PrometheusJacopo Nardiello
This talk is a quick intro to Prometheus with an overview on all its components. The presentation points to a generally available demo so that you can see all its components in action.
Introduction to Prometheus and Cortex (WOUG)Weaveworks
Have you been wanting to learn about open source Prometheus? Prometheus contributor Bryan Boreham will give you an intro about top things to know about the open source monitoring solution (which was the second project after Kubernetes to go into the CNCF). Bryan will also talk about the value of Cortex. Cortex is an open source project in the CNCF sandbox (and started by Weaveworks) that extends Prometheus by making horizontal scaling and long-term storage possible. He will also cover a little bit about PromQL, the Prometheus Query Language, and key use cases to understand the power of PromQL.
Slides used in following Udemy training: https://www.udemy.com/course/monitoring-and-alerting-with-prometheus/?referralCode=6E2F738124DB09FA4C21
Prometheus is the leading open-source monitoring system that can collect metrics from all your systems, including Linux servers, Windows Servers, Database Servers and any application you have written. It's inspired on Google's Borgmon, which uses time-series data as a datasource, to then send alerts based on this data.
This course will show you how to install and configure Prometheus on a Linux server. This course will use a VM on DigitalOcean, but you can install Prometheus on any modern Linux OS. We'll show you how to make visualizations (graphs) using Grafana. When building these graphs, you'll get to know PromQL, the language to query Prometheus and get meaningful data displayed. You'll also learn how to setup alerts to receive notifications when something goes wrong. Lastly, we have a section on use-cases to showcase you some real world examples.
Skynet project: Monitor, analyze, scale, and maintain a system in the CloudSylvain Kalache
The goal of Skynet is to avoid human doing repetitive things and make a system doing them in a better way. System automation should be the way to go for any system management so that human can focus on stuff that really matters.
Related blog post for more informations https://engineering.linkedin.com/slideshare/skynet-project-_-monitor-scale-and-auto-heal-system-cloud
RedLambda (USA)
Red Lambda is an innovation company from Florida with 8 awarded patents and 8 others pending final award; focused on Cloud Security and Big Data Analytics technologies.
Red Lambda offers one unified, seamlessly integrated platform and a suite of bold solutions unlike anything in the industry today.
Anatomy of a Prometheus Client Library (PromCon 2018)Brian Brazil
Prometheus client libraries are notably different from most other options in the space. In order to get the best insights into your applications it helps to know how they are designed, and why they are designed that way. This talk will look at how client libraries are structured, how that makes them easy to use, some tips for instrumentation, and why you should use them even if you aren't using Prometheus.
Prometheus: A Next Generation Monitoring System (FOSDEM 2016)Brian Brazil
A look at how Prometheus's instrumentation, data model, query language, manageability and reliability make it a next generation solution.
Video: https://www.youtube.com/watch?v=cwRmXqXKGtk
Contact us: prometheus@robustperception.io
Prometheus for Monitoring Metrics (Fermilab 2018)Brian Brazil
From its humble beginnings in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will give an overview of the core ideas behind Prometheus, its feature set and how it has grown to met the challenges of modern cloud-based systems.
An Introduction to Prometheus (GrafanaCon 2016)Brian Brazil
Often what you monitor and get alerted on is defined by your tools, rather than what makes the most sense to you and your organisation. Alerts on metrics such as CPU usage which are noisy and rarely spot real problems, while outages go undetected. Monitoring systems can also be challenging to maintain, and overall provide a poor return on investment.
In the past few years several new monitoring systems have appeared with more powerful semantics and which are easier to run, which offer a way to vastly improve how your organisation operates and prepare you for a Cloud Native environment. Prometheus is one such system. This talk will look at the monitoring ideal and how whitebox monitoring with a time series database, multi-dimensional labels and a powerful querying/alerting language can free you from midnight pages.
Cloud Native Night August 2016, Munich: Talk by Julius Volz (@juliusvolz, Co-founder at Prometheus).
Join our Meetup: www.meetup.com/cloud-native-muc
Abstract: This talk is on monitoring dynamic cloud environments with Prometheus.
Prometheus Design and Philosophy by Julius Volz at Docker Distributed System Summit
Prometheus - https://github.com/Prometheus
Liveblogging: http://canopy.mirage.io/Liveblog/MonitoringDDS2016
Ansible at FOSDEM (Ansible Dublin, 2016)Brian Brazil
At FOSDEM 2016 we used Ansible for the first time to manage the infrastructure. This talk looks at how we did that, and tips for getting the most out of your Ansible setup.
OpenMetrics: What Does It Mean for You (PromCon 2019, Munich)Brian Brazil
The OpenMetrics format intends to standardise metric exposition, making it easy for both those developing and operating systems to monitor them. It is however a new format. Will it be supported by your monitoring system? Will you need to rewrite your existing instrumentation? What's needed to transition? What about 3rd party systems you don't control? How does this differ and expand, and improve on the existing Prometheus format? This session will cover all of these questions.
Prometheus is a open-source time series database with a powerful query language designed for operational monitoring.
Contact us at prometheus@robustperception.io
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)Brian Brazil
Prometheus is a next-generation monitoring system with a time series database at it's core. Once you have a time series database, what do you do with it though? This talk will look at getting data in, and more importantly how to use the data you collect productively.
Contact us at prometheus@robustperception.io
Container environments make it easy to deploy hundreds of microservices in today’s infrastructures. Monitoring thousands of metrics efficiently introduces new challenges to not lose insight, avoid alert fatigue and maintain a high development velocity. In this talk I’ll present an overview of important metrics including the 4 golden signals, discuss strategies to organize alerting efficiently, give insight into SoundCloud’s monitoring history and highlight a few success and failure stories.
Monitoring Cloud Native Applications with PrometheusJacopo Nardiello
This talk is a quick intro to Prometheus with an overview on all its components. The presentation points to a generally available demo so that you can see all its components in action.
Introduction to Prometheus and Cortex (WOUG)Weaveworks
Have you been wanting to learn about open source Prometheus? Prometheus contributor Bryan Boreham will give you an intro about top things to know about the open source monitoring solution (which was the second project after Kubernetes to go into the CNCF). Bryan will also talk about the value of Cortex. Cortex is an open source project in the CNCF sandbox (and started by Weaveworks) that extends Prometheus by making horizontal scaling and long-term storage possible. He will also cover a little bit about PromQL, the Prometheus Query Language, and key use cases to understand the power of PromQL.
Slides used in following Udemy training: https://www.udemy.com/course/monitoring-and-alerting-with-prometheus/?referralCode=6E2F738124DB09FA4C21
Prometheus is the leading open-source monitoring system that can collect metrics from all your systems, including Linux servers, Windows Servers, Database Servers and any application you have written. It's inspired on Google's Borgmon, which uses time-series data as a datasource, to then send alerts based on this data.
This course will show you how to install and configure Prometheus on a Linux server. This course will use a VM on DigitalOcean, but you can install Prometheus on any modern Linux OS. We'll show you how to make visualizations (graphs) using Grafana. When building these graphs, you'll get to know PromQL, the language to query Prometheus and get meaningful data displayed. You'll also learn how to setup alerts to receive notifications when something goes wrong. Lastly, we have a section on use-cases to showcase you some real world examples.
Skynet project: Monitor, analyze, scale, and maintain a system in the CloudSylvain Kalache
The goal of Skynet is to avoid human doing repetitive things and make a system doing them in a better way. System automation should be the way to go for any system management so that human can focus on stuff that really matters.
Related blog post for more informations https://engineering.linkedin.com/slideshare/skynet-project-_-monitor-scale-and-auto-heal-system-cloud
RedLambda (USA)
Red Lambda is an innovation company from Florida with 8 awarded patents and 8 others pending final award; focused on Cloud Security and Big Data Analytics technologies.
Red Lambda offers one unified, seamlessly integrated platform and a suite of bold solutions unlike anything in the industry today.
Cartographer, or Building A Next Generation Management Frameworkansmtug
Dr. Bobby Krupczak's slides about the Cartographer management agent and the underlying XMP management framework. Presented at the February 10, 2009 meeting of the Atlanta Network and Systems Management Technical User Group (ANSMTUG).
Microservices architecture involves many services that are being distributed over the network resulting in many more ways of failure. This session will try to cover the available tools that can help you when designing/building such distributed system in Go
You are already the Duke of DevOps: you have a master in CI/CD, some feature teams including ops skills, your TTM rocks ! But you have some difficulties to scale it. You have some quality issues, Qos at risk. You are quick to adopt practices that: increase flexibility of development and velocity of deployment. An urgent question follows on the heels of these benefits: how much confidence we can have in the complex systems that we put into production? Let’s talk about the next hype of DevOps: SRE, error budget, continuous quality, observability, Chaos Engineering.
Effective Data Erasure and Anti Forensics Techniquesijtsrd
Deleting sensitive data after usage is just as important as storing of data in a safe location. In the verge of cyber attacks such as data theft happening, it is best to delete or purge or destroy unwanted sensitive data after its use as soon as possible. Data stored offline, for example in hard disks are just as prone to get stolen as the data stored online. For destroying the data to ensure cybercriminals should not get hold of this, techniques such as Data Wiping and Anti Forensics are used. A study is done on how these techniques can be used to the advantage of our system and against the cyber criminals. Anand V | Dr. MN Nachappa "Effective Data Erasure and Anti-Forensics Techniques" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-1 , December 2020, URL: https://www.ijtsrd.com/papers/ijtsrd38043.pdf Paper URL : https://www.ijtsrd.com/computer-science/computer-security/38043/effective-data-erasure-and-antiforensics-techniques/anand-v
NetFlow Auditor Anomaly Detection Plus Forensics February 2010 08NetFlowAuditor
NetFlow Auditor software uses NetFlow and sFlow to detect anomalies & analyze full network traffic forensics. The objective of our software is to provide easy to use full-featured anomaly detection and analysis of Flows to quickly identify who is doing what, where, when, with whom and for how long on a network and provide alerts, scheduled reports, SNMP Traps and or filter lists. It allows organizations to quickly identify and alert on network anomalies to help resolve performance problems and manage network security and compliance across business services and applications, dramatically reducing the risk of potential downtime.
Infrastructure - a journey from datacentres to cloudEqual Experts
What is infrastructure, and how do I avoid it forever? Where does the software that runs so much of the world, actually run? In this talk, we look at the terms "infrastructure" and "platform", what they currently mean and how they are built and managed; we rant about how bad a metaphor "The Cloud" is; and we speculate wildly about the future for our servers, our planet and ourselves
You’ve probably heard the statement that there is no cloud, there’s just somebody else’s computer. How can we monitor what we don’t own?
Developers and operations teams are increasingly relying on cloud providers to manage and operate their infrastructure. While this can offer many benefits, it also presents new challenges when it comes to observability. In this talk, we’ll explore the unique challenges of observability in a cloud-native environment, and discuss some best practices for ensuring that you can effectively monitor and troubleshoot your applications, even when you don’t have direct access to the underlying infrastructure.
We’ll begin by discussing the basic principles of observability in a cloud-native context, including the importance of monitoring not just the application itself, but also the underlying infrastructure and the interactions between different components. We’ll then explore some common challenges that can arise when it comes to observability in a cloud-native environment, including issues with data access and the need to deal with large volumes of data from multiple sources.
We’ll also discuss some practical strategies for addressing these challenges, including the use of cloud-native observability tools such as Kubernetes metrics and logging frameworks, as well as best practices for configuring and deploying these tools effectively. We’ll also explore the role of observability in incident response and how it can help teams quickly diagnose and resolve issues in a cloud-native environment.
Whether you’re just getting started with cloud-native observability or you’re looking to take your observability practices to the next level, this talk will provide valuable insights and practical tips for ensuring that you can effectively monitor and troubleshoot your applications, even when they’re running on somebody else’s computer.
Performance monitoring and call tracing in microservice environmentsMartin Gutenbrunner
Performance analysis can easily be done with on-board tools of nearly any programming language. In microservice environments, the real challenge is not in single, high-performing services, but in resiliently running a complex ecosystem of many services.This talk will introduce open-source tools for analysis and call tracing. Concluding, we will briefly get to know Dynatrace Ruxit - a commercial alternative. After this session, the audience will know about how to get started in performance analysis and call-tracing and some according tools.
Similar to Evolution of Monitoring and Prometheus (Dublin 2018) (20)
Evaluating Prometheus Knowledge in Interviews (PromCon 2018)Brian Brazil
With the growth in usage of Prometheus and increased need to hire those with relevant skills, the need to be able to evaluate Prometheus knowledge is important. In this talk I'll show how standard interview questions from related fields can be applied.
Staleness and Isolation in Prometheus 2.0 (PromCon 2017)Brian Brazil
The biggest semantic change in Prometheus 2.0 is the new staleness handling. This long awaited feature means there's no longer a fixed 5 minute staleness. Now time series go stale when they're no longer exposed, and targets that no longer exist don't hang around for a full 5 minutes. Learn about how it works and how to take advantage of it.
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)Brian Brazil
Counters are one of the two core metric types in Prometheus, allowing for tracking of request rates, error ratios and other key measurements. Learn why are they designed the way they are, how client libraries implement them and how rate() works.
If you'd like more information about Prometheus, contact us at prometheus@robustperception.io
Provisioning and Capacity Planning (Travel Meets Big Data)Brian Brazil
Ever worried that you’ll have an outage someday because your production servers can’t handle increased user traffic?
Then this workshop will help put you at ease! Learn the foundations and how to apply it to your services.
At the end of the workshop you will be able to:
– Estimate how much spare capacity you have in less than 5 minutes
– Estimate how much runway that capacity provides
– Determine how many servers you need
– Spot common potential problems as you scale
In the glorious future, cancer will be cured, world hunger will solved and all because everything was directly instrumented for Prometheus. Until then however, we need to write exporters. This talk will look at how to go about this and all the tradeoffs involved in writing a good exporter.
An Exploration of the Formal Properties of PromQLBrian Brazil
Prometheus is often considered in a production sense. But what about the more formal and academic aspects? Is PromQL interesting from a Computer Science standpoint?
Labels are at the core of Prometheus's dimensional data model. The Prometheus server and its surrounding ecosystem components all either attach, modify, or act on labels in various ways. In this talk, Brian explains the entire life cycle of labels, including their generation in the client libraries, their transformation in relabeling, as well as their use in service discovery and alerting.
Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)Brian Brazil
Prometheus is a next-generation monitoring system. Since being publicly announced last year it has seen wide-spread interest and adoption. This talk will look at the concepts behind monitoring with Prometheus, and how to use it with Kubernetes which has direct support for Prometheus.
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...Brian Brazil
Monitoring should be part of your solution, not a problem. This lightening talk takes a brief look at the ideas behind Inclusive Monitoring and how to use them with Python.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
2. Who am I?
Engineer passionate about running software reliably in production.
● One of the main developers of Prometheus
● Author of Prometheus: Up&Running
● Founder of Robust Perception
● Based in Dublin
3. Historical Roots
A lot of what we do today for monitoring is based on tools and techniques that
were awesome decades ago.
A lot of our practices still come from that time, and often we are still feeding the
machine with human blood.
Treating systems as pets rather than cattle.
4. A little history - MRTG and RRD
In 1994 Tobias Oetiker created a perl script, which became MRTG 1.0 in 1995.
Used to graph metrics from SNMP or external programs. Stored data in constantly
rewritten ASCII file.
MRTG 2.0 moved some code to C, released 1997.
RRD started in 1997 by Tobias Oetiker to further improve performance, released
1999.
Many tools use/used RRD, e.g. Graphite and Munin.
6. A little history - Nagios
Written initially by Ethan Galstad in 1996. MS-DOS application to do pings.
Started more properly in 1998, first release in 1999 as NetSaint (renamed in 2002
for legal reasons).
Runs scripts on a regular basis, and sends alerts based on their exit code.
Many projects inspired by/based off Nagios such as Icinga, Sensu and Zmon.
8. Historical Heritage
These very popular tools and their offspring left us with a world where graphing is
whitebox and alerting is blackbox, and they are separate concerns.
They come from a world where machines are pets, and services tend to live on
one machine.
They come from a world where even slight deviance would be immediately
jumped upon by heroic engineers in a NOC.
We need a new perspective in a cloud native environment.
9. What is Different Now?
It's no longer one service on one machine that will live there for years.
Services are dynamically assigned to machines, and can be moved around on an
hourly basis.
Microservices rather than monoliths mean more services created more often.
More dynamic, more churn, more to monitor.
10. So what should be monitoring?
We need a view that goes beyond alerting, graphing and jumping on every
potential problem.
We need to consider additional data sources, such as logs and browser events.
What about looking at the problem statement rather than what the tools can give
us?
What is the ultimate goal of all this "monitoring"?
11. Why do we monitor?
● Know when things go wrong
● Be able to debug and gain insight
● Trending to see changes over time
● Plumbing data to other systems/processes
12. Knowing when things go wrong
The first thing many people think of you say monitoring is alerting.
What is the wrongness we want to detect and alert on?
A blip with no real consequence, or a latency issue affecting users?
13. Symptoms vs Causes
Humans are limited in what they can handle.
If you alert on every single thing that might be a problem, you'll get overwhelmed
and suffer from alert fatigue.
Key problem: You care about things like user facing latency. There are hundreds
of things that could cause that.
Alerting on every possible cause is a Sisyphean task, but alerting on the symptom
of high latency is just one alert.
14. Example: CPU usage
Some monitoring systems don't allow you to alert on the latency of your servers.
The closest you can get is CPU usage.
False positives due to e.g. logrotate running too long.
False negatives due to deadlocks.
End result: Spammy alerts which operators learn to ignore, missing real problems.
15. Human attention is limited
Alerts should require intelligent human action!
Alerts should relate to actual end-user problems!
Your users don't care if a machine has a load of 4.
They care if they can't view their cat videos.
16. Debugging to Gain Insight
After you receive an alert notification you need to investigate it.
How do you work from a high level symptom alert such as increased latency?
You methodically drill down through your stack with dashboards to find the
subsystem that's the cause.
Break out more tools as you drill down into the suspect process.
18. Trending and Reporting
Alerting and debugging is short term.
Trending is medium to long term.
How is cache hit rate changing over time?
Is anyone still using that obscure feature?
When will I need more machines?
19. Plumbing
When all you have is a hammer, everything starts to look like a nail.
Often it'd be really convenient to use a monitoring system as a data transport as
part of some other process (often a control loop of some form).
This isn't monitoring, but it's going to happen.
If it's ~free, it's not necessarily even a bad idea.
20. How do we monitor?
Now knowing the three general goals of monitoring (plus plumbing), how do we go
about it?
What data do we collect? How do we process it?
What tradeoffs do we make?
Monitoring resources aren't free, so everything all the time is rarely an option.
21. The core is the Event
Events are what monitoring systems work off.
An event might be a HTTP request coming in, a packet being sent out, a library
call being made or a blackbox probe failing.
Events have context, such as the customer ultimately making the request, which
machines are involved, and how much data is being processed.
A single event might accumulate thousands of pieces of context as it goes through
the system, and there can be millions of events per second.
22. How do we monitor?
We're going to look at 4 classes of approaches to monitoring.
● Profiling
● Metrics
● Logs
● Distributed Tracing
When someone says "monitoring" they often have one of these stuck in their
head, however they're all complementary with different tradeoffs.
23. Profiling
Profiling is using tools like tcpdump, gdb, strace, dtrace, BPF and pretty much
anything Brendan Gregg talks about.
They give you very detailed information about individual events.
So detailed that you can't keep it all, so use must be targeted and temporary.
Great for debugging if have an idea what's wrong, not so much for anything else.
24. Metrics
Metrics make the tradeoff that we're going to ignore individual events, but track
how often particular contexts show up. Examples: Prometheus, Graphite.
For example you wouldn't track the exact time each HTTP request came in, but
you do track enough to tell that there were 3594 in the past minute.
And 14 of them failed. And 85 were for the login subsystem. And 5.4MB was
transferred. With 2023 cache hits.
And one of those requests hit a weird code path everyone has forgotten about.
25. Metric Tradeoffs
Metrics give you breadth.
You can easily track tens of thousands of metrics, but you can't break them out by
too many dimensions. E.g. breaking out metrics by email address is rarely a good
idea.
Great for figuring out what's generally going on, writing meaningful alerts,
narrowing down the scope of what needs debugging.
Not so great for tracking individual events.
26. Logs
Logs take the opposite approach to metrics.
They track individual events. So you can tell that Mr. Foo visited /myendpoint
yesterday at 7pm and received a 7892 byte response with status code 200.
The downside is you're limited in how many fields you can track due, likely <100.
Data volumes involved can also mean it takes some time for data to be available.
Examples: ELK stack, Graylog
27. Distributed Tracing
Distributed tracing is really a special case of logging.
It gives each request an unique ID that is logged as the request is handled
through various services in your architecture.
It ties these logs back together to see how the request flowed through the system,
and where time was spent.
Essential for debugging large-scale distributed systems.
Examples: OpenTracing, OpenZipkin
29. Where does Prometheus fit in?
Prometheus is a metrics-based monitoring system.
It tracks overall statistics over time, not individual events.
It has a Time Series DataBase (TSDB) at its core.
30. Powerful Data Model and Query Language
All metrics have arbitrary multi-dimensional labels.
Supports any double value with millisecond resolution timestamps.
Can multiply, add, aggregate, join, predict, take quantiles across many metrics in
the same query. Can evaluate right now, and graph back in time.
Can alert on any query.
31. Prometheus and the Cloud
Dynamic environments mean that new application instances continuously appear
and disappear.
Service Discovery can automatically detect these changes, and monitor all the
current instances.
Even better as Prometheus is pull-based, we can tell the difference between an
instance being down and an instance being turned off on purpose!
32. Heterogeneity
Not all Cloud VMs are equal.
Noisy neighbours mean different application instances have different performance.
Alerting on individual instance latency would be spammy.
But PromQL can aggregate latency across instances, allowing you to alert on
overall end-user visible latency rather than outliers.
33. Reliability is Key
Core Prometheus server is a single binary.
Each Prometheus server is independent, it only relies on local SSD.
No clustering or attempts to backfill "missing" data when scrapes fail. Such
approaches are difficult/impossible to get right, and often cause the type of
outages you're trying to prevent.
Option for remote storage for long term storage.
35. Prometheus History - 1
Prometheus started in 2012 by Matt Proud and Julius Volz in Berlin.
In 2013 developed within SoundCloud, expanded to support Bazooka (cluster
manager/scheduler), Go, Java and Ruby clients.
In 2014 other companies start using it, including myself working at Boxever.
Project matures: new v2 storage by Beorn, new text format.
In January 2015 we "publicly release", adoption increases.
36. Prometheus History - 2
In May 2016 Prometheus is 2nd project to join the CNCF.
In July 2016 Prometheus releases 1.0.
In September 2016, first Promcon in Berlin.
In early 2017, Fabian starts work on a new TSDB.
In November 2017, Prometheus releases 2.0.
In August 2018, Prometheus is 2nd project to graduate within the CNCF. This is
announced at the 3rd Promcon.
37. Community Growth
2016: 100+ users, 250+ contributors, 35+ integrations
2017: 500+ users, 300+ contributors, 100+ integrations, 600+ on lists
2018: 10k+ users, 900+ contributors, 300+ integrations, 1.2k+ on lists
38. Prometheus: The Book
Released in August 2018 with O'Reilly, 386
pages.
Core content was written of the course of 61
days.
Copies to give out today!