The document discusses open source monitoring tools and trends from 2008 to 2016. It begins by providing background on the speaker and their experience transitioning from a developer to operations role. It then outlines reasons why traditional monitoring was disliked (#monitoringsucks), before the rise of tools enabling more automated, metrics-based monitoring brought renewed interest (#monitoringlove). The rest of the document summarizes popular open source monitoring tools for collecting, storing, analyzing, and visualizing metrics and logs at different stages of the monitoring pipeline. It concludes by noting an improved ability to reduce false alarms compared to earlier approaches.
Adopting Devops , Stories from the trenchesKris Buytaert
As presented at Baltic Devops in Talllinn ,
Starting with devops is either the most trivial, or the hardest thing to do.
This talk will teach you a number of tricks on how to make life easier for your team. How to work together with your management and how to convince them devops is a relevant thing
Monitoring an infrastructure is still one of the most complex tasks at hand. Yet in this age of Infrastructure as Code a lot of people are still stuck using tools that haven't adapted. This talk will explain you how to look at monitoring your infrastructure from another angle, where you can benefit from using Puppet to deploy and provision your monitoring platforms. This talk will explain you which monitoring tools are suitable for monitoring at scale and Puppet driven provisioning and which ones will only make your life harder than it should be. Where do tools like Icinga, Sensu, Graphite or even CheckMK fit into the picture ?
Kris Buytaert
Chief Travel Officer, Inuits.eu
Kris Buytaert is a long time Linux and Open Source Consultant. He's one of instigators of the devops movement, currently working for Inuits He is frequently speaking at, or organizing different international conferences and has written about the same subjects in different Books, Papers and Articles He spends most of his time working on bridging the gap between developers and operations with a strong focus on High Availability, Scalability , Virtualisation and Large Infrastructure Management projects hence trying to build infrastructures that can survive the 10th floor test, better known today as the cloud while actively promoting the devops idea ! His blog titled "Everything is a Freaking DNS Problem" can be found at http://www.krisbuytaert.be/blog/
Adopting Devops , Stories from the trenchesKris Buytaert
As presented at Baltic Devops in Talllinn ,
Starting with devops is either the most trivial, or the hardest thing to do.
This talk will teach you a number of tricks on how to make life easier for your team. How to work together with your management and how to convince them devops is a relevant thing
Monitoring an infrastructure is still one of the most complex tasks at hand. Yet in this age of Infrastructure as Code a lot of people are still stuck using tools that haven't adapted. This talk will explain you how to look at monitoring your infrastructure from another angle, where you can benefit from using Puppet to deploy and provision your monitoring platforms. This talk will explain you which monitoring tools are suitable for monitoring at scale and Puppet driven provisioning and which ones will only make your life harder than it should be. Where do tools like Icinga, Sensu, Graphite or even CheckMK fit into the picture ?
Kris Buytaert
Chief Travel Officer, Inuits.eu
Kris Buytaert is a long time Linux and Open Source Consultant. He's one of instigators of the devops movement, currently working for Inuits He is frequently speaking at, or organizing different international conferences and has written about the same subjects in different Books, Papers and Articles He spends most of his time working on bridging the gap between developers and operations with a strong focus on High Availability, Scalability , Virtualisation and Large Infrastructure Management projects hence trying to build infrastructures that can survive the 10th floor test, better known today as the cloud while actively promoting the devops idea ! His blog titled "Everything is a Freaking DNS Problem" can be found at http://www.krisbuytaert.be/blog/
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris BuytaertNETWAYS
Back in June 2011 John Vincent ranted on twitter that #monitoringsucks, and for a lot of us he was absolutely right.
At #devopsdays Rome 2012, in November, Ulf Mansson proclaimed his new found love for monitoring and we changed the hashtag into #monitoringlove.
Based on a new era of open source tools, Ulf started loving monitoring again. And for a lot of us he was absolutely right. Over the past 5 years an enormous amount of new tools and new patterns has come out of the community sometimes tagged with #devops, pretty much all of them open source. Do you still know what you should be using for what? And what the differences are?
An opinionated overview of the open source monitoring landscape to clear up the confusion on what you should use, or make the decision even more difficult on you :)
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris BuytaertNETWAYS
Im Juni 2011 wetterte John Vincent auf Twitter über Monitoring mit dem Hashtag #monitoringsucks und für viele von uns, traf er damit genau ins Schwarze.
Bei den #devopsdays im November 2012 in Rom verkündete Ulf Mansson seine neu gefundene Liebe zu Monitoring und der Hashtag #monitoringlove wurde ins Leben gerufen. Aufgrund einer neuen Ära von Open Source Monitoring Tools, entwickelt er wieder eine Leidenschaft für Monitoring und viele von uns taten es ihm gleich.
Während der letzten 5 Jahre veröffentlichte die Community eine enorme Anzahl an neuen Tools und Mustern, die manchmal mit #devops getaggt wurden und fast alle Open Source waren.
Haben Sie noch einen Überblick darüber, welches Tool man für was verwendet und was die Unterschiede sind?
Dieser Vortrag wird Ihnen einen Überblick über die Open-Source- Monitoring-Landschaft verschaffen und aufzeigen, welche Tools man am besten nutzt oder aber Ihnen die Entscheidung sogar noch schwerer machen:)
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLoveNETWAYS
In the beginning there was CFEngine, and the learning curve was high, then came Puppet , Chef and the learning curve was still high.
Now we have Ansible , for everyone that wasn't smart enough to learn the original tools. Or wasn't that the problem ?
For some people Infrastructure as Code became a goal alone, not caring about the infrastructure, Junior people wanted to learn Puppet, but forgot about the service they were configuring. Too Complex, Too much effort, .. And then containers came.
What to do when you must monitor the whole infrastructure of the biggest European hosting and cloud provider? How to choose a tool when the most used ones fail to scale to your needs? How to build an Metrics platform to unify, conciliate and replace years of fragmented legacy partial solutions? In this talk we will relate our experience building and maintaining OVH Metrics, the platform used to monitor all OVH infrastructure. We needed to go to places where most monitoring solutions hadn’t gone before, it needed to operate at the scale of the biggest European hosting and cloud providers.
Most people will claim that this never happens, others hope it never happens, but it happened on March 10, 2021, and it was not just the 1 datacenter that got impacted, but the whole campus of the provider that got powered down. This talk will explain how our customers survived this outage, how our culture, opensource tooling and automation saved the da(y,ta). A talk about disaster recovery, business continuity plans and building cloud agnostic stacks that survive disasters.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 4
From MonitoringSucks to Monitoring Love , 2016 Edition
1. From #MonitoringSucks toFrom #MonitoringSucks to
#MonitoringLove#MonitoringLove
Open Source Monitoring in 2016Open Source Monitoring in 2016
@KrisBuytaert
FlossUK 2016, London , UK
2. Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source
Consultant @inuits.euConsultant @inuits.eu
● Everything is an effing DNS ProblemEverything is an effing DNS Problem
● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore
● Organising ConferencesOrganising Conferences
● Evangelizing devopsEvangelizing devops
3. An opinionated talk about the Open SourceAn opinionated talk about the Open Source
Monitoring tooling landscapeMonitoring tooling landscape
In which I hope to learn from YOUIn which I hope to learn from YOU
5. Monitoring is usually anMonitoring is usually an
aftertoughtaftertought
ENOBUDGET, ENOTIMEENOBUDGET, ENOTIME
6. An 2008 OLS PaperAn 2008 OLS Paper
● We have bloated Java toolsWe have bloated Java tools
● Some open Core stuffSome open Core stuff
● DYI folks want traditional NagiosDYI folks want traditional Nagios
● DBA RequiredDBA Required
7. #monitoringsucks#monitoringsucks
● John Vincent (@lusis), june 2011John Vincent (@lusis), june 2011
● A sub #devops movementA sub #devops movement
● https://github.com/monitoringsucks/https://github.com/monitoringsucks/
8. Why #monitoringsucksWhy #monitoringsucks
● Manual config (gui)Manual config (gui)
● Not in sync with realityNot in sync with reality
● Hosts onlyHosts only
● Services sometimesServices sometimes
● Application neverApplication never
● Chaos or out of sync with realityChaos or out of sync with reality
● Alert FatigueAlert Fatigue
9. #monitoringlove#monitoringlove
•
•
Ulf Mansson #devopsdays Rome 2011Ulf Mansson #devopsdays Rome 2011
•
A new era of toolingA new era of tooling
•
#monitoringlove hacksessions @inuits#monitoringlove hacksessions @inuits
•
#monitorama#monitorama
10. What we wantWhat we want
● Small , well suited componentsSmall , well suited components
•
CollectCollect
•
Transport / MangleTransport / Mangle
•
StoreStore
•
AnalyseAnalyse
•
Act / AlertAct / Alert
•
VisualizeVisualize
15. SensuSensu
● Awesome for non staticAwesome for non static
environmentsenvironments
● Scaling a clustered RabbitMQ ?Scaling a clustered RabbitMQ ?
● This is Europe, U no do cloudThis is Europe, U no do cloud
25. Draw as InfiniteDraw as Infinite
● Time To DeployTime To Deploy
● DeployDeploy
FrequencyFrequency
● LifecycleLifecycle
frequencyfrequency
● Map to otherMap to other
metricsmetrics
28. Graphs to KnowledgeGraphs to Knowledge
SkylineSkyline
•
OculusOculus
•
Creating Information out of this dataCreating Information out of this data
•
Big dataBig data
•
Machine LearningMachine Learning
32. RiemannRiemann
● I still don't get it ?I still don't get it ?
● Distributed TopDistributed Top
● Do you like Clojure ?Do you like Clojure ?
● Riemann Health plugin ?Riemann Health plugin ?
● s/riemann-health/collectd/g;s/riemann-health/collectd/g;
● Output to graphiteOutput to graphite
33. PrometheusPrometheus
● Started 2012Started 2012
● SoundCloudSoundCloud
● Metrics BasedMetrics Based
● ScrapesScrapes
EndpointsEndpoints
•
ExistingExisting
endpoints forendpoints for
limited toolslimited tools
● GraphiteGraphite
ExporterExporter
● Push GatewayPush Gateway
● Great AlertingGreat Alerting
● Might needMight need
some codingsome coding
45. Beats ?Beats ?
● Elasti.coElasti.co
● Collect, Parse and ShipCollect, Parse and Ship
● Q: Is all the data you care aboutQ: Is all the data you care about
suitable for Elastic Search ?suitable for Elastic Search ?
● What about Long Term Storage ?What about Long Term Storage ?
● Do you even want to build alertingDo you even want to build alerting
from this ?from this ?
46. Checking for FailureChecking for Failure
● IcingaIcinga
•
Automated config generationAutomated config generation
● SensuSensu
•
CloudstyleCloudstyle
● PrometheusPrometheus
•
Metric basedMetric based
47. Waking you up at nightWaking you up at night
● FlapjackFlapjack
flapjack.ioflapjack.io
monitoring notification routing +monitoring notification routing +
event processing systemevent processing system
● OpenDutyOpenDuty
github.com/szechuen/OpenDutygithub.com/szechuen/OpenDuty
Duty managementDuty management
50. I love where Monitoring is headingI love where Monitoring is heading
We have much less false positives these daysWe have much less false positives these days