This document provides an overview of OpenTelemetry for operators. It discusses some of the limitations of current observability platforms and how OpenTelemetry addresses these issues. It introduces the OpenTelemetry project which combines distributed tracing, metrics, and logging APIs. It describes the OpenTelemetry Collector which receives, processes, and exports telemetry data. It provides examples of Collector configuration and running it in production. It also discusses some innovations in the observability space from vendors like Dynatrace, New Relic, Splunk SignalFX, and others.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the fundamental concepts of observability and then demonstrates how to instrument your applications using the OpenTelemetry libraries.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk gives an overview of the OpenTelemetry project and then outlines some production-proven architectures for improving the observability of your applications and systems.
This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.
Observability in Java: Getting Started with OpenTelemetryDevOps.com
Our software is more complex than ever: applications must be reliable, predictable, and easy to use to meet modern expectations. As developers, this means our responsibilities have grown while the things we can control have stayed the same. In order to better understand our systems and create truly modern software, we need observability.
This workshop will walk through what observability means for Java developers and how to achieve it in our systems with the least amount of work using the open source observability project OpenTelemetry.
This presentation has been presented at the "Vienna DevOps & Security Meetup" in 2021.
It discusses the state of monitoring, what Opentelemetry is and why should you care about it.
Concepts and basics are discussed and presented in a full example extracting traces, metrics and logs.
Demo: https://github.com/secustor/opentelemetry-meetup
Everything You wanted to Know About Distributed TracingAmuhinda Hungai
In the age of microservices, understanding how applications are executing in a highly distributed environment can be complicated. Looking at log files only gives a snapshot of the whole story and looking at a single service in isolation simply does not give enough information. Each service is just one side of a bigger story. Distributed tracing has emerged as an invaluable technique that succeeds in summarizing all sides of the story into a shared timeline. Yet deploying it can be quite challenging, especially in the large scale, polyglot environments of modern companies that mix together many different technologies. During this session, we will take a look at patterns and means to implement Tracing for services. After introducing the basic concepts we will cover how the tracing model works, and how to safely use it in production to troubleshoot and diagnose issues.
Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent and collectors.
In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyze the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others.
I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the fundamental concepts of observability and then demonstrates how to instrument your applications using the OpenTelemetry libraries.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk gives an overview of the OpenTelemetry project and then outlines some production-proven architectures for improving the observability of your applications and systems.
This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.
Observability in Java: Getting Started with OpenTelemetryDevOps.com
Our software is more complex than ever: applications must be reliable, predictable, and easy to use to meet modern expectations. As developers, this means our responsibilities have grown while the things we can control have stayed the same. In order to better understand our systems and create truly modern software, we need observability.
This workshop will walk through what observability means for Java developers and how to achieve it in our systems with the least amount of work using the open source observability project OpenTelemetry.
This presentation has been presented at the "Vienna DevOps & Security Meetup" in 2021.
It discusses the state of monitoring, what Opentelemetry is and why should you care about it.
Concepts and basics are discussed and presented in a full example extracting traces, metrics and logs.
Demo: https://github.com/secustor/opentelemetry-meetup
Everything You wanted to Know About Distributed TracingAmuhinda Hungai
In the age of microservices, understanding how applications are executing in a highly distributed environment can be complicated. Looking at log files only gives a snapshot of the whole story and looking at a single service in isolation simply does not give enough information. Each service is just one side of a bigger story. Distributed tracing has emerged as an invaluable technique that succeeds in summarizing all sides of the story into a shared timeline. Yet deploying it can be quite challenging, especially in the large scale, polyglot environments of modern companies that mix together many different technologies. During this session, we will take a look at patterns and means to implement Tracing for services. After introducing the basic concepts we will cover how the tracing model works, and how to safely use it in production to troubleshoot and diagnose issues.
Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent and collectors.
In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyze the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others.
I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaSridhar Kumar N
https://www.youtube.com/playlist?list=PLAiEy9H6ItrKC5PbH7KiELiSEIKv3tuov
-What is Prometheus?
-Difference Between Nagios vs Prometheus
-Architecture
-Alertmanager
-Time series DB
-PromQL (Prometheus Query Language)
-Live Demo
-Grafana
My contribution to the "Grafana & Friends" Meetup.
This presentation goes into the context in the Observability landscape, the basics of OpenTelemetry with its signals and lookout what to expect next.
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
MeetUp Monitoring with Prometheus and Grafana (September 2018)Lucas Jellema
This presentation introduces the concept of monitoring - focusing on why and how and finally on the tools to use. It introduces Prometheus (metrics gathering, processing, alerting), application instrumentation and Prometheus exporters and finally it introduces Grafana as a common companion for dashboarding, alerting and notifications. This presentations also introduces the handson workshop - for which materials are available from https://github.com/lucasjellema/monitoring-workshop-prometheus-grafana
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...NETWAYS
Tracking a request’s flow across different components in distributed systems is essential. With the rise of microservices, their importance has risen to critical levels. Some proprietary tools for tracking have been used already: Jaeger and Zipkin naturally come to mind. Observability is built on three pillars: logging, metrics, and tracing. OpenTelemetry is a joint effort to bring an open standard to them. Jaeger and Zipkin joined the effort so that they are now OpenTelemetry compatible. In this talk, I’ll describe the above in more detail and showcase a (simple) use case to demo how you could benefit from OpenTelemetry in your distributed architecture.
Improve monitoring and observability for kubernetes with oss toolsNilesh Gule
Slide deck from the ASEAN Cloud Summit meetup on 27 January 2022. The session cover the following topics
1 - Centralized Loggin with Elasticsearch, Fluentbit and Kibana
2 - Monitoring and Alerting with Prometheus and Grafana
3 - Exception aggregation with Sentry
The live demo showcased these aspects using Azure Kubernetes Service (AKS)
Everybody in our team knows how to create stable and scalable software products. But in this case, we are using Docker... and it really helps us to concentrate on development and spend more time on code review & tests instead of troubleshooting issues with servers.
Cinder On-boarding Room - Berlin (11-13-2018)Jay Bryant
These are the slides presented during the Berlin OpenStack Summit. The presentation includes information about the Cinder Team and our processes. Intended to help new contributors to get involved developing with the team upstream.
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaSridhar Kumar N
https://www.youtube.com/playlist?list=PLAiEy9H6ItrKC5PbH7KiELiSEIKv3tuov
-What is Prometheus?
-Difference Between Nagios vs Prometheus
-Architecture
-Alertmanager
-Time series DB
-PromQL (Prometheus Query Language)
-Live Demo
-Grafana
My contribution to the "Grafana & Friends" Meetup.
This presentation goes into the context in the Observability landscape, the basics of OpenTelemetry with its signals and lookout what to expect next.
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
MeetUp Monitoring with Prometheus and Grafana (September 2018)Lucas Jellema
This presentation introduces the concept of monitoring - focusing on why and how and finally on the tools to use. It introduces Prometheus (metrics gathering, processing, alerting), application instrumentation and Prometheus exporters and finally it introduces Grafana as a common companion for dashboarding, alerting and notifications. This presentations also introduces the handson workshop - for which materials are available from https://github.com/lucasjellema/monitoring-workshop-prometheus-grafana
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...NETWAYS
Tracking a request’s flow across different components in distributed systems is essential. With the rise of microservices, their importance has risen to critical levels. Some proprietary tools for tracking have been used already: Jaeger and Zipkin naturally come to mind. Observability is built on three pillars: logging, metrics, and tracing. OpenTelemetry is a joint effort to bring an open standard to them. Jaeger and Zipkin joined the effort so that they are now OpenTelemetry compatible. In this talk, I’ll describe the above in more detail and showcase a (simple) use case to demo how you could benefit from OpenTelemetry in your distributed architecture.
Improve monitoring and observability for kubernetes with oss toolsNilesh Gule
Slide deck from the ASEAN Cloud Summit meetup on 27 January 2022. The session cover the following topics
1 - Centralized Loggin with Elasticsearch, Fluentbit and Kibana
2 - Monitoring and Alerting with Prometheus and Grafana
3 - Exception aggregation with Sentry
The live demo showcased these aspects using Azure Kubernetes Service (AKS)
Everybody in our team knows how to create stable and scalable software products. But in this case, we are using Docker... and it really helps us to concentrate on development and spend more time on code review & tests instead of troubleshooting issues with servers.
Cinder On-boarding Room - Berlin (11-13-2018)Jay Bryant
These are the slides presented during the Berlin OpenStack Summit. The presentation includes information about the Cinder Team and our processes. Intended to help new contributors to get involved developing with the team upstream.
Prometheus: What is is, what is new, what is comingJulien Pivotto
Prometheus is a metrics-based monitoring and alerting system and also the project with the second longest tenure within the CNCF. As such you have probably heard about it by now. We will give you a short introduction to Prometheus, what it is and why it was such a big deal when it was initially released. In all those years since then, the project has only gained speed, which provides us with the opportunity to tell you about all the exciting new features that have just been released or are in the pipeline, including opportunities to contribute to the project and its wider ecosystem.
Talk at kubecon 2021
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018Jay Bryant
These are the slides presented in the Cinder On-Boarding room at the OpenStack Summit in Vancouver on May 22, 2018. Includes an overview of the Cinder project, team members and processes.
Modern cloud-native applications are incredibly complex systems. Keeping the systems healthy and meeting SLAs for our customers is crucial for long-term success. In this session, we will dive into the three pillars of observability - metrics, logs, tracing - the foundation of successful troubleshooting in distributed systems. You'll learn the gotchas and pitfalls of rolling out the OpenTelemetry stack on Kubernetes to effectively collect all your signals without worrying about a vendor lock in. Additionally we will replace parts of the Prometheus stack to scrape metrics with OpenTelemetry collector and operator.
OpenStack Cinder On-Boarding Education - Boston Summit - 2017Jay Bryant
These slides were presented at the Boston Summit for people interested in learning how to start contributing to OpenStack's Block Storage project, Cinder. Includes an overview of Cinder's architecture, an introduction to our development processes and a description of our code tree.
Introducing KRaft: Kafka Without Zookeeper With Colin McCabe | Current 2022HostedbyConfluent
Introducing KRaft: Kafka Without Zookeeper With Colin McCabe | Current 2022
Apache Kafka without Zookeeper is now production ready! This talk is about how you can run without ZooKeeper, and why you should.
20180503 kube con eu kubernetes metrics deep diveBob Cotton
Kubernetes generates a wealth of metrics. Some explicitly within the Kubernetes API server, the Kublet, and cAdvisor or implicitly by observing events such as the kube-state-metrics project. A subset of these metrics are used within Kubernetes itself to make scheduling decisions, however, other metrics can be used to determine the overall health of the system or for capacity planning purposes.
Kubernetes exposes metrics from several places, some available internally, others through add-on projects. In this session you will learn about:
- Node level metrics, as exposed from the node_exporter
- Kublet metrics
- API server metrics
- etcd metrics
- cAdvisor metrics
- Metrics exposed from kube-state-metrics
Integrating Puppet and Gitolite for sysadmins cooperationsLuca Mazzaferro
In this slides is presented a light solution based on the integration between Puppet-Foreman and Gitolite to the problem: How to enable many sysadmins to work together on one work environment without interfering with each other?
Presented at Stream Processing Meetup (7/19/2018)(https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/).
At Uber, we operate 20+ Kafka clusters to collect system and application logs as well as event data from rider and driver apps. We need a Kafka replication solution to replicate data between Kafka clusters across multiple data centers for different purposes. This talk will introduce the history behind uReplicator and the high level architecture. As the original uReplicator ran into scalability challenges and operational overhead as the scale of Kafka clusters increased, we built the Federated uReplicator which addressed above issues and provide an extensible architecture for further scaling.
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...DoKC
Link: https://youtu.be/cegd3Exg05w
https://go.dok.community/slack
https://dok.community/
Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB
ABSTRACT OF THE TALK
Imagine this: you have a virtual infrastructure based on Kubernetes, made up of virtual data centers, possibly spread across multiple Kubernetes clusters and regions. Your infrastructure could even be hosted on premises or on different cloud service providers. Infrastructure as Code is a requirement. You’ve been tasked to run Postgres databases, alongside your applications.
The good news is that you can leverage a fully open source stack with Kubernetes, PostgreSQL and the CloudNativePG operator, and deploy your Postgres database in the same way you deploy applications.
Join me in this webinar to discover the key role that you have to make this succeed, starting from day 0 through day 2 operations.
I’ll share some examples and best practices for running Postgres databases in Kubernetes, before peeking at the new features we are developing for the months to come.
A short introductory talk given as part of the April 2018 Kong meetup "Introducing Kubernetes Ingress Controller for Kong".
This talk covers the new features and improvements made to Kong from 2017 to 2018, including the groundwork conducted by Kong Inc. and open source contributors that allowed for the development of the Kong Ingress Controller for Kubernetes.
The Kong Ingress Controller for Kubernetes was then announced during the meetup:
https://github.com/Kong/kubernetes-ingress-controller
BKK16-302: Android Optimizing Compiler: New Member Assimilation GuideLinaro
A tour of essential topics for working on the Android Optimizing Compiler, with a special emphasis on helping new engineers integrate and hit the ground running. Learn how to work on intrinsics, instruction simplification, platform specific optimizations, how to submit good patches, write Checker tests, analyse IR, take boot.oat measurements, and debug performance and execution issues with Streamline and GDB.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
2. Our
Agenda
● Why are current observability platforms
falling short?
● What OpenTelemetry features address
these issues?
● How do I run OpenTelemetry
components in production?
● Who are the innovators in the
observability space?
3. Level
Setting
● Have you used ELK stack or other log
aggregator?
● Have you used an APM system?
● Have you used distributed tracing
before?
4. Who am I?
● Kevin Brockhoff - Senior
Consultant, Daugherty Business
Solutions
○ Solving difficult cloud adoption
challenges for Daugherty's
Fortune 500 clients
○ OpenTelemetry committer since
early stages of the project
○ Github:
https://github.com/kbrockhoff
○ Linkedin:
https://www.linkedin.com/in/kevi
n-brockhoff-a557877/
6. 6
Enterprise Applications
● Only instrumented with logging during initial development.
○ Logging oriented toward development, not operations
● Metrics and tracing only added later if at all as a separate project.
○ Each team creates their own system using familiar tools
○ Or enterprise commits to a specific APM vendor
● Logs, metrics and traces are never connected.
7. 7
First Generation Observability Platforms
Search logs in ELK,
Lack context
Homegrown tracing per
app mainly accessible by
developers
Customer experience
metrics
Low-level metrics
and alerts
9. OpenCensus + OpenTracing = OpenTelemetry
● OpenTracing:
○ Provides APIs and instrumentation for distributed tracing
● OpenCensus:
○ Provides APIs and instrumentation that allow you to collect application metrics and
distributed tracing.
● OpenTelemetry:
○ An effort to combine distributed tracing, metrics and logging into a single set of system
components and language-specific libraries.
10. 10
OpenTelemetry Project
● Specification
○ API (for application developers)
○ SDK Implementations
○ Transport Protocol (Protobuf)
● Collector (middleware)
● SDK’s (various stages of maturity)
○ C++
○ C# (Auto-instrument/Manual)
○ Erlang
○ Go
○ JavaScript (Browser/Node)
○ Java (Auto-instrument/Manual)
■ Android compatibility
○ PHP
○ Python (Auto-instrument/Manual)
○ Ruby
○ Rust
○ Swift
13. 13
OpenTelemetry Collector
● Offers a vendor-agnostic implementation on how to receive, process and
export telemetry data.
● Removes the need to run, operate and maintain multiple
agents/collectors.
● Support open-source telemetry data formats (e.g. OTLP, Jaeger,
Prometheus, etc.) sending to multiple open-source or commercial back-
ends.
14. 14
Collector Concepts
● Telemetry data processing pipelines
○ Per pipeline: Receiver(s) -> Processors -> Exporter(s)
○ Currently only single telemetry type pipelines supported
● Extensions
○ Supporting functionality
○ Core collector extensions
■ health_check - HTTP endpoint for load balancer or k8s controller
■ zpages - Internal processing metrics and traces accessible via HTTP
■ pprof - Performance profiler enables the golang net/http/pprof endpoint
17. Collector Bundled Processors
● Attributes
○ Modifies span attributes
● Batch
○ Groups data into batches
● Filter
○ Include/exclude metrics by name
● Group by Trace
○ Holds all spans for a trace for a set time
and then sends to next processor
● Memory Limiter
○ Prevents out-of-memory issues by
triggering GC
○ Configuration must be matched with
ballast setting collector is launched with
● Queued Retry
○ Deprecated, each exporter now
implements
● Resource
○ Applies changes to Resource attributes
● Probabilistic Sampling
○ Adjusts TraceID hash-based sampling
decisions by sampling.priority
attribute value
● Tail Sampling
○ Sampling decisions based on configured
attribute values and rate limits
● Span
○ Modifies span name or attributes based
on span name
18. 18
Recommended Processor Configuration
Traces
memory_limiter
any sampling processors
batch
any other processors
Metrics
memory_limiter
any filtering processors
batch
any other processors
Memory limiter ballast_size_mib must match --mem-ballast-size-mib command line
parameter. Trigger GC with either limit_mib / spike_limit_mib or limit_percentage /
spike_limit_percentage.
19. Collector Contrib Processors
● Kubernetes
○ Adds metadata from pod
● Metrics Transform
○ Renames/aggregations within individual
metrics
● Resource Detection
○ OTEL_RESOURCE environment variable
○ GCE metadata server
○ EC2 instance metadata server
● Routing
○ Route to particular exporter based on
incoming header value
TODO
● Span data sharding by TraceID
20. Collector Bundled Exporters
Traces
● File
○ JSON format
● Jaeger
○ v2 gRPC
● Kafka
○ OTLP, Jaeger, Zipkin
● Logging
○ Debugging
● OpenCensus
● OTLP (OpenTelemetry Protocol)
● Zipkin
○ v2 JSON or Protobuf
Metrics
● File
○ JSON format
● Logging
○ Debugging
● OpenCensus
● OTLP (OpenTelemetry Protocol)
● Prometheus
○ Metrics endpoint for Prometheus to pull
from
● Prometheus Remote Write
○ Pushes metrics in Prometheus
TimeSeries format (Cortex, etc.)
24. Collector Command Line Example
/usr/local/bin/otelcol
--config=/usr/local/etc/otel-collector-config.yaml
--mem-ballast-size-mib=192
--log-level=DEBUG
25. 25
Collector Docker Images
● otel/opentelemetry-collector
○ Core receivers, processors, and exporters bundled in
● otel/opentelemetry-collector-contrib
○ All core and contrib receivers, processors, and exporters bundled in
● OpenTelemetry Collector builder
○ https://github.com/observatorium/opentelemetry-collector-builder
26. 26
Other Collector Installs
● RPM
○ Produced by opentelemetry-collector build
● Debian
○ Produced by opentelemetry-collector build
27. 27
Observing the Collector
● health_check
○ http://<hostname>:13133/ returns basic
pipeline availability
● zpages
○ RPC metric aggregations at
http://<hostname>:55679/debug/rpcz
○ Trace summaries at
http://<hostname>:55679/debug/tracez
● prometheus
○ Pipeline metrics scrap endpoint at
http://<hostname>:8888/metrics
28. 28
Current Gotchas
● Errors propagated back through pipelines and instances in the chain
○ Errors reported by SDK exporters in the applications may be coming from two hops
downstream
● TraceID sharding not working correctly
○ Can only do tail-based sampling if running single instance of collector
30. 30
Latest Innovations
● Dynatrace automates manual quality validation processes using AI-
assisted SLI/SLO-based quality gates.
● New Relic Incident Intelligence continuously analyzes alerts and incident
data to find patterns in event sequences and offers suggested correlation
decisions that merge incidents to reduce alert noise further.
● Splunk SignalFX provides high cardinality exploration of traces across
different regions, hosts, versions or users.
● Lightstep provides rapid root cause analysis using unlimited cardinality
and a high-fidelity dataset uncompromised by head or tail sampling,
31. 31
Latest Innovations
● Datadog provides automated tagging and correlation of logs so can jump
from any log entry to related metrics.
● Honeycomb lets you break down on every dimension in your data both
the obvious fields, and the surprising ones.
● Grafana Loki datasource provides switching from metrics to logs with
preserved label filters.
● Elastic Observability bring your logs, metrics, and APM traces together at
scale in a single stack.
Copyright 2020, The OpenTelemetry Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.