The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the fundamental concepts of observability and then demonstrates how to instrument your applications using the OpenTelemetry libraries.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk gives an overview of the OpenTelemetry project and then outlines some production-proven architectures for improving the observability of your applications and systems.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the latest concepts in observability and then demonstrates how to configure and deploy various OpenTelemetry components to effectively meet your SLO's.
Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent and collectors.
In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyze the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others.
I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.
This presentation has been presented at the "Vienna DevOps & Security Meetup" in 2021.
It discusses the state of monitoring, what Opentelemetry is and why should you care about it.
Concepts and basics are discussed and presented in a full example extracting traces, metrics and logs.
Demo: https://github.com/secustor/opentelemetry-meetup
This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk gives an overview of the OpenTelemetry project and then outlines some production-proven architectures for improving the observability of your applications and systems.
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the latest concepts in observability and then demonstrates how to configure and deploy various OpenTelemetry components to effectively meet your SLO's.
Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent and collectors.
In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyze the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others.
I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.
This presentation has been presented at the "Vienna DevOps & Security Meetup" in 2021.
It discusses the state of monitoring, what Opentelemetry is and why should you care about it.
Concepts and basics are discussed and presented in a full example extracting traces, metrics and logs.
Demo: https://github.com/secustor/opentelemetry-meetup
This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.
My contribution to the "Grafana & Friends" Meetup.
This presentation goes into the context in the Observability landscape, the basics of OpenTelemetry with its signals and lookout what to expect next.
Everything You wanted to Know About Distributed TracingAmuhinda Hungai
In the age of microservices, understanding how applications are executing in a highly distributed environment can be complicated. Looking at log files only gives a snapshot of the whole story and looking at a single service in isolation simply does not give enough information. Each service is just one side of a bigger story. Distributed tracing has emerged as an invaluable technique that succeeds in summarizing all sides of the story into a shared timeline. Yet deploying it can be quite challenging, especially in the large scale, polyglot environments of modern companies that mix together many different technologies. During this session, we will take a look at patterns and means to implement Tracing for services. After introducing the basic concepts we will cover how the tracing model works, and how to safely use it in production to troubleshoot and diagnose issues.
Observability in Java: Getting Started with OpenTelemetryDevOps.com
Our software is more complex than ever: applications must be reliable, predictable, and easy to use to meet modern expectations. As developers, this means our responsibilities have grown while the things we can control have stayed the same. In order to better understand our systems and create truly modern software, we need observability.
This workshop will walk through what observability means for Java developers and how to achieve it in our systems with the least amount of work using the open source observability project OpenTelemetry.
Prometheus: What is is, what is new, what is comingJulien Pivotto
Prometheus is a metrics-based monitoring and alerting system and also the project with the second longest tenure within the CNCF. As such you have probably heard about it by now. We will give you a short introduction to Prometheus, what it is and why it was such a big deal when it was initially released. In all those years since then, the project has only gained speed, which provides us with the opportunity to tell you about all the exciting new features that have just been released or are in the pipeline, including opportunities to contribute to the project and its wider ecosystem.
Talk at kubecon 2021
Juraci Paixão Kröhling - All you need to know about OpenTelemetryJuliano Costa
OpenTelemetry is one of the newest projects in the realm of Observability at the CNCF and is already the second most active project there. In this session, Juraci Paixão Kröhling will talk about the different subprojects and how to get started using them. Even if you heard about OpenTelemetry before, you'll leave this session with a better understanding of what this is all about, the several faces of OpenTelemetry, and what you can do to make your projects more observable.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOSs Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
OSMC 2022 | OpenTelemetry 101 by Dotan Horovit s.pdfNETWAYS
Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent, and collectors. In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyse the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others. I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.
Improve monitoring and observability for kubernetes with oss toolsNilesh Gule
Slide deck from the ASEAN Cloud Summit meetup on 27 January 2022. The session cover the following topics
1 - Centralized Loggin with Elasticsearch, Fluentbit and Kibana
2 - Monitoring and Alerting with Prometheus and Grafana
3 - Exception aggregation with Sentry
The live demo showcased these aspects using Azure Kubernetes Service (AKS)
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaSridhar Kumar N
https://www.youtube.com/playlist?list=PLAiEy9H6ItrKC5PbH7KiELiSEIKv3tuov
-What is Prometheus?
-Difference Between Nagios vs Prometheus
-Architecture
-Alertmanager
-Time series DB
-PromQL (Prometheus Query Language)
-Live Demo
-Grafana
Securing TodoMVC Using the Web Cryptography APIKevin Hakanson
The open source TodoMVC project implements a Todo application using popular JavaScript MV* frameworks. Some of the implementations add support for compile to JavaScript languages, module loaders and real time backends. This presentation will demonstrate a TodoMVC implementation which adds support for the forthcoming W3C Web Cryptography API, as well as review some key cryptographic concepts and definitions.
Instead of storing the Todo list as plaintext in localStorage, this "secure" TodoMVC implementation encrypts Todos using a password derived key. The PBKDF2 algorithm is used for the deriveKey operation, with getRandomValues generating a cryptographically random salt. The importKey method sets up usage of AES-CBC for both encrypt and decrypt operations. The final solution helps address item "A6-Sensitive Data Exposure" from the OWASP Top 10.
With the Web Cryptography API being a recommendation in 2014, any Q&A time will likely include browser implementations and limitations, and whether JavaScript cryptography adds any value.
My contribution to the "Grafana & Friends" Meetup.
This presentation goes into the context in the Observability landscape, the basics of OpenTelemetry with its signals and lookout what to expect next.
Everything You wanted to Know About Distributed TracingAmuhinda Hungai
In the age of microservices, understanding how applications are executing in a highly distributed environment can be complicated. Looking at log files only gives a snapshot of the whole story and looking at a single service in isolation simply does not give enough information. Each service is just one side of a bigger story. Distributed tracing has emerged as an invaluable technique that succeeds in summarizing all sides of the story into a shared timeline. Yet deploying it can be quite challenging, especially in the large scale, polyglot environments of modern companies that mix together many different technologies. During this session, we will take a look at patterns and means to implement Tracing for services. After introducing the basic concepts we will cover how the tracing model works, and how to safely use it in production to troubleshoot and diagnose issues.
Observability in Java: Getting Started with OpenTelemetryDevOps.com
Our software is more complex than ever: applications must be reliable, predictable, and easy to use to meet modern expectations. As developers, this means our responsibilities have grown while the things we can control have stayed the same. In order to better understand our systems and create truly modern software, we need observability.
This workshop will walk through what observability means for Java developers and how to achieve it in our systems with the least amount of work using the open source observability project OpenTelemetry.
Prometheus: What is is, what is new, what is comingJulien Pivotto
Prometheus is a metrics-based monitoring and alerting system and also the project with the second longest tenure within the CNCF. As such you have probably heard about it by now. We will give you a short introduction to Prometheus, what it is and why it was such a big deal when it was initially released. In all those years since then, the project has only gained speed, which provides us with the opportunity to tell you about all the exciting new features that have just been released or are in the pipeline, including opportunities to contribute to the project and its wider ecosystem.
Talk at kubecon 2021
Juraci Paixão Kröhling - All you need to know about OpenTelemetryJuliano Costa
OpenTelemetry is one of the newest projects in the realm of Observability at the CNCF and is already the second most active project there. In this session, Juraci Paixão Kröhling will talk about the different subprojects and how to get started using them. Even if you heard about OpenTelemetry before, you'll leave this session with a better understanding of what this is all about, the several faces of OpenTelemetry, and what you can do to make your projects more observable.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOSs Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
OSMC 2022 | OpenTelemetry 101 by Dotan Horovit s.pdfNETWAYS
Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent, and collectors. In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyse the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others. I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.
Improve monitoring and observability for kubernetes with oss toolsNilesh Gule
Slide deck from the ASEAN Cloud Summit meetup on 27 January 2022. The session cover the following topics
1 - Centralized Loggin with Elasticsearch, Fluentbit and Kibana
2 - Monitoring and Alerting with Prometheus and Grafana
3 - Exception aggregation with Sentry
The live demo showcased these aspects using Azure Kubernetes Service (AKS)
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaSridhar Kumar N
https://www.youtube.com/playlist?list=PLAiEy9H6ItrKC5PbH7KiELiSEIKv3tuov
-What is Prometheus?
-Difference Between Nagios vs Prometheus
-Architecture
-Alertmanager
-Time series DB
-PromQL (Prometheus Query Language)
-Live Demo
-Grafana
Securing TodoMVC Using the Web Cryptography APIKevin Hakanson
The open source TodoMVC project implements a Todo application using popular JavaScript MV* frameworks. Some of the implementations add support for compile to JavaScript languages, module loaders and real time backends. This presentation will demonstrate a TodoMVC implementation which adds support for the forthcoming W3C Web Cryptography API, as well as review some key cryptographic concepts and definitions.
Instead of storing the Todo list as plaintext in localStorage, this "secure" TodoMVC implementation encrypts Todos using a password derived key. The PBKDF2 algorithm is used for the deriveKey operation, with getRandomValues generating a cryptographically random salt. The importKey method sets up usage of AES-CBC for both encrypt and decrypt operations. The final solution helps address item "A6-Sensitive Data Exposure" from the OWASP Top 10.
With the Web Cryptography API being a recommendation in 2014, any Q&A time will likely include browser implementations and limitations, and whether JavaScript cryptography adds any value.
Using Groovy to empower WebRTC Network Systemsantonry
This workshop introduces to WebRTC and how you can utilize its open API to develop Rich Communication services in your web applications. It describes a general Java gateway to allow the translation of WebRTC to SIP, the protocol used internally in the majority of VoIP networks and how Groovy is used to empower the process. In the end of the workshop, the assistants will be able to develop and deploy a basic webpage and they will have the knowledge to customize the example and develop more complex features, adding audio, video or data to HTML5, iOS or Android applications.
We will introduce Airflow, an Apache Project for scheduling and workflow orchestration. We will discuss use cases, applicability and how best to use Airflow, mainly in the context of building data engineering pipelines. We have been running Airflow in production for about 2 years, we will also go over some learnings, best practices and some tools we have built around it.
Speakers: Robert Sanders, Shekhar Vemuri
This talk describes the current state of the Veil-Framework and the different tools included in it such as Veil-Evasion, Veil-Catapult, Veil-Powerview, Veil-Pillage, Veil-Ordnance
"Node.js vs workers — A comparison of two JavaScript runtimes", James M SnellFwdays
Workers is the open source kernel of the Cloudflare Workers platform, and despite being built around v8, and running JavaScript and WebAssembly, it is quite different from Node.js. This talk will explore the differences and similarities and hopefully give you a bit more insight into how both operate.
With distributed tracing, we can track requests as they pass through multiple services, emitting timing and other metadata throughout, and this information can then be reassembled to provide a complete picture of the application’s behavior at runtime - Read more in https://blog.buoyant.io/2016/05/17/distributed-tracing-for-polyglot-microservices/ and https://www.rookout.com/
Thinking DevOps in the era of the Cloud - Demi Ben-AriDemi Ben-Ari
The lines between Development and Operations people have gotten blurry and lots of skills needs to be held by both sides.
In the talk we'll talk about all of the considerations that are needed to be taken when creating a development and production environment, mentioning Continuous Integration, Continuous Deployment and the Buzzword "DevOps", also talking about some real implementations in the industry.
Of course how can we leave out the real enabler of the whole deal,
"The Cloud", Giving us a tool set that makes life much easier when implementing all of these practices.
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseVictoriaMetrics
Monitoring is the key to successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast growing, high performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Altinity Ltd
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHouse Webinar Slides
Monitoring is the key to the successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open-source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast-growing, high-performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Presented by:
Roman Khavronenko, Co-Founder at VictoriaMetrics
Robert Hodges, CEO at Altinity
OSMC 2021 | inspectIT Ocelot: Dynamic OpenTelemetry Instrumentation at RuntimeNETWAYS
If you want to trace or extract specific data from a Java application with OpenTelemetry, you usually have to modify the application’s code. However, this is often not possible, especially with bought-in software. We would like to show, how the open source inspectIT Ocelot Java agent can be used to dynamically inject OpenTelemetry code at runtime for extracting specific application and business data – and all this without having to adapt the application itself.
No production system is complete without a way to monitor it. In software, we define observability as the ability to understand how our system is performing. This talk dives into capabilities and tools that are recommended for implementing observability when running K8s in production as the main platform today for deploying and maintaining containers with cloud-native solutions.
We start by introducing the concept of observability in the context of distributed systems such as K8s and the difference with monitoring. We continue by reviewing the observability stack in K8s and the main functionalities. Finally, we will review the tools K8s provides for monitoring and logging, and get metrics from applications and infrastructure.
Between the points to be discussed we can highlight:
-Introducing the concept of observability
-Observability stack in K8s
-Tools and apps for implementing Kubernetes observability
-Integrating Prometheus with OpenMetrics
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...Codemotion
Once you start working with Big Data systems, you discover a whole bunch of problems you won’t find in monolithic systems. Monitoring all of the components becomes a big data problem itself. In the talk, we’ll mention all of the aspects that you should take into consideration when monitoring a distributed system using tools like Web Services, Spark, Cassandra, MongoDB, AWS. Not only the tools, what should you monitor about the actual data that flows in the system? We’ll cover the simplest solution with your day to day open source tools, the surprising thing, that it comes not from an Ops Guy.
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...Demi Ben-Ari
Once you start working with distributed Big Data systems, you start discovering a whole bunch of problems you won’t find in monolithic systems.
All of a sudden to monitor all of the components becomes a big data problem itself.
In the talk we’ll mention all of the aspects that you should take in consideration when monitoring a distributed system once you’re using tools like:
Web Services, Apache Spark, Cassandra, MongoDB, Amazon Web Services.
Not only the tools, what should you monitor about the actual data that flows in the system?
And we’ll cover the simplest solution with your day to day open source tools, the surprising thing, that it comes not from an Ops Guy.
Everybody is consuming NuGet packages these days. It’s easy, right? But how can we create and share our own packages? What is .NET Standard? How should we version, create, publish and share our package?
Once we have those things covered, we’ll look beyond what everyone is doing. How can we use the NuGet client API to fetch data from NuGet? Can we build an application plugin system based on NuGet? What hidden gems are there in the NuGet server API? Can we create a full copy of NuGet.org?
Good questions! In this talk, we will get them answered.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
2. Our
Agenda
● What is observability?
● How does OpenTelemetry relate to
observability?
● What concepts do I need to use
OpenTelemetry?
● How do I record data using
OpenTelemetry?
● Where can I send my data?
3. Level
Setting
● Are you responsible for writing software?
● Are you responsible for operating
software?
● Have you used distributed tracing
before?
● Have you used OpenCensus?
● Have you used OpenTracing?
4. Who am I?
● Kevin Brockhoff - Senior
Consultant, Daugherty Business
Solutions
○ Solving difficult cloud adoption
challenges for Daugherty's
Fortune 500 clients
○ OpenTelemetry committer since
early stages of the project
○ Github:
https://github.com/kbrockhoff
○ Linkedin:
https://www.linkedin.com/in/kevi
n-brockhoff-a557877/
6. 6
Why observability?
● Microservices create complex interactions.
● Failures don't exactly repeat.
● Debugging multi-tenancy is painful.
● Monitoring no longer can help us.
Cynefin Framework
Complex
7. 7
What is observability?
● We need to answer questions about our systems.
What characteristics did the queries that timed out at 500ms share in
common? Service versions? Browser plugins?
● Instrumentation produces data.
● Querying data answers our questions.
8. 8
Telemetry aids observability
● Telemetry data isn't observability itself.
● Instrumentation code is how we get telemetry.
● Telemetry data can include traces, logs, and/or metrics.
All different views into the same underlying truth.
9. 9
Metrics, logs, and traces, oh my!
● Metrics
○ Aggregated summary statistics.
● Logs
○ Detailed debugging information emitted by processes.
● Distributed Tracing
○ Provides insights into the full lifecycles, aka traces of requests to a system, allowing you
to pinpoint failures and performance issues.
Structured data can be transmuted into any of these!
10. 10
Metrics Concepts
● Gauges
○ Instantaneous point-in-time value (e.g.
CPU utilization)
● Cumulative counters
○ Cumulative sums of data since process
start (e.g. request counts)
● Cumulative histogram
○ Grouped counters for a range of buckets
(e.g. 0-10ms, 11-20ms)
● Rates
○ The derivative of a counter, typically. (e.g.
requests per second)
11. 11
Tracing Concepts
● Span
○ Represents a single unit of work in a
system.
● Trace
○ Defined implicitly by its spans. A trace
can be thought of as a directed acyclic
graph of spans where the edges
between spans are defined as
parent/child relationships.
● Distributed Context
○ Contains the tracing identifiers, tags, and
options that are propagated from parent
to child spans.
12. 12
Add more context to traces with Span Events
● Span Events are context-aware logging.
● An event contains timestamped information added to a span. You can
think of this as a structured log, or a way to annotate your spans with
specific details about what happened along the way.
○ Contains:
■ the name of the event
■ one or more attributes
■ a timestamp
13. 13
But how do I implement these?
● You need an instrumentation framework!
● and a place to send the data!
● and a way to visualize the data!
15. OpenCensus + OpenTracing = OpenTelemetry
● OpenCensus:
○ Provides APIs and instrumentation that allow you to collect application metrics and
distributed tracing.
● OpenTracing:
○ Provides APIs and instrumentation for distributed tracing
● OpenTelemetry:
○ An effort to combine distributed tracing, metrics and logging into a single set of system
components and language-specific libraries.
16. 16
OpenTelemetry Project
● Specification
○ API (for application developers)
○ SDK Implementations
○ Transport Protocol (Protobuf)
● Collector (middleware)
● SDK’s (various stages of maturity)
○ C++
○ C# (Auto-instrument/Manual)
○ Erlang
○ Go
○ JavaScript (Browser/Node)
○ Java (Auto-instrument/Manual)
■ Android compatibility
○ PHP
○ Python (Auto-instrument/Manual)
○ Ruby
○ Rust
○ Swift
17. 17
OTel API - packages, methods, & when to call
● Tracer
○ A Tracer is responsible for tracking the currently active span.
● Meter
○ A Meter is responsible for accumulating a collection of statistics.
● BaggageManager
○ A BaggageManager is responsible for propagating key-value pairs across systems.
You can have more than one. Why?
Ensures uniqueness of name prefixes.
23. 23
Resource SDK Concepts
● Resource is an immutable representation of the entity producing
telemetry.
○ Populated on executable startup.
○ Created by either SDK or Collector.
○ Different Resource objects can be merged.
○ Read from environment variable OTEL_RESOURCE_ATTRIBUTES is the default.
○ One copy of Resource passed to exporters with each batch..
25. 25
Baggage API Concepts
● Provides context and information to trace, metrics and logs telemetry
○ Set of name/value pairs
○ Contains info such as user, session, A/B testing parameters
○ Usually modeled as an immutable Context with each attribute add creating a new Context
○ Can be cleared if sending to untrusted process
● May be propagated across process boundaries
○ Using the header otcorrelations until W3C Baggage specification reaches
recommendation status
● BaggageManager creates new Baggage objects based on current context
27. 27
Tracing API Concepts
● TracerProvider is the entry point of the API. It provides access to Tracers.
○ Stateful object holding configuration with a global provider and possibly additional ones.
● Tracer is the class responsible for creating Spans.
○ Named and optionally versioned with each instrumentation library using values
guaranteed to be globally-unique.
○ Delegates getting active Span and marking a given Span as active to the Context.
● Span is the API to trace an operation.
○ Immutable SpanContext represents the serialized and propagated portion of a Span.
28. 28
Span Fields
● Name - Concisely identifies the work
● SpanContext - Immutable unique id
○ TraceId (16-bytes) globally-unique
○ SpanId (8-bytes) unique-within-trace
○ TraceFlags - Currently only sampled
○ TraceState - Vendor-specific info
● ParentSpan - Span which spawned this
span
● SpanKind - Relationship to other spans
○ [ CLIENT | SERVER | PRODUCER |
CONSUMER | INTERNAL ]
● StartTimestamp - Unix nanos
● EndTimestamp - Unix nanos
● Attributes collection
○ Key-value pairs describing the
operational context
○ OTel semantic conventions define
common keys
● Links collection
○ Associated non-parent spans
● Events collection
○ Significant data points during span
● Status - Success or failure details
29. 29
Span Names
Span Name Guidance
get Too general
get_account/42 Too specific
get_account Good, and account_id=42 would make a nice Span attribute
get_account/{accountId} Also good (using the “HTTP route”)
30. Span Attribute Examples
HTTP Server RDBMS Client
http.method PUT
http.scheme https
http.server_nam
e
api22.opentelemetry.io
net.host.port 443
http.target /blog/posts/774356
http.client_ip 2001:506:71f0:16e::1
http.user_agent Mozilla/5.0 (Macintosh; Intel
Mac OS X 10_15_4)
AppleWebKit/537.36 (KHTML,
like Gecko)
Chrome/81.0.4044.138
Safari/537.36
db.system mysql
db.connection_
string
Server=shopdb.example.com;Datab
ase=ShopDb;Uid=billing_user;Tab
leCache=true;UseCompression=Tru
e;MinimumPoolSize=10;MaximumPoo
lSize=50;
net.host.ip 192.0.3.122
net.host.port 51306
net.peer.ip 192.0.2.12
net.peer.port 3306
db.statement SELECT * FROM orders WHERE
order_id = :oid
31. 31
Tracing SDK Concepts
● Sampler controls the number of traces collected and sent to the backend.
○ Sampling decision values [ DROP | RECORD_ONLY | RECORD_AND_SAMPLE ]
○ SDK built-in samplers [ AlwaysOn | AlwaysOff | TraceIdRatioBased ]
○ Use AlwaysOn if doing tail-based sampling
● SpanProcessor enables hooks for span start and end method invocations.
○ SDK built-in processors [ ExportFormatConverter | Batching ]
○ Only invoked when IsRecording is true
● SpanExporter defines interface for protocol-specific exporters
32. Tracing Code - Java
Tracer tracer = OpenTelemetry.getTracer("instrumentation-library-name","semver:1.0.0");
Span span = tracer.spanBuilder("my span").startSpan();
span.setAttribute("http.method", "GET");
span.setAttribute("http.url", url.toString());
try (Scope scope = tracer.withSpan(span)) {
// your use case
Attributes eventAttributes = Attributes.of(
"key", AttributeValue.stringAttributeValue("value"),
"result", AttributeValue.longAttributeValue(0L));
span.addEvent("End Computation", eventAttributes);
} catch (Throwable t) {
Status status = Status.INTERNAL.withDescription(t.getMessage());
span.setStatus(status);
} finally {
span.end(); // closing the scope does not end the span, this has to be done manually
}
33. Tracing Code - Typescript (Web)
import { SimpleSpanProcessor } from '@opentelemetry/tracing';
import { WebTracerProvider } from '@opentelemetry/web';
import { CollectorTraceExporter } from '@opentelemetry/exporter-collector';
const collectorOptions = {
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is
http://localhost:55681/v1/trace
headers: {}, //an optional object containing custom headers to be sent with each request
};
const provider = new WebTracerProvider();
const exporter = new CollectorTraceExporter(collectorOptions);
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();
34. Tracing Code - Python
# tracing.py
from opentelemetry import trace
from opentelemetry.exporter.otlp.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchExportSpanProcessor
trace.set_tracer_provider(TracerProvider())
otlp_exporter = OTLPSpanExporter(endpoint="localhost:55680")
trace.get_tracer_provider().add_span_processor(
BatchExportSpanProcessor(otlp_exporter)
)
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("foo"):
with tracer.start_as_current_span("bar"):
with tracer.start_as_current_span("baz"):
print("Hello world from OpenTelemetry Python!")
35. 35
Metrics API Concepts
● MeterProvider is the entry point of the API. It provides access to Meters.
○ Stateful object holding configuration with a global provider and possibly additional ones.
● Meter is class responsible for creating and managing Instruments.
○ Named and optionally versioned with each instrumentation library using values
guaranteed to be globally-unique.
○ Provides means of capturing a batch of measurements in atomic way.
● Instrument is a device for capturing raw measurements.
○ Identified by a LabelSet (set of key-value pairs)
○ Bound instruments have pre-associated set of labels
○ SDK provides means for aggregating measurements captured by an instrument before
export.
36. 36
Metric Instrument Types
Name Instrument Kind Function(argument) Default Aggregation
Counter Synchronous additive
monotonic
Add(increment) Sum
UpDownCounter Synchronous additive Add(increment) Sum
ValueRecorder Synchronous Record(value) MinMaxSumCount / DDSketch
SumObserver Asynchronous additive
monotonic
Observe(sum) Sum
UpDownSumObserver Asynchronous additive Observe(sum) Sum
ValueObserver Asynchronous Observe(value) MinMaxSumCount / DDSketch
37. 37
Metrics SDK Concepts
● Aggregator aggregates one or more measurements in a useful way.
● Accumulator receives metric events from the API, computes one
Accumulation per active Instrument and Label Set pair.
● Processor receives Accumulations from the Accumulator, transforms into
ExportRecordSet.
● Exporter receives ExportRecordSet, transforms into some protocol and
sends it somewhere.
38. Metrics Code - Java
Meter meter = OpenTelemetry.getMeter("instrumentation-library-name","semver:1.0.0");
// Build counter e.g. LongCounter
LongCounter counter = meter.longCounterBuilder("processed_jobs").setDescription("Processed jobs").setUnit("1").build();
// It is recommended that the API user keep a reference to a Bound Counter for the entire time or
// call unbind when no-longer needed.
BoundLongCounter someWorkCounter = counter.bind(Labels.of("Key", "SomeWork"));
// Record data
someWorkCounter.add(123);
// Alternatively, the user can use the unbounded counter and explicitly
// specify the labels set at call-time:
counter.add(123, Labels.of("Key", "SomeWork"));
// Build observer e.g. LongObserver
LongObserver observer = meter.observerLongBuilder("cpu_usage").setDescription("CPU Usage").setUnit("ms").build();
observer.setCallback(
new LongObserver.Callback<LongObserver.ResultLongObserver>() {
@Override
public void update(ResultLongObserver result) {
// long getCpuUsage()
result.observe(getCpuUsage(), Labels.of("Key", "SomeWork"));
}
});
39. Metrics Code - Javascript (Node)
const { MeterProvider } = require('@opentelemetry/metrics');
const { CollectorMetricExporter } = require('@opentelemetry/exporter-collector');
const collectorOptions = {
serviceName: 'basic-service',
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is
http://localhost:55681/v1/metrics
};
const exporter = new CollectorMetricExporter(collectorOptions);
// Register the exporter
const meter = new MeterProvider({
exporter,
interval: 60000,
}).getMeter('example-meter');
// Now, start recording data
const counter = meter.createCounter('metric_name');
counter.add(10, { 'key': 'value' });
40. Metrics Code - Python
from opentelemetry import metrics
from opentelemetry.sdk.metrics import Counter, MeterProvider
from opentelemetry.sdk.metrics.export import ConsoleMetricsExporter
from opentelemetry.sdk.metrics.export.controller import PushController
metrics.set_meter_provider(MeterProvider())
meter = metrics.get_meter(__name__, True)
controller = PushController(meter, ConsoleMetricsExporter(), 5)
requests_counter = meter.create_metric(
name="requests",
description="number of requests",
unit="1",
value_type=int,
metric_type=Counter,
label_keys=("environment",),
)
requests_counter.add(25, staging_labels)
41. 41
Logging Concepts
● No API, use existing libraries
● Log events enhanced with key-value pairs from SpanContext and
Baggage by either SDK or Collector processor.
43. 43
Exporter Wiring
● Export to console logging or in-memory
○ Best solution for unit tests
● Export directly to observability frontend
○ Simple architecture but side processing in the application
● Export to otel-collector running as agent/sidecar which forwards to main
otel-collector
○ Static configuration because exporter always pointed to localhost
● Export to otel-collector running as an aggregator
○ All telemetry processing done in isolation from user-facing functionality
Copyright 2020, The OpenTelemetry Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.