Optimizing Observability Spend: Metrics
•Download as PPTX, PDF•
0 likes•60 views
Are you collecting just about every metric under the sun and the kitchen sink too? Understanding the cost of collecting metrics and the usefulness of those metrics is the only way to scale in a cloud native world. You can’t get away with just collecting everything as you grow. Your observability teams need to make decisions about what to collect, what to drop, what to aggregate, and still be able to alert, triage, remediate, and do their root cause analysis on a daily basis. Gain immediate insights into high cost data (DPPS), when to drop time series data, and how to determine when the value of that data is at its lowest. Session includes a recorded demo video of it in action.
Report
Share
Report
Share
Recommended
SRECon EU 2023 - Three Phases to Better Observability Outcomes
We all want to have better business outcomes for our organizations solutions, such as faster remediation of problems, easier problem detection, greater revenue generation, happier customers, and engineering teams that can remain focused on delivering more business value. The problem with the popular three pillars (metrics, logs, tracing) is that you are talking about technology aspects and not about solutions. It's like talking about the tools in a mechanics toolbox used to make your convertible run again, instead of focusing on the blue smoke coming out of the exhaust, the rising engine temperature, and using that data to quickly remediate the problem by replacing the seals to prevent oil leaking in the engine. Let’s quickly tour the phases that lead to better outcomes and get our focus back on effective observability goals.
Key takeaways - Modern cloud native observability needs three guiding phases to provide better outcomes, not tooling.
Based on article: https://www.schabell.org/2022/09/o11y-guide-cloud-native-observability-needs-phases.html
Checking the pulse of your cloud native architecture
The daily choices you make as an engineer when shipping code contributes to the feedback loop. In cloud native environments a surprising amount of data is generated from the application layer down to infrastructure and along the delivery path. Regulatory and compliance pressures force us to store audit and observability data. Understanding the pressures on our engineering teams around the collection, storage, and maintenance of your cloud data can mean the difference between successful teams and burnout. Let us take you on a journey, looking closely at the current state of observability based on a recent research conducted with 500 cloud native engineers and find out what it’s like to be in the trenches.
Observability For You and Me with OpenTelemetry
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Observability For You and Me with OpenTelemetry (with demo)
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community. (includes demo)
DZone webinar - Shift left Observability
Whether you’re an enterprise migrating to cloud-native or born in the cloud, most of today’s APM and Observability tools don’t support how your engineers and DevOps teams need to develop, deploy, and support their software. Observability needs to shift left and reflect the modern way companies organize their development teams and their vital interdependencies.
Chronosphere is the only vendor addressing the unique requirements for observability in a cloud-native world.
Join this webinar to learn:
• What cloud native observability is and how it is different from the promises made by traditional cloud APM and observability vendors
• How to use cloud-native observability to do more “Dev” and less “Ops” so you can dramatically improve developer and engineer workflows and productivity
• How to make on-call shifts less stressful so that your engineers aren’t getting burned out
Observability For You and Me with openTelemetry
Presented at Open Source 101 2023 - Charlotte
Presented by Eric D Schabell, Chronosphere
Title: Observability For You and Me with openTelemetry
Abstract: Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
To find more talks by Eric D Schabell: https://www.slideshare.net/eschabell
Open Source 101 - Observability For You and Me with OpenTelemetry
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Observability For You and Me with OpenTelemetry
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing your microservices and applications on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of opportunities for getting started with telemetry data. The project, openTelemetry (OTEL), is where we start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation to lay a foundation. Then we’ll explore the OTEL community and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OTEL protocol (OTLP), its APIs, and its SDKs. Attendees will leave with an understanding of key observability concepts in distributed tracing!
Key takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Recommended
SRECon EU 2023 - Three Phases to Better Observability Outcomes
We all want to have better business outcomes for our organizations solutions, such as faster remediation of problems, easier problem detection, greater revenue generation, happier customers, and engineering teams that can remain focused on delivering more business value. The problem with the popular three pillars (metrics, logs, tracing) is that you are talking about technology aspects and not about solutions. It's like talking about the tools in a mechanics toolbox used to make your convertible run again, instead of focusing on the blue smoke coming out of the exhaust, the rising engine temperature, and using that data to quickly remediate the problem by replacing the seals to prevent oil leaking in the engine. Let’s quickly tour the phases that lead to better outcomes and get our focus back on effective observability goals.
Key takeaways - Modern cloud native observability needs three guiding phases to provide better outcomes, not tooling.
Based on article: https://www.schabell.org/2022/09/o11y-guide-cloud-native-observability-needs-phases.html
Checking the pulse of your cloud native architecture
The daily choices you make as an engineer when shipping code contributes to the feedback loop. In cloud native environments a surprising amount of data is generated from the application layer down to infrastructure and along the delivery path. Regulatory and compliance pressures force us to store audit and observability data. Understanding the pressures on our engineering teams around the collection, storage, and maintenance of your cloud data can mean the difference between successful teams and burnout. Let us take you on a journey, looking closely at the current state of observability based on a recent research conducted with 500 cloud native engineers and find out what it’s like to be in the trenches.
Observability For You and Me with OpenTelemetry
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Observability For You and Me with OpenTelemetry (with demo)
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community. (includes demo)
DZone webinar - Shift left Observability
Whether you’re an enterprise migrating to cloud-native or born in the cloud, most of today’s APM and Observability tools don’t support how your engineers and DevOps teams need to develop, deploy, and support their software. Observability needs to shift left and reflect the modern way companies organize their development teams and their vital interdependencies.
Chronosphere is the only vendor addressing the unique requirements for observability in a cloud-native world.
Join this webinar to learn:
• What cloud native observability is and how it is different from the promises made by traditional cloud APM and observability vendors
• How to use cloud-native observability to do more “Dev” and less “Ops” so you can dramatically improve developer and engineer workflows and productivity
• How to make on-call shifts less stressful so that your engineers aren’t getting burned out
Observability For You and Me with openTelemetry
Presented at Open Source 101 2023 - Charlotte
Presented by Eric D Schabell, Chronosphere
Title: Observability For You and Me with openTelemetry
Abstract: Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
To find more talks by Eric D Schabell: https://www.slideshare.net/eschabell
Open Source 101 - Observability For You and Me with OpenTelemetry
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Observability For You and Me with OpenTelemetry
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing your microservices and applications on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of opportunities for getting started with telemetry data. The project, openTelemetry (OTEL), is where we start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation to lay a foundation. Then we’ll explore the OTEL community and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OTEL protocol (OTLP), its APIs, and its SDKs. Attendees will leave with an understanding of key observability concepts in distributed tracing!
Key takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
3 Pitfalls Everyone Should Avoid with Cloud Native Observability
Are you looking at your organization's efforts to enter or expand into the cloud native landscape and feeling a bit daunted by the vast expanse of information surrounding cloud native observability? When you're moving so fast with agile practices across your DevOps, SRE's, and platform engineering teams, it's no wonder this can seem a bit confusing. Unfortunately, the choices being made have a great impact on both your business, your budgets, and the ultimate success of your cloud native initiatives. That hasty decision up front leads to big headaches very quickly down the road. In this talk, I'll introduce the problem facing everyone with cloud native observability followed by 3 common mistakes that I'm seeing organizations make and how you can avoid them!
Working together with SURF Raymond Oonk Annette Langedijk SURF
Presentation for the Seminar Enabling Copernicus Big Data Analytics through European Open Science Cloud on 27 October 2021
GlobusWorld 2020 Keynote
This presentation was given at the GlobusWorld 2020 Virtual Conference, by Ian Foster, Rachana Ananthakrishnan, and Vas Vasiliadis from the University of Chicago.
Whats new in Autonomous Database in 2022
This session covers the new features and happenings in the autonomous database world and will help answer more questions DBAs and Developers will have on the Autonomous Database, from provisioning to backups, troubleshooting, tips and tricks, security and HA. This is a good introduction for on-prem DBAs who want to learn how this works quickly without spending too much time on it. Questions like what does the free tier cover, how do I do backup or if it's automated, how do I manage it, how to scale up and down, how to secure their environment, how to use mtls, how to use tools like SQLDeveloper and SQLModeler, performance tuning all in a quick 45-minute session which might take weeks to pick up reading documentation or spanning several presentations
Workload Automation for Cloud Migration and Machine Learning Platform
Activeeon has developed two Innovative Solutions based on workflows for:
1. Workload Automation for Cloud Migration
2. Data Science and Machine Learning Platform
Linking Scientific Instruments and Computation
[Talk presented at Monterey Data Conference, August 31, 2022]
Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subsets of such massive data streams, such as by explicitly discarding some data elements or by directing instruments to relevant areas of experimental space. Thus, methods are required for configuring and running distributed computing pipelines—what we call flows—that link instruments, computers (e.g., for analysis, simulation, AI model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. We review common patterns associated with such flows and describe methods for instantiating these patterns. We present experiences with the application of these methods to the processing of data from five different scientific instruments, each of which engages powerful computers for data inversion, machine learning model training, or other purposes. We also discuss implications of such methods for operators and users of scientific facilities.
I/O & virtualization performance with a search engine based on an xml databa...
See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011
HPC Cluster Computing from 64 to 156,000 Cores 
In this video from ChefConf 2014 in San Francisco, Cycle Computing CEO Jason Stowe outlines the biggest challenge facing us today, Climate Change, and suggests how Cloud HPC can help find a solution, including ideas around Climate Engineering, and Renewable Energy.
"As proof points, Jason uses three use cases from Cycle Computing customers, including from companies like HGST (a Western Digital Company), Aerospace Corporation, Novartis, and the University of Southern California. It’s clear that with these new tools that leverage both Cloud Computing, and HPC – the power of Cloud HPC enables researchers, and designers to ask the right questions, to help them find better answers, faster. This all delivers a more powerful future, and means to solving these really difficult problems."
Watch the video presentation: http://insidehpc.com/2014/09/video-hpc-cluster-computing-64-156000-cores/
Data Automation at Light Sources
Plenary talk at the international Synchrotron Radiation Instrumentation conference in Taiwan, on work with great colleagues Ben Blaiszik, Ryan Chard, Logan Ward, and others.
Rapidly growing data volumes at light sources demand increasingly automated data collection, distribution, and analysis processes, in order to enable new scientific discoveries while not overwhelming finite human capabilities. I present here three projects that use cloud-hosted data automation and enrichment services, institutional computing resources, and high- performance computing facilities to provide cost-effective, scalable, and reliable implementations of such processes. In the first, Globus cloud-hosted data automation services are used to implement data capture, distribution, and analysis workflows for Advanced Photon Source and Advanced Light Source beamlines, leveraging institutional storage and computing. In the second, such services are combined with cloud-hosted data indexing and institutional storage to create a collaborative data publication, indexing, and discovery service, the Materials Data Facility (MDF), built to support a host of informatics applications in materials science. The third integrates components of the previous two projects with machine learning capabilities provided by the Data and Learning Hub for science (DLHub) to enable on-demand access to machine learning models from light source data capture and analysis workflows, and provides simplified interfaces to train new models on data from sources such as MDF on leadership scale computing resources. I draw conclusions about best practices for building next-generation data automation systems for future light sources.
The Discovery Cloud: Accelerating Science via Outsourcing and Automation
Director's Colloquium at Los Alamos National Laboratory, September 18, 2014.
We have made much progress over the past decade toward harnessing the collective power of IT resources distributed across the globe. In high-energy physics, astronomy, and climate, thousands work daily within virtual computing systems with global scope. But we now face a far greater challenge: Exploding data volumes and powerful simulation tools mean that many more--ultimately most?--researchers will soon require capabilities not so different from those used by such big-science teams. How are we to meet these needs? Must every lab be filled with computers and every researcher become an IT specialist? Perhaps the solution is rather to move research IT out of the lab entirely: to leverage the “cloud” (whether private or public) to achieve economies of scale and reduce cognitive load. In this talk, I explore the past, current, and potential future of large-scale outsourcing and automation for science.
Science as a Service: How On-Demand Computing can Accelerate Discovery
My talk at ScienceCloud 2013 in NYC. Thanks to the organizers for the invitation to talk.
A bit of new material relative to previous talks posted, e.g., on Globus Genomics.
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Ever more data- and compute-intensive science makes computing increasingly important for research. But for advanced computing infrastructure to benefit more than the scientific 1%, we need new delivery methods that slash access costs, new sustainability models beyond direct research funding, and new platform capabilities to accelerate the development of new, interoperable tools and services.
The Globus team has been working towards these goals since 2010. We have developed software-as-a-service methods that move complex and time-consuming research IT tasks out of the lab and into the cloud, thus greatly reducing the expertise and resources required to use them. We have demonstrated a subscription-based funding model that engages research institutions in supporting service operations. And we are now also showing how the platform services that underpin Globus applications can accelerate the development and use of an integrated ecosystem of advanced science applications, such as NCAR’s Research Data Archive and OSG Connect, thus enabling access to powerful data and compute resources by many more people than is possible today.
In this talk, I introduce Globus services and the underlying Globus platform. I present representative applications and discuss opportunities that this platform presents for both small science and large facilities.
Estimating the Total Costs of Your Cloud Analytics Platform
Organizations today need a broad set of enterprise data cloud services with key data functionality to modernize applications and utilize machine learning. They need a platform designed to address multi-faceted needs by offering multi-function Data Management and analytics to solve the enterprise’s most pressing data and analytic challenges in a streamlined fashion. They need a worry-free experience with the architecture and its components.
Real-time Inverted Search in the Cloud Using Lucene and Storm
Building real-time notification systems is often limited to basic filtering and pattern matching against incoming records. Allowing users to query incoming documents using Solr's full range of capabilities is much more powerful. In our environment we needed a way to allow for tens of thousands of such query subscriptions, meaning we needed to find a way to distribute the query processing in the cloud. By creating in-memory Lucene indices from our Solr configuration, we were able to parallelize our queries across our cluster. To achieve this distribution, we wrapped the processing in a Storm topology to provide a flexible way to scale and manage our infrastructure. This presentation will describe our experiences creating this distributed, real-time inverted search notification framework.
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Benjamin Wright-Jones, Simon Lidberg. Are you interested in near real-time data processing but confused about Azure capabilities and product positioning? Spark, StreamInsight, Storm (HDInsight) and Stream Analytics offer ways to ingest data but there is uncertainty about when and how we should use these capabilities. For example, what are the differences and key solution design decision points? Come to this session to learn about current and new near real-time data processing engines. Go to https://channel9.msdn.com/ to find the recording of this session.
Windows Azure: Lessons From The Field
Presentation at CodeStock 2010 on the lessons learned while building an application that handles post-processing of scientific data in the Cloud.
Federated Cloud Computing
Presentation given to Oxfordshire BCS on 5th March 2015 talking about the OeRC and our work on the federation of cloud computing resources.
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Alluxio foresaw the need for agility when accessing data across silos separated from compute engines like Spark, Presto, Tensorflow and PyTorch. Embracing the separation of storage from compute, the Alluxio data orchestration platform simplifies adoption of the data lake and data mesh paradigm for analytics and AI/ML. In this talk, Bin Fan will share observations to help identify ways to use the platform to meet the needs of your data environment and workloads.
越來越多的企業架構已轉向混合雲和多雲環境。雖然這種轉變帶來了更大的靈活性和敏捷性，但也意味著必須將計算與存儲分離，這就對企業跨框架、跨雲和跨存儲系統的數據管理和編排提出了新的挑戰。此分享將讓聽眾深入了解Alluxio數據編排理念在數據中台對存儲和計算的解耦作用，以及數據編排針對存算分離場景提出的創新架構，同時結合來自金融、運營商、互聯網等行業的典型應用場景來展現Alluxio如何為大數據計算帶來真正的加速，以及如何將數據編排技術用於AI模型訓練！
Tutorial: What's New with Globus
This tutorial was given at the 2019 GlobusWorld Conference in Chicago, IL by Globus Head of Products Rachana Ananthakrishnan and Director of Customer Engagement Greg Nawrocki.
Power Up with Podman - Cloud Native + K8s Meetup
Curious about containers beyond Docker? There’s a new generation of containers on the scene, Podman! Supporting secure, rootless containers for Kubernetes microservices, it was designed and built with the cloud in mind. Benefitting from the lessons learned out in the open from Docker, this next generation of containers will quickly become a trusted daily driver in your dev workflow.
Covering what you need to know as an end-user from the UI to the backend, sharing a real world use case leveraging Podman for open source observability workshops https://o11y-workshops.gitlab.io. Paige will share how Podman and the adorable seal mascots Caitlín, Maighréad and Róisín have transformed her local development!
Choose Your Own Adventure - Cloud Native Observability Pitfalls
Are you looking at your organization's efforts to enter or expand into the cloud native landscape and feeling a bit daunted by the vast expanse of information surrounding cloud native observability? When you're moving so fast with agile practices across your DevOps, SRE's, and platform engineering teams, it's no wonder this can seem a bit confusing. Unfortunately, the choices being made have a great impact on both your business, your budgets, and the ultimate success of your cloud native initiatives. That hasty decision up front leads to big headaches very quickly down the road. In this talk, I'll introduce the problem facing everyone with cloud native observability followed by 3 common mistakes that I'm seeing organizations make and how you can avoid them!
Key takeaways - This session is never the same twice as you the audience / attendees choose from a list of cloud native observability pitfalls that DevOps have to contend with in their daily cloud native lives! Super engaging and fun to tour the challenges that interest you most!
More Related Content
Similar to Optimizing Observability Spend: Metrics
3 Pitfalls Everyone Should Avoid with Cloud Native Observability
Are you looking at your organization's efforts to enter or expand into the cloud native landscape and feeling a bit daunted by the vast expanse of information surrounding cloud native observability? When you're moving so fast with agile practices across your DevOps, SRE's, and platform engineering teams, it's no wonder this can seem a bit confusing. Unfortunately, the choices being made have a great impact on both your business, your budgets, and the ultimate success of your cloud native initiatives. That hasty decision up front leads to big headaches very quickly down the road. In this talk, I'll introduce the problem facing everyone with cloud native observability followed by 3 common mistakes that I'm seeing organizations make and how you can avoid them!
Working together with SURF Raymond Oonk Annette Langedijk SURF
Presentation for the Seminar Enabling Copernicus Big Data Analytics through European Open Science Cloud on 27 October 2021
GlobusWorld 2020 Keynote
This presentation was given at the GlobusWorld 2020 Virtual Conference, by Ian Foster, Rachana Ananthakrishnan, and Vas Vasiliadis from the University of Chicago.
Whats new in Autonomous Database in 2022
This session covers the new features and happenings in the autonomous database world and will help answer more questions DBAs and Developers will have on the Autonomous Database, from provisioning to backups, troubleshooting, tips and tricks, security and HA. This is a good introduction for on-prem DBAs who want to learn how this works quickly without spending too much time on it. Questions like what does the free tier cover, how do I do backup or if it's automated, how do I manage it, how to scale up and down, how to secure their environment, how to use mtls, how to use tools like SQLDeveloper and SQLModeler, performance tuning all in a quick 45-minute session which might take weeks to pick up reading documentation or spanning several presentations
Workload Automation for Cloud Migration and Machine Learning Platform
Activeeon has developed two Innovative Solutions based on workflows for:
1. Workload Automation for Cloud Migration
2. Data Science and Machine Learning Platform
Linking Scientific Instruments and Computation
[Talk presented at Monterey Data Conference, August 31, 2022]
Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subsets of such massive data streams, such as by explicitly discarding some data elements or by directing instruments to relevant areas of experimental space. Thus, methods are required for configuring and running distributed computing pipelines—what we call flows—that link instruments, computers (e.g., for analysis, simulation, AI model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. We review common patterns associated with such flows and describe methods for instantiating these patterns. We present experiences with the application of these methods to the processing of data from five different scientific instruments, each of which engages powerful computers for data inversion, machine learning model training, or other purposes. We also discuss implications of such methods for operators and users of scientific facilities.
I/O & virtualization performance with a search engine based on an xml databa...
See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011
HPC Cluster Computing from 64 to 156,000 Cores 
In this video from ChefConf 2014 in San Francisco, Cycle Computing CEO Jason Stowe outlines the biggest challenge facing us today, Climate Change, and suggests how Cloud HPC can help find a solution, including ideas around Climate Engineering, and Renewable Energy.
"As proof points, Jason uses three use cases from Cycle Computing customers, including from companies like HGST (a Western Digital Company), Aerospace Corporation, Novartis, and the University of Southern California. It’s clear that with these new tools that leverage both Cloud Computing, and HPC – the power of Cloud HPC enables researchers, and designers to ask the right questions, to help them find better answers, faster. This all delivers a more powerful future, and means to solving these really difficult problems."
Watch the video presentation: http://insidehpc.com/2014/09/video-hpc-cluster-computing-64-156000-cores/
Data Automation at Light Sources
Plenary talk at the international Synchrotron Radiation Instrumentation conference in Taiwan, on work with great colleagues Ben Blaiszik, Ryan Chard, Logan Ward, and others.
Rapidly growing data volumes at light sources demand increasingly automated data collection, distribution, and analysis processes, in order to enable new scientific discoveries while not overwhelming finite human capabilities. I present here three projects that use cloud-hosted data automation and enrichment services, institutional computing resources, and high- performance computing facilities to provide cost-effective, scalable, and reliable implementations of such processes. In the first, Globus cloud-hosted data automation services are used to implement data capture, distribution, and analysis workflows for Advanced Photon Source and Advanced Light Source beamlines, leveraging institutional storage and computing. In the second, such services are combined with cloud-hosted data indexing and institutional storage to create a collaborative data publication, indexing, and discovery service, the Materials Data Facility (MDF), built to support a host of informatics applications in materials science. The third integrates components of the previous two projects with machine learning capabilities provided by the Data and Learning Hub for science (DLHub) to enable on-demand access to machine learning models from light source data capture and analysis workflows, and provides simplified interfaces to train new models on data from sources such as MDF on leadership scale computing resources. I draw conclusions about best practices for building next-generation data automation systems for future light sources.
The Discovery Cloud: Accelerating Science via Outsourcing and Automation
Director's Colloquium at Los Alamos National Laboratory, September 18, 2014.
We have made much progress over the past decade toward harnessing the collective power of IT resources distributed across the globe. In high-energy physics, astronomy, and climate, thousands work daily within virtual computing systems with global scope. But we now face a far greater challenge: Exploding data volumes and powerful simulation tools mean that many more--ultimately most?--researchers will soon require capabilities not so different from those used by such big-science teams. How are we to meet these needs? Must every lab be filled with computers and every researcher become an IT specialist? Perhaps the solution is rather to move research IT out of the lab entirely: to leverage the “cloud” (whether private or public) to achieve economies of scale and reduce cognitive load. In this talk, I explore the past, current, and potential future of large-scale outsourcing and automation for science.
Science as a Service: How On-Demand Computing can Accelerate Discovery
My talk at ScienceCloud 2013 in NYC. Thanks to the organizers for the invitation to talk.
A bit of new material relative to previous talks posted, e.g., on Globus Genomics.
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Ever more data- and compute-intensive science makes computing increasingly important for research. But for advanced computing infrastructure to benefit more than the scientific 1%, we need new delivery methods that slash access costs, new sustainability models beyond direct research funding, and new platform capabilities to accelerate the development of new, interoperable tools and services.
The Globus team has been working towards these goals since 2010. We have developed software-as-a-service methods that move complex and time-consuming research IT tasks out of the lab and into the cloud, thus greatly reducing the expertise and resources required to use them. We have demonstrated a subscription-based funding model that engages research institutions in supporting service operations. And we are now also showing how the platform services that underpin Globus applications can accelerate the development and use of an integrated ecosystem of advanced science applications, such as NCAR’s Research Data Archive and OSG Connect, thus enabling access to powerful data and compute resources by many more people than is possible today.
In this talk, I introduce Globus services and the underlying Globus platform. I present representative applications and discuss opportunities that this platform presents for both small science and large facilities.
Estimating the Total Costs of Your Cloud Analytics Platform
Organizations today need a broad set of enterprise data cloud services with key data functionality to modernize applications and utilize machine learning. They need a platform designed to address multi-faceted needs by offering multi-function Data Management and analytics to solve the enterprise’s most pressing data and analytic challenges in a streamlined fashion. They need a worry-free experience with the architecture and its components.
Real-time Inverted Search in the Cloud Using Lucene and Storm
Building real-time notification systems is often limited to basic filtering and pattern matching against incoming records. Allowing users to query incoming documents using Solr's full range of capabilities is much more powerful. In our environment we needed a way to allow for tens of thousands of such query subscriptions, meaning we needed to find a way to distribute the query processing in the cloud. By creating in-memory Lucene indices from our Solr configuration, we were able to parallelize our queries across our cluster. To achieve this distribution, we wrapped the processing in a Storm topology to provide a flexible way to scale and manage our infrastructure. This presentation will describe our experiences creating this distributed, real-time inverted search notification framework.
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Benjamin Wright-Jones, Simon Lidberg. Are you interested in near real-time data processing but confused about Azure capabilities and product positioning? Spark, StreamInsight, Storm (HDInsight) and Stream Analytics offer ways to ingest data but there is uncertainty about when and how we should use these capabilities. For example, what are the differences and key solution design decision points? Come to this session to learn about current and new near real-time data processing engines. Go to https://channel9.msdn.com/ to find the recording of this session.
Windows Azure: Lessons From The Field
Presentation at CodeStock 2010 on the lessons learned while building an application that handles post-processing of scientific data in the Cloud.
Federated Cloud Computing
Presentation given to Oxfordshire BCS on 5th March 2015 talking about the OeRC and our work on the federation of cloud computing resources.
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Alluxio foresaw the need for agility when accessing data across silos separated from compute engines like Spark, Presto, Tensorflow and PyTorch. Embracing the separation of storage from compute, the Alluxio data orchestration platform simplifies adoption of the data lake and data mesh paradigm for analytics and AI/ML. In this talk, Bin Fan will share observations to help identify ways to use the platform to meet the needs of your data environment and workloads.
越來越多的企業架構已轉向混合雲和多雲環境。雖然這種轉變帶來了更大的靈活性和敏捷性，但也意味著必須將計算與存儲分離，這就對企業跨框架、跨雲和跨存儲系統的數據管理和編排提出了新的挑戰。此分享將讓聽眾深入了解Alluxio數據編排理念在數據中台對存儲和計算的解耦作用，以及數據編排針對存算分離場景提出的創新架構，同時結合來自金融、運營商、互聯網等行業的典型應用場景來展現Alluxio如何為大數據計算帶來真正的加速，以及如何將數據編排技術用於AI模型訓練！
Tutorial: What's New with Globus
This tutorial was given at the 2019 GlobusWorld Conference in Chicago, IL by Globus Head of Products Rachana Ananthakrishnan and Director of Customer Engagement Greg Nawrocki.
Similar to Optimizing Observability Spend: Metrics (20)
3 Pitfalls Everyone Should Avoid with Cloud Native Observability
3 Pitfalls Everyone Should Avoid with Cloud Native Observability
Working together with SURF Raymond Oonk Annette Langedijk SURF
Working together with SURF Raymond Oonk Annette Langedijk SURF
Workload Automation for Cloud Migration and Machine Learning Platform
Workload Automation for Cloud Migration and Machine Learning Platform
I/O & virtualization performance with a search engine based on an xml databa...
I/O & virtualization performance with a search engine based on an xml databa...
The Discovery Cloud: Accelerating Science via Outsourcing and Automation
The Discovery Cloud: Accelerating Science via Outsourcing and Automation
Science as a Service: How On-Demand Computing can Accelerate Discovery
Science as a Service: How On-Demand Computing can Accelerate Discovery
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Estimating the Total Costs of Your Cloud Analytics Platform
Estimating the Total Costs of Your Cloud Analytics Platform
Real-time Inverted Search in the Cloud Using Lucene and Storm
Real-time Inverted Search in the Cloud Using Lucene and Storm
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
More from Eric D. Schabell
Power Up with Podman - Cloud Native + K8s Meetup
Curious about containers beyond Docker? There’s a new generation of containers on the scene, Podman! Supporting secure, rootless containers for Kubernetes microservices, it was designed and built with the cloud in mind. Benefitting from the lessons learned out in the open from Docker, this next generation of containers will quickly become a trusted daily driver in your dev workflow.
Covering what you need to know as an end-user from the UI to the backend, sharing a real world use case leveraging Podman for open source observability workshops https://o11y-workshops.gitlab.io. Paige will share how Podman and the adorable seal mascots Caitlín, Maighréad and Róisín have transformed her local development!
Choose Your Own Adventure - Cloud Native Observability Pitfalls
Are you looking at your organization's efforts to enter or expand into the cloud native landscape and feeling a bit daunted by the vast expanse of information surrounding cloud native observability? When you're moving so fast with agile practices across your DevOps, SRE's, and platform engineering teams, it's no wonder this can seem a bit confusing. Unfortunately, the choices being made have a great impact on both your business, your budgets, and the ultimate success of your cloud native initiatives. That hasty decision up front leads to big headaches very quickly down the road. In this talk, I'll introduce the problem facing everyone with cloud native observability followed by 3 common mistakes that I'm seeing organizations make and how you can avoid them!
Key takeaways - This session is never the same twice as you the audience / attendees choose from a list of cloud native observability pitfalls that DevOps have to contend with in their daily cloud native lives! Super engaging and fun to tour the challenges that interest you most!
OpenShift Commons Paris - Choose Your Own Observability Adventure
Great observability begins with great instrumentation! We know it's hard to decide where to start your observability journey, so we've come up with a perfect introduction to observability workshop collection getting you hands-on with the best open source cloud native observability projects available. Attendees can pick their own cloud native observability learning path (https://o11y-workshops.gitlab.io) in this session from the following workshops:
OpenTelemetry (traces) - Learn how to adopt OpenTelemetry by instrumenting a sample application with spans and metrics. You’ll leave with an understanding of how telemetry travels and be ready to bring OpenTelemetry to your project. The workshop is self-paced and available online, so attendees can continue to explore after the event: https://o11y-workshops.gitlab.io/workshop-opentelemetry
Prometheus (metrics) - During the workshop, you will install Prometheus, collect metrics, and learn how to effectively run it in your observability stack. The workshop is self-paced and available online, so attendees can continue to explore after the event: https://o11y-workshops.gitlab.io/workshop-prometheus
Fluent Bit (pipelines) - This workshop will guide you through the open source project Fluent Bit, what it is, a basic installation, and setting up a first cloud native observability pipeline project. The workshop is self-paced and available online, so attendees can continue to explore after the event: https://o11y-workshops.gitlab.io/workshop-fluentbit
Perses (visualization) - Great observability is impossible without great visualization! Learn how to adopt truly open visualization by installing Perses, exploring the provided tooling, tinkering with its API, and then get your hands dirty building your first dashboard in no time! The workshop is self-paced and available online, so attendees can continue to explore after the event: https://o11y-workshops.gitlab.io/workshop-perses
3 Pitfalls Everyone Should Avoid with Cloud Data
The daily hype is all around you. From cloud native, multicloud, to hybrid cloud, this is the path to your digital future. The choices you make as a developer does not preclude the daily work of enhancing your customer's experience and agile delivery of your applications. With all this delivery and infrastructure, there is a lot of data generated when engaging with any cloud experience. Regulatory and compliance pressures force us to store audit and observability data. Understanding the pitfalls around the collection, storage, and maintenance of your cloud data can mean the difference between bankruptcy and success with our cloud native strategy. Let us take you on a journey, looking closely at the decisions you are making as a developer delivering and dealing with monitoring your applications. Join us for an hour of power, where real customer experiences are used to highlight the three top lessons learned as their developers transitioned their data needs into cloud native environments.
Key Takeaways: Attendees to this session will gain insights into the data explosion that is part of the large scale cloud native world. Real customer experiences are used to highlight the three top lessons learned as their developers transitioned their data needs into cloud native environments.
Roadmap to Becoming a CNCF Ambassador
The CNCF Ambassador program is designed for individuals who are passionate about cloud native technologies and want to contribute to the community. Becoming an ambassador is a great opportunity to enhance your knowledge, gain visibility within the industry, and help drive the adoption of cloud-native technologies. The road to becoming an ambassador might seem intimidating, scary, or just impossible, but it does not have to be. We've put together a roadmap that leads you to the title of CNCF Ambassador. In this session a current ambassador and the community manager share the stage to bring you insights into achieving the title of CNCF Ambassador. Whether you are a developer, student, or seasoned professional, this talk provides attendees with 5 actionable insights needed to take your cloud-native skills to the next level and become a CNCF Ambassador. Join us to learn how you can contribute to the community and advance your career by taking the road to the CNCF Ambassador community.
Cloud Native Bedtime Stories - Terrifying Execs into Action
Anyone embedded in the cloud native teams in any organization can voice their frustrations at not being taken seriously by their executive decision makers. This leads to way too much on-call stress, frustrations, and eventual burnout. With research showing us DevOps spending over 10 hrs a week on issues in their environments, we could all use quick action by our executives when we find ways to fix our cloud native issues. The trick is to tell the tales we accumulate in such a way as to engage, inspire, and effect change in our organizations. This session provides attendees with ample cloud native bedtime stories, tricks that make your tales land within the executive human mind, and actionable insights to head home with immediate results. Join me for a half hour of power where you are empowered to tell better cloud native stories for better executive decision outcomes.
Key takeaways - Attendees to this session will be given a small yet powerful set of examples to help them effectively tell their cloud native observability tales to motivate their executives into action. Humans listen to stories (tales) more than they pay attention to pages of charts, dashboards, and data. Learn how to tell your tales, terrifying and educational, with tips and tricks to engage your executives into believing your need for organization’s observability improvements.
Optimizing Observability Spend: Metrics
Are you collecting just about every metric under the sun and the kitchen sink too? Understanding the cost of collecting metrics and the usefulness of those metrics is the only way to scale in a cloud native world. You can’t get away with just collecting everything as you grow. Your observability teams need to make decisions about what to collect, what to drop, what to aggregate, and still be able to alert, triage, remediate, and do their root cause analysis on a daily basis. Gain immediate insights into high cost data (DPPS), when to drop time series data, and how to determine when the value of that data is at its lowest. Session includes a recorded demo video of it in action.
Engaging Your Execs - Telling Great Observability Tales Inspiring Action
Anyone embedded in the cloud native observability teams in any organization can voice their frustrations at not being taken seriously by their executive decision makers. This leads to way too much on-call stress, frustrations, and eventual burnout. With research showing us DevOps spending over 10 hrs a week on issues in their environments, we could all use quick action by our executives when we find ways to fix our cloud native issues. The trick is to tell the tales we accumulate in such a way as to engage, inspire, and effect change in our organizations. This session provides attendees with ample cloud native bedtime stories, tricks that make your tales land within the executive human mind, and actionable insights to head home with immediate results. Join me for a half hour of power where you are empowered to tell better observability stories for better executive decision outcomes.
WTF is SRE - Telling Effective Tales about Production
Storytelling is as old as time itself…. Since the beginning of humankind, we share our experiences, we teach, we inspire, we relate to stories as told all around us. How can we learn to use this powerful mechanism to tell effective tales about our production environments when dealing with our management teams?
Learn how humans listen to stories (tales) more than they pay attention to pages of charts, dashboards, and data. If you want to learn how to make sure your message lands and how to effectively manage upwards in your organization, this is the session for you. Attendees will depart with a small yet powerful set of actionable examples that almost ensure your stories will capture your management's attention. One thing is certain, stories are being told, but what are your production stories and how can you become adept at telling them?
3 Pitfalls Everyone Should Avoid with Cloud Data
The daily hype is all around you. From cloud native, multicloud, to hybrid cloud, this is the path to your digital future. The choices you make as a developer does not preclude the daily work of enhancing your customer's experience and agile delivery of your applications. With all this delivery and infrastructure, there is a lot of data generated when engaging with any cloud experience. Regulatory and compliance pressures force us to store audit and observability data. Understanding the pitfalls around the collection, storage, and maintenance of your cloud data can mean the difference between bankruptcy and success with our cloud native strategy. Let us take you on a journey, looking closely at the decisions you are making as a developer delivering and dealing with monitoring your applications. Join us for an hour of power, where real customer experiences are used to highlight the three top lessons learned as their developers transitioned their data needs into cloud native environments.
Key Takeaways: Attendees to this session will gain insights into the data explosion that is part of the large scale cloud native world. Real customer experiences are used to highlight the three top lessons learned as their developers transitioned their data needs into cloud native environments.
3 Pitfalls Everyone Should Avoid with Cloud Native Data
The daily hype is all around you. From cloud native, multicloud, to hybrid cloud, this is the path to your digital future. The choices you make as a developer does not preclude the daily work of enhancing your customer's experience and agile delivery of your applications. With all this delivery and infrastructure, there is a lot of data generated when engaging with any cloud experience. Regulatory and compliance pressures force us to store audit and observability data. Understanding the pitfalls around the collection, storage, and maintenance of your cloud data can mean the difference between bankruptcy and success with our cloud native strategy. Let us take you on a journey, looking closely at the decisions you are making as a developer delivering and dealing with monitoring your applications. Join us for an hour of power, where real customer experiences are used to highlight the three top lessons learned as their developers transitioned their data needs into cloud native environments.
Key Takeaways: Attendees to this session will gain insights into the data explosion that is part of the large scale cloud native world. Real customer experiences are used to highlight the three top lessons learned as their developers transitioned their data needs into cloud native environments.
Storytelling - How to build and delivery a story
Storytelling is as old as time itself…. Since the beginning of humankind, we share our experiences, we teach, we inspire, we relate to stories as told all around us. They are told by elders, they are told by kids at the dinner table, they are written down in books, and they are captured on video or tape. One thing is certain, stories are being told, but what are your stories and how can you become adept at telling them?
These slides: http://go/storytelling
Shift left Observability
Whether you’re an enterprise migrating to cloud native or born in the cloud, most of today’s APM and Observability tools don’t support how your engineers and DevOps teams need to develop, deploy, and support their software. Observability needs to shift left and reflect the modern way companies organize their development teams and their vital interdependencies.
Chronosphere is the only vendor addressing the unique requirements for observability in a cloud native world. Join this webinar to learn:
- What cloud native observability is and how it is different from the promises made by traditional cloud APM and observability vendors
- How to use cloud native observability to do more “Dev” and less “Ops” so you can dramatically improve developer and engineer workflows and productivity
- How to make on-call shifts less stressful so your engineers aren’t getting burned out
Trajectory 2022 - Shifting Cloud Native Observability to the Left
The emergence of cloud native environments has changed our platform world. Not only are you coding, but expected to deliver, maintain, and support all aspects of the platform. This change means the Ops in DevOps has been overshadowing why we got into development in the first place. There is hope for us yet if we can just manage to shift left to close the gap in our time spent on non-operational activities. How do we do that, you ask? It’s imperative that developers are given not only the organizational structure to succeed, and even more critical is the ability to reduce time on triaging failures in their deployments. Attendees will be provided with insights into DevOps cloud native tips and tricks to spending more time on projects that matter and less on call. Join us in learning how you can shift left with your DevOps activities to bring more joy to your days.
PromCon EU 2022 - Centralized vs Decentralized Prometheus Scraping Architectu...
There are two primary approaches to scrape and collect metrics using Prometheus - using a centralized set of dedicated scrapers or decentralized scrapers that run as an agent. With centralized scraping, Prometheus is deployed as a central scraper to pull metrics from all discoverable endpoints and sometimes can be split across multiple centralized instances using a few different approaches. However, with a decentralized approach, Prometheus runs as an agent and in Kubernetes is deployed as a DaemonSet on each node in a cluster and only collects metrics from the node it runs on. Each model has pros and cons - especially when operating at large scale - which can make it difficult when deciding which model to use.
In this session, the speaakers will provide an overview of Prometheus metrics collection at DoorDash, where having highly reliable resources, easy endpoint discovery, and real-time insights is critical. They will share insights and best practices into DoorDash’s decision to implement a decentralized model by offering pros and cons of each approach. Leave with a better understanding of the “right” model for your use case(s).
How to Wrestle Your Observability Data Demons and Win!
The crazy growth of observability data often outpaces the growth of your applications and infrastructure. Data volume and complexity can quickly grow out of control – reducing system reliability, lengthening remediation time, and increasing costs.
To get data under control, central observability teams and engineers need detailed visibility into data usage, the ability to filter and aggregate data intelligently, and manage high cardinality data without weakening issue detection or query performance.
In this webinar, you will learn:
- About different architectures and methods for ingesting, filtering, and aggregating complex data
- Why Chronosphere’s unique server-side approach is optimally designed for the scale and complexity of cloud native observability data
- How some of the most successful cloud native businesses have reduced data volumes by 42% while improving issue detection and remediation
DevConf.US 2022 - Exploring Open Source Edge Success at Scale
You've heard of large scale open source architectures, but have you ever wanted to take a serious look at real life enterprise edge implementations that scale? This session takes attendees on a tour of multiple use cases for enterprise challenges on the edge with integration, telco, healthcare, manufacturing, and much more. Not only are these architectures interesting, but they are successful real life implementations featuring open source technologies and power many of your own edge experiences.
The attendee departs this session with a working knowledge of how to map general open source technologies to their own edge solutions. Material covered is available freely online and attendees can use these solutions as starting points for aligning to their own solution architectures. Join us for an hour of power as we talk architecture shop!
DevConf.US 2022 - Exploring Open Source Telco Success at Scale
(Note the title of the slides was updated so does not match, but content is correct)
You've heard of large scale open source architectures, but have you ever wanted to take a serious look at real life enterprise telco implementations that scale? This session takes attendees on a tour of multiple use cases for enterprise challenges in the telco domain with a look at use cases for integration, networks, 5G, hyperscalers, and much more. Not only are these architectures interesting, but they are successful real life implementations featuring open source technologies and power many of your own edge experiences.
The attendee departs this session with a working knowledge of how to map general open source technologies to their own telco solutions. Material covered is available freely online and attendees can use these solutions as starting points for aligning to their own solution architectures. Join us for an hour of power as we talk architecture shop!
OpenShift Commons Dublin 2022 - 3 Pitfalls Everyone Should Avoid with Cloud Data
The daily hype is all around you. From cloud, multicloud, to hybrid cloud, this is the path to your digital future. These choices you’ve got to make don’t preclude the daily work of enhancing your customer's experience and agile delivery of those applications. With all this delivery and infrastructure, there is a lot of data to consider when engaging with any cloud experience. Regulatory and compliance pressures force us to store audit and observability data. Understanding the pitfalls around the collection, storage, and maintenance of our data can mean the difference between bankruptcy and success with our cloud strategy. Let us take you on a journey, looking closely at the decisions being made for delivering and dealing with monitoring those applications. Join us for an hour of power, where real customer experiences are used to highlight the three top lessons learned as they transitioned their data needs into cloud environments.
Designing Your Best Architectural Diagrams
Diagraming is one of the most important communication tools for sharing your project and architectural ideas to your colleagues and teams. In this workshop walkthrough, attendees are exposed to an open source tool we host online for designing architecture diagrams like an expert. Attendees are walked through the following in just 30 mins:
- open and explore the tooling in your favourite web browser
- explore the provided asset libraries for drag-and-drop designing
- learn about the three types of diagrams that make up a good design
- create your first simple logical diagram
- create your first simple schematic diagram
- create a detailed diagram
- how to import and export diagrams and elements from a diagram
This session is an introduction to the free online workshop available for attendees to jump right into after the session. Each of the individual labs in this workshop are stand alone, allowing the attendee to focus on anything of interest without having to work through the previous labs. If you're looking to become more proficient in sharing your ideas, architectures, and projects visually to wider audiences you can't underestimate the value of a good diagram. Join us to learn the tips and tricks that make a good diagram such a good communication vehicle and how our tooling eases your design tasks. Then head homewards with a free online workshop just waiting for you to explore!
https://bit.ly/your-best-diagrams
More from Eric D. Schabell (20)
Choose Your Own Adventure - Cloud Native Observability Pitfalls
Choose Your Own Adventure - Cloud Native Observability Pitfalls
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
Cloud Native Bedtime Stories - Terrifying Execs into Action
Cloud Native Bedtime Stories - Terrifying Execs into Action
Engaging Your Execs - Telling Great Observability Tales Inspiring Action
Engaging Your Execs - Telling Great Observability Tales Inspiring Action
WTF is SRE - Telling Effective Tales about Production
WTF is SRE - Telling Effective Tales about Production
3 Pitfalls Everyone Should Avoid with Cloud Native Data
3 Pitfalls Everyone Should Avoid with Cloud Native Data
Trajectory 2022 - Shifting Cloud Native Observability to the Left
Trajectory 2022 - Shifting Cloud Native Observability to the Left
PromCon EU 2022 - Centralized vs Decentralized Prometheus Scraping Architectu...
PromCon EU 2022 - Centralized vs Decentralized Prometheus Scraping Architectu...
How to Wrestle Your Observability Data Demons and Win!
How to Wrestle Your Observability Data Demons and Win!
DevConf.US 2022 - Exploring Open Source Edge Success at Scale
DevConf.US 2022 - Exploring Open Source Edge Success at Scale
DevConf.US 2022 - Exploring Open Source Telco Success at Scale
DevConf.US 2022 - Exploring Open Source Telco Success at Scale
OpenShift Commons Dublin 2022 - 3 Pitfalls Everyone Should Avoid with Cloud Data
OpenShift Commons Dublin 2022 - 3 Pitfalls Everyone Should Avoid with Cloud Data
Recently uploaded
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovation With Your Product by VP of Product Design, Warner Music Group
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Presented by Suzanne Phillips and Alex Marcotte
Key Trends Shaping the Future of Infrastructure.pdf
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
JMeter webinar - integration with InfluxDB and Grafana
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP of Product, Amplitude
Knowledge engineering: from people to machines and back
Keynote at the 21st European Semantic Web Conference
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, Product School
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Epistemic Interaction - tuning interfaces to provide information for AI support
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Elevating Tactical DDD Patterns Through Object Calisthenics
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Connector Corner: Automate dynamic content and events by pushing a button
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Recently uploaded (20)
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Optimizing Observability Spend: Metrics
- 1. Optimizing Observability Spend: Metrics Eric D. Schabell Director Evangelism @ericschabell{@fosstodon.org} Apr 2023 KubeCon EU 2023
- 4. chronosphere.io Data volume Experiment: - Hello World app on 4 node Kubernetes cluster with Tracing, End User Metrics (EUM), Logs, Metrics (containers / nodes) - 30 days == +450 GB
- 5. chronosphere.io “If you have to ask, you can’t afford it…” Only if we get better outcomes… chronosphere.io
- 6. Chronosphere Overview Chronosphere Collector Supports all standard open source formats Chronosphere Data Store Proven to scale to 2B data points per second Chronosphere Control Plane Analyze, shape, and manage your data based on need, context, and utility Chronosphere Query Engine 100% PromQL compatible Dashboards & Alerts Grafana & Alertmanager compatible
- 7. chronosphere.io Chronosphere Benefits The leading cloud native companies use Chronosphere enterprise-wide Engineers spend 50% less time troubleshooting, on average Reduce observability data volumes by 50% or more Most reliable platform 99.9% uptime SLA 100% open source compatible
- 8. chronosphere.io Customer Impact - Control Plane optimization of data volumes optimization of data volumes metrics reduction via aggregation 98% 80% 41% query latency improvement 8x
- 9. Booth #S8 Eric D. Schabell Director Evangelism @ericschabell{@fosstodon.org}