Three Pillars, No Answers: Helping Platform Teams Solve Real Observability Problems
•
0 likes•165 views
Observability has never been more important: the complexity of microservices makes it harder and harder to answer basic questions about system behavior. The conventional wisdom claims that Metrics, Logging and Tracing are “the three pillars” of observability… yet software organizations check these three boxes and are still grasping at straws during emergencies. In this session we’ll illustrate the problem with the three pillars: metrics, logs, and traces are just data – they are the fuel, not the car.
Report
Share
Report
Share
Download to read offline
Recommended
More Than Monitoring: How Observability Takes You From Firefighting to Fire P...
For some, observability is just a hollow rebranding of monitoring, for others it’s monitoring on steroids. But what if we told you observability is the new way to find out why—not just if—your distributed system or application isn’t working as expected? Today, we see that traditional monitoring approaches can fall short if a system or application doesn’t adequately externalize its state.
This is truer as workloads move into the cloud and leverage ephemeral technologies, such as microservices and containers. To reach observability, IT and DevOps teams need to correlate different sources from logs, metrics, traces, events and more. This becomes even more challenging when defining the online revenue impact of a failed container—after all, this is what really matters to the business.
This webinar will cover:
The differences between observability and monitoring
Why it is a bigger challenge in a multicloud and containerized world
How observability results in less firefighting and more fire prevention
How new platforms can help gain observability (on premises and in the cloud) for containers, microservices and even SAP or mainframes
Achieving observability-in-modern-applications
Modern applications are complex with microservices and containers which makes observability challenging. Observability refers to understanding an application's state through logs, events and other data. It is more comprehensive than monitoring alone. To achieve effective observability, organizations should use error tracking, distributed tracing, APM, infrastructure monitoring, log aggregation and incident management tools. They should also implement development best practices like shift-left processes and continuous communication between teams. The goal of observability is to deliver the best user experience and maximize business value.
Do You Really Need to Evolve From Monitoring to Observability?
The document discusses the concepts of monitoring and observability. It defines observability as focusing on what can't be seen or the unknowns in a system. Observability provides visibility into the state of applications, systems, and services through logs, metrics, and traces to understand problems and take actions. The document then summarizes SignalFx's approach to observability, which combines metrics, traces, and logs in a streaming architecture to provide insights in seconds and help troubleshoot issues.
Splunk Discovery Köln - 17-01-2020 - Willkommen!
The document outlines the agenda for the Splunk Discovery event in Cologne on January 17, 2020. The event will include a keynote and demo on Splunk's data-to-everything platform, followed by breakout sessions in both a tech track and business track on various topics such as data value creation, customer use cases, security analytics methods, and the future of IT operations and security operations. There will also be lunch, customer presentations, and networking opportunities. Hands-on demonstrations of Splunk for IT operations and security will be available in the showcase area.
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
The document discusses Splunk Incident Response, orchestration and automation capabilities. It notes that incident response currently takes significant time, from months for detection to days for containment and remediation. Splunk aims to accelerate this process through automation, orchestration and its security operations platform to integrate tools, streamline workflows and automate repetitive tasks. The presentation demonstrates Splunk's Phantom security orchestration product and how it can automate security tasks like malware investigations to reduce response times.
The Top 10 Glasstable Design Principles to Boost Your Career and Your Business
The document provides 10 principles for effective data visualization and dashboard design on glass tables. It discusses choosing the right visualization for different types of data, using colors and fonts effectively, following principles of visual hierarchy and proximity, leveraging diagrams and flows, and using alerts and a dramatic approach when needed. It also recommends starting with paper prototypes, using simple tools like PowerPoint, and following current design trends like flat and minimalist styles.
Splunk Platform 2020 & Beyond
Looking into 2020 and beyond, we are certainly going to continue this trend of strategic technology investment and architecture evolution. This session’s aim is to highlight Splunk platform evolutionary approach to address key technology trends. Additionally, many customers are adopting Serverless cloud services to deliver their cloud solutions. This session will include a live demo of a new library of functions which provides Google Cloud Platform (GCP) serverless “push” capability to send data into Splunk, via HTTP Event Collector (HEC).
Machine Learning and Social Good
AI and Machine Learning can be used for the greater good. Why should AI be applied and what are the main challenges that have to be tackled when harnessing the power of AI for social good? Find out more here.
Recommended
More Than Monitoring: How Observability Takes You From Firefighting to Fire P...
For some, observability is just a hollow rebranding of monitoring, for others it’s monitoring on steroids. But what if we told you observability is the new way to find out why—not just if—your distributed system or application isn’t working as expected? Today, we see that traditional monitoring approaches can fall short if a system or application doesn’t adequately externalize its state.
This is truer as workloads move into the cloud and leverage ephemeral technologies, such as microservices and containers. To reach observability, IT and DevOps teams need to correlate different sources from logs, metrics, traces, events and more. This becomes even more challenging when defining the online revenue impact of a failed container—after all, this is what really matters to the business.
This webinar will cover:
The differences between observability and monitoring
Why it is a bigger challenge in a multicloud and containerized world
How observability results in less firefighting and more fire prevention
How new platforms can help gain observability (on premises and in the cloud) for containers, microservices and even SAP or mainframes
Achieving observability-in-modern-applications
Modern applications are complex with microservices and containers which makes observability challenging. Observability refers to understanding an application's state through logs, events and other data. It is more comprehensive than monitoring alone. To achieve effective observability, organizations should use error tracking, distributed tracing, APM, infrastructure monitoring, log aggregation and incident management tools. They should also implement development best practices like shift-left processes and continuous communication between teams. The goal of observability is to deliver the best user experience and maximize business value.
Do You Really Need to Evolve From Monitoring to Observability?
The document discusses the concepts of monitoring and observability. It defines observability as focusing on what can't be seen or the unknowns in a system. Observability provides visibility into the state of applications, systems, and services through logs, metrics, and traces to understand problems and take actions. The document then summarizes SignalFx's approach to observability, which combines metrics, traces, and logs in a streaming architecture to provide insights in seconds and help troubleshoot issues.
Splunk Discovery Köln - 17-01-2020 - Willkommen!
The document outlines the agenda for the Splunk Discovery event in Cologne on January 17, 2020. The event will include a keynote and demo on Splunk's data-to-everything platform, followed by breakout sessions in both a tech track and business track on various topics such as data value creation, customer use cases, security analytics methods, and the future of IT operations and security operations. There will also be lunch, customer presentations, and networking opportunities. Hands-on demonstrations of Splunk for IT operations and security will be available in the showcase area.
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
The document discusses Splunk Incident Response, orchestration and automation capabilities. It notes that incident response currently takes significant time, from months for detection to days for containment and remediation. Splunk aims to accelerate this process through automation, orchestration and its security operations platform to integrate tools, streamline workflows and automate repetitive tasks. The presentation demonstrates Splunk's Phantom security orchestration product and how it can automate security tasks like malware investigations to reduce response times.
The Top 10 Glasstable Design Principles to Boost Your Career and Your Business
The document provides 10 principles for effective data visualization and dashboard design on glass tables. It discusses choosing the right visualization for different types of data, using colors and fonts effectively, following principles of visual hierarchy and proximity, leveraging diagrams and flows, and using alerts and a dramatic approach when needed. It also recommends starting with paper prototypes, using simple tools like PowerPoint, and following current design trends like flat and minimalist styles.
Splunk Platform 2020 & Beyond
Looking into 2020 and beyond, we are certainly going to continue this trend of strategic technology investment and architecture evolution. This session’s aim is to highlight Splunk platform evolutionary approach to address key technology trends. Additionally, many customers are adopting Serverless cloud services to deliver their cloud solutions. This session will include a live demo of a new library of functions which provides Google Cloud Platform (GCP) serverless “push” capability to send data into Splunk, via HTTP Event Collector (HEC).
Machine Learning and Social Good
AI and Machine Learning can be used for the greater good. Why should AI be applied and what are the main challenges that have to be tackled when harnessing the power of AI for social good? Find out more here.
Wie erkenne ich die Auswirkungen von IT Ausfallen auf meine Produktion?
This presentation discusses how Splunk software can help companies identify the impacts of IT outages on production. It provides examples of how Splunk has helped companies like Porsche transform digitally and implement AIOps. Key benefits highlighted include collecting all types of machine data, reducing alert noise through clustering and correlation, adding context through indicators and business metadata, performing anomaly detection and predictive analytics, and assisting with deep dive investigations.
Splunk Discovery Köln - 17-01-2020 - Splunk for ITOps
This document provides an agenda for a Splunk Discovery event taking place in Cologne, Germany on January 17th, 2020. The event will include a keynote and demo on Splunk's data platform in the morning. In the afternoon, there will be parallel tech and business tracks covering topics like security analytics, incident response, and the future of IT operations and monitoring. The document also includes disclaimers about forward-looking statements and that product roadmaps are subject to change.
Splunk Overview
The document appears to be a presentation by Splunk Inc. discussing their data platform. Some key points:
1. Splunk's platform allows customers to investigate, monitor, analyze and act on data from any source in real-time.
2. It addresses challenges of collecting and making sense of massive amounts of data from various systems and devices across IT, security, and IoT use cases.
3. Splunk provides solutions and services to help customers accelerate their data journey from initial investigation to taking action.
.conf21 - The Best of
This document provides an agenda and summaries of sessions from the Splunk .conf21 conference on DevOps and Observability topics. It introduces new features for Splunk APM like code-level profiling and database visibility. It also covers new mobile monitoring capabilities and an auto-detection feature for infrastructure monitoring. Recommended sessions are listed on monitoring environments, troubleshooting, and using observability data for AIOps processes. The document encourages attendees to access the online sessions on demand.
Security Automation & Orchestration
On your marks, get set GO!
Take a more in-depth look at the automation and orchestration journey and the future of SOAR.
Watch the SOCtails video here: https://www.youtube.com/watch?v=YzsGQzqaDYw&t=2s
Better Threat Analytics: From Getting Started to Cloud Security Analytics and...
Fighting the Eternal Challenge: Dealing with Alert Fatigue and Getting Insights into Security Productivity.
Lessons for a Fast Start in Automation and Orchestration.
Manufacturing Webinar AMS
The document provides an introduction to Amorph Systems and their SMARTUNIFIER product. It introduces Frank Frauenhoffer as the Managing Director of Amorph Systems and discusses their key business domains of Airport Solutions and Industrial Solutions. It then focuses on describing SMARTUNIFIER, their integration platform that aims to provide unified IT integration across heterogeneous systems in a simple, fast and powerful way. It discusses challenges of current industry landscapes and how SMARTUNIFIER addresses interoperability needs. The document provides examples of how SMARTUNIFIER can be deployed and operated in different scenarios, as well as its application areas and use cases.
SplunkLive! Paris 2018: Splunk Overview
The document provides an overview of Splunk, including:
- Splunk allows users to search and analyze machine-generated data from websites, applications, sensors and other sources to gain operational intelligence and security insights.
- Splunk's platform can index and correlate data from various sources in real-time to enable log search, monitoring, and analytics across IT, security, and business functions.
- Splunk provides solutions for IT operations, security, IoT and industrial data, and business analytics to help customers address challenges in those areas.
The Risks and Rewards of AI
The document discusses how artificial intelligence and machine learning can help organizations address challenges around data and analytics. It notes that while CEOs expect AI to have a major impact, most organizations have not introduced AI initiatives. The document outlines different types of machine learning and provides examples of applications in various industries. It also discusses challenges with data quality and "dark data" that can limit the effectiveness of AI/ML. Finally, it presents how Splunk's AIops capabilities can help organizations leverage machine learning to drive business outcomes by gaining insights from infrastructure, applications and other digital systems data.
IoT Analytics @ splunk
During the presentation, forward-looking statements were made regarding Splunk's plans and estimates that are subject to risks and uncertainties. Any information about Splunk's roadmap outlines general product direction but is subject to change without notice. Splunk undertakes no obligation to develop or include any described feature in a future release. The presentation demonstrated Splunk's IoT analytics capabilities for manufacturing including predictive maintenance, advanced monitoring, and self-service analytics.
Catch these Sessions on-demand at .conf Online
This document lists session titles and codes for on-demand sessions covering various security topics from Splunk's .conf Online conference, including automated vulnerability detection, incident response, security orchestration, automation and response (SOAR), asset management, machine learning for detections, software supply chain security, zero trust architectures, security information and event management (SIEM), and tips for advancing skills in Splunk security.
Introduction into Security Analytics Methods
This document provides an overview and demo of Splunk Security Essentials. It begins with an introduction to the app and its capabilities for detecting threats both external and internal. It then demonstrates how to install and navigate the app to evaluate security use cases and review analytics methods. A scenario of a malicious insider exfiltrating data is presented and it shows how the app's searches could be used to detect anomalous activity related to Salesforce and Box downloads. The summary concludes by emphasizing how the app teaches detection use cases that can then be customized and integrated with Splunk's security products.
Splunk Artificial Intelligence & Machine Learning Webinar
These are the slides from the webinar broadcast on April 1st 2020, presented by Philipp Drieger. Content covers:
- Introduction to AI and ML Features in Splunk
- Customer Use Case Examples
- Live Demo of Machine Learning Toolkit, with examples for:
Methods for Anomaly Detection, Predictive Analytics and Forecasting, and Clustering
- Custom Machine Learning, incl.: Advanced Containerization and Expansion with MLSPL API
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
- The document is a presentation about turning data into business outcomes using the Splunk platform.
- It discusses how traditional approaches cannot keep up with digital evolution and increasing data volumes, while Splunk provides a holistic approach to ingest and analyze all types of machine data across multiple use cases.
- Splunk supports security, IT, business users, developers, and IoT use cases, and can help organizations progress from reactive to predictive and automated actions based on data insights.
Best Practices for Forwarder Hierarchies
Splunk is like an iceberg, on the surface we see the major components: indexers, search heads, license master, cluster master but under the water line we have a huge number of forwarders collecting and aggregating data streams. These forwarders are the foundations of any installation and configuration issues translate into problems with alerts, search performance, cluster stability and scaling out. This talk shows you to various ways to measure the efficiency of data collection and how to improve it. Prepare for lots of complex searches to identify common problems and charts that show good and bad. The talk aims to revolutionise how you think about forwarders and data collection in Splunk and turbo charge your platform performance and improve stability.
Observe 2020-d mc
Learn how RED monitoring impacts our view of observability, in any request-driven or microservices world.
From RED to deep-dices, the multi-dimensional nature of the observability problem both clarifies and leads to solution.
SplunkLive! Stockholm 2019 - Customer presentation: ISS
ISS is a global facility services company founded in 1901 with over 485,000 employees in 74 countries. It provides a wide range of services including cleaning, security, catering, and facility management. The company's strategy is to focus on large, global key accounts. Its Global Information Security team replaced a team of one to help enable compliance, secure the IT environment, and support business units using Splunk. Splunk provides centralized security information and event management, and is used by 90% of Fortune 100 companies. The team has established visibility, reduced threats, supported compliance, and started threat detection in five months of using Splunk Cloud.
Monitoring Modern Architectures with Data Science
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2A6lx09.
Dave Casper talks about how modern data science and algorithms are being applied to "fight machines with machines". Filmed at qconsf.com.
Dave Casper is CTO at Moogsoft. Previously, he served as Senior Vice President and Executive Director at several of the world's largest financial institutions, including UBS, Merrill Lynch, Bank of America, and Deutsche Bank and also co-chaired the Open Data Center Alliance Infrastructure Group.
Splunk for Monitoring and Diagnostics Breakout Session
Splunk for Monitoring and Diagnostics is a presentation about using Splunk software to gain real-time insights from industrial machine data. The document discusses how Splunk can be used to collect, index, enrich, search, analyze, and report on data from industrial IoT sensors, equipment, and systems. It provides examples of how Splunk has helped companies in oil/gas, manufacturing, and other industries improve operations, maintenance, safety and security by turning their machine data into business value. The presentation includes a demo of Splunk's capabilities for industrial use cases.
SplunkLive! Munich 2018: Getting Started with Splunk Enterprise
The document provides an agenda for a SplunkLive! presentation on installing and using Splunk. It includes downloading required files, importing sample data, conducting searches on the data, and exploring various Splunk features through a live demonstration. Common installation problems are also addressed. The presentation aims to provide attendees with the knowledge and skills to get started using Splunk through hands-on learning and a question and answer session.
Three Pillars, Zero Answers: Rethinking Observability
Observability has never been more important: the complexity of microservices makes it harder and harder to answer basic questions about system behavior. The conventional wisdom claims that Metrics, Logging and Tracing are “the three pillars” of observability… yet software organizations check these three boxes and are still grasping at straws during emergencies.
In this session, we’ll illustrate the problem with the three pillars: metrics, logs, and traces are just data – they are the fuel, not the car.
Three Pillars with Zero Answers: A New Observability Scorecard
Nobody denies the importance of observability in modern production software: with microservices adding scale, concurrency, and frequent deploys, it’s getting harder and harder to answer even basic questions about application behavior. The conventional wisdom has been that metrics, logging and tracing are “the three pillars” of observability, yet organizations check these boxes and still find themselves grasping at straws during emergencies. The problem is that metrics, logs, and traces are just data – if what we need is a car, all we’re talking about is the fuel. We will continue to disappoint ourselves until we reframe observability around two fundamental activities: (1) detection and
(2) refinement.
In this session, we’ll summarize the contemporary observability dogma, then present a new observability scorecard for objectively reasoning about and assessing observability solutions for modern distributed systems.
More Related Content
What's hot
Wie erkenne ich die Auswirkungen von IT Ausfallen auf meine Produktion?
This presentation discusses how Splunk software can help companies identify the impacts of IT outages on production. It provides examples of how Splunk has helped companies like Porsche transform digitally and implement AIOps. Key benefits highlighted include collecting all types of machine data, reducing alert noise through clustering and correlation, adding context through indicators and business metadata, performing anomaly detection and predictive analytics, and assisting with deep dive investigations.
Splunk Discovery Köln - 17-01-2020 - Splunk for ITOps
This document provides an agenda for a Splunk Discovery event taking place in Cologne, Germany on January 17th, 2020. The event will include a keynote and demo on Splunk's data platform in the morning. In the afternoon, there will be parallel tech and business tracks covering topics like security analytics, incident response, and the future of IT operations and monitoring. The document also includes disclaimers about forward-looking statements and that product roadmaps are subject to change.
Splunk Overview
The document appears to be a presentation by Splunk Inc. discussing their data platform. Some key points:
1. Splunk's platform allows customers to investigate, monitor, analyze and act on data from any source in real-time.
2. It addresses challenges of collecting and making sense of massive amounts of data from various systems and devices across IT, security, and IoT use cases.
3. Splunk provides solutions and services to help customers accelerate their data journey from initial investigation to taking action.
.conf21 - The Best of
This document provides an agenda and summaries of sessions from the Splunk .conf21 conference on DevOps and Observability topics. It introduces new features for Splunk APM like code-level profiling and database visibility. It also covers new mobile monitoring capabilities and an auto-detection feature for infrastructure monitoring. Recommended sessions are listed on monitoring environments, troubleshooting, and using observability data for AIOps processes. The document encourages attendees to access the online sessions on demand.
Security Automation & Orchestration
On your marks, get set GO!
Take a more in-depth look at the automation and orchestration journey and the future of SOAR.
Watch the SOCtails video here: https://www.youtube.com/watch?v=YzsGQzqaDYw&t=2s
Better Threat Analytics: From Getting Started to Cloud Security Analytics and...
Fighting the Eternal Challenge: Dealing with Alert Fatigue and Getting Insights into Security Productivity.
Lessons for a Fast Start in Automation and Orchestration.
Manufacturing Webinar AMS
The document provides an introduction to Amorph Systems and their SMARTUNIFIER product. It introduces Frank Frauenhoffer as the Managing Director of Amorph Systems and discusses their key business domains of Airport Solutions and Industrial Solutions. It then focuses on describing SMARTUNIFIER, their integration platform that aims to provide unified IT integration across heterogeneous systems in a simple, fast and powerful way. It discusses challenges of current industry landscapes and how SMARTUNIFIER addresses interoperability needs. The document provides examples of how SMARTUNIFIER can be deployed and operated in different scenarios, as well as its application areas and use cases.
SplunkLive! Paris 2018: Splunk Overview
The document provides an overview of Splunk, including:
- Splunk allows users to search and analyze machine-generated data from websites, applications, sensors and other sources to gain operational intelligence and security insights.
- Splunk's platform can index and correlate data from various sources in real-time to enable log search, monitoring, and analytics across IT, security, and business functions.
- Splunk provides solutions for IT operations, security, IoT and industrial data, and business analytics to help customers address challenges in those areas.
The Risks and Rewards of AI
The document discusses how artificial intelligence and machine learning can help organizations address challenges around data and analytics. It notes that while CEOs expect AI to have a major impact, most organizations have not introduced AI initiatives. The document outlines different types of machine learning and provides examples of applications in various industries. It also discusses challenges with data quality and "dark data" that can limit the effectiveness of AI/ML. Finally, it presents how Splunk's AIops capabilities can help organizations leverage machine learning to drive business outcomes by gaining insights from infrastructure, applications and other digital systems data.
IoT Analytics @ splunk
During the presentation, forward-looking statements were made regarding Splunk's plans and estimates that are subject to risks and uncertainties. Any information about Splunk's roadmap outlines general product direction but is subject to change without notice. Splunk undertakes no obligation to develop or include any described feature in a future release. The presentation demonstrated Splunk's IoT analytics capabilities for manufacturing including predictive maintenance, advanced monitoring, and self-service analytics.
Catch these Sessions on-demand at .conf Online
This document lists session titles and codes for on-demand sessions covering various security topics from Splunk's .conf Online conference, including automated vulnerability detection, incident response, security orchestration, automation and response (SOAR), asset management, machine learning for detections, software supply chain security, zero trust architectures, security information and event management (SIEM), and tips for advancing skills in Splunk security.
Introduction into Security Analytics Methods
This document provides an overview and demo of Splunk Security Essentials. It begins with an introduction to the app and its capabilities for detecting threats both external and internal. It then demonstrates how to install and navigate the app to evaluate security use cases and review analytics methods. A scenario of a malicious insider exfiltrating data is presented and it shows how the app's searches could be used to detect anomalous activity related to Salesforce and Box downloads. The summary concludes by emphasizing how the app teaches detection use cases that can then be customized and integrated with Splunk's security products.
Splunk Artificial Intelligence & Machine Learning Webinar
These are the slides from the webinar broadcast on April 1st 2020, presented by Philipp Drieger. Content covers:
- Introduction to AI and ML Features in Splunk
- Customer Use Case Examples
- Live Demo of Machine Learning Toolkit, with examples for:
Methods for Anomaly Detection, Predictive Analytics and Forecasting, and Clustering
- Custom Machine Learning, incl.: Advanced Containerization and Expansion with MLSPL API
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
- The document is a presentation about turning data into business outcomes using the Splunk platform.
- It discusses how traditional approaches cannot keep up with digital evolution and increasing data volumes, while Splunk provides a holistic approach to ingest and analyze all types of machine data across multiple use cases.
- Splunk supports security, IT, business users, developers, and IoT use cases, and can help organizations progress from reactive to predictive and automated actions based on data insights.
Best Practices for Forwarder Hierarchies
Splunk is like an iceberg, on the surface we see the major components: indexers, search heads, license master, cluster master but under the water line we have a huge number of forwarders collecting and aggregating data streams. These forwarders are the foundations of any installation and configuration issues translate into problems with alerts, search performance, cluster stability and scaling out. This talk shows you to various ways to measure the efficiency of data collection and how to improve it. Prepare for lots of complex searches to identify common problems and charts that show good and bad. The talk aims to revolutionise how you think about forwarders and data collection in Splunk and turbo charge your platform performance and improve stability.
Observe 2020-d mc
Learn how RED monitoring impacts our view of observability, in any request-driven or microservices world.
From RED to deep-dices, the multi-dimensional nature of the observability problem both clarifies and leads to solution.
SplunkLive! Stockholm 2019 - Customer presentation: ISS
ISS is a global facility services company founded in 1901 with over 485,000 employees in 74 countries. It provides a wide range of services including cleaning, security, catering, and facility management. The company's strategy is to focus on large, global key accounts. Its Global Information Security team replaced a team of one to help enable compliance, secure the IT environment, and support business units using Splunk. Splunk provides centralized security information and event management, and is used by 90% of Fortune 100 companies. The team has established visibility, reduced threats, supported compliance, and started threat detection in five months of using Splunk Cloud.
Monitoring Modern Architectures with Data Science
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2A6lx09.
Dave Casper talks about how modern data science and algorithms are being applied to "fight machines with machines". Filmed at qconsf.com.
Dave Casper is CTO at Moogsoft. Previously, he served as Senior Vice President and Executive Director at several of the world's largest financial institutions, including UBS, Merrill Lynch, Bank of America, and Deutsche Bank and also co-chaired the Open Data Center Alliance Infrastructure Group.
Splunk for Monitoring and Diagnostics Breakout Session
Splunk for Monitoring and Diagnostics is a presentation about using Splunk software to gain real-time insights from industrial machine data. The document discusses how Splunk can be used to collect, index, enrich, search, analyze, and report on data from industrial IoT sensors, equipment, and systems. It provides examples of how Splunk has helped companies in oil/gas, manufacturing, and other industries improve operations, maintenance, safety and security by turning their machine data into business value. The presentation includes a demo of Splunk's capabilities for industrial use cases.
SplunkLive! Munich 2018: Getting Started with Splunk Enterprise
The document provides an agenda for a SplunkLive! presentation on installing and using Splunk. It includes downloading required files, importing sample data, conducting searches on the data, and exploring various Splunk features through a live demonstration. Common installation problems are also addressed. The presentation aims to provide attendees with the knowledge and skills to get started using Splunk through hands-on learning and a question and answer session.
What's hot (20)
Wie erkenne ich die Auswirkungen von IT Ausfallen auf meine Produktion?
Wie erkenne ich die Auswirkungen von IT Ausfallen auf meine Produktion?
Splunk Discovery Köln - 17-01-2020 - Splunk for ITOps
Splunk Discovery Köln - 17-01-2020 - Splunk for ITOps
Better Threat Analytics: From Getting Started to Cloud Security Analytics and...
Better Threat Analytics: From Getting Started to Cloud Security Analytics and...
Splunk Artificial Intelligence & Machine Learning Webinar
Splunk Artificial Intelligence & Machine Learning Webinar
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
SplunkLive! Stockholm 2019 - Customer presentation: ISS
SplunkLive! Stockholm 2019 - Customer presentation: ISS
Splunk for Monitoring and Diagnostics Breakout Session
Splunk for Monitoring and Diagnostics Breakout Session
SplunkLive! Munich 2018: Getting Started with Splunk Enterprise
SplunkLive! Munich 2018: Getting Started with Splunk Enterprise
Similar to Three Pillars, No Answers: Helping Platform Teams Solve Real Observability Problems
Three Pillars, Zero Answers: Rethinking Observability
Observability has never been more important: the complexity of microservices makes it harder and harder to answer basic questions about system behavior. The conventional wisdom claims that Metrics, Logging and Tracing are “the three pillars” of observability… yet software organizations check these three boxes and are still grasping at straws during emergencies.
In this session, we’ll illustrate the problem with the three pillars: metrics, logs, and traces are just data – they are the fuel, not the car.
Three Pillars with Zero Answers: A New Observability Scorecard
Nobody denies the importance of observability in modern production software: with microservices adding scale, concurrency, and frequent deploys, it’s getting harder and harder to answer even basic questions about application behavior. The conventional wisdom has been that metrics, logging and tracing are “the three pillars” of observability, yet organizations check these boxes and still find themselves grasping at straws during emergencies. The problem is that metrics, logs, and traces are just data – if what we need is a car, all we’re talking about is the fuel. We will continue to disappoint ourselves until we reframe observability around two fundamental activities: (1) detection and
(2) refinement.
In this session, we’ll summarize the contemporary observability dogma, then present a new observability scorecard for objectively reasoning about and assessing observability solutions for modern distributed systems.
Mantis: Netflix's Event Stream Processing System
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1yyaHb8.
The authors discuss Netflix's new stream processing system that supports a reactive programming model, allows auto scaling, and is capable of processing millions of messages per second. Filmed at qconsf.com.
Danny Yuan is an architect and software developer in Netflix’s Platform Engineering team. Justin Becker is Senior Software Engineer at Netflix.
High Availability HPC ~ Microservice Architectures for Supercomputing
In this deck from the Stanford HPC Conference, Ryan Quick from Providentia Worldwide presents: High Availability HPC ~ Microservice Architectures for Supercomputing.
"Microservices power cloud-native applications to scale thousands of times larger than single deployments. We introduce the notion of microservices for traditional HPC workloads. We will describe microservices generally, highlighting some of the more popular and large-scale applications. Then we examine similarities between large-scale cloud configurations and HPC environments. Finally we propose a microservice application for solving a traditional HPC problem, illustrating improved time-to-market and workload resiliency."
Watch the video: https://insidehpc.com/2018/02/high-availability-hpc-microservice-architectures-supercomputing/
Learn more: http://www.providentiaworldwide.com/
and
http://hpcadvisorycouncil.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Big Data : Bits of History, Words of Advice
Big Data: Bits of History, Words of Advice by Venu Vasudevan discusses the history and challenges of big data. The document provides examples from Iridium satellite communications and an industrial IoT project to illustrate issues with fast, unpredictable data streams and the challenges of integrating IT and OT systems. It advocates for hybrid human-machine approaches and emphasizes starting simply before over-engineering architectures to fit extreme needs. The key lessons are to focus on the core problem, allow for data variability, and consider lightweight data augmentation when starting with limited real-world datasets.
Is this normal?
There are many modern techniques for identifying anomalies in datasets. There are fewer that work as online algorithms suitable for application to real-time streaming data. What’s worse? Most of these methodologies require a deep understanding of the data itself. In this talk, we tour what the options are for identifying anomalies in real-time data and discuss how much we really need to know before hand to guess at the ever-useful question: is this normal?
Real-Time Analytics With StarRocks (DWH+DL).pdf
StarRocks, an open-source distributed columnar engine for real-time analytics, is carving its niche in the big data landscape. Its ability to handle high-velocity data streams and deliver blazing-fast query responses makes it a compelling choice for modern analytics workloads. Let's delve into the intricacies of real-time analytics with StarRocks and explore its capabilities.
Data Ingestion:
The journey begins with ingesting data into StarRocks. It supports a variety of real-time data sources, including Kafka, Pulsar, and custom streaming protocols. These integrations allow seamless data flow from streaming sources to StarRocks, ensuring minimal latency.
Stream Processing and Storage:
StarRocks employs a hybrid architecture for real-time processing. Incoming data streams are first processed by lightweight stream engines like Flink or Spark. These engines perform initial aggregations and transformations, preparing the data for efficient storage in StarRocks' columnar format. This format facilitates rapid data retrieval and filtering, crucial for real-time querying.
Real-time Querying:
The true power of StarRocks lies in its real-time query engine. Once data lands in StarRocks, users can leverage SQL-like queries to analyze it with minimal lag. StarRocks optimizes queries by exploiting its columnar storage and parallel processing capabilities. This enables sub-second response times for even complex queries, empowering users to gain immediate insights from their data.
Advanced Features:
StarRocks packs several features that further enhance its real-time analytics prowess. Materialized views act as pre-computed summaries of data, accelerating frequently used queries. Additionally, StarRocks' automatic tiered storage seamlessly migrates less frequently accessed data to cost-effective storage solutions, optimizing resource utilization.
Intelligent Monitoring
This presentation describes a intelligent IT monitoring solution that uses Nagios as source of information, Esper as the CEP engine and a PCA algorithm.
Machine Learning Impact on IoT - Part 2
How to Introduce Machine Learning in a IoT scenario:
- Map of ML algorithms and uses cases
- Case Study: Jet engines predictive maintenance.
Data Platform at Twitter: Enabling Real-time & Batch Analytics at Scale
The Data Platform at Twitter supports engineers and data scientists running batch jobs on Hadoop clusters that are several 1000s of nodes, and real-time jobs on top of systems such as Storm. In this presentation, I discuss the overall Data Platform stack at Twitter. In particular, I talk about enabling real-time and batch analytics at scale with the help of Scalding, which is a Scala DSL for batch jobs using MapReduce, Summingbird, which is a framework for combined real-time and batch processing, and Tsar, which is a framework for real-time time-series aggregations.
Data Science Challenge presentation given to the CinBITools Meetup Group
The document describes the Cloudera Data Science Challenge, which involves solving three data science problems using large datasets. For the first problem, Smartfly, the goal is to predict flight delays using historical flight data and machine learning algorithms like logistic regression and SVM. The second problem, Almost Famous, involves statistical analysis of web log data and filtering for spam. The third problem, Winklr, requires social network analysis to recommend users to follow on a social media platform based on click data. The document discusses the approaches, tools, and algorithms used to solve each problem at scale using Apache Spark and Hadoop technologies.
Cloudera Data Science Challenge
The document describes the Cloudera Data Science Challenge, which involves solving three data science problems using large datasets. For the first problem, Smartfly, the goal is to predict flight delays using historical flight data and machine learning algorithms like logistic regression and SVM. The second problem, Almost Famous, involves statistical analysis of web log data and filtering for spam. The third problem, Winklr, requires analyzing a social network graph to recommend users to follow. The document discusses the approaches, tools, and results for each problem.
How to not fail at security data analytics (by CxOSidekick)
1. The document discusses the challenges of obtaining security-related data from different sources and transporting it to a central platform for analysis. It addresses questions about data volume, collection methods, filtering and formatting.
2. Setting up a security data pipeline involves determining what data to collect from various host systems, networks, and applications. Data must then be forwarded from collectors to a central platform while managing bandwidth, latency, and failures.
3. Collecting the right security-related data is vital for detecting threats and being able to investigate incidents. The document argues for collecting most available data by default and filtering out exceptions, rather than only collecting predefined types of data.
QConSF 2014 talk on Netflix Mantis, a stream processing system
Justin and I gave this talk in QCon SF 2014 about the Mantis, a stream processing system that features a reactive programming API, auto scaling, and stream locality
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
At Netflix, we have traditionally approached cloud efficiency from a human standpoint, whether it be in-person meetings with the largest service teams or manually flipping reservations. Over time, we realized that these manual processes are not scalable as the business continues to grow. Therefore, in the past year, we have focused on building out tools that allow us to make more insightful, data-driven decisions around capacity and efficiency. In this session, we discuss the DIY applications, dashboards, and processes we built to help with capacity and efficiency. We start at the ten thousand foot view to understand the unique business and cloud problems that drove us to create these products, and discuss implementation details, including the challenges encountered along the way. Tools discussed include Picsou, the successor to our AWS billing file cost analyzer; Libra, an easy-to-use reservation conversion application; and cost and efficiency dashboards that relay useful financial context to 50+ engineering teams and managers.
Cloudera Movies Data Science Project On Big Data
The document describes a data science project conducted on streaming log data from Cloudera Movies, an online streaming video service. The goals of the project were to understand which user accounts are used most by younger viewers, segment user sessions to improve site usability, and build a recommendation engine. Key steps included exploring and cleaning the data, classifying users as children or adults using a SimRank approach, clustering user sessions to identify behavior patterns, and predicting user ratings through user-user and item-item similarity models to build a recommendation system. Accuracy of 99.64% was achieved in classifying users.
No specimen (software) left behind
A short talk at the iEvoBio (Informatics for Phylogenetics, Evolution, and Biodiversity) conference at the The University of Oklahoma, Embassy Suites Hotel and Conference Center, Norman, Oklahoma, USA. June 21-22, 2011.
Build User-Facing Analytics Application That Scales Using StarRocks (DLH).pdf
This article isn't just about building an application, it's about empowering your users to unleash the full potential of their data. We'll delve into the world of StarRocks, a cutting-edge engine for real-time analytics, and guide you through the process of constructing user-facing applications that deliver fast, insightful, and visually captivating experiences.
Dive into the Why:
First, we'll explore the compelling reasons to choose StarRocks. We'll demystify its ability to handle high-velocity data streams and deliver sub-second query responses, making it ideal for applications where real-time insights are king.
Laying the Foundation:
Next, we'll embark on a step-by-step journey through the application development process. We'll discuss:
Data Ingestion: Learn how to seamlessly integrate real-time data sources like Kafka and Pulsar into your application, ensuring a constant flow of fresh insights.
Building the Core: We'll explore various options for structuring your application's backend, showcasing the pros and cons of different frameworks and libraries.
Querying Made Easy: Delve into StarRocks' intuitive SQL interface and discover how to craft powerful queries that unlock the treasure trove of data hiding within.
Visualization Wonderland: Explore the vast landscape of data visualization tools compatible with StarRocks. We'll showcase ways to transform complex data into interactive dashboards, compelling charts, and captivating reports that resonate with your users.
Beyond the Basics:
We'll push the boundaries by exploring advanced features like:
Materialized Views: Discover how to pre-compute data summaries for even faster querying.
Security and Access Control: Learn how to implement robust security measures to protect sensitive data while granting appropriate access levels to different user groups.
Customization and Scalability: Explore options for tailoring the application to your specific needs and ensuring it can gracefully handle growing data volumes and user traffic.
Conclusion:
By the end of this article, you'll be equipped with the knowledge and tools to build user-facing analytics applications that empower your users to harness the power of real-time data and make confident, data-driven decisions. This journey isn't just about building an application, it's about transforming how your users interact with data, unlocking a world of actionable insights and limitless possibilities.
So, are you ready to conquer complexity and unleash the power of StarRocks? Dive into this article and start building your next game-changing user-facing analytics application!
Spark Summit EU 2016: The Next AMPLab: Real-time Intelligent Secure Execution
Committed to the goal of building open-source frameworks, tools, and algorithms that make building real-time applications decisions on live data with stronger security, The RISELab is set to innovate and enhance Spark
"Traffic Speed Control System in the Cloud using Machine Learning" by Albert ...
This talk will be about difficulties they have met on the road, and how they overcame some of them. It will reveal both - technical and management side.
Alberts Bušinskis (Garuda Dev Giri) - Yoga & Meditation teacher, IT enthusiast, software developer. Many years ago he had left a very good job in Switzerland and went to Indian Himalayas to practice Yoga. After he realized that Software Development is not different from Yoga, he came back to homeland trying to be helpful and inspiring.
Similar to Three Pillars, No Answers: Helping Platform Teams Solve Real Observability Problems (20)
Three Pillars, Zero Answers: Rethinking Observability
Three Pillars, Zero Answers: Rethinking Observability
Three Pillars with Zero Answers: A New Observability Scorecard
Three Pillars with Zero Answers: A New Observability Scorecard
High Availability HPC ~ Microservice Architectures for Supercomputing
High Availability HPC ~ Microservice Architectures for Supercomputing
Data Platform at Twitter: Enabling Real-time & Batch Analytics at Scale
Data Platform at Twitter: Enabling Real-time & Batch Analytics at Scale
Data Science Challenge presentation given to the CinBITools Meetup Group
Data Science Challenge presentation given to the CinBITools Meetup Group
How to not fail at security data analytics (by CxOSidekick)
How to not fail at security data analytics (by CxOSidekick)
QConSF 2014 talk on Netflix Mantis, a stream processing system
QConSF 2014 talk on Netflix Mantis, a stream processing system
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Build User-Facing Analytics Application That Scales Using StarRocks (DLH).pdf
Build User-Facing Analytics Application That Scales Using StarRocks (DLH).pdf
Spark Summit EU 2016: The Next AMPLab: Real-time Intelligent Secure Execution
Spark Summit EU 2016: The Next AMPLab: Real-time Intelligent Secure Execution
"Traffic Speed Control System in the Cloud using Machine Learning" by Albert ...
"Traffic Speed Control System in the Cloud using Machine Learning" by Albert ...
More from DevOps.com
Modernizing on IBM Z Made Easier With Open Source Software
In the past decade, IDC has seen IBM Z evolve first from a siloed platform to what they call a "connected" platform, and then to a "transformative" platform. This transition has been driven by IBM, by the IBM Z software vendors, like Rocket Software, and by businesses themselves.
IDC research shows that businesses that choose to modernize IBM Z achieve higher satisfaction than re-platformers and many are using open source software (OSS) in their modernization initiatives. Employing OSS makes it possible to crack the platform open and enable it to connect to the rest of the datacenter and the outside world. Join IDC guest speaker, Al Gillen and Peter Fandel as they take a deeper look at the value proposition associated with using commercially supported OSS in mission-critical environments, like IBM Z. In this webinar we’ll discuss:
How OSS can neutralize the disparity between seasoned IBM Z and emerging developers
The modernization initiatives that involve OSS
What to consider before bringing OSS to IBM Z
How Rocket Software is delivering commercially supported OSS to IBM Z
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
With the growing adoption of Kubernetes, organizations want to take advantage of containerized Microsoft SQL Server 2019 to optimize transactional performance and accelerate time-to-insights from their business-critical data. However, as enterprises embrace hybrid cloud strategy, they need to consider several aspects based on the performance, cost and data protection requirements for running enterprise-grade SQL Server databases.
In this webinar, we will compare and contrast various cloud-native platforms for SQL Server that would help CIOs, DevOps engineers, database administrators and applications architects to determine the most suitable platform that fits their business needs.
Join us as we explore some exciting results from a recent performance benchmark study conducted by McKnight Consulting Group, an independent consulting firm, to compare the performance of Microsoft SQL Server 2019 on the best possible configurations of the following Kubernetes platforms:
Diamanti Enterprise Kubernetes Platform
Amazon Web Services Elastic Kubernetes Service (AWS EKS)
Azure Kubernetes Service (AKS)
Topics will include:
Platform considerations and requirements for running Microsoft SQL Server 2019
Performance comparison and analysis of running SQL Server on various platform
Best practices for running containerized SQL Server databases in Kubernetes environment
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
With the growing adoption of Kubernetes, organizations want to take advantage of containerized Microsoft SQL Server 2019 to optimize transactional performance and accelerate time-to-insights from their business-critical data. However, as enterprises embrace hybrid cloud strategy, they need to consider several aspects based on the performance, cost and data protection requirements for running enterprise-grade SQL Server databases.
In this webinar, we will compare and contrast various cloud-native platforms for SQL Server that would help CIOs, DevOps engineers, database administrators and applications architects to determine the most suitable platform that fits their business needs.
Join us as we explore some exciting results from a recent performance benchmark study conducted by McKnight Consulting Group, an independent consulting firm, to compare the performance of Microsoft SQL Server 2019 on the best possible configurations of the following Kubernetes platforms:
Diamanti Enterprise Kubernetes Platform
Amazon Web Services Elastic Kubernetes Service (AWS EKS)
Azure Kubernetes Service (AKS)
Topics will include:
Platform considerations and requirements for running Microsoft SQL Server 2019
Performance comparison and analysis of running SQL Server on various platform
Best practices for running containerized SQL Server databases in Kubernetes environment
Next Generation Vulnerability Assessment Using Datadog and Snyk
Vulnerability assessment for teams can often be overwhelming. The dependency graph could be thousands of packages depending on the application. Triaging vulnerability data and prioritizing actions has historically been a very manual process, until now. With Datadog and Snyk, learn how to trace security and performance issues by leveraging continuous profiling capabilities for actionable insight that help developers remediate problems.
Join us on Thursday, January 21 for a unique opportunity to learn more about continuous profiling, vulnerability management, and the benefit to customers from using both of these products. In this webinar, you will:
Bust some myths around continuous profiling and learn how Datadog differentiates itself
See decorated traces in action for sample Java applications and understand how Snyk + Datadog reduce time to triage supply chain vulnerabilities
Learn roadmap information for upcoming public announcements from both partners
Vulnerability Discovery in the Cloud
In the era of cloud generation, the constant activity around workloads and containers create more vulnerabilities than an organization can keep up with. Using legacy security vendors doesn't set you up for success in the cloud. You’re likely spending undue hours chasing, triaging and patching a countless stream of cloud vulnerabilities with little prioritization.
Join us for this live webinar as we detail how to streamline host and container vulnerability workflows for your software teams wanting to build fast in the cloud. We'll be covering how to:
Get visibility into active packages and associated vulnerabilities
Reduce false positives by 98%
Reduce investigation time by 30%
Spot a legacy vendor looking to do some cloud washing
2021 Open Source Governance: Top Ten Trends and Predictions
If you work in software development, jumpstart your engineering team in 2021—get ahead of the engineering curve and your competitors—by attending this must-watch open source trends and predictions webinar.
Alex Rybak, Director of Product Management at Revenera, and Russ Eling, founder and CEO of OSS Engineering Consultants, share their top 10 open source usage, license compliance and security insights for the new year.
Just a few hints at what you’ll learn more about:
Where the adoption of shift-left is headed and the decisions you’ll face going forward
The impact of a lack of software developer security training relative to pandemic fallout
The broader role of the engineering team in open source management and governance
The expanding role and impact of open source marketplaces such as GitHub
Don’t miss the discussion for valuable insight and learning for software engineering teams
A New Year’s Ransomware Resolution
2020 was a brutal year for ransomware. Cybercriminals operated without any human decency, targeting the most vulnerable and at-risk parties, such as hospitals, scientists, and global manufacturers. The approach has become more sophisticated and life-threatening, shifting from individual targets to global enterprises, destroying backups, blackmailing victims with public leakage of exfiltrated data, and paralyzing critical systems and infrastructure.
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
This document discusses runtime security on Azure Kubernetes Service (AKS). It begins by introducing AKS and how it simplifies Kubernetes deployment and management. It then discusses the security concerns with containers and the need for runtime security. Runtime security involves monitoring activity within containers to detect unwanted behaviors. The document outlines how Sysdig provides runtime security for AKS through its agents that collect syscall data and Kubernetes audit logs. It analyzes this data using policies to detect anomalies and threats across containers, hosts, and Kubernetes clusters. Sysdig also integrates with other tools like Falco and Anchore to provide breadth and depth of security.
Don't Panic! Effective Incident Response
In any fast-paced engineering environment, unexpected incidents can arise and escalate without warning. Without strong leadership within teams, you get chaotic, stressful, and tiring situations that waste valuable engineering time, slow down resolution, and most importantly, impact your customers.
Operationally mature organisations use proven incident response systems led by Incident Commanders. Incident Commanders provide the leadership needed to help stabilize major incidents fast.
In this webinar, we’ll take lessons learned from formalized incident response, such as those used by first responders, and show you how to apply those same practices to your organization. By utilising these methods you’ll improve both the speed and effectiveness of your team’s response, reducing the amount of downtime experienced.
In this workshop, attendees will:
Be introduced to the Incident Command System and learn how it can be adapted to their organisation
Walk through the basics of incident response best practices
Discuss examples of formal incident response from multiple organisations
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Chaos engineering is becoming a critical part of the DevOps toolchain when adopting Site Reliability Engineering (SRE) practices. Every system is becoming a distributed system and chaos engineering proclaims many advantages for them.
It improves infrastructure automation, increases reliability and transforms incident management. However, an often-overlooked benefit of chaos engineering and SRE involves culture transformation. Culture is often touched upon when talking about chaos engineering and SRE but not as often as skills and process.
In this webinar, we will discuss how you can build out a chaos engineering practice and how you can adopt a true blameless culture and maximize the potential of your team.
You will learn how to:
Hold blameless postmortems
Share post mortems with other teams
Run regular fire drills and game days
Automate chaos experiments for continuous validation
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Enterprises are best served by leveraging an RBAC system to manage access to their SSH and Kubernetes resources. With Teleport, an open source software, employers are able to provide granular access controls to developers based on the access they need and when they need it. This makes it possible for employers to maintain secure access without getting in the way of their developers’ daily operations.
Join Steven Martin, solution engineer at Teleport, as he demonstrates how to assign access to developers and SRE’s across environments with Teleport through roles mapped from enterprises’ identity providers or SSOs.
Monitoring Serverless Applications with Datadog
Join Datadog for a webinar on monitoring serverless applications with AWS Lambda. You'll learn how to get the most of Datadog's platform, as well ask the following key takeaways:
Learn how to set up a Twitter bot that makes API calls with Node.js
Deploying Serverless Applications
What does observability look like with less infrastructure?
Deliver your App Anywhere … Publicly or Privately
Developers are increasingly adopting a microservices approach for their apps in order to gain rapid iteration capabilities required for delivering new services faster. However, delivering the App still requires multiple steps such as allocation of virtual IPs, provisioning the front load balancer, configuring firewall rules, configuring a public domain, and DDOS. At present, each of these steps requires coordination across multiple teams with multiple iterations per team. The time efficiencies gained by adopting microservices and cloud-native technologies is negated due to the time taken to deliver the App.
In this session, Pranav Dharwadkar, VP of products at Volterra, and Jakub Pavlik, director of engineering, will help you understand these challenges and introduce a distributed proxy architecture that can alleviate the challenges across different cloud environments. This webinar will include a live demo using a distributed proxy architecture to advertise an App publicly and privately.
In this webinar, you will learn:
The steps required to deliver an App using the current approaches
How a distributed proxy architecture can be used to deliver the app publicly and privately
The operational benefits of a distributed proxy architecture for delivering new services
Securing medical apps in the age of covid final
The COVID-19 pandemic has drastically altered the connected healthcare landscape, accelerating the usage of telemedicine and other remote healthcare delivery systems by as much as 11,000% for some populations. How has this unprecedented push affected healthcare and medical device application security? The security team at Intertrust recently analyzed 100 Android and iOS medical apps to find out.
In this webinar, we'll discuss:
Medical application and device threat trends
The top mHealth security vulnerabilities uncovered in our analysis
Strategies to keep your mHealth apps safe
Future advances in digital healthcare and how your security can evolve with it
How to Build a Healthy On-Call Culture
Raise your hand if you enjoy being buried in alerts or woken up at 2 a.m. — yeah … thought so. Ever-rising customer expectations around high availability and performance put massive pressure on the teams who develop and support SaaS products. And teams are literally losing sleep over it. Until outages and other incidents are a thing of the past, organizations need to invest in a way of dealing with them that won’t lead to burn-out.
In this session, you’ll learn how to combine the latest tooling with DevOps practices in the pursuit of a sustainable incident response workflow. It’s all about transparency, actionable alerts, resilience and learning from each incident.
The Evolving Role of the Developer in 2021
The role of the developer continues to change as they sit on the front line of application and even cloud infrastructure security. Today, developers are focused on innovating fast and improving security, but how do high-performing teams accomplish this? They commit code frequently, release often and update dependencies regularly (608x faster than others).
In this webinar, we'll discuss the key traits of high-performing teams and how that impacts the role of the developer.
Key Takeaways:
Choose the best third party dependencies
Determine the lowest effort upgrades between open source versions
Solve for issues in both direct and transitive dependencies with a single-click
Block and quarantine suspicious open source components
Service Mesh: Two Big Words But Do You Need It?
Today, one of the big concepts buzzing in the app development world is service mesh. A service mesh is a configurable infrastructure layer for microservices application that makes communication flexible, reliable and fast. Let’s take a step back, though, and answer this question: Do you need a service mesh?
Join this webinar to learn:
What a service mesh is; when and why you need it — or when and why you may not
App modernization journey and traffic management approaches for microservices-based apps
How to make an informed decision based on cost and complexity before adopting service mesh
Learn about NGINX Service Mesh in a live demo, and how it provides the best service mesh option for container-based L7 traffic management
Secure Data Sharing in OpenShift Environments
Red Hat OpenShift is enabling quicker adoption of DevOps practices. Containers are an essential component of DevOps and the OpenShift Kubernetes Container Platform is integral for orchestration within these environments. Data security is now challenged to keep pace with the size and scope of container usage. The migration from legacy in-house deployments to hybrid-cloud installations has created new attack surfaces as data is shared more freely in Kubernetes deployments.
Protecting data at rest and in motions is a necessity. Learn how you can keep data protected and securely share data in OpenShift environments with real-time data protection solutions.
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
Managing access permissions in the public cloud can be a very complex process. In fact, by 2023, 75% of cloud security failures will result from the inadequate management of identities, access and privileges, according to Gartner.
Join us as Guy Flechter, CISO of AppsFlyer, presents a real-world case of how his company works to enforce least-privilege and to govern identities in their cloud. This webinar will also provide an overview of how to govern access and achieve least privilege by analyzing the access permissions and activity in your public cloud environment. With thousands of human and machine identities, roles, policies and entitlements, this webinar will give you the tools to examine the access open to people and services in your public cloud, and determine whether that access is necessary.
In this workshop, you will learn about:
The risks of IAM misconfiguration and excessive entitlements in cloud environments
The challenges in identifying and mitigating Identity and access risks for both human and machine identities
How to automate cloud identity governance and entitlement management with Ermetic
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Open-source machine learning can be transformative, but without the proper tools in place, enterprises struggle to balance the IT security and governance requirements with the need to deliver these powerpoint tools into the hands of their developers and modelers.
How can organizations get the latest technology from the open-source brain trust, while ensuring enterprise-grade management and security? In this webinar, we will discuss how Anaconda Team Edition, available on RedHat Marketplace, enables IT departments to mirror a curated set of packages into their organization in a safe and governed way.
Join Michael Grant, VP of services at Anaconda, to discuss:
How IT organizations are using Anaconda Team Edition to curate, govern and secure Python and R packages
Tips for how development and data science teams can get the most out of Team Edition, from uploading your own packages to building custom channels for groups or projects
How to distribute conda environments to desktops, servers and clusters:
GUI-based installers for desktop users
“Conda packs” for automated delivery to remote servers and distributed computing clusters
Conda-enabled Docker containers for application deployment
More from DevOps.com (20)
Modernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source Software
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and Snyk
2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Recently uploaded
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Mutation Testing for Task-Oriented Chatbots
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays, June 2024 with Maria Copot 20
Skybuffer SAM4U tool for SAP license adoption
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
AppSec PNW: Android and iOS Application Security with MobSF
Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application.
This talk covers:
Using MobSF for static analysis of mobile applications.
Interactive dynamic security assessment of Android and iOS applications.
Solving Mobile app CTF challenges.
Reverse engineering and runtime analysis of Mobile malware.
How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
5th LF Energy Power Grid Model Meet-up Slides
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Dandelion Hashtable: beyond billion requests per second on a commodity server
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Astute Business Solutions | Oracle Cloud Partner |
Your goto partner for Oracle Cloud, PeopleSoft, E-Business Suite, and Ellucian Banner. We are a firm specialized in managed services and consulting.
JavaLand 2024: Application Development Green Masterplan
My presentation slides I used at JavaLand 2024
Recently uploaded (20)
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Three Pillars, No Answers: Helping Platform Teams Solve Real Observability Problems
- 1. Three Pillars, No Answers: Helping Platform Teams Solve Real Observability Problems Austin Parker, Principal Developer Advocate at Lightstep
- 2. Who Am I? Austin Parker Principal Developer Advocate @austinlparker austin@lightstep.com✉
- 4. The Conventional Wisdom ● Observing microservices is hard ● Google and Facebook solved this (right???) ● They used Metrics, Logging, and Distributed Tracing… ● So we should, too.
- 5. The Three Pillars of Observability - Metrics - Logging - Distributed Tracing
- 6. Metrics
- 7. Logging
- 8. Tracing
- 10. Fatal Flaws
- 11. A word nobody knew in 2015… Dimensions (aka “tags”) can explain variance in timeseries data (aka “metrics”) …… but cardinality
- 12. Logging Data Volume: a reality check transaction rate x all microservices x cost of net+storage x weeks of retention ----------------------- way too much $$$$
- 13. The Life of Transaction Data: Dapper Stage Overhead affects… Retained Instrumentation Executed App 100.00% Buffered within app process App 000.10% Flushed out of process App 000.10% Centralized regionally Regional network + storage 000.10% Centralized globally WAN + storage 000.01%
- 14. Fatal Flaws: A Review Logs Metrics Dist. Traces TCO scales gracefully – ✓ ✓ Accounts for all data (i.e., unsampled) ✓ ✓ – Immune to cardinality ✓ – ✓
- 16. Data vs UI
- 17. Data vs UI
- 19. Metrics, Logs, and Traces are Just Data, … not a feature or use case.
- 20. Part 2: A New Scorecard for Observability
- 21. Mental Model: Goals and Activities ● Goals: how our services perform in the eyes of their consumers ● Activities: what we (as operators) actually do to further our goals
- 22. Quick Vocab Refresher: SLIs “SLI” = “Service Level Indicator” TL;DR: An SLI is an indicator of health that a service’s consumers would care about. … not an indicator of its inner workings
- 23. Observability: 2 Fundamental Goals Gradually improving an SLI Rapidly restoring an SLI Reminder: “SLI” = “Service Level Indicator” NOW!!!! days, weeks, months…
- 24. Observability: 2 Fundamental Activities 1. Detection: measuring SLIs precisely 2. Refinement: reducing the search space for plausible explanations
- 25. An interlude about stats frequency
- 26. Scorecard: Detection 1. Specificity: - Cost of cardinality ($ per tag value) - Stack support (mobile/web platforms, managed services, “black-box OSS infra” like Kafka/Cassandra) 2. Fidelity: - Correct stats!!! (global p95, p99) - High stats frequency (stats sampling frequency, in seconds) 3. Freshness (lag from real-time, in seconds)
- 27. Why “Refinement”? # of things your users actually care about # of microservices # of failure modes Must reduce the search space!
- 28. The Refinement Process Discover Variance Explain Variance Deploy Fix
- 30. Scorecard: Refinement Identifying Variance: - Cardinality ($ per tag value) - Robust stats (histograms (see prev slide)) - Retention horizons for plausible queries (time duration) Explaining variance: - Correct stats!!! (global p95, p99) - “Suppress the messengers” of microservice failures
- 31. Wrapping Up...
- 32. (first, a hint at my perspective)
- 33. A fun game! (“Observability Whack-a-Mole”) Design your own observability system: ❏ High-throughput ❏ High-cardinality ❏ Lengthy retention window ❏ Unsampled Choose three
- 34. The Life of Trace Data: Dapper Stage Overhead affects… Retained Instrumentation Executed App 100.00% Buffered within app process App 000.10% Flushed out of process App 000.10% Centralized regionally Regional network + storage 000.10% Centralized globally WAN + storage 000.01%
- 35. The Life of Trace Data: Dapper Other Approaches Stage Overhead affects… Retained Instrumentation Executed App 100.00% Buffered within app process App 100.00% Flushed out of process App 100.00% Centralized regionally Regional network + storage 100.00% Centralized globally WAN + storage “fancy”
- 36. An Observability Scorecard Detection - Specificity: cardinality cost, stack coverage - Fidelity: correct stats, high stats frequency - Freshness: ≤ 5 seconds Refinement - Identifying variance: cardinality cost, correct stats, hi-fi histograms, retention horizons - “Suppress the messengers”
- 37. LightStep: Observability with context Automatic deployment and regression detection System and service diagrams Real-time and historical root cause analysis Correlations Custom alerting Easy Setup with no vendor lock-in No cardinality limitations, really
- 39. Extra Slides
- 42. Ideal Refinement: Real-time Must be able to test and eliminate hypotheses quickly Actual data must be ≤10s fresh UI / API latency must be very low
- 44. Ideal Refinement: Context-Rich We can’t expect humans to know what’s normal