SlideShare a Scribd company logo

OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn

NETWAYS
NETWAYS

While OpenTelemetry tracing is the de-facto standard today and metrics are also getting more and more established, logging is still lagging far behind. This talk gives an overview of: – Why should you change your logging to work with OpenTelemetry? – Where is the OTel standard in terms of logging? – How can you use it in your application today? We will focus on Java with three different approaches for the implementation. But most of the concepts will translate well to other ecosystems as well.

1 of 12
Download to read offline
OpenTelemetry for Logging
Philipp Krenn, @xeraa
OpenTelemetry
https://github.com/cncf/tag-observability/blob/whitepaper-v1.0.0/whitepaper.md
Create a unified data path for routing traces,
metrics, and logs from apps to backends
Unified
Signal
Routing
Logs have been around forever.
What new value can OpenTelemetry bring?
Easily include common metadata (including
baggage from REST or RPC) in traces, metrics, and
logs to enable correlation
Common
Metadata
Host
Signals and Routing
OTel Collector
Tracing VM
Metrics
App
OTel APM Library
Log Lib
Log Files
Custom
Metrics
Logs
Logs Infra
Metrics
Host Stats K8s API
OTel “hooks”
standard log
frameworks
OTel Collector can
scrape log files and
infra stats in a
vendor-agnostic way
JVM,node,.NET
OTLP to Native OpenTelemetry-compatible backend
Correlation
Decomposing Logs
[2011-10-05T14:48:00] [DEBUG] : Processing Record
[2011-10-05T14:48:00] [INFO] : records_processed=25
[2011-10-05T14:48:00] [WARN] : Record malformatted!
[2011-10-05T14:48:00] [FATAL] :
Exception in thread "main" java.lang.RuntimeException: A test exception
at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13)
at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9)
at com.stackify.StackTraceExample.main(StackTraceExample.java:5)

Recommended

Debugging Planning Issues Using Calcite's Built-in Loggers
Debugging Planning Issues Using Calcite's Built-in LoggersDebugging Planning Issues Using Calcite's Built-in Loggers
Debugging Planning Issues Using Calcite's Built-in LoggersStamatis Zampetakis
 
Managing the Machine Learning Lifecycle with MLOps
Managing the Machine Learning Lifecycle with MLOpsManaging the Machine Learning Lifecycle with MLOps
Managing the Machine Learning Lifecycle with MLOpsFatih Baltacı
 
Kettleetltool 090522005630-phpapp01
Kettleetltool 090522005630-phpapp01Kettleetltool 090522005630-phpapp01
Kettleetltool 090522005630-phpapp01jade_22
 
Squeezing Blood From a Stone V1.2
Squeezing Blood From a Stone V1.2Squeezing Blood From a Stone V1.2
Squeezing Blood From a Stone V1.2Jen Costillo
 
OpenTelemetry 101 FTW
OpenTelemetry 101 FTWOpenTelemetry 101 FTW
OpenTelemetry 101 FTWNGINX, Inc.
 
QTP&UFT Automation Framework
QTP&UFT Automation FrameworkQTP&UFT Automation Framework
QTP&UFT Automation FrameworkYu Tao Zhang
 
Ceph Day Beijing: Newstore: Design, Implementation, and Next Step Work
Ceph Day Beijing: Newstore: Design, Implementation, and Next Step Work Ceph Day Beijing: Newstore: Design, Implementation, and Next Step Work
Ceph Day Beijing: Newstore: Design, Implementation, and Next Step Work Ceph Community
 

More Related Content

Similar to OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn

Oracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionOracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionAditya Trivedi
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityElasticsearch
 
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LMESet your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LMEconfluent
 
Jboss World 2011 Infinispan
Jboss World 2011 InfinispanJboss World 2011 Infinispan
Jboss World 2011 Infinispancbo_
 
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Miguel Pérez Colino
 
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...NETWAYS
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...James Anderson
 
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...Alex Rayón Jerez
 
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaCombinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaElasticsearch
 
Lap around .net 4
Lap around .net 4Lap around .net 4
Lap around .net 4Abdul Khan
 
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...Zalando Technology
 
Combinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaCombinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaElasticsearch
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to KubernetesSamuel Dratwa
 
Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015aioughydchapter
 
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]Animesh Singh
 

Similar to OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn (20)

Oracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionOracle 11g data warehouse introdution
Oracle 11g data warehouse introdution
 
Combining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observabilityCombining logs, metrics, and traces for unified observability
Combining logs, metrics, and traces for unified observability
 
Oracle Management Cloud
Oracle Management Cloud Oracle Management Cloud
Oracle Management Cloud
 
Oracle Management Cloud
Oracle Management CloudOracle Management Cloud
Oracle Management Cloud
 
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LMESet your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
Set your Data in Motion with Confluent & Apache Kafka Tech Talk Series LME
 
Jboss World 2011 Infinispan
Jboss World 2011 InfinispanJboss World 2011 Infinispan
Jboss World 2011 Infinispan
 
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
 
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
 
Oracle
OracleOracle
Oracle
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
 
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
Pentaho Data Integration: Extrayendo, integrando, normalizando y preparando m...
 
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizadaCombinación de logs, métricas y seguimiento para una visibilidad centralizada
Combinación de logs, métricas y seguimiento para una visibilidad centralizada
 
Lap around .net 4
Lap around .net 4Lap around .net 4
Lap around .net 4
 
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
Stream Processing using Apache Flink in Zalando's World of Microservices - Re...
 
Combinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificadaCombinação de logs, métricas e rastreamentos para observabilidade unificada
Combinação de logs, métricas e rastreamentos para observabilidade unificada
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015Aioug ha day oct2015 goldengate- High Availability Day 2015
Aioug ha day oct2015 goldengate- High Availability Day 2015
 
11g R2
11g R211g R2
11g R2
 
AKS: k8s e azure
AKS: k8s e azureAKS: k8s e azure
AKS: k8s e azure
 
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
 

Recently uploaded

Auditorium Session 3 - Resilience - Financial Resilience and Collaboration
Auditorium Session 3 - Resilience - Financial Resilience and CollaborationAuditorium Session 3 - Resilience - Financial Resilience and Collaboration
Auditorium Session 3 - Resilience - Financial Resilience and CollaborationMuseums Galleries Scotland
 
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptxGarcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx0461620
 
Auditorium Session 2 - Workforce - Diversity/Skills & Confidence
Auditorium Session 2 - Workforce - Diversity/Skills & ConfidenceAuditorium Session 2 - Workforce - Diversity/Skills & Confidence
Auditorium Session 2 - Workforce - Diversity/Skills & ConfidenceMuseums Galleries Scotland
 
Instructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdf
Instructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdfInstructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdf
Instructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdfDr. Cherinet Aytenfsu Weldearegay
 
Monthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSEMonthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSEOlgaOliveaJohn
 
IE Application: Express Yourself - Sofia Merizalde
IE Application: Express Yourself - Sofia MerizaldeIE Application: Express Yourself - Sofia Merizalde
IE Application: Express Yourself - Sofia Merizaldesofiamerizaldev
 
Freeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand ExplorationFreeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand Explorationabbytoliver
 
Teams Nation 2024 - #Copilot & Teams or Just Premium.pptx
Teams Nation 2024 - #Copilot & Teams or Just Premium.pptxTeams Nation 2024 - #Copilot & Teams or Just Premium.pptx
Teams Nation 2024 - #Copilot & Teams or Just Premium.pptxKai Stenberg
 
Issues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptxIssues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptxbill846304
 
TheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptxTheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptxStevenLuker3
 

Recently uploaded (11)

Auditorium Session 3 - Resilience - Financial Resilience and Collaboration
Auditorium Session 3 - Resilience - Financial Resilience and CollaborationAuditorium Session 3 - Resilience - Financial Resilience and Collaboration
Auditorium Session 3 - Resilience - Financial Resilience and Collaboration
 
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptxGarcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
Garcia_RobertDaniel_SPCSTA_PB1_2024-02.pptx
 
Auditorium Session 1 - Connection - Inclusion
Auditorium Session 1 - Connection - InclusionAuditorium Session 1 - Connection - Inclusion
Auditorium Session 1 - Connection - Inclusion
 
Auditorium Session 2 - Workforce - Diversity/Skills & Confidence
Auditorium Session 2 - Workforce - Diversity/Skills & ConfidenceAuditorium Session 2 - Workforce - Diversity/Skills & Confidence
Auditorium Session 2 - Workforce - Diversity/Skills & Confidence
 
Instructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdf
Instructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdfInstructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdf
Instructional Supervision - By Dr. Cherinet Aytenfsu Weldearegay.pdf
 
Monthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSEMonthly HSE Report March for overall HSE
Monthly HSE Report March for overall HSE
 
IE Application: Express Yourself - Sofia Merizalde
IE Application: Express Yourself - Sofia MerizaldeIE Application: Express Yourself - Sofia Merizalde
IE Application: Express Yourself - Sofia Merizalde
 
Freeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand ExplorationFreeman_Abigail Personal Brand Exploration
Freeman_Abigail Personal Brand Exploration
 
Teams Nation 2024 - #Copilot & Teams or Just Premium.pptx
Teams Nation 2024 - #Copilot & Teams or Just Premium.pptxTeams Nation 2024 - #Copilot & Teams or Just Premium.pptx
Teams Nation 2024 - #Copilot & Teams or Just Premium.pptx
 
Issues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptxIssues affecting LGBT as they grow older.pptx
Issues affecting LGBT as they grow older.pptx
 
TheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptxTheSimpsons_Fandom_Assignment_4.5pc.pptx
TheSimpsons_Fandom_Assignment_4.5pc.pptx
 

OSMC 2023 | OpenTelemetry for Logging by Philipp Krenn

  • 3. Create a unified data path for routing traces, metrics, and logs from apps to backends Unified Signal Routing Logs have been around forever. What new value can OpenTelemetry bring? Easily include common metadata (including baggage from REST or RPC) in traces, metrics, and logs to enable correlation Common Metadata
  • 4. Host Signals and Routing OTel Collector Tracing VM Metrics App OTel APM Library Log Lib Log Files Custom Metrics Logs Logs Infra Metrics Host Stats K8s API OTel “hooks” standard log frameworks OTel Collector can scrape log files and infra stats in a vendor-agnostic way JVM,node,.NET OTLP to Native OpenTelemetry-compatible backend
  • 6. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5)
  • 7. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OpenTelemetry Span Added Value: ● Latency and success / failure analytics ● Distributed tracing ● Correlate with Business Analytics ● Verbosity is controlled through intelligent sampling instead of static log-level
  • 8. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OpenTelemetry Metric Added Value: ● Metric rollups ● Much lower storage costs ● Easily visualized ● Derive Business Intelligence ● Framed in context of transaction
  • 9. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OTel Span Event (the new “log line”) Added Value: ● Framed in context of transaction
  • 10. Decomposing Logs [2011-10-05T14:48:00] [DEBUG] : Processing Record [2011-10-05T14:48:00] [INFO] : records_processed=25 [2011-10-05T14:48:00] [WARN] : Record malformatted! [2011-10-05T14:48:00] [FATAL] : Exception in thread "main" java.lang.RuntimeException: A test exception at com.stackify.StackTraceExample.methodB(StackTraceExample.java:13) at com.stackify.StackTraceExample.methodA(StackTraceExample.java:9) at com.stackify.StackTraceExample.main(StackTraceExample.java:5) Record an OTel Span Exception Added Value: ● Derive quality metrics ● Framed in context of transaction
  • 11. Logs (OTLP) File to Elastic Agent Log Models Host Collector Tracing (OTLP) App Instrumentation Lib Log Lib (eg Logback) Log File Metrics (OTLP) Logs (OTLP) OTel Hook log(“hello!”) Hook adds traceID and common metadata Filebeat Logs (Elastic) OTel logging leverages existing logging frameworks Supported Models: 1 2 3 most mature 1 2 3 least mature 2 3 Direct OTLP File to Collector OTLP