The three pillars of observability, namely logs, metrics, and traces, have been raising our industry’s awareness of the three categorical data sources that are indispensable to implement true observability. The need for observability becomes even more obvious and pressing as an organization’s infrastructure and workloads evolve to be more hybrid, distributed, and, inevitably, complex. However, in reality, DevOps professionals a lot of times still face the challenges diversified tool chains and disparate data silos. It is no secret that every single signal from applications or infrastructure in itself is limited in its scope and usefulness. The true power only comes in when logs, metrics, and traces are properly combined and correlated together. Everyone is talking about it, but only a handful few really knows the devil details. In this talk, we will show various scenarios where the combination of logs, metrics, and traces truly make a difference. We will also review some of the challenges and best practices of integrating the three pillars together using OSS tools (and its ecosystem) such as the Elastic Stack.

1. Steve Shangguan
Elastic
The Unreasonable Effectiveness
of Combining and Correlating
Logs, Metrics, and Traces

2. 2
Complicated Modern IT Architectures
My serverless containerized hybrid cloud microservices
sync with geo-dispersed distributed messaging queues
microservice
s
{...}

3. 3
Observability: A Must-Have
Source: Peter Bourgon
Charity Majors
on Observability
“Observability is about swiftly
isolating the source of the
problem”
“Observability is about being
able to ask arbitrary questions
about your environment
without—and this is the key
part—having to know ahead
of time what you wanted to
ask.”
Observability: A measure of how well internal states
of a system can be inferred from knowledge of its
external outputs.

4. 4
Tracing Logging
Custom Assemblage of Point Solutions
Storage & Analysis
Metrics

5. 5

6. 6
Image credit of Adrien Liard

7. 7
3 Pillars of Observability : Logging, Metrics and APM
APM
Logs
Metrics
Reference : Peter Bourgon https://peter.bourgon.org/blog/2017/02/21/metrics-tracing-and-logging.html:

8. Benefits of Logs + Metrics + APM in one stack

9. 9
Correlate Logs, Metrics, Traces by Time
Same UI for KPI summaries and root-cause analysis

10. 1
0
Detect Anomalies with Simultaneous Indicators
Correlate multiple data sources for more intelligent anomaly detection

11. 1
1
Alert with Composite Rules
Trigger off any operational data to provide unified SLA monitoring

12. 1
2
Operational Gains
Using a single technology for operational data saves on administrative costs

13. 13
DEMO

14. Thank you!