This document discusses Elastic's approach to unified observability using the Elastic Stack. It describes how the Elastic Stack can be used to ingest and correlate logs, metrics, APM, and uptime data to provide a single unified view. It highlights benefits like being able to reuse analysis and content across data sources. It also summarizes Elastic's Common Schema, licensing model, and provides an overview of logs, metrics, APM, and uptime monitoring capabilities in the Elastic Stack.

1. Gabriel Moskovicz
Principal Solutions Architect
June, 2020
Logs, Metrics, and APM for
Unified Observability

2. Elastic Enterprise Search Elastic SecurityElastic
Observability

3. Logs Metrics APM Uptime
Elastic Observability

4. 4
#ObservaBLT
Observability
=
+
+

6. Higher resource utilization
increases monitoring complexity
• Orchestration/Hypervisor
• Dynamic/ephemeral jobs
• You can no longer "point" to where
that job lives
Monitoring Complexity
Hardware & software trends are evolving in tandem
Evolving Architectures ~↑ Monitoring Complexity
Shift to cloud-native yields
maintainable code, with costs
• Traditional licensing models don't
scale as well as your applications
• Hurdles with autoscaling

7. Applications VMs/Containers
Other DBs,
Services &
Middleware
Orchestration InfrastructureAPM
Metrics
Logs
Uptime
Uptime
APM Metrics
APM Logs
APM
APM
Metrics
Logs
Uptime
Metrics
Logs
Uptime
APM

8. Development
Team
Ops: Log
Monitoring
Uptime
Response Time
Uptime Tool
Ops: Infra
Monitoring
Web Logs
App Logs
Database Logs
Container Logs
Log Tool
Ops: Service
Monitoring
Real User Monitoring
Txn Perf Monitoring
Distributed Tracing
APM Tool
Container Metrics
Host Metrics
Database Metics
Network Metrics
Storage Metrics
Metrics Tool
Status Quo: Siloed Collection of Tools

9. Observability is
a search use case

10. APM Data Uptime DataMetrics DataLog Data
Elastic Approach to Observability
Uptime
Response Time
Web Logs
App Logs
Database Logs
Container Logs
Real User Monitoring
Txn Perf Monitoring
Distributed Tracing
Container Metrics
Host Metrics
Database Metics
Network Metrics
Storage Metrics
Dev & Ops Teams

11. ¿Está interesado en implementar
una solución de Observabilidad?

12. • Correlate data from different sources
• Ability to re-use analysis content
• Ability to re-use Elastic-provided content
Benefits
• Published at: github.com/elastic/ecs
• Supported in Beats and APM since 7.0
• Community feedback welcome!
Status
Elastic Common Schema (ECS)
Supports ad hoc analysis in Kibana Dashboards

13. Unified User Interface
Same UI for KPI summaries and root-cause analysis

14. Correlate multiple data sources for more intelligent anomaly detection
Unified Machine Learning

15. Trigger off any operational data to provide unified SLA monitoring
Unified Alerting

16. Pricing aligned with business value
Unified Licensing Model
PER
AGENT
$$$$
PER
HOST
$$$$
PER
INGEST
$$$$
PER
MONITOR
$$$$
PER
ADD-ON
$$$$
• Intuitive
Single, unified pricing model. No add-ons.
• Cloud native
No problem using with container workloads and serverless.
• Future proof
You pay for capacity and are not locked into a specific use case.

17. Elastic Stack for logs

18. Logs
64.242.88.10 - - [07/Jan/2019:16:10:02 -0800] "GET /mailman/listinfo/hsdivision HTTP/1.1" 200 6291
64.242.88.10 - - [07/Jan/2019:16:10:02 -0800] "POST /twiki/bin/view/TWiki/WikiSyntax HTTP/1.1" 404 7352
64.242.88.10 - - [07/Jan/2019:16:10:02 -0800] "GET /twiki/bin/view/Main/DCCAndPostFix HTTP/1.1" 200 5253
For each event, print out what happened.
Logs are chronological records of events

20. Elastic Stack for metrics

21. Metrics vs Logs
Metrics are periodic measurement of numeric KPIs
07/Jan/2019 16:10:00 all 2.58 0.00 0.70 1.12 0.05 95.55 server1 containerX regionA
07/Jan/2019 16:20:00 all 2.56 0.00 0.69 1.05 0.04 95.66 server2 containerY regionB
07/Jan/2019 16:30:00 all 2.64 0.00 0.65 1.15 0.05 95.50 server2 containerZ regionC
Every x minutes, measure the CPU load, print it out, and annotate with meta-data.
64.242.88.10 - - [07/Jan/2019:16:10:02 -0800] "GET /mailman/listinfo/hsdivision HTTP/1.1" 200 6291
64.242.88.10 - - [07/Jan/2019:16:10:02 -0800] "POST /twiki/bin/view/TWiki/WikiSyntax HTTP/1.1" 404 7352
64.242.88.10 - - [07/Jan/2019:16:10:02 -0800] "GET /twiki/bin/view/Main/DCCAndPostFix HTTP/1.1" 200 5253
For each event, print out what happened.
Logs are chronological records of events

23. Elastic Stack for APM

24. 24
Why APM?
03:43:45 Request "GET cyclops.ESProductDetailView"
03:43:57 Response "cyclops.ESProductDetailView 200 OK"
12 seconds - zZzzZZz
Example: Slow response or load times

25. Why APM?
03:43:59 Request "POST /api/checkout"
03:43:59 Response "/api/checkout 500 ERROR"
Example: Errors & Exceptions

26. APM Agents
● Java
● Go
● .NET
● Javascript (React / Angular)
● RUM (Real User Monitoring)
Language Support
● Python
● Ruby
● Node.js
• Easy to add to your applications
• Designed to be lightweight
• Open source
• Support distributed tracing
• OpenTracing compatible
• Distributed Tracing & OpenTracing + Jaeger intake
Auto-instrumentation of common programming frameworks

27. Distributed Tracing & OpenTracing + Jaeger intake
End-to-end transaction tracking with auto-instrumentation or OpenTracing IDs

29. 29
Elastic Stack for uptime

30. Heartbeat: Uptime Monitoring

32. 32
All together now

33. 33
Demo

34. 34
Demo

35. What now?
Try it yourself!

36. While you monitor, why not protect?
Elastic SIEM

37. Pruébalo gratis
por 14 días
- Incluye acceso a todos los productos
y features disponibles
- Elastic Observability y Elastic
Enterprise Seach tambien incluidos
- Fácil de convertir a un ambiente en
producción en minutos

38. Thank you