Découvrez comment Elasticsearch centralise le stockage des données et comment exploiter Kibana pour les analyser. Sans oublier l'accélération de l'identification, du diagnostic et de la résolution des problèmes.
Les logs, traces et indicateurs au service d'une observabilité unifiée
1. Cyrille Le Clerc
Director of Product Management, Observability
June 2020
Combining Logs, Metrics, and Traces
for Unified Observability
2.
3. Higher resource utilization
increases monitoring complexity
• Orchestration/Hypervisor
• Dynamic/ephemeral jobs
• You can no longer "point" to where
that job lives
Shift to cloud-native yields
maintainable code, with costs
• Traditional licensing models don't
scale as well as your applications
• Hurdles with autoscaling
Monitoring Complexity
Hardware & software trends are evolving in tandem
Evolving Architectures ~↑ Monitoring Complexity
4.
5. Development
Team
Ops: Log
Monitoring
Uptime
Response Time
Uptime Tool
Ops: Infra
Monitoring
Web Logs
App Logs
Database Logs
Container Logs
Log Tool
Ops: Service
Monitoring
Real User Monitoring
Txn Perf Monitoring
Distributed Tracing
APM Tool
Container Metrics
Host Metrics
Database Metics
Network Metrics
Storage Metrics
Metrics Tool
Status Quo: Siloed Collection of Tools
6. APM Data Uptime DataMetrics DataLog Data
Elastic Approach to Observability
Uptime
Response Time
Correctness
Certificate Validation
Web Logs
App Logs
Database Logs
Container Logs
Real User Monitoring
Txn Perf Monitoring
Distributed Tracing
Dependency Mapping
Host/Container Metrics
Database Metics
Network Metrics
Storage Metrics
Dev & Ops Teams
Elastic Common Schema
8. • Ship data from anywhere -- and correlate across your data sources
• The data is yours -- no API rate limiting, no data black boxes
• Cloud native scale -- no constraints on dimensions and cardinalities
Unified Data Layer with Common Schema
Open data keeps your data out of silos
9. Correlate multiple data sources for more intelligent anomaly detection
Unified Machine Learning and Alerting
10. Pricing aligned with business value
Unified Licensing Model
PER
AGENT
$$$$
PER
HOST
$$$$
PER
INGEST
$$$$
PER
MONITOR
$$$$
PER
ADD-ON
$$$$
• Intuitive
Single, unified pricing model. No add-ons.
• Cloud native
No problem using with container workloads and serverless.
• Future proof
You pay for capacity and are not locked into a specific use case.
14. Meet Audit Requirements with Log Lifecycle Management
You’re in control of how your data is tiered
Hot. Warm. Cold. Frozen.
Index Lifecycle Management
Policy based data management that optimize your
cluster behind the scene
Cold storage with online search
Specialized indices for efficient long-term storage
Log archival and rehydratation
Robust snapshot management via API and UI
16. Evolution of Elastic Stack to a Metrics Store
BKD trees
Data structures optimized for numerical
time series analysis.
Columnar storage
Structured data storage, resulting in
compact storage and faster analytics
Rollups
Aggregate older data into bigger time
buckets
Aggregations framework
Analytics features to slice and dice data
along various dimensions
2012
2016
2014
2018
2019
2020
Prometheus support
Support for ingesting data from
Prometheus exporters and server
Improved support for histograms
Dedicated histogram data type in
Elasticsearch
17. Turnkey Data Onboarding
100s of data sources at your fingertips
Now 100+ integrations and growing!
Recently added
AWS LAmbda
AWS Virtual Private Cloud
Amazon Aurora DB
Azure Database Metrics
Azure Container Metrics
Google Cloud Platform Pub/Sub
Istio
...
18. Turn Metrics into Intelligence
Flexible time-series analytics and data visualization
21. 21
Elastic APM
Elastic joins forces with Opbeat
A next-generation APM solution designed
for developers
2017
2018
Distributed tracing
Distributed tracing with W3C Trace Context
standard
2020
2019
Elastic APM GA
Native Elastic integration, Agents for
Python, Node.js, Ruby, Javascript; Real User
Monitoring
● Java
● Go
● .NET
● Node.js
● Javascript
● RUM
Language Support
● Python
● Ruby
● Go
● PHP (in dev)
• Turnkey agents
• Auto-instrument for common
frameworks
• OpenTracing &
OpenTelemetry to avoid
lock-in
22. 22
Evolution of Elastic Stack to Open Source APM
Elastic joins forces with Opbeat
A next-generation APM solution designed
for developers
2017
2018
Distributed tracing
Distributed tracing with W3C Trace Context
standard
2020
2019
Service Maps, annotations
Full featured UI to navigate APM data
Elastic APM GA
Native Elastic integration, Agents for
Python, Node.js, Ruby, Javascript; Real User
Monitoring
45. Demo Key Takeaways
• Unified Observability
○ Single tool
○ Investigate deep in any dimension
○ All data types: logs, metrics, APM, and synthetics
○ All layers: application and infrastructure
○ High cardinalities for powerful search