Download to read offline
Uploaded byGDG Cloud Bengaluru
Building observable infrastructure and code
This document discusses collecting and correlating different types of infrastructure and application data for observability purposes. It notes that data like logs, metrics and traces are typically collected by different tools that store data in different formats. It recommends using the Elastic Common Schema to define a common data schema, which allows ingesting and correlating data from different sources into a single Elasticsearch cluster. An upcoming session is also promoted for discussing ingesting Jaeger traces from a Python app.
More Related Content
Similar to Building observable infrastructure and code
Building observable infrastructure and code
- 1. Building Observable Infrastructure &Code @aravindputrevu
- 2. @aravindputrevu Hello Aravind Putrevu Developer Advocate aravind.dev| @aravindputrevu
- 3. @aravindputrevu Upcoming Sessions ● IngestingJaeger Traces from Python Flask App - 27th Jun ● Powering a Community Platform using Elasticsearch - 11th July **community.elastic.co**
- 4. @aravindputrevu Problem You are awho loves coding but wants less bugs. You are a who wants all systems to function well. You are a who wants to secure & protect data. You are a product owner looking after product perf.
- 5.
- 6. @aravindputrevu Usual Scenario Web Logs AppLogs Database Logs Container Logs Container Metrics Host Metrics Database Metics Network Metrics Storage Metrics Uptime Response Time Real User Monitoring Txn Perf Monitoring Code Tracing Errors APM SLA Metrics Logs
- 7. @aravindputrevu Usual Scenario Web Logs AppLogs Database Logs Container Logs Container Metrics Host Metrics Database Metics Network Metrics Storage Metrics Uptime Response Time Real User Monitoring Txn Perf Monitoring Code Tracing Errors APM Tool SLA Tool Metrics Platform Logs Platform {HH:MM:SS}
- 8. @aravindputrevu Best Approach Web Logs AppLogs Database Logs Container Logs Container Metrics Host Metrics Database Metics Network Metrics Storage Metrics Uptime Response Time Real User Monitoring Txn Perf Monitoring Code Tracing Errors APM SLA Metrics Logs {HH:MM:SS}
- 9. @aravindputrevu Best Approach Web Logs AppLogs Database Logs Container Logs Container Metrics Host Metrics Database Metics Network Metrics Storage Metrics Uptime Response Time Real User Monitoring Txn Perf Monitoring Code Tracing Errors APM SLA Metrics Logs {HH:MM:SS}
- 10.
- 11. @aravindputrevu But then, eachtool ships/stores data in different schema?
- 12.
- 13.
- 14. @aravindputrevu Common Data Schema github.com/elastic/ecs ●Defines a common set of fields for ingesting data into Elasticsearch. ● Helps you correlate data from different source types Logs, Metrics and Traces ● Designed to be extensible and reusable
- 15. @aravindputrevu*Prometheus ships toES via metricbeat module ECS ECS
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.