Download as PDF, PPTX
Uploaded byDocker, Inc.
DockerCon EU 2015: Monitoring Docker
This document discusses monitoring Docker containers. It provides an overview of Datadog and how it can be used to monitor dynamic infrastructure like Docker. It then discusses the challenges in monitoring Docker environments due to their dynamic and distributed nature. Finally, it outlines different methods for collecting key metrics on Docker containers like CPU, memory, I/O and network metrics using Docker's pseudo-files, stats command and API.
More Related Content
Similar to DockerCon EU 2015: Monitoring Docker
DockerCon EU 2015: Monitoring Docker
- 1. Monitoring Docker Ilan Rabinovitch Director,Community - Datadog
- 2. Agenda • Monitoring 101- Crash Course • Challenges in Monitoring Dynamic Infrastructure • Key Docker Metrics
- 3. About Me •Long time Datadog user. • Prior to Datadog built automation and monitoring tooling at Ooyala and Edmunds.com • Community Event Organizer: SCALE and Texas Linux Fest DevOpsDay LA DevOpsDay Silicon Valley Ilan Rabinovitch Datadog
- 4. Quick Overview ofDatadog Datadog gathers performance data from all your application and infrastructure components. • Monitoring for modern applications 1. Dynamic Infrastructure 2. Containers (Docker, ECS, Mesos, k8s, and more…) 3. Microservices • Time series storage of metrics and events • Trending, alerting and anomaly detection. • We’re hiring! (Remote, NY, Boston, Paris)
- 5.
- 6. Docker Adoption Growth We’vesee 5x increase of Docker adoption over the last year.
- 7. Monitor Everything Culture Automation Metrics Sharing Damon Edwardsand John Willis DevOpsDay LA
- 10. Monitoring 101: CategorizingYour Metrics More at: http://goo.gl/t1Rgcg
- 11. Monitoring 101: Focuson symptoms More at: http://goo.gl/t1Rgcg
- 12. Recurse until youfind root cause. More at: http://goo.gl/t1Rgcg
- 14. Operational Complexity • Averagecontainers per host: N (N=4, 10/2015) • N-times as many “hosts” to manage • Affects • provisioning: prep’ing & building containers • configuration: passing config to containers • orchestration: deciding where/when containers run • monitoring: making sure containers run properly
- 15. So what doesthat mean for monitoring and management?
- 17. Fact 7: DockerHosts Often Run Four Containers at a Time
- 19.
- 20. Operational Complexity: Scale 160 metricsper host 640 metrics per host Assuming 4 containers per host
- 21. Operational Complexity: Scale 100 instances 64,000 metrics Assuming4 containers per host
- 22.
- 24. Monitoring Needs andPains • Avoid Static config files tracking dynamic infrastructure.
- 25. Monitoring Needs andPains: Static vs Dynamic • Avoid Static config files tracking dynamic infrastructure. Configuration management is awesome, but….
- 26. Monitoring Needs andPains • Avoid Static config files tracking dynamic infrastructure. • Avoid a host centric view. Focus on service level.
- 27. Monitoring Needs andPains: Host Centric vs Service Centric
- 28. Monitoring Needs andPains: Host Centric vs Service Centric
- 29. Monitoring Needs andPains • Avoid Static config files tracking dynamic infrastructure. • Avoid a host centric view. Focus on service level. • Use tags, labels, etc on your hosts and metrics to form queries.
- 30. Monitoring Needs andPains: Query Based Monitoring “Show me rate of HTTP 500 responses from nginx” “… in region us-east-1 across all availability zones” “… running my app version 2….” • Use tags, labels, etc on your hosts and metrics to form queries. • Pull in labels from your infrastructure whether EC2, Docker or your scheduler. • Ask questions that will ring true regardless of your scale that day.
- 31. Monitoring Needs andPains • Avoid Static config files tracking dynamic infrastructure. • Avoid a host centric view. Focus on service level. • Use tags, labels, etc on your hosts and metrics to form queries. • Know your underlying tech. In this case Docker and how to pull metrics from it.
- 32. Collecting Docker Metrics Moreat http://bit.ly/1j4QXek
- 33. Collecting Docker Metrics:Pseudo Files • Access via sysfs in /sys/fs/cgroup or /cgroup • By default do not require root access. • Fast and light weight • Limited I/O and Network metrics.
- 34. Collecting Docker Metrics:stats • Continuous live stream of basic CPU, memory, & network metrics. • Available via API in JSON (see unix:///var/run/docker.sock) • At least version 1.5.0 of Docker (released Feb 2015)
- 35. Collecting Docker Metrics:stats api • Similar to stats command provides a stream of metrics but more details. • Data is returned in JSON (see unix:///var/run/docker.sock)
- 37. Collecting Docker Metrics:Summary CPU METRICS MEMORY METRICS I/O METRICS NETWORK METRICS pseudo-files Yes Yes Some Yes, in 1.6.1+ stats command Basic Basic No Basic API Yes Yes Some Yes