SlideShare a Scribd company logo
1 of 23
Download to read offline
The State of Logging on Docker
March 26, 2015
1
Housekeeping
© 2015 Logentries. All Rights Reserved. Confidential. 2
•  Please ask questions via the chat pane
•  All registrants will receive a recorded copy of the webinar following
the presentation.
•  Follow us @Logentries
Meet The Presenters
© 2015 Logentries. All Rights Reserved. Confidential. 3
Trevor Parsons, PhD
Logentries Co-founder
& Chief Scientist
Peter Elger
NearForm Director of
Engineering
Bright Fulton
Swipely Director of
Engineering Operations
Webinar Outline
© 2015 Logentries. All Rights Reserved. Confidential. 4
•  Challenges with monitoring micro-services
•  What should you be logging?
•  The State of Logging on Docker
•  Inside the container
•  Outside the container
•  Specialized containers
•  Useful APIs
•  Common Questions
•  What about CoreOS?
Monitoring Micro-Services
© 2015 Logentries. All Rights Reserved. Confidential. 5
Containers: Ephemeral in Nature
© 2015 Logentries. All Rights Reserved. Confidential. 6
What to Log?
© 2015 Logentries. All Rights Reserved. Confidential. 7
Simple best practices for what to log:
•  When the event occurred: the timestamp
•  Structured Logs: JSON, KVP
•  Context: if someone bought a book… what book was it…
•  Unique Identifiers: request ID, User ID, Session ID, thread ID
•  Host Info: hostname & IP
•  Container Info: Image ID, Container ID, Process ID
Logging on Docker: No Standard Approach
© 2015 Logentries. All Rights Reserved. Confidential. 8
Docker Logging Support includes:
•  Capturing container stdout logs
•  Access via Docker_remote_api
•  Access via Docker CLI
•  $ docker logs [ID]
•  Will fetch the logs from a container
•  can be used to tail also
•  $ docker logs –tf –tail [ID]
Logging on Docker: No Standard Approach
© 2015 Logentries. All Rights Reserved. Confidential. 9
•  Inside the Container
•  Log from the application
•  Run a logging Daemon/collector within the container
•  Outside the Container (on the host)
•  Containers write logs to the host
•  Collector runs on the host
•  Logging Container
•  Collector runs in a specialized container
Inside the Container
© 2015 Logentries. All Rights Reserved. Confidential. 10
SERVER
LOG MANAGEMENT
& ANALYTICS
PLATFORMHOST OS
DOCKER
LOGSLOGS
APP APP
CONTAINER 1 CONTAINER 2
Daemon Daemon
Inside the Container
© 2015 Logentries. All Rights Reserved. Confidential. 11
•  Logging libs (log from your app code)
•  Install a collector inside a container
•  Use syslog inside a container
Outside the Container (on host)
© 2015 Logentries. All Rights Reserved. Confidential. 12
SERVER
LOG MANAGEMENT
& ANALYTICS
PLATFORM
HOST OS
DOCKER
CONTAINER 1 CONTAINER 2
APP APP
LOGS LOGS
LOGSAGENT
Outside the container (on host)
© 2015 Logentries. All Rights Reserved. Confidential. 13
•  Configure a container to bind to a volume & write logs to the volume
•  Write logs to stdout
•  Collector runs on the host
Logging Containers
© 2015 Logentries. All Rights Reserved. Confidential. 14
SERVER
LOG MANAGEMENT
& ANALYTICS
PLATFORMHOST OS
DOCKER
CONTAINER 1 CONTAINER 2 LOGGING CONTAINER
APP APP Agent
LOGS LOGS
LOGS
Logging Containers
© 2015 Logentries. All Rights Reserved. Confidential. 15
•  Syslog in a container
•  Collector in a container
•  E.g. Logentries container will collect Stats and Logs
•  https://github.com/nearform/docker-logentries
•  Logspout: https://github.com/gliderlabs/logspout
Common Questions
© 2015 Logentries. All Rights Reserved. Confidential. 16
•  Do I need to rotate any logs?
•  Is there a big performance hit?
•  E.g. collecting metrics from every container
•  Any security concerns?
What about CoreOS?
© 2015 Logentries. All Rights Reserved. Confidential. 17
•  Enables warehouse-scale computing
•  Used for managing clusters of containers
•  Distributed process manager, Fleet (based on systemd)
•  etcd for service discovery and global storage of your config settings
•  How you can log from your CoreOS Clusters
•  Journal - a modern logging system (e.g. JSON export, forward sealing)
•  Journal2Logentries: https://github.com/kelseyhightower/journal-2-logentries
Useful Resources:
© 2015 Logentries. All Rights Reserved. Confidential. 18
•  Docker CLI
•  https://docs.docker.com/reference/commandline/cli/
•  Docker Remote API
•  https://docs.docker.com/reference/api/docker_remote_api/
•  Stats API (From Docker 1.5)
•  https://github.com/docker/docker/pull/9984 (e.g. used by Logentries container)
•  Docker RunTime Metrics
•  Uses the native cgroup accounting metrics
•  https://docs.docker.com/articles/runmetrics/ (e.g. used by DataDog)
•  Docker Ruby Gem
•  https://github.com/swipely/docker-api
Useful Resources: Blogs
© 2015 Logentries. All Rights Reserved. Confidential. 19
•  The State of Logging on Docker
•  https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/
•  What is the Docker Stats API
•  https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/
•  Docker Log Management Using Fluentd
•  http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/
•  How to Centralize Logs from CoreOS Clusters
•  https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/
Useful Resources: Blogs
© 2015 Logentries. All Rights Reserved. Confidential. 20
•  The State of Logging on Docker
•  https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/
•  What is the Docker Stats API
•  https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/
•  Docker Log Management Using Fluentd
•  http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/
•  How to Centralize Logs from CoreOS Clusters
•  https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/
Our Docker Community Pack
© 2015 Logentries. All Rights Reserved. Confidential. 21
Getting Started
© 2015 Logentries. All Rights Reserved. Confidential. 22
•  Don’t have a Logentries Free Account Yet?
•  Sign-up for a free 30-day trial: http://logentries.com/quick-start
•  Docker & Node.Js Development:
•  http://www.nearform.com/
•  http://nscale.nearform.com/
•  Contact us directly!
•  Trevor.parsons@logentries.com
•  support@logentries.com
•  peter.elger@nearform.com
•  Let’s take a few questions!
Thank you!

More Related Content

What's hot

Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014D
 
CoreOS @Codetalks Hamburg
CoreOS @Codetalks HamburgCoreOS @Codetalks Hamburg
CoreOS @Codetalks HamburgTimo Derstappen
 
Comprehensive Monitoring for Docker
Comprehensive Monitoring for DockerComprehensive Monitoring for Docker
Comprehensive Monitoring for DockerChristian Beedgen
 
ContainerCon sysdig Slides
ContainerCon sysdig Slides ContainerCon sysdig Slides
ContainerCon sysdig Slides Loris Degioanni
 
Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationDockerizing a Symfony2 application
Dockerizing a Symfony2 applicationRoman Rodomansky
 
Scaling Next-Generation Internet TV on AWS With Docker, Packer, and Chef
Scaling Next-Generation Internet TV on AWS With Docker, Packer, and ChefScaling Next-Generation Internet TV on AWS With Docker, Packer, and Chef
Scaling Next-Generation Internet TV on AWS With Docker, Packer, and Chefbridgetkromhout
 
Docker Multi Host Networking, Rachit Arora, IBM
Docker Multi Host Networking, Rachit Arora, IBMDocker Multi Host Networking, Rachit Arora, IBM
Docker Multi Host Networking, Rachit Arora, IBMNeependra Khare
 
Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014André Rømcke
 
Docker Swarm 0.2.0
Docker Swarm 0.2.0Docker Swarm 0.2.0
Docker Swarm 0.2.0Docker, Inc.
 
Container Performance Analysis
Container Performance AnalysisContainer Performance Analysis
Container Performance AnalysisBrendan Gregg
 
Go 1.8 'new' networking features
Go 1.8 'new' networking featuresGo 1.8 'new' networking features
Go 1.8 'new' networking featuresstrikr .
 
Docker and Maestro for fun, development and profit
Docker and Maestro for fun, development and profitDocker and Maestro for fun, development and profit
Docker and Maestro for fun, development and profitMaxime Petazzoni
 
runC: The little engine that could (run Docker containers) by Docker Captain ...
runC: The little engine that could (run Docker containers) by Docker Captain ...runC: The little engine that could (run Docker containers) by Docker Captain ...
runC: The little engine that could (run Docker containers) by Docker Captain ...Docker, Inc.
 
Docker Networking – Running multi-host applications
Docker Networking – Running multi-host applicationsDocker Networking – Running multi-host applications
Docker Networking – Running multi-host applicationsChristina Rasimus
 
Containerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyContainerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyDocker, Inc.
 
DCUS17 : Docker networking deep dive
DCUS17 : Docker networking deep diveDCUS17 : Docker networking deep dive
DCUS17 : Docker networking deep diveMadhu Venugopal
 
Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...
Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...
Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...Fwdays
 

What's hot (20)

Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014
 
CoreOS @Codetalks Hamburg
CoreOS @Codetalks HamburgCoreOS @Codetalks Hamburg
CoreOS @Codetalks Hamburg
 
Docker practical solutions
Docker practical solutionsDocker practical solutions
Docker practical solutions
 
Comprehensive Monitoring for Docker
Comprehensive Monitoring for DockerComprehensive Monitoring for Docker
Comprehensive Monitoring for Docker
 
ContainerCon sysdig Slides
ContainerCon sysdig Slides ContainerCon sysdig Slides
ContainerCon sysdig Slides
 
Project Atomic-Nulecule
Project Atomic-NuleculeProject Atomic-Nulecule
Project Atomic-Nulecule
 
Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationDockerizing a Symfony2 application
Dockerizing a Symfony2 application
 
Scaling Next-Generation Internet TV on AWS With Docker, Packer, and Chef
Scaling Next-Generation Internet TV on AWS With Docker, Packer, and ChefScaling Next-Generation Internet TV on AWS With Docker, Packer, and Chef
Scaling Next-Generation Internet TV on AWS With Docker, Packer, and Chef
 
Docker Multi Host Networking, Rachit Arora, IBM
Docker Multi Host Networking, Rachit Arora, IBMDocker Multi Host Networking, Rachit Arora, IBM
Docker Multi Host Networking, Rachit Arora, IBM
 
What's New in Docker 1.12?
What's New in Docker 1.12?What's New in Docker 1.12?
What's New in Docker 1.12?
 
Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014
 
Docker Swarm 0.2.0
Docker Swarm 0.2.0Docker Swarm 0.2.0
Docker Swarm 0.2.0
 
Container Performance Analysis
Container Performance AnalysisContainer Performance Analysis
Container Performance Analysis
 
Go 1.8 'new' networking features
Go 1.8 'new' networking featuresGo 1.8 'new' networking features
Go 1.8 'new' networking features
 
Docker and Maestro for fun, development and profit
Docker and Maestro for fun, development and profitDocker and Maestro for fun, development and profit
Docker and Maestro for fun, development and profit
 
runC: The little engine that could (run Docker containers) by Docker Captain ...
runC: The little engine that could (run Docker containers) by Docker Captain ...runC: The little engine that could (run Docker containers) by Docker Captain ...
runC: The little engine that could (run Docker containers) by Docker Captain ...
 
Docker Networking – Running multi-host applications
Docker Networking – Running multi-host applicationsDocker Networking – Running multi-host applications
Docker Networking – Running multi-host applications
 
Containerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyContainerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael Crosby
 
DCUS17 : Docker networking deep dive
DCUS17 : Docker networking deep diveDCUS17 : Docker networking deep dive
DCUS17 : Docker networking deep dive
 
Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...
Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...
Алексей Петров "Dockerize Me: Distributed PHP applications with Symfony, Dock...
 

Similar to The State of Logging on Docker

컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Docker컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Dockerseungdon Choi
 
Python Linters at Scale.pdf
Python Linters at Scale.pdfPython Linters at Scale.pdf
Python Linters at Scale.pdfJimmy Lai
 
Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10MagaliDavidCruz
 
Docker at and with SignalFx
Docker at and with SignalFxDocker at and with SignalFx
Docker at and with SignalFxSignalFx
 
DevOps, Kubernetes and Istio
DevOps, Kubernetes and IstioDevOps, Kubernetes and Istio
DevOps, Kubernetes and IstioJohn Jardin
 
Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019Olivier Loverde
 
PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux Neotys
 
Adopting OpenTelemetry
Adopting OpenTelemetryAdopting OpenTelemetry
Adopting OpenTelemetryVincent Behar
 
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...NETWAYS
 
Disenchantment: Netflix Titus, Its Feisty Team, and Daemons
Disenchantment: Netflix Titus, Its Feisty Team, and DaemonsDisenchantment: Netflix Titus, Its Feisty Team, and Daemons
Disenchantment: Netflix Titus, Its Feisty Team, and DaemonsC4Media
 
Manage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with ObservabilityManage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with ObservabilityNGINX, Inc.
 
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)Lee Myring
 
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015Datadog
 
Fluentd and docker monitoring
Fluentd and docker monitoringFluentd and docker monitoring
Fluentd and docker monitoringVinay Krishna
 
from Docker to Moby and back. what changed ?
from Docker to Moby and back. what changed ?from Docker to Moby and back. what changed ?
from Docker to Moby and back. what changed ?strikr .
 
OpenNTF - The Lotus Notes and Domino Open Source Organization
OpenNTF - The Lotus Notes and Domino Open Source OrganizationOpenNTF - The Lotus Notes and Domino Open Source Organization
OpenNTF - The Lotus Notes and Domino Open Source OrganizationBruce Elgort
 
12 Factor App Methodology
12 Factor App Methodology12 Factor App Methodology
12 Factor App Methodologylaeshin park
 
Docker Enterprise Deployment Planning
Docker Enterprise Deployment PlanningDocker Enterprise Deployment Planning
Docker Enterprise Deployment PlanningStephane Woillez
 
Strategy, planning and governance for enterprise deployments of containers - ...
Strategy, planning and governance for enterprise deployments of containers - ...Strategy, planning and governance for enterprise deployments of containers - ...
Strategy, planning and governance for enterprise deployments of containers - ...The Incredible Automation Day
 

Similar to The State of Logging on Docker (20)

컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Docker컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Docker
 
Python Linters at Scale.pdf
Python Linters at Scale.pdfPython Linters at Scale.pdf
Python Linters at Scale.pdf
 
Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10
 
Docker at and with SignalFx
Docker at and with SignalFxDocker at and with SignalFx
Docker at and with SignalFx
 
Docker {at,with} SignalFx
Docker {at,with} SignalFxDocker {at,with} SignalFx
Docker {at,with} SignalFx
 
DevOps, Kubernetes and Istio
DevOps, Kubernetes and IstioDevOps, Kubernetes and Istio
DevOps, Kubernetes and Istio
 
Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019
 
PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux
 
Adopting OpenTelemetry
Adopting OpenTelemetryAdopting OpenTelemetry
Adopting OpenTelemetry
 
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
 
Disenchantment: Netflix Titus, Its Feisty Team, and Daemons
Disenchantment: Netflix Titus, Its Feisty Team, and DaemonsDisenchantment: Netflix Titus, Its Feisty Team, and Daemons
Disenchantment: Netflix Titus, Its Feisty Team, and Daemons
 
Manage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with ObservabilityManage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with Observability
 
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)
 
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
 
Fluentd and docker monitoring
Fluentd and docker monitoringFluentd and docker monitoring
Fluentd and docker monitoring
 
from Docker to Moby and back. what changed ?
from Docker to Moby and back. what changed ?from Docker to Moby and back. what changed ?
from Docker to Moby and back. what changed ?
 
OpenNTF - The Lotus Notes and Domino Open Source Organization
OpenNTF - The Lotus Notes and Domino Open Source OrganizationOpenNTF - The Lotus Notes and Domino Open Source Organization
OpenNTF - The Lotus Notes and Domino Open Source Organization
 
12 Factor App Methodology
12 Factor App Methodology12 Factor App Methodology
12 Factor App Methodology
 
Docker Enterprise Deployment Planning
Docker Enterprise Deployment PlanningDocker Enterprise Deployment Planning
Docker Enterprise Deployment Planning
 
Strategy, planning and governance for enterprise deployments of containers - ...
Strategy, planning and governance for enterprise deployments of containers - ...Strategy, planning and governance for enterprise deployments of containers - ...
Strategy, planning and governance for enterprise deployments of containers - ...
 

Recently uploaded

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 

Recently uploaded (20)

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 

The State of Logging on Docker

  • 1. The State of Logging on Docker March 26, 2015 1
  • 2. Housekeeping © 2015 Logentries. All Rights Reserved. Confidential. 2 •  Please ask questions via the chat pane •  All registrants will receive a recorded copy of the webinar following the presentation. •  Follow us @Logentries
  • 3. Meet The Presenters © 2015 Logentries. All Rights Reserved. Confidential. 3 Trevor Parsons, PhD Logentries Co-founder & Chief Scientist Peter Elger NearForm Director of Engineering Bright Fulton Swipely Director of Engineering Operations
  • 4. Webinar Outline © 2015 Logentries. All Rights Reserved. Confidential. 4 •  Challenges with monitoring micro-services •  What should you be logging? •  The State of Logging on Docker •  Inside the container •  Outside the container •  Specialized containers •  Useful APIs •  Common Questions •  What about CoreOS?
  • 5. Monitoring Micro-Services © 2015 Logentries. All Rights Reserved. Confidential. 5
  • 6. Containers: Ephemeral in Nature © 2015 Logentries. All Rights Reserved. Confidential. 6
  • 7. What to Log? © 2015 Logentries. All Rights Reserved. Confidential. 7 Simple best practices for what to log: •  When the event occurred: the timestamp •  Structured Logs: JSON, KVP •  Context: if someone bought a book… what book was it… •  Unique Identifiers: request ID, User ID, Session ID, thread ID •  Host Info: hostname & IP •  Container Info: Image ID, Container ID, Process ID
  • 8. Logging on Docker: No Standard Approach © 2015 Logentries. All Rights Reserved. Confidential. 8 Docker Logging Support includes: •  Capturing container stdout logs •  Access via Docker_remote_api •  Access via Docker CLI •  $ docker logs [ID] •  Will fetch the logs from a container •  can be used to tail also •  $ docker logs –tf –tail [ID]
  • 9. Logging on Docker: No Standard Approach © 2015 Logentries. All Rights Reserved. Confidential. 9 •  Inside the Container •  Log from the application •  Run a logging Daemon/collector within the container •  Outside the Container (on the host) •  Containers write logs to the host •  Collector runs on the host •  Logging Container •  Collector runs in a specialized container
  • 10. Inside the Container © 2015 Logentries. All Rights Reserved. Confidential. 10 SERVER LOG MANAGEMENT & ANALYTICS PLATFORMHOST OS DOCKER LOGSLOGS APP APP CONTAINER 1 CONTAINER 2 Daemon Daemon
  • 11. Inside the Container © 2015 Logentries. All Rights Reserved. Confidential. 11 •  Logging libs (log from your app code) •  Install a collector inside a container •  Use syslog inside a container
  • 12. Outside the Container (on host) © 2015 Logentries. All Rights Reserved. Confidential. 12 SERVER LOG MANAGEMENT & ANALYTICS PLATFORM HOST OS DOCKER CONTAINER 1 CONTAINER 2 APP APP LOGS LOGS LOGSAGENT
  • 13. Outside the container (on host) © 2015 Logentries. All Rights Reserved. Confidential. 13 •  Configure a container to bind to a volume & write logs to the volume •  Write logs to stdout •  Collector runs on the host
  • 14. Logging Containers © 2015 Logentries. All Rights Reserved. Confidential. 14 SERVER LOG MANAGEMENT & ANALYTICS PLATFORMHOST OS DOCKER CONTAINER 1 CONTAINER 2 LOGGING CONTAINER APP APP Agent LOGS LOGS LOGS
  • 15. Logging Containers © 2015 Logentries. All Rights Reserved. Confidential. 15 •  Syslog in a container •  Collector in a container •  E.g. Logentries container will collect Stats and Logs •  https://github.com/nearform/docker-logentries •  Logspout: https://github.com/gliderlabs/logspout
  • 16. Common Questions © 2015 Logentries. All Rights Reserved. Confidential. 16 •  Do I need to rotate any logs? •  Is there a big performance hit? •  E.g. collecting metrics from every container •  Any security concerns?
  • 17. What about CoreOS? © 2015 Logentries. All Rights Reserved. Confidential. 17 •  Enables warehouse-scale computing •  Used for managing clusters of containers •  Distributed process manager, Fleet (based on systemd) •  etcd for service discovery and global storage of your config settings •  How you can log from your CoreOS Clusters •  Journal - a modern logging system (e.g. JSON export, forward sealing) •  Journal2Logentries: https://github.com/kelseyhightower/journal-2-logentries
  • 18. Useful Resources: © 2015 Logentries. All Rights Reserved. Confidential. 18 •  Docker CLI •  https://docs.docker.com/reference/commandline/cli/ •  Docker Remote API •  https://docs.docker.com/reference/api/docker_remote_api/ •  Stats API (From Docker 1.5) •  https://github.com/docker/docker/pull/9984 (e.g. used by Logentries container) •  Docker RunTime Metrics •  Uses the native cgroup accounting metrics •  https://docs.docker.com/articles/runmetrics/ (e.g. used by DataDog) •  Docker Ruby Gem •  https://github.com/swipely/docker-api
  • 19. Useful Resources: Blogs © 2015 Logentries. All Rights Reserved. Confidential. 19 •  The State of Logging on Docker •  https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/ •  What is the Docker Stats API •  https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/ •  Docker Log Management Using Fluentd •  http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/ •  How to Centralize Logs from CoreOS Clusters •  https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/
  • 20. Useful Resources: Blogs © 2015 Logentries. All Rights Reserved. Confidential. 20 •  The State of Logging on Docker •  https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/ •  What is the Docker Stats API •  https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/ •  Docker Log Management Using Fluentd •  http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/ •  How to Centralize Logs from CoreOS Clusters •  https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/
  • 21. Our Docker Community Pack © 2015 Logentries. All Rights Reserved. Confidential. 21
  • 22. Getting Started © 2015 Logentries. All Rights Reserved. Confidential. 22 •  Don’t have a Logentries Free Account Yet? •  Sign-up for a free 30-day trial: http://logentries.com/quick-start •  Docker & Node.Js Development: •  http://www.nearform.com/ •  http://nscale.nearform.com/ •  Contact us directly! •  Trevor.parsons@logentries.com •  support@logentries.com •  peter.elger@nearform.com •  Let’s take a few questions!