DockerCon EU 2015: Monitoring Docker
•
9 likes•3,536 views
This document discusses monitoring Docker containers. It provides an overview of Datadog and how it can be used to monitor dynamic infrastructure like Docker. It then discusses the challenges in monitoring Docker environments due to their dynamic and distributed nature. Finally, it outlines different methods for collecting key metrics on Docker containers like CPU, memory, I/O and network metrics using Docker's pseudo-files, stats command and API.
Report
Share
Report
Share
Download to read offline
Recommended
DockerCon EU 2015: Nesting Containers: Real Life Observations
Nested containers provide efficiency benefits over virtual machines by avoiding hardware virtualization overhead and allowing for elastic resizing without downtime. Running Docker containers within system containers implements nested containerization and provides additional tenant isolation through solid Linux container security and resource capping abilities. However, Docker's layered storage model poses challenges for nested environments due to security restrictions on block device mounts. The storage extpoint framework aims to address this by moving storage driver functionality outside of Docker containers to enable better storage solutions, though more work is still needed to fully protect images and manage storage space when containers are removed. Further compatibility issues may also arise as Docker continues to evolve.
Infinit's Next Generation Key-value Store - Julien Quintard and Quentin Hocqu...
Key-value store projects have been widely adopted as a way to store metadata, but also as a low-level construct on top of which can be built more advanced storage solutions from file systems, object storage APIs and more. Unfortunately, most key-value store constructs suffer the same limitations when it comes to scalability, performance, and resilience. Infinit's key-value store takes a different approach, relying on a decentralized architecture rather than a master/slave model while offering strong consistency.
Take an Analytics-driven Approach to Container Performance with Splunk for Co...
Docker containers add portability but can also introduce complexity into your environment. In this session learn about why monitoring your container environment is essential to maintaining service reliability, and how Splunk software can help you monitor different layers of infrastructure running in a Docker environment, including third-party tools, instances, and custom code.
Learn how to use Splunk software to collect, search and correlate container data with other infrastructure data for better service context, root cause monitoring and reporting. Additionally, receive introduction to the product integrations between Splunk and Docker such as the Splunk Logging Driver, Splunk Forwarder, and Splunk Logging Libraries.
Practical Container Security by Mrunal Patel and Thomas Cameron, Red Hat
You can secure your containerized microservices without slowing down development. Through a combination of Linux kernel features and open source tools, you can isolate the host from the container and the containers from each other, as well as finding vulnerabilities and securing data. Two of Red Hat's Docker contributors will discuss the state of container security today, covering Linux namespaces, SElinux, cgroups, capabilities, scan, seccomp, and other tools you can use right now.
Production Ready Containers from IBM and Docker
Containers are quickly becoming the default foundation for modern applications. As a public cloud provider, IBM has been an early champion of containers in the cloud and has built an enterprise ready container service as part of IBM Bluemix. IBM has a long heritage of supporting, contributing to, and building offerings on top of open technologies and IBM carries this commitment to the open development of container solutions by being an active/founding member of the Open Containers Initiative and Cloud Native Computing Foundation. In this session, we will explore the enduring commitment to open technology as well as the advantages of using a pure containers service where the user has access to total solution life cycle management through integration of lessons learned, cutting edge enhancements/development and end-to-end support on the user's underlying infrastructure.
We will explore topics such as exploiting bare metal servers, applying overlay networking to containers, ensuring isolation and security in a truly multi-tenant container environment and managing a global service deployment.
DockerCon EU 2015: Docker Universal Control Plane (Gordon's Special Session)
This document discusses Docker Universal Control Plane (UCP), a solution from Docker for deploying and managing containerized applications across teams and environments. UCP provides agility for developers through integrated Docker tools and APIs, while also giving IT operations control, security, and visibility. It enables both developers and IT ops to quickly deploy and run Dockerized apps on any infrastructure. The presentation highlights key UCP capabilities like user management, resource management, security, and its integration with the Docker ecosystem. A demo of UCP is offered at the end.
Securing your Containers
Learn best practices in container security to make your containers seaworthy through the build, ship, and run lifecycle.
Demos temporarily living at github.com/endophage/apps (look under wordpress dir)
Introduction to Docker - 2017
This document provides an introduction to Docker and discusses how it helps address challenges in the modern IT landscape. Some key points:
- Applications are increasingly being broken up into microservices and deployed across multiple servers and environments, making portability and scalability important.
- Docker containers help address these issues by allowing applications to run reliably across different infrastructures through package dependencies and resources together. This improves portability.
- Docker provides a platform for building, shipping and running applications. It helps bridge the needs of developers who want fast innovation and operations teams who need security and control.
Recommended
DockerCon EU 2015: Nesting Containers: Real Life Observations
Nested containers provide efficiency benefits over virtual machines by avoiding hardware virtualization overhead and allowing for elastic resizing without downtime. Running Docker containers within system containers implements nested containerization and provides additional tenant isolation through solid Linux container security and resource capping abilities. However, Docker's layered storage model poses challenges for nested environments due to security restrictions on block device mounts. The storage extpoint framework aims to address this by moving storage driver functionality outside of Docker containers to enable better storage solutions, though more work is still needed to fully protect images and manage storage space when containers are removed. Further compatibility issues may also arise as Docker continues to evolve.
Infinit's Next Generation Key-value Store - Julien Quintard and Quentin Hocqu...
Key-value store projects have been widely adopted as a way to store metadata, but also as a low-level construct on top of which can be built more advanced storage solutions from file systems, object storage APIs and more. Unfortunately, most key-value store constructs suffer the same limitations when it comes to scalability, performance, and resilience. Infinit's key-value store takes a different approach, relying on a decentralized architecture rather than a master/slave model while offering strong consistency.
Take an Analytics-driven Approach to Container Performance with Splunk for Co...
Docker containers add portability but can also introduce complexity into your environment. In this session learn about why monitoring your container environment is essential to maintaining service reliability, and how Splunk software can help you monitor different layers of infrastructure running in a Docker environment, including third-party tools, instances, and custom code.
Learn how to use Splunk software to collect, search and correlate container data with other infrastructure data for better service context, root cause monitoring and reporting. Additionally, receive introduction to the product integrations between Splunk and Docker such as the Splunk Logging Driver, Splunk Forwarder, and Splunk Logging Libraries.
Practical Container Security by Mrunal Patel and Thomas Cameron, Red Hat
You can secure your containerized microservices without slowing down development. Through a combination of Linux kernel features and open source tools, you can isolate the host from the container and the containers from each other, as well as finding vulnerabilities and securing data. Two of Red Hat's Docker contributors will discuss the state of container security today, covering Linux namespaces, SElinux, cgroups, capabilities, scan, seccomp, and other tools you can use right now.
Production Ready Containers from IBM and Docker
Containers are quickly becoming the default foundation for modern applications. As a public cloud provider, IBM has been an early champion of containers in the cloud and has built an enterprise ready container service as part of IBM Bluemix. IBM has a long heritage of supporting, contributing to, and building offerings on top of open technologies and IBM carries this commitment to the open development of container solutions by being an active/founding member of the Open Containers Initiative and Cloud Native Computing Foundation. In this session, we will explore the enduring commitment to open technology as well as the advantages of using a pure containers service where the user has access to total solution life cycle management through integration of lessons learned, cutting edge enhancements/development and end-to-end support on the user's underlying infrastructure.
We will explore topics such as exploiting bare metal servers, applying overlay networking to containers, ensuring isolation and security in a truly multi-tenant container environment and managing a global service deployment.
DockerCon EU 2015: Docker Universal Control Plane (Gordon's Special Session)
This document discusses Docker Universal Control Plane (UCP), a solution from Docker for deploying and managing containerized applications across teams and environments. UCP provides agility for developers through integrated Docker tools and APIs, while also giving IT operations control, security, and visibility. It enables both developers and IT ops to quickly deploy and run Dockerized apps on any infrastructure. The presentation highlights key UCP capabilities like user management, resource management, security, and its integration with the Docker ecosystem. A demo of UCP is offered at the end.
Securing your Containers
Learn best practices in container security to make your containers seaworthy through the build, ship, and run lifecycle.
Demos temporarily living at github.com/endophage/apps (look under wordpress dir)
Introduction to Docker - 2017
This document provides an introduction to Docker and discusses how it helps address challenges in the modern IT landscape. Some key points:
- Applications are increasingly being broken up into microservices and deployed across multiple servers and environments, making portability and scalability important.
- Docker containers help address these issues by allowing applications to run reliably across different infrastructures through package dependencies and resources together. This improves portability.
- Docker provides a platform for building, shipping and running applications. It helps bridge the needs of developers who want fast innovation and operations teams who need security and control.
DockerCon 16 General Session Day 1
This document discusses Docker and its goals of making development and operations tools easier to use. It outlines Docker's focus on three areas: 1) improving the developer experience by creating tools that are simple and intuitive, 2) building orchestration capabilities directly into Docker to simplify managing multiple containers, and 3) enhancing the operations experience through deep integration with cloud platforms and a new application bundle format. The document promotes Docker 1.12 which includes built-in orchestration functionality and announces public betas of Docker for Mac and Windows.
Securing the Container Pipeline at Salesforce by Cem Gurkok
Customer trust and security is paramount for Salesforce. While containerization is great for DevOps due to flexibility, speed, isolation, transient existence, ease of management and patching, it becomes a challenging environment when the sensitivity level of the data traversing the environment increases. Monitoring systems, applications and network; performing disk, memory and network forensics in case of an incident; and vulnerability detection can easily become daunting tasks in such a volatile environment.
In this presentation we would like to discuss the infrastructure we have built to address these issues and to secure our Docker container platform while we rapidly containerize Salesforce. Our solutions focus on securing the container pipeline, building security into the architecture, monitoring, Docker forensics (disk, memory, network), and automation. We also would like to demonstrate some of our live memory analysis capabilities we leverage to assure container and application integrity during execution.
Online Meetup: Intro to LinuxKit
At DockerCon 2017 we introduced LinuxKit: A toolkit for building secure, lean and portable Linux subsystems. In this Online Meetup, we'll provide an introduction and some hands on examples on how to use LinuxKit.
Speaker Biography: Rolf Neugebauer is a Member of the Technical Staff at Docker. He is one of the LinuxKit developers and previously worked on Docker for Mac and Windows. Rolf tweets at @neugebar
Highly Available Persistent Applications in Containers by Kendrick Coleman, E...
Persistent applications are typically last on the list when it comes to container strategy, but the benefits that containers bring to general applications can go beyond what most recognize. Adding persistence extends the types of applications that can be containerized and opens the door to new opportunities for operating these applications. Getting there requires the right container platform that includes awareness of storage at all levels. Learn how they both play a critical role in making sure your new container strategy is inclusive of all applications.
DockerCon EU 2015: Day 1 General Session
DockerCon EU 2015: Day 1 General Session
More information on the Docker announcements: http://blog.docker.com/2015/11/dockercon-eu-2015-day-1/
Docker Roadshow 2016
This presentation is from the 2016 Enterprise Roadshow series in North America and Europe. This presentation explains the Docker enterprise solution including Containers as a Service workflows powered by Docker Datacenter and the integration with HPE to deliver a container platform on hybrid cloud infrastructure.
Learn more: www.docker.com/enterprise
DockerCon SF 2015: DHE/DTR
Docker Trusted Registry is a private registry for Docker images that provides key features like role-based access control, audit logs, and storage drivers. It has a modular architecture with components like registry servers, load balancers, and authentication servers. It can be deployed on any infrastructure and supports integration with directory services. Docker Trusted Registry comes in different editions for teams of various sizes and needs and pricing starts from free to $150 per month depending on the edition.
DCSF19 Containerized Databases for Enterprise Applications
Containerized Databases for Enterprise Applications
Containers are now being used in organizations of all sizes. From small startups to established enterprises, data persistence is necessary in many mission critical applications. “Containers are not for database applications” is a misconception and nothing could be further from the truth.
This session aims to help practitioners navigate the minefield of database containerization and avoid some of the major pitfalls that can occur. Discussion includes traditional enterprise database concerns surrounding data persistence and data security, and how they mesh with containerized deployment.
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
This document summarizes the experiences of two software engineering teams at Cornell University in migrating their applications to Docker containers. The first team dockerized the university's central financial system (KFS) to enable easier local development and automated testing/deployment. The second team built a new research analytics dashboard from the ground up using Docker to containerize the front-end, API, and data processing components. Both projects saw significant benefits from standardized environments and workflows using Docker, including faster setup for new developers, consistent environments, and easier continuous integration/deployment.
Docker, Innovation Accelerator
Sailabove is a container as a service offering from OVH Group that provides features like private registries, networks, volumes, load balancing, and log/metric collection. It is used internally by OVH for their code engine and microservices and allows customers to deploy custom code in containers for processing IoT sensor data and building IoT platforms as a service with on-demand resources. Sailabove is part of OVH's focus on containers and leverages their experience with Docker and containers internally over the past few years.
Docker Enterprise Edition: Building a Secure Supply Chain for the Enterprise ...
Learn from the development team as we dive into some of the latest and upcoming features in Docker EE, our enterprise container management solution. We will focus on the architecture and configuration of the features and how they can be used with both modern apps and containerized legacy apps. Stay for some tips on monitoring and troubleshooting to help you prevent your production environment from going sideways.
Proof of Concept: Serverless with Swarm by Nirmal Mehta, Booz Allen Hamilton
The document discusses serverless computing with Docker containers. It provides definitions of serverless and functions-as-a-service. It then summarizes several serverless platforms like Serverless.com, Iron.io IronWorker, and iopipe Dockaless and how they integrate Docker containers or allow running code on multiple clouds. It demonstrates running a simple Docker container function locally using the Dockaless library. In conclusion, while serverless is still early, Docker containers provide a way to build serverless applications that are infrastructure agnostic.
Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov
The document discusses how modern hardware has become more complex with multi-core, multi-socket CPUs and deep cache hierarchies. This complexity introduces latency and performance issues for software. The author describes their service that processes millions of requests per second spending a large amount of time on garbage collection, context switching, and CPU stalls. They developed a tool called Tesson that analyzes hardware topology and shards containerized applications across CPU cores, pinning linked components closer together to improve locality and performance. Tesson integrates with a local load balancer to distribute workloads efficiently utilizing the system resources.
Journey to Docker Production: Evolving Your Infrastructure and Processes - Br...
DevOps in the Real World is far from perfect, and we're all somewhere on the path to one day writing that "Amazing-Hacker-News-Post about your chat-bot fully-automated micro-service infrastructure." But until then, how can you *really* start using containers today, in meaningful ways that impact yours and your customers productivity? This session is designed for practitioners who are looking for ways to get started now with Docker and Swarm in production. No Docker 101 here, this is for helping you be successful on your way to Dockerizing your production systems. Attendees will get tactics, example configs, real working infrastructure designs, and see the (sometimes messy) internals of Docker in production today.
Docker Container Lifecycles, Problem or Opportunity? by Baruch Sadogursky, JFrog
Docker is hot. However, as Docker container use spreads into more mature production pipelines, there can be issues about control of Docker images to ensure they are production-ready. Is a promotion-based model appropriate to control and track the flow of Docker images from development to production? We will demonstrate how to implement a promotion model for docker images, and then show how to distribute them to any kind of consumer, being it a customer or a data center.
Docker Datacenter Overview and Production Setup Slides
An overview on Docker Data Center and Universal Control Plane. We will cover how to install for production and integrate Docker Trusted Registry.
Led by DDC + UCP Champ:
Vivek Saraswat
Experience Level: Attendees need no prior experience with Docker, but should be familiar with basic linux command-line.
Docker Datacenter - CaaS
This was the deck I presented for a meetup organized by Software Circus.
Docker Datacenter (DDC) delivers Containers as a Service (CaaS) for enterprises to build, ship and run any application anywhere. With an integrated technology platform that spans across the application lifecycle with tooling and support for both developers and IT operations, Docker Datacenter delivers a secure software supply chain at enterprise scale. Join this talk to understand how DDC delivers CaaS, and hear examples of customer who have adopted DDC and their journey with it. A live demo will conclude the presentation.
Full video here:
https://www.youtube.com/watch?v=qboZCZfb0mc
Back to the Future: Containerize Legacy Applications - Rob Tanner, Northern T...
People typically think of Docker for microservices and try to make the smallest container they can. There are tremendous benefits to a microservices model but those are not the only apps that qualify for containers. Traditional, homegrown, monolithic apps are also great candidates for Docker - why? By containerizing these apps, many of the same agility, portability, security and cost savings benefits can be applied to the hundreds (if not thousands) of apps in your datacenters. But where to begin? Attend this session to learn how to approach modernizing traditional apps (MTA), considerations, the available tools and possibilities.
DCSF19 Container Security: Theory & Practice at Netflix
Michael Wardrop, Netflix
Usage of containers has undergone rapid growth at Netflix and it is still accelerating. Our container story started organically with developers downloading Docker and using it to improve their developer experience. The first production workloads were simple batch jobs, pioneering micro-services followed, then status as a first class platform running critical workloads.
As the types of workloads changed and their importance increased, the security of our container ecosystem needed to evolve and adapt. This session will cover some security theory, architecture, along with practical considerations, and lessons we learnt along the way.
Dell Trials and Triumphs using Docker on Client Systems by Sean McGinnis and ...
Join engineers from ClusterHQ and Dell Storage to learn about persistent storage integration with Docker. See how Flocker enables native support for enterprise storage infrastructure use in containers, enabling you to leverage Dell storage for new workloads.
Monitoring docker containers and dockerized applications
This document summarizes an approach for monitoring Docker containers and dockerized applications. It outlines challenges with monitoring at the container level, including differentiating metrics between linked containers and collecting data without overloading containers. It then describes a proposed design that involves monitoring at the host, container, and application levels using a driver-based agent approach. Key aspects of the design include abstracting monitoring methods, intelligently correlating data between levels, and including proactive, reactive, and adaptive monitoring strategies.
Measuring Micro-services. Richard Rodger
Microservice systems have emergent properties related to message flow that can be measured. Key measures include message flow rates which indicate system health, and detecting changes after deploying new services. Useful message patterns to measure include actors, subscribers, and chains, which have predictable relationships between messages that can serve as invariants. Tracking these measures and invariants over time helps validate the system and detect potential problems.
More Related Content
What's hot
DockerCon 16 General Session Day 1
This document discusses Docker and its goals of making development and operations tools easier to use. It outlines Docker's focus on three areas: 1) improving the developer experience by creating tools that are simple and intuitive, 2) building orchestration capabilities directly into Docker to simplify managing multiple containers, and 3) enhancing the operations experience through deep integration with cloud platforms and a new application bundle format. The document promotes Docker 1.12 which includes built-in orchestration functionality and announces public betas of Docker for Mac and Windows.
Securing the Container Pipeline at Salesforce by Cem Gurkok
Customer trust and security is paramount for Salesforce. While containerization is great for DevOps due to flexibility, speed, isolation, transient existence, ease of management and patching, it becomes a challenging environment when the sensitivity level of the data traversing the environment increases. Monitoring systems, applications and network; performing disk, memory and network forensics in case of an incident; and vulnerability detection can easily become daunting tasks in such a volatile environment.
In this presentation we would like to discuss the infrastructure we have built to address these issues and to secure our Docker container platform while we rapidly containerize Salesforce. Our solutions focus on securing the container pipeline, building security into the architecture, monitoring, Docker forensics (disk, memory, network), and automation. We also would like to demonstrate some of our live memory analysis capabilities we leverage to assure container and application integrity during execution.
Online Meetup: Intro to LinuxKit
At DockerCon 2017 we introduced LinuxKit: A toolkit for building secure, lean and portable Linux subsystems. In this Online Meetup, we'll provide an introduction and some hands on examples on how to use LinuxKit.
Speaker Biography: Rolf Neugebauer is a Member of the Technical Staff at Docker. He is one of the LinuxKit developers and previously worked on Docker for Mac and Windows. Rolf tweets at @neugebar
Highly Available Persistent Applications in Containers by Kendrick Coleman, E...
Persistent applications are typically last on the list when it comes to container strategy, but the benefits that containers bring to general applications can go beyond what most recognize. Adding persistence extends the types of applications that can be containerized and opens the door to new opportunities for operating these applications. Getting there requires the right container platform that includes awareness of storage at all levels. Learn how they both play a critical role in making sure your new container strategy is inclusive of all applications.
DockerCon EU 2015: Day 1 General Session
DockerCon EU 2015: Day 1 General Session
More information on the Docker announcements: http://blog.docker.com/2015/11/dockercon-eu-2015-day-1/
Docker Roadshow 2016
This presentation is from the 2016 Enterprise Roadshow series in North America and Europe. This presentation explains the Docker enterprise solution including Containers as a Service workflows powered by Docker Datacenter and the integration with HPE to deliver a container platform on hybrid cloud infrastructure.
Learn more: www.docker.com/enterprise
DockerCon SF 2015: DHE/DTR
Docker Trusted Registry is a private registry for Docker images that provides key features like role-based access control, audit logs, and storage drivers. It has a modular architecture with components like registry servers, load balancers, and authentication servers. It can be deployed on any infrastructure and supports integration with directory services. Docker Trusted Registry comes in different editions for teams of various sizes and needs and pricing starts from free to $150 per month depending on the edition.
DCSF19 Containerized Databases for Enterprise Applications
Containerized Databases for Enterprise Applications
Containers are now being used in organizations of all sizes. From small startups to established enterprises, data persistence is necessary in many mission critical applications. “Containers are not for database applications” is a misconception and nothing could be further from the truth.
This session aims to help practitioners navigate the minefield of database containerization and avoid some of the major pitfalls that can occur. Discussion includes traditional enterprise database concerns surrounding data persistence and data security, and how they mesh with containerized deployment.
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
This document summarizes the experiences of two software engineering teams at Cornell University in migrating their applications to Docker containers. The first team dockerized the university's central financial system (KFS) to enable easier local development and automated testing/deployment. The second team built a new research analytics dashboard from the ground up using Docker to containerize the front-end, API, and data processing components. Both projects saw significant benefits from standardized environments and workflows using Docker, including faster setup for new developers, consistent environments, and easier continuous integration/deployment.
Docker, Innovation Accelerator
Sailabove is a container as a service offering from OVH Group that provides features like private registries, networks, volumes, load balancing, and log/metric collection. It is used internally by OVH for their code engine and microservices and allows customers to deploy custom code in containers for processing IoT sensor data and building IoT platforms as a service with on-demand resources. Sailabove is part of OVH's focus on containers and leverages their experience with Docker and containers internally over the past few years.
Docker Enterprise Edition: Building a Secure Supply Chain for the Enterprise ...
Learn from the development team as we dive into some of the latest and upcoming features in Docker EE, our enterprise container management solution. We will focus on the architecture and configuration of the features and how they can be used with both modern apps and containerized legacy apps. Stay for some tips on monitoring and troubleshooting to help you prevent your production environment from going sideways.
Proof of Concept: Serverless with Swarm by Nirmal Mehta, Booz Allen Hamilton
The document discusses serverless computing with Docker containers. It provides definitions of serverless and functions-as-a-service. It then summarizes several serverless platforms like Serverless.com, Iron.io IronWorker, and iopipe Dockaless and how they integrate Docker containers or allow running code on multiple clouds. It demonstrates running a simple Docker container function locally using the Dockaless library. In conclusion, while serverless is still early, Docker containers provide a way to build serverless applications that are infrastructure agnostic.
Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov
The document discusses how modern hardware has become more complex with multi-core, multi-socket CPUs and deep cache hierarchies. This complexity introduces latency and performance issues for software. The author describes their service that processes millions of requests per second spending a large amount of time on garbage collection, context switching, and CPU stalls. They developed a tool called Tesson that analyzes hardware topology and shards containerized applications across CPU cores, pinning linked components closer together to improve locality and performance. Tesson integrates with a local load balancer to distribute workloads efficiently utilizing the system resources.
Journey to Docker Production: Evolving Your Infrastructure and Processes - Br...
DevOps in the Real World is far from perfect, and we're all somewhere on the path to one day writing that "Amazing-Hacker-News-Post about your chat-bot fully-automated micro-service infrastructure." But until then, how can you *really* start using containers today, in meaningful ways that impact yours and your customers productivity? This session is designed for practitioners who are looking for ways to get started now with Docker and Swarm in production. No Docker 101 here, this is for helping you be successful on your way to Dockerizing your production systems. Attendees will get tactics, example configs, real working infrastructure designs, and see the (sometimes messy) internals of Docker in production today.
Docker Container Lifecycles, Problem or Opportunity? by Baruch Sadogursky, JFrog
Docker is hot. However, as Docker container use spreads into more mature production pipelines, there can be issues about control of Docker images to ensure they are production-ready. Is a promotion-based model appropriate to control and track the flow of Docker images from development to production? We will demonstrate how to implement a promotion model for docker images, and then show how to distribute them to any kind of consumer, being it a customer or a data center.
Docker Datacenter Overview and Production Setup Slides
An overview on Docker Data Center and Universal Control Plane. We will cover how to install for production and integrate Docker Trusted Registry.
Led by DDC + UCP Champ:
Vivek Saraswat
Experience Level: Attendees need no prior experience with Docker, but should be familiar with basic linux command-line.
Docker Datacenter - CaaS
This was the deck I presented for a meetup organized by Software Circus.
Docker Datacenter (DDC) delivers Containers as a Service (CaaS) for enterprises to build, ship and run any application anywhere. With an integrated technology platform that spans across the application lifecycle with tooling and support for both developers and IT operations, Docker Datacenter delivers a secure software supply chain at enterprise scale. Join this talk to understand how DDC delivers CaaS, and hear examples of customer who have adopted DDC and their journey with it. A live demo will conclude the presentation.
Full video here:
https://www.youtube.com/watch?v=qboZCZfb0mc
Back to the Future: Containerize Legacy Applications - Rob Tanner, Northern T...
People typically think of Docker for microservices and try to make the smallest container they can. There are tremendous benefits to a microservices model but those are not the only apps that qualify for containers. Traditional, homegrown, monolithic apps are also great candidates for Docker - why? By containerizing these apps, many of the same agility, portability, security and cost savings benefits can be applied to the hundreds (if not thousands) of apps in your datacenters. But where to begin? Attend this session to learn how to approach modernizing traditional apps (MTA), considerations, the available tools and possibilities.
DCSF19 Container Security: Theory & Practice at Netflix
Michael Wardrop, Netflix
Usage of containers has undergone rapid growth at Netflix and it is still accelerating. Our container story started organically with developers downloading Docker and using it to improve their developer experience. The first production workloads were simple batch jobs, pioneering micro-services followed, then status as a first class platform running critical workloads.
As the types of workloads changed and their importance increased, the security of our container ecosystem needed to evolve and adapt. This session will cover some security theory, architecture, along with practical considerations, and lessons we learnt along the way.
Dell Trials and Triumphs using Docker on Client Systems by Sean McGinnis and ...
Join engineers from ClusterHQ and Dell Storage to learn about persistent storage integration with Docker. See how Flocker enables native support for enterprise storage infrastructure use in containers, enabling you to leverage Dell storage for new workloads.
What's hot (20)
Securing the Container Pipeline at Salesforce by Cem Gurkok
Securing the Container Pipeline at Salesforce by Cem Gurkok
Highly Available Persistent Applications in Containers by Kendrick Coleman, E...
Highly Available Persistent Applications in Containers by Kendrick Coleman, E...
DCSF19 Containerized Databases for Enterprise Applications
DCSF19 Containerized Databases for Enterprise Applications
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
Docker Enterprise Edition: Building a Secure Supply Chain for the Enterprise ...
Docker Enterprise Edition: Building a Secure Supply Chain for the Enterprise ...
Proof of Concept: Serverless with Swarm by Nirmal Mehta, Booz Allen Hamilton
Proof of Concept: Serverless with Swarm by Nirmal Mehta, Booz Allen Hamilton
Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov
Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov
Journey to Docker Production: Evolving Your Infrastructure and Processes - Br...
Journey to Docker Production: Evolving Your Infrastructure and Processes - Br...
Docker Container Lifecycles, Problem or Opportunity? by Baruch Sadogursky, JFrog
Docker Container Lifecycles, Problem or Opportunity? by Baruch Sadogursky, JFrog
Docker Datacenter Overview and Production Setup Slides
Docker Datacenter Overview and Production Setup Slides
Back to the Future: Containerize Legacy Applications - Rob Tanner, Northern T...
Back to the Future: Containerize Legacy Applications - Rob Tanner, Northern T...
DCSF19 Container Security: Theory & Practice at Netflix
DCSF19 Container Security: Theory & Practice at Netflix
Dell Trials and Triumphs using Docker on Client Systems by Sean McGinnis and ...
Dell Trials and Triumphs using Docker on Client Systems by Sean McGinnis and ...
Viewers also liked
Monitoring docker containers and dockerized applications
This document summarizes an approach for monitoring Docker containers and dockerized applications. It outlines challenges with monitoring at the container level, including differentiating metrics between linked containers and collecting data without overloading containers. It then describes a proposed design that involves monitoring at the host, container, and application levels using a driver-based agent approach. Key aspects of the design include abstracting monitoring methods, intelligently correlating data between levels, and including proactive, reactive, and adaptive monitoring strategies.
Measuring Micro-services. Richard Rodger
Microservice systems have emergent properties related to message flow that can be measured. Key measures include message flow rates which indicate system health, and detecting changes after deploying new services. Useful message patterns to measure include actors, subscribers, and chains, which have predictable relationships between messages that can serve as invariants. Tracking these measures and invariants over time helps validate the system and detect potential problems.
Monitoring Docker containers - Docker NYC Feb 2015
Alexis goals this presentation are three-fold:
1) Dive into key Docker metrics
2) Explain operational complexity. In other words I want to take what we have seen on the field and show you where the pain points will be.
3) Rethink monitoring of Docker containers. The old tricks won’t work.
ContainerDays NYC 2016: "Observability and Manageability in a Container Envir...
- The document discusses observability in containerized systems and microservices. It provides examples of logging output from containerized applications like Nginx and how to parse the logs to extract useful fields.
- The examples show how Docker log drivers can mangle structured logs by adding metadata, and suggest directly logging from applications to collectors.
- Configurations for Logstash are shown to parse logs from Nginx and other applications running in containers in order to extract fields like timestamps, requests IDs, and HTTP information for further analysis.
Voxxed Days Thessaloniki 2016 - Microservices in production
This document discusses best practices for running microservices in production. It advocates for a metrics-first approach and using containers and Kubernetes for infrastructure. Key points include collecting standard metrics in Kubernetes for visibility, packaging apps with all dependencies like metrics endpoints, and taking an iterative approach to continuously evolve systems through experimentation, measurement, and learning from experience.
BFF Pattern in Action: SoundCloud’s Microservices
At SoundCloud we managed to break away from the monolith while delivering key business features. Our journey towards a microservices architecture has not been a straightforward one. We experimented a lot to reach the set of tools and technologies that we use today. We changed how we build our applications. We introduced specific apis for our mobile and web clients. We call them BFFs (backend for the frontend). They became the central piece of SoundCloud’s architecture. We rethought how we monitor our services. We created a service registry for knowledge sharing. While making all these changes, we benefited from the learnings of our peer companies. This talk will share our learnings from this journey: what worked for us and what we moved away from.
2008 "An overview of Methods for analysis of Identifiability and Observabilit...
The document discusses various methods for analyzing identifiability and observability in nonlinear state and parameter estimation models, which are important concepts in system identification. It describes methods like local sensitivity analysis, empirical observability Gramians, asymptotic analysis, and the Alternating Conditional Expectation algorithm for testing structural and practical identifiability. It also discusses differential algebra approaches for testing observability in nonlinear systems.
Microservice Architecture
The document discusses microservice architecture and compares it to monolithic architecture. It describes microservices as small, discrete, isolated services that can be deployed separately. A monolith is a single application combining all business logic and data access. The document outlines characteristics of microservices such as single responsibility, statelessness, independent data management and communication through APIs or message queues. It also covers deployment, testing, monitoring, metrics and the need for automation and a culture open to change when using microservice architecture.
(APP309) Running and Monitoring Docker Containers at Scale | AWS re:Invent 2014
If you have tried Docker but are unsure about how to run it at scale, you will benefit from this session. Like virtualization before, containerization (à; la Docker) is increasing the elastic nature of cloud infrastructure by an order of magnitude. But maybe you still have questions: How many containers can you run on a given Amazon EC2 instance type? Which metric should you look at to measure contention? How do you manage fleets of containers at scale?
Datadog is a monitoring service for IT, operations, and development teams who write and run applications at scale. In this session, the cofounder of Datadog presents the challenges and benefits of running containers at scale and how to use quantitative performance patterns to monitor your infrastructure at this magnitude and complexity. Sponsored by Datadog.
Tracing 2000+ polyglot microservices at Uber with Jaeger and OpenTracing
Slides from my talk & demo at Go NYC Meeetup 19-Jan-2017.
We present Jaeger, Uber’s open source distributed tracing system, featuring Go backend, React based UI, and OpenTracing API support. We show examples of instrumenting application code for tracing and using distributed context propagation to attribute backend resource usage to top level consumers.
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Prometheus is an open-source monitoring system that allows for whitebox monitoring through metrics collected from inside systems and applications. It provides the ability to alert on high-level symptoms, debug issues through customizable dashboards, and perform complex queries across metrics. Prometheus empowers building monitoring that matters through alerting on important business metrics, gaining insight via dashboards, and integrating with other systems via open interfaces.
Monitoring and observability
In this session we’ll leave the need for performance a foregone conclusion and take a whirlwind tour through the complexity of modern Internet architectures. The complexities lead to evil optimization problems and significant challenges troubleshooting production issues to a speedy and successful end.
Starting with the simple facts that you can’t fix what you can’t see and you can’t improve what you can’t measure, we’ll discuss what needs monitoring and why. We’ll talk about unlikely allies in the fight for time and budget to instrument systems, applications and processes for observability.
You’ll leave the session with a better understanding of what it looks like to troubleshoot the storm of a malfunctioning large architecture and some tools and techniques you can use to not be swallowed by the Kraken.
Performance Analysis: The USE Method
Delivered at the FISL13 conference in Brazil: http://www.youtube.com/watch?v=K9w2cipqfvc
This talk introduces the USE Method: a simple strategy for performing a complete check of system performance health, identifying common bottlenecks and errors. This methodology can be used early in a performance investigation to quickly identify the most severe system performance issues, and is a methodology the speaker has used successfully for years in both enterprise and cloud computing environments. Checklists have been developed to show how the USE Method can be applied to Solaris/illumos-based and Linux-based systems.
Many hardware and software resource types have been commonly overlooked, including memory and I/O busses, CPU interconnects, and kernel locks. Any of these can become a system bottleneck. The USE Method provides a way to find and identify these.
This approach focuses on the questions to ask of the system, before reaching for the tools. Tools that are ultimately used include all the standard performance tools (vmstat, iostat, top), and more advanced tools, including dynamic tracing (DTrace), and hardware performance counters.
Other performance methodologies are included for comparison: the Problem Statement Method, Workload Characterization Method, and Drill-Down Analysis Method.
SREcon 2016 Performance Checklists for SREs
Talk from SREcon2016 by Brendan Gregg. Video: https://www.usenix.org/conference/srecon16/program/presentation/gregg . "There's limited time for performance analysis in the emergency room. When there is a performance-related site outage, the SRE team must analyze and solve complex performance issues as quickly as possible, and under pressure. Many performance tools and techniques are designed for a different environment: an engineer analyzing their system over the course of hours or days, and given time to try dozens of tools: profilers, tracers, monitoring tools, benchmarks, as well as different tunings and configurations. But when Netflix is down, minutes matter, and there's little time for such traditional systems analysis. As with aviation emergencies, short checklists and quick procedures can be applied by the on-call SRE staff to help solve performance issues as quickly as possible.
In this talk, I'll cover a checklist for Linux performance analysis in 60 seconds, as well as other methodology-derived checklists and procedures for cloud computing, with examples of performance issues for context. Whether you are solving crises in the SRE war room, or just have limited time for performance engineering, these checklists and approaches should help you find some quick performance wins. Safe flying."
Mobycraft - Docker in 8-bit by Aditya Gupta
Mobycraft is a Minecraft client-side mod to manage and visualize Docker containers in Minecraft. This mod can be installed in any standard Minecraft client and allow young kids to learn Docker fundamentals in a fun way. It allowed a 13-year old boy to apply his Minecraft modding skills to pick up Docker concepts such as Engine, Machine, Swarm, and Remote API.
This project became a great bonding experience between a father and a son. It allowed them to engage in fun and geeky conversations, such as code reviews and tooling discussion, and thereby building memories for a lifetime.
DockerCon SF 2015: Networking Breakout
This document provides an overview of Docker's new networking capabilities through libnetwork. It introduces libnetwork, which provides a pluggable driver-based networking stack for containers. Libnetwork implements the Container Network Model and provides APIs for creating and managing networks and endpoints. It supports multiple networking drivers like bridge and overlay. The goals are to make networking and services first-class objects in Docker and span networks across multiple hosts. The presentation encourages trying the new networking features in Docker experimental and contributing to libnetwork.
A Gentle Introduction to Docker and Containers
This document provides an introduction to Docker and containers. It outlines that Docker is an open source tool that makes it easy to deploy applications by using containers. Containers allow applications to be isolated for easier management and deployment. The document discusses how Docker builds on existing container technologies and provides a standardized way to build, share, and run application containers.
How to Use Your Own Private Registry
This document summarizes a Docker workshop discussing how to set up and use a private Docker registry. It outlines the differences between an index and a registry, and provides instructions for installing a local private registry using Docker or by manually installing dependencies. It then demonstrates how to tag and push images to the private registry, and shows a zero downtime deployment process using the local registry to seamlessly update a running application.
LXC to Docker Via Continuous Delivery
- Three disparate ways of containerization (cgroups, LXC, Docker) evolved over time at the company
- Docker provided a unified standard and improved the workflow, such as pre-building container images ahead of time
- Challenges remained around distributing and storing the large number of container images generated by continuous delivery
Mobycraft:Docker in 8-bit (Meetup at Docker HQ 4/7)
This Minecraft mod called Mobycraft allows players to manage Docker containers from within the game. Containers are represented as boxes that can be started and stopped. Commands like "docker ps" are available in-game. The mod aims to simplify container management and visualization, as well as teach Docker concepts to young players. It's open source on GitHub where the creator welcomes feedback to improve the project.
Viewers also liked (20)
Monitoring docker containers and dockerized applications
Monitoring docker containers and dockerized applications
Monitoring Docker containers - Docker NYC Feb 2015
Monitoring Docker containers - Docker NYC Feb 2015
ContainerDays NYC 2016: "Observability and Manageability in a Container Envir...
ContainerDays NYC 2016: "Observability and Manageability in a Container Envir...
Voxxed Days Thessaloniki 2016 - Microservices in production
Voxxed Days Thessaloniki 2016 - Microservices in production
2008 "An overview of Methods for analysis of Identifiability and Observabilit...
2008 "An overview of Methods for analysis of Identifiability and Observabilit...
(APP309) Running and Monitoring Docker Containers at Scale | AWS re:Invent 2014
(APP309) Running and Monitoring Docker Containers at Scale | AWS re:Invent 2014
Tracing 2000+ polyglot microservices at Uber with Jaeger and OpenTracing
Tracing 2000+ polyglot microservices at Uber with Jaeger and OpenTracing
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Mobycraft:Docker in 8-bit (Meetup at Docker HQ 4/7)
Mobycraft:Docker in 8-bit (Meetup at Docker HQ 4/7)
Similar to DockerCon EU 2015: Monitoring Docker
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Go through the result of our latest large-scale study about Docker usage in real environment. Analyze and see the impact for operations and monitoring.
Monitoring in Motion: Monitoring Containers and Amazon ECS
Containers and other forms of dynamic infrastructure can prove challenging to monitor. How do you define normal, when your infrastructure is intentionally in motion and change from minute to minute? Join us as we discuss proven strategies for monitoring your containerized infrastructure on AWS and ECS.
10 tips for Cloud Native Security
A list of action items you want to keep in mind when you're devsecops'ing for your cloudnative environments. Given as a part of a talk on the Modern Security series (
https://info.signalsciences.com/securing-cloud-native-ten-tips-better-container-security).
You're monitoring Kubernetes Wrong
This document discusses monitoring Kubernetes clusters and provides best practices. It notes that monitoring has become more complex with distributed architectures like microservices and containers. Key challenges discussed are getting monitoring data from containers, making sense of large and varied data, troubleshooting distributed systems, and ensuring monitoring works for people as systems become more distributed. The document demonstrates how Sysdig provides container-level monitoring data and tools for segmenting and troubleshooting that data. It also introduces Sysdig Teams for access control and customization based on organizational teams.
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
Think you have big data? What about high availability
requirements? At DataDog we process billions of data points every day including metrics and events, as we help the world
monitor the their applications and infrastructure. Being the world’s monitoring system is a big responsibility, and thanks to
Redis we are up to the task. Join us as we discuss how the DataDog team monitors and scales Redis to power our SaaS based monitoring offering. We will discuss our usage and deployment patterns, as well as dive into monitoring best practices for production Redis workloads
How to Monitor Microservices
How to Monitor Microservices running in Docker, Kubernetes, Openshift, DC/OS Mesos or any other container orchestration platform.
The eBay-Way Meetup IL - CI/CD with Microservices
Practices and lessons learned for improving your CI/CD pipelines, from the developer's local environment, through the CI server, and up to deployment to production.
Sukumar Nayak-Agile-DevOps-Cloud Management
The document provides an overview of Agile, DevOps and Cloud Management from a security, risk management and audit compliance perspective. It discusses how the IT industry paradigm is shifting towards microservices, containers, continuous delivery and cloud platforms. DevOps is described as development and operations engineers participating together in the entire service lifecycle. Key differences in DevOps include changes to configuration management, release and change management, and event monitoring. Factors for DevOps success include culture, collaboration, eliminating waste, unified processes, tooling and automation.
It's What's Inside that Counts!
This document provides an overview of New Relic's app-centric Docker monitoring capabilities. It begins with introductions of Andrew Marshall, a senior product marketing manager, and Adam Larson, a software engineering manager. The webinar agenda is then outlined, covering why software monitoring is needed, an introduction to New Relic, Docker, and how Docker is used. It then demonstrates Docker monitoring in New Relic before providing steps to get started. Key points about New Relic's Docker monitoring include that it prioritizes useful over impressive information, optimizes for the present since the future is unclear, and fixes issues like not treating containers as first-class nodes.
Tampere Docker meetup - Happy 5th Birthday Docker
Part of official docker meetup events by Docker Inc.
https://events.docker.com/events/docker-bday-5/
Meetup event:
https://www.meetup.com/Docker-Tampere/events/248566945/
Where and When to Docker
This document provides an overview of Docker including what it is, why it's useful, considerations around security, when and where it should be used, and examples of how three customers have implemented Docker. Docker allows for consistent packaging of applications and their environments, improved resource usage, and more rapid deployments. Key benefits include standardization, security, auditability, and bridging the gap between development and operations. Docker is well-suited for web applications, microservices, and environments requiring frequent deployments, while legacy applications may not be the best fit. The document outlines lessons learned around each customer case study.
Containers, microservices and serverless for realists
The document discusses containers, microservices, and serverless applications for developers. It provides an overview of these topics, including how containers and microservices fit into the DevOps paradigm and allow for better collaboration between development and operations teams. It also discusses trends in container usage and orchestration as well as differences between platforms as a service (PaaS) and serverless applications.
Getting Started with Docker - Nick Stinemates
This document summarizes a presentation about Docker and containers. It discusses how applications have changed from monolithic to distributed microservices, creating challenges around managing different stacks and environments. Docker addresses this by providing lightweight containers that package code and dependencies to run consistently on any infrastructure. The presentation outlines how Docker works, its adoption by companies, and its open platform for building, shipping, and running distributed applications. It aims to create an ecosystem similar to how shipping containers standardized cargo transportation globally.
Kubernetes Security
My cloud native security talk I gave at Innotech Austin 2018. I cover container and Kubernetes security topics, security features in Kubernetes, including opensource projects you will want to consider while building and maintaining cloud native applications.
Docker Birthday #5 Meetup Cluj - Presentation
- Docker celebrated its 5th birthday with events worldwide including one in Cluj, Romania. Over 100 user and customer events were held.
- The Docker platform now has over 450 commercial customers, 37 billion container downloads, and 15,000 Docker-related jobs on LinkedIn.
- The event in Cluj included presentations on Docker and hands-on labs to learn Docker, as well as social activities like taking selfies with a birthday banner.
Container DevOps in Azure
This document discusses DevOps practices and container technologies like Docker and Kubernetes. It provides an overview of containers as an application delivery mechanism and how Docker and Kubernetes can be used to deploy and manage containerized applications at scale. It also summarizes key benefits of containers like speed, efficiency, and portability as well as how Docker and Kubernetes address challenges like application deployment, scaling, networking, and reliability.
Continuous Integration with Amazon ECS and Docker
One of the most fundamental challenges of CI/CD is the ability to balance between Quality, Time, and Cost. Amazon EC2 Container Service (ECS), along with Docker and Amazon EC2 Container Registry (ECR), has changed the game for many by making resource management very simple. For Okta, it has enabled the Continuous Integration team to maximize throughput while minimizing cost. In this session we will show you how Okta has created a flexible CI system with ECS, Docker, ECR, AWS Lambda, AWS CloudFormation, Amazon RDS, and Amazon SQS. Okta runs 30,000 tests with each developer commit, and releases 10,000 new lines of code each week to production. The CI system, built 100% on AWS, must be able to handle load while keeping cost under control. This talk is oriented toward developers looking to achieve efficient resource and cost management without compromising speed or quality.
Deep-Dive to Application Insights
This document provides an overview of Application Insights, a telemetry service that monitors application performance and usage. It discusses why application monitoring is useful, how Application Insights collects data from various sources, and how different roles can use the insights. The document then demonstrates Application Insights features like tracking requests, exceptions, dependencies and custom events. It shows how to export telemetry data to Azure Blob Storage and analyze it with Stream Analytics and Power BI.
Applications Performance Monitoring with Applications Manager part 1
Explore the components of a simple web app and then see how Applications Manager helps you discover and map relationships between your apps and take a look at the wide range of business critical metrics monitored for every app/ server in your network. We will also explore how to logically group your apps so as to achieve maximum productivity from your IT department.
Security for devs
This document provides an overview of security best practices for developers. It discusses the software development lifecycle (SDLC) and phases like planning, architecture, testing, and release. It also summarizes Microsoft's recommendations for securing the SDLC, which include training, defining security requirements, threat modeling, using cryptography standards, and regularly penetration testing. The document then covers topics like how HTTP works with different request and response types, common vulnerabilities from the OWASP Top 10, and ways to test applications through penetration testing and bug bounty programs. It provides tips on applying security best practices and knowing about new vulnerabilities, and recommends securing continuous integration/continuous delivery (CI/CD) through techniques like code analysis, container hardening, and
Similar to DockerCon EU 2015: Monitoring Docker (20)
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Docker Usage Patterns - Meetup Docker Paris - November, 10th 2015
Monitoring in Motion: Monitoring Containers and Amazon ECS
Monitoring in Motion: Monitoring Containers and Amazon ECS
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
Containers, microservices and serverless for realists
Containers, microservices and serverless for realists
Applications Performance Monitoring with Applications Manager part 1
Applications Performance Monitoring with Applications Manager part 1
More from Docker, Inc.
Containerize Your Game Server for the Best Multiplayer Experience
Raymond Arifianto, AccelByte and
Mark Mandel, Google -
We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.
How to Improve Your Image Builds Using Advance Docker Build
Nicholas Dille, Haufe-Lexware + Docker Captain -
Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.
Build & Deploy Multi-Container Applications to AWS
Lukonde Mwila, Entelect -
As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud.
In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics:
- Docker Compose
- Containerizing an Nginx Server
- Containerizing an React App
- Containerizing an Node.JS App
- Containerizing anMongoDB App
- Runing Multi-Container App Locally
- Creating a CI/CD Pipeline
- Adding a build stage to test containers and push images to Docker Hub
- Deploying Multi-Container App to AWS Elastic Beanstalk
Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking.
After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.
Securing Your Containerized Applications with NGINX
The document summarizes Kevin Jones' presentation on securing containerized applications with NGINX. It discusses the benefits of using a reverse proxy for security, NGINX best practices for TLS configuration, and deploying NGINX in Docker containers. It also provides code examples and configurations for setting up NGINX as a reverse proxy, optimizing TLS, and using NGINX as a sidecar proxy.
How To Build and Run Node Apps with Docker and Compose
Kathleen Juell, Digital Ocean -
Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself.
This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.
Hands-on Helm
Jessica Deen, Microsoft -
Helm 3 is here; let's go hands-on! In this demo-fueled session, I'll walk you through the differences between Helm 2 and Helm 3. I'll offer tips for a successful rollout or upgrade, go over how to easily use charts created for Helm 2 with Helm 3 (without changing your syntax), and review opportunities where you can participate in the project's future.
Distributed Deep Learning with Docker at Salesforce
Jeff Hajewski, Salesforce -
There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.
The First 10M Pulls: Building The Official Curl Image for Docker Hub
James Fuller, webcomposite s.r.o. -
Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.
Monitoring in a Microservices World
Fabian Stäber, Instana -
In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
Clemente Biondo, Engineering Ingegneria Informatica -
When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.
Predicting Space Weather with Docker
The document discusses how NOAA's Space Weather Prediction Center transitioned from a monolithic architecture to microservices using Docker. It describes how they started with a small verification project, then replaced their critical GOES satellite data source. This improved developers' morale and delivery speed. They encountered some security issues initially but learned from them. The transition was very successful and allowed them to quickly expand their mission to forecast aviation impacts using scientists' models packaged as Docker services.
Become a Docker Power User With Microsoft Visual Studio Code
Brian Christner, 56k + Docker Captain -
In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.
How to Use Mirroring and Caching to Optimize your Container Registry
Brandon Mitchell, Boxboat + Docker Captain -
How do you make your builds more performant? This talk looks at options to configure caching and mirroring of images that you need to save on bandwidth costs and to keep running even if something goes down upstream.
Monolithic to Microservices + Docker = SDLC on Steroids!
Ashish Sharma, SS&C Eze -
SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.
Kubernetes at Datadog Scale
Kubernetes networking can be complex to scale due to issues like growing iptables rules, but newer solutions are helping. Pod networking uses CNI plugins like flannel or Calico to assign each pod an IP and allow communication. Service networking uses kube-proxy and iptables or IPVS for load balancing to pods. DNS is used to resolve service names to IPs. While Kubernetes networking brings flexibility, operators must learn the nuances of their specific CNI plugin and issues can arise, but the ecosystem adapts quickly to new needs and changes don't impact all workloads.
Labels, Labels, Labels
Andy Clemenko, StackRox -
One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Micro Focus uses Docker Hub at scale to support its software delivery and deployment model. Some key points:
- Docker Hub is used as the registry service for Micro Focus container images
- It allows for optimized, secure, reliable and cost-effective software delivery through deployments and updates of container images to customers and partners
- Micro Focus leverages features like private repositories, offline/online access, signing and scanning of images, and integration with CI/CD pipelines
- Over 1,650 organizations, 450 repositories, and 18 teams are used on Docker Hub to manage access and deliver software from Micro Focus
Build & Deploy Multi-Container Applications to AWS
Lukonde Mwila, Entelect
As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud.
In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics:
- Docker Compose
- Containerizing an Nginx Server
- Containerizing an React App
- Containerizing an Node.JS App
- Containerizing anMongoDB App
- Runing Multi-Container App Locally
- Creating a CI/CD Pipeline
- Adding a build stage to test containers and push images to Docker Hub
- Deploying Multi-Container App to AWS Elastic Beanstalk
Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking.
After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
Elton Stoneman, Docker Captain + Container Consultant and Trainer
How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least).
In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product.
There's a lot of technology here from a real world case study, and I'll focus on:
- running Windows apps in Docker containers
- building a .NET Core API which can run in Linux or Windows containers
- running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud
- configuring AKS workloads in Azure to burst out to Azure Container Instances
And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.
Developing with Docker for the Arm Architecture
This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.
More from Docker, Inc. (20)
Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
Recently uploaded
5th LF Energy Power Grid Model Meet-up Slides
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Digital Marketing Trends in 2024 | Guide for Staying Ahead
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Best 20 SEO Techniques To Improve Website Visibility In SERP
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Project Management Semester Long Project - Acuity
Acuity is an innovative learning app designed to transform the way you engage with knowledge. Powered by AI technology, Acuity takes complex topics and distills them into concise, interactive summaries that are easy to read & understand. Whether you're exploring the depths of quantum mechanics or seeking insight into historical events, Acuity provides the key information you need without the burden of lengthy texts.
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Choosing The Best AWS Service For Your Website + API.pptx
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays, June 2024 with Maria Copot 20
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Recently uploaded (20)
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DockerCon EU 2015: Monitoring Docker
- 1. Monitoring Docker Ilan Rabinovitch Director, Community - Datadog
- 2. Agenda • Monitoring 101 - Crash Course • Challenges in Monitoring Dynamic Infrastructure • Key Docker Metrics
- 3. About Me • Long time Datadog user. • Prior to Datadog built automation and monitoring tooling at Ooyala and Edmunds.com • Community Event Organizer: SCALE and Texas Linux Fest DevOpsDay LA DevOpsDay Silicon Valley Ilan Rabinovitch Datadog
- 4. Quick Overview of Datadog Datadog gathers performance data from all your application and infrastructure components. • Monitoring for modern applications 1. Dynamic Infrastructure 2. Containers (Docker, ECS, Mesos, k8s, and more…) 3. Microservices • Time series storage of metrics and events • Trending, alerting and anomaly detection. • We’re hiring! (Remote, NY, Boston, Paris)
- 6. Docker Adoption Growth We’ve see 5x increase of Docker adoption over the last year.
- 7. Monitor Everything Culture Automation Metrics Sharing Damon Edwards and John Willis DevOpsDay LA
- 10. Monitoring 101: Categorizing Your Metrics More at: http://goo.gl/t1Rgcg
- 11. Monitoring 101: Focus on symptoms More at: http://goo.gl/t1Rgcg
- 12. Recurse until you find root cause. More at: http://goo.gl/t1Rgcg
- 14. Operational Complexity • Average containers per host: N (N=4, 10/2015) • N-times as many “hosts” to manage • Affects • provisioning: prep’ing & building containers • configuration: passing config to containers • orchestration: deciding where/when containers run • monitoring: making sure containers run properly
- 15. So what does that mean for monitoring and management?
- 17. Fact 7: Docker Hosts Often Run Four Containers at a Time
- 20. Operational Complexity: Scale 160 metrics per host 640 metrics per host Assuming 4 containers per host
- 21. Operational Complexity: Scale 100 instances 64,000 metrics Assuming 4 containers per host
- 24. Monitoring Needs and Pains • Avoid Static config files tracking dynamic infrastructure.
- 25. Monitoring Needs and Pains: Static vs Dynamic • Avoid Static config files tracking dynamic infrastructure. Configuration management is awesome, but….
- 26. Monitoring Needs and Pains • Avoid Static config files tracking dynamic infrastructure. • Avoid a host centric view. Focus on service level.
- 27. Monitoring Needs and Pains: Host Centric vs Service Centric
- 28. Monitoring Needs and Pains: Host Centric vs Service Centric
- 29. Monitoring Needs and Pains • Avoid Static config files tracking dynamic infrastructure. • Avoid a host centric view. Focus on service level. • Use tags, labels, etc on your hosts and metrics to form queries.
- 30. Monitoring Needs and Pains: Query Based Monitoring “Show me rate of HTTP 500 responses from nginx” “… in region us-east-1 across all availability zones” “… running my app version 2….” • Use tags, labels, etc on your hosts and metrics to form queries. • Pull in labels from your infrastructure whether EC2, Docker or your scheduler. • Ask questions that will ring true regardless of your scale that day.
- 31. Monitoring Needs and Pains • Avoid Static config files tracking dynamic infrastructure. • Avoid a host centric view. Focus on service level. • Use tags, labels, etc on your hosts and metrics to form queries. • Know your underlying tech. In this case Docker and how to pull metrics from it.
- 32. Collecting Docker Metrics More at http://bit.ly/1j4QXek
- 33. Collecting Docker Metrics: Pseudo Files • Access via sysfs in /sys/fs/cgroup or /cgroup • By default do not require root access. • Fast and light weight • Limited I/O and Network metrics.
- 34. Collecting Docker Metrics: stats • Continuous live stream of basic CPU, memory, & network metrics. • Available via API in JSON (see unix:///var/run/docker.sock) • At least version 1.5.0 of Docker (released Feb 2015)
- 35. Collecting Docker Metrics: stats api • Similar to stats command provides a stream of metrics but more details. • Data is returned in JSON (see unix:///var/run/docker.sock)
- 37. Collecting Docker Metrics: Summary CPU METRICS MEMORY METRICS I/O METRICS NETWORK METRICS pseudo-files Yes Yes Some Yes, in 1.6.1+ stats command Basic Basic No Basic API Yes Yes Some Yes