ConHub is a metadata management system for Docker containers built on PostgreSQL. It includes:
1) ConSQL, which models container metadata in a relational schema called ConSchema with tables for entities and relationships.
2) CQL, an extension of SQL for querying container metadata with APIs like TAG and INTERSECTION.
3) Applications like ConQ for querying, ConViz for visualizing relationships, and ConRecovery for facilitating recovery from failures.
Docker allows creating isolated environments called containers from images. Containers provide a standard way to develop, ship, and run applications. The document discusses how Docker can be used for scientific computing including running different versions of software, automating computations, sharing research environments and results, and providing isolated development environments for users through Docker IaaS tools. K-scope is a code analysis tool that previously required complex installation of its Omni XMP dependency, but could now be run as a containerized application to simplify deployment.
This document discusses integrating Docker containers with the libvirt API to allow Docker management using libvirt. It begins by providing background on Docker, containers, and libvirt. It then proposes implementing the Docker API in C and integrating it with the libvirt API. This would allow clouds to provide a single libvirt API for managing both containers and virtual machines, without needing separate Docker APIs. It would also provide a generic Docker interface across clouds.
An operational view into docker registry with scalability, access control and...Conference Papers
This document discusses improvements to the Docker registry to address scalability, access control, and image vulnerability assessment. It proposes:
1) Using a proxy like NGINX in front of the registry to load balance requests and scale the registry across multiple servers.
2) Adding user authentication and authorization to the registry to restrict access to images based on user permissions.
3) Integrating the Anchore image scanning tool to analyze images pushed to the registry for vulnerabilities before use.
Together these changes aim to make the Docker registry more scalable, secure, and provide visibility into image vulnerabilities.
Server virtualization is a fundamental technological innovation that is used extensively in IT enterprises. Server virtualization enables creation of multiple virtual machines on single underlying physical machine. It is realized either in form of hypervisors or containers. Hypervisor is an extra layer of abstraction between the hardware and virtual machines that emulates underlying hardware. In contrast, the more recent container-based virtualization technology runs on host kernel without additional layer of abstraction. Thus container technology is expected to provide near native performance compared to hypervisor based technology. We have conducted a series of experiments to measure and compare the performance of workloads over hypervisor based virtual machines, Docker containers and native bare metal machine. We use a standard benchmark workload suite that stresses CPU, memory, disk IO and system. The results obtained show that Docker containers provide better or similar performance compared to traditional hypervisor based virtual machines in almost all the tests. However as expected the native system still provides the best performance as compared to either containers or hypervisors.
The document provides instructions for installing Red Hat Enterprise Linux 6 (RHEL 6) using the basic graphical installation process, including requirements for hardware, partitioning disks, setting the hostname and time zone, creating users and passwords, and selecting installation options. It outlines the steps to boot from the installation media, navigate the installation screens to configure language and keyboard settings, storage selection, networking configuration, and partitioning disks for the root, boot and swap partitions.
This is one of 7 reports provided in work package 3: Micro services for small and medium institutions.
Authors:
Odo Benda, Gerda Koch and Walter Koch
AIT Forschungsgesellschaft mbH
The document provides an overview of containers and Docker. It discusses why containers are important for organizing software, improving portability, and protecting infrastructure. It describes key Docker concepts like images, containers, Dockerfile for building images, and tools like Docker Compose and Docker Swarm for defining and running multi-container apps. The document recommends reading "The Art of War" and scanning systems without being detected before potentially more intrusive activities. It also briefly introduces network security pillars and buffer overflows as an attack technique.
Cohesive Networks Support Docs: VNS3 3.5 Container System Add-OnsCohesive Networks
Use the VNS3 Network Security Container Plugin System to allow customized plugins in your secure network.
In this guide, you will learn how container networking with VNS3:net works, how to upload an image or Dockerfile, allocate a container via the VNS3 UI, save a running container, and access considerations to go along with the container setup. \
This guide is intended for VSN3:net versions 3.5 and higher, bespoke Lite or standard SME and Enterprise editions. We recommend familiarizing yourself with VNS3 and the other documentation before using container systems.
Docker allows creating isolated environments called containers from images. Containers provide a standard way to develop, ship, and run applications. The document discusses how Docker can be used for scientific computing including running different versions of software, automating computations, sharing research environments and results, and providing isolated development environments for users through Docker IaaS tools. K-scope is a code analysis tool that previously required complex installation of its Omni XMP dependency, but could now be run as a containerized application to simplify deployment.
This document discusses integrating Docker containers with the libvirt API to allow Docker management using libvirt. It begins by providing background on Docker, containers, and libvirt. It then proposes implementing the Docker API in C and integrating it with the libvirt API. This would allow clouds to provide a single libvirt API for managing both containers and virtual machines, without needing separate Docker APIs. It would also provide a generic Docker interface across clouds.
An operational view into docker registry with scalability, access control and...Conference Papers
This document discusses improvements to the Docker registry to address scalability, access control, and image vulnerability assessment. It proposes:
1) Using a proxy like NGINX in front of the registry to load balance requests and scale the registry across multiple servers.
2) Adding user authentication and authorization to the registry to restrict access to images based on user permissions.
3) Integrating the Anchore image scanning tool to analyze images pushed to the registry for vulnerabilities before use.
Together these changes aim to make the Docker registry more scalable, secure, and provide visibility into image vulnerabilities.
Server virtualization is a fundamental technological innovation that is used extensively in IT enterprises. Server virtualization enables creation of multiple virtual machines on single underlying physical machine. It is realized either in form of hypervisors or containers. Hypervisor is an extra layer of abstraction between the hardware and virtual machines that emulates underlying hardware. In contrast, the more recent container-based virtualization technology runs on host kernel without additional layer of abstraction. Thus container technology is expected to provide near native performance compared to hypervisor based technology. We have conducted a series of experiments to measure and compare the performance of workloads over hypervisor based virtual machines, Docker containers and native bare metal machine. We use a standard benchmark workload suite that stresses CPU, memory, disk IO and system. The results obtained show that Docker containers provide better or similar performance compared to traditional hypervisor based virtual machines in almost all the tests. However as expected the native system still provides the best performance as compared to either containers or hypervisors.
The document provides instructions for installing Red Hat Enterprise Linux 6 (RHEL 6) using the basic graphical installation process, including requirements for hardware, partitioning disks, setting the hostname and time zone, creating users and passwords, and selecting installation options. It outlines the steps to boot from the installation media, navigate the installation screens to configure language and keyboard settings, storage selection, networking configuration, and partitioning disks for the root, boot and swap partitions.
This is one of 7 reports provided in work package 3: Micro services for small and medium institutions.
Authors:
Odo Benda, Gerda Koch and Walter Koch
AIT Forschungsgesellschaft mbH
The document provides an overview of containers and Docker. It discusses why containers are important for organizing software, improving portability, and protecting infrastructure. It describes key Docker concepts like images, containers, Dockerfile for building images, and tools like Docker Compose and Docker Swarm for defining and running multi-container apps. The document recommends reading "The Art of War" and scanning systems without being detected before potentially more intrusive activities. It also briefly introduces network security pillars and buffer overflows as an attack technique.
Cohesive Networks Support Docs: VNS3 3.5 Container System Add-OnsCohesive Networks
Use the VNS3 Network Security Container Plugin System to allow customized plugins in your secure network.
In this guide, you will learn how container networking with VNS3:net works, how to upload an image or Dockerfile, allocate a container via the VNS3 UI, save a running container, and access considerations to go along with the container setup. \
This guide is intended for VSN3:net versions 3.5 and higher, bespoke Lite or standard SME and Enterprise editions. We recommend familiarizing yourself with VNS3 and the other documentation before using container systems.
The document contains a resume for Virendra Kumar Mishra. It summarizes his objective of seeking a challenging role in software development, lists his 3 years of experience developing Java and J2EE applications, and provides details on 4 projects he worked on between 2014-2016 developing applications for clients like Alert Enterprise, MGH Logistics, and Deutsche Bank. It also includes his academic qualifications, personal details, and contact information.
Jose Rizal arrived in San Francisco on April 28, 1888 after a long ship journey. On May 4th, he witnessed racial discrimination and inequality as white Americans prejudiced against black African Americans. Despite staying in fine hotels, this left Rizal with a poor impression of the United States. He then traveled by train to New York, stopping to see Niagara Falls along the way, before arriving in New York City on May 13th, where he later departed for London, concluding his experiences traveling through the US.
This document summarizes Jose Rizal's second travels abroad from 1888 to 1892. It mentions that he stayed in a grand hotel in Yokohama, Japan in February 1888 after arriving by oceanic steamer. It also discusses two individuals Rizal met - Juan Perez y Caballero, a Spanish politician and diplomat, and Seiko Usui, a 23-year-old Japanese woman with whom Rizal had a relationship during his time in Yokohama. The document notes Rizal departed Japan on April 13, 1888 aboard the steamer "Belgic" and eventually arrived in London on December 1, 1888.
The document discusses competency models, which are clusters of knowledge, skills, abilities, behaviors, and attitudes related to job success. It outlines different approaches for developing competency models, including universal, functional, job-specific, and multiple job models. The document also discusses how competency models can be used for human resource processes like recruitment, selection, performance management, and career development.
VADI: Viirtualización del Data Center - José Manuel FloresNextel S.A.
Este documento describe la solución ADC-VX de Radware, un hipervisor especializado para administrador de tráfico de aplicaciones (ADC) que ejecuta múltiples instancias virtuales de ADC (vADC). ADC-VX ofrece consolidación, aislamiento, escalabilidad y gestión centralizada de los vADC.
Este documento presenta 15 consejos para tener éxito en la búsqueda de empleo. Se divide en secciones que cubren temas como la orientación laboral, la actitud positiva, la perseverancia, la planificación, el currículum y la entrevista. El objetivo es ayudar a las personas que buscan trabajo ofreciendo recomendaciones basadas en la experiencia de consultores de empleo.
The document provides an agenda for a Class 4 EWRT 1B course. It includes presentations on course terms and author Langston Hughes. Students will discuss the stories "Passing" and "Passing" in groups and generate QHQ questions. They will also learn about writing summaries and paraphrasing poetry. The rest of the class covers team assignments, earning participation points, and instructions for an in-class writing assignment summarizing and paraphrasing a poem by Langston Hughes. Teams will remain the same for class discussions and will be required to change members and compositions between essay assignments.
Servicio de Asistencia a la víctima. Memoria 2015Irekia - EJGV
El documento presenta el informe anual de 2015 del Servicio de Asistencia a la Víctima (SAV). El SAV atendió a 2.674 personas, la mayoría mujeres (79,43%). La violencia de género y la violencia intrafamiliar fueron los principales motivos de asistencia, representando el 70% de los casos. El SAV ofreció diversas atenciones como intervenciones psicológicas y asesoramiento jurídico. Además, el SAV asumió funciones de comunicación penitenciaria preventiva, realizando 1.009 comunic
Uang merupakan alat tukar yang diterima secara sah untuk membeli barang dan jasa, beberapa contoh mata uang yang digunakan adalah Rupiah, Dolar, Euro. Nilai mata uang dapat ditukar dengan pecahan lain sesuai kesetaraannya.
Get prepared for camping by setting up your shelter before dark to avoid struggling in the low light. Pack extra clothes for kids since they will get dirty playing outdoors. Consider bringing camping-appropriate pillows and sleeping bags suited to the climate. Prepare kids for camping activities like fishing and pitching tents by teaching them at home beforehand. With proper planning, your camping trip can be a relaxing experience instead of a miserable one.
Triforce & Company presenta nuevos productos Apple como el Iphone 6 y 6 Plus, el nuevo reloj inteligente Iwatch, portátiles Apple con mayor capacidad y velocidad, un nuevo modelo de Imac con mejor diseño, el Icar con accesorios Apple y un prototipo de televisor controlable con el Iphone.
When seeking to implement microservices architecture in an organization, these are the benefits of deploying Docker as the platform as a service (PaaS); Docker helps manage costs, complexity, service continuity and production times.
Dev opsec dockerimage_patch_n_lifecyclemanagement_kanedafromparis
Lors de cette présentation, nous allons dans un premier temps rappeler la spécificité de docker par rapport à une VM (PID, cgroups, etc) parler du système de layer et de la différence entre images et instances puis nous présenterons succinctement kubernetes.
Ensuite, nous présenterons un processus « standard » de propagation d’une version CI/CD (développement, préproduction, production) à travers les tags docker.
Enfin, nous parlerons des différents composants constituant une application docker (base-image, tooling, librairie, code).
Une fois cette introduction réalisée, nous parlerons du cycle de vie d’une application à travers ses phases de développement, BAU pour mettre en avant que les failles de sécurité en période de développement sont rapidement corrigées par de nouvelles releases, mais pas nécessairement en BAU où les releases sont plus rares. Nous parlerons des diverses solutions (jfrog Xray, clair, …) pour le suivie des automatique des CVE et l’automatisation des mises à jour. Enfin, nous ferons un bref retour d’expérience pour parler des difficultés rencontrées et des propositions d’organisation mises en oeuvre.
Cette présentation bien qu’illustrée par des implémentations techniques est principalement organisationnelle.
Cette présentation donne une vue d’ensemble et les concepts généraux, permettant d’appréhender OpenShift et de faciliter les premières étales de prises en mains.
On y parle de Pods, de services, de source-to-image, etc.
The document contains a resume for Virendra Kumar Mishra. It summarizes his objective of seeking a challenging role in software development, lists his 3 years of experience developing Java and J2EE applications, and provides details on 4 projects he worked on between 2014-2016 developing applications for clients like Alert Enterprise, MGH Logistics, and Deutsche Bank. It also includes his academic qualifications, personal details, and contact information.
Jose Rizal arrived in San Francisco on April 28, 1888 after a long ship journey. On May 4th, he witnessed racial discrimination and inequality as white Americans prejudiced against black African Americans. Despite staying in fine hotels, this left Rizal with a poor impression of the United States. He then traveled by train to New York, stopping to see Niagara Falls along the way, before arriving in New York City on May 13th, where he later departed for London, concluding his experiences traveling through the US.
This document summarizes Jose Rizal's second travels abroad from 1888 to 1892. It mentions that he stayed in a grand hotel in Yokohama, Japan in February 1888 after arriving by oceanic steamer. It also discusses two individuals Rizal met - Juan Perez y Caballero, a Spanish politician and diplomat, and Seiko Usui, a 23-year-old Japanese woman with whom Rizal had a relationship during his time in Yokohama. The document notes Rizal departed Japan on April 13, 1888 aboard the steamer "Belgic" and eventually arrived in London on December 1, 1888.
The document discusses competency models, which are clusters of knowledge, skills, abilities, behaviors, and attitudes related to job success. It outlines different approaches for developing competency models, including universal, functional, job-specific, and multiple job models. The document also discusses how competency models can be used for human resource processes like recruitment, selection, performance management, and career development.
VADI: Viirtualización del Data Center - José Manuel FloresNextel S.A.
Este documento describe la solución ADC-VX de Radware, un hipervisor especializado para administrador de tráfico de aplicaciones (ADC) que ejecuta múltiples instancias virtuales de ADC (vADC). ADC-VX ofrece consolidación, aislamiento, escalabilidad y gestión centralizada de los vADC.
Este documento presenta 15 consejos para tener éxito en la búsqueda de empleo. Se divide en secciones que cubren temas como la orientación laboral, la actitud positiva, la perseverancia, la planificación, el currículum y la entrevista. El objetivo es ayudar a las personas que buscan trabajo ofreciendo recomendaciones basadas en la experiencia de consultores de empleo.
The document provides an agenda for a Class 4 EWRT 1B course. It includes presentations on course terms and author Langston Hughes. Students will discuss the stories "Passing" and "Passing" in groups and generate QHQ questions. They will also learn about writing summaries and paraphrasing poetry. The rest of the class covers team assignments, earning participation points, and instructions for an in-class writing assignment summarizing and paraphrasing a poem by Langston Hughes. Teams will remain the same for class discussions and will be required to change members and compositions between essay assignments.
Servicio de Asistencia a la víctima. Memoria 2015Irekia - EJGV
El documento presenta el informe anual de 2015 del Servicio de Asistencia a la Víctima (SAV). El SAV atendió a 2.674 personas, la mayoría mujeres (79,43%). La violencia de género y la violencia intrafamiliar fueron los principales motivos de asistencia, representando el 70% de los casos. El SAV ofreció diversas atenciones como intervenciones psicológicas y asesoramiento jurídico. Además, el SAV asumió funciones de comunicación penitenciaria preventiva, realizando 1.009 comunic
Uang merupakan alat tukar yang diterima secara sah untuk membeli barang dan jasa, beberapa contoh mata uang yang digunakan adalah Rupiah, Dolar, Euro. Nilai mata uang dapat ditukar dengan pecahan lain sesuai kesetaraannya.
Get prepared for camping by setting up your shelter before dark to avoid struggling in the low light. Pack extra clothes for kids since they will get dirty playing outdoors. Consider bringing camping-appropriate pillows and sleeping bags suited to the climate. Prepare kids for camping activities like fishing and pitching tents by teaching them at home beforehand. With proper planning, your camping trip can be a relaxing experience instead of a miserable one.
Triforce & Company presenta nuevos productos Apple como el Iphone 6 y 6 Plus, el nuevo reloj inteligente Iwatch, portátiles Apple con mayor capacidad y velocidad, un nuevo modelo de Imac con mejor diseño, el Icar con accesorios Apple y un prototipo de televisor controlable con el Iphone.
When seeking to implement microservices architecture in an organization, these are the benefits of deploying Docker as the platform as a service (PaaS); Docker helps manage costs, complexity, service continuity and production times.
Dev opsec dockerimage_patch_n_lifecyclemanagement_kanedafromparis
Lors de cette présentation, nous allons dans un premier temps rappeler la spécificité de docker par rapport à une VM (PID, cgroups, etc) parler du système de layer et de la différence entre images et instances puis nous présenterons succinctement kubernetes.
Ensuite, nous présenterons un processus « standard » de propagation d’une version CI/CD (développement, préproduction, production) à travers les tags docker.
Enfin, nous parlerons des différents composants constituant une application docker (base-image, tooling, librairie, code).
Une fois cette introduction réalisée, nous parlerons du cycle de vie d’une application à travers ses phases de développement, BAU pour mettre en avant que les failles de sécurité en période de développement sont rapidement corrigées par de nouvelles releases, mais pas nécessairement en BAU où les releases sont plus rares. Nous parlerons des diverses solutions (jfrog Xray, clair, …) pour le suivie des automatique des CVE et l’automatisation des mises à jour. Enfin, nous ferons un bref retour d’expérience pour parler des difficultés rencontrées et des propositions d’organisation mises en oeuvre.
Cette présentation bien qu’illustrée par des implémentations techniques est principalement organisationnelle.
Cette présentation donne une vue d’ensemble et les concepts généraux, permettant d’appréhender OpenShift et de faciliter les premières étales de prises en mains.
On y parle de Pods, de services, de source-to-image, etc.
This document provides an introduction to containers and Docker. It defines key Docker terminology like images, containers, registries and explains Docker's benefits like isolation, portability and scalability. It compares containers to virtual machines and outlines Docker's components including Community Edition for development and Enterprise Edition for production.
Docker Announces Open Source Compose for AWS ECS & Microsoft ACI9 series
Docker has announced that the code for the Microsoft Azure Container Instances (ACI) and Amazon Elastic Container Service (ECS) integrations will be open-sourced.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It was originally developed by Google based on years of experience running production workloads at scale. Kubernetes groups containers into logical units called pods and handles tasks like scheduling, health checking, scaling and rollbacks. The main components include a master node that manages the cluster and worker nodes that run application containers scheduled by the master.
An overview of Mesos and Kubernetes ecosystem including overview, architecture, customers and partners. For a beginner it will give a good covering of all the basics!
The purpose of this solution is to go over the Docker basics which explain containers, images, how they work, where to find them, the architecture (client, daemon), the difference between Docker and VMs, and we will see Docker and an image and see some commands.
Summary:
- Virtual machines VS containers
- Containers
- What is Docker ?
- LXC vs Docker
- Docker basic concepts
- The Open Container Initiative (OCI)
- Runtime containers
- OCI Containers images
- Write a Docker File
- Build an image with Docker
- Docker Compose
- Images Registry
- Docker Engine
- Run a container with Docker
The Axigen Docker image is provided for users to be able to run an Axigen based mail service within a Docker container.
The following services are enabled and mapped as 'exposed' TCP ports in Docker:
§ SMTP (25 - non secure, 465 - TLS)
§ IMAP (143 - non secure, 993 - TLS)
§ POP3 (110 - non secure, 995 - TLS)
§ WEBMAIL (80 - non secure, 443 - TLS)
§ WEBADMIN (9000 - non secure, 9443 - TLS)
CLI (7000 - non secure
Using Docker container technology with F5 Networks products and servicesF5 Networks
This document discusses how Docker containerization technology can be used with F5 products and services. It provides an overview of Docker, comparing it to virtual machines. Docker allows for higher resource utilization and faster application deployment than VMs. The document outlines how F5 supports using containers and integrating with Docker for application delivery and security services. It describes Docker networking and how F5 solutions can provide services like load balancing within Docker container environments.
Understanding the container landscape and it associated projectsAnthony Chow
The document discusses containers and container technologies. It provides an overview of the history and key components of containers like Docker, including namespaces, control groups, AUFS, Docker images, registries, networking solutions, security concerns and orchestration tools. It also discusses how OpenStack projects are embracing containers to provide container orchestration platforms and run OpenStack services as containers to make them more scalable and efficient. The document encourages learning more about containers to stay relevant in today's technologies.
This document provides an overview of Docker and containers for data science. It begins with definitions of containers and discusses the history and benefits of containers. It then explains how Docker containers work using namespaces, cgroups, and union file systems. Key Docker concepts are introduced like Dockerfiles, images, containers, and the Docker architecture. Practical examples are given for building simple machine learning models and databases in containers. Advanced topics covered include Docker Compose, DevOps workflows, continuous delivery, and Kubernetes. The document is intended to provide data scientists with an introduction to using Docker for their work.
This document provides an introduction to Docker and OpenShift. It begins with an overview of containers and Docker, then discusses OpenShift as a platform for developing, running and managing applications using containers. Key concepts covered include Docker images, OpenShift's use of Kubernetes to manage pods and container orchestration, build configurations, deployment configurations, routes and services for network communication, and the use of projects/namespaces for resource isolation and security. The document concludes with a demonstration of Docker and OpenShift.
My college ppt on topic Docker. Through this ppt, you will understand the following:- What is a container? What is Docker? Why its important for developers? and many more!
Docker is an open source containerization platform that allows users to package applications and their dependencies into standardized executable units called containers. Docker relies on features of the Linux kernel like namespaces and cgroups to provide operating-system-level virtualization and allow containers to run isolated on a shared kernel. This makes Docker highly portable and allows applications to run consistently regardless of the underlying infrastructure. Docker uses a client-server architecture where the Docker Engine runs in the cloud or on-premises and clients interact with it via Docker APIs or the command line. Common commands include build to create images from Dockerfiles, run to launch containers, and push/pull to distribute images to registries. Docker is often used for microservices and multi-container
This document provides an overview and comparison of Docker, Kubernetes, OpenShift, Fabric8, and Jube container technologies. It discusses key concepts like containers, images, and Dockerfiles. It explains how Kubernetes provides horizontal scaling of Docker through replication controllers and services. OpenShift builds on Kubernetes to provide a platform as a service with routing, multi-tenancy, and a build/deploy pipeline. Fabric8 and Jube add additional functionality for developers, with tools, libraries, logging, and pure Java Kubernetes implementations respectively.
Containerization is a lightweight alternative to full machine virtualization that involves encapsulating an application in a container with its own operating environment. This provides many of the benefits of loading an application onto a virtual machine, as the application can be run on any suitable physical machine without any worries about dependencies.
Building Distributed Systems without Docker, Using Docker Plumbing Projects -...Patrick Chanezon
Docker provides an integrated and opinionated toolset to build, ship and run distributed applications. Over the past year, the Docker codebase has been refactored extensively to extract infrastructure plumbing components that can be used independently, following the UNIX philosophy of small tools doing one thing well: runC, containerd, swarmkit, hyperkit, vpnkit, datakit and the newly introduced InfraKit.
This talk will give an overview of these tools and how you can use them to build your own distributed systems without Docker.
Patrick Chanezon & David Chung, Docker & Phil Estes, IBM
Containers allow multiple isolated user space instances to run on a single host operating system. Containers are seen as less flexible than virtual machines since they generally can only run the same operating system as the host. Docker adds an application deployment engine on top of a container execution environment. Docker aims to provide a lightweight way to model applications and a fast development lifecycle by reducing the time between code writing and deployment. Docker has components like the client/server, images used to create containers, and public/private registries for storing images.
Similar to ConHub A Metadata Management System for Docker Containers (20)
ConHub A Metadata Management System for Docker Containers
1. ConHub: A Metadata Management System
for Docker Containers
Chris Xing Tian
National University of Singapore
tianxing@comp.nus.edu.sg
Aditya Pan
Amity University
aditya.pan@student.amity.edu
Y.C. Tay
National University of Singapore
dcstayyc@nus.edu.sg
ABSTRACT
For many years now, enterprises and cloud providers have been
using virtualization to run their workloads. Until recently, this
means running an application in a virtual machine (hardware
virtualization). However, virtual machines are increasingly
replaced by containers (operating system virtualization), as
evidenced by the rapid rise of Docker. A containerized software
environment can generate a large amount of metadata. If properly
managed, these metadata can greatly facilitate the management of
containers themselves.
This demonstration introduces ConHub, a PostgreSQL-based
container metadata management system. Visitors will see that
(1) ConHub has a language CQL that supports Docker commands;
(2) it has a user-friendly interface for querying and visualizing
container relationships; and (3) they can use CQL to formulate
sophisticated queries to facilitate container management.
A video of the demonstration can be found at
https://youtu.be/aWPgbeo_79g
Keywords
OS Virtualization; Container Metadata; Relational Database
1. INTRODUCTION
Datacenters and cloud providers largely rely on virtualization to
run enterprise workloads and user applications. Until recently,
this means hardware virtualization, where execution happens in
virtual machines (VMs).
Previously, a user process would run on a bare-metal machine that
consists of a physical machine and an operating system (OS).
Now, the user process and the guest OS that it calls are contained
in a VM (another process) that runs on a hypervisor that takes on
the role of a host OS. The hardware is thus virtualized.
The guest OS becomes largely redundant if it shares the same
kernel as the host OS. In this case, the VM can be replaced by a
container that consists of the application, and the files, libraries
and binaries that it needs. Two containers on the same bare-metal
may have different OS versions, distributions (libraries, tools,
window system, etc.) or namespaces (file system, process
identifiers, etc.). The OS is thus virtualized.
Interest in container-based virtualization is rising rapidly, as
evidenced by the viral adoption of the Docker engine 1
for
containers, replacing the hypervisor for VMs. In fact, even before
the rise of Docker, Google [1] and Facebook2
have both been
using containerized infrastructures for some years. There are 2
main reasons for the surging interest in containerization:
C1: A container image is generated by scripts that specify
dependencies, thus facilitating code development, debugging and
deployment; the container is a running instance of the image (like
a process is a running instance of its code).
C2: The removal of the guest OS makes these images much
smaller, so a physical machine can host many more containers
than VMs, and spawning containers (in response to a flash crowd,
say) is also much faster than booting VMs.
A containerized system has a lot of metadata that can facilitate
code development and debugging (C1): which OS version does
image X use? who is the developer for X? when was X last
modified? which containers are running X? Etc.
There are also a lot of metadata that can facilitate container
deployment (C2): how much free memory is there on a particular
node? which hosts are running replicas of a container? which
containers can be collocated without performance interference?
which containers are using a particular port? Etc.
While there are already systems for managing containers, there is
none so far for managing metadata for images and containers.
This demonstration introduces such a system, namely ConHub.
1.1 Related Work
Examples of OS virtualization include HP-UX3
, BSD jails [2],
Solaris Zones [3] and Linux containers (LXC 4
), which were
recently extended by Docker. Queries on the Docker metadata for
images and containers can only be expressed as keyword search.
Docker Datacenter5
and Kubernetes6
are systems for managing
Docker containers in a cluster environment; other container
management systems include Huawei CCE7
and Netease Hive8
.
1
http://www.docker.com/
2
http://www.slideshare.net/Docker/aravindnarayanan-
facebook140613153626phpapp02-37588997
3
http://www.hpl.hp.com/hpjournal/pdfs/IssuePDFs/1985-10.pdf
4
https://linuxcontainers.org
5
https://www.docker.com/products/docker-datacenter
6
http://kubernetes.io/
7
http://console.hwclouds.com/cce
8
http://c.163.com
Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights
for third-party components of this work must be honored. For all other
uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
CIKM’16, October 24 – November 28, 2016, Indianapolis, IN, USA
ACM 978-1-4503-4073-1/16/10.
DOI: http://dx.doi.org/10.1145/2983323.2983331
2. These systems dynamically generate a lot of metadata for
container deployment (failures, replication, resource allocation
etc.). The dynamic setting in a datacenter calls for a powerful
metadata management system to support container management.
We designed ConHub to fill this need. ConHub is built on
PostgreSQL 9
, so it brings to bear well-developed, industrial-
strength relational database technology on the management and
querying of metadata for images and containers.
2. CONHUB ARCHITECTURE
ConHub has 3 key components: (1) ConSQL, a metadata
management system. (2) A query processor that supports CQL, a
language for querying and generating metadata for images and
containers; it includes a set of APIs for developers to implement
third-party applications that suit their purpose. (3) An ecosystem
of applications, built on the APIs, for queries and visualization.
These components are illustrated in Figure 1, and described below:
Figure 1. ConHub Architecture
Figure 2. ConSchema has 9 tables for entities and 5 tables for
relationships (arrows point from primary key to foreign key).
9
http://www.postgresql.org/
(1) ConSQL
ConSQL is a database system implemented with PostgreSQL.
We have designed a relational schema called ConSchema,
shown in Figure 2, to model the metadata underlying a
container system. ConSchema has 9 tables for entities
(Images, Containers, Users, Dockerfiles, etc.) and 5
relationship tables (ConToImage, Labels, etc.).
Much of these metadata are extracted from the JSON files
generated by Docker. Users can also generate metadata, via
Docker labels or CQL tags. If integrated with a container
management system like Docker Datacenter or Kubernetes,
ConSQL can also manage metadata from that system.
(2) CQL
CQL is an extension of SQL, the standard language for
managing relational data. CQL thus inherits the power of
SQL in the declarative formulation of semantically rich
queries, like those illustrated in the Introduction for
provenance (C1) and management (C2). This is a
tremendous improvement over the simple keyword search
provided by current container systems.
The CQL extension of SQL consists of the following APIs:
• TAG (Set objects, String label): Tags a set of objects
with the specified label.
• INTERSECTION (id1, id2): Returns the lowest
common ancestor that two images or containers share.
• CHILD (imageId): Returns all the child images that
derived from the specified image in the form of a Set.
• IMAGE(conId): Returns the id of the image that
generated the specified container.
• CONTAINER(imageId): Returns the containers
generated from the specified image.
• DISTANCE(id1, id2): Returns the distance between
two images in the version chain; returns –1 if they are
unrelated.
(3) Application Ecosystem
The APIs accessible from CQL can also be used to build
applications for image and container management. So far,
we have implemented the following applications:
ConQ: A tool for formulating CQL queries by manipulating
ConSchema tables using a graphical user interface. We
designed this tool to help the Docker user who is unfamiliar
with SQL syntax and semantics.
ConViz: A tool for visualizing the provenance relationship
among images and containers. For example, a user can
specify two containers, and ConViz will display the image
paths that lead from the containers to their common ancestor
image (if any).
ConRecovery: A tool to facilitate recovery from a container
failure (using a reported incident10
as a guide). Suppose
there is a code change to an image X, and the container CX
spawned from X crashes a service. A user can use
ConRecovery to find a previous, safe image version Y, spawn
a new container CY to replace CX, and notify the developer
that made the change to X.
10
http://blog.flux7.com/blogs/docker/docker-saves-the-day-at-
flux7
3. 3. DEMONSTRATION SCENARIOS
Our demonstration of ConHub will proceed in 3 stages:
Stage I: Docker
Visitors at the conference who are familiar with Docker can
verify that ConHub is an extension of Docker: they can query
the ConHub repository (preloaded), create new images, and
label, spawn or shutdown containers, like they can with a
Docker client. For visitors who are unfamiliar with Docker,
we will demonstrate how these can be done with Docker
commands.
Stage II: ConQ and ConViz
In Stage II, the visitors can use ConQ’s table manipulation
interface (see Fig.2) to formulate queries of the metadata
stored in ConSchema, including those for any new images or
containers created in Stage I. They can also use ConViz to
visualize the version tree for an image, or the provenance
among images and containers (see Fig.3).
Stage III: ConRecovery and CQL
Finally, we use ConRecovery to demonstrate the scenario
described above for recovering from a container failure (see
Fig.4).
We will also demonstrate how the power of CQL can help a
development team deal with bugs. Suppose the team
identifies two containers, conIdX and conIdY, with similar
faulty behavior; they believe the fault is inherited from some
common image Z that both are based on, and want to find all
containers and images derived from Z and label them
“hazard”. This can be done with a CQL statement:
TAG ((SELECT C.conId, I.imageId
FROM Containers C, Images I WHERE I.imageId
IN CHILD(INTERSECTION(conIdX, conIdY))
AND C.imageId= I.imageId) , “hazard” )
The team suspects that Z has a virus that came from an
infected IDE downloaded by a developer when creating Z.
They decide to identify all developers of “hazard” images
and label the images and containers they produced as
“potential hazard”, using the CQL statement:
TAG((SELECT C.conId, I2.imageId
FROM Containers C, Images I1, Images I2, Labels L
WHERE L.key="hazard" AND I1.imageId=L.imageId
AND I1.maintainer = I2.maintainer
AND C.imageId = I2.imageId) , “potential hazard”)
Note the use of APIs TAG, CHILD and INTERSECTION.
Also, the second CQL statement uses four joins; it will be
hard to do the same thing with just keyword search.
We will encourage visitors to suggest queries so we can
demonstrate how they can be formulated with CQL.
4. FUTURE WORK
A video of the demonstration can be found at
https://youtu.be/aWPgbeo_79g
The metadata in this demonstration are generated by the user and
a Docker client. We plan to integrate ConHub with a container
management system, like Docker Datacenter or Kubernetes. That
will require expansion of ConSchema and addition of APIs, so the
application ecosystem can host more tools for facilitating
container deployment in a datacenter.
5. REFERENCES
[1] A. Verma, L. Pedrosa, M. Korupolu, D. Oppenheimer, E.
Tune, and J. Wilkes. Large-scale cluster management at
Google with Borg. In Proc. EuroSys, page 18, April 2015.
[2] P.-H. Kamp and R.N. Watson. Jails: Confining the
omnipotent root. In Proc SANE, vol.43, 2000.
[3] J. Beck, D. Comay, Ozgur L., D. Price, Andy T., Andrew G.,
and Blaise S. Virtualization and namespace isolation in the
Solaris operating system (psarc/2002/174), 2006.
Figure 2. ConQ: graphical interface for query formulation.
Figure 3. ConViz: visualizing provenance.
Figure 4. ConRecovery: a tool for recovering from container
failure.