A Tail of Two Containers: How docker made ci great again
•
1 like•400 views
Codeship has been powered by containers from the very beginning. In this talk we will discuss our initial implementation using LXC, how we use it and the limitations that we have encountered. Using the lessons we learned from our first gen implementation, we’ll look at the evolution of our next generation Docker service. We’ll dive into how this implementation works and discover some of the benefits and challenges of using Docker for CI.
Report
Share
Report
Share
Download to read offline
Recommended
ElasticKube, a Container Management Platform for Kubernetes
ElasticBox Lead Architect Arnaud Bonnet's presentation from the March 18, 2016 Seattle Kubernetes meetup hosted by StackPointCloud. Arnaud gives us a great overview of ElasticKube, a Kubernetes container management platform, and what's ahead for the open source project. Join us in Seattle: http://www.meetup.com/Seattle-Kubernetes-Meetup/
DockerCon SF 2015: The Distributed System Toolkit
Slides from Brendan Burns Advanced Tech Talk at DockerCon SF 2015:
Talk Description:
People often adopt containers for the dramatic improvements in application packaging and deployment that they provide. However, possibly more important, is the abstraction layer that containers provide. By encouraging users to build their distributed applications through containerized modules, rather than monolithic systems, developers are building composable, reusable distributed applications. In this talk we will explore both the development of abstract application patterns for distributed systems, as well as introduce a set of reusable, composable containers that radically simplify the process of distributed application design and construction.
HLayer / Kubernetes for CI/CD
Kubernetes can schedule workloads for CI/CD pipelines by running containerized workloads across Kubernetes clusters. It provides features like self-healing, horizontal pod autoscaling, automated rollouts and rollbacks, service discovery, storage orchestration, and secret management. Kubernetes was born out of Borg at Google and is now a top-level Apache project donated to the CNCF. It uses a controller manager to reconcile the desired state with the actual cluster state. Key Kubernetes objects include deployments, replicasets, pods, services, and secrets. Kubernetes is well-suited for scheduling CI/CD workloads across clusters.
GWT Enterprise Edition
GWT brings a lot to the table on the client side: the comprehensive browser compatibility and the ease of writing in java are just a few examples to name. But, when looking at the server side, GWT can be a bit lacking with the technologies it uses. Learn how to build powerful end to end Enterprise applications with GWT as your frontend and how to back it up with your favorite arsenal of tools, including, Spring, Guice and Hibernate
Docker Store: The New Destination for Enterprise Software - Lily Guo and Alfr...
Docker Store is the place to find trusted community and enterprise content. Independent software vendors, startups and developers alike now have a marketplace to create and distribute Enterprise-ready content through the Store. Join Chinmayee and Alfred from the Docker Store team to learn how ISVs, big and small, are using Docker Store. They will cover: How to publish and distribute high quality, reusable containers and plugins; and How Enterprise customers can simplify procurement and management of their software assets using Docker Store.
DockerCon SF 2015: DHE/DTR
Docker Trusted Registry is a private registry for Docker images that provides key features like role-based access control, audit logs, and storage drivers. It has a modular architecture with components like registry servers, load balancers, and authentication servers. It can be deployed on any infrastructure and supports integration with directory services. Docker Trusted Registry comes in different editions for teams of various sizes and needs and pricing starts from free to $150 per month depending on the edition.
Docker presentation
Docker provides a standard way to package applications into containers that are portable and can run on any infrastructure. The containers isolate applications from one another and ensure that dependencies and configurations are included so applications always run the same. This allows developers to build applications once and deploy them anywhere without worrying about compatibility or missing dependencies. It also allows operators to configure applications once and run anything on their infrastructure efficiently by eliminating inconsistencies between environments.
KubeCon 2017: Kubernetes from Dev to Prod
Kubernetes from Dev to Prod summarizes GoEuro's transition from a legacy environment to using Kubernetes and CI/CD pipelines from development to production. Key points:
- GoEuro transitioned 50+ services across 150 engineers from separate development and operations teams to using Kubernetes, Docker, and CI/CD pipelines in 4 months.
- They developed "hyper-vm" single-node Kubernetes VMs for local development and testing and "y8s" for sharing Kubernetes configurations across environments from development through production.
- CI/CD pipelines were automated using GitLab CI and custom implementations running jobs on "hyper-vm" agents to deploy to environments from preview through production.
- Additional services
Recommended
ElasticKube, a Container Management Platform for Kubernetes
ElasticBox Lead Architect Arnaud Bonnet's presentation from the March 18, 2016 Seattle Kubernetes meetup hosted by StackPointCloud. Arnaud gives us a great overview of ElasticKube, a Kubernetes container management platform, and what's ahead for the open source project. Join us in Seattle: http://www.meetup.com/Seattle-Kubernetes-Meetup/
DockerCon SF 2015: The Distributed System Toolkit
Slides from Brendan Burns Advanced Tech Talk at DockerCon SF 2015:
Talk Description:
People often adopt containers for the dramatic improvements in application packaging and deployment that they provide. However, possibly more important, is the abstraction layer that containers provide. By encouraging users to build their distributed applications through containerized modules, rather than monolithic systems, developers are building composable, reusable distributed applications. In this talk we will explore both the development of abstract application patterns for distributed systems, as well as introduce a set of reusable, composable containers that radically simplify the process of distributed application design and construction.
HLayer / Kubernetes for CI/CD
Kubernetes can schedule workloads for CI/CD pipelines by running containerized workloads across Kubernetes clusters. It provides features like self-healing, horizontal pod autoscaling, automated rollouts and rollbacks, service discovery, storage orchestration, and secret management. Kubernetes was born out of Borg at Google and is now a top-level Apache project donated to the CNCF. It uses a controller manager to reconcile the desired state with the actual cluster state. Key Kubernetes objects include deployments, replicasets, pods, services, and secrets. Kubernetes is well-suited for scheduling CI/CD workloads across clusters.
GWT Enterprise Edition
GWT brings a lot to the table on the client side: the comprehensive browser compatibility and the ease of writing in java are just a few examples to name. But, when looking at the server side, GWT can be a bit lacking with the technologies it uses. Learn how to build powerful end to end Enterprise applications with GWT as your frontend and how to back it up with your favorite arsenal of tools, including, Spring, Guice and Hibernate
Docker Store: The New Destination for Enterprise Software - Lily Guo and Alfr...
Docker Store is the place to find trusted community and enterprise content. Independent software vendors, startups and developers alike now have a marketplace to create and distribute Enterprise-ready content through the Store. Join Chinmayee and Alfred from the Docker Store team to learn how ISVs, big and small, are using Docker Store. They will cover: How to publish and distribute high quality, reusable containers and plugins; and How Enterprise customers can simplify procurement and management of their software assets using Docker Store.
DockerCon SF 2015: DHE/DTR
Docker Trusted Registry is a private registry for Docker images that provides key features like role-based access control, audit logs, and storage drivers. It has a modular architecture with components like registry servers, load balancers, and authentication servers. It can be deployed on any infrastructure and supports integration with directory services. Docker Trusted Registry comes in different editions for teams of various sizes and needs and pricing starts from free to $150 per month depending on the edition.
Docker presentation
Docker provides a standard way to package applications into containers that are portable and can run on any infrastructure. The containers isolate applications from one another and ensure that dependencies and configurations are included so applications always run the same. This allows developers to build applications once and deploy them anywhere without worrying about compatibility or missing dependencies. It also allows operators to configure applications once and run anything on their infrastructure efficiently by eliminating inconsistencies between environments.
KubeCon 2017: Kubernetes from Dev to Prod
Kubernetes from Dev to Prod summarizes GoEuro's transition from a legacy environment to using Kubernetes and CI/CD pipelines from development to production. Key points:
- GoEuro transitioned 50+ services across 150 engineers from separate development and operations teams to using Kubernetes, Docker, and CI/CD pipelines in 4 months.
- They developed "hyper-vm" single-node Kubernetes VMs for local development and testing and "y8s" for sharing Kubernetes configurations across environments from development through production.
- CI/CD pipelines were automated using GitLab CI and custom implementations running jobs on "hyper-vm" agents to deploy to environments from preview through production.
- Additional services
Automation CI CD with Gitlab, Java, docker on Hidora - Jelastic
How to commit, build, deploy and run with Gitlab for a Java application used on docker, hosted on the swiss hoster Hidora, Jelastic platform.
Build pipelines with bitbucket for Magento
Pradip Shah discusses using Bitbucket Pipelines to build continuous integration and delivery pipelines for WordPress and Magento applications. Bitbucket Pipelines uses Docker containers to run build steps defined in a YAML configuration file. Key benefits include reducing the cost of testing with containerization, and the ability to automate deployments through continuous delivery. The presentation provides an example Magento 2 pipeline and discusses concepts like separating concerns between Docker images and pipeline steps.
Kubernetes in Docker
1. Docker EE will include an unmodified Kubernetes distribution to provide orchestration capabilities alongside Docker Swarm.
2. When running mixed workloads across orchestrators, resource contention is a risk and it is recommended to separate workloads by orchestrator on each node for now.
3. Docker EE aims to address the shortcomings of running mixed workloads to better support this in the future.
Deep Dive into Kubernetes - Part 1
This presentation includes information on Kubernetes Architecture, Container Orchestration, Internal Routing, External Routing, Configuration Management, Credentials Management, Persistent Volumes, Rolling Out Updates, Autoscaling, Package Management, and a Hello World example using Helm.
Docker microservices and the service mesh
The nature of containerized, cloud-native applications is rapidly advancing with a fundamentally different architecture that will rely on service meshes with smarter proxies, traffic management, and enhanced observability for cooperating microservices, serverless functions, and complex workflows. In this session we will highlight the features that characterize this architectural transformation in the Docker cloud-native ecosystem.
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
The document discusses containers and how they can be used for stateless and stateful applications. It notes that while containers provide CPU and memory isolation, traditional storage systems cannot handle the scale and performance needs of containerized applications. The Robin Containerization Platform aims to address these issues with its container-aware storage layer that can support large volumes of variable sized volumes, quickly create and destroy snapshots and clones, and provide end-to-end performance isolation and QoS controls from applications to storage. It also decouples compute and storage to allow independent scaling and portability without data loss.
berne.*tesday1
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for self-healing, horizontal scaling, and load balancing. Kubernetes manages the computing infrastructure, networking, storage, and applications running on containers. It has features like automated scheduling, rollouts and rollbacks, load balancing, environment consistency, and auto-scaling of infrastructure. Kubernetes uses objects like pods, deployments, replica sets, and stateful sets to define and run application containers.
NetflixOSS for Triangle Devops Oct 2013
My @TriangleDevops talk from 2013-10-17. I covered the work that led us to @NetflixOSS (Acme Air), the work we did on the cloud prize (NetflixOSS on IBM SoftLayer/RightScale) and the @NetflixOSS platform (Karyon, Archaius, Eureka, Ribbon, Asgard, Hystrix, Turbine, Zuul, Servo, Edda, Ice, Denominator, Aminator, Janitor/Conformity/Chaos Monkeys of the Simian Army).
Microservices Architecture and Containers.
This document discusses microservices architecture and how containers can help solve problems with monolithic architectures. It describes how microservices break applications into independent, scalable services that communicate over standard protocols. Containers provide standardized, isolated environments for microservices. Using containers for microservices allows for easier deployment and management of distributed applications across infrastructure.
The Mushroom Cloud Effect or What Happens When Containers Fail? by Alois Mayr...
This document discusses the "mushroom cloud effect" that can occur when containers fail in highly dynamic container environments. It describes how a failure in one container due to a lack of disk space on the host led to cascading failures that affected many dependent services. The failure spread as container health checks failed and orchestration rescheduled containers, eventually exhausting disk space and preventing any new containers from running. Automated monitoring is needed to pinpoint the root cause of such cascading failures in complex systems with many interdependent containers and services.
Preparing your dockerised application for production deployment
You’ve got your application dockerised for development. That process is working smoothly, and you’re gaining a lot of the benefits that docker gives you - environments are trivial to setup, independent of platform, and they are consistent for everyone on your team.
How do you go about taking the next step so that your application is deployed into a scalable and reliable production setup?
How do you create deployment artefacts which are built with consistency and transparency? How do you manage environment variables between staging and production environments? How do you perform actions / schedule processes in one environment and not another?
In this talk we will discuss what you need to do to get your dockerised application ready for deployment into a production environment.
Webinar: Achieving Economies of Web Scale in Your Enterprise with Containeriz...
To watch the recording of this webinar please use the below URL:
http://wso2.com/library/webinars/2016/07/achieving-economies-of-web-scale-in-your-enterprise-with-containerization/
Containers are the next big step in cloud architectures. Unlike virtual machines, containers share the same underlying operating systems. This leads to significantly smaller overhead and higher efficiency. Orchestration technologies such as Kubernetes allow to achieve scale by managing multi-container deployments. Containers enable microservice architectures, near-instant scaling, application portability and cross-datacenter solutions. In this webinar, Dmitry will explore how these technologies can bring the cloud economies of scale to your data center.
Introduction to Kubernetes
Kubernetes is an open-source system for managing containerized applications across multiple hosts. It includes key components like Pods, Services, ReplicationControllers, and a master node for managing the cluster. The master maintains state using etcd and schedules containers on worker nodes, while nodes run the kubelet daemon to manage Pods and their containers. Kubernetes handles tasks like replication, rollouts, and health checking through its API objects.
Deploying WSO2 Middleware on Kubernetes
This slide deck will explore the advantages and requirements of Kubernetes. It will also demonstrate how to deploy WSO2 middleware on Kubernetes.
The state of containerd
containerd is an industry-standard core container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc..
2016 10-26 docker meetup - kubernetes on open stack
This document discusses Kubernetes implementation on OpenStack. Key points include:
1. Kubernetes was chosen for its powerful features like auto-scaling and multi-AZ capabilities, though storage on OpenStack Ceph presented challenges.
2. Authentication was implemented using certificates and authorization used RBAC. Ingress was implemented using HAProxy for layer 4 and 7 load balancing.
3. Immutable CoreOS servers were created using cloud-config templates for easy, automated cluster creation with infrastructure as code.
4. Logging was set up with Fluentd and monitoring with Prometheus/Grafana. Configuration files were managed in Git.
Docker Advanced registry usage
The document discusses advanced configuration and usage of Docker registries. It describes what a registry is for storing Docker images and supporting various storage backends. It then covers configuring a registry to add features like a search index using SQLite, mirroring another registry, and adding a Redis cache. The document concludes by discussing extending the registry code for customization.
Structured Container Delivery by Oscar Renalias, Accenture
With tools like Docker Toolbox, the entry barrier to Docker and containers is rather low. However, it takes a lot more to design, build and run an entire container platform, at scale, for production applications.
This talk will focus on why it is important to have a well-defined reference model for building container platforms that guides container engineers and architects through the process of identifying platform concerns, patterns, components as well as the interactions between them in order to deliver a set of platform capabilities (service discovery, load balancing, security, and others) to support containerized applications using existing tooling.
As part of this session will also see how a container architecture has enabled real projects in their delivery of container platforms.
Service Discovery In Kubernetes
Service Discovery in kubernetes is all about how services of kubernetes get discovered internally and externally. How does a single POD communicate to another POD the within the cluster and how does a user request reach to a specific POD in the cluster? These are some questions that are answered by this TOPIC.
Persistent Data Storage for Docker Containers by Andre Moruga
This talk explores the best approaches to integrating storage with application containers such as Docker. The statelessness of application containers presents challenges when it comes to the use and management of storage resources in a dynamic and multi-server environment. This talk particularly explores the ways in which Virtuozzo Storage offer a compelling solution to these challenges.
Efficient Parallel Testing with Docker by Laura Frank
Fast and efficient software testing is easy with Docker. We often
use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, Machine, and Compose can work together to make your tests fast.
Efficient Parallel Testing with Docker
Fast and efficient software testing is easy with Docker. We often
use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, and Compose can work together to make your tests fast.
More Related Content
What's hot
Automation CI CD with Gitlab, Java, docker on Hidora - Jelastic
How to commit, build, deploy and run with Gitlab for a Java application used on docker, hosted on the swiss hoster Hidora, Jelastic platform.
Build pipelines with bitbucket for Magento
Pradip Shah discusses using Bitbucket Pipelines to build continuous integration and delivery pipelines for WordPress and Magento applications. Bitbucket Pipelines uses Docker containers to run build steps defined in a YAML configuration file. Key benefits include reducing the cost of testing with containerization, and the ability to automate deployments through continuous delivery. The presentation provides an example Magento 2 pipeline and discusses concepts like separating concerns between Docker images and pipeline steps.
Kubernetes in Docker
1. Docker EE will include an unmodified Kubernetes distribution to provide orchestration capabilities alongside Docker Swarm.
2. When running mixed workloads across orchestrators, resource contention is a risk and it is recommended to separate workloads by orchestrator on each node for now.
3. Docker EE aims to address the shortcomings of running mixed workloads to better support this in the future.
Deep Dive into Kubernetes - Part 1
This presentation includes information on Kubernetes Architecture, Container Orchestration, Internal Routing, External Routing, Configuration Management, Credentials Management, Persistent Volumes, Rolling Out Updates, Autoscaling, Package Management, and a Hello World example using Helm.
Docker microservices and the service mesh
The nature of containerized, cloud-native applications is rapidly advancing with a fundamentally different architecture that will rely on service meshes with smarter proxies, traffic management, and enhanced observability for cooperating microservices, serverless functions, and complex workflows. In this session we will highlight the features that characterize this architectural transformation in the Docker cloud-native ecosystem.
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
The document discusses containers and how they can be used for stateless and stateful applications. It notes that while containers provide CPU and memory isolation, traditional storage systems cannot handle the scale and performance needs of containerized applications. The Robin Containerization Platform aims to address these issues with its container-aware storage layer that can support large volumes of variable sized volumes, quickly create and destroy snapshots and clones, and provide end-to-end performance isolation and QoS controls from applications to storage. It also decouples compute and storage to allow independent scaling and portability without data loss.
berne.*tesday1
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for self-healing, horizontal scaling, and load balancing. Kubernetes manages the computing infrastructure, networking, storage, and applications running on containers. It has features like automated scheduling, rollouts and rollbacks, load balancing, environment consistency, and auto-scaling of infrastructure. Kubernetes uses objects like pods, deployments, replica sets, and stateful sets to define and run application containers.
NetflixOSS for Triangle Devops Oct 2013
My @TriangleDevops talk from 2013-10-17. I covered the work that led us to @NetflixOSS (Acme Air), the work we did on the cloud prize (NetflixOSS on IBM SoftLayer/RightScale) and the @NetflixOSS platform (Karyon, Archaius, Eureka, Ribbon, Asgard, Hystrix, Turbine, Zuul, Servo, Edda, Ice, Denominator, Aminator, Janitor/Conformity/Chaos Monkeys of the Simian Army).
Microservices Architecture and Containers.
This document discusses microservices architecture and how containers can help solve problems with monolithic architectures. It describes how microservices break applications into independent, scalable services that communicate over standard protocols. Containers provide standardized, isolated environments for microservices. Using containers for microservices allows for easier deployment and management of distributed applications across infrastructure.
The Mushroom Cloud Effect or What Happens When Containers Fail? by Alois Mayr...
This document discusses the "mushroom cloud effect" that can occur when containers fail in highly dynamic container environments. It describes how a failure in one container due to a lack of disk space on the host led to cascading failures that affected many dependent services. The failure spread as container health checks failed and orchestration rescheduled containers, eventually exhausting disk space and preventing any new containers from running. Automated monitoring is needed to pinpoint the root cause of such cascading failures in complex systems with many interdependent containers and services.
Preparing your dockerised application for production deployment
You’ve got your application dockerised for development. That process is working smoothly, and you’re gaining a lot of the benefits that docker gives you - environments are trivial to setup, independent of platform, and they are consistent for everyone on your team.
How do you go about taking the next step so that your application is deployed into a scalable and reliable production setup?
How do you create deployment artefacts which are built with consistency and transparency? How do you manage environment variables between staging and production environments? How do you perform actions / schedule processes in one environment and not another?
In this talk we will discuss what you need to do to get your dockerised application ready for deployment into a production environment.
Webinar: Achieving Economies of Web Scale in Your Enterprise with Containeriz...
To watch the recording of this webinar please use the below URL:
http://wso2.com/library/webinars/2016/07/achieving-economies-of-web-scale-in-your-enterprise-with-containerization/
Containers are the next big step in cloud architectures. Unlike virtual machines, containers share the same underlying operating systems. This leads to significantly smaller overhead and higher efficiency. Orchestration technologies such as Kubernetes allow to achieve scale by managing multi-container deployments. Containers enable microservice architectures, near-instant scaling, application portability and cross-datacenter solutions. In this webinar, Dmitry will explore how these technologies can bring the cloud economies of scale to your data center.
Introduction to Kubernetes
Kubernetes is an open-source system for managing containerized applications across multiple hosts. It includes key components like Pods, Services, ReplicationControllers, and a master node for managing the cluster. The master maintains state using etcd and schedules containers on worker nodes, while nodes run the kubelet daemon to manage Pods and their containers. Kubernetes handles tasks like replication, rollouts, and health checking through its API objects.
Deploying WSO2 Middleware on Kubernetes
This slide deck will explore the advantages and requirements of Kubernetes. It will also demonstrate how to deploy WSO2 middleware on Kubernetes.
The state of containerd
containerd is an industry-standard core container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc..
2016 10-26 docker meetup - kubernetes on open stack
This document discusses Kubernetes implementation on OpenStack. Key points include:
1. Kubernetes was chosen for its powerful features like auto-scaling and multi-AZ capabilities, though storage on OpenStack Ceph presented challenges.
2. Authentication was implemented using certificates and authorization used RBAC. Ingress was implemented using HAProxy for layer 4 and 7 load balancing.
3. Immutable CoreOS servers were created using cloud-config templates for easy, automated cluster creation with infrastructure as code.
4. Logging was set up with Fluentd and monitoring with Prometheus/Grafana. Configuration files were managed in Git.
Docker Advanced registry usage
The document discusses advanced configuration and usage of Docker registries. It describes what a registry is for storing Docker images and supporting various storage backends. It then covers configuring a registry to add features like a search index using SQLite, mirroring another registry, and adding a Redis cache. The document concludes by discussing extending the registry code for customization.
Structured Container Delivery by Oscar Renalias, Accenture
With tools like Docker Toolbox, the entry barrier to Docker and containers is rather low. However, it takes a lot more to design, build and run an entire container platform, at scale, for production applications.
This talk will focus on why it is important to have a well-defined reference model for building container platforms that guides container engineers and architects through the process of identifying platform concerns, patterns, components as well as the interactions between them in order to deliver a set of platform capabilities (service discovery, load balancing, security, and others) to support containerized applications using existing tooling.
As part of this session will also see how a container architecture has enabled real projects in their delivery of container platforms.
Service Discovery In Kubernetes
Service Discovery in kubernetes is all about how services of kubernetes get discovered internally and externally. How does a single POD communicate to another POD the within the cluster and how does a user request reach to a specific POD in the cluster? These are some questions that are answered by this TOPIC.
Persistent Data Storage for Docker Containers by Andre Moruga
This talk explores the best approaches to integrating storage with application containers such as Docker. The statelessness of application containers presents challenges when it comes to the use and management of storage resources in a dynamic and multi-server environment. This talk particularly explores the ways in which Virtuozzo Storage offer a compelling solution to these challenges.
What's hot (20)
Automation CI CD with Gitlab, Java, docker on Hidora - Jelastic
Automation CI CD with Gitlab, Java, docker on Hidora - Jelastic
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
The Mushroom Cloud Effect or What Happens When Containers Fail? by Alois Mayr...
The Mushroom Cloud Effect or What Happens When Containers Fail? by Alois Mayr...
Preparing your dockerised application for production deployment
Preparing your dockerised application for production deployment
Webinar: Achieving Economies of Web Scale in Your Enterprise with Containeriz...
Webinar: Achieving Economies of Web Scale in Your Enterprise with Containeriz...
2016 10-26 docker meetup - kubernetes on open stack
2016 10-26 docker meetup - kubernetes on open stack
Structured Container Delivery by Oscar Renalias, Accenture
Structured Container Delivery by Oscar Renalias, Accenture
Persistent Data Storage for Docker Containers by Andre Moruga
Persistent Data Storage for Docker Containers by Andre Moruga
Similar to A Tail of Two Containers: How docker made ci great again
Efficient Parallel Testing with Docker by Laura Frank
Fast and efficient software testing is easy with Docker. We often
use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, Machine, and Compose can work together to make your tests fast.
Efficient Parallel Testing with Docker
Fast and efficient software testing is easy with Docker. We often
use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, and Compose can work together to make your tests fast.
Building Efficient Parallel Testing Platforms with Docker
We often use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, and Compose can work together to make your tests fast.
Continuous Delivery the hard way with Kubernetes
This talk shows three increasingly advanced levels of continuous delivery with Kubernetes and GitLab (as an example), arguing for a continuous delivery architecture which has an explicit _Release Manager_ component. We then propose Flux, the open source project which powers the _Deploy_ feature of Weave Cloud, as an implementation of that idea. This approach is the precursor to GitOps.
Docker
Docker is a tool designed to make it easier to create, deploy, and run applications
by using containers. Containers allow a developer to package up
an application with all of the parts it needs, such as libraries and other dependencies,
and ship it all out as one package. By doing so, thanks to the
container, the developer can rest assured that the application will run on
any other Linux machine regardless of any customized settings that machine
might have that could differ from the machine used for writing and testing
the code.
In a way, Docker is a bit like a virtual machine. But unlike a virtual
machine, rather than creating a whole virtual operating system, Docker allows
applications to use the same Linux kernel as the system that they’re
running on and only requires applications be shipped with things not already
running on the host computer. This gives a significant performance boost
and reduces the size of the application.
Continuous Delivery the Hard Way with Kubernetes
The document discusses continuous delivery of software to Kubernetes using various tools. It proposes three architectures (V1, V2, V3) to connect version control, CI/CD systems, container registries, and Kubernetes. V3 introduces a "release manager" to decouple code versions from deployments, automate configuration updates, and allow rollbacks without pushing new code. The release manager watches for image changes, commits configuration updates, and controls deployments based on release policies for each environment.
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
This was a talk given at the second CT Software Developers Meetup (http://www.meetup.com/CT-Software-Developers-Meetup/). It covers how NorthPage is using Docker and Vagrant with a home grown Preview tool to increase the efficiency of the GitHub Pull Request Workflow.
Rami Sayar - Node microservices with Docker
The document discusses converting a monolithic Node.js application into microservices and deploying them using Docker. It begins by defining microservices and their benefits. It then describes converting a sample pizza ordering application into independent microservices for handling messages, serving the frontend, and providing an API. Next, it covers patterns for networking microservices, including using an API gateway. It concludes by demonstrating how to deploy the microservices to Docker containers and use an orchestration tool like Kubernetes to manage them.
Webinar: From Development to Production with Docker and MongoDB
In this talk we review what Docker is and why it's important to Developers, Admins and DevOps.
We also cover the following topics:
- Using Docker to Orchestrate a multi container application (Flask + MongoDB)
- Injecting HAProxy and other production requirements as we deploy to production
- Scaling the Web and MongoDB cluster to grow to meet demand
This presentation includes an interactive demo showcasing the core Docker components (Machine, Engine, Swarm and Compose) as well as some of Docker's new components (libnetowrk, runC) from the experimental branch along with MongoDB. We hope you will see how much simpler Docker can make building and deploying multi-node applications.
<hr>
<b>What's next?</b>
See how you can push MongoDB performance to meed the needs of your mission-critical app with our best practices for MongoDB operations.
<a>Read the guide</a>
Docker Hub: Past, Present and Future by Ken Cochrane & BC Wong
This document provides an overview of Docker Hub, including its history and features. Docker Hub is a cloud registry service that allows users to share applications and automate workflows. It currently has over 240,000 users, 150,000 repositories, and handles over 1 billion pulls annually. The document discusses Docker Hub's growth over time and upcoming features like improved performance, automated builds using Kubernetes, and a redesigned user interface.
Containers 101
Dockerized containers are the current wave that promising to revolutionize IT. Everybody is talking about containers, but a lot of people remain confused on how they work and why they are different or better than virtual machines. In this session, Black Duck container and virtualization expert Tim Mackey will demystify containers, explain their core concepts, and compare and contrast them with the virtual machine architectures that have been the staple of IT for the last decade.
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
They provide the workload isolation and security advantages of VMs. but at the same time maintain the speed of deployment and usability of containers.by using kata containers, instead of namespace, small virtual machines are created on the kernel and be strongly isolated. The technology of Kata Containers is based on KVM hypervisor. That’s why the level of isolation is equivalent to typical hypervisors. This session will focus on a live production phase when choosing kata instead of docker, and why they are preferable
Although containers provides software-level isolation of resources, the kernel needs to be shared. That’s why the isolation level in terms of security is not so high when compared with hypervisors.This learns to shift from Docker as the de facto standard to Kata containers and learn how to obtain higherl level of security
Getting Started with Docker
This document discusses Docker and provides an introduction and overview for getting started with Docker. It begins with discussing the challenges of managing complex software stacks across different environments and how Docker addresses this through containerization and separation of concerns. It then covers downloading and installing Docker, basic Docker commands like run, images, ps, and explains a "Hello World" example. Finally, it demonstrates building a simple Whalesay image and running MySQL and WordPress in linked Docker containers using both the Docker CLI and Docker Compose.
Docker and the K computer
The document discusses using Docker on the K computer supercomputer. Docker could help with compiling and running software by providing consistent software environments. At compile time, Docker containers could be used on login nodes to install cross-compilers and compile software in a consistent environment. At runtime, Docker could potentially be installed on K computer compute nodes, but the operating system kernel would need to be updated from 2.6.25 to 3.8 to support Docker, requiring patching 84 files. Issues include compiler licenses and administrative policies. A Docker-based IaaS approach is proposed to allow compiling software in containers on a server to overcome dependency and installation challenges.
IBM Bluemix Paris Meetup #14 - Le Village by CA - 20160413 - Introduction à D...
This document discusses Docker and how IBM uses Docker for ODM (Operational Decision Management). Some key points:
- Docker allows decoupling applications from the underlying infrastructure and providing consistent runtime environments and operations.
- IBM leverages Docker for ODM on Cloud, running ODM in Docker containers on a predefined set of VMs managed by Docker Swarm.
- Internally, IBM is working to Dockerize existing ODM runtimes by running product components like RES and Decision Center in separate Docker containers connected via REST APIs. This aims to provide a homogeneous software delivery, topology and operations using Docker.
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
Docker Desktop and Enterprise Edition now both include Kubernetes as an optional orchestration component. This talk will explain how to use Docker Desktop (Mac or Windows) to develop and debug a cloud native application, then how Docker Enterprise Edition helps you deploy it to Kubernetes in production.
DCEU 18: Building Your Development Pipeline
This document discusses building a development pipeline using containers. It outlines using containers for building images, automated testing, security scanning, and deploying to production. Containers make environments consistent and reproducible. The pipeline includes building images, testing, security scanning, and promoting images to production. Methods discussed include using multi-stage builds to optimize images, leveraging Buildkit for faster builds, and parallel testing across containers. Automated tools are available to implement rolling updates and rollbacks during deployments.
A curtain-raiser to the container world Docker & Kubernetes
Agenda
1. The changing landscape of IT Infrastructure
2. Containers - An introduction
3. Container management systems
4. Kubernetes
5. Containers and DevOps
6. Future of Infrastructure Mgmt
About the talk
In this talk, you will get a review of the components & the benefits of Container technologies - Docker & Kubernetes. The talk focuses on making the solution platform-independent. It gives an insight into Docker and Kubernetes for consistent and reliable Deployment. We talk about how the containers fit and improve your DevOps ecosystem and how to get started with containerization. Learn new deployment approach to effectively use your infrastructure resources to minimize the overall cost.
Dockercon 2015 Recap
The document recaps announcements from Dockercon including improved security features for Docker like process monitoring and fine-grained access control. It outlines updates to networking including multi-host networking and plugins for networking and volumes. Docker 1.7 was announced with improvements to building, networking stack, and Swarm orchestration. The OpenContainer Project was also announced to develop universal container standards.
HOW TO DRONE.IO IN CI/CD WORLD
eugene.starchenko@gmail.com
https://github.com/eugenestarchenko
Cloud Engineer
@ AWS Meetup Kiev #4
Similar to A Tail of Two Containers: How docker made ci great again (20)
Efficient Parallel Testing with Docker by Laura Frank
Efficient Parallel Testing with Docker by Laura Frank
Building Efficient Parallel Testing Platforms with Docker
Building Efficient Parallel Testing Platforms with Docker
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
Webinar: From Development to Production with Docker and MongoDB
Webinar: From Development to Production with Docker and MongoDB
Docker Hub: Past, Present and Future by Ken Cochrane & BC Wong
Docker Hub: Past, Present and Future by Ken Cochrane & BC Wong
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
stackconf 2020 | Replace your Docker based Containers with Cri-o Kata Contain...
IBM Bluemix Paris Meetup #14 - Le Village by CA - 20160413 - Introduction à D...
IBM Bluemix Paris Meetup #14 - Le Village by CA - 20160413 - Introduction à D...
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
A curtain-raiser to the container world Docker & Kubernetes
A curtain-raiser to the container world Docker & Kubernetes
Recently uploaded
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Best 20 SEO Techniques To Improve Website Visibility In SERP
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Infrastructure Challenges in Scaling RAG with Custom AI models
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Recently uploaded (20)
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
A Tail of Two Containers: How docker made ci great again
- 1. A Tale of Two Containers How Docker made CI Great Again
- 2. About us Kyle Rames @krames github.com/krames Brendan Fosberry @brendanfosberry github.com/bfosberry 2
- 4. Agenda ● History of Codeship Platform ● Checkbot (LXC) ● Jet (Docker) ● Future of CI/CD
- 6. Checkbot
- 7. Checkbot 39K Builds Per Day 7.8M Builds Per Year
- 8. Jet
- 9. Jet 1.5K Builds Per Day 115K Builds total
- 11. Benefits of LXC • Impose resource limits • Containers are isolated • Separation of build environment and Codeship infrastructure • Customer code is secure • Consistent build environment and experience • Automatable
- 12. How we use LXC • Universal build image • Pre-installed languages, services, libraries
- 13. Screenshot of test commands and setup
- 15. Checkbot (LXC)
- 16. How we use LXC • Build server scaling group • Assign user to build server
- 17. Issues for Developers • Shared system requirements for every user. • Maintaining parity with development & production
- 18. Issues for Codeship • Many parallel services leave a large footprint • Lack of tooling around LXC
- 19. It works!
- 21. Benefits of Docker • Standardization • Strong tooling and automation support • Parity between dev, test, and prod • BYO Container and OS • Empowered Users
- 22. How we use docker • Provision Docker resources • Repo-controlled build - zero config • Layer on Codeship features • Docker-centric support and integrations
- 23. How users run builds • Describe services • Write custom images where needed • Committed to SCM
- 24. How users run builds
- 25. How users run builds • Define pipeline • References service • Tests, compilation, deployments, notifications • Control parallelism • Control flow • Committed to SCM
- 26. How users run builds
- 27. How users run buids
- 28. How users run builds Parity through Docker
- 29. Maintain Parity ● Credential & Dockercfg support ● Share Image build cache ● Tooling ● Development environments
- 30. Issues for Developers • High Barrier to entry • Requires Docker understanding • Requires careful configuration • More isolation <-> Less visibility
- 31. Issues for Codeship • Heavily tied to Docker • Upstream changes break our features • Docker bugs are our bugs
- 32. Demo
- 33. The future of CI/CD Powered by Codeship
- 34. Containers ● Isolation ● Portability ● Resource control
- 35. Docker ● Phase 1: Docker ● Phase 2: ??? ● Phase 3: Profit
- 36. Docker ● Control ○ Automation ○ Configurability ● Parity ● Lowest common denominator
- 37. CI/CD ● Configuration ● Execution ● Notification
- 38. CI/CD ● Native ● Personal ● Instantly available ● Intelligent
- 39. Q&A