It Takes an Ecosystem to Build a Production Application by Ross Gardler

•Download as PPTX, PDF•

2 likes•3,831 views

Docker has made containerizing applications easy. This ease of use has given birth to a vibrant ecosystem of innovation. As more workloads become containerized and more tooling emerges from this ecosystem it is becoming harder to define the best dev/test/prod environment for your work. In this session we’ll discuss Microsoft’s strategy for supporting your Docker container work. We’ll discuss how important the open source ecosystem is to this strategy and we’ll take a look at the Azure Container Service which provides a Docker native experience using only the open source tools and APIs built by the community. We’ll see how this approach allows established tooling to be used within ACS and how ACS can help facilitate innovation in the ecosystem. Finally, we’ll take a look at some of the Microsoft led innovations in the pipeline.

Technology

It takes an ecosystem to build a
production application
Ross Gardler
@rgardler

Docker, Docker, Docker
• They created open source software to build and manage containers
• Even I can create and run them
• Docker CLI, Docker Engine, Docker Swarm, Docker Compose, Docker Machine and more…

Docker makes it Easy
• Many people agree that:
• Development is easier with Docker
• Scaling is easier with Docker
• Continuous Integration is easier with Docker
• Testing is easier with Docker
• So just adopt Docker and life is easier, right?
• Applications don’t consist of a single container
• How do we manage applications?

Docker is an Ecosystem
• Docker make Containerization accessible
• It has spawned an Ecosystem
• This session is about why the Ecosystem is improtant

“A model where IT organizations and developers
can work together to build, ship and run their
applications anywhere.” https://blog.docker.com/2016/02/containers-as-a-service-caas/

Docker offers portability
• Docker containers are portable
• Orchestrate them with Docker Swarm, Kubernetes, Apache Mesos, DC/OS etc.
• Run them on Azure, AWS, Google, Digital Ocean etc.
• But, what about applications consisting of many containers?

An orchestrator is not enough
• Continuous Delivery
• Resource Management
• High Availability
• Storage
• Networking
• Load Balancing
• Service Discovery
• Monitoring
• … and more

What about everything else?
• Continuous Delivery
• Resource Management
• High Availability
• Storage
• Networking
• Load Balancing
• Service Discovery
• Monitoring
• … and more
• Who’s going to configure it all?
• What about optimizing for the
infrastructure of choice?
• Are you locking into a cloud
provider because of your choice
of orchestrator?

Building a Container Hosting Platform
Docker Engine
Container ToolingService Tooling • Not everything is open
source software
• There is a limit to portability
when it comes to electricity
and CPUs
• How do you minimize the
lock-in to specific hardware
and software infrastructure

Azure Container Service:
Maintaining Portability of Workloads
Containers
Container ToolingService Tooling
Layer Supported Technologies
Configuration as
Code
ARM, Dockerfile, Docker
Compose, Marathon.json
High Availability Azure, VM Scale Sets
Container
orchestration
Docker Swarm, Chronos,
Marathon, Apache Mesos
Continuous
Integration
Jenkins, VSTS
Monitoring OMS, Statsd, Cadvisor,
Datadog
Networking Docker Networking
Storage Docker volumes
??? ???

Infrastructure Layer
(Electricity, CPUs, Cables restrict portability)
Azure Container Service:
Maintaining Portability of Workloads
Management Layer
(Variety of solutions, some more portable than others)
Application Layer
(Docker Containers promise portability)

Demo:
ACS Deployment
http://portal.azure.com

Bringing it All Together
Azure
SQL Database
Load Balancer
Containers in Swarm

Private Preview:
Windows Server Containers on
Azure Container Service
Self nominate at https://aka.ms/acswin

Where will the ecosystem take
you from here?

Viewers also liked

Docker Community Edition (CE) and Enterprise Edition (EE) are the best expressions of the Docker Platform to date. Whether you’re a developer, an ops team or a enterprise IT-team member, and no matter the infrastructure, Docker CE and EE gives you a way to install, upgrade and maintain Docker with the support and assurances required for your particular workload. Both Docker CE and EE are available on a wide range of popular operating systems (including Windows Server 2016) and cloud infrastructure. Developers and devOps have the freedom to run Docker on their favorite infrastructure without risk of lock-in. Michael Friis will give an overview of both editions and highlight the big enhancements to the lifecycle, maintainability and upgradability of Docker.

Docker Online Meetup: Announcing Docker CE + EE

Docker, Inc.

A session covering the container workflow from the developers inner loop, CI/CD, to deployment in a container orchestration solution. We'll cover Visual Studio Code from a Mac, Visual Studio Code from Windows with Bash and Visual Studio as an in-container local development environment targeting both Windows and Linux Containers. We'll walk through CI, Validation and CD to the Azure Container Service running Docker Swarm as one example of how you can convert your existing config as code and VM deployments to the containerized workflows startups and early adopter enterprises are using today.

Dockerizing Windows Server Applications by Ender Barillas and Taylor Brown

Docker, Inc.

Fugro is a multinational enterprise that collects and provides highly specialized interpretation of geological data for a number of industries, at land and at sea. The company recently launched OARS (Office Assisted Remote Services), an innovation which uses advanced technology to reduce, and potentially eliminate, the need for surveyors onboard sea-going vessels, optimizing project crewing, safety and efficiency. By keeping skilled staff onshore and using an Internet of Things platform model, Fugro’s OARS project provides faster interpretation of data and decisions, better access to information across regions Hear how Fugro and consulting partner Flux7 created a solution with Docker and Amazon Web Services at its center that provides a high degree of uptime, ensures data is secure and enables portability so that environments that can be quickly replicated in new global regions on demand. Learn how Docker is being used as a key component in Fugro’s continuous delivery cycle and see how Docker is also used to create redundancy that ensures high uptime for Fugro’s 24X7 requirements.

Overseeing Ship's Surveys and Surveyors Globally Using IoT and Docker by Jay ...

Docker, Inc.

What is the right balance between moving fast, innovating, experimenting with new technology, and protecting the personal data of our customers and interests of our stakeholders? How can we safely try new ideas in production without risking costly downtime? Does the utopia where developers are free from lock-in and operators enjoy the calm of a steadily running system exist in the real world? Is it possible to have open platforms with better security? At Kroger Digital we are still working through these questions every day but are redesigning our systems with the goals of true operational maturity and security. Discover how we are building capabilities for monitoring, A/B testing, and continuous delivery with Docker Datacenter, plugins, and open source building blocks such as NGiNX, ElasticSearch, and more.

Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...

Docker, Inc.

Customer trust and security is paramount for Salesforce. While containerization is great for DevOps due to flexibility, speed, isolation, transient existence, ease of management and patching, it becomes a challenging environment when the sensitivity level of the data traversing the environment increases. Monitoring systems, applications and network; performing disk, memory and network forensics in case of an incident; and vulnerability detection can easily become daunting tasks in such a volatile environment. In this presentation we would like to discuss the infrastructure we have built to address these issues and to secure our Docker container platform while we rapidly containerize Salesforce. Our solutions focus on securing the container pipeline, building security into the architecture, monitoring, Docker forensics (disk, memory, network), and automation. We also would like to demonstrate some of our live memory analysis capabilities we leverage to assure container and application integrity during execution.

Securing the Container Pipeline at Salesforce by Cem Gurkok

Docker, Inc.

Docker for Mac and Windows were released in beta in March, and provide lots of new features that users have been clamouring for including: file system notifications, simpler file sharing, and no Virtualbox hassles. During this talk, I will give the inside guide to how these products work. We will look at all the major components and how they fit together to make up the product. This includes a technical deep dive covering the hypervisors for OSX and Windows, the custom file sharing code, the networking, the embedded Alpine Linux distribution, and more.

Docker for Mac and Windows: The Insider's Guide by Justin Cormack

Docker, Inc.

In this session we will talk about HealthDirect’s journey with Docker. We will follow the life cycle of a container through our CD process to its home in our swarm cluster with just a git commit thanks to configuration management. We will cover the CD process for Docker, Docker swarm, Docker networking and service discovery. The audience will leave with a solid foundation of how to build a production ready swarm cluster (A github repo with code will be given). They will also have the knowledge of how to implement a CD framework using Docker.

Docker in Production, Look No Hands! by Scott Coulton

Docker, Inc.

Our motto "Imagination at work" is the belief in driving innovation that builds, powers, moves and cures the world. At GE, we have 9,000+ legacy apps powering 9 business units across every major industry from oil and gas, healthcare to household appliances generating over $148B in revenue. With legacy apps and infrastructure, our app teams were facing issues with long development cycles, deploying apps and scaling features and services. How do you migrate legacy data center built apps to a new microservices and hybrid cloud architecture at this organizational scale and business diversity? In this talk, the GE Digital team will share their journey to a modern microservices platform built with Docker Datacenter, Rails, Chef, Sensu, Gems, AWS, Azure and Rackspace on-prem to modernize these apps and automate processes to enable agile development and rapid deployment. This session will cover both the technical and organizational sides of the project to take legacy apps and infrastructure at GE to multi cloud microservices.

Build Fast, Deploy Fast: Innovating in the Enterprise by Imran Raja and Andy Lim

Docker, Inc.

The SDACK architecture stands for Spark, Docker, Akka, Cassandra, and Kafka. At TrendMicro, we adopted the SDACK architecture to implement a security event inspection platform for APT attack analysis. In this talk, we will introduce SDACK stack with Spark lambda architecture, Akka and Kafka for streaming data pipeline, Cassandra for time series data, and Docker for microservices. Specifically, we will show you how we Dockerize each SDACK component to facilitate the RD team of algorithms development, help the QA team test the product easily, and use the Docker as a Service strategy to ship our products to customers. Next, we will show you how we monitor each Docker container and adjust the resource usage based on monitoring metrics. And then, we will share our Docker security policy which ensures our products are safety before shipping to customers. After that, we'll show you how we develop an all-in-one Docker based data product and scale it out to multi-host Docker cluster to solve the big data problem. Finally, we will share some challenges we faced during the product development and some lesson learned.

Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...

Docker, Inc.

Riot builds a lot of software. At the start of 2015 we were looking at 3000 build jobs over a hundred different applications and dozens of teams. We were handling nearly 750 jobs per hour and our build infrastructure needed to grow rapidly to meet demand. We needed to give teams total control of the “stack” used to build their applications and we needed a solution that enabled agile delivery to our players. On top of that, we needed a scalable system that would allow a team of four engineers to support over 250. After as few explorations, we built an integrated Docker solution using Jenkins that accepts docker images submitted as build environments by engineers around the company . Our “containerized” farm now creates over 10,000 containers a week and handles nearly 1000 jobs at a rate of about 100 jobs an hour. In this occasionally technical talk, we’ll explore the decisions that led Riot to consider Docker, the evolutionary stages of our build infrastructure, and how the open source and in-house software we combined to achieve our goals at scale. You’ll come away with some best practices, plenty of lessons learned, and insight into some of the more unique aspects of our system (like automated testing of submitted build environments, or testing node.js apps in containers with Chromium and xvfb).

Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart

Docker, Inc.

Docker leverages capabilities in Linux like namespaces and cgroups to enable containers and then builds tooling on top to enable users to build distributed apps. A common question is "What about Docker support for Windows?" In this session the Windows engineering leads will dive deep into the primitives within Windows to enable an awesome Docker experience on Windows. This session will also include a live demo of Docker and Windows Server.

Windows Server and Docker - The Internals Behind Bringing Docker and Containe...

Docker, Inc.

As a multi-national bank, Societe General IT infrastructure has thousands of apps, almost every bit of technology deployed and compliance requirements. Our vision is to broadly transform traditional bank IT to be agile and fast. Speed is critical in a digital economy and at Societe Generale we are building a new execution platform with Docker that provides IT containers, middleware and infrastructure as a service and orchestration. In this session we will share the technical and organizational steps of our journey from how we defined and architected a PaaS for our entity; with service catalog, service topologies, ambassadors with Docker Datacenter, continuous integration and what’s next.

Fully Orchestrating Applications, Microservices and Enterprise Services with ...

Docker, Inc.

Watson developer cloud delivers Watson Cognitive services as micro services on the cloud that are being used by many IBM Watson customers. The micro services were packaged in ova at the first release. There were some drawbacks in ova deployment in the cloud. We gradually switched to use docker. As a result, the service deployment time and start up time are significantly improved. It also greatly simplified our continuous delivery process since our services run on both Intel and Power platform and we have offerings on our public cloud, dedicated cloud as well as customers’ on premise cloud. With minimal deployment time and quick startup time, Docker makes our dynamic creation of service instance on the fly per customer request possible.

Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...

Docker, Inc.

Small is the new big, and for good reason. The benefits of microservices and service-oriented architecture have been extolled for a number of years, yet many forge ahead without thinking of the impact the users of the services. Consuming on micro services can be enjoyable as long as the developer experience has been crafted as finely as the service itself. But just like with any other product, there isn’t a single kind of consumer. Together we will walk through some typical kinds of consumers, what their needs are, and how we can create a great developer experience using brains and tools like Docker.

Making Friendly Microservices by Michele Titlol

Docker, Inc.

Contribute 101: Engine/Swarm/ContainerD by Phil Estes

Docker, Inc.

Docker Online Meetup #22: Docker Networking

Docker, Inc.

Unikernels: the rise of the library hypervisor in MirageOS

Docker, Inc.

Talk from Docker SF Meetup #50 Abstract: Docker swarm mode enables users to manage their applications with service primitives. In this talk we demonstrate how to do service upgrades without impacting your application. The Healthcheck feature provides health indication for a container. Coming up in Docker 1.13 release, Docker Swarm can connect healthcheck result with load balancer to implement no-loss service upgrade. Speaker Biographies: Nishant Totla is a software engineer at Docker, and works on the core open source team. He is currently working on Docker SwarmKit and Docker Swarm. Prior to Docker, he was a PhD student at UC Berkeley, doing research on programming languages. In his spare time, he enjoys long-distance running, biking, and other outdoor activities. Nishant tweets at @nishanttotla. Dongluo Chen is a software engineer at Docker focusing on orchestration and container development. Before Docker he was software engineer manager at Microsoft Azure building and automating global data centers. He worked at France Telecom (Orange) and the Ohio State University as research scientist in networking area.

Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...

Docker, Inc.

Docker and Microsoft - Windows Server 2016 Technical Deep Dive

Docker, Inc.

Containerd - core container runtime component

Docker, Inc.

Viewers also liked (20)

Docker Online Meetup: Announcing Docker CE + EE

Dockerizing Windows Server Applications by Ender Barillas and Taylor Brown

Overseeing Ship's Surveys and Surveyors Globally Using IoT and Docker by Jay ...

Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...

Securing the Container Pipeline at Salesforce by Cem Gurkok

Docker for Mac and Windows: The Insider's Guide by Justin Cormack

Docker in Production, Look No Hands! by Scott Coulton

Build Fast, Deploy Fast: Innovating in the Enterprise by Imran Raja and Andy Lim

Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...

Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart

Windows Server and Docker - The Internals Behind Bringing Docker and Containe...

Fully Orchestrating Applications, Microservices and Enterprise Services with ...

Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Clo...

Making Friendly Microservices by Michele Titlol

Contribute 101: Engine/Swarm/ContainerD by Phil Estes

Docker Online Meetup #22: Docker Networking

Unikernels: the rise of the library hypervisor in MirageOS

Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...

Docker and Microsoft - Windows Server 2016 Technical Deep Dive

Containerd - core container runtime component

More from Docker, Inc.

Raymond Arifianto, AccelByte and Mark Mandel, Google - We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.

Containerize Your Game Server for the Best Multiplayer Experience

Docker, Inc.

Nicholas Dille, Haufe-Lexware + Docker Captain - Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.

How to Improve Your Image Builds Using Advance Docker Build

Docker, Inc.

Lukonde Mwila, Entelect - As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Kevin Jones, NGNIX - NGINX is one of the most popular images on Docker Hub and has been at the forefront of the web since the early 2000's. In this talk we will discuss how and why NGINX's lightweight and powerful architecture makes it a very popular choice for securing containerized applications as a sidecar reverse proxy within containers. We will highlight important aspects of application security that NGINX can help with, such as TLS, HTTP, AuthN, AuthZ and traffic control.

Securing Your Containerized Applications with NGINX

Docker, Inc.

Kathleen Juell, Digital Ocean - Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself. This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.

How To Build and Run Node Apps with Docker and Compose

Docker, Inc.

Hands-on Helm

Docker, Inc.

Jeff Hajewski, Salesforce - There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.

Distributed Deep Learning with Docker at Salesforce

Docker, Inc.

James Fuller, webcomposite s.r.o. - Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Docker, Inc.

Fabian Stäber, Instana - In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.

Monitoring in a Microservices World

Docker, Inc.

Clemente Biondo, Engineering Ingegneria Informatica - When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Docker, Inc.

Chris Lauer, NOAA Space Weather Prediction Center - This is the story of how adopting a containerized workflow changed the way our small software team works at NOAA’s Space Weather Prediction Center. Our old architecture, a big ball of mud shared-database integration, just wasn’t cutting it - it was killing our agility. Over the past two years, our small team has adopted a microservice style architecture, using Docker with docker-compose and environment files as our deployment strategy for all new development. We’ve discovered the joys of using containers for identical dev, staging, and production environments. We work closely with scientists: much of the code we’re running has complicated and conflicting library dependencies. Docker captures these beautifully - we’ve even had some success teaching our scientists to use it! I’ll share what we’ve learned, some of the persistent challenges we face, and one place we really got it wrong. This talk builds off of a popular hallway track from DockerCon 2019.

Predicting Space Weather with Docker

Docker, Inc.

Brian Christner, 56k + Docker Captain - In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.

Become a Docker Power User With Microsoft Visual Studio Code

Docker, Inc.

How to Use Mirroring and Caching to Optimize your Container Registry

Docker, Inc.

Ashish Sharma, SS&C Eze - SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.

Monolithic to Microservices + Docker = SDLC on Steroids!

Docker, Inc.

Ara Pulido, Datadog - Container technologies, although not new, have increased their popularity in the past few years, with container orchestrators allowing companies around the world to adopt these technologies to help them ship and scale microservices with precision and velocity. Kubernetes is currently the most popular container orchestration platform, and while many organizations are migrating their workloads to it, Kubernetes is still relatively immature. New corner cases, errors, and quirks are regularly discovered as users push the boundaries of size and scale. When Datadog adopted Kubernetes we discovered some of these boundaries the hard way, and we continuously challenge and modify our infrastructure decisions in order to fit our use case. Join me in this talk for our story on what we learned while we scaled our Kubernetes clusters, the contributions to Kubernetes we made along the way, and how you can apply those learnings when growing your Kubernetes clusters from a handful to hundreds or thousands of nodes.

Kubernetes at Datadog Scale

Docker, Inc.

Andy Clemenko, StackRox - One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.

Labels, Labels, Labels

Docker, Inc.

Patrick Deloulay, Micro Focus - Micro Focus started their digital transformation 3 years ago, moving the entire portfolio into hundreds of container images. Leveraging Docker Hub as our primary registry service, we will cover how we ended up building a simple but secure push/pull model to publish and deliver our premium assets to our customers and partners to both meet the high agility of our DevOps teams while greatly simplifying the deployment of our applications.

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Docker, Inc.

Lukonde Mwila, Entelect As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Elton Stoneman, Docker Captain + Container Consultant and Trainer How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least). In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product. There's a lot of technology here from a real world case study, and I'll focus on: - running Windows apps in Docker containers - building a .NET Core API which can run in Linux or Windows containers - running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud - configuring AKS workloads in Azure to burst out to Azure Container Instances And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Docker, Inc.

This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.

Developing with Docker for the Arm Architecture

Docker, Inc.

More from Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience

How to Improve Your Image Builds Using Advance Docker Build

Build & Deploy Multi-Container Applications to AWS

Securing Your Containerized Applications with NGINX

How To Build and Run Node Apps with Docker and Compose

Hands-on Helm

Distributed Deep Learning with Docker at Salesforce

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Monitoring in a Microservices World

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Predicting Space Weather with Docker

Become a Docker Power User With Microsoft Visual Studio Code

How to Use Mirroring and Caching to Optimize your Container Registry

Monolithic to Microservices + Docker = SDLC on Steroids!

Kubernetes at Datadog Scale

Labels, Labels, Labels

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Build & Deploy Multi-Container Applications to AWS

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Developing with Docker for the Arm Architecture

Recently uploaded

DBX First Quarter 2024 Investor Presentation

Dropbox

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

Architecting Cloud Native Applications

WSO2

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

Keynote 2: APIs in 2030: The Risk of Technological Sleepwalk Paolo Malinverno, Growth Advisor - The Business of Technology Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

apidays

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Victor Rentea

[BuildWithAI] Introduction to Gemini.pdf

Sandro Moreira

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Bhuvaneswari Subramani

Exploring Multimodal Embeddings with Milvus

Zilliz

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Recently uploaded (20)

DBX First Quarter 2024 Investor Presentation

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Architecting Cloud Native Applications

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

Corporate and higher education May webinar.pptx

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

[BuildWithAI] Introduction to Gemini.pdf

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Why Teams call analytics are critical to your entire business

Boost Fertility New Invention Ups Success Rates.pdf

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Exploring Multimodal Embeddings with Milvus

Strategies for Landing an Oracle DBA Job as a Fresher

It Takes an Ecosystem to Build a Production Application by Ross Gardler

1. It takes an ecosystem to build a production application Ross Gardler @rgardler

2. Docker, Docker, Docker • They created open source software to build and manage containers • Even I can create and run them • Docker CLI, Docker Engine, Docker Swarm, Docker Compose, Docker Machine and more…

3. Docker makes it Easy • Many people agree that: • Development is easier with Docker • Scaling is easier with Docker • Continuous Integration is easier with Docker • Testing is easier with Docker • So just adopt Docker and life is easier, right? • Applications don’t consist of a single container • How do we manage applications?

4. Docker is an Ecosystem • Docker make Containerization accessible • It has spawned an Ecosystem • This session is about why the Ecosystem is improtant

5. About Me

6. “A model where IT organizations and developers can work together to build, ship and run their applications anywhere.” https://blog.docker.com/2016/02/containers-as-a-service-caas/

7. Docker offers portability • Docker containers are portable • Orchestrate them with Docker Swarm, Kubernetes, Apache Mesos, DC/OS etc. • Run them on Azure, AWS, Google, Digital Ocean etc. • But, what about applications consisting of many containers?

8. An orchestrator is not enough • Continuous Delivery • Resource Management • High Availability • Storage • Networking • Load Balancing • Service Discovery • Monitoring • … and more

9. What about everything else? • Continuous Delivery • Resource Management • High Availability • Storage • Networking • Load Balancing • Service Discovery • Monitoring • … and more • Who’s going to configure it all? • What about optimizing for the infrastructure of choice? • Are you locking into a cloud provider because of your choice of orchestrator?

10. Building a Container Hosting Platform Docker Engine Container ToolingService Tooling • Not everything is open source software • There is a limit to portability when it comes to electricity and CPUs • How do you minimize the lock-in to specific hardware and software infrastructure

11. Azure Container Service: Maintaining Portability of Workloads Containers Container ToolingService Tooling Layer Supported Technologies Configuration as Code ARM, Dockerfile, Docker Compose, Marathon.json High Availability Azure, VM Scale Sets Container orchestration Docker Swarm, Chronos, Marathon, Apache Mesos Continuous Integration Jenkins, VSTS Monitoring OMS, Statsd, Cadvisor, Datadog Networking Docker Networking Storage Docker volumes ??? ???

12. Here’s what you get…

13. Infrastructure Layer (Electricity, CPUs, Cables restrict portability) Azure Container Service: Maintaining Portability of Workloads Management Layer (Variety of solutions, some more portable than others) Application Layer (Docker Containers promise portability)

14. Demo: ACS Deployment http://portal.azure.com

15. Demo: Standard Docker Tooling

16. Contributing to the Ecosystem

17. Demo: Jenkins and ACS

18. Choice, flexibility, scalability

19. Swarm in Azure Container Service

20. Preview Demo: Windows Server Containers

21. Bringing it All Together Azure SQL Database Load Balancer Containers in Swarm

22. Private Preview: Windows Server Containers on Azure Container Service Self nominate at https://aka.ms/acswin

23. Bringing the Ecosystem to You

24. Demo: Monitoring solutions

25. Demo: Microscaling

26. Where will the ecosystem take you from here?

27.

28. Private Preview: Windows Server Containers on Azure Container Service Self nominate at https://aka.ms/acswin

Editor's Notes

Mesos now provides more advanced scheduling as well as fault tolerance But now it’s getting really complicated to configure my environment: Docker CLI Docker Machine Docker Host with Docker Engine Docker containers Apache Mesos Docker Swarm (and the diagram doesn’t even include the required registry and multiple masters with Apache Zookeeper that you would use in production, we’re also ignoring the need to monitor what is happening – more on monitoring soon)
Azure Stack and Azure provides the foundations VMs provide the initial unit of computation for management purposes Windows Server and Linux provide the OS for those VMs and allow BYO management software Today we offer containers, managed by Docker compatible Container tooling The infrastructure is hand crafted using ARM templates Azure Container Services will ease the management of 1st and 3rd party container technologies We are taking a layered approach to providing these services
Azure Stack and Azure provides the foundations VMs provide the initial unit of computation for management purposes Windows Server and Linux provide the OS for those VMs and allow BYO management software Today we offer containers, managed by Docker compatible Container tooling The infrastructure is hand crafted using ARM templates Azure Container Services will ease the management of 1st and 3rd party container technologies We are taking a layered approach to providing these services
ACS says nothing about the application layer, you are portable here ACS uses only open source at the management layer and we provide choice. We optimize for portability. ACS is not portable at the infrastructure layer, we optimize for Azure. Azure Stack is coming, which will provide more options, but since our orchestration layer is open source you can host always host elsewhere.
Using Portal Key message: we take away all the complexity and make it a simple set of parameters. After that it is IaaS with open source software installed
Key message: Once the infrastructure is up and running your applications are just Docker Containers, with all available tooling you normally use. On rgdev box cd ~/projects/acs-demos/demo_scripts/dockercon2016 ssh -N -L 2375:localhost:2375 azureuser@rgacsdcmgmt.westus.cloudapp.azure.com -p 2200 & export DOCKER_HOST=:2375 docker info docker-compose up –d docker-compose ps Clean Up docker-compose scale producer=1 docker-compose stop docker stop $(docker ps –a)
Key message: our open API for standing up the infrastructure means you can use any tool to create your cluster and then manage your workloads on that cluster Server: http://13.91.0.150/jenkins
Docker containers running in 2 VMs in Azure – deployed through template Each container is assigned an IP address from the Virtual Network Direct communication between VMs using private IPs (no encapsulation/port re-use) Leverage Azure SDN stack (NSG) to set container specific ACL Demonstrate the ACLing taking effect Stretch: Show container talking to on-premises using Azure SDN.
Show existing resource group in portal.azure.com that comprises my small cluster – “already deployed using ARM template” 1 master 2 agents Existing SQL Azure instance - "production ready, designed for Azure cloud, so I'll use it" "This is the same Swarm you're used to" Bring up plink w/ port forwarding "Swarm master running in Azure" Docker info - point out Windows hosts Conveniently mapped a local port $ENV:DOCKER_HOST=":2375" Run docker info on the laptop Let's deploy an app across the nodes Docker-Compose build Docker-compose scale web=2 Open http://flatwhite.westus.cloudapp.azure.com http://flatwhite.westus.cloudapp.azure.com/MusicStore And of course, it's still Windows Mstsc /v:flatwhite.westus.cloudapp.azure.com "If you want it, the familiar Windows admin experience is available"
Key message: Since the cluster is “just Docker” you can use any tooling in the cluster you like, for example monitoring can be done with any of …, … http://104.41.133.84:8080/containers/ https://app.datadoghq.com/account/login?next=%2Fevent%2Fstream https://azurecon-containers.portal.mms.microsoft.com/#Workspace/overview/Containers/details/index?_timeInterval.intervalDuration=86400
https://app.microscaling.com User: swarm@microscaling.com Password: bees147hivequeenbee docker-compose scale producer=8

It Takes an Ecosystem to Build a Production Application by Ross Gardler

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from Docker, Inc.

More from Docker, Inc. (20)

Recently uploaded

Recently uploaded (20)

It Takes an Ecosystem to Build a Production Application by Ross Gardler

Editor's Notes