A vision of persistence

•

3 likes•261 views

Pull, push, clone, it is all in your daily workflow. But what if this wasn't your source code or your container, but the state of your whole computer? Push your production database over to another machine? No problem! This talk shows how you can use Dotmesh with LinuxKit to work with persistent data on your server as simply as you work with git. This workflow helps unleash new ways of working with servers and data. Immutable infrastructure from LinuxKit meets controlled and manageable data storage from Dotmesh. Combining these two open source projects allows new possibilities in how to manage your infrastructure.

Presentations & Public Speaking

Luke Marsden @lmarsden
Founder & CEO, Dotmesh
Justin Cormack @justincormack
Engineer, Docker

“In the cloud, we know exactly what we
want a server to be, and if we want to
change that we simply terminate it and
launch a new server with a new AMI.
Building with Legos, 2011

https://www.oreilly.com/ideas/an-introduction-to-immutable-infrastructure

● Built from containers, embeds containerd
● Small, fast, secure, cloud native, build in CI
● S million users, as part of Docker Desktop
● Base of a certified Kubernetes build
● amd64, arm64, s390x support

● LinuxKit has a clean separation between code (read
only) and data
● These have different lifecycles
● Different management tooling
● Update OS code while preserving data

● Commit a snapshot
● Push for backups
● Clone for copies
● Fork for experiments
Sounds great! How can I do that?

● LinuxKit manages runtime state in immutable way
● Dotmesh moves the data to where it is needed
● Version the data independently from the image
● Infrastructure independent - can do sort of similar things
if you use cloud provider APIs but locked in

● docker CoW is a poor match for databases
● Databases tend to use a small number of large files
● dotmesh uses zfs
● zfs snapshots are very efficient

• dotmesh.yml –>
• dm-linuxkit binary which launches
etcd and then dotmesh (and clones
a dot if 'seed' is specified)
• onboot vs service

● github.com/linuxkit/linuxkit
● github.com/dotmesh-io/dotmesh
● github.com/dotmesh-io/dm-linuxkit

● Docker Community Slack #linuxkit
● Dotmesh Slack http://bit.ly/dotmesh-slack
● Search DockerCon talks for "Dotmesh"

● Find more excuses to drink beer together
● More reliable on local Mac (qemu and
hyperkit both broken in different ways!)
● Make dotmesh work with newest zfs so that
it to works on latest kernels (4.14+)

● Cherry-pick files between dots in dothub
● ARM support
● /var/lib/docker in a dot
● What would you like to see?

Two things are for certain – space is hard, and Docker is not just for web content! Space software development traditionally lags behind state of the art software process for good reason – our missions are long (7+ years), we run on highly constrained embedded hardware, and the software cannot fail. Docker, along with a devops mentality, has helped us create a scalable, parallelizable and rapidly deployable test infrastructure for DART, NASA’s mission to hit an asteroid at 6 km/s. During the presentation, we will walk through how our dev cycle has changed from a human based testing system to an automated one. We will outline how we are using Docker (and NASA Goddard’s Core Flight Executive) for both our embedded development environment and our scalable test environment. Next, we will discuss what deployment means to us (and how different it is from web deployment). Lastly, we will explore lessons learned on how our hardware-centric testing approach was adapted into a software-based approach: what worked, what didn’t, what we wish we could do someday. How can you help? We are new to Docker. We are excited to share our experiences and hear from the Docker community on our use cases, technological hurdles that we faced, our solutions to these problems, and how we can harness Docker to the fullest extent.

Proactive ops for container orchestration environments

Docker, Inc.

Break -> inspect -> fix is the Ops workflow for infrastructure stacks of the past. Distributed infrastructure and applications claim to be the new generation, but why is it so much more painful to maintain and troubleshoot them? Much of the pain comes from outdated operational models relying on reactive or, worse yet, manual monitoring and Ops. This talk lays out a proactive Ops model for container infrastructure. By focusing on event monitoring, infrastructure state monitoring, trend analysis, and distributed log collection, a proactive Ops model delivers observability for distributed apps that was not possible before. Using real-world examples from Swarm and Kubernetes, we'll demonstrate the tools used and how we relieve Ops pain in container orchestration.

Docker on docker leveraging kubernetes in docker ee

Docker, Inc.

How to accelerate docker adoption with a simple and powerful user experience

Docker, Inc.

Windows container security

Docker, Inc.

"The majority of the container security discussion revolves around containers on Linux while the security of containers in Windows is left as a mystical black box. In this talk we'll peel back the curtain and dive in to how Windows containers are secured. Does Windows have namespaces? How does it compose the layers of a container's filesystem? How does it limit resource usage of containers? I heard there's a Hyper-V isolation thing, what's that about? We'll answer all these questions and more!"

Production sec ops with kubernetes in docker

Docker, Inc.

In this talk, Scott Coulton will walk through how to build a container as a service platform with Docker EE. Starting from scratch he will help you figure out what orchestrator to choose by deep diving into the technical differences between swarm and kubernetes on the EE platform as well as cover some of the practical considerations that could influence your decision. He will also share various automation solutions to deploy your cluster into production. Once the cluster is up and and running, Scott will delve into sec ops and discuss security best practices - including signing images in DTR (Docker Trusted Registry) and CVE scanning to provide a secure supply chain into production. You’ll leave this talk with the knowledge needed to build your own container platform in production. And did I mention it will all be done live, step-by-step?

Gordon's secret session kubernetes on windows

Docker, Inc.

Your developers just walked into your cube and said: "Here's the new app, I built it with Docker, and it's ready to go live." What do you do next? In this session, we'll talk about what containers are and what they are not. And we'll step through a series of considerations that need to be examined when deploying containerized workloads - VMs or Container? Bare Metal or Cloud? What about capacity planning? Security? Disaster Recovery? How do I even get started?

Activision's Skypilot: Delivering Amazing Game Experiences Through Containeri...

Docker, Inc.

"Technologies that are going to affect our lives in the next decade are being tested and developed in the video game sphere." In January 2016 Activision approved a pilot project to build a containerised continuous delivery pipeline using Docker. This project spanned multiple devops teams and would culminate in launching a production title "Skylanders Imaginators" in October 2016. The Mission Statement : “Our mission is to deliver an amazing build, test and deploy pipeline that aims to be so reliable, effective and easy to use that our product and title departments will end up writing high value gaming services all day long without giving a second thought to how they may reliably deliver these in record time.” This talk will discuss the cultural and technical challenges faced throughout the pilot. Spoiler alert: Not everyone was happy with the decision to use Docker. The talk will cover the concerns and how we handled them. It will cover why it is important, especially in the games industry, to be evaluating and integrating technologies like Docker in order to remain relevant. For the first time in Demonware history developers were responsible for the launch and support of a title. We are also the first studio under Activision to be running Docker in Production.

Docker on Docker

Docker, Inc.

At Docker, we like to “eat our own dog food” or “drink our own champagne.” Whatever your favorite phrase, the importance of a software company using their own software is critical to relating to our customers. In this talk, we will discuss how the Docker Infrastructure and engineering teams have deployed and operationalized Docker Enterprise Edition (EE) for our staging and production environments, what we have learned in the process, and how it's making Docker EE better.

DockerCon EU 2015: What's New with Docker Trusted Registry

Docker, Inc.

Considerations for operating docker at scale

Docker, Inc.

"Scale" happens along 3 different aspects: (1) applications and their services scale up and down leading to (2) the infrastructure scaling up to meet the needs of the applications, and finally (3) sites scale across multiple locations, including movement to public cloud. In this session, we will talk about how Docker EE scales along all three of these dimensions to give you a consistent platform for running your applications: 1. At the application level: how do you manage application state & health along with resource and security constraints to scale containers up and down up in a controlled fashion? 2. The infrastructure level: as your application estate grows on the Docker EE platform you will need to scale across more nodes. How do automate the provisioning of these new nodes and how do you integrate the Docker EE platform layer with your existing infrastructure systems and tools. 3. Finally, we'll talk about distributed scale: how do you take what works for applications in one data center and spread it across multiple sites, in an integrated fashion so you can operate seamlessly?

DockerCon EU 2015: Placing a container on a train at 200mph

Docker, Inc.

Presented by Casper S. Jensen, Software Engineer, Uber At Uber, we've been introducing Docker to give service owners more control over their environments. However, everything at Uber is moving very fast so we have had to do it a way such that Docker fitted into the existing infrastructure and services could be migrated seamlessly to Docker without any service interruptions. In this talk we will talk about the challenges we faced while doing this, such as handling both non-Docker and Docker builds, image replication, integration with our deployment systems and other challenges when deploying Docker at scale.

DCEU 18: Docker Containers in a Serverless World

Docker, Inc.

Jules Testard - Software Engineer, Docker Inc Since the advent of AWS Lambda in 2014, the Function as a Service (FaaS) programming paradigm has gained a lot of traction in the cloud community. Since then, interest has increased for developers and entreprises to build their own open source solutions on top of Kubernetes. A number of competing frameworks in this space have been developed. In this talk, we will look at three specific frameworks (OpenFaas, Nuclio and FN) and for each framework we will: Show how to create, deploy, and invoke a function using that framework Show how Docker images and containers are used by each framework under the hood Investigate how the frameworks leverage KNative to build, ship and run applications on Kubernetes

Docker for developers on mac and windows

Docker, Inc.

The whole Docker ecosystem exists today because of every single developer who found ways of using Docker to improve how they build software; whether streamlining production deployments, speeding up continuous integration systems or standing up an application on your laptop to hack on. In this talk we want to take a step back and look at where Docker sits today from the software developers point of view - and then jump ahead and talk about where it might go in the future. In this talk, we’ll discuss: * Making Docker an everyday part of the developing software on the desktop, with Docker for Windows and Docker for Mac * Docker Compose, and the future of describing applications as code * How Docker provides the best tools for developing applications destined to run on any Kubernetes cluster This session should be of interest to anyone who writes software; from people who want to hack on a few personal projects, to polyglot open source programmers and to professional developers working in tightly controlled environments. Everyone deserves a better developer experience.

Accessible hpc for everyone with docker and containers

Docker, Inc.

Building your production tech stack for docker container platform

Docker, Inc.

This session will focus on the practicals of building a fully-functional stack of container cluster tools, with different options for stacking those tools from the OS-up. We’ve all seen examples of common technologies stacks, like the good ol’ LAMP and MEAN stacks for apps, but what about lower-level infrastructure? And can we get it without cloud vendor lock in please? Oh and pure containers and infrastructure-as-code too? With Docker, sure thing! This session will cover: Which OS/Distro and Kernel to use VM’s or Bare Metal Recommended Swarm architectures Tool stacks for “pure open source”, “cloud-service based”, and “Docker EE” scenarios Demos of these tools working together including InfraKit, Docker, Swarm, Flow-Proxy, ELK, Prometheus, REX-Ray, and more.

Packaging software for the distribution on the edge

Docker, Inc.

At GE Digital, in the Asset Performance Management space, we need to supply an edge solution that impacts both on-premise and data transmission to the cloud. Our current edge solutions are relatively simplistic, but as our technologies mature along with our customers’ needs, we’re finding that we need to grasp a more fog computing-based approach where we include more intelligence, more computing power, at the edge. Along with this computative power, we need to better remotely manage these systems – to be able to monitor progress and diagnose problems – a technology that would enable us to containerize, to better manage, our software bundlings and deployments. We found that Windows Docker seemed to fit the bill -- much of the technologies that live at our edge solutions are Windows OS based (as the customers’ main platforms are Windows OS based). This presentation reviews the approach that we took to repackage one of our main APM on-premise solutions using Windows Docker. We’ve created a prototype which we’re looking forward to productize and enable the capability of remote management to thousands of deployments. The presentation also contains a video demo of the running system. The on-prem APM system will demonstrate the usage of Docker networking along with docker volumes and three (3) docker containers – will discuss the construction of the images, and nuances, of execution of the running docker containers.

Docker Platform Internals: Taking runtimes and image creation to the next lev...

Docker, Inc.

In this session, we'll go into details about the latest developments around some of the components behind the core features of the Docker Platform. We'll cover the containerd runtime that was built to serve as an underlying daemon for Docker and Kubernetes, and BuildKit, a toolkit that builds on containerd to provide next-generation capabilities for building software with the help of containers. You will learn about the architecture and design choices of these projects, for example, the power of containerd's rich client library and BuildKit's frontend model that allows introducing new build languages or Dockerfile features. You can discover how you can use these projects directly and how they are being integrated into the Docker Platform.

5 patterns for success for application transformation

Docker, Inc.

Legacy apps weren't designed to run in a modern distributed platform like Docker. They have their own ideas about logging, configuration and health which don't translate to the world of containers and make transformation projects hard. This session shows you how to bring your old apps into the modern world, and integrate them with Docker - without changing code. We'll cover patterns for all the core application concerns: * logging * configuration * monitoring * health * dependency management The sample apps will be in .NET and Java, and will show you how to turn your existing apps into good Docker citizens.

DCSF19 Deploying Istio as an Ingress Controller

Docker, Inc.

Deploying containers on Heterogeneous IOT devices by Daniel Bruzual

Docker, Inc.

Android Meets Docker

Docker, Inc.

How does awesome Android benefit from dazzling Docker? With an introduction to the Docker way of continuous integration for Android, it would help you get a consistent Android development and build environment with minimal effort. Syllabus: * Background story - where the problem happened * Why Android dev should care? * How to provision an Android environment to build project usually? How Docker is gonna solve it? * Typical machine dependant failing unit tests examples, which could be revealed by using Docker * Philosophy, features, improvements, comparison against traditional approach * Limitations * Quick setup of Jenkins using AndroidSDK docker slaves (live demo)

DockerCon EU 2015: Cultural Revolution - How to Mange the Change Docker Brings

Docker, Inc.

Presentation by Ian Miell, author of 'Docker in Practice' The adoption of Docker brings with it many challenges, not all of themtechnical. There is a clear vision of Docker regularly articulated thatencompasses microservices, decoupled applications, agile and continuous deliveryand integration, and DevOps. But the application of this vision often flounders when confronted by the legacy structures of working and successful businesses. This talk draws on the experienced gained from several attempts to lead changewithin technical organisations. The speaker experienced both failure to effectchange (through productization of a bespoke software stack, and the introduction of Erlang), and success (moving to a Continuous Improvement modelof complex software maintenance). These experiences informed a successful drive to Docker as the build and delivery system of a 700-strong software company,with significant efficiency improvements. As with most historical revolutions, the integration of the vision with realityresulted in corners cut and principles compromised in order to effect change.This talk is a discussion of the lessons learned from these experiences in an effort to help the listener clarify and overcome the unique challenges their own organisation brings to making change happen.

DockerCon EU 2015: Docker Universal Control Plane (Gordon's Special Session)

Docker, Inc.

It takes a Village to do the Impossible - Jeff Lindsay

Docker, Inc.

Java in a world of containers

Docker, Inc.

"Container technologies such as Docker are rapidly becoming the de-facto way to deploy cloud applications, and Java is committed to being a good container citizen. This session will explain how OpenJDK fits into the world of containers, specifically how it fits with Docker images and containers. The session will focus on the production of optimized Docker images containing a JDK. We will introduce technologies such as jlink, that can be used to reduce the size of the created image. The session will explain Alpine/musl support for an effective image and runtime. The session will also talk about and the inclusion of Class Data Sharing (CDS) archives and Ahead of Time (AOT) shared object libraries for improving startup time. The attendees will learn about the recent work that has gone into OpenJDK for interacting with container resource limitations."

Kubernetes 101

Stanislav Pogrebnyak

Instant developer onboarding with self contained repositories

Yshay Yaacobi

What's hot

Docker?!?! But I'm a SysAdmin

Docker, Inc.

Activision's Skypilot: Delivering Amazing Game Experiences Through Containeri...

Docker, Inc.

Docker on Docker

Docker, Inc.

DockerCon EU 2015: What's New with Docker Trusted Registry

Docker, Inc.

Considerations for operating docker at scale

Docker, Inc.

DockerCon EU 2015: Placing a container on a train at 200mph

Docker, Inc.

DCEU 18: Docker Containers in a Serverless World

Docker, Inc.

Docker for developers on mac and windows

Docker, Inc.

Accessible hpc for everyone with docker and containers

Docker, Inc.

Building your production tech stack for docker container platform

Docker, Inc.

Packaging software for the distribution on the edge

Docker, Inc.

Docker Platform Internals: Taking runtimes and image creation to the next lev...

Docker, Inc.

5 patterns for success for application transformation

Docker, Inc.

DCSF19 Deploying Istio as an Ingress Controller

Docker, Inc.

Deploying containers on Heterogeneous IOT devices by Daniel Bruzual

Docker, Inc.

Android Meets Docker

Docker, Inc.

DockerCon EU 2015: Cultural Revolution - How to Mange the Change Docker Brings

Docker, Inc.

DockerCon EU 2015: Docker Universal Control Plane (Gordon's Special Session)

Docker, Inc.

It takes a Village to do the Impossible - Jeff Lindsay

Docker, Inc.

Java in a world of containers

Docker, Inc.

What's hot (20)

Docker?!?! But I'm a SysAdmin

Activision's Skypilot: Delivering Amazing Game Experiences Through Containeri...

Docker on Docker

DockerCon EU 2015: What's New with Docker Trusted Registry

Considerations for operating docker at scale

DockerCon EU 2015: Placing a container on a train at 200mph

DCEU 18: Docker Containers in a Serverless World

Docker for developers on mac and windows

Accessible hpc for everyone with docker and containers

Building your production tech stack for docker container platform

Packaging software for the distribution on the edge

Docker Platform Internals: Taking runtimes and image creation to the next lev...

5 patterns for success for application transformation

DCSF19 Deploying Istio as an Ingress Controller

Deploying containers on Heterogeneous IOT devices by Daniel Bruzual

Android Meets Docker

DockerCon EU 2015: Cultural Revolution - How to Mange the Change Docker Brings

DockerCon EU 2015: Docker Universal Control Plane (Gordon's Special Session)

It takes a Village to do the Impossible - Jeff Lindsay

Java in a world of containers

Similar to A vision of persistence

Kubernetes 101

Stanislav Pogrebnyak

Instant developer onboarding with self contained repositories

Yshay Yaacobi

Adventures in docker compose

LinkMe Srl

JOSA TechTalks - Docker in Production

Jordan Open Source Association

Docker is an open platform for developers and system administrators to build, ship and run distributed applications. Using Docker, companies in Jordan have been able to build powerful system architectures that allow speeding up delivery, easing deployment processes and at the same time cutting major hosting costs. George Khoury shares his experience at Salalem in building flexible and cost effective architectures using Docker and other tools for infrastructure orchestration. The result allows them to easily and quickly move between different cloud providers.

Accelerate your development with Docker

Andrey Hristov

Accelerate your software development with Docker

Andrey Hristov

Discussing the difference between docker dontainers and virtual machines

Steven Grzbielok

Getting started with docker (2017)

JEMLI Fathi

Taking Docker to Production: What You Need to Know and Decide

Bret Fisher

DevOps in the Real World is far from perfect, yet we all dream of that amazing auto-healing fully-automated CI/CD micro-service infrastructure that we'll have "someday." But until then, how can you really start using containers today, and what decisions do you need to make to get there? This session is designed for practitioners who are looking for ways to get started now with Docker and Swarm in production. This is not a Docker 101, but rather it's to help you be successful on your way to Dockerizing your production systems. Attendees will get tactics, example configs, real working infrastructure designs, and see the (sometimes messy) internals of Docker in production today.

Taking Docker to Production: What You Need to Know and Decide

Docker, Inc.

Introduction to Containers

Dharmit Shah

DevEx | there’s no place like k3s

Haggai Philip Zagury

Introduction to Modern DevOps Technologies

Kriangkrai Chaonithi

Sean schofield & Richard Lister, Spree Commerce_ Fearless deployment @ Open C...

Spark Solutions

Docker_AGH_v0.1.3Witold 'Ficio' Kopel

A Gentle Introduction to Docker and ContainersDocker, Inc.

Introduction to Docker and Linux Containers @ Cloud Computing Rhein MainPuja Abbassi

Docker on Power Systems

Cesar Maciel

Introduction to Docker, December 2014 "Tour de France" Bordeaux Special Edition

Jérôme Petazzoni

Docker, the Open Source container Engine, lets you build, ship and run, any app, anywhere. This is the presentation which was shown in December 2014 for the last stop of the "Tour de France" in Bordeaux. It is slightly different from the presentation which was shown in the other cities (http://www.slideshare.net/jpetazzo/introduction-to-docker-december-2014-tour-de-france-edition), and includes a detailed history of dotCloud and Docker and a few other differences. Special thanks to https://twitter.com/LilliJane and https://twitter.com/zirkome, who gave me the necessary motivation to put together this slightly different presentation, since they had already seen the other presentation in Paris :-)

Docker 0.11 at MaxCDN meetup in Los Angeles

Jérôme Petazzoni

Similar to A vision of persistence (20)

Kubernetes 101

Instant developer onboarding with self contained repositories

Adventures in docker compose

JOSA TechTalks - Docker in Production

Accelerate your development with Docker

Accelerate your software development with Docker

Discussing the difference between docker dontainers and virtual machines

Getting started with docker (2017)

Taking Docker to Production: What You Need to Know and Decide

Introduction to Containers

DevEx | there’s no place like k3s

Introduction to Modern DevOps Technologies

Sean schofield & Richard Lister, Spree Commerce_ Fearless deployment @ Open C...

Docker_AGH_v0.1.3

A Gentle Introduction to Docker and Containers

Introduction to Docker and Linux Containers @ Cloud Computing Rhein Main

Docker on Power Systems

Introduction to Docker, December 2014 "Tour de France" Bordeaux Special Edition

Docker 0.11 at MaxCDN meetup in Los Angeles

More from Docker, Inc.

Containerize Your Game Server for the Best Multiplayer Experience

Docker, Inc.

Raymond Arifianto, AccelByte and Mark Mandel, Google - We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.

How to Improve Your Image Builds Using Advance Docker Build

Docker, Inc.

Nicholas Dille, Haufe-Lexware + Docker Captain - Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Lukonde Mwila, Entelect - As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

Securing Your Containerized Applications with NGINX

Docker, Inc.

Kevin Jones, NGNIX - NGINX is one of the most popular images on Docker Hub and has been at the forefront of the web since the early 2000's. In this talk we will discuss how and why NGINX's lightweight and powerful architecture makes it a very popular choice for securing containerized applications as a sidecar reverse proxy within containers. We will highlight important aspects of application security that NGINX can help with, such as TLS, HTTP, AuthN, AuthZ and traffic control.

How To Build and Run Node Apps with Docker and Compose

Docker, Inc.

Kathleen Juell, Digital Ocean - Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself. This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.

Hands-on Helm

Docker, Inc.

Distributed Deep Learning with Docker at Salesforce

Docker, Inc.

Jeff Hajewski, Salesforce - There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Docker, Inc.

James Fuller, webcomposite s.r.o. - Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.

Monitoring in a Microservices World

Docker, Inc.

Fabian Stäber, Instana - In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Docker, Inc.

Clemente Biondo, Engineering Ingegneria Informatica - When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.

Predicting Space Weather with Docker

Docker, Inc.

Chris Lauer, NOAA Space Weather Prediction Center - This is the story of how adopting a containerized workflow changed the way our small software team works at NOAA’s Space Weather Prediction Center. Our old architecture, a big ball of mud shared-database integration, just wasn’t cutting it - it was killing our agility. Over the past two years, our small team has adopted a microservice style architecture, using Docker with docker-compose and environment files as our deployment strategy for all new development. We’ve discovered the joys of using containers for identical dev, staging, and production environments. We work closely with scientists: much of the code we’re running has complicated and conflicting library dependencies. Docker captures these beautifully - we’ve even had some success teaching our scientists to use it! I’ll share what we’ve learned, some of the persistent challenges we face, and one place we really got it wrong. This talk builds off of a popular hallway track from DockerCon 2019.

Become a Docker Power User With Microsoft Visual Studio Code

Docker, Inc.

Brian Christner, 56k + Docker Captain - In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.

How to Use Mirroring and Caching to Optimize your Container Registry

Docker, Inc.

Monolithic to Microservices + Docker = SDLC on Steroids!

Docker, Inc.

Ashish Sharma, SS&C Eze - SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.

Kubernetes at Datadog Scale

Docker, Inc.

Ara Pulido, Datadog - Container technologies, although not new, have increased their popularity in the past few years, with container orchestrators allowing companies around the world to adopt these technologies to help them ship and scale microservices with precision and velocity. Kubernetes is currently the most popular container orchestration platform, and while many organizations are migrating their workloads to it, Kubernetes is still relatively immature. New corner cases, errors, and quirks are regularly discovered as users push the boundaries of size and scale. When Datadog adopted Kubernetes we discovered some of these boundaries the hard way, and we continuously challenge and modify our infrastructure decisions in order to fit our use case. Join me in this talk for our story on what we learned while we scaled our Kubernetes clusters, the contributions to Kubernetes we made along the way, and how you can apply those learnings when growing your Kubernetes clusters from a handful to hundreds or thousands of nodes.

Labels, Labels, Labels

Docker, Inc.

Andy Clemenko, StackRox - One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Docker, Inc.

Patrick Deloulay, Micro Focus - Micro Focus started their digital transformation 3 years ago, moving the entire portfolio into hundreds of container images. Leveraging Docker Hub as our primary registry service, we will cover how we ended up building a simple but secure push/pull model to publish and deliver our premium assets to our customers and partners to both meet the high agility of our DevOps teams while greatly simplifying the deployment of our applications.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Lukonde Mwila, Entelect As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Docker, Inc.

Elton Stoneman, Docker Captain + Container Consultant and Trainer How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least). In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product. There's a lot of technology here from a real world case study, and I'll focus on: - running Windows apps in Docker containers - building a .NET Core API which can run in Linux or Windows containers - running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud - configuring AKS workloads in Azure to burst out to Azure Container Instances And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.

Developing with Docker for the Arm Architecture

Docker, Inc.

This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.

More from Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience

How to Improve Your Image Builds Using Advance Docker Build

Build & Deploy Multi-Container Applications to AWS

Securing Your Containerized Applications with NGINX

How To Build and Run Node Apps with Docker and Compose

Hands-on Helm

Distributed Deep Learning with Docker at Salesforce

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Monitoring in a Microservices World

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Predicting Space Weather with Docker

Become a Docker Power User With Microsoft Visual Studio Code

How to Use Mirroring and Caching to Optimize your Container Registry

Monolithic to Microservices + Docker = SDLC on Steroids!

Kubernetes at Datadog Scale

Labels, Labels, Labels

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Build & Deploy Multi-Container Applications to AWS

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Developing with Docker for the Arm Architecture

Recently uploaded

Bonzo subscription_hjjjjjjjj5hhhhhhh_2024.pdf

khadija278284

Obesity causes and management and associated medical conditions

Faculty of Medicine And Health Sciences

Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...

Orkestra

Competition and Regulation in Professional Services – KLEINER – June 2024 OEC...

OECD Directorate for Financial and Enterprise Affairs

Acorn Recovery: Restore IT infra within minutes

IP ServerOne

Announcement of 18th IEEE International Conference on Software Testing, Verif...

Sebastiano Panichella

Eureka, I found it! - Special Libraries Association 2021 Presentation

Access Innovations, Inc.

Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.

Bitcoin Lightning wallet and tic-tac-toe game XOXO

Matjaž Lipuš

0x01 - Newton's Third Law: Static vs. Dynamic Abusers

OWASP Beja

f you offer a service on the web, odds are that someone will abuse it. Be it an API, a SaaS, a PaaS, or even a static website, someone somewhere will try to figure out a way to use it to their own needs. In this talk we'll compare measures that are effective against static attackers and how to battle a dynamic attacker who adapts to your counter-measures. About the Speaker =============== Diogo Sousa, Engineering Manager @ Canonical An opinionated individual with an interest in cryptography and its intersection with secure software development.

Getting started with Amazon Bedrock Studio and Control Tower

Vladimir Samoylov

Doctoral Symposium at the 17th IEEE International Conference on Software Test...

Sebastiano Panichella

somanykidsbutsofewfathers-140705000023-phpapp02.pptx

Howard Spence

International Workshop on Artificial Intelligence in Software Testing

Sebastiano Panichella

Recently uploaded (13)

Bonzo subscription_hjjjjjjjj5hhhhhhh_2024.pdf

Obesity causes and management and associated medical conditions

Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...

Competition and Regulation in Professional Services – KLEINER – June 2024 OEC...

Acorn Recovery: Restore IT infra within minutes

Announcement of 18th IEEE International Conference on Software Testing, Verif...

Eureka, I found it! - Special Libraries Association 2021 Presentation

Bitcoin Lightning wallet and tic-tac-toe game XOXO

0x01 - Newton's Third Law: Static vs. Dynamic Abusers

Getting started with Amazon Bedrock Studio and Control Tower

Doctoral Symposium at the 17th IEEE International Conference on Software Test...

somanykidsbutsofewfathers-140705000023-phpapp02.pptx

International Workshop on Artificial Intelligence in Software Testing

A vision of persistence

1. Luke Marsden @lmarsden Founder & CEO, Dotmesh Justin Cormack @justincormack Engineer, Docker

2. v What and why?

3. “In the cloud, we know exactly what we want a server to be, and if we want to change that we simply terminate it and launch a new server with a new AMI. Building with Legos, 2011

4. https://www.oreilly.com/ideas/an-introduction-to-immutable-infrastructure

5. v Immutable infrastructure the easy way

6. ● Built from containers, embeds containerd ● Small, fast, secure, cloud native, build in CI ● S million users, as part of Docker Desktop ● Base of a certified Kubernetes build ● amd64, arm64, s390x support

7. v Code without data is useless

8. ● LinuxKit has a clean separation between code (read only) and data ● These have different lifecycles ● Different management tooling ● Update OS code while preserving data

9. ● Commit a snapshot ● Push for backups ● Clone for copies ● Fork for experiments Sounds great! How can I do that?

10. v git for data

11.

12. ● LinuxKit manages runtime state in immutable way ● Dotmesh moves the data to where it is needed ● Version the data independently from the image ● Infrastructure independent - can do sort of similar things if you use cloud provider APIs but locked in

13. ● docker CoW is a poor match for databases ● Databases tend to use a small number of large files ● dotmesh uses zfs ● zfs snapshots are very efficient

14.

15.

16. How did that work?

17. • dotmesh.yml –> • dm-linuxkit binary which launches etcd and then dotmesh (and clones a dot if 'seed' is specified) • onboot vs service

18. v Get involved!

19. ● github.com/linuxkit/linuxkit ● github.com/dotmesh-io/dotmesh ● github.com/dotmesh-io/dm-linuxkit

20. ● Docker Community Slack #linuxkit ● Dotmesh Slack http://bit.ly/dotmesh-slack ● Search DockerCon talks for "Dotmesh"

21. ● Find more excuses to drink beer together ● More reliable on local Mac (qemu and hyperkit both broken in different ways!) ● Make dotmesh work with newest zfs so that it to works on latest kernels (4.14+)

22. ● Cherry-pick files between dots in dothub ● ARM support ● /var/lib/docker in a dot ● What would you like to see?

23. v Questions?

A vision of persistence

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to A vision of persistence

Similar to A vision of persistence (20)

More from Docker, Inc.

More from Docker, Inc. (20)

Recently uploaded

Recently uploaded (13)

A vision of persistence