SlideShare a Scribd company logo
ssThe State of Linux Containers
2
Gaikai
PS Now announcement at CES 2014
3
Gaikai
- caching
+ controller feedback
1. “Linux Container” / “Docker Ecosystem” in a Nutshell
2. Confusion about Ecosystem / Vision to tackle it
3. Docker -> SWARM -> SLURM 

-> BigData
4. Discussion of Opportunities and Problems
4
Agenda
The Bits and Pieces…
Userland	(OS)Userland	(OS) Userland	(OS)
Userland	(OS)
Ubuntu:14.04 Ubuntu:15.10 RHEL7.2
Tiny	Core	Linux	
Linux Containers
6
SERVER
HOST	KERNEL
HYPERVISOR
KERNEL
SERVICE
Userland	(OS)
KERNEL KERNEL
Userland	(OS)Userland	(OS) Userland	(OS)
SERVICE SERVICE
SERVER
HOST	KERNEL
SERVICE SERVICE SERVICE
Traditional Virtualisation Containerisation
Containers do not spin up a distinct kernel
all containers & the host share the same
user-lands are independent
they are separated by Kernel Namespaces
Containers are ‘grouped processes’
isolated by Kernel Namespaces
resource restrictions applicable through CGroups (disk/netIO)
HOST
container1
7
Kernel Namespaces
bash
ls -l
container2
apache
container3
mysqld
consul consul
PIDNamespaces: Network Mount IPC UTS
container4
slurmd
ssh
consul
Container Runtime Daemon
creates/…/removes containers, exposes REST API
handles Namespaces, CGroups, bind-mounts, etc.
IP connectivity by default via ‘host-only’ network bridge
Docker Engine
8
SERVER
eth0
docker0
container1
container2
Docker-Engine
Docker Compose
9
Describes stack of container configurations
instead of writing a small bash script…
… it holds the runtime configuration as YAML file.
Docker Networking spans networks across engines
KV-store to synchronise (Zookeeper, etcd, Consul)
VXLAN to pass messages along
SERVER0 SERVER1 SERVER<n>
Docker Networking
10
Consul
Docker-Engine
Consul Consul
Docker-Engine Docker-Engine
Consul DC
global
container0 container1 containerN
Docker Swarm proxies docker-engines
serves an API endpoint in front of multiple docker-engines
does placement decisions.
SERVER0 SERVER1 SERVER<n>
Docker Swarm
11
Docker-Engine Docker-Engine Docker-Engine
swarm-client swarm-client swarm-client
swarm-master
:2376 :2376 :2376
:2375
container1
-e constraint:node==SERVER0
Docker Swarm [cont]
12
query docker-enginequery docker-swarm
Introduce new Technologies
Introducing new Tech
14
Self-perception when introducing new tech…
credit: TF2 - Meet the Pyro
Introducing new Tech
15
… not always the same as the perception of others.
credit: TF2 - Meet the Pyro
Docker Buzzword Chaos!
Distributions
Solutions
Auto-Scaling
On-Premise & OverSpill
Orchestration
self-healing
16
production-ready
enterprise-grade
1. No special distributions
useful for certain use-cases, such as elasticity and green-field
deployment
not so much for an on-premise datacenter w/ legacy in it.
2. Leverage existing processes/resources
install workflow, syslog, monitoring
security (ssh infrastructure), user auth.
3. keep up with docker ecosystem
incorporate new features of engine, swarm, compose
networking, volumes, user-namespaces
17
Vision
Reduce to the max!
Hardware (courtesy of )
8x Sun Fire x2250, 2x 4core XEON, 32GB, Mellanox ConnectX-2)
Software
Base installation
CentOS 7.2 base installation (updated from 7-alpha)
Ansible
consul, sensu
docker v1.10, docker-compose
docker SWARM
19
Testbed
node1
node2
node8
20
Docker Networking
Synchronised by Consul
Consul
Consul

DC
Consul
Consul
Docker-Engine
Docker-Engine
Docker-Engine
node1
node2
node8
21
Docker SWARM
Docker SWARM
Synchronised by Consul KV-
store
Consul
Consul

DC
Consul
Consul
Docker-Engine
Docker-Engine
Docker-Engine
swarm
swarm
SWARM
swarm master
node8
node2
node1
22
SLURM Cluster
Consul
Consul

DC
Consul
Consul
SLURM within SWARM
slurmctld slurmd
slurmd
slurmd
Docker-Engine
Docker-Engine
Docker-Engine
swarm
swarm
SWARM
swarm master
SLURM
23
SLURM Cluster [cont]
node8
node2
node1
24
SLURM Cluster [cont]
Consul
Consul

DC
Consul
Consul
SLURM within SWARM
slurmd within app-container
pre-stage containers slurmctld slurmd
slurmd
slurmd
Docker-Engine
Docker-Engine
Docker-Engine
swarm
swarm
hpcg
hpcg
SWARM
hpcg
swarm master
SLURM
25
MPI Benchmark
http://qnib.org/mpi
http://qnib.org/mpi-paper
node8
node2
node1
26
SLURM Cluster [cont]
Consul
Consul

DC
Consul
Consul
SLURM within SWARM
slurmd within app-container
pre-stage containers slurmctld slurmd
slurmd
slurmd
Docker-Engine
Docker-Engine
Docker-Engine
swarm
swarm
hpcg
hpcg
SWARM
hpcg
OpenFOAM
OpenFOAM
OpenFOAM
swarm master
SLURM
27
OpenFOAM Benchmark
http://qnib.org/immutable
http://qnib.org/immutable-paper
node1
node2
node8
28
Samza Cluster
Consul
Consul

DC
Consul
Consul
Distributed Samza
Zookeeper and Kafka cluster
Samza instances to run jobs Docker-Engine
Docker-Engine
Docker-Engine
swarm
swarm
SWARM
swarm masterzookeeper
zookeeper
zookeeper
kafka
kafka
kafka
samza
samza
samza
$ cat test.log |awk ‘{print $1}’ |sed -e ’s/HPC/BigData/g’ |tee out.log
To Be Explored
1. Where to base images on?
Ubuntu/Fedora: ~200MB
Debian: ~100MB
Alpine Linux: 5MB (musl-libc)
2. Trimm the Images down at all cost?
How about debugging tools? Possibility to run tools on the host
and ‘inspect’ namespaced processes inside of a container.
If PID-sharing arrives, carving out (e.g.) monitoring could be a
thing.
30
Small vs. Big
1. In an ideal world…
a container only runs one process, e.g. the HPC solver.
2. In reality…
MPI want’s to connect to a sshd within the job-peers
monitoring, syslog, service discovery should be present as well.
3. How fast / aggressive to break traditional
approaches?
31
One vs. Many Processes
Plugin System
VXLAN
MACVLAN
How about IPoIB?
32
Docker Network
Running OpenFOAM on small scale is cumbersome
manually install OpenFOAM on a workstation
be confident that the installation works correctly
A containerised OpenFOAM installation tackles both
33
Reproducibility / Downscaling
http://qnib.org/immutablehttp://qnib.org/immutable-paper
1. Since the environments are rather dynamic…
how does the containers discover services?
external registry as part of the framework?
discovery service as part of the container stacks?
34
Service Discovery
With Docker Swarm it is rather easy
to spin up a Kubernetes or Mesos cluster within Swarm.
35
Orchestration Frameworks
SERVER0 SERVER1 SERVER<n>
Docker-Engine Docker-Engine Docker-Engine
swarm-client swarm-client swarm-client
swarm-master
etcd
kubelet
scheduler apiserver
etcd
kubelet
etcd
kubelet
1. Containers should be controlled via ENV or flags
External access/change of a running container is discouraged
2. Configuration management
Downgraded to bootstrap a host?
36
Immutable vs. Config Mgmt
If containers are immutable within pipeline
testing/deployment should be automated
developers should have a production replica
37
Continuous Dev./Integration
38
Docker Momentum
Software Dev
DatacenterOps
IT Tinkering (Hello World)
Continuous Dev/Int/Dep
Microservices, hyper scale
Big Data
High Performance Computing
HPC
Disclaimer: subjective exaggeration
Spinning up production-like environment is great
MongoDB, PostreSQL, memcached as separate containers
python2.7, python3.4
39
Docker in Software Development
Like python’s virtualenv on steroids,
iteration speedup through reproducibility
Spinning up production-like environment is…
…not that easy
focus more on engineer/scientist, not the software-developer
1. For development it might work
close to non-HPC software dev
2. But is that the iteration-focus?
rather job settings / input data?
40
Docker in HPC development
Split input iteration / development from operation
non-distributed stays vanilla
transition to HPC cluster using tech to foster operation
41
Separation of Concerns?
http://gmkurtzer.github.io/singularity
Input/Dev
Docker-Engine 1.11 will not be the parent of containers
runC usage under the hood
42
containerd Integration
1. Separat Dev and Ops
don’t block the momentum fostering 

iteration speed in Development 
2. Using vanilla docker-tech
keep up with the ecosystem and prevent vendor/ecosystem
lock-in
3. 80/20 rule
have caveats on the radar but don’t bother too much
everything is so fast moving - it’s hard to predict
43
Recap aka. IMHO
Q&A
https://github.com/qnib/hpcac-cluster2016
http://qnib.org
eGalea Workshop (Pisa)

<plz ping me if you are interested>
23.06.2016

More Related Content

What's hot

Cloud Computing using OpenStack
Cloud Computing using OpenStackCloud Computing using OpenStack
Cloud Computing using OpenStack
Jobayer Almahmud Hossain (RHCA, RHCDS, RHCSS)
 
Introduction to Red Hat
Introduction to Red HatIntroduction to Red Hat
Introduction to Red Hat
Albert Wong
 
Introduction to the Helium release of OpenDaylight
Introduction to the Helium release of OpenDaylightIntroduction to the Helium release of OpenDaylight
Introduction to the Helium release of OpenDaylight
SDN Hub
 
VMware NSX + Cumulus Networks: Software Defined Networking
VMware NSX + Cumulus Networks: Software Defined NetworkingVMware NSX + Cumulus Networks: Software Defined Networking
VMware NSX + Cumulus Networks: Software Defined Networking
Cumulus Networks
 
Building Resilient Applications with Cloudflare DNS
Building Resilient Applications with Cloudflare DNSBuilding Resilient Applications with Cloudflare DNS
Building Resilient Applications with Cloudflare DNS
DevOps.com
 
BRKSDN-2115
BRKSDN-2115 BRKSDN-2115
BRKSDN-2115
Rohit Agarwalla
 
VMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep DiveVMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep Dive
VMworld
 
OpenSolaris Introduction
OpenSolaris IntroductionOpenSolaris Introduction
OpenSolaris Introduction
satyajit_t
 
SDN Service Provider use cases Network Function Virtualization (NFV)
SDN Service Provider use cases Network Function Virtualization (NFV)SDN Service Provider use cases Network Function Virtualization (NFV)
SDN Service Provider use cases Network Function Virtualization (NFV)
Brent Salisbury
 
Virtualization technolegys for amdocs
Virtualization technolegys for amdocsVirtualization technolegys for amdocs
Virtualization technolegys for amdocs
Samuel Dratwa
 
VMware NSX - Lessons Learned from real project
VMware NSX - Lessons Learned from real projectVMware NSX - Lessons Learned from real project
VMware NSX - Lessons Learned from real project
David Pasek
 
Cont0519
Cont0519Cont0519
Cont0519
Samuel Dratwa
 
SDN Scale-out Testing at OpenStack Innovation Center (OSIC)
SDN Scale-out Testing at OpenStack Innovation Center (OSIC)SDN Scale-out Testing at OpenStack Innovation Center (OSIC)
SDN Scale-out Testing at OpenStack Innovation Center (OSIC)
PLUMgrid
 
NSX Reference Design version 3.0
NSX Reference Design version 3.0NSX Reference Design version 3.0
NSX Reference Design version 3.0
Doddi Priyambodo
 
Opening Up Your Network with SDN
Opening Up Your Network with SDNOpening Up Your Network with SDN
Opening Up Your Network with SDN
Open Networking Summits
 
Introduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OSIntroduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OS
Steve Wong
 
Presentation cloud orchestration
Presentation   cloud orchestrationPresentation   cloud orchestration
Presentation cloud orchestration
xKinAnx
 
Cisco UCS for OpenStack Cloud
Cisco UCS for OpenStack CloudCisco UCS for OpenStack Cloud
Cisco UCS for OpenStack Cloud
Lora O'Haver
 
DPDK IPSec performance benchmark ~ Georgii Tkachuk
DPDK IPSec performance benchmark ~ Georgii TkachukDPDK IPSec performance benchmark ~ Georgii Tkachuk
DPDK IPSec performance benchmark ~ Georgii Tkachuk
Intel
 
OpenStack: Everything You Need to Know To Get Started
OpenStack: Everything You Need to Know To Get StartedOpenStack: Everything You Need to Know To Get Started
OpenStack: Everything You Need to Know To Get Started
All Things Open
 

What's hot (20)

Cloud Computing using OpenStack
Cloud Computing using OpenStackCloud Computing using OpenStack
Cloud Computing using OpenStack
 
Introduction to Red Hat
Introduction to Red HatIntroduction to Red Hat
Introduction to Red Hat
 
Introduction to the Helium release of OpenDaylight
Introduction to the Helium release of OpenDaylightIntroduction to the Helium release of OpenDaylight
Introduction to the Helium release of OpenDaylight
 
VMware NSX + Cumulus Networks: Software Defined Networking
VMware NSX + Cumulus Networks: Software Defined NetworkingVMware NSX + Cumulus Networks: Software Defined Networking
VMware NSX + Cumulus Networks: Software Defined Networking
 
Building Resilient Applications with Cloudflare DNS
Building Resilient Applications with Cloudflare DNSBuilding Resilient Applications with Cloudflare DNS
Building Resilient Applications with Cloudflare DNS
 
BRKSDN-2115
BRKSDN-2115 BRKSDN-2115
BRKSDN-2115
 
VMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep DiveVMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep Dive
 
OpenSolaris Introduction
OpenSolaris IntroductionOpenSolaris Introduction
OpenSolaris Introduction
 
SDN Service Provider use cases Network Function Virtualization (NFV)
SDN Service Provider use cases Network Function Virtualization (NFV)SDN Service Provider use cases Network Function Virtualization (NFV)
SDN Service Provider use cases Network Function Virtualization (NFV)
 
Virtualization technolegys for amdocs
Virtualization technolegys for amdocsVirtualization technolegys for amdocs
Virtualization technolegys for amdocs
 
VMware NSX - Lessons Learned from real project
VMware NSX - Lessons Learned from real projectVMware NSX - Lessons Learned from real project
VMware NSX - Lessons Learned from real project
 
Cont0519
Cont0519Cont0519
Cont0519
 
SDN Scale-out Testing at OpenStack Innovation Center (OSIC)
SDN Scale-out Testing at OpenStack Innovation Center (OSIC)SDN Scale-out Testing at OpenStack Innovation Center (OSIC)
SDN Scale-out Testing at OpenStack Innovation Center (OSIC)
 
NSX Reference Design version 3.0
NSX Reference Design version 3.0NSX Reference Design version 3.0
NSX Reference Design version 3.0
 
Opening Up Your Network with SDN
Opening Up Your Network with SDNOpening Up Your Network with SDN
Opening Up Your Network with SDN
 
Introduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OSIntroduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OS
 
Presentation cloud orchestration
Presentation   cloud orchestrationPresentation   cloud orchestration
Presentation cloud orchestration
 
Cisco UCS for OpenStack Cloud
Cisco UCS for OpenStack CloudCisco UCS for OpenStack Cloud
Cisco UCS for OpenStack Cloud
 
DPDK IPSec performance benchmark ~ Georgii Tkachuk
DPDK IPSec performance benchmark ~ Georgii TkachukDPDK IPSec performance benchmark ~ Georgii Tkachuk
DPDK IPSec performance benchmark ~ Georgii Tkachuk
 
OpenStack: Everything You Need to Know To Get Started
OpenStack: Everything You Need to Know To Get StartedOpenStack: Everything You Need to Know To Get Started
OpenStack: Everything You Need to Know To Get Started
 

Viewers also liked

OpenPOWER Update
OpenPOWER UpdateOpenPOWER Update
OpenPOWER Update
inside-BigData.com
 
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_finalPresentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
Diego Alberto Tamayo
 
Oracle Solaris Cloud Management and Deployment with OpenStack
Oracle Solaris Cloud Management and Deployment with OpenStackOracle Solaris Cloud Management and Deployment with OpenStack
Oracle Solaris Cloud Management and Deployment with OpenStack
OTN Systems Hub
 
Understanding the IBM Power Systems Advantage
Understanding the IBM Power Systems AdvantageUnderstanding the IBM Power Systems Advantage
Understanding the IBM Power Systems Advantage
IBM Power Systems
 
Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3
Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3
Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3
OTN Systems Hub
 
Co-Design Architecture for Exascale
Co-Design Architecture for ExascaleCo-Design Architecture for Exascale
Co-Design Architecture for Exascale
inside-BigData.com
 
Solaris 11.2 What's New
Solaris 11.2 What's NewSolaris 11.2 What's New
Solaris 11.2 What's New
Orgad Kimchi
 
Accelerating Business Intelligence Solutions with Microsoft Azure pass
Accelerating Business Intelligence Solutions with Microsoft Azure   passAccelerating Business Intelligence Solutions with Microsoft Azure   pass
Accelerating Business Intelligence Solutions with Microsoft Azure pass
Jason Strate
 
Georgia Azure Event - Scalable cloud games using Microsoft Azure
Georgia Azure Event - Scalable cloud games using Microsoft AzureGeorgia Azure Event - Scalable cloud games using Microsoft Azure
Georgia Azure Event - Scalable cloud games using Microsoft Azure
Microsoft
 
OpenPOWER Roadmap Toward CORAL
OpenPOWER Roadmap Toward CORALOpenPOWER Roadmap Toward CORAL
OpenPOWER Roadmap Toward CORAL
inside-BigData.com
 
IBM POWER8 as an HPC platform
IBM POWER8 as an HPC platformIBM POWER8 as an HPC platform
IBM POWER8 as an HPC platform
Alexander Pozdneev
 
Bitcoin explained
Bitcoin explainedBitcoin explained
Bitcoin explained
Benjamin Fuentes
 
Blockchain
BlockchainBlockchain
Blockchain
Benjamin Fuentes
 
Oracle Solaris Software Integration
Oracle Solaris Software IntegrationOracle Solaris Software Integration
Oracle Solaris Software Integration
OTN Systems Hub
 
Open Innovation with Power Systems
Open Innovation with Power Systems Open Innovation with Power Systems
Open Innovation with Power Systems
IBM Power Systems
 
IBM Power8 announce
IBM Power8 announceIBM Power8 announce
IBM Power8 announce
Anna Landolfi
 
Expert summit SQL Server 2016
Expert summit   SQL Server 2016Expert summit   SQL Server 2016
Expert summit SQL Server 2016
Łukasz Grala
 
Puppet + Windows Nano Server
Puppet + Windows Nano ServerPuppet + Windows Nano Server
Puppet + Windows Nano Server
Alessandro Pilotti
 
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
Mark Rittman
 
Oracle Solaris Build and Run Applications Better on 11.3
Oracle Solaris  Build and Run Applications Better on 11.3Oracle Solaris  Build and Run Applications Better on 11.3
Oracle Solaris Build and Run Applications Better on 11.3
OTN Systems Hub
 

Viewers also liked (20)

OpenPOWER Update
OpenPOWER UpdateOpenPOWER Update
OpenPOWER Update
 
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_finalPresentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
 
Oracle Solaris Cloud Management and Deployment with OpenStack
Oracle Solaris Cloud Management and Deployment with OpenStackOracle Solaris Cloud Management and Deployment with OpenStack
Oracle Solaris Cloud Management and Deployment with OpenStack
 
Understanding the IBM Power Systems Advantage
Understanding the IBM Power Systems AdvantageUnderstanding the IBM Power Systems Advantage
Understanding the IBM Power Systems Advantage
 
Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3
Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3
Oracle Solaris Simple, Flexible, Fast: Virtualization in 11.3
 
Co-Design Architecture for Exascale
Co-Design Architecture for ExascaleCo-Design Architecture for Exascale
Co-Design Architecture for Exascale
 
Solaris 11.2 What's New
Solaris 11.2 What's NewSolaris 11.2 What's New
Solaris 11.2 What's New
 
Accelerating Business Intelligence Solutions with Microsoft Azure pass
Accelerating Business Intelligence Solutions with Microsoft Azure   passAccelerating Business Intelligence Solutions with Microsoft Azure   pass
Accelerating Business Intelligence Solutions with Microsoft Azure pass
 
Georgia Azure Event - Scalable cloud games using Microsoft Azure
Georgia Azure Event - Scalable cloud games using Microsoft AzureGeorgia Azure Event - Scalable cloud games using Microsoft Azure
Georgia Azure Event - Scalable cloud games using Microsoft Azure
 
OpenPOWER Roadmap Toward CORAL
OpenPOWER Roadmap Toward CORALOpenPOWER Roadmap Toward CORAL
OpenPOWER Roadmap Toward CORAL
 
IBM POWER8 as an HPC platform
IBM POWER8 as an HPC platformIBM POWER8 as an HPC platform
IBM POWER8 as an HPC platform
 
Bitcoin explained
Bitcoin explainedBitcoin explained
Bitcoin explained
 
Blockchain
BlockchainBlockchain
Blockchain
 
Oracle Solaris Software Integration
Oracle Solaris Software IntegrationOracle Solaris Software Integration
Oracle Solaris Software Integration
 
Open Innovation with Power Systems
Open Innovation with Power Systems Open Innovation with Power Systems
Open Innovation with Power Systems
 
IBM Power8 announce
IBM Power8 announceIBM Power8 announce
IBM Power8 announce
 
Expert summit SQL Server 2016
Expert summit   SQL Server 2016Expert summit   SQL Server 2016
Expert summit SQL Server 2016
 
Puppet + Windows Nano Server
Puppet + Windows Nano ServerPuppet + Windows Nano Server
Puppet + Windows Nano Server
 
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
IlOUG Tech Days 2016 - Big Data for Oracle Developers - Towards Spark, Real-T...
 
Oracle Solaris Build and Run Applications Better on 11.3
Oracle Solaris  Build and Run Applications Better on 11.3Oracle Solaris  Build and Run Applications Better on 11.3
Oracle Solaris Build and Run Applications Better on 11.3
 

Similar to The State of Linux Containers

State of Linux Containers for HPC
State of Linux Containers for HPCState of Linux Containers for HPC
State of Linux Containers for HPC
inside-BigData.com
 
Dockers zero to hero
Dockers zero to heroDockers zero to hero
Dockers zero to hero
Nicolas De Loof
 
Containerize! Between Docker and Jube.
Containerize! Between Docker and Jube.Containerize! Between Docker and Jube.
Containerize! Between Docker and Jube.
Henryk Konsek
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
Eueung Mulyana
 
Evolution of Linux Containerization
Evolution of Linux Containerization Evolution of Linux Containerization
Evolution of Linux Containerization
WSO2
 
Evoluation of Linux Container Virtualization
Evoluation of Linux Container VirtualizationEvoluation of Linux Container Virtualization
Evoluation of Linux Container Virtualization
Imesh Gunaratne
 
Containers and Nutanix - Acropolis Container Services
Containers and Nutanix - Acropolis Container ServicesContainers and Nutanix - Acropolis Container Services
Containers and Nutanix - Acropolis Container Services
NEXTtour
 
Docker Devoxx UK - Never mind the bollocks here's the Linux Containers
Docker Devoxx UK - Never mind the bollocks here's the Linux ContainersDocker Devoxx UK - Never mind the bollocks here's the Linux Containers
Docker Devoxx UK - Never mind the bollocks here's the Linux ContainersPatrick Chanezon
 
Docker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshitDocker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshit
Yusuf Hadiwinata Sutandar
 
Revolutionizing WSO2 PaaS with Kubernetes & App Factory
Revolutionizing WSO2 PaaS with Kubernetes & App FactoryRevolutionizing WSO2 PaaS with Kubernetes & App Factory
Revolutionizing WSO2 PaaS with Kubernetes & App Factory
Imesh Gunaratne
 
Revolutionizing the cloud with container virtualization
Revolutionizing the cloud with container virtualizationRevolutionizing the cloud with container virtualization
Revolutionizing the cloud with container virtualizationWSO2
 
Accelerate your development with Docker
Accelerate your development with DockerAccelerate your development with Docker
Accelerate your development with Docker
Andrey Hristov
 
Accelerate your software development with Docker
Accelerate your software development with DockerAccelerate your software development with Docker
Accelerate your software development with Docker
Andrey Hristov
 
Academy PRO: Docker. Lecture 1
Academy PRO: Docker. Lecture 1Academy PRO: Docker. Lecture 1
Academy PRO: Docker. Lecture 1
Binary Studio
 
Linux container & docker
Linux container & dockerLinux container & docker
Linux container & docker
ejlp12
 
Weave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapWeave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 Recap
Patrick Chanezon
 
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Michelle Antebi
 
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael CrosbyDocker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker, Inc.
 
Docker at the gate
Docker at the gateDocker at the gate
Docker at the gate
Tony Deng
 
Evolution of containers to kubernetes
Evolution of containers to kubernetesEvolution of containers to kubernetes
Evolution of containers to kubernetes
Krishna-Kumar
 

Similar to The State of Linux Containers (20)

State of Linux Containers for HPC
State of Linux Containers for HPCState of Linux Containers for HPC
State of Linux Containers for HPC
 
Dockers zero to hero
Dockers zero to heroDockers zero to hero
Dockers zero to hero
 
Containerize! Between Docker and Jube.
Containerize! Between Docker and Jube.Containerize! Between Docker and Jube.
Containerize! Between Docker and Jube.
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Evolution of Linux Containerization
Evolution of Linux Containerization Evolution of Linux Containerization
Evolution of Linux Containerization
 
Evoluation of Linux Container Virtualization
Evoluation of Linux Container VirtualizationEvoluation of Linux Container Virtualization
Evoluation of Linux Container Virtualization
 
Containers and Nutanix - Acropolis Container Services
Containers and Nutanix - Acropolis Container ServicesContainers and Nutanix - Acropolis Container Services
Containers and Nutanix - Acropolis Container Services
 
Docker Devoxx UK - Never mind the bollocks here's the Linux Containers
Docker Devoxx UK - Never mind the bollocks here's the Linux ContainersDocker Devoxx UK - Never mind the bollocks here's the Linux Containers
Docker Devoxx UK - Never mind the bollocks here's the Linux Containers
 
Docker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshitDocker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshit
 
Revolutionizing WSO2 PaaS with Kubernetes & App Factory
Revolutionizing WSO2 PaaS with Kubernetes & App FactoryRevolutionizing WSO2 PaaS with Kubernetes & App Factory
Revolutionizing WSO2 PaaS with Kubernetes & App Factory
 
Revolutionizing the cloud with container virtualization
Revolutionizing the cloud with container virtualizationRevolutionizing the cloud with container virtualization
Revolutionizing the cloud with container virtualization
 
Accelerate your development with Docker
Accelerate your development with DockerAccelerate your development with Docker
Accelerate your development with Docker
 
Accelerate your software development with Docker
Accelerate your software development with DockerAccelerate your software development with Docker
Accelerate your software development with Docker
 
Academy PRO: Docker. Lecture 1
Academy PRO: Docker. Lecture 1Academy PRO: Docker. Lecture 1
Academy PRO: Docker. Lecture 1
 
Linux container & docker
Linux container & dockerLinux container & docker
Linux container & docker
 
Weave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapWeave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 Recap
 
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
 
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael CrosbyDocker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
Docker 1.11 Meetup: Containerd and runc, by Arnaud Porterie and Michael Crosby
 
Docker at the gate
Docker at the gateDocker at the gate
Docker at the gate
 
Evolution of containers to kubernetes
Evolution of containers to kubernetesEvolution of containers to kubernetes
Evolution of containers to kubernetes
 

More from inside-BigData.com

Major Market Shifts in IT
Major Market Shifts in ITMajor Market Shifts in IT
Major Market Shifts in IT
inside-BigData.com
 
Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...
inside-BigData.com
 
Transforming Private 5G Networks
Transforming Private 5G NetworksTransforming Private 5G Networks
Transforming Private 5G Networks
inside-BigData.com
 
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
inside-BigData.com
 
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
inside-BigData.com
 
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
inside-BigData.com
 
HPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural NetworksHPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural Networks
inside-BigData.com
 
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean MonitoringBiohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
inside-BigData.com
 
Machine Learning for Weather Forecasts
Machine Learning for Weather ForecastsMachine Learning for Weather Forecasts
Machine Learning for Weather Forecasts
inside-BigData.com
 
HPC AI Advisory Council Update
HPC AI Advisory Council UpdateHPC AI Advisory Council Update
HPC AI Advisory Council Update
inside-BigData.com
 
Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19
inside-BigData.com
 
Energy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic TuningEnergy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic Tuning
inside-BigData.com
 
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPODHPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
inside-BigData.com
 
State of ARM-based HPC
State of ARM-based HPCState of ARM-based HPC
State of ARM-based HPC
inside-BigData.com
 
Versal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud AccelerationVersal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud Acceleration
inside-BigData.com
 
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance EfficientlyZettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
inside-BigData.com
 
Scaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's EraScaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's Era
inside-BigData.com
 
CUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computingCUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computing
inside-BigData.com
 
Introducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi ClusterIntroducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi Cluster
inside-BigData.com
 
Overview of HPC Interconnects
Overview of HPC InterconnectsOverview of HPC Interconnects
Overview of HPC Interconnects
inside-BigData.com
 

More from inside-BigData.com (20)

Major Market Shifts in IT
Major Market Shifts in ITMajor Market Shifts in IT
Major Market Shifts in IT
 
Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...
 
Transforming Private 5G Networks
Transforming Private 5G NetworksTransforming Private 5G Networks
Transforming Private 5G Networks
 
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
 
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
 
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
 
HPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural NetworksHPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural Networks
 
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean MonitoringBiohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
 
Machine Learning for Weather Forecasts
Machine Learning for Weather ForecastsMachine Learning for Weather Forecasts
Machine Learning for Weather Forecasts
 
HPC AI Advisory Council Update
HPC AI Advisory Council UpdateHPC AI Advisory Council Update
HPC AI Advisory Council Update
 
Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19
 
Energy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic TuningEnergy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic Tuning
 
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPODHPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
 
State of ARM-based HPC
State of ARM-based HPCState of ARM-based HPC
State of ARM-based HPC
 
Versal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud AccelerationVersal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud Acceleration
 
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance EfficientlyZettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
 
Scaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's EraScaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's Era
 
CUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computingCUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computing
 
Introducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi ClusterIntroducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi Cluster
 
Overview of HPC Interconnects
Overview of HPC InterconnectsOverview of HPC Interconnects
Overview of HPC Interconnects
 

Recently uploaded

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
Fwdays
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 

Recently uploaded (20)

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 

The State of Linux Containers

  • 1. ssThe State of Linux Containers
  • 4. 1. “Linux Container” / “Docker Ecosystem” in a Nutshell 2. Confusion about Ecosystem / Vision to tackle it 3. Docker -> SWARM -> SLURM 
 -> BigData 4. Discussion of Opportunities and Problems 4 Agenda
  • 5. The Bits and Pieces…
  • 6. Userland (OS)Userland (OS) Userland (OS) Userland (OS) Ubuntu:14.04 Ubuntu:15.10 RHEL7.2 Tiny Core Linux Linux Containers 6 SERVER HOST KERNEL HYPERVISOR KERNEL SERVICE Userland (OS) KERNEL KERNEL Userland (OS)Userland (OS) Userland (OS) SERVICE SERVICE SERVER HOST KERNEL SERVICE SERVICE SERVICE Traditional Virtualisation Containerisation Containers do not spin up a distinct kernel all containers & the host share the same user-lands are independent they are separated by Kernel Namespaces
  • 7. Containers are ‘grouped processes’ isolated by Kernel Namespaces resource restrictions applicable through CGroups (disk/netIO) HOST container1 7 Kernel Namespaces bash ls -l container2 apache container3 mysqld consul consul PIDNamespaces: Network Mount IPC UTS container4 slurmd ssh consul
  • 8. Container Runtime Daemon creates/…/removes containers, exposes REST API handles Namespaces, CGroups, bind-mounts, etc. IP connectivity by default via ‘host-only’ network bridge Docker Engine 8 SERVER eth0 docker0 container1 container2 Docker-Engine
  • 9. Docker Compose 9 Describes stack of container configurations instead of writing a small bash script… … it holds the runtime configuration as YAML file.
  • 10. Docker Networking spans networks across engines KV-store to synchronise (Zookeeper, etcd, Consul) VXLAN to pass messages along SERVER0 SERVER1 SERVER<n> Docker Networking 10 Consul Docker-Engine Consul Consul Docker-Engine Docker-Engine Consul DC global container0 container1 containerN
  • 11. Docker Swarm proxies docker-engines serves an API endpoint in front of multiple docker-engines does placement decisions. SERVER0 SERVER1 SERVER<n> Docker Swarm 11 Docker-Engine Docker-Engine Docker-Engine swarm-client swarm-client swarm-client swarm-master :2376 :2376 :2376 :2375 container1 -e constraint:node==SERVER0
  • 12. Docker Swarm [cont] 12 query docker-enginequery docker-swarm
  • 14. Introducing new Tech 14 Self-perception when introducing new tech… credit: TF2 - Meet the Pyro
  • 15. Introducing new Tech 15 … not always the same as the perception of others. credit: TF2 - Meet the Pyro
  • 16. Docker Buzzword Chaos! Distributions Solutions Auto-Scaling On-Premise & OverSpill Orchestration self-healing 16 production-ready enterprise-grade
  • 17. 1. No special distributions useful for certain use-cases, such as elasticity and green-field deployment not so much for an on-premise datacenter w/ legacy in it. 2. Leverage existing processes/resources install workflow, syslog, monitoring security (ssh infrastructure), user auth. 3. keep up with docker ecosystem incorporate new features of engine, swarm, compose networking, volumes, user-namespaces 17 Vision
  • 19. Hardware (courtesy of ) 8x Sun Fire x2250, 2x 4core XEON, 32GB, Mellanox ConnectX-2) Software Base installation CentOS 7.2 base installation (updated from 7-alpha) Ansible consul, sensu docker v1.10, docker-compose docker SWARM 19 Testbed
  • 20. node1 node2 node8 20 Docker Networking Synchronised by Consul Consul Consul
 DC Consul Consul Docker-Engine Docker-Engine Docker-Engine
  • 21. node1 node2 node8 21 Docker SWARM Docker SWARM Synchronised by Consul KV- store Consul Consul
 DC Consul Consul Docker-Engine Docker-Engine Docker-Engine swarm swarm SWARM swarm master
  • 22. node8 node2 node1 22 SLURM Cluster Consul Consul
 DC Consul Consul SLURM within SWARM slurmctld slurmd slurmd slurmd Docker-Engine Docker-Engine Docker-Engine swarm swarm SWARM swarm master SLURM
  • 24. node8 node2 node1 24 SLURM Cluster [cont] Consul Consul
 DC Consul Consul SLURM within SWARM slurmd within app-container pre-stage containers slurmctld slurmd slurmd slurmd Docker-Engine Docker-Engine Docker-Engine swarm swarm hpcg hpcg SWARM hpcg swarm master SLURM
  • 26. node8 node2 node1 26 SLURM Cluster [cont] Consul Consul
 DC Consul Consul SLURM within SWARM slurmd within app-container pre-stage containers slurmctld slurmd slurmd slurmd Docker-Engine Docker-Engine Docker-Engine swarm swarm hpcg hpcg SWARM hpcg OpenFOAM OpenFOAM OpenFOAM swarm master SLURM
  • 28. node1 node2 node8 28 Samza Cluster Consul Consul
 DC Consul Consul Distributed Samza Zookeeper and Kafka cluster Samza instances to run jobs Docker-Engine Docker-Engine Docker-Engine swarm swarm SWARM swarm masterzookeeper zookeeper zookeeper kafka kafka kafka samza samza samza $ cat test.log |awk ‘{print $1}’ |sed -e ’s/HPC/BigData/g’ |tee out.log
  • 30. 1. Where to base images on? Ubuntu/Fedora: ~200MB Debian: ~100MB Alpine Linux: 5MB (musl-libc) 2. Trimm the Images down at all cost? How about debugging tools? Possibility to run tools on the host and ‘inspect’ namespaced processes inside of a container. If PID-sharing arrives, carving out (e.g.) monitoring could be a thing. 30 Small vs. Big
  • 31. 1. In an ideal world… a container only runs one process, e.g. the HPC solver. 2. In reality… MPI want’s to connect to a sshd within the job-peers monitoring, syslog, service discovery should be present as well. 3. How fast / aggressive to break traditional approaches? 31 One vs. Many Processes
  • 32. Plugin System VXLAN MACVLAN How about IPoIB? 32 Docker Network
  • 33. Running OpenFOAM on small scale is cumbersome manually install OpenFOAM on a workstation be confident that the installation works correctly A containerised OpenFOAM installation tackles both 33 Reproducibility / Downscaling http://qnib.org/immutablehttp://qnib.org/immutable-paper
  • 34. 1. Since the environments are rather dynamic… how does the containers discover services? external registry as part of the framework? discovery service as part of the container stacks? 34 Service Discovery
  • 35. With Docker Swarm it is rather easy to spin up a Kubernetes or Mesos cluster within Swarm. 35 Orchestration Frameworks SERVER0 SERVER1 SERVER<n> Docker-Engine Docker-Engine Docker-Engine swarm-client swarm-client swarm-client swarm-master etcd kubelet scheduler apiserver etcd kubelet etcd kubelet
  • 36. 1. Containers should be controlled via ENV or flags External access/change of a running container is discouraged 2. Configuration management Downgraded to bootstrap a host? 36 Immutable vs. Config Mgmt
  • 37. If containers are immutable within pipeline testing/deployment should be automated developers should have a production replica 37 Continuous Dev./Integration
  • 38. 38 Docker Momentum Software Dev DatacenterOps IT Tinkering (Hello World) Continuous Dev/Int/Dep Microservices, hyper scale Big Data High Performance Computing HPC Disclaimer: subjective exaggeration
  • 39. Spinning up production-like environment is great MongoDB, PostreSQL, memcached as separate containers python2.7, python3.4 39 Docker in Software Development Like python’s virtualenv on steroids, iteration speedup through reproducibility
  • 40. Spinning up production-like environment is… …not that easy focus more on engineer/scientist, not the software-developer 1. For development it might work close to non-HPC software dev 2. But is that the iteration-focus? rather job settings / input data? 40 Docker in HPC development
  • 41. Split input iteration / development from operation non-distributed stays vanilla transition to HPC cluster using tech to foster operation 41 Separation of Concerns? http://gmkurtzer.github.io/singularity Input/Dev
  • 42. Docker-Engine 1.11 will not be the parent of containers runC usage under the hood 42 containerd Integration
  • 43. 1. Separat Dev and Ops don’t block the momentum fostering 
 iteration speed in Development  2. Using vanilla docker-tech keep up with the ecosystem and prevent vendor/ecosystem lock-in 3. 80/20 rule have caveats on the radar but don’t bother too much everything is so fast moving - it’s hard to predict 43 Recap aka. IMHO