• Jeff Victor
• Principal Author, “Oracle Solaris 10 System Virtualization Essentials”
• Kir Kolyshkin
• OpenVZ Containers Lead
History of Containers
Copyright 2015 Oracle Corp.
• Origin of Virtualization
• Timelines
• Concepts
• OSV vs. VMs
• Shallow Dives
Agenda
Copyright 2015 Oracle Corp.
The Mists of Antiquity
1960 19751965 1970
MIT CTSS CP-40
CP-67 VM/370
Batch
The Three (4? 5?) “Laws”
of Virtualization
• Initial
• Equivalence
• Resource Control
• Efficiency
• Later
• Security
• Administrative Observability
Copyright 2015 Oracle Corp.
The Dark Ages
1975 19991980
chroot
Unix V7* 4BSD
1990
“jail”
Sun
Dynamic
Domains
*And thereafter:
Sun3 w/s
Xenix/8086
Unix/32V (Vax)
JVM
Copyright 2015 Oracle Corp.
The System Virtualization Spectrum
More Flexibility
OS
CPU
RAM
I/O
Partition
OS
CPU
RAM
I/O
Partition
OS
CPU
RAM
I/O
Partition
Partitions
Interconnect
OS
Virtual
Machine
OS
Virtual
Machine
OS
Virtual
Machine
Virtual Machines
Hypervisor
Hardware
Zone
Operating System
Zone Zone
OS Virtualization
Hardware
More Isolation
Copyright 2015 Oracle Corp.
Container / Zone
A collection of software processes unified by one
namespace, with access to an operating system kernel
that it shares with other containers, and little to no
access between containers.
Copyright 2015 Oracle Corp.
Container Advantages
• Leverage mature OS
• Lightweight
• Fewer resources: shared kernel, optional shared text pages
• Faster to provision, boot, shutdown
• “Zero” overhead: faster, better scalability, more predictable consolidation
• Better resource sharing
• Faster context switch
• Direct path to I/O
• More sophisticated “Control Program”: one control point
• Better Observability
• More flexible access to hardware
Copyright 2015 Oracle Corp.
Container Disadvantages
• Homogeneity
• Most require all containers to run at the same kernel patch level
• Heterogenous guest OS is uncommon
• Must sacrifice performance to run heterogeneous guests
• Less isolation
• More sophisticated Control Program: more code to fail
• Slightly greater chance for downtime for multiple workloads
• Can be mitigated…
Copyright 2015 Oracle Corp.
The Virtual Renaissance
1999 20152005 2010
VMware
W/S HP
nPars
FreeBSD
jails
VMware
ESX
Virtuozzo
Power
LPARs
Linux
VServers
Solaris
Zones
OpenVZ
HP
vPars
Integrity VM
HP-UX
SRP
Sun
LDoms
HP
Dynamic
nPars
AIX
WPars
LXC
HP-UX
System
Containers,
HP 9000
Containers
Solaris
Kernel
Zones
VirtualBox
Xen HyperV
VMs
OSV
KVM
Solaris
“Containers”
All dates are approximate, v1.0/stable.Copyright 2015 Oracle Corp.
The Virtual Renaissance
(non-Unix)
1999 20152005 2010
VMware
W/S VMware
ESX
Virtuozzo
Linux
VServers
OpenVZ
LXC
VirtualBox
Xen HyperV
VMs
OSV
KVM
All dates are approximate, v1.0/stable.Copyright 2015 Oracle Corp.
The Virtual Renaissance
(Unix)
1999 20152005 2010
HP
nPars
FreeBSD
jails
Power
LPARs
Solaris
Zones
HP
vPars
Integrity VM
HP-UX
SRPv2
Sun
LDoms
HP
Dynamic
nPars
AIX
WPars
HP-UX
System
Containers,
HP 9000
Containers
Solaris
Kernel
Zones
VirtualBox
VMs
OSV
Solaris
“Containers”
All dates are approximate, v1.0/stable.Copyright 2015 Oracle Corp.
The “Three” “Laws” of V12N:
Oracle Solaris Zones
• Initial
• Equivalence: very difficult to tell you're in a Zone
• Resource Control: controls for CPU, RAM, VM, BW
• Efficiency: no software layer… no perf overhead
• Later
• Security: fine-grained, configurable privileges
• Administrative Observability
• Global Zone can “see” into all native zones, but they can't see
back into GZ, or into each other.
Copyright 2015 Oracle Corp.
Solaris Zones
• Privileges
• Namespace
• File system
• Networking
• Resource Controls
(aka “Solaris Containers” 2007-2011)
Copyright 2015 Oracle Corp.
Solaris Zones
• Privileges
• Fine-grained abilities (e.g. SYS_TIME, SYS_SMB)
• Define the security boundary
• Cannot be changed while the zone runs
• Configurable
• Tighten or loosen security boundary from defaults
• Tight for web-facing, loose for well-protected, specialty
• Trusted Extensions
• DoD-grade features: compartmentalization, etc.
Copyright 2015 Oracle Corp.
Solaris Zones
• Namespace
• Naming service, users/groups
• Network services
• Configuration choices
Copyright 2015 Oracle Corp.
Solaris Zones
• File system
• Entirely separate storage pool, or just a file system
• GZ can add other mounts (ZFS, UFS, VxFS, lofs, ...)
• Zone can mount or share NFS shares
• Zone can use LUNs configured into it
• Mandatory Access Control (“Immutable Zones”)
• Can choose from 4 levels of hardening
• Most secure: can't modify any Solaris files
• Great for web-facing environments
Copyright 2015 Oracle Corp.
Solaris Zones
• Networking
• By default, Zones use Solaris vNICs
• Individual routing, firewall config
• Solaris network v12n also includes vSwitches, vRouters
• Elastic Virtual Switch - spans computers
• Plumbing via VLAN or VXLAN (routable)
• Can build:
• an arbitrary network structure in one Solaris instance
• ...with multiple subnets, per-zone routing, firewall rules, NAT
• ...and bandwidth controls and load-balancing
• Great for prototyping networks
• IP, MAC address spoof prevention, ...
Copyright 2015 Oracle Corp.
Solaris Zones
• Resource Controls
• CPU
• Pools: assign a zone to specific cores, strands
• CPU Cap: accounting cap on CPU time
• FSS: Fair Share Scheduler
• RAM Cap, VM cap: accounting cap: RAM, Virtual Memory
• Max-Processes cap
• Shared-memory cap, ...
• Network bandwidth controls
Copyright 2015 Oracle Corp.
Solaris Zones
• Non-native Zones
• Solaris 11 Kernel Zones
• Separate kernel and patch level, more like a VM
• Solaris 10 Zone in Solaris 11 system
• System-call translation layer
• Takes advantage of underlying Solaris 11 features
• Network virtualization, transparent encryption, …
Copyright 2015 Oracle Corp.
The Future: Container Management
2010 2015
Docker
OpenStack
?
2005
Ops Center
SolarWinds
VirtualCenter
Joyent
Triton
Copyright 2015 Oracle Corp.
The End

ContainerDays Boston 2015: "A Brief History of Containers" (Jeff Victor & Kir Kolyshkin)

  • 1.
    • Jeff Victor •Principal Author, “Oracle Solaris 10 System Virtualization Essentials” • Kir Kolyshkin • OpenVZ Containers Lead History of Containers Copyright 2015 Oracle Corp.
  • 2.
    • Origin ofVirtualization • Timelines • Concepts • OSV vs. VMs • Shallow Dives Agenda Copyright 2015 Oracle Corp.
  • 3.
    The Mists ofAntiquity 1960 19751965 1970 MIT CTSS CP-40 CP-67 VM/370 Batch
  • 4.
    The Three (4?5?) “Laws” of Virtualization • Initial • Equivalence • Resource Control • Efficiency • Later • Security • Administrative Observability Copyright 2015 Oracle Corp.
  • 5.
    The Dark Ages 197519991980 chroot Unix V7* 4BSD 1990 “jail” Sun Dynamic Domains *And thereafter: Sun3 w/s Xenix/8086 Unix/32V (Vax) JVM Copyright 2015 Oracle Corp.
  • 6.
    The System VirtualizationSpectrum More Flexibility OS CPU RAM I/O Partition OS CPU RAM I/O Partition OS CPU RAM I/O Partition Partitions Interconnect OS Virtual Machine OS Virtual Machine OS Virtual Machine Virtual Machines Hypervisor Hardware Zone Operating System Zone Zone OS Virtualization Hardware More Isolation Copyright 2015 Oracle Corp.
  • 7.
    Container / Zone Acollection of software processes unified by one namespace, with access to an operating system kernel that it shares with other containers, and little to no access between containers. Copyright 2015 Oracle Corp.
  • 8.
    Container Advantages • Leveragemature OS • Lightweight • Fewer resources: shared kernel, optional shared text pages • Faster to provision, boot, shutdown • “Zero” overhead: faster, better scalability, more predictable consolidation • Better resource sharing • Faster context switch • Direct path to I/O • More sophisticated “Control Program”: one control point • Better Observability • More flexible access to hardware Copyright 2015 Oracle Corp.
  • 9.
    Container Disadvantages • Homogeneity •Most require all containers to run at the same kernel patch level • Heterogenous guest OS is uncommon • Must sacrifice performance to run heterogeneous guests • Less isolation • More sophisticated Control Program: more code to fail • Slightly greater chance for downtime for multiple workloads • Can be mitigated… Copyright 2015 Oracle Corp.
  • 10.
    The Virtual Renaissance 199920152005 2010 VMware W/S HP nPars FreeBSD jails VMware ESX Virtuozzo Power LPARs Linux VServers Solaris Zones OpenVZ HP vPars Integrity VM HP-UX SRP Sun LDoms HP Dynamic nPars AIX WPars LXC HP-UX System Containers, HP 9000 Containers Solaris Kernel Zones VirtualBox Xen HyperV VMs OSV KVM Solaris “Containers” All dates are approximate, v1.0/stable.Copyright 2015 Oracle Corp.
  • 11.
    The Virtual Renaissance (non-Unix) 199920152005 2010 VMware W/S VMware ESX Virtuozzo Linux VServers OpenVZ LXC VirtualBox Xen HyperV VMs OSV KVM All dates are approximate, v1.0/stable.Copyright 2015 Oracle Corp.
  • 12.
    The Virtual Renaissance (Unix) 199920152005 2010 HP nPars FreeBSD jails Power LPARs Solaris Zones HP vPars Integrity VM HP-UX SRPv2 Sun LDoms HP Dynamic nPars AIX WPars HP-UX System Containers, HP 9000 Containers Solaris Kernel Zones VirtualBox VMs OSV Solaris “Containers” All dates are approximate, v1.0/stable.Copyright 2015 Oracle Corp.
  • 13.
    The “Three” “Laws”of V12N: Oracle Solaris Zones • Initial • Equivalence: very difficult to tell you're in a Zone • Resource Control: controls for CPU, RAM, VM, BW • Efficiency: no software layer… no perf overhead • Later • Security: fine-grained, configurable privileges • Administrative Observability • Global Zone can “see” into all native zones, but they can't see back into GZ, or into each other. Copyright 2015 Oracle Corp.
  • 14.
    Solaris Zones • Privileges •Namespace • File system • Networking • Resource Controls (aka “Solaris Containers” 2007-2011) Copyright 2015 Oracle Corp.
  • 15.
    Solaris Zones • Privileges •Fine-grained abilities (e.g. SYS_TIME, SYS_SMB) • Define the security boundary • Cannot be changed while the zone runs • Configurable • Tighten or loosen security boundary from defaults • Tight for web-facing, loose for well-protected, specialty • Trusted Extensions • DoD-grade features: compartmentalization, etc. Copyright 2015 Oracle Corp.
  • 16.
    Solaris Zones • Namespace •Naming service, users/groups • Network services • Configuration choices Copyright 2015 Oracle Corp.
  • 17.
    Solaris Zones • Filesystem • Entirely separate storage pool, or just a file system • GZ can add other mounts (ZFS, UFS, VxFS, lofs, ...) • Zone can mount or share NFS shares • Zone can use LUNs configured into it • Mandatory Access Control (“Immutable Zones”) • Can choose from 4 levels of hardening • Most secure: can't modify any Solaris files • Great for web-facing environments Copyright 2015 Oracle Corp.
  • 18.
    Solaris Zones • Networking •By default, Zones use Solaris vNICs • Individual routing, firewall config • Solaris network v12n also includes vSwitches, vRouters • Elastic Virtual Switch - spans computers • Plumbing via VLAN or VXLAN (routable) • Can build: • an arbitrary network structure in one Solaris instance • ...with multiple subnets, per-zone routing, firewall rules, NAT • ...and bandwidth controls and load-balancing • Great for prototyping networks • IP, MAC address spoof prevention, ... Copyright 2015 Oracle Corp.
  • 19.
    Solaris Zones • ResourceControls • CPU • Pools: assign a zone to specific cores, strands • CPU Cap: accounting cap on CPU time • FSS: Fair Share Scheduler • RAM Cap, VM cap: accounting cap: RAM, Virtual Memory • Max-Processes cap • Shared-memory cap, ... • Network bandwidth controls Copyright 2015 Oracle Corp.
  • 20.
    Solaris Zones • Non-nativeZones • Solaris 11 Kernel Zones • Separate kernel and patch level, more like a VM • Solaris 10 Zone in Solaris 11 system • System-call translation layer • Takes advantage of underlying Solaris 11 features • Network virtualization, transparent encryption, … Copyright 2015 Oracle Corp.
  • 21.
    The Future: ContainerManagement 2010 2015 Docker OpenStack ? 2005 Ops Center SolarWinds VirtualCenter Joyent Triton Copyright 2015 Oracle Corp.
  • 22.