1.
Papers We Love:
Jails and Zones
CTO
bryan@joyent.com
Bryan Cantrill
@bcantrill

2.
Papers we love: Jails and Zones
• Discussing two important papers that form the foundation of
thinking about OS-based virtualization and containers:
• Jails: Confining the Omnipotent Root by Poul-Henning Kamp
and Robert Watson, presented at SANE 2000
• Solaris Zones: Operating System Support for Consolidating
Commercial Workloads by Dan Price and Andy Tucker,
presented at LISA 2004
• As much as possible, want to let these papers speak for
themselves — and provoke discussion!

3.
Jails: Problem statement

4.
Jails: Prior work

5.
Jails aside: chroot(2)

6.
Jails: Proposed solution

7.
Jails: Advantages

8.
Jails: jail(2)

9.
Jails: Confining the filesystem

10.
Jails: Confining the network

11.
Jails: Implementation

12.
Jails: Network management complexities

13.
Jails: Filesystem management complexities

14.
Jails: User management complexities

15.
Jails: Unintended consequences

16.
Jails: Networking limitations

17.
Jails: Resource management limitations

18.
Jails: Management limitations

19.
Jails: Epilogue
• Jails became easier to manage with jls/jps/ezjail/iocage
• Jails were allowed to have multiple IPv4 addresses
• Some jail-based resource management was added, including
CPU binding and
• System V IPC was virtualized, but remains out-of-tree
• VIMAGE added exclusive IP stacks to jails, but it remains a build-
time option and “is considered experimental”

20.
Zones: Problem statement

21.
Zones: Problem statement detail

22.
Zones: Proposed solution

23.
Zones: Block diagram

24.
Zones: Design principles

25.
Zones: Design principles, cont.

26.
Zones: State model

27.
Zones: Configuration

28.
Zones: Installation

29.
Zones: Application environment

30.
Zones: Virtual platform

31.
Zones: Console

32.
Zones: Process model

33.
Zones: Process model, cont.

34.
Zones: IPC

35.
Zones: System V IPC

36.
Zones: Networking

37.
Zones: Filesystem

38.
Zones: Resource management

39.
Zones: Observability and debugging

40.
Zones: Security experience

41.
Zones: Workloads

42.
Zones: Epilogue
• Crossbow added virtual NICs and exclusive IP stacks — and anti-
spoof allowed exclusive IP stacks to be deployed safely
• Resource management became much more complete, adding
memory capping, CPU capping, I/O throttling
• ZFS revolutionized zone installation/configuration
• With introduction of IPS packaging, Solaris got rid of so-called
“sparse root” zones...
• ...and Joyent added sparse root zones back to SmartOS (thanks
to no IPS and no global zone package management)

43.
Zones: Epilogue, cont.
• Sun added notion of branded zones in 2006, including a nascent
Linux brand (LX) — and then ripped LX out in 2010
• LX brand revived by Joyent in 2014 in SmartOS and completed
(first deployed into production in early 2015)
• Overlay network support added to SmartOS by Joyent, allowing
software-defined VXLAN-based networks in non-global zones

44.
Jails and Zones: Conclusions
• Each of these technologies has served to inspire the other: zones
was explicitly inspired by jails — and the jails networking work
has been explicitly inspired by Crossbow
• These two papers are important because they capture not just the
what, but the why of their respective works
• These technologies were both ahead of their time; it’s invaluable
now to be able to understand their motivations!
• In the words of the late, great Jim Gray: You need to write more!