Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
LX Branded Zones
1
Friday, March 27, 15
Topic Outline
2
• Brief description of container mechanisms
• SmartOS Zones
• Branded Zones
• LX Branded Zones
• Demos
• R...
3
OS OSOS
Hypervisor
Hardware
OS OSOS
Hypervisor
Hardware
OS
AppsApps Apps
OS
Hardware
Apps Apps Apps Apps Apps Apps
Conta...
OS Virtualization
4
•Each container sees their own (virtualized) OS
•In reality, only one OS
•From within a container you ...
SmartOS LX-Branded Zones
5
•Execute Linux binaries directly on SmartOS kernel
•Uses zfs datasets for file system
•Uses zone...
LX-branded Zones -
Implementation
•Processes within LX-branded zone use library
interpositioning (done transparently) to h...
Why LX Branded Zones
•Allows you to run docker containers on SmartOS
•Gets rid of problems with running Docker on a
hardwa...
Images
•The data and metadata required to create a new
container.
•Existing images can be “imported”.
•New images can be c...
Creating an LX-Branded Zone -
Import an Image
9
# imgadm sources -a https://updates.joyent.com
Added "imgapi" image source...
Creating an LX Branded Zone -
JSON config file
•File used when creating a new virtual system
•Contains properties of the new...
Creating an LX Branded Zone -
Example JSON config file
11
# cat /var/tmp/lx.json
{
"alias": "lxtest03242015",
"brand": "lx",...
vmadm(1M) - Create the new
machine
12
# vmadm create -f lx.json
Successfully created VM 203e8515-a1fa-4150-ad93-6e3ce1ee3b...
SmartOS Commands within LX
Branded Zones
•SmartOS binaries are available within the zone
•lofi mounted from global zone
•/n...
Some Example Commands
14
# /native/usr/bin/prstat
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
30597 root 25...
Using DTrace in LX Branded
Zone
•Login via zlogin or ssh is slow.
•Internet search for slow login on Ubuntu yields
(among ...
DTrace Example
16
# export PATH=/native/usr/sbin
# dtrace -qn 'BEGIN{printf("PID PPID COMMANDn");} 
proc:::exec-success/pr...
DTrace Example - Continued
17
# run-parts --lsbsysinit /etc/update-motd.d
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0 i6...
DTrace Example - Continued
18
# dtrace -q -n 'proc:::exec-success/progenyof($target)/{printf("%d %d
%sn", pid, ppid, curps...
DTrace Example - Continued
•In a more recent ubuntu image, /etc/update-
motd.d had several files removed, including /90-
up...
DEMO
20
Friday, March 27, 15
References
• Creating an LX image - http://us-east.manta.joyent.com/jperkin/public/lximg/README
• General wiki page on lx ...
Upcoming SlideShare
Loading in …5
×

Lxbrand

929 views

Published on

My talk on LX branded zones at Snow.nl March 26 and 27, 2015.

Published in: Software
  • Be the first to comment

Lxbrand

  1. 1. LX Branded Zones 1 Friday, March 27, 15
  2. 2. Topic Outline 2 • Brief description of container mechanisms • SmartOS Zones • Branded Zones • LX Branded Zones • Demos • References • slides at http://www.slideshare.net/mrbruning/lxbrand Friday, March 27, 15
  3. 3. 3 OS OSOS Hypervisor Hardware OS OSOS Hypervisor Hardware OS AppsApps Apps OS Hardware Apps Apps Apps Apps Apps Apps Container Architectures • Protection via OS per Container and Hypervisor • Visibility within hypervisor is limited • Duplication of effort • Example: xen • Can be done without OS support • Protection via OS per container and host OS • Duplication of effort • Example: qemu/kvm • Fast (no duplication of work) • High visibility of all components • Protection via OS • Requires OS changes • Example: SmartOS Friday, March 27, 15
  4. 4. OS Virtualization 4 •Each container sees their own (virtualized) OS •In reality, only one OS •From within a container you can only “see” (depending on permissions) things within that container •Minimizes code and data path between application and hardware •Problem •Apps must be built for the OS Friday, March 27, 15
  5. 5. SmartOS LX-Branded Zones 5 •Execute Linux binaries directly on SmartOS kernel •Uses zfs datasets for file system •Uses zones for protection •Processes running in a zone can only see what is in the zone •vxlan networking •Allows use of SmartOS tools within the zone •DTrace, mdb, proc tools, zlogin, etc. Friday, March 27, 15
  6. 6. LX-branded Zones - Implementation •Processes within LX-branded zone use library interpositioning (done transparently) to handle system calls •Some system calls are passed directly to the SmartOS system call • read(2), write(2), getpid(2), etc. •System calls not in SmartOS are implemented using library or in SmartOS •Each thread within an lx process has 2 stacks •One for the native stack, and one for the “brand” stack. Also used for signal handling. 6 Friday, March 27, 15
  7. 7. Why LX Branded Zones •Allows you to run docker containers on SmartOS •Gets rid of problems with running Docker on a hardware virtualized environment •Security •Observability •Bare Metal Performance •Start at http://www.joyent.com to read about Triton, our Elastic Container-Native Infrastructure 7 Friday, March 27, 15
  8. 8. Images •The data and metadata required to create a new container. •Existing images can be “imported”. •New images can be created and provisioned. 8 Friday, March 27, 15
  9. 9. Creating an LX-Branded Zone - Import an Image 9 # imgadm sources -a https://updates.joyent.com Added "imgapi" image source "https://updates.joyent.com" # imgadm avail | grep -i lx f7c19252-c998-11e4-be95-3315493f3741 lx-centos-6 20150313 other 2015-03-13T15:52:35Z 818cc79e-ceb3-11e4-99ee-7bc8c674e754 lx-ubuntu-14.04 20150320 other 2015-03-20T03:45:09Z ... # imgadm import 818cc79e-ceb3-11e4-99ee-7bc8c674e754 Importing 818cc79e-ceb3-11e4-99ee-7bc8c674e754 (lx-ubuntu-14.04@20150320) from "https://images.joyent.com” # imgadm list UUID NAME VERSION OS PUBLISHED 818cc79e-ceb3-11e4-99ee-7bc8c674e754 lx-ubuntu-14.04 20150320 other 2015-03-20T03:45:09Z # Friday, March 27, 15
  10. 10. Creating an LX Branded Zone - JSON config file •File used when creating a new virtual system •Contains properties of the new system •image uuid, alias brand, cpu cap and shares, metadata, file system info, network info, etc. •See vmadm(1M)for list of properties 10 Friday, March 27, 15
  11. 11. Creating an LX Branded Zone - Example JSON config file 11 # cat /var/tmp/lx.json { "alias": "lxtest03242015", "brand": "lx", "kernel_version": "3.13.0", "max_physical_memory": 2048, "image_uuid": "818cc79e-ceb3-11e4-99ee-7bc8c674e754", "resolvers": ["8.8.8.8","8.8.4.4"], "nics": [ { "nic_tag": "admin", "ip": "10.88.88.71", "netmask": "255.255.255.0", "gateway": "10.88.88.2" } ], "customer_metadata": { "root_authorized_keys": "ssh-rsa AAAA...” } } # Friday, March 27, 15
  12. 12. vmadm(1M) - Create the new machine 12 # vmadm create -f lx.json Successfully created VM 203e8515-a1fa-4150-ad93-6e3ce1ee3b21 # # zlogin 203e8515-a1fa-4150-ad93-6e3ce1ee3b21 [Connected to zone '203e8515-a1fa-4150-ad93-6e3ce1ee3b21' pts/4] Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0 x86_64) * Documentation: https://help.ubuntu.com/ ... #ifconfig eth0 eth0 Link encap:Ethernet HWaddr 72:af:f4:3c:a0:23 inet addr:10.88.88.71 Mask:255.255.255.0 ... # # ssh root@10.88.88.71 <-- from another host [Connected to zone '203e8515-a1fa-4150-ad93-6e3ce1ee3b21' pts/4] Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0 i686) ... # Friday, March 27, 15
  13. 13. SmartOS Commands within LX Branded Zones •SmartOS binaries are available within the zone •lofi mounted from global zone •/native/usr •/native/sbin •/native/lib •Others can be added •Apps in LX zone should look and act like apps on linux •Subset of Linux /proc •/sys exists but is empty 13 Friday, March 27, 15
  14. 14. Some Example Commands 14 # /native/usr/bin/prstat PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 30597 root 25M 3796K sleep 59 0 0:00:00 0.0% top/1 13593 root 18M 2408K sleep 20 0 0:00:00 0.0% upstart- socket-/1 30389 root 108M 7372K sleep 59 0 0:00:00 0.0% sshd/1 ... Total: 17 processes, 23 lwps, load averages: 0.10, 0.03, 0.01 # top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 30597 root 59 0 26076 3796 0 R 0.2 0.1 0:00.06 top 30564 root 59 0 4220 3448 0 S 0.1 0.1 0:00.02 prstat 30389 root 59 0 110912 7372 0 S 0.0 0.2 0:00.06 sshd ... top - 23:11:33 up 2 days, 1:08, 2 users, load average: 0.11, 0.02, 0.00 Tasks: 16 total, 1 running, 15 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.6 us, 7.9 sy, 0.0 ni, 89.3 id, 0.0 wa, 0.2 hi, 0.0 si, 0.0 st KiB Mem: 4194304 total, 45480 used, 4148824 free, 0 buffers KiB Swap: 8388608 total, 24632 used, 8363976 free. 0 cached Mem Friday, March 27, 15
  15. 15. Using DTrace in LX Branded Zone •Login via zlogin or ssh is slow. •Internet search for slow login on Ubuntu yields (among many others) •http://askubuntu.com/questions/11538/long-wait- time-on-login •Suggested solution is to comment out pam_motd lines in /etc/pam.d/sshd and /etc/pam.d/ login •We’ll use DTrace within the LX zone to come up with a solution •Note that we shall not “root cause” the problem (but we could) 15 Friday, March 27, 15
  16. 16. DTrace Example 16 # export PATH=/native/usr/sbin # dtrace -qn 'BEGIN{printf("PID PPID COMMANDn");} proc:::exec-success/progenyof($target)/ {printf("%d %d %sn", pid, ppid, curpsinfo->pr_psargs);}' -p `pgrep sshd` PID PPID COMMAND 4483 2824 /usr/sbin/sshd -D -R ... 4488 4487 run-parts --lsbsysinit /etc/update-motd.d 4489 4488 /bin/sh /etc/update-motd.d/00-header ... 4493 4488 /bin/sh /etc/update-motd.d/10-help-text 4496 4488 /bin/sh /etc/update-motd.d/50-landscape-sysinfo 4497 4496 grep -c ^processor /proc/cpuinfo ... 4773 4753 /bin/sh -e /usr/lib/update-notifier/update-motd-updates- available ... 4516 4488 /bin/sh /etc/update-motd.d/98-fsck-at-reboot 4516 4488 /bin/sh /usr/lib/update-notifier/update-motd-fsck-at-reboot 4517 4516 stat -c %Y /var/lib/update-notifier/fsck-at-reboot ... 4535 4523 mesg n ^C Friday, March 27, 15
  17. 17. DTrace Example - Continued 17 # run-parts --lsbsysinit /etc/update-motd.d Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0 i686) * Documentation: https://help.ubuntu.com/ System information as of Fri Mar 20 03:08:11 MDT 2015 System load: 0.0 Memory usage: 0% Users logged in: 1 Usage of /home: unknown Swap usage: 1% => There were exceptions while processing one or more plugins. See /var/log/landscape/sysinfo.log for more information. Graph this data and manage this system at: https://landscape.canonical.com/ pause occurs here... 154 packages can be updated. 78 updates are security updates. Friday, March 27, 15
  18. 18. DTrace Example - Continued 18 # dtrace -q -n 'proc:::exec-success/progenyof($target)/{printf("%d %d %sn", pid, ppid, curpsinfo->pr_psargs);}' -c "run-parts --lsbsysinit / etc/update-motd.d" Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0 i686) ... Graph this data and manage this system at: https://landscape.canonical.com/ 4923 4921 /bin/sh4924 4923 uname -o ... 4942 4921 /bin/sh /etc/update-motd.d/90-updates-available 4942 4921 /bin/sh -e /usr/lib/update-notifier/update-motd-updates- available ... 154 packages can be updated. 78 updates are security updates. Friday, March 27, 15
  19. 19. DTrace Example - Continued •In a more recent ubuntu image, /etc/update- motd.d had several files removed, including /90- updates-available. Problem is gone. 19 # /bin/sh -e /usr/lib/update-notifier/update-motd-updates-available long pause here... 154 packages can be updated. 78 updates are security updates. # Friday, March 27, 15
  20. 20. DEMO 20 Friday, March 27, 15
  21. 21. References • Creating an LX image - http://us-east.manta.joyent.com/jperkin/public/lximg/README • General wiki page on lx branded zones - https://wiki.smartos.org/display/DOC/LX+Branded+Zones • The source. A mid-level description of how it all works - /smartos-live/projects/illumos/usr/src/lib/brand/lx/ lx_brand/common/lx_brand.c has a large comment explaining a lot of the internals • http://www.slideshare.net/bcantrill/illumos-lx - Some history and other details • https://www.joyent.com/blog/container-service-preview - instructions for docker setup • Thanks to Snow.nl for inviting me to speak. 21 Friday, March 27, 15

×