Linux boot-time
Upcoming SlideShare
Loading in...5
×
 

Linux boot-time

on

  • 1,773 views

 

Statistics

Views

Total Views
1,773
Views on SlideShare
1,744
Embed Views
29

Actions

Likes
1
Downloads
36
Comments
0

3 Embeds 29

https://twitter.com 19
http://eventifier.co 8
http://eventifier.com 2

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Linux boot-time Linux boot-time Presentation Transcript

  • Andrea Righi - andrea@betterlinux.com Ottimizzare i tempi di boot di Linux
  • Andrea Righi - andrea@betterlinux.com Agenda ● Overview ● Case study: Raspberry Pi ● Kernel optimizations ● rootfs optimizations ● Q/A
  • Andrea Righi - andrea@betterlinux.com Overview of boot phases ● Firmware (boot-loader) ● Hardware probing ● Hardware initialization ● Kernel load and decompression ● Kernel execution ● Core init (start_kernel) ● Driver init (initcalls) ● User-space init ● /sbin/init ● RC scripts ● Application start (first user impression) View slide
  • Andrea Righi - andrea@betterlinux.com Generic concepts ● Identify boot time functionalities ● Measure boot time of each functionality ● Remove unnecessary functionalities ● Optimize required functions ● Defer loading of less-important features (modularization - LKM) ● Re-order initialization / parallelization ● Asynchronous initialization View slide
  • Andrea Righi - andrea@betterlinux.com Generic concepts ● Identify boot time functionalities ● Measure boot time of each functionality ● Remove unnecessary functionalities ● Optimize required functions ● Defer loading of less-important features (modularization - LKM) ● Re-order initialization / parallelization ● Asynchronous initialization Premature optimization is the root of all evil. -Donald Knuth
  • Andrea Righi - andrea@betterlinux.com Optimization ● Remove features ● The fastest code is the code you don't run! ● Defer loading of less important features ● Modularization (LKM) ● Parallelization ● async initialization (kernel/async.c) ● Reduce probing delays ● Filesystem tricks
  • Andrea Righi - andrea@betterlinux.com Case study ● Raspberry Pi ● Boot time functionality ● Reponsive ssh login ● Tools and techniques used to improve boot time ● Focusing on cold- boot optimizations
  • Andrea Righi - andrea@betterlinux.com Boot steps ● RPi boot steps: ● bootcode.bin: 1st-stage boot loader from SoC BCM2835 ROM ● bootloader.bin: 2nd-stage boot loader from external SD card ● kernel.img: Linux ● rootfs: external SD card ● application: ssh (dropbear)
  • Andrea Righi - andrea@betterlinux.com Boot steps ● Rpi boot steps: ● bootcode.bin: 1st-stage boot loader from SoC BCM2835 ROM ● bootloader.bin: 2nd-stage boot loader from external SD card ● kernel.img: Linux ● rootfs: external SD card ● application: ssh (dropbear)
  • Andrea Righi - andrea@betterlinux.com Measure boot-time functionality (kernel) ● CONFIG_PRINTK_TIME ● Configure it in “Kernel hacking” section ● Adds timing informations to kernel messages (dmesg) # dmesg ... [ 0.022030] initcall bcm_mbox_init+0x0/0x38 returned 0 after 0 usecs [ 0.022054] calling bcm_power_init+0x0/0xa4 @ 1 [ 0.022065] bcm_power: Broadcom power driver [ 0.022080] bcm_power_open() -> 0 [ 0.022090] bcm_power_request(0, 8) [ 0.522766] bcm_mailbox_read -> 00000080, 0 [ 0.522783] bcm_power_request -> 0 [ 0.522812] initcall bcm_power_init+0x0/0xa4 returned 0 after 488281 usecs
  • Andrea Righi - andrea@betterlinux.com Kernel initcall tracer ● Introduced in Linux 2.6.28 ● Add “initcall_debug” to the kernel boot options ● Allows to record the timings of each initcall in dmesg # dmesg | grep " initcall " | sed "s/ */ /g" | sort -n -t' ' -k8 | tail -5 [ 0.701759] initcall bcm2708_fb_init+0x0/0xc returned 0 after 32518 usecs [ 0.794306] initcall pty_init+0x0/0x3e0 returned 0 after 90313 usecs [ 0.660366] initcall init_kprobes+0x0/0x108 returned 0 after 94523 usecs [ 1.224203] initcall dwc_otg_driver_init+0x0/0xb8 returned 0 after 402667 usecs [ 0.518454] initcall bcm_power_init+0x0/0xac returned 0 after 488281 usecs
  • Andrea Righi - andrea@betterlinux.com Kernel initcall tracer (example) ● dmesg > dmesg.log ● cat dmesg.log | perl scripts/bootgraph.pl > /boot/dmesg.svg
  • Andrea Righi - andrea@betterlinux.com Raspbian ● http://www.raspbian.org ● General-purpose distro based on Debian ● Optimized for the Raspberry Pi ● compilation settings adjusted to produce hard-float code ● 2013-05-25-wheezy-raspbian.img
  • Andrea Righi - andrea@betterlinux.com Raspbian: boot time # dmesg ... [ 3.107163] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:a9:d6:24 [ 5.664780] EXT4-fs (mmcblk0p2): recovery complete [ 5.677326] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null) [ 5.689427] VFS: Mounted root (ext4 filesystem) on device 179:2. [ 5.700523] devtmpfs: mounted [ 5.706028] Freeing init memory: 128K [ 6.257687] init start [ 7.337703] udevd[154]: starting version 175 [ 8.511002] Registered led device: led0 [ 16.041293] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null) [ 16.522927] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null) [ 25.553157] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps, full-duplex, lpa 0xC5E1 [ 28.672178] Adding 102396k swap on /var/swap. Priority:-1 extents:2 across:507900k SS [ 30.436046] init done - Shell after ~6.3sec - SSH after ~30.5sec
  • Andrea Righi - andrea@betterlinux.com sysv-rc-conf ● Run-level configuration for SysV like init scripts ● disable npt ● disable plymouth ● disable rsync ● disable x11-common and lightdm ● disable alsa-utils ● disable swap (dphys-swapfile) ● disable checkfs ● disable ntp
  • Andrea Righi - andrea@betterlinux.com Static IP vs DHCP ● Assign a static IP to the board (save the time to get an IP via DHCP) ● Disable CONFIG_IP_PNP in the kernel .config ● Used to mount rootfs via NFS
  • Andrea Righi - andrea@betterlinux.com Raspbian: boot time after simple optimizations # dmesg ... [ 3.132731] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:a9:d6:24 [ 5.730131] EXT4-fs (mmcblk0p2): recovery complete [ 5.742783] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null) [ 5.754881] VFS: Mounted root (ext4 filesystem) on device 179:2. [ 5.765958] devtmpfs: mounted [ 5.771460] Freeing init memory: 128K [ 6.310549] init start [ 7.389321] udevd[154]: starting version 175 [ 8.536932] calling leds_init+0x0/0x60 [led_class] @ 229 [ 8.537065] initcall leds_init+0x0/0x60 [led_class] returned 0 after 82 usecs [ 8.543802] calling gpio_led_driver_init+0x0/0xc [leds_gpio] @ 229 [ 8.544067] Registered led device: led0 [ 8.545242] initcall gpio_led_driver_init+0x0/0xc [leds_gpio] returned 0 after 1344 usecs [ 14.724169] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null) [ 15.202445] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null) [ 24.493346] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps, full-duplex, lpa 0xC5E1 [ 24.991030] init done About 25 sec to login via ssh, instead of 30 sec (16% faster)
  • Andrea Righi - andrea@betterlinux.com Build a minimal distro from scratch: requirements ● build toolchain (gcc, glibc, binutils) ● git://github.com/raspberrypi/tools.git ● HINT: use hard-float toolchain (arm-bcm2708hardfp-linux- gnueabi) ● Linux (kernel) ● git://github.com/raspberrypi/linux.git ● busybox (rootfs) ● git://busybox.net/busybox.git ● dropbear (ssh server) ● https://matt.ucc.asn.au/dropbear/
  • Andrea Righi - andrea@betterlinux.com Kernel initcalls (before optimization) # dmesg | grep " initcall " | sed "s/ */ /g" | sort -n -t' ' -k8 ... [ 0.810927] initcall iscsi_transport_init+0x0/0x154 returned 0 after 446 usecs [ 1.225990] initcall bcm2835_thermal_driver_init+0x0/0xc returned 0 after 451 usecs [ 1.232472] initcall pm_qos_power_init+0x0/0xac returned 0 after 713 usecs [ 1.230686] initcall sysctl_ipv4_init+0x0/0x98 returned 0 after 755 usecs [ 0.810406] initcall vchiq_init+0x0/0x1dc returned 0 after 1565 usecs [ 1.228797] initcall sdhci_drv_init+0x0/0xc returned 0 after 1829 usecs [ 0.560116] initcall inet_init+0x0/0x260 returned 0 after 2739 usecs [ 0.808747] initcall loop_init+0x0/0x11c returned 0 after 4852 usecs [ 0.803722] initcall brd_init+0x0/0x1c0 returned 0 after 8230 usecs [ 0.540586] initcall genhd_device_init+0x0/0x84 returned 0 after 9765 usecs [ 0.556774] initcall chr_dev_init+0x0/0xd8 returned 0 after 12234 usecs [ 0.538982] initcall param_sysfs_init+0x0/0x1d4 returned 0 after 19531 usecs [ 0.701759] initcall bcm2708_fb_init+0x0/0xc returned 0 after 32518 usecs [ 0.794306] initcall pty_init+0x0/0x3e0 returned 0 after 90313 usecs [ 0.660366] initcall init_kprobes+0x0/0x108 returned 0 after 94523 usecs [ 1.224203] initcall dwc_otg_driver_init+0x0/0xb8 returned 0 after 402667 usecs [ 0.518454] initcall bcm_power_init+0x0/0xac returned 0 after 488281 usecs
  • Andrea Righi - andrea@betterlinux.com Kernel optimizations (initcalls) ● disable kprobes (CONFIG_KPROBES): ~95ms ● reduce the number of PTYs: ~90ms ● CONFIG_LEGACY_PTY_COUNT=256 => CONFIG_LEGACY_PTY_COUNT=1 ● disable framebuffer: ~33ms ● remove loop device: ~5ms
  • Andrea Righi - andrea@betterlinux.com Kernel initcalls (after optimization) # dmesg | grep " initcall " | sed "s/ */ /g" | sort -n -t' ' -k8 ... [ 0.570870] initcall tun_init+0x0/0x8c returned 0 after 259 usecs [ 0.568035] initcall des_generic_mod_init+0x0/0x10 returned 0 after 341 usecs [ 0.975476] initcall bcm2835_cpufreq_module_init+0x0/0xc returned 0 after 358 usecs [ 0.569678] initcall pty_init+0x0/0x194 returned 0 after 375 usecs [ 0.561542] initcall vc_mem_init+0x0/0x1b4 returned 0 after 393 usecs [ 0.974729] initcall bcm2835_thermal_driver_init+0x0/0xc returned 0 after 393 usecs [ 1.018751] initcall deferred_probe_initcall+0x0/0x6c returned 0 after 395 usecs [ 0.561078] initcall bcm2708_gpio_init+0x0/0xc returned 0 after 405 usecs [ 1.018028] initcall pm_qos_power_init+0x0/0x60 returned 0 after 506 usecs [ 0.559402] initcall inet_init+0x0/0x260 returned 0 after 1540 usecs [ 1.021006] initcall net_secret_init+0x0/0x1c returned 0 after 2145 usecs [ 1.024343] initcall initialize_hashrnd+0x0/0x1c returned 0 after 3052 usecs [ 0.557293] initcall chr_dev_init+0x0/0xd8 returned 0 after 10978 usecs [ 0.541305] initcall param_sysfs_init+0x0/0x19c returned 0 after 19531 usecs [ 1.016084] initcall sdhci_drv_init+0x0/0xc returned 0 after 39250 usecs [ 0.973739] initcall dwc_otg_driver_init+0x0/0xc4 returned 0 after 392536 usecs [ 0.522812] initcall bcm_power_init+0x0/0xa4 returned 0 after 488281 usecs
  • Andrea Righi - andrea@betterlinux.com Kernel optimizations (other optimizations) ● preset loops_per_jiffy ● At each boot the kernel calibrates a delay loop (used later by the udelay() function) ● 1 jiffy = time between 2 timer interrupts ● CONFIG_HZ=100 => 250ms!!! ● loop ● disable console output ● remove “console=xxx” and add “quiet” to the kernel boot parameters ● use LZO kernel decompression (CONFIG_KERNEL_LZO) ● LZO is a compression algorithm that is much faster than gzip, at the cost of a slightly degrade compression ratio (+10%) ● reduce kernel size... ● A smaller kernel is faster to load, less code also means smaller working set (good for caches)
  • Andrea Righi - andrea@betterlinux.com Kernel image (before => after)
  • Andrea Righi - andrea@betterlinux.com rootfs ● Generated using busybox and dropbear ● Hints about busybox (reduce fork()s): ● CONFIG_FEATURE_SH_STANDALONE: use applets instead of fork/exec/wait external binaries ● CONFIG_FEATURE_SH_NOFORK: call <applet>_main() directly without spawning another task ● Build everything with -Os ● Use mklibs to strip system libraries (rootfs is mounted to /mnt) ● mklibs -v -D -d lib2 -L /mnt/lib --ldlib lib/ld-linux.so.3 --target=arm-bcm2708- linux-gnueabi /mnt/bin/* /mnt/sbin/* /mnt/usr/sbin/* /mnt/usr/bin/*; rm -rf lib; mv lib2 lib ● After all these stops total rootfs size is ~3.2MB
  • Andrea Righi - andrea@betterlinux.com filesystem optimizations ● Split the filesystem into read-only portion and read/write portion ● Read-only filesystem mounts faster ● Use Squashfs + tmpfs (or aufs
  • Andrea Righi - andrea@betterlinux.com root filesystem: boot time
  • Andrea Righi - andrea@betterlinux.com Kernel image (before => after) 1.5GB 3.2MB
  • Andrea Righi - andrea@betterlinux.com Demo: custom kernel+rootfs boot time ... [ 1.144025] Freeing init memory: 92K [ 1.228550] init start [ 1.249067] waiting eth0 to come up [ 1.345882] usb 1-1: new high-speed USB device number 2 using dwc_otg [ 1.346059] Indeed it is in host mode hprt0 = 00001101 [ 1.556942] hub 1-1:1.0: USB hub found [ 1.557074] hub 1-1:1.0: 3 ports detected [ 1.836029] usb 1-1.1: new high-speed USB device number 3 using dwc_otg [ 1.940066] smsc95xx v1.0.4 [ 1.951858] smsc95xx 1-1.1:1.0 eth0: register 'smsc95xx' at usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:a9:d6:24 [ 2.086695] init done [ 3.406004] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xC5E1 - Shell after ~1.2 sec - SSH after ~3.4 sec
  • Andrea Righi - andrea@betterlinux.com Conclusion ● Why a faster boot? ● Consumer electronics products require very fast boot times (digital camera, mobile phone, etc.) ● Fast recovery upon system failures (crashes) ● You can show to your friends your new awesome board booting in a couple of seconds ;-)
  • Andrea Righi - andrea@betterlinux.com References ● “LPC: Booting Linux in 5 Seconds” - Arjan van de Ven's talk: http://lwn.net/Articles/299483/ ● elinux wiki - Tim's fastboot tools: http://elinux.org/Tims_Fastboot_Tools ● “Update on boot time reduction techniques” - Michael Opdenacker ● Linux: http:.//www.kernel.org ● Busybox: http://busybox.net ● Dropbear: https://matt.ucc.asn.au/dropbear/dropbear.html ● Raspberry Pi: http://www.raspberrypi.org ● http://it.emcelettronica.com/embedded-gnulinux-partendo-da-zero- test-sulla-raspberry-pi
  • Andrea Righi - andrea@betterlinux.com Q/A ● You're very welcome! ● Twitter ● @arighi ● #bem2013