Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Embedded Fest 2019. Руслан Биловол. Linux Boot: The Big Bang theory

17 views

Published on

Many Linux developers and users know that Bootloader boots Linux Kernel which starts Userspace.
But what is behind that?
Linux Boot process is very similar to the Big Bang which happened in our world: from singularity to the whole beautiful Universe of running Linux processes. What do you think is a black hole equivalent in the Linux? Come to hear about all of that from an engineer who worked on a new processors and boards bringup, Linux boot process modifications and did boot time optimization many times.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Embedded Fest 2019. Руслан Биловол. Linux Boot: The Big Bang theory

  1. 1. ©2016 GlobalLogic Inc. Linux Boot The Big Bang theory Ruslan Bilovol 2019
  2. 2. 2 Who am I? • 10+ years background in Embedded Systems development • Texas Instruments, Marvell, Cisco • OpenSource contributor and enthusiast 0295 E4A8 BEA7 3111 24B8 E490 0A02 52FB 7C55 614B
  3. 3. 3 How Linux is booting? BIOS/UEFI/ ROM Bootloader Linux Kernel Userspace
  4. 4. ©2016 GlobalLogic Inc. Questions?
  5. 5. 5 Questions • Why so many steps? • Why bootloader is needed? • How multiple CPU cores are started? • When Virtual Memory appears in the system? • How first process starts? • What Linux Kernel is doing after starting userspace?
  6. 6. ©2016 GlobalLogic Inc. What is “Linux Boot”?
  7. 7. 7 What is “Linux Boot”? • Boot is short for bootstrap • A self-starting process to init HW and prepare SW • Power on, wait - and universe is ready
  8. 8. 8 How Linux is booting? BIOS/UEFI/ ROM Bootloader Linux Kernel Userspace Initial HW startup Fetch and execute bootloader Low-level HW init Fetch and execute Linux Kernel System initialization Preparation for UserSpace execution Starting processes. System initialization from UserSpace
  9. 9. 9 The Big Bang theory •
  10. 10. ©2016 GlobalLogic Inc. Singularity of hardware
  11. 11. 11 vim Chrome Office VLC Chrome Userspace processes
  12. 12. 12 vim Chrome Office VLC Chrome CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM
  13. 13. 13 vim Chrome Office VLC Chrome CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM
  14. 14. 14 vim Chrome Office VLC Chrome Timer CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM
  15. 15. 15 vim Chrome Office VLC Chrome Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM
  16. 16. 16 vim Chrome Office VLC Chrome Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM MMU
  17. 17. 17 vim Chrome Office VLC Chrome Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM MMU HDD GPU Keybd Mouse Network
  18. 18. 18 vim Chrome Office VLC Chrome Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM MMU Linux Kernel HDD GPU Keybd Mouse Network
  19. 19. 19 Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) RAM MMU HDD GPU Keybd Mouse Network
  20. 20. 20 The Big Bang theory •
  21. 21. ©2016 GlobalLogic Inc. The Big Bang: powering up your machine
  22. 22. 22 I've got the power! • x86 - BIOS/UEFI – RAM detection – Hardware detection/Initialization – Boot sequence • BIOS: loads a 512 byte from the MBR and jumps into it • UEFI: loads UEFI application from a partition on a GPT boot device
  23. 23. 23 I've got the power! • ARM - ROM – Very minimal init (varies) – Load first-stage bootloader (SD, FLASH, UART)
  24. 24. 24 Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) RAM MMU HDD GPU Keybd Mouse Network
  25. 25. 25 The Big Bang theory •
  26. 26. ©2016 GlobalLogic Inc. Bootloader: why is it really needed?
  27. 27. 27 Bootloader • Is a program that does: – Low-level hardware initialization – Fetch Linux Kernel image (disk, network, etc) and put it to the RAM – Prepare system for kernel boot (CPUs, caches, MMU, interrupts) – Jump into first kernel instruction BIOS/UEFI/ ROM Bootloader Linux Kernel Userspace
  28. 28. 28 Bootloaders • GRUB (GRand Unified Bootloader) – x86/x86-64 • LILO (Linux Loader) • U-Boot (“the Universal Boot Loader”) – ARM and other embedded systems
  29. 29. 29 Bootloader • ARM64 bootloader requirements (minimal) – Setup and initialise the RAM – Setup the device tree – Decompress the kernel image (if needed) – Call the kernel image
  30. 30. 30 ARM64 Kernel starting conditions • Following conditions must be met by bootloader: – Quiesce all DMA capable devices – All interrupts disabled (IRQ, FIQ, Debug, SError) – The MMU must be off – Same coherency domain on all CPUs – Primary (boot) CPU jumps directly to the kernel – Secondary CPUs a either disabled or spinning outside of Kernel
  31. 31. 31 Kernel starting conditions Power ON Bootloader Kernel CPU 0 (boot CPU) CPU 1 CPU 2 CPU 3 time
  32. 32. ©2016 GlobalLogic Inc. Dark Ages: The beginning of the Linux Kernel execution
  33. 33. 33 Linux Kernel compression • Saves disk space • Faster to decompress in RAM than load full uncompressed image Kernel Image Type Decompressor vmlinux NA (raw ELF file) Image Not compressed Image.gz Bootloader zImage Kernel (self-decompressing)
  34. 34. 34 Linux Kernel compression (ARM example) vmlinux Image piggy_data piggy.S piggy.o head.o decompress.o misc.o ... vmlinux zImage Raw kernel executable (ELF) Stripped binary vmlinux gzip-ped Image included into piggy.S Combined with decompressor and bootstrapping part Combined kernel image (ELF) Stripped binary vmlinux
  35. 35. 35 The Big Bang theory •
  36. 36. ©2016 GlobalLogic Inc. Black hole in the Linux?
  37. 37. 37 Black hole in the Linux?
  38. 38. 38 Black hole in the Linux? • /dev/null !
  39. 39. ©2016 GlobalLogic Inc. Kernel and Hardware initialization: from darkness to first stars
  40. 40. 40 Linux kernel start vmlinux head.o main.o Architecture-dependent low-level initialization: - check CPU type and capabilities - enable MMU and virtual memory - call start_kernel() from main.o Initialization of all the kernel features (a lot of them!) Rest of the kernel binary jmp
  41. 41. 41 arch/*/kernel/head*[S,c] init/main.c: start_kernel() jmp init/main.c: rest_init() init/main.c: kernel_init() arch/*/kernel arch/*/mm cpu_idle (PID 0) init process (PID 1) Initialization Operation
  42. 42. 42 Linux init/main.c • start_kernel() – local_irq_disable() – boot_cpu_init() – setup_arch() /* architecture-specific initialization */ – mm_init() /* Kernel Memory Management */ – sched_init() /* Scheduler init (doesn’t run at this moment) */ – preempt_disable() – init_IRQ() /* IRQ subsystem initialization */ – tick_init() / init_timers() /* Timer sybsystem init */ – rand_initialize() – time_init() /* Platform timers init */ – local_irq_enable() /* IRQs enabled on local CPU */ – console_init()
  43. 43. 43 Linux init/main.c • rest_init() – pid = kernel_thread(kernel_init, NULL, CLONE_FS); – pid = kernel_thread(kthreadd, NULL, CLONE_FS | CLONE_FILES); – cpu_startup_entry() /* Call into cpu_idle */
  44. 44. 44 Linux init/main.c • kernel_init() – smp_init() – do_initcalls() /* drivers initialization */ – prepare_namespace() /* mount initrd if needed */ – free_initmem() – try to run init process: if (!try_to_run_init_process("/sbin/init") || !try_to_run_init_process("/etc/init") || !try_to_run_init_process("/bin/init") || !try_to_run_init_process("/bin/sh")) return 0;
  45. 45. 45 Linux initrd • Problem – need to mount root filesystem, but driver for it lives in root fs – chicken and egg issue • Solution – bootloader loads small filesystem with needed drivers to RAM – Kernel mounts it as root fs – initrd does all needed initialization – initrd switches root to actual root filesystem pivot_root bootloader kernel initrd /dev/sda2 Load drivers
  46. 46. 46 Linux initrd - typical boot sequence • bootloader loads the kernel and the initial RAM disk • kernel converts initrd into a “normal” RAM disk • root device (initrd) is mounted • /sbin/init is executed • init mounts the “real” root file system • init switches to the “real” root fs using pivot_root system call • init execs the /sbin/init on the new root filesystem – important: PID 1 is preserved • the initrd file system is removed
  47. 47. 47 init idletsk Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM MMU Linux Kernel HDD GPU Keybd Mouse Network
  48. 48. 48 The Big Bang theory •
  49. 49. ©2016 GlobalLogic Inc. Galaxies and planets: Init spawns user Space
  50. 50. 50 Init process • The first process that is executed once the Linux kernel loads • Does system initialization on userspace level • Runs all other processes (bootstraps userspace) • Handles processes whose parent died (“zombie”) Kernel initialization time init (PID 0) User Space process 1 process 2 process 3 process n
  51. 51. 51 Typical Init system tasks • Desktop systems – find init configuration files/scripts – check and mount filesystems – run daemons (background tasks) – setup networking – run display manager • Embedded systems – depends on Embedded system purpose – often similar to desktop, but without graphics
  52. 52. 52 Init process • Popular init systems – SysVinit • survived from UNIX System V days – systemd • modern init system aimed to address the limitations of SysVinit – OpenRC • Gentoo users – busybox • simple, for small/embedded systems
  53. 53. 53 Run levels / Targets SysV Runlevel systemd Target Description 0 poweroff.target Shut down the system. 1, s, single rescue.target Single user mode. 2, 4 multi-user.target User-defined/Site-specific runlevels. By default, identical to 3. 3 multi-user.target Multi-user, non-graphical. Users can usually login via multiple consoles or via the network. 5 graphical.target Multi-user, graphical. Usually has all the services of runlevel 3 plus a graphical login. 6 reboot.target Reboot emergency emergency.target Emergency shell
  54. 54. 54 vim Chrome Office VLC Chrome Timer Interrupt controller CPU0 CPU1 CPU2 CPU3 Caches (L1, L2, L3) Userspace processes RAM MMU Linux Kernel HDD GPU Keybd Mouse Network
  55. 55. 55 The Big Bang theory •
  56. 56. ©2016 GlobalLogic Inc. Questions?
  57. 57. 57 References https://kernel.org https://wiki.archlinux.org
  58. 58. ©2016 GlobalLogic Inc.

×