Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction To Linux Kernel Modules

Some basic knowledges required for beginners in writing linux kernel module - with a description of linux source tree, so that the idea of where and how develops. The working of insmod and rmmod commands are described also.

  • Login to see the comments

Introduction To Linux Kernel Modules

  1. 1. Introduction To Kernel Modules - Dibyajyoti Ghosh
  2. 2. Introduction What is kernel Kernel Space and User Space Kernel Categories Linux Kernel Source Build your own Kernel Kernel Files Kernel Modules Kernel Module Build System Few Special Files Use Cases Kernel Module Vs Kernel Built-in Kernel Module Vs User Application How “insmod” works How “rmmod” works Outline
  3. 3. Introduction Let us start our discussion with a diagrammatic representation of a Linux system User Space Applications Hardware Operating System Kernel Kernel modules
  4. 4. What is Kernel What is kernel? - Central core of an Operating System - Kernel is loaded first during booting and stays till the system is up. So image size should be minimum. - Usually loaded into a protected memory area – Kernel Space. - Kernel and BIOS are completely separate entity.
  5. 5. What is Kernel (Contd.) Main Roles of kernel? - Kernel executes jobs or handles interrupts etc. in Kernel Space. - Kernel provides a set of portable, architecture and hardware independent Kernel APIs to allow user space applications to use the hardware resources. - User Space applications request for basic services (Memory management; Process management; I/O management etc.) through system calls – Services provided by kernel. - Kernel handles concurrent access and usage of hardware resources.
  6. 6. Kernel Space and User Space  Kernel codes [including kernel modules] execute in a separate address space [protected from being overwritten by external programs] with super-user privilege : This environment [Address space + privilege] is called Kernel Space  User applications run in another separate address space with lowest privilege mode : This environment is called User Space  Linux system can switch from User Space to Kernel Space, whenever an application issues a system call, or an hardware interrupt suspends the application process  Kernel code executing a system call executes in process context; whereas the code handling interrupts [Interrupt Handlers or Interrupt Service Routines] works asynchronously in Interrupt context.
  7. 7. Kernel Categories Micro kernel: - Provides minimal services, such as defining Memory Address Space, IPC and CPU management - All other services such as Hardware management etc. are implemented separately as User Space Processes. - Examples: AIX, Mac OS X, MINIX etc. Monolithic kernel: - Contains all the core functions of OS and device drivers - Some can load modules dynamically to extend kernel features on demand - Examples: Linux, FreeBSD etc. Hybrid kernel: - Similar to micro kernel, except that they include additional code in kernel space so that such code can run more swiftly compared to if those code were made to run from User Space. - Examples: Windows etc. Exo kernel: - Still experimental
  8. 8. Linux Kernel Source arch/<ARCH>: - Architecture specific code block/: - Block layer core COPYING: - Linux copying conditions (GNU GPL). CREDITS: - Linux main contributors crypto: - Cryptographic libraries Documentation/: - Kernel documentation. Don't miss it! drivers/: - All device drivers except sound ones (usb, pci...) firmware/: - Legacy: firmware images extracted from old drivers
  9. 9. Linux Kernel Source (Contd) fs/: - Filesystems (fs/ext3/, etc.) include/: - Kernel headers include/linux/: - Linux kernel core headers include/uapi/: - User space API headers init/: - Linux initialization (including main.c) ipc/: - Code used for process communication Kbuild: - Part of the kernel build system
  10. 10. Linux Kernel Source (Contd) Kconfig: - Top level description file for conguration parameters kernel/: - Linux kernel core (very small!) lib/: - Misc library routines (zlib, crc32...) MAINTAINERS: - Maintainers of each kernel part. Very useful! Makefile: - Top Linux Makefile (sets arch and version) mm/: - I Memory management code (small too!) net/: - Network support code (not drivers) README: - Overview and building instructions
  11. 11. Linux Kernel Source (Contd) REPORTING­BUGS: - Bug report instructions samples/: - Sample code (markers, kprobes, kobjects...) scripts/: - Scripts for internal or external use security/: - Security model implementations (SELinux...) sound/: - Sound support code and drivers tools/: - Code for various user space tools (mostly C) usr/: - Code to generate an initramfs cpio archive virt/: - Virtualization support (KVM)
  12. 12. Build your own Kernel [root@asl­host169]# make defconfig /** Creates a default .config file for kernel configuration, based on underlying system */ [root@asl­host169]# make menuconfig /** Opens a text based GUI, where we can custom-configure the kernel as per our requirement. The previous .config file is kept as backup with name .config.old, and a new .config file is created along with the modifications. Put a [*] against “Enable loadable module support” */ [root@asl­host169]# make bzImage modules modules_install  install /** Builds the kernel and its modules; Prepares various files like vmlinux, vmlinux.bin, vmlinuz, zImage, bzImage,, initrd.img etc. */
  13. 13. Kernel Files vmlinux:Binary image of Linux kernel in a Statically Linked Executable File Format. vmlinux.bin: Same as vmlinux, but in a bootable raw binary file format. All symbols and relocation information is discarded.Binary image of Linux kernel in a Statically Linked Executable File Format. Generated from vmlinux by objcopy ­O binary vmlinux  vmlinux.bin. vmlinuz:Binary compressed [with zlib/LZMA/bzip2] vmlinux file. During bootup, it is decompressed to get the boot image.
  14. 14. Kernel Files (Contd.) zImage:Prepared with make zImage. This is an old format for small kernels. bzImage:Prepared with make bzImage. This big zImage was created while the kernel grew bigger enough, and accordingly the size of the binary image. kernel symbol table. initrd.img:A small file that does some initiations, and extracts and executes the actual kernel file.
  15. 15. Kernel Modules - Linux Kernel Module (LKM) are nothing but a piece of kernel code, that can be loaded as and when required to the running kernel. However, kernel must should “Enable loadable module support” [CONFIG_MODULES=y] for this. - The prototype for the init and exit function of a module: init function: static int __init <init_fn_name>(void) exit function: static void __exit <exit_fn_name>(void) - The __init and __exit Macros: __init macro: causes the init function to be discarded and its memory freed once the init function finishes for built-in codes, but not loadable modules. __exit macro: causes the omission of the function when the module is built into the kernel, and has no effect for loadable modules. These macros are defined in linux/init.h
  16. 16. Kernel Modules (Contd.) Fine, it’s time for an example: Step1: Write a HelloWorld module #include <linux/init.h> #include <linux/module.h> MODULE_LICENSE("Dual BSD/GPL"); static int hello_init(void) {         printk(KERN_ALERT "Hello, worldn");         return 0; } static void hello_exit(void) {         printk(KERN_ALERT "Goodbye, cruel worldn"); } module_init(hello_init); module_exit(hello_exit);
  17. 17. Kernel Modules (Contd.) Step2: Prepare a Makefile to build it to HelloWorld.ko kernel object KERNELDIR=/lib/modules/$(shell uname ­r)/build obj­m += HelloWorld.o all:       make ­C $(KERNELDIR) M=$(PWD) modules clean:       make ­C $(KERNELDIR) M=$(PWD) clean Step3: Keep the HelloWorld.c and the Makefile in same directory, and issue make [root@asl­host169 my_module]# make make[1]: Entering directory `/home/kernel/linux­3.19'   CC [M]  /home/dibyajyoti/Study_mats/my_module/HelloWorld.o   Building modules, stage 2.   MODPOST 1 modules   CC       /home/dibyajyoti/Study_mats/my_module/HelloWorld.mod.o   LD [M]  /home/dibyajyoti/Study_mats/my_module/HelloWorld.ko make[1]: Leaving directory `/home/kernel/linux­3.19'
  18. 18. Kernel Modules (Contd.) Step4: Load the module [root@asl­host169 my_module]# insmod HelloWorld.ko [root@asl­host169 my_module]# dmesg ­c [706754.319134] Hello, world [root@asl­host169 my_module]# Step5: Unload the module [root@asl­host169 my_module]# rmmod HelloWorld [root@asl­host169 my_module]# dmesg ­c [706891.948856] Goodbye, cruel world [root@asl­host169 my_module]# Step6: Bingo! The journey of our module is end :-)
  19. 19. Kernel Module Build System Ref: <Kernel_Home>/Documentation/kbuild/modules.txt - "kbuild" is the build system used by Linux kernel (Our example uses it) - Modules' build system should comply with this for easy compatibility to changes in build infrastructure and pick right flags to gcc - Functionality for both in-tree and out-of-tree module building are provided - External modules are supplied with Makefiles to hide complexity - The command to build an external module is: $ make -C <path_to_kernel_src> M=$PWD - To build against the running kernel use: $ make -C /lib/modules/`uname -r`/build M=$PWD - Then to install the module(s) just built, add the target "modules_install" to the command: $ make -C /lib/modules/`uname -r`/build M=$PWD modules_install
  20. 20. Kernel Module Build System (Contd.) - make -C $KDIR M=$PWD ($KDIR refers to the path of the kernel source directory.) - “-C $KDIR” The directory where the kernel source is located. "make" will actually change to the specified directory when executing and will change back when finished. - “M=$PWD” Informs kbuild that an external module is being built. The value given to "M" is the absolute path of the directory where the external module (kbuild file) is located.
  21. 21. Kernel Module Build System (Contd.) - When building an external module, make line looks like (generally): make -C $KDIR M=$PWD [target] “[target]” is optional, and can have following values: - “modules” Default target for external modules. It has the same functionality as if no target was specified. - “modules_install” Installs the external module(s). Default location is /lib/modules/<kernel_release>/extra/, but a prefix may be added with INSTALL_MOD_PATH - “clean” Removes all generated files in the module directory only. - “help” Lists the available targets for external modules.
  22. 22. Kernel Module Build System (Contd.) - “obj-<y/m> := <module_name>.o” kbuild system will build <module_name>.o from <module_name>.c, and, after linking, will result in the kernel module <module_name>.ko. The above line can be put in either a "Kbuild" file or a "Makefile. NOTE: Many a times, we may encounter lines like: “obj-$(CONFIG_BT) += bluetooth/” Means, value of the tri-state “CONFIG_BT” ['m' if module; 'y' if built- in; 'n/<NULL>' if not defined] is appended to make the line obj-m / obj-y / not-defined correspondingly.
  23. 23. Kernel Module Build System (Contd.) - When the module is built from multiple sources, an additional line is needed listing the files: “<module_name>-objs := <src1>.o <src2>.o ...”
  24. 24. Few Special Files Once the module is built, many files are generated, like: - <module_name>.ko : Actual kernel module file to “insmod”. “ko” Stands for Kernel Object. - Module.symvers : When your module “exports” symbols using EXPORT_SYMBOL or its variants, those symbols are listed here along with the EXPORTer module name - modules.order : This file tells the modules should be loaded in which order [Top-down ordering]
  25. 25. Use case I built kernel 3.3 with mmc_core as module and kernel 3.9 with mmc_core as built-in. Result: The Module.symvers of both contain the symbols. However, of 3.9.0 contains symbols from mmc_core sub- section, while that of 3.3.0 does not contain the symbols. - The file /proc/kallsyms in proc filesystem is the dynamic list of all the kernel symbols
  26. 26. Use case Most of us probably faced: insmod: ERROR: could not insert module <module_name>.ko: Unknown symbol in module - The above occurs for modules, but never originates for kernel built-in. Reason: - Kernel built-in are built during kernel compilation phase, and symbols are resolved when preparing the kernel binary image. At this phase, all the symbol references are resolved - Kernel modules are external agent, that refers to kernel symbols. When we call insmod utility to load the module, first the symbols are resolved. During this phase, if the kernel symbol is not available in /proc/kallsyms file, we face the above error (irritation!).
  27. 27. Kernel Modules Vs. Kernel built-in Similarity: - Both seats and executes in kernel space completely - Both has direct access to whole kernel - Once loaded, both have equal rights and responsibilities Dissimilarity: - Unlike module, kernel built-in is part of the kernel binary image - kernel built-in can access the whole source [if not static function]. But modules can access kernel code only if they are any of macro / inline fn or symbol exported by kernel using EXPORT_SYMBOL or its variants - Kernel built-in has lesser memory footprint compared to module - Kernel built-in usually outperforms kernel modules
  28. 28. Kernel Modules Vs. User Applications Full of Dissimilarities: - Unlike Applications, Kernel modules do not define a main() function - Kernel modules link only to kernel; Aplications link to libraries - Kernel modules use different set of header files than user applications - Kernel modules should avoid global variables, as those must be unique kernel-wide. - Kernel module may be hardware specific; applications are generally not - Kernel modules can be dynamically loaded. - Kernel module runs in kernel space; application runs in user space - Kernel modules typically are not sequential in nature. - Kernel modules typically can be interrupted. - Faults may be fatal to system for kernel codes.
  29. 29. How “insmod” works Steps in brief: - Allocates a buffer, and scales it up if required - Opens the <module_name>.ko file [O_RDONLY mode], and copies [read] the entire file to the userspace buffer - Now it issues init_module system call; C Compiler supported ABI assigns a unique number [128 for init_module] to it - eventually it maps to kernel call sys_init_module Ref: File: arch/x86/include/generated/uapi/asm/unistd_32.h #define __NR_init_module 128 File: arch/x86/include/generated/asm/syscalls_32.h __SYSCALL_I386(128, sys_init_module, sys_init_module) - However, sys_init_module code does not directly find any. It is defined by using the following macro [defined in kernel/module.c] SYSCALL_DEFINE3(init_module, void __user *, umod, unsigned long, len, const char __user *, uargs);
  30. 30. How “insmod” works (Contd.) - This call does the following major steps to actually load the module - copy_module_from_user() : Allocates memory in kernel space [vmalloc()]; & copies the userspace buffer to kernel space [copy_from_user()] - load_module() : Does the following 1. various sanity checks [elf header etc.] 2. some basic setup [refcnt; MODINFO_ATTR field etc] (setup_modinfo()) 3. some preparation for relocation 4. copies the arguments 5. set module state as MODULE_STATE_COMING (complete_formation()) 6. setup of link to sysfs and 7. call do_init_module() : This actually calls the module initialization function (<fn>) that is specified by our module using module_init(<fn>). Additionally, the module state is set to MODULE_STATE_LIVE
  31. 31. How “rmmod” works Steps in brief: - It determines the module name and checks if it is in use presently - Now it issues delete_module system call; C Compiler supported ABI assigns a unique number [129 for delete_module] to it - eventually it maps to kernel call sys_delete_module Ref: File: arch/x86/include/generated/uapi/asm/unistd_32.h #define __NR_delete_module 129 File: arch/x86/include/generated/asm/syscalls_32.h __SYSCALL_I386(129, sys_delete_module, sys_delete_module) - However, sys_delete_module code does not directly find any. It is defined by using the following macro [defined in kernel/module.c] SYSCALL_DEFINE2(delete_module, const char __user *, name_user, unsigned int, flags);
  32. 32. How “rmmod” works (Contd.) - This call does the following major steps to actually load the module -Copies the module name from userspace - Traverse the list of modules and find the specified module by name - Checks if other modules are dependent on it - Synchronizes all asynchronous function calls. This function waits until all asynchronous functiona calls are done - free_module(): Frees up a module, its memory, removes from list etc. 1. unlinks the sys file system from the module 2. set module state tp MODULE_STATE_UNFORMED 3. Removes dynamic debug info. 4. Arch specific cleanup 5. Clear the unload stuff from module 6. Frees up any allocated parameters 7. Some more free up are done; Frees up the core containing the module structure 8. Frees up module memory – the kernel space buffer allocated during load