Getting the Linux kernel running on a new processor architecture is a difficult process. Worse still, there is not much documentation available describing the porting process. After spending countless hours becoming almost fluent in many of the supported architectures, I discovered that a well-defined skeleton shared by the majority of ports exists. Such a skeleton can logically be split into two parts that intersect a great deal. The first part is the boot code, meaning the architecture-specific code that is executed from the moment the kernel takes over from the bootloader until init is finally executed. The second part concerns the architecture-specific code that is regularly executed once the booting phase has been completed and the kernel is running normally. This second part includes starting new threads, dealing with hardware interrupts or software exceptions, copying data from/to user applications, serving system calls, and so on. In this talk I will provide an overview of the procedure, or at least one possible procedure, that can be followed when porting the Linux kernel to a new processor architecture. Joël Porquet – Joël was a post-doc at Pierre and Marie Curie University (UPMC) where he ported Linux to TSAR, an academic processor. He is now looking for new adventures.

1. Porting Linux to a new processor architecture
Kernel Recipes 2015
Joël Porquet
September 30th, 2015

2. Background
Research engineer at LIP6 from 2013 to 2015
TSAR
EU-funded project
Massively parallel architecture
Shared and hardware-maintained coherent memory
cluster
(n,n)
cluster
(0,0)
MIPS32
L1 cache
+ MMU
Memory
Cache
MIPS32
L1 cache
+ MMU
MIPS32
L1 cache
+ MMU
MIPS32
L1 cache
+ MMU
Local Crossbar
External
RAM
INT/timer
controller
DMA
controller
Joël Porquet Kernel Recipes 2015 September 30th, 2015 2 / 14

3. Is a new port necessary?*
New board with already-supported processor
New processor within an existing, already supported processor family
New processor architecture
$ mkdir arch/tsar
*https://lwn.net/Articles/597351/
Joël Porquet Kernel Recipes 2015 September 30th, 2015 3 / 14

4. How to start?
Two-step process
1 Minimal set of files that define a minimal set of symbols
2 Gradual implementation of the boot functions
$ ls -l arch/tsar/
configs/
tsar_defconfig
include/
asm/
uapi/asm/
kernel/
vmlinux.lds.S
lib/
mm/
Kconfig
Makefile
Joël Porquet Kernel Recipes 2015 September 30th, 2015 4 / 14

5. The boot sequence
kernel_entry*
start_kernel
setup_arch*
trap_init*
mm_init
mem_init*
init_IRQ*
time_init*
rest_init
kernel_thread(kernel_init)
kernel_thread(kthreadd)
cpu_startup_entry
Joël Porquet Kernel Recipes 2015 September 30th, 2015 5 / 14

6. Early assembly boot code
kernel_entry()
resets the processor to a default state
clears the bss segment
saves the bootloader argument(s) (e.g. device tree)
initializes the first page table
maps the kernel image
enables the virtual memory and jumps into the virtual address space
sets up the stack register (and optionally the current thread info register)
jumps to start_kernel()
Joël Porquet Kernel Recipes 2015 September 30th, 2015 6 / 14

7. setup_arch()
Scans the flattened device tree, discovers the physical memory banks and
registers them into the memblock layer
Parses the early arguments (e.g. early_printk)
Configures memblock and maps the low memory
Memory zones (ZONE_DMA, ZONE_NORMAL, ZONE_HIGHMEM)
Physical
Memory
Virtual
Memory
user space kernel space
Direct
Mapping vmalloc
0GiB 3GiB 4GiB
lowmem highmem
pkmap
Joël Porquet Kernel Recipes 2015 September 30th, 2015 7 / 14

8. trap_init()
Exception vector
The exception vector acts as a dispatcher:
mfc0 k1, CP0_CAUSE
andi k1, k1, 0x7c
lw k0, exception_handlers(k1)
jr k0
Configures the processor to use this exception vector
Initializes exception_handlers with the sub-handlers (handle_int,
handle_bp, etc.)
Joël Porquet Kernel Recipes 2015 September 30th, 2015 8 / 14

9. Trap infrastructure
ENTRY(handle_int)
SAVE_ALL
CLI
move a0, sp
la ra, ret_from_intr
j do_IRQ
ENDPROC(handle_int)
ENTRY(handle_bp)
SAVE_ALL
STI
move a0, sp
la ra, ret_from_exception
j do_bp
ENDPROC(handle_bp)
/* CLI: switch to pure kernel mode and disable interruptions */
/* STI: switch to pure kernel mode and enable interruptions */
do_* are C functions:
void do_bp(struct pt_regs *regs)
{
die_if_kernel("do_bp in kernel",
regs);
force_sig(SIGTRAP, current);
}
Joël Porquet Kernel Recipes 2015 September 30th, 2015 9 / 14

10. mem_init()
Releases the free memory from memblock to the buddy allocator (aka the
page allocator)
Memory: 257916k/262144k available (1412k kernel code, 4228k reserved
267k data, 84k bss, 169k init, 0k highmem)
Virtual kernel memory layout:
vmalloc : 0xd0800000 - 0xfffff000 ( 759 MB)
lowmem : 0xc0000000 - 0xd0000000 ( 256 MB)
.init : 0xc01a5000 - 0xc01ba000 ( 84 kB)
.data : 0xc01621f8 - 0xc01a4fe0 ( 267 kB)
.text : 0xc00010c0 - 0xc01621f8 (1412 kB)
Joël Porquet Kernel Recipes 2015 September 30th, 2015 10 / 14

11. init_IRQ()
Scans device tree and finds all the nodes identified as interrupt controllers.
icu: xicu {
compatible = "soclib,vci_xicu";
interrupt-controller;
#interrupt-cells = <1>;
reg = <0x0 0xf0000000 0x1000>;
};
→ First device driver!
Joël Porquet Kernel Recipes 2015 September 30th, 2015 11 / 14

12. time_init()
Parses clock provider nodes
clocks {
freq: frequency@25MHz {
#clock-cells = <0>;
compatible = "fixed-clock";
clock-frequency = <25000000>;
};
};
Parses clocksource nodes
Clock-source device (monotonic counter)
Clock-event device (counts periods of time and raises interrupts)
→ Second device driver!
Joël Porquet Kernel Recipes 2015 September 30th, 2015 12 / 14

13. To init
Process management
Setting up the stack for new threads
Switching between threads (switch_to())
Page fault handler
Catching memory faults
System calls
List of system calls
Enhancement of the interrupt and exception handler
Signal management
Execution of the signal handlers
User-space memory access
Setting up the exception table
Joël Porquet Kernel Recipes 2015 September 30th, 2015 13 / 14

14. Conclusion
After the initial port
Push it upstream if possible
Keep it up-to-date
Enhance it: SMP, NUMA, drivers, etc.
Bedtime reading
Series of articles on LWN.net
The basics: https://lwn.net/Articles/654783/
The early code: https://lwn.net/Articles/656286/
To the finish line: https://lwn.net/Articles/657939/
Joël Porquet Kernel Recipes 2015 September 30th, 2015 14 / 14