This document outlines the agenda and content for a talk on basics of Linux memory management. The talk will cover physical memory overview, the memory management unit, memory nodes and zones, boot time memory management, and memory allocation techniques including SLOB, SLAB, and SLUB allocators. It will also discuss the userspace memory allocator.

1. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Yet another talk about basics in Linux
Memory
Vasyl Gomonovych
2020

2. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Agenda
I Physical Memory Overview
I Memory Management Unit
I Memory Nodes, Zones, Frames
I Boot time memory management
I Memblock
I Bootmem
I Physical memory allocation
I Slab allocator
I Userspace allocator

3. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Overview

4. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Virtual Memory

5. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Physical Memory Hierarchy

6. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Physical Memory Segmentation

7. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Memory Banks

8. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit

9. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit

10. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit access
I Cp15:c1:c0 -¿ MCR p15, 0, val, c1, c0, 0
I C0 – TCM status Tightly coupled memory
I C1 – control register
I C2 –
I C3 – domain access
I C4 –
I C5 –
I C6 –
I C7 – cache setup, clean, drain write buffer
I C8 –
I C9 – cache operation (victim pointer, cache lock down)
I C10 –
I C11 –
I C12 –
I C13 – current process ID
I C14 –
I C15 – round-robin replacemant

11. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit page table

12. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit tlb

13. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit tlb
I Data TLB
I Instruction TLB

14. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit cache
I Virtually indexed
I Homonyms - the same virtual addresses in different
processes
I Synonyms - different virtual addresses maps to the same
physical
I Physically indexed
I Virtually Indexed Physically Tagged

15. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit cache

16. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit cache structure
I Cache line is transfer unit between cache and RAM
I Make write asynchronous
I Clustering read and write
I Direct map cache
I Way Associative cache
I Fully Associative cache

17. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit cache miss
I Compulsory miss - miss after cache flash
I Capacity miss - cache is full no free lines
I Conflict miss - cache is full no free lines
I Coherence miss - miss triggered by coherence protocol

18. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit cache replacement
I Pseudo LRU
I FIFO
I Random

19. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit mapping

20. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit mapping

21. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit switch
I FCSE – Fast context switch
I Elimination of the cache and TLB flush
I Process ID
I 32MB VM limit for task size
I MVM = VA + (0x2000000 * process ID)
I Task switch change process ID and add entries to
domain access reg

22. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
memory management unit functionality
I Write policy
I Writeback – only to cache write
I Writethrue – write to cache and memory
I Allocation policy (miss case policy)
I Read allocate
I Write allocate - allocate line in cache read and store
I No write allocate - write directly to memory
I Lockdown – exempt from eviction (irq vector, ISR,
global data)
I Logical cache
I Physical cache
I Unified cache
I Split cache
I Cache hit vs cache miss (time penalty)
I Eviction
I Data streaming
I Thrashing
I Direct map vs set associativity

23. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Memory Nodes, Zones, Frames

24. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Boot time memory management

25. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Flat memory model

26. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Discontiguous memory model

27. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Sparse memory model

28. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Memblock

29. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Memblock

30. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Mips memblock

31. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Bootmem

32. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Buddy allocator

33. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLOB

34. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLAB

35. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLAB

36. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLUB

37. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLUB debug

38. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLUB debug

39. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLUB debug

40. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
SLUB debug

41. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
malloc chank

42. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
malloc buddy

43. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
thread local storage descriptor

44. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Software setup
I (1) - architecture defined register point on TCB start
I (2) - first TCB entry is a pointer on Dynamic Thread
Vector DTV
I (3) - TCP Thread Control Block
I (4) - TLS Thread Local Storage
I (5) - Static TLS
I (6) - Dynamic TLS
Static TLS blocks are memory blocks for thread-local
variables for modules or dynamically loaded libraries that load
in memory by dynamic linker during the main program load.
Dynamic TLS blocks are memory blocks for thread-local
variables for modules that load in runtime by dlopen()
syscall.
Static TLS module number 1 is always main program
executable region for own thread-local variables.

45. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
thread local storage

46. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
END
Questions?

47. Yet another talk
about basics in
Linux Memory
Vasyl Gomonovych
Physical Memory
Overview
Memory
Management Unit
Memory Nodes,
Zones, Frames
Boot time memory
management
Physical memory
allocation
SLOB SLAB SLUB
Userspace allocator
END
Links
I What Every Programmer Should Know About Memory
I The Linux Virtual Memory Manager
I Carnegie Mellon Computer Architecture
I ELF Handling For Thread-Local Storage
I elixir.bootlin.com