The document discusses Linux kernel memory management and the ION memory allocator. It provides an overview of kernel memory management concepts like pages, zones, and functions for allocating and freeing memory. It then describes the ION memory allocator which manages memory pools for devices and helps prevent fragmentation. It explains how to allocate and share ION buffers between user space processes and the kernel.

1. Memory Management
Feng Qin
CSE Dept., The Ohio State University

2. Group Discussion
• Applications of Information Flow
Checking Mechanisms?

3. Outline
• Background
– Linux Kernel Memory Management
• ION Memory Management

4. Kernel Memory Management
• Physical memory
• It is not easy:
– Often kernel cannot sleep
– Not easy to handle memory errors
4/25/2012 CSE662: Operating Systems Laboratory 4

5. Pages
• <linux/mm.h>
struct page {
page_flags_t flags;
atomic_t _count;
atomic_t _mapcount;
unsigned long private;
struct address_space *mapping;
pgoff_t index;
struct list_head lru;
void *virtual;
}
4/25/2012 CSE662: Operating Systems Laboratory 5

6. Zones
• Reasons:
– Some devices are capable of performing DMA to
only certain memory addresses
– Some architectures are capable of physically
addressing larger amounts of memory than they
can virtually address
• Zones: ( for x86)
– ZONE_DMA (< 16 MB)
– ZONE_NORMAL (16-896 MB)
– ZONE_HIGHMEM(> 896 MB)
4/25/2012 CSE662: Operating Systems Laboratory 6

7. Getting Memory (I)
• Contiguous physical memory pages:
– struct page *alloc_pages (unsigned int
gfp_mask, unsigned int order)
– unsigned long __get_free_pages(…)
– unsigned long get_zeroed_pages(…)
– void free_pages(…)
– void __free_pages(…)
4/25/2012 CSE662: Operating Systems Laboratory 7

8. Getting Memory (II)
• Contiguous physical memory in bytes:
– void *kmalloc(size_t size, int flags)
– void kfree(const void *ptr)
• Contiguous virtual memory in bytes:
– void *vmalloc(unsigned long size)
– void vfree(void *addr)
– Performance may not be good (TLB threshing)
– E.g. when loading modules
4/25/2012 CSE662: Operating Systems Laboratory 8

9. Getting Memory (III)
• gfp_mask flags:
– GFP_ATOMIC (high priority and must not
sleep)
– GFP_KERNEL (normal allocation and
might sleep)
– GFP_USER (allocate memory for user-space
processes)
– GFP_DMA (allocate memory for device
drivers)
– ….
4/25/2012 CSE662: Operating Systems Laboratory 9

10. High Memory Mappings
• Permanent mappings (might sleep)
– void *kmap(struct page *page)
– void kunmap(struct page *page)
• Temporary mappings (not sleep)
– void *kmap_atomic(…)
– void *kunmap_atomic(…)
4/25/2012 CSE662: Operating Systems Laboratory 10

11. Slab Layer (I)
• Cache objects that are frequently allocated
and deallocated
Cache
Slab
Slab
obj
obj
obj
obj
4/25/2012 CSE662: Operating Systems Laboratory 11

12. Slab Layer (II)
struct slab {
struct list_head list;
unsigned long colouroff;
void *s_mem;
unsigned int inuse;
kmem_bufctl_t free;
};
4/25/2012 CSE662: Operating Systems Laboratory 12

13. Statically Allocating on the
Stack
• Per-process kernel stack
• Keep stack usage to a minimum
– The size is really small ( 4 KB option in 2.6
kernels)
4/25/2012 CSE662: Operating Systems Laboratory 13

14. Per-CPU Allocations
• In modern SMP architectures
• Benefits:
– Reduce locking
– Avoid cache invalidation
• Usage example:
unsigned long my_percpu[NR_CPUS];
cpu = get_cpu();
my_percpu[cpu];
put_cpu();
4/25/2012 CSE662: Operating Systems Laboratory 14

15. ION Memory Allocator
• Manage one or more memory pools, i.e.,
ION heaps
– Each device can have a different set of ION
heaps
• Combat fragmentation
• Serve special hardware needs:
– E.g., GPU, display controller, cameras

16. Providing an ION Heap
struct ion_heap_ops {
int (*allocate) (struct ion_heap *heap,
struct ion_buffer *buffer, unsigned long len,
unsigned long align, unsigned long flags);
void (*free) (struct ion_buffer *buffer);
int (*phys) (struct ion_heap *heap, struct ion_buffer *buffer,
ion_phys_addr_t *addr, size_t *len);
struct scatterlist *(*map_dma) (struct ion_heap *heap,
struct ion_buffer *buffer);
void (*unmap_dma) (struct ion_heap *heap,
struct ion_buffer *buffer);
void * (*map_kernel) (struct ion_heap *heap,
struct ion_buffer *buffer);
void (*unmap_kernel) (struct ion_heap *heap,
struct ion_buffer *buffer);
int (*map_user) (struct ion_heap *heap, struct ion_buffer *buffer,
struct vm_area_struct *vma);
};

17. Default ION driver
• ION_HEAP_TYPE_SYSTEM: memory
allocated via vmalloc_user().
• ION_HEAP_TYPE_SYSTEM_CONTIG:
memory allocated via kzalloc.
• ION_HEAP_TYPE_CARVEOUT: carveout
memory is physically contiguous and set aside
at boot.

18. Using ION From User Space
• Who uses ION?
– User space device access libraries
• Why needs ION?
– Allocate large contiguous media buffers
– Sharing between two processes or processes
and kernel
– E.g., still camera library, when buffer is filled
with data, it will pass the buffer to the kernel
to be processed by JPEG encoder hardware

19. Allocate Buffer from ION
• Require Access permission to device /dev/ion
• Retrieve a file descriptor (one client per
user process)
client_fd = open (“/dev/ion”, O_RDONLY)
• Fill the following data structure
struct ion_allocation_data {
size_t len;
size_t align;
unsigned int flags; // specify which ION heap the buffer is from
(i.e., ION_HEAP_TYPE_CARVEOUT | ION_HEAP_TYPE_CONTIG)
struct ion_handle *handle; // output parameter, not CPU-accessible buffer pointer
}
• Use ioctl system call to interact with ION
int ioctl(int client_fd, ION_IOC_ALLOC, struct ion_allocation_data *allocation_data)

20. ION Buffer Sharing in User
Space
• Convert ion_handle to a file descriptor
int ioctl(int client_fd, ION_IOC_SHARE, struct ion_fd_data *fd_data);
struct ion_fd_data {
struct ion_handle *handle;
int fd;
}
• BINDER IPC mechanism can be used to
send fd to another process for sharing
• To obtain the shared buffer, the second process
needs to retrieve the file descriptor via
client_fd = open (“/dev/ion”, O_RDONLY)

21. Free ION Buffer
• Go through ioctl again
int ioctl(int client_fd, ION_IOC_FREE, struct ion_handle_data *handle_data);
struct ion_handle_data {
struct ion_handle *handle;
}

22. ION Buffers in Kernel
• ION supports multiple clients in the kernel
– One for each driver that uses ION
• A kernel driver calls the following function
to obtain an ION client handle
struct ion_client *ion_client_create(
struct ion_device *dev, // /dev/ion
unsigned int heap_mask, // selects ION heaps
const char *debug_name
)

23. Sharing ION Buffer between
User Space and Kernel (I)
• User process
– Allocates the ION buffer
– Obtains a file descriptor using
ION_IOC_SHARE
– Pass the file descriptor to a kernel driver

24. Sharing ION Buffer between
User Space and Kernel (II)
• Kernel driver
– Converts the file descriptor to an ion_handle
struct ion_handle *ion_import_fd(struct ion_client *client,
int fd_from_user);
// based on the physical address of the buffer
– Some hardware blocks need physically-contiguous
buffers with physical addresses
int ion_phys(struct ion_client *client, struct ion_handle
*handle, ion_phys_addr_t *addr, size_t *len)

25. Source Code Cross References
• Linux kernel
– http://lxr.linux.no/
• Android
– http://androidxref.com/source/xref/