Virtual memory 20070222-en


Published on

This is an old presentation salvaged from archive.
This is English translated version. Japanese version is here.

Published in: Technology

Virtual memory 20070222-en

  1. 1. 1Introduction to Linux,for Embedded EngineersTutorial on Virtual MemoryFeb. 22, 2007Tetsuyuki KobayashiAplix Corporation[translated by ikoma]
  2. 2. 2Target Audience of this Presentation• People who have been engaged inprojects on embedded devices, andwho are now using Linux as operatingsystem
  3. 3. 3Goals of this Presentation• To understand the mechanism of virtualmemory in Linux, and to make use of it forthe current project– Although programs work withoutunderstanding the mechanism, it is importantto understand the mechanism to extractsufficient performance
  4. 4. 4Basic Concepts, First of All• Virtual ..., Logical ...– Virtual addresses, logical devices, logicalsectors, virtual machines– To handle as if it is ...• Real ..., Physical ...– Real addresses, physical devices, phisicalsectors– Itself, as it is
  5. 5. 5Virtualization: As if ... but ...• As if it is large, but it actually small• As if it is flat, but it actually uneven• As if there are many, but there is actually one• As if exclusively usable, actually sharedVirtualization is magic to hide complexity orindividual depedency;as it is magic, there is a trick= Mapping between the real and the virtualTranslating so that it looks as if ...
  6. 6. 6Cost of Virtualization• We do virtualize as its merits aregreater than its demerts, butvirtualization does not always meanpositive results
  7. 7. 7Physical Memory and VirtualMemory• Most of ordinary embedded deviceprojects so far have handled only physicalmemory• Recently, as the size of embeddedsystems grow, PC-oriented OSes such asLinux and Windows CE are getting widelyused; these operating systems providevirtual memory systems• This presentation explains Linux
  8. 8. 8Physical Memory• Single memory space• As each device isimplemented with differentaddresses for ROM, RAMand I/O, programmersshould code accordinglyROMRAMI/OI/OI/O
  9. 9. 9Virtual Memory• Merits– User programs do not depend on actual memory map(implementation address, implementation size) anymore– Can use non-contiguous physical memory fragmentsas contiguous virtual memory– Memory protection: Can prevent irrelevant memoryfrom being destroyed by bugs• Introducing new concepts– Address translation– Multiple memory spaces– Demand paging
  10. 10. 10Conceptual Schema of Virtual MemorySource: WikipediaSwapping AreaPhysical MemorySpaceVirtual Address Space
  11. 11. 11Address TranslationROMRAMI/OI/OI/Oaddress xPhysical Addr SpaceVirtual Add Spaceaddress yMMULinux kernelcontrols
  12. 12. 12Virtual Memory is only in CPUROMRAMI/OI/OI/Oaddress xPhysical Addr SpaceVirtual Addr Spaceaddress yMMUCPUOnly physical addresses come out of CPU onto address bus.Virtual addresses can not be observed with the logic analyzer
  13. 13. 13User Program Handlesonly Virtual AddressesROMRAMI/OI/OI/Oaddress xPhysical Addr SpaceVirtual Addr Spaceaddress yMMUCPUPhysical addresses are handled only in kernel mode,i.e. kernel itself and device drivers
  14. 14. 14Address Translation with MMUDirectoryEntryPage TableEntryPhysical Addr011122131 22Page DirectoryPage TablePagePage OffsetPage Table IndexDirectory IndexVirtual AddrPage DirectoryBase RegisterNo page! ... page fault
  15. 15. 15TLB• Translation Lookaside Buffers• Something like hashtable getting a physicaladdress by a using virtual address as a key• In most address translation, page is found inTLB, so there is no need to access pagedirectory or page table...Virtual Address Pages Physical Address Pages
  16. 16. 16Multiple Memory SpacesROMI/OI/OPhysical Addr SpaceIndependent virtual memory spaces per processRAM
  17. 17. 17Demand Paging• Mapped per page– Page size is usually 4Kbytes• Two phase execution1. Virtual memory is allocated(mmap); justregistered in management table2. At actual access, physical momory isallocated for the pageAs no physcal page is allocated unless the page is accessedVirtual memory size >= Actually required physical memory size
  18. 18. 18Example of Demand Paging BehaviorVirtual Addr Space Physical Addr Spaceread accessNoCorrespondingPhysical page!Page fault occurs;Transits into kernel mode(1)
  19. 19. 19Example of Demand PagingBehavior (cont.)DMA TransferMappingVirtual Addr Space Physical Addr SpaceKernel loads the data and maps physical address(2)
  20. 20. 20Example of Demand PagingBehavior (cont.)Virtual Addr. Space Physical Addr SpaceReturn to user mode;User program can read the data as if nothing happened(but time has elapsed actually)(3)data
  21. 21. 21Page CachePhysical Addr SpaceData read from disk are kept on memory as far as space allows.Access tends to be sequential, so several pages are readat a time in advance;Thus disk access does not occur every time in (2)
  22. 22. 22Page OutIf no physical memory is available in (2), a page assumed to be leastused is released. If the contents of this page is not modified, it is justdiscarded; otherwise, the page is swapped out onto swap device.Virtual Addr Space Physicial Addr Spaceswap deviceMany of embedded Linux does not havea swap devicea page to beswapped out
  23. 23. 23Page Out (cont.)A requested page is allocated using area of a page released.This “juggling” enables to larger size of virtual memory thanphysical memory size actually installedDMA TransferVirtual Addr Space Physical Addr Spaceno page
  24. 24. 24Page SharingVirtual Addr SpacePhysical Addr SpaceProcess AProcess BThe same pages of a same file will beshared among more than one processes;for both read-only pages and writable pages
  25. 25. 25Copy on WriteProcess AProcess Bwriteaccessread onlyPage FaultOccursr/w privateIf write operation occurs on writableand private page...
  26. 26. 26Copy on Write (cont.)Process AProcess Bwriteaccessread onlyKernel copies the page and changesthe page status to read/writeread/writecopy
  27. 27. 27Memory Spaces of Processes0x000000000xffffffffTASK_SIZEprocess A B C...user spacekernelspaceKernel space is shared among processes;kernel space is not allowed toread/write/execute in user mode;user memory spaces are switchedwhen processes switchedAbout 3 GB user memory space per processTASK_SIZE is 0xc0000000 for i386;0xbf000000 for ARM
  28. 28. 28Example of Memory Spaceof a User Process00101000-0011a000 r-xp 00000000 fd:00 15172739 /lib/ld-2.4.so0011a000-0011b000 r-xp 00018000 fd:00 15172739 /lib/ld-2.4.so0011b000-0011c000 rwxp 00019000 fd:00 15172739 /lib/ld-2.4.so0011e000-0024a000 r-xp 00000000 fd:00 15172740 /lib/libc-2.4.so0024a000-0024d000 r-xp 0012b000 fd:00 15172740 /lib/libc-2.4.so0024d000-0024e000 rwxp 0012e000 fd:00 15172740 /lib/libc-2.4.so0024e000-00251000 rwxp 0024e000 00:00 008048000-08049000 r-xp 00000000 fd:00 11666681 /home/koba/lab/loop/a.out08049000-0804a000 rw-p 00000000 fd:00 11666681 /home/koba/lab/loop/a.outb7fef000-b7ff1000 rw-p b7fef000 00:00 0b7fff000-b8000000 rw-p b7fff000 00:00 0bffeb000-c0000000 rw-p bffeb000 00:00 0 [stack]cat /proc/<PROCESS_ID>/mapsfile nameinodedevicemajor:minorAddress Range file offsetr: readw: writex: executes: sharedp: private (copy on write)
  29. 29. 29Example of Memory Space of aUser Process (Detail)....0011e000-0024a000 r-xp 00000000 fd:00 15172740 /lib/libc-2.4.soSize: 1200 kBRss: 136 kBShared_Clean: 136 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 0 kB0024a000-0024d000 r-xp 0012b000 fd:00 15172740 /lib/libc-2.4.soSize: 12 kBRss: 8 kBShared_Clean: 0 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 8 kB0024d000-0024e000 rwxp 0012e000 fd:00 15172740 /lib/libc-2.4.soSize: 4 kBRss: 4 kBShared_Clean: 0 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 4 /proc/<PROCESS_ID>/smapsRSS = Physical Memory Size
  30. 30. 30mmap System Call• Map/Unmap files or devices onto memory• Argument prot– PROT_NONE, or OR operation of PROT_EXEC,PROT_READ, and PROT_WRITE• Argument flags– MAP_FIXED, MAP_SHARED, MAP_PRIVATE,MAP_ANONYMOUS, ...#include <sys/mman.h>void *mmap(void *start, size_t length, int prot, int flags,int fd, off_t offset);int munmap(void *start, sizt_t length);
  31. 31. 31mmap tips• Unless specified as MAP_FIXED, kernelsearches available pages• If MAP_FIXED is specified and it overlapsexisting pages, the pages are mumpappedinternally– Thus this option is usually not used• File offset must be multiple of page size• Addresses and sizes of mmap andmunmap need not be identical
  32. 32. 32Usage of mmap (1)• As substitute of malloc for large size– No data copy, such as compaction, occurs– Unlike malloc/free, addr and size at munmapcan be different than those at mmap• It is possible to allocate a large chunk with asingle mmap, and to release piecemeal withmultiple munmaps– In malloc of glibc implementation, mmap iscalled for a certain size or larger• DEFAULT_MMAP_THRESHOLD = (128*1024)
  33. 33. 33Usage of mmap (2)• Fast file access– In system calls read and write, data isinternally buffered in physical pages; fromthere data is copied to array specified by user– Using mmap enables to access page directly,thus number data copies can be reduced– java.nio.MappedByteBuffer in Java1.4
  34. 34. 34Usage of mmap (3)• Shared memory among processes– Map the same file as readable/writable andshared from more than one processes– IPC shared memory system calls (shmget,shmat, ...) does above internally
  35. 35. 35Usage of mmap (4)• Access to physical memory, I/O ports– By mapping device file /dev/mem, it becomespossible to read/write physical memory spacein user mode– To access /dev/mem, root privilege is required
  36. 36. 36Summary• Virtual memory usage and physicalmemory usage are not same. Physical onematters in practice• Be careful when overhead of virtualmemory occurs.– TLB miss– Page fault• Make use of system call mmap
  37. 37. 37References• Linux kernel source code• GNU C library source code• “Understanding the Linux Kernel (2nd Edition)”by Daniel P. Bovet (O’Reilly) [Japanese translation; 3rdEdition available in English]• “Linux kernel 2.6 Kaidokushitsu”, by Hirokazu Takahashiet. al. (SoftBank Creative) [in Japanese]• Linux man command• And other search results on web
  38. 38. 38One more thing: hot topics• From CELF BootTimeResources–KernelXIP–ApplicationXIP–(DataReadInPlace)• From CELF MemoryManagementResouces–Huge/large/superpages–Page cache compression