Simple and efficient way to get the last log using MMAP


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Simple and efficient way to get the last log using MMAP

  1. 1. 1Simple and efficient wayto get ”the last log”using MMAPTetsuyuki Kobayashi2013.5.31 LinuxCon Japan
  2. 2. 2 The latest version of this slide willbe available from here
  3. 3. 3Who am I? 20+ years involved in embedded systems 10 years in real time OS, such as iTRON 10 years in embedded Java Virtual Machine Now GCC, Linux, QEMU, Android, … Blogs (Personal) (Corporate) Twitter @tetsu_koba
  4. 4. 4Motivation We use log output fortroubleshooting fprintf(log, ”...”, …); more and more logs … Problem: too slow
  5. 5. 5Limitation The target is embedded system Limited RAM Slow storage (SD card) Slow or no network
  6. 6. 6Buffered vs Unbuffered Buffered + fast - lose the last log when abort by SEGV Unbuffered + never lose the last log - slow
  7. 7. 7I need ”fast” and ”the last log” any idea? I focus ”the last log”rather than ”whole log”
  8. 8. 8Basic idea Ring buffer on fileassociated sharedmemory
  9. 9. 9ImplementationfileTarget processring bufferwriting logUse mmap(2) to map file and memory
  10. 10. 10Even if the process crush andabort ..fileTarget processring bufferThe ring buffer still remains as a file
  11. 11. 11After that, I can examine the logfileLog utilityreading log
  12. 12. 12Log format: text vs binary Text + Easy, flexible, human readable - Slow, bigger data Binary + Fast, smaller data - Less flexible - Human unreadable You need decoder utility
  13. 13. 13How to allocate file associatedshared memoryfd = open(filename, O_CREAT |O_RDWR, S_IRWXU | S_IRWXO);...fill_file(fd, size);addr = mmap(NULL, size, PROT_READ|PROT_WRITE,MAP_SHARED, fd, 0);if (MAP_FAILED == addr) {perror("mmap");abort();}close(fd);The file should not be empty, so fill it.After mmap, you can close(fd).
  14. 14. 14void fill_file(int fd, int size){int cnt, ret;char c = 0for (cnt = 0; cnt < size; cnt++) {ret = write(fd, &c, 1);}}Simple version of fill_file()
  15. 15. 15Synchronize ? When memory and filesynchronize in file associatedmemory? Not always. To get guarantee msync(2) explicitly munmap(2)
  16. 16. 16Synchronize ? Even if the process aborts,kernel takes care of unmappingall the process memory. It means the last moment ofshared memory is flushed tothe file.
  17. 17. 17When the process forks .. Usual memory is private memory Kernel does copy-on-write The process dont care about it For shared memory, kernel doesnot copy-on-write
  18. 18. 18Quick summary of copy-on-writeprocessVirtual memoryPhysical memory
  19. 19. 19Quick summary of copy-on-writeprocessVirtual memoryPhysical memorychild processfork
  20. 20. 20Quick summary of copy-on-writeprocessVirtual memoryPhysical memorychild processfork 1: Write to the page2:Kernel copy the page and remap
  21. 21. 21In case of file associated sharedmemoryprocessVirtual memoryPhysical memoryfile
  22. 22. 22processVirtual memoryPhysical memoryfilechild processfork
  23. 23. 23processVirtual memoryPhysical memoryfilechild processforkwriting log writing logBoth processs logsmixed to the same file!!
  24. 24. 24To avoid mixing log.. Child process have to re-initialize log buffer on new fileright after fork(2) Register hook bypthread_atfork(3) file name should have process idprefix not to use the same filename
  25. 25. 25Fork problem is solvedprocess(PID=358)Virtual memoryPhysical memoryfile.358child process(PID=359)file.359fork
  26. 26. 26Summary File associated shared memory bymmap(2) is useful for the ringbuffer of process crush log. Consider about the processforking. Use pthread_atfork(3) to registerhook function
  27. 27. 27Q & A@tetsu_kobaThank you for listening!