Your SlideShare is downloading. ×
Simple and efficient way to get the last log using MMAP
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Simple and efficient way to get the last log using MMAP


Published on

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. 1Simple and efficient wayto get ”the last log”using MMAPTetsuyuki Kobayashi2013.5.31 LinuxCon Japan
  • 2. 2 The latest version of this slide willbe available from here
  • 3. 3Who am I? 20+ years involved in embedded systems 10 years in real time OS, such as iTRON 10 years in embedded Java Virtual Machine Now GCC, Linux, QEMU, Android, … Blogs (Personal) (Corporate) Twitter @tetsu_koba
  • 4. 4Motivation We use log output fortroubleshooting fprintf(log, ”...”, …); more and more logs … Problem: too slow
  • 5. 5Limitation The target is embedded system Limited RAM Slow storage (SD card) Slow or no network
  • 6. 6Buffered vs Unbuffered Buffered + fast - lose the last log when abort by SEGV Unbuffered + never lose the last log - slow
  • 7. 7I need ”fast” and ”the last log” any idea? I focus ”the last log”rather than ”whole log”
  • 8. 8Basic idea Ring buffer on fileassociated sharedmemory
  • 9. 9ImplementationfileTarget processring bufferwriting logUse mmap(2) to map file and memory
  • 10. 10Even if the process crush andabort ..fileTarget processring bufferThe ring buffer still remains as a file
  • 11. 11After that, I can examine the logfileLog utilityreading log
  • 12. 12Log format: text vs binary Text + Easy, flexible, human readable - Slow, bigger data Binary + Fast, smaller data - Less flexible - Human unreadable You need decoder utility
  • 13. 13How to allocate file associatedshared memoryfd = open(filename, O_CREAT |O_RDWR, S_IRWXU | S_IRWXO);...fill_file(fd, size);addr = mmap(NULL, size, PROT_READ|PROT_WRITE,MAP_SHARED, fd, 0);if (MAP_FAILED == addr) {perror("mmap");abort();}close(fd);The file should not be empty, so fill it.After mmap, you can close(fd).
  • 14. 14void fill_file(int fd, int size){int cnt, ret;char c = 0for (cnt = 0; cnt < size; cnt++) {ret = write(fd, &c, 1);}}Simple version of fill_file()
  • 15. 15Synchronize ? When memory and filesynchronize in file associatedmemory? Not always. To get guarantee msync(2) explicitly munmap(2)
  • 16. 16Synchronize ? Even if the process aborts,kernel takes care of unmappingall the process memory. It means the last moment ofshared memory is flushed tothe file.
  • 17. 17When the process forks .. Usual memory is private memory Kernel does copy-on-write The process dont care about it For shared memory, kernel doesnot copy-on-write
  • 18. 18Quick summary of copy-on-writeprocessVirtual memoryPhysical memory
  • 19. 19Quick summary of copy-on-writeprocessVirtual memoryPhysical memorychild processfork
  • 20. 20Quick summary of copy-on-writeprocessVirtual memoryPhysical memorychild processfork 1: Write to the page2:Kernel copy the page and remap
  • 21. 21In case of file associated sharedmemoryprocessVirtual memoryPhysical memoryfile
  • 22. 22processVirtual memoryPhysical memoryfilechild processfork
  • 23. 23processVirtual memoryPhysical memoryfilechild processforkwriting log writing logBoth processs logsmixed to the same file!!
  • 24. 24To avoid mixing log.. Child process have to re-initialize log buffer on new fileright after fork(2) Register hook bypthread_atfork(3) file name should have process idprefix not to use the same filename
  • 25. 25Fork problem is solvedprocess(PID=358)Virtual memoryPhysical memoryfile.358child process(PID=359)file.359fork
  • 26. 26Summary File associated shared memory bymmap(2) is useful for the ringbuffer of process crush log. Consider about the processforking. Use pthread_atfork(3) to registerhook function
  • 27. 27Q & A@tetsu_kobaThank you for listening!