• Save
Simple and efficient way to get the last log using MMAP
Upcoming SlideShare
Loading in...5

Simple and efficient way to get the last log using MMAP






Total Views
Views on SlideShare
Embed Views



10 Embeds 4,447

http://d.hatena.ne.jp 2597
http://blog.kmckk.com 1782
http://eventifier.co 25
https://twitter.com 19
http://hatenatunnel.appspot.com 8
http://eventifier.com 7
http://webcache.googleusercontent.com 4
http://translate.googleusercontent.com 3
http://www.google.co.jp 1
http://cache.yahoofs.jp 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Simple and efficient way to get the last log using MMAP Simple and efficient way to get the last log using MMAP Presentation Transcript

  • 1Simple and efficient wayto get ”the last log”using MMAPTetsuyuki Kobayashi2013.5.31 LinuxCon Japan
  • 2 The latest version of this slide willbe available from here http://www.slideshare.net/tetsu.koba/presentations
  • 3Who am I? 20+ years involved in embedded systems 10 years in real time OS, such as iTRON 10 years in embedded Java Virtual Machine Now GCC, Linux, QEMU, Android, … Blogs http://d.hatena.ne.jp/embedded/ (Personal) http://blog.kmckk.com/ (Corporate) http://kobablog.wordpress.com/(English) Twitter @tetsu_koba
  • 4Motivation We use log output fortroubleshooting fprintf(log, ”...”, …); more and more logs … Problem: too slow
  • 5Limitation The target is embedded system Limited RAM Slow storage (SD card) Slow or no network
  • 6Buffered vs Unbuffered Buffered + fast - lose the last log when abort by SEGV Unbuffered + never lose the last log - slow
  • 7I need ”fast” and ”the last log” any idea? I focus ”the last log”rather than ”whole log”
  • 8Basic idea Ring buffer on fileassociated sharedmemory
  • 9ImplementationfileTarget processring bufferwriting logUse mmap(2) to map file and memory
  • 10Even if the process crush andabort ..fileTarget processring bufferThe ring buffer still remains as a file
  • 11After that, I can examine the logfileLog utilityreading log
  • 12Log format: text vs binary Text + Easy, flexible, human readable - Slow, bigger data Binary + Fast, smaller data - Less flexible - Human unreadable You need decoder utility
  • 13How to allocate file associatedshared memoryfd = open(filename, O_CREAT |O_RDWR, S_IRWXU | S_IRWXO);...fill_file(fd, size);addr = mmap(NULL, size, PROT_READ|PROT_WRITE,MAP_SHARED, fd, 0);if (MAP_FAILED == addr) {perror("mmap");abort();}close(fd);The file should not be empty, so fill it.After mmap, you can close(fd).
  • 14void fill_file(int fd, int size){int cnt, ret;char c = 0for (cnt = 0; cnt < size; cnt++) {ret = write(fd, &c, 1);}}Simple version of fill_file()
  • 15Synchronize ? When memory and filesynchronize in file associatedmemory? Not always. To get guarantee msync(2) explicitly munmap(2)
  • 16Synchronize ? Even if the process aborts,kernel takes care of unmappingall the process memory. It means the last moment ofshared memory is flushed tothe file.
  • 17When the process forks .. Usual memory is private memory Kernel does copy-on-write The process dont care about it For shared memory, kernel doesnot copy-on-write
  • 18Quick summary of copy-on-writeprocessVirtual memoryPhysical memory
  • 19Quick summary of copy-on-writeprocessVirtual memoryPhysical memorychild processfork
  • 20Quick summary of copy-on-writeprocessVirtual memoryPhysical memorychild processfork 1: Write to the page2:Kernel copy the page and remap
  • 21In case of file associated sharedmemoryprocessVirtual memoryPhysical memoryfile
  • 22processVirtual memoryPhysical memoryfilechild processfork
  • 23processVirtual memoryPhysical memoryfilechild processforkwriting log writing logBoth processs logsmixed to the same file!!
  • 24To avoid mixing log.. Child process have to re-initialize log buffer on new fileright after fork(2) Register hook bypthread_atfork(3) file name should have process idprefix not to use the same filename
  • 25Fork problem is solvedprocess(PID=358)Virtual memoryPhysical memoryfile.358child process(PID=359)file.359fork
  • 26Summary File associated shared memory bymmap(2) is useful for the ringbuffer of process crush log. Consider about the processforking. Use pthread_atfork(3) to registerhook function
  • 27Q & A@tetsu_kobaThank you for listening!