Successfully reported this slideshow.
Your SlideShare is downloading. ×

Simple and efficient way to get the last log using MMAP

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 27 Ad
Advertisement

More Related Content

Slideshows for you (20)

Similar to Simple and efficient way to get the last log using MMAP (20)

Advertisement

More from Tetsuyuki Kobayashi (20)

Recently uploaded (20)

Advertisement

Simple and efficient way to get the last log using MMAP

  1. 1. 1 Simple and efficient way to get ”the last log” using MMAP Tetsuyuki Kobayashi 2013.5.31 LinuxCon Japan
  2. 2. 2  The latest version of this slide will be available from here  http://www.slideshare.net/tetsu.koba/presentati ons
  3. 3. 3 Who am I?  20+ years involved in embedded systems  10 years in real time OS, such as iTRON  10 years in embedded Java Virtual Machine  Now GCC, Linux, QEMU, Android, …  Blogs  http://d.hatena.ne.jp/embedded/ (Personal)  http://blog.kmckk.com/ (Corporate)  http://kobablog.wordpress.com/(English)  Twitter  @tetsu_koba
  4. 4. 4 Motivation  We use log output for troubleshooting  fprintf(log, ”...”, …);  more and more logs …  Problem: too slow
  5. 5. 5 Limitation  The target is embedded system  Limited RAM  Slow storage (SD card)  Slow or no network
  6. 6. 6 Buffered vs Unbuffered  Buffered  + fast  - lose the last log  when abort by SEGV  Unbuffered  + never lose the last log  - slow
  7. 7. 7 I need ”fast” and ”the last log”  any idea?  I focus ”the last log” rather than ”whole log”
  8. 8. 8 Basic idea  Ring buffer on file associated shared memory
  9. 9. 9 Implementation file Target process ring buffer writing log Use mmap(2) to map file and memory
  10. 10. 10 Even if the process crush and abort .. file Target process ring buffer The ring buffer still remains as a file
  11. 11. 11 After that, I can examine the log file Log utility reading log
  12. 12. 12 Log format: text vs binary  Text  + Easy, flexible, human readable  - Slow, bigger data  Binary  + Fast, smaller data  - Less flexible  - Human unreadable  You need decoder utility
  13. 13. 13 How to allocate file associated shared memory fd = open(filename, O_CREAT |O_RDWR, S_IRWXU | S_IRWXO); ... fill_file(fd, size); addr = mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); if (MAP_FAILED == addr) { perror("mmap"); abort(); } close(fd); The file should not be empty, so fill it. After mmap, you can close(fd).
  14. 14. 14 void fill_file(int fd, int size) { int cnt, ret; char c = '0' for (cnt = 0; cnt < size; cnt++) { ret = write(fd, &c, 1); } } Simple version of fill_file()
  15. 15. 15 Synchronize ?  When memory and file synchronize in file associated memory?  Not always.  To get guarantee  msync(2) explicitly  munmap(2)
  16. 16. 16 Synchronize ?  Even if the process aborts, kernel takes care of unmapping all the process memory.  It means the last moment of shared memory is flushed to the file.
  17. 17. 17 When the process forks ..  Usual memory is private memory  Kernel does copy-on-write  The process don't care about it  For shared memory, kernel does not copy-on-write
  18. 18. 18 Quick summary of copy-on-write process Virtual memory Physical memory
  19. 19. 19 Quick summary of copy-on-write process Virtual memory Physical memory child process fork
  20. 20. 20 Quick summary of copy-on-write process Virtual memory Physical memory child process fork 1: Write to the page 2:Kernel copy the page and remap
  21. 21. 21 In case of file associated shared memory process Virtual memory Physical memory file
  22. 22. 22 process Virtual memory Physical memory file child process fork
  23. 23. 23 process Virtual memory Physical memory file child process fork writing log writing log Both process's logs mixed to the same file!!
  24. 24. 24 To avoid mixing log..  Child process have to re- initialize log buffer on new file right after fork(2)  Register hook by pthread_atfork(3)  file name should have process id prefix not to use the same file name
  25. 25. 25 Fork problem is solved process(PID=358) Virtual memory Physical memory file.358 child process(PID=359) file.359 fork
  26. 26. 26 Summary  File associated shared memory by mmap(2) is useful for the ring buffer of process crush log.  Consider about the process forking.  Use pthread_atfork(3) to register hook function
  27. 27. 27 Q & A @tetsu_koba Thank you for listening!

×