Kernel Recipes 2013 - Persistent logs using UBI
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Kernel Recipes 2013 - Persistent logs using UBI

on

  • 683 views

A short presentation about a library for persistent log based on the UBI layer. The goal is to enable an embedded software component to log persistent messages (even after a full update of the ...

A short presentation about a library for persistent log based on the UBI layer. The goal is to enable an embedded software component to log persistent messages (even after a full update of the system) without requiring the use of a file system.

Statistics

Views

Total Views
683
Views on SlideShare
626
Embed Views
57

Actions

Likes
0
Downloads
14
Comments
0

2 Embeds 57

https://kernel-recipes.org 47
https://twitter.com 10

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Kernel Recipes 2013 - Persistent logs using UBI Presentation Transcript

  • 1. Persistent log with UBI Matthieu CASTET - www.parrot.com 25 September 2013 Persistent log with UBI
  • 2. Persistent log with UBI ulog
  • 3. Goal Log must be persistent on product : Nand flash Can be used to trace system updates Has to be independent/hidden of application filesystem not on / filesystem Has to be usable on shipped products difficult to resize all partitions Persistent log with UBI ulog
  • 4. Which interface to use ? Raw flash (MTD) UBI Nand filesystem (UBIFS, ...) Persistent log with UBI ulog
  • 5. Flash device vs block device 1/2 Block device Consists of sectors Sectors are small (512, 1024 B) Maintains 2 main operations: Flash device Consists of eraseblocks Eraseblocks are larger ( typically 128KB) Maintains 3 main operations: read sector read from eraseblock write sector write to eraseblock erase eraseblock Persistent log with UBI ulog
  • 6. Flash device vs block device 2/2 Block device Bad sectors are re-mapped and hidden by hardware (at least in modern LBA hard drives) Sectors are devoid of the wear-out property Flash device Bad eraseblocks are not hidden and should be dealt with in software Eraseblocks wear-out and become bad and unusable after about 103 (for MLC NAND) - 105 (NOR, SLC NAND) erase cycles Flash device is more difficult to handle (ecc, bad block, eraseblock, ...) Persistent log with UBI ulog
  • 7. MTD MTD stands for "Memory Technology Devices" Provides an abstraction of flash devices Hides many aspects specific to particular flash technologies Provides uniform API to access various types of flashes E.g., MTD supports NAND, NOR, ECC-ed NOR, DataFlash, OneNAND, etc Provides partitioning capabilities Persistent log with UBI ulog
  • 8. UBI Stands for "Unsorted Block Images" Provides an abstraction of "UBI volume" Has kernel API and user space API (/dev/ubi0) Provides wear-leveling Hides bad eraseblocks Allows run-time volume creation, deletion, and re-size Is somewhat similar to LVM, but for MTD devices Persistent log with UBI ulog
  • 9. UBI volume vs MTD device MTD device Consists of physical (PEB), typically 128 KB Has 3 main operations eraseblocks UBI device Consists of logical eraseblocks (LEB), slightly smaller than PEB (e.g 126/124 KB) Has 3 main operations read from PEB read from LEB write to PEB write to LEB erase PEB erase LEB May have bad PEBs PEBs wear out MTD devices are static Persistent log with UBI ulog Does not have bad LEB (handle a certain amount of bad PEB) LEBs do not wear out - UBI spread the I/O load across the whole flash UBI volumes are dynamic
  • 10. Main idea behind UBI Maps LEBs to PEBs Any LEB may be mapped to any PEB Eraseblock headers store mapping information and erase count Persistent log with UBI ulog
  • 11. Other Handle bit-flips by moving data to a different PEB Configurable wear-leveling threshold Atomic LEB change Volume update/rename operation Suitable for MLC NAND Performs operations in background Works on NAND, NOR and other flash types Tolerant to power cuts Simple and robust design easy support in bootloader Persistent log with UBI ulog
  • 12. UBIFS Filesystem on top of UBI (2.6.27 2008-10) Needs a minimal number of LEBs to work : 17 http://www.linuxmtd.infradead.org/faq/ubifs.html#L_few_lebs Complex filesystem : few (rare) corruptions seen on products Persistent log with UBI ulog
  • 13. log over UBI we have a UBI device (for linux kernel) with free space UBIFS has too much overhead Persistent log with UBI ulog
  • 14. UBI user API include/mtd/ubi-user.h device attach device detach volume create volume delete volume resize volume rename volume update (static volume) LEB erase LEB atomic change LEB map LEB unmap lseek read write Persistent log with UBI ulog
  • 15. ulog A log entry is much smaller than a page size (512B-4KB) A cache is needed (in flash) ulog uses a dynamic volume to have per LEB write Persistent log with UBI ulog
  • 16. ulog Persistent log with UBI ulog
  • 17. ulog : flush 1 Persistent log with UBI ulog
  • 18. ulog : flush 2 Persistent log with UBI ulog
  • 19. ulog : flush 2 Persistent log with UBI ulog
  • 20. ulog : flush 3 Persistent log with UBI ulog
  • 21. ulog : flush 4 Persistent log with UBI ulog
  • 22. Data flush Copy data from level N to level N+1 Merge pages to try to fill pages from level N+1 if level 0 has 64 entries of about 20 B, it can be merged in a page of 2 KB. Can be recursive ! flush of L0, but L1 is full flush of L1 needed ... Persistent log with UBI ulog
  • 23. ulog : rotate 1 Persistent log with UBI ulog
  • 24. ulog : rotate 2 Persistent log with UBI ulog
  • 25. ulog : rotate 3 Persistent log with UBI ulog
  • 26. ulog : rotate 4 Persistent log with UBI ulog
  • 27. ulog : rotate 5 Persistent log with UBI ulog
  • 28. Data rotate Clean next LEB (if not empty) Write data on it Persistent log with UBI ulog
  • 29. Data format Header (32 bits) size (24 bits) version (3 bits) (relevant for first page) Log data Persistent log with UBI ulog
  • 30. Scan Parse header for all LEBs LEB version LEB index (which pages have data) For last level (L2), find current LEB All empty use first (L2) LEB Use LEB version Persistent log with UBI ulog
  • 31. Scan Parse header for all LEBs LEB version LEB index (which pages have data) For last level (L2), find current LEB All empty use first (L2) LEB Use LEB version Persistent log with UBI ulog
  • 32. ulog API ulog_init ulog_destroy ulog_read ulog_printf ulog_vprintf ulog_flush Persistent log with UBI ulog
  • 33. ulog demo Persistent log with UBI ulog
  • 34. TODO any remark? Persistent log with UBI ulog
  • 35. TODO any remark? Need to correctly handle all power failure cases during a flush Can be complex in case of cascaded flush Some data are currently duplicated Persistent log with UBI ulog
  • 36. Questions ? Merci pour votre attention ! Thanks for your attention! Questions? Persistent log with UBI ulog