3. 3
Motivation
• Indexing in WAL/DB
• Synchronous heat recording
• No Batch Processing
• SQL Creates are expensive : ACID
• Read-Modify-Update Expensive
4. 4
What is ImeTaL?
• In Memory Circular Log
• Updates are captured and processed in Batches
5. 5
ImeTaL Components
• Log Record : Each record that is inserted in IMeTaL
• Segment: Segments is logical partitioning of the log. Each
segment will hold a fixed amount of Log Records.
• Locator: Locator is a consultant that will tell the FOP
thread where to insert the Log Record in the log or
precisely speaking which location in a segment.
• Flush Queue : Full or Expired Segments go to the flush
queue.
• Worker Thread Pool: This is a thread pool of worker
threads that works on the segments that are places in the
flush queue.
6. 6
ImeTaL Components
• Flush Function/Data-Structure: Flush function is the
function that is called when a segment, which is nothing
but a batch of Log Records needs to be flushed to the
database.
• Flush Data-Structure : Before the flush we need to sort the
Log Records for redundant entries and have only one
entry per GFID. For this we need a data structure like
balanced binary search tree - red-black tree (insertion and
search of O(log(n))), that would help us sort the Log
records. Once the Log records are sorted in the data
structure without duplication, we traverse the flush data-
structure and start flushing the Log records into the
database.
7. 7
Configurations:
• Size of segment : How many gfdb_db_records in a
segments (set during : init of IMTL + dynamic growth**)
• Size of the IMeTaL : How many segments in IMeTaL (set
during : init of IMTL + dynamic growth**)
• 3. Thread pool size : How many worker threads (set during
: init of IMeTaL + dynamic growth)
• 4. On-Demand Flush : On/Off (set during : init of IMeTaL
+ dynamic)
