Take-away point: 85% of all disk blocks written were never updated within the hour.93% of all disk blocks written were never updated within a day.Data is first stored in NVRAM / RAZ until it reaches a certain age.
one—day trace.With larger blocks, it’s more likely that larger percentage of blocks will be updated..Trade off: Larger block reduce each block overhead. But has higher update rate.Negligible for blocks updated more than 2-4 times.
A shingled-disk aware filesystem might be more helpful to optimize this situation.
Stripe: Stripe, and randomly interleaving the four workloads with burst size.Pure: No stripe, no interleaving of different data sources: i.e. data source 1, data source 2…Dedicated: Each disk is dedicated to an individual source workload.Relocating disk bans.Unrelated data being written in adjacent positions that increases the likelihood of an update in the band.
The Big Data Storage Storage: Magnetic disks High storage density Current: 400GB/in2 - 550GB/in2 30-50% increase per year. It’s reaching its physical limit… “Superparamagnetic Limit” Predicted limit: around 1TB/in2
Conventionally Written Track Non-overlap Track width w (e.g. 25nm) Guard gaps between tracks (e.g. g = 5nm). Bottleneck is the writing track width. Current read heads can work on much narrower track. But it is hard to write narrower track.
Shingled Disk: Overlap Tracks Wilder Track written. (e.g. w = 70nm). Shingled writing overlaps tracks. The remaining residual track could be much narrower. (e.g. r = 10nm).
Characteristics Higher Density without significant hardware change. 2-3 times of the conventional disk density. Support Random Read / Sequential Write A single write will destroy the next k tracks Typically, k = 4~8 Can we do better than a “tape with random read support”?
Two High-Level Strategies Mask the operational difference of a Shingled Disk. Drop-in replacement for current disks. Uses the standard block interface. Specialized file system with no/little hardware mask. More flexibility in the data layout and block management. Increased knowledge at file system layer.
Strategy One: Masking the Operational Difference Synergy with SSD: Slow erasure of block in SSD SSD: Flash Translation Layer (FTL) Shingled Disk: Shingled Translation Layer (STL) Translate from Virtual Block Address to Logical Block Address on disk. How to perform random write. One extreme: Read-modify-write. Another extreme: Remap the physical location of written data. Benefit No change for user and system. “Drop-in” replacement for current system.
Strategy One: Masking the Operational Difference Drawback Experience with SSDs indicates the performance will be hard to predict. Reverse engineering on SSD to achieve higher level goals. Sophisticated STL could be expensive. Data stored in Continuous Virtual Block Address could be far away on disk. Database table with frequent edits. Concurrent downloads of movies. Might use large NVRAM (as cache) to mitigate the problem.
Virtual Block Address Translation Need to quickly translate Virtual Block Address to Logical Block Address. Translation Table could be very large. Capacity 2T, each entry 8 bytes. Block Size 4K: Translation Table 4GB. Block Size 512 bytes: Translation Table 32GB. Some B+-Tree type structure.
Strategy Two: Specialized System Simple Shingled Translation Layer. Random Update: Read-Modify-Write. TRIM Command: Tell hardware that overwriting subsequent tracks is fine. Support to format some part of the disk unshingled. More Sophisticated System Software Avoid writing to the middle of the band. Conceptualize writing as appending to a log. Perform necessary data remapping and garbage collection.
Band Abstraction Store the bulk of data in band. A collection of b-contiguous tracks. A buffer of k tracks at the end of each band. Bands are not interfered with each other More flexible.
Proportional Capacity Loss c = 1 – b/(b+k): proportional capacity loss • k = 5 and want to control c < 0.1 • b > 45 • Each band have 67.5MB • Reasonable for modern LFS.
Band Usage1. Only writes complete bands. Each band contains a segment of Log structured File System. Assumes data is buffering in NVRAM.2. Only appends to bands. Less efficient.3. Circular Log inside each band Consume data from head. Append data to the tail. Require additional k track gap between head and tail.4. Flexible band size Neighboring bands could be joined. Not suitable for a general purpose SWD: Just for completeness.
Reserved Space for Random Update Option 1: NVRAM. Option 2: Random Access Zone (RAZ) Every track is followed by k unused tracks. Density of RAZ is lower than current disk.
How Large Random Access Zone Can we Have? Assume without RAZ, the capacity of shingled disk is 2.3 times of conventional disk. • If we want to guarantee L = 2 times of the conventional disk. • k = 5: 3.75% of total storage capacity.
Trade offs for Two Options Reserved Space for Random Access Option 1: NVRAM Faster More Expensive: cost 10 times to RAZ Option 2: Random Access Zone (RAZ) Use some part of the disk for Random Access Zone. Cheaper but slower. Trade-offs would be interesting.
Usage of NVRAM1. Buffering data for writing bands. Be careful about the limited number of write-erase cycles of flash memory.2. Use NVRAM to store metadata. Metadata tends to have a higher amount of activity. In Write Anywhere File System, NVRAM could be used to maintain the log of file system activities.3. Store recently created objects. Temporal locality: a block/object created long long ago is less likely to be updated. If data is first written to NVRAM, we can also have better placement of data on disk.
Number of Logs Here Log-structured File System is assumed to be used. What’s the benefit to have more than a single log? 1. Separation between metadata and data. E.g. Access Time. 2. Allocate files for more efficient read access later. E.g. Downloads several movies at the same time. If only one log, all the movie objects will be interspersed. Inefficient for read.
Workloads Evaluation Rate of Block Updates If few blocks are updated frequently. Less need for Random Access Zone / NVRAM. Shingled Disk is more usable to replace conventional disk. Evaluated Workloads 1. General purpose personal usage for 1 month 2. Specialized workload: Video Edit for 3 hours. 3. Specialized workload: Music Library Management. Negligible block update. Not surprised.
Some Points From Workload1. Identifying hot blocks is important since the volume of hot blocks is small enough to be held in the Random Access Zone / NVRAM.2. Larger block sizes reduces the accuracy of identifying hot blocks. But it’s not that significant.3. File system distinguish between metadata from user data would be helpful.
Shingled Disk Arrays Can the shingled disk used in a server environment? Probably part of the disk array. It could have writing originating from different sources. Two Impacts in Workload Data Striping. Workload Interleaving. Replay workloads against a simulated drive. Log-structured Writing Scheme to perform in-band update.