An insight into Fatcache - SSD backed memcache.

1. Twitter Fatcache
Satish Mittal
Data Platform
Inmobi

2. Overview
• Fatcache is memcache on SSD
• Memcache is volatile in-memory <K, V> cache, whereas
fatcache persists <K,V> on SSD
• Memory is much faster than SSD (~1000X)
• But memory is much costlier as well
• DRAM cost per server increases dramatically beyond 150 GB
• Power cost increases similarly
• Not a viable option to horizontally scale to TBs of data across nodes
• Network-connected SSD design makes sense if network
latencies dominate over SSD latencies by a large factor

3. Latency comparison
• Intel 320 series SSD:
o Read latency: 75 us
o Write latency: 90 us
o Sequential Read Bandwidth: 270 MB/s
o Sequential Write Bandwidth: 220 MB/s
• Memory:
o Latency: 50-70 ns
o Bandwidth: 15-25 GB/s
• Rotational disk:
o Seek time: 3-15 ms
o Data transfer rate: 130 MB/s

4. SSD I/O characteristics
• SSD reads happen at page-level granularity, usually 4 KB.
o Single page read access time is ~70 us. Hence SSD access needs to be
minimized to keep SSD latency under network latency.
o Fatcache reduces SSD reads by maintaining an in-memory
index for all on-disk data.
• SSD writes are essentially erase-and-rewrites.
o In-place updates to SSD degrade performance.
o Small, random writes reduce SSD lifetime and need to be eliminated.
o Fatcache aggregates all writes in memory and write to SSD in
batches in a log structured fashion.

5. Scaling
• Network I/O in fatcache is async, but SSD I/O is sync.
• To exploit full SSD parallelism, we need to run multiple
instances of fatcache against single SSD. Each instance works
on a fixed ‘hard’ partition of SSD.
• Further scale the SSD throughput by scaling the number of
SSDs on a single machine and the number of machines.

6. Accessibility
• Fatcache supports the Memcache protocol to get/set data.
• Storage commands: SET, ADD, REPLACE, APPEND, PREPEND, CAS
• Retrieval commands: GET, GETS
• Delete command: DELETE
• Arithmetic commands: INCR, DECR
• Quit command: QUIT
• Clients support at least one method of hashing keys among
servers
• Clients should support consistent hashing scheme to handle
scenarios of server additions/removals.

7. Durability
• Fatcache is NOT a <K,V> store
• Any item written to fatcache is subject to cache eviction
• Capacity-triggered eviction happens if at the time of adding a
new item, there are no free chunks and no free pages
available on SSD
• Page level eviction would result in a cache miss if client needs
to access a key belonging to that page
• Ideally server should expose various stats that helps figure out
whether fatcache is frequently peaking capacity and needs
scaling out
o Currently no observability through stats

8. Availability
• If a fatcache instance becomes unavailable, client can take
either of two approaches – failover or failure
• Failover: If client supports consistent hashing, it would
‘reroute’ the request to next available instance in the list
• Initially client would have to deal with cache miss
• Client can choose to start updating keys in the new instance
• When the failed instance comes back, client would start
seeing older versions since request gets ‘rerouted’ back to the
original instance
o Any updates made during failover will not be visible after restart

9. Availability
• Failure: In this approach, client can simply treat server
unavailable scenario as a cache miss
• Depending upon cache miss strategy, client can choose to
connect to secondary store until server comes back
• Manual monitoring is needed to detect failures and restart
instances quickly
• Restart will load the snapshot of data last persisted to SSD
o All pending writes batched in-memory last time will be lost

10. Performance
• Published performance results:
o A single fatcache can do close to 100K set/sec for 100 byte item sizes.
o A single fatcache can do close to 4.5K get/sec for 100 byte item sizes.
o All 8 fatcache instances in aggregate do 32K get/sec to a single 600 GB
SSD.
• Need to run our own in-house tests to get real
numbers.

11. Appendix
• Documentation:
https://github.com/twitter/fatcache/blob/master/README.m
d
• Performance results:
https://github.com/twitter/fatcache/blob/master/notes/perf
ormance.md

12. Thanks!