Use cases like high-performance computing (HPC), AI, and IoTA can generate a huge volume of data. Learn how Intel® Optane™ DC persistent memory can be an alternative to DRAM for applications that benefit from a very large volatile memory capacity.
2. SPDK, PMDK & Vtune™ Summit
Agenda
• Why Use Persistent Memory as Volatile
• How to Create Volatile Region over
Persistent Memory
▪ libmemkind
▪ libvmemcache
2
3. SPDK, PMDK & Intel® VTune™ Amplifier Summit 3
ModesataGlance
• Memory Mode (volatile Memory)
• Data Placement
• Application does not have control
over data placement between
DRAM and Intel Persistent
Memory
• Ease of adoption: No Code Changes
• Performance slower than DRAM
• App Direct Mode (persistent memory)
• Data Placement
• Application has control over data
placement between DRAM and Intel
Persistent Memory
• Ease of Adoption: Need to Re-architect
Application
• Performance at Native hardware latencies
APPLICATION
VOLATILE MEMORY POOL
O P T A N E P E R S I S T E N T M E M O R Y
D R A M A S C A C H E
APPLICATION
OPTANE PERSISTENT
MEMORY
DRAM
4. SPDK, PMDK & Intel® VTune™ Amplifier Summit 4
Motivation
• Application Controlled Data Placement
• Has Different Tiers of Data
• Opportunity to use Bigger and Cheaper Volatile Memory than DRAM
• Persistence not Required
• Reduce Development Effort
5. SPDK, PMDK & Vtune™ Summit 5
Whatislibmemkind
DRAM
Unified Memory Management
High BW
Memory
Intel® Optane™ DC
Persistent Memory
Distros
Allocator Familiar API
stdlib like API
Availability
https://github.com/memkindJeMalloc 5.0
6. SPDK, PMDK & Intel® VTune™ Amplifier Summit 6
HowDoesitWORK
• pmem kind is file-backed
• Temp file created on DAX-enabled
filesystem
• Temp file is memory mapped into
application virtual address space
• Temp file deleted when application
terminates
• Jemalloc manages memory allocation from
this address space
libmemkind is a Wrapper; Jemalloc Manages Heap
13. SPDK, PMDK & Vtune™ Summit
Problemstatement
Local LRU cache
Support for large capacities available with persistent memory (many terabytes per server)
Lightweight, efficient and embeddable
In-memory
Scalable
13
14. SPDK, PMDK & Vtune™ Summit
Existingsolutions
In-memory databases tend to rely on malloc() in some form for allocating memory for
entries
▪ Which means allocating anonymous memory
Persistent Memory is exposed by the operating system through normal file-system
operations
▪ Which means allocating byte-addressable PMEM needs to use file memory mapping
(fsdax).
We could modify the allocator of an existing in-memory database and be done with it,
right? ☺
14
15. SPDK, PMDK & Vtune™ Summit 15
Fragmentation
Manual dynamic memory management a’la
dlmalloc/jemalloc/tcmalloc/palloc causes
fragmentation
Applications with substantial expected
runtime durations need a way to combat this
problem
▪ Compacting GC (Java, .NET)
▪ Defragmentation (Redis, Apache Ignite)
▪ Slab allocation (memcached)
Especially so if there’s substantial expected
variety in allocated sizes
16. SPDK, PMDK & Vtune™ Summit 16
Extentallocation
If fragmentation is unavoidable, and
defragmentation/compacting is CPU and
memory bandwidth intensive, let’s embrace it!
Usually only done in relatively large blocks in
file-systems.
But on PMEM, we are no longer restricted by
large transfer units (sectors, pages etc)
17. SPDK, PMDK & Vtune™ Summit 17
Scalablereplacementpolicy
Performance of libvmemcache was
bottlenecked by naïve implementation of
LRU based on a doubly-linked list.
With 100st of threads, most of the time of
any request was spent
waiting on a list lock…
Locking per-node doesn’t solve the
problem…
18. SPDK, PMDK & Vtune™ Summit 18
BufferedLRU
Our solution was quite simple.
We’ve added a non-blocking ring-buffer
which buffers the list-move operations
This way, the list only needs to get locked
during eviction or when the ring-buffer is
full.
19. SPDK, PMDK & Vtune™ Summit
Lightweight,embeddable,in-memorycaching
https://github.com/pmem/vmemcache
VMEMcache *cache = vmemcache_new("/tmp", VMEMCACHE_MIN_POOL,
VMEMCACHE_MIN_EXTENT, VMEMCACHE_REPLACEMENT_LRU);
const char *key = "foo";
vmemcache_put(cache, key, strlen(key), "bar", sizeof("bar"));
char buf[128];
ssize_t len = vmemcache_get(cache, key, strlen(key),
buf, sizeof(buf), 0, NULL);
vmemcache_delete(cache);
libvmemcache has normal get/put APIs, optional replacement policy, and configurable extent
size. Works with terabyte-sized in-memory workloads without a sweat, with very high space
utilization. Also works on regular DRAM.
19