Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Key-Value Store for Data Acquisition Systems

Get an overview of the Data Acquisition Database design. It's based on the Persistent Memory Development Kit (PMDK) and Storage Performance Development Kit (SPDK) to leverage Intel® Optane™ DC persistent memory and non-volatile memory express (NVMe) drives.

  • Be the first to comment

A Key-Value Store for Data Acquisition Systems

  1. 1. Jakub Radtke, Maciej Maciejewski April 2019
  2. 2. SPDK, PMDK & Vtune™ Summit 2 DataAcQuisiontypicalusecase Key – timestamp, camera ID, geo, ... Value - frame Processing at the edge Filtered data Cloud storage DAQ DB enables better ways to process data at the edge Data stream
  3. 3. SPDK, PMDK & Vtune™ Summit 3 LHCexperimentswillbeproducing ©CERN
  4. 4. SPDK, PMDK & Vtune™ Summit 4 Dataacquisitionsystems(DAQ) 1 in 100 20..40x 5..10x >20k data fragments every 1ms ©Atlas Experiment©Atlas Experiment
  5. 5. SPDK, PMDK & Vtune™ Summit 5 PMDK Persistent Memory Development Kit • Optimal performance of persistent memory SPDK Storage Performance Development Kit • User-mode access to NVMe devices (SSDs)
  6. 6. SPDK, PMDK & Vtune™ Summit 6 DAQDB–aKVSforDAQ • First-line buffer for fast pre-computing and second-line buffer for longer term storage • Data structure based on optimized Adaptive Radix Trie • Distributed locking Compute farm ~2k nodes Offline storage DAQDB nodes ~500 NVMe Replication Persistent Memory Key/Value O(100) Gbps O(100) Gbps Request router ©Atlas Experiment
  7. 7. SPDK, PMDK & Vtune™ Summit 7 DAQ–specificAPI struct MinidaqKey { uint64_t eventId; uint16_t subdetectorId; uint16_t runId; } User-defined key structure kvPairVector = kvs->GetRange(keyMin, keyMax) Range queries with compound keys kvs->GetRangeAsync(keyMin, keyMax, cb) Asynchronous mode for even higher performance eventKey = kvs->GetAny(options=(lock)) Distributed locking for next event retrieval value = kvs->Alloc(key, 10 * 1024) DAQDB memory allocator minimizing copy operations
  8. 8. SPDK, PMDK & Vtune™ Summit 8 reserve/publishwork-flow • Application requests allocation for key data AllocKey() • Application requests allocation for value data AllocValue() • Application moves the value data • Signals it has finished Put() • Optional • Changes the location of object from AEP to NVMe • Changes locks state • Instructs persistency Update() DAQ DB tree makes object visible Copy the value data to SPDK ring buffer Publish actions Cancel actions pmemobj_x reserve() DAQ DB tree DAQ DB tree DAQ DB tree ©MuhannadAjjan - CC BY-SA 4.0
  9. 9. SPDK, PMDK & Vtune™ Summit  Throughput matching 2x100Gbps  Distributed Range Queries  RDMA networking 9 DAQDB Multi-TB/s hot storage solution of a petascale capacity for DAQ
  10. 10.