A distributed database.
Sir Tariq Mehmood Presented By:
Voldemort is a distributed data store that is
designed as a key-value store used by
LinkedIn for high-scalability storage.
It is named after the fictional Harry Potter
villain Lord Voldemort.
there are job titles, job openings for people, Groups
and companies that offerings jobs.
Need frequently read write
LinkedIn Big Data Problem
Voldemort Scale Both
• The amount of data we can store (write)
• The number of requests for that data (read)
Why Not Using Hadopp
Naturally the only way to do this is to spread both
the load and the data across many servers.
1. Need to find a way to split the data so that all
servers have different data
2. Need to find a way to handle server failures
without interrupting service
3. HBase still be write-heavy (due to horizontal
partitioning and use of SSTables, which are write
• Data is automatically replicated over multiple
• Data is automatically partitioned so each server
contains only a subset of the total data
• Provides tunable consistency (strict quorum or
• Server failure is handled transparently
• Pluggable Storage Engines -- BDB-JE, MySQL,
• Pluggable serialization -- Protocol Buffers, Thrift,
Avro and Java Serialization
• Data items are versioned to maximize data
integrity in failure scenarios without
compromising availability of the system
• Each node is independent of other nodes with no
central point of failure or coordination
• Good single node performance: you can expect 10-
20k operations per second depending on the
machines, the network, the disk system, and the
data replication factor
• Support for pluggable data placement strategies to
support things like distribution across data
centers that are geographically far apart.
Voldemort Storage Engines
Trivial to integrate new persistence mechanisms
Conﬁg(data) & Storage Engine(servers info)
put(k, v), get(k), delete(k)
k is Versioned<Key>
• To enable high performance and availability it
allow only very simple key-value data access.
• Both keys and values can be complex compound
objects including lists or maps, but none-the-less
the only supported queries are effectively the
value = store.get(key)
• Voldemort supports hashtable semantics, so a
single value can be modified at a time and
retrieval is by primary key.
• This makes distribution across machines
particularly easy since everything can be split by
the primary key.
Consistent Hashing Mechanism
• In order to effectively Scaling , the data in
Voldemort is split-up in such a way that each item
is stored on multiple Servers.
• For retrieving data first figure out which is the
correct server to use. This partitioning is done via
a consistent hashing mechanism that let’s any
server calculate the location of data without doing
any expensive look ups
• Voldemort set an SLA (service level agreement)
for the requests and ban servers who cannot meet
their SLA (this could be because they are down,
because requests are timing out).
• Servers that violate this SLA get banned for a short
period of time, after which they attempt to restore
Dealing With Failure
Since each value is stored in multiple places it is
possible that one of these servers will not get
updated (say because it is crashed when the
To solve this problem Voldemort uses a data
versioning mechanism called Vector Clocks.
This data versioning allows the servers to detect
stale data when it is read and repair it.
Comparison to Hbase databases
Voldemort API calls
Durability Yes Yes
Locking model Optimistic Locking MVCC
Free for commercial use Yes Yes
Linkdin founder of voldemort
shopping cart used by Gilt Groupe, which is
powered by Voldemort.
High-Performance Key-Value Store (Amazon
treats the key‐value store as an API and adds an
in‐memory caching layer, which means that you
can plug into the back end that makes the most
sense for your particular needs.
• only efficient queries are possible, very predictable
• easy to distribute across a cluster.
• clean separation of storage and logic.
• The storage layer is completely mockable so
development and unit testing can be done against a
throw-away in-memory storage system without
needing a real cluster.
• no complex query filters
• no foreign key constraints
• no triggers
• No built-in support for “multiple data center”-
aware routing (there must be 1 copy of each key in
at least one data center)
• It is basically just a big, distributed, persistent,
fault-tolerant hash table.
• The redundancy of storage makes the system
more resilient to server failure. Since each value is
stored N times, you can tolerate as many as N –
1 machine failures without data loss.