Project Voldemort is a distributed key-value store developed by LinkedIn for high scalability, featuring automatic data replication, partitioning, and tunable consistency to handle server failures efficiently. It offers a simple key-value access model and supports various storage engines and data serialization formats, making it suitable for high-performance applications. The system is resilient to server failures due to its design, allowing for multiple data copies and data versioning to maintain data integrity.

1. Project Voldemort
A distributed database.
Presented To:
Sir Tariq Mehmood Presented By:
Fasiha Ikram
Aniqa Naeem

2. Voldemort
Voldemort is a distributed data store that is
designed as a key-value store used by
LinkedIn for high-scalability storage.
It is named after the fictional Harry Potter
villain Lord Voldemort.

3. there are job titles, job openings for people, Groups
and companies that offerings jobs.
Big
Data
Variety
velocity
volum
e
Need frequently read write
LinkedIn Big Data Problem

4. Voldemort Scale Both
• The amount of data we can store (write)
• The number of requests for that data (read)

5. Why Not Using Hadopp
Naturally the only way to do this is to spread both
the load and the data across many servers.
1. Need to find a way to split the data so that all
servers have different data
2. Need to find a way to handle server failures
without interrupting service
3. HBase still be write-heavy (due to horizontal
partitioning and use of SSTables, which are write
optimized)

6. Why voldemort
• Data is automatically replicated over multiple
servers.
• Data is automatically partitioned so each server
contains only a subset of the total data
• Provides tunable consistency (strict quorum or
eventual consistency)
• Server failure is handled transparently
• Pluggable Storage Engines -- BDB-JE, MySQL,
Read-Only

7. Why voldemort
• Pluggable serialization -- Protocol Buffers, Thrift,
Avro and Java Serialization
• Data items are versioned to maximize data
integrity in failure scenarios without
compromising availability of the system
• Each node is independent of other nodes with no
central point of failure or coordination

8. Why voldemort
• Good single node performance: you can expect 10-
20k operations per second depending on the
machines, the network, the disk system, and the
data replication factor
• Support for pluggable data placement strategies to
support things like distribution across data
centers that are geographically far apart.

9. Voldemort Storage Engines
 Trivial to integrate new persistence mechanisms
with Voldemort
 2 Classes:
Config(data) & Storage Engine(servers info)
 3 Operations:
put(k, v), get(k), delete(k)
 Complication:
k is Versioned<Key>

10. Architecture of Queue

13. Key-Value Storage
• To enable high performance and availability it
allow only very simple key-value data access.
• Both keys and values can be complex compound
objects including lists or maps, but none-the-less
the only supported queries are effectively the
following:
value = store.get(key)
store.put(key, value)
store.delete(key)

14. Query execution
• Voldemort supports hashtable semantics, so a
single value can be modified at a time and
retrieval is by primary key.
• This makes distribution across machines
particularly easy since everything can be split by
the primary key.

15. Consistent Hashing Mechanism
• In order to effectively Scaling , the data in
Voldemort is split-up in such a way that each item
is stored on multiple Servers.
• For retrieving data first figure out which is the
correct server to use. This partitioning is done via
a consistent hashing mechanism that let’s any
server calculate the location of data without doing
any expensive look ups

16. Detecting Failure
• Voldemort set an SLA (service level agreement)
for the requests and ban servers who cannot meet
their SLA (this could be because they are down,
because requests are timing out).
• Servers that violate this SLA get banned for a short
period of time, after which they attempt to restore
them.

17. Dealing With Failure
 Since each value is stored in multiple places it is
possible that one of these servers will not get
updated (say because it is crashed when the
update occurs).
 To solve this problem Voldemort uses a data
versioning mechanism called Vector Clocks.
 This data versioning allows the servers to detect
stale data when it is read and repair it.

18. Comparison to Hbase databases
Query
language
Architecture
Database
Model
Replication Issues
Voldemort API calls
Big Unordered
Map
Key-value
NoSQL
Distributed
data
structure
Topology
Aware
Routing
Strategies
Not
Satisfyin
g ACID
Properti
es.
Hbase
API calls
REST
XML
Thrift
Big Multi-
dimensional
Sorted Map
HDFS
Master-
slave/Master
-master
replication
Master
Slave
Which Is
Not
Highly
Availabl
e

19. Acid Properties
Voldemort Hbase
Atomicity
Yes Yes
Consistency
Yes Yes
Isolation
No Yes
Durability Yes Yes
Locking model Optimistic Locking MVCC
Free for commercial use Yes Yes

20. Industry Implementation
 Linkdin founder of voldemort
 shopping cart used by Gilt Groupe, which is
powered by Voldemort.

21. Use Case
 High-Performance Key-Value Store (Amazon
Dynamo clone)
 treats the key‐value store as an API and adds an
in‐memory caching layer, which means that you
can plug into the back end that makes the most
sense for your particular needs.

22. Pros
• only efficient queries are possible, very predictable
performance.
• easy to distribute across a cluster.
• clean separation of storage and logic.
• The storage layer is completely mockable so
development and unit testing can be done against a
throw-away in-memory storage system without
needing a real cluster.

23. Cons
• no complex query filters
• no foreign key constraints
• no triggers
• No built-in support for “multiple data center”-
aware routing (there must be 1 copy of each key in
at least one data center)

24. Conclusion
• It is basically just a big, distributed, persistent,
fault-tolerant hash table.
• The redundancy of storage makes the system
more resilient to server failure. Since each value is
stored N times, you can tolerate as many as N –
1 machine failures without data loss.

25. Refrences
1. http://blog.linkedin.com/2009/04/01/project-voldemort-part-ii-how-it-works/
2. http://blog.linkedin.com/2009/03/20/project-voldemort-scaling-simple-storage-at-
linkedin/
3. http://en.wikipedia.org/wiki/Voldemort_(distributed_data_store)
4. http://highscalability.com/product-project-voldemort-distributed-database
5. http://www.dummies.com/how-to/content/using-pluggable-storage-with-nosql.html
6. http://vschart.com/compare/project-voldemort/vs/hbase
7. http://www.project-voldemort.com/voldemort/design.html