Project Voldemort<br />Whenrelationdatabaseis not enough (too much?)<br />
Tomasz Nurkiewicz<br />Graduated Warsaw University Of Technology (Faculty of Electronics and Information Technology)<br />...
WhatisVoldemort not?<br />Relationaldatabase<br />Alternative API to RDB<br />Distributedcache<br />
...so?<br />Key-valuedatabase<br />„PersistentHashMap”<br />Keywords:<br />Clusterready<br />Scalability (no master)<br />...
Voldemortvs. HashMap<br />Partitionsvs. hash-buckets<br />Whereis my stuff? – clientsiderouting<br />0<br />4<br />8<br />...
API – StoreClient<K, V><br />getValue(K key): V<br />getAll(Iterable<K> keys): Map<K, V><br />put(K key, V value)<br />del...
Voldemortcluster<br />DEMO<br />node1<br />node2<br />node3<br />node4<br />localhost:6101<br />localhost:6102<br />localh...
Connection (One to rulethemall)<br />ClientConfigconfig = newClientConfig();<br />config.setBootstrapUrls("tcp://server:61...
Storeconfiguration (stores.xml)<br />N (<replication-factor>)<br />R (<required-reads>)<br />W (<required-writes>)<br />Se...
Clusterconfiguration (cluster.xml)<br /><cluster><br />	<name>Usercluster</name><br />	<server><br />		<id>1</id><br />		<...
Rebalancing<br />0<br />4<br />8<br />12<br />16<br />node1<br />1<br />5<br />9<br />13<br />17<br />node2<br />2<br />6<...
Facts of life (LinkedInand Gilt)<br /><ul><li>Uptimemeasuredinmonths (Gilt Groupe)
Last Thursday we lost a server and no-one noticed (as above)</li></li></ul><li>Transactional/consistent?<br />Logic time (...
Whatelse?<br />JMX (bothclient and server)<br />Views (non-materialized)<br />HTTP interface (not REST though)<br />Hinted...
Happy clustering!<br />http://project-voldemort.com<br />http://www.infoq.com/presentations/Project-Voldemort-Scaling-Simp...
Upcoming SlideShare
Loading in …5
×

Project voldemort - When relation database is not enough (too much?)

6,567 views

Published on

Published in: Technology, News & Politics
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,567
On SlideShare
0
From Embeds
0
Number of Embeds
3,680
Actions
Shares
0
Downloads
56
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Project voldemort - When relation database is not enough (too much?)

  1. 1. Project Voldemort<br />Whenrelationdatabaseis not enough (too much?)<br />
  2. 2. Tomasz Nurkiewicz<br />Graduated Warsaw University Of Technology (Faculty of Electronics and Information Technology)<br />SCJP, SCJD, SCWCD, SCBCD<br />Brown belt on JavaBlackBelt.com<br />Fewcommitsin:<br />Commons Lang, Logback, Mule ESB<br />Currentlyworking as a developer atJAVART<br />http://nurkiewicz.blogspot.com<br />
  3. 3. WhatisVoldemort not?<br />Relationaldatabase<br />Alternative API to RDB<br />Distributedcache<br />
  4. 4. ...so?<br />Key-valuedatabase<br />„PersistentHashMap”<br />Keywords:<br />Clusterready<br />Scalability (no master)<br />Efficient (partitioning)<br />High availability<br />„Less than” relationdatabase<br />
  5. 5. Voldemortvs. HashMap<br />Partitionsvs. hash-buckets<br />Whereis my stuff? – clientsiderouting<br />0<br />4<br />8<br />12<br />16<br />node1<br />1<br />5<br />9<br />13<br />17<br />node2<br />2<br />6<br />10<br />14<br />18<br />node3<br />3<br />7<br />11<br />15<br />19<br />node4<br />
  6. 6. API – StoreClient<K, V><br />getValue(K key): V<br />getAll(Iterable<K> keys): Map<K, V><br />put(K key, V value)<br />delete(K key): boolean<br />getResponsibleNodes(K key): List<Node><br />… <br />extendsjava.util.Map<K, V>?<br />
  7. 7. Voldemortcluster<br />DEMO<br />node1<br />node2<br />node3<br />node4<br />localhost:6101<br />localhost:6102<br />localhost:6103<br />localhost:6104<br />
  8. 8. Connection (One to rulethemall)<br />ClientConfigconfig = newClientConfig();<br />config.setBootstrapUrls("tcp://server:6101");<br />SocketStoreClientFactoryfactory = newSocketStoreClientFactory(config);<br />StoreClient<K, V> client = factory.getStoreClient("users");<br />No replication – efficient, but risky<br />
  9. 9. Storeconfiguration (stores.xml)<br />N (<replication-factor>)<br />R (<required-reads>)<br />W (<required-writes>)<br />Serialization<br />JSON – onlyprimitives, dates and collections<br />protobuf – (Google)<br />String (e.g. XML)<br />Java (Serializable)<br />
  10. 10. Clusterconfiguration (cluster.xml)<br /><cluster><br /> <name>Usercluster</name><br /> <server><br /> <id>1</id><br /> <host>192.168.0.17</host><br /> <socket-port>6101</socket-port><br /> <partitions>0, 4, 8, 12, 16</partitions><br /> </server><br />Persistency: MySQL, Berkeley DB, RAM<br />
  11. 11. Rebalancing<br />0<br />4<br />8<br />12<br />16<br />node1<br />1<br />5<br />9<br />13<br />17<br />node2<br />2<br />6<br />10<br />14<br />18<br />node3<br />3<br />7<br />11<br />15<br />19<br />node4<br />node5<br />
  12. 12. Facts of life (LinkedInand Gilt)<br /><ul><li>Uptimemeasuredinmonths (Gilt Groupe)
  13. 13. Last Thursday we lost a server and no-one noticed (as above)</li></li></ul><li>Transactional/consistent?<br />Logic time (vectorclock)<br />Conflict resolution duringread<br />Optimisticlocking<br />putIfNotObsolete(K key, Versioned<V> versioned)<br />applyUpdate(UpdateAction<K, V> action)<br />ObsoleteVersionException<br />R+W>N<br />
  14. 14. Whatelse?<br />JMX (bothclient and server)<br />Views (non-materialized)<br />HTTP interface (not REST though)<br />Hintedhandoff<br />
  15. 15. Happy clustering!<br />http://project-voldemort.com<br />http://www.infoq.com/presentations/Project-Voldemort-Scaling-Simple-Storage<br />http://www.infoq.com/presentations/Project-Voldemort-at-Gilt-Groupe<br />http://behemoth.strlen.net/~alex/voldemort-nosql_live.ppt<br />http://www.slideshare.net/hadoopusergroup/hadoop-and-voldemort-linkedin<br />http://www.vineetgupta.com/2010/01/nosql-databases-part-1-landscape.html<br />http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores<br />

×