Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cook your KV

787 views

Published on

In this talk we will try to create an in-memory distributed KV store using riak_core library.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Cook your KV

  1. 1. Cook your KV How to create your in memory KV store
  2. 2. GPad Born to be a developer with an interest in distributed system. I develop using many languages like C++, C#, js and ruby. I had fallen in love with functional programming, especially with elixir, erlang. Twitter: https://twitter.com/gpad619 Github: https://github.com/gpad/ Medium: https://medium.com/@gpad CTO & founder of coders51
  3. 3. Schedule What is riak_core? How to implement a simple distributed command How to implement a command that involves all nodes If we had more time, we would talk about: What happens when I remove a node? How to handle a node crash? Conflict-resolution http://github.com/gpad/no_slides http://medium.com/@GPad/
  4. 4. What is riak_core? From readme page on github: Riak Core is the distributed systems framework that forms the basis of how Riak distributes data and scales. More generally, it can be thought of as a toolkit for building distributed, scalable, fault-tolerant applications.
  5. 5. What is riak_core? DO It manages the addition and removal of nodes from the cluster It manages how the requests should be routed inside the cluster It manages how to upgrade cluster having nodes with various capabilities It manages information cluster-wide etc ... DON’T It doesn’t help to store data It isn’t DB centric It doesn’t care what type of data do you use (more or less) It’ doesn’t care what type of computation you do ... etc ...
  6. 6. How does a riak_core cluster work? Every node is equivalent (masterless) Any client can connect to any node of the cluster The node executes the work or forwards it to correct node Consistent Hashing / VNode
  7. 7. How to partition data
  8. 8. How to partition data Node 1Physical node Virtual node
  9. 9. How to partition data Every virtual node is an Erlang process Every virtual node is an gen_fsm Riak_core defines a behaviour to implement the vnode
  10. 10. Riak_core_vnode behaviour Define the functions that a module should implement How to manage a command How to manage a coverage command Etc ...
  11. 11. Implement a distributed command
  12. 12. Implement a distributed command
  13. 13. Implement a distributed command
  14. 14. Implement a distributed command
  15. 15. Implement a distributed command
  16. 16. Implement a distributed command
  17. 17. Implement a distributed command
  18. 18. Distributed command - Demo!
  19. 19. All nodes Can we get only one key at time from the cluster? Is it possible to execute a command on all nodes? What do you mean for all nodes?
  20. 20. All nodes
  21. 21. All nodes
  22. 22. All nodes
  23. 23. All nodes
  24. 24. All nodes
  25. 25. All nodes
  26. 26. All nodes
  27. 27. All nodes - VNode
  28. 28. All nodes - Service
  29. 29. All nodes - Supervisor
  30. 30. All nodes - Demo!
  31. 31. What we have done... Create an in-memory KV store that: Can be used from every nodes Can execute a command on a specific node Can execute a command on the entire cluster
  32. 32. What we have done... Main concepts: Consistent hashing Virtual Node vs Physical Node Preference List
  33. 33. What we have done... Next concepts: Handoff - Manage the addition and removal of nodes Manage the crash of a node through replications How to use conflict-resolution strategy http://github.com/gpad/no_slides http://medium.com/@GPad/
  34. 34. Thank you!
  35. 35. Q&A

×