Downloaded 12 times
Uploaded byGianluca Padovani
Cook your KV
Riak Core is a distributed systems framework that forms the basis of how Riak distributes data and scales across nodes. It manages the addition and removal of nodes, routes requests within the cluster, and manages information cluster-wide. The presenter demonstrated implementing a distributed command in Riak Core that can execute on a single node or all nodes in the cluster using consistent hashing, virtual nodes, and preference lists.
Cook your KV
- 1. Cook your KV Howto create your in memory KV store
- 2. GPad Born to bea developer with an interest in distributed system. I develop using many languages like C++, C#, js and ruby. I had fallen in love with functional programming, especially with elixir, erlang. Twitter: https://twitter.com/gpad619 Github: https://github.com/gpad/ Medium: https://medium.com/@gpad CTO & founder of coders51
- 3. Schedule What is riak_core? Howto implement a simple distributed command How to implement a command that involves all nodes If we had more time, we would talk about: What happens when I remove a node? How to handle a node crash? Conflict-resolution http://github.com/gpad/no_slides http://medium.com/@GPad/
- 4. What is riak_core? Fromreadme page on github: Riak Core is the distributed systems framework that forms the basis of how Riak distributes data and scales. More generally, it can be thought of as a toolkit for building distributed, scalable, fault-tolerant applications.
- 5. What is riak_core? DO Itmanages the addition and removal of nodes from the cluster It manages how the requests should be routed inside the cluster It manages how to upgrade cluster having nodes with various capabilities It manages information cluster-wide etc ... DON’T It doesn’t help to store data It isn’t DB centric It doesn’t care what type of data do you use (more or less) It’ doesn’t care what type of computation you do ... etc ...
- 6. How does ariak_core cluster work? Every node is equivalent (masterless) Any client can connect to any node of the cluster The node executes the work or forwards it to correct node Consistent Hashing / VNode
- 7. How to partitiondata
- 8. How to partitiondata Node 1Physical node Virtual node
- 9. How to partitiondata Every virtual node is an Erlang process Every virtual node is an gen_fsm Riak_core defines a behaviour to implement the vnode
- 10. Riak_core_vnode behaviour Define thefunctions that a module should implement How to manage a command How to manage a coverage command Etc ...
- 11. Implement a distributedcommand
- 12. Implement a distributedcommand
- 13. Implement a distributedcommand
- 14. Implement a distributedcommand
- 15. Implement a distributedcommand
- 16. Implement a distributedcommand
- 17. Implement a distributedcommand
- 18.
- 19. All nodes Can weget only one key at time from the cluster? Is it possible to execute a command on all nodes? What do you mean for all nodes?
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27. All nodes -VNode
- 28. All nodes -Service
- 29. All nodes -Supervisor
- 30. All nodes -Demo!
- 31. What we havedone... Create an in-memory KV store that: Can be used from every nodes Can execute a command on a specific node Can execute a command on the entire cluster
- 32. What we havedone... Main concepts: Consistent hashing Virtual Node vs Physical Node Preference List
- 33. What we havedone... Next concepts: Handoff - Manage the addition and removal of nodes Manage the crash of a node through replications How to use conflict-resolution strategy http://github.com/gpad/no_slides http://medium.com/@GPad/
- 34.
- 35.
Editor's Notes
- #19 On node 1 NoSlides.Service.put(:gpad, 42) NoSlides.Service.put(:joe, 1) NoSlides.Service.put(:asdf, 45) NoSlides.Service.put(:gino, 666) NoSlides.Service.get(:gpad) NoSlides.Service.get(:joe) On node 2: NoSlides.Service.get(:gpad) NoSlides.Service.get(:joe) Remove node … :riak_core.leave On node 1: NoSlides.Service.get(:gpad)