Lock Service with Paxos in Erlang
•
2 likes•1,304 views
Overview of my implementation of a wannabe highly available lock service over Paxos algorithm in Erlang
Recommended
More Related Content
More from Save Manos
More from Save Manos (14)
Recently uploaded
Recently uploaded (20)
Lock Service with Paxos in Erlang
- 1. Motivation Implementation Evaluation Conclusions . . . Lock Service with Paxos .. . . Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Supervisor: Johan Montelius C CC BY: EMDC KTH December 5, 2012 . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 1/19 CC BY:
- 2. Motivation Implementation Evaluation Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance & Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 2/19 CC BY:
- 3. Motivation Implementation Evaluation Conclusions . Paxos . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 3/19 CC BY:
- 4. Motivation Implementation Evaluation Conclusions . Paxos . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 3/19 CC BY:
- 5. Motivation Implementation Evaluation Conclusions . From Paxos to a Lock Service Not Trivial . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
- 6. Motivation Implementation Evaluation Conclusions . From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
- 7. Motivation Implementation Evaluation Conclusions . From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
- 8. Motivation Implementation Evaluation Conclusions . From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders Paxos round number = Slot number . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
- 9. Motivation Implementation Evaluation Conclusions . From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders Paxos round number = Slot number Paxos decides who gets each slot . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
- 10. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance & Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 5/19 CC BY:
- 11. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Abstract View . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 6/19 CC BY:
- 12. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Detailed View . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 7/19 CC BY:
- 13. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 14. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 15. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 16. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 17. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 18. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 19. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 20. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 21. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 22. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
- 23. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Process Fault Tolerance Supervisor Tree Process restarts aware that it has crashed . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 9/19 CC BY:
- 24. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Paxos Fault Tolerance Acceptor persists asynchronously its state . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:
- 25. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Paxos Fault Tolerance Acceptor persists asynchronously its state Learner persists asynchronously the slot queue . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:
- 26. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Paxos Fault Tolerance Acceptor persists asynchronously its state Learner persists asynchronously the slot queue Assignor persists asynchronously its state . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:
- 27. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . Optimizations @COMPLETED: Receiver handles the huge amount of requests Shared Queue between Receiver and Proposer Learner and Acceptor spawn separate writing process . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 11/19 CC BY:
- 28. Motivation Implementation Architecture Evaluation Fault Tolerance Optimizations Conclusions . Optimizations @COMPLETED: Receiver handles the huge amount of requests Shared Queue between Receiver and Proposer Learner and Acceptor spawn separate writing process @TODO: Propose lock requests in batches Leader Garbage Collection . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 11/19 CC BY:
- 29. Motivation Implementation Central Lock Evaluation Experiments Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 12/19 CC BY:
- 30. Motivation Implementation Central Lock Evaluation Experiments Conclusions . Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 13/19 CC BY:
- 31. Motivation Implementation Central Lock Evaluation Experiments Conclusions . Latency . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 14/19 CC BY:
- 32. Motivation Implementation Central Lock Evaluation Experiments Conclusions . Cost of Availability # of Paxos Nodes VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 15/19 CC BY:
- 33. Motivation Implementation Central Lock Evaluation Experiments Conclusions . Cost of Availability # of Paxos Nodes VS Latency and/or Throughput Will look like: . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 15/19 CC BY:
- 34. Motivation Implementation Central Lock Evaluation Experiments Conclusions . Cost of Fault Tolerance Latency of Node Recovery VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 16/19 CC BY:
- 35. Motivation Implementation Central Lock Evaluation Experiments Conclusions . Cost of Fault Tolerance Latency of Node Recovery VS Latency and/or Throughput Will look like: . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 16/19 CC BY:
- 36. Motivation Implementation Central Lock Evaluation Experiments Conclusions . More Proposer Timeouts VS Latency and/or Throughput Paxos +/-Persistence VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 17/19 CC BY:
- 37. Motivation Implementation Evaluation Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 18/19 CC BY:
- 38. Motivation Implementation Evaluation Conclusions Find golden ratio: Performance vs Availability Find golden ratio: Performance vs Fault Tolerance Their combination? . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 19/19 CC BY:
- 39. Motivation Implementation Evaluation Conclusions Find golden ratio: Performance vs Availability Find golden ratio: Performance vs Fault Tolerance Their combination? Questions ? . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 19/19 CC BY:
- 40. Part I . . Appendix . .. . . . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 20/19 CC BY:
- 41. . Benchmarking Tool Basho bench Ability to write custom clients Can simulate network dispatcher process . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 1/1 CC BY: