Lock Service with Paxos in Erlang

1. Motivation Implementation Evaluation Conclusions . . . Lock Service with Paxos .. . . Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Supervisor: Johan Montelius C CC BY: EMDC KTH December 5, 2012 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 1/19 CC BY:

2. Motivation Implementation Evaluation Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance & Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 2/19 CC BY:

3. Motivation Implementation Evaluation Conclusions. Paxos . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 3/19 CC BY:

4. Motivation Implementation Evaluation Conclusions. Paxos . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 3/19 CC BY:

5. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:

6. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:

7. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:

8. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders Paxos round number = Slot number . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:

9. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders Paxos round number = Slot number Paxos decides who gets each slot . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:

10. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance & Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 5/19 CC BY:

11. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Abstract View . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 6/19 CC BY:

12. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Detailed View . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 7/19 CC BY:

13. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:

23. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Process Fault Tolerance Supervisor Tree Process restarts aware that it has crashed . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 9/19 CC BY:

24. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Paxos Fault Tolerance Acceptor persists asynchronously its state . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:

25. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Paxos Fault Tolerance Acceptor persists asynchronously its state Learner persists asynchronously the slot queue . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:

26. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Paxos Fault Tolerance Acceptor persists asynchronously its state Learner persists asynchronously the slot queue Assignor persists asynchronously its state . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:

27. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Optimizations @COMPLETED: Receiver handles the huge amount of requests Shared Queue between Receiver and Proposer Learner and Acceptor spawn separate writing process . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 11/19 CC BY:

28. Motivation Implementation Architecture Evaluation Fault Tolerance Optimizations Conclusions. Optimizations @COMPLETED: Receiver handles the huge amount of requests Shared Queue between Receiver and Proposer Learner and Acceptor spawn separate writing process @TODO: Propose lock requests in batches Leader Garbage Collection . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 11/19 CC BY:

29. Motivation Implementation Central Lock Evaluation Experiments Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 12/19 CC BY:

30. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 13/19 CC BY:

31. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Latency . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 14/19 CC BY:

32. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Availability # of Paxos Nodes VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 15/19 CC BY:

33. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Availability # of Paxos Nodes VS Latency and/or Throughput Will look like: . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 15/19 CC BY:

34. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Fault Tolerance Latency of Node Recovery VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 16/19 CC BY:

35. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Fault Tolerance Latency of Node Recovery VS Latency and/or Throughput Will look like: . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 16/19 CC BY:

36. Motivation Implementation Central Lock Evaluation Experiments Conclusions. More Proposer Timeouts VS Latency and/or Throughput Paxos +/-Persistence VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 17/19 CC BY:

37. Motivation Implementation Evaluation Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 18/19 CC BY:

38. Motivation Implementation Evaluation Conclusions Find golden ratio: Performance vs Availability Find golden ratio: Performance vs Fault Tolerance Their combination? . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 19/19 CC BY:

39. Motivation Implementation Evaluation Conclusions Find golden ratio: Performance vs Availability Find golden ratio: Performance vs Fault Tolerance Their combination? Questions ? . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 19/19 CC BY:

40. Part I . . Appendix . .. . . . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 20/19 CC BY:

41. . Benchmarking Tool Basho bench Ability to write custom clients Can simulate network dispatcher process . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 1/1 CC BY:

Lock Service with Paxos in Erlang

Save Manos

Lock Service with Paxos in Erlang