Lock Service with Paxos in Erlang

1,163 views
1,003 views

Published on

Overview of my implementation of a wannabe highly available lock service over Paxos algorithm in Erlang

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,163
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Lock Service with Paxos in Erlang

  1. 1. Motivation Implementation Evaluation Conclusions . . . Lock Service with Paxos .. . . Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Supervisor: Johan Montelius C CC BY: EMDC KTH December 5, 2012 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 1/19 CC BY:
  2. 2. Motivation Implementation Evaluation Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance & Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 2/19 CC BY:
  3. 3. Motivation Implementation Evaluation Conclusions. Paxos . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 3/19 CC BY:
  4. 4. Motivation Implementation Evaluation Conclusions. Paxos . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 3/19 CC BY:
  5. 5. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
  6. 6. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
  7. 7. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
  8. 8. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders Paxos round number = Slot number . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
  9. 9. Motivation Implementation Evaluation Conclusions. From Paxos to a Lock Service Not Trivial Paxos Replicated State Machine Slot queue → Sequence of lock holders Paxos round number = Slot number Paxos decides who gets each slot . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 4/19 CC BY:
  10. 10. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance & Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 5/19 CC BY:
  11. 11. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Abstract View . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 6/19 CC BY:
  12. 12. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Detailed View . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 7/19 CC BY:
  13. 13. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  14. 14. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  15. 15. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  16. 16. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  17. 17. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  18. 18. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  19. 19. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  20. 20. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  21. 21. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  22. 22. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Use Case . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 8/19 CC BY:
  23. 23. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Process Fault Tolerance Supervisor Tree Process restarts aware that it has crashed . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 9/19 CC BY:
  24. 24. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Paxos Fault Tolerance Acceptor persists asynchronously its state . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:
  25. 25. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Paxos Fault Tolerance Acceptor persists asynchronously its state Learner persists asynchronously the slot queue . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:
  26. 26. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Paxos Fault Tolerance Acceptor persists asynchronously its state Learner persists asynchronously the slot queue Assignor persists asynchronously its state . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 10/19 CC BY:
  27. 27. Motivation Implementation Architecture Evaluation Fault Tolerance & Optimizations Conclusions. Optimizations @COMPLETED: Receiver handles the huge amount of requests Shared Queue between Receiver and Proposer Learner and Acceptor spawn separate writing process . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 11/19 CC BY:
  28. 28. Motivation Implementation Architecture Evaluation Fault Tolerance Optimizations Conclusions. Optimizations @COMPLETED: Receiver handles the huge amount of requests Shared Queue between Receiver and Proposer Learner and Acceptor spawn separate writing process @TODO: Propose lock requests in batches Leader Garbage Collection . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 11/19 CC BY:
  29. 29. Motivation Implementation Central Lock Evaluation Experiments Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 12/19 CC BY:
  30. 30. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 13/19 CC BY:
  31. 31. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Latency . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 14/19 CC BY:
  32. 32. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Availability # of Paxos Nodes VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 15/19 CC BY:
  33. 33. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Availability # of Paxos Nodes VS Latency and/or Throughput Will look like: . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 15/19 CC BY:
  34. 34. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Fault Tolerance Latency of Node Recovery VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 16/19 CC BY:
  35. 35. Motivation Implementation Central Lock Evaluation Experiments Conclusions. Cost of Fault Tolerance Latency of Node Recovery VS Latency and/or Throughput Will look like: . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 16/19 CC BY:
  36. 36. Motivation Implementation Central Lock Evaluation Experiments Conclusions. More Proposer Timeouts VS Latency and/or Throughput Paxos +/-Persistence VS Latency and/or Throughput . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 17/19 CC BY:
  37. 37. Motivation Implementation Evaluation Conclusions . . . Motivation 1 . . . Implementation 2 Architecture Fault Tolerance Optimizations . . . Evaluation 3 Central Lock Experiments . . . Conclusions 4 . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 18/19 CC BY:
  38. 38. Motivation Implementation Evaluation Conclusions Find golden ratio: Performance vs Availability Find golden ratio: Performance vs Fault Tolerance Their combination? . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 19/19 CC BY:
  39. 39. Motivation Implementation Evaluation Conclusions Find golden ratio: Performance vs Availability Find golden ratio: Performance vs Fault Tolerance Their combination? Questions ? . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 19/19 CC BY:
  40. 40. Part I . . Appendix . .. . . . . . . . . CPresenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 20/19 CC BY:
  41. 41. . Benchmarking Tool Basho bench Ability to write custom clients Can simulate network dispatcher process . . . . . . C Presenter: Emmanouil Dimogerontakis Teammate: Muhammet Orazov Service with Johan Montelius Lock Supervisor: Paxos 1/1 CC BY:

×