Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Elixir

192 views

Published on

A brief introduction to the problems that can arise when building a distributed system and the libraries that can be solved.

Published in: Software
  • Be the first to comment

Elixir

  1. 1. ELIXIR ELIXIR AND DISTRIBUTION COMPUTING
  2. 2. 2012 2017 Elixir 1.5.x
  3. 3. FUNCTION A B ELIXIR FUNCTIONAL PROGRAMMING
  4. 4. FUNCTION FUNCTION FUNCTION
  5. 5. ELIXIR PIPE OPERATOR
  6. 6. ACTOR MODEL ACTOR MAIL BOX CODE DATA PROCESS ACTOR MAIL BOX CODE DATA PROCESS message
  7. 7. ERALNG
  8. 8. 1. ERLANG 2. BEAM (ERLANG VM) 3. ERLANG OTP 4. ERLANG CLUSTER
  9. 9. ERLANG
  10. 10. 1986 1998 2017 Open source Erlang/OTP 20.x Ericsson
  11. 11. 1986 1998 2017 Open source Erlang/OTP 20.x Ericsson 1995 Java
  12. 12. SWITCH SWITCH
  13. 13. SERVER SERVER CLIENT CLIENT CLIENT CLIENT CLIENT
  14. 14. BEAM (ERLANG VM)
  15. 15. https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html SCHEDULER SCHEDULING
  16. 16. https://tkowal.wordpress.com/2015/01/27/the-unintuitive-latency-over-throughput-problem/?utm_content=buffer17ab0&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer SCHEDULER PREEMPTIVE SCHEDULER
  17. 17. https://viethip.com/category/erlang/ GARBAGE COLLECTOR
  18. 18. ERLANG OTP
  19. 19. pen elecom latformO T P
  20. 20. OTP BEHAVIOR ▸ gen_server ▸ supervisor ▸ gen_fsm (old) ▸ getn_statem (new) ▸ gen_event
  21. 21. OTP BEHAVIOR ▸ gen_server ▸ supervisor ▸ gen_fsm (old) ▸ getn_statem (new) ▸ gen_event
  22. 22. GENSERVER GENERIC SERVER SERVER CLIENT CLIENT request - sync response - sync request - async response - async
  23. 23. SUPERVISOR SUPERVISOR WORKER WORKER WORKER
  24. 24. SUPERVISOR SUPERVISOR WORKER WORKER WORKER
  25. 25. SUPERVISOR SUPERVISOR WORKER WORKER WORKER WORKER
  26. 26. SUPERVISOR - SUPERVICSON TREES SUPERVISOR WORKER WORKER SUPERVISOR
  27. 27. ERLANG CLUSTER
  28. 28. ERLANG CLUSTER NODE NODE
  29. 29. ERLANG CLUSTER NODE NODE CLUSTER
  30. 30. CLUSTER ERLANG CLUSTER NODE#1 NODE#2
  31. 31. CLUSTER ERLANG CLUSTER NODE NODE NEW spawn
  32. 32. ERLANG CLUSTER NODE NODE NODE
  33. 33. ERLANG CLUSTER DEMO
  34. 34. DISTRIBUTED SYSTEM
  35. 35. 1. LOAD BALANCING 2. HIGH AVAILABILITY 3. RELIABILITY
  36. 36. LOAD BALANCING
  37. 37. LOAD BALANCING NODE CLIENT
  38. 38. LOAD BALANCING NODE CLIENT
  39. 39. LOAD BALANCING NODE CLIENT NODE NODE LOADBALNANCER OLTP
  40. 40. LOAD BALANCING BUSINESS NODE CLIENT BUSINESS NODE BUSINESS NODE LOADBALNANCER ENPOINT
 NODE ENPOINT
 NODE ENPOINT
 NODE QUEUE OLAP
  41. 41. HOW TO STATEFUL
  42. 42. LOAD BALANCING BUSINESS NODE BUSINESS NODE BUSINESS NODE HAPROXY CLIENTCLIENTCLIENTCLIENTCLIENTCLIENT CLIENTCLIENTCLIENTCLIENTCLIENTCLIENT CLIENTCLIENTCLIENTCLIENTCLIENT CLIENT A Message to CLIENT A STATEFUL
  43. 43. LOAD BALANCING BUSINESS NODE BUSINESS NODE BUSINESS NODE HAPROXY CLIENTCLIENTCLIENTCLIENTCLIENTCLIENT CLIENTCLIENTCLIENTCLIENTCLIENTCLIENT CLIENTCLIENTCLIENTCLIENTCLIENT CLIENT A Message to CLIENT ASTATEFUL
  44. 44. LOAD BALANCING BUSINESS NODE BUSINESS NODE BUSINESS NODE CLIENTCLIENTCLIENTCLIENTCLIENTCLIENT CLIENTCLIENTCLIENTCLIENTCLIENTCLIENT CLIENTCLIENTCLIENTCLIENTCLIENT CLIENT A Message to CLIENT ASTATEFUL ENPOINT NODE
  45. 45. HIGH AVAILABILITY
  46. 46. High availability refers to systems with very low downtime, software maintenance and upgrades included. - Desinging for scalabilit with Erlang/OTP
  47. 47. FAULT TOLERANCE WEB LOGIC CLIENT WEB LOGIC CLIENT #1 #2 Error / Timeout Error response
  48. 48. FAULT TOLERANCE WEB ACTOR LOGIC ACTOR ERROR monitor(logic_actor_pid)
  49. 49. FAULT TOLERANCE WEB ACTOR LOGIC ACTOR ERROR {'DOWN', MonitorRef, Type, Object, Info}
  50. 50. FAULT TOLERANCE WEB ACTOR LOGIC ACTOR TIMEOUT GenServer.call(logic_actor_pid, message, 5000)
  51. 51. FAULT TOLERANCE WEB ACTOR LOGIC ACTOR TIMEOUT {:error, timeout}
  52. 52. RESILIENCE WEB LOGIC CLIENT WEB LOGIC CLIENT #1 #2 Error response
  53. 53. RESILIENCE SUPERVISOR WORKER WORKER WORKER SUPERVISOR
  54. 54. HOT UPGRADES ACTOR CODE DATA PROCESS
  55. 55. HOT UPGRADES ACTOR CODE DATA PROCESS NEW CODE
  56. 56. DEMO HOT UPGRADES
  57. 57. RELIABILITY
  58. 58. SINGLE POINT OF FAILURE
  59. 59. ENDPOINT NODE CLIENT LOGIC NODE RELIABILITY SINGLE POINT OF FAILURE
  60. 60. ENDPOINT NODE LOADBALNANCER ENDPOINT NODE LOGIC NODE LOGIC NODE RELIABILITY … CLIENT
  61. 61. ENDPOINT NODE CLIENT LOADBALNANCER ENDPOINT NODE LOGIC NODE LOGIC NODE RELIABILITY …
  62. 62. ENDPOINT NODE CLIENT LOADBALNANCER ENDPOINT NODE LOGIC NODE LOGIC NODE RELIABILITY … Sharing data
  63. 63. RELIABILITY CAP THEOREM
  64. 64. RELIABILITY CAP THEOREM
  65. 65. RELIABILITY CAP THEOREM CONSISTENCY AVAILABILITY PARTITION TOLERANCE
  66. 66. RELIABILITY CAP THEOREM NODE NODE NODE NODE CLIENT CLIENT CLIENT CLIENT
  67. 67. RELIABILITY CONSISTENCY NODE NODE NODE NODE CLIENT CLIENT CLIENT CLIENT
  68. 68. RELIABILITY AVAILABILITY NODE NODE NODE NODE CLIENT CLIENT CLIENT CLIENT
  69. 69. RELIABILITY AVAILABILITY NODE NODE NODE NODE CLIENT CLIENT CLIENT CLIENT
  70. 70. RELIABILITY AVAILABILITY NODE NODE NODE NODE CLIENT CLIENT CLIENT CLIENT
  71. 71. DISTRIBUTED SYSTEM PROBLEM
  72. 72. https://www.youtube.com/watch?v=hwBuR0UCwYA
  73. 73. 1. CONSENSUS PROBLEM 2. DETERMINE ORDER OF EVENTS 3. SHARE DATA CONSISTENTLY
  74. 74. CONSENSUS PROBLEM
  75. 75. CONSENSUS PROBLEM BYZANTINE GENERALS PROBLEM
  76. 76. CONSENSUS PROBLEM NODE PAXOS CLUSTER NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE APPLE
  77. 77. CONSENSUS PROBLEM NODE PAXOS CLUSTER NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE APPLE
  78. 78. CONSENSUS PROBLEM NODE PAXOS CLUSTER NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE APPLE
  79. 79. CONSENSUS PROBLEM NODE PAXOS CLUSTER NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE APPLE
  80. 80. CONSENSUS PROBLEM NODE PAXOS CLUSTER NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE NODE APPLE
  81. 81. CONSENSUS PROBLEM PAXOS-SCENARIO#1 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner
  82. 82. CONSENSUS PROBLEM PAXOS-SCENARIO#1 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner Prepare Request
  83. 83. CONSENSUS PROBLEM PAXOS-SCENARIO#1 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner Prepare response
  84. 84. CONSENSUS PROBLEM PAXOS-SCENARIO#1 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner Accept request
  85. 85. CONSENSUS PROBLEM PAXOS-SCENARIO#1 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner Accepted
  86. 86. CONSENSUS PROBLEM PAXOS-SCENARIO#2 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner
  87. 87. CONSENSUS PROBLEM PAXOS-SCENARIO#2 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner req [n=2, v=4] req [n=4, v=8] [n=2, v=4] [n=2, v=4] [n=4, v=8] [n=4, v=8] [n=4, v=8]
  88. 88. CONSENSUS PROBLEM PAXOS-SCENARIO#2 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner [n=2, v=4] [n=2, v=4] [n=4, v=8] [n=4, v=8] [n=4, v=8] res [] res []res [n=2, v=4]
  89. 89. CONSENSUS PROBLEM PAXOS-SCENARIO#2 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner [n=2, v=4] [n=2, v=4] [n=4, v=8] [n=4, v=8] [n=4, v=8] accept req [n=4, v=8] [n=4, v=8] [n=4, v=8] [n=4, v=8]
  90. 90. CONSENSUS PROBLEM PAXOS-SCENARIO#2 Proposer A Proposer B Acceptors A Acceptors B Acceptors D Learner accepted [v=8] accepted [v=8] accepted [v=8]
  91. 91. CONSENSUS PROBLEM RIAK_ENSEMBLE
  92. 92. DETERMINE ORDER OF EVENTS
  93. 93. DETERMINE ORDER OF EVENTS VECTOR CLOCK Node 1 Node 2 Node 3 [0, 0, 1] [0, 0, 2] [0, 0, 3] [1, 0, 0] [1, 1, 0] [1, 2, 3] [2, 2, 3]
  94. 94. DETERMINE ORDER OF EVENTS VECTOR CLOCK [1, 0, 0] Event X [2, 2, 0] Event Y <=
  95. 95. DETERMINE ORDER OF EVENTS VECTOR CLOCK [1, 0, 0] Event X [0, 2, 0] Event Y ?
  96. 96. DETERMINE ORDER OF EVENTS RIACK_CORE - DVVS (Dotted version vectors)
  97. 97. DETERMINE ORDER OF EVENTS RIACK_CORE - DVVS (Dotted version vectors)
  98. 98. DETERMINE ORDER OF EVENTS RIACK_CORE - DVVS (Dotted version vectors)
  99. 99. DETERMINE ORDER OF EVENTS RIACK_CORE - DVVS (Dotted version vectors)
  100. 100. SHARE DATA CONSISTENTLY
  101. 101. SHARE DATA CONSISTENTLY NODE
  102. 102. SHARE DATA CONSISTENTLY NODE
  103. 103. SHARE DATA CONSISTENTLY NODE
  104. 104. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING NODE#2 NODE#3 NODE#1
  105. 105. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING NODE#2 NODE#3 NODE#1 NODE#4
  106. 106. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING NODE#2 NODE#3 NODE#1 NODE#4
  107. 107. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING NODE#2 NODE#3 NODE#1 NODE#4 NODE#1 NODE#2 NODE#1 NODE#3 NODE#4 NODE#2 NODE#3 NODE#1
  108. 108. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING - N NODE#2 NODE#3 NODE#1 NODE#4 NODE#1 NODE#2 NODE#1 NODE#3 NODE#4 NODE#2 NODE#3 NODE#1 N = 3
  109. 109. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING - W NODE#2 NODE#3 NODE#1 NODE#4 NODE#1 NODE#2 NODE#1 NODE#3 NODE#4 NODE#2 NODE#3 NODE#1 W = N/2+1
  110. 110. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING - R NODE#2 NODE#3 NODE#1 NODE#4 NODE#1 NODE#2 NODE#1 NODE#3 NODE#4 NODE#2 NODE#3 NODE#1 R = N/2+1
  111. 111. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING NODE#1 NODE#2 NODE#3 NODE#4 VNODE#1 VNODE#2 VNODE#3 VNODE#4 VNODE#5 VNODE#6 VNODE#7 VNODE#8 hash(id)
  112. 112. SHARE DATA CONSISTENTLY CONSISTENT HASHING RING
  113. 113. EASE OF ADMINISTRATION AND MAINTERANCE
  114. 114. MAINTERANCE
  115. 115. ADMINISTRATION OBSERVER
  116. 116. REFERENCE ▸ scheduler ▸ https://tkowal.wordpress.com/2015/01/27/the-unintuitive-latency-over-throughput-problem/? utm_content=buffer17ab0&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer ▸ paxos ▸ https://www.youtube.com/watch?v=ITstwAQYYag ▸ https://angus.nyc/2012/paxos-by-example/ ▸ vector clock ▸ https://8thlight.com/blog/rylan-dirksen/2013/10/04/synchronization-in-a-distributed-system.html ▸ other ▸ http://erlang.org/doc/design_principles/release_handling.html ▸ http://blog.plataformatec.com.br/2017/07/the-fallacies-of-web-application-performance/? utm_content=buffer99d6d&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer ▸ https://www.youtube.com/watch?v=hwBuR0UCwYA&t=2185s

×