Successfully reported this slideshow.
Your SlideShare is downloading. ×

Building Distributed Systems

Check these out next

1 of 87 Ad
1 of 87 Ad

Building Distributed Systems

Download to read offline

Presentation by Oleksiy Kurnenkov, OnApp.
During 7th #pivorak

Video from the MeetUp is available here https://www.youtube.com/watch?v=ENxqIvLEIck

Presentation by Oleksiy Kurnenkov, OnApp.
During 7th #pivorak

Video from the MeetUp is available here https://www.youtube.com/watch?v=ENxqIvLEIck

Advertisement
Advertisement

More Related Content

Advertisement

More from Pivorak MeetUp (20)

Advertisement

Building Distributed Systems

  1. 1. Building Distributed Systems decision guide
  2. 2. Oleksiy Kurnenkov Core Infrastructure Domain lead OnApp
  3. 3. Building Distributed Systems decision guide
  4. 4. Introduction Part I
  5. 5. Distributed system is a set of interconnected functional units with separate runtime context
  6. 6. MONOLITH : DISTRIBUTED 1 Runtime context Coroutines Threads Process N Runtime contexts Nodes Components Clusters Grids
  7. 7. SCALE Small -> Middle -> Global -> OS, LAN LAN WAN
  8. 8. TYPES CDN Grids Clouds Clusters Enterprise
  9. 9. CDN
  10. 10. Grid
  11. 11. Globus Toolkit 6 BOINC x-Torrent
  12. 12. Cloud
  13. 13. Cluster
  14. 14. Enterprise
  15. 15. TASKS MMOG Analysis Sensors Real Time Enterprise Media Storage Computation Transactions
  16. 16. Challenges Availability Scalability Security Openness Heterogeneity Concurrency Transparency Feasibility
  17. 17. It IS there 99.9999999% Availability
  18. 18. It grows and shrinks Scalability
  19. 19. U ain’ gonna crack it >:| Security
  20. 20. It is open for development and integration Opennes
  21. 21. Everything changes Heterogeneity
  22. 22. Parallel workloads Concurrency
  23. 23. Less know - less depend Transparency
  24. 24. Simplify as possible Feasibility
  25. 25. Fallacies The network is reliable Latency is zero Bandwidth is infinite The network is secure Topology doesn't change There is one administrator Transport cost is zero The network is homogeneous
  26. 26. Mechanics Part II
  27. 27. Mechanics I Membership Concurrency / Parallelism Synchronization Coordination Consensus Clustering
  28. 28. Who’s there? Membership
  29. 29. Membership Heartbeats Up-2-date list of nodes
  30. 30. We are Legion Concurrency / Parallelism
  31. 31. Concurrency / Parallelism Actors Process Calculi Shared memory
  32. 32. MY precious! Synchronization
  33. 33. Synchronisation Mutex | Semaphore Distributed Lock
  34. 34. Roger that! Coordination
  35. 35. Coordination Leader election Orchestration the ‘Two Generals Problem’
  36. 36. Deal Consensus
  37. 37. Consensus Quorum Consistency Availability Partition tolerance
  38. 38. Altogether Clustering
  39. 39. Clustering Load Balancing Redundancy Availability Replication Partition tolerance
  40. 40. Mechanics II Scaling Fault Detection Failover Recover Fail back
  41. 41. Scaling Vertical Horisontal Automatic
  42. 42. Fault Detection Monitoring Supervision Activity Analysis
  43. 43. Failover Data | State Accessibility Resource allocation Spare Capacity Business Continuity
  44. 44. Recovery State Reconstruction Data backup State dump
  45. 45. Failback Data consistency Conflict resolution
  46. 46. Apache YARN Apache Zookeeper Apache Mesos Apache Drill Google Chubby CoreOS Google Kubernetes Google Dremel
  47. 47. Architecture Part III
  48. 48. Client - Server
  49. 49. N-tier
  50. 50. Service oriented Architecture
  51. 51. Event Driven Architecture
  52. 52. P2P
  53. 53. Context
  54. 54. System Allocation
  55. 55. Architectural entities decomposition Subsystem Service Component Process Object Middleware
  56. 56. Fractal
  57. 57. Subsystem -> Component -> Process -> Object
  58. 58. Subsystem -> Component -> Process -> Object
  59. 59. Subsystem -> Component -> Process -> Object
  60. 60. Communication Infrastructure aka Middleware IPC ----------------------------- Remote Invocation Group Communication Messaging
  61. 61. IPC Share data Send data
  62. 62. Remote Invocation Request - Reply RPC RMI
  63. 63. Group Communication { Uni | Multi | Broad } cast Publish | Subscribe
  64. 64. Messaging Message Queue Message Broker Pipe | Filter | Translator
  65. 65. Implementation Part IV
  66. 66. Microcosm -> Macrocosm Middleware [ * ] Process Host Network
  67. 67. Within a process Threads Shared memory Mutex | Monitor
  68. 68. require 'thread' class RaceCondition def initialize(resource, concurrency_level) @resource = resource @concurrency_level = concurrency_level end def perform @concurrency_level.times do threads << Thread.new { action } end threads.each(&:join) end private def threads @threads ||= [] end def action @resource += 1 puts @resource end end RaceCondition.new(0, 10).perform # 12 # 3 # 5 # 6 # 7 # 8 # 910 # 4
  69. 69. require 'thread' class Sycnhronizer def initialize(resource, concurrency_level) @resource = resource @concurrency_level = concurrency_level end def perform @concurrency_level.times do threads << Thread.new { action } end threads.each(&:join) end private def threads @threads ||= [] end def action lock.synchronize do @resource += 1 puts @resource end end def lock @lock ||= Mutex.new end end Sycnhronizer.new(0, 10).perform # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10
  70. 70. Microcosm -> Macrocosm Process [ * ] Host NetworkMiddleware
  71. 71. Within a host Processes MQ | Shared memory | Pipes | UNIX Socket Semaphore
  72. 72. # http://bogomips.org/ruby_posix_mq/ require 'posix_mq' class Producer attr_reader :mq def initialize(mq_name) @mq = POSIX_MQ.new("/foo", :rw) end def send(message, prio = 0) puts "Send: #{message}. Priority #{prio}" mq.send("#{message} #{prio}", prio) end end p = Producer.new("/test_ipc") p.send("Hello from #{Process.pid}", 10) p.send("Hello from #{Process.pid}", 2) p.send("Hello from #{Process.pid}", 0) p.send("Hello from #{Process.pid}", 1) p.send("Hello from #{Process.pid}", 20) # ruby posix_mq/producer.rb # # Send: Hello from 12635. Priority 10 # Send: Hello from 12635. Priority 2 # Send: Hello from 12635. Priority 0 # Send: Hello from 12635. Priority 1 # Send: Hello from 12635. Priority 20
  73. 73. # http://bogomips.org/ruby_posix_mq/ require 'posix_mq' class Consumer attr_reader :mq def initialize(mq_name) @mq = POSIX_MQ.new("/foo", :rw) end def receive mq.receive.first end def receive_non_block mq.nonblock = true begin receive rescue Errno::EAGAIN mq.nonblock = false puts "Nothing" end end def shift mq.tryshift end end c = Consumer.new("/test_ipc") while m = c.shift puts "got: #{m}" end # ruby posix_mq/consumer.rb # got: Hello from 12635 10 # got: Hello from 12635 20 # got: Hello from 12635 2 # got: Hello from 12635 1 # got: Hello from 12635 0
  74. 74. # https://github.com/pmahoney/process_shared require 'process_shared' mutex = ProcessShared::Mutex.new mem = ProcessShared::SharedMemory.new(:int) mem.put_int(0, 0) pid1 = fork do puts "in process 1 (#{Process.pid})" 10.times do sleep 0.01 mutex.synchronize do value = mem.get_int(0) sleep 0.01 puts "process 1 (#{Process.pid}) incrementing" mem.put_int(0, value + 1) end end end pid2 = fork do puts "in process 2 (#{Process.pid})" 10.times do sleep 0.01 mutex.synchronize do value = mem.get_int(0) sleep 0.01 puts "process 2 (#{Process.pid}) decrementing" mem.put_int(0, value - 1) end end end Process.wait(pid1) Process.wait(pid2) # ruby shm.rb # # in process 1 (8038) # in process 2 (8041) # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # process 1 (8038) incrementing # process 2 (8041) decrementing # value should be zero: 0
  75. 75. rd_child, wr_parent = IO.pipe rd_parent, wr_child = IO.pipe pid = fork do rd_parent.close wr_parent.close wr_child.puts "sent from child process" puts rd_child.gets end rd_child.close wr_child.close wr_parent.write "sent from parent process" puts rd_parent.gets # ruby pipes/pipes.rb # # sent from child process # sent from parent process
  76. 76. require 'eventmachine' module UnixServer def receive_data(data) puts data EM.stop if data.chomp == "exit" send_data("Server #{Process.pid}: Got #{data.chomp} from you") close_connection_after_writing end end EM.run do puts "Started UNIX socket server on /tmp/sock" EM::start_unix_domain_server("/tmp/sock", UnixServer) end # ruby server.rb # # Started UNIX socket server on /tmp/sock # # HELLO! My pid is 13847
  77. 77. require 'socket' UNIXSocket.open("/tmp/sock") do |c| c.write("HELLO! My pid is #{Process.pid}") puts c.read end # ruby client.rb # # Server 13843: Got HELLO! My pid is 13847 from you
  78. 78. Microcosm -> Macrocosm Process Host [ * ] NetworkMiddleware
  79. 79. Over the Network { TCP | UDP } Socket HTTP Client - Server Async messaging
  80. 80. require "bunny" conn = Bunny.new conn.start ch = conn.create_channel q = ch.queue("hello") puts " [*] Waiting for messages in #{q.name}. To exit press CTRL+C" q.subscribe(:block => true) do |delivery_info, properties, body| puts " [x] Received #{body}" # cancel the consumer to exit delivery_info.consumer.cancel end require "bunny" conn = Bunny.new(:hostname => "rabbit.local") conn.start ch = conn.create_channel q = ch.queue("hello") ch.default_exchange.publish("Hello World!", :routing_key => q.name) puts " [x] Sent 'Hello World!'" conn.close
  81. 81. require "bunny" conn = Bunny.new conn.start ch = conn.create_channel x = ch.fanout("logs") msg = ARGV.empty? ? "Hello World!" : ARGV.join(" ") x.publish(msg) puts " [x] Sent #{msg}" conn.close require "bunny" conn = Bunny.new conn.start ch = conn.create_channel x = ch.fanout("logs") q = ch.queue("", :exclusive => true) q.bind(x) puts " [*] Waiting for logs. To exit press CTRL+C" begin q.subscribe(:block => true) do |delivery_info, properties, body| puts " [x] #{body}" end rescue Interrupt => _ ch.close conn.close end
  82. 82. Apache Kafka RabbitMQ
  83. 83. Next time To be continued: Algorithms, patterns, code!
  84. 84. Thank you! Q&A

×