Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euZeroMQ as scriptable socketsRealtimec...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euWhy this talk ?Monday, April 22, 13
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euOperations / PlatformMonday, April 22...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euCredentialsZeroMQ monitoring supportc...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu99% of deploymentsdon’t need enterpri...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euThen ...Monday, April 22, 13
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu... nowMonday, April 22, 13
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euZero broker, admin, cost,waste, price...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euWhat it isn’tMonday, April 22, 13
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euIM for appsNot a server or message qu...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euBSD APIsock.bind(“inproc://workers”)s...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euCommunity30+ language bindings ...Mon...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euSuper (scriptable) socketsWhat is a Z...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eubind () or connect () tomultiple endp...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eubind () or connect ()in any orderAsyn...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euSocket typesPUB SUBREQ REPPUSH PULLMo...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euRequest / replyREQREPREQREP REPREPREQ...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euRequest / Reply usageLock stepBi-dire...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPublish / subscribePUBSUB SUB SUBMond...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euInversion - loggingSUBPUB PUB PUBMond...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPublish / subscribe usageMarket data,...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPipelinePUSHPULL PULL PULLMonday, Apr...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPipeline usageExtendable - UNIX pipes...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euTransport agnosticinproc:// - threads...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euLightweight switchesDevicesMonday, Ap...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eupub/sub persistencePERSISTENCEDEVICEM...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euDevice main looploop dom = sub_socket...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euUse casesProxyRouting (LRU)Filtering ...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euWork distributionPUSHPULLPUSHPULLPULL...
PUSHPULLPUSHPULLPULLPUSHPULLPUSHVENTILATORSINKPUSHPULLBear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | ...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euInterjection PrincipleScalable topolo...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euSaving bandwidthSUBPUBPUBSUB SUBSUB S...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euAsynchronous externaldependencies.Res...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euAtomicityAtomic message deliveryUnbou...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euAvailabilityDelayed retry on failureM...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euReliability as patternsLazy Pirate (R...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euZeroMQ concurrencyPUSHPULL PULL PULLM...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eus = ctx.bind(:PUSH, "inproc://w")3.ti...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euThread.new dos = ctx.connect(:PULL, “...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2Case studyMonday, April 22, 13
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euHow it worksMulti-protocolZeroMQ for ...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 topologyPULLPUBPULLPUBPUSHSU...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 protocolUUID ID PATH SIZE:HE...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 HTTP requestPUSH PULLAB23 4 ...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 HTTP responseSUB PUBAB23 4 /...
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euQuestions ?(... or sync up after)O fi...
Upcoming SlideShare
Loading in …5
×

ZeroMQ as scriptable sockets

4,982 views

Published on

ZeroMQ is a socket abstraction and concurrency framework that's changing the way we think and reason about distributed systems. In this talk we'll discuss :

The importance of lightweight messaging for applications
Composable messaging patterns
Swappable transports
Refactoring at the transport layer
Resiliency and reliability
Small case study

Published in: Technology
  • Be the first to comment

ZeroMQ as scriptable sockets

  1. 1. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euZeroMQ as scriptable socketsRealtimeconf EU 2013Lyon - http://realtimeconf.eu/Lourens NaudéMonday, April 22, 13
  2. 2. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euWhy this talk ?Monday, April 22, 13
  3. 3. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euOperations / PlatformMonday, April 22, 13
  4. 4. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euCredentialsZeroMQ monitoring supportczmq co-maintainerRuby bindingExperimental TLS supportMonday, April 22, 13
  5. 5. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu99% of deploymentsdon’t need enterprisereliabilityOf reliabilityMonday, April 22, 13
  6. 6. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euThen ...Monday, April 22, 13
  7. 7. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu... nowMonday, April 22, 13
  8. 8. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euZero broker, admin, cost,waste, price tag and(almost) zero latencyWhat is ZeroMQ ?Monday, April 22, 13
  9. 9. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euWhat it isn’tMonday, April 22, 13
  10. 10. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euIM for appsNot a server or message queueA libraryCommunication / concurrencyMessaging semantics for socketsMonday, April 22, 13
  11. 11. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euBSD APIsock.bind(“inproc://workers”)sock.connect (“tcp://0.0.0.0:30”)sock.send (“message”)msg = sock.recv ()sock.close ()Monday, April 22, 13
  12. 12. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euCommunity30+ language bindings ...Monday, April 22, 13
  13. 13. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euSuper (scriptable) socketsWhat is a ZeroMQ socket ?Monday, April 22, 13
  14. 14. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eubind () or connect () tomultiple endpoints,simultaneouslyMultiple endpointsMonday, April 22, 13
  15. 15. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eubind () or connect ()in any orderAsync connectMonday, April 22, 13
  16. 16. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euSocket typesPUB SUBREQ REPPUSH PULLMonday, April 22, 13
  17. 17. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euRequest / replyREQREPREQREP REPREPREQ REQMonday, April 22, 13
  18. 18. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euRequest / Reply usageLock stepBi-directionalRoutableFlow control, orchestration etc.Monday, April 22, 13
  19. 19. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPublish / subscribePUBSUB SUB SUBMonday, April 22, 13
  20. 20. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euInversion - loggingSUBPUB PUB PUBMonday, April 22, 13
  21. 21. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPublish / subscribe usageMarket data, game state etc.PUB socket distributesSUB socket filtersSubscription forwarding (3.x)Monday, April 22, 13
  22. 22. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPipelinePUSHPULL PULL PULLMonday, April 22, 13
  23. 23. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euPipeline usageExtendable - UNIX pipesRound-robin distributionTasks with variable processingMonday, April 22, 13
  24. 24. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euTransport agnosticinproc:// - threads in a processipc:// - processes on a boxtcp:// - nodes in a networkUniform APIMonday, April 22, 13
  25. 25. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euLightweight switchesDevicesMonday, April 22, 13
  26. 26. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eupub/sub persistencePERSISTENCEDEVICEMonday, April 22, 13
  27. 27. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euDevice main looploop dom = sub_socket.recv ()persist (m)pub_socket.send (m)endMonday, April 22, 13
  28. 28. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euUse casesProxyRouting (LRU)Filtering / de-dupTranslationMonday, April 22, 13
  29. 29. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euWork distributionPUSHPULLPUSHPULLPULLPUSHPULLPUSHVENTILATORSINKMonday, April 22, 13
  30. 30. PUSHPULLPUSHPULLPULLPUSHPULLPUSHVENTILATORSINKPUSHPULLBear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euScalable work distributionMonday, April 22, 13
  31. 31. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euInterjection PrincipleScalable topologiesMonday, April 22, 13
  32. 32. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euSaving bandwidthSUBPUBPUBSUB SUBSUB SUBMonday, April 22, 13
  33. 33. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euAsynchronous externaldependencies.ResiliencyMonday, April 22, 13
  34. 34. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euAtomicityAtomic message deliveryUnbound message sizeSend 5GB, recv all or nothingMonday, April 22, 13
  35. 35. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euAvailabilityDelayed retry on failureMulti bind() and connect()Unordered bind() or connect()Monday, April 22, 13
  36. 36. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euReliability as patternsLazy Pirate (REQ / REP)Majordomo (queue)Titanic (persistence)Binary Star (High Availability)See http://zguide.zeromq.orgMonday, April 22, 13
  37. 37. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euZeroMQ concurrencyPUSHPULL PULL PULLMAIN THREADTHREAD 1 THREAD 2 THREAD 3I/OTHREADMonday, April 22, 13
  38. 38. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eus = ctx.bind(:PUSH, "inproc://w")3.times{ spawn_worker(ctx) }loop domsg = work.shiftbreak if msg == “exit”s.send(msg)endMaster (work distributor)Monday, April 22, 13
  39. 39. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euThread.new dos = ctx.connect(:PULL, “inproc://w”)loop domsg = s.recvbreak if msg == "quit"process(msg)endendWorkers (consumer threads)Monday, April 22, 13
  40. 40. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2Case studyMonday, April 22, 13
  41. 41. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euHow it worksMulti-protocolZeroMQ for backendsAsync backend requestsLanguage agnostic (bindings)Monday, April 22, 13
  42. 42. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 topologyPULLPUBPULLPUBPUSHSUBBACKEND A BACKEND BMONGREL2CLIENTMonday, April 22, 13
  43. 43. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 protocolUUID ID PATH SIZE:HEADERS,SIZE:BODYMonday, April 22, 13
  44. 44. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 HTTP requestPUSH PULLAB23 4 /uri SIZE:HEADERS,SIZE:BODYMonday, April 22, 13
  45. 45. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euMongrel2 HTTP responseSUB PUBAB23 4 /uri SIZE:HEADERS,SIZE:BODYMonday, April 22, 13
  46. 46. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.euQuestions ?(... or sync up after)O fimfork @methodmissingfollow @methodmissingMonday, April 22, 13

×