Uploaded on


More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Messaging Patterns with ZeroMQ In Python “It’s time to think distributedly” By Victor Lin
  • 2. The Age of Distribution
  • 3. Different messaging methods Node Node Process Process TCP/IP UDP IPCNetwork UDP Process Node Node Node Thread Thread ITC
  • 4. Background• It’s all about messaging• Too many messaging transports• Common patterns cannot be reused• Auto reconnect is needed
  • 5. Superhero
  • 6. ZeroMQ features• Lightweight overhead• Auto reconnect• One-to-many connection• Support different transports• Support different patterns• Support different languages
  • 7. Patterns• REQ/REP• SUB/PUB• PUSH/PULL• PAIR
  • 8. REQ/REP• Round-robin• One-to-many• One request, one response
  • 9. SUB/PUB• One-to-many• All subscribers receive messages from publisher
  • 10. PUSH/PULL• Round-robin• One-to-many• Many-to-one
  • 11. PAIR• Just like BSD socket• Bidirectional communication
  • 12. Different transports• TCP/IP – tcp://• Unix domain socket (IPC) – ipc://• Inter-thread Communication – inproc://• Multicast – pgm://
  • 13. Same program, different transport Node Node Process Process tcp:// ipc:// pgm://Network Process Node Node Node Thread Thread inproc://
  • 14. TCP/IP Worker tcp:// NodeManager Worker Node tcp:// REQ/REP tcp:// Node Worker tcp:// Node
  • 15. Unix domain socket Node Worker ipc:// ProcessManager WorkerProcess ipc:// REQ/REP ipc:// Process Worker ipc:// Process
  • 16. Inter-thread Communication Node Process Worker inproc:// ThreadManager WorkerThread inproc:// REQ/REP inproc:// Thread Worker inproc:// Thread
  • 17. Different languages• Python• C/C++• Java• .Net• Ruby• Go• PHP• Perl• …
  • 18. Example: authentication service• Updating authentication module requires restarting server• Authentication module could be the bottleneck Broadcast Mr.DJ TCP/IP Authentication module
  • 19. Example: authentication service• Authentication service can be restarted anytime• Working load can be distributed Authentication Broadcast REQ/REP (IPC) service Mr.DJ
  • 20. Example: authentication serviceBroadcast Authentication service AuthenticationBroadcast broker Authentication serviceBroadcast Authentication service
  • 21. Broker in ZMQ
  • 22. Example: recording service• Record audio when broadcasting• Upload file chunks to Amazon S3• Must be reliable
  • 23. Amazon S3 BroadcastPUB/SUB (IPC) Uploading Uploading Uploading worker worker workerWriting service REQ/REP (IPC) File Uploading service Local DB
  • 24. Server level plugins• Attach whatever you want Record service ipc:// Broadcast ipc:// Audio PUB/SUB ipc:// identifying service ipc:// …
  • 25. Conclusions• Flexible• High performance• Easy to use• Multi-language supported• Focus on messaging and topology
  • 26. Q&A• Any question?