Uploaded on

 

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
703
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
19
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Messaging Patterns with ZeroMQ In Python “It’s time to think distributedly” By Victor Lin
  • 2. The Age of Distribution
  • 3. Different messaging methods Node Node Process Process TCP/IP UDP IPCNetwork UDP Process Node Node Node Thread Thread ITC
  • 4. Background• It’s all about messaging• Too many messaging transports• Common patterns cannot be reused• Auto reconnect is needed
  • 5. Superhero
  • 6. ZeroMQ features• Lightweight overhead• Auto reconnect• One-to-many connection• Support different transports• Support different patterns• Support different languages
  • 7. Patterns• REQ/REP• SUB/PUB• PUSH/PULL• PAIR
  • 8. REQ/REP• Round-robin• One-to-many• One request, one response
  • 9. SUB/PUB• One-to-many• All subscribers receive messages from publisher
  • 10. PUSH/PULL• Round-robin• One-to-many• Many-to-one
  • 11. PAIR• Just like BSD socket• Bidirectional communication
  • 12. Different transports• TCP/IP – tcp://• Unix domain socket (IPC) – ipc://• Inter-thread Communication – inproc://• Multicast – pgm://
  • 13. Same program, different transport Node Node Process Process tcp:// ipc:// pgm://Network Process Node Node Node Thread Thread inproc://
  • 14. TCP/IP Worker tcp:// NodeManager Worker Node tcp:// REQ/REP tcp:// Node Worker tcp:// Node
  • 15. Unix domain socket Node Worker ipc:// ProcessManager WorkerProcess ipc:// REQ/REP ipc:// Process Worker ipc:// Process
  • 16. Inter-thread Communication Node Process Worker inproc:// ThreadManager WorkerThread inproc:// REQ/REP inproc:// Thread Worker inproc:// Thread
  • 17. Different languages• Python• C/C++• Java• .Net• Ruby• Go• PHP• Perl• …
  • 18. Example: authentication service• Updating authentication module requires restarting server• Authentication module could be the bottleneck Broadcast Mr.DJ TCP/IP Authentication module
  • 19. Example: authentication service• Authentication service can be restarted anytime• Working load can be distributed Authentication Broadcast REQ/REP (IPC) service Mr.DJ
  • 20. Example: authentication serviceBroadcast Authentication service AuthenticationBroadcast broker Authentication serviceBroadcast Authentication service
  • 21. Broker in ZMQ
  • 22. Example: recording service• Record audio when broadcasting• Upload file chunks to Amazon S3• Must be reliable
  • 23. Amazon S3 BroadcastPUB/SUB (IPC) Uploading Uploading Uploading worker worker workerWriting service REQ/REP (IPC) File Uploading service Local DB
  • 24. Server level plugins• Attach whatever you want Record service ipc:// Broadcast ipc:// Audio PUB/SUB ipc:// identifying service ipc:// …
  • 25. Conclusions• Flexible• High performance• Easy to use• Multi-language supported• Focus on messaging and topology
  • 26. Q&A• Any question?