Downloaded 28 times
Uploaded by芳本 林
Pyhug zmq
This document discusses messaging patterns using ZeroMQ in Python. It begins with background on messaging and common patterns. ZeroMQ features lightweight overhead, auto-reconnect, different transports including TCP/IP, IPC, and inter-thread. Common patterns with ZeroMQ include REQ/REP for request-response, SUB/PUB for publish-subscribe, PUSH/PULL for routing, and PAIR for bidirectional communication. Examples demonstrate using different transports and how ZeroMQ supports flexible, high performance, and multi-language messaging architectures.
More Related Content
Pyhug zmq
- 1. Messaging Patterns withZeroMQ In Python “It’s time to think distributedly” By Victor Lin
- 2. The Age ofDistribution
- 3. Different messaging methods Node Node Process Process TCP/IP UDP IPC Network UDP Process Node Node Node Thread Thread ITC
- 4. Background • It’s all about messaging • Too many messaging transports • Common patterns cannot be reused • Auto reconnect is needed
- 5.
- 6. ZeroMQ features • Lightweight overhead • Auto reconnect • One-to-many connection • Support different transports • Support different patterns • Support different languages
- 7. Patterns • REQ/REP • SUB/PUB • PUSH/PULL • PAIR
- 8. REQ/REP • Round-robin • One-to-many •One request, one response
- 9. SUB/PUB • One-to-many • Allsubscribers receive messages from publisher
- 10. PUSH/PULL • Round-robin • One-to-many •Many-to-one
- 11. PAIR • Just likeBSD socket • Bidirectional communication
- 12. Different transports • TCP/IP – tcp:// • Unix domain socket (IPC) – ipc:// • Inter-thread Communication – inproc:// • Multicast – pgm://
- 13. Same program, differenttransport Node Node Process Process tcp:// ipc:// pgm:// Network Process Node Node Node Thread Thread inproc://
- 14. TCP/IP Worker tcp:// Node Manager Worker Node tcp:// REQ/REP tcp:// Node Worker tcp:// Node
- 15. Unix domain socket Node Worker ipc:// Process Manager Worker Process ipc:// REQ/REP ipc:// Process Worker ipc:// Process
- 16. Inter-thread Communication Node Process Worker inproc:// Thread Manager Worker Thread inproc:// REQ/REP inproc:// Thread Worker inproc:// Thread
- 17. Different languages • Python • C/C++ • Java • .Net • Ruby • Go • PHP • Perl • …
- 18. Example: authentication service •Updating authentication module requires restarting server • Authentication module could be the bottleneck Broadcast Mr.DJ TCP/IP Authentication module
- 19. Example: authentication service •Authentication service can be restarted anytime • Working load can be distributed Authentication Broadcast REQ/REP (IPC) service Mr.DJ
- 20. Example: authentication service Broadcast Authentication service Authentication Broadcast broker Authentication service Broadcast Authentication service
- 21.
- 22. Example: recording service •Record audio when broadcasting • Upload file chunks to Amazon S3 • Must be reliable
- 23. Amazon S3 Broadcast PUB/SUB (IPC) Uploading Uploading Uploading worker worker worker Writing service REQ/REP (IPC) File Uploading service Local DB
- 24. Server level plugins •Attach whatever you want Record service ipc:// Broadcast ipc:// Audio PUB/SUB ipc:// identifying service ipc:// …
- 25. Conclusions • Flexible • High performance • Easy to use • Multi-language supported • Focus on messaging and topology
- 26.