Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Implementing distributed     applications with...and some other bad guys...         Crippa Francesco
Scalability vs Complexity
Scalability vs Complexity     what we want                                      Complexity1 Thread    2 Threads   2 Nodes ...
Scalability vs Complexity      reality                                          Complexity1 Thread        2 Threads   2 No...
The Q in ØMQ
The Ø in ØMQ
The Ø in ØMQZero Broker
The Ø in ØMQZero BrokerZero Latency (as close as possible...)
The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administration
The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero cost
The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero costZero waste
Sockets
SocketsUnicast transports (inproc, ipc, tcp)
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independent
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are...
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are...
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are...
...and, of course...Cross Platform (Linux, Windows, Mac, etc...)Multiple Languages (c, c++, python, java,ruby, erlang, php...
if you have a laptop...http://www.zeromq.org/http://zguide.zeromq.org/
Basic Message Patterns
Server
Client
Demo
Basic Message Patterns
Server
Client
Demo
Publisher                SubscriberThe PUB-SUB socket pair is asynchronouswhen you use a SUB socket you must set asubscrip...
Basic Message Patterns
Ventilator
Worker
Sink
Demo
WorkersAlways synchronize the start of the batchThe ventilators PUSH socket distributestasks to workers (load balancing)Th...
Fair-Queuing
Basic Message Patterns
Sendmessages  back
Demo
Allowed PatternsPUB and SUB         DEALER and DEALERREQ and REP         ROUTER and ROUTERREQ and ROUTER      PUSH and PUL...
Scalability
Scalability
Scalability
Scalability
A Publish-Subscribe Proxy
Built-in DevicesQUEUE (request-reply broker.)FORWARDER (pub-sub proxy server)STREAMER (like FORWARDER but forpipeline flows)
Dynamic Scalability...
Dynamic Scalability...              SCA   LOW                  LAB                       ILI                  !!!!     TY
Dynamic Scalability...
ØMQ : Sockets = Python : C++
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Upcoming SlideShare
Loading in …5
×

Europycon2011: Implementing distributed application using ZeroMQ

83,419 views

Published on

Europycon2011: Implementing distributed application using ZeroMQ

  1. 1. Implementing distributed applications with...and some other bad guys... Crippa Francesco
  2. 2. Scalability vs Complexity
  3. 3. Scalability vs Complexity what we want Complexity1 Thread 2 Threads 2 Nodes 4 Nodes
  4. 4. Scalability vs Complexity reality Complexity1 Thread 2 Threads 2 Nodes 4 Nodes
  5. 5. The Q in ØMQ
  6. 6. The Ø in ØMQ
  7. 7. The Ø in ØMQZero Broker
  8. 8. The Ø in ØMQZero BrokerZero Latency (as close as possible...)
  9. 9. The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administration
  10. 10. The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero cost
  11. 11. The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero costZero waste
  12. 12. Sockets
  13. 13. SocketsUnicast transports (inproc, ipc, tcp)
  14. 14. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)
  15. 15. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independent
  16. 16. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)
  17. 17. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)They express a certain "messaging pattern"
  18. 18. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)They express a certain "messaging pattern"They are not necessarily one-to-one
  19. 19. ...and, of course...Cross Platform (Linux, Windows, Mac, etc...)Multiple Languages (c, c++, python, java,ruby, erlang, php, perl, ada, c#, lua, scala,objective-c, go, haskell, racket, cl, basic...)OpenSource
  20. 20. if you have a laptop...http://www.zeromq.org/http://zguide.zeromq.org/
  21. 21. Basic Message Patterns
  22. 22. Server
  23. 23. Client
  24. 24. Demo
  25. 25. Basic Message Patterns
  26. 26. Server
  27. 27. Client
  28. 28. Demo
  29. 29. Publisher SubscriberThe PUB-SUB socket pair is asynchronouswhen you use a SUB socket you must set asubscription using zmq_setsockopt andSUBSCRIBE"slow joiner" symptom
  30. 30. Basic Message Patterns
  31. 31. Ventilator
  32. 32. Worker
  33. 33. Sink
  34. 34. Demo
  35. 35. WorkersAlways synchronize the start of the batchThe ventilators PUSH socket distributestasks to workers (load balancing)The sinks PULL socket collects results fromworkers evenly (fair-queuing)
  36. 36. Fair-Queuing
  37. 37. Basic Message Patterns
  38. 38. Sendmessages back
  39. 39. Demo
  40. 40. Allowed PatternsPUB and SUB DEALER and DEALERREQ and REP ROUTER and ROUTERREQ and ROUTER PUSH and PULLDEALER and REP PAIR and PAIRDEALER and ROUTER
  41. 41. Scalability
  42. 42. Scalability
  43. 43. Scalability
  44. 44. Scalability
  45. 45. A Publish-Subscribe Proxy
  46. 46. Built-in DevicesQUEUE (request-reply broker.)FORWARDER (pub-sub proxy server)STREAMER (like FORWARDER but forpipeline flows)
  47. 47. Dynamic Scalability...
  48. 48. Dynamic Scalability... SCA LOW LAB ILI !!!! TY
  49. 49. Dynamic Scalability...
  50. 50. ØMQ : Sockets = Python : C++

×