0
Implementing distributed     applications with...and some other bad guys...         Crippa Francesco
Scalability vs Complexity
Scalability vs Complexity     what we want                                      Complexity1 Thread    2 Threads   2 Nodes ...
Scalability vs Complexity      reality                                          Complexity1 Thread        2 Threads   2 No...
The Q in ØMQ
The Ø in ØMQ
The Ø in ØMQZero Broker
The Ø in ØMQZero BrokerZero Latency (as close as possible...)
The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administration
The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero cost
The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero costZero waste
Sockets
SocketsUnicast transports (inproc, ipc, tcp)
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independent
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are...
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are...
SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are...
...and, of course...Cross Platform (Linux, Windows, Mac, etc...)Multiple Languages (c, c++, python, java,ruby, erlang, php...
if you have a laptop...http://www.zeromq.org/http://zguide.zeromq.org/
Basic Message Patterns
Server
Client
Demo
Basic Message Patterns
Server
Client
Demo
Publisher                SubscriberThe PUB-SUB socket pair is asynchronouswhen you use a SUB socket you must set asubscrip...
Basic Message Patterns
Ventilator
Worker
Sink
Demo
WorkersAlways synchronize the start of the batchThe ventilators PUSH socket distributestasks to workers (load balancing)Th...
Fair-Queuing
Basic Message Patterns
Sendmessages  back
Demo
Allowed PatternsPUB and SUB         DEALER and DEALERREQ and REP         ROUTER and ROUTERREQ and ROUTER      PUSH and PUL...
Scalability
Scalability
Scalability
Scalability
A Publish-Subscribe Proxy
Built-in DevicesQUEUE (request-reply broker.)FORWARDER (pub-sub proxy server)STREAMER (like FORWARDER but forpipeline flows)
Dynamic Scalability...
Dynamic Scalability...              SCA   LOW                  LAB                       ILI                  !!!!     TY
Dynamic Scalability...
ØMQ : Sockets = Python : C++
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Europycon2011: Implementing distributed application using ZeroMQ
Upcoming SlideShare
Loading in...5
×

Europycon2011: Implementing distributed application using ZeroMQ

47,747

Published on

3 Comments
88 Likes
Statistics
Notes
No Downloads
Views
Total Views
47,747
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
1,113
Comments
3
Likes
88
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript of "Europycon2011: Implementing distributed application using ZeroMQ"

    1. 1. Implementing distributed applications with...and some other bad guys... Crippa Francesco
    2. 2. Scalability vs Complexity
    3. 3. Scalability vs Complexity what we want Complexity1 Thread 2 Threads 2 Nodes 4 Nodes
    4. 4. Scalability vs Complexity reality Complexity1 Thread 2 Threads 2 Nodes 4 Nodes
    5. 5. The Q in ØMQ
    6. 6. The Ø in ØMQ
    7. 7. The Ø in ØMQZero Broker
    8. 8. The Ø in ØMQZero BrokerZero Latency (as close as possible...)
    9. 9. The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administration
    10. 10. The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero cost
    11. 11. The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero costZero waste
    12. 12. Sockets
    13. 13. SocketsUnicast transports (inproc, ipc, tcp)
    14. 14. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)
    15. 15. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independent
    16. 16. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)
    17. 17. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)They express a certain "messaging pattern"
    18. 18. SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)They express a certain "messaging pattern"They are not necessarily one-to-one
    19. 19. ...and, of course...Cross Platform (Linux, Windows, Mac, etc...)Multiple Languages (c, c++, python, java,ruby, erlang, php, perl, ada, c#, lua, scala,objective-c, go, haskell, racket, cl, basic...)OpenSource
    20. 20. if you have a laptop...http://www.zeromq.org/http://zguide.zeromq.org/
    21. 21. Basic Message Patterns
    22. 22. Server
    23. 23. Client
    24. 24. Demo
    25. 25. Basic Message Patterns
    26. 26. Server
    27. 27. Client
    28. 28. Demo
    29. 29. Publisher SubscriberThe PUB-SUB socket pair is asynchronouswhen you use a SUB socket you must set asubscription using zmq_setsockopt andSUBSCRIBE"slow joiner" symptom
    30. 30. Basic Message Patterns
    31. 31. Ventilator
    32. 32. Worker
    33. 33. Sink
    34. 34. Demo
    35. 35. WorkersAlways synchronize the start of the batchThe ventilators PUSH socket distributestasks to workers (load balancing)The sinks PULL socket collects results fromworkers evenly (fair-queuing)
    36. 36. Fair-Queuing
    37. 37. Basic Message Patterns
    38. 38. Sendmessages back
    39. 39. Demo
    40. 40. Allowed PatternsPUB and SUB DEALER and DEALERREQ and REP ROUTER and ROUTERREQ and ROUTER PUSH and PULLDEALER and REP PAIR and PAIRDEALER and ROUTER
    41. 41. Scalability
    42. 42. Scalability
    43. 43. Scalability
    44. 44. Scalability
    45. 45. A Publish-Subscribe Proxy
    46. 46. Built-in DevicesQUEUE (request-reply broker.)FORWARDER (pub-sub proxy server)STREAMER (like FORWARDER but forpipeline flows)
    47. 47. Dynamic Scalability...
    48. 48. Dynamic Scalability... SCA LOW LAB ILI !!!! TY
    49. 49. Dynamic Scalability...
    50. 50. ØMQ : Sockets = Python : C++
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×