• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Europycon2011: Implementing distributed application using ZeroMQ
 

Europycon2011: Implementing distributed application using ZeroMQ

on

  • 30,507 views

 

Statistics

Views

Total Views
30,507
Views on SlideShare
30,124
Embed Views
383

Actions

Likes
58
Downloads
744
Comments
1

3 Embeds 383

http://lanyrd.com 217
https://pulse.neudesic.com 159
https://twitter.com 7

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • nice
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Europycon2011: Implementing distributed application using ZeroMQ Europycon2011: Implementing distributed application using ZeroMQ Presentation Transcript

  • Implementing distributed applications with...and some other bad guys... Crippa Francesco
  • Scalability vs Complexity
  • Scalability vs Complexity what we want Complexity1 Thread 2 Threads 2 Nodes 4 Nodes
  • Scalability vs Complexity reality Complexity1 Thread 2 Threads 2 Nodes 4 Nodes
  • The Q in ØMQ
  • The Ø in ØMQ
  • The Ø in ØMQZero Broker
  • The Ø in ØMQZero BrokerZero Latency (as close as possible...)
  • The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administration
  • The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero cost
  • The Ø in ØMQZero BrokerZero Latency (as close as possible...)Zero administrationZero costZero waste
  • Sockets
  • SocketsUnicast transports (inproc, ipc, tcp)
  • SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)
  • SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independent
  • SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)
  • SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)They express a certain "messaging pattern"
  • SocketsUnicast transports (inproc, ipc, tcp)Multicast transports (pgm or epgm)connect() and bind() are independentThey are asynchronous (with queues)They express a certain "messaging pattern"They are not necessarily one-to-one
  • ...and, of course...Cross Platform (Linux, Windows, Mac, etc...)Multiple Languages (c, c++, python, java,ruby, erlang, php, perl, ada, c#, lua, scala,objective-c, go, haskell, racket, cl, basic...)OpenSource
  • if you have a laptop...http://www.zeromq.org/http://zguide.zeromq.org/
  • Basic Message Patterns
  • Server
  • Client
  • Demo
  • Basic Message Patterns
  • Server
  • Client
  • Demo
  • Publisher SubscriberThe PUB-SUB socket pair is asynchronouswhen you use a SUB socket you must set asubscription using zmq_setsockopt andSUBSCRIBE"slow joiner" symptom
  • Basic Message Patterns
  • Ventilator
  • Worker
  • Sink
  • Demo
  • WorkersAlways synchronize the start of the batchThe ventilators PUSH socket distributestasks to workers (load balancing)The sinks PULL socket collects results fromworkers evenly (fair-queuing)
  • Fair-Queuing
  • Basic Message Patterns
  • Sendmessages back
  • Demo
  • Allowed PatternsPUB and SUB DEALER and DEALERREQ and REP ROUTER and ROUTERREQ and ROUTER PUSH and PULLDEALER and REP PAIR and PAIRDEALER and ROUTER
  • Scalability
  • Scalability
  • Scalability
  • Scalability
  • A Publish-Subscribe Proxy
  • Built-in DevicesQUEUE (request-reply broker.)FORWARDER (pub-sub proxy server)STREAMER (like FORWARDER but forpipeline flows)
  • Dynamic Scalability...
  • Dynamic Scalability... SCA LOW LAB ILI !!!! TY
  • Dynamic Scalability...
  • ØMQ : Sockets = Python : C++