Build your next project -
Super fast, distributed
and infinitely scalable.
Using ZeroMQ
Web Framework Tutorial
Application
Client
(Browser)
Real World
Queues
• Apache ActiveMQ
• RabbitMQ
• StormMQ
• Redis
…
ZeroMQ
0MQ
ØMQ
Basic Patterns
• Request/Reply
• Publish/Subscribe
• Pipelining
• Paired Sockets
Request - Reply
Hello World Server
import zmq
context = zmq.Context()
print "Starting hello world server..."
socket = context.socket(zmq.R...
Hello World Client
import zmq
context = zmq.Context()
# Socket to talk to server
print "Connecting to hello world server.....
Hello World Client 2
import sys
import time
import zmq
context = zmq.Context()
print "Connecting to hello world server..."...
Publish - Subscribe
Stock Ticker Server
import zmq
import time
import random
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.b...
Stock Ticker Client
import sys
import zmq
context = zmq.Context()
socket = context.socket(zmq.SUB)
print "Collecting updat...
Parallel Pipeline
import antigravity
XKCD Task Ventilator
import sys
import zmq
context = zmq.Context()
sender = context.socket(zmq.PUSH)
sender.bind("tcp://*:...
XKCD Task Worker
import json
import zmq
import requests
from lxml import etree
def get_title(url):
"""Given a XKCD url, re...
XKCD Task Worker
context = zmq.Context()
# Socket to receive messages on
receiver = context.socket(zmq.PULL)
receiver.conn...
XKCD Sink
import zmq
context = zmq.Context()
# Socket to receive messages on
receiver = context.socket(zmq.PULL)
receiver....
Transport Types
• inproc://name
• ipc:///tmp/filename
• tcp://hostname:port
• pgm://interface:address:port &
epgm://interf...
Languages
• C/C++
• C#
• Java
• Erlang
• Go
• Ruby
• Node.js
• Objective-C
…
http://zero.mq/
http://zguide.zeromq.org/py:all
https://github.com/cnu/zeromq-talk
Links
Upcoming SlideShare
Loading in …5
×

Zeromq - Pycon India 2013

4,489 views

Published on

Published in: Technology, Education

Zeromq - Pycon India 2013

  1. 1. Build your next project - Super fast, distributed and infinitely scalable. Using ZeroMQ
  2. 2. Web Framework Tutorial Application Client (Browser)
  3. 3. Real World
  4. 4. Queues • Apache ActiveMQ • RabbitMQ • StormMQ • Redis …
  5. 5. ZeroMQ 0MQ ØMQ
  6. 6. Basic Patterns • Request/Reply • Publish/Subscribe • Pipelining • Paired Sockets
  7. 7. Request - Reply
  8. 8. Hello World Server import zmq context = zmq.Context() print "Starting hello world server..." socket = context.socket(zmq.REP) socket.bind("tcp://*:5555") while True: message = socket.recv() print "Got: ", message # Send the reply. socket.send ("World")
  9. 9. Hello World Client import zmq context = zmq.Context() # Socket to talk to server print "Connecting to hello world server..." socket = context.socket(zmq.REQ) socket.connect ("tcp://localhost:5555") # Do 10 requests, waiting each time for a response for req_no in range (10): socket.send ("Hello") # Get the reply. message = socket.recv() print "Received reply ", req_no, "[", message, "]"
  10. 10. Hello World Client 2 import sys import time import zmq context = zmq.Context() print "Connecting to hello world server..." socket = context.socket(zmq.REQ) socket.connect ("tcp://localhost:5555") while True: socket.send(sys.argv[2]) # Get the reply. message = socket.recv() print "Received reply ", "[", message, "]" time.sleep(float(sys.argv[1]))
  11. 11. Publish - Subscribe
  12. 12. Stock Ticker Server import zmq import time import random context = zmq.Context() socket = context.socket(zmq.PUB) socket.bind("tcp://*:5556") scrips = ['AAPL', 'GOOG', 'MSFT', 'AMZN'] while True: scrip = random.choice(scrips) price = random.randrange(20,700) msg = "%s: %d" % (scrip, price) print msg socket.send(msg) time.sleep(0.5)
  13. 13. Stock Ticker Client import sys import zmq context = zmq.Context() socket = context.socket(zmq.SUB) print "Collecting updates from stock server..." socket.connect ("tcp://localhost:5556”) scrip_filter = sys.argv[1:] if len(sys.argv) > 1 else ["AAPL"] for scrip in scrip_filter: socket.setsockopt(zmq.SUBSCRIBE, scrip) while True: string = socket.recv() stock, price = string.split() print "%s: %s" %(stock, price)
  14. 14. Parallel Pipeline
  15. 15. import antigravity
  16. 16. XKCD Task Ventilator import sys import zmq context = zmq.Context() sender = context.socket(zmq.PUSH) sender.bind("tcp://*:5557") print "Press Enter when the workers are ready: " _ = raw_input() print "Sending tasks to workers..." base_url = 'http://xkcd.com/' urls = [base_url + str(i) for i in xrange(1, int(sys.argv[1]))] for url in urls: sender.send(url)
  17. 17. XKCD Task Worker import json import zmq import requests from lxml import etree def get_title(url): """Given a XKCD url, retrieve the title of the image.""" r = requests.get(url) tree = etree.HTML(r.content) title = tree.xpath('//div[@id="comic"]/img/@title') if title: return title[0] else: return None
  18. 18. XKCD Task Worker context = zmq.Context() # Socket to receive messages on receiver = context.socket(zmq.PULL) receiver.connect("tcp://localhost:5557") # Socket to send messages to sender = context.socket(zmq.PUSH) sender.connect("tcp://localhost:5558") # Process tasks forever while True: s = receiver.recv() title = get_title(s) print s, title # Send a JSON payload to sink sender.send(json.dumps({"url":s, "title":title}))
  19. 19. XKCD Sink import zmq context = zmq.Context() # Socket to receive messages on receiver = context.socket(zmq.PULL) receiver.bind("tcp://*:5558") while True: s = receiver.recv() print s
  20. 20. Transport Types • inproc://name • ipc:///tmp/filename • tcp://hostname:port • pgm://interface:address:port & epgm://interface:address:port
  21. 21. Languages • C/C++ • C# • Java • Erlang • Go • Ruby • Node.js • Objective-C …
  22. 22. http://zero.mq/ http://zguide.zeromq.org/py:all https://github.com/cnu/zeromq-talk Links

×