ZeroMQ is a socket abstraction and concurrency framework that's changing the way we think and reason about distributed systems. In this talk we'll discuss : The importance of lightweight messaging for applications Composable messaging patterns Swappable transports Refactoring at the transport layer Resiliency and reliability Small case study

1. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
ZeroMQ as scriptable sockets
Realtimeconf EU 2013
Lyon - http://realtimeconf.eu/
Lourens Naudé
Monday, April 22, 13

2. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Why this talk ?
Monday, April 22, 13

3. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Operations / Platform
Monday, April 22, 13

4. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Credentials
ZeroMQ monitoring support
czmq co-maintainer
Ruby binding
Experimental TLS support
Monday, April 22, 13

5. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
99% of deployments
don’t need enterprise
reliability
Of reliability
Monday, April 22, 13

6. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Then ...
Monday, April 22, 13

7. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
... now
Monday, April 22, 13

8. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Zero broker, admin, cost,
waste, price tag and
(almost) zero latency
What is ZeroMQ ?
Monday, April 22, 13

9. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
What it isn’t
Monday, April 22, 13

10. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
IM for apps
Not a server or message queue
A library
Communication / concurrency
Messaging semantics for sockets
Monday, April 22, 13

11. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
BSD API
sock.bind(“inproc://workers”)
sock.connect (“tcp://0.0.0.0:30”)
sock.send (“message”)
msg = sock.recv ()
sock.close ()
Monday, April 22, 13

12. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Community
30+ language bindings ...
Monday, April 22, 13

13. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Super (scriptable) sockets
What is a ZeroMQ socket ?
Monday, April 22, 13

14. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
bind () or connect () to
multiple endpoints,
simultaneously
Multiple endpoints
Monday, April 22, 13

15. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
bind () or connect ()
in any order
Async connect
Monday, April 22, 13

16. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Socket types
PUB SUB
REQ REP
PUSH PULL
Monday, April 22, 13

17. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Request / reply
REQ
REP
REQ
REP REP
REP
REQ REQ
Monday, April 22, 13

18. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Request / Reply usage
Lock step
Bi-directional
Routable
Flow control, orchestration etc.
Monday, April 22, 13

19. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Publish / subscribe
PUB
SUB SUB SUB
Monday, April 22, 13

20. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Inversion - logging
SUB
PUB PUB PUB
Monday, April 22, 13

21. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Publish / subscribe usage
Market data, game state etc.
PUB socket distributes
SUB socket filters
Subscription forwarding (3.x)
Monday, April 22, 13

22. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Pipeline
PUSH
PULL PULL PULL
Monday, April 22, 13

23. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Pipeline usage
Extendable - UNIX pipes
Round-robin distribution
Tasks with variable processing
Monday, April 22, 13

24. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Transport agnostic
inproc:// - threads in a process
ipc:// - processes on a box
tcp:// - nodes in a network
Uniform API
Monday, April 22, 13

25. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Lightweight switches
Devices
Monday, April 22, 13

26. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
pub/sub persistence
PERSISTENCE
DEVICE
Monday, April 22, 13

27. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Device main loop
loop do
m = sub_socket.recv ()
persist (m)
pub_socket.send (m)
end
Monday, April 22, 13

28. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Use cases
Proxy
Routing (LRU)
Filtering / de-dup
Translation
Monday, April 22, 13

29. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Work distribution
PUSH
PULL
PUSH
PULL
PULL
PUSH
PULL
PUSH
VENTILATOR
SINK
Monday, April 22, 13

30. PUSH
PULL
PUSH
PULL
PULL
PUSH
PULL
PUSH
VENTILATOR
SINK
PUSH
PULL
Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Scalable work distribution
Monday, April 22, 13

31. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Interjection Principle
Scalable topologies
Monday, April 22, 13

32. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Saving bandwidth
SUB
PUB
PUB
SUB SUB
SUB SUB
Monday, April 22, 13

33. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Asynchronous external
dependencies.
Resiliency
Monday, April 22, 13

34. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Atomicity
Atomic message delivery
Unbound message size
Send 5GB, recv all or nothing
Monday, April 22, 13

35. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Availability
Delayed retry on failure
Multi bind() and connect()
Unordered bind() or connect()
Monday, April 22, 13

36. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Reliability as patterns
Lazy Pirate (REQ / REP)
Majordomo (queue)
Titanic (persistence)
Binary Star (High Availability)
See http://zguide.zeromq.org
Monday, April 22, 13

37. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
ZeroMQ concurrency
PUSH
PULL PULL PULL
MAIN THREAD
THREAD 1 THREAD 2 THREAD 3
I/O
THREAD
Monday, April 22, 13

38. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
s = ctx.bind(:PUSH, "inproc://w")
3.times{ spawn_worker(ctx) }
loop do
msg = work.shift
break if msg == “exit”
s.send(msg)
end
Master (work distributor)
Monday, April 22, 13

39. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Thread.new do
s = ctx.connect(:PULL, “inproc://w”)
loop do
msg = s.recv
break if msg == "quit"
process(msg)
end
end
Workers (consumer threads)
Monday, April 22, 13

40. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Mongrel2
Case study
Monday, April 22, 13

41. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
How it works
Multi-protocol
ZeroMQ for backends
Async backend requests
Language agnostic (bindings)
Monday, April 22, 13

42. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Mongrel2 topology
PULL
PUB
PULL
PUB
PUSH
SUB
BACKEND A BACKEND B
MONGREL2
CLIENT
Monday, April 22, 13

43. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Mongrel2 protocol
UUID ID PATH SIZE:HEADERS,SIZE:BODY
Monday, April 22, 13

44. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Mongrel2 HTTP request
PUSH PULL
AB23 4 /uri SIZE:HEADERS,SIZE:BODY
Monday, April 22, 13

45. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Mongrel2 HTTP response
SUB PUB
AB23 4 /uri SIZE:HEADERS,SIZE:BODY
Monday, April 22, 13

46. Bear Metal OÜ| Õismäe tee 78-25, 13513 Tallinn, Harjumaa, Estonia | info@bearmetal.eu
Questions ?
(... or sync up after)
O fim
fork @methodmissing
follow @methodmissing
Monday, April 22, 13