Your SlideShare is downloading. ×
Zmq in context of openstack
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Zmq in context of openstack

960
views

Published on

ZMQ socket based networking library, which can also be molded as broker-less messaging framework. The slides talks about how zmq been woven to work with openstack services.

ZMQ socket based networking library, which can also be molded as broker-less messaging framework. The slides talks about how zmq been woven to work with openstack services.

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
960
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Openstack Messaging with ZMQ (Distributed RPC) By Yatin Kumbhare yatinkumbhare@gmail.com
  • 2. ZMQ / 0MQ / ZeroMQ -Introduction ● Zero Broker, Zero latency, zero cost - culture of minimalism. ● Brokerless - No SPOF ● High Scalability ● Flexible: inproc, ipc, tcp ● Order of bind and connect - Doesn’t matter ● Fast and reliable
  • 3. REQ-REP # Client import zmq context = zmq.Context() print("Connecting to hello world server…") socket = context.socket(zmq.REQ) socket.connect("tcp://localhost:5555") for request in range(10): print("Sending request %s …" % request) socket.send("Hello") # Get the reply. message = socket.recv() print("Received reply %s [ %s ]" % (request, message)) # Server import zmq context = zmq.Context() # Socket to talk to server print("Connecting to hello world server…") socket = context.socket(zmq.REP) socket.bind("tcp://*:5555") while True: print("waiting for client request") msg = socket.recv() print("Message received %s “ % msg) message = socket.send("World")
  • 4. PUB-SUB import zmq from random import randint context = zmq.Context() socket = context.socket(zmq.PUB) socket.bind("tcp://*:5556") import zmq import sys context = zmq.Context() socket = context.socket(zmq.SUB) socket.connect("tcp://localhost:5556") while True: zipcode = randint(1, 10) temperature = randint(-20, 50) zip_filter = sys.argv[1] if len(sys.argv) > 1 else “” humidity = randint(10, 15) print “Publish data:”,(zipcode, temperature, humidity) socket.send("%d %d %d" % (zipcode, temperature, relhumidity)) print “Collectin updates from weather service %s” % zip_filter socket.setsockopt(zmq.SUBSCRIBE, zip_filter) #process 5 records for record in range(5): data = socket.recv() print data
  • 5. PUB-SUB
  • 6. PUSH-PULL # Ventilator # socket to send messages on sender = context.socket(zmq.PUSH) sender.bind( “tcp://*:5557” ) sink = context.socket(zmq.PUSH) sink.connect( “tcp://localhost:5558” ) sink.send( “0”) # Workers: receiver = context .socket(zmq .PULL) receiver .connect( "tcp://localhost:5557" ) # Socket to send messages to to sink sender = context .socket(zmq .PUSH) sender.connect( "tcp://localhost:5558" ) # Sink: # Socket to receive messages on receiver = context .socket(zmq .PULL) receiver .bind("tcp://*:5558" ) # Wait for start of batch s = receiver .recv()
  • 7. Single-Point-Failure
  • 8. rpc.cast
  • 9. rpc.call
  • 10. ZMQ for Openstack ● In openstack, zeromq makes use of two protocols, TCP and IPC. ● TCP Socket, for sending message from nova services to rpc-zmq-receiver service. ● IPC Socket, for forwarding message received by rpc-zmq-receiver to the local nova services, which is listening on IPC socket. ● ZMQ uses PUB-SUB and PUSH-PULL socket types. ● PUB-SUB - to get reply, in case of rpc.call, this work as direct-consumer/publisher in context of rabbitmq. ● PUSH-PULL - for sending messages to other services, for example rpc.cast.
  • 11. zmq-receiver service ● rpc-zmq-receiver service with different sockets. ● Instantiate ZmqProxy class from oslo.messaging, impl_zmq. py file. ● Service creates TCP://*:9501, PULL socket, and listen for messages from other services. ● Topic: fanout~service or zmq_replies.<hostname>, zmq.PUB socket ● Topic: service.<hostname>, (conductor.node) zmq.PUSH socket is created. ● The socket protocol will be IPC. ● Here, data is received on TCP://*:9501 and forwarded to IPC socket.
  • 12. Nova-* services ● Instantiate ZmqReactor from oslo.messaging, impl_zmq.py file. ● Mostly the nova service manages with zmq.PUSH, zmq.PULL and zmq.SUB socket type. ● For consuming messages nova service uses IPC sockets with zmq.PULL, and to cast a message it creates TCP, zmq.PUSH socket. ● example: when zmq-receiver, publish message on ipc: ///var/run/openstack/zmq_topic_conductor.node, with PUSH socket, which is consumed by ipc: ///var/run/openstack/zmq_topic_conductor.node with PULL socket at conductor service.
  • 13. Nova-compute ● topic: fanout~compute, bind to ipc:// , zmq.SUB socket ● topic: compute.<hostname>, bind to ipc:// , zmq.PULL ● nova compute does rpc.call on conductor at the start of the service. ● The hostname for conductor service is read out from MatchMakerRing file. ● compute service, creates tcp://<conductor-hostname>:9501, PUSH socket and send message, which is received by zmqreceiver, and forwarded to nova-conductor service. ● As, this is rpc.call, to receive reply, nova compute create, ipc: ///var/run/openstack/zmq_topic_zmq_replies.<hostname> with SUB socket type and subscribe to msg-id (unique uuid), and await for response from conductor.
  • 14. MatchMakerRing file { "conductor": [ "controller" ], "scheduler": [ "controller" ], "compute": [ "controller","computenode" ], "network": [ "controller","computenode" ], "zmq_replies": [ "controller","computenode" ], "cert": [ "controller" ], "cinderscheduler": [ "controller" ], "cinder-volume": [ "controller" ], "consoleauth": [ "controller" ] }
  • 15. Message routing compute make rpc.call to conductor. Sends data on tcp://gravity:9501 with PUSH, which is received by zmq-receiver service of gravity host with topic:conductor. gravity 'conductor' {'args': {'service': {u'binary': u'nova-compute', u'created_at': u'2014-0219T11:02:32.000000', u'deleted': 0, before rpc.call, include following data ● reply-topic as zmq_replies.nodex u'deleted_at': None, ● create unique msg_id, u'disabled': False, ● extra key "method":"-reply" u'disabled_reason': None, u'host': u'nodex', To receive reply on topic:zmq_replies.nodex, creates socket ipc://...zmq_topic_zmq_replies.nodex with SUB and subscribe to msg_id. u'id': 2, u'report_count': 22, u'topic': u'compute', The conductor consume data and takes out reply-topic as zmq_replies.nodex Conductor sends back the response by creating tcp://nodex: 9501 with PUSH and send data. u'updated_at': u'2014-0219T11:08:10.000000'}, 'values': {'report_count': 23}}, 'method': 'service_update', 'namespace': None, 'version': '1.34'}
  • 16. Message routing ● nodex zmq-receiver service, receiver data and check for topic, based on topic which is u'zmq_replies.nodex' {'args': {' msg_id': u'103b95cc64ab4e39924b6240a8dbaac8', 'response': [[{'binary': u'nova-compute', zmq_replies.nodex, create ipc://.... /zmq_topic_zmq_replies.nodex with PUB and 'created_at': '2014-0219T11:02:32.000000', send data. ● 'deleted': 0L, Now, the compute already has reply socket ipc:// 'deleted_at': None, with SUB, and is already subscribe to msg_id, 'disabled': False, which will get back the response of rpc.call. 'disabled_reason': None, 'host': u'nodex', 'id': 2L, 'report_count': 23, 'topic': u'compute', 'updated_at': '2014-0219T11:08:33.572453'}]]}, 'method': '-process_reply'}
  • 17. Devstack with zmq #localrc ● ENABLED_SERVICES+=,-rabbit,-qpid,zeromq ● #controller ENABLED_SERVICES=n-cpu,n-net,zeromq #compute node Apply Patch: https://review.openstack.org/#/c/59875/3 Due to an issue with zeromq support for notifications in Glance, glance-api fails to start when configured to use ZeroMQ. Commnet line in lib/glance in devstack repository #iniset_rpc_backend glance $GLANCE_API_CONF DEFAULT