Your SlideShare is downloading. ×
0
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Real-Time Python Web: Gevent and Socket.io

20,157

Published on

Published in: Technology, Education
0 Comments
35 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
20,157
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
302
Comments
0
Likes
35
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Real-Time Web: Gevent and Socket.io Rick Copeland @rick446 [email_address]
  • 2. <ul><li>Getting started with Gevent </li></ul><ul><li>AJAX, push, WebSockets, wha? </li></ul><ul><li>ZeroMQ for fun and multiprocessing </li></ul><ul><li>Putting it all together </li></ul>
  • 3. A (very) Brief Survey of Python Asynchronous Programming <ul><li>AsynCore </li></ul><ul><ul><li>In stdlib, used for stdlib SMTP server </li></ul></ul><ul><ul><li>Nobody cares about it anymore  </li></ul></ul><ul><li>Twisted </li></ul><ul><ul><li>Large community, vast amounts of code </li></ul></ul><ul><ul><li>Callbacks hurt my brain </li></ul></ul><ul><li>Stackless </li></ul><ul><ul><li>Cool, cooperative multithreading </li></ul></ul><ul><ul><li>Needs a custom Python </li></ul></ul><ul><li>Event-based green threads </li></ul><ul><ul><li>Like stackless, but in regular Python </li></ul></ul><ul><ul><li>Know when you yield </li></ul></ul>
  • 4. Let’s Go Green: Async that Doesn’t Hurt Your Brain <ul><li>Greenlets: Cooperative, lightweight threads </li></ul><ul><li>Very forgiving – mutexes rarely needed </li></ul><ul><li>Use it for IO! </li></ul>
  • 5. Gevent: Greenlets <ul><li>Spawn helpers </li></ul><ul><ul><li>spawn(my_python_function, *args, **kwargs) </li></ul></ul><ul><ul><li>Also spawn_later(), spawn_link(), etc. </li></ul></ul><ul><li>Greenlet class </li></ul><ul><ul><li>Like threads but cooperative </li></ul></ul><ul><ul><li>Useful properties: .get(), .join(), .kill(), .link() </li></ul></ul><ul><li>Timeouts </li></ul><ul><ul><li>Timeout(seconds, exception).start() </li></ul></ul><ul><li>Pools: for limiting concurrency, use Pool.spawn </li></ul>
  • 6. Gevent: Communication <ul><li>Event </li></ul><ul><ul><li>set() </li></ul></ul><ul><ul><li>clear() </li></ul></ul><ul><ul><li>wait() </li></ul></ul><ul><li>Queue </li></ul><ul><ul><li>Modeled after Queue.Queue </li></ul></ul><ul><ul><li>.get() </li></ul></ul><ul><ul><li>.put() </li></ul></ul><ul><ul><li>__iter__() </li></ul></ul><ul><ul><li>PriorityQueue, LifoQueue, JoinableQueue </li></ul></ul>
  • 7. Gevent: Networking <ul><li>“ Green” versions of sockets, select(), ssl, and dns </li></ul><ul><li>Quick and dirty: </li></ul>import gevent.monkey gevent .monkey.patch_all()
  • 8. Gevent: Servers <ul><li>Simple callback interface </li></ul><ul><li>Creates one greenlet per connection (but remember, that’s OK!) </li></ul>def handle(socket, address): print 'new connection!’ server = StreamServer( ( '127.0.0.1', 1234), handle) # creates a new server server .start() # start accepting new connections
  • 9. Gevent: WSGI <ul><li>gevent.wsgi </li></ul><ul><ul><li>Fast (~4k requests/s) </li></ul></ul><ul><ul><li>No streaming, pipelining, or ssl  </li></ul></ul><ul><li>gevent.pywsgi: </li></ul><ul><ul><li>Full featured </li></ul></ul><ul><ul><li>Slower (“only” 3k requests/s) </li></ul></ul>from gevent import pywsgi def hello_world(env, start_response): start_response( '200 OK', [('Content-Type', 'text/html')]) yield '<b>Hello world</b>’ server = pywsgi.WSGIServer( ( '0.0.0.0', 8080), hello_world) server .serve_forever()
  • 10. <ul><li>Getting started with Gevent </li></ul><ul><li>AJAX, push, WebSockets, wha? </li></ul><ul><li>ZeroMQ for fun and multiprocessing </li></ul><ul><li>Putting it all together </li></ul>
  • 11. What is the real-time web? <ul><li>No page refreshes </li></ul><ul><li>Server push </li></ul><ul><li>Examples: chat, realtime analytics, … </li></ul><ul><li>Implementation </li></ul><ul><ul><li>Flash (eww…) </li></ul></ul><ul><ul><li>Polling (2x eww…) </li></ul></ul><ul><ul><li>Long polling (wow – that’s clever :-/ ) </li></ul></ul><ul><ul><li>HTML5 WebSockets ( Super-easy! Awesome! Security vulnerabilities! Immature spec! ) </li></ul></ul>
  • 12. SocketIO to the Rescue “ Socket.IO aims to make realtime apps possible in every browser and mobile device, blurring the differences between the different transport mechanisms.”
  • 13. Socket.io Example <script src=&quot;/socket.io/socket.io.js&quot; ></script> <script> var socket = io.connect( 'http://localhost'); socket.on('news', function (data) { console.log(data); socket.emit( 'my other event’, { my : 'data' }); }); </script>
  • 14. gevent_socketio def hello_world(environ, start_response): if not environ[ 'PATH_INFO'] .startswith( '/socket.io'): return serve_file(environ, start_response) socketio = environ[ 'socketio'] while True: socketio .send( 'Hello, world') gevent .sleep( 2)
  • 15. <ul><li>Getting started with Gevent </li></ul><ul><li>AJAX, push, WebSockets, wha? </li></ul><ul><li>ZeroMQ for fun and multiprocessing </li></ul><ul><li>Putting it all together </li></ul>
  • 16. ZeroMQ Overview <ul><li>C library with Python bindings </li></ul><ul><li>ZMQ “sockets” are message based, delivery is via a dedicated communication thread </li></ul><ul><li>ZMQ transports (tcp, inproc, unix, multicast) </li></ul><ul><li>ZMQ socket types </li></ul><ul><ul><li>REQ/RES </li></ul></ul><ul><ul><li>PUSH/PULL </li></ul></ul><ul><ul><li>PUB/SUB </li></ul></ul><ul><ul><li>… </li></ul></ul>
  • 17. pyzmq and gevent_zmq <ul><li>pyzmq works great for threading </li></ul><ul><li>gevent_zmq is necessary for gevent (single-threaded) </li></ul><ul><li>Be careful when forking or otherwise using multiprocessing! </li></ul><ul><ul><li>Gevent has a global “hub” of greenlets </li></ul></ul><ul><ul><li>ZeroMQ has a global thread & “context” for communication </li></ul></ul><ul><ul><li>Best to wait till done forking before initializing ZeroMQ </li></ul></ul>
  • 18. ZeroMQ: bind/connect and pub/sub <ul><li>from gevent_zeromq import zmq </li></ul><ul><li>context = zmq.Context() </li></ul><ul><li>sock_queue = context.socket(zmq.PUB) </li></ul><ul><li>sock_queue.bind( 'inproc://chat') </li></ul>zmq_sock = context.socket(zmq.SUB) zmq_sock.setsockopt(zmq.SUBSCRIBE, &quot;&quot;) zmq_sock .connect( 'inproc://chat')
  • 19. <ul><li>Getting started with Gevent </li></ul><ul><li>AJAX, push, WebSockets, wha? </li></ul><ul><li>ZeroMQ for fun and multiprocessing </li></ul><ul><li>Putting it all together </li></ul>
  • 20. WebChat: Design Incoming Greenlet ZMQ send Outgoing Greenlet Socket.io ZMQ recv JSON Messages JSON Messages Socket.io
  • 21. WebChat: HTML <h1> Socket.io Chatterbox </h1> <div id=&quot;status&quot; style=&quot;border:1px solid black;&quot; > Disconnected </div> <form> <input id=&quot;input&quot; style=&quot;width: 35em;&quot; > </form> <div id=&quot;data&quot; style=&quot;border:1px solid black;&quot; > </div> <script src=&quot;/js/jquery.min.js&quot; ></script> <script src=&quot;/js/socket.io.js&quot; ></script> <script src=&quot;/js/test.js&quot; ></script>
  • 22. WebChat: Javascript Setup ( function () { // Create and connect socket var socket = new io.Socket( 'localhost'); socket.connect(); // Socket status var $status = $( '#status'); socket.on('connect', function () { $status.html( '<b>Connected: ' + socket.transport.type + '</b>'); }); socket.on('error', function () { $status.html( '<b>Error</b>'); }); socket.on('disconnect', function () { $status.html( '<b>Closed</b>'); });
  • 23. WebChat: Javascript Communication // Send data to the server var $form = $( 'form'); var $input = $( '#input'); $form.bind('submit', function () { socket.send($input.val()); $input.val( ''); return false ; }); // Get data back from the server var $data = $( '#data'); socket.on('message', function (msg) { msg = $.parseJSON(msg) ; var u = msg.u || 'SYSTEM’; $data.prepend($( '<em>' + u + '</em>: ' + msg.m + '<br/>')); }); })();
  • 24. WebChat: Server def chat(environ, start_response): if not environ[ 'PATH_INFO'] .startswith( '/socket.io): return serve_file(environ, start_response) socketio = environ[ 'socketio'] #... handle auth ... zmq_sock = context.socket(zmq.SUB) zmq_sock.setsockopt(zmq.SUBSCRIBE, &quot;&quot;) zmq_sock .connect( 'inproc://chat') greenlets = [ gevent.spawn(incoming, uname, socketio), gevent.spawn(outgoing, zmq_sock, socketio) ] gevent .joinall(greenlets)
  • 25. WebChat: Greenlets def incoming(uname, socketio): while True: for part in socketio .recv(): sock_queue.send(json.dumps( dict( u =uname, m=part))) def outgoing(zmq_sock, socketio): while True: socketio.send(zmq_sock.recv())
  • 26. Get the Code! Socket.io http://socket.io MIT License Chatterbox http://sf.net/u/rick446/pygotham Apache License ZeroMQ http://www.zeromq.org LGPL License Gevent http://gevent.org MIT License
  • 27. Rick Copeland @rick446 [email_address]

×