Communication in Python and the C10k problem
•
6 likes•3,125 views
Talk at the Codemotion Spain 2014 about how to handle communication (polling, long polling, websockets, SSE), concurrency (processes, threads, coroutines, green threads) and the C10K problem in python
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Communication in Python and the C10k problem
Similar to Communication in Python and the C10k problem (20)
Recently uploaded
Recently uploaded (20)
Communication in Python and the C10k problem
- 1. Communication in Python and the C10k problem Jose Ignacio Galarza @igalarzab
- 4. select the best method to communicate
- 5. select the best method to scale
- 7. Index ✤ From polling… to pushing ✤ Concurrency ✤ C10k ✤ Asynchronous I/O
- 8. PYTHON 3 PYTHON 3 EVERYWHERE
- 9. From polling to pushing
- 11. polling ➡ Nothing new to implement ➡ Good support in all platforms ➡ Proxy friendly ➡ Uni-directional ➡ Very low efficiency ➡ No cross-domain
- 12. @app.route("/messages") def messages(): message = check_new_message() return message while True: polling r = requests.get('http://api.yo.com/messages') messages.extend(r.json()) time.sleep(5)
- 13. long polling Client Server
- 14. long polling ➡ Good support in all platforms ➡ “Proxy friendly” ➡ Uni-directional ➡ Low efficiency (better than polling) ➡ No cross-domain
- 15. long polling @app.route("/messages") def messages(): message = wait_until_new_message() return message while True: try: r = requests.get('http://yo.com/messages', timeout=60) except TimeOut: continue messages.extend(r.json())
- 16. HTTP Streaming Client Server
- 17. HTTP Streaming ➡ Better efficiency than (long) polling ➡ “Proxy friendly” ➡ Uni-directional ➡ You need to parse the data manually
- 18. HTTP Streaming ➡ Type 1: Connection Close HTTP/1.1 200 OK Content-Type: text/plain Connection: close Hello world This a connection close response
- 19. HTTP Streaming ➡ Type 2: Chunked response HTTP/1.1 200 OK Content-Type: text/plain Transfer-Encoding: chunked E Hello World! 19 I am a chunked response 0
- 20. HTTP Streaming @app.route("/messages") def messages(): def content(): for sentence in wait_until_next_sentence(): yield sentence return Response(content()) r = requests.get('http://yo.com/messages', stream=True) for message in r.iter_lines(): messages.extend(message.decode(‘utf8’))
- 22. SSE ➡ Well known protocol (HTTP) ➡ Good efficiency ➡ JS API ➡ Few client implementations ➡ Uni-directional
- 24. WebSockets ➡ Bidirectional ➡ Great efficiency ➡ A lot of implementations (JS API) ➡ Cross-domain ➡ Handshake to update from HTTP ➡ Complete different protocol
- 25. WS Handshake GET /chat HTTP/1.1 Host: server.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw== Sec-WebSocket-Protocol: chat, superchat Sec-WebSocket-Version: 13 Origin: http://example.com HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat Wikipedia
- 26. WebSockets @sockets.route('/messages') def messages(ws): while True: message = wait_until_new_message() ws.send(message) def msg(ws, msg): messages.extend(msg) ws = websocket.WebSocketApp(‘ws://api.yo.com/', on_message=msg) ws.on_open = on_open ws.run_forever()
- 28. Bytes IN Bytes Out Total Bytes Time (seconds) Polling 14640 13564 28204 159 Polling 2 7503 7636 15139 200 Long Polling 7503 7636 15139 120 Streaming 183 1549 1732 120 WebSockets 209 1605 1814 120
- 29. 30000 27000 24000 21000 18000 15000 12000 9000 6000 3000 0 Polling Polling 2 Long Polling Streaming WebSockets Total Bytes Bytes Out Bytes IN
- 30. Concurrency and the C10k
- 31. C1 0k? how to handle more than 10k connections simultaneously?
- 32. concurrency is… the ability of running in overlapping time periods, not necessarily at the same time
- 34. processes ➡ Preemptive scheduling by the OS ➡ Separate memory space ➡ No GIL related issues ➡ How to communicate them? ➡ They are really heavy
- 35. threads ➡ Preemptive scheduling by the OS ➡ Same memory space ➡ Faster and lighter than processes ➡ You may suffer the GIL ➡ What about synchronisation? ➡ Race conditions
- 36. user threads ➡ Same space address ➡ Lightest alternative ➡ No race-conditions ➡ You may suffer the GIL ➡ Very bad with CPU bound tasks
- 37. GIL
- 38. I/O methods
- 39. again… how to handle more than 10k connections simultaneously?
- 40. option 1 use 10k machines, one per client, VIP service
- 41. option 2 use blocking and synchronous calls, one per client
- 42. option 3 non-blocking calls to start I/O and then readiness notifications to know when the socket is ready
- 43. option 4 asynchronous calls to start I/O and completion notifications to know when they’ve finished
- 44. but… in Python?
- 47. asyncio ➡ Single-threaded ➡ Async I/O ➡ Coroutines ➡ Multiplexes I/O events ➡ …
- 48. COROUTINES
- 50. demo
- 55. X Game Of Life
- 56. 10K connections
- 59. thanks!