Realtime Streaming using Autobahn Websockets

5,399 views

Published on

This is a talk presented at #LSPE (Large-Scale Production Engineering) in the Bay Area on November 21, 2013.

Published in: Technology, Business

Realtime Streaming using Autobahn Websockets

  1. 1. Realtime Streaming using Autobahn Websockets TOM SHEFFLER NOVEMBER 2013
  2. 2. Realtime Telemetry: Wazwot Cloud Service ‣ websocket HTML5+Javascript realtime multimedia ‣ audio, images, telemetry ‣ rendered using HTML5+Javascript ‣ portable across iOS, IE10, Android, Chrome, Chromebook, Firefox, Aurora Sheffler
  3. 3. Outline Twisted ‣ background Autobahn ‣ Websockets on Twisted Building an in-memory application ‣ resource hierarchy ‣ mixed resources - no middle tier Real-Time considerations in Application layer ‣ tcp flow control Performance in the real world What’s Next ‣ Twisted, Autobahn, Wazwot Sheffler
  4. 4. Twisted • Asynchronous framework begun 2001 • granddaddy of async • inspired Tornado(Facebook) • response to the “c10K” problem [Kegel] • epoll/kqueue • rock-solid foundation (my opinion). • deferred - a programming construct • a promise that a result will be resolved with data later • related to futures • adopted by jQuery:Deferred Sheffler
  5. 5. Autobahn • A WebSocket framework built on Twisted • scales to 10s of thousands of connections • Robust Test Suite - 300 tests • pseudo-random “fuzzing” • self-checking Sheffler
  6. 6. Twisted 101 Protocols and Transports Sheffler
  7. 7. Protocols and Transports Protocol Internal State bi-di data Transport write() def dataReceived(): pass Twisted machinery connects a transport to protocol. Base methods: ‣ makeConnection, connectionMade, connectionLost Sheffler
  8. 8. Customizing a Protocol Protocol bi-di data Internal State Transport write() def dataReceived(): pass inherit MyProtocol def dataReceived(self, data): self.transport.write(data) Sheffler
  9. 9. Websocket Protocol Sheffler
  10. 10. Autobahn Websocket WebSocketServerProtocol Internal State bi-di data Transport write() def dataReceived(): ... self.onMessage(payload) def sendMessage(payload): ... self.transport.write(...) Sheffler
  11. 11. Autobahn Echo WebSocketServerProtocol bi-di data Internal State Transport write() def dataReceived(): ... self.onMessage(payload) def sendMessage(payload): ... self.transport.write(...) inherit MyProtocol def onOpen(self): pass def onConnect(self): pass def onMessage(self, payload): self.sendMessage(payload) Sheffler
  12. 12. Twisted Resources • Persistent Data Structures •contrast to routes and req/resp •most Twisted apps are an in-memory DB Sheffler
  13. 13. Resource Hierarchy In-Memory Data Structure SITE Site-Map /res1 /res1/child1 /res1/child2 /res2 /res2/child1 /res2/child2 res1 Port 80 HTTP: GET/POST PUT/ DELETE chidl1 child2 res2 child1 child2 Sheffler
  14. 14. Resources: Code class MyResource(Resource): def __init__(self, name): self.name = name def render_GET(self, request): data = {‘value’:‘hello world’, ‘name’:self.name} return json.dumps(data) def render_POST(self, request): jdata = json.loads(request.content.read()) .... Sheffler
  15. 15. Resources: hierarchy # Define the hierarchy root = Resource() # Define resource One and its children r1 = MyResource(“resource1”) r1c1 = MyResource(“resource1.child1”) r1.putChild(“c1”, r1c1) r1c2 = MyResource(“resource1.child2”) r1.putChild(“c2”, r1c2) # Define resource Two and its children ... skipped ... # Put the two resources under the root root.putChild(“res1”, r1) root.putChild(“res2”, r2) # start serving the site on port 80 site = server.Site(root) http_service = internet.TCPServer(80, Site) Sheffler
  16. 16. Mixed Hierarchy In-Memory Data Structure SITE Site-Map /res1 /res1/child1 /res1/child2 ws:/res1/wx /res2 /res2/child1 /res2/child2 ws:/res1/wx res1 Port 80 HTTP: GET/POST PUT/ DELETE chidl1 child2 wx res2 child1 wx • no middle tier WebSocket: message child2 Sheffler
  17. 17. WebSocket Resources class MyWebsocketResource(WebSocketResource): def __init__(self, name): f = WebSocketServerFactory(“ws://myhost:80”) f.name = name f.protocol = MyWebSocketProtocol WebSocketResource.__init__(self, f) class MyWebsocketProtocol(WebSocketServerProtocol): def connectionMade(self): self.sendMessage({“hello from” : self.factory.name}) def onMessage(self, payload): self.sendMessage(payload) Sheffler
  18. 18. Instantiate Hierarchy # Define the hierarchy root = Resource() # Define resource One and its children r1 = MyResource(“resource1”) ... r1wx = MyWebSocketResource(“resource1.websocket1”) r1.putChild(“wx”, r1wx) # Define resource Two and its children ... r2wx = MyWebSocketResource(“resource2.websocket2”) r2.putChild(“wx”, res1c1) # Put the two resources under the root root.putChild(“res1”, r1) root.putChild(“res2”, r2) # start serving the site on port 80 site = server.Site(root) http_service = internet.TCPServer(80, Site) Sheffler
  19. 19. TCP Backpressure • Twisted is producer/consumer all the way down. • So is Autobahn. Sheffler
  20. 20. Wazwot Cloud Server websocket HTML5+Javascript assembly buffer approximates continuous stream as series of audio packets over websockets Sheffler
  21. 21. Real-time data is buffered until memory is gone Protocol Internal State Transport write() dataReceived() sendMessage() inherit MyProtocol def onOpen(self): pass def onConnect(self): pass what happens when more data is pushed than the transport can deliver? Sheffler
  22. 22. Producer-Consumer Protocol bi-di data Internal State Transport write() dataReceived() IPushProducer sendMessage() pause() resume() inherit push() MyProtocol def onOpen(self): self.registerProducer(producer) def onConnect(self): pass def mySender(self): self.producer.push() the producer coordinates the use of the consumer (e.g. - the transport) Sheffler
  23. 23. using a producer # callback after handshake completed def onOpen(self): # Rather than calling sendMessage directly use FlowControl self.producer = FlowControl(self) self.registerProducer(self.producer, True) self.producer.resumeProducing() Sheffler
  24. 24. a producer for an infinite stream class FlowControl: implements(interfaces.IPushProducer) def __init__(self, proto): self.proto = proto self.started = False self.paused = False def pauseProducing(self): self.paused = True def resumeProducing(self): self.paused = False def stopProducing(self): pass # This method puts multimedia data onto the channel def push(self, message): if not self.paused: self.proto.sendMessage(message, binary=False) else: ... implement priority policy for queued messages ... Sheffler
  25. 25. Performance Sheffler
  26. 26. Twisted Node at Sensr 80GB/day 1/4 $20 CPU >50Mbps http://photo/camN HTTP POST depot camN HTTP POST switchyard haproxy depot camP HTTP POST depot camQ HTTP POST depot camR TWISTED Sheffler
  27. 27. Twisted, haproxy and CV 2TB/day fxp txp00 txp10 FTP/S or HTTP/S fxp txp01 txp11 ha txp02 txp12 fxp fxp balanced sanitized fxp fxp txp09 txp19 ha fxp fxp Auth, sanitization and BW shaping moved way out to the edge. Reboot every month. Just because. fxp public ntwk private ntwk Sheffler
  28. 28. Summary Twisted + Autobahn + Wazwot ‣ mixed resource hierarchy ‣ producer/consumer all the way down ‣ real-time constraints can be handled in application: back-pressure ‣ scalable and reliable Sheffler
  29. 29. Future and Trends • Autobahn • multiprocessor support soon (pre-fork. nothing shared) • http://crossbar.io open-source multi-protocol app router • more info: http://autobahn.ws/ • Twisted • inspiring Python’s Tulip • will there be a merger? • Tulip is an Async core. Twisted is a protocol framework. • Likely they will both exist going forward. • Wazwot • https://itunes.apple.com/us/app/wazwot/id684986597?mt=8 • media-source extensions to smooth playback experience Sheffler

×