Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kamaelia Protocol Walkthrough

4,353 views

Published on

A walkthrough of what's going on in this example: http://tinyurl.com/4k3df2

  • Be the first to comment

Kamaelia Protocol Walkthrough

  1. A walk through building & stacking protocols using Kamaelia Michael Sparks October 2008
  2. The aim of this document is to guide you through building a system where: ● A user connects to a server and ● Over a secure connection, ● Receives a sequence of JSON encode-able objects
  3. The aim of this document is to guide you through building a system where: ● A user connects to a server and ● Over a secure connection, This doc also uses speech ● Receives a sequence of JSON make “aside” bubbles like this to encode-able comments regarding particlar objects panels. You can skip these safely.
  4. ServerCore First of all, we start off with the ServerCore component. The hole is for fitting a protocol handler factory. eg a class, or function.
  5. ServerCore As an aside, this hole is pretty crucial. Without filling this hole, this component is pretty useless. So we call this sort of component a CHASSIS First of all, we start off with the ServerCore component. The hole is for fitting a protocol handler factory. eg a class, or function.
  6. ServerCore «factory» protocol So, we provide a protocol factory. ServerCore(protocol=protocol, port=2345)
  7. ServerCore «factory» protocol In retrospect, “protocol” as the name of an example protocol handler is a bad idea, so let's change this slightly. So, we provide a protocol factory. ServerCore(protocol=protocol, port=2345)
  8. ServerCore «factory» stackedjson So, we provide a protocol factory. ServerCore(protocol=stackedjson, port=2345)
  9. ServerCore «factory» stackedjson We then activate it, which causes it to start a subcomponent. ServerCore( .... )
  10. ServerCore TCPServer «factory» stackedjson We then activate it. It starts TCPServer subcomponent, which listens for connections. ServerCore( .... ).activate()
  11. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson A client then connects to the TCPServer which creates a ConnectedSocketAdapter to handle the mechanics of the connection.
  12. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson stackedjson TCPServer tells ServerCore it has done this. ServerCore in response calls the protocol factory to create a handler component...
  13. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson stackedjson ...which is wired up. The protocol handler just receives bytes and sends bytes, which are sent to the client via the ConnectedSocketAdapter
  14. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson stackedjson There, the key thing in creating a server, is to create an appropriate protocol handler component.
  15. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson stackedjson There, the key thing in creating a server, is to create an appropriate protocol handler component.
  16. stackedjson So, let's look inside this component
  17. stackedjson def protocol(*args,**argd): return Pipeline( PeriodicWakeup(message=quot;NEXTquot;, interval=1), Chooser(messages), MarshallJSON(), Encrypter(), DataChunker(), ) So, let's look inside this component
  18. stackedjson def protocol(*args,**argd): return Pipeline( PeriodicWakeup(message=quot;NEXTquot;, interval=1), Chooser(messages), MarshallJSON(), Encrypter(), DataChunker(), ) So, let's look inside this component. Clearly this is what actually handles the connection.
  19. PeriodicWakeup Chooser Pipeline( PeriodicWakeup(message=quot;NEXTquot;, MarshallJSON interval=1), Chooser(messages), Encrypter MarshallJSON(), Encrypter(), DataChunker DataChunker(), ) Let's expand that...
  20. PeriodicWakeup Chooser MarshallJSON Direction of Encrypter data flow DataChunker We see this is a uni-directional protocol – as soon as this pipeline starts up. Encrypted JSON data chunks get sent, ignoring client data.
  21. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Direction of Encrypter data flow Decrypter DataChunker DataDeChunker TCPClient So, given the ServerCore infrastructure means we can more or less ignore the network, what does the client side pipeline look like?
  22. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient Each component in the server, that is part of the user's protocol, has a matching component in the client.
  23. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient Fundamentally, this protocol allows one side of it to securely throw dictionary objects to clients which can then do something with them
  24. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient To understand what a client sees, we just look at these three components. The 2 highlighted server side components are the core behaviour.
  25. PeriodicWakeup Chooser ConsoleEchoer Let's rearrange these, showing the logical pipeline that drives what the user sees.
  26. PeriodicWakeup( PeriodicWakeup message=quot;NEXTquot;, interval=1) Chooser ConsoleEchoer First of all the PeriodicWakeupComponent sends out the message “NEXT” once per second.
  27. Messages = [ PeriodicWakeup {quot;helloquot;: quot;worldquot; }, {quot;helloquot;: [1,2,3] }, {quot;worldquot;: [1,2,3] }, {quot;worldquot;: {quot;gamequot;:quot;overquot;} }, Chooser ]*10 Chooser(messages) ConsoleEchoer These “NEXT” messages control the Chooser. The chooser steps through a list of messages, and when told “NEXT”, sends the next message.
  28. Messages = [ PeriodicWakeup {quot;helloquot;: quot;worldquot; }, {quot;helloquot;: [1,2,3] }, {quot;worldquot;: [1,2,3] }, It's worth noting that the Chooser Chooser can back backwards{quot;gamequot;:quot;overquot;} }, {quot;worldquot;: as well, just to ]*10 the start and end etc. For more Chooser(messages)at the docs :) details look ConsoleEchoer These “NEXT” messages control the Chooser. The chooser steps through a list of messages, and when told “NEXT”, sends the next message.
  29. PeriodicWakeup Chooser ConsoleEchoer(use_repr=True) ConsoleEchoer These messages arrive intact at the client - in this case a ConsoleEchoer. However it could be something more interesting – eg a game system.
  30. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient Going back to the overview slide, this diagram shows the various protocols which are stacked on top of each other.
  31. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient We have our core application protocol, stacked on top of a secure serialising communications channel. This truly is at the application level.
  32. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient We have our serialisation protocol. Clearly this is a message oriented protocol – it requires the lower layer to preserve boundaries.
  33. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient We have a secure transmission protocol. This is admittedly simplistic – it assumes predistributed keys. However it gives a flavour
  34. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient We then have our chunking protocol, which preserves message boundaries – necessary because TCP does not guarantee to do so.
  35. PeriodicWakeup Chooser ConsoleEchoer MarshallJSON DeMarshallJSON Encrypter Decrypter DataChunker DataDeChunker TCPClient So let's go back and see where these things fit back into the system.
  36. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson stackedjson So this is what we had ...
  37. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson stackedjson The client really looks like this... Also, this is a one way protocol
  38. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson The client really looks like this & this is a one way protocol. The protocol itself looks like this. And that's how those parts fit together.
  39. ServerCore Connected TCPServer SocketAdapter «factory» stackedjson So, finally, for completeness, we'll include the code.
  40. So, finally, for completeness, we'll include the code. Though, for code, this is a better location: http://tinyurl.com/4k3df2
  41. Bunch of imports... import cjson import Axon from Kamaelia.Chassis.Pipeline import Pipeline from Kamaelia.Util.Chooser import Chooser from Kamaelia.Util.Console import ConsoleEchoer from Kamaelia.Internet.TCPClient import TCPClient from Kamaelia.Chassis.ConnectedServer import ServerCore from Kamaelia.Protocol.Framing import DataChunker, DataDeChunker from Kamaelia.Apps.Grey.PeriodicWakeup import PeriodicWakeup from Crypto.Cipher import DES
  42. Define the messages being slung to clients messages = [ {quot;helloquot;: quot;worldquot; }, {quot;helloquot;: [1,2,3] }, {quot;worldquot;: [1,2,3] }, {quot;worldquot;: {quot;gamequot;:quot;overquot;} }, ]*10
  43. Then the JSON De/Marshalling components class MarshallJSON(Axon.Component.component): def main(self): while not self.dataReady(quot;controlquot;): for j in self.Inbox(quot;inboxquot;): j_encoded = cjson.encode(j) self.send(j_encoded, quot;outboxquot;) if not self.anyReady(): self.pause() yield 1 class DeMarshallJSON(Axon.Component.component): def main(self): while not self.dataReady(quot;controlquot;): for j in self.Inbox(quot;inboxquot;): j_decoded = cjson.decode(j) self.send(j_decoded, quot;outboxquot;) if not self.anyReady(): self.pause() yield 1
  44. Define our encoding layer using normal code...1 class Encoder(object): quot;quot;quot;Null encoder/base encoder - returns the same string for encode/decodequot;quot;quot; def __init__(self, key, **argd): super(Encoder, self).__init__(**argd) self.__dict__.update(argd) self.key = key def encode(self, some_string): return some_string def decode(self, some_string): return some_string
  45. Define our encoding layer using normal code...2 class DES_CRYPT(Encoder): def __init__(self, key, **argd): super(DES_CRYPT, self).__init__(key, **argd) self.key = self.pad_eight(key)[:8] self.obj = obj=DES.new(self.key, DES.MODE_ECB) def encode(self, some_string): padded = self.pad_eight(some_string) encrypted = self.obj.encrypt(padded) return encrypted def decode(self, some_string): padded = self.obj.decrypt(some_string) decoded = self.unpad(padded) return decoded # ... continued
  46. Define our encoding layer using normal code...3 # class DES_CRYPT(Encoder): # ... continued from before def pad_eight(self, some_string): X = len(some_string) if X % 8 != 0: pad_needed = 8-X % 8 else: pad_needed = 8 pad_needed = 8-(X % 8) PAD = pad_needed * chr(pad_needed) return some_string+PAD def unpad(self, some_string): x = ord(some_string[-1]) return some_string[:-x]
  47. Create Encryption components using this class Encrypter(Axon.Component.component): key = quot;ABCDquot; def main(self): crypter = DES_CRYPT(self.key) while not self.dataReady(quot;controlquot;): for j in self.Inbox(quot;inboxquot;): j_encoded = crypter.encode(j) self.send(j_encoded, quot;outboxquot;) if not self.anyReady(): self.pause() yield 1 Note, this is not a serious protocol, it is an illustration. This form of pre-shared key would be a bad idea, but the pattern of usage will be useful.
  48. Create Decryption components using this class Decrypter(Axon.Component.component): key = quot;ABCDquot; def main(self): crypter = DES_CRYPT(self.key) while not self.dataReady(quot;controlquot;): for j in self.Inbox(quot;inboxquot;): j_decoded = crypter.decode(j) self.send(j_decoded, quot;outboxquot;) if not self.anyReady(): self.pause() yield 1 Note, this is not a serious protocol, it is an illustration. This form of pre-shared key would be a bad idea, but the pattern of usage will be useful.
  49. Create the protocol handler factory & Client def stackedjson(*args,**argd): return Pipeline( PeriodicWakeup(message=quot;NEXTquot;, interval=1), Chooser(messages), MarshallJSON(), Encrypter(), # Encrypt on the way out DataChunker(), ) def json_client_prefab(ip, port): return Pipeline( TCPClient(ip, port=port), DataDeChunker(), Decrypter(), # Decrypt on the way in DeMarshallJSON(), ConsoleEchoer(use_repr=True) )
  50. And finally start the server and point a client at the server. ServerCore(protocol=protocol, port=2345).activate() json_client_prefab(quot;127.0.0.1quot;, 2345).run() ServerCore Connected TCPServer SocketAdapter «factory» stackedjson

×