PyClientServer: a Python powererd Client-Server Architecture over a compressed JSON/TCP channel
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

PyClientServer: a Python powererd Client-Server Architecture over a compressed JSON/TCP channel

  • 1,432 views
Uploaded on

This presentation aims at introducing the interested reader to the very important topics of (i) Data Compression and (ii) Lightweight Network Communication in SoSE (Systems of Systems Engineering).......

This presentation aims at introducing the interested reader to the very important topics of (i) Data Compression and (ii) Lightweight Network Communication in SoSE (Systems of Systems Engineering). These very interesting topics in the SOA context, in particular for data-centric application integrations. A working implementation is provided in Python, a very powerful main stream programming language.

Check the project out from GitHub at https://github.com/hailpam/pyclientserver.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,432
On Slideshare
1,432
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PYClientServer Python powered Client-Server architecture on compressed JSON channel
  • 2. Outline ❖ Intro ❖ Requirements ❖ Architecture ❖ Implementation ❖ Tests ❖ Conclusion
  • 3. Introduction - 1 of 2 ❏ A need: seamless integration using a lightweight data exchange format ❏ JSON is emerging as the modern de-facto standard data format for services integration ❏ TCP is a must, it is reliable and can be secure ❏ JSON over TCP is a trustable solution ❏ Why compression? JSON strings are naturally verbose, but less than XML, so ❏ compression can help in reducing the footprint ❏ bandwidth utilisation can be effectively improved ❏ RTT latency benefits from (much more responsiveness)
  • 4. Introduction - 2 of 2 ❏ And, Python? Well, it is an agile main stream programming language ❏ very well suited for network communication ❏ very well supported ❏ platform-independent (so, portable solutions) ❏ powerful in string/data processing ❏ ... ❏ PYClientServer represents an attempt to implement a working example of the previously outlined needs
  • 5. Requirements ❏ Let’s go through the needs introduced a. services integration over the network b. seamless lightweight communication c. effective bandwidth utilisation d. easy extensibility e. rapid development f. working code to extend for enhancements ❏ Let’s summarize as follows a. A Server offering a data-centric service b. A Client asking for the service c. A flexible Data Model offering ❏ marshalling/unmarshalling facility ❏ compression/decompression facility
  • 6. Architecture Client Server TCP Channel Decoder Encoder Process#1 Thread#1 Process#2 Thread#1 File System 1. JSON request 2. File Reading 3. Text compression 4. Server response 5. Decompressed Text 0101010101 { GET, 100 }
  • 7. Implementation - 1 of 5 ❏ Key implementation aspects concerning the Server a. it takes advantage of Python’s powerful network library: 1 server handler in few LoC b. it receives text-based request and replies providing compressed data ❏ Let’s have a look at the code class TCPRequestHandler(SocketServer.BaseRequestHandler): … try: print "[TCPRequestHandler][handle] Connection accepted... processing" # Reading request (assuming a small amount of bytes) data = self.request.recv(1024).strip() # Unmarshall the request request = Request('', 0) data = request.from_json(data) …
  • 8. Implementation - 2 of 5 # Read lines from a text file resource = [] resource.append(self.__resourcepath) resource.append(self.__filename) testfile = open(''.join(resource), 'r') # Prepare the response data response = Data(True, [], 0) list = testfile.readlines() response.vector = list response.nrbytes = len(list) # Marshall JSON representation json_str = response.to_json() c_response = self.__compression.compress(json_str) start = time.time() self.request.sendall(c_response) print "[TCPRequestHandler][handle] Bunch of compressed data sent back!" end = time.time() if (self.__isdebug): print "Delivery::Time Elapsed::", str(end - start)
  • 9. Implementation - 3 of 5 ❏ As already seen in the Server code, system Data Model is very important, because a. it allows to seamlessly marshall/unmarshall the data b. it allows to perform compression/decompression, according to the communication protocol class Request(object): def __init__(self, name, code): self.name = name self.code = code def to_json(self): self.__dict__['name'] = self.name self.__dict__['code'] = self.code return json.dumps(self.__dict__, indent=4) def from_json(self, sting): dict = json.loads(sting) self.__dict__ = dict
  • 10. Implementation - 4 of 5 class Utility(object): def compress(self, data): string = str(data) return zlib.compress(string, zlib.Z_BEST_COMPRESSION) def decompress(self, data): string = str(zlib.decompress(data)) return string ❏ Till now, Server and Data Model have been briefly discussed, but those are only simple snippets of code ❏ To have a look at the full code of Server and Data Model, check the project out on GitHub.
  • 11. Implementation - 5 of 5 ❏ Client thread is the most simple entity involved in the scenario sketched on the architecture, in fact, it acts as follows ❏ send a pure JSON request (Data Model aids in doing that) ❏ upon a reception of server data, it performs the decompression as dictated by the defined communication protocol # Request creation data = Request('GET', 100) # Client socket binding sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((self.__serverhost, self.__serverport)) # Sending JSON data over the socket sock.send(data.to_json()) response = self.__receive_data(sock) # Treating compressed data result = self.__compression.decompress(response) data = Data(False, [], 0) data.from_json(result)
  • 12. Tests ❏ Hereafter some screenshots captured during a session test are proposed Server-side: it is important to consider how the compression reduces drastically the dimensions (form 4.2MB to 1MB) Client-Side: in less than 1 second, the data are received. Afterwards, data are decompressed and the first line of the document is disaplyed
  • 13. Conclusion - 1 of 2 ❏ Architecting the communication protocol for the service integration as shown in the architectural sketch, brings several advantages a. effective bandwidth utilization: the compression reduces the verbosity b. much more responsiveness: the entire data frame is trasferred in less than a second ❏ Python network programming is really powerful: with few lines of code, the application programmer can define and deploy a working multithreaded service ❏ JSON format is really amazing to work with in Python: as shown, with few lines of code, the library makes the serialization and deserialization a pretty easy job
  • 14. ❏ Despite the simplified snippets presented, in the previous slides ❏ Client, Server and Data Model have been presented as the three main entities involved by design, but some other interesting features have to be discovered ❏ System Architecture has been depicted and, as seen, it is simple but powerful at the same time: there is the reason to believe that that Architecture might scale very well ❏ If you want to contribute or to have a deeper look at the code, do not hesitate to check the project out from GitHub. Conclusion - 2 of 2