PYClientServer
Python powered Client-Server architecture on
compressed JSON channel
Outline
❖ Intro
❖ Requirements
❖ Architecture
❖ Implementation
❖ Tests
❖ Conclusion
Introduction - 1 of 2
❏ A need: seamless integration using a lightweight data
exchange format
❏ JSON is emerging as the mo...
Introduction - 2 of 2
❏ And, Python? Well, it is an agile main stream
programming language
❏ very well suited for network ...
Requirements
❏ Let’s go through the needs introduced
a. services integration over the network
b. seamless lightweight comm...
Architecture
Client Server
TCP
Channel
Decoder Encoder
Process#1
Thread#1
Process#2
Thread#1
File System
1. JSON request
2...
Implementation - 1 of 5
❏ Key implementation aspects concerning the Server
a. it takes advantage of Python’s powerful netw...
Implementation - 2 of 5
# Read lines from a text file
resource = []
resource.append(self.__resourcepath)
resource.append(s...
Implementation - 3 of 5
❏ As already seen in the Server code, system Data Model
is very important, because
a. it allows to...
Implementation - 4 of 5
class Utility(object):
def compress(self, data):
string = str(data)
return zlib.compress(string, z...
Implementation - 5 of 5
❏ Client thread is the most simple entity involved in the
scenario sketched on the architecture, i...
Tests
❏ Hereafter some screenshots captured during a session
test are proposed
Server-side: it is important to consider ho...
Conclusion - 1 of 2
❏ Architecting the communication protocol for the service
integration as shown in the architectural sk...
❏ Despite the simplified snippets presented, in the
previous slides
❏ Client, Server and Data Model have been presented as...
Upcoming SlideShare
Loading in...5
×

PyClientServer: a Python powererd Client-Server Architecture over a compressed JSON/TCP channel

1,465

Published on

This presentation aims at introducing the interested reader to the very important topics of (i) Data Compression and (ii) Lightweight Network Communication in SoSE (Systems of Systems Engineering). These very interesting topics in the SOA context, in particular for data-centric application integrations. A working implementation is provided in Python, a very powerful main stream programming language.

Check the project out from GitHub at https://github.com/hailpam/pyclientserver.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,465
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

PyClientServer: a Python powererd Client-Server Architecture over a compressed JSON/TCP channel

  1. 1. PYClientServer Python powered Client-Server architecture on compressed JSON channel
  2. 2. Outline ❖ Intro ❖ Requirements ❖ Architecture ❖ Implementation ❖ Tests ❖ Conclusion
  3. 3. Introduction - 1 of 2 ❏ A need: seamless integration using a lightweight data exchange format ❏ JSON is emerging as the modern de-facto standard data format for services integration ❏ TCP is a must, it is reliable and can be secure ❏ JSON over TCP is a trustable solution ❏ Why compression? JSON strings are naturally verbose, but less than XML, so ❏ compression can help in reducing the footprint ❏ bandwidth utilisation can be effectively improved ❏ RTT latency benefits from (much more responsiveness)
  4. 4. Introduction - 2 of 2 ❏ And, Python? Well, it is an agile main stream programming language ❏ very well suited for network communication ❏ very well supported ❏ platform-independent (so, portable solutions) ❏ powerful in string/data processing ❏ ... ❏ PYClientServer represents an attempt to implement a working example of the previously outlined needs
  5. 5. Requirements ❏ Let’s go through the needs introduced a. services integration over the network b. seamless lightweight communication c. effective bandwidth utilisation d. easy extensibility e. rapid development f. working code to extend for enhancements ❏ Let’s summarize as follows a. A Server offering a data-centric service b. A Client asking for the service c. A flexible Data Model offering ❏ marshalling/unmarshalling facility ❏ compression/decompression facility
  6. 6. Architecture Client Server TCP Channel Decoder Encoder Process#1 Thread#1 Process#2 Thread#1 File System 1. JSON request 2. File Reading 3. Text compression 4. Server response 5. Decompressed Text 0101010101 { GET, 100 }
  7. 7. Implementation - 1 of 5 ❏ Key implementation aspects concerning the Server a. it takes advantage of Python’s powerful network library: 1 server handler in few LoC b. it receives text-based request and replies providing compressed data ❏ Let’s have a look at the code class TCPRequestHandler(SocketServer.BaseRequestHandler): … try: print "[TCPRequestHandler][handle] Connection accepted... processing" # Reading request (assuming a small amount of bytes) data = self.request.recv(1024).strip() # Unmarshall the request request = Request('', 0) data = request.from_json(data) …
  8. 8. Implementation - 2 of 5 # Read lines from a text file resource = [] resource.append(self.__resourcepath) resource.append(self.__filename) testfile = open(''.join(resource), 'r') # Prepare the response data response = Data(True, [], 0) list = testfile.readlines() response.vector = list response.nrbytes = len(list) # Marshall JSON representation json_str = response.to_json() c_response = self.__compression.compress(json_str) start = time.time() self.request.sendall(c_response) print "[TCPRequestHandler][handle] Bunch of compressed data sent back!" end = time.time() if (self.__isdebug): print "Delivery::Time Elapsed::", str(end - start)
  9. 9. Implementation - 3 of 5 ❏ As already seen in the Server code, system Data Model is very important, because a. it allows to seamlessly marshall/unmarshall the data b. it allows to perform compression/decompression, according to the communication protocol class Request(object): def __init__(self, name, code): self.name = name self.code = code def to_json(self): self.__dict__['name'] = self.name self.__dict__['code'] = self.code return json.dumps(self.__dict__, indent=4) def from_json(self, sting): dict = json.loads(sting) self.__dict__ = dict
  10. 10. Implementation - 4 of 5 class Utility(object): def compress(self, data): string = str(data) return zlib.compress(string, zlib.Z_BEST_COMPRESSION) def decompress(self, data): string = str(zlib.decompress(data)) return string ❏ Till now, Server and Data Model have been briefly discussed, but those are only simple snippets of code ❏ To have a look at the full code of Server and Data Model, check the project out on GitHub.
  11. 11. Implementation - 5 of 5 ❏ Client thread is the most simple entity involved in the scenario sketched on the architecture, in fact, it acts as follows ❏ send a pure JSON request (Data Model aids in doing that) ❏ upon a reception of server data, it performs the decompression as dictated by the defined communication protocol # Request creation data = Request('GET', 100) # Client socket binding sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((self.__serverhost, self.__serverport)) # Sending JSON data over the socket sock.send(data.to_json()) response = self.__receive_data(sock) # Treating compressed data result = self.__compression.decompress(response) data = Data(False, [], 0) data.from_json(result)
  12. 12. Tests ❏ Hereafter some screenshots captured during a session test are proposed Server-side: it is important to consider how the compression reduces drastically the dimensions (form 4.2MB to 1MB) Client-Side: in less than 1 second, the data are received. Afterwards, data are decompressed and the first line of the document is disaplyed
  13. 13. Conclusion - 1 of 2 ❏ Architecting the communication protocol for the service integration as shown in the architectural sketch, brings several advantages a. effective bandwidth utilization: the compression reduces the verbosity b. much more responsiveness: the entire data frame is trasferred in less than a second ❏ Python network programming is really powerful: with few lines of code, the application programmer can define and deploy a working multithreaded service ❏ JSON format is really amazing to work with in Python: as shown, with few lines of code, the library makes the serialization and deserialization a pretty easy job
  14. 14. ❏ Despite the simplified snippets presented, in the previous slides ❏ Client, Server and Data Model have been presented as the three main entities involved by design, but some other interesting features have to be discovered ❏ System Architecture has been depicted and, as seen, it is simple but powerful at the same time: there is the reason to believe that that Architecture might scale very well ❏ If you want to contribute or to have a deeper look at the code, do not hesitate to check the project out from GitHub. Conclusion - 2 of 2
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×