Architecture of MessagePack

• Efﬁcient serialization library • Rich data structures - compatible with JSON • Dynamic typing • Remote Procedure Call (RPC) • Synchronous, Asynchronous and Callback style • Concurrent calls with multiple servers • Event-driven I/O • Interface Deﬁnition Language (IDL) - compatible with Thrift

1. Compact 2. Fast Binary-based format Zero-copy (C++) Embed type information Stream deserialization

JSON MessagePack null null c0 Integer 10 0a Array [20] 91 14 String ”30” a2 ‘3’ ‘0’ Map {“40”:null} 81 a1 ‘4’ ‘0’ c0

JSON MessagePack null null 4 bytes c0 1 byte Integer 10 2 bytes 0a 1 byte Array [20] 4 bytes 91 14 2 bytes String ”30” 4 bytes a2 ‘3’ ‘0’ bytes 3 Map {“40”:null}bytes 11 5 bytes 81 a1 ‘4’ ‘0’ c0

Fixed length types Variable length types Integer Raw bytes Floating point Array Boolean Map Nil type value type length body... Type information

Type information Types 0x00 nil 0xc2 false 0xc3 true 0xca float 0xcb double 0xcc uint8 0xcd uint16 0xc0 0xce uint32 0xe0 0xcf uint64 0xdf int8 ... ...

Type information Types 0x00 0x00 nil 0xc2 false 0xc3 true Positive FixNum 0xca float 0xcb double 0x80 FixMap 0xcc uint8 0x90 0xa0 FixArray 0xcd uint16 0xc0 FixRaw 0xce uint32 0xe0 0xcf uint64 Negative FixNum 0xdf int8 ... ...

It measured the elapsed time of serializing and deserializing 200,000 target objects. The target object consists of the three integers and 512 bytes string.

MessagePack-RPC Inter-process messaging library for clients, servers and cluster applications.

MessagePack-RPC Inter-process messaging library for clients, servers and cluster applications. Concept of Communicates with multiple Future servers concurrently Multithreaded event-driven I/O

require 'msgpack/rpc' client = MessagePack::RPC::Client.new(host, port) result = client.call(:method, arg1, arg2)

require 'msgpack/rpc' client = MessagePack::RPC::Client.new(host, port) future1 = client.call_async(:methodA, arg1, arg2) future2 = client.call_async(:methodB, arg1, arg2) result1 = future1.get result2 = future2.get

require 'msgpack/rpc' client = MessagePack::RPC::Client.new(host, port) client.callback(:method, arg, arg2) do |future| result = future.get end client.join

require 'msgpack/rpc' loop = MessagePack::RPC::Loop.new client1 = MessagePack::RPC::Client.new(host1, port1, loop) client2 = MessagePack::RPC::Client.new(host2, port2, loop) future1 = client1.call_async(:methodA, arg1, arg2) future2 = client2.call_async(:methodB, arg1, arg2) result1 = future1.get result2 = future2.get

require 'msgpack/rpc' sp = MessagePack::RPC::SessionPool.new session1 = sp.get_session(host1, port1) session2 = sp.get_session(host2, port2) future1 = session1.call_async(:methodA, arg1, arg2) future2 = session2.call_async(:methodB, arg1, arg2) result1 = future1.get result2 = future2.get

Client Session Loop Server Client Session Loop Server

Client Session Server Loop Client shared Session event loop Server

Session Server Session Pool pools these Loop connections Session Server

Client Server Dispatcher Loop Client

• Performance of the Loop is important. • Java version uses Netty (JBoss’ I/O framework) •Multithreaded •Utilizes Java’s NIO • C++ version uses mpio (Kumofs’ I/O architecture) •Multithreaded •Utilizes epoll or kqueue • Ruby version uses Rev (libev for Ruby) •Utilizes epoll or kqueue

The MessagePack Project http://msgpack.sourceforge.net/

http://msgpack.wordpress.com	2006
http://log.medcl.net	516
http://blog.netzhou.net	344
http://www.webguo.com	198
http://www.slideshare.net	39
http://paper.li	30
http://cnfist.blogcn.com	14
http://a0.twimg.com	5
http://webcache.googleusercontent.com	5
url_unknown	5
http://cache.baidu.com	5
http://www.ofelio.com	4
http://www.techgig.com	2
https://msgpack.wordpress.com	2
http://twitter.com	1
http://us-w1.rockmelt.com	1
http://flashattackcrew.blogspot.com	1
http://static.slidesharecdn.com	1
http://passing.tk	1
http://www.sogou.com	1

Architecture of MessagePack

by Sadayuki Furuhashi on Jun 03, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

20 Embeds 3,181

Statistics

Architecture of MessagePack — Presentation Transcript