Architecture of MessagePack

35,510 views

Published on

Published in: Technology, Education
4 Comments
53 Likes
Statistics
Notes
No Downloads
Views
Total views
35,510
On SlideShare
0
From Embeds
0
Number of Embeds
11,101
Actions
Shares
0
Downloads
367
Comments
4
Likes
53
Embeds 0
No embeds

No notes for slide

Architecture of MessagePack

  1. Architecture of MessagePack
  2. • Efficient serialization library • Rich data structures - compatible with JSON • Dynamic typing • Remote Procedure Call (RPC) • Synchronous, Asynchronous and Callback style • Concurrent calls with multiple servers • Event-driven I/O • Interface Definition Language (IDL) - compatible with Thrift
  3. 1. Compact 2. Fast Binary-based format Zero-copy (C++) Embed type information Stream deserialization
  4. JSON MessagePack null null c0 Integer 10 0a Array [20] 91 14 String ”30” a2 ‘3’ ‘0’ Map {“40”:null} 81 a1 ‘4’ ‘0’ c0
  5. JSON MessagePack null null 4 bytes c0 1 byte Integer 10 2 bytes 0a 1 byte Array [20] 4 bytes 91 14 2 bytes String ”30” 4 bytes a2 ‘3’ ‘0’ bytes 3 Map {“40”:null}bytes 11 5 bytes 81 a1 ‘4’ ‘0’ c0
  6. Fixed length types Variable length types Integer Raw bytes Floating point Array Boolean Map Nil type value type length body... Type information
  7. Type information Types 0x00 nil 0xc2 false 0xc3 true 0xca float 0xcb double 0xcc uint8 0xcd uint16 0xc0 0xce uint32 0xe0 0xcf uint64 0xdf int8 ... ...
  8. Type information Types 0x00 0x00 nil 0xc2 false 0xc3 true Positive FixNum 0xca float 0xcb double 0x80 FixMap 0xcc uint8 0x90 0xa0 FixArray 0xcd uint16 0xc0 FixRaw 0xce uint32 0xe0 0xcf uint64 Negative FixNum 0xdf int8 ... ...
  9. It measured the elapsed time of serializing and deserializing 200,000 target objects. The target object consists of the three integers and 512 bytes string.
  10. MessagePack-RPC Inter-process messaging library for clients, servers and cluster applications.
  11. MessagePack-RPC Inter-process messaging library for clients, servers and cluster applications. Concept of Communicates with multiple Future servers concurrently Multithreaded event-driven I/O
  12. require 'msgpack/rpc' client = MessagePack::RPC::Client.new(host, port) result = client.call(:method, arg1, arg2)
  13. require 'msgpack/rpc' client = MessagePack::RPC::Client.new(host, port) future1 = client.call_async(:methodA, arg1, arg2) future2 = client.call_async(:methodB, arg1, arg2) result1 = future1.get result2 = future2.get
  14. require 'msgpack/rpc' client = MessagePack::RPC::Client.new(host, port) client.callback(:method, arg, arg2) do |future| result = future.get end client.join
  15. require 'msgpack/rpc' loop = MessagePack::RPC::Loop.new client1 = MessagePack::RPC::Client.new(host1, port1, loop) client2 = MessagePack::RPC::Client.new(host2, port2, loop) future1 = client1.call_async(:methodA, arg1, arg2) future2 = client2.call_async(:methodB, arg1, arg2) result1 = future1.get result2 = future2.get
  16. require 'msgpack/rpc' sp = MessagePack::RPC::SessionPool.new session1 = sp.get_session(host1, port1) session2 = sp.get_session(host2, port2) future1 = session1.call_async(:methodA, arg1, arg2) future2 = session2.call_async(:methodB, arg1, arg2) result1 = future1.get result2 = future2.get
  17. Client Session Loop Server Client Session Loop Server
  18. Client Session Server Loop Client shared Session event loop Server
  19. Session Server Session Pool pools these Loop connections Session Server
  20. Client Server Dispatcher Loop Client
  21. • Performance of the Loop is important. • Java version uses Netty (JBoss’ I/O framework) •Multithreaded •Utilizes Java’s NIO • C++ version uses mpio (Kumofs’ I/O architecture) •Multithreaded •Utilizes epoll or kqueue • Ruby version uses Rev (libev for Ruby) •Utilizes epoll or kqueue
  22. The MessagePack Project http://msgpack.sourceforge.net/

×