The document discusses MessagePack, an efficient binary serialization format that is like JSON but faster and more compact. It describes MessagePack's format which uses a single byte to identify the type of data, such as integers, floats, booleans, strings, arrays and maps. It also demonstrates how to serialize and deserialize objects to and from MessagePack format using the MsgPack.Cli library in C#. Finally, it compares the performance of MessagePack to Protocol Buffers by measuring the size and speed of serializing and deserializing different amounts of data.

1. MESSAGEPACK - AN EFFICIENT
BINARY SERIALIZATION FORMAT
Larry Nung

2. AGENDA
Introduction
Format
MsgPack.Cli
Messagepack VS Protobuf
Reference
Q & A
2

3. INTRODUCTION
3

4. INTRODUCTION
 An efficient binary serialization format
 Like JSON. but fast and small.
4

5. FORMAT
5

6. FORMAT
6

7. FORMAT
7

8. FORMAT
format name first byte (in binary) first byte (in hex)
positive fixint 0xxxxxxx 0x00 - 0x7f
fixmap 1000xxxx 0x80 - 0x8f
fixarray 1001xxxx 0x90 - 0x9f
fixstr 101xxxxx 0xa0 - 0xbf
nil 11000000 0xc0
(never used) 11000001 0xc1
false 11000010 0xc2
true 11000011 0xc3
bin 8 11000100 0xc4
bin 16 11000101 0xc5
bin 32 11000110 0xc6 8

9. FORMAT
format name first byte (in binary) first byte (in hex)
ext 8 11000111 0xc7
ext 16 11001000 0xc8
ext 32 11001001 0xc9
float 32 11001010 0xca
float 64 11001011 0xcb
uint 8 11001100 0xcc
uint 16 11001101 0xcd
uint 32 11001110 0xce
uint 64 11001111 0xcf
int 8 11010000 0xd0
int 16 11010001 0xd1 9

10. FORMAT
format name first byte (in binary) first byte (in hex)
int 32 11010010 0xd2
int 64 11010011 0xd3
fixext 1 11010100 0xd4
fixext 2 11010101 0xd5
fixext 4 11010110 0xd6
fixext 8 11010111 0xd7
fixext 16 11011000 0xd8
str 8 11011001 0xd9
str 16 11011010 0xda
str 32 11011011 0xdb
array 16 11011100 0xdc 10

11. FORMAT
format name first byte (in binary) first byte (in hex)
array 32 11011101 0xdd
map 16 11011110 0xde
map 32 11011111 0xdf
negative fixint 111xxxxx 0xe0 - 0xff
11

12. FORMAT
12
 Fixed length types
 Integer
 Floating point
 Boolean
 Nil

13. FORMAT
13
 Nil => 11000000 => 0xc0
 Boolean
 False => 11000010 => 0xc2
 True => 11000011 => 0xc3
 Positive fixint
 0 => 00000000 => 0x00
 1 => 00000001 => 0x01

14. FORMAT
14
 Variable length types
 Raw bytes
 Array
 Map

15. FORMAT
 Fixstr
 Compact => 10100007 01100011 01101111 01101101
01110000 01100001 01100011 01110100 => a7 63 6f 6d
70 61 63 74
15

16. MSGPACK.CLI
16

17. MSGPACK.CLI
 Install-Package MsgPack.Cli
17

18. MSGPACK.CLI
using MsgPack.Serialization;
...
public static byte[] Serialize<T>(T thisObj) {
var serializer = SerializationContext.Default.GetSerializer<T>();
using (var ms = new MemoryStream()) {
serializer.Pack(ms, thisObj);
return ms.ToArray();
}
}
public static T Deserialize<T>(byte[] bytes) {
var serializer = SerializationContext.Default.GetSerializer<T>();
using (var byteStream = new MemoryStream(bytes)) {
return serializer.Unpack(byteStream);
}
} 18

19. MSGPACK.CLI
public class OldPerson {
[MessagePackMember(0)]
public String Name { get; set; }
[MessagePackMember(1)]
public String NickName { get; set; }
[MessagePackIgnore]
public Object Tag { get; set; }
}
public class NewPerson {
[MessagePackMember(2)]
public String ID { get; set; }
[MessagePackMember(0)]
public String Name { get; set; }
[MessagePackMember(1)]
public String NickName { get; set; }
}
19

20. MSGPACK.CLI
…
var larry = new OldPerson {
Name = "Larry Nung",
NickName = "Larry"
};
var bytes = Serialize(larry);
var person = Deserialize<NewPerson>(bytes);
Console.WriteLine("{0} ({1})", person.NickName,
person.Name);
…
20

21. MESSAGEPACK VS PROTOBUF
21

22. MESSAGEPACK VS PROTOBUF
private static void SerializeToStream<T>(Stream stream, T obj) {
stream.Seek(0, SeekOrigin.Begin);
Serializer.Serialize(stream, obj);
}
private static T DeSerializeFromStream<T>(Stream stream) {
stream.Seek(0, SeekOrigin.Begin);
return Serializer.Deserialize<T>(stream);
}
public static void Serialize<T>(Stream stream, T thisObj) {
stream.Seek(0, SeekOrigin.Begin);
var serializer = SerializationContext.Default.GetSerializer<T>();
serializer.Pack(stream, thisObj);
}
public static T Deserialize<T>(Stream stream) {
stream.Seek(0, SeekOrigin.Begin);
var serializer = SerializationContext.Default.GetSerializer<T>();
return serializer.Unpack(stream);
}
22

23. MESSAGEPACK VS PROTOBUF
var counts = new int[] { 100, 1000, 10000, 100000, 1000000, 10000000, 100000000 };
...
Serialize(ms, larry);
Console.WriteLine("MsgPack Size: {0}", ms.Length);
SerializeToStream(ms, larry);
Console.WriteLine("ProtoBuf Size: {0}", ms.Length);
...
foreach (var count in counts)
{
…
Console.WriteLine("MsgPack Serialize: {0} ms", DoTest(count, () => { Serialize(ms,
larry); }));
Console.WriteLine("ProtoBuf Serialize: {0} ms", DoTest(count, () =>
{ SerializeToStream(ms, larry); }));
Console.WriteLine("MsgPack DeSerialize: {0} ms", DoTest(count, () =>
{ Deserialize<Person>(ms); }));
Console.WriteLine("ProtoBuf DeSerialize: {0} ms", DoTest(count, () =>
{ DeSerializeFromStream<Person>(ms); }));
}
23

24. MESSAGEPACK VS PROTOBUF
24

25. MESSAGEPACK VS PROTOBUF
Serialize
25
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
100 1000 10000 100000 1000000 10000000 100000000
MsgPack
Protobuf

26. MESSAGEPACK VS PROTOBUF
Deserialize
26
0
10000
20000
30000
40000
50000
60000
70000
80000
100 1000 10000 100000 1000000 10000000 100000000
MsgPack
Protobuf

27. REFERENCE
27

28. REFERENCE
 MessagePack: It's like JSON. but fast and small.
 http://msgpack.org/
28

29. Q&A
29

30. QUESTION & ANSWER
30