The current solutions for mapping data to java objects come in many shapes and sizes, with JSON leading as the most familiar and “coolest” solution. Yes, JSON is very readable, convenient, and offers great support libraries like GSON and Jackson, but these options don’t always get the job done. Accessing and parsing serialized data is a common source of runtime inefficiency within applications, and this delay can crush your app’s overall user experience. How can you bypass this inefficiency? Google’s FlatBuffer library is the answer. FlatBuffers are similar to Google’s Protocol Buffers, but with one key differentiator: the ability to access serialized data without parsing or unpacking it first. Imagine a serialization process with no temporary objects, no additional allocation, and no copying. Join us for a deep dive into Google’s FlatBuffers library to learn more about the advantages of using FlatBuffers and what makes it different from other commonly used libraries.

1. Don’t Get Lost in Translation
for Serializing Data Structures
Christopher Brown
Principal Mobile Developer – Team Lead, priceline.com
DroidCon NYC
August 28th, 2015

3. {
“trips”:
[
“offerDetails”:
{...},
“customer”:
{...},
“requestId”:
“...”,
...
]
}
10
trips
Light Booker

4. {
“trips”:
[
“offerDetails”:
{...},
“customer”:
{...},
“requestId”:
“...”,
...
...
...
...
...
...
...
]
}
100
trips
More Frequent Booker

5. {
“trips”:
[
“offerDetails”:
{...},
“customer”:
{...},
“requestId”:
“...”,
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
]
}
500
trips
Heavy Booker

6. JSONObject

7. new
JSONObject(
);
{
“trips”:
[
“offerDetails”:
{...},
“customer”:
{...},
“requestId”:
“...”,
...
...
...
...
...
...
...
...
...
...
...
...
...
...
]
}

8. new
JSONObject(
);
{
“trips”:
[
“offerDetails”:
{...},
“customer”:
{...},
“requestId”:
“...”,
...
...
...
...
...
...
...
...
...
...
...
...
...
...
]
}

9. new
JSONObject(
);
{
“trips”:
[
“offerDetails”:
{...},
“customer”:
{...},
“requestId”:
“...”,
...
...
...
...
...
...
...
...
...
...
...
...
...
...
]
}

10. How can we improve the
performance?
Let’s experiment with different
serialization libraries...

11. Gson

12. Gson
Provides simple toJson() and fromJson() methods

13. Gson
Provides simple toJson() and fromJson() methods
Provides streaming access

14. Gson
Provides simple toJson() and fromJson() methods
Provides streaming access
Operates as sequence of tokens traversed in depth-first order

15. Gson
Provides simple toJson() and fromJson() methods
Provides streaming access
Operates as sequence of tokens traversed in depth-first order
Minimal memory overhead

16. Implementation example
public
List<Trip>
getListOfTrips(
InputStream
inputStream
)
throws
IOException
{
Gson
gson
=
new
Gson();
JsonReader
reader
=
new
JsonReader(
new
InputStreamReader(
inputStream
)
);
List<Trip>
trips
=
new
ArrayList<>();
reader.beginArray();
while
(
reader.hasNext()
)
{
Trip
trip
=
gson.fromJson(
reader,
Trip.class
);
trips.add(
trip
);
}
reader.endArray();
reader.close();
return
trips;
}

17. Implementation example
public
List<Trip>
getListOfTrips(
InputStream
inputStream
)
throws
IOException
{
Gson
gson
=
new
Gson();
JsonReader
reader
=
new
JsonReader(
new
InputStreamReader(
inputStream
)
);
List<Trip>
trips
=
new
ArrayList<>();
reader.beginArray();
while
(
reader.hasNext()
)
{
Trip
trip
=
gson.fromJson(
reader,
Trip.class
);
trips.add(
trip
);
}
reader.endArray();
reader.close();
return
trips;
}

18. Implementation example
public
List<Trip>
getListOfTrips(
InputStream
inputStream
)
throws
IOException
{
Gson
gson
=
new
Gson();
JsonReader
reader
=
new
JsonReader(
new
InputStreamReader(
inputStream
)
);
List<Trip>
trips
=
new
ArrayList<>();
reader.beginArray();
while
(
reader.hasNext()
)
{
Trip
trip
=
gson.fromJson(
reader,
Trip.class
);
trips.add(
trip
);
}
reader.endArray();
reader.close();
return
trips;
}

19. Implementation example
public
List<Trip>
getListOfTrips(
InputStream
inputStream
)
throws
IOException
{
Gson
gson
=
new
Gson();
JsonReader
reader
=
new
JsonReader(
new
InputStreamReader(
inputStream
)
);
List<Trip>
trips
=
new
ArrayList<>();
reader.beginArray();
while
(
reader.hasNext()
)
{
Trip
trip
=
gson.fromJson(
reader,
Trip.class
);
trips.add(
trip
);
}
reader.endArray();
reader.close();
return
trips;
}

20. Gson performance
87.3 ms 56 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
10 Trips
100 Trips
500 Trips

21. Gson performance
87.3 ms 56 KB
142.9 ms 399 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
10 Trips
100 Trips
500 Trips

22. Gson performance
10 Trips
100 Trips
500 Trips
87.3 ms 56 KB
142.9 ms 399 KB
459.2 ms 599 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1

23. Jackson

24. Jackson
Offers 3 alternative methods for processing JSON (Streaming API, Tree Model,
Data Binding)
Default Jackson only handles strictly valid JSON but can be configured
Provides simple readValue() and readTree() methods
ObjectMapper caching

25. Jackson implementation example
public
Trips
getTrips(
String
json
)
{
ObjectMapper
objectMapper
=
new
ObjectMapper();
return
objectMapper.readValue(
json,
Trips.class
);
}

26. Jackson performance
351 ms 146 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
10 Trips
100 Trips
500 Trips

27. Jackson performance
351 ms 146 KB
455 ms 386 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
10 Trips
100 Trips
500 Trips

28. Jackson performance
10 Trips
100 Trips
500 Trips
351 ms 146 KB
636 ms 837 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
455 ms 386 KB

29. Protocol Buffers

30. Protocol Buffers
Language and platform neutral mechanism for serializing structured data

31. Protocol Buffers
Language and platform neutral mechanism for serializing structured data
You specify how you want the information you’re serializing to be structured by
defining protocol buffer message types in .proto files

32. Protocol Buffers
Language and platform neutral mechanism for serializing structured data
You specify how you want the information you’re serializing to be structured by
defining protocol buffer message types in .proto files
Each protocol buffer message is a logical record of information containing name-
value pairs

33. Protocol Buffers
Language and platform neutral mechanism for serializing structured data
You specify how you want the information you’re serializing to be structured by
defining protocol buffer message types in .proto files
Each protocol buffer message is a logical record of information containing name-
value pairs
.proto files are used to generate special source code which allows you to
populate, serialize, and parse protocol buffer objects

34. Protocol Buffers
Language and platform neutral mechanism for serializing structured data
You specify how you want the information you’re serializing to be structured by
defining protocol buffer message types in .proto files
Each protocol buffer message is a logical record of information containing name-
value pairs
.proto files are used to generate special source code which allows you to
populate, serialize, and parse protocol buffer objects
Protocol buffer messages are serialized into a tiny binary format. Serializing and
parsing protocol buffer messages is much faster than JSON.

35. Protocol Buffers .proto file
package
TripsList;
option
java_package
=
"com.priceline.droidcon.model.proto";
option
java_outer_classname
=
"TripsProtos";
message
Trips
{
...
}
message
Trip
{
...
}
message
OfferDetails
{
...
}
message
PrimaryOffer
{
...
}
message
Customer
{
...
}

36. Protocol Buffers .proto file
package
TripsList;
option
java_package
=
"com.priceline.droidcon.model.proto";
option
java_outer_classname
=
"TripsProtos";
message
Trips
{
...
}
message
Trip
{
...
}
message
OfferDetails
{
...
}
message
PrimaryOffer
{
...
}
message
Customer
{
...
}

37. Protocol Buffers .proto file
package
TripsList;
option
java_package
=
"com.priceline.droidcon.model.proto";
option
java_outer_classname
=
"TripsProtos";
message
Trips
{
...
}
message
Trip
{
...
}
message
OfferDetails
{
...
}
message
PrimaryOffer
{
...
}
message
Customer
{
...
}

38. Protocol Buffers .proto file
package
TripsList;
option
java_package
=
"com.priceline.droidcon.model.proto";
option
java_outer_classname
=
"TripsProtos";
message
Trips
{
...
}
message
Trip
{
...
}
message
OfferDetails
{
...
}
message
PrimaryOffer
{
...
}
message
Customer
{
...
}

39. Protocol Buffers implementation example
public
List<Trip>
getListOfTrips(
byte[]
data
)
{
Trips
trips
=
Trips.parseFrom(
data
);
return
trips.getTripList();
}

40. Protocol Buffers implementation example
public
List<Trip>
getListOfTrips(
byte[]
data
)
{
Trips
trips
=
Trips.parseFrom(
data
);
return
trips.getTripList();
}

41. Protocol Buffers implementation example
public
List<Trip>
getListOfTrips(
byte[]
data
)
{
Trips
trips
=
Trips.parseFrom(
data
);
return
trips.getTripList();
}

42. Protocol Buffers performance
3.816 ms 264 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
10 Trips
100 Trips
500 Trips

43. Protocol Buffers performance
3.816 ms 264 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
24.231 ms 352 KB
10 Trips
100 Trips
500 Trips

44. Protocol Buffers performance
10 Trips
100 Trips
500 Trips
3.816 ms 264 KB
24.231 ms 352 KB
102.355 ms 1,728 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1

45. ...but, there’s something
faster!

46. FlatBuffers

47. FlatBuffers
Efficient open-source, cross platform serialization library

48. Efficient open-source, cross platform serialization library
Similar to Protocol Buffers but allows you to access information without parsing
the data which results in much better memory performance
FlatBuffers

49. FlatBuffers
Efficient open-source, cross platform serialization library
Similar to Protocol Buffers but allows you to access information without parsing
the data which results in much better memory performance
A schema let’s you define how you want to structure your data

50. FlatBuffers
Efficient open-source, cross platform serialization library
Similar to Protocol Buffers but allows you to access information without parsing
the data which results in much better memory performance
A schema let’s you define how you want to structure your data
FlatBuffer is a binary buffer containing data organized by offsets

51. FlatBuffers
Efficient open-source, cross platform serialization library
Similar to Protocol Buffers but allows you to access information without parsing
the data which results in much better memory performance
A schema let’s you define how you want to structure your data
FlatBuffer is a binary buffer containing data organized by offsets
Faster than JSON and smaller memory footprint than Protocol Buffers

52. FlatBuffers schema
namespace
TripsList;
table
Trips
{
trips
:
[Trip];
}
table
Trip
{
...
}
table
OfferDetails
{
...
}
...
root_type
Trips;

53. FlatBuffers schema
namespace
TripsList;
table
Trips
{
trips
:
[Trip];
}
table
Trip
{
...
}
table
OfferDetails
{
...
}
...
root_type
Trips;

54. FlatBuffers schema
namespace
TripsList;
table
Trips
{
trips
:
[Trip];
}
table
Trip
{
...
}
table
OfferDetails
{
...
}
...
root_type
Trips;

55. FlatBuffers schema
namespace
TripsList;
table
Trips
{
trips
:
[Trip];
}
table
Trip
{
...
}
table
OfferDetails
{
...
}
...
root_type
Trips;

56. FlatBuffers implementation example
public
List<Trip>
getListOfTrips(
byte[]
data
)
{
Trips
trips
=
Trips.getRootAsTrips(
ByteBuffer.wrap(
data
)
);
List<Trip>
list
=
new
ArrayList<>();
for
(
int
i
=
0;
i
<
trips.tripsLength();
i++
)
{
list.add(
trips.trips(
i
)
);
}
return
list;
}

57. FlatBuffers implementation example
public
List<Trip>
getListOfTrips(
byte[]
data
)
{
Trips
trips
=
Trips.getRootAsTrips(
ByteBuffer.wrap(
data
)
);
List<Trip>
list
=
new
ArrayList<>();
for
(
int
i
=
0;
i
<
trips.tripsLength();
i++
)
{
list.add(
trips.trips(
i
)
);
}
return
list;
}

58. FlatBuffers implementation example
public
List<Trip>
getListOfTrips(
byte[]
data
)
{
Trips
trips
=
Trips.getRootAsTrips(
ByteBuffer.wrap(
data
)
);
List<Trip>
list
=
new
ArrayList<>();
for
(
int
i
=
0;
i
<
trips.tripsLength();
i++
)
{
list.add(
trips.trips(
i
)
);
}
return
list;
}

59. FlatBuffers performance
10 Trips 1.304 ms 3.8 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
100 Trips
500 Trips

60. FlatBuffers performance
10 Trips 1.304 ms 3.8 KB
6.153 ms 6.944 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
100 Trips
500 Trips

61. FlatBuffers performance
10 Trips
100 Trips
500 Trips
1.304 ms 3.8 KB
6.153 ms 6.944 KB
13.704 ms 13 KB
Incl. CPU Time
Memory Allocation
Tests were performed on a Motorola Nexus 6 running Android 5.1.1
Protocol Buffers
On average 350% faster than
Protocol Buffers

62. Overall Time Results
Gson
Jackson
Protocol
Buffers
Flat Buffers
459 ms
14 ms
Note: Time results shown for 500 trips
102 ms
636 ms

63. Our Selection
FlatBuffers was faster and more efficient, but…
Image credits: The Noun Project – Dalpat Prajapati

64. Our Selection
FlatBuffers was faster and more efficient, but…
Our final decision was Protocol Buffers

65. Our Selection
FlatBuffers was faster and more efficient, but…
Our final decision was Protocol Buffers
...it’s not the fastest, but it was the most optimal for us

66. Our Selection
FlatBuffers was faster and more efficient, but…
Our final decision was Protocol Buffers
...it’s not the fastest, but it was the most optimal for us
FlatBuffers is still a great way to serialize and deserialize data

67. Wrap Up
Don’t get lost in translation for serializing your data structures
Image credits: The Noun Project – Hello Many

68. Wrap Up
Don’t get lost in translation for serializing your data structures
Different options available (Gson → Jackson → Protocol Buffers →
FlatBuffers)

69. Wrap Up
Don’t get lost in translation for serializing your data structures
Different options available (Gson → Jackson → Protocol Buffers →
FlatBuffers)
FlatBuffers are the best balance of speed and memory efficiency
(FlatBuffers won the battle, it did not win the war)
Image credits: The Noun Project - Rediffusion

70. Questions?
christopher.brown@priceline.com