Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Maxim Zaks
Beyond JSON -
An Introduction to FlatBuffers
Agenda
• About me
• The Why and How of Data Serialisation
• Deep dive into FlatBuffers
RESI News -
Nachrichten, die Spaß machen
VS.
Why do we need data
serialisation?
• Persisting State
• Machine to Machine Communication
• Representation of Configuration
How can we persist data
on mobile?
• Custom binary representation
• Language provided binary serialisation
• Text based re...
Important criteria for persisting
data
• Size on disk
• Speed of read & write / partial read /
memory consumption
• Human ...
VS.
JSON
• Size on disk 👍👎
• Speed of read & write / partial read /
memory consumption 👎
• Human readable and writable 👍 👍
• S...
FlatBuffers
• Size on disk 👍👎
• Speed of read & write / partial read /
memory consumption 👍 👍
• Human readable and writabl...
JSON FlatBuffers
size on disk 👍👎 👍👎
read & write
performance
👎 👍👍
human readable &
writable
👍👍 👍👎
typed API 👍👎 👍👍
multi ve...
But what about sending
data?
Important criteria for sending
data
• Size on disk wire
• Speed of read & write / partial read /
memory consumption
• Huma...
Enough theory
let’s do some practice
FlatBuffers IDL
./flatc -n person.fbs
FlatBuffers Schema Editor
FlatBuffers IDL
FlatBuffers IDL
FlatBuffers IDL
FlatBuffers IDL
What does it all mean
in practice?
Size on disk / wire
Speed of read, write
and memory allocation
Partial Read
• over 3M cities
• ≈200 MB
• real time search
• in memory
• from file 10x
slower
Human readable & writable
is a manner of tooling
• ./flatc -b person.fbs person.json
• ./flatc -t person.fbs -- person.bin -...
Support of OO
Language type system
Data is backwards &
forwards compatible
More information
• https://google.github.io/flatbuffers/
• https://github.com/mzaks/
FlatBuffersSchemaEditor
• https://gith...
Questions?
Thank you!
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
Upcoming SlideShare
Loading in …5
×

Beyond JSON - An Introduction to FlatBuffers

6,493 views

Published on

Presentations was given at Mobile Tech Con in Berlin

Published in: Software
  • Be the first to comment

  • Be the first to like this

Beyond JSON - An Introduction to FlatBuffers

  1. 1. Maxim Zaks Beyond JSON - An Introduction to FlatBuffers
  2. 2. Agenda • About me • The Why and How of Data Serialisation • Deep dive into FlatBuffers
  3. 3. RESI News - Nachrichten, die Spaß machen
  4. 4. VS.
  5. 5. Why do we need data serialisation? • Persisting State • Machine to Machine Communication • Representation of Configuration
  6. 6. How can we persist data on mobile? • Custom binary representation • Language provided binary serialisation • Text based representation • CSV, XML, JSON,YAML • Embedded SQL or NoSQL DB • Binary cross platform serialisation library • FlatBuffers, Protocol Buffers, Cap’n Proto, SBE, Apache Thrift
  7. 7. Important criteria for persisting data • Size on disk • Speed of read & write / partial read / memory consumption • Human readable and writable • Support of OO language type system • Data versioning / evolution / migration
  8. 8. VS.
  9. 9. JSON • Size on disk 👍👎 • Speed of read & write / partial read / memory consumption 👎 • Human readable and writable 👍 👍 • Support of OO language type system 👍👎 • Data versioning / evolution / migration 👍👎
  10. 10. FlatBuffers • Size on disk 👍👎 • Speed of read & write / partial read / memory consumption 👍 👍 • Human readable and writable 👍👎 • Support of OO language type system 👍 👍 • Data versioning / evolution / migration 👍 👍
  11. 11. JSON FlatBuffers size on disk 👍👎 👍👎 read & write performance 👎 👍👍 human readable & writable 👍👍 👍👎 typed API 👍👎 👍👍 multi version support 👍👎 👍👍
  12. 12. But what about sending data?
  13. 13. Important criteria for sending data • Size on disk wire • Speed of read & write / partial read / memory consumption • Human readable and writable • Support of OO language type system • Data versioning / evolution / migration
  14. 14. Enough theory let’s do some practice
  15. 15. FlatBuffers IDL
  16. 16. ./flatc -n person.fbs
  17. 17. FlatBuffers Schema Editor
  18. 18. FlatBuffers IDL
  19. 19. FlatBuffers IDL
  20. 20. FlatBuffers IDL
  21. 21. FlatBuffers IDL
  22. 22. What does it all mean in practice?
  23. 23. Size on disk / wire
  24. 24. Speed of read, write and memory allocation
  25. 25. Partial Read
  26. 26. • over 3M cities • ≈200 MB • real time search • in memory • from file 10x slower
  27. 27. Human readable & writable is a manner of tooling • ./flatc -b person.fbs person.json • ./flatc -t person.fbs -- person.bin --raw- binary --strict-json
  28. 28. Support of OO Language type system
  29. 29. Data is backwards & forwards compatible
  30. 30. More information • https://google.github.io/flatbuffers/ • https://github.com/mzaks/ FlatBuffersSchemaEditor • https://github.com/mzaks/FlatBuffersSwift • https://twitter.com/iceX33
  31. 31. Questions?
  32. 32. Thank you!

×