Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beyond JSON @ Mobile.Warsaw

603 views

Published on

Introduction to FlatBuffers

Published in: Education
  • Be the first to comment

  • Be the first to like this

Beyond JSON @ Mobile.Warsaw

  1. 1. Beyond JSON Maxim Zaks @iceX33
  2. 2. Agenda • War Stories • And very technical details
  3. 3. 2007
  4. 4. 2007
  5. 5. JSON vs. XML • much more human readable • no schema • no streaming parser • too early for financial sector
  6. 6. 2015
  7. 7. Simulation Games
  8. 8. Deterministic simulaiton running on mobile device And validated on the BackEnd
  9. 9. Share code between BackEnd & FrontEnd
  10. 10. Stateless Worker
  11. 11. Parsing a JSON config (400kb) and game state (30kb) takes 60 ms
  12. 12. Did we gave up?
  13. 13. No! We Google
  14. 14. FlatBuffers is an efficient cross platform serialization library for C++, Java, C#, Go, Python and JavaScript (C, PHP & Ruby in progress).
  15. 15. How does it work?
  16. 16. Define a schema table Person { name: string; age : int; } root_type Person;
  17. 17. Call code generator ./flatc -n person01.fbs
  18. 18. Use generated code for reading public sealed class Person : Table { public static Person GetRootAsPerson(ByteBuffer _bb) {...} public static Person GetRootAsPerson(ByteBuffer _bb, Person obj) {...} public string Name { get { ... } } public int Age { get { ... } } }; byte[] data = ...; Person person = Person.GetRootAsPerson(new ByteBuffer(data));
  19. 19. Suffer through generated code for writing public sealed class Person : Table { public static Offset<Person> CreatePerson(FlatBufferBuilder builder, StringOffset name = default(StringOffset), int age = 0) { ... } public static void FinishPersonBuffer(FlatBufferBuilder builder, Offset<Person> offset) { ... } }; FlatBufferBuilder fbb = new FlatBufferBuilder(1); var nameOffset = fbb.CreateString("maxim"); var personOffset = Person.CreatePerson(fbb, nameOffset, 34); Person.FinishPersonBuffer(fbb, personOffset); byte[] data = fbb.SizedByteArray();
  20. 20. What is underneath? Why is it so fast to read And cumbersome to write
  21. 21. Call FlatBuffers compiler ./flatc -b person01.fbs person01.json
  22. 22. And if you want binary to JSON ./flatc -t person01.fbs -- person01.bin --raw-binary --strict-json = { "name": "maxim", "age": 34 }
  23. 23. FlatBuffers has evolution strategy
  24. 24. table Person { name: string; age : int; } root_type Person; -> table Person { name: string; age : int (deprecated); birthday : double; } root_type Person;
  25. 25. table Person { name: string; age : int; } root_type Person; -> table Person { name: string (id: 0); birthday : double (id: 2); age : int (deprecated, id: 1); } root_type Person;
  26. 26. It's backwards and forwards compatible new code can read old data & old code can read new data
  27. 27. I wanted to make writing more usable specially for unit testing
  28. 28. Eager serialization public sealed class Person : Table { public string name; public int age; public static Person FromByteArray(byte[] data) {...} public byte[] ToByteArray(){...} } // var person = new Person(); byte[] data = ...; var person = Person.FromByteArray(data); person.name = "Alex"; byte[] data2 = person.ToByteArray();
  29. 29. let person = Person(name: "Maxim", age: 34) let fbData = person.toByteArray NSData( bytes: UnsafePointer<UInt8>(fbData), length: fbData.count) .writeToFile("person.bin", atomically: true)
  30. 30. Facebook is using FlatBuffers !
  31. 31. FlatBuffersSwift benchmark • 65.015 mseconds for parsing JSON (3.6MB) • 36.478 mseconds for eager FlatBuffers (2.1MB) • 00.002 mseconds for lazy FlatBuffers (2.1MB)
  32. 32. Some rain on FlatBuffers parade • Zip JSON: 3.6MB -> 739KB • Zip FB: 2.1MB -> 1.1MB
  33. 33. Outlook for FlatBuffers • real Graph not a Tree • could support streaming • is easy to encrypt • could be very memory efficient with custom compression
  34. 34. Questions?
  35. 35. Thank you! @iceX33 https://google.github.io/flatbuffers https://github.com/mzaks/FlatBuffersSchemaEditor https://github.com/mzaks/FlatBuffersSwift

×