Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CBOR - The Better JSON

5,431 views

Published on

JSON, by now, became a regular part of most applications and services. Do we, how ever, really want to transfer human readable information or are we looking for a binary protocol to be as debuggable as JSON? CBOR the Concise Binary Object Representation offers the best of JSON + an extremely efficient, binary representation.

http://www.cbor.io

Published in: Technology

CBOR - The Better JSON

  1. 1. CBOR The Better JSON
  2. 2. www.hazelcast.com@noctarius2k Who’s that dude? • Chris Engelbert • Manager of Developer Relations @Hazelcast • Java-Passionate (10+ years) • Performance • Garbage Collection • JVM / Benchmark Fairytales
  3. 3. www.hazelcast.com@noctarius2k Brief look at the history
  4. 4. www.hazelcast.com@noctarius2k In the beginning there was darkness…
  5. 5. www.hazelcast.com@noctarius2k …and binary data
  6. 6. www.hazelcast.com@noctarius2k …and binary data
  7. 7. www.hazelcast.com@noctarius2k Debugging binary data is hard
  8. 8. www.hazelcast.com@noctarius2k Binary data is just a big blob
  9. 9. www.hazelcast.com@noctarius2k Possible with custom tooling
  10. 10. www.hazelcast.com@noctarius2k Possible with custom tooling • unique to this specific use case • involves model classes to be known • utilizes workarounds • generally feels weird :-) Often:
  11. 11. www.hazelcast.com@noctarius2k “I need to understand it!”
  12. 12. www.hazelcast.com@noctarius2k Human Readability, Expressiveness
  13. 13. www.hazelcast.com@noctarius2k Human Readability, Expressiveness <fun>
 <with>
 <xml dont:you="think" />
 </with>
 </fun>
  14. 14. www.hazelcast.com@noctarius2k Human Readability, Expressiveness
  15. 15. www.hazelcast.com@noctarius2k Human Readability, Expressiveness This is not a flame graph!
  16. 16. www.hazelcast.com@noctarius2k Human Readability, Expressiveness This is not a flame graph! <plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-failsafe-plugin</artifactId>
 <version>${maven.failsafe.plugin.version}</version>
 <configuration combine.self="override">
 <redirectTestOutputToFile>true</redirectTestOutputToFile>
 <argLine>
 ${jacoco.agent.argLine}
 -Xms129m -Xmx1G -XX:MaxPermSize=129M
 -Dxyz.feature1.enabled=false
 -Dxyz.feature2.enabled=false
 -Dxyz.feature3.type=none
 -Dxyz.feature4=true
 </argLine>
 <includes>
 <include>**/TestCast*.java</include>
 <include>**/IntegrationTestCase*.java</include>
 </includes>
 <groups>
 com.xyz.test.annotation.QuickTest,
 com.xyz.test.annotation.SlowTest,
 com.xyz.test.annotation.NightlyTest
 </groups>
 
 </configuration>
 <executions>
 <execution>
 <goals>
 <goal>integration-test</goal>
 </goals>
 </execution>
 </executions> </plugin>
  17. 17. www.hazelcast.com@noctarius2k Too verbose!
  18. 18. www.hazelcast.com@noctarius2k Too understandable?!
  19. 19. www.hazelcast.com@noctarius2k Just gross?
  20. 20. www.hazelcast.com@noctarius2k Grossbusters!
  21. 21. www.hazelcast.com@noctarius2k There is no Dana, only Zuul!JSON
  22. 22. www.hazelcast.com@noctarius2k Human Readability, Concise
  23. 23. www.hazelcast.com@noctarius2k Human Readability, Concise
  24. 24. www.hazelcast.com@noctarius2k Human Readability, Concise base64 encoded image
  25. 25. www.hazelcast.com@noctarius2k Human Readability, Concise Does not play “well” with binary data :-( base64 encoded image
  26. 26. www.hazelcast.com@noctarius2k But let’s be honest…
  27. 27. www.hazelcast.com@noctarius2k …do we need to store human readable?
  28. 28. www.hazelcast.com@noctarius2k Binary but understandable!
  29. 29. www.hazelcast.com@noctarius2k So what’s the next evolution step?
  30. 30. www.hazelcast.com@noctarius2k C(oncise) B(inary) O(bject) R(epresentation)
  31. 31. www.hazelcast.com@noctarius2k Features • Stores JSON-alike content • Extremely concise • Binary • Streamable • Type safe • Schemaless • Highly Extensible • Well defined specification • Standard (by the IETF, RFC7049)
  32. 32. www.hazelcast.com@noctarius2k Real World (Size) Example www.pathway-game.com • Stores Geometry Information • Mainly Float Data • Often floatToIntBits for Speed • Lots of Sequences • Only a few Dictionaries Really simple data structure
  33. 33. www.hazelcast.com@noctarius2k Real World (Size) Example 2250000 4500000 6750000 9000000 XML JSON BSON CBOR MSGPACK MSGPACK
 CBOR BSON JSON XML 1.408.737 bytes 1.415.215 bytes 5.575.783 bytes 5.755.188 bytes 8.592.541 bytes 1.0x 1.0x 3.9x 4.0x 6.0x
  34. 34. www.hazelcast.com@noctarius2k Debugging capability built-in Dictionary [ ByteString{ ANI0 }=Sequence [ Sequence [ UInt{ 7264 }, UInt{ 4 }, UInt{ 2724 }, UInt{ 37 }, Sequence [ NInt{ -1053818880 }, UInt{ 1077214225 }, UInt{ 0 }, UInt{ 0 }, UInt{ 1068825617 }, ] ] ] ] Since items carry their type information, writing a quick debugger or using the provided to-string capabilities of libs
  35. 35. www.hazelcast.com@noctarius2k Implementations • JavaScript • browser • node.js • Julia • C# • Java • Lua • PHP • Python • Go • C • C++ • Rust • Perl • Ruby • D • Swift • Erlang • Elixir
  36. 36. www.hazelcast.com@noctarius2k Implementations • JavaScript • browser • node.js • Julia • C# • Java • Lua • PHP • Python • Go • C • C++ • Rust • Perl • Ruby • D • Swift • Erlang • Elixir CBOR-java:
 https://github.com/peteroupc/CBOR-Java Jackson Dataformat: https://github.com/FasterXML/jackson-dataformat-cbor cbor-java: https://github.com/c-rack/cbor-java JACOB: https://github.com/jawi/jacob borabora: https://github.com/noctarius/borabora
  37. 37. www.hazelcast.com@noctarius2k Implementations • JavaScript • browser • node.js • Julia • C# • Java • Lua • PHP • Python • Go • C • C++ • Rust • Perl • Ruby • D • Swift • Erlang • Elixir CBOR-java:
 https://github.com/peteroupc/CBOR-Java Jackson Dataformat: https://github.com/FasterXML/jackson-dataformat-cbor cbor-java: https://github.com/c-rack/cbor-java JACOB: https://github.com/jawi/jacob borabora: https://github.com/noctarius/borabora
  38. 38. www.hazelcast.com@noctarius2k borabora •Skip-scan parser •Simple, fluent API •Supports object queries •Reading + writing •Supports mutating (hacky prototype) •Extended datatype support
  39. 39. www.hazelcast.com@noctarius2k borabora •Skip-scan parser •Simple, fluent API •Supports object queries •Reading + writing •Supports mutating (hacky prototype) •Extended datatype support Not just a blur of color
  40. 40. www.hazelcast.com@noctarius2k borabora •Skip-scan parser •Simple, fluent API •Supports object queries •Reading + writing •Supports mutating (hacky prototype) •Extended datatype support but the island ;-)
  41. 41. www.hazelcast.com@noctarius2k Time for some demos, isn’t it?
  42. 42. www.hazelcast.com@noctarius2k Comparisons
  43. 43. www.hazelcast.com@noctarius2k Comparisons “Alternative Facts” in the comparisons, are all fake news!
  44. 44. www.hazelcast.com@noctarius2k MessagePack • Extremely concise • Available in almost every language • UTF-8 and binary strings • No map keys datatype restriction • Schemaless • Binary • 32bit size limitation for certain data types • Specification is concise too (~450 lines) • No official standard
  45. 45. www.hazelcast.com@noctarius2k MessagePack • Extremely concise • Available in almost every language • UTF-8 and binary strings • No map keys datatype restriction • Schemaless • Binary • 32bit size limitation for certain data types • Specification is concise too (~450 lines) • No official standard 1:0 2:0 Draw Draw Draw Draw 2:1 2:2 2:3
  46. 46. www.hazelcast.com@noctarius2k JSON • Human readable • Official standard (ECMA-404) • Subset of JavaScript (strong support) • Schemaless • Kind of type safe • Human readable • Text-based • Map keys can be strings only • Large bytesize
  47. 47. www.hazelcast.com@noctarius2k • Human readable • Official standard (ECMA-404) • Subset of JavaScript (strong support) • Schemaless • Kind of type safe • Human readable • Text-based • Map keys can be strings only • Large bytesize 1:0 Draw 2:0 Draw Draw 2:1 2:2 2:3 2:4 JSON
  48. 48. www.hazelcast.com@noctarius2k • Binary • Available in a lot of languages • Derived from JSON, interoperability 1+ • Schemaless • Type safe • Supports in-place updates • No official standard • Extremely concise specification • Map keys can be strings only • Large bytesize BSON
  49. 49. www.hazelcast.com@noctarius2k • Binary • Available in a lot of languages • Derived from JSON, interoperability 1+ • Schemaless • Type safe • Supports in-place updates • No official standard • Extremely concise specification • Map keys can be strings only • Large bytesize Draw 1:0 2:0 Draw Draw 3:0 3:1 3:2 3:3 3:4 BSON
  50. 50. www.hazelcast.com@noctarius2k • Human readable • Official standard (W3C REC-xml) • Strong commercial support (e.g. IBM) • Supported everywhere • Type safe or schemaless • Human readable • Text-based • Very verbose • Large bytesize XML
  51. 51. www.hazelcast.com@noctarius2k • Human readable • Official standard (W3C REC-xml) • Strong commercial support (e.g. IBM) • Supported everywhere • Type safe or schemaless • Human readable • Text-based • Very verbose • Large bytesize 1:0 Draw 2:0 3:0 3:1 3:2 3:3 3:4 3:5 XML
  52. 52. www.hazelcast.com@noctarius2k More alternatives Text-based: • bencode • YAML • … Binary: • Amazon ION • Smile • Apache Thrift • Apache Avro • Protobuf • … ASCII based encoding, unaffected by endianess, type safe commonly for configuration, superset of JSON, type safe skip-scan parsable, type safe, commercial support (Amazon) based on JSON, property-name back-references DSL, schema-bound, developed by Facebook, RPC services DSL, schema-bound, primarily used by Hadoop DSL, schema-bound, developed by Google, RPC services
  53. 53. www.hazelcast.com@noctarius2k More information: • http://cbor.io/ • http://bsonspec.org/ • http://www.json.org/ • https://www.w3.org/TR/REC-xml/ • http://msgpack.org/ • https://github.com/noctarius/borabora • https://en.wikipedia.org/wiki/Bencode • https://en.wikipedia.org/wiki/YAML • https://amznlabs.github.io/ion-docs/ • https://en.wikipedia.org/wiki/Apache_Thrift • https://en.wikipedia.org/wiki/Apache_Avro • https://en.wikipedia.org/wiki/Protocol_Buffers • https://en.wikipedia.org/wiki/Smile_(data_interchange_format) Thank You!
  54. 54. www.hazelcast.com@noctarius2k Thank You! Any Questions? @noctarius2k http://www.sourceprojects.org http://github.com/noctarius @hazelcast http://www.hazelcast.com http://www.hazelcast.org http://github.com/hazelcast

×