Tour de Jacksonfrom Basics to Forgotten Features
Tour De JacksonJackson is quickly becoming the de facto standard toolfor processing JSON on Java platform, due to itsexten...
About AuthorTatu Saloranta aka @cowtowncoder● Programming since 1983 (VIC-20 FTW!)● for Public Benefit since ~1988 (AmigaS...
About Author, Open SourceNotable Open Source events:● Java UUID Generator (JUG): 2002● Woodstox XML processor (Stax): 2004...
Jackson BioJackson JSON (*) processing library● Born in 2007 -- initially (0.5) skimpystreaming parser, generator● Data-bi...
Jackson Basic FeaturesSince beginning of time (Jackson 1.0)● Read JSON○ as token stream (streaming) -- JsonParser○ into PO...
Jackson Basic FeaturesAdditions over time● More configurable data-binding (annotate)● Convert values, update (merge) POJOs...
Jackson Advanced FeaturesObject Serialization requires handling:1. Polymorphism (@JsonTypeInfo, "defaulttyping"), type met...
Extensibility via ModulesDatatypes● 3rd-party libs: Joda, Guava, org.json● frameworks: Hibernate (lazy-loading)● JVM langu...
Extensibility via ModulesJAX-RS (Jersey, RESTeasy, CXF), DropWizard● Providers (MessageBodyReader/-Writer)○ JSON○ Smile (b...
Forgotten Features: overview1. Mr Bean: "abstract type materialization"2. Value conversions3. Smile! (binary JSON)4. Mix-i...
Forgotten Feature: Mr BeanEasy to write monkey code,but its a chore… so why dowe keep doing it this? ->How about we just d...
Forgotten Feature: Mr BeanAnd so we should!ObjectMapper mapper = new ObjectMapper();mapper.registerModule(new MrBeanModule...
Forgotten Feature: ValueConversionsConvert any structurally compatible types:List<String> keys = ...;String[] keyArray = m...
Forgotten Feature: Smile!● Virtues of binary formats (protobuf, Thrift,MsgPack, Avro, BSON) often overrated --esp. speed; ...
Forgotten Feature: Smile!● What if there was “binary JSON” format thatwas:○ 100% compatible (same model, types) with JSON(...
Forgotten Feature: Smile!● Simple usage: extends JsonFactory● ObjectMapper m = new ObjectMapper(new SmileFactory());● … an...
Forgotten Feature: mix-inannotations● Databinding highly configurable withAnnotations, but you can not always modifyvalue ...
Forgotten Feature: mix-inannotationspublic class Rectangle {private int w, h;public Rectangle(int w, int h) {this.w = w;th...
Forgotten Feature: mix-inannotationspublic class MyModule extends SimpleModule{public MyModule() {super("ModuleName", Vers...
Forgotten Feature:ObjectReader, ObjectWriterMost code uses ObjectMapper, which worksfine but:● Can not be reconfigured (co...
Forgotten Feature:ObjectReader, ObjectWriterTo serialize using JSON View, indented:mapper.writerWithView(View1.class).with...
Forgotten Feature:ObjectReader.readValues()● Large data sets often sequences of objects:{ "x":1,"y":3 }{ "x":5,"y":2 }...●...
Forgotten: @JsonValueProvide your own String representation:public class MyStuff {@JsonValue public String toString() { }}...
Forgotten: @JsonUnwrapped● Used to "flatten" POJOs● Can add prefix/suffixpublic class Box{@JsonUnwrapped(prefix="topLeft")...
Forgotten: Dyna-beans● Sometimes want to handle set of additional "extra"properties (key/value pairs)public class NamedBea...
Forgotten Feature: Dyna-beanspublic class NamedBeanWithStuff// continued...public Object get(String name) { // non-Bean ac...
Forgotten Feature: OtherObjectReader.updateValue()● Update existing (root) value● Shallow merge● Good for configuration ov...
Forgotten Feature: Other@JacksonInject: injecting values from context● Indicate logical name of value to injectwith annota...
Thats All Folks!Questions? Comments?Follow me (@cowtowncoder), or email(tsaloranta@gmail.com / tatu@fasterxml.com)ps. Fast...
Upcoming SlideShare
Loading in …5
×

Tour de Jackson: Forgotten Features of Jackson JSON processor

9,778 views

Published on

Presentation originally given at: https://www.airbnb.com/meetups/kbmjfabje-tech-talk-tatu-saloranta

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
9,778
On SlideShare
0
From Embeds
0
Number of Embeds
116
Actions
Shares
0
Downloads
45
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Tour de Jackson: Forgotten Features of Jackson JSON processor

  1. 1. Tour de Jacksonfrom Basics to Forgotten Features
  2. 2. Tour De JacksonJackson is quickly becoming the de facto standard toolfor processing JSON on Java platform, due to itsextensive feature set and high configurability.But learning all the available power can bechallenging. This talk tries to shine light on some oflesser-known features of Jackson.
  3. 3. About AuthorTatu Saloranta aka @cowtowncoder● Programming since 1983 (VIC-20 FTW!)● for Public Benefit since ~1988 (AmigaSoundTracker Packer)● Open source since 1998 (Java VT-100 emu)● for Money, too; most recently:○ Sun (2001-2005)○ Amazon (2005-2009)○ Ning (2009-2012)○ Salesforce (2012-)
  4. 4. About Author, Open SourceNotable Open Source events:● Java UUID Generator (JUG): 2002● Woodstox XML processor (Stax): 2004● StaxMate, helper for Stax: 2004● Jackson JSON processor: 2007● Aalto XML parser (non-blocking): 2008● @Ning: LZF-compress, tr13, jvm-compressor-benchmark: 2010● Other: ClassMate, CacheMate, java-merge-sort
  5. 5. Jackson BioJackson JSON (*) processing library● Born in 2007 -- initially (0.5) skimpystreaming parser, generator● Data-binding added in 2008● New versions every 3 - 6 months● 3 core components (annotations, streaming,databind), ~20 extension modules● of many, many Java JSON libs○ Most popular: 10,000/day via Maven Central○ Fastest (google or ask for links)● (*) ... and not just JSON any more...
  6. 6. Jackson Basic FeaturesSince beginning of time (Jackson 1.0)● Read JSON○ as token stream (streaming) -- JsonParser○ into POJOs (databind) -- ObjectMapper/-Reader○ as JSON Trees (tree model) -- JsonNode● Write JSON○ from POJOs -- ObjectMapper/-Writer○ from Trees○ or directly using token stream -- JsonGenerator● POJO/Tree access built on top of streamingmodel (layering)
  7. 7. Jackson Basic FeaturesAdditions over time● More configurable data-binding (annotate)● Convert values, update (merge) POJOs○ shortcut for "write-X-as-JSON, read-Y-from-JSON"== "x -> y"○ Base64 for binary● Modular extensibility (modules)○ support for 3rd party data types, formats● JSON Schema support (mostly, generation)● Convenient conversions between models:tree <-> POJO <-> Streaming
  8. 8. Jackson Advanced FeaturesObject Serialization requires handling:1. Polymorphism (@JsonTypeInfo, "defaulttyping"), type metadata handlinga. Type id: class name or custom type nameb. Inclusion: as-property, wrapper-object, wrapper-array, external-propertyc. Resolution mechanism (type name to/from class)2. Object Identity (@JsonIdentityInfo)a. Id generators (sequence, UUID, from-property)b. Property name (no other inclusion methods)
  9. 9. Extensibility via ModulesDatatypes● 3rd-party libs: Joda, Guava, org.json● frameworks: Hibernate (lazy-loading)● JVM language-specific types: ScalaData formats● Smile (binary JSON) -- more on this later● XML -- move over JAXB!● CSV● YAML, BSON (mongoDB, external), Avro● future: Protobuf, Thrift?
  10. 10. Extensibility via ModulesJAX-RS (Jersey, RESTeasy, CXF), DropWizard● Providers (MessageBodyReader/-Writer)○ JSON○ Smile (binary JSON)○ XML○ ... add others (CSV?) as requestedOther● Mr Bean (more later on)● Afterburner: Mach3 with bytecode gen● JAXB annotation support● JSON Schema generator (Jackson 2.2)
  11. 11. Forgotten Features: overview1. Mr Bean: "abstract type materialization"2. Value conversions3. Smile! (binary JSON)4. Mix-in Annotations5. ObjectReader / ObjectWriter6. ObjectReader.readValues()7. @JsonValue8. @JsonUnwrapped9. Dyna-beans (@JsonAnySetter/-Getter)10. Lots more (misc other) -- just pointers
  12. 12. Forgotten Feature: Mr BeanEasy to write monkey code,but its a chore… so why dowe keep doing it this? ->How about we just define:public interface Name {public String getFirst();public String getLast();}and use it like:Name n=m.readValue(json);public class NameImpl{private String first, last;public Name() { }public void setFirst(String f){first=f;}public void setLast(String l){last=l;}public String getFirst(){return first;}public String getLast(){return last;}}
  13. 13. Forgotten Feature: Mr BeanAnd so we should!ObjectMapper mapper = new ObjectMapper();mapper.registerModule(new MrBeanModule());Name n = mapper.readValue(in, Name.class);Brilliant! But how?1. Take interface/abstract class, discover properties2. Implement getters AND setters (ASM)3. Define new class4. Rest works based on auto-discovery!Works for generic and transitive types too.Can even be used outside of Jackson...
  14. 14. Forgotten Feature: ValueConversionsConvert any structurally compatible types:List<String> keys = ...;String[] keyArray = mapper.convertValue(keys, String[].class);Map<String,Integer> map = mapper.convertValue(pojo, new TypeReference<String,Integer>(){});or compatible scalar types:Date date = mapper.convertValue("2013-04-19T23:09Z", Date.class);or even binary data with Base64!byte[] data = mapper.convertValue("TWFuIGlzIGRpc3", byte[].class);
  15. 15. Forgotten Feature: Smile!● Virtues of binary formats (protobuf, Thrift,MsgPack, Avro, BSON) often overrated --esp. speed; compression for data size● Still: binary typically more compact, MAY befaster to process● But is it necessary to give up all benefits ofJSON – self-descriptiveness, open content(no mandatory schema), extensibility,interoperability – to get there?
  16. 16. Forgotten Feature: Smile!● What if there was “binary JSON” format thatwas:○ 100% compatible (same model, types) with JSON(bit like "Fast Infoset" for XML)○ More compact than JSON○ Strictly faster to process – fast writes, not just reads○ Auto-detectable, so with proper parser/generatorcan be used “just like JSON” from app perspective● Wouldnt this be useful?● http://wiki.fasterxml.com/SmileFormatSpecdefines such format: "Smile"
  17. 17. Forgotten Feature: Smile!● Simple usage: extends JsonFactory● ObjectMapper m = new ObjectMapper(new SmileFactory());● … and it “Just Works”● Use cases:○ “Big Data” – logging, Hadoop data○ Search indexing -- used by Elastic Search○ Internal (for external JSON may be better)● Efficient transcoding to/from JSON – best ofboth!● Public tests show efficiency (jvm-serializers)
  18. 18. Forgotten Feature: mix-inannotations● Databinding highly configurable withAnnotations, but you can not always modifyvalue classes (or dont want to)● How to externalize such configuration?● What if you could "mix in" annotations, fromoutside?● Due to the way Java handles Annotations,always associated with class/interface● Mix-in annotations: define "mix-in"class/interface, add annotations to target
  19. 19. Forgotten Feature: mix-inannotationspublic class Rectangle {private int w, h;public Rectangle(int w, int h) {this.w = w;this.h = h;}public int getW() { return w; }public int getH() { return h; }public int getSize() { return w * h; }}● Property names "w","h"?● Shouldnt serialize "size"● How to construct?● Mix-ins for the win!abstract class RectangleMixInextends Rectangle // optional{@JsonCreator // important!MixIn(@JsonProperty("width") int ww, @JsonProperty("height") int h) {}@JsonProperty("width")abstract int getW(); // rename@JsonProperty("height")abstract int getH(); // rename@JsonIgnoreabstract int getSize(); // remove}
  20. 20. Forgotten Feature: mix-inannotationspublic class MyModule extends SimpleModule{public MyModule() {super("ModuleName", Version.unknownVersion());}@Overridepublic void setupModule(SetupContext context){// Mix-ins so that annotations from Targetcontext.setMixInAnnotations(Rectangle.class, RectangleMixIn.class);}}
  21. 21. Forgotten Feature:ObjectReader, ObjectWriterMost code uses ObjectMapper, which worksfine but:● Can not be reconfigured (config-then-use)● Does not contain all functionalityInstead, be Jackson Pro, and use:1. ObjectReader for deserialization2. ObjectWriter for serializationso that you can:● define per-call JSON View, Features● use new functionality (readValues())
  22. 22. Forgotten Feature:ObjectReader, ObjectWriterTo serialize using JSON View, indented:mapper.writerWithView(View1.class).withDefaultPrettyPrinter().writeValue(pojo, outputStream);or read instance of certain type, view:MyType value = mapper.reader(MyType.class).withView(View2.class).readValue(source);
  23. 23. Forgotten Feature:ObjectReader.readValues()● Large data sets often sequences of objects:{ "x":1,"y":3 }{ "x":5,"y":2 }...● Can read using JsonParser + ObjectMapper● But there is a simpler way:ObjectReader r = mapper.reader(Point.class);MappingIterator<Point> it = r.readValues(in);while (it.hasNextValue()) {Point p = it.nextValue();// ... process}
  24. 24. Forgotten: @JsonValueProvide your own String representation:public class MyStuff {@JsonValue public String toString() { }}or anything Jackson knows how to serialize:@JsonValuepublic Map<String,Object> toJson() { }Perfect match for custom constructors:@JsonCreatorpublic MyStuff(Map<String,Object> map) {}
  25. 25. Forgotten: @JsonUnwrapped● Used to "flatten" POJOs● Can add prefix/suffixpublic class Box{@JsonUnwrapped(prefix="topLeft")Point tl;@JsonUnwrapped(prefix="bottomRight")Point br;}public class Point {int x, y;}● would produce/consumeJSON like:{"topLeft.x" : 0,"topLeft.y" : 0,"bottomRight.x" : 100,"bottomRight.y" : 80}● instead of{ "tl": { "x":0, "y":o },"br" : { "x":100, "y":80 }}
  26. 26. Forgotten: Dyna-beans● Sometimes want to handle set of additional "extra"properties (key/value pairs)public class NamedBeanWithStuff{protected final String name; // mandatory property// and then dynamic propertiesprotected Map<String,Object> other = new HashMap<String,Object>();@JsonCreatorpublic NamedBeanWithStuff@JsonProperty("name") String name){this.name = name;}public String getName() { return name; }
  27. 27. Forgotten Feature: Dyna-beanspublic class NamedBeanWithStuff// continued...public Object get(String name) { // non-Bean accessorreturn other.get(name);}// "any getter" needed for writing out dynamic props@JsonAnyGetterpublic Map<String,Object> any() {return other;}// "any setter" for reading JSON, adding in bean@JsonAnySetterpublic void set(String name, Object value) {other.put(name, value);}}● Can be used for storing unknown properties
  28. 28. Forgotten Feature: OtherObjectReader.updateValue()● Update existing (root) value● Shallow merge● Good for configuration overrides, defaultvaluesData-format auto-detection (alas, too long forexample!)● Detectable formats: JSON, Smile, XML,YAML (maybe CSV in future)● Feed an InputStream, get parser/reader
  29. 29. Forgotten Feature: Other@JacksonInject: injecting values from context● Indicate logical name of value to injectwith annotation● Provide mapping from name to value, perdeserialization call● Can provide value defaultingSupport for "Builder" pattern● @JsonDeserialize(builder=Builder.class)● by default, "withX" methods (overridable)
  30. 30. Thats All Folks!Questions? Comments?Follow me (@cowtowncoder), or email(tsaloranta@gmail.com / tatu@fasterxml.com)ps. FasterXML (http://fasterxml.com) -- theBenevolent Corporate Entity behind Jackson etal -- offers support, training, for Jackson andfriends (Woodstox, Aalto)

×