Java Serialization
Facts and Fallacies
© 2013, Roman Elizarov, Devexperts
Serialization
… is the process of translating data structures or object
state into a format that can be stored (for exampl...
Use-cases
• Transfer data over network between cluster nodes or between
servers and clients
- Serialization is a key techn...
Java serialization facility (key facts)
• Had appeared in 1997 as a part of JDK 1.1 release
• Uses java.io.Serializable as...
It is a joke
… and the only JEP joke (!)
… and it has a big grain of truth:
There are lots of folk myths about serializati...
Hazelcast Portable
Google protobuf
Hessian
Protostuff

Kryo

Coherence POF
JBoss serialization

Apache Avro

fast-serializ...
The obvious fact of live

Real hackers write everything from scratch 
Every mature appserver/framework/cache/eventbus/soa...
How to choose a serialization lib?
• Cross-language or Java-only
• Binary or text (XML/JSON/other) format
• Custom or auto...
Java built-in serialization lib?
• Cross-language or Java-only
• Binary or text (XML/JSON/other) format
• Custom or automa...
Serious serialization lib choice diagram
YES

YES

Java to JS?

Cross-platform
data transfer?

NO

Use built-in Java
Seria...
Serialization myth #1
I’ve found this framework called XYZ…
… it has really simple and fast serialization

All you have to...
In fact this is corollary to “The obvious fact of life”
• Of course it is the fastest possible way! But this does not solv...
Serialization myth #2

java.io.Serializable …
“This interface limits how its implementing classes can
change in the future...
So what about this code evolution?

write
read

got
serialVersionUID
• serialVersionUID = crazy hash of your class
- If you change your class -> serialVersionUID changes -> i...
serialver
• That’s the tool to compute serialVersionUID as some crazy hash
of your class
- You only need it when your clas...
So what about this code evolution (2)?

wrie
got
symbol – as written
quantity – as written
price – as written
orderType – ...
What if I want to make my class incompatible?
• You have two choices
- serialVersionUID++
- Rename class

• So, having ser...
Complex changes made compatible
• Java serialization has a bunch of features to support radical
evolution of serializable ...
What about “renaming private fields is unsafe”?
• It is just a matter of developer’s choice on what actually constitutes
“...
http://www.organisationscience.com/styled-6/
Serialization myth #3
In practice
• It really matters when very verbose text formats like XML are used
in a combination with not-so-fast seriali...
Popular myth #4

© Stack Overflow authors, emphasis is mine
A popular way to benchmark serialization

https://github.com/eishay/jvm-serializers/blob/master/tpc/src/serializers/JavaBu...
The real fact about serialization performance
• ObjectOutputStream/ObjectInputStream are expensive to create
- >1KB of int...
The real fact about serialization performance (2)
• Each ObjectOutputStream writes class descriptor on the first
encounter...
Beware of benchmarks
• A typical benchmark compares different serialization frameworks
doing tasks of absolutely different...
Real cons of Java serialization
• Just one reset() to rule them all
- No way to “resetObjects()” to clear all message-leve...
Summary
• Java serialization is there out-of-the box
• It handles most Java-to-Java serialization needs well
- While provi...
46000

305400
Sent orders in one day

Users on-line in one day

We create professional financial software for
Brokerage an...
Headquarters
197110, 10/1 Barochnaya st.
Saint Petersburg, Russia
+7 812 438 16 26
mail@devexperts.com

www.devexperts.com
Contact me by email: elizarov at devexperts.com
Read my blog: http://elizarov.livejournal.com/
Java Serialization Facts and Fallacies
Java Serialization Facts and Fallacies
Upcoming SlideShare
Loading in...5
×

Java Serialization Facts and Fallacies

2,428

Published on

Published in: Technology, Education
3 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,428
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
70
Comments
3
Likes
6
Embeds 0
No embeds

No notes for slide

Java Serialization Facts and Fallacies

  1. 1. Java Serialization Facts and Fallacies © 2013, Roman Elizarov, Devexperts
  2. 2. Serialization … is the process of translating data structures or object state into a format that can be stored (for example, in a file or memory buffer, or transmitted across a network connection link) and resurrected later in the same or another computer environment. -- WIkipedia Object Bytes Network/Storage
  3. 3. Use-cases • Transfer data over network between cluster nodes or between servers and clients - Serialization is a key technology behind any RPC/RMI • Serialization if often used for storage - but then data transfer is still often a part of the picture
  4. 4. Java serialization facility (key facts) • Had appeared in 1997 as a part of JDK 1.1 release • Uses java.io.Serializable as a marker interface • Consists of java.io.ObjectInputStream/ObjectOutputStream with related classes and interfaces • Has “Java Object Serialization Specification” that documents both API and object serialization stream protocol • Is part of any Java platform (even Android has it) • Somehow has a lot of myths around it
  5. 5. It is a joke … and the only JEP joke (!) … and it has a big grain of truth: There are lots of folk myths about serialization
  6. 6. Hazelcast Portable Google protobuf Hessian Protostuff Kryo Coherence POF JBoss serialization Apache Avro fast-serialization Apache Thrift Hadoop Writable jserial quickser GridGain serialization msgpack wobly Burlap EclipseLink MOXy Gson Json-lib JAXB XStream FLEXJSON Javolution XML Jackson svenson java.beans.XMLEncoder/XMLDecoder jsonij
  7. 7. The obvious fact of live Real hackers write everything from scratch  Every mature appserver/framework/cache/eventbus/soa/rpc/db/etc must have its own serialization that is better than everybody else’s 
  8. 8. How to choose a serialization lib? • Cross-language or Java-only • Binary or text (XML/JSON/other) format • Custom or automatic object-to-serialized format mapping • Needs format description/mapping/annotation or not • Writes object scheme (e.g. field names) with each object, once per request, once per session, never at all, or some mixed approach • Writes object fields or bean properties • Serializes object graph with shared objects or not • Performance - CPU consumption to serialize/deserialize - Network consumption (bytes per object)
  9. 9. Java built-in serialization lib? • Cross-language or Java-only • Binary or text (XML/JSON/other) format • Custom or automatic object-to-serialized format mapping • Needs format description/mapping/annotation or not • Writes object scheme (e.g. field names) with each object, once per request, once per session, never at all, or some mixed approach • Writes object fields or bean properties • Serializes object graph with shared objects or not • Performance - CPU consumption to serialize/deserialize - Network consumption (bytes per object)
  10. 10. Serious serialization lib choice diagram YES YES Java to JS? Cross-platform data transfer? NO Use built-in Java Serialization YES Jackson/gson/etc do further research NO Do you have problems with it? NO
  11. 11. Serialization myth #1 I’ve found this framework called XYZ… … it has really simple and fast serialization All you have to do is to implement this interface! Easy! I’ve tested it! It is really fast! It outperforms everything else I’ve tried! /* This is almost actual real-life conversation. * Names changed */
  12. 12. In fact this is corollary to “The obvious fact of life” • Of course it is the fastest possible way! But this does not solve the most complex serialization problems - You can implement java.io.Externalizable if you are up to writing a custom serialization code for each object. • Maintenance cost/efforts are often underestimated - How are you planning to evolve your system? Add new fields? New classes? • What will keep you from forgetting to read/write them? • Ah… yes, you write a lot of extra tests in addition to writing your write/read methods to make sure your serialization works - How about on-the-wire compatibility of old code and new one? • But I deploy all of my system with a new version at once! • But how do you work with it during development?
  13. 13. Serialization myth #2 java.io.Serializable … “This interface limits how its implementing classes can change in the future. By implementing Serializable you expose your flexible in-memory implementation details as a rigid binary representation. Simple code changes--like renaming private fields--are not safe when the changed class is serializable.” http://developer.android.com/reference/java/io/Serializable.html
  14. 14. So what about this code evolution? write read got
  15. 15. serialVersionUID • serialVersionUID = crazy hash of your class - If you change your class -> serialVersionUID changes -> incompatible • But you can set serialVersionUID as a “private static final long” field in your class to fix it. • With a fixed serialVersionUID java serialization supports: - Adding/removing fields without any additional hurdles and annotations - Moving fields around in class definition - Extracting/inlining super-classes (but without moving serializable fields up/down class hierarchy). • It basically means you can evolve your class at will, just a you would do with key-value map in json/other-untyped-system
  16. 16. serialver • That’s the tool to compute serialVersionUID as some crazy hash of your class - You only need it when your class is “in the wild” without explicitly assigned serialVersionUID and you want to be backwards compatible with that version of it • DO NOT USE serialver for freshly created classes - It is a pseudo random number that serves no purpose, but to make your gzipped class files and gzipped serial files larger - Just set serialVersionUID = 0 • It is no worse, but better than any other value
  17. 17. So what about this code evolution (2)? wrie got symbol – as written quantity – as written price – as written orderType – null read Profit! It works the other way around, too!
  18. 18. What if I want to make my class incompatible? • You have two choices - serialVersionUID++ - Rename class • So, having serialVersionUID in a modern serialization framework is an optional feature - They just did not have refactorings back then in 1996, so renaming a class was not an option for them - They chose a default of “your changes break serialization” and force you to explicitly declare serialVersionUID if you want “your changes are backwards compatible” • I would have preferred the other default, but having to manually add serialVersionUID is a small price to pay for a serialization facility that you have out-of-the box in Java.
  19. 19. Complex changes made compatible • Java serialization has a bunch of features to support radical evolution of serializable classes in backwards-compatible way: - writeReplace/readResolve • Lets you migrate your code to a completely new class, but still use the old class for serialization. - putFields/writeFields/readFields Lets you completely redo the fields of the object, but still be compatible with the old set of fields for serialization. It really helps if you are working on, evolving, improving, and refactoring really big projects
  20. 20. What about “renaming private fields is unsafe”? • It is just a matter of developer’s choice on what actually constitutes “serial form” of the class: - It’s private fields • like in Java serialization - It’s public API with its getter/setter methods • aka “Bean Serialization” • many serialization frameworks follow this approach • good match when you need to expose public API via serialization - A totally separate code, file, or annotation defines serial form • you have to write it in addition to your class • more work, but the approach is the most flexible and pays of in case of highly heterogeneous system (many platforms/languages)
  21. 21. http://www.organisationscience.com/styled-6/
  22. 22. Serialization myth #3
  23. 23. In practice • It really matters when very verbose text formats like XML are used in a combination with not-so-fast serializer implementations • Most binary formats are fast and compact enough, so that they do not constitute a bottleneck in the system - Data is usually expensive to acquire (compute, fetch from db, etc) - Otherwise (if data is in cache) and the only action of the system is to retrieve it from cache and send, then cache serialized data in binary form, thus making serialization performance irrelevant aka “marshalled object”
  24. 24. Popular myth #4 © Stack Overflow authors, emphasis is mine
  25. 25. A popular way to benchmark serialization https://github.com/eishay/jvm-serializers/blob/master/tpc/src/serializers/JavaBuiltIn.java
  26. 26. The real fact about serialization performance • ObjectOutputStream/ObjectInputStream are expensive to create - >1KB of internal buffers allocated - It matters if you really do a lot of serialization • You can reuse OOS/OIS instances if you need - It is easy when you write a single stream of values • Just keep doing writeObject(!) • That is what it was designed for • The added benefit is that class descriptors are written just once - It is slightly tricky if you want to optimize it for one time writes • The actual gain is modest and depends on the kinds of objects you are serializing (their size, your heap/gc load, etc)
  27. 27. The real fact about serialization performance (2) • Each ObjectOutputStream writes class descriptor on the first encounter - reset() clears internal handles table that keep both written objects (for object graph serialization) and written class descriptors • BEWARE: All written objects are retained until you call reset() • Writing class descriptor is a signification portion of both CPU time consumption and resulting space - That is the price you pay for all the automagic class evolution features without having to manually define field ids, create mappings, or write custom code - The data itself is relatively fast to write and has straightforward binary format (4 bytes per int, etc)
  28. 28. Beware of benchmarks • A typical benchmark compares different serialization frameworks doing tasks of absolutely different complexity • With Java serialization facility you necessary pay for - No hassle object-to-serial format mapping • (no writing of interface descriptions, no code generation) - Support for wide range of class evolution scenarios - Support for object graph writes - Support for machines with different byte-endianness • Most significant performance/size improvements are gained by forfeiting one of these qualities - Be sure to understand what exactly you forfeit
  29. 29. Real cons of Java serialization • Just one reset() to rule them all - No way to “resetObjects()” to clear all message-level object graph info (referenced objects), but keep session-level information (written class descriptors) to avoid repetition when keeping a network session to recipient open • Anyone knows ins-and-outs of JEP process to improve it? • Can use some performance improvement  - Some obviously unnecessary allocations of internal objects can be removed, code simplified, streamlined • I’m looking for help in “contributing to JDK” area - But do remember, that it rarely matters in practice
  30. 30. Summary • Java serialization is there out-of-the box • It handles most Java-to-Java serialization needs well - While providing a wide range of goodies • It provides non-compact, but binary representation - It works especially good, if you have collections of repeated business objects • It covers a wide range of class evolution scenarios - Set serialVersionUID = 0 to be happy • It has decent performance that does not often show on profiler’s radar in practice • We use it in Devexperts for all our Java-to-Java non-market-data
  31. 31. 46000 305400 Sent orders in one day Users on-line in one day We create professional financial software for Brokerage and Exchanges since 2002 99.99% Our software index of trouble-free operation 350 Peoples in our team 365 7 24 We support our products around-the-clock
  32. 32. Headquarters 197110, 10/1 Barochnaya st. Saint Petersburg, Russia +7 812 438 16 26 mail@devexperts.com www.devexperts.com
  33. 33. Contact me by email: elizarov at devexperts.com Read my blog: http://elizarov.livejournal.com/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×