Java IncrementalDeserialization/reserialization (JID)              Bill La Forge      CTO, Jactor Consulting     http://ja...
The Problem●   Data is stored on disk as byte arrays, which    may be quite large.●   These arrays must be deserialized be...
Incremental Deserialization and       ReserializationJID Benchmark results for deserializing, updating and thenreserializi...
Types of Serialization●   Many applications use      ●   Hard-coded serialization    reflection-based               direct...
JID is Complex●   Every JID object must track the serialized data    that has not been outdated and any data that    has b...
Factories●   Registered factory objects are needed to instantiate    and initialize JID objects.●   Each registered facory...
User Classpublic class User extends AppJid {  private StringJid getNameJid() throws Exception {     return (StringJid) _iG...
UserFactoryClasspublic class UserFactory extends AppJidFactory {  final public static UserFactory fac = new UserFactory();...
User Test Environmentpublic class Apps {  public static void main(String[] args) throws Exception {     JAFactory factory ...
SerializingRootJid rootJid0 = (RootJid)   factory.newActor(JidFactories.ROOT_JID_TYPE);rootJid0.setValue("User");User user...
DeserializationRootJid rootJid1 = (RootJid)  factory.newActor(JidFactories.ROOT_JID_TYPE);rootJid1.load(serializedData0, 0...
JID ProjectJID Project Page:  http://jactorconsulting.com/product/jid/License: LGPL
Upcoming SlideShare
Loading in …5
×

Incremental Java Deserialization/Reserialization

1,064 views

Published on

Fast deserialization/reserialization is critical for databases. The key is to deserialize only what you need and to keep the serialized data for items that have not been changed for reserialization after an update.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,064
On SlideShare
0
From Embeds
0
Number of Embeds
124
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Incremental Java Deserialization/Reserialization

  1. 1. Java IncrementalDeserialization/reserialization (JID) Bill La Forge CTO, Jactor Consulting http://jactorconsulting.com
  2. 2. The Problem● Data is stored on disk as byte arrays, which may be quite large.● These arrays must be deserialized before the data can be updated and then reserialized.● There are various techniques for deserialization and reserialization but they are all quite slow as all the data is deserialized and then reserialized.
  3. 3. Incremental Deserialization and ReserializationJID Benchmark results for deserializing, updating and thenreserializing large collections. Entries List Insertion Map Update 1,000 .014 ms .027 ms 10.000 .035 ms .056 ms 100,000 .24 ms .48 ms 1,000,000 2.9 ms 6.9 msHigh performance is achieved by deserializing onlythe data that is needed and reserializing only thedata that has been updated.
  4. 4. Types of Serialization● Many applications use ● Hard-coded serialization reflection-based directly converst data serialization. from/to arrays of bytes.● Reflection-based ● This is harder to serialization is the implement, but runs easiest to use, but also faster. the slowest. ● But mixing applicaiton● There are three main logic with serialization types: native logic detracts from the serialization, XML-based clarity of the code. and Jason.
  5. 5. JID is Complex● Every JID object must track the serialized data that has not been outdated and any data that has been deserialized, along with notifying its container when there is a change.● Serializable data structures are trees of JID objects that are mostly instances of pre-defined classes.● Application classes contain no serialization logic, but subclass a JID vector of homogenious JID types.
  6. 6. Factories● Registered factory objects are needed to instantiate and initialize JID objects.● Each registered facory object represents a different JID type and has a unique name.● All factory objects are instances of subclasses of the JidFactory class.● Factory objects used to instantiate application JID objects are instances of subclasses of AppJidFactory.● These factory objects provide the type names of the registered factory objects needed to create the tuple elements of the application objects.
  7. 7. User Classpublic class User extends AppJid { private StringJid getNameJid() throws Exception { return (StringJid) _iGet(0);} private IntegerJid getAgeJid() throws Exception { return (IntegerJid) _iGet(1);} public String getName() throws Exception { return getNameJid().getValue();} public void setName(String name) throws Exception { getNameJid().setValue(name);} public int getAge() throws Exception { return getAgeJid().getValue();} public void setAge(int age) throws Exception { getAgeJid().setValue(age);}}
  8. 8. UserFactoryClasspublic class UserFactory extends AppJidFactory { final public static UserFactory fac = new UserFactory(); public UserFactory() { super("User", JidFactories.STRING_JID_TYPE, JidFactories.INTEGER_JID_TYPE); } protected User instantiateActor() throws Exception { return new User(); }}
  9. 9. User Test Environmentpublic class Apps { public static void main(String[] args) throws Exception { JAFactory factory = new JAFactory(); (new JidFactories()).initialize(factory); factory.registerActorFactory(UserFactory.fac); . . . }}
  10. 10. SerializingRootJid rootJid0 = (RootJid) factory.newActor(JidFactories.ROOT_JID_TYPE);rootJid0.setValue("User");User user0 = (User) rootJid0.getValue();user0.setName("Fred");user0.setAge(38);int serializedLength0 = rootJid0.getSerializedLength();byte[] serializedData0 = new byte[serializedLength0];rootJid0.save(serializedData0, 0);
  11. 11. DeserializationRootJid rootJid1 = (RootJid) factory.newActor(JidFactories.ROOT_JID_TYPE);rootJid1.load(serializedData0, 0, serializedLength0);User user1 = (User) rootJid1.getValue();if (!"Fred".equals(user1.getName())) throw new Exception("unexpected result");if (38 != user1.getAge()) throw new Exception("unexpected result");
  12. 12. JID ProjectJID Project Page: http://jactorconsulting.com/product/jid/License: LGPL

×