More Related Content Similar to Wed 1630 greene_robert_color Similar to Wed 1630 greene_robert_color (20) More from DATAVERSITY (20) 2. Overview NoSQL at it’s Core
Pole-Position – Overview
About the Code
Circuits Description
RDB JPA Code
MongoDB Code
Versant JPA Code
Results – Winners of the Race
Developer Challenge
4. A Shift In Application Architecture
Inefficient
CPU destroying
Mapping
• Google – Soft-Schema Excessive
• IBM – Schema-Less Repetitive data
movement and JOIN
calculation
5. Why the Shift is Needed
• Think about it – How Often do Relations Change?
– Blog : BlogEntry , Order : OrderItem , You : Friend
Stop Banging Your Head on the Relational Wall
Relations Rarely Change, Stop Recalculating Them
You don’t need ALL your Data, you can distribute
7. PolePosition
Performance Benchmark
• Established in 2003
– NoSQL -vs- ORM
• Code Complexity Circuits
– Flat Objects
– Graphs
– Inherited Objects
– Collections
• Data Operations
– CRUD
– Concurrency
• Scalability
8. Contenders | Methodology
• Contenders
– ORM
• Hibernate – MySQL / Postgres
• OpenJPA – MySQL / Postgres (other RDB, cannot publish – same
basic results)
– NoSQL
• MongoDB
• Versant Database Engine
• Methodology
– External RDB Experts, Internal NoSQL Experts
– Open Source Benchmark Code for all contenders
9. About the Code
• ORM - Common Code Base
– Hibernate – MySQL / Postgres
– OpenJPA – MySQL / Postgres
– MongoDB
– Versant
10. Circuit Structure
• Simulate CRUD on embedded graph of different Classes.
• Class model - Complex Circuit:
class Holder0 {
String _name;
List <Holder0> _children;
Holder0[] _array; }
class Holder1 extends Holder0 {
int _i1; }
class Holder2 extends Holder1 {
int _i2 <indexed>; } ...class HolderN extends…..
11. Hibernate JPA Code
• Write – Generates Holder’s to user spec depth
ComplexHolder cp = new ComplexHolder( ); em.makePersistent(cp);
• Read – Access root of graph and traverse
cp = em.find( ComplexHolder.class, id ); cp.getChildren().touch();
• Query
String query = "from org.polepos.teams.hibernate.data.Holder2 where
i2=" + currentInt; Iterator it = em.iterate(query);
• Delete – Deletes Graph - cascading operation
12. ORM JPA Mapping
XML Mapping File for each Persistent Class ( 11ea Files )
• - <hibernate-mapping package="org.polepos.teams.hibernate.data" default-cascade="none" default-access="property" default-lazy="true" auto-import="true">
• - <class name="ComplexHolder0" table="tComplexHolderNew0" polymorphism="implicit" mutable="true" dynamic-update="false" dynamic-insert="false" select-before-update="false"
optimistic-lock="version">
• <id name="id" column="fid" type="long" />
• - <discriminator type="string" not-null="true" force="false" insert="true">
• <column name="DISCRIMINATOR" />
• </discriminator>
• <property name="name" column="fname" type="string" unique="false" optimistic-lock="true" lazy="false" generated="never" />
• - <list name="children" access="field" cascade="all" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">
• <key column="parentId" on-delete="noaction" />
• <index column="elementIndex" />
• <many-to-many class="ComplexHolder0" embed-xml="true" not-found="exception" unique="false" />
• </list>
• - <array name="array" access="field" cascade="all" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">
• <key column="parentId" on-delete="noaction" />
• <index column="elementIndex" />
• <many-to-many class="ComplexHolder0" embed-xml="true" not-found="exception" unique="false" />
• </array>
• - <subclass name="ComplexHolder1" discriminator-value="D" dynamic-update="false" dynamic-insert="false" select-before-update="false">
• <property name="i1" column="i1" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />
• - <subclass name="ComplexHolder2" discriminator-value="E" dynamic-update="false" dynamic-insert="false" select-before-update="false">
• <property name="i2" column="i2" type="int" index="i2_idx" unique="false" optimistic-lock="true" lazy="false" generated="never" />
• - <subclass name="ComplexHolder3" discriminator-value="F" dynamic-update="false" dynamic-insert="false" select-before-update="false">
• <property name="i3" column="i3" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />
• - <subclass name="ComplexHolder4" discriminator-value="G" dynamic-update="false" dynamic-insert="false" select-before-update="false">
• <property name="i4" column="i4" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />
• </subclass>
• </subclass>
• </subclass>
• </subclass>
• </class>
• </hibernate-mapping>
13. MongoDB Code
private void storeData(Mongo mongo) {
collection(mongo).insert(new BasicDBObject("test", "object"));
}
public ComplexHolder0 convertFromDocument(DBObject data, DeserializationOptions deserializationOption) {
ComplexHolder0 instance = createInstance(getAsString(data, TYPE_ATTRIBUTE));
instance.setName(getAsString(data, NAME_ATTRIBUTE));
if (null != data.get(CHILDREN)) {
instance.setChildren(fromMongoObjectToList(getAsList(data, CHILDREN), deserializationOption));
}
if (null != data.get(ARRAY)) {
final List<ComplexHolder0> arrayContent = fromMongoObjectToList(getAsList(data, ARRAY),
deserializationOption);
instance.setArray(arrayContent.toArray(new ComplexHolder0[arrayContent.size()]));
}
readAttributes(data, instance);
return instance;
}
14. MongoDB Mapping
private static void writeSubTypeAttributes(ComplexHolder0 holder, BasicDBObject
dataStorage) {
if (holder instanceof ComplexHolder1) {
dataStorage.put(FIELD_I1, ((ComplexHolder1) holder)._i1);
}
if (holder instanceof ComplexHolder2) {
dataStorage.put(FIELD_I2, ((ComplexHolder2) holder)._i2);
}
private static void readAttributes(DBObject dataStorage, ComplexHolder0 holder) {
if (holder instanceof ComplexHolder1) {
((ComplexHolder1) holder)._i1 = (Integer) dataStorage.get(FIELD_I1);
}
if (holder instanceof ComplexHolder2) {
((ComplexHolder2) holder)._i2 = (Integer) dataStorage.get(FIELD_I2);
}
15. MongoDB Mapping
public SerialisationResult convertToDocument(ComplexHolder0 holder)new ArrayList<ComplexHolder0>(data.size());
List<ComplexHolder0> objects = {
List<BasicDBObject> holder2Objects = new(Object o : data) {
for ArrayList<BasicDBObject>();
class Serialisation { BasicDBObject document = convertToDocument(holder, holder2Objects); }
static final String TYPE_ATTRIBUTE = "_t"; objects.add(restoreDocumentOrReference(o, deserializationOption));
if (holder instanceof ComplexHolder4) {
return new SerialisationResult(document, holder2Objects);
}
private static final String PACKAGE_NAME = ComplexHolder0.class.getPackage().getName();
} dataStorage.put(FIELD_I4, ((ComplexHolder4) holder)._i4);
static final String NAME_ATTRIBUTE = "name"; return objects; }
static final String CHILDREN = "children"; } }
public ComplexHolder0 convertFromDocument(DBObject data) {
static final String ARRAY = "array";return convertFromDocument(data,private ComplexHolder0 restoreDocumentOrReference(Object o, DeserializationOptions deserializationOption) {
DeserializationOptions.FULL_DESERIALISATION);
static final String REFERENCE_TO_ORIGINAL_DOCUMENT = "_refToOriginal";
} private static void readAttributes(DBObject dataStorage, if (holder }
static final String FIELD_I1 = "_i1"; DBObject dbObj = (DBObject) o; if (holder instanceof ComplexHolder2) {
static final String FIELD_I2 = "_i2"; if (null != dbObj.get(REFERENCE_TO_ORIGINAL_DOCUMENT)) {
public ComplexHolder0 convertFromDocument(DBObject data, DeserializationOptions deserializationOption) { holder)._i2 = (Integer) dataStorage.get(FIELD_I2);
((ComplexHolder2)
return deserializationOption.deserialize(this, dbObj);
static final String FIELD_I3 = "_i3";
ComplexHolder0 instance = createInstance(getAsString(data, TYPE_ATTRIBUTE)); }
static final String FIELD_I4 = "_i4"; } else { if (holder instanceof ComplexHolder3) {
instance.setName(getAsString(data, NAME_ATTRIBUTE));
return convertFromDocument(dbObj, deserializationOption); holder)._i3 = (Integer) dataStorage.get(FIELD_I3);
if (null != data.get(CHILDREN)) { ((ComplexHolder3)
} }
instance.setChildren(fromMongoObjectToList(getAsList(data, CHILDREN), deserializationOption));
}
private final OneArgFunction<BasicDBObject, DBRef> refCreator;
} if (holder instanceof ComplexHolder4) {
if (null != data.get(ARRAY)) { ((ComplexHolder4) holder)._i4 = (Integer) dataStorage.get(FIELD_I4);
private static List getAsList(DBObject data, String attributeName) {
}
final List<ComplexHolder0> arrayContent = fromMongoObjectToList(getAsList(data, ARRAY), deserializationOption);
return (List) data.get(attributeName); }
Serialisation(OneArgFunction<BasicDBObject, DBRef> refCreator) {
instance.setArray(arrayContent.toArray(new ComplexHolder0[arrayContent.size()]));
this.refCreator = refCreator; } }
} readAttributes(data, instance); private List<Object> toMongoObject(List<ComplexHolder0> children,) {
private static String getAsString(DBObject data, String attribute) {= new ArrayList<Object>(children.size());
List<Object> objects
return instance;
public static Serialisation create(OneArgFunction<BasicDBObject, DBRef> refCreator) { data.get(attribute);
}
return (String) for (ComplexHolder0 child : children) {
if (null == refCreator) { } final BasicDBObject document = convertToDocument(child,
throw new ArgumentNullException("requires a reference creator"); if (isDocumentOnlyReferenced(child)) {
private BasicDBObject convertToDocument(ComplexHolder0 holder, List<BasicDBObject> referencedDocumentsCollector) {
} private static ComplexHolder0 createInstance(String className) {
referencedDocumentsCollector.add(document);
BasicDBObject dbObject = createDocumentWithAttributesOnly(holder);
try {
return new Serialisation(refCreator);
dbObject.put(CHILDREN, toMongoObject(holder.getChildren(), referencedDocumentsCollector)); DBObject copy = createDocumentWithAttributesOnly(child);
} return (ComplexHolder0) Thread.currentThread().getContextClassLoader()
copy.put(REFERENCE_TO_ORIGINAL_DOCUMENT,
if (null != holder.getArray()) { .loadClass(PACKAGE_NAME + "." + className).newInstance();
objects.add(copy);
dbObject.put(ARRAY, toMongoObject(asList(holder.getArray()), referencedDocumentsCollector));
} catch (Exception e) {
public static OneArgFunction<BasicDBObject, DBRef> createReferenceCreator(final DBCollection collection) { } else {
} throw rethrow(e);
if (null == collection) { return dbObject; objects.add(document);
throw new ArgumentNullException("requires a collection"); } }
} }
} }
return new OneArgFunction<BasicDBObject, DBRef>() {
private BasicDBObject createDocumentWithAttributesOnly(ComplexHolder0 holder) { return objects; holder, BasicDBObject dataStorage) {
@Override private static void writeSubTypeAttributes(ComplexHolder0
}
BasicDBObject dbObject = new BasicDBObject("_id", new ObjectId());
if (holder instanceof ComplexHolder1) {
public DBRef invoke(BasicDBObject basicDBObject) {
dbObject.put(NAME_ATTRIBUTE, holder.getName());
final DB db = collection.getDB(); dataStorage.put(FIELD_I1, ((ComplexHolder1) holder)._i1); isDocumentOnlyReferenced(ComplexHolder0 child) {
private static boolean
dbObject.put(TYPE_ATTRIBUTE, holder.getClass().getSimpleName());
}
final Object id = basicDBObject.get("_id");
writeSubTypeAttributes(holder, dbObject); return child.getClass().equals(ComplexHolder2.class);
if (null == id) { if (holder instanceof ComplexHolder2) {}
return dbObject; dataStorage.put(FIELD_I2, ((ComplexHolder2) holder)._i2);
throw new IllegalStateException("Expected an '_id' on the object");
}
} }
return new DBRef(db, collection.getName(), id); if (holder instanceof ComplexHolder3) {
public List<ComplexHolder0> fromMongoObjectToList(List data, DeserializationOptions deserializationOption) {
dataStorage.put(FIELD_I3, ((ComplexHolder3) holder)._i3);
}
}; }
} if (holder instanceof ComplexHolder4) {
16. Versant JPA Code
• Write – Generates Holder’s to user spec depth
ComplexHolder cp = new ComplexHolder( ); em.makePersistent(cp);
• Read – Access root of graph and traverse
cp = em.find( ComplexHolder.class, id ); cp.getChildren().touch();
• Query
String query = "from org.polepos.teams.versant.data.Holder2 where i2="
+ currentInt; Iterator it = session.iterate(query);
• Delete – Deletes Graph – Cascading Operation
17. Versant JPA Mapping
single XML File with primitive declaration entries
<class name="ComplexHolder0">
<field name="name" />
<field name="array" />
<field name="children">
<collection element-type="ComplexHolder0"/> </field>
</class>
18. Sh@t’s and Grins
• JDBC Code
JDBC StringBuilder sb = new StringBuilder(); sb.append("select " +
HOLDER_TABLE0 + ".id from " + HOLDER_TABLE0); sb.append(" INNER
JOIN " + HOLDER_TABLES[0]); sb.append(" on " + HOLDER_TABLE0 + ".id =
" + HOLDER_TABLES[0] + ".id "); sb.append(" INNER JOIN " +
HOLDER_TABLES[1]); sb.append(" on " + HOLDER_TABLE0 + ".id = " +
HOLDER_TABLES[1] + ".id "); sb.append(" LEFT OUTER JOIN " +
HOLDER_TABLES[2]); sb.append(" on " + HOLDER_TABLE0 + ".id = " +
HOLDER_TABLES[2] + ".id "); sb.append(" LEFT OUTER JOIN " +
HOLDER_TABLES[3]); sb.append(" on " + HOLDER_TABLE0 + ".id = " +
HOLDER_TABLES[3] + ".id "); sb.append(" where " + HOLDER_TABLES[1] +
".i2 = ?"); PreparedStatement stat = prepareStatement(sb.toString());
ResultSet resultSet stat.executeQuery();
27. Conclusions
• Wake-up, smell the coffee
– JOINs are not needed - unless adhoc analytics
• Take care of business, dump a load of code
– Serialization is bad, data useful once structured
– Mapping is Mapping, even when it’s not an ORM
• Get on your Bad Motor Scooter and Ride
– NoSQL without Mapping rules the road
Get the Code: http://www.polepos.org
28. Contact
Robert Greene
Vice President, Technology
Versant Corporation
rgreene@versant.com
650-232-2431
