VersantInnovationLeveraging your Knowledge of ORM TowardsPerformance-based NoSQL TechnologyVersant Corporation U.S. Headqu...
Overview   NoSQL at it’s Core           Pole-Position – Overview           About the Code               Circuits Descripti...
NoSQL at its Core
A Shift In Application Architecture                              Inefficient                            CPU destroying    ...
Why the Shift is Needed• Think about it – How Often do Relations Change?    – Blog : BlogEntry , Order : OrderItem , You :...
Measured ValueHow NoSQL performance stacks up to Relational
PolePosition             Performance Benchmark• Established in 2003   – NoSQL -vs- ORM• Code Complexity Circuits   –   Fla...
Contenders | Methodology• Contenders  – ORM     • Hibernate – MySQL / Postgres     • OpenJPA – MySQL / Postgres (other RDB...
About the Code• ORM - Common Code Base  – Hibernate – MySQL / Postgres  – OpenJPA – MySQL / Postgres  – MongoDB  – Versant
Circuit Structure• Simulate CRUD on embedded graph of different Classes.• Class model - Complex Circuit:      class Holder...
Hibernate JPA Code• Write – Generates Holder’s to user spec depth  ComplexHolder cp = new ComplexHolder( ); em.makePersist...
ORM JPA Mapping                            XML Mapping File for each Persistent Class ( 11ea Files )•   - <hibernate-mappi...
MongoDB Codeprivate void storeData(Mongo mongo) {    collection(mongo).insert(new BasicDBObject("test", "object")); }publi...
MongoDB Mappingprivate static void writeSubTypeAttributes(ComplexHolder0 holder, BasicDBObject   dataStorage) {    if (hol...
MongoDB Mapping                                    public SerialisationResult convertToDocument(ComplexHolder0 holder)new ...
Versant JPA Code• Write – Generates Holder’s to user spec depth  ComplexHolder cp = new ComplexHolder( ); em.makePersisten...
Versant JPA Mapping single XML File with primitive declaration entries<class name="ComplexHolder0">     <field name="name"...
Sh@t’s and Grins• JDBC CodeJDBC StringBuilder sb = new StringBuilder(); sb.append("select " +   HOLDER_TABLE0 + ".id from ...
PolePositionBenchMark Summary Results
FlatObject - Single Thread
Results – ComplexConcurrency        @mongo - no safe mode
Results – QueryConcurrency       @mongo – no safe mode
Results – InsertConcurrent       @mongo – no safe mode
Results – ComplexConcurrency        @mongo – safe mode
Results – QueryConcurrency        @mongo – safe mode
Results – InsertConcurrent       @mongo – safe mode
Conclusions• Wake-up, smell the coffee  – JOINs are not needed - unless adhoc analytics• Take care of business, dump a loa...
Contact      Robert GreeneVice President, Technology   Versant Corporation  rgreene@versant.com      650-232-2431
Upcoming SlideShare
Loading in...5
×

Wed 1630 greene_robert_color

425

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
425
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Wed 1630 greene_robert_color

  1. 1. VersantInnovationLeveraging your Knowledge of ORM TowardsPerformance-based NoSQL TechnologyVersant Corporation U.S. Headquarters255 Shoreline Dr. Suite 450, Redwood City, CA 94065www.versant.com | 650-232-2400
  2. 2. Overview NoSQL at it’s Core Pole-Position – Overview About the Code Circuits Description RDB JPA Code MongoDB Code Versant JPA Code Results – Winners of the Race Developer Challenge
  3. 3. NoSQL at its Core
  4. 4. A Shift In Application Architecture Inefficient CPU destroying Mapping • Google – Soft-Schema Excessive • IBM – Schema-Less Repetitive data movement and JOIN calculation
  5. 5. Why the Shift is Needed• Think about it – How Often do Relations Change? – Blog : BlogEntry , Order : OrderItem , You : FriendStop Banging Your Head on the Relational WallRelations Rarely Change, Stop Recalculating ThemYou don’t need ALL your Data, you can distribute
  6. 6. Measured ValueHow NoSQL performance stacks up to Relational
  7. 7. PolePosition Performance Benchmark• Established in 2003 – NoSQL -vs- ORM• Code Complexity Circuits – Flat Objects – Graphs – Inherited Objects – Collections• Data Operations – CRUD – Concurrency• Scalability
  8. 8. Contenders | Methodology• Contenders – ORM • Hibernate – MySQL / Postgres • OpenJPA – MySQL / Postgres (other RDB, cannot publish – same basic results) – NoSQL • MongoDB • Versant Database Engine• Methodology – External RDB Experts, Internal NoSQL Experts – Open Source Benchmark Code for all contenders
  9. 9. About the Code• ORM - Common Code Base – Hibernate – MySQL / Postgres – OpenJPA – MySQL / Postgres – MongoDB – Versant
  10. 10. Circuit Structure• Simulate CRUD on embedded graph of different Classes.• Class model - Complex Circuit: class Holder0 { String _name; List <Holder0> _children; Holder0[] _array; } class Holder1 extends Holder0 { int _i1; } class Holder2 extends Holder1 { int _i2 <indexed>; } ...class HolderN extends…..
  11. 11. Hibernate JPA Code• Write – Generates Holder’s to user spec depth ComplexHolder cp = new ComplexHolder( ); em.makePersistent(cp);• Read – Access root of graph and traverse cp = em.find( ComplexHolder.class, id ); cp.getChildren().touch();• Query String query = "from org.polepos.teams.hibernate.data.Holder2 where i2=" + currentInt; Iterator it = em.iterate(query);• Delete – Deletes Graph - cascading operation
  12. 12. ORM JPA Mapping XML Mapping File for each Persistent Class ( 11ea Files )• - <hibernate-mapping package="org.polepos.teams.hibernate.data" default-cascade="none" default-access="property" default-lazy="true" auto-import="true">• - <class name="ComplexHolder0" table="tComplexHolderNew0" polymorphism="implicit" mutable="true" dynamic-update="false" dynamic-insert="false" select-before-update="false" optimistic-lock="version">• <id name="id" column="fid" type="long" />• - <discriminator type="string" not-null="true" force="false" insert="true">• <column name="DISCRIMINATOR" />• </discriminator>• <property name="name" column="fname" type="string" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <list name="children" access="field" cascade="all" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">• <key column="parentId" on-delete="noaction" />• <index column="elementIndex" />• <many-to-many class="ComplexHolder0" embed-xml="true" not-found="exception" unique="false" />• </list>• - <array name="array" access="field" cascade="all" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">• <key column="parentId" on-delete="noaction" />• <index column="elementIndex" />• <many-to-many class="ComplexHolder0" embed-xml="true" not-found="exception" unique="false" />• </array>• - <subclass name="ComplexHolder1" discriminator-value="D" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i1" column="i1" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <subclass name="ComplexHolder2" discriminator-value="E" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i2" column="i2" type="int" index="i2_idx" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <subclass name="ComplexHolder3" discriminator-value="F" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i3" column="i3" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <subclass name="ComplexHolder4" discriminator-value="G" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i4" column="i4" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />• </subclass>• </subclass>• </subclass>• </subclass>• </class>• </hibernate-mapping>
  13. 13. MongoDB Codeprivate void storeData(Mongo mongo) { collection(mongo).insert(new BasicDBObject("test", "object")); }public ComplexHolder0 convertFromDocument(DBObject data, DeserializationOptions deserializationOption) { ComplexHolder0 instance = createInstance(getAsString(data, TYPE_ATTRIBUTE)); instance.setName(getAsString(data, NAME_ATTRIBUTE)); if (null != data.get(CHILDREN)) { instance.setChildren(fromMongoObjectToList(getAsList(data, CHILDREN), deserializationOption)); } if (null != data.get(ARRAY)) { final List<ComplexHolder0> arrayContent = fromMongoObjectToList(getAsList(data, ARRAY), deserializationOption); instance.setArray(arrayContent.toArray(new ComplexHolder0[arrayContent.size()])); } readAttributes(data, instance); return instance; }
  14. 14. MongoDB Mappingprivate static void writeSubTypeAttributes(ComplexHolder0 holder, BasicDBObject dataStorage) { if (holder instanceof ComplexHolder1) { dataStorage.put(FIELD_I1, ((ComplexHolder1) holder)._i1); } if (holder instanceof ComplexHolder2) { dataStorage.put(FIELD_I2, ((ComplexHolder2) holder)._i2); }private static void readAttributes(DBObject dataStorage, ComplexHolder0 holder) { if (holder instanceof ComplexHolder1) { ((ComplexHolder1) holder)._i1 = (Integer) dataStorage.get(FIELD_I1); } if (holder instanceof ComplexHolder2) { ((ComplexHolder2) holder)._i2 = (Integer) dataStorage.get(FIELD_I2); }
  15. 15. MongoDB Mapping public SerialisationResult convertToDocument(ComplexHolder0 holder)new ArrayList<ComplexHolder0>(data.size()); List<ComplexHolder0> objects = { List<BasicDBObject> holder2Objects = new(Object o : data) { for ArrayList<BasicDBObject>();class Serialisation { BasicDBObject document = convertToDocument(holder, holder2Objects); } static final String TYPE_ATTRIBUTE = "_t"; objects.add(restoreDocumentOrReference(o, deserializationOption)); if (holder instanceof ComplexHolder4) { return new SerialisationResult(document, holder2Objects); } private static final String PACKAGE_NAME = ComplexHolder0.class.getPackage().getName(); } dataStorage.put(FIELD_I4, ((ComplexHolder4) holder)._i4); static final String NAME_ATTRIBUTE = "name"; return objects; } static final String CHILDREN = "children"; } } public ComplexHolder0 convertFromDocument(DBObject data) { static final String ARRAY = "array";return convertFromDocument(data,private ComplexHolder0 restoreDocumentOrReference(Object o, DeserializationOptions deserializationOption) { DeserializationOptions.FULL_DESERIALISATION); static final String REFERENCE_TO_ORIGINAL_DOCUMENT = "_refToOriginal"; } private static void readAttributes(DBObject dataStorage, if (holder } static final String FIELD_I1 = "_i1"; DBObject dbObj = (DBObject) o; if (holder instanceof ComplexHolder2) { static final String FIELD_I2 = "_i2"; if (null != dbObj.get(REFERENCE_TO_ORIGINAL_DOCUMENT)) { public ComplexHolder0 convertFromDocument(DBObject data, DeserializationOptions deserializationOption) { holder)._i2 = (Integer) dataStorage.get(FIELD_I2); ((ComplexHolder2) return deserializationOption.deserialize(this, dbObj); static final String FIELD_I3 = "_i3"; ComplexHolder0 instance = createInstance(getAsString(data, TYPE_ATTRIBUTE)); } static final String FIELD_I4 = "_i4"; } else { if (holder instanceof ComplexHolder3) { instance.setName(getAsString(data, NAME_ATTRIBUTE)); return convertFromDocument(dbObj, deserializationOption); holder)._i3 = (Integer) dataStorage.get(FIELD_I3); if (null != data.get(CHILDREN)) { ((ComplexHolder3) } } instance.setChildren(fromMongoObjectToList(getAsList(data, CHILDREN), deserializationOption)); } private final OneArgFunction<BasicDBObject, DBRef> refCreator; } if (holder instanceof ComplexHolder4) { if (null != data.get(ARRAY)) { ((ComplexHolder4) holder)._i4 = (Integer) dataStorage.get(FIELD_I4); private static List getAsList(DBObject data, String attributeName) { } final List<ComplexHolder0> arrayContent = fromMongoObjectToList(getAsList(data, ARRAY), deserializationOption); return (List) data.get(attributeName); } Serialisation(OneArgFunction<BasicDBObject, DBRef> refCreator) { instance.setArray(arrayContent.toArray(new ComplexHolder0[arrayContent.size()])); this.refCreator = refCreator; } } } readAttributes(data, instance); private List<Object> toMongoObject(List<ComplexHolder0> children,) { private static String getAsString(DBObject data, String attribute) {= new ArrayList<Object>(children.size()); List<Object> objects return instance; public static Serialisation create(OneArgFunction<BasicDBObject, DBRef> refCreator) { data.get(attribute); } return (String) for (ComplexHolder0 child : children) { if (null == refCreator) { } final BasicDBObject document = convertToDocument(child, throw new ArgumentNullException("requires a reference creator"); if (isDocumentOnlyReferenced(child)) { private BasicDBObject convertToDocument(ComplexHolder0 holder, List<BasicDBObject> referencedDocumentsCollector) { } private static ComplexHolder0 createInstance(String className) { referencedDocumentsCollector.add(document); BasicDBObject dbObject = createDocumentWithAttributesOnly(holder); try { return new Serialisation(refCreator); dbObject.put(CHILDREN, toMongoObject(holder.getChildren(), referencedDocumentsCollector)); DBObject copy = createDocumentWithAttributesOnly(child); } return (ComplexHolder0) Thread.currentThread().getContextClassLoader() copy.put(REFERENCE_TO_ORIGINAL_DOCUMENT, if (null != holder.getArray()) { .loadClass(PACKAGE_NAME + "." + className).newInstance(); objects.add(copy); dbObject.put(ARRAY, toMongoObject(asList(holder.getArray()), referencedDocumentsCollector)); } catch (Exception e) { public static OneArgFunction<BasicDBObject, DBRef> createReferenceCreator(final DBCollection collection) { } else { } throw rethrow(e); if (null == collection) { return dbObject; objects.add(document); throw new ArgumentNullException("requires a collection"); } } } } } } return new OneArgFunction<BasicDBObject, DBRef>() { private BasicDBObject createDocumentWithAttributesOnly(ComplexHolder0 holder) { return objects; holder, BasicDBObject dataStorage) { @Override private static void writeSubTypeAttributes(ComplexHolder0 } BasicDBObject dbObject = new BasicDBObject("_id", new ObjectId()); if (holder instanceof ComplexHolder1) { public DBRef invoke(BasicDBObject basicDBObject) { dbObject.put(NAME_ATTRIBUTE, holder.getName()); final DB db = collection.getDB(); dataStorage.put(FIELD_I1, ((ComplexHolder1) holder)._i1); isDocumentOnlyReferenced(ComplexHolder0 child) { private static boolean dbObject.put(TYPE_ATTRIBUTE, holder.getClass().getSimpleName()); } final Object id = basicDBObject.get("_id"); writeSubTypeAttributes(holder, dbObject); return child.getClass().equals(ComplexHolder2.class); if (null == id) { if (holder instanceof ComplexHolder2) {} return dbObject; dataStorage.put(FIELD_I2, ((ComplexHolder2) holder)._i2); throw new IllegalStateException("Expected an _id on the object"); } } } return new DBRef(db, collection.getName(), id); if (holder instanceof ComplexHolder3) { public List<ComplexHolder0> fromMongoObjectToList(List data, DeserializationOptions deserializationOption) { dataStorage.put(FIELD_I3, ((ComplexHolder3) holder)._i3); } }; } } if (holder instanceof ComplexHolder4) {
  16. 16. Versant JPA Code• Write – Generates Holder’s to user spec depth ComplexHolder cp = new ComplexHolder( ); em.makePersistent(cp);• Read – Access root of graph and traverse cp = em.find( ComplexHolder.class, id ); cp.getChildren().touch();• Query String query = "from org.polepos.teams.versant.data.Holder2 where i2=" + currentInt; Iterator it = session.iterate(query);• Delete – Deletes Graph – Cascading Operation
  17. 17. Versant JPA Mapping single XML File with primitive declaration entries<class name="ComplexHolder0"> <field name="name" /> <field name="array" /> <field name="children"> <collection element-type="ComplexHolder0"/> </field></class>
  18. 18. Sh@t’s and Grins• JDBC CodeJDBC StringBuilder sb = new StringBuilder(); sb.append("select " + HOLDER_TABLE0 + ".id from " + HOLDER_TABLE0); sb.append(" INNER JOIN " + HOLDER_TABLES[0]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[0] + ".id "); sb.append(" INNER JOIN " + HOLDER_TABLES[1]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[1] + ".id "); sb.append(" LEFT OUTER JOIN " + HOLDER_TABLES[2]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[2] + ".id "); sb.append(" LEFT OUTER JOIN " + HOLDER_TABLES[3]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[3] + ".id "); sb.append(" where " + HOLDER_TABLES[1] + ".i2 = ?"); PreparedStatement stat = prepareStatement(sb.toString()); ResultSet resultSet stat.executeQuery();
  19. 19. PolePositionBenchMark Summary Results
  20. 20. FlatObject - Single Thread
  21. 21. Results – ComplexConcurrency @mongo - no safe mode
  22. 22. Results – QueryConcurrency @mongo – no safe mode
  23. 23. Results – InsertConcurrent @mongo – no safe mode
  24. 24. Results – ComplexConcurrency @mongo – safe mode
  25. 25. Results – QueryConcurrency @mongo – safe mode
  26. 26. Results – InsertConcurrent @mongo – safe mode
  27. 27. Conclusions• Wake-up, smell the coffee – JOINs are not needed - unless adhoc analytics• Take care of business, dump a load of code – Serialization is bad, data useful once structured – Mapping is Mapping, even when it’s not an ORM• Get on your Bad Motor Scooter and Ride – NoSQL without Mapping rules the road Get the Code: http://www.polepos.org
  28. 28. Contact Robert GreeneVice President, Technology Versant Corporation rgreene@versant.com 650-232-2431
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×