Wed 1630 greene_robert_color
 

Wed 1630 greene_robert_color

on

  • 539 views

 

Statistics

Views

Total Views
539
Views on SlideShare
539
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Wed 1630 greene_robert_color Wed 1630 greene_robert_color Presentation Transcript

    • VersantInnovationLeveraging your Knowledge of ORM TowardsPerformance-based NoSQL TechnologyVersant Corporation U.S. Headquarters255 Shoreline Dr. Suite 450, Redwood City, CA 94065www.versant.com | 650-232-2400
    • Overview NoSQL at it’s Core Pole-Position – Overview About the Code Circuits Description RDB JPA Code MongoDB Code Versant JPA Code Results – Winners of the Race Developer Challenge
    • NoSQL at its Core
    • A Shift In Application Architecture Inefficient CPU destroying Mapping • Google – Soft-Schema Excessive • IBM – Schema-Less Repetitive data movement and JOIN calculation
    • Why the Shift is Needed• Think about it – How Often do Relations Change? – Blog : BlogEntry , Order : OrderItem , You : FriendStop Banging Your Head on the Relational WallRelations Rarely Change, Stop Recalculating ThemYou don’t need ALL your Data, you can distribute
    • Measured ValueHow NoSQL performance stacks up to Relational
    • PolePosition Performance Benchmark• Established in 2003 – NoSQL -vs- ORM• Code Complexity Circuits – Flat Objects – Graphs – Inherited Objects – Collections• Data Operations – CRUD – Concurrency• Scalability
    • Contenders | Methodology• Contenders – ORM • Hibernate – MySQL / Postgres • OpenJPA – MySQL / Postgres (other RDB, cannot publish – same basic results) – NoSQL • MongoDB • Versant Database Engine• Methodology – External RDB Experts, Internal NoSQL Experts – Open Source Benchmark Code for all contenders
    • About the Code• ORM - Common Code Base – Hibernate – MySQL / Postgres – OpenJPA – MySQL / Postgres – MongoDB – Versant
    • Circuit Structure• Simulate CRUD on embedded graph of different Classes.• Class model - Complex Circuit: class Holder0 { String _name; List <Holder0> _children; Holder0[] _array; } class Holder1 extends Holder0 { int _i1; } class Holder2 extends Holder1 { int _i2 <indexed>; } ...class HolderN extends…..
    • Hibernate JPA Code• Write – Generates Holder’s to user spec depth ComplexHolder cp = new ComplexHolder( ); em.makePersistent(cp);• Read – Access root of graph and traverse cp = em.find( ComplexHolder.class, id ); cp.getChildren().touch();• Query String query = "from org.polepos.teams.hibernate.data.Holder2 where i2=" + currentInt; Iterator it = em.iterate(query);• Delete – Deletes Graph - cascading operation
    • ORM JPA Mapping XML Mapping File for each Persistent Class ( 11ea Files )• - <hibernate-mapping package="org.polepos.teams.hibernate.data" default-cascade="none" default-access="property" default-lazy="true" auto-import="true">• - <class name="ComplexHolder0" table="tComplexHolderNew0" polymorphism="implicit" mutable="true" dynamic-update="false" dynamic-insert="false" select-before-update="false" optimistic-lock="version">• <id name="id" column="fid" type="long" />• - <discriminator type="string" not-null="true" force="false" insert="true">• <column name="DISCRIMINATOR" />• </discriminator>• <property name="name" column="fname" type="string" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <list name="children" access="field" cascade="all" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">• <key column="parentId" on-delete="noaction" />• <index column="elementIndex" />• <many-to-many class="ComplexHolder0" embed-xml="true" not-found="exception" unique="false" />• </list>• - <array name="array" access="field" cascade="all" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">• <key column="parentId" on-delete="noaction" />• <index column="elementIndex" />• <many-to-many class="ComplexHolder0" embed-xml="true" not-found="exception" unique="false" />• </array>• - <subclass name="ComplexHolder1" discriminator-value="D" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i1" column="i1" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <subclass name="ComplexHolder2" discriminator-value="E" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i2" column="i2" type="int" index="i2_idx" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <subclass name="ComplexHolder3" discriminator-value="F" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i3" column="i3" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />• - <subclass name="ComplexHolder4" discriminator-value="G" dynamic-update="false" dynamic-insert="false" select-before-update="false">• <property name="i4" column="i4" type="int" unique="false" optimistic-lock="true" lazy="false" generated="never" />• </subclass>• </subclass>• </subclass>• </subclass>• </class>• </hibernate-mapping>
    • MongoDB Codeprivate void storeData(Mongo mongo) { collection(mongo).insert(new BasicDBObject("test", "object")); }public ComplexHolder0 convertFromDocument(DBObject data, DeserializationOptions deserializationOption) { ComplexHolder0 instance = createInstance(getAsString(data, TYPE_ATTRIBUTE)); instance.setName(getAsString(data, NAME_ATTRIBUTE)); if (null != data.get(CHILDREN)) { instance.setChildren(fromMongoObjectToList(getAsList(data, CHILDREN), deserializationOption)); } if (null != data.get(ARRAY)) { final List<ComplexHolder0> arrayContent = fromMongoObjectToList(getAsList(data, ARRAY), deserializationOption); instance.setArray(arrayContent.toArray(new ComplexHolder0[arrayContent.size()])); } readAttributes(data, instance); return instance; }
    • MongoDB Mappingprivate static void writeSubTypeAttributes(ComplexHolder0 holder, BasicDBObject dataStorage) { if (holder instanceof ComplexHolder1) { dataStorage.put(FIELD_I1, ((ComplexHolder1) holder)._i1); } if (holder instanceof ComplexHolder2) { dataStorage.put(FIELD_I2, ((ComplexHolder2) holder)._i2); }private static void readAttributes(DBObject dataStorage, ComplexHolder0 holder) { if (holder instanceof ComplexHolder1) { ((ComplexHolder1) holder)._i1 = (Integer) dataStorage.get(FIELD_I1); } if (holder instanceof ComplexHolder2) { ((ComplexHolder2) holder)._i2 = (Integer) dataStorage.get(FIELD_I2); }
    • MongoDB Mapping public SerialisationResult convertToDocument(ComplexHolder0 holder)new ArrayList<ComplexHolder0>(data.size()); List<ComplexHolder0> objects = { List<BasicDBObject> holder2Objects = new(Object o : data) { for ArrayList<BasicDBObject>();class Serialisation { BasicDBObject document = convertToDocument(holder, holder2Objects); } static final String TYPE_ATTRIBUTE = "_t"; objects.add(restoreDocumentOrReference(o, deserializationOption)); if (holder instanceof ComplexHolder4) { return new SerialisationResult(document, holder2Objects); } private static final String PACKAGE_NAME = ComplexHolder0.class.getPackage().getName(); } dataStorage.put(FIELD_I4, ((ComplexHolder4) holder)._i4); static final String NAME_ATTRIBUTE = "name"; return objects; } static final String CHILDREN = "children"; } } public ComplexHolder0 convertFromDocument(DBObject data) { static final String ARRAY = "array";return convertFromDocument(data,private ComplexHolder0 restoreDocumentOrReference(Object o, DeserializationOptions deserializationOption) { DeserializationOptions.FULL_DESERIALISATION); static final String REFERENCE_TO_ORIGINAL_DOCUMENT = "_refToOriginal"; } private static void readAttributes(DBObject dataStorage, if (holder } static final String FIELD_I1 = "_i1"; DBObject dbObj = (DBObject) o; if (holder instanceof ComplexHolder2) { static final String FIELD_I2 = "_i2"; if (null != dbObj.get(REFERENCE_TO_ORIGINAL_DOCUMENT)) { public ComplexHolder0 convertFromDocument(DBObject data, DeserializationOptions deserializationOption) { holder)._i2 = (Integer) dataStorage.get(FIELD_I2); ((ComplexHolder2) return deserializationOption.deserialize(this, dbObj); static final String FIELD_I3 = "_i3"; ComplexHolder0 instance = createInstance(getAsString(data, TYPE_ATTRIBUTE)); } static final String FIELD_I4 = "_i4"; } else { if (holder instanceof ComplexHolder3) { instance.setName(getAsString(data, NAME_ATTRIBUTE)); return convertFromDocument(dbObj, deserializationOption); holder)._i3 = (Integer) dataStorage.get(FIELD_I3); if (null != data.get(CHILDREN)) { ((ComplexHolder3) } } instance.setChildren(fromMongoObjectToList(getAsList(data, CHILDREN), deserializationOption)); } private final OneArgFunction<BasicDBObject, DBRef> refCreator; } if (holder instanceof ComplexHolder4) { if (null != data.get(ARRAY)) { ((ComplexHolder4) holder)._i4 = (Integer) dataStorage.get(FIELD_I4); private static List getAsList(DBObject data, String attributeName) { } final List<ComplexHolder0> arrayContent = fromMongoObjectToList(getAsList(data, ARRAY), deserializationOption); return (List) data.get(attributeName); } Serialisation(OneArgFunction<BasicDBObject, DBRef> refCreator) { instance.setArray(arrayContent.toArray(new ComplexHolder0[arrayContent.size()])); this.refCreator = refCreator; } } } readAttributes(data, instance); private List<Object> toMongoObject(List<ComplexHolder0> children,) { private static String getAsString(DBObject data, String attribute) {= new ArrayList<Object>(children.size()); List<Object> objects return instance; public static Serialisation create(OneArgFunction<BasicDBObject, DBRef> refCreator) { data.get(attribute); } return (String) for (ComplexHolder0 child : children) { if (null == refCreator) { } final BasicDBObject document = convertToDocument(child, throw new ArgumentNullException("requires a reference creator"); if (isDocumentOnlyReferenced(child)) { private BasicDBObject convertToDocument(ComplexHolder0 holder, List<BasicDBObject> referencedDocumentsCollector) { } private static ComplexHolder0 createInstance(String className) { referencedDocumentsCollector.add(document); BasicDBObject dbObject = createDocumentWithAttributesOnly(holder); try { return new Serialisation(refCreator); dbObject.put(CHILDREN, toMongoObject(holder.getChildren(), referencedDocumentsCollector)); DBObject copy = createDocumentWithAttributesOnly(child); } return (ComplexHolder0) Thread.currentThread().getContextClassLoader() copy.put(REFERENCE_TO_ORIGINAL_DOCUMENT, if (null != holder.getArray()) { .loadClass(PACKAGE_NAME + "." + className).newInstance(); objects.add(copy); dbObject.put(ARRAY, toMongoObject(asList(holder.getArray()), referencedDocumentsCollector)); } catch (Exception e) { public static OneArgFunction<BasicDBObject, DBRef> createReferenceCreator(final DBCollection collection) { } else { } throw rethrow(e); if (null == collection) { return dbObject; objects.add(document); throw new ArgumentNullException("requires a collection"); } } } } } } return new OneArgFunction<BasicDBObject, DBRef>() { private BasicDBObject createDocumentWithAttributesOnly(ComplexHolder0 holder) { return objects; holder, BasicDBObject dataStorage) { @Override private static void writeSubTypeAttributes(ComplexHolder0 } BasicDBObject dbObject = new BasicDBObject("_id", new ObjectId()); if (holder instanceof ComplexHolder1) { public DBRef invoke(BasicDBObject basicDBObject) { dbObject.put(NAME_ATTRIBUTE, holder.getName()); final DB db = collection.getDB(); dataStorage.put(FIELD_I1, ((ComplexHolder1) holder)._i1); isDocumentOnlyReferenced(ComplexHolder0 child) { private static boolean dbObject.put(TYPE_ATTRIBUTE, holder.getClass().getSimpleName()); } final Object id = basicDBObject.get("_id"); writeSubTypeAttributes(holder, dbObject); return child.getClass().equals(ComplexHolder2.class); if (null == id) { if (holder instanceof ComplexHolder2) {} return dbObject; dataStorage.put(FIELD_I2, ((ComplexHolder2) holder)._i2); throw new IllegalStateException("Expected an _id on the object"); } } } return new DBRef(db, collection.getName(), id); if (holder instanceof ComplexHolder3) { public List<ComplexHolder0> fromMongoObjectToList(List data, DeserializationOptions deserializationOption) { dataStorage.put(FIELD_I3, ((ComplexHolder3) holder)._i3); } }; } } if (holder instanceof ComplexHolder4) {
    • Versant JPA Code• Write – Generates Holder’s to user spec depth ComplexHolder cp = new ComplexHolder( ); em.makePersistent(cp);• Read – Access root of graph and traverse cp = em.find( ComplexHolder.class, id ); cp.getChildren().touch();• Query String query = "from org.polepos.teams.versant.data.Holder2 where i2=" + currentInt; Iterator it = session.iterate(query);• Delete – Deletes Graph – Cascading Operation
    • Versant JPA Mapping single XML File with primitive declaration entries<class name="ComplexHolder0"> <field name="name" /> <field name="array" /> <field name="children"> <collection element-type="ComplexHolder0"/> </field></class>
    • Sh@t’s and Grins• JDBC CodeJDBC StringBuilder sb = new StringBuilder(); sb.append("select " + HOLDER_TABLE0 + ".id from " + HOLDER_TABLE0); sb.append(" INNER JOIN " + HOLDER_TABLES[0]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[0] + ".id "); sb.append(" INNER JOIN " + HOLDER_TABLES[1]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[1] + ".id "); sb.append(" LEFT OUTER JOIN " + HOLDER_TABLES[2]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[2] + ".id "); sb.append(" LEFT OUTER JOIN " + HOLDER_TABLES[3]); sb.append(" on " + HOLDER_TABLE0 + ".id = " + HOLDER_TABLES[3] + ".id "); sb.append(" where " + HOLDER_TABLES[1] + ".i2 = ?"); PreparedStatement stat = prepareStatement(sb.toString()); ResultSet resultSet stat.executeQuery();
    • PolePositionBenchMark Summary Results
    • FlatObject - Single Thread
    • Results – ComplexConcurrency @mongo - no safe mode
    • Results – QueryConcurrency @mongo – no safe mode
    • Results – InsertConcurrent @mongo – no safe mode
    • Results – ComplexConcurrency @mongo – safe mode
    • Results – QueryConcurrency @mongo – safe mode
    • Results – InsertConcurrent @mongo – safe mode
    • Conclusions• Wake-up, smell the coffee – JOINs are not needed - unless adhoc analytics• Take care of business, dump a load of code – Serialization is bad, data useful once structured – Mapping is Mapping, even when it’s not an ORM• Get on your Bad Motor Scooter and Ride – NoSQL without Mapping rules the road Get the Code: http://www.polepos.org
    • Contact Robert GreeneVice President, Technology Versant Corporation rgreene@versant.com 650-232-2431