The InfoGrid™
Graph Database

    Big Data Workshop

       April 2010




                        infogrid.org
InfoGrid Parts
REST-ful GUI framework                       User-centric Identity
      “Viewlets”                                  "OpenID"
                        ig-ui                                        ig-lid



  Semantic Schemas                   Mapping-in of External Data
      “Models”                               “Probes”
             ig-model-library                                   ig-probe



        Graph Database                            Federation Protocol
          “MeshBase”                                   “XPRISO”
                                ig-graphdb                ig-graphdb-grid


                                                          ig-graphdb-grid
  Store Abstraction &
                                                   Utilities
   Implementations
                     ig-store                                    ig-utils




                                                                              infogrid.org
InfoGrid Parts
REST-ful GUI framework                       User-centric Identity
      “Viewlets”                                  "OpenID"
                        ig-ui                                        ig-lid



  Semantic Schemas                   Mapping-in of External Data
      “Models”                               “Probes”
             ig-model-library                                   ig-probe



        Graph Database                            Federation Protocol
          “MeshBase”                                   “XPRISO”
                                ig-graphdb                ig-graphdb-grid


                                                          ig-graphdb-grid
  Store Abstraction &
                                                   Utilities
   Implementations
                     ig-store                                    ig-utils




                                                                              infogrid.org
What is a Graph Database?
• Relational Database:                 ID        First      Last
                                            ID     First      Last
     • Tables, rows, columns                1      John           Doe
     • SQL                                  2      Jane           Doe

• Hierarchical Database:
     • Nodes, children of nodes
     • XML
• Key-Value Database:
                                        Key               Value
     • Identifier maps to Blob           1234          <html><…
     • persistent java.util.Map         5678         0xd8ffe0…

 …
• Graph Database:
     • Nodes, edges, perhaps blessed
     • Neo4j, InfoGrid, “web itself”



                                                                        infogrid.org
Graph Primitives
Basic manipulation:                        Manipulation with properties & types:
  •   create   node                          •   set property on node
  •   delete   node                          •   get property on node
                             a
  •   create   edge                    d
                                             •   bless node with type(s)
  •   delete   edge                          •   unbless node from type(s)
                        b          c
                                             •   bless relationship with type(s)
                                             •   unbless relationship from type(s)


Basic traversal:                           Traversal with properties & types:
  • Given start node, get set of             • Given start node, get set of
    neighbor nodes                             neighbor nodes related with type(s)
                                             • Subset by type or property value
Example 1 (untyped)
MeshBase mb = …;
MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager();
Transaction tx = null;
try {
    tx = mb.createTransactionAsap();                                a

   MeshObject   a   =   life.createMeshObject();                                 d
   MeshObject   b   =   life.createMeshObject();
   MeshObject   c   =   life.createMeshObject();
   MeshObject   d   =   life.createMeshObject();           b            c

   a.relate(   b   );
   a.relate(   c   );
   b.relate(   d   );
   c.relate(   d   );

} finally {
    if( tx != null ) tx.commitTransaction();
}




                                                                            infogrid.org
Example 1 (untyped)
MeshObject a = …

MeshObjectSet neighbors = a.traverseToNeighbors();

                                                               a
for( MeshObject n : neighbors ) {
    // finds b, c
                                                                            d
    // do something with n
}

                                                           b       c
MeshObjectSet oneMore = neighbors.traverseToNeighbors();

for( MeshObject n : oneMore ) {
    // finds a, d
    // do something with n
}
// etc.




                                                                       infogrid.org
Example 2 (typed: RSS & tagging)
MeshBase mb = …;
MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager();
Transaction tx = null;
try {
    tx = mb.createTransactionAsap();                                a

   MeshObject   a   =   life.createMeshObject(   RSSFEED );                   d
   MeshObject   b   =   life.createMeshObject(   ITEM );
   MeshObject   c   =   life.createMeshObject(   ITEM );
   MeshObject   d   =   life.createMeshObject(   TAG );           b       c

   a.relateAndBless(       b,   FEED_CONTAINS_ITEM.getSource() );
   a.relateAndBless(       c,   FEED_CONTAINS_ITEM.getSource() );
   b.relateAndBless(       d,   TAG_TAGS_MESHOBJECT.getDestination() );
   c.relateAndBless(       d,   TAG_TAGS_MESHOBJECT.getDestination() );

} finally {
    if( tx != null ) tx.commitTransaction();
}
Models (in InfoGrid)
Model definition:                        Example: feeds
 • Define types for nodes                 • Feed, with subtypes RssFeed and
   ("EntityType")                          AtomFeed
 • Define types for edges, allowed        • Feed has name, author, date last
   sources/dest. ("RelationshipType")      published etc.
 • Define properties and their types      • FeedItem
   ("PropertyType")                      • FeedItem has title, content, …
 • Define multiplicities                 Example: tags
   for relationships
                                         •   Tag
Run-time:
                                         •   Tag has a name
 • Enforced at run-time                  •   TagLibrary
 • Dynamic blessing and unblessing       •   Tag_Tags_MeshObject (allow any
 • Multiple types per node or edge           MeshObject be tagged)
Example 3 (REST-ful identifiers)
MeshBase mb = …;
MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager();
Transaction tx = null;
try {
    tx = mb.createTransactionAsap();                                a

      MeshObject           a   =   life.createMeshObject(                       "rssfeed" );                   d
      MeshObject           b   =   life.createMeshObject(                       "rssfeed/item1" );
      MeshObject           c   =   life.createMeshObject(                       "rssfeed/item2" );
      MeshObject           d   =   life.createMeshObject(                       "tags/cool" );    b   c

      a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() );
      a.relate( c, FEED_CONTAINS_ITEM.getSource() );
      b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() );
      c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() );

} finally {
    if( tx != null ) tx.commitTransaction();
}
*API slightly simplified for this example. And: yes, there is security. Just not on this slide.




                                                                                                          infogrid.org
Example 3 (REST-ful identifiers)
MeshBase mb = …;                  Web-accessible as:
MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager();
Transaction tx = null;            http://example.com/rssfeed
try {
    tx = mb.createTransactionAsap();                                a

      MeshObject           a   =   life.createMeshObject(                       "rssfeed" );                   d
      MeshObject           b   =   life.createMeshObject(                       "rssfeed/item1" );
      MeshObject           c   =   life.createMeshObject(                       "rssfeed/item2" );
      MeshObject           d   =   life.createMeshObject(                       "tags/cool" );    b   c

      a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() );
      a.relate( c, FEED_CONTAINS_ITEM.getSource() );
      b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() );
      c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() );

} finally {
    if( tx != null ) tx.commitTransaction();
}
*API slightly simplified for this example. And: yes, there is security. Just not on this slide.




                                                                                                          infogrid.org
Example 3 (REST-ful identifiers)
MeshBase mb = …;                  Web-accessible as:
MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager();
Transaction tx = null;            http://example.com/rssfeed
try {
    tx = mb.createTransactionAsap();                                a

      MeshObject           a   =   life.createMeshObject(                       "rssfeed" );                   d
      MeshObject           b   =   life.createMeshObject(                       "rssfeed/item1" );
      MeshObject           c   =   life.createMeshObject(                       "rssfeed/item2" );
      MeshObject           d   =   life.createMeshObject(                       "tags/cool" );    b   c

      a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() );
      a.relate( c, FEED_CONTAINS_ITEM.getSource() ); as:
                                     Web-accessible
      b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() );
                                     http://example.com/rssfeed/item2
      c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() );

} finally {
    if( tx != null ) tx.commitTransaction();
}
*API slightly simplified for this example. And: yes, there is security. Just not on this slide.




                                                                                                          infogrid.org
Why a Graph Database?

• Close to the way humans think
 • thing, type, property, relationship, …
• Much more flexible than a relational database
 • relate anything to anything without schema changes
 • can define schemas/models as components, not monolithic
• Often much faster (depends on application)
 • no joins
 • local operations
• Higher developer productivity




                                                            infogrid.org
This concludes:



            The InfoGrid™
           Graph Database

                  For more information:
                      infogrid.org
                   Twitter: @InfoGrid




                                          infogrid.org

The InfoGrid Graph DataBase

  • 1.
    The InfoGrid™ Graph Database Big Data Workshop April 2010 infogrid.org
  • 2.
    InfoGrid Parts REST-ful GUIframework User-centric Identity “Viewlets” "OpenID" ig-ui ig-lid Semantic Schemas Mapping-in of External Data “Models” “Probes” ig-model-library ig-probe Graph Database Federation Protocol “MeshBase” “XPRISO” ig-graphdb ig-graphdb-grid ig-graphdb-grid Store Abstraction & Utilities Implementations ig-store ig-utils infogrid.org
  • 3.
    InfoGrid Parts REST-ful GUIframework User-centric Identity “Viewlets” "OpenID" ig-ui ig-lid Semantic Schemas Mapping-in of External Data “Models” “Probes” ig-model-library ig-probe Graph Database Federation Protocol “MeshBase” “XPRISO” ig-graphdb ig-graphdb-grid ig-graphdb-grid Store Abstraction & Utilities Implementations ig-store ig-utils infogrid.org
  • 4.
    What is aGraph Database? • Relational Database: ID First Last ID First Last • Tables, rows, columns 1 John Doe • SQL 2 Jane Doe • Hierarchical Database: • Nodes, children of nodes • XML • Key-Value Database: Key Value • Identifier maps to Blob 1234 <html><… • persistent java.util.Map 5678 0xd8ffe0… … • Graph Database: • Nodes, edges, perhaps blessed • Neo4j, InfoGrid, “web itself” infogrid.org
  • 5.
    Graph Primitives Basic manipulation: Manipulation with properties & types: • create node • set property on node • delete node • get property on node a • create edge d • bless node with type(s) • delete edge • unbless node from type(s) b c • bless relationship with type(s) • unbless relationship from type(s) Basic traversal: Traversal with properties & types: • Given start node, get set of • Given start node, get set of neighbor nodes neighbor nodes related with type(s) • Subset by type or property value
  • 6.
    Example 1 (untyped) MeshBasemb = …; MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject(); d MeshObject b = life.createMeshObject(); MeshObject c = life.createMeshObject(); MeshObject d = life.createMeshObject(); b c a.relate( b ); a.relate( c ); b.relate( d ); c.relate( d ); } finally { if( tx != null ) tx.commitTransaction(); } infogrid.org
  • 7.
    Example 1 (untyped) MeshObjecta = … MeshObjectSet neighbors = a.traverseToNeighbors(); a for( MeshObject n : neighbors ) { // finds b, c d // do something with n } b c MeshObjectSet oneMore = neighbors.traverseToNeighbors(); for( MeshObject n : oneMore ) { // finds a, d // do something with n } // etc. infogrid.org
  • 8.
    Example 2 (typed:RSS & tagging) MeshBase mb = …; MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( RSSFEED ); d MeshObject b = life.createMeshObject( ITEM ); MeshObject c = life.createMeshObject( ITEM ); MeshObject d = life.createMeshObject( TAG ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relateAndBless( c, FEED_CONTAINS_ITEM.getSource() ); b.relateAndBless( d, TAG_TAGS_MESHOBJECT.getDestination() ); c.relateAndBless( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); }
  • 9.
    Models (in InfoGrid) Modeldefinition: Example: feeds • Define types for nodes • Feed, with subtypes RssFeed and ("EntityType") AtomFeed • Define types for edges, allowed • Feed has name, author, date last sources/dest. ("RelationshipType") published etc. • Define properties and their types • FeedItem ("PropertyType") • FeedItem has title, content, … • Define multiplicities Example: tags for relationships • Tag Run-time: • Tag has a name • Enforced at run-time • TagLibrary • Dynamic blessing and unblessing • Tag_Tags_MeshObject (allow any • Multiple types per node or edge MeshObject be tagged)
  • 10.
    Example 3 (REST-fulidentifiers) MeshBase mb = …; MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( "rssfeed" ); d MeshObject b = life.createMeshObject( "rssfeed/item1" ); MeshObject c = life.createMeshObject( "rssfeed/item2" ); MeshObject d = life.createMeshObject( "tags/cool" ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relate( c, FEED_CONTAINS_ITEM.getSource() ); b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); } *API slightly simplified for this example. And: yes, there is security. Just not on this slide. infogrid.org
  • 11.
    Example 3 (REST-fulidentifiers) MeshBase mb = …; Web-accessible as: MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; http://example.com/rssfeed try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( "rssfeed" ); d MeshObject b = life.createMeshObject( "rssfeed/item1" ); MeshObject c = life.createMeshObject( "rssfeed/item2" ); MeshObject d = life.createMeshObject( "tags/cool" ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relate( c, FEED_CONTAINS_ITEM.getSource() ); b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); } *API slightly simplified for this example. And: yes, there is security. Just not on this slide. infogrid.org
  • 12.
    Example 3 (REST-fulidentifiers) MeshBase mb = …; Web-accessible as: MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; http://example.com/rssfeed try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( "rssfeed" ); d MeshObject b = life.createMeshObject( "rssfeed/item1" ); MeshObject c = life.createMeshObject( "rssfeed/item2" ); MeshObject d = life.createMeshObject( "tags/cool" ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relate( c, FEED_CONTAINS_ITEM.getSource() ); as: Web-accessible b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); http://example.com/rssfeed/item2 c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); } *API slightly simplified for this example. And: yes, there is security. Just not on this slide. infogrid.org
  • 13.
    Why a GraphDatabase? • Close to the way humans think • thing, type, property, relationship, … • Much more flexible than a relational database • relate anything to anything without schema changes • can define schemas/models as components, not monolithic • Often much faster (depends on application) • no joins • local operations • Higher developer productivity infogrid.org
  • 14.
    This concludes: The InfoGrid™ Graph Database For more information: infogrid.org Twitter: @InfoGrid infogrid.org