The InfoGrid Graph DataBase

4,660 views

Published on

Slides from Big Data Workshop in Mountain View, CA, April 2010

Published in: Technology

The InfoGrid Graph DataBase

  1. 1. The InfoGrid™ Graph Database Big Data Workshop April 2010 infogrid.org
  2. 2. InfoGrid Parts REST-ful GUI framework User-centric Identity “Viewlets” "OpenID" ig-ui ig-lid Semantic Schemas Mapping-in of External Data “Models” “Probes” ig-model-library ig-probe Graph Database Federation Protocol “MeshBase” “XPRISO” ig-graphdb ig-graphdb-grid ig-graphdb-grid Store Abstraction & Utilities Implementations ig-store ig-utils infogrid.org
  3. 3. InfoGrid Parts REST-ful GUI framework User-centric Identity “Viewlets” "OpenID" ig-ui ig-lid Semantic Schemas Mapping-in of External Data “Models” “Probes” ig-model-library ig-probe Graph Database Federation Protocol “MeshBase” “XPRISO” ig-graphdb ig-graphdb-grid ig-graphdb-grid Store Abstraction & Utilities Implementations ig-store ig-utils infogrid.org
  4. 4. What is a Graph Database? • Relational Database: ID First Last ID First Last • Tables, rows, columns 1 John Doe • SQL 2 Jane Doe • Hierarchical Database: • Nodes, children of nodes • XML • Key-Value Database: Key Value • Identifier maps to Blob 1234 <html><… • persistent java.util.Map 5678 0xd8ffe0… … • Graph Database: • Nodes, edges, perhaps blessed • Neo4j, InfoGrid, “web itself” infogrid.org
  5. 5. Graph Primitives Basic manipulation: Manipulation with properties & types: • create node • set property on node • delete node • get property on node a • create edge d • bless node with type(s) • delete edge • unbless node from type(s) b c • bless relationship with type(s) • unbless relationship from type(s) Basic traversal: Traversal with properties & types: • Given start node, get set of • Given start node, get set of neighbor nodes neighbor nodes related with type(s) • Subset by type or property value
  6. 6. Example 1 (untyped) MeshBase mb = …; MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject(); d MeshObject b = life.createMeshObject(); MeshObject c = life.createMeshObject(); MeshObject d = life.createMeshObject(); b c a.relate( b ); a.relate( c ); b.relate( d ); c.relate( d ); } finally { if( tx != null ) tx.commitTransaction(); } infogrid.org
  7. 7. Example 1 (untyped) MeshObject a = … MeshObjectSet neighbors = a.traverseToNeighbors(); a for( MeshObject n : neighbors ) { // finds b, c d // do something with n } b c MeshObjectSet oneMore = neighbors.traverseToNeighbors(); for( MeshObject n : oneMore ) { // finds a, d // do something with n } // etc. infogrid.org
  8. 8. Example 2 (typed: RSS & tagging) MeshBase mb = …; MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( RSSFEED ); d MeshObject b = life.createMeshObject( ITEM ); MeshObject c = life.createMeshObject( ITEM ); MeshObject d = life.createMeshObject( TAG ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relateAndBless( c, FEED_CONTAINS_ITEM.getSource() ); b.relateAndBless( d, TAG_TAGS_MESHOBJECT.getDestination() ); c.relateAndBless( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); }
  9. 9. Models (in InfoGrid) Model definition: Example: feeds • Define types for nodes • Feed, with subtypes RssFeed and ("EntityType") AtomFeed • Define types for edges, allowed • Feed has name, author, date last sources/dest. ("RelationshipType") published etc. • Define properties and their types • FeedItem ("PropertyType") • FeedItem has title, content, … • Define multiplicities Example: tags for relationships • Tag Run-time: • Tag has a name • Enforced at run-time • TagLibrary • Dynamic blessing and unblessing • Tag_Tags_MeshObject (allow any • Multiple types per node or edge MeshObject be tagged)
  10. 10. Example 3 (REST-ful identifiers) MeshBase mb = …; MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( "rssfeed" ); d MeshObject b = life.createMeshObject( "rssfeed/item1" ); MeshObject c = life.createMeshObject( "rssfeed/item2" ); MeshObject d = life.createMeshObject( "tags/cool" ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relate( c, FEED_CONTAINS_ITEM.getSource() ); b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); } *API slightly simplified for this example. And: yes, there is security. Just not on this slide. infogrid.org
  11. 11. Example 3 (REST-ful identifiers) MeshBase mb = …; Web-accessible as: MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; http://example.com/rssfeed try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( "rssfeed" ); d MeshObject b = life.createMeshObject( "rssfeed/item1" ); MeshObject c = life.createMeshObject( "rssfeed/item2" ); MeshObject d = life.createMeshObject( "tags/cool" ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relate( c, FEED_CONTAINS_ITEM.getSource() ); b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); } *API slightly simplified for this example. And: yes, there is security. Just not on this slide. infogrid.org
  12. 12. Example 3 (REST-ful identifiers) MeshBase mb = …; Web-accessible as: MeshObjectLifecycleManager life = mb.getMeshObjectLifecycleManager(); Transaction tx = null; http://example.com/rssfeed try { tx = mb.createTransactionAsap(); a MeshObject a = life.createMeshObject( "rssfeed" ); d MeshObject b = life.createMeshObject( "rssfeed/item1" ); MeshObject c = life.createMeshObject( "rssfeed/item2" ); MeshObject d = life.createMeshObject( "tags/cool" ); b c a.relateAndBless( b, FEED_CONTAINS_ITEM.getSource() ); a.relate( c, FEED_CONTAINS_ITEM.getSource() ); as: Web-accessible b.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); http://example.com/rssfeed/item2 c.relate( d, TAG_TAGS_MESHOBJECT.getDestination() ); } finally { if( tx != null ) tx.commitTransaction(); } *API slightly simplified for this example. And: yes, there is security. Just not on this slide. infogrid.org
  13. 13. Why a Graph Database? • Close to the way humans think • thing, type, property, relationship, … • Much more flexible than a relational database • relate anything to anything without schema changes • can define schemas/models as components, not monolithic • Often much faster (depends on application) • no joins • local operations • Higher developer productivity infogrid.org
  14. 14. This concludes: The InfoGrid™ Graph Database For more information: infogrid.org Twitter: @InfoGrid infogrid.org

×