Dexjava Technical Seminar Dec 2011

11,556 views
11,531 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
11,556
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dexjava Technical Seminar Dec 2011

  1. 1. Technical Seminar December, 2011 Sergio GómezDex Graph Database http://www.sparsity-technologies.com
  2. 2. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  3. 3. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  4. 4. Introduction Graph database  Graph databases focus on the structure of the model.  Nodes and edges instead of tables.  Relationships are first-class citizens.  Explicit in the model.  DEX is a programming library which allows to manage aDex Graph Database graph database.  Very large datasets.  High performance query processing. http://www.sparsity-technologies.com
  5. 5. Introduction Dex Definition  Persistent and temporary graph management programming library.  Data model: Typed and attributed directed multigraph.  Typed: Node and edge instances belong to a type (label).  Attributed: Node and edge instances may have attribute values.  Directed: Edge can be directed or undirected.Dex Graph Database  Multigraph: Multiple edges between two nodes. http://www.sparsity-technologies.com
  6. 6. Introduction Graph ModelDex Graph Database http://www.sparsity-technologies.com
  7. 7. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  8. 8. Basic Concepts  Java library  public API  Private native dynamic library  Automatically loaded  System requirements:  Java Runtime Environment, v1.5 or higher.Dex Graph Database  Operative system:  Windows, MacOSX, Linux  32 and 64 bits http://www.sparsity-technologies.com
  9. 9. Basic Concepts Dexjava Class Diagram Dex 1 Database N Session 1 Graph N 1 1 1 Persistent DBDex Graph Database N Objects Set of OIDs http://www.sparsity-technologies.com
  10. 10. Basic Concepts Main methods Database Dex newSession()  Session open(filename)  Database create(filename)  Database Session close() getGraph()  Graph close() Objects add(long) Graph exists(long) newNodeType(name)  int copy(objs) newEdgeType(name)  int union(objs) newNode(type)  long newEdge(type)  longDex Graph Database intersection(objs) difference(objs) newAttribute(type, name)  int setAttribute(oid, attr, value) ObjectsIterator getAttribute(oid, attr)  value hasNext()  boolean select(type)  Objects next()  long select(attr, op, value)  Objects explode(oid, type)  Objects neigbors(oid, type)  Objects http://www.sparsity-technologies.com
  11. 11. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  12. 12. Database construction  Graph  Dex: Loads library and manages graph db instances.  Database: Manages a graph db instance.  Session: Manages queries and temporary data.  Nodes & Edges  Type:  Dex identifier (integer)  Public identifier (string)  Instance:  DEX identifier (long) – OID  belongs to a typeDex Graph Database  Attributes  Attribute:  DEX identifier (int)  public identifier (string)  Scope: type or global  Temporary (per Session) or persistent http://www.sparsity-technologies.com
  13. 13. Database construction Create a graph database Database Dex#create(String path, String alias) Creates a new graph database instance. Returns the Database instance to manage a new persistent graph. Database Dex#open(String path, bool read) Opens an existing graph database instance. Read-only mode. Returns the Database instance to manage the persistent graph.Dex Graph Database Session Database#newSession() Initiates a new user Session. Graph Session#getGraph() Gets the Graph instance which represents the graph data. http://www.sparsity-technologies.com
  14. 14. Database construction Create a graph database example import com.sparsity.dex.gdb.*; … Dex dex = new Dex(new DexConfig()); Database db = dex.create(“C:/image.dex”, “graphdb”); Session s = db.newSession(); … … s.close(); db.close();Dex Graph Database dex.close(); http://www.sparsity-technologies.com
  15. 15. Database construction Add nodes int Graph#newNodeType(String name) Creates a new node type with the given unique name. Returns the Dex node type identifier. long Graph#newNode(int nodeType) Creates a new node belonging to the given node type. Returns the Dex object identifier.Dex Graph Database http://www.sparsity-technologies.com
  16. 16. Database construction Add edges int Graph#newEdgeType(String name, bool directed, bool neighbors) Creates a new edge type with the given unique name. Directed or undirected edge type. Create neighbor-index or not. Returns the Dex edge type identifier. int Graph#newRestrictedEdgeType(String name, int srcNodeType, int dstNodeType, bool neighbors) Creates a new directed edge type with the given unique name.Dex Graph Database (Integrity restriction) Source and destination of the edge instances are restricted to the given node types. Create neighbor-index or not. Returns the Dex edge type identifier. http://www.sparsity-technologies.com
  17. 17. Database construction Add edges int Graph#newEdge(int edgeType, long tail, long head) Creates a new edge belonging to the given edge type. Tail is the source and head is the target. Returns the Dex edge identifier.Dex Graph Database http://www.sparsity-technologies.com
  18. 18. Database construction Add nodes and edges example … Graph g = s.getGraph(); p1 int person = g.newNodeType(“PERSON”); p2 long p1 = g.newNode(person); long p2 = g.newNode(person); p3 long p3 = g.newNode(person); int friend = g.newEdgeType(“FRIEND”, false, false); long e1 = g.newEdge(friend, p1, p2);Dex Graph Database long e2 = g.newEdge(friend, p2, p3); p1 int loves = g.newEdgeType(“LOVES”, p2 true, false); long e3 = g.newEdge(loves, p1, p3); p3 … http://www.sparsity-technologies.com
  19. 19. Database construction Manage attributes class Value Encapsulates a value and its domain (data type). Use them to set and get attribute values for the objects. int Graph#newAttribute(int type, String name, DataType dt, AttributeKind kind) Creates a new attribute with the given unique name for the given node or edge type. Returns the Dex attribute identifier. “dt” can be:Dex Graph Database Boolean, Integer, Long, Double, String, Text, Timestamp, OID. “kind” can be: Basic: Just set and get operations are allowed. Indexed: Select operations are allowed as well as set and get operations. Unique: As indexed. Unique integrity restriction: no two objects with the same value for the attribute but NULL. http://www.sparsity-technologies.com
  20. 20. Database construction Manage attributes Graph#setAttribute(long oid, int attr, Value v) Sets the given Value for the given attribute to the given object identifier. Given attribute must be defined for the object‟s type. Value „s data type must match attribute‟s data type or NULL. Graph#getAttribute(long oid, int attr, Value v) Gets the Value for the given attribute and for the given object identifier. Given attribute identifier must be defined for the object‟sDex Graph Database type. http://www.sparsity-technologies.com
  21. 21. Database construction Manage attributes example … int name = g.newAttribute(person, “NAME”, String, Unique); int age = g.newAttribute(person, “AGE”, Integer, Indexed); JOHN Value v = new Value(); 18 KELLY g.setAttribute(p1, name, v.setString(“JOHN”)); g.setAttribute(p1, age, v.setInteger(18)); MARY g.setAttribute(p2, name, v.setString(“KELLY")); g.setAttribute(p3, name, v.setString(“MARY")); …Dex Graph Database int since = g.newAttribute(friend, “SINCE”, JOHN 2000 Integer, Indexed); 18 g.setAttribute(e1, since, v.setInt(2000)); KELLY g.setAttribute(e2, since, v.setInt(1995)); … MARY 1995 http://www.sparsity-technologies.com
  22. 22. Database construction Manage attributes example … int phones = g.newEdgeType("phones“, true, true); int when = g.newAttribute(phones, "when", String, Indexed); long e4 = g.newEdge(phones, p1, p3); g.setAttribute(e4, when, v.setString("4pm"))); JOHN 2000 long e5 = g.newEdge(phones, p1, p3); 18 g.setAttribute(e5, when, v.setString("5pm")); KELLY 4pm 5pm long e6 = g.newEdge(phones, p3, p2); g.setAttribute(e6, when, v.setString("6pm")); MARY 1995 …Dex Graph Database g.getAttribute(p1, name, v); 6pm System.out.println(v); g.getAttribute(e5, when, v); System.out.println(v); g.getAttribute(e4, when, v); System.out.println(v); http://www.sparsity-technologies.com
  23. 23. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  24. 24. Query database Manage node and edge types int Graph#findType(String name) Returns the Dex type identifier for the given type name. Type Graph#getType(int type) Returns the metadata for the given Dex type identifier. TypeList Graph#findTypes() Returns the list of all existing Dex type identifiers.Dex Graph Database http://www.sparsity-technologies.com
  25. 25. Query database Manage attributes int Graph#findAttribute(int type, String name) Returns the Dex attribute identifier for the given Dex type identifier and attribute name. Attribute Graph#getAttribute(int attr) Returns the metadata for the given Dex attribute identifier. AttributeList Graph#findAttributes(int type) Returns the list of all existing Dex attribute identifiers for the given Dex type identifier.Dex Graph Database http://www.sparsity-technologies.com
  26. 26. Query database Objects class Objects Unordered set of OIDs for large collections. Implements Set<Long>, Iterable<Long>. boolean Objects#add(long oid) Adds the given OID to the collection. Returns true if added, false if the OID was already into the collection. boolean Objects#exists(long oid) Returns true if the given OID exists into the collection, falseDex Graph Database otherwise. boolean Objects#remove(long oid) Removes the given OID from the collection. Returns true if removed or false if the OID was not into the collection. http://www.sparsity-technologies.com
  27. 27. Query database Objects long Objects#union(Objects objs) this = this UNION objs Returns the new size of the collection. long Objects#intersection(Objects objs) this = this INTERSECTION objs Returns the new size of the collection. long Objects#difference(Objects objs) this = this DIFFERENCE objs Returns the new size of the collection.Dex Graph Database http://www.sparsity-technologies.com
  28. 28. Query database Retrieve data Objects Graph#select(int t) Retrieves object identifiers belonging to the given node or edge type. Objects Graph#select(int attr, Condition c, Value v) Retrieves object identifiers which satisfy the condition for the given Value. “c” can be: Equal, NotEqual, GreaterEqual, GreaterThan, LessEqual, LessThan, Between. Also, for String attributes: Like, LikeNoCase, RegExp.Dex Graph Database long Graph#findObject(int attr, Value v) Randomly retrieves an object identifier which has the given value for the given attribute (or Objects.InvalidOID if not found). Useful for unique attributes. http://www.sparsity-technologies.com
  29. 29. Query database Navigation Objects Graph#explode(long oid, int edgeType, EdgesDirection dir) Retrieves out-going or in-going edges (or both) from or to the given node identifier and for the given edge type. “dir” can be: Ingoing, Outgoing, Any. Objects Graph#neighbors(long oid, int edgeType, EdgesDirection dir) Retrieves neighbor nodes to the given node identifier whichDex Graph Database can be reached through the given edge type and direction. “dir” can be: Ingoing, Outgoing, Any. http://www.sparsity-technologies.com
  30. 30. Query database Retrieve data example … Graph g = s.getGraph(); Objects persons = g.select(person); ObjectsIterator it = persons.iterator(); while (it.hasNext()) { long p = it.next(); dbg.getAttribute(p, name, v); String name = v.toString(); } it.close(); JOHN persons.close(); 18 … 2000 5pm KELLY KELLYDex Graph Database 4pm 1995 MARY 6pm http://www.sparsity-technologies.com
  31. 31. Query database Navigation & Objects operations example … Objects objs1 = g.select(when, GreaterThan, “5pm”); // objs1 = { e5, e6 } Objects objs2 = g.explode(p1, phones, Outgoing); // objs2 = { e4, e5 } objs1.intersection(objs2); // objs1 = { e5, e6 } ∩ { e4, e5 } = { e5 } … objs1.close(); objs2.close(); JOHN 18 … 2000Dex Graph Database 5pm KELLY KELLY 4pm 1995 MARY 6pm http://www.sparsity-technologies.com
  32. 32. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  33. 33. Loaders  Package com.sparsity.dex.io  NodeTypeLoader  Requires a RowReader  CSVReader  … or write your own implementation.  Creates a node instance for each row and sets its attributes with the values within each column.  EdgeTypeLoader  Requires a RowReader, too.Dex Graph Database  Creates an edge instance for each row and sets its attributes with the values within each column.  Two special columns to identify source and target nodes for the edge. http://www.sparsity-technologies.com
  34. 34. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  35. 35. Script loaders Schema definition (CREATE|USE) GDB alias INTO „filename„ CREATE NODE node_type_name "(“ [attribute_name (INTEGER|LONG|DOUBLE|STRING|BOOLEAN|TIMESTAMP|TEXT) [INDEXED|UNIQUE|BASIC] , ...] ")“ CREATE [UNDIRECTED] EDGE edge_type_name [FROM node_type_name TO node_type_name] "(“ [attribute_nameDex Graph Database (INTEGER|LONG|DOUBLE|STRING|BOOLEAN|TIMESTAMP|TEXT) [INDEXED|UNIQUE|BASIC] , ...] ") [MATERIALIZE NEIGHBORS]" http://www.sparsity-technologies.com
  36. 36. Script loaders Load nodes LOAD NODES „file_name‟ [LOCALE loc] COLUMNS attribute_name [alias_name], … INTO node_type_name [IGNORE (attribute_name|alias_name), …] [FIELDS [TERMINATED char] [ENCLOSED char] [ALLOW_MULTILINE [max]]] [FROM num] [MAX num] [MODE (ROWS|COLUMNS [SPLIT [PARTITIONS num]])]Dex Graph Database http://www.sparsity-technologies.com
  37. 37. Script loaders Load edges LOAD EDGES „file_name‟ [LOCALE loc] COLUMNS attribute_name [alias_name], … INTO node_type_name [IGNORE (attribute_name|alias_name), …] WHERE TAIL (attribute_name|alias_name) = node_type_name.attribute_name HEAD (attribute_name|alias_name) = node_type_name.attribute_name [FIELDS [TERMINATED char] [ENCLOSED char] [ALLOW_MULTILINE [max]]]Dex Graph Database [FROM num] [MAX num] [MODE (ROWS|COLUMNS [SPLIT [PARTITIONS num]])] http://www.sparsity-technologies.com
  38. 38. Script loaders Examples create gdb WIKIPEDIA into wikipedia.dex create node TITLES ( ID int unique, TEXT string, NLC string, TITLE string indexed ) create node IMAGES ( ID int unique, NLC string, FILENAME string indexed )Dex Graph Database create edge REFS from TITLES to TITLES ( NLC string, "TEXT" string, TYPE string ) materialize neighbors create undirected edge IMGS http://www.sparsity-technologies.com
  39. 39. Script loaders Examples use gdb WIKIPEDIA into wikipedia.dex load nodes images.csv columns ID, NLC, FILENAME into IMAGES from 2 max 10000 load edges references.csv columns NLC, TEXT, TYPE, FROM F, TO T into REFS ignore F, T where tail F = TITLES.ID head T = TITLES.IDDex Graph Database fields terminated „|‟ enclosed „”‟ allow_multiline mode columns split partitions 3 http://www.sparsity-technologies.com
  40. 40. Index  Introduction  Basic Concepts  Database construction  Query database  Loaders  Script loadersDex Graph Database  Tips & tricks http://www.sparsity-technologies.com
  41. 41. Tips & tricks  Index or not?  Attributes:  Attributes used at select operations must be indexed.  Optionally, index once the attribute has been created/loaded.  Neighbors:Dex Graph Database  It is recommended to index those edge types used at neighbors operations. http://www.sparsity-technologies.com
  42. 42. Tips & tricks  String attributes  String  Maximum length = 2047.  Indexed or not.  Select [==, !=, >, >=, <, <=, Like, LikeNoCase, RegExp]  Text (Character large object)  Unlimited length.Dex Graph Database  Not Indexed.  Just get and set.  Streaming read and write operations. http://www.sparsity-technologies.com
  43. 43. Tips & tricks  Others:  DB cross-platform format.  32 – 64 bits, OS independent.  Just take into account platform endianness.  Read only mode.  Configuration:  com.sparsity.dex.gdb.DexConfigDex Graph Database  Set the maximum memory usage.  0 means unlimited.  License.  No license means evaluation version. http://www.sparsity-technologies.com
  44. 44. Thanks for your attention Any questions?Dex Graph Database SPARSITY-TECHNOLOGIES Jordi Girona, 1-3, Edifici K2M 08034 Barcelona info@sparsity-technologies.com http://www.sparsity-technologies.com http://www.sparsity-technologies.com

×