Dexjava Technical Seminar Dec 2011

Technical Seminar
December, 2011
Sergio Gómez
Dex Graph Database

http://www.sparsity-technologies.com

Index
 Introduction

 Basic Concepts

 Database construction

 Query database

 Loaders

 Script loaders
Dex Graph Database

 Tips & tricks


Introduction

Graph database

 Graph databases focus on the structure of the model.
 Nodes and edges instead of tables.

 Relationships are first-class citizens.

 Explicit in the model.

 DEX is a programming library
which allows to manage a
Dex Graph Database

graph database.
 Very large datasets.

 High performance

query processing.


Introduction
Dex Definition
 Persistent and temporary graph management
programming library.

 Data model:
Typed and attributed directed multigraph.
 Typed: Node and edge instances belong to a type
(label).
 Attributed: Node and edge instances may have attribute
values.
 Directed: Edge can be directed or undirected.
Dex Graph Database

 Multigraph: Multiple edges between two nodes.


Introduction
Graph Model
Dex Graph Database


Basic Concepts
 Java library  public API
 Private native
dynamic library
 Automatically
loaded

 System requirements:
 Java Runtime Environment, v1.5 or higher.
Dex Graph Database

 Operative system:
 Windows, MacOSX, Linux
 32 and 64 bits


Basic Concepts
Dexjava Class Diagram

Dex 1 Database N Session 1 Graph
N 1 1

1

Persistent DB
Dex Graph Database

N

Objects
Set of OIDs


Basic Concepts
Main methods
Database
Dex newSession()  Session
open(filename)  Database
create(filename)  Database Session
close()
getGraph()  Graph
close()
Objects
add(long) Graph
exists(long)
newNodeType(name)  int
copy(objs) newEdgeType(name)  int
union(objs) newNode(type)  long
newEdge(type)  long
Dex Graph Database

intersection(objs)
difference(objs) newAttribute(type, name)  int
setAttribute(oid, attr, value)
ObjectsIterator getAttribute(oid, attr)  value

hasNext()  boolean select(type)  Objects
next()  long select(attr, op, value)  Objects
explode(oid, type)  Objects
neigbors(oid, type)  Objects

Database construction
 Graph
 Dex: Loads library and manages graph db instances.
 Database: Manages a graph db instance.
 Session: Manages queries and temporary data.

 Nodes & Edges
 Type:
 Dex identifier (integer)
 Public identifier (string)
 Instance:
 DEX identifier (long) – OID
 belongs to a type
Dex Graph Database

 Attributes
 Attribute:
 DEX identifier (int)
 public identifier (string)
 Scope: type or global
 Temporary (per Session) or persistent


Create a graph database
Database Dex#create(String path, String alias)
Creates a new graph database instance.
Returns the Database instance to manage a new
persistent graph.

Database Dex#open(String path, bool read)
Opens an existing graph database instance.
Read-only mode.
Returns the Database instance to manage the persistent
graph.
Dex Graph Database

Session Database#newSession()
Initiates a new user Session.

Graph Session#getGraph()
Gets the Graph instance which represents the graph data.


Create a graph database example

import com.sparsity.dex.gdb.*;
…
Dex dex = new Dex(new DexConfig());
Database db = dex.create(“C:/image.dex”, “graphdb”);
Session s = db.newSession();
…
…
s.close();
db.close();
Dex Graph Database

dex.close();


Add nodes

int Graph#newNodeType(String name)
Creates a new node type with the given unique name.
Returns the Dex node type identifier.

long Graph#newNode(int nodeType)
Creates a new node belonging to the given node type.
Returns the Dex object identifier.
Dex Graph Database


Add edges
int Graph#newEdgeType(String name, bool directed,
bool neighbors)
Creates a new edge type with the given unique name.
Directed or undirected edge type.
Create neighbor-index or not.
Returns the Dex edge type identifier.

int Graph#newRestrictedEdgeType(String name, int
srcNodeType, int dstNodeType, bool neighbors)
Creates a new directed edge type with the given unique
name.
Dex Graph Database

(Integrity restriction) Source and destination of the edge
instances are restricted to the given node types.
Create neighbor-index or not.
Returns the Dex edge type identifier.


Add edges

int Graph#newEdge(int edgeType, long tail, long head)
Creates a new edge belonging to the given edge type.
Tail is the source and head is the target.
Returns the Dex edge identifier.
Dex Graph Database


Add nodes and edges example

…
Graph g = s.getGraph();
p1
int person = g.newNodeType(“PERSON”);
p2
long p1 = g.newNode(person);
p3

int friend = g.newEdgeType(“FRIEND”,
false, false);
long e1 = g.newEdge(friend, p1, p2);
Dex Graph Database

long e2 = g.newEdge(friend, p2, p3);
p1

int loves = g.newEdgeType(“LOVES”, p2
true, false);
long e3 = g.newEdge(loves, p1, p3);
p3
…


Manage attributes

class Value
Encapsulates a value and its domain (data type).
Use them to set and get attribute values for the objects.

int Graph#newAttribute(int type, String name,
DataType dt, AttributeKind kind)
Creates a new attribute with the given unique name for the
given node or edge type.
Returns the Dex attribute identifier.

“dt” can be:
Dex Graph Database

Boolean, Integer, Long, Double, String, Text, Timestamp, OID.
“kind” can be:
Basic: Just set and get operations are allowed.
Indexed: Select operations are allowed as well as set and get operations.
Unique: As indexed. Unique integrity restriction: no two objects with the
same value for the attribute but NULL.


Manage attributes

Graph#setAttribute(long oid, int attr, Value v)
Sets the given Value for the given attribute to the given
object identifier.
Given attribute must be defined for the object‟s type.
Value „s data type must match attribute‟s data type or NULL.

Graph#getAttribute(long oid, int attr, Value v)
Gets the Value for the given attribute and for the given
object identifier.
Given attribute identifier must be defined for the object‟s
Dex Graph Database

type.


Manage attributes example
…
int name = g.newAttribute(person, “NAME”,
String, Unique);
int age = g.newAttribute(person, “AGE”,
Integer, Indexed); JOHN
Value v = new Value(); 18
KELLY

g.setAttribute(p1, name, v.setString(“JOHN”));
g.setAttribute(p1, age, v.setInteger(18)); MARY
g.setAttribute(p2, name, v.setString(“KELLY"));
g.setAttribute(p3, name, v.setString(“MARY"));
…
Dex Graph Database

int since = g.newAttribute(friend, “SINCE”, JOHN 2000
Integer, Indexed); 18
g.setAttribute(e1, since, v.setInt(2000)); KELLY
g.setAttribute(e2, since, v.setInt(1995));
…
MARY 1995


Manage attributes example
…
int phones = g.newEdgeType("phones“, true, true);
int when = g.newAttribute(phones, "when",
String, Indexed);

long e4 = g.newEdge(phones, p1, p3);
g.setAttribute(e4, when, v.setString("4pm")));
JOHN 2000
long e5 = g.newEdge(phones, p1, p3); 18
g.setAttribute(e5, when, v.setString("5pm")); KELLY
4pm
5pm
long e6 = g.newEdge(phones, p3, p2);
g.setAttribute(e6, when, v.setString("6pm")); MARY 1995
…
Dex Graph Database

g.getAttribute(p1, name, v); 6pm
System.out.println(v);
g.getAttribute(e5, when, v);
g.getAttribute(e4, when, v);


Query database
Manage node and edge types

int Graph#findType(String name)
Returns the Dex type identifier for the given type name.

Type Graph#getType(int type)
Returns the metadata for the given Dex type identifier.

TypeList Graph#findTypes()
Returns the list of all existing Dex type identifiers.
Dex Graph Database


Query database
Manage attributes

int Graph#findAttribute(int type, String name)
Returns the Dex attribute identifier for the given Dex type
identifier and attribute name.

Attribute Graph#getAttribute(int attr)
Returns the metadata for the given Dex attribute identifier.

AttributeList Graph#findAttributes(int type)
Returns the list of all existing Dex attribute identifiers for the
given Dex type identifier.
Dex Graph Database


Query database
Objects
class Objects
Unordered set of OIDs for large collections.
Implements Set<Long>, Iterable<Long>.

boolean Objects#add(long oid)
Adds the given OID to the collection.
Returns true if added, false if the OID was already into the
collection.

boolean Objects#exists(long oid)
Returns true if the given OID exists into the collection, false
Dex Graph Database

otherwise.

boolean Objects#remove(long oid)
Removes the given OID from the collection.
Returns true if removed or false if the OID was not into the
collection.


Query database
Objects

long Objects#union(Objects objs)
this = this UNION objs
Returns the new size of the collection.

long Objects#intersection(Objects objs)
this = this INTERSECTION objs

long Objects#difference(Objects objs)
this = this DIFFERENCE objs
Dex Graph Database


Query database
Retrieve data
Objects Graph#select(int t)
Retrieves object identifiers belonging to the given node or
edge type.

Objects Graph#select(int attr, Condition c, Value v)
Retrieves object identifiers which satisfy the condition for the
given Value.
“c” can be:
Equal, NotEqual, GreaterEqual, GreaterThan, LessEqual,
LessThan, Between.
Also, for String attributes: Like, LikeNoCase, RegExp.
Dex Graph Database

long Graph#findObject(int attr, Value v)
Randomly retrieves an object identifier which has the given
value for the given attribute (or Objects.InvalidOID if not
found).
Useful for unique attributes.


Query database
Navigation

Objects Graph#explode(long oid, int edgeType,
EdgesDirection dir)
Retrieves out-going or in-going edges (or both) from or to
the given node identifier and for the given edge type.
“dir” can be:
Ingoing, Outgoing, Any.

Objects Graph#neighbors(long oid, int edgeType,
EdgesDirection dir)
Retrieves neighbor nodes to the given node identifier which
Dex Graph Database

can be reached through the given edge type and direction.
“dir” can be:
Ingoing, Outgoing, Any.


Query database
Retrieve data example
…
Graph g = s.getGraph();
Objects persons = g.select(person);
ObjectsIterator it = persons.iterator();
while (it.hasNext()) {
long p = it.next();
dbg.getAttribute(p, name, v);
String name = v.toString();
}
it.close(); JOHN
persons.close(); 18
… 2000
5pm
KELLY
KELLY
Dex Graph Database

4pm

1995
MARY
6pm


Query database
Navigation & Objects operations example
…
Objects objs1 = g.select(when, GreaterThan, “5pm”);
// objs1 = { e5, e6 }
Objects objs2 = g.explode(p1, phones, Outgoing);
// objs2 = { e4, e5 }
objs1.intersection(objs2);
// objs1 = { e5, e6 } ∩ { e4, e5 } = { e5 }
…
objs1.close();
objs2.close(); JOHN
18
…
2000
Dex Graph Database

5pm
KELLY
KELLY

4pm
1995
MARY

6pm


Loaders
 Package com.sparsity.dex.io

 NodeTypeLoader
 Requires a RowReader
 CSVReader
 … or write your own implementation.
 Creates a node instance for each row and sets its
attributes with the values within each column.

 EdgeTypeLoader
 Requires a RowReader, too.
Dex Graph Database

 Creates an edge instance for each row and sets its
attributes with the values within each column.
 Two special columns to identify source and target
nodes for the edge.


Script loaders
Load nodes
LOAD NODES „file_name‟ [LOCALE loc]
COLUMNS attribute_name [alias_name], …
INTO node_type_name
[IGNORE (attribute_name|alias_name), …]
[FIELDS
[TERMINATED char]
[ENCLOSED char]
[ALLOW_MULTILINE [max]]]
[FROM num]
[MAX num]
[MODE (ROWS|COLUMNS [SPLIT [PARTITIONS num]])]
Dex Graph Database


Script loaders
Load edges

LOAD EDGES „file_name‟ [LOCALE loc]
COLUMNS attribute_name [alias_name], …
INTO node_type_name
[IGNORE (attribute_name|alias_name), …]
WHERE
TAIL (attribute_name|alias_name) = node_type_name.attribute_name
HEAD (attribute_name|alias_name) = node_type_name.attribute_name
[FIELDS
[TERMINATED char]
[ENCLOSED char]
[ALLOW_MULTILINE [max]]]
Dex Graph Database

[FROM num]
[MAX num]
[MODE (ROWS|COLUMNS [SPLIT [PARTITIONS num]])]


Script loaders
Examples
create gdb WIKIPEDIA into 'wikipedia.dex'
create node TITLES (
ID int unique,
'TEXT' string,
NLC string,
TITLE string indexed
)
create node IMAGES (
ID int unique,
NLC string,
FILENAME string indexed
)
Dex Graph Database

create edge REFS from TITLES to TITLES (
NLC string,
"TEXT" string,
TYPE string
) materialize neighbors
create undirected edge IMGS


Script loaders
Examples
use gdb WIKIPEDIA into 'wikipedia.dex'
load nodes 'images.csv'
columns ID, NLC, FILENAME
into IMAGES
from 2 max 10000
load edges 'references.csv'
columns NLC, 'TEXT', TYPE, FROM F, TO T
into REFS
ignore F, T
where
tail F = TITLES.ID
head T = TITLES.ID
Dex Graph Database

fields
terminated „|‟
enclosed „”‟
allow_multiline
mode columns split partitions 3


Tips & tricks

 Index or not?
 Attributes:
 Attributes used at select operations
must be indexed.
 Optionally, index once the attribute
has been created/loaded.
 Neighbors:
Dex Graph Database

 It is recommended to index those
edge types used at neighbors
operations.


Tips & tricks

 String attributes
 String
 Maximum length = 2047.
 Indexed or not.
 Select
[==, !=, >, >=, <, <=, Like, LikeNoCase, RegExp]

 Text (Character large object)
 Unlimited length.
Dex Graph Database

 Not Indexed.
 Just get and set.
 Streaming read and write operations.


Tips & tricks

 Others:
 DB cross-platform format.
 32 – 64 bits, OS independent.
 Just take into account platform endianness.
 Read only mode.
 Configuration:
 com.sparsity.dex.gdb.DexConfig
Dex Graph Database

 Set the maximum memory usage.
 0 means unlimited.
 License.
 No license means evaluation version.


Thanks for your
attention
Any questions?
Dex Graph Database

SPARSITY-TECHNOLOGIES
Jordi Girona, 1-3, Edifici K2M
08034 Barcelona
info@sparsity-technologies.com


Dexjava Technical Seminar Dec 2011

More Related Content

What's hot

Viewers also liked

Similar to Dexjava Technical Seminar Dec 2011

Recently uploaded

Dexjava Technical Seminar Dec 2011