SlideShare a Scribd company logo
1 of 26
Download to read offline
Ditch SQL, use C# to store C# objects
in VelocityDB & VelocityGraph
NoSQL databases
Mats Persson
@VelocityDB
Tired of mapping layers from .NET to SQL, Json
or XML? There is another way to do it!
Introduction to VelocityGraph
Property Graph Database basic structure
Graph An object that contains vertices and edges
Vertex An object that has incoming and outgoing edges
Edge An object that has a tail and head vertex
Property A key/value associated with a Vertex or Edge
Edges and Vertices also have an associated id
Vertex
669
id
Property name (key) Property Value
“rating” 4
“Established” Nov 14, 2015
... …
Edge
tail head
Can also be bidirectional
454
id
Property name (key) Property Value
rating 4
“Established” Nov 14, 2015
... …
out in
Property Graph Standard Interfaces
TinkerPop/Blueprints is a standardized
interface for Java
Frontenac.Blueprints is a C# port of the
Java interfaces
VelocityGraph implements the
Frontenac.Blueprints interfaces
Are graph databases schema less?
No, no database application is schema less.
Database Schema is either:
• explicit (relational databases)
• implicit (most of NoSQL)
VelocityGraph additional structure
VertexType Useful to distinguish different types of vertices, e.g. user
vertices and product vertices.
EdgeType Useful to distinguish different types of edges, e.g. friends
edges and enemies edges.
PropertyType Useful to distinguish different types of properties, e.g. age
property and birthday properties
Polymorphism VertexType and EdgeType can have a base type. Api like
EdgeType.GetEdges(bool polymorphic = false) enables
sub types in return value.
Any C# type
object as property
value.
A Neo4J property value can only be a string, a byte or a
number while VelocityGraph supports most C# types as
property value.
VertexType
VertexType userType = graph.NewVertexType("User");
PropertyType genderType = userType.NewProperty("Gender", DataType.Integer, PropertyKind.Indexed);
Vertex aUser = userType.NewVertex();
aUser.SetProperty(genderType, (int)Gender.Male);
Without VertexType (as in BluePrints standard interfaces)
Vertex aVertex = graph.AddVertex();
aVertex.SetProperty("Gender", (int) Gender.Male);
Also found in OrientDB and sparksee
EdgeType
EdgeType ratingEdgeType= graph.NewEdgeType("UserToRating", true, userType, ratingType);
public EdgeType NewEdgeType(string name, bool biderectional, VertexType tailType,
VertexType headType, EdgeType baseType = null)
Using API on class Graph
Also found in OrientDB and sparksee
PropertyType
PropertyType genderType = userType.NewProperty("Gender", DataType.Integer, PropertyKind.Indexed);
public PropertyType NewProperty(string name, DataType dt, PropertyKind kind)
Using API on class VertexType and EdgeType
Polymorphism
VertexType userType = graph.NewVertexType("User");
VertexType powerUserType = g.NewVertexType("PowerUser", userType);
public VertexType NewVertexType(string name, VertexType baseType = null)
public EdgeType NewEdgeType(string name, bool biderectional, EdgeType baseType = null)
Using polymorphic API on class Graph
public IEnumerable<Edge> GetEdges(bool polymorphic = false)
on class EdgeType
abstract public Vertex GetPropertyVertex(IComparable value, bool polymorphic = false, bool errorIfNotFound = true);
on class PropertyType
public Vertex GetVertex(VertexId vertexId, bool polymorphic = false, bool errorIfNotFound = true)
public IEnumerable<Vertex> GetVertices(bool polymorphic = false)
on class VertexType
Also found in OrientDB
Any C# type object as property value
VelocityGraph is build as an extension of object database VelocityDB. You
can use graphs as a component of your overall application design and you
can use any C# type objects as property values.
PropertyType objectPropertyTypeIndexed = graph.NewVertexProperty(movieType, "director", DataType.IOptimizedPersistable,
PropertyKind.Indexed);
Vertex mVickyCB = movieType.NewVertex();
mVickyCB.SetProperty(movieTitleType, "Vicky Cristina Barcelona");
Person pObj = new Person();
mVickyCB.SetProperty(objectPropertyTypeIndexed, pObj);
Vertex lookup = objectPropertyTypeIndexed.GetPropertyVertex(pObj);
Downloading VelocityGraph
Direct link: www.VelocityDB.com/VelocityDbNoServer.exe (14 MB)
Generate 10 day trial license: www.VelocityDB.com/Secure/License.aspx
Then download (click Download button) it to c:/4.odb
Samples pick up license file from c:/4.odb
After 10 days, generate another trial license or pay $200 for a permanent
license.
VelocityGraph and all VelocityDB & VelocityGraph sample projects are all
open source on GitHub.
VelocityDB and VelocityGraph are available as NuGet.
Quick Start sample application
using Frontenac.Blueprints.Util.IO.GraphJson;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using VelocityDb.Session;
using VelocityGraph;
namespace QuickStartVelocityGraph
{
class QuickStartVelocityGraph
{
static readonly string systemDir = "QuickStartVelocityGraph"; // appended to SessionBase.BaseDatabasePath
static readonly string s_licenseDbFile = "c:/4.odb"; // (download from https://www.velocitydb.com/Secure/Download.aspx)
static void CreateGraph()
{
using (SessionNoServer session = new SessionNoServer(systemDir))
{
if (Directory.Exists(session.SystemDirectory))
Directory.Delete(session.SystemDirectory, true); // remove systemDir from prior runs and all its databases.
// Start an update transaction
session.BeginUpdate();
// Copy VelocityDB license database to this database directory
File.Copy(s_licenseDbFile, Path.Combine(session.SystemDirectory, "4.odb"));
Graph g = new Graph(session);
session.Persist(g);
// Add a node type for the movies, with a unique identifier and two indexed Propertys
VertexType movieType = g.NewVertexType("Movie");
PropertyType movieTitleType = g.NewVertexProperty(movieType, "title", DataType.String, PropertyKind.Indexed);
PropertyType movieYearType = g.NewVertexProperty(movieType, "year", DataType.Integer, PropertyKind.Indexed);
// Add a node type for the actor
VertexType actorType = g.NewVertexType("Actor");
PropertyType actorNameType = g.NewVertexProperty(actorType, "name", DataType.String, PropertyKind.Indexed);
// Add a directed edge type with a Property for the cast of a movie
EdgeType castType = g.NewEdgeType("ACTS_IN", false);
PropertyType castCharacterType = g.NewEdgeProperty(castType, "role", DataType.String, PropertyKind.Indexed);
Visualize graph with Alchemy.js
First export graph to GraphJson
g.ExportToGraphJson("c:/QuickStart.json");
and then include in html doc
Browse graph with Database Manager
SupplierTracking
To summarize we start of with a bunch of leaf stores which all are associated with a particular
supplier, which is a property on the Store node. Inventory is then moved along to other stores
and the proportion from each supplier corresponds to their contribution to the original store.
So for node B02, S2 contributed 750/1250 = 60% and S3 contributed 40%. We then move
600 units our of B02 of which 60% belongs to S2 and 40% to S3 and so on.
What we want to know what percentage of the final 700 units into D01 belong to each
supplier. Where suppliers with the same name are the same supplier.
Simple to do with C# and VelocityGraph!
Console.WriteLine("Supplier 1 to Warehouse D total: " + supplierTracking.CalculateTotalTo(supplier1, supplierWarehouseEdgeType, moveToS1EdgeType, howManyS1Property, wareHouseD1));
Console.WriteLine("Supplier 2 to Warehouse D total: " + supplierTracking.CalculateTotalTo(supplier2, supplierWarehouseEdgeType, moveToS2EdgeType, howManyS2Property, wareHouseD1));
Console.WriteLine("Supplier 3 to Warehouse D total: " + supplierTracking.CalculateTotalTo(supplier3, supplierWarehouseEdgeType, moveToS3EdgeType, howManyS3Property, wareHouseD1));
int CalculateTotalTo(Vertex supplier, EdgeType supplierWarehouseEdgeType, EdgeType moveToEdgeType, PropertyType howManyProperty,
Vertex toVertex)
{
int total = 0;
HashSet<Vertex> excludeSet = new HashSet<Vertex>();
foreach (IEdge wareHouseEdge in supplier.GetEdges(supplierWarehouseEdgeType, Direction.Out))
{
Vertex supplierWareHouse = (Vertex)wareHouseEdge.GetVertex(Direction.In);
var allPaths = supplierWareHouse.Traverse(toVertex, moveToEdgeType, 10, true, null, excludeSet);
foreach (List<Edge> path in allPaths)
{
if (path.Count > 0)
total += (int)path.Last().GetProperty(howManyProperty); // last because that is all we care about in this simplified sample
foreach (Edge edge in path)
{
excludeSet.Add(edge.Tail);
}
}
}
return total;
}
Same problem using Neo4J
MATCH p =s<-[:MOVE_TO*]-sup WHERE HAS (sup.Supplier) AND NOT HAS
(s.Supplier) WITH s,sup,reduce(totalSupplier = 0, r IN relationships(p)| totalSupplier
+ r.Quantity) AS TotalAmountMoved WITH sum(TotalAmountMoved) AS
sumMoved, collect(DISTINCT ([sup.Supplier, TotalAmountMoved])) AS
MyDataPart1,s WITH reduce(b=[], c IN MyDataPart1| b +[{ Supplier: c[0], Quantity:
c[1], Percentile: ((c[1]*1.00))/(sumMoved*1.00)*100.00 }]) AS MyData, s, sumMoved
RETURN s.Name, sumMoved, MyData
???
Dating Recommendations
Dating site with ratings. Can be used to recommend other people a user might like.
17 million ratings, 168 000 profiles that are rated by 135 000 users.
All ratings are contained in the file "ratings.dat" and are in the following format:
UserID,ProfileID,Rating
- UserID is user who provided rating
- ProfileID is user who has been rated
- UserIDs range between 1 and 135,359
- ProfileIDs range between 1 and 220,970 (not every profile has been rated)
- Ratings are on a 1-10 scale where 10 is best (integer ratings only)
- Only users who provided at least 20 ratings were included
- Users who provided constant ratings were excluded
User gender information is in the file "gender.dat" and is in the following format:
UserID,Gender
- Gender is denoted by a "M" for male and "F" for female and "U" for unknown
Graph structure selected
Id: 1
Gender: M
User Vertices
Id: 2
Gender: F
Id: 3
Gender: M
Id: 4
Gender: U
Id: 5
Gender: M
Id: 168,791
Gender: M
Id: 1
Rating: 1
Rating Vertices
Id: 2
Rating: 2
Id: 3
Rating: 3
Id: 4
Rating: 4
Id: 5
Rating: 5
Id: 6
Rating: 6
Id: 7
Rating: 7
Id: 8
Rating: 8
Id: 9
Rating: 9
Id: 10
Rating: 10
User to Rating Edge
Rating Of Edge
For each rating, add 2 edges. One
relating user with a rating and
another one relating rated by with
rated user.
Queries
Complex queries
- Given a user id, and based on the rated profiles, find other users who have
rated similarly on the same profiles, and find other profiles to recommend based
on what those other users have rated
- Get all female users with less than 50 ratings
- Get all male users with at least one 10 rating
- Get the first 10 male users who have rated at least 3 of the same profiles as
the given user.
Statistics queries
- Get the 20 profiles with the most ratings
- Get the 20 best rated profiles regardless of gender
- Get the 20 best rated males
- Get the 20 best rated females
Get the 20 best rated profiles regardless of gender
var ratingsVertexEnum = from v in ratingType.GetVertices() orderby v.GetProperty(ratingValuePropertyType) descending select v;
Vertex rating10Vertex = ratingsVertexEnum.First();
// Get the 20 best rated profiles regardless of gender
var top = from u in userType.GetVertices()
let edgeCt = u.GetNumberOfEdges(ratingEdgeType, rating10Vertex, Direction.Out)
orderby edgeCt descending
select new { u, edgeCt };
Console.WriteLine("20 best rated profiles regardless of gender");
ct = 0;
foreach (var v in top)
{
if (++ct > 20)
break;
Console.WriteLine("User id: " + v.u.VertexId + “t10 ratings: " + v.edgeCt);
}
Triangle Counter
This test idea started with a Vertica comparison with Hadoop and
PIG. Given 86,220,856 tuples compute the number of triangles that
can be formed from these edges.
Number of Nodes found: 4,846,609
Number of Triangles found: 285,730,264
[c:]more edges.txt
208761 293938
208761 293946
208761 299583
208761 316917
208761 377488
208761 377492
Kevin Bacon Distance
Computes the degree of separation between Kevin Bacon and all other actors and actresses (~
2 million) taking part in about 350,000 movies.
0 degrees means you are Kevin Bacon
1 degree, you acted in one of Kevin Bacon's movies
2 degrees, you acted in a movie with someone who acted in the same movie as Kevin Bacon
and so on...
The output looks similar to Oracle of Bacon (which by the way is not using an Oracle database!)
[c:VelocityDbRelease]timer& KevinBaconNumbers.exe& timer
Timer 1 on: 12:07:04
Degree 0 has # of people: 1
Degree 1 has # of people: 3475
Degree 2 has # of people: 385345
Degree 3 has # of people: 894996
Degree 4 has # of people: 121903
Degree 5 has # of people: 7462
Degree 6 has # of people: 446
Degree 7 has # of people: 28
Degree 8 has # of people: 0
Degree 9 has # of people: 0
Timer 1 off: 12:07:33 Elapsed: 0:00:29.81

More Related Content

What's hot

Automatically generating-json-from-java-objects-java-objects268
Automatically generating-json-from-java-objects-java-objects268Automatically generating-json-from-java-objects-java-objects268
Automatically generating-json-from-java-objects-java-objects268
Ramamohan Chokkam
 
Javascript basic course
Javascript basic courseJavascript basic course
Javascript basic course
Tran Khoa
 

What's hot (20)

EMF Tips n Tricks
EMF Tips n TricksEMF Tips n Tricks
EMF Tips n Tricks
 
Xtext Webinar
Xtext WebinarXtext Webinar
Xtext Webinar
 
Xtext's new Formatter API
Xtext's new Formatter APIXtext's new Formatter API
Xtext's new Formatter API
 
Xtext Eclipse Con
Xtext Eclipse ConXtext Eclipse Con
Xtext Eclipse Con
 
Prototype Js
Prototype JsPrototype Js
Prototype Js
 
Turning Ideas Into Code Faster
Turning Ideas Into Code FasterTurning Ideas Into Code Faster
Turning Ideas Into Code Faster
 
Hamcrest
HamcrestHamcrest
Hamcrest
 
Nikita Popov "What’s new in PHP 8.0?"
Nikita Popov "What’s new in PHP 8.0?"Nikita Popov "What’s new in PHP 8.0?"
Nikita Popov "What’s new in PHP 8.0?"
 
Codegeneration Goodies
Codegeneration GoodiesCodegeneration Goodies
Codegeneration Goodies
 
Java best practices
Java best practicesJava best practices
Java best practices
 
Intro To Hibernate
Intro To HibernateIntro To Hibernate
Intro To Hibernate
 
Iphone course 1
Iphone course 1Iphone course 1
Iphone course 1
 
Textual Modeling Framework Xtext
Textual Modeling Framework XtextTextual Modeling Framework Xtext
Textual Modeling Framework Xtext
 
Automatically generating-json-from-java-objects-java-objects268
Automatically generating-json-from-java-objects-java-objects268Automatically generating-json-from-java-objects-java-objects268
Automatically generating-json-from-java-objects-java-objects268
 
Json the-x-in-ajax1588
Json the-x-in-ajax1588Json the-x-in-ajax1588
Json the-x-in-ajax1588
 
Few simple-type-tricks in scala
Few simple-type-tricks in scalaFew simple-type-tricks in scala
Few simple-type-tricks in scala
 
Advanced JavaScript
Advanced JavaScriptAdvanced JavaScript
Advanced JavaScript
 
Javascript basic course
Javascript basic courseJavascript basic course
Javascript basic course
 
Advanced JavaScript
Advanced JavaScriptAdvanced JavaScript
Advanced JavaScript
 
Javascript Basics
Javascript BasicsJavascript Basics
Javascript Basics
 

Similar to VelocityGraph Introduction

Overview of Android Infrastructure
Overview of Android InfrastructureOverview of Android Infrastructure
Overview of Android Infrastructure
C.T.Co
 
Overview of Android Infrastructure
Overview of Android InfrastructureOverview of Android Infrastructure
Overview of Android Infrastructure
Alexey Buzdin
 

Similar to VelocityGraph Introduction (20)

Getting the most out of Java [Nordic Coding-2010]
Getting the most out of Java [Nordic Coding-2010]Getting the most out of Java [Nordic Coding-2010]
Getting the most out of Java [Nordic Coding-2010]
 
Introduction To Groovy 2005
Introduction To Groovy 2005Introduction To Groovy 2005
Introduction To Groovy 2005
 
Using the Windows 8 Runtime from C++
Using the Windows 8 Runtime from C++Using the Windows 8 Runtime from C++
Using the Windows 8 Runtime from C++
 
Twitter Author Prediction from Tweets using Bayesian Network
Twitter Author Prediction from Tweets using Bayesian NetworkTwitter Author Prediction from Tweets using Bayesian Network
Twitter Author Prediction from Tweets using Bayesian Network
 
Real World MVC
Real World MVCReal World MVC
Real World MVC
 
Adding a modern twist to legacy web applications
Adding a modern twist to legacy web applicationsAdding a modern twist to legacy web applications
Adding a modern twist to legacy web applications
 
Reactive clean architecture
Reactive clean architectureReactive clean architecture
Reactive clean architecture
 
Overview of GraphQL & Clients
Overview of GraphQL & ClientsOverview of GraphQL & Clients
Overview of GraphQL & Clients
 
Svcc Building Rich Applications with Groovy's SwingBuilder
Svcc Building Rich Applications with Groovy's SwingBuilderSvcc Building Rich Applications with Groovy's SwingBuilder
Svcc Building Rich Applications with Groovy's SwingBuilder
 
Overview of Android Infrastructure
Overview of Android InfrastructureOverview of Android Infrastructure
Overview of Android Infrastructure
 
Overview of Android Infrastructure
Overview of Android InfrastructureOverview of Android Infrastructure
Overview of Android Infrastructure
 
Vertx SouJava
Vertx SouJavaVertx SouJava
Vertx SouJava
 
Vertx daitan
Vertx daitanVertx daitan
Vertx daitan
 
Einführung in TypeScript
Einführung in TypeScriptEinführung in TypeScript
Einführung in TypeScript
 
Introduction to Client-Side Javascript
Introduction to Client-Side JavascriptIntroduction to Client-Side Javascript
Introduction to Client-Side Javascript
 
Having Fun with Play
Having Fun with PlayHaving Fun with Play
Having Fun with Play
 
Android Automated Testing
Android Automated TestingAndroid Automated Testing
Android Automated Testing
 
Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008
 
Data access 2.0? Please welcome: Spring Data!
Data access 2.0? Please welcome: Spring Data!Data access 2.0? Please welcome: Spring Data!
Data access 2.0? Please welcome: Spring Data!
 
Ian 2014.10.24 weekly report
Ian 2014.10.24 weekly reportIan 2014.10.24 weekly report
Ian 2014.10.24 weekly report
 

Recently uploaded

JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
Max Lee
 
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfMastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
mbmh111980
 

Recently uploaded (20)

What need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java DevelopersWhat need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java Developers
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdf
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdfStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdf
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdf
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
 
OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
CompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfCompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdf
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in Michelangelo
 
Secure Software Ecosystem Teqnation 2024
Secure Software Ecosystem Teqnation 2024Secure Software Ecosystem Teqnation 2024
Secure Software Ecosystem Teqnation 2024
 
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
 
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfMastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
 
10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf
 
how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdf
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
 
APVP,apvp apvp High quality supplier safe spot transport, 98% purity
APVP,apvp apvp High quality supplier safe spot transport, 98% purityAPVP,apvp apvp High quality supplier safe spot transport, 98% purity
APVP,apvp apvp High quality supplier safe spot transport, 98% purity
 
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAGAI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
 
Workforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdfWorkforce Efficiency with Employee Time Tracking Software.pdf
Workforce Efficiency with Employee Time Tracking Software.pdf
 
IT Software Development Resume, Vaibhav jha 2024
IT Software Development Resume, Vaibhav jha 2024IT Software Development Resume, Vaibhav jha 2024
IT Software Development Resume, Vaibhav jha 2024
 
SQL Injection Introduction and Prevention
SQL Injection Introduction and PreventionSQL Injection Introduction and Prevention
SQL Injection Introduction and Prevention
 
Top Mobile App Development Companies 2024
Top Mobile App Development Companies 2024Top Mobile App Development Companies 2024
Top Mobile App Development Companies 2024
 

VelocityGraph Introduction

  • 1. Ditch SQL, use C# to store C# objects in VelocityDB & VelocityGraph NoSQL databases Mats Persson @VelocityDB Tired of mapping layers from .NET to SQL, Json or XML? There is another way to do it!
  • 3. Property Graph Database basic structure Graph An object that contains vertices and edges Vertex An object that has incoming and outgoing edges Edge An object that has a tail and head vertex Property A key/value associated with a Vertex or Edge Edges and Vertices also have an associated id
  • 4. Vertex 669 id Property name (key) Property Value “rating” 4 “Established” Nov 14, 2015 ... …
  • 5. Edge tail head Can also be bidirectional 454 id Property name (key) Property Value rating 4 “Established” Nov 14, 2015 ... … out in
  • 6. Property Graph Standard Interfaces TinkerPop/Blueprints is a standardized interface for Java Frontenac.Blueprints is a C# port of the Java interfaces VelocityGraph implements the Frontenac.Blueprints interfaces
  • 7. Are graph databases schema less? No, no database application is schema less. Database Schema is either: • explicit (relational databases) • implicit (most of NoSQL)
  • 8. VelocityGraph additional structure VertexType Useful to distinguish different types of vertices, e.g. user vertices and product vertices. EdgeType Useful to distinguish different types of edges, e.g. friends edges and enemies edges. PropertyType Useful to distinguish different types of properties, e.g. age property and birthday properties Polymorphism VertexType and EdgeType can have a base type. Api like EdgeType.GetEdges(bool polymorphic = false) enables sub types in return value. Any C# type object as property value. A Neo4J property value can only be a string, a byte or a number while VelocityGraph supports most C# types as property value.
  • 9. VertexType VertexType userType = graph.NewVertexType("User"); PropertyType genderType = userType.NewProperty("Gender", DataType.Integer, PropertyKind.Indexed); Vertex aUser = userType.NewVertex(); aUser.SetProperty(genderType, (int)Gender.Male); Without VertexType (as in BluePrints standard interfaces) Vertex aVertex = graph.AddVertex(); aVertex.SetProperty("Gender", (int) Gender.Male); Also found in OrientDB and sparksee
  • 10. EdgeType EdgeType ratingEdgeType= graph.NewEdgeType("UserToRating", true, userType, ratingType); public EdgeType NewEdgeType(string name, bool biderectional, VertexType tailType, VertexType headType, EdgeType baseType = null) Using API on class Graph Also found in OrientDB and sparksee
  • 11. PropertyType PropertyType genderType = userType.NewProperty("Gender", DataType.Integer, PropertyKind.Indexed); public PropertyType NewProperty(string name, DataType dt, PropertyKind kind) Using API on class VertexType and EdgeType
  • 12. Polymorphism VertexType userType = graph.NewVertexType("User"); VertexType powerUserType = g.NewVertexType("PowerUser", userType); public VertexType NewVertexType(string name, VertexType baseType = null) public EdgeType NewEdgeType(string name, bool biderectional, EdgeType baseType = null) Using polymorphic API on class Graph public IEnumerable<Edge> GetEdges(bool polymorphic = false) on class EdgeType abstract public Vertex GetPropertyVertex(IComparable value, bool polymorphic = false, bool errorIfNotFound = true); on class PropertyType public Vertex GetVertex(VertexId vertexId, bool polymorphic = false, bool errorIfNotFound = true) public IEnumerable<Vertex> GetVertices(bool polymorphic = false) on class VertexType Also found in OrientDB
  • 13. Any C# type object as property value VelocityGraph is build as an extension of object database VelocityDB. You can use graphs as a component of your overall application design and you can use any C# type objects as property values. PropertyType objectPropertyTypeIndexed = graph.NewVertexProperty(movieType, "director", DataType.IOptimizedPersistable, PropertyKind.Indexed); Vertex mVickyCB = movieType.NewVertex(); mVickyCB.SetProperty(movieTitleType, "Vicky Cristina Barcelona"); Person pObj = new Person(); mVickyCB.SetProperty(objectPropertyTypeIndexed, pObj); Vertex lookup = objectPropertyTypeIndexed.GetPropertyVertex(pObj);
  • 14. Downloading VelocityGraph Direct link: www.VelocityDB.com/VelocityDbNoServer.exe (14 MB) Generate 10 day trial license: www.VelocityDB.com/Secure/License.aspx Then download (click Download button) it to c:/4.odb Samples pick up license file from c:/4.odb After 10 days, generate another trial license or pay $200 for a permanent license. VelocityGraph and all VelocityDB & VelocityGraph sample projects are all open source on GitHub. VelocityDB and VelocityGraph are available as NuGet.
  • 15. Quick Start sample application using Frontenac.Blueprints.Util.IO.GraphJson; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using VelocityDb.Session; using VelocityGraph; namespace QuickStartVelocityGraph { class QuickStartVelocityGraph { static readonly string systemDir = "QuickStartVelocityGraph"; // appended to SessionBase.BaseDatabasePath static readonly string s_licenseDbFile = "c:/4.odb"; // (download from https://www.velocitydb.com/Secure/Download.aspx) static void CreateGraph() { using (SessionNoServer session = new SessionNoServer(systemDir)) { if (Directory.Exists(session.SystemDirectory)) Directory.Delete(session.SystemDirectory, true); // remove systemDir from prior runs and all its databases. // Start an update transaction session.BeginUpdate(); // Copy VelocityDB license database to this database directory File.Copy(s_licenseDbFile, Path.Combine(session.SystemDirectory, "4.odb")); Graph g = new Graph(session); session.Persist(g); // Add a node type for the movies, with a unique identifier and two indexed Propertys VertexType movieType = g.NewVertexType("Movie"); PropertyType movieTitleType = g.NewVertexProperty(movieType, "title", DataType.String, PropertyKind.Indexed); PropertyType movieYearType = g.NewVertexProperty(movieType, "year", DataType.Integer, PropertyKind.Indexed); // Add a node type for the actor VertexType actorType = g.NewVertexType("Actor"); PropertyType actorNameType = g.NewVertexProperty(actorType, "name", DataType.String, PropertyKind.Indexed); // Add a directed edge type with a Property for the cast of a movie EdgeType castType = g.NewEdgeType("ACTS_IN", false); PropertyType castCharacterType = g.NewEdgeProperty(castType, "role", DataType.String, PropertyKind.Indexed);
  • 16. Visualize graph with Alchemy.js First export graph to GraphJson g.ExportToGraphJson("c:/QuickStart.json"); and then include in html doc
  • 17. Browse graph with Database Manager
  • 18. SupplierTracking To summarize we start of with a bunch of leaf stores which all are associated with a particular supplier, which is a property on the Store node. Inventory is then moved along to other stores and the proportion from each supplier corresponds to their contribution to the original store. So for node B02, S2 contributed 750/1250 = 60% and S3 contributed 40%. We then move 600 units our of B02 of which 60% belongs to S2 and 40% to S3 and so on. What we want to know what percentage of the final 700 units into D01 belong to each supplier. Where suppliers with the same name are the same supplier.
  • 19. Simple to do with C# and VelocityGraph! Console.WriteLine("Supplier 1 to Warehouse D total: " + supplierTracking.CalculateTotalTo(supplier1, supplierWarehouseEdgeType, moveToS1EdgeType, howManyS1Property, wareHouseD1)); Console.WriteLine("Supplier 2 to Warehouse D total: " + supplierTracking.CalculateTotalTo(supplier2, supplierWarehouseEdgeType, moveToS2EdgeType, howManyS2Property, wareHouseD1)); Console.WriteLine("Supplier 3 to Warehouse D total: " + supplierTracking.CalculateTotalTo(supplier3, supplierWarehouseEdgeType, moveToS3EdgeType, howManyS3Property, wareHouseD1)); int CalculateTotalTo(Vertex supplier, EdgeType supplierWarehouseEdgeType, EdgeType moveToEdgeType, PropertyType howManyProperty, Vertex toVertex) { int total = 0; HashSet<Vertex> excludeSet = new HashSet<Vertex>(); foreach (IEdge wareHouseEdge in supplier.GetEdges(supplierWarehouseEdgeType, Direction.Out)) { Vertex supplierWareHouse = (Vertex)wareHouseEdge.GetVertex(Direction.In); var allPaths = supplierWareHouse.Traverse(toVertex, moveToEdgeType, 10, true, null, excludeSet); foreach (List<Edge> path in allPaths) { if (path.Count > 0) total += (int)path.Last().GetProperty(howManyProperty); // last because that is all we care about in this simplified sample foreach (Edge edge in path) { excludeSet.Add(edge.Tail); } } } return total; }
  • 20. Same problem using Neo4J MATCH p =s<-[:MOVE_TO*]-sup WHERE HAS (sup.Supplier) AND NOT HAS (s.Supplier) WITH s,sup,reduce(totalSupplier = 0, r IN relationships(p)| totalSupplier + r.Quantity) AS TotalAmountMoved WITH sum(TotalAmountMoved) AS sumMoved, collect(DISTINCT ([sup.Supplier, TotalAmountMoved])) AS MyDataPart1,s WITH reduce(b=[], c IN MyDataPart1| b +[{ Supplier: c[0], Quantity: c[1], Percentile: ((c[1]*1.00))/(sumMoved*1.00)*100.00 }]) AS MyData, s, sumMoved RETURN s.Name, sumMoved, MyData ???
  • 21. Dating Recommendations Dating site with ratings. Can be used to recommend other people a user might like. 17 million ratings, 168 000 profiles that are rated by 135 000 users. All ratings are contained in the file "ratings.dat" and are in the following format: UserID,ProfileID,Rating - UserID is user who provided rating - ProfileID is user who has been rated - UserIDs range between 1 and 135,359 - ProfileIDs range between 1 and 220,970 (not every profile has been rated) - Ratings are on a 1-10 scale where 10 is best (integer ratings only) - Only users who provided at least 20 ratings were included - Users who provided constant ratings were excluded User gender information is in the file "gender.dat" and is in the following format: UserID,Gender - Gender is denoted by a "M" for male and "F" for female and "U" for unknown
  • 22. Graph structure selected Id: 1 Gender: M User Vertices Id: 2 Gender: F Id: 3 Gender: M Id: 4 Gender: U Id: 5 Gender: M Id: 168,791 Gender: M Id: 1 Rating: 1 Rating Vertices Id: 2 Rating: 2 Id: 3 Rating: 3 Id: 4 Rating: 4 Id: 5 Rating: 5 Id: 6 Rating: 6 Id: 7 Rating: 7 Id: 8 Rating: 8 Id: 9 Rating: 9 Id: 10 Rating: 10 User to Rating Edge Rating Of Edge For each rating, add 2 edges. One relating user with a rating and another one relating rated by with rated user.
  • 23. Queries Complex queries - Given a user id, and based on the rated profiles, find other users who have rated similarly on the same profiles, and find other profiles to recommend based on what those other users have rated - Get all female users with less than 50 ratings - Get all male users with at least one 10 rating - Get the first 10 male users who have rated at least 3 of the same profiles as the given user. Statistics queries - Get the 20 profiles with the most ratings - Get the 20 best rated profiles regardless of gender - Get the 20 best rated males - Get the 20 best rated females
  • 24. Get the 20 best rated profiles regardless of gender var ratingsVertexEnum = from v in ratingType.GetVertices() orderby v.GetProperty(ratingValuePropertyType) descending select v; Vertex rating10Vertex = ratingsVertexEnum.First(); // Get the 20 best rated profiles regardless of gender var top = from u in userType.GetVertices() let edgeCt = u.GetNumberOfEdges(ratingEdgeType, rating10Vertex, Direction.Out) orderby edgeCt descending select new { u, edgeCt }; Console.WriteLine("20 best rated profiles regardless of gender"); ct = 0; foreach (var v in top) { if (++ct > 20) break; Console.WriteLine("User id: " + v.u.VertexId + “t10 ratings: " + v.edgeCt); }
  • 25. Triangle Counter This test idea started with a Vertica comparison with Hadoop and PIG. Given 86,220,856 tuples compute the number of triangles that can be formed from these edges. Number of Nodes found: 4,846,609 Number of Triangles found: 285,730,264 [c:]more edges.txt 208761 293938 208761 293946 208761 299583 208761 316917 208761 377488 208761 377492
  • 26. Kevin Bacon Distance Computes the degree of separation between Kevin Bacon and all other actors and actresses (~ 2 million) taking part in about 350,000 movies. 0 degrees means you are Kevin Bacon 1 degree, you acted in one of Kevin Bacon's movies 2 degrees, you acted in a movie with someone who acted in the same movie as Kevin Bacon and so on... The output looks similar to Oracle of Bacon (which by the way is not using an Oracle database!) [c:VelocityDbRelease]timer& KevinBaconNumbers.exe& timer Timer 1 on: 12:07:04 Degree 0 has # of people: 1 Degree 1 has # of people: 3475 Degree 2 has # of people: 385345 Degree 3 has # of people: 894996 Degree 4 has # of people: 121903 Degree 5 has # of people: 7462 Degree 6 has # of people: 446 Degree 7 has # of people: 28 Degree 8 has # of people: 0 Degree 9 has # of people: 0 Timer 1 off: 12:07:33 Elapsed: 0:00:29.81

Editor's Notes

  1. My name is Mats. I have been building NoSQL database engines for the past 18 years. The first 14 years I build an object database engine called Objectivity/DB. This was using C++. Today I will show how to build a database with only C# code. I started building VelocityDB a few years ago. The hardest part is keeping it simple to use. Using C# instead of C++ helps, it is making it easier to use. With an object database you can still choose to design your data classes with total freedom. This is great but it also makes each application unique, each with a different design. Graph databases reduces the freedom of choice and makes you design your database with a fixed small set of options. This should make it simpler to use and as a builder of a graph database engine we can focus on optimizing these few data structures.
  2. A couple of years ago I got a request to do a graph database as an extension to VelocityDB. I got input from other graph databases such as DEX – now called sparcsee but mainly it was driven by input from VelocityDB user’s around the world. Most of the input came from Vidar at WebNodes in Norway but I also got input from a user in India and another one in Austrailia. Louis-Pierre in Canada is also an important collaborator.
  3. Let’s start by talking about the basic structure of a property graph database. It is really simple: graph, vertex, edge and properties. Vertex is called node is some other graph databases such as Neo4J. Edge is a relation or connection between vertices. Both edges and vertices can have one or more key value properties. The key of property is always a string. Each edge and Vertex have an associated id.
  4. An object that has incoming and outgoing edges. A vertex has an id and can have multiple properties.
  5. An object that has a tail and head vertex. At the tail end, the outgoing side, of an edge we have the tail vertex. At the head end, the incoming side, of an edge we have the head vertex. An edge can be either one directional or biderectional.
  6. For Java property graph databases we have an interface standard called TinkerPop/Blueprints. A couple of years ago, Louis-Pierre in Canada ported the Java interfaces to C#, he named it Frontenac.Blueprints. Obviously, we made VelocityGraph implement the Frontenac.Blueprints interfaces.
  7. Neo4J claims it is schema less but …
  8. We need more structure, a light weight schema, so that a complex graph can be done efficiently and is readable. VelocityGraph is an extension to the object database VelocityDB. Each VelocityDB application has a unique schema but a typical VelocityGraph application has a fixed schema as far as VelocityDB is concerned. However, we add some structure to VelocityGraph by introducing Vertex, Edge and property types. As requested we also added polymorphism to vertex and edge types. Since VelocityGraph is build on VelocityDB, we can easily support property values of any C# type.
  9. VetexType We want a C# type associated with a particular type of Vertex. In this case a User Vertex. We create the new VertexType by specifying the name “User”, the value type as being integer and we want values to be indexed for quick lookup by value. The equivalent to a VertexType is found in other databases supporting graph api, for instance OrientDB and Sparksee
  10. Similarly for EdgeType, we want to create specific a specific type for edges of a certain kind. In this case we have an EdgeType for edges from a user Vertex to a rating Vertex. We can specify that these type of edges are bidirectional and that the head VertexType is userType and tail VertexType is ratingType.
  11. Same idea with PropertyType, we want an object representing a certain type of property. In this case a property “Gender” on the VertexType userType. When we create the property object, we specify what type of values this type of property will have. Values can be indexed or not.
  12. We may want to have VertexType “User” and another VertexType based on “User” called “PowerUser”. We can do that by specifying userType as the base type when we create the “PowerUser” VertexType. We can take advantage of this polymorphism is api such as getting all the edges of an EdgeType. If we specify true for the parameter, we also can all edges that are based on this edge type.
  13. Show how to download and license management.
  14. Switch over to use Visual Studio.
  15. Show DatabaseManager live with VelocityGraphQuickStart data
  16. Show StackOverflow web site and describe problem.
  17. One way to represent this in a graph is like this. Create 168,000 user vertices and 10 rating vertices (one for each possible rating) For each rating create an edge from rater user to rated user and another edge from rated user to its rating. Do so for all 17 million ratings.
  18. Given this data and our graph we want to be able to do the following queries.
  19. Let’s pick one of them
  20. Show Vertica web page.
  21. Describe and Show Oracle of Bacon web page