Neo4j the benefits ofgraph databases
Whats the plan?  Why now? – Four trends  NoSQL overview  Graph databases && Neo4j  Conclusions  Food
988Trend 1:data set size402007            2010
Trend 2: connectedness                                                                                       Giant        ...
Trend 3: semi-structure  Individualization of content!     In the salary lists of the 1970s, all elements had exactly     ...
Aside: RDBMS performance                                                            Relational database Performance       ...
Trend 4: architecture          1990s: Database as integration hub
Trend 4: architecture                 2000s: (Slowly towards...)         Decoupled services with own backend
Why NoSQL 2009? Trend 1: Size. Trend 2: Connectivity. Trend 3: Semi-structure. Trend 4: Architecture.
NoSQLoverview
First off: the damn name  NoSQL is NOT “Never SQL”  NoSQL is NOT “No To SQL”  NoSQL is NOT “WE HATE CHRIS DOG”
NoSQL     is simplyot    nly        !
Four (emerging) NoSQL categories Key-value stores   Based on Amazons Dynamo paper   Data model: (global) collection of K-V...
Four (emerging) NoSQL categories Document databases   Inspired by Lotus Notes   Data model: collections of K-V collections...
NoSQL data models Size        Key-value stores                      Bigtable clones                                       ...
NoSQL data models    Size           Key-value stores                         Bigtable clones                              ...
Graph DBs& Neo4j intro
The Graph DB model: representation Core abstractions:                       name = “Emil”                                 ...
Example: The Matrix                                                                           name = “The Architect”      ...
Code (1): Building a node spaceNeoService neo = ... // Get factory// Create Thomas Neo AndersonNode mrAnderson = neo.creat...
Code (1): Building a node spaceNeoService neo = ... // Get factoryTransaction tx = neo.beginTx();// Create Thomas Neo Ande...
Code (1b): Defining RelationshipTypes// In package org.neo4j.api.corepublic interface RelationshipType{   String name();}//...
Whiteboard friendly                                 owns                      Björn                  Big Car              ...
The Graph DB model: traversal Traverser framework for                    name = “Emil” high-performance traversing        ...
Example: Mr Andersonʼs friends                                                                           name = “The Archi...
Code (2): Traversing a node space// Instantiate a traverser that returns Mr Andersons friendsTraverser friendsTraverser = ...
name = “The Architect”                             name = “Morpheus”                             rank = “Captain”         ...
Example: Friends in love?                                                                          name = “The Architect” ...
Code (3a): Custom traverser// Create a traverser that returns all “friends in love”Traverser loveTraverser = mrAnderson.tr...
Code (3a): Custom traverser// Traverse the node space and print out the resultSystem.out.println( "Who’s a lover?" );for (...
name = “The Architect”                                  name = “Morpheus”                                  rank = “Captain...
Bonus code: domain model     How do you implement your domain model?     Use the delegator pattern, i.e. every domain enti...
Domain layer frameworks Qi4j (www.qi4j.org)   Framework for doing DDD in pure Java5   Defines Entities / Associations / Pro...
Neo4j system characteristics  Disk-based     Native graph storage engine with custom binary on-disk     format  Transactio...
Social network pathExists()                ~1k persons                Avg 50 friends per                person            ...
Social network pathExists()               Emil        Mike                               Kevin                            ...
Pros & Cons compared to RDBMS+ No O/R impedance mismatch (whiteboard friendly)+ Can easily evolve schemas+ Can represent s...
Language bindings Neo4j.py – bindings for Jython and CPython   http://components.neo4j.org/neo4j.py Neo4jrb – bindings for...
Grails Neoclipse screendump
Conclusion Graphs && Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license   AGPLv3: “if youʼre open sourc...
Poop 1 Key-value stores?   => the awesome   … if you have 1000s of BILLIONS records OR you dont   care about programmer pr...
Poop 2 In a not-suck architecture... … the only thing that makes sense is to have an embedded database.
Looking ahead: polyglot persistence
Questions?             Image credit: lost again! Sorry :(
http://neotechnology.com
Eifrem neo4j
Eifrem neo4j
Eifrem neo4j
Eifrem neo4j
Eifrem neo4j
Upcoming SlideShare
Loading in...5
×

Eifrem neo4j

832

Published on

presentation on Nodes and relationships

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
832
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
24
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Eifrem neo4j

  1. 1. Neo4j the benefits ofgraph databases
  2. 2. Whats the plan? Why now? – Four trends NoSQL overview Graph databases && Neo4j Conclusions Food
  3. 3. 988Trend 1:data set size402007 2010
  4. 4. Trend 2: connectedness Giant Global Graph (GGG) Information connectivity Ontologies RDF Folksonomies Tagging Wikis User- generated content Blogs RSS Hypertext Text documents web 1.0 web 2.0 “web 3.0” 1990 2000 2010 2020
  5. 5. Trend 3: semi-structure Individualization of content! In the salary lists of the 1970s, all elements had exactly one job In the salary lists of the 2000s, we need 5 job columns! Or 8? Or 15? Trend accelerated by the decentralization of content generation that is the hallmark of the age of participation (“web 2.0”)
  6. 6. Aside: RDBMS performance Relational database Performance Salary List Majority of Webapps Social network } Semantic Trading custom Data complexity
  7. 7. Trend 4: architecture 1990s: Database as integration hub
  8. 8. Trend 4: architecture 2000s: (Slowly towards...) Decoupled services with own backend
  9. 9. Why NoSQL 2009? Trend 1: Size. Trend 2: Connectivity. Trend 3: Semi-structure. Trend 4: Architecture.
  10. 10. NoSQLoverview
  11. 11. First off: the damn name NoSQL is NOT “Never SQL” NoSQL is NOT “No To SQL” NoSQL is NOT “WE HATE CHRIS DOG”
  12. 12. NoSQL is simplyot nly !
  13. 13. Four (emerging) NoSQL categories Key-value stores Based on Amazons Dynamo paper Data model: (global) collection of K-V pairs Example: Dynomite, Voldemort, Tokyo BigTable clones Based on Googles BigTable paper Data model: big table, column families Example: Hbase, Hypertable
  14. 14. Four (emerging) NoSQL categories Document databases Inspired by Lotus Notes Data model: collections of K-V collections Example: CouchDB, MongoDB Graph databases Inspired by Euler & graph theory Data model: nodes, rels, K-V on both Example: AllegroGraph, VertexDB, Neo4j
  15. 15. NoSQL data models Size Key-value stores Bigtable clones Document databases Graph databases Complexity
  16. 16. NoSQL data models Size Key-value stores Bigtable clones Document databases Graph databases (This is still of 90% nodes & relationships) of usecases Complexity
  17. 17. Graph DBs& Neo4j intro
  18. 18. The Graph DB model: representation Core abstractions: name = “Emil” age = 29 Nodes sex = “yes” Relationships between nodes Properties on both type = KNOWS time = 4 years type = car vendor = “SAAB” model = “95 Aero”
  19. 19. Example: The Matrix name = “The Architect” name = “Morpheus” rank = “Captain” occupation = “Total badass”name = “Thomas Anderson”age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY WS KN S KNO W OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ name = “Trinity”
  20. 20. Code (1): Building a node spaceNeoService neo = ... // Get factory// Create Thomas Neo AndersonNode mrAnderson = neo.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );// Create MorpheusNode morpheus = neo.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly
  21. 21. Code (1): Building a node spaceNeoService neo = ... // Get factoryTransaction tx = neo.beginTx();// Create Thomas Neo AndersonNode mrAnderson = neo.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );// Create MorpheusNode morpheus = neo.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarlytx.commit();
  22. 22. Code (1b): Defining RelationshipTypes// In package org.neo4j.api.corepublic interface RelationshipType{ String name();}// In package org.yourdomain.yourapp// Example on how to roll dynamic RelationshipTypesclass MyDynamicRelType implements RelationshipType{ private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; }}// Example on how to kick it, static-RelationshipType-likeenum MyStaticRelTypes implements RelationshipType{ KNOWS, WORKS_FOR,}
  23. 23. Whiteboard friendly owns Björn Big Car build drives DayCare
  24. 24. The Graph DB model: traversal Traverser framework for name = “Emil” high-performance traversing age = 29 sex = “yes” across the node space type = KNOWS time = 4 years type = car vendor = “SAAB” model = “95 Aero”
  25. 25. Example: Mr Andersonʼs friends name = “The Architect” name = “Morpheus” rank = “Captain” occupation = “Total badass”name = “Thomas Anderson”age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY WS KN S KNO W OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ name = “Trinity”
  26. 26. Code (2): Traversing a node space// Instantiate a traverser that returns Mr Andersons friendsTraverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING );// Traverse the node space and print out the resultSystem.out.println( "Mr Andersons friends:" );for ( Node friend : friendsTraverser ){ System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) );}
  27. 27. name = “The Architect” name = “Morpheus” rank = “Captain” occupation = “Total badass” name = “Thomas Anderson” age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY WS KN S KNO W OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ name = “Trinity” $ bin/start-neo-example Mr Andersons friends: At depth 1 => MorpheusfriendsTraverser = mrAnderson.traverse( Traverser.Order. BREADTH_FIRST , At depth 1 => Trinity StopEvaluator. END_OF_GRAPH , At depth 2 => Cypher ReturnableEvaluator. ALL_BUT_START_NODE , RelTypes. KNOWS , At depth 3 => Agent Smith Direction. OUTGOING ); $
  28. 28. Example: Friends in love? name = “The Architect” name = “Morpheus” rank = “Captain” occupation = “Total badass”name = “Thomas Anderson”age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY WS KN S K NO W OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” LO disclosure = secret version = 1.0b VE language = C++ S age = 6 months name = “Trinity”
  29. 29. Code (3a): Custom traverser// Create a traverser that returns all “friends in love”Traverser loveTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, new ReturnableEvaluator() { public boolean isReturnableNode( TraversalPosition pos ) { return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } }, RelTypes.KNOWS, Direction.OUTGOING );
  30. 30. Code (3a): Custom traverser// Traverse the node space and print out the resultSystem.out.println( "Who’s a lover?" );for ( Node person : loveTraverser ){ System.out.printf( "At depth %d => %s%n", loveTraverser.currentPosition().getDepth(), person.getProperty( "name" ) );}
  31. 31. name = “The Architect” name = “Morpheus” rank = “Captain” occupation = “Total badass” name = “Thomas Anderson” age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY WS KN S K NO W OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” LO disclosure = secret version = 1.0b VE language = C++ S age = 6 months name = “Trinity” $ bin/start-neo-example Who’s a lover?new ReturnableEvaluator(){ public boolean isReturnableNode( At depth 1 => Trinity TraversalPosition pos) { $ return pos.currentNode(). hasRelationship( RelTypes. LOVES, Direction .OUTGOING ); }},
  32. 32. Bonus code: domain model How do you implement your domain model? Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive:// In package org.yourdomain.yourappclass PersonImpl implements Person{ private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; } public String getName() { return this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); }}
  33. 33. Domain layer frameworks Qi4j (www.qi4j.org) Framework for doing DDD in pure Java5 Defines Entities / Associations / Properties Sound familiar? Nodes / Relʼs / Properties! Neo4j is an “EntityStore” backend NeoWeaver (http://components.neo4j.org/neo-weaver) Weaves Neo4j-backed persistence into domain objects in runtime (dynamic proxy / cglib based) Veeeery alpha
  34. 34. Neo4j system characteristics Disk-based Native graph storage engine with custom binary on-disk format Transactional JTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc Scales up (whats the x and the y?) Several billions of nodes/rels/props on single JVM Robust 6+ years in 24/7 production
  35. 35. Social network pathExists() ~1k persons Avg 50 friends per person pathExists(a, b) limit depth 4 Two backends Eliminate disk IO so warm up caches
  36. 36. Social network pathExists() Emil Mike Kevin John Marcus Bruce Leigh # persons query timeRelational database 1 000 2 000 msGraph database (Neo4j) 1 000 2 msGraph database (Neo4j) 1 000 000 2 ms
  37. 37. Pros & Cons compared to RDBMS+ No O/R impedance mismatch (whiteboard friendly)+ Can easily evolve schemas+ Can represent semi-structured info+ Can represent graphs/networks (with performance)- Lacks in tool and framework support- Few other implementations => potential lock in- No support for ad-hoc queries
  38. 38. Language bindings Neo4j.py – bindings for Jython and CPython http://components.neo4j.org/neo4j.py Neo4jrb – bindings for JRuby (incl RESTful API) http://wiki.neo4j.org/content/Ruby Clojure http://wiki.neo4j.org/content/Clojure Scala (incl RESTful API) http://wiki.neo4j.org/content/Scala … .NET? Erlang?
  39. 39. Grails Neoclipse screendump
  40. 40. Conclusion Graphs && Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license AGPLv3: “if youʼre open source, weʼre open source” If you have proprietary software? Must buy a commercial license But up to 1M primitives itʼs free for all uses! Download http://neo4j.org Feedback http://lists.neo4j.org
  41. 41. Poop 1 Key-value stores? => the awesome … if you have 1000s of BILLIONS records OR you dont care about programmer productivity What if you had no variables at all in your programs except a single globally accessible hashtable? Would your software be maintainable?
  42. 42. Poop 2 In a not-suck architecture... … the only thing that makes sense is to have an embedded database.
  43. 43. Looking ahead: polyglot persistence
  44. 44. Questions? Image credit: lost again! Sorry :(
  45. 45. http://neotechnology.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×