No SQL?




          Image credit: http://browsertoolkit.com/fault-tolerance.png
Neo4j   the benefits of
                graph databases



                                   #neo4j
Peter Neubauer       ...
Why NoSQL 2009?

 Trend 1: Size.

 Trend 2: Connectivity.

 Trend 3: Semi-structure.

 Trend 4: Architecture.
NOSQL data models
            Key-value stores
Data size




                          Bigtable clones


                 ...
NOSQL data models
               Key-value stores
   Data size




                             Bigtable clones


        ...
Graph DBs
& Neo4j intro
The Graph DB model: representation
 Core abstractions:
                                              name = “Emil”

   Nod...
Example: The Matrix
                                                                           name = “The Architect”
    ...
Code (1): Building a node space
GraphDatabaseService graphDb = ... // Get factory


// Create Thomas 'Neo' Anderson
Node m...
Code (1): Building a node space
GraphDatabaseService graphDb = ... // Get factory
Transaction tx = graphdb.beginTx();

// ...
Code (1b): Def ning RelationshipTypes
             i
// In package org.neo4j.graphdb
public interface RelationshipType
{
 ...
Whiteboard friendly



                                 owns
                      Björn                  Big Car
        ...
The Graph DB model: traversal
 Traverser framework for
 high-performance traversing                    name = “Emil”
     ...
Example: Mr Anderson’s friends
                                                                           name = “The Arch...
Code (2): Traversing a node space
// Instantiate a traverser that returns Mr Anderson's friends
Traverser friendsTraverser...
name = “The Architect”
                              name = “Morpheus”
                              rank = “Captain”
 nam...
Example: Friends in love?
                                                                             name = “The Archite...
Code (3a): Custom traverser
// Create a traverser that returns all “friends in love”
Traverser loveTraverser = mrAnderson....
Code (3a): Custom traverser
// Traverse the node space and print out the result
System.out.println( "Who’s a lover?" );
fo...
name = “The Architect”
                                name = “Morpheus”
                                rank = “Captain”
...
Domain layer frameworks
 Spring (SpringDM, Grails, Griffon etc), Sling, Karaf
 Rails, Django
 Qi4j (www.qi4j.org)
    Fram...
Code (4): Groovy/Grails
import grails.plugins.neo4j.*
@Neo4jEntity
class Author {

    @Neo4jIndex
    String name
}
Result: Groovy/Grails
Code (5): JRuby
require "rubygems"
require "neo4j"

class Person
    include Neo4j::NodeMixin

      property :name, :sala...
Neo4j system characteristics
  Disk-based
    Native graph storage engine with custom binary on-disk
    format
  Robust
 ...
What do I use this stuff for?
  Modeling, productivity, stability
    Bindings, migrations, robustness
  Fast graph algos ...
Query languages
 SPARQL – “SQL for linked data”
    Ex: ”SELECT ?person WHERE {
          ?person neo4j:KNOWS ?friend .
  ...
The Neo4j ecosystem
 Neo4j-kernel is an embedded database
   Tiny teeny lil jar f le
                      i
 Component ec...
Example: Neo4j-RDF

         Neo4j-RDF triple/quad store
                    REST

               OWL          SPARQL

   ...
Language bindings
 Neo4j.py – bindings for Jython and CPython
    http://components.neo4j.org/neo4j.py
 Neo4jrb – bindings...
Scale out – replication
  Rolling out Neo4j HA... soon :)
  Master-slave replication, 1st configuration
     MySQL style.....
Scale out – partitioning
  Sharding possible today
     … but you have to do manual work
     … just as with MySQL
     Gr...
Conclusion
 Graphs && Neo4j => teh awesome!
 Available NOW under AGPLv3 / commercial license
   AGPLv3: “if you’re open so...
Looking ahead: polyglot persistence


       SQL       &&      N oSQL
Questions?




             Image credit: lost again! Sorry :(
http://neotechnology.com
Neo4j Nosqllive
Neo4j Nosqllive
Upcoming SlideShare
Loading in...5
×

Neo4j Nosqllive

5,279

Published on

Peter Neubauer presents Neo4j at NOSQL Live in Boston.

Published in: Technology, Sports

Neo4j Nosqllive

  1. 1. No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
  2. 2. Neo4j the benefits of graph databases #neo4j Peter Neubauer @peterneubauer COO, Neo Technology peter@neotechnology.com
  3. 3. Why NoSQL 2009? Trend 1: Size. Trend 2: Connectivity. Trend 3: Semi-structure. Trend 4: Architecture.
  4. 4. NOSQL data models Key-value stores Data size Bigtable clones Document databases Graph databases Data complexity
  5. 5. NOSQL data models Key-value stores Data size Bigtable clones Document databases Graph databases (This is still b i l l i on s of 90% nodes & relationships) of use cases Data complexity
  6. 6. Graph DBs & Neo4j intro
  7. 7. The Graph DB model: representation Core abstractions: name = “Emil” Nodes age = 29 sex = “yes” Relationships between nodes Properties on both 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  8. 8. Example: The Matrix name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S OW name = “Cypher” KNOW S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
  9. 9. Code (1): Building a node space GraphDatabaseService graphDb = ... // Get factory // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly
  10. 10. Code (1): Building a node space GraphDatabaseService graphDb = ... // Get factory Transaction tx = graphdb.beginTx(); // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly tx.commit();
  11. 11. Code (1b): Def ning RelationshipTypes i // In package org.neo4j.graphdb public interface RelationshipType { String name(); } // In package org.yourdomain.yourapp // Example on how to roll dynamic RelationshipTypes RelationshipsType myRelType = DynamicRelatoinshipType. withName(“myRelType”); // Example on how to kick it, static-RelationshipType-like enum MyStaticRelTypes implements RelationshipType { KNOWS, WORKS_FOR, }
  12. 12. Whiteboard friendly owns Björn Big Car build drives DayCare
  13. 13. The Graph DB model: traversal Traverser framework for high-performance traversing name = “Emil” age = 31 across the node space sex = “yes” 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  14. 14. Example: Mr Anderson’s friends name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S OW name = “Cypher” KNOW S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
  15. 15. Code (2): Traversing a node space // Instantiate a traverser that returns Mr Anderson's friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING ); // Traverse the node space and print out the result System.out.println( "Mr Anderson's friends:" ); for ( Node friend : friendsTraverser ) { System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) ); }
  16. 16. name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S OW name = “Cypher” KNOW S last name = “Reagan” name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity” $ bin/start-neo-example Mr Anderson's friends: At depth 1 => Morpheus friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, At depth 1 => Trinity StopEvaluator.END_OF_GRAPH, At depth 2 => Cypher ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, At depth 3 => Agent Smith Direction.OUTGOING ); $
  17. 17. Example: Friends in love? name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY KNO 1 7 3 W S KN 13 S KNOW OW name = “Cypher” S last name = “Reagan” name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity”
  18. 18. Code (3a): Custom traverser // Create a traverser that returns all “friends in love” Traverser loveTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, new ReturnableEvaluator() { public boolean isReturnableNode( TraversalPosition pos ) { return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } }, RelTypes.KNOWS, Direction.OUTGOING );
  19. 19. Code (3a): Custom traverser // Traverse the node space and print out the result System.out.println( "Who’s a lover?" ); for ( Node person : loveTraverser ) { System.out.printf( "At depth %d => %s%n", loveTraverser.currentPosition().getDepth(), person.getProperty( "name" ) ); }
  20. 20. name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS KNO CODED_BY 1 7 3 W S KN 13 S name = “Cypher” KNOW OW S last name = “Reagan” name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity” $ bin/start-neo-example new ReturnableEvaluator() Who’s a lover? { public boolean isReturnableNode( TraversalPosition pos) At depth 1 => Trinity { $ return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } },
  21. 21. Domain layer frameworks Spring (SpringDM, Grails, Griffon etc), Sling, Karaf Rails, Django Qi4j (www.qi4j.org) Framework for doing DDD in pure Java5 Def nes Entities / Associations / Properties i Sound familiar? Nodes / Rel’s / Properties! Neo4j is an “EntityStore” backend Jo4neo (http://code.google.com/p/jo4neo) Annotation driven Weaves Neo4j-backed persistence into domain objects at runtime
  22. 22. Code (4): Groovy/Grails import grails.plugins.neo4j.* @Neo4jEntity class Author { @Neo4jIndex String name }
  23. 23. Result: Groovy/Grails
  24. 24. Code (5): JRuby require "rubygems" require "neo4j" class Person include Neo4j::NodeMixin property :name, :salary, :age, :country has_n :friends index :name, :salary, :age, :country end neo = Person.new :name => 'Neo' morpheus = Person.new :name => 'Morpheus' neo.friends << morpheus
  25. 25. Neo4j system characteristics Disk-based Native graph storage engine with custom binary on-disk format Robust 6+ years in 24/7 production Fast 1-2 million nodes per second Transactional JTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc Scales up Many b i l l i on s of nodes/rels/props on single JVM
  26. 26. What do I use this stuff for? Modeling, productivity, stability Bindings, migrations, robustness Fast graph algos on critical path Shortest path (Spatial, GIS, Social Web, Routing) Fast traversals Hierarchies, Networks (Automation, ERP, Intelligence, Finance) Recommendation engines (social, bioinfo) Multidimensional indexes (BI, Analytics, versioning) Activity Streams, Write Load -> Read Load RDF, SAIL, SemWeb
  27. 27. Query languages SPARQL – “SQL for linked data” Ex: ”SELECT ?person WHERE { ?person neo4j:KNOWS ?friend . ?friend neo4j:KNOWS ?foe . ?foe neo4j:name “Larry Ellison” . }” Gremlin – “perl for graphs” Ex: ”./outE[@label='KNOWS']/inV[@age > 30]/@name”
  28. 28. The Neo4j ecosystem Neo4j-kernel is an embedded database Tiny teeny lil jar f le i Component ecosystem Index REST meta-model graph-matching remote-graphdb sparql-engine ... See http://components.neo4j.org
  29. 29. Example: Neo4j-RDF Neo4j-RDF triple/quad store REST OWL SPARQL RDF Metamodel Graph match Neo4j
  30. 30. Language bindings Neo4j.py – bindings for Jython and CPython http://components.neo4j.org/neo4j.py Neo4jrb – bindings for JRuby (incl RESTful API) http://wiki.neo4j.org/content/Ruby Clojure http://wiki.neo4j.org/content/Clojure Scala (incl RESTful API) http://wiki.neo4j.org/content/Scala PHP http://wiki.neo4j.org/content/PHP Erlang, Compojure ...
  31. 31. Scale out – replication Rolling out Neo4j HA... soon :) Master-slave replication, 1st configuration MySQL style... ish Except all instances can write, synchronously between writing slave & master (strong consistency) Updates are asynchronously propagated to the other slaves (eventual consistency) This can handle billions of entities... … but not 100B
  32. 32. Scale out – partitioning Sharding possible today … but you have to do manual work … just as with MySQL Great option: shard on top of resilient, scalable OSS app server , see: www.codecauldron.org Transparent partitioning? Neo4j 2.0 100B? Easy to say. Sliiiiightly harder to do. Fundamentals: BASE & eventual consistency Generic clustering algorithm as base case, but give lots of knobs for developers Cross-shard edges->Node migration/replication->Partition schemes
  33. 33. Conclusion Graphs && Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license AGPLv3: “if you’re open source, we’re open source” If you have proprietary software? Must buy a commercial license But the first one is free! Download http://neo4j.org Feedback http://lists.neo4j.org
  34. 34. Looking ahead: polyglot persistence SQL && N oSQL
  35. 35. Questions? Image credit: lost again! Sorry :(
  36. 36. http://neotechnology.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×