SlideShare a Scribd company logo
1 of 45
No SQL?




    Image credit: http://browsertoolkit.com/fault-tolerance.png
Neo4j   the benefits of
               graph databases



Emil Eifrem
CEO, Neo Technology
Death?
Community
experimentation:
  CouchDB
  SimpleDB
  Hypertable
  Cassandra
  Scalaris
  ...
?
Trend 1: data is getting more connected
                                                                                  Giant
                                                                                 Global
 Information connectivity


                                                                                 Graph
                                                                                 (GGG)

                                                                    Ontologies


                                                              RDF

                                                                          Folksonomies
                                                          Tagging

                                                                User-
                                              Wikis
                                                              generated
                                                               content
                                                      Blogs


                                             RSS


                                 Hypertext


                       Text
                    documents      web 1.0            web 2.0              “web 3.0”

                                1990         2000                   2010                  2020
Trend 2: ... and more semi-structured
 Individualization of content!
   In the salary lists of the 1970s, all elements had
   exactly one job
   In the salary lists of the 2000s, we need 5 job
   columns! Or 8? Or 15?
 Trend accelerated by the decentralization of
 content generation that is the hallmark of the age
 of participation (“web 2.0”)
Relational database
              Salary List
Performance




                            Majority of
                            Webapps



                                             Social network

                                                                     Semantic




                                                    }
                                                                      Trading




                                                                custom


                                          Information complexity
We = hackers!
   So that’s vCPU...
what about vhackers?
Whiteboard friendly?




       ?
                                 owns
                       Björn               Big Car
                        build           transport
                                             ,
                                          Kids &
                                DayCare Veggies
Alternative?
 a graph database
The Graph DB model: representation
 Core abstractions:                          name = “Emil”
                                             age = 29
   Nodes                                     sex = “yes”



   Relationships between nodes
                                         1                         2
   Properties on both

                        type = KNOWS
                        time = 4 years                       3

                                                                 type = car
                                                                 vendor = “SAAB”
                                                                 model = “95 Aero”
Example: The Matrix
                                                                            name = “The Architect”
                               name = “Morpheus”
                               rank = “Captain”
name = “Thomas Anderson”
                               occupation = “Total badass”                                        42
age = 29
                                                  disclosure = public


                 KNOWS                             KNOWS                                             CODED_BY
     1                                                                           KN O
                                          7                             3            WS

                                                                                                 13
                                           S
                 KN                                       name = “Cypher”
                                          KNOW


                      OW                                  last name = “Reagan”
                           S
                                                                                              name = “Agent Smith”
                                                                        disclosure = secret   version = 1.0b
         age = 3 days                                                   age = 6 months        language = C++
                                      2

                               name = “Trinity”
Code (1): Building a node space
NeoService neo = ... // Get factory


// Create Thomas 'Neo' Anderson
Node mrAnderson = neo.createNode();
mrAnderson.setProperty( "name", "Thomas Anderson" );
mrAnderson.setProperty( "age", 29 );

// Create Morpheus
Node morpheus = neo.createNode();
morpheus.setProperty( "name", "Morpheus" );
morpheus.setProperty( "rank", "Captain" );
morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each other
mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );
// ...create Trinity, Cypher, Agent Smith, Architect similarly
Code (1): Building a node space
NeoService neo = ... // Get factory
Transaction tx = neo.beginTx();

// Create Thomas 'Neo' Anderson
Node mrAnderson = neo.createNode();
mrAnderson.setProperty( "name", "Thomas Anderson" );
mrAnderson.setProperty( "age", 29 );

// Create Morpheus
Node morpheus = neo.createNode();
morpheus.setProperty( "name", "Morpheus" );
morpheus.setProperty( "rank", "Captain" );
morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each other
mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );
// ...create Trinity, Cypher, Agent Smith, Architect similarly

tx.commit();
Code (1b): Defining RelationshipTypes
// In package org.neo4j.api.core
public interface RelationshipType
{
   String name();
}

// In package org.yourdomain.yourapp
// Example on how to roll dynamic RelationshipTypes
class MyDynamicRelType implements RelationshipType
{
   private final String name;
   MyDynamicRelType( String name ){ this.name = name; }
   public String name() { return this.name; }
}

// Example on how to kick it, static-RelationshipType-like
enum MyStaticRelTypes implements RelationshipType
{
   KNOWS,
   WORKS_FOR,
}
The Graph DB model: traversal
 Traverser framework for                    name = “Emil”
 high-performance traversing                age = 29
                                            sex = “yes”
 across the node space
                                        1                         2



                       type = KNOWS
                       time = 4 years                       3

                                                                type = car
                                                                vendor = “SAAB”
                                                                model = “95 Aero”
Example: Mr Anderson’s friends
                                                                            name = “The Architect”
                               name = “Morpheus”
                               rank = “Captain”
name = “Thomas Anderson”
                               occupation = “Total badass”                                        42
age = 29
                                                  disclosure = public


                 KNOWS                             KNOWS                                             CODED_BY
     1                                                                           KN O
                                          7                             3            WS

                                                                                                 13
                                           S
                 KN                                       name = “Cypher”
                                          KNOW


                      OW                                  last name = “Reagan”
                           S
                                                                                              name = “Agent Smith”
                                                                        disclosure = secret   version = 1.0b
         age = 3 days                                                   age = 6 months        language = C++
                                      2

                               name = “Trinity”
Code (2): Traversing a node space
// Instantiate a traverser that returns Mr Anderson's friends
Traverser friendsTraverser = mrAnderson.traverse(
   Traverser.Order.BREADTH_FIRST,
   StopEvaluator.END_OF_GRAPH,
   ReturnableEvaluator.ALL_BUT_START_NODE,
   RelTypes.KNOWS,
   Direction.OUTGOING );

// Traverse the node space and print out the result
System.out.println( "Mr Anderson's friends:" );
for ( Node friend : friendsTraverser )
{
   System.out.printf( "At depth %d => %s%n",
       friendsTraverser.currentPosition().getDepth(),
       friend.getProperty( "name" ) );
}
name = “The Architect”
                                name = “Morpheus”
                                rank = “Captain”
 name = “Thomas Anderson”
                                occupation = “Total badass”                                         42
 age = 29
                                                   disclosure = public


                  KNOWS                             KNOWS                                              CODED_BY
      1                                                                           KN O
                                           7                             3            WS

                                                                                                   13

                                            S
                  KN                                       name = “Cypher”

                                           KNOW
                       OW                                  last name = “Reagan”
                            S
                                                                                                name = “Agent Smith”
                                                                         disclosure = secret    version = 1.0b
          age = 3 days                                                   age = 6 months         language = C++
                                       2

                                name = “Trinity”
                                                                     $ bin/start-neo-example
                                                                     Mr Anderson's friends:

                                                                     At      depth     1   =>   Morpheus
friendsTraverser = mrAnderson.traverse(
  Traverser.Order.BREADTH_FIRST,                                     At      depth     1   =>   Trinity
  StopEvaluator.END_OF_GRAPH,                                        At      depth     2   =>   Cypher
  ReturnableEvaluator.ALL_BUT_START_NODE,
  RelTypes.KNOWS,
                                                                     At      depth     3   =>   Agent Smith
  Direction.OUTGOING );                                              $
Example: Friends in love?
                                                                                  name = “The Architect”
                                     name = “Morpheus”
                                     rank = “Captain”
name = “Thomas Anderson”
                                     occupation = “Total badass”                                        42
age = 29
                                                        disclosure = public


                       KNOWS                             KNOWS                                             CODED_BY
     1                                          7                             3        KN O
                                                                                           WS

                                                                                                       13
                                                 S

                       KN
                                                KNOW


                                                                name = “Cypher”
                            OW                                  last name = “Reagan”
                                 S
                                                                                                    name = “Agent Smith”
         LO                                                                   disclosure = secret   version = 1.0b
              VE                                                              age = 6 months        language = C++
                   S
                                            2

                                     name = “Trinity”
Code (3a): Custom traverser
// Create a traverser that returns all “friends in love”
Traverser loveTraverser = mrAnderson.traverse(
   Traverser.Order.BREADTH_FIRST,
   StopEvaluator.END_OF_GRAPH,
   new ReturnableEvaluator()
   {
       public boolean isReturnableNode( TraversalPosition pos )
       {
          return pos.currentNode().hasRelationship(
              RelTypes.LOVES, Direction.OUTGOING );
       }
   },
   RelTypes.KNOWS,
   Direction.OUTGOING );
Code (3a): Custom traverser
// Traverse the node space and print out the result
System.out.println( "Who’s a lover?" );
for ( Node person : loveTraverser )
{
   System.out.printf( "At depth %d => %s%n",
       loveTraverser.currentPosition().getDepth(),
       person.getProperty( "name" ) );
}
name = “The Architect”
                                   name = “Morpheus”
                                   rank = “Captain”
 name = “Thomas Anderson”
                                   occupation = “Total badass”                                        42
 age = 29
                                                      disclosure = public


                     KNOWS                             KNOWS                         KN O                CODED_BY
      1                                       7                             3            WS

                                                                                                     13

                                               S
                     KN

                                              KNOW
                                                              name = “Cypher”
                          OW                                  last name = “Reagan”
                               S
                                                                                                  name = “Agent Smith”
          LO                                                                disclosure = secret   version = 1.0b
            VE                                                              age = 6 months        language = C++
                 S                        2

                                   name = “Trinity”
                                                                       $ bin/start-neo-example
new ReturnableEvaluator()
                                                                       Who’s a lover?
{
  public boolean isReturnableNode(
    TraversalPosition pos)
                                                                       At depth 1 => Trinity
  {                                                                    $
    return pos.currentNode().
      hasRelationship( RelTypes.LOVES,
         Direction.OUTGOING );
  }
},
Bonus code: domain model
    How do you implement your domain model?
    Use the delegator pattern, i.e. every domain entity
    wraps a Neo4j primitive:
// In package org.yourdomain.yourapp
class PersonImpl implements Person
{
   private final Node underlyingNode;
   PersonImpl( Node node ){ this.underlyingNode = node; }

    public String getName()
    {
       return this.underlyingNode.getProperty( "name" );
    }
    public void setName( String name )
    {
       this.underlyingNode.setProperty( "name", name );
    }
}
Domain layer frameworks
 Qi4j (www.qi4j.org)
   Framework for doing DDD in pure Java5
   Defines Entities / Associations / Properties
     Sound familiar? Nodes / Rel’s / Properties!
   Neo4j is an “EntityStore” backend

 NeoWeaver (http://components.neo4j.org/neo-weaver)
   Weaves Neo4j-backed persistence into domain
   objects in runtime (dynamic proxy / cglib based)
   Veeeery alpha
Neo4j system characteristics
 Disk-based
   Native graph storage engine with custom (“SSD-
   ready”) binary on-disk format
 Transactional
   JTA/JTS, XA, 2PC, Tx recovery, deadlock
   detection, etc
 Scalable
   Several billions of nodes/rels/props on single JVM
 Robust
   6+ years in 24/7 production
Social network pathExists()
          12
                               ~1k persons
                           3
7         1                    Avg 50 friends per
                               person
                               pathExists(a, b) limit
    36
         41           77       depth 4
                 5
                               Two backends
                               Eliminate disk IO so
                               warm up caches
Social network pathExists()

                 2
                Emil
         1                                    5
                                      7
        Mike                                Kevin
                           3        John
                         Marcus
                   9                4
                 Bruce            Leigh

                                  # persons query time
Relational database                   1 000 2 000 ms
Graph database (Neo4j)                1 000      2 ms
Graph database (Neo4j)            1 000 000      2 ms
Pros & Cons compared to RDBMS
+ No O/R impedance mismatch (whiteboard friendly)
+ Can easily evolve schemas
+ Can represent semi-structured info
+ Can represent graphs/networks (with performance)


- Lacks in tool and framework support
- No other implementations => potential lock in
- No support for ad-hoc queries
+
More consequences
 Ability to capture semi-structured information
   => allowing individualization of content
 No predefined schema
   => easier to evolve model
   => can capture ad-hoc relationships
 Can capture non-normative relations
   => easy to model specific links to specific sets
 All state is kept in transactional memory
   => improves application concurrency
The Neo4j ecosystem
 Neo4j is an embedded database
   Tiny teeny lil jar file
 Component ecosystem
   index-util
   neo-meta
   neo-utils
   owl2neo
   sparql-engine
   ...
 See http://components.neo4j.org
Example: NeoRDF

       NeoRDF triple/quad store

            OWL         SPARQL

      RDF
             Metamodel      Graph
                            match



                Neo4j
Language bindings
 Neo4j.py – bindings for Jython and CPython
   http://components.neo4j.org/neo4j.py

 Neo4jrb – bindings for JRuby (incl RESTful API)
   http://wiki.neo4j.org/content/Ruby

 Clojure
   http://wiki.neo4j.org/content/Clojure

 Scala (incl RESTful API)
   http://wiki.neo4j.org/content/Scala

 … .NET? Erlang?
Scale out – replication
 Rolling out Neo4j HA before end-of-year
   Side note: ppl roll it today w/ REST frontends & onlinebackup

 Master-slave replication, 1st configuration
   MySQL style... ish
   Except all instances can write, synchronously
   between writing slave & master (strict consistency)
   Updates are asynchronously propagated to the
   other slaves (eventual consistency)
 This can handle billions of entities...
 … but not 100B
Scale out – partitioning
 Sharding possible today
   … but you have to do a lot of manual work
   … just as with MySQL
   Great option: shard on top of resilient, scalable
   OSS app server             , see: www.codecauldron.org
 Transparent partitioning? Neo4j 2.0
   100B? Easy to say. Sliiiiightly harder to do.
   Fundamentals: BASE & eventual consistency
   Generic clustering algorithm as base case, but
   give lots of knobs for developers
How ego are you? (aka other impls?)
 Franz’ AllegroGraph      (http://agraph.franz.com)

   Proprietary, Lisp, RDF-oriented but real graphdb
 FreeBase graphd     (http://bit.ly/13VITB)

   In-house at Metaweb
 Kloudshare   (http://kloudshare.com)

   Graph database in the cloud, still stealth mode
 Google Pregel   (http://bit.ly/dP9IP)

   We are oh-so-secret
 Some academic papers from ~10 years ago
   G = {V, E}   #FAIL
Conclusion
 Graphs && Neo4j => teh awesome!
 Available NOW under AGPLv3 / commercial license
   AGPLv3: “if you’re open source, we’re open source”
   If you have proprietary software? Must buy a
   commercial license
   But up to 1M primitives it’s free for all uses!
 Download
   http://neo4j.org
 Feedback
   http://lists.neo4j.org
Questions?




             Image credit: lost again! Sorry :(
http://neotechnology.com

More Related Content

What's hot

Neo4j - 5 cool graph examples
Neo4j - 5 cool graph examplesNeo4j - 5 cool graph examples
Neo4j - 5 cool graph examples
Peter Neubauer
 

What's hot (12)

Neo4j - 5 cool graph examples
Neo4j - 5 cool graph examplesNeo4j - 5 cool graph examples
Neo4j - 5 cool graph examples
 
Intro to Neo4j presentation
Intro to Neo4j presentationIntro to Neo4j presentation
Intro to Neo4j presentation
 
Grails goes Graph
Grails goes GraphGrails goes Graph
Grails goes Graph
 
Rc173 010d-json 2
Rc173 010d-json 2Rc173 010d-json 2
Rc173 010d-json 2
 
An overview of NOSQL (JFokus 2011)
An overview of NOSQL (JFokus 2011)An overview of NOSQL (JFokus 2011)
An overview of NOSQL (JFokus 2011)
 
2010 09-neo4j-deutsche-telekom
2010 09-neo4j-deutsche-telekom2010 09-neo4j-deutsche-telekom
2010 09-neo4j-deutsche-telekom
 
RDBMS to Graph
RDBMS to GraphRDBMS to Graph
RDBMS to Graph
 
Graph databases
Graph databasesGraph databases
Graph databases
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
 
Publishing Linked Open Data in 15 minutes
Publishing Linked Open Data in 15 minutesPublishing Linked Open Data in 15 minutes
Publishing Linked Open Data in 15 minutes
 
Sector CloudSlam 09
Sector CloudSlam 09Sector CloudSlam 09
Sector CloudSlam 09
 
Granularity in linked open data
Granularity in linked open dataGranularity in linked open data
Granularity in linked open data
 

Viewers also liked

Paul Fabretti Blogging Presentation 11th April 2008
Paul Fabretti Blogging Presentation   11th April 2008Paul Fabretti Blogging Presentation   11th April 2008
Paul Fabretti Blogging Presentation 11th April 2008
InBlackandWhite
 

Viewers also liked (6)

OSCON 2009: Building a Corporate Blog Portal using WordPress MU (WPMU)
OSCON 2009: Building a Corporate Blog Portal using WordPress MU (WPMU)OSCON 2009: Building a Corporate Blog Portal using WordPress MU (WPMU)
OSCON 2009: Building a Corporate Blog Portal using WordPress MU (WPMU)
 
Paul Fabretti Blogging Presentation 11th April 2008
Paul Fabretti Blogging Presentation   11th April 2008Paul Fabretti Blogging Presentation   11th April 2008
Paul Fabretti Blogging Presentation 11th April 2008
 
Introduction to Corporate Blogging - WATConsult.com
Introduction to Corporate Blogging - WATConsult.comIntroduction to Corporate Blogging - WATConsult.com
Introduction to Corporate Blogging - WATConsult.com
 
Blogging for Business - Sweet for all sizes
Blogging for Business - Sweet for all sizesBlogging for Business - Sweet for all sizes
Blogging for Business - Sweet for all sizes
 
Blogging for Business
Blogging for BusinessBlogging for Business
Blogging for Business
 
Social Media & Social Marketing. The Power of Conversations
Social Media & Social Marketing. The Power of ConversationsSocial Media & Social Marketing. The Power of Conversations
Social Media & Social Marketing. The Power of Conversations
 

Similar to Neo4j - The Benefits of Graph Databases (OSCON 2009)

Neo4j spatial-nosql-frankfurt
Neo4j spatial-nosql-frankfurtNeo4j spatial-nosql-frankfurt
Neo4j spatial-nosql-frankfurt
Peter Neubauer
 
1st UIM-GDB - Connections to the Real World
1st UIM-GDB - Connections to the Real World1st UIM-GDB - Connections to the Real World
1st UIM-GDB - Connections to the Real World
Achim Friedland
 
Building RESTful Java Applications with EMF
Building RESTful Java Applications with EMFBuilding RESTful Java Applications with EMF
Building RESTful Java Applications with EMF
Kenn Hussey
 

Similar to Neo4j - The Benefits of Graph Databases (OSCON 2009) (20)

Neo4j Nosqllive
Neo4j NosqlliveNeo4j Nosqllive
Neo4j Nosqllive
 
Neo4j
Neo4jNeo4j
Neo4j
 
Neo4j spatial-nosql-frankfurt
Neo4j spatial-nosql-frankfurtNeo4j spatial-nosql-frankfurt
Neo4j spatial-nosql-frankfurt
 
No Sql
No SqlNo Sql
No Sql
 
Querying Riak Just Got Easier - Introducing Secondary Indices
Querying Riak Just Got Easier - Introducing Secondary IndicesQuerying Riak Just Got Easier - Introducing Secondary Indices
Querying Riak Just Got Easier - Introducing Secondary Indices
 
Non Relational Databases
Non Relational DatabasesNon Relational Databases
Non Relational Databases
 
2009 - Node XL v.84+ - Social Media Network Visualization Tools For Excel 2007
2009 - Node XL v.84+ - Social Media Network Visualization Tools For Excel 20072009 - Node XL v.84+ - Social Media Network Visualization Tools For Excel 2007
2009 - Node XL v.84+ - Social Media Network Visualization Tools For Excel 2007
 
Taming NoSQL with Spring Data
Taming NoSQL with Spring DataTaming NoSQL with Spring Data
Taming NoSQL with Spring Data
 
1st UIM-GDB - Connections to the Real World
1st UIM-GDB - Connections to the Real World1st UIM-GDB - Connections to the Real World
1st UIM-GDB - Connections to the Real World
 
High-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and ModelingHigh-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and Modeling
 
Django and Neo4j - Domain modeling that kicks ass
Django and Neo4j - Domain modeling that kicks assDjango and Neo4j - Domain modeling that kicks ass
Django and Neo4j - Domain modeling that kicks ass
 
Dexjava Technical Seminar Dec 2011
Dexjava Technical Seminar Dec 2011Dexjava Technical Seminar Dec 2011
Dexjava Technical Seminar Dec 2011
 
using Spring and MongoDB on Cloud Foundry
using Spring and MongoDB on Cloud Foundryusing Spring and MongoDB on Cloud Foundry
using Spring and MongoDB on Cloud Foundry
 
Using Architectures for Semantic Interoperability to Create Journal Clubs for...
Using Architectures for Semantic Interoperability to Create Journal Clubs for...Using Architectures for Semantic Interoperability to Create Journal Clubs for...
Using Architectures for Semantic Interoperability to Create Journal Clubs for...
 
web 3.0 part1
web 3.0 part1web 3.0 part1
web 3.0 part1
 
An intro to Neo4j and some use cases (JFokus 2011)
An intro to Neo4j and some use cases (JFokus 2011)An intro to Neo4j and some use cases (JFokus 2011)
An intro to Neo4j and some use cases (JFokus 2011)
 
When?
When?When?
When?
 
Building RESTful Java Applications with EMF
Building RESTful Java Applications with EMFBuilding RESTful Java Applications with EMF
Building RESTful Java Applications with EMF
 
Scaling Big Data Mining Infrastructure Twitter Experience
Scaling Big Data Mining Infrastructure Twitter ExperienceScaling Big Data Mining Infrastructure Twitter Experience
Scaling Big Data Mining Infrastructure Twitter Experience
 
MongoDb and Windows Azure
MongoDb and Windows AzureMongoDb and Windows Azure
MongoDb and Windows Azure
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational Performance
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 

Neo4j - The Benefits of Graph Databases (OSCON 2009)

  • 1. No SQL? Image credit: http://browsertoolkit.com/fault-tolerance.png
  • 2. Neo4j the benefits of graph databases Emil Eifrem CEO, Neo Technology
  • 3.
  • 4.
  • 5. Death? Community experimentation: CouchDB SimpleDB Hypertable Cassandra Scalaris ...
  • 6. ?
  • 7. Trend 1: data is getting more connected Giant Global Information connectivity Graph (GGG) Ontologies RDF Folksonomies Tagging User- Wikis generated content Blogs RSS Hypertext Text documents web 1.0 web 2.0 “web 3.0” 1990 2000 2010 2020
  • 8. Trend 2: ... and more semi-structured Individualization of content! In the salary lists of the 1970s, all elements had exactly one job In the salary lists of the 2000s, we need 5 job columns! Or 8? Or 15? Trend accelerated by the decentralization of content generation that is the hallmark of the age of participation (“web 2.0”)
  • 9. Relational database Salary List Performance Majority of Webapps Social network Semantic } Trading custom Information complexity
  • 10. We = hackers! So that’s vCPU... what about vhackers?
  • 11. Whiteboard friendly? ? owns Björn Big Car build transport , Kids & DayCare Veggies
  • 12.
  • 14. The Graph DB model: representation Core abstractions: name = “Emil” age = 29 Nodes sex = “yes” Relationships between nodes 1 2 Properties on both type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  • 15. Example: The Matrix name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
  • 16. Code (1): Building a node space NeoService neo = ... // Get factory // Create Thomas 'Neo' Anderson Node mrAnderson = neo.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = neo.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly
  • 17. Code (1): Building a node space NeoService neo = ... // Get factory Transaction tx = neo.beginTx(); // Create Thomas 'Neo' Anderson Node mrAnderson = neo.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = neo.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly tx.commit();
  • 18. Code (1b): Defining RelationshipTypes // In package org.neo4j.api.core public interface RelationshipType { String name(); } // In package org.yourdomain.yourapp // Example on how to roll dynamic RelationshipTypes class MyDynamicRelType implements RelationshipType { private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; } } // Example on how to kick it, static-RelationshipType-like enum MyStaticRelTypes implements RelationshipType { KNOWS, WORKS_FOR, }
  • 19. The Graph DB model: traversal Traverser framework for name = “Emil” high-performance traversing age = 29 sex = “yes” across the node space 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  • 20. Example: Mr Anderson’s friends name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity”
  • 21. Code (2): Traversing a node space // Instantiate a traverser that returns Mr Anderson's friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING ); // Traverse the node space and print out the result System.out.println( "Mr Anderson's friends:" ); for ( Node friend : friendsTraverser ) { System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) ); }
  • 22. name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 KN O 7 3 WS 13 S KN name = “Cypher” KNOW OW last name = “Reagan” S name = “Agent Smith” disclosure = secret version = 1.0b age = 3 days age = 6 months language = C++ 2 name = “Trinity” $ bin/start-neo-example Mr Anderson's friends: At depth 1 => Morpheus friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, At depth 1 => Trinity StopEvaluator.END_OF_GRAPH, At depth 2 => Cypher ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, At depth 3 => Agent Smith Direction.OUTGOING ); $
  • 23. Example: Friends in love? name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS CODED_BY 1 7 3 KN O WS 13 S KN KNOW name = “Cypher” OW last name = “Reagan” S name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity”
  • 24. Code (3a): Custom traverser // Create a traverser that returns all “friends in love” Traverser loveTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, new ReturnableEvaluator() { public boolean isReturnableNode( TraversalPosition pos ) { return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } }, RelTypes.KNOWS, Direction.OUTGOING );
  • 25. Code (3a): Custom traverser // Traverse the node space and print out the result System.out.println( "Who’s a lover?" ); for ( Node person : loveTraverser ) { System.out.printf( "At depth %d => %s%n", loveTraverser.currentPosition().getDepth(), person.getProperty( "name" ) ); }
  • 26. name = “The Architect” name = “Morpheus” rank = “Captain” name = “Thomas Anderson” occupation = “Total badass” 42 age = 29 disclosure = public KNOWS KNOWS KN O CODED_BY 1 7 3 WS 13 S KN KNOW name = “Cypher” OW last name = “Reagan” S name = “Agent Smith” LO disclosure = secret version = 1.0b VE age = 6 months language = C++ S 2 name = “Trinity” $ bin/start-neo-example new ReturnableEvaluator() Who’s a lover? { public boolean isReturnableNode( TraversalPosition pos) At depth 1 => Trinity { $ return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); } },
  • 27. Bonus code: domain model How do you implement your domain model? Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive: // In package org.yourdomain.yourapp class PersonImpl implements Person { private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; } public String getName() { return this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); } }
  • 28. Domain layer frameworks Qi4j (www.qi4j.org) Framework for doing DDD in pure Java5 Defines Entities / Associations / Properties Sound familiar? Nodes / Rel’s / Properties! Neo4j is an “EntityStore” backend NeoWeaver (http://components.neo4j.org/neo-weaver) Weaves Neo4j-backed persistence into domain objects in runtime (dynamic proxy / cglib based) Veeeery alpha
  • 29. Neo4j system characteristics Disk-based Native graph storage engine with custom (“SSD- ready”) binary on-disk format Transactional JTA/JTS, XA, 2PC, Tx recovery, deadlock detection, etc Scalable Several billions of nodes/rels/props on single JVM Robust 6+ years in 24/7 production
  • 30. Social network pathExists() 12 ~1k persons 3 7 1 Avg 50 friends per person pathExists(a, b) limit 36 41 77 depth 4 5 Two backends Eliminate disk IO so warm up caches
  • 31. Social network pathExists() 2 Emil 1 5 7 Mike Kevin 3 John Marcus 9 4 Bruce Leigh # persons query time Relational database 1 000 2 000 ms Graph database (Neo4j) 1 000 2 ms Graph database (Neo4j) 1 000 000 2 ms
  • 32.
  • 33.
  • 34. Pros & Cons compared to RDBMS + No O/R impedance mismatch (whiteboard friendly) + Can easily evolve schemas + Can represent semi-structured info + Can represent graphs/networks (with performance) - Lacks in tool and framework support - No other implementations => potential lock in - No support for ad-hoc queries +
  • 35. More consequences Ability to capture semi-structured information => allowing individualization of content No predefined schema => easier to evolve model => can capture ad-hoc relationships Can capture non-normative relations => easy to model specific links to specific sets All state is kept in transactional memory => improves application concurrency
  • 36. The Neo4j ecosystem Neo4j is an embedded database Tiny teeny lil jar file Component ecosystem index-util neo-meta neo-utils owl2neo sparql-engine ... See http://components.neo4j.org
  • 37. Example: NeoRDF NeoRDF triple/quad store OWL SPARQL RDF Metamodel Graph match Neo4j
  • 38. Language bindings Neo4j.py – bindings for Jython and CPython http://components.neo4j.org/neo4j.py Neo4jrb – bindings for JRuby (incl RESTful API) http://wiki.neo4j.org/content/Ruby Clojure http://wiki.neo4j.org/content/Clojure Scala (incl RESTful API) http://wiki.neo4j.org/content/Scala … .NET? Erlang?
  • 39. Scale out – replication Rolling out Neo4j HA before end-of-year Side note: ppl roll it today w/ REST frontends & onlinebackup Master-slave replication, 1st configuration MySQL style... ish Except all instances can write, synchronously between writing slave & master (strict consistency) Updates are asynchronously propagated to the other slaves (eventual consistency) This can handle billions of entities... … but not 100B
  • 40. Scale out – partitioning Sharding possible today … but you have to do a lot of manual work … just as with MySQL Great option: shard on top of resilient, scalable OSS app server , see: www.codecauldron.org Transparent partitioning? Neo4j 2.0 100B? Easy to say. Sliiiiightly harder to do. Fundamentals: BASE & eventual consistency Generic clustering algorithm as base case, but give lots of knobs for developers
  • 41. How ego are you? (aka other impls?) Franz’ AllegroGraph (http://agraph.franz.com) Proprietary, Lisp, RDF-oriented but real graphdb FreeBase graphd (http://bit.ly/13VITB) In-house at Metaweb Kloudshare (http://kloudshare.com) Graph database in the cloud, still stealth mode Google Pregel (http://bit.ly/dP9IP) We are oh-so-secret Some academic papers from ~10 years ago G = {V, E} #FAIL
  • 42. Conclusion Graphs && Neo4j => teh awesome! Available NOW under AGPLv3 / commercial license AGPLv3: “if you’re open source, we’re open source” If you have proprietary software? Must buy a commercial license But up to 1M primitives it’s free for all uses! Download http://neo4j.org Feedback http://lists.neo4j.org
  • 43.
  • 44. Questions? Image credit: lost again! Sorry :(