SlideShare a Scribd company logo
GRAPH DATABASES
    Henning Rauch
          1
AGENDA


•   Overview

•   Neo4J

•   InfiniteGraph

•   Fallen-8


                   2
OVERVIEW
         Why it makes sense to know about graph databases


„Graph databases will come into vogue. One key gap in the Hadoop
ecosystem is for graph databases, which support rich mining and visualization of
relationships, influence, and behavioral propensities. The market for graph
databases will boom in 2012 as companies everywhere adopt them for social
media analytics, marketing campaign optimization, and customer experience
fine-tuning. We will see VCs put big money behind graph database and analytics
startups. Many big data platform and tool vendors will acquire the startups to
supplement their expanding Hadoop, NoSQL, and enterprise data warehousing
(EDW) portfolios. Social graph analysis, although not a brand-new field, will
become one of the most prestigious specialties in the data science arena,
focusing on high-powered drilldown into polystructured behavioral data sets.“
                              Source: http://blogs.forrester.com/james_kobielus/11-12-19-the_year_ahead_in_big_data_big_cool_new_stuff_looms_large

                                                 3
OVERVIEW
Example of a real-world graph - facebook




                       Source: http://www.facebook.com/press/info.php?statistics

          4
OVERVIEW
Example of a real-world graph - NYT „Cascade“




                               Source: http://nytlabs.com/projects/cascade.html

                5
OVERVIEW
Example of a real-world graph - phone bill




           6
OVERVIEW
Delimitation to RDBMS - property graph
      RDBMS GraphDB




           7
OVERVIEW
                       Delimitation to RDBMS - property graph
                             RDBMS GraphDB
     Person
Id       Name
0     Henning Rauch
1       René Peinl
2        Foo Bar
3     Bruce Schneier
4     Linus Torwalds




                                  7
OVERVIEW
                       Delimitation to RDBMS - property graph
                             RDBMS GraphDB
     Person
                                                  2
Id       Name                                             3
0     Henning Rauch
1       René Peinl
2        Foo Bar
3     Bruce Schneier
                                                              4
4     Linus Torwalds
                                             1




                                                      0




                                  7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                2
Id       Name          Id_1   Id_2                                      3
0     Henning Rauch     1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                            4
4     Linus Torwalds    0      1                           1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                    0




                                                7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                2
Id       Name          Id_1   Id_2                                      3
0     Henning Rauch     1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                            4
4     Linus Torwalds    0      1                           1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                    0




                                                7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                2
Id       Name          Id_1   Id_2                                      3
0     Henning Rauch     1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                            4
4     Linus Torwalds    0      1                           1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                    0



      Tag
Id       Name
0           .NET
1           Java
2           PKI
3        NoSQL



                                                7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                 2
Id       Name          Id_1   Id_2                                             3
0     Henning Rauch                                 Java
                        1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                                      4
4     Linus Torwalds    0      1                            1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                     0



      Tag                                           NoSQL
Id       Name                                                                      .NET

0           .NET
                                                                         PKI
1           Java
2           PKI
3        NoSQL



                                                7
OVERVIEW
                                                  Delimitation to RDBMS - property graph
                                                                RDBMS GraphDB
     Person                     Knows_rel
                                                                                      2
Id       Name                   Id_1       Id_2                                                     3
0     Henning Rauch                                                      Java
                                 1          0
1       René Peinl               1          2
2        Foo Bar                 1          3
3     Bruce Schneier             1          4
                                                                                                           4
4     Linus Torwalds             0          1                                    1
                                 0          2
                                 0          3
                                 0          4
                                 3          4
                                 4          3                                             0



      Tag                         Tags_rel                               NoSQL
Id       Name          Tag_Id   Person_Id         Significance                                           .NET

0           .NET         0             0               5
                                                                                              PKI
1           Java         1             1               5
2           PKI          2             1               6
3        NoSQL           2             3              10
                         3             0               7
                         3             1               7
                                                                     7
OVERVIEW
                                                  Delimitation to RDBMS - property graph
                                                                RDBMS GraphDB
     Person                     Knows_rel
                                                                                             2
Id       Name                   Id_1       Id_2                                                                3
0     Henning Rauch                                                      Java
                                 1          0
1       René Peinl               1          2
                                                                                                                   10
2        Foo Bar                 1          3
                                                                                5
3     Bruce Schneier             1          4
                                                                                                                           4
4     Linus Torwalds             0          1                                       1
                                 0          2
                                 0          3
                                 0          4
                                 3          4                            7

                                 4          3                                                    0


                                                                                                     5
      Tag                         Tags_rel                               NoSQL           7
Id       Name          Tag_Id   Person_Id         Significance                                                           .NET

0           .NET         0             0               5
                                                                                             6           PKI
1           Java         1             1               5
2           PKI          2             1               6
3        NoSQL           2             3              10
                         3             0               7
                         3             1               7
                                                                     7
OVERVIEW
                                                  Delimitation to RDBMS - property graph
                                                                RDBMS GraphDB
     Person                     Knows_rel
                                                                                              2
Id       Name                   Id_1       Id_2                                                                 3
0     Henning Rauch                                                      Java
                                 1          0
1       René Peinl               1          2
                                                                                                                    10
2        Foo Bar                 1          3
                                                                                5
3     Bruce Schneier             1          4
                                                                                                                            4
4     Linus Torwalds             0          1                                       1
                                 0          2
                                 0          3
                                 0          4
                                 3          4                            7

                                 4          3                                                     0


                                                                                                      5
      Tag                         Tags_rel                               NoSQL            7
Id       Name          Tag_Id   Person_Id         Significance                                                            .NET

0           .NET         0             0               5
                                                                                              6           PKI
1           Java         1             1               5
2           PKI          2             1               6
3        NoSQL           2
                         3
                                       3
                                       0
                                                      10
                                                       7
                                                                                            vertex =
                         3             1               7
                                                                     7                  properties + edges
OVERVIEW
                               Delimitation to RDBMS - Scalability
                                                                    Knows_rel
                                                                    Id_1       Id_2

•   Relation tables act as a global index over linked                1
                                                                     1
                                                                                0
                                                                                2

    data                                                             1
                                                                     1
                                                                                3
                                                                                4
                                                                     0          1


    The bigger the relation table the longer it takes to
                                                                     0          2
•                                                                    0          3


    get the interesting information (e.g. local                      0
                                                                     3
                                                                                4
                                                                                4

    neighbourhood of data)                                           4          3



                                                                     Tags_rel

•   Solution of graph databases: Information on            Tag_Id
                                                             0
                                                                    Person_Id
                                                                           0
                                                                                      Significance
                                                                                           5

    relationships (aka edges) are stored locally on the      1             1               5


    vertex
                                                             2             1               6
                                                             2             3              10
                                                             3             0               7
                                                             3             1               7

                                     8
OVERVIEW
                 Delimitation to RDBMS - example of complexity

•   Task: Find the persons that are known to Id 0.
                                                        Knows_rel

    •   Linear table scan: O(n)                         Id_1
                                                         1
                                                               Id_2
                                                                0
                                                         1      2


        Index scan: O(log n)
                                                         1      3
    •                                                    1      4
                                                         0      1
                                                         0      2

•   Because of the dependency to n RDBMS do not          0
                                                         0
                                                                3
                                                                4

    perform well on recursive search algorithms          3      4
                                                         4      3



•   Graph database solve this task in O(1)

                                    9
OVERVIEW
       Delimitation to other NoSQL products
Size




           > 90% of use cases



                                                                                               Complexity

                                Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                    10
OVERVIEW
                   Delimitation to other NoSQL products
Size
       Key/Value
        stores




                       > 90% of use cases



                                                                                                           Complexity

                                            Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                10
OVERVIEW
                              Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones




                                  > 90% of use cases



                                                                                                                      Complexity

                                                       Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                           10
OVERVIEW
                                  Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones
                              Document
                              databases




                                          > 90% of use cases



                                                                                                                              Complexity

                                                               Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                                   10
OVERVIEW
                                  Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones
                              Document
                              databases


                                                        Graph databases




                                          > 90% of use cases



                                                                                                                                  Complexity

                                                                   Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                                   10
OVERVIEW
                                  Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones
                              Document
                              databases


                                                        Graph databases
                                                                                                           In-memory
                                                                                                         graph databases




                                          > 90% of use cases



                                                                                                                                  Complexity

                                                                   Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                                   10
OVERVIEW
       Graph processing vs. graph database




OLAP       Graph affine
           Universal                                                              OLTP




                         Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                   11
OVERVIEW
       Graph processing vs. graph database




OLAP       Graph affine                                                               OLTP




                         RDBMS
           Universal




                            Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                   11
OVERVIEW
                Graph processing vs. graph database




OLAP                Graph affine                                                               OLTP


       Hadoop


                                  RDBMS
                    Universal




                                     Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                            11
OVERVIEW
                         Graph processing vs. graph database




                             Graph affine
       Pregel




OLAP                                                                                                   OLTP


                Hadoop


                                           RDBMS
                             Universal




                                              Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                                     11
OVERVIEW
                         Graph processing vs. graph database




                             Graph affine
                                           Graph databases


       Pregel




OLAP                                                                                                      OLTP


                Hadoop


                                           RDBMS
                             Universal




                                                 Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                                     11
OVERVIEW
                         Graph processing vs. graph database




                             Graph affine
                                                                             In-memory
                                           Graph databases
                                                                           graph database


       Pregel




OLAP                                                                                                      OLTP


                Hadoop


                                           RDBMS
                             Universal




                                                 Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                                     11
OVERVIEW
                                                                      Graph databases

•   Neo4J                             •   HypergraphDB
•   InfiniteGraph (Objectivity)        •   DEX
•   Sones GraphDB                     •   FlockDB (Twitter)
•   AllegroGraph                      •   Trinity (Microsoft)
•   OrientDB                          •   Fallen-8


                                             Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                 12
13
NEO4J
                                                               Overview

•   Graph database + Lucene index

•   ACID (isolation level read committed)

•   High availability in enterprise edition

•   32 billion vertices, 32 billion edges, 64 billion properties

•   Embedded or via REST-API

•   Support for the Blueprints project
                                      14
NEO4J
                                                                   Architecture


    Cypher/Gremlin                    Java/Ruby/.../C# API

                        REST API

                     Core API (Java)

Caches (files and objects)                     HA

      Record files                       Transaction-log

                            Disk(s)

                                                    Source: http://www.slideshare.net/rheehot/eo4j-12713065

                              15
NEO4J
                  knows
                           Example of the on-disk layout
                                              Name: Bob
                                              Age: 42




Name: Alice
Age: 23
                                             knows




              knows


                                    Name: Carol
                                    Age: 22



                                                  Source: https://github.com/thobe/presentations

                          16
NEO4J
                          knows
                                   Example of the on-disk layout
                                                      Name: Bob
                                                      Age: 42




                                                                          Name
        Name: Alice                                                         Bob
Name    Age: 23
Alice                                                knows
                                                                            Age
                                                                             42
Age
 23
                      knows
                                                               Name
                                                               Carol
                                            Name: Carol
                                            Age: 22
                                                                Age
                                                                 22
                                                          Source: https://github.com/thobe/presentations

                                  16
NEO4J
                                  knows
                                                 Example of the on-disk layout
                                      SP   EP                            Name: Bob
                                      SN EN                              Age: 42
                                       knows



                                                                                             Name
               Name: Alice                           SP   EP                                   Bob
Name           Age: 23
                             SP   EP                 SN EN
Alice                                                                   knows
                             SN EN                    knows                                    Age
                              knows                                                             42
Age
 23
                             knows
                                                                                  Name
 SP Source Previous
                                                                                  Carol
SN Source Next                                                 Name: Carol
EP End Previous                                                Age: 22
EN End Next                                                                        Age
                                                                                    22
        Existent
        Nonexistent                                                          Source: https://github.com/thobe/presentations

                                                16
NEO4J
                                  knows
                                                 Example of the on-disk layout
                                      SP   EP                            Name: Bob
                                      SN EN                              Age: 42
                                       knows



                                                                                             Name
               Name: Alice                           SP   EP                                   Bob
Name           Age: 23
                             SP   EP                 SN EN
Alice                                                                   knows
                             SN EN                    knows                                    Age
                              knows                                                             42
Age
 23
                             knows
                                                                                  Name
 SP Source Previous
                                                                                  Carol
SN Source Next                                                 Name: Carol
EP End Previous                                                Age: 22
EN End Next                                                                        Age
                                                                                    22
        Existent
        Nonexistent                                                          Source: https://github.com/thobe/presentations

                                                16
NEO4J
                                                                       In-memory layout (cache)
         ID
                                Relationship ID refs
                         in:    R1      R2       ...   Rn
         Type 1
                        out     R1       R2      ...   Rn


Vertex        ...        Grouped by type (type = „knows“)
                                                                        •   Transformation of the
                                                                            double linked list (on-disk)
                        in:     R1       R2     ...    Rn
         Type n
                        out     R1       R2     ...    Rn
                                                                            to objects
           Key 1                Key 2            ...          Key n
                                                                        •   Increases the traversal
                Val 1




                                 Val 2




                                                               Val n
                                                                            speed
         ID             start            end           type
Edge
           Key 1                Key 2            ...          Key n
                Val 1




                                 Val 2




                                                               Val n




                                                                                     Source: https://github.com/thobe/presentations

                                                        17
NEO4J
                                                                         Traversal


•   Relationship-expander (delivers edges of a vertex)

•   Evaluators (evaluate if a vertex is going to be traversed or if it
    should be taken to the result set)

•   Projection of the result set (e.g. „take the last vertex of the path“

•   Uniqueness level (sets in steps, whether a node could be visited
    several times)
                                                         Source: https://github.com/thobe/presentations

                                     18
NEO4J
                                                                                     Cypher & Gremlin
 Feature                          Gremlin                                                 Cypher

 Paradigm                 Imperative programming                                  Declarative programming

            •Developed Marko Rodriguez (Tinkerpop)            •In-house development
Description •                                                 •Cypher provides greater opportunities for optimization
              Based on xpath to describe the traversal
            •Developed using Groovy                           •Good for traversals that need back tracking
            •30-50% faster on „simple“ traversals             •Output is a table
                                                              START
                                                               me=node:people(name={myname})
                                                              MATCH
                                                               me-[:HAS_CART]->cart-[:CONTAINS_ITEM]->item
             outE[label=HAS_CART].inV                          item<-[:PURCHASED]-user-[:PURCHASED]->recommendation
              .outE[label=CONTAINS_ITEM].inV                  RETURN recommendation
 Example
              .inE[label=PURCHASED].outV
              .outE[label=PURCHASED].inV                      START
                                                               d=node(1), e=node(2)
                                                              MATCH
                                                               p = shortestPath( d-[*..15]->e )
                                                              RETURN p
                                                                                                  Source: https://github.com/thobe/presentations

                                                         19
NEO4J
                                                   WebAdmin




     Quelle: http://docs.neo4j.org/chunked/stable/images/operations/webadmin-overview.png

20
NEO4J
                                                                               Pricing

                                                                      Price
  Edition            License                    Description
                                                                    (annual)


                                            Complete database
                  Open Source
„Community“                                  including a basic        0 €
                   (GPLv3)
                                           management frontend



                                                    +
                                             Monitoring, better
„Advanced“     Commercial and AGPL                                  6,000 €
                                          management frontend and
                                                 support



                                                    +
„Enterprise“   Commercial and AGPL    Enterprise frontend, HA and   24,000 €
                                           premium support



                                     21
22
INFINITEGRAPH
                                                              Overview

•   Distributed graph database

    •   Implemented in C++ (APIs in Java, C#, Python, etc.)

•   Based on Objectivity/DB (distributed object database)

    •   Established 1988 in Sunnyvale, California

    •   Enterprise-customers + US-government

•   Support for Blueprints
                                      23
INFINITEGRAPH
                                                                      Architecture
               User Apps
                                                     Blueprints#


                                IG#Core/API#


Management#       Naviga0on#                    Session#/#TX#
                                 Placement#                        Configura0on#
 Extensions#      Execu0on#                     Management#



                     Objec0vity/DB#Distributed#Database#




                                                                              Copyright © InfiniteGraph

                                     24
INFINITEGRAPH
                                                                         Ingest
                              AppD2#
   AppD1#                                              AppD3#
                           (Ingest#V2)#
(Ingest#V1)#                                        (Ingest#V3)#
                                #

                          IG#Core/API#


        Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                        Objec@vity/DB#




                                                                   Copyright © InfiniteGraph

                               25
INFINITEGRAPH
                                                                          Ingest
                              AppD2#
   AppD1#                                              AppD3#
                           (Ingest#V2)#
(Ingest#V1)#                                        (Ingest#V3)#
                                #

                          IG#Core/API#


        Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                        Objec@vity/DB#



V1#                             V2#                           V3#



                                                                    Copyright © InfiniteGraph

                               25
INFINITEGRAPH
                                                                            Ingest
                               AppD2#
   App#1%
    AppD1#                     App#2%                    App#3%
                                                         AppD3#
                            (Ingest#V2)#
 (Ingest#V )#
(E1%2{%V1V21})%             (E23{%V2V3})%             (Ingest#V3)#
                                                           %
                                  #

                            IG#Core/API#


          Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                          Objec@vity/DB#



V1#                              V2#                            V3#



                                                                      Copyright © InfiniteGraph

                                25
INFINITEGRAPH
                                                                             Ingest
                               AppD2#
   App#1%
    AppD1#                     App#2%                     App#3%
                                                          AppD3#
                            (Ingest#V2)#
 (Ingest#V )#
(E1%2{%V1V21})%             (E23{%V2V3})%              (Ingest#V3)#
                                                            %
                                  #

                            IG#Core/API#


          Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                          Objec@vity/DB#



V1#               E12$           V2#            E23$             V3#



                                                                       Copyright © InfiniteGraph

                                25
INFINITEGRAPH
                                                                             Ingest
                               AppD2#
   App#1%
    AppD1#                     App#2%                     App#3%
                                                          AppD3#
                            (Ingest#V2)#
 (Ingest#V )#
(E1%2{%V1V21})%             (E23{%V2V3})%              (Ingest#V3)#
                                                            %
                                  #

                            IG#Core/API#


          Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                          Objec@vity/DB#



V1#               E12$           V2#            E23$             V3#



                                                                       Copyright © InfiniteGraph

                                25
INFINITEGRAPH
                                                             Code (ingest)



Vertex alice = myGraph.addVertex(new Person(“Alice”));
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);
bob.addEdge(new Call(timestamp), charlie);



                                                                  Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                     Code (ingest)

                                                             Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);
bob.addEdge(new Call(timestamp), charlie);



                                                                          Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                     Code (ingest)

                                                             Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));                        Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);
bob.addEdge(new Call(timestamp), charlie);



                                                                          Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                     Code (ingest)

                                                             Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));                        Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                            Carlos
bob.addEdge(new Call(timestamp), charlie);



                                                                          Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));                          Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                              Carlos
bob.addEdge(new Call(timestamp), charlie);


                                                             Charlie
                                                                            Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice
                                                                         meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                               Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                   Carlos
bob.addEdge(new Call(timestamp), charlie);


                                                             Charlie
                                                                                 Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice
                                                                         meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                               Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));
                                                                                          calls

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                   Carlos
bob.addEdge(new Call(timestamp), charlie);


                                                             Charlie
                                                                                 Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice
                                                                         meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                               Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));
                                                                                          calls

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                   Carlos
bob.addEdge(new Call(timestamp), charlie);
                                                                         pays


                                                             Charlie
                                                                                 Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                        Code (ingest)

                                                              Alice
                                                                               meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                                     Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));
                                                                       calls                    calls

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                         Carlos
bob.addEdge(new Call(timestamp), charlie);
                                                                               pays


                                                             Charlie
                                                                                       Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (Navigator)

// Create a qualifier that describes the target vertex
Qualifier findCharliePredicate =
	

 new VertexPredicate(personType, "name == ’Charlie'");

// Construct a navigator which starts with Alice and uses a result qualifier
// to find all paths in the graph to Charlie
Navigator charlieFinder = alice.navigate(
	

 Guide.SIMPLE_BREADTH_FIRST,	

 // default guide 	

     Qualifier.ANY, 	

 	

 	

 // no path constraints
	

 findCharliePredicate ,	

	

 // find paths ending with Charlie
     myResultHandler);	

       // fire results to supplied handler

// Start the navigator
charlieFinder.start();
                                                                               Copyright © InfiniteGraph

                                                 27
INFINITEGRAPH
        Visualization




            Copyright © InfiniteGraph

28
INFINITEGRAPH
                                                                                                           Pricing

                                                                                           Price
        Edition             License                Description
                                                                                         (annual)


                                             Complete database but
„InfiniteGraph FREE“           Free            limitation to 1 million                       0 €
                                                 vertices or edges



                                                                               starts at app. 5000 $
   „Pay as you go“         Commercial             No limitation               (depends on count of
                                                                                vertices and edges)




                                               Focus on „bigger“                      >..... €
„Unit or site licensing“   Commercial
                                                 environments                   (No price available)


                                                                    Source: http://objectivity.com/products/infinitegraph/overview

                                        29
30
FALLEN-8
                                                           Overview

•   In-memory graph database

•   Implemented in C# (platform independent because of mono)

•   4 billion vertices or edges, each element can have app. 65000
    properties

•   Indexes on vertices and/or edges

•   Core is open source (MIT-license), plugins can have any license

                                   31
FALLEN-8
                                                          Persistence


•   Persistence in form of „save-points“ (all vertices and edges are
    serialized en bloc)

•   Commodity hardware allows to (de)serialize app. 2 million
    vertices or edges per second

•   Saving blocks only write operations

•   Performance + reliability

                                    32
FALLEN-8
                                    Architecture


        Services

                           Index-
Traversal-framework
                        framework

        Core API

   Vertices and edges

          RAM




           33
FALLEN-8
                         Architecture and some plugins


            HA + ACID Transaktionen

REST API (via JSON) + Management/query frontend

       Traversal-framework            Index-framework
        (incl. path analysis)        (incl R* tree index)

                     Core API

                Vertices and edges

                       RAM


                         34
FALLEN-8
Benchmark - friends of a friend




35
FALLEN-8
    Benchmark - friends of a friend




1




    35
FALLEN-8
        Benchmark - friends of a friend



    2




    3


1


    4




    5




        35
FALLEN-8
        Benchmark - friends of a friend
                    6


                    7


                    8
    2

                    9



    3               10


1
                    11

    4
                    12


                    13
    5

                    14


                    15


                    16
        35
FALLEN-8
                             Benchmark - friends of a friend

                  Fallen-8              Neo4J
          5,000


          4,000


          3,000
t in ms




          2,000


          1,000


             0
                                  run
                                                    Source: Martin Junghanns

                             36
FALLEN-8
Benchmark - traversals per second




                         Source: Sebastian Dechant

   37
FALLEN-8
    Benchmark - traversals per second

1




                             Source: Sebastian Dechant

       37
FALLEN-8
    Benchmark - traversals per second

1




                             Source: Sebastian Dechant

       37
FALLEN-8
    Benchmark - traversals per second

1                     2




                             Source: Sebastian Dechant

       37
FALLEN-8
                                      Benchmark - traversals per second

             1                                                           2



Graph: |V| = 10000, |E| = 600.000 (equally distributed)
System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM




                                                                             Source: Sebastian Dechant

                                               37
FALLEN-8
                                                           Benchmark - traversals per second

                                 1                                                                        2



                    Graph: |V| = 10000, |E| = 600.000 (equally distributed)
                    System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM


     MySQL                                                      62,168
 PostgreSQL                                                      78,449
      Neo4J                                                                  943,580
InfiniteGraph                                                              1,243,084
     Fallen-8                                                                                           196,930,256
                1          10          100         1,000        10,000       100,000        1,000,000    10,000,000   100,000,000 1,000,000,000

                                                                 traversals / sec
                                                                                                                          Source: Sebastian Dechant

                                                                   37
FALLEN-8
                                                      Roadmap


•   Release: July 2012

•   Q3 2012

    •   High availability (MIT) using Amazon EC2

•   2013

    •   Graph partitioning (MIT or ???)

                                      38
THANKS
                          &
                         Q&A
Email: Henning@RauchEntwicklung.biz
Url: http://www.NoSQL-Database.com
Twitter: http://www.twitter.com/cosh23


                                39

More Related Content

What's hot

Intro to Cypher
Intro to CypherIntro to Cypher
Intro to Cypher
Neo4j
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
Tobias Lindaaker
 
NoSQL Graph Databases - Why, When and Where
NoSQL Graph Databases - Why, When and WhereNoSQL Graph Databases - Why, When and Where
NoSQL Graph Databases - Why, When and Where
Eugene Hanikblum
 
Introduction to Graph Database
Introduction to Graph DatabaseIntroduction to Graph Database
Introduction to Graph Database
Eric Lee
 
Redis overview
Redis overviewRedis overview
Redis overview
Ahmad El-khuja
 
Introduction to redis
Introduction to redisIntroduction to redis
Introduction to redis
NexThoughts Technologies
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redis
Zhichao Liang
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
TO THE NEW | Technology
 
Introdução ao Spring Framework MVC
Introdução ao Spring Framework MVCIntrodução ao Spring Framework MVC
Introdução ao Spring Framework MVC
Messias Batista
 
Graph databases
Graph databasesGraph databases
Graph databases
Karol Grzegorczyk
 
Amazon DynamoDB 키 디자인 패턴
Amazon DynamoDB 키 디자인 패턴Amazon DynamoDB 키 디자인 패턴
Amazon DynamoDB 키 디자인 패턴
Amazon Web Services Korea
 
Hive Training -- Motivations and Real World Use Cases
Hive Training -- Motivations and Real World Use CasesHive Training -- Motivations and Real World Use Cases
Hive Training -- Motivations and Real World Use Cases
nzhang
 
MySQL SQL Tutorial
MySQL SQL TutorialMySQL SQL Tutorial
MySQL SQL Tutorial
Chien Chung Shen
 
Evolution of the Graph Schema
Evolution of the Graph SchemaEvolution of the Graph Schema
Evolution of the Graph Schema
Joshua Shinavier
 
PostgreSQL
PostgreSQLPostgreSQL
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
Simplilearn
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
Maarten Smeets
 
Seminar Presentation Hadoop
Seminar Presentation HadoopSeminar Presentation Hadoop
Seminar Presentation Hadoop
Varun Narang
 
Murach: An introduction to web programming with ASP.NET Core MVC
Murach: An introduction to web programming with ASP.NET Core MVCMurach: An introduction to web programming with ASP.NET Core MVC
Murach: An introduction to web programming with ASP.NET Core MVC
MahmoudOHassouna
 
Sql vs NoSQL-Presentation
 Sql vs NoSQL-Presentation Sql vs NoSQL-Presentation
Sql vs NoSQL-Presentation
Shubham Tomar
 

What's hot (20)

Intro to Cypher
Intro to CypherIntro to Cypher
Intro to Cypher
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
 
NoSQL Graph Databases - Why, When and Where
NoSQL Graph Databases - Why, When and WhereNoSQL Graph Databases - Why, When and Where
NoSQL Graph Databases - Why, When and Where
 
Introduction to Graph Database
Introduction to Graph DatabaseIntroduction to Graph Database
Introduction to Graph Database
 
Redis overview
Redis overviewRedis overview
Redis overview
 
Introduction to redis
Introduction to redisIntroduction to redis
Introduction to redis
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redis
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Introdução ao Spring Framework MVC
Introdução ao Spring Framework MVCIntrodução ao Spring Framework MVC
Introdução ao Spring Framework MVC
 
Graph databases
Graph databasesGraph databases
Graph databases
 
Amazon DynamoDB 키 디자인 패턴
Amazon DynamoDB 키 디자인 패턴Amazon DynamoDB 키 디자인 패턴
Amazon DynamoDB 키 디자인 패턴
 
Hive Training -- Motivations and Real World Use Cases
Hive Training -- Motivations and Real World Use CasesHive Training -- Motivations and Real World Use Cases
Hive Training -- Motivations and Real World Use Cases
 
MySQL SQL Tutorial
MySQL SQL TutorialMySQL SQL Tutorial
MySQL SQL Tutorial
 
Evolution of the Graph Schema
Evolution of the Graph SchemaEvolution of the Graph Schema
Evolution of the Graph Schema
 
PostgreSQL
PostgreSQLPostgreSQL
PostgreSQL
 
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
HBase Tutorial For Beginners | HBase Architecture | HBase Tutorial | Hadoop T...
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Seminar Presentation Hadoop
Seminar Presentation HadoopSeminar Presentation Hadoop
Seminar Presentation Hadoop
 
Murach: An introduction to web programming with ASP.NET Core MVC
Murach: An introduction to web programming with ASP.NET Core MVCMurach: An introduction to web programming with ASP.NET Core MVC
Murach: An introduction to web programming with ASP.NET Core MVC
 
Sql vs NoSQL-Presentation
 Sql vs NoSQL-Presentation Sql vs NoSQL-Presentation
Sql vs NoSQL-Presentation
 

Viewers also liked

Solving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDBSolving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDB
MongoDB
 
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سومآموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
faradars
 
An Introduction to MongoDB Compass
An Introduction to MongoDB CompassAn Introduction to MongoDB Compass
An Introduction to MongoDB Compass
MongoDB
 
Data Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data WarehouseData Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data Warehouse
Daniel Upton
 
Data Modeling for NoSQL
Data Modeling for NoSQLData Modeling for NoSQL
Data Modeling for NoSQL
Tony Tam
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
Neo4j
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?
Venu Anuganti
 
RDF and OWL
RDF and OWLRDF and OWL
RDF and OWL
Rachel Lovinger
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
Venu Anuganti
 
Webinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in DocumentsWebinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in Documents
MongoDB
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
Amazon Web Services
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
Amazon Web Services
 

Viewers also liked (12)

Solving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDBSolving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDB
 
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سومآموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
 
An Introduction to MongoDB Compass
An Introduction to MongoDB CompassAn Introduction to MongoDB Compass
An Introduction to MongoDB Compass
 
Data Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data WarehouseData Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data Warehouse
 
Data Modeling for NoSQL
Data Modeling for NoSQLData Modeling for NoSQL
Data Modeling for NoSQL
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?
 
RDF and OWL
RDF and OWLRDF and OWL
RDF and OWL
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
 
Webinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in DocumentsWebinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in Documents
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
 

Recently uploaded

Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 

Recently uploaded (20)

Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 

Graphdatabases

  • 1. GRAPH DATABASES Henning Rauch 1
  • 2. AGENDA • Overview • Neo4J • InfiniteGraph • Fallen-8 2
  • 3. OVERVIEW Why it makes sense to know about graph databases „Graph databases will come into vogue. One key gap in the Hadoop ecosystem is for graph databases, which support rich mining and visualization of relationships, influence, and behavioral propensities. The market for graph databases will boom in 2012 as companies everywhere adopt them for social media analytics, marketing campaign optimization, and customer experience fine-tuning. We will see VCs put big money behind graph database and analytics startups. Many big data platform and tool vendors will acquire the startups to supplement their expanding Hadoop, NoSQL, and enterprise data warehousing (EDW) portfolios. Social graph analysis, although not a brand-new field, will become one of the most prestigious specialties in the data science arena, focusing on high-powered drilldown into polystructured behavioral data sets.“ Source: http://blogs.forrester.com/james_kobielus/11-12-19-the_year_ahead_in_big_data_big_cool_new_stuff_looms_large 3
  • 4. OVERVIEW Example of a real-world graph - facebook Source: http://www.facebook.com/press/info.php?statistics 4
  • 5. OVERVIEW Example of a real-world graph - NYT „Cascade“ Source: http://nytlabs.com/projects/cascade.html 5
  • 6. OVERVIEW Example of a real-world graph - phone bill 6
  • 7. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB 7
  • 8. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Id Name 0 Henning Rauch 1 René Peinl 2 Foo Bar 3 Bruce Schneier 4 Linus Torwalds 7
  • 9. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person 2 Id Name 3 0 Henning Rauch 1 René Peinl 2 Foo Bar 3 Bruce Schneier 4 4 Linus Torwalds 1 0 7
  • 10. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 7
  • 11. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 7
  • 12. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 Tag Id Name 0 .NET 1 Java 2 PKI 3 NoSQL 7
  • 13. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 Tag NoSQL Id Name .NET 0 .NET PKI 1 Java 2 PKI 3 NoSQL 7
  • 14. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 Tag Tags_rel NoSQL Id Name Tag_Id Person_Id Significance .NET 0 .NET 0 0 5 PKI 1 Java 1 1 5 2 PKI 2 1 6 3 NoSQL 2 3 10 3 0 7 3 1 7 7
  • 15. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 10 2 Foo Bar 1 3 5 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 7 4 3 0 5 Tag Tags_rel NoSQL 7 Id Name Tag_Id Person_Id Significance .NET 0 .NET 0 0 5 6 PKI 1 Java 1 1 5 2 PKI 2 1 6 3 NoSQL 2 3 10 3 0 7 3 1 7 7
  • 16. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 10 2 Foo Bar 1 3 5 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 7 4 3 0 5 Tag Tags_rel NoSQL 7 Id Name Tag_Id Person_Id Significance .NET 0 .NET 0 0 5 6 PKI 1 Java 1 1 5 2 PKI 2 1 6 3 NoSQL 2 3 3 0 10 7 vertex = 3 1 7 7 properties + edges
  • 17. OVERVIEW Delimitation to RDBMS - Scalability Knows_rel Id_1 Id_2 • Relation tables act as a global index over linked 1 1 0 2 data 1 1 3 4 0 1 The bigger the relation table the longer it takes to 0 2 • 0 3 get the interesting information (e.g. local 0 3 4 4 neighbourhood of data) 4 3 Tags_rel • Solution of graph databases: Information on Tag_Id 0 Person_Id 0 Significance 5 relationships (aka edges) are stored locally on the 1 1 5 vertex 2 1 6 2 3 10 3 0 7 3 1 7 8
  • 18. OVERVIEW Delimitation to RDBMS - example of complexity • Task: Find the persons that are known to Id 0. Knows_rel • Linear table scan: O(n) Id_1 1 Id_2 0 1 2 Index scan: O(log n) 1 3 • 1 4 0 1 0 2 • Because of the dependency to n RDBMS do not 0 0 3 4 perform well on recursive search algorithms 3 4 4 3 • Graph database solve this task in O(1) 9
  • 19. OVERVIEW Delimitation to other NoSQL products Size > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 20. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 21. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 22. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones Document databases > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 23. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones Document databases Graph databases > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 24. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones Document databases Graph databases In-memory graph databases > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 25. OVERVIEW Graph processing vs. graph database OLAP Graph affine Universal OLTP Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 26. OVERVIEW Graph processing vs. graph database OLAP Graph affine OLTP RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 27. OVERVIEW Graph processing vs. graph database OLAP Graph affine OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 28. OVERVIEW Graph processing vs. graph database Graph affine Pregel OLAP OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 29. OVERVIEW Graph processing vs. graph database Graph affine Graph databases Pregel OLAP OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 30. OVERVIEW Graph processing vs. graph database Graph affine In-memory Graph databases graph database Pregel OLAP OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 31. OVERVIEW Graph databases • Neo4J • HypergraphDB • InfiniteGraph (Objectivity) • DEX • Sones GraphDB • FlockDB (Twitter) • AllegroGraph • Trinity (Microsoft) • OrientDB • Fallen-8 Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 12
  • 32. 13
  • 33. NEO4J Overview • Graph database + Lucene index • ACID (isolation level read committed) • High availability in enterprise edition • 32 billion vertices, 32 billion edges, 64 billion properties • Embedded or via REST-API • Support for the Blueprints project 14
  • 34. NEO4J Architecture Cypher/Gremlin Java/Ruby/.../C# API REST API Core API (Java) Caches (files and objects) HA Record files Transaction-log Disk(s) Source: http://www.slideshare.net/rheehot/eo4j-12713065 15
  • 35. NEO4J knows Example of the on-disk layout Name: Bob Age: 42 Name: Alice Age: 23 knows knows Name: Carol Age: 22 Source: https://github.com/thobe/presentations 16
  • 36. NEO4J knows Example of the on-disk layout Name: Bob Age: 42 Name Name: Alice Bob Name Age: 23 Alice knows Age 42 Age 23 knows Name Carol Name: Carol Age: 22 Age 22 Source: https://github.com/thobe/presentations 16
  • 37. NEO4J knows Example of the on-disk layout SP EP Name: Bob SN EN Age: 42 knows Name Name: Alice SP EP Bob Name Age: 23 SP EP SN EN Alice knows SN EN knows Age knows 42 Age 23 knows Name SP Source Previous Carol SN Source Next Name: Carol EP End Previous Age: 22 EN End Next Age 22 Existent Nonexistent Source: https://github.com/thobe/presentations 16
  • 38. NEO4J knows Example of the on-disk layout SP EP Name: Bob SN EN Age: 42 knows Name Name: Alice SP EP Bob Name Age: 23 SP EP SN EN Alice knows SN EN knows Age knows 42 Age 23 knows Name SP Source Previous Carol SN Source Next Name: Carol EP End Previous Age: 22 EN End Next Age 22 Existent Nonexistent Source: https://github.com/thobe/presentations 16
  • 39. NEO4J In-memory layout (cache) ID Relationship ID refs in: R1 R2 ... Rn Type 1 out R1 R2 ... Rn Vertex ... Grouped by type (type = „knows“) • Transformation of the double linked list (on-disk) in: R1 R2 ... Rn Type n out R1 R2 ... Rn to objects Key 1 Key 2 ... Key n • Increases the traversal Val 1 Val 2 Val n speed ID start end type Edge Key 1 Key 2 ... Key n Val 1 Val 2 Val n Source: https://github.com/thobe/presentations 17
  • 40. NEO4J Traversal • Relationship-expander (delivers edges of a vertex) • Evaluators (evaluate if a vertex is going to be traversed or if it should be taken to the result set) • Projection of the result set (e.g. „take the last vertex of the path“ • Uniqueness level (sets in steps, whether a node could be visited several times) Source: https://github.com/thobe/presentations 18
  • 41. NEO4J Cypher & Gremlin Feature Gremlin Cypher Paradigm Imperative programming Declarative programming •Developed Marko Rodriguez (Tinkerpop) •In-house development Description • •Cypher provides greater opportunities for optimization Based on xpath to describe the traversal •Developed using Groovy •Good for traversals that need back tracking •30-50% faster on „simple“ traversals •Output is a table START me=node:people(name={myname}) MATCH me-[:HAS_CART]->cart-[:CONTAINS_ITEM]->item outE[label=HAS_CART].inV item<-[:PURCHASED]-user-[:PURCHASED]->recommendation .outE[label=CONTAINS_ITEM].inV RETURN recommendation Example .inE[label=PURCHASED].outV .outE[label=PURCHASED].inV START d=node(1), e=node(2) MATCH p = shortestPath( d-[*..15]->e ) RETURN p Source: https://github.com/thobe/presentations 19
  • 42. NEO4J WebAdmin Quelle: http://docs.neo4j.org/chunked/stable/images/operations/webadmin-overview.png 20
  • 43. NEO4J Pricing Price Edition License Description (annual) Complete database Open Source „Community“ including a basic 0 € (GPLv3) management frontend + Monitoring, better „Advanced“ Commercial and AGPL 6,000 € management frontend and support + „Enterprise“ Commercial and AGPL Enterprise frontend, HA and 24,000 € premium support 21
  • 44. 22
  • 45. INFINITEGRAPH Overview • Distributed graph database • Implemented in C++ (APIs in Java, C#, Python, etc.) • Based on Objectivity/DB (distributed object database) • Established 1988 in Sunnyvale, California • Enterprise-customers + US-government • Support for Blueprints 23
  • 46. INFINITEGRAPH Architecture User Apps Blueprints# IG#Core/API# Management# Naviga0on# Session#/#TX# Placement# Configura0on# Extensions# Execu0on# Management# Objec0vity/DB#Distributed#Database# Copyright © InfiniteGraph 24
  • 47. INFINITEGRAPH Ingest AppD2# AppD1# AppD3# (Ingest#V2)# (Ingest#V1)# (Ingest#V3)# # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# Copyright © InfiniteGraph 25
  • 48. INFINITEGRAPH Ingest AppD2# AppD1# AppD3# (Ingest#V2)# (Ingest#V1)# (Ingest#V3)# # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# V2# V3# Copyright © InfiniteGraph 25
  • 49. INFINITEGRAPH Ingest AppD2# App#1% AppD1# App#2% App#3% AppD3# (Ingest#V2)# (Ingest#V )# (E1%2{%V1V21})% (E23{%V2V3})% (Ingest#V3)# % # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# V2# V3# Copyright © InfiniteGraph 25
  • 50. INFINITEGRAPH Ingest AppD2# App#1% AppD1# App#2% App#3% AppD3# (Ingest#V2)# (Ingest#V )# (E1%2{%V1V21})% (E23{%V2V3})% (Ingest#V3)# % # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# E12$ V2# E23$ V3# Copyright © InfiniteGraph 25
  • 51. INFINITEGRAPH Ingest AppD2# App#1% AppD1# App#2% App#3% AppD3# (Ingest#V2)# (Ingest#V )# (E1%2{%V1V21})% (E23{%V2V3})% (Ingest#V3)# % # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# E12$ V2# E23$ V3# Copyright © InfiniteGraph 25
  • 52. INFINITEGRAPH Code (ingest) Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 53. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 54. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 55. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 56. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Charlie Copyright © InfiniteGraph 26
  • 57. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Charlie Copyright © InfiniteGraph 26
  • 58. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); calls alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Charlie Copyright © InfiniteGraph 26
  • 59. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); calls alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); pays Charlie Copyright © InfiniteGraph 26
  • 60. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); calls calls alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); pays Charlie Copyright © InfiniteGraph 26
  • 61. INFINITEGRAPH Code (Navigator) // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, "name == ’Charlie'"); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start(); Copyright © InfiniteGraph 27
  • 62. INFINITEGRAPH Visualization Copyright © InfiniteGraph 28
  • 63. INFINITEGRAPH Pricing Price Edition License Description (annual) Complete database but „InfiniteGraph FREE“ Free limitation to 1 million 0 € vertices or edges starts at app. 5000 $ „Pay as you go“ Commercial No limitation (depends on count of vertices and edges) Focus on „bigger“ >..... € „Unit or site licensing“ Commercial environments (No price available) Source: http://objectivity.com/products/infinitegraph/overview 29
  • 64. 30
  • 65. FALLEN-8 Overview • In-memory graph database • Implemented in C# (platform independent because of mono) • 4 billion vertices or edges, each element can have app. 65000 properties • Indexes on vertices and/or edges • Core is open source (MIT-license), plugins can have any license 31
  • 66. FALLEN-8 Persistence • Persistence in form of „save-points“ (all vertices and edges are serialized en bloc) • Commodity hardware allows to (de)serialize app. 2 million vertices or edges per second • Saving blocks only write operations • Performance + reliability 32
  • 67. FALLEN-8 Architecture Services Index- Traversal-framework framework Core API Vertices and edges RAM 33
  • 68. FALLEN-8 Architecture and some plugins HA + ACID Transaktionen REST API (via JSON) + Management/query frontend Traversal-framework Index-framework (incl. path analysis) (incl R* tree index) Core API Vertices and edges RAM 34
  • 70. FALLEN-8 Benchmark - friends of a friend 1 35
  • 71. FALLEN-8 Benchmark - friends of a friend 2 3 1 4 5 35
  • 72. FALLEN-8 Benchmark - friends of a friend 6 7 8 2 9 3 10 1 11 4 12 13 5 14 15 16 35
  • 73. FALLEN-8 Benchmark - friends of a friend Fallen-8 Neo4J 5,000 4,000 3,000 t in ms 2,000 1,000 0 run Source: Martin Junghanns 36
  • 74. FALLEN-8 Benchmark - traversals per second Source: Sebastian Dechant 37
  • 75. FALLEN-8 Benchmark - traversals per second 1 Source: Sebastian Dechant 37
  • 76. FALLEN-8 Benchmark - traversals per second 1 Source: Sebastian Dechant 37
  • 77. FALLEN-8 Benchmark - traversals per second 1 2 Source: Sebastian Dechant 37
  • 78. FALLEN-8 Benchmark - traversals per second 1 2 Graph: |V| = 10000, |E| = 600.000 (equally distributed) System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM Source: Sebastian Dechant 37
  • 79. FALLEN-8 Benchmark - traversals per second 1 2 Graph: |V| = 10000, |E| = 600.000 (equally distributed) System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM MySQL 62,168 PostgreSQL 78,449 Neo4J 943,580 InfiniteGraph 1,243,084 Fallen-8 196,930,256 1 10 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000 1,000,000,000 traversals / sec Source: Sebastian Dechant 37
  • 80. FALLEN-8 Roadmap • Release: July 2012 • Q3 2012 • High availability (MIT) using Amazon EC2 • 2013 • Graph partitioning (MIT or ???) 38
  • 81. THANKS & Q&A Email: Henning@RauchEntwicklung.biz Url: http://www.NoSQL-Database.com Twitter: http://www.twitter.com/cosh23 39

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. \n
  63. \n
  64. \n
  65. \n
  66. \n
  67. \n
  68. \n
  69. \n
  70. \n
  71. \n
  72. \n
  73. \n
  74. \n
  75. \n
  76. \n
  77. \n
  78. \n
  79. \n
  80. \n
  81. \n
  82. \n
  83. \n
  84. \n
  85. \n
  86. \n
  87. \n
  88. \n
  89. \n
  90. \n
  91. \n
  92. \n
  93. \n
  94. \n
  95. \n
  96. \n
  97. \n
  98. \n
  99. \n
  100. \n
  101. \n
  102. \n
  103. \n
  104. \n
  105. \n
  106. \n
  107. \n
  108. \n
  109. \n
  110. \n
  111. \n
  112. \n
  113. \n
  114. \n
  115. \n
  116. \n
  117. \n
  118. \n
  119. \n
  120. \n
  121. \n
  122. \n
  123. \n
  124. \n
  125. \n
  126. \n
  127. \n
  128. \n
  129. \n
  130. \n
  131. \n
  132. \n
  133. \n
  134. \n
  135. \n
  136. \n
  137. \n
  138. \n
  139. \n
  140. \n
  141. \n
  142. \n
  143. \n
  144. \n
  145. \n
  146. \n
  147. \n
  148. \n
  149. \n
  150. \n
  151. \n
  152. \n
  153. \n
  154. \n
  155. \n
  156. \n
  157. \n
  158. \n
  159. \n
  160. \n
  161. \n
  162. \n
  163. \n
  164. \n
  165. \n
  166. \n
  167. \n
  168. \n
  169. \n
  170. \n
  171. \n
  172. \n
  173. \n
  174. \n
  175. \n
  176. \n
  177. \n
  178. \n
  179. \n
  180. \n
  181. \n
  182. \n
  183. \n
  184. \n
  185. \n
  186. \n
  187. \n
  188. \n
  189. \n
  190. \n
  191. \n
  192. \n
  193. \n
  194. \n
  195. \n
  196. \n
  197. \n
  198. \n
  199. \n
  200. \n
  201. \n
  202. \n