SlideShare a Scribd company logo
GRAPH DATABASES
    Henning Rauch
          1
AGENDA


•   Overview

•   Neo4J

•   InfiniteGraph

•   Fallen-8


                   2
OVERVIEW
         Why it makes sense to know about graph databases


„Graph databases will come into vogue. One key gap in the Hadoop
ecosystem is for graph databases, which support rich mining and visualization of
relationships, influence, and behavioral propensities. The market for graph
databases will boom in 2012 as companies everywhere adopt them for social
media analytics, marketing campaign optimization, and customer experience
fine-tuning. We will see VCs put big money behind graph database and analytics
startups. Many big data platform and tool vendors will acquire the startups to
supplement their expanding Hadoop, NoSQL, and enterprise data warehousing
(EDW) portfolios. Social graph analysis, although not a brand-new field, will
become one of the most prestigious specialties in the data science arena,
focusing on high-powered drilldown into polystructured behavioral data sets.“
                              Source: http://blogs.forrester.com/james_kobielus/11-12-19-the_year_ahead_in_big_data_big_cool_new_stuff_looms_large

                                                 3
OVERVIEW
Example of a real-world graph - facebook




                       Source: http://www.facebook.com/press/info.php?statistics

          4
OVERVIEW
Example of a real-world graph - NYT „Cascade“




                               Source: http://nytlabs.com/projects/cascade.html

                5
OVERVIEW
Example of a real-world graph - phone bill




           6
OVERVIEW
Delimitation to RDBMS - property graph
      RDBMS GraphDB




           7
OVERVIEW
                       Delimitation to RDBMS - property graph
                             RDBMS GraphDB
     Person
Id       Name
0     Henning Rauch
1       René Peinl
2        Foo Bar
3     Bruce Schneier
4     Linus Torwalds




                                  7
OVERVIEW
                       Delimitation to RDBMS - property graph
                             RDBMS GraphDB
     Person
                                                  2
Id       Name                                             3
0     Henning Rauch
1       René Peinl
2        Foo Bar
3     Bruce Schneier
                                                              4
4     Linus Torwalds
                                             1




                                                      0




                                  7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                2
Id       Name          Id_1   Id_2                                      3
0     Henning Rauch     1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                            4
4     Linus Torwalds    0      1                           1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                    0




                                                7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                2
Id       Name          Id_1   Id_2                                      3
0     Henning Rauch     1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                            4
4     Linus Torwalds    0      1                           1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                    0




                                                7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                2
Id       Name          Id_1   Id_2                                      3
0     Henning Rauch     1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                            4
4     Linus Torwalds    0      1                           1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                    0



      Tag
Id       Name
0           .NET
1           Java
2           PKI
3        NoSQL



                                                7
OVERVIEW
                                     Delimitation to RDBMS - property graph
                                           RDBMS GraphDB
     Person            Knows_rel
                                                                 2
Id       Name          Id_1   Id_2                                             3
0     Henning Rauch                                 Java
                        1      0
1       René Peinl      1      2
2        Foo Bar        1      3
3     Bruce Schneier    1      4
                                                                                      4
4     Linus Torwalds    0      1                            1
                        0      2
                        0      3
                        0      4
                        3      4
                        4      3                                     0



      Tag                                           NoSQL
Id       Name                                                                      .NET

0           .NET
                                                                         PKI
1           Java
2           PKI
3        NoSQL



                                                7
OVERVIEW
                                                  Delimitation to RDBMS - property graph
                                                                RDBMS GraphDB
     Person                     Knows_rel
                                                                                      2
Id       Name                   Id_1       Id_2                                                     3
0     Henning Rauch                                                      Java
                                 1          0
1       René Peinl               1          2
2        Foo Bar                 1          3
3     Bruce Schneier             1          4
                                                                                                           4
4     Linus Torwalds             0          1                                    1
                                 0          2
                                 0          3
                                 0          4
                                 3          4
                                 4          3                                             0



      Tag                         Tags_rel                               NoSQL
Id       Name          Tag_Id   Person_Id         Significance                                           .NET

0           .NET         0             0               5
                                                                                              PKI
1           Java         1             1               5
2           PKI          2             1               6
3        NoSQL           2             3              10
                         3             0               7
                         3             1               7
                                                                     7
OVERVIEW
                                                  Delimitation to RDBMS - property graph
                                                                RDBMS GraphDB
     Person                     Knows_rel
                                                                                             2
Id       Name                   Id_1       Id_2                                                                3
0     Henning Rauch                                                      Java
                                 1          0
1       René Peinl               1          2
                                                                                                                   10
2        Foo Bar                 1          3
                                                                                5
3     Bruce Schneier             1          4
                                                                                                                           4
4     Linus Torwalds             0          1                                       1
                                 0          2
                                 0          3
                                 0          4
                                 3          4                            7

                                 4          3                                                    0


                                                                                                     5
      Tag                         Tags_rel                               NoSQL           7
Id       Name          Tag_Id   Person_Id         Significance                                                           .NET

0           .NET         0             0               5
                                                                                             6           PKI
1           Java         1             1               5
2           PKI          2             1               6
3        NoSQL           2             3              10
                         3             0               7
                         3             1               7
                                                                     7
OVERVIEW
                                                  Delimitation to RDBMS - property graph
                                                                RDBMS GraphDB
     Person                     Knows_rel
                                                                                              2
Id       Name                   Id_1       Id_2                                                                 3
0     Henning Rauch                                                      Java
                                 1          0
1       René Peinl               1          2
                                                                                                                    10
2        Foo Bar                 1          3
                                                                                5
3     Bruce Schneier             1          4
                                                                                                                            4
4     Linus Torwalds             0          1                                       1
                                 0          2
                                 0          3
                                 0          4
                                 3          4                            7

                                 4          3                                                     0


                                                                                                      5
      Tag                         Tags_rel                               NoSQL            7
Id       Name          Tag_Id   Person_Id         Significance                                                            .NET

0           .NET         0             0               5
                                                                                              6           PKI
1           Java         1             1               5
2           PKI          2             1               6
3        NoSQL           2
                         3
                                       3
                                       0
                                                      10
                                                       7
                                                                                            vertex =
                         3             1               7
                                                                     7                  properties + edges
OVERVIEW
                               Delimitation to RDBMS - Scalability
                                                                    Knows_rel
                                                                    Id_1       Id_2

•   Relation tables act as a global index over linked                1
                                                                     1
                                                                                0
                                                                                2

    data                                                             1
                                                                     1
                                                                                3
                                                                                4
                                                                     0          1


    The bigger the relation table the longer it takes to
                                                                     0          2
•                                                                    0          3


    get the interesting information (e.g. local                      0
                                                                     3
                                                                                4
                                                                                4

    neighbourhood of data)                                           4          3



                                                                     Tags_rel

•   Solution of graph databases: Information on            Tag_Id
                                                             0
                                                                    Person_Id
                                                                           0
                                                                                      Significance
                                                                                           5

    relationships (aka edges) are stored locally on the      1             1               5


    vertex
                                                             2             1               6
                                                             2             3              10
                                                             3             0               7
                                                             3             1               7

                                     8
OVERVIEW
                 Delimitation to RDBMS - example of complexity

•   Task: Find the persons that are known to Id 0.
                                                        Knows_rel

    •   Linear table scan: O(n)                         Id_1
                                                         1
                                                               Id_2
                                                                0
                                                         1      2


        Index scan: O(log n)
                                                         1      3
    •                                                    1      4
                                                         0      1
                                                         0      2

•   Because of the dependency to n RDBMS do not          0
                                                         0
                                                                3
                                                                4

    perform well on recursive search algorithms          3      4
                                                         4      3



•   Graph database solve this task in O(1)

                                    9
OVERVIEW
       Delimitation to other NoSQL products
Size




           > 90% of use cases



                                                                                               Complexity

                                Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                    10
OVERVIEW
                   Delimitation to other NoSQL products
Size
       Key/Value
        stores




                       > 90% of use cases



                                                                                                           Complexity

                                            Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                10
OVERVIEW
                              Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones




                                  > 90% of use cases



                                                                                                                      Complexity

                                                       Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                           10
OVERVIEW
                                  Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones
                              Document
                              databases




                                          > 90% of use cases



                                                                                                                              Complexity

                                                               Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                                   10
OVERVIEW
                                  Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones
                              Document
                              databases


                                                        Graph databases




                                          > 90% of use cases



                                                                                                                                  Complexity

                                                                   Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                                   10
OVERVIEW
                                  Delimitation to other NoSQL products
Size
       Key/Value
        stores
                   Bigtable
                    clones
                              Document
                              databases


                                                        Graph databases
                                                                                                           In-memory
                                                                                                         graph databases




                                          > 90% of use cases



                                                                                                                                  Complexity

                                                                   Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                                   10
OVERVIEW
       Graph processing vs. graph database




OLAP       Graph affine
           Universal                                                              OLTP




                         Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                   11
OVERVIEW
       Graph processing vs. graph database




OLAP       Graph affine                                                               OLTP




                         RDBMS
           Universal




                            Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                   11
OVERVIEW
                Graph processing vs. graph database




OLAP                Graph affine                                                               OLTP


       Hadoop


                                  RDBMS
                    Universal




                                     Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                            11
OVERVIEW
                         Graph processing vs. graph database




                             Graph affine
       Pregel




OLAP                                                                                                   OLTP


                Hadoop


                                           RDBMS
                             Universal




                                              Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                                     11
OVERVIEW
                         Graph processing vs. graph database




                             Graph affine
                                           Graph databases


       Pregel




OLAP                                                                                                      OLTP


                Hadoop


                                           RDBMS
                             Universal




                                                 Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                                     11
OVERVIEW
                         Graph processing vs. graph database




                             Graph affine
                                                                             In-memory
                                           Graph databases
                                                                           graph database


       Pregel




OLAP                                                                                                      OLTP


                Hadoop


                                           RDBMS
                             Universal




                                                 Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx

                                     11
OVERVIEW
                                                                      Graph databases

•   Neo4J                             •   HypergraphDB
•   InfiniteGraph (Objectivity)        •   DEX
•   Sones GraphDB                     •   FlockDB (Twitter)
•   AllegroGraph                      •   Trinity (Microsoft)
•   OrientDB                          •   Fallen-8


                                             Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german

                                 12
13
NEO4J
                                                               Overview

•   Graph database + Lucene index

•   ACID (isolation level read committed)

•   High availability in enterprise edition

•   32 billion vertices, 32 billion edges, 64 billion properties

•   Embedded or via REST-API

•   Support for the Blueprints project
                                      14
NEO4J
                                                                   Architecture


    Cypher/Gremlin                    Java/Ruby/.../C# API

                        REST API

                     Core API (Java)

Caches (files and objects)                     HA

      Record files                       Transaction-log

                            Disk(s)

                                                    Source: http://www.slideshare.net/rheehot/eo4j-12713065

                              15
NEO4J
                  knows
                           Example of the on-disk layout
                                              Name: Bob
                                              Age: 42




Name: Alice
Age: 23
                                             knows




              knows


                                    Name: Carol
                                    Age: 22



                                                  Source: https://github.com/thobe/presentations

                          16
NEO4J
                          knows
                                   Example of the on-disk layout
                                                      Name: Bob
                                                      Age: 42




                                                                          Name
        Name: Alice                                                         Bob
Name    Age: 23
Alice                                                knows
                                                                            Age
                                                                             42
Age
 23
                      knows
                                                               Name
                                                               Carol
                                            Name: Carol
                                            Age: 22
                                                                Age
                                                                 22
                                                          Source: https://github.com/thobe/presentations

                                  16
NEO4J
                                  knows
                                                 Example of the on-disk layout
                                      SP   EP                            Name: Bob
                                      SN EN                              Age: 42
                                       knows



                                                                                             Name
               Name: Alice                           SP   EP                                   Bob
Name           Age: 23
                             SP   EP                 SN EN
Alice                                                                   knows
                             SN EN                    knows                                    Age
                              knows                                                             42
Age
 23
                             knows
                                                                                  Name
 SP Source Previous
                                                                                  Carol
SN Source Next                                                 Name: Carol
EP End Previous                                                Age: 22
EN End Next                                                                        Age
                                                                                    22
        Existent
        Nonexistent                                                          Source: https://github.com/thobe/presentations

                                                16
NEO4J
                                  knows
                                                 Example of the on-disk layout
                                      SP   EP                            Name: Bob
                                      SN EN                              Age: 42
                                       knows



                                                                                             Name
               Name: Alice                           SP   EP                                   Bob
Name           Age: 23
                             SP   EP                 SN EN
Alice                                                                   knows
                             SN EN                    knows                                    Age
                              knows                                                             42
Age
 23
                             knows
                                                                                  Name
 SP Source Previous
                                                                                  Carol
SN Source Next                                                 Name: Carol
EP End Previous                                                Age: 22
EN End Next                                                                        Age
                                                                                    22
        Existent
        Nonexistent                                                          Source: https://github.com/thobe/presentations

                                                16
NEO4J
                                                                       In-memory layout (cache)
         ID
                                Relationship ID refs
                         in:    R1      R2       ...   Rn
         Type 1
                        out     R1       R2      ...   Rn


Vertex        ...        Grouped by type (type = „knows“)
                                                                        •   Transformation of the
                                                                            double linked list (on-disk)
                        in:     R1       R2     ...    Rn
         Type n
                        out     R1       R2     ...    Rn
                                                                            to objects
           Key 1                Key 2            ...          Key n
                                                                        •   Increases the traversal
                Val 1




                                 Val 2




                                                               Val n
                                                                            speed
         ID             start            end           type
Edge
           Key 1                Key 2            ...          Key n
                Val 1




                                 Val 2




                                                               Val n




                                                                                     Source: https://github.com/thobe/presentations

                                                        17
NEO4J
                                                                         Traversal


•   Relationship-expander (delivers edges of a vertex)

•   Evaluators (evaluate if a vertex is going to be traversed or if it
    should be taken to the result set)

•   Projection of the result set (e.g. „take the last vertex of the path“

•   Uniqueness level (sets in steps, whether a node could be visited
    several times)
                                                         Source: https://github.com/thobe/presentations

                                     18
NEO4J
                                                                                     Cypher & Gremlin
 Feature                          Gremlin                                                 Cypher

 Paradigm                 Imperative programming                                  Declarative programming

            •Developed Marko Rodriguez (Tinkerpop)            •In-house development
Description •                                                 •Cypher provides greater opportunities for optimization
              Based on xpath to describe the traversal
            •Developed using Groovy                           •Good for traversals that need back tracking
            •30-50% faster on „simple“ traversals             •Output is a table
                                                              START
                                                               me=node:people(name={myname})
                                                              MATCH
                                                               me-[:HAS_CART]->cart-[:CONTAINS_ITEM]->item
             outE[label=HAS_CART].inV                          item<-[:PURCHASED]-user-[:PURCHASED]->recommendation
              .outE[label=CONTAINS_ITEM].inV                  RETURN recommendation
 Example
              .inE[label=PURCHASED].outV
              .outE[label=PURCHASED].inV                      START
                                                               d=node(1), e=node(2)
                                                              MATCH
                                                               p = shortestPath( d-[*..15]->e )
                                                              RETURN p
                                                                                                  Source: https://github.com/thobe/presentations

                                                         19
NEO4J
                                                   WebAdmin




     Quelle: http://docs.neo4j.org/chunked/stable/images/operations/webadmin-overview.png

20
NEO4J
                                                                               Pricing

                                                                      Price
  Edition            License                    Description
                                                                    (annual)


                                            Complete database
                  Open Source
„Community“                                  including a basic        0 €
                   (GPLv3)
                                           management frontend



                                                    +
                                             Monitoring, better
„Advanced“     Commercial and AGPL                                  6,000 €
                                          management frontend and
                                                 support



                                                    +
„Enterprise“   Commercial and AGPL    Enterprise frontend, HA and   24,000 €
                                           premium support



                                     21
22
INFINITEGRAPH
                                                              Overview

•   Distributed graph database

    •   Implemented in C++ (APIs in Java, C#, Python, etc.)

•   Based on Objectivity/DB (distributed object database)

    •   Established 1988 in Sunnyvale, California

    •   Enterprise-customers + US-government

•   Support for Blueprints
                                      23
INFINITEGRAPH
                                                                      Architecture
               User Apps
                                                     Blueprints#


                                IG#Core/API#


Management#       Naviga0on#                    Session#/#TX#
                                 Placement#                        Configura0on#
 Extensions#      Execu0on#                     Management#



                     Objec0vity/DB#Distributed#Database#




                                                                              Copyright © InfiniteGraph

                                     24
INFINITEGRAPH
                                                                         Ingest
                              AppD2#
   AppD1#                                              AppD3#
                           (Ingest#V2)#
(Ingest#V1)#                                        (Ingest#V3)#
                                #

                          IG#Core/API#


        Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                        Objec@vity/DB#




                                                                   Copyright © InfiniteGraph

                               25
INFINITEGRAPH
                                                                          Ingest
                              AppD2#
   AppD1#                                              AppD3#
                           (Ingest#V2)#
(Ingest#V1)#                                        (Ingest#V3)#
                                #

                          IG#Core/API#


        Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                        Objec@vity/DB#



V1#                             V2#                           V3#



                                                                    Copyright © InfiniteGraph

                               25
INFINITEGRAPH
                                                                            Ingest
                               AppD2#
   App#1%
    AppD1#                     App#2%                    App#3%
                                                         AppD3#
                            (Ingest#V2)#
 (Ingest#V )#
(E1%2{%V1V21})%             (E23{%V2V3})%             (Ingest#V3)#
                                                           %
                                  #

                            IG#Core/API#


          Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                          Objec@vity/DB#



V1#                              V2#                            V3#



                                                                      Copyright © InfiniteGraph

                                25
INFINITEGRAPH
                                                                             Ingest
                               AppD2#
   App#1%
    AppD1#                     App#2%                     App#3%
                                                          AppD3#
                            (Ingest#V2)#
 (Ingest#V )#
(E1%2{%V1V21})%             (E23{%V2V3})%              (Ingest#V3)#
                                                            %
                                  #

                            IG#Core/API#


          Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                          Objec@vity/DB#



V1#               E12$           V2#            E23$             V3#



                                                                       Copyright © InfiniteGraph

                                25
INFINITEGRAPH
                                                                             Ingest
                               AppD2#
   App#1%
    AppD1#                     App#2%                     App#3%
                                                          AppD3#
                            (Ingest#V2)#
 (Ingest#V )#
(E1%2{%V1V21})%             (E23{%V2V3})%              (Ingest#V3)#
                                                            %
                                  #

                            IG#Core/API#


          Standard#Blocking#Ingest/Placement#(MDP#Plugin)#
                          Objec@vity/DB#



V1#               E12$           V2#            E23$             V3#



                                                                       Copyright © InfiniteGraph

                                25
INFINITEGRAPH
                                                             Code (ingest)



Vertex alice = myGraph.addVertex(new Person(“Alice”));
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);
bob.addEdge(new Call(timestamp), charlie);



                                                                  Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                     Code (ingest)

                                                             Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);
bob.addEdge(new Call(timestamp), charlie);



                                                                          Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                     Code (ingest)

                                                             Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));                        Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);
bob.addEdge(new Call(timestamp), charlie);



                                                                          Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                     Code (ingest)

                                                             Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));                        Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                            Carlos
bob.addEdge(new Call(timestamp), charlie);



                                                                          Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice



Vertex alice = myGraph.addVertex(new Person(“Alice”));                          Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                              Carlos
bob.addEdge(new Call(timestamp), charlie);


                                                             Charlie
                                                                            Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice
                                                                         meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                               Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                   Carlos
bob.addEdge(new Call(timestamp), charlie);


                                                             Charlie
                                                                                 Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice
                                                                         meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                               Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));
                                                                                          calls

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                   Carlos
bob.addEdge(new Call(timestamp), charlie);


                                                             Charlie
                                                                                 Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (ingest)

                                                              Alice
                                                                         meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                               Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));
                                                                                          calls

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                   Carlos
bob.addEdge(new Call(timestamp), charlie);
                                                                         pays


                                                             Charlie
                                                                                 Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                        Code (ingest)

                                                              Alice
                                                                               meets



Vertex alice = myGraph.addVertex(new Person(“Alice”));                                     Bob
Vertex bob = myGraph.addVertex(new Person(“Bob”));
Vertex carlos = myGraph.addVertex(new Person(“Carlos”));
Vertex charlie = myGraph.addVertex(new Person(“Charlie”));
                                                                       calls                    calls

alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob);
bob.addEdge(new Call(timestamp), carlos);
carlos.addEdge(new Payment(100000.00), charlie);                                         Carlos
bob.addEdge(new Call(timestamp), charlie);
                                                                               pays


                                                             Charlie
                                                                                       Copyright © InfiniteGraph

                                              26
INFINITEGRAPH
                                                                       Code (Navigator)

// Create a qualifier that describes the target vertex
Qualifier findCharliePredicate =
	

 new VertexPredicate(personType, "name == ’Charlie'");

// Construct a navigator which starts with Alice and uses a result qualifier
// to find all paths in the graph to Charlie
Navigator charlieFinder = alice.navigate(
	

 Guide.SIMPLE_BREADTH_FIRST,	

 // default guide 	

     Qualifier.ANY, 	

 	

 	

 // no path constraints
	

 findCharliePredicate ,	

	

 // find paths ending with Charlie
     myResultHandler);	

       // fire results to supplied handler

// Start the navigator
charlieFinder.start();
                                                                               Copyright © InfiniteGraph

                                                 27
INFINITEGRAPH
        Visualization




            Copyright © InfiniteGraph

28
INFINITEGRAPH
                                                                                                           Pricing

                                                                                           Price
        Edition             License                Description
                                                                                         (annual)


                                             Complete database but
„InfiniteGraph FREE“           Free            limitation to 1 million                       0 €
                                                 vertices or edges



                                                                               starts at app. 5000 $
   „Pay as you go“         Commercial             No limitation               (depends on count of
                                                                                vertices and edges)




                                               Focus on „bigger“                      >..... €
„Unit or site licensing“   Commercial
                                                 environments                   (No price available)


                                                                    Source: http://objectivity.com/products/infinitegraph/overview

                                        29
30
FALLEN-8
                                                           Overview

•   In-memory graph database

•   Implemented in C# (platform independent because of mono)

•   4 billion vertices or edges, each element can have app. 65000
    properties

•   Indexes on vertices and/or edges

•   Core is open source (MIT-license), plugins can have any license

                                   31
FALLEN-8
                                                          Persistence


•   Persistence in form of „save-points“ (all vertices and edges are
    serialized en bloc)

•   Commodity hardware allows to (de)serialize app. 2 million
    vertices or edges per second

•   Saving blocks only write operations

•   Performance + reliability

                                    32
FALLEN-8
                                    Architecture


        Services

                           Index-
Traversal-framework
                        framework

        Core API

   Vertices and edges

          RAM




           33
FALLEN-8
                         Architecture and some plugins


            HA + ACID Transaktionen

REST API (via JSON) + Management/query frontend

       Traversal-framework            Index-framework
        (incl. path analysis)        (incl R* tree index)

                     Core API

                Vertices and edges

                       RAM


                         34
FALLEN-8
Benchmark - friends of a friend




35
FALLEN-8
    Benchmark - friends of a friend




1




    35
FALLEN-8
        Benchmark - friends of a friend



    2




    3


1


    4




    5




        35
FALLEN-8
        Benchmark - friends of a friend
                    6


                    7


                    8
    2

                    9



    3               10


1
                    11

    4
                    12


                    13
    5

                    14


                    15


                    16
        35
FALLEN-8
                             Benchmark - friends of a friend

                  Fallen-8              Neo4J
          5,000


          4,000


          3,000
t in ms




          2,000


          1,000


             0
                                  run
                                                    Source: Martin Junghanns

                             36
FALLEN-8
Benchmark - traversals per second




                         Source: Sebastian Dechant

   37
FALLEN-8
    Benchmark - traversals per second

1




                             Source: Sebastian Dechant

       37
FALLEN-8
    Benchmark - traversals per second

1




                             Source: Sebastian Dechant

       37
FALLEN-8
    Benchmark - traversals per second

1                     2




                             Source: Sebastian Dechant

       37
FALLEN-8
                                      Benchmark - traversals per second

             1                                                           2



Graph: |V| = 10000, |E| = 600.000 (equally distributed)
System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM




                                                                             Source: Sebastian Dechant

                                               37
FALLEN-8
                                                           Benchmark - traversals per second

                                 1                                                                        2



                    Graph: |V| = 10000, |E| = 600.000 (equally distributed)
                    System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM


     MySQL                                                      62,168
 PostgreSQL                                                      78,449
      Neo4J                                                                  943,580
InfiniteGraph                                                              1,243,084
     Fallen-8                                                                                           196,930,256
                1          10          100         1,000        10,000       100,000        1,000,000    10,000,000   100,000,000 1,000,000,000

                                                                 traversals / sec
                                                                                                                          Source: Sebastian Dechant

                                                                   37
FALLEN-8
                                                      Roadmap


•   Release: July 2012

•   Q3 2012

    •   High availability (MIT) using Amazon EC2

•   2013

    •   Graph partitioning (MIT or ???)

                                      38
THANKS
                          &
                         Q&A
Email: Henning@RauchEntwicklung.biz
Url: http://www.NoSQL-Database.com
Twitter: http://www.twitter.com/cosh23


                                39

More Related Content

What's hot

Graph databases
Graph databasesGraph databases
Graph databases
Vinoth Kannan
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
Tobias Lindaaker
 
NOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jNOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4j
Tobias Lindaaker
 
Intro to Neo4j and Graph Databases
Intro to Neo4j and Graph DatabasesIntro to Neo4j and Graph Databases
Intro to Neo4j and Graph Databases
Neo4j
 
Intro to Graphs and Neo4j
Intro to Graphs and Neo4jIntro to Graphs and Neo4j
Intro to Graphs and Neo4j
jexp
 
An Introduction to Graph Databases
An Introduction to Graph DatabasesAn Introduction to Graph Databases
An Introduction to Graph Databases
InfiniteGraph
 
Introducing DocumentDB
Introducing DocumentDB Introducing DocumentDB
Introducing DocumentDB
James Serra
 
HBase
HBaseHBase
Graph database Use Cases
Graph database Use CasesGraph database Use Cases
Graph database Use Cases
Max De Marzi
 
Copy of MongoDB .pptx
Copy of MongoDB .pptxCopy of MongoDB .pptx
Copy of MongoDB .pptx
nehabsairam
 
Gremlin's Graph Traversal Machinery
Gremlin's Graph Traversal MachineryGremlin's Graph Traversal Machinery
Gremlin's Graph Traversal Machinery
Marko Rodriguez
 
Introduction to Graph Databases
Introduction to Graph DatabasesIntroduction to Graph Databases
Introduction to Graph Databases
DataStax
 
Python and MongoDB
Python and MongoDBPython and MongoDB
Python and MongoDB
Christiano Anderson
 
Hive(ppt)
Hive(ppt)Hive(ppt)
Hive(ppt)
Abhinav Tyagi
 
RedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter Cailliau
Redis Labs
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
Mike Dirolf
 
Cross language information retrieval (clir)slide
Cross language information retrieval (clir)slideCross language information retrieval (clir)slide
Cross language information retrieval (clir)slide
Mohd Iqbal Al-farabi
 
Introduction to RDF
Introduction to RDFIntroduction to RDF
Introduction to RDF
Dr Sukhpal Singh Gill
 
Mongo db intro.pptx
Mongo db intro.pptxMongo db intro.pptx
Mongo db intro.pptx
JWORKS powered by Ordina
 
Introduction to column oriented databases
Introduction to column oriented databasesIntroduction to column oriented databases
Introduction to column oriented databases
ArangoDB Database
 

What's hot (20)

Graph databases
Graph databasesGraph databases
Graph databases
 
An overview of Neo4j Internals
An overview of Neo4j InternalsAn overview of Neo4j Internals
An overview of Neo4j Internals
 
NOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4jNOSQLEU - Graph Databases and Neo4j
NOSQLEU - Graph Databases and Neo4j
 
Intro to Neo4j and Graph Databases
Intro to Neo4j and Graph DatabasesIntro to Neo4j and Graph Databases
Intro to Neo4j and Graph Databases
 
Intro to Graphs and Neo4j
Intro to Graphs and Neo4jIntro to Graphs and Neo4j
Intro to Graphs and Neo4j
 
An Introduction to Graph Databases
An Introduction to Graph DatabasesAn Introduction to Graph Databases
An Introduction to Graph Databases
 
Introducing DocumentDB
Introducing DocumentDB Introducing DocumentDB
Introducing DocumentDB
 
HBase
HBaseHBase
HBase
 
Graph database Use Cases
Graph database Use CasesGraph database Use Cases
Graph database Use Cases
 
Copy of MongoDB .pptx
Copy of MongoDB .pptxCopy of MongoDB .pptx
Copy of MongoDB .pptx
 
Gremlin's Graph Traversal Machinery
Gremlin's Graph Traversal MachineryGremlin's Graph Traversal Machinery
Gremlin's Graph Traversal Machinery
 
Introduction to Graph Databases
Introduction to Graph DatabasesIntroduction to Graph Databases
Introduction to Graph Databases
 
Python and MongoDB
Python and MongoDBPython and MongoDB
Python and MongoDB
 
Hive(ppt)
Hive(ppt)Hive(ppt)
Hive(ppt)
 
RedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter Cailliau
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Cross language information retrieval (clir)slide
Cross language information retrieval (clir)slideCross language information retrieval (clir)slide
Cross language information retrieval (clir)slide
 
Introduction to RDF
Introduction to RDFIntroduction to RDF
Introduction to RDF
 
Mongo db intro.pptx
Mongo db intro.pptxMongo db intro.pptx
Mongo db intro.pptx
 
Introduction to column oriented databases
Introduction to column oriented databasesIntroduction to column oriented databases
Introduction to column oriented databases
 

Viewers also liked

Solving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDBSolving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDB
MongoDB
 
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سومآموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
faradars
 
An Introduction to MongoDB Compass
An Introduction to MongoDB CompassAn Introduction to MongoDB Compass
An Introduction to MongoDB Compass
MongoDB
 
Data Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data WarehouseData Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data Warehouse
Daniel Upton
 
Data Modeling for NoSQL
Data Modeling for NoSQLData Modeling for NoSQL
Data Modeling for NoSQL
Tony Tam
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
Neo4j
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?
Venu Anuganti
 
RDF and OWL
RDF and OWLRDF and OWL
RDF and OWL
Rachel Lovinger
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
Venu Anuganti
 
Webinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in DocumentsWebinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in Documents
MongoDB
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
Amazon Web Services
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
Amazon Web Services
 

Viewers also liked (12)

Solving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDBSolving the Disconnected Data Problem in Healthcare Using MongoDB
Solving the Disconnected Data Problem in Healthcare Using MongoDB
 
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سومآموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
آموزش مدیریت بانک اطلاعاتی اوراکل - بخش سوم
 
An Introduction to MongoDB Compass
An Introduction to MongoDB CompassAn Introduction to MongoDB Compass
An Introduction to MongoDB Compass
 
Data Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data WarehouseData Modeling for Integration of NoSQL with a Data Warehouse
Data Modeling for Integration of NoSQL with a Data Warehouse
 
Data Modeling for NoSQL
Data Modeling for NoSQLData Modeling for NoSQL
Data Modeling for NoSQL
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?
 
RDF and OWL
RDF and OWLRDF and OWL
RDF and OWL
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
 
Webinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in DocumentsWebinar: Back to Basics: Thinking in Documents
Webinar: Back to Basics: Thinking in Documents
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
 
Building Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon RedshiftBuilding Your Data Warehouse with Amazon Redshift
Building Your Data Warehouse with Amazon Redshift
 

Recently uploaded

The History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal EmbeddingsThe History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal Embeddings
Zilliz
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
Enterprise Knowledge
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
AmandaCheung15
 
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Zilliz
 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
Priyanka Aash
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
SAI KAILASH R
 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
Zilliz
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
SelfMade bd
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
Retrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with RagasRetrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with Ragas
Zilliz
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 

Recently uploaded (20)

The History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal EmbeddingsThe History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal Embeddings
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
 
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
Retrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with RagasRetrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with Ragas
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 

Graphdatabases

  • 1. GRAPH DATABASES Henning Rauch 1
  • 2. AGENDA • Overview • Neo4J • InfiniteGraph • Fallen-8 2
  • 3. OVERVIEW Why it makes sense to know about graph databases „Graph databases will come into vogue. One key gap in the Hadoop ecosystem is for graph databases, which support rich mining and visualization of relationships, influence, and behavioral propensities. The market for graph databases will boom in 2012 as companies everywhere adopt them for social media analytics, marketing campaign optimization, and customer experience fine-tuning. We will see VCs put big money behind graph database and analytics startups. Many big data platform and tool vendors will acquire the startups to supplement their expanding Hadoop, NoSQL, and enterprise data warehousing (EDW) portfolios. Social graph analysis, although not a brand-new field, will become one of the most prestigious specialties in the data science arena, focusing on high-powered drilldown into polystructured behavioral data sets.“ Source: http://blogs.forrester.com/james_kobielus/11-12-19-the_year_ahead_in_big_data_big_cool_new_stuff_looms_large 3
  • 4. OVERVIEW Example of a real-world graph - facebook Source: http://www.facebook.com/press/info.php?statistics 4
  • 5. OVERVIEW Example of a real-world graph - NYT „Cascade“ Source: http://nytlabs.com/projects/cascade.html 5
  • 6. OVERVIEW Example of a real-world graph - phone bill 6
  • 7. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB 7
  • 8. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Id Name 0 Henning Rauch 1 René Peinl 2 Foo Bar 3 Bruce Schneier 4 Linus Torwalds 7
  • 9. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person 2 Id Name 3 0 Henning Rauch 1 René Peinl 2 Foo Bar 3 Bruce Schneier 4 4 Linus Torwalds 1 0 7
  • 10. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 7
  • 11. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 7
  • 12. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 Tag Id Name 0 .NET 1 Java 2 PKI 3 NoSQL 7
  • 13. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 Tag NoSQL Id Name .NET 0 .NET PKI 1 Java 2 PKI 3 NoSQL 7
  • 14. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 2 Foo Bar 1 3 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 4 3 0 Tag Tags_rel NoSQL Id Name Tag_Id Person_Id Significance .NET 0 .NET 0 0 5 PKI 1 Java 1 1 5 2 PKI 2 1 6 3 NoSQL 2 3 10 3 0 7 3 1 7 7
  • 15. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 10 2 Foo Bar 1 3 5 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 7 4 3 0 5 Tag Tags_rel NoSQL 7 Id Name Tag_Id Person_Id Significance .NET 0 .NET 0 0 5 6 PKI 1 Java 1 1 5 2 PKI 2 1 6 3 NoSQL 2 3 10 3 0 7 3 1 7 7
  • 16. OVERVIEW Delimitation to RDBMS - property graph RDBMS GraphDB Person Knows_rel 2 Id Name Id_1 Id_2 3 0 Henning Rauch Java 1 0 1 René Peinl 1 2 10 2 Foo Bar 1 3 5 3 Bruce Schneier 1 4 4 4 Linus Torwalds 0 1 1 0 2 0 3 0 4 3 4 7 4 3 0 5 Tag Tags_rel NoSQL 7 Id Name Tag_Id Person_Id Significance .NET 0 .NET 0 0 5 6 PKI 1 Java 1 1 5 2 PKI 2 1 6 3 NoSQL 2 3 3 0 10 7 vertex = 3 1 7 7 properties + edges
  • 17. OVERVIEW Delimitation to RDBMS - Scalability Knows_rel Id_1 Id_2 • Relation tables act as a global index over linked 1 1 0 2 data 1 1 3 4 0 1 The bigger the relation table the longer it takes to 0 2 • 0 3 get the interesting information (e.g. local 0 3 4 4 neighbourhood of data) 4 3 Tags_rel • Solution of graph databases: Information on Tag_Id 0 Person_Id 0 Significance 5 relationships (aka edges) are stored locally on the 1 1 5 vertex 2 1 6 2 3 10 3 0 7 3 1 7 8
  • 18. OVERVIEW Delimitation to RDBMS - example of complexity • Task: Find the persons that are known to Id 0. Knows_rel • Linear table scan: O(n) Id_1 1 Id_2 0 1 2 Index scan: O(log n) 1 3 • 1 4 0 1 0 2 • Because of the dependency to n RDBMS do not 0 0 3 4 perform well on recursive search algorithms 3 4 4 3 • Graph database solve this task in O(1) 9
  • 19. OVERVIEW Delimitation to other NoSQL products Size > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 20. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 21. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 22. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones Document databases > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 23. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones Document databases Graph databases > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 24. OVERVIEW Delimitation to other NoSQL products Size Key/Value stores Bigtable clones Document databases Graph databases In-memory graph databases > 90% of use cases Complexity Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 10
  • 25. OVERVIEW Graph processing vs. graph database OLAP Graph affine Universal OLTP Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 26. OVERVIEW Graph processing vs. graph database OLAP Graph affine OLTP RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 27. OVERVIEW Graph processing vs. graph database OLAP Graph affine OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 28. OVERVIEW Graph processing vs. graph database Graph affine Pregel OLAP OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 29. OVERVIEW Graph processing vs. graph database Graph affine Graph databases Pregel OLAP OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 30. OVERVIEW Graph processing vs. graph database Graph affine In-memory Graph databases graph database Pregel OLAP OLTP Hadoop RDBMS Universal Quelle: http://jim.webber.name/2011/08/24/66f1fb4b-83c3-4f52-af40-ee6382ad2155.aspx 11
  • 31. OVERVIEW Graph databases • Neo4J • HypergraphDB • InfiniteGraph (Objectivity) • DEX • Sones GraphDB • FlockDB (Twitter) • AllegroGraph • Trinity (Microsoft) • OrientDB • Fallen-8 Source: http://www.slideshare.net/jexp/neo4j-graph-database-presentation-german 12
  • 32. 13
  • 33. NEO4J Overview • Graph database + Lucene index • ACID (isolation level read committed) • High availability in enterprise edition • 32 billion vertices, 32 billion edges, 64 billion properties • Embedded or via REST-API • Support for the Blueprints project 14
  • 34. NEO4J Architecture Cypher/Gremlin Java/Ruby/.../C# API REST API Core API (Java) Caches (files and objects) HA Record files Transaction-log Disk(s) Source: http://www.slideshare.net/rheehot/eo4j-12713065 15
  • 35. NEO4J knows Example of the on-disk layout Name: Bob Age: 42 Name: Alice Age: 23 knows knows Name: Carol Age: 22 Source: https://github.com/thobe/presentations 16
  • 36. NEO4J knows Example of the on-disk layout Name: Bob Age: 42 Name Name: Alice Bob Name Age: 23 Alice knows Age 42 Age 23 knows Name Carol Name: Carol Age: 22 Age 22 Source: https://github.com/thobe/presentations 16
  • 37. NEO4J knows Example of the on-disk layout SP EP Name: Bob SN EN Age: 42 knows Name Name: Alice SP EP Bob Name Age: 23 SP EP SN EN Alice knows SN EN knows Age knows 42 Age 23 knows Name SP Source Previous Carol SN Source Next Name: Carol EP End Previous Age: 22 EN End Next Age 22 Existent Nonexistent Source: https://github.com/thobe/presentations 16
  • 38. NEO4J knows Example of the on-disk layout SP EP Name: Bob SN EN Age: 42 knows Name Name: Alice SP EP Bob Name Age: 23 SP EP SN EN Alice knows SN EN knows Age knows 42 Age 23 knows Name SP Source Previous Carol SN Source Next Name: Carol EP End Previous Age: 22 EN End Next Age 22 Existent Nonexistent Source: https://github.com/thobe/presentations 16
  • 39. NEO4J In-memory layout (cache) ID Relationship ID refs in: R1 R2 ... Rn Type 1 out R1 R2 ... Rn Vertex ... Grouped by type (type = „knows“) • Transformation of the double linked list (on-disk) in: R1 R2 ... Rn Type n out R1 R2 ... Rn to objects Key 1 Key 2 ... Key n • Increases the traversal Val 1 Val 2 Val n speed ID start end type Edge Key 1 Key 2 ... Key n Val 1 Val 2 Val n Source: https://github.com/thobe/presentations 17
  • 40. NEO4J Traversal • Relationship-expander (delivers edges of a vertex) • Evaluators (evaluate if a vertex is going to be traversed or if it should be taken to the result set) • Projection of the result set (e.g. „take the last vertex of the path“ • Uniqueness level (sets in steps, whether a node could be visited several times) Source: https://github.com/thobe/presentations 18
  • 41. NEO4J Cypher & Gremlin Feature Gremlin Cypher Paradigm Imperative programming Declarative programming •Developed Marko Rodriguez (Tinkerpop) •In-house development Description • •Cypher provides greater opportunities for optimization Based on xpath to describe the traversal •Developed using Groovy •Good for traversals that need back tracking •30-50% faster on „simple“ traversals •Output is a table START me=node:people(name={myname}) MATCH me-[:HAS_CART]->cart-[:CONTAINS_ITEM]->item outE[label=HAS_CART].inV item<-[:PURCHASED]-user-[:PURCHASED]->recommendation .outE[label=CONTAINS_ITEM].inV RETURN recommendation Example .inE[label=PURCHASED].outV .outE[label=PURCHASED].inV START d=node(1), e=node(2) MATCH p = shortestPath( d-[*..15]->e ) RETURN p Source: https://github.com/thobe/presentations 19
  • 42. NEO4J WebAdmin Quelle: http://docs.neo4j.org/chunked/stable/images/operations/webadmin-overview.png 20
  • 43. NEO4J Pricing Price Edition License Description (annual) Complete database Open Source „Community“ including a basic 0 € (GPLv3) management frontend + Monitoring, better „Advanced“ Commercial and AGPL 6,000 € management frontend and support + „Enterprise“ Commercial and AGPL Enterprise frontend, HA and 24,000 € premium support 21
  • 44. 22
  • 45. INFINITEGRAPH Overview • Distributed graph database • Implemented in C++ (APIs in Java, C#, Python, etc.) • Based on Objectivity/DB (distributed object database) • Established 1988 in Sunnyvale, California • Enterprise-customers + US-government • Support for Blueprints 23
  • 46. INFINITEGRAPH Architecture User Apps Blueprints# IG#Core/API# Management# Naviga0on# Session#/#TX# Placement# Configura0on# Extensions# Execu0on# Management# Objec0vity/DB#Distributed#Database# Copyright © InfiniteGraph 24
  • 47. INFINITEGRAPH Ingest AppD2# AppD1# AppD3# (Ingest#V2)# (Ingest#V1)# (Ingest#V3)# # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# Copyright © InfiniteGraph 25
  • 48. INFINITEGRAPH Ingest AppD2# AppD1# AppD3# (Ingest#V2)# (Ingest#V1)# (Ingest#V3)# # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# V2# V3# Copyright © InfiniteGraph 25
  • 49. INFINITEGRAPH Ingest AppD2# App#1% AppD1# App#2% App#3% AppD3# (Ingest#V2)# (Ingest#V )# (E1%2{%V1V21})% (E23{%V2V3})% (Ingest#V3)# % # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# V2# V3# Copyright © InfiniteGraph 25
  • 50. INFINITEGRAPH Ingest AppD2# App#1% AppD1# App#2% App#3% AppD3# (Ingest#V2)# (Ingest#V )# (E1%2{%V1V21})% (E23{%V2V3})% (Ingest#V3)# % # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# E12$ V2# E23$ V3# Copyright © InfiniteGraph 25
  • 51. INFINITEGRAPH Ingest AppD2# App#1% AppD1# App#2% App#3% AppD3# (Ingest#V2)# (Ingest#V )# (E1%2{%V1V21})% (E23{%V2V3})% (Ingest#V3)# % # IG#Core/API# Standard#Blocking#Ingest/Placement#(MDP#Plugin)# Objec@vity/DB# V1# E12$ V2# E23$ V3# Copyright © InfiniteGraph 25
  • 52. INFINITEGRAPH Code (ingest) Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 53. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 54. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 55. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Copyright © InfiniteGraph 26
  • 56. INFINITEGRAPH Code (ingest) Alice Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Charlie Copyright © InfiniteGraph 26
  • 57. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Charlie Copyright © InfiniteGraph 26
  • 58. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); calls alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); Charlie Copyright © InfiniteGraph 26
  • 59. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); calls alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); pays Charlie Copyright © InfiniteGraph 26
  • 60. INFINITEGRAPH Code (ingest) Alice meets Vertex alice = myGraph.addVertex(new Person(“Alice”)); Bob Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); calls calls alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); Carlos bob.addEdge(new Call(timestamp), charlie); pays Charlie Copyright © InfiniteGraph 26
  • 61. INFINITEGRAPH Code (Navigator) // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, "name == ’Charlie'"); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start(); Copyright © InfiniteGraph 27
  • 62. INFINITEGRAPH Visualization Copyright © InfiniteGraph 28
  • 63. INFINITEGRAPH Pricing Price Edition License Description (annual) Complete database but „InfiniteGraph FREE“ Free limitation to 1 million 0 € vertices or edges starts at app. 5000 $ „Pay as you go“ Commercial No limitation (depends on count of vertices and edges) Focus on „bigger“ >..... € „Unit or site licensing“ Commercial environments (No price available) Source: http://objectivity.com/products/infinitegraph/overview 29
  • 64. 30
  • 65. FALLEN-8 Overview • In-memory graph database • Implemented in C# (platform independent because of mono) • 4 billion vertices or edges, each element can have app. 65000 properties • Indexes on vertices and/or edges • Core is open source (MIT-license), plugins can have any license 31
  • 66. FALLEN-8 Persistence • Persistence in form of „save-points“ (all vertices and edges are serialized en bloc) • Commodity hardware allows to (de)serialize app. 2 million vertices or edges per second • Saving blocks only write operations • Performance + reliability 32
  • 67. FALLEN-8 Architecture Services Index- Traversal-framework framework Core API Vertices and edges RAM 33
  • 68. FALLEN-8 Architecture and some plugins HA + ACID Transaktionen REST API (via JSON) + Management/query frontend Traversal-framework Index-framework (incl. path analysis) (incl R* tree index) Core API Vertices and edges RAM 34
  • 70. FALLEN-8 Benchmark - friends of a friend 1 35
  • 71. FALLEN-8 Benchmark - friends of a friend 2 3 1 4 5 35
  • 72. FALLEN-8 Benchmark - friends of a friend 6 7 8 2 9 3 10 1 11 4 12 13 5 14 15 16 35
  • 73. FALLEN-8 Benchmark - friends of a friend Fallen-8 Neo4J 5,000 4,000 3,000 t in ms 2,000 1,000 0 run Source: Martin Junghanns 36
  • 74. FALLEN-8 Benchmark - traversals per second Source: Sebastian Dechant 37
  • 75. FALLEN-8 Benchmark - traversals per second 1 Source: Sebastian Dechant 37
  • 76. FALLEN-8 Benchmark - traversals per second 1 Source: Sebastian Dechant 37
  • 77. FALLEN-8 Benchmark - traversals per second 1 2 Source: Sebastian Dechant 37
  • 78. FALLEN-8 Benchmark - traversals per second 1 2 Graph: |V| = 10000, |E| = 600.000 (equally distributed) System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM Source: Sebastian Dechant 37
  • 79. FALLEN-8 Benchmark - traversals per second 1 2 Graph: |V| = 10000, |E| = 600.000 (equally distributed) System: Windows Server 2008 R2, Intel Xeon E5620 (2,40 GHz), 6 GB RAM MySQL 62,168 PostgreSQL 78,449 Neo4J 943,580 InfiniteGraph 1,243,084 Fallen-8 196,930,256 1 10 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000 1,000,000,000 traversals / sec Source: Sebastian Dechant 37
  • 80. FALLEN-8 Roadmap • Release: July 2012 • Q3 2012 • High availability (MIT) using Amazon EC2 • 2013 • Graph partitioning (MIT or ???) 38
  • 81. THANKS & Q&A Email: Henning@RauchEntwicklung.biz Url: http://www.NoSQL-Database.com Twitter: http://www.twitter.com/cosh23 39

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. \n
  63. \n
  64. \n
  65. \n
  66. \n
  67. \n
  68. \n
  69. \n
  70. \n
  71. \n
  72. \n
  73. \n
  74. \n
  75. \n
  76. \n
  77. \n
  78. \n
  79. \n
  80. \n
  81. \n
  82. \n
  83. \n
  84. \n
  85. \n
  86. \n
  87. \n
  88. \n
  89. \n
  90. \n
  91. \n
  92. \n
  93. \n
  94. \n
  95. \n
  96. \n
  97. \n
  98. \n
  99. \n
  100. \n
  101. \n
  102. \n
  103. \n
  104. \n
  105. \n
  106. \n
  107. \n
  108. \n
  109. \n
  110. \n
  111. \n
  112. \n
  113. \n
  114. \n
  115. \n
  116. \n
  117. \n
  118. \n
  119. \n
  120. \n
  121. \n
  122. \n
  123. \n
  124. \n
  125. \n
  126. \n
  127. \n
  128. \n
  129. \n
  130. \n
  131. \n
  132. \n
  133. \n
  134. \n
  135. \n
  136. \n
  137. \n
  138. \n
  139. \n
  140. \n
  141. \n
  142. \n
  143. \n
  144. \n
  145. \n
  146. \n
  147. \n
  148. \n
  149. \n
  150. \n
  151. \n
  152. \n
  153. \n
  154. \n
  155. \n
  156. \n
  157. \n
  158. \n
  159. \n
  160. \n
  161. \n
  162. \n
  163. \n
  164. \n
  165. \n
  166. \n
  167. \n
  168. \n
  169. \n
  170. \n
  171. \n
  172. \n
  173. \n
  174. \n
  175. \n
  176. \n
  177. \n
  178. \n
  179. \n
  180. \n
  181. \n
  182. \n
  183. \n
  184. \n
  185. \n
  186. \n
  187. \n
  188. \n
  189. \n
  190. \n
  191. \n
  192. \n
  193. \n
  194. \n
  195. \n
  196. \n
  197. \n
  198. \n
  199. \n
  200. \n
  201. \n
  202. \n