SlideShare a Scribd company logo
FluxGraph: A time-machine for your graphs
                   Davy Suvee
              Michel Van Speybroeck

                Janssen Pharmaceutica
about me

                 who am i ...
                 ➡ working as an it lead / software architect @ janssen pharmaceutica
                   • dealing with big scientific data sets
                   • hands-on expertise in big data and NoSQL technologies



                 ➡ founder of datablend
                   • provide big data and NoSQL consultancy
    Davy Suvee     • share practical knowledge and big data use cases via blog

      @DSUVEE
graphs and time ...
➡ graphs are continuously changing ...
graphs and time ...
       ➡ graphs are continuously changing ...

       ➡ graphs and time ...
          ★ neo-versioning by david montag  1
                                                                              2
          ★ representing time dependent graphs in neo4j by the isi foundation
          ★ modeling a multilevel index in neo4j by peter neubauer 3




1. http://github.com/dmontag/neo4j-versioning   2. http://github.com/ccattuto/neo4j-dynagraph/wiki   3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
graphs and time ...
       ➡ graphs are continuously changing ...

       ➡ graphs and time ...
          ★ neo-versioning by david montag  1
                                                                              2
          ★ representing time dependent graphs in neo4j by the isi foundation
          ★ modeling a multilevel index in neo4j by peter neubauer 3


                                                                            copy and relink semantics
                                                                                                     ๏ graph size
                                                                                                     ๏ object identity
                                                                                                     ๏ mixing data-model and time-model
1. http://github.com/dmontag/neo4j-versioning   2. http://github.com/ccattuto/neo4j-dynagraph/wiki      3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
FluxGraph ...
➡ towards a time-aware graph ...
FluxGraph ...
➡ towards a time-aware graph ...


➡ implement a blueprints-compatible graph on top of Datomic
FluxGraph ...
➡ towards a time-aware graph ...


➡ implement a blueprints-compatible graph on top of Datomic


➡ make FluxGraph fully time-aware
   ★ travel your graph through time
   ★ time-scoped iteration of vertices and edges
   ★ temporal graph comparison
travel through time
FluxGraph fg = new FluxGraph();
travel through time
FluxGraph fg = new FluxGraph();
                                   Davy

Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
travel through time
FluxGraph fg = new FluxGraph();
                                   Davy

Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
                                          Peter
Vertex peter = ...
travel through time
FluxGraph fg = new FluxGraph();
                                   Davy

Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
                                                    Peter
Vertex peter = ...
Vertex michael = ...

                                          Michael
travel through time
FluxGraph fg = new FluxGraph();
                                     Davy




                                                      kn
                                                       ow
Vertex davy = fg.addVertex();




                                                           s
davy.setProperty(“name”,”Davy”);
                                                       Peter
Vertex peter = ...
Vertex michael = ...

Edge e1 =                                   Michael
  fg.addEdge(davy, peter,“knows”);
travel through time

                                Davy
Date checkpoint = new Date();




                                                 kn
                                                  ow
                                                      s
                                                  Peter




                                       Michael
travel through time

                                    David
Date checkpoint = new Date();




                                                      kn
                                                       ow
                                                           s
davy.setProperty(“name”,”David”);                      Peter




                                            Michael
travel through time

                                       David
Date checkpoint = new Date();




                                                         kn
                                                          ow
                                                              s
davy.setProperty(“name”,”David”);                         Peter




                                       kn
Edge e2 =




                                        ow
  fg.addEdge(davy, michael,“knows”);




                                            s
                                               Michael
travel through time
time


                        kn
       Davy                  ow
                                  s



                                      Peter




              Michael
travel through time
time


                        kn
       Davy                  ow
                                  s




                                              checkpoint
                                      Peter




              Michael
travel through time
time


                        kn
       Davy                  ow                            David
                                                           Davy
                                  s




                                                                             kn
                                                                              ow
                                              checkpoint




                                                                                  s
                                      Peter                                   Peter




                                                           kn
                                                            ow
                                                                s
              Michael                                              Michael
travel through time
time


                        kn
       Davy                  ow                            David
                                                           Davy
                                  s




                                                                             kn
                                                                              ow
                                              checkpoint




                                                                                  s



                                                                                      current
                                      Peter                                   Peter




                                                           kn
                                                            ow
                                                                s
              Michael                                              Michael
travel through time                                           by default
time


                        kn
       Davy                  ow                            David
                                                           Davy
                                  s




                                                                             kn
                                                                              ow
                                              checkpoint




                                                                                  s



                                                                                          current
                                      Peter                                   Peter




                                                           kn
                                                            ow
                                                                s
              Michael                                              Michael
travel through time
time


                         kn
       Davy                   ow                            David
                                                            Davy
                                   s




                                                                              kn
                                                                               ow
                                               checkpoint




                                                                                   s



                                                                                       current
                                       Peter                                   Peter




                                                            kn
                                                             ow
                                                                 s
              Michael                                               Michael




                        fg.setCheckpointTime(checkpoint);
time-scoped iteration

       t1




     Davy
time-scoped iteration

       t1               t2


            change



     Davy            Davy’
time-scoped iteration

       t1               t2               t3


            change           change



     Davy            Davy’            Davy’’
time-scoped iteration

       t1               t2               t3                 tcurrrent


            change           change            change



     Davy            Davy’            Davy’’            Davy’’’
time-scoped iteration

         t1               t2               t3                 tcurrrent


              change           change            change



      Davy             Davy’            Davy’’            Davy’’’




  ➡ how to find the version of the vertex you are interested in?
time-scoped iteration
      t1        t2         t3        tcurrrent




    Davy     Davy’      Davy’’   Davy’’’
time-scoped iteration
      t1                 t2                 t3                   tcurrrent




             next              next                next

    Davy              Davy’              Davy’’              Davy’’’
           previous           previous            previous
time-scoped iteration
       t1                 t2                 t3                   tcurrrent




              next              next                next

     Davy              Davy’              Davy’’              Davy’’’
            previous           previous            previous




Vertex previousDavy = davy.getPreviousVersion();
time-scoped iteration
         t1                 t2                 t3                   tcurrrent




                next              next                next

       Davy              Davy’              Davy’’              Davy’’’
              previous           previous            previous




 Vertex previousDavy = davy.getPreviousVersion();
Iterable<Vertex> allDavy = davy.getNextVersions();
time-scoped iteration
            t1                 t2                 t3                   tcurrrent




                   next              next                next

          Davy              Davy’              Davy’’              Davy’’’
                 previous           previous            previous




     Vertex previousDavy = davy.getPreviousVersion();
   Iterable<Vertex> allDavy = davy.getNextVersions();
Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
time-scoped iteration
            t1                 t2                 t3                   tcurrrent




                   next              next                next

          Davy              Davy’              Davy’’              Davy’’’
                 previous           previous            previous




     Vertex previousDavy = davy.getPreviousVersion();
   Iterable<Vertex> allDavy = davy.getNextVersions();
Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
       Interval valid = davy.getTimerInterval();
time-scoped iteration
➡ When does an element change?
time-scoped iteration
➡ When does an element change?


➡ vertex:
   ★ setting or removing a property
   ★ add or remove it from an edge
   ★ being removed
time-scoped iteration
➡ When does an element change?


➡ vertex:                             ➡ edge:
   ★ setting or removing a property      ★ setting or removing a property
   ★ add or remove it from an edge       ★ being removed
   ★ being removed
time-scoped iteration
➡ When does an element change?


➡ vertex:                                ➡ edge:
   ★ setting or removing a property         ★ setting or removing a property
   ★ add or remove it from an edge          ★ being removed
   ★ being removed



➡ ... and each element is time-scoped!
temporal graph comparison

David
Davy                                          Davy




                                                                kn
                     kn




                                                                     ow
                      ow




                                                                      s
                          s
                      Peter   what changed?                          Peter
kn
 ow
     s




        Michael                                      Michael


           current                                      checkpoint
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!                   David




  difference (                  ,          )=




                                                kn
                                                 ow
                                                     s
FluxGraph ...

➡ available on github
      http://github.com/datablend/fluxgraph
use case: longitudinal patient data
    t1        t2        t3        t4        t5




          smoking   smoking             death




patient   patient   patient   patient   patient




                              cancer    cancer
use case: longitudinal patient data

➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
use case: longitudinal patient data

➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)


➡ example analysis:
   ★ if a male patient is no longer smoking in 2005
   ★ what are the chances of getting lung cancer in 2010, comparing
        patients that smoked before 2005
        patients that never smoked
use case: longitudinal patient data
➡ get all male non-smokers in 2005

fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
use case: longitudinal patient data
➡ get all male non-smokers in 2005

fg.setCheckpointTime(new DateTime(2005,12,31).toDate());

Iterator<Vertex> males =
  fg.getVertices("gender", "male").iterator()
use case: longitudinal patient data
➡ get all male non-smokers in 2005

fg.setCheckpointTime(new DateTime(2005,12,31).toDate());

Iterator<Vertex> males =
  fg.getVertices("gender", "male").iterator()

while (males.hasNext()) {
   Vertex p2005 = males.next();
   boolean smoking2005 =
     p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();
}
use case: longitudinal patient data
➡ which patients were smoking before 2005?


boolean smokingBefore2005 =
  ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() {

    public TimeAwareElement filter(TimeAwareVertex element) {
      return element.getEdges(OUT, "smokingStatus").iterator().hasNext()
        ? element : null;
    }

  }).iterator().hasNext();
use case: longitudinal patient data
➡ which patients have cancer in 2010

                                       working set of smokers
 Graph g =
   fg.difference(smokerws,
                 time2010.toDate(),
                 time2005.toDate());
use case: longitudinal patient data
➡ which patients have cancer in 2010

                                       working set of smokers
 Graph g =
   fg.difference(smokerws,
                 time2010.toDate(),
                 time2005.toDate());



➡ extract the patients that have an edge to the cancer node
gephi plugin for fluxgraph   2010
gephi plugin for fluxgraph   2001
gephi plugin for blueprints!

                                          1
     ➡ available on github
      http://github.com/datablend/gephi-blueprints-plugin

     ➡ Support for neo4j, orientdb, dex, rexter, ...

1. Kudos to Timmy Storms (@timmystorms)
Questions?

More Related Content

Viewers also liked

Graph DB + Bioinformatics: Bio4j, recent applications and future directions
Graph DB + Bioinformatics:  Bio4j, recent applications and future directions Graph DB + Bioinformatics:  Bio4j, recent applications and future directions
Graph DB + Bioinformatics: Bio4j, recent applications and future directions Pablo Pareja Tobes
 
Graph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDBGraph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDB
Andrei KUCHARAVY
 
PresentationFinal
PresentationFinalPresentationFinal
PresentationFinalLin Han
 
Cancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren KibbeCancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren Kibbe
imgcommcall
 
Visual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient StratificationVisual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient Stratification
Nils Gehlenborg
 
how to sell
how to sellhow to sell
how to sell
dkhsurvey
 
Linked Cancer Genome Atlas Database
Linked Cancer Genome Atlas DatabaseLinked Cancer Genome Atlas Database
Linked Cancer Genome Atlas Database
Muhammad Saleem
 
LSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries DatasetLSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries Dataset
Muhammad Saleem
 
Introduction to STINGER
Introduction to STINGERIntroduction to STINGER
Introduction to STINGER
robertmccoll
 
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
ipposi
 
City of hope research informatics common data elements
City of hope research informatics common data elementsCity of hope research informatics common data elements
City of hope research informatics common data elements
Abdul-Malik Shakir
 
Patient profiling disaggregating the data
Patient profiling disaggregating the dataPatient profiling disaggregating the data
Patient profiling disaggregating the data
nhsnwHELP
 
Patient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and ManagementPatient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and Management
Tommy Snitz
 
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Emad Shash
 
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
Health IT Conference – iHT2
 
Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role
NHS Improvement
 
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberGraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
Neo4j
 
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
建豪 陳
 

Viewers also liked (20)

Graph DB + Bioinformatics: Bio4j, recent applications and future directions
Graph DB + Bioinformatics:  Bio4j, recent applications and future directions Graph DB + Bioinformatics:  Bio4j, recent applications and future directions
Graph DB + Bioinformatics: Bio4j, recent applications and future directions
 
Graph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDBGraph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDB
 
Malmo 11.11.2008
Malmo 11.11.2008Malmo 11.11.2008
Malmo 11.11.2008
 
PresentationFinal
PresentationFinalPresentationFinal
PresentationFinal
 
Cancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren KibbeCancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren Kibbe
 
Data collection
Data collectionData collection
Data collection
 
Visual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient StratificationVisual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient Stratification
 
how to sell
how to sellhow to sell
how to sell
 
Linked Cancer Genome Atlas Database
Linked Cancer Genome Atlas DatabaseLinked Cancer Genome Atlas Database
Linked Cancer Genome Atlas Database
 
LSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries DatasetLSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries Dataset
 
Introduction to STINGER
Introduction to STINGERIntroduction to STINGER
Introduction to STINGER
 
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
 
City of hope research informatics common data elements
City of hope research informatics common data elementsCity of hope research informatics common data elements
City of hope research informatics common data elements
 
Patient profiling disaggregating the data
Patient profiling disaggregating the dataPatient profiling disaggregating the data
Patient profiling disaggregating the data
 
Patient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and ManagementPatient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and Management
 
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
 
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
 
Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role
 
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberGraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
 
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
 

Recently uploaded

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 

Recently uploaded (20)

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 

FluxGraph: a time-machine for your graphs

  • 1. FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica
  • 2. about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE
  • 3.
  • 4. graphs and time ... ➡ graphs are continuously changing ...
  • 5. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 6. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model 1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 7. FluxGraph ... ➡ towards a time-aware graph ...
  • 8. FluxGraph ... ➡ towards a time-aware graph ... ➡ implement a blueprints-compatible graph on top of Datomic
  • 9. FluxGraph ... ➡ towards a time-aware graph ... ➡ implement a blueprints-compatible graph on top of Datomic ➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison
  • 10. travel through time FluxGraph fg = new FluxGraph();
  • 11. travel through time FluxGraph fg = new FluxGraph(); Davy Vertex davy = fg.addVertex(); davy.setProperty(“name”,”Davy”);
  • 12. travel through time FluxGraph fg = new FluxGraph(); Davy Vertex davy = fg.addVertex(); davy.setProperty(“name”,”Davy”); Peter Vertex peter = ...
  • 13. travel through time FluxGraph fg = new FluxGraph(); Davy Vertex davy = fg.addVertex(); davy.setProperty(“name”,”Davy”); Peter Vertex peter = ... Vertex michael = ... Michael
  • 14. travel through time FluxGraph fg = new FluxGraph(); Davy kn ow Vertex davy = fg.addVertex(); s davy.setProperty(“name”,”Davy”); Peter Vertex peter = ... Vertex michael = ... Edge e1 = Michael fg.addEdge(davy, peter,“knows”);
  • 15. travel through time Davy Date checkpoint = new Date(); kn ow s Peter Michael
  • 16. travel through time David Date checkpoint = new Date(); kn ow s davy.setProperty(“name”,”David”); Peter Michael
  • 17. travel through time David Date checkpoint = new Date(); kn ow s davy.setProperty(“name”,”David”); Peter kn Edge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael
  • 18. travel through time time kn Davy ow s Peter Michael
  • 19. travel through time time kn Davy ow s checkpoint Peter Michael
  • 20. travel through time time kn Davy ow David Davy s kn ow checkpoint s Peter Peter kn ow s Michael Michael
  • 21. travel through time time kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 22. travel through time by default time kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 23. travel through time time kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);
  • 25. time-scoped iteration t1 t2 change Davy Davy’
  • 26. time-scoped iteration t1 t2 t3 change change Davy Davy’ Davy’’
  • 27. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’
  • 28. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?
  • 29. time-scoped iteration t1 t2 t3 tcurrrent Davy Davy’ Davy’’ Davy’’’
  • 30. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous
  • 31. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();
  • 32. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();
  • 33. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions(); Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
  • 34. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions(); Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();
  • 35. time-scoped iteration ➡ When does an element change?
  • 36. time-scoped iteration ➡ When does an element change? ➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed
  • 37. time-scoped iteration ➡ When does an element change? ➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed
  • 38. time-scoped iteration ➡ When does an element change? ➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed ➡ ... and each element is time-scoped!
  • 39. temporal graph comparison David Davy Davy kn kn ow ow s s Peter what changed? Peter kn ow s Michael Michael current checkpoint
  • 40. temporal graph comparison ➡ difference (A , B) = union (A , B) - B ➡ ... as a (immutable) graph!
  • 41. temporal graph comparison ➡ difference (A , B) = union (A , B) - B ➡ ... as a (immutable) graph!
  • 42. temporal graph comparison ➡ difference (A , B) = union (A , B) - B ➡ ... as a (immutable) graph! David difference ( , )= kn ow s
  • 43. FluxGraph ... ➡ available on github http://github.com/datablend/fluxgraph
  • 44. use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking death patient patient patient patient patient cancer cancer
  • 45. use case: longitudinal patient data ➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
  • 46. use case: longitudinal patient data ➡ historical data for 15.000 patients over a period of 10 years (2001- 2010) ➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked
  • 47. use case: longitudinal patient data ➡ get all male non-smokers in 2005 fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
  • 48. use case: longitudinal patient data ➡ get all male non-smokers in 2005 fg.setCheckpointTime(new DateTime(2005,12,31).toDate()); Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
  • 49. use case: longitudinal patient data ➡ get all male non-smokers in 2005 fg.setCheckpointTime(new DateTime(2005,12,31).toDate()); Iterator<Vertex> males = fg.getVertices("gender", "male").iterator() while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext(); }
  • 50. use case: longitudinal patient data ➡ which patients were smoking before 2005? boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator().hasNext();
  • 51. use case: longitudinal patient data ➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  • 52. use case: longitudinal patient data ➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate()); ➡ extract the patients that have an edge to the cancer node
  • 53.
  • 54. gephi plugin for fluxgraph 2010
  • 55. gephi plugin for fluxgraph 2001
  • 56. gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ... 1. Kudos to Timmy Storms (@timmystorms)