Your SlideShare is downloading. ×
11:40 Adopt: Suvee - Fluxgraph
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

11:40 Adopt: Suvee - Fluxgraph

325
views

Published on

A retrospective cohort study is a medical research study in which the patient records of a group of similar individuals are compared for a particular outcome. For instance, a study can try to assess …

A retrospective cohort study is a medical research study in which the patient records of a group of similar individuals are compared for a particular outcome. For instance, a study can try to assess the impact of smoking behavior with respect to getting lung cancer in a group of 40-year old construction workers who also have been exposed to asbestos. As retrospective case studies are historical in nature, researchers require accurate representations of patient records over time in order to correctly assess the importance of particular time-dependent patient characteristics.

During this presentation, we will show how a state-of-the-art Graph Database such as Neo4J can be extended with a set of temporal primitives that effectively aid researchers at gathering the required insights from a set of longitudinal medical records. Graph Databases are the ideal platform to model and store the multi-dimensional data points of the individual patient records and the cohorts to which they are belonging. By introducing a temporal notion within Graph Databases, physicians are given the power to query beyond time boundaries and get historical access to individual patient characteristics or combinations thereof. Patterns for individual patients can be compared and evaluated against the patterns for the cohort.

In order to validate our proposed approach, we have implemented FluxGraph, a proof-of-concept Temporal Graph Database. Being Blueprints-compatible, it should be straightforward to integrate the proposed API changes within mature Graph Database products such as Neo4J. The explicit notion of time, combined with the flexible modelling offered by Graph Databases, provides users with an expressive and powerful data store and analysis platform which is difficult or even impossible to implement with traditional relational database technologies.
Davy Suvée
(IT Lead - Software Architect at Johnson & Johnson)

Davy Suvee is currently working as an IT Lead/Software Architect in the Research and Development IT division of Janssen Pharmaceutica (Johnson & Johnson). Required to work with big and unstructured scientific data sets, Davy gathered hands-on expertise and insights in the best practices on Big Data and NOSQL. He is also the founder of Datablend and frequently blogs about the practical application of various NOSQL technologies.


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
325
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica
  • 2. about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE
  • 3. graphs and time ...➡ graphs are continuously changing ...
  • 4. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 31. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 5. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 6. FluxGraph ...➡ towards a time-aware graph ...
  • 7. FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic
  • 8. FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison
  • 9. travel through timeFluxGraph fg = new FluxGraph();
  • 10. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
  • 11. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...
  • 12. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ... Michael
  • 13. travel through timeFluxGraph fg = new FluxGraph(); Davy kn owVertex davy = fg.addVertex(); sdavy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ...Edge e1 = Michael fg.addEdge(davy, peter,“knows”);
  • 14. travel through time DavyDate checkpoint = new Date(); kn ow s Peter Michael
  • 15. travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter Michael
  • 16. travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter knEdge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael
  • 17. travel through timetime kn Davy ow s Peter Michael
  • 18. travel through timetime kn Davy ow s checkpoint Peter Michael
  • 19. travel through timetime kn Davy ow David Davy s kn ow checkpoint s Peter Peter kn ow s Michael Michael
  • 20. travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 21. travel through time by defaulttime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 22. travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);
  • 23. time-scoped iteration t1 Davy
  • 24. time-scoped iteration t1 t2 change Davy Davy’
  • 25. time-scoped iteration t1 t2 t3 change change Davy Davy’ Davy’’
  • 26. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’
  • 27. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?
  • 28. time-scoped iteration t1 t2 t3 tcurrrent Davy Davy’ Davy’’ Davy’’’
  • 29. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous
  • 30. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previousVertex previousDavy = davy.getPreviousVersion();
  • 31. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
  • 32. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
  • 33. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();
  • 34. time-scoped iteration➡ When does an element change?
  • 35. time-scoped iteration➡ When does an element change?➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed
  • 36. time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed
  • 37. time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed➡ ... and each element is time-scoped!
  • 38. temporal graph comparisonDavidDavy Davy kn kn ow ow s s Peter what changed? Peterkn ow s Michael Michael current checkpoint
  • 39. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  • 40. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  • 41. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph! David difference ( , )= kn ow s
  • 42. FluxGraph ...➡ available on github http://github.com/datablend/fluxgraph
  • 43. use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking deathpatient patient patient patient patient cancer cancer
  • 44. use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
  • 45. use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked
  • 46. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
  • 47. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
  • 48. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();}
  • 49. use case: longitudinal patient data➡ which patients were smoking before 2005?boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator().hasNext();
  • 50. use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  • 51. use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());➡ extract the patients that have an edge to the cancer node
  • 52. gephi plugin for fluxgraph 2010
  • 53. gephi plugin for fluxgraph 2001
  • 54. gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ...1. Kudos to Timmy Storms (@timmystorms)
  • 55. Questions?

×