Your SlideShare is downloading. ×
0
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
FluxGraph @ GraphDevRoom
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

FluxGraph @ GraphDevRoom

647

Published on

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
647
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica
  • 2. about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE
  • 3. relational databases ... timeid col1 col2 created_at updated_at1 “ba” “ba” “1-jan-2011” “5-jun-2012”2 “da” “da” “2-jan-2011”3 “boem” “bam” “3-jan-2011” “12-jun-2012”
  • 4. relational databases ... time id col1 col2 created_at updated_at 1 “ba” “ba” “1-jan-2011” “5-jun-2012” 2 “da” “da” “2-jan-2011” 3 “boem” “bam” “3-jan-2011” “12-jun-2012”entity_id property old_value new_value date 1 “col1” “bi” “ba” “5-jun-2012” 2 “col1” “bim” “boem” “1-mar-2012” 2 “col2” “bim” “bam” “12-jun-2012” audit trail
  • 5. graphs ...➡ graphs are continuously changing ...
  • 6. graphs ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 7. graphs ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 8. FluxGraph ...➡ towards a time-aware graph ...
  • 9. FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic
  • 10. FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison
  • 11. travel through timeFluxGraph fg = new FluxGraph();
  • 12. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
  • 13. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...
  • 14. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ... Michael
  • 15. travel through timeFluxGraph fg = new FluxGraph(); Davy kn owVertex davy = fg.addVertex(); sdavy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ...Edge e1 = Michael fg.addEdge(davy, peter,“knows”);
  • 16. travel through time DavyDate checkpoint = new Date(); kn ow s Peter Michael
  • 17. travel through time DavyDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter Michael
  • 18. travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter Michael
  • 19. travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter knEdge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael
  • 20. travel through time by defaulttime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 21. travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);
  • 22. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?
  • 23. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previousVertex previousDavy = davy.getPreviousVersion();
  • 24. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
  • 25. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
  • 26. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();
  • 27. time-scoped iteration➡ When does an element change?
  • 28. time-scoped iteration➡ When does an element change?➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed
  • 29. time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed
  • 30. time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed➡ ... and each element is time-scoped!
  • 31. temporal graph comparisonDavidDavy Davy kn kn ow ow s s Peter what changed? Peterkn ow s Michael Michael current checkpoint
  • 32. temporal graph comparison➡ difference (A , B) = union (A , B) - B
  • 33. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  • 34. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph! David difference ( , )= kn ow s
  • 35. FluxGraph ...➡ available on github http://github.com/datablend/fluxgraph
  • 36. use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking deathpatient patient patient patient patient cancer cancer
  • 37. use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
  • 38. use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked
  • 39. use case: longitudinal patient data➡ get all male non-smokers in 2005
  • 40. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
  • 41. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
  • 42. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();}
  • 43. use case: longitudinal patient data➡ which patients were smoking before 2005?
  • 44. use case: longitudinal patient data➡ which patients were smoking before 2005?boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareElement element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator();
  • 45. use case: longitudinal patient data➡ which patients have cancer in 2010
  • 46. use case: longitudinal patient data➡ which patients have cancer in 2010 Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  • 47. use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  • 48. use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());➡ extract the patients that have an edge to the cancer node
  • 49. gephi plugin for fluxgraph 2010
  • 50. gephi plugin for fluxgraph 2001
  • 51. gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ...1. Kudos to Timmy Storms (@timmystorms)
  • 52. Questions?

×