DevFest Istanbul - a free guided tour of Neo4J

537
-1

Published on

2013-11-02 : DevFest Türkiye, Istanbul.

Slightly modified version of my previous Neo4J introduction talk about Neo4J in Soft-Shake Event, Geneva, Switzerland.

Published in: Technology, Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
537
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

DevFest Istanbul - a free guided tour of Neo4J

  1. 1. A free guided tour of (Reality)-[:IS_A]->(Graph)
  2. 2. What we have always known selection sort (O(n2))
  3. 3. What we have always known selection sort (O(n2)) | heap sort (O(n*log(n))
  4. 4. What we have always known selection sort (O(n2)) | heap sort (O(n*log(n)) same algorithm, different data structure, better execution time !
  5. 5. What we have always known 1 data structure 1 usage
  6. 6. One NOSQL lesson? 1 data STORE 1 usage
  7. 7. One NOSQL lesson? polyglot persistence, anyone ?
  8. 8. ZOOM on Graph Databases graph = nodes/vertices + edges/relationships/arcs
  9. 9. Graph DB : a common model property graph = nodes + labeled relationships + K/V pairs
  10. 10. Graph DB : a common model property graph = labeledneov2 nodes + labeled relationships + K/V pairs
  11. 11. Property Graph DBs lock DB F
  12. 12. Property Graph DBs WHY DO THEY KICK ASS?
  13. 13. BECAUSE
  14. 14. Graph-based computing Intuitive model Expressive querying Powerful analyses
  15. 15. Graph-based computing Intuitive model Whiteboard-friendliness Expressive querying Pregel (GOOG), TAO (FB) Powerful analyses Pattern matching, path finding...
  16. 16. A glimpse at usecases: mantra RELATIONSHIPS ARE AS IMPORTANT AS E N T I T I E S
  17. 17. A glimpse at usecases Recommendations People I may know ex: people known by contacts I have worked with in the past
  18. 18. A glimpse at usecases Recommendations People I may know ex: people known by contacts I have worked with in the past Products I should buy ex: people who bought “Twilight” and “Justin Bieber biography” like you also bought “The ultimate emo guide”
  19. 19. A glimpse at usecases Recommendations People I may know ex: people known by contacts I have worked with in the past Products I should buy ex: people who bought “Twilight” and “Justin Bieber biography” like you also bought “The ultimate emo guide” Movies I should watch with whom and where...
  20. 20. A glimpse at usecases Pattern detection Fraud ex: many IPs from Fraudistan have made a purchase of game X in the last hour
  21. 21. A glimpse at usecases Pattern detection Fraud ex: many IPs from Fraudistan have made a purchase of game X in the last hour Disease detection ex: DNA sequencing
  22. 22. A glimpse at usecases Pattern detection Fraud ex: many IPs from Fraudistan have made a purchase of game X in the last hour Disease detection ex: DNA sequencing Trend detection ex: the term Flu has been tweeted 789% times more in Guatemala area in the last 24 hours
  23. 23. A glimpse at usecases Path finding Genealogy ex: is François Mitterand related to Elizabeth II ? (yes)
  24. 24. A glimpse at usecases Path finding Genealogy ex: is François Mitterand related to Elizabeth II ? (yes) Navigation ex: what is the cheapest way to go to a sushi place < 15€ for me (Place de Clichy) and my friend (Place d’ Italie)?
  25. 25. A glimpse at usecases Path finding Genealogy ex: is François Mitterand related to Elizabeth II ? (yes) Navigation ex: what is the cheapest way to go to a sushi place < 15€ for me (Place de Clichy) and my friend (Place d’ Italie)? Impact analysis ex: which customers are impacted if network switch XYZ fails?
  26. 26. A glimpse at usecases and more... Topological ordering ex: given a set of dependencies, in which order should I include them? Community detection ex: tag clustering on annotated resources to detect groups of interest (targeted advertising) and much more...
  27. 27. A glimpse at usecases http://www.opentreeoflife.org/ http://bio4j.com/ http://www.reco4j.org/ http://structr.org/ https://github.com/neo4j-contrib/graphgist/wiki
  28. 28. In short GRAPHS ARE EVERYWHERE!
  29. 29. THAT’S WHY YOU SHOULD TRY
  30. 30. Neo4J - the origins Circa 2000, somewhere in Sweden 2 swedish guys hacking in a garage
  31. 31. Neo4J - the origins Dialogue - Man, I cannot stand Informix anymore - Right, we’ve pushed it to the limit - All these documents, these workflows… - Right, it’s way too densely connected. - Connected… connections? CONNECTIONS??
  32. 32. Flash-forward: Neo Technology! 1 company Neo Technology 1 main product Neo4J ~50 employees All over the world Sweden, US, Germany, France, Malaysia, NZ...
  33. 33. Neo4J - moaar facts & figures Versions 2.0.0.M06 / 1.9.4 Licenses GPL, AGPL, OEM, Commercial 235 nodes 34_359_738_368 235 relationships > 236 properties at most 238 … capacity can be tailored on demand
  34. 34. Neo4J anatomy
  35. 35. GRAPH ON DISK (roughly) original presentation: http://www.ustream.tv/channel/neo4j
  36. 36. CORE API
  37. 37. Neo4J anatomy
  38. 38. Node CRUD (JVM) GraphDatabaseService graphDB = new TestGraphDatabaseFactory () .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel .label("CHARACTER" )); character.setProperty( "name", "Homer Simpson" ); transaction .success(); }
  39. 39. Node CRUD (JVM) officially distributed test version! GraphDatabaseService graphDB = new TestGraphDatabaseFactory() .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel .label("CHARACTER" )); character.setProperty( "name", "Homer Simpson" ); transaction .success(); }
  40. 40. Node CRUD (JVM) GraphDatabaseService graphDB = new TestGraphDatabaseFactory () .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel .label("CHARACTER" )); character.setProperty( "name", "Homer Simpson" ); transaction .success(); } transaction is MANDATORY Java 7 required since 2.0
  41. 41. Node CRUD (JVM) GraphDatabaseService graphDB = new TestGraphDatabaseFactory () .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel.label("CHARACTER")); character.setProperty( "name", "Homer Simpson" ); transaction .success(); } labels are a way to semi-structure your nodes (since 2.0)
  42. 42. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { for (Node node: graphDB.findNodesByLabelAndProperty( DynamicLabel .label("CHARACTER" ), "name", "Homer Simpson" )) { /* do something very useful */ } transaction .success(); }
  43. 43. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { for (Node node: graphDB.findNodesByLabelAndProperty( DynamicLabel .label("CHARACTER" ), "name", "Homer Simpson" )) { /* do something very useful */ } transaction .success(); } Gotchas ● avoid graphDB.findById !!! ● transaction is MANDATORY for reads as well (new in 2.0)
  44. 44. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node character = /*lookup*/ ; character.delete(); transaction .success(); }
  45. 45. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node character = /*lookup*/ ; character.delete(); transaction .success(); } Gotchas ● no relationships must be attached when transaction commits ● all properties will be automatically removed
  46. 46. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node homer = /*lookup*/ ; Node doughnut = /*lookup*/ ; Relationship eating = homer.createRelationshipTo( doughnut, DynamicRelationshipType .withName("LOVES_EATING" ) ); eating.setProperty( "quantity" , Long.MAX_VALUE); transaction .success(); }
  47. 47. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node homer = /*lookup*/ ; Node doughnut = /*lookup*/ ; Relationship eating = homer.createRelationshipTo( doughnut, DynamicRelationshipType.withName("LOVES_EATING") ); eating.setProperty( "quantity" , Long.MAX_VALUE); transaction .success(); } unrelated to Node labels
  48. 48. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node homer = /*lookup*/ ; Node doughnut = /*lookup*/ ; Relationship eating = homer.createRelationshipTo( doughnut, DynamicRelationshipType .withName("LOVES_EATING" ) ); eating.setProperty( "quantity" , Long.MAX_VALUE); transaction .success(); } Gotchas ● relationship direction matters at query time ● avoid human-eating doughnuts ;-)
  49. 49. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = /*lookup*/ ; relationship .delete(); transaction .success(); }
  50. 50. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = /*lookup*/ ; relationship .delete(); transaction .success(); } Gotcha ● a write lock is set on the relationship, as well as both start AND end nodes of the relationship
  51. 51. Core API Low Level Transactions are ALWAYS required (v2.0) Technical IDs are dangerous (findById) have a look at github.com/sarmbruster/neo4j-uuid SAME capabilities with REST API
  52. 52. QUERYING DATA
  53. 53. Neo4J anatomy
  54. 54. Two strategies IMPERATIVE VERY extensive Totally customizable 100% under your responsability DECLARATIVE VERY intuitive 90% of your needs No free lunch (yet)! Cypher PROFILE on its way
  55. 55. Two strategies TRAVERSALS CYPHER QL (/GREMLIN)
  56. 56. Traversals DEPTH FIRST BREADTH FIRST
  57. 57. Traversals
  58. 58. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) .relationships( DynRelType .withName("PARENT_COMMIT" ), INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); }
  59. 59. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) lazy traversal .relationships( definition DynRelType.withName("PARENT_COMMIT"), Direction.INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); }
  60. 60. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) .relationships( DynRelType.withName("PARENT_COMMIT" ), INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); } start traversal with node
  61. 61. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) .relationships( DynRelType.withName("PARENT_COMMIT" ), INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); } keeps track of current position & visited nodes/rels
  62. 62. Traversals Extensive but verbose and error-prone WE <3 ASCII ART!
  63. 63. Pattern matching and ASCII art WE <3 CYPHER
  64. 64. Pattern matching and ASCII art
  65. 65. Pattern matching and ASCII art
  66. 66. Pattern matching and ASCII art
  67. 67. Pattern matching and ASCII art
  68. 68. Pattern matching and ASCII art
  69. 69. Pattern matching and ASCII art
  70. 70. Cypher syntax with <3 Cypher ()-->()
  71. 71. Cypher syntax with <3 Cypher (A)-->(B)
  72. 72. Cypher syntax with <3 Cypher LOVES (A)-[:LOVES]->(B)
  73. 73. Cypher syntax with <3 Cypher (C)<--(A)-->(B)-->(C) A-->B-->C,A-->C
  74. 74. Cypher reads START <lookup> (optional) MATCH <pattern> WHERE <filtering> RETURN <expression>
  75. 75. Cypher reads MATCH (homer:HUMAN)-[:LOVES_EATING]->(doughnut:FOOD) WHERE homer.name = "Homer Simpson" AND doughnut.brand = "Fattylicious!" RETURN homer
  76. 76. Cypher reads MATCH (sugg:CONTACT)-[:IN_CONTACT*2..10]-(me:CONTACT) WHERE me.name = "Florent Biville" AND me <> sugg RETURN me, sugg
  77. 77. Cypher reads RULES OF THUMB ● MATCH for results ● use WHERE to filter (WHERE ) a-[:F]->b or NOT(a-[:F]->b) ● favour parameters over literals (exec. plan reuse) ● javacodegeeks.com: “optimizing Neo4j Cypher Queries”
  78. 78. Cypher writes CREATE (UNIQUE) <expression> MERGE <expression>
  79. 79. Cypher writes CREATE (homer:HUMAN:DAD {name: "Homer Simpson"}) RETURN homer
  80. 80. Cypher writes START homer = node:characters("name:Hom*") MATCH (d:JUNK:FOOD) WHERE d.brand = "Fattylicious!" CREATE (homer)-[luv:LOVES_EATING {quantity:∞}]->(d) RETURN luv
  81. 81. Cypher writes MERGE (keanu:ACTOR {name:'Keanu Reeves'}) ON CREATE keanu SET keanu.created = timestamp() ON MATCH keanu SET keanu.lastSeen = timestamp() RETURN keanu
  82. 82. Cypher - I want moaaar Declarative power Super nice syntax Evolutionary design with MERGE! http://console.neo4j.org to try it out! Cypher will the #1 way to query data!
  83. 83. OBJECT-GRAPH MAPPING
  84. 84. With...
  85. 85. Spring Data History ~2010 Rod Johnson, Scala last poet Emil Eifrem, Neo Tech. founder & CEO
  86. 86. Spring Data Familiar model for Spring apps THIN common layer Embraces diversity MongoDB Redis Neo4J ElasticSearch… Current version 2.3.1.RELEASE
  87. 87. Vanilla Neo4J repositories with Spring @Repository public class BranchRepository { public Relationship createBranch (Node p, Node c, Map<String,?> props) { try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = p.createRelationshipTo( c, DynamicRelationshipType .name("HAS_BRANCH" ) ); for (Entry<String,?> entry:props.entrySet()) { relationship .setProperty(entry .getKey(), entry .getValue()); } transaction .success(); return relationship; }}}
  88. 88. Vanilla Neo4J repositories with Spring @Repository public class BranchRepository { public Relationship createBranch (Node p, Node c, Map<String,?> props) { try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = p.createRelationshipTo( commit, DynamicRelationshipType .name("HAS_BRANCH" ) ); for (Entry<String,?> entry:props.entrySet()) { relationship .setProperty(entry .getKey(), entry .getValue()); } transaction .success(); return relationship; }}}
  89. 89. Spring Data Neo4J repositories public interface BranchRepository extends GraphRepository< Branch> { // look ma! no code! }
  90. 90. Moaaar Spring Data Neo4J repositories public interface BranchRepository extends GraphRepository< Branch> { Iterable<Branch> findByNameLike (String name); @Query("MATCH (p:PROJECT)-[b:HAS_BRANCH]->(c:COMMIT) RETURN b" ) Page<Branch> lookMaIveGotPages (); Branch findByNameAndCommitIdentifierLike (String name, String commit); }
  91. 91. Moaaar Spring Data Neo4J repositories public interface BranchRepository extends GraphRepository< Branch> { Iterable<Branch> findByNameLike (String name); @Query("MATCH (p:PROJECT)-[b:HAS_BRANCH]->(c:COMMIT) RETURN b" ) Cool things Page<Branch> lookMaIveGotPages (); ● Branch findByNameAndCommitIdentifierLike (String boilerplate methods already provided }● name, String commit); you declare methods following a naming convention, Spring Data Neo4J generates the right implementation for ya! ● YOU EXPOSE YOUR DOMAIN, no Nodes, no Relationships!
  92. 92. Spring Data Neo4J node entities @NodeEntity public class Person { @GraphId private Long id; @Indexed(indexName = "people", type=FULLTEXT) private String name; @RelatedTo (type="OWNS", enforceTargetType = true) private Car car; @RelatedToVia (type="FRIEND_OF" , direction = Direction.INCOMING) private Iterable<Friendship> friendships; @GraphTraversal (traversal = PeopleTraversalBuilder .class, elementClass = Person.class, params = "persons") private Iterable<Person> people; }
  93. 93. Spring Data Neo4J relationship entities @RelationshipEntity(type ="FRIEND_OF" ) public class Friendship { @StartNode private Person person; @EndNode private Dog humansBestFriend; @GraphProperty /* optional here ;-) */ private Date since; /** * moaaaaar properties */ }
  94. 94. And much more Neo4jTemplate Geospatial queries Cross-store support Dynamic relationships “Advanced” mapping
  95. 95. Conclusion
  96. 96. So much to talk about, so little time ● moaaar Cypher ● REST ○ standard API ○ unmanaged extensions ○ streaming ● Tinkerpop abstractions, http://www.tinkerpop.com/ ● dataviz ○ auto : http://linkurio.us/, Neoclipse, Gephi ○ custom : d3.js, sigma.js… ● NeoAAS : http://www.graphenedb.com/, Heroku ● misc. : backup, batch-import, JDBC drivers
  97. 97. And one more thing AssertJ-Neo4J 1.0 is coming soon! Fluent test assertions for Neo4J https://github.com/joel-costigliola/assertj-neo4j
  98. 98. SORUSU OLAN?
  99. 99. Bana ulaşın @fbiville @LateraIThoughts florent.biville.net www.lateral-thoughts.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×