DevFest Istanbul - a free guided tour of Neo4J

  • 414 views
Uploaded on

2013-11-02 : DevFest Türkiye, Istanbul. …

2013-11-02 : DevFest Türkiye, Istanbul.

Slightly modified version of my previous Neo4J introduction talk about Neo4J in Soft-Shake Event, Geneva, Switzerland.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
414
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
14
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A free guided tour of (Reality)-[:IS_A]->(Graph)
  • 2. What we have always known selection sort (O(n2))
  • 3. What we have always known selection sort (O(n2)) | heap sort (O(n*log(n))
  • 4. What we have always known selection sort (O(n2)) | heap sort (O(n*log(n)) same algorithm, different data structure, better execution time !
  • 5. What we have always known 1 data structure 1 usage
  • 6. One NOSQL lesson? 1 data STORE 1 usage
  • 7. One NOSQL lesson? polyglot persistence, anyone ?
  • 8. ZOOM on Graph Databases graph = nodes/vertices + edges/relationships/arcs
  • 9. Graph DB : a common model property graph = nodes + labeled relationships + K/V pairs
  • 10. Graph DB : a common model property graph = labeledneov2 nodes + labeled relationships + K/V pairs
  • 11. Property Graph DBs lock DB F
  • 12. Property Graph DBs WHY DO THEY KICK ASS?
  • 13. BECAUSE
  • 14. Graph-based computing Intuitive model Expressive querying Powerful analyses
  • 15. Graph-based computing Intuitive model Whiteboard-friendliness Expressive querying Pregel (GOOG), TAO (FB) Powerful analyses Pattern matching, path finding...
  • 16. A glimpse at usecases: mantra RELATIONSHIPS ARE AS IMPORTANT AS E N T I T I E S
  • 17. A glimpse at usecases Recommendations People I may know ex: people known by contacts I have worked with in the past
  • 18. A glimpse at usecases Recommendations People I may know ex: people known by contacts I have worked with in the past Products I should buy ex: people who bought “Twilight” and “Justin Bieber biography” like you also bought “The ultimate emo guide”
  • 19. A glimpse at usecases Recommendations People I may know ex: people known by contacts I have worked with in the past Products I should buy ex: people who bought “Twilight” and “Justin Bieber biography” like you also bought “The ultimate emo guide” Movies I should watch with whom and where...
  • 20. A glimpse at usecases Pattern detection Fraud ex: many IPs from Fraudistan have made a purchase of game X in the last hour
  • 21. A glimpse at usecases Pattern detection Fraud ex: many IPs from Fraudistan have made a purchase of game X in the last hour Disease detection ex: DNA sequencing
  • 22. A glimpse at usecases Pattern detection Fraud ex: many IPs from Fraudistan have made a purchase of game X in the last hour Disease detection ex: DNA sequencing Trend detection ex: the term Flu has been tweeted 789% times more in Guatemala area in the last 24 hours
  • 23. A glimpse at usecases Path finding Genealogy ex: is François Mitterand related to Elizabeth II ? (yes)
  • 24. A glimpse at usecases Path finding Genealogy ex: is François Mitterand related to Elizabeth II ? (yes) Navigation ex: what is the cheapest way to go to a sushi place < 15€ for me (Place de Clichy) and my friend (Place d’ Italie)?
  • 25. A glimpse at usecases Path finding Genealogy ex: is François Mitterand related to Elizabeth II ? (yes) Navigation ex: what is the cheapest way to go to a sushi place < 15€ for me (Place de Clichy) and my friend (Place d’ Italie)? Impact analysis ex: which customers are impacted if network switch XYZ fails?
  • 26. A glimpse at usecases and more... Topological ordering ex: given a set of dependencies, in which order should I include them? Community detection ex: tag clustering on annotated resources to detect groups of interest (targeted advertising) and much more...
  • 27. A glimpse at usecases http://www.opentreeoflife.org/ http://bio4j.com/ http://www.reco4j.org/ http://structr.org/ https://github.com/neo4j-contrib/graphgist/wiki
  • 28. In short GRAPHS ARE EVERYWHERE!
  • 29. THAT’S WHY YOU SHOULD TRY
  • 30. Neo4J - the origins Circa 2000, somewhere in Sweden 2 swedish guys hacking in a garage
  • 31. Neo4J - the origins Dialogue - Man, I cannot stand Informix anymore - Right, we’ve pushed it to the limit - All these documents, these workflows… - Right, it’s way too densely connected. - Connected… connections? CONNECTIONS??
  • 32. Flash-forward: Neo Technology! 1 company Neo Technology 1 main product Neo4J ~50 employees All over the world Sweden, US, Germany, France, Malaysia, NZ...
  • 33. Neo4J - moaar facts & figures Versions 2.0.0.M06 / 1.9.4 Licenses GPL, AGPL, OEM, Commercial 235 nodes 34_359_738_368 235 relationships > 236 properties at most 238 … capacity can be tailored on demand
  • 34. Neo4J anatomy
  • 35. GRAPH ON DISK (roughly) original presentation: http://www.ustream.tv/channel/neo4j
  • 36. CORE API
  • 37. Neo4J anatomy
  • 38. Node CRUD (JVM) GraphDatabaseService graphDB = new TestGraphDatabaseFactory () .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel .label("CHARACTER" )); character.setProperty( "name", "Homer Simpson" ); transaction .success(); }
  • 39. Node CRUD (JVM) officially distributed test version! GraphDatabaseService graphDB = new TestGraphDatabaseFactory() .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel .label("CHARACTER" )); character.setProperty( "name", "Homer Simpson" ); transaction .success(); }
  • 40. Node CRUD (JVM) GraphDatabaseService graphDB = new TestGraphDatabaseFactory () .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel .label("CHARACTER" )); character.setProperty( "name", "Homer Simpson" ); transaction .success(); } transaction is MANDATORY Java 7 required since 2.0
  • 41. Node CRUD (JVM) GraphDatabaseService graphDB = new TestGraphDatabaseFactory () .newImpermanentDatabase(); try (Transaction transaction = graphDB.beginTx()) { Node character =graphDB.createNode( DynamicLabel.label("CHARACTER")); character.setProperty( "name", "Homer Simpson" ); transaction .success(); } labels are a way to semi-structure your nodes (since 2.0)
  • 42. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { for (Node node: graphDB.findNodesByLabelAndProperty( DynamicLabel .label("CHARACTER" ), "name", "Homer Simpson" )) { /* do something very useful */ } transaction .success(); }
  • 43. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { for (Node node: graphDB.findNodesByLabelAndProperty( DynamicLabel .label("CHARACTER" ), "name", "Homer Simpson" )) { /* do something very useful */ } transaction .success(); } Gotchas ● avoid graphDB.findById !!! ● transaction is MANDATORY for reads as well (new in 2.0)
  • 44. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node character = /*lookup*/ ; character.delete(); transaction .success(); }
  • 45. Node CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node character = /*lookup*/ ; character.delete(); transaction .success(); } Gotchas ● no relationships must be attached when transaction commits ● all properties will be automatically removed
  • 46. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node homer = /*lookup*/ ; Node doughnut = /*lookup*/ ; Relationship eating = homer.createRelationshipTo( doughnut, DynamicRelationshipType .withName("LOVES_EATING" ) ); eating.setProperty( "quantity" , Long.MAX_VALUE); transaction .success(); }
  • 47. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node homer = /*lookup*/ ; Node doughnut = /*lookup*/ ; Relationship eating = homer.createRelationshipTo( doughnut, DynamicRelationshipType.withName("LOVES_EATING") ); eating.setProperty( "quantity" , Long.MAX_VALUE); transaction .success(); } unrelated to Node labels
  • 48. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Node homer = /*lookup*/ ; Node doughnut = /*lookup*/ ; Relationship eating = homer.createRelationshipTo( doughnut, DynamicRelationshipType .withName("LOVES_EATING" ) ); eating.setProperty( "quantity" , Long.MAX_VALUE); transaction .success(); } Gotchas ● relationship direction matters at query time ● avoid human-eating doughnuts ;-)
  • 49. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = /*lookup*/ ; relationship .delete(); transaction .success(); }
  • 50. Relationship CRUD (JVM) try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = /*lookup*/ ; relationship .delete(); transaction .success(); } Gotcha ● a write lock is set on the relationship, as well as both start AND end nodes of the relationship
  • 51. Core API Low Level Transactions are ALWAYS required (v2.0) Technical IDs are dangerous (findById) have a look at github.com/sarmbruster/neo4j-uuid SAME capabilities with REST API
  • 52. QUERYING DATA
  • 53. Neo4J anatomy
  • 54. Two strategies IMPERATIVE VERY extensive Totally customizable 100% under your responsability DECLARATIVE VERY intuitive 90% of your needs No free lunch (yet)! Cypher PROFILE on its way
  • 55. Two strategies TRAVERSALS CYPHER QL (/GREMLIN)
  • 56. Traversals DEPTH FIRST BREADTH FIRST
  • 57. Traversals
  • 58. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) .relationships( DynRelType .withName("PARENT_COMMIT" ), INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); }
  • 59. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) lazy traversal .relationships( definition DynRelType.withName("PARENT_COMMIT"), Direction.INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); }
  • 60. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) .relationships( DynRelType.withName("PARENT_COMMIT" ), INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); } start traversal with node
  • 61. Traversal - basic git log try (Transaction transaction = graphDB.beginTx()) { for (Path position : Traversal.description() .depthFirst() .evaluator(toDepth(LOG_DEFAULT_SIZE)) .relationships( DynRelType.withName("PARENT_COMMIT" ), INCOMING ).traverse(headCommit)) { Node currentNode = position.endNode; logs.add(currentNode); } transaction .success(); } keeps track of current position & visited nodes/rels
  • 62. Traversals Extensive but verbose and error-prone WE <3 ASCII ART!
  • 63. Pattern matching and ASCII art WE <3 CYPHER
  • 64. Pattern matching and ASCII art
  • 65. Pattern matching and ASCII art
  • 66. Pattern matching and ASCII art
  • 67. Pattern matching and ASCII art
  • 68. Pattern matching and ASCII art
  • 69. Pattern matching and ASCII art
  • 70. Cypher syntax with <3 Cypher ()-->()
  • 71. Cypher syntax with <3 Cypher (A)-->(B)
  • 72. Cypher syntax with <3 Cypher LOVES (A)-[:LOVES]->(B)
  • 73. Cypher syntax with <3 Cypher (C)<--(A)-->(B)-->(C) A-->B-->C,A-->C
  • 74. Cypher reads START <lookup> (optional) MATCH <pattern> WHERE <filtering> RETURN <expression>
  • 75. Cypher reads MATCH (homer:HUMAN)-[:LOVES_EATING]->(doughnut:FOOD) WHERE homer.name = "Homer Simpson" AND doughnut.brand = "Fattylicious!" RETURN homer
  • 76. Cypher reads MATCH (sugg:CONTACT)-[:IN_CONTACT*2..10]-(me:CONTACT) WHERE me.name = "Florent Biville" AND me <> sugg RETURN me, sugg
  • 77. Cypher reads RULES OF THUMB ● MATCH for results ● use WHERE to filter (WHERE ) a-[:F]->b or NOT(a-[:F]->b) ● favour parameters over literals (exec. plan reuse) ● javacodegeeks.com: “optimizing Neo4j Cypher Queries”
  • 78. Cypher writes CREATE (UNIQUE) <expression> MERGE <expression>
  • 79. Cypher writes CREATE (homer:HUMAN:DAD {name: "Homer Simpson"}) RETURN homer
  • 80. Cypher writes START homer = node:characters("name:Hom*") MATCH (d:JUNK:FOOD) WHERE d.brand = "Fattylicious!" CREATE (homer)-[luv:LOVES_EATING {quantity:∞}]->(d) RETURN luv
  • 81. Cypher writes MERGE (keanu:ACTOR {name:'Keanu Reeves'}) ON CREATE keanu SET keanu.created = timestamp() ON MATCH keanu SET keanu.lastSeen = timestamp() RETURN keanu
  • 82. Cypher - I want moaaar Declarative power Super nice syntax Evolutionary design with MERGE! http://console.neo4j.org to try it out! Cypher will the #1 way to query data!
  • 83. OBJECT-GRAPH MAPPING
  • 84. With...
  • 85. Spring Data History ~2010 Rod Johnson, Scala last poet Emil Eifrem, Neo Tech. founder & CEO
  • 86. Spring Data Familiar model for Spring apps THIN common layer Embraces diversity MongoDB Redis Neo4J ElasticSearch… Current version 2.3.1.RELEASE
  • 87. Vanilla Neo4J repositories with Spring @Repository public class BranchRepository { public Relationship createBranch (Node p, Node c, Map<String,?> props) { try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = p.createRelationshipTo( c, DynamicRelationshipType .name("HAS_BRANCH" ) ); for (Entry<String,?> entry:props.entrySet()) { relationship .setProperty(entry .getKey(), entry .getValue()); } transaction .success(); return relationship; }}}
  • 88. Vanilla Neo4J repositories with Spring @Repository public class BranchRepository { public Relationship createBranch (Node p, Node c, Map<String,?> props) { try (Transaction transaction = graphDB.beginTx()) { Relationship relationship = p.createRelationshipTo( commit, DynamicRelationshipType .name("HAS_BRANCH" ) ); for (Entry<String,?> entry:props.entrySet()) { relationship .setProperty(entry .getKey(), entry .getValue()); } transaction .success(); return relationship; }}}
  • 89. Spring Data Neo4J repositories public interface BranchRepository extends GraphRepository< Branch> { // look ma! no code! }
  • 90. Moaaar Spring Data Neo4J repositories public interface BranchRepository extends GraphRepository< Branch> { Iterable<Branch> findByNameLike (String name); @Query("MATCH (p:PROJECT)-[b:HAS_BRANCH]->(c:COMMIT) RETURN b" ) Page<Branch> lookMaIveGotPages (); Branch findByNameAndCommitIdentifierLike (String name, String commit); }
  • 91. Moaaar Spring Data Neo4J repositories public interface BranchRepository extends GraphRepository< Branch> { Iterable<Branch> findByNameLike (String name); @Query("MATCH (p:PROJECT)-[b:HAS_BRANCH]->(c:COMMIT) RETURN b" ) Cool things Page<Branch> lookMaIveGotPages (); ● Branch findByNameAndCommitIdentifierLike (String boilerplate methods already provided }● name, String commit); you declare methods following a naming convention, Spring Data Neo4J generates the right implementation for ya! ● YOU EXPOSE YOUR DOMAIN, no Nodes, no Relationships!
  • 92. Spring Data Neo4J node entities @NodeEntity public class Person { @GraphId private Long id; @Indexed(indexName = "people", type=FULLTEXT) private String name; @RelatedTo (type="OWNS", enforceTargetType = true) private Car car; @RelatedToVia (type="FRIEND_OF" , direction = Direction.INCOMING) private Iterable<Friendship> friendships; @GraphTraversal (traversal = PeopleTraversalBuilder .class, elementClass = Person.class, params = "persons") private Iterable<Person> people; }
  • 93. Spring Data Neo4J relationship entities @RelationshipEntity(type ="FRIEND_OF" ) public class Friendship { @StartNode private Person person; @EndNode private Dog humansBestFriend; @GraphProperty /* optional here ;-) */ private Date since; /** * moaaaaar properties */ }
  • 94. And much more Neo4jTemplate Geospatial queries Cross-store support Dynamic relationships “Advanced” mapping
  • 95. Conclusion
  • 96. So much to talk about, so little time ● moaaar Cypher ● REST ○ standard API ○ unmanaged extensions ○ streaming ● Tinkerpop abstractions, http://www.tinkerpop.com/ ● dataviz ○ auto : http://linkurio.us/, Neoclipse, Gephi ○ custom : d3.js, sigma.js… ● NeoAAS : http://www.graphenedb.com/, Heroku ● misc. : backup, batch-import, JDBC drivers
  • 97. And one more thing AssertJ-Neo4J 1.0 is coming soon! Fluent test assertions for Neo4J https://github.com/joel-costigliola/assertj-neo4j
  • 98. SORUSU OLAN?
  • 99. Bana ulaşın @fbiville @LateraIThoughts florent.biville.net www.lateral-thoughts.com