Neo4j Introduction (for Techies)

1,870 views

Published on

The social graph of Facebook is the most popular application for a graph database. In addition, there are far more exciting applications, such as spatial data, financial trail, indexing, and others. If you combine different graphs, you are able to evaluate those together with the algorithms known from the graph theory. As a graph, a domain can often be easier and more natural designed. This talk introduces the topic of graph databases and shows how to implement mediated models with large, complex and highly connected data with Neo4j. Subsequently, topics like querying, indexing, import / export are considered as well.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,870
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
39
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Neo4j Introduction (for Techies)

  1. 1. Neo4j from the Command Line Perspective (Introduction) JUG MK | Skopje | 02.10.2013
  2. 2. 2 Speaker Profile Patrick Baumgartner •  Senior Software Consultant | Partner @ Swiftmind •  Spring Framework, OSGi, Neo4j •  Spring Trainer, Neo4j Fanboy, Agilista •  Co-Author of „OSGi für Praktiker“ •  Co-Organizer of Neo4j Meetup Zurich •  @patbaumgartner
  3. 3. 3 Swiftmind Your experts for Enterprise Java Areas of expertise •  Java EE •  Spring Framework •  OSGi •  Agile Methodologies •  Software Engineering Best Practices Headquarter •  Zürich, Schweiz •  @swiftmind •  http://www.swiftmind.com
  4. 4. 4 Agenda •  Introduction to Graphs •  Use cases •  Neo4j •  Querying •  Import / Export •  Tools und APIs •  Polyglot Persistence •  Q & A
  5. 5. 5 New Demands on Data Access •  Structured and unstructured data •  Massive amounts of data •  Inexpensive horizontal scaling •  Apps and data in the cloud •  Social network features •  …
  6. 6. 6 New Types of Data Stores
  7. 7. 7 NoSQL Data Volume Data Complexity Key-Value Stores Column Family Document Databases Graph Databases Relational Databases 90% of all use cases
  8. 8. 8 Property Graph Model •  Graphs are made of –  Vertices / Nodes –  Edges / Relationships –  Properties Alice Bob Car name:Bob age:63 sex:male name:Alice age:21 sex:female type:car vendor:tesla type:OWNS type:DRIVES
  9. 9. 9 Where is my graph? Use cases
  10. 10. 10 Social Data Alice Carol Luke Bob Sam Pete KNOWS KNOWS KNOWS KNOWS
  11. 11. 11 Is it the only use case? Social Graph
  12. 12. 12 Spatial Data 1 2 5 3 13 8 ROAD ROAD ROAD type:ROADlength:3km type:Hotel name:Radisson lat:47.452549 long:8.564615 type:Hotel name:Widder lat:47.373517 long:8.539252
  13. 13. 13 Social & Spatial Data Alice Mat 5 Bob 13 8 ROAD ROAD LIKES KNOWS type:Hotel name:Radisson Blu lat:47.452549 long:8.564615
  14. 14. 14 Financial Data Alice Hotel Luke Bank ATM Pete WITHDRAW TRANSFER OWNS
  15. 15. 15 Other graph use cases?
  16. 16. 16 Railroad in Switzerland
  17. 17. 17 Your Business Domain Process KPI Device Activity Function Service CONTAINS DEPENDS
  18. 18. 18 Neo4j The Big Picture
  19. 19. 19 Neo4j – Big Picture •  Most used graph data base –  Robust & Mature: in production 24/7 since 2003 –  Community: Ecosystem with Tools, Bindings, Frameworks •  Licenses –  AGPLv3 Community Edition – OpenSource –  Advanced/Enterprise – for commercial applications •  Neo Technology –  Development & Support –  ~ 50 Persons / 6 Countries / 3 Continents
  20. 20. 20 Features •  Object oriented, flexible network structure •  ACID transactions •  Horizontal scalable •  Java API –  Java, Groovy, Scala, JRuby •  Runtime –  Standalone Server –  Embedded Database •  Disk based storage manager •  Plugin and testing support
  21. 21. 21 How do I find my data? Querying
  22. 22. 22 Graph Querying •  How do I query the right nodes from the graph? –  Tell me all friends of friends of friends who liked the movie XYZ with more than 3 stars –  ... •  Querying methods –  Traversing with Neo4j API –  Traversal descriptions –  Graph algorithms –  Index queries –  Cypher queries
  23. 23. 23 Neo4j API Node bob = ...; for ((Relationship rel: bob.getRelationships(RelTypes.KNOWS)) { Node friend = rel.getEndNode(); System.out.println(friend.getProperty("name")); } •  Simple, but very low level API •  To verbose for complex traversals
  24. 24. 24 Traversal API •  Querying API for graphs •  Describes paths trough the graph •  Call back based •  Fluent API •  Several predefined constructs •  Neo4j implements the following graph algorithms: –  A* –  Dijkstra –  pathsWithLength –  shortestPath –  allSimplePaths –  allPaths
  25. 25. 25 Traversal API – Constructs
  26. 26. 26 Example: Traversal API TraversalDescription directFriends = Traversal.description() .breadthFirst() .relationships(RelTypes.KNOWS) .evaluator(Evaluators.toDepth(1)) .evaluator(Evaluators.excludeStartPosition()); for(Path p : directFriends.traverse(bob)) { System.out.println(p.endNode().getProperty(„firstname“)); }
  27. 27. 27 Index Queries •  Support for node and relationship indices •  Lucene as default implementation node id property value 15 name Alice 15 yearOfBirth 1972 16 name Bob 46 name Carol 46 yearOfBirth 1983
  28. 28. 28 Cypher Query Language •  Graph query language similar to SQL •  Declarative language •  Defines „What“ and not „How“ •  Uses pattern matching •  Support for –  Filters –  Pagination •  Supports CRUD functionality
  29. 29. 29 Cypher Query Language – Elemente Element Description START Start element(s) (index- or id-lookup) MATCH Pattern to find nodes, describes paths (a) –[knows]-> (b) WHERE Result filter (boolean expression) SKIP LIMIT Pagination RETURN Defines return elements ORDER BY Sorting by properties PARAMETERS Parameter-Map, die im Cypher mittels Key oder Position verwendet werden kann CREATE Creates node nodes and relations SET Updates properties on nodes and relations DELETE Deletes unused nodes or relationships
  30. 30. 30 Example: Cypher Query String query = "START person=node(12) “ + “MATCH person-[:KNOWS]-friend “ + “RETURN friend“; ExecutionEngine engine = new ExecutionEngine(graphDb); ExecutionResult result = engine.execute(query); Iterator<Object> column = result.columnAs("friend"); while(column.hasNext()) { Node friendNode = (Node)column.next(); System.out.println(friendNode.getProperty(PROP_FIRSTNAME)); } person friend KNOWS
  31. 31. 31 Example: Cypher Create Query CREATE (carol {name:“Carol“}) return carol; +------------------------+ | carol | +------------------------+ | Node[1]{name:“carol“} | +------------------------+ START carol=node(1) CREATE (bob {name:“Bob"}), (bob)-[:knows]->(carol) return carol; START carol=node(1) CREATE (apple {name:“Apple“,type:“Fruit“}), (carol)-[:owns {number: 5}]->(apple); Carol Bob KNOWS Carol Carol Apple type: Fruit OWNS number: 5
  32. 32. 32 Shipping data Import / Export – Node Manipulation
  33. 33. 33 Node Creation with Java org.neo4j.graphdb.GraphDatabaseService graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH); Transaction tx = graphDb.beginTx(); try { Node peter = createNode(graphDb, “Alice"); Node carol = createNode(graphDb, “Carol"); peter.createRelationshipTo(carol, RelTypes.KNOWS); .. tx.success(); } finally {tx.finish();} public Node createNode(GraphDatabaseService graphDb, String firstname) { Node person = graphDb.createNode(); person.setProperty(„firstname“, firstname); return person; } Alice Carol KNOWS
  34. 34. 34 NeoClipse
  35. 35. 35 Spring Data Neo4j (SDN) •  POJOs mapped as nodes or relationships – type safe •  Works directly Database, typically embedded mode •  Data is fetched very fast and lazy •  Stores everything within a @Transaction •  Uses heavily AspectJ magic to enhance the POJOs •  …
  36. 36. 36 Spring Data Neo4j – Entity _type_: com.example.Person name: "Alice" age: 42 @NodeEntity public class Person { private String name; private int age; // getters/setters… } Person alice = new Person(); alice.setName("Alice"); alice.setAge(42); alice.persist(); Node
  37. 37. 37 Spring Data Neo4j – NodeEntity @NodeEntity public class Person { private String name; private int yearOfBirth; @RelatedTo(type = "KNOWS", direction = Direction.OUTGOING) private Set<Person> knownPersons; public void knows(Person p) { knownPersons.add(p); } public Set<Person> getFriends() { return knownPersons; } } Person alice = ...; alice.knows(bob); alice.knows(carol); Alice Bob KNOWS Carol KNOWS
  38. 38. 38 Spring Data Neo4j – Relationship @RelationshipEntity public class Knows { private int sinceYear; public Knows since(int year) { this.sinceYear = year; return this; } } @NodeEntity public class Person { public Known knows(Person p) { return this.relateTo(p, Knows.class, "KNOWS"); } } Person alice = ...; Person bob ...; alice.knows(bob).since(2012); Alice Bob KNOWS since: 2012
  39. 39. 39 Spring Data Neo4j – Repository public interface PersonRepository extends GraphRepository<Person> { @Query("start person = {0} match ... return ...") Iterable<Product> getOwnedServices(Person person); Iterable<Person> findByName(String name); Iterable<Person> findByNameLike(String name); } @Autowired PersonRepository personRepository; Person alice = personRepository.findByName("Alice");
  40. 40. 40 Thinkerpop Blueprint Thinkerpop Blueprint Extensions •  GraphML Reader/Writer Library •  GraphSon Reader/Writer Library •  GML Reader/Writer Library •  Java Universal Network/ Graph Framework
  41. 41. 41 GraphML •  GraphML (http://graphml.graphdrawing.org/ ) –  Standardized format –  Tool support –  Pure and simple XML <graph id="G" edgedefault="directed"> <node id="n0"/> <node id="n1"/> <edge source="n0" target="n1"/> </graph>
  42. 42. 42 Neo4j Tools •  GraphML –  Gremlin Plugin (no further development) –  Thinkerpop Blueprint Extensions •  CSV –  BatchInserter –  ParallelBatchInserter
  43. 43. 43 Useful tools for your daily work Tools & APIs
  44. 44. 44 Tools & APIs (2) •  Web console –  Data overview –  Querying and visual graph presentation –  „Command line“ •  Neo4j Shell –  Command line –  Sends UNIX-like Commands via RMI (cd, ls, pwd) –  Cypher Queries Support •  REST API –  JSON –  Clients for Java, .NET, PHP, Ruby, … •  Neoclipse –  Visual presentation of the graph –  Filter options for partial graph presentation
  45. 45. 45 What?! Your app uses only one DB? Polyglot Persistence
  46. 46. 46 Polyglot Persistence Web shop application User Sessions Shopping Cart Recommendations Product Catalog Analytics Financial Data Redis Riak Neo4j MongoDB Casandra MySQL
  47. 47. 47 Neo4j on Meetup.com
  48. 48. 48 Q&A
  49. 49. 49 Thank you! Patrick Baumgartner, @patbaumgartner patrick.baumgartner [at] swiftmind [dot] com http://www.swiftmind.com @swiftmind

×