• Save
Introduction data structure for GraphDB
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Introduction data structure for GraphDB

on

  • 6,118 views

Introduce graphDB structure with Neo4j.

Introduce graphDB structure with Neo4j.

Statistics

Views

Total Views
6,118
Views on SlideShare
6,010
Embed Views
108

Actions

Likes
26
Downloads
0
Comments
4

14 Embeds 108

http://a0.twimg.com 39
http://paper.li 25
http://us-w1.rockmelt.com 18
https://twitter.com 7
http://pdfslidestore.blogspot.com 6
https://si0.twimg.com 3
http://www.linkedin.com 2
https://www.linkedin.com 2
http://littlemisslizzyandfriends.blogspot.de 1
http://alem61mar.blogspot.co.uk 1
http://cordobamediterranea.blogspot.co.il 1
http://alem61mar.blogspot.com 1
http://www.twylah.com 1
http://cordobamediterranea.blogspot.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • thanks
    Are you sure you want to
    Your message goes here
    Processing…
  • Awesome. Let's connect on that!
    Are you sure you want to
    Your message goes here
    Processing…
  • Peter Neubauer,
    Thank you, I feel proud of your comment.

    > Would you mind doing a Neo4j User group in Japan one of these days?

    Yes, I agree to creating a Neo4j user group in Japan.
    I will send you some message after creating plan.
    Are you sure you want to
    Your message goes here
    Processing…
  • Very nice presentation! Would you mind doing a Neo4j User group in Japan one of these days?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Introduction data structure for GraphDB Presentation Transcript

  • 1. Introduction datastructure for GraphDB mixi.inc Shunya Kimura @kimuras
  • 2. agenda• Motivation• Introduction• Structure for Node• Create Node• Appendix
  • 3. Motivation
  • 4. • Just interesting in GraphDB data structure• Why fast• Mysterious
  • 5. Introduction
  • 6. Neo4j• OSS implement of GraphDB• High availability• Scales to billions of nods and relationships• High speed querying through traversals• For more Information about Neo4j • http://www.slideshare.net/skimura/
  • 7. Introduction for Graph Node Name: Kimura Job: Eng Relationship
  • 8. Introduction for Graph Node Name: Kimura Job: Eng Relationship
  • 9. Introduction for Graph Node Data Structure Create Node Relationship
  • 10. Structure for Node
  • 11. A node data structureNode Manager
  • 12. A node data structureNode Manager Node Cache
  • 13. A node data structureNode Manager Node Cache Relation Cache
  • 14. A node data structureNode Manager Node Cache Relation Cache Cache Manager
  • 15. A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Manager
  • 16. A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Manager
  • 17. A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager
  • 18. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager id id id idrelationships relationships relationships relationships Node Node Node Node
  • 19. Node Manager• NodeCache: Caching node id and node object• RelationCache: Caching rel id and Relation object• Cache Manager: Management cache threads• Property Index Manager: Management all nodes Properties• Persistence Manager: Management store
  • 20. Relationship structure id relationships Node
  • 21. Relationship structure idrelationships Node
  • 22. Relationship structure Array structure for relationships idrelationships Node
  • 23. Relationship structure relationships In Blocks Out Blocks Node
  • 24. Relationship structureNode Node relationships In BlocksNode Node Out Blocks NodeNode NodeNode Node
  • 25. Relationship structure In DirectionNode Node relationships In BlocksNode Node Out Blocks NodeNode NodeNode Node
  • 26. Relationship structure In DirectionNode Node relationships In BlocksNode Node Out Blocks NodeNode Node Out DirectionNode Node
  • 27. Relationship structure Node id relationships
  • 28. Relationship structure Node id relationshipsrelationships
  • 29. Relationship structure Node id relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9
  • 30. Relationship structure Node id relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
  • 31. Relationship structure Node id Last In block relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
  • 32. Relationship structure Node id Last In block relationships Last Out blockrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
  • 33. Create Nodes
  • 34. Create Node• Understanding data structure with creating nodes • Create Node A • Create Node B • Create Relationship A to B
  • 35. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS );
  • 36. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA
  • 37. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
  • 38. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
  • 39. Introduction Create Node
  • 40. Introduction Create Node NodeA
  • 41. Introduction Create Node Node Manager Node Cache Persistence Manager Relation Cache NodeA
  • 42. Introduction Create Node Create New ID id Node Manager Node Cache Persistence Manager Relation Cache NodeA
  • 43. Introduction Create Node Create New ID id Create Node Node Manager id Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
  • 44. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
  • 45. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
  • 46. Cache• You can choice some cache type • Strong Reference Cache • Soft LRU Cache (default) • Weak LRU Cache • No Cache
  • 47. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms
  • 48. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 3 VAL_3 4 VAL_4 5 VAL_5
  • 49. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
  • 50. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEGET KEY:5 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
  • 51. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEGET KEY:5 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
  • 52. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
  • 53. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEPUT KEY:6 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
  • 54. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEPUT KEY:6 6 VAL_6 5 VAL_5 Priority 1 VAL_1 2 VAL_2 3 VAL_3
  • 55. Soft LRU Cache
  • 56. Soft LRU Cache Soft LRU CacheCache (ConcurrentHashMap)Queue (SoftReferenceQueue)
  • 57. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3Cache (ConcurrentHashMap)Queue (SoftReferenceQueue)
  • 58. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memorydemand.http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
  • 59. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memorydemand.http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
  • 60. Comparing Cache type Cache Type Queue Type Reference Type Weak LRU Cache WeakReferenceQueue WeakReference Soft LRU Cache SoftRefrenceQueue SoftReference Strong Reference None None CacheRemaining Reference on memoryStrong > Soft > Weak
  • 61. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
  • 62. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA NodeB
  • 63. Introduction Create Node NodeA NodeB
  • 64. Introduction Create Node NodeA NodeB
  • 65. Introduction Create Node id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • 66. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeBPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • 67. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeBPersistence Manager id Relation Cache relationships Register IN direction from NodeA NodeB NodeA NodeB
  • 68. Introduction Create Node id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • 69. Introduction Create Node Create New Relation ID rel id id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • 70. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA RelationPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • 71. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA RelationPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • 72. Appendix
  • 73. Implemented Graph Algorithms• Centrality • Betweenness Centrality • Closeness Centrality• Path • A* • Dijkstart • Shortest Path
  • 74. announcement
  • 75. (NL ) (DSIRNLP)(Tokyo NLP)(Japan.R) ( ) (TokyoWebmining) (MongoDB Japan) http://www.ipsj.or.jp/10jigyo/taikai/74kai/event_2-6.html
  • 76. Thanks!