Introduction data structure for GraphDB
•29 likes•3,779 views
Report
Share
Introduce graphDB structure with Neo4j.
More Related Content
Similar to Introduction data structure for GraphDB
Similar to Introduction data structure for GraphDB(20)
Recently uploaded
Recently uploaded(20)
Introduction data structure for GraphDB
- 1. Introduction data structure for GraphDB mixi.inc Shunya Kimura @kimuras
- 2. agenda • Motivation • Introduction • Structure for Node • Create Node • Appendix
- 3. Motivation
- 4. • Just interesting in GraphDB data structure • Why fast • Mysterious
- 5. Introduction
- 6. Neo4j • OSS implement of GraphDB • High availability • Scales to billions of nods and relationships • High speed querying through traversals • For more Information about Neo4j • http://www.slideshare.net/skimura/
- 7. Introduction for Graph Node Name: Kimura Job: Eng Relationship
- 8. Introduction for Graph Node Name: Kimura Job: Eng Relationship
- 9. Introduction for Graph Node Data Structure Create Node Relationship
- 11. A node data structure Node Manager
- 12. A node data structure Node Manager Node Cache
- 13. A node data structure Node Manager Node Cache Relation Cache
- 14. A node data structure Node Manager Node Cache Relation Cache Cache Manager
- 15. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Manager
- 16. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Manager
- 17. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager
- 18. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager id id id id relationships relationships relationships relationships Node Node Node Node
- 19. Node Manager • NodeCache: Caching node id and node object • RelationCache: Caching rel id and Relation object • Cache Manager: Management cache threads • Property Index Manager: Management all nodes Properties • Persistence Manager: Management store
- 20. Relationship structure id relationships Node
- 21. Relationship structure id relationships Node
- 22. Relationship structure Array structure for relationships id relationships Node
- 23. Relationship structure relationships In Blocks Out Blocks Node
- 24. Relationship structure Node Node relationships In Blocks Node Node Out Blocks Node Node Node Node Node
- 25. Relationship structure In Direction Node Node relationships In Blocks Node Node Out Blocks Node Node Node Node Node
- 26. Relationship structure In Direction Node Node relationships In Blocks Node Node Out Blocks Node Node Node Out Direction Node Node
- 27. Relationship structure Node id relationships
- 28. Relationship structure Node id relationships relationships
- 29. Relationship structure Node id relationships relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9
- 30. Relationship structure Node id relationships relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
- 31. Relationship structure Node id Last In block relationships relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
- 32. Relationship structure Node id Last In block relationships Last Out block relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
- 33. Create Nodes
- 34. Create Node • Understanding data structure with creating nodes • Create Node A • Create Node B • Create Relationship A to B
- 35. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS );
- 36. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA
- 37. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
- 38. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
- 40. Introduction Create Node NodeA
- 41. Introduction Create Node Node Manager Node Cache Persistence Manager Relation Cache NodeA
- 42. Introduction Create Node Create New ID id Node Manager Node Cache Persistence Manager Relation Cache NodeA
- 43. Introduction Create Node Create New ID id Create Node Node Manager id Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
- 44. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
- 45. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
- 46. Cache • You can choice some cache type • Strong Reference Cache • Soft LRU Cache (default) • Weak LRU Cache • No Cache
- 47. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms
- 48. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 3 VAL_3 4 VAL_4 5 VAL_5
- 49. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
- 50. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE GET KEY:5 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
- 51. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE GET KEY:5 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
- 52. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
- 53. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE PUT KEY:6 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
- 54. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE PUT KEY:6 6 VAL_6 5 VAL_5 Priority 1 VAL_1 2 VAL_2 3 VAL_3
- 55. Soft LRU Cache
- 56. Soft LRU Cache Soft LRU Cache Cache (ConcurrentHashMap) Queue (SoftReferenceQueue)
- 57. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) Queue (SoftReferenceQueue)
- 58. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memory demand. http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
- 59. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memory demand. http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
- 60. Comparing Cache type Cache Type Queue Type Reference Type Weak LRU Cache WeakReferenceQueue WeakReference Soft LRU Cache SoftRefrenceQueue SoftReference Strong Reference None None Cache Remaining Reference on memory Strong > Soft > Weak
- 61. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
- 62. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA NodeB
- 63. Introduction Create Node NodeA NodeB
- 64. Introduction Create Node NodeA NodeB
- 65. Introduction Create Node id Node Manager relationships Node Cache NodeA Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 66. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeB Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 67. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeB Persistence Manager id Relation Cache relationships Register IN direction from NodeA NodeB NodeA NodeB
- 68. Introduction Create Node id Node Manager relationships Node Cache NodeA Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 69. Introduction Create Node Create New Relation ID rel id id Node Manager relationships Node Cache NodeA Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 70. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA Relation Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 71. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA Relation Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 72. Appendix
- 73. Implemented Graph Algorithms • Centrality • Betweenness Centrality • Closeness Centrality • Path • A* • Dijkstart • Shortest Path
- 74. announcement
- 75. (NL ) (DSIRNLP) (Tokyo NLP) (Japan.R) ( ) (TokyoWebmining) (MongoDB Japan) http://www.ipsj.or.jp/10jigyo/taikai/74kai/event_2-6.html
- 76. Thanks!
Editor's Notes
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n