Your SlideShare is downloading. ×
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Introduction data structure for GraphDB
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Introduction data structure for GraphDB

5,788

Published on

Introduce graphDB structure with Neo4j.

Introduce graphDB structure with Neo4j.

Published in: Technology, Business
4 Comments
27 Likes
Statistics
Notes
No Downloads
Views
Total Views
5,788
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
4
Likes
27
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. Introduction datastructure for GraphDB mixi.inc Shunya Kimura @kimuras
    • 2. agenda• Motivation• Introduction• Structure for Node• Create Node• Appendix
    • 3. Motivation
    • 4. • Just interesting in GraphDB data structure• Why fast• Mysterious
    • 5. Introduction
    • 6. Neo4j• OSS implement of GraphDB• High availability• Scales to billions of nods and relationships• High speed querying through traversals• For more Information about Neo4j • http://www.slideshare.net/skimura/
    • 7. Introduction for Graph Node Name: Kimura Job: Eng Relationship
    • 8. Introduction for Graph Node Name: Kimura Job: Eng Relationship
    • 9. Introduction for Graph Node Data Structure Create Node Relationship
    • 10. Structure for Node
    • 11. A node data structureNode Manager
    • 12. A node data structureNode Manager Node Cache
    • 13. A node data structureNode Manager Node Cache Relation Cache
    • 14. A node data structureNode Manager Node Cache Relation Cache Cache Manager
    • 15. A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Manager
    • 16. A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Manager
    • 17. A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager
    • 18. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager id id id idrelationships relationships relationships relationships Node Node Node Node
    • 19. Node Manager• NodeCache: Caching node id and node object• RelationCache: Caching rel id and Relation object• Cache Manager: Management cache threads• Property Index Manager: Management all nodes Properties• Persistence Manager: Management store
    • 20. Relationship structure id relationships Node
    • 21. Relationship structure idrelationships Node
    • 22. Relationship structure Array structure for relationships idrelationships Node
    • 23. Relationship structure relationships In Blocks Out Blocks Node
    • 24. Relationship structureNode Node relationships In BlocksNode Node Out Blocks NodeNode NodeNode Node
    • 25. Relationship structure In DirectionNode Node relationships In BlocksNode Node Out Blocks NodeNode NodeNode Node
    • 26. Relationship structure In DirectionNode Node relationships In BlocksNode Node Out Blocks NodeNode Node Out DirectionNode Node
    • 27. Relationship structure Node id relationships
    • 28. Relationship structure Node id relationshipsrelationships
    • 29. Relationship structure Node id relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9
    • 30. Relationship structure Node id relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
    • 31. Relationship structure Node id Last In block relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
    • 32. Relationship structure Node id Last In block relationships Last Out blockrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
    • 33. Create Nodes
    • 34. Create Node• Understanding data structure with creating nodes • Create Node A • Create Node B • Create Relationship A to B
    • 35. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS );
    • 36. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA
    • 37. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
    • 38. Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
    • 39. Introduction Create Node
    • 40. Introduction Create Node NodeA
    • 41. Introduction Create Node Node Manager Node Cache Persistence Manager Relation Cache NodeA
    • 42. Introduction Create Node Create New ID id Node Manager Node Cache Persistence Manager Relation Cache NodeA
    • 43. Introduction Create Node Create New ID id Create Node Node Manager id Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
    • 44. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
    • 45. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
    • 46. Cache• You can choice some cache type • Strong Reference Cache • Soft LRU Cache (default) • Weak LRU Cache • No Cache
    • 47. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms
    • 48. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 3 VAL_3 4 VAL_4 5 VAL_5
    • 49. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
    • 50. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEGET KEY:5 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
    • 51. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEGET KEY:5 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
    • 52. What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
    • 53. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEPUT KEY:6 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
    • 54. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEPUT KEY:6 6 VAL_6 5 VAL_5 Priority 1 VAL_1 2 VAL_2 3 VAL_3
    • 55. Soft LRU Cache
    • 56. Soft LRU Cache Soft LRU CacheCache (ConcurrentHashMap)Queue (SoftReferenceQueue)
    • 57. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3Cache (ConcurrentHashMap)Queue (SoftReferenceQueue)
    • 58. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memorydemand.http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
    • 59. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memorydemand.http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
    • 60. Comparing Cache type Cache Type Queue Type Reference Type Weak LRU Cache WeakReferenceQueue WeakReference Soft LRU Cache SoftRefrenceQueue SoftReference Strong Reference None None CacheRemaining Reference on memoryStrong > Soft > Weak
    • 61. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
    • 62. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA NodeB
    • 63. Introduction Create Node NodeA NodeB
    • 64. Introduction Create Node NodeA NodeB
    • 65. Introduction Create Node id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
    • 66. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeBPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
    • 67. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeBPersistence Manager id Relation Cache relationships Register IN direction from NodeA NodeB NodeA NodeB
    • 68. Introduction Create Node id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
    • 69. Introduction Create Node Create New Relation ID rel id id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
    • 70. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA RelationPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
    • 71. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA RelationPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
    • 72. Appendix
    • 73. Implemented Graph Algorithms• Centrality • Betweenness Centrality • Closeness Centrality• Path • A* • Dijkstart • Shortest Path
    • 74. announcement
    • 75. (NL ) (DSIRNLP)(Tokyo NLP)(Japan.R) ( ) (TokyoWebmining) (MongoDB Japan) http://www.ipsj.or.jp/10jigyo/taikai/74kai/event_2-6.html
    • 76. Thanks!

    ×