• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Introduction data structure for GraphDB
 

Introduction data structure for GraphDB

on

  • 5,798 views

Introduce graphDB structure with Neo4j.

Introduce graphDB structure with Neo4j.

Statistics

Views

Total Views
5,798
Views on SlideShare
5,699
Embed Views
99

Actions

Likes
26
Downloads
0
Comments
4

12 Embeds 99

http://a0.twimg.com 39
http://paper.li 25
http://us-w1.rockmelt.com 18
http://pdfslidestore.blogspot.com 5
https://si0.twimg.com 3
http://www.linkedin.com 2
https://twitter.com 2
http://alem61mar.blogspot.co.uk 1
http://cordobamediterranea.blogspot.co.il 1
http://alem61mar.blogspot.com 1
http://www.twylah.com 1
http://littlemisslizzyandfriends.blogspot.de 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

14 of 4 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • thanks
    Are you sure you want to
    Your message goes here
    Processing…
  • Awesome. Let's connect on that!
    Are you sure you want to
    Your message goes here
    Processing…
  • Peter Neubauer,
    Thank you, I feel proud of your comment.

    > Would you mind doing a Neo4j User group in Japan one of these days?

    Yes, I agree to creating a Neo4j user group in Japan.
    I will send you some message after creating plan.
    Are you sure you want to
    Your message goes here
    Processing…
  • Very nice presentation! Would you mind doing a Neo4j User group in Japan one of these days?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Introduction data structure for GraphDB Introduction data structure for GraphDB Presentation Transcript

  • Introduction datastructure for GraphDB mixi.inc Shunya Kimura @kimuras
  • agenda• Motivation• Introduction• Structure for Node• Create Node• Appendix
  • Motivation
  • • Just interesting in GraphDB data structure• Why fast• Mysterious
  • Introduction
  • Neo4j• OSS implement of GraphDB• High availability• Scales to billions of nods and relationships• High speed querying through traversals• For more Information about Neo4j • http://www.slideshare.net/skimura/
  • Introduction for Graph Node Name: Kimura Job: Eng Relationship
  • Introduction for Graph Node Name: Kimura Job: Eng Relationship
  • Introduction for Graph Node Data Structure Create Node Relationship
  • Structure for Node
  • A node data structureNode Manager
  • A node data structureNode Manager Node Cache
  • A node data structureNode Manager Node Cache Relation Cache
  • A node data structureNode Manager Node Cache Relation Cache Cache Manager
  • A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Manager
  • A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Manager
  • A node data structureNode Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager
  • A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager id id id idrelationships relationships relationships relationships Node Node Node Node
  • Node Manager• NodeCache: Caching node id and node object• RelationCache: Caching rel id and Relation object• Cache Manager: Management cache threads• Property Index Manager: Management all nodes Properties• Persistence Manager: Management store
  • Relationship structure id relationships Node
  • Relationship structure idrelationships Node
  • Relationship structure Array structure for relationships idrelationships Node
  • Relationship structure relationships In Blocks Out Blocks Node
  • Relationship structureNode Node relationships In BlocksNode Node Out Blocks NodeNode NodeNode Node
  • Relationship structure In DirectionNode Node relationships In BlocksNode Node Out Blocks NodeNode NodeNode Node
  • Relationship structure In DirectionNode Node relationships In BlocksNode Node Out Blocks NodeNode Node Out DirectionNode Node
  • Relationship structure Node id relationships
  • Relationship structure Node id relationshipsrelationships
  • Relationship structure Node id relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9
  • Relationship structure Node id relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
  • Relationship structure Node id Last In block relationshipsrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
  • Relationship structure Node id Last In block relationships Last Out blockrelationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
  • Create Nodes
  • Create Node• Understanding data structure with creating nodes • Create Node A • Create Node B • Create Relationship A to B
  • Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS );
  • Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA
  • Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
  • Create NodeGraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH );Node NodeA = graphDb.createNode();Node NodeB = graphDb.createNode();Relationship relationship =NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
  • Introduction Create Node
  • Introduction Create Node NodeA
  • Introduction Create Node Node Manager Node Cache Persistence Manager Relation Cache NodeA
  • Introduction Create Node Create New ID id Node Manager Node Cache Persistence Manager Relation Cache NodeA
  • Introduction Create Node Create New ID id Create Node Node Manager id Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
  • Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
  • Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
  • Cache• You can choice some cache type • Strong Reference Cache • Soft LRU Cache (default) • Weak LRU Cache • No Cache
  • What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms
  • What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 3 VAL_3 4 VAL_4 5 VAL_5
  • What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
  • What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEGET KEY:5 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
  • What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEGET KEY:5 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
  • What is LRU Cache• discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
  • What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEPUT KEY:6 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
  • What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUEPUT KEY:6 6 VAL_6 5 VAL_5 Priority 1 VAL_1 2 VAL_2 3 VAL_3
  • Soft LRU Cache
  • Soft LRU Cache Soft LRU CacheCache (ConcurrentHashMap)Queue (SoftReferenceQueue)
  • Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3Cache (ConcurrentHashMap)Queue (SoftReferenceQueue)
  • Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memorydemand.http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
  • Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memorydemand.http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
  • Comparing Cache type Cache Type Queue Type Reference Type Weak LRU Cache WeakReferenceQueue WeakReference Soft LRU Cache SoftRefrenceQueue SoftReference Strong Reference None None CacheRemaining Reference on memoryStrong > Soft > Weak
  • Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
  • Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA NodeB
  • Introduction Create Node NodeA NodeB
  • Introduction Create Node NodeA NodeB
  • Introduction Create Node id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeBPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeBPersistence Manager id Relation Cache relationships Register IN direction from NodeA NodeB NodeA NodeB
  • Introduction Create Node id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • Introduction Create Node Create New Relation ID rel id id Node Manager relationships Node Cache NodeAPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA RelationPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA RelationPersistence Manager id Relation Cache relationships NodeB NodeA NodeB
  • Appendix
  • Implemented Graph Algorithms• Centrality • Betweenness Centrality • Closeness Centrality• Path • A* • Dijkstart • Shortest Path
  • announcement
  • (NL ) (DSIRNLP)(Tokyo NLP)(Japan.R) ( ) (TokyoWebmining) (MongoDB Japan) http://www.ipsj.or.jp/10jigyo/taikai/74kai/event_2-6.html
  • Thanks!