ソーシャルグラフ分析

shunya kimura
shunya kimuradevelopment at mixi
ソーシャルグラフ分析
•                  (   )

    •   @kimuras

•                          G(2007   )

•
    •
•
Agenda

• Introduction
• The past work
• Introduction to GraphDB
• Introduction to Neo4j
• Introduction to analysis sample
Introduction
Motivation for social graph analysis
mixi
                 30000000

                                   ID
                 22500000
# of member id




                 15000000



                  7500000



                        0
                            2007        2008   2009   2010   2011
                                               year
What is Social Graph?
ソーシャルグラフ分析
ソーシャルグラフ分析
ソーシャルグラフ分析
ソーシャルグラフ分析
ソーシャルグラフ分析
ソーシャルグラフ分析
ソーシャルグラフ分析
Feed Back
Feed Back
Feed Back
Feed Back
Feed Back
Approach for SG
    analysis


       Feed Back
The past work
•
•
Relational Databases

                                        Dump &
                                        Denormalization




from_id    to_id    id   name     age                     Key      value

1          2        1    Kimura   18                      From:1   2,3

1          3        2    kato     45                      From:2   3

2          3        3    ito      21                      Prof:1   Kimura,18
                                                          Prof:2   Kato,45
Relational Databases

                                        Dump &

                   reimplementation     Denormalization




from_id    to_id    id   name     age                     Key      value

1
1
           2
           3
                   maintenance cost
                    1
                    2
                         Kimura
                         kato
                                  18
                                  45
                                                          From:1
                                                          From:2
                                                                   2,3
                                                                   3

2          3        3    ito      21                      Prof:1   Kimuras,18
                                                          Prof:2   Kato,45

                               scalability
Introduction to GraphDB
What is graph
What is graph
 Vertex (node :   )
What is graph
         Vertex (node :   )




Edge (     )
What is graph
         Vertex (node :   )


          Undirected graph (   )

Edge (     )
What is graph
         Vertex (node :   )




Edge (     )
What is graph
         Vertex (node :   )




Edge (     )
What is graph
         Vertex (node :   )




Edge (     )
What is graph
         Vertex (node :     )


               Directed graph (   )

Edge (     )
What is GraphDB
         Vertex (node :   )




Edge (     )
What is GraphDB
ID:   1
               Vertex (node :   )
NAME: kimura
PROP: Male
AGE: 18




Edge (           )
What is GraphDB
ID:   1
               Vertex (node :       )
NAME: kimura
PROP: Male
AGE: 18




Edge (           )
                     ID:   2
                     NAME: ITO
                     PROP: Female
                     AGE: 21
What is GraphDB
ID:   1
               Vertex (node :       )
NAME: kimura
PROP: Male
AGE: 18




Edge (           )
                     ID:   2
                     NAME: ITO
                     PROP: Female
                     AGE: 21
What is GraphDB
ID:   1
               Vertex (node :       )
NAME: kimura
PROP: Male
AGE: 18




Edge (           )
                     ID:   2
                     NAME: ITO
                     PROP: Female
                     AGE: 21
What is GraphDB
ID:   1
                       Vertex (node :       )
NAME: kimura
PROP: Male
AGE: 18




 Edge (                  )
                             ID:   2
ID:       3                  NAME: ITO
LABEL:    Like               PROP: Female
Since:    2011/08/06         AGE: 21
OutGoing: 2
What is GraphDB
ID:   1
                       Vertex (node :       )
NAME: kimura
PROP: Male
AGE: 18




 Edge (                  )
                             ID:   2
ID:       3                  NAME: ITO
LABEL:    Like               PROP: Female
Since:    2011/08/06         AGE: 21
OutGoing: 2
What is GraphDB
ID:   1
                       Vertex (node :       )
NAME: kimura
PROP: Male
AGE: 18




 Edge (                  )
                             ID:   2
ID:       3                  NAME: ITO
LABEL:    Like               PROP: Female
Since:    2011/08/06         AGE: 21
OutGoing: 2
The implementations
   for GraphDB




  http://en.wikipedia.org/wiki/GraphDB
Introduction to Neo4j
GraphDB Neo4j
       •     True ACID transactions
       •     High availability
       •     Scales to billions of nods and relationships
       •     High speed querying through traversals


               Single instance(GPLv3)   Multiple instance(AGPLv3)
Embedded       EmbeddedGraphDatabase    HighlyAvailableGraphDatabase
Standalone     Neo4j Server             Neo4j Server high availability mode


                                                   http://neo4j.org/
Other my favorite features
       for Neo4j
• RESTful APIs
• Query Language(Cypher)
• Full indexing
   – lucene
• Implemented graph algorithm
 – A*, Dijkstra
 – High speed traverse
• Gremlin supported
 – Like a query language

                         http://www.tinkerpop.com/post/4633229547/tinkerpop-graph-stack
Introduction simple Neo4j usecase
           Single node           Multi node
Embedded



           Analyses system       Analyses system




           Analyses system       Analyses system
Server
Introduction simple Neo4j usecase
           Single node           Multi node
Embedded



           Analyses system       Analyses system




           Analyses system       Analyses system
Server
Introduction simple Neo4j usecase
              Single node          Multi node
           Analyses system
Embedded



                                   Analyses system




             Analyses system       Analyses system
Server
Introduction simple Neo4j usecase
              Single node          Multi node
           Analyses system
Embedded



                                   Analyses system




             Analyses system       Analyses system
Server
Introduction to simple
   embedded Neo4j

• Insert Vertices & make Relationships
 • Single node & Embedded
• Traversal sample
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {
        GraphDatabaseService graphDb = new
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
            firstNode.setProperty("Name", "Kimura");
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {
            tx.finish();
        }
        graphDb.shutdown();
    }
}
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {
        GraphDatabaseService graphDb = new
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
            firstNode.setProperty("Name", "Kimura");
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {
            tx.finish();
        }
        graphDb.shutdown();
    }
}
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {            ID:   1
        GraphDatabaseService graphDb = new                    NAME: kimura
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
            firstNode.setProperty("Name", "Kimura");
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {
            tx.finish();
        }
        graphDb.shutdown();
    }
}
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {            ID:   1
        GraphDatabaseService graphDb = new                    NAME: kimura
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
            firstNode.setProperty("Name", "Kimura");
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {
            tx.finish();
        }
        graphDb.shutdown();
    }
}
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {            ID:   1
        GraphDatabaseService graphDb = new                    NAME: kimura
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
            firstNode.setProperty("Name", "Kimura");
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {                                           ID:   2
            tx.finish();                                      NAME: Kato
        }
        graphDb.shutdown();
    }
}
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {            ID:   1
        GraphDatabaseService graphDb = new                    NAME: kimura
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
            firstNode.setProperty("Name", "Kimura");
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {                                           ID:   2
            tx.finish();                                      NAME: Kato
        }
        graphDb.shutdown();
    }
}
Insert vertices,
                   make relationship
public final class InputVertex {
    public static void main(final String[] args) {                             ID:   1
        GraphDatabaseService graphDb = new                                     NAME: kimura
                       EmbeddedGraphDatabase("/tmp/neo4j");
        Transaction tx = graphDb.beginTx();
        try {
            Node firstNode = graphDb.createNode();
                                                              ID:       3
            firstNode.setProperty("Name", "Kimura");          Relation: Like
            Node secondNode = graphDb.createNode();
            secondNode.setProperty("Name", "Kato");
            firstNode.createRelationshipTo(secondNode,
                 DynamicRelationshipType.withName("LIKE"));
            tx.success();
        } finally {                                                            ID:   2
            tx.finish();                                                       NAME: Kato
        }
        graphDb.shutdown();
    }
}
Batch Insert
     • Non thread safe, non transaction
     • But very fast!
public final class Batch {
    public static void main(final String[] args) {
        BatchInserter inserter = new BatchInserterImpl("/tmp/neo4j",
                BatchInserterImpl.loadProperties("/tmp/neo4j.props"));
        Map<String, Object> prop = new HashMap<String, Object>();
        prop.put("Name", "Kimura");
        prop.put("Age", 21);
        long node1 = inserter.createNode(prop);

        prop.put("Name", "Kato");
        prop.put("Age", 21);
        long node2 = inserter.createNode(prop);
        inserter.createRelationship(node1, node2,
                DynamicRelationshipType.withName("LIKE"), null);
        inserter.shutdown();
    }
}
Traversal sample
    •
public static void main(final String[] args) {
        GraphDatabaseService graphDB = new EmbeddedGraphDatabase(args[0]);
        Node node = graphDB.getNodeById(1);
        Traverser friends = node.traverse(
          //
          Order.DEPTH_FIRST,   BREADTH_FIRST
          //
          StopEvaluator.END_OF_GRAPH,   DEPTH_ONE
          //
          ReturnableEvaluator.ALL_BUT_START_NODE,     ALL, isReturnableNode()
          //
          DynamicRelationshipType.withName("LIKE"),
          //
          Direction.OUTGOING);   INCOMING, BOTH
        for (Node nodeBuf : friends) {
            TraversalPosition currentPosition = friends.currentPosition();
        }
   }
Traversal sample
    Order.BREADTH_FIRST
•
Traversal sample
    Order.BREADTH_FIRST
•
Traversal sample
    Order.BREADTH_FIRST
•
Traversal sample
    Order.BREADTH_FIRST
•
Traversal sample
    Order.BREADTH_FIRST
•
Traversal sample
    Order.BREADTH_FIRST
•
Traversal sample
    Order.DEPTH_FIRST
•
Traversal sample
    Order.DEPTH_FIRST
•
Traversal sample
    Order.DEPTH_FIRST
•
Traversal sample
    Order.DEPTH_FIRST
•
Traversal sample
    Order.DEPTH_FIRST
•
Traversal sample
    Order.DEPTH_FIRST
•
Neoclipse sample




       http://wiki.neo4j.org/content/Neoclipse
experiment
•   mixi             Neo4j

•
    •   Machine: 24 core CPU, Memory 65GB

    •   Neo4j: BatchInsert, community, embedded

•   Data

    •                1500                    6

                   513m17sec (about 8.6h)
Network Dataset
•   Stanford Large Network Dataset Collection

    •    SNAP has a Wide variety of graph data!
             Social Networks             Communication networks

            Citation networks             Collaboration networks

               Web graphs             Product co-purchasing networks

     Internet peer-to-peer networks           Road networks

        Autonomous systems graphs            Signed networks

    Wikipedia networks and metadata      Memetracker and Twitter


                            http://snap.stanford.edu/data/index.html
Introduction to Analysis
        Sample
Architecture

   Service
                  Database   Analysis   Visualization
(Social Graph)
Introduction Analyses
          Sample


• Centrality (       )

• Clustering coefficient (   )
Centrality (       )

•       =


            Pagerank
•
•   =   Vertex   (   )
•
•   =   Vertex       (   )



    1            1



                 1
•
•   =       Vertex        (   )


              2
    1                 1


        2
                      1
                  2
•
•   =       Vertex        (   )


              2
    1                 1


        2
                      1
                  2
•
•   =       Vertex        (   )


              2
    1                 1
                  4
        2
                      1
                  2
•
•   =       Vertex        (   )


              2
    1                 1
                  4
        2
                      1
                  2
mixi

 •     1000

 •             summary


Min      1st Que. Median     Mean    3rd Que.    Max

1.00          3.00   10.00   25.69    30.00     903.00
•
    •   ≒
•
    •   ≒
            =0/3=0
•
    •   ≒
            =0/3=0


            =1/3
•
    •   ≒
            =0/3=0


            =1/3


            =2/3
•
    •   ≒
            =0/3=0


            =1/3


            =2/3


            =3/3=1
•   1000

•                     summary


Min        1st Que. Median   Mean     3rd Que.   Max

0.00        0.00    0.1157   0.2071    0.2667    1.000
ソーシャルグラフ分析
ソーシャルグラフ分析
•   25   0.08
•   14   0.17
•   10   0.68
•   4   1
Visualization Sample
•          2hop      Social Graph


•   Edge


    •                              (              )


•   Vertex


    •                       (                 )


•                 Gephi
                          http://gephi.org/
ソーシャルグラフ分析
•   Social Graph

    •
•   GraphDB

•   Neo4j

•   R

•   Visualization
Thanks!
1 of 102

Recommended

Mining the social graph by
Mining the social graphMining the social graph
Mining the social graphshunya kimura
2K views142 slides
ソーシャルグラフのデータ解析 by
ソーシャルグラフのデータ解析ソーシャルグラフのデータ解析
ソーシャルグラフのデータ解析Chiaki Hatanaka
752 views143 slides
DL'12 mastro at work by
DL'12 mastro at workDL'12 mastro at work
DL'12 mastro at workMariano Rodriguez-Muro
493 views49 slides
Collaborative Cuisine's 1 Hour JNDI Cookbook by
Collaborative Cuisine's 1 Hour JNDI CookbookCollaborative Cuisine's 1 Hour JNDI Cookbook
Collaborative Cuisine's 1 Hour JNDI CookbookKen Lin
794 views36 slides
情報処理学会第74回全国大会 私的勉強会と学会の未来 by
情報処理学会第74回全国大会 私的勉強会と学会の未来情報処理学会第74回全国大会 私的勉強会と学会の未来
情報処理学会第74回全国大会 私的勉強会と学会の未来shunya kimura
2.7K views47 slides
Introduction data structure for GraphDB by
Introduction data structure for GraphDBIntroduction data structure for GraphDB
Introduction data structure for GraphDBshunya kimura
3.8K views76 slides

More Related Content

Similar to ソーシャルグラフ分析

Better DSL Support for Groovy-Eclipse by
Better DSL Support for Groovy-EclipseBetter DSL Support for Groovy-Eclipse
Better DSL Support for Groovy-EclipseAndrew Eisenberg
2.5K views36 slides
Web-Scale Graph Analytics with Apache® Spark™ by
Web-Scale Graph Analytics with Apache® Spark™Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™Databricks
2.7K views59 slides
Challenging Web-Scale Graph Analytics with Apache Spark by
Challenging Web-Scale Graph Analytics with Apache SparkChallenging Web-Scale Graph Analytics with Apache Spark
Challenging Web-Scale Graph Analytics with Apache SparkDatabricks
965 views37 slides
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng by
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui MengChallenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui MengDatabricks
1.2K views37 slides
The InfoGrid Graph DataBase by
The InfoGrid Graph DataBaseThe InfoGrid Graph DataBase
The InfoGrid Graph DataBaseInfoGrid.org
3.1K views14 slides
Processing Large Graphs by
Processing Large GraphsProcessing Large Graphs
Processing Large GraphsNishant Gandhi
261 views68 slides

Similar to ソーシャルグラフ分析(20)

Better DSL Support for Groovy-Eclipse by Andrew Eisenberg
Better DSL Support for Groovy-EclipseBetter DSL Support for Groovy-Eclipse
Better DSL Support for Groovy-Eclipse
Andrew Eisenberg2.5K views
Web-Scale Graph Analytics with Apache® Spark™ by Databricks
Web-Scale Graph Analytics with Apache® Spark™Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™
Databricks2.7K views
Challenging Web-Scale Graph Analytics with Apache Spark by Databricks
Challenging Web-Scale Graph Analytics with Apache SparkChallenging Web-Scale Graph Analytics with Apache Spark
Challenging Web-Scale Graph Analytics with Apache Spark
Databricks965 views
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng by Databricks
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui MengChallenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng
Databricks1.2K views
The InfoGrid Graph DataBase by InfoGrid.org
The InfoGrid Graph DataBaseThe InfoGrid Graph DataBase
The InfoGrid Graph DataBase
InfoGrid.org3.1K views
Odessapy2013 - Graph databases and Python by Max Klymyshyn
Odessapy2013 - Graph databases and PythonOdessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and Python
Max Klymyshyn56.8K views
Triton and symbolic execution on gdb by Wei-Bo Chen
Triton and symbolic execution on gdbTriton and symbolic execution on gdb
Triton and symbolic execution on gdb
Wei-Bo Chen595 views
Web-Scale Graph Analytics with Apache® Spark™ by Databricks
Web-Scale Graph Analytics with Apache® Spark™Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™
Databricks1.4K views
Designing an Objective-C Framework about 3D by rsebbe
Designing an Objective-C Framework about 3DDesigning an Objective-C Framework about 3D
Designing an Objective-C Framework about 3D
rsebbe2.3K views
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono by Achim Friedland
Fosdem 2011 - A Common Graph Database Access Layer for .Net and MonoFosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Achim Friedland1.8K views
Scaling into Billions of Nodes and Relationships with Neo4j Graph Data Science by Neo4j
Scaling into Billions of Nodes and Relationships with Neo4j Graph Data ScienceScaling into Billions of Nodes and Relationships with Neo4j Graph Data Science
Scaling into Billions of Nodes and Relationships with Neo4j Graph Data Science
Neo4j355 views
HBaseCon 2015: HBase @ CyberAgent by HBaseCon
HBaseCon 2015: HBase @ CyberAgentHBaseCon 2015: HBase @ CyberAgent
HBaseCon 2015: HBase @ CyberAgent
HBaseCon5.5K views
Graph technology meetup slides by Sean Mulvehill
Graph technology meetup slidesGraph technology meetup slides
Graph technology meetup slides
Sean Mulvehill215 views
High-Performance Graph Analysis and Modeling by Nesreen K. Ahmed
High-Performance Graph Analysis and ModelingHigh-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and Modeling
Nesreen K. Ahmed200 views
MongoDB, Hadoop and humongous data - MongoSV 2012 by Steven Francia
MongoDB, Hadoop and humongous data - MongoSV 2012MongoDB, Hadoop and humongous data - MongoSV 2012
MongoDB, Hadoop and humongous data - MongoSV 2012
Steven Francia10.2K views
Triton and Symbolic execution on GDB@DEF CON China by Wei-Bo Chen
Triton and Symbolic execution on GDB@DEF CON ChinaTriton and Symbolic execution on GDB@DEF CON China
Triton and Symbolic execution on GDB@DEF CON China
Wei-Bo Chen2.2K views
MathWorks Interview Lecture by John Yates
MathWorks Interview LectureMathWorks Interview Lecture
MathWorks Interview Lecture
John Yates532 views

Recently uploaded

AMAZON PRODUCT RESEARCH.pdf by
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdfJerikkLaureta
19 views13 slides
HTTP headers that make your website go faster - devs.gent November 2023 by
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023Thijs Feryn
21 views151 slides
Java Platform Approach 1.0 - Picnic Meetup by
Java Platform Approach 1.0 - Picnic MeetupJava Platform Approach 1.0 - Picnic Meetup
Java Platform Approach 1.0 - Picnic MeetupRick Ossendrijver
27 views39 slides
From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!sammart93
9 views39 slides
Microsoft Power Platform.pptx by
Microsoft Power Platform.pptxMicrosoft Power Platform.pptx
Microsoft Power Platform.pptxUni Systems S.M.S.A.
52 views38 slides
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors by
TouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective SensorsTouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective Sensors
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensorssugiuralab
19 views15 slides

Recently uploaded(20)

AMAZON PRODUCT RESEARCH.pdf by JerikkLaureta
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdf
JerikkLaureta19 views
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn21 views
From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by sammart93
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
sammart939 views
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors by sugiuralab
TouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective SensorsTouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective Sensors
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors
sugiuralab19 views
Empathic Computing: Delivering the Potential of the Metaverse by Mark Billinghurst
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
Mark Billinghurst476 views
Web Dev - 1 PPT.pdf by gdsczhcet
Web Dev - 1 PPT.pdfWeb Dev - 1 PPT.pdf
Web Dev - 1 PPT.pdf
gdsczhcet60 views
Voice Logger - Telephony Integration Solution at Aegis by Nirmal Sharma
Voice Logger - Telephony Integration Solution at AegisVoice Logger - Telephony Integration Solution at Aegis
Voice Logger - Telephony Integration Solution at Aegis
Nirmal Sharma31 views
6g - REPORT.pdf by Liveplex
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdf
Liveplex10 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi126 views
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 -  Week 3 Session 5Business Analyst Series 2023 -  Week 3 Session 5
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10237 views
Spesifikasi Lengkap ASUS Vivobook Go 14 by Dot Semarang
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14
Dot Semarang37 views
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N... by James Anderson
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
James Anderson66 views
handbook for web 3 adoption.pdf by Liveplex
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdf
Liveplex22 views

ソーシャルグラフ分析

  • 2. ( ) • @kimuras • G(2007 ) • • •
  • 3. Agenda • Introduction • The past work • Introduction to GraphDB • Introduction to Neo4j • Introduction to analysis sample
  • 5. Motivation for social graph analysis
  • 6. mixi 30000000 ID 22500000 # of member id 15000000 7500000 0 2007 2008 2009 2010 2011 year
  • 7. What is Social Graph?
  • 20. Approach for SG analysis Feed Back
  • 23. Relational Databases Dump & Denormalization from_id to_id id name age Key value 1 2 1 Kimura 18 From:1 2,3 1 3 2 kato 45 From:2 3 2 3 3 ito 21 Prof:1 Kimura,18 Prof:2 Kato,45
  • 24. Relational Databases Dump & reimplementation Denormalization from_id to_id id name age Key value 1 1 2 3 maintenance cost 1 2 Kimura kato 18 45 From:1 From:2 2,3 3 2 3 3 ito 21 Prof:1 Kimuras,18 Prof:2 Kato,45 scalability
  • 27. What is graph Vertex (node : )
  • 28. What is graph Vertex (node : ) Edge ( )
  • 29. What is graph Vertex (node : ) Undirected graph ( ) Edge ( )
  • 30. What is graph Vertex (node : ) Edge ( )
  • 31. What is graph Vertex (node : ) Edge ( )
  • 32. What is graph Vertex (node : ) Edge ( )
  • 33. What is graph Vertex (node : ) Directed graph ( ) Edge ( )
  • 34. What is GraphDB Vertex (node : ) Edge ( )
  • 35. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( )
  • 36. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( ) ID: 2 NAME: ITO PROP: Female AGE: 21
  • 37. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( ) ID: 2 NAME: ITO PROP: Female AGE: 21
  • 38. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( ) ID: 2 NAME: ITO PROP: Female AGE: 21
  • 39. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( ) ID: 2 ID: 3 NAME: ITO LABEL: Like PROP: Female Since: 2011/08/06 AGE: 21 OutGoing: 2
  • 40. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( ) ID: 2 ID: 3 NAME: ITO LABEL: Like PROP: Female Since: 2011/08/06 AGE: 21 OutGoing: 2
  • 41. What is GraphDB ID: 1 Vertex (node : ) NAME: kimura PROP: Male AGE: 18 Edge ( ) ID: 2 ID: 3 NAME: ITO LABEL: Like PROP: Female Since: 2011/08/06 AGE: 21 OutGoing: 2
  • 42. The implementations for GraphDB http://en.wikipedia.org/wiki/GraphDB
  • 44. GraphDB Neo4j • True ACID transactions • High availability • Scales to billions of nods and relationships • High speed querying through traversals Single instance(GPLv3) Multiple instance(AGPLv3) Embedded EmbeddedGraphDatabase HighlyAvailableGraphDatabase Standalone Neo4j Server Neo4j Server high availability mode http://neo4j.org/
  • 45. Other my favorite features for Neo4j • RESTful APIs • Query Language(Cypher) • Full indexing – lucene • Implemented graph algorithm – A*, Dijkstra – High speed traverse • Gremlin supported – Like a query language http://www.tinkerpop.com/post/4633229547/tinkerpop-graph-stack
  • 46. Introduction simple Neo4j usecase Single node Multi node Embedded Analyses system Analyses system Analyses system Analyses system Server
  • 47. Introduction simple Neo4j usecase Single node Multi node Embedded Analyses system Analyses system Analyses system Analyses system Server
  • 48. Introduction simple Neo4j usecase Single node Multi node Analyses system Embedded Analyses system Analyses system Analyses system Server
  • 49. Introduction simple Neo4j usecase Single node Multi node Analyses system Embedded Analyses system Analyses system Analyses system Server
  • 50. Introduction to simple embedded Neo4j • Insert Vertices & make Relationships • Single node & Embedded • Traversal sample
  • 51. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { GraphDatabaseService graphDb = new EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); firstNode.setProperty("Name", "Kimura"); Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { tx.finish(); } graphDb.shutdown(); } }
  • 52. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { GraphDatabaseService graphDb = new EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); firstNode.setProperty("Name", "Kimura"); Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { tx.finish(); } graphDb.shutdown(); } }
  • 53. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { ID: 1 GraphDatabaseService graphDb = new NAME: kimura EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); firstNode.setProperty("Name", "Kimura"); Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { tx.finish(); } graphDb.shutdown(); } }
  • 54. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { ID: 1 GraphDatabaseService graphDb = new NAME: kimura EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); firstNode.setProperty("Name", "Kimura"); Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { tx.finish(); } graphDb.shutdown(); } }
  • 55. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { ID: 1 GraphDatabaseService graphDb = new NAME: kimura EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); firstNode.setProperty("Name", "Kimura"); Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { ID: 2 tx.finish(); NAME: Kato } graphDb.shutdown(); } }
  • 56. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { ID: 1 GraphDatabaseService graphDb = new NAME: kimura EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); firstNode.setProperty("Name", "Kimura"); Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { ID: 2 tx.finish(); NAME: Kato } graphDb.shutdown(); } }
  • 57. Insert vertices, make relationship public final class InputVertex { public static void main(final String[] args) { ID: 1 GraphDatabaseService graphDb = new NAME: kimura EmbeddedGraphDatabase("/tmp/neo4j"); Transaction tx = graphDb.beginTx(); try { Node firstNode = graphDb.createNode(); ID: 3 firstNode.setProperty("Name", "Kimura"); Relation: Like Node secondNode = graphDb.createNode(); secondNode.setProperty("Name", "Kato"); firstNode.createRelationshipTo(secondNode, DynamicRelationshipType.withName("LIKE")); tx.success(); } finally { ID: 2 tx.finish(); NAME: Kato } graphDb.shutdown(); } }
  • 58. Batch Insert • Non thread safe, non transaction • But very fast! public final class Batch { public static void main(final String[] args) { BatchInserter inserter = new BatchInserterImpl("/tmp/neo4j", BatchInserterImpl.loadProperties("/tmp/neo4j.props")); Map<String, Object> prop = new HashMap<String, Object>(); prop.put("Name", "Kimura"); prop.put("Age", 21); long node1 = inserter.createNode(prop); prop.put("Name", "Kato"); prop.put("Age", 21); long node2 = inserter.createNode(prop); inserter.createRelationship(node1, node2, DynamicRelationshipType.withName("LIKE"), null); inserter.shutdown(); } }
  • 59. Traversal sample • public static void main(final String[] args) { GraphDatabaseService graphDB = new EmbeddedGraphDatabase(args[0]); Node node = graphDB.getNodeById(1); Traverser friends = node.traverse( // Order.DEPTH_FIRST, BREADTH_FIRST // StopEvaluator.END_OF_GRAPH, DEPTH_ONE // ReturnableEvaluator.ALL_BUT_START_NODE, ALL, isReturnableNode() // DynamicRelationshipType.withName("LIKE"), // Direction.OUTGOING); INCOMING, BOTH for (Node nodeBuf : friends) { TraversalPosition currentPosition = friends.currentPosition(); } }
  • 60. Traversal sample Order.BREADTH_FIRST •
  • 61. Traversal sample Order.BREADTH_FIRST •
  • 62. Traversal sample Order.BREADTH_FIRST •
  • 63. Traversal sample Order.BREADTH_FIRST •
  • 64. Traversal sample Order.BREADTH_FIRST •
  • 65. Traversal sample Order.BREADTH_FIRST •
  • 66. Traversal sample Order.DEPTH_FIRST •
  • 67. Traversal sample Order.DEPTH_FIRST •
  • 68. Traversal sample Order.DEPTH_FIRST •
  • 69. Traversal sample Order.DEPTH_FIRST •
  • 70. Traversal sample Order.DEPTH_FIRST •
  • 71. Traversal sample Order.DEPTH_FIRST •
  • 72. Neoclipse sample http://wiki.neo4j.org/content/Neoclipse
  • 73. experiment • mixi Neo4j • • Machine: 24 core CPU, Memory 65GB • Neo4j: BatchInsert, community, embedded • Data • 1500 6 513m17sec (about 8.6h)
  • 74. Network Dataset • Stanford Large Network Dataset Collection • SNAP has a Wide variety of graph data! Social Networks Communication networks Citation networks Collaboration networks Web graphs Product co-purchasing networks Internet peer-to-peer networks Road networks Autonomous systems graphs Signed networks Wikipedia networks and metadata Memetracker and Twitter http://snap.stanford.edu/data/index.html
  • 76. Architecture Service Database Analysis Visualization (Social Graph)
  • 77. Introduction Analyses Sample • Centrality ( ) • Clustering coefficient ( )
  • 78. Centrality ( ) • = Pagerank
  • 79. • • = Vertex ( )
  • 80. • • = Vertex ( ) 1 1 1
  • 81. • • = Vertex ( ) 2 1 1 2 1 2
  • 82. • • = Vertex ( ) 2 1 1 2 1 2
  • 83. • • = Vertex ( ) 2 1 1 4 2 1 2
  • 84. • • = Vertex ( ) 2 1 1 4 2 1 2
  • 85. mixi • 1000 • summary Min 1st Que. Median Mean 3rd Que. Max 1.00 3.00 10.00 25.69 30.00 903.00
  • 86. • ≒
  • 87. • ≒ =0/3=0
  • 88. • ≒ =0/3=0 =1/3
  • 89. • ≒ =0/3=0 =1/3 =2/3
  • 90. • ≒ =0/3=0 =1/3 =2/3 =3/3=1
  • 91. 1000 • summary Min 1st Que. Median Mean 3rd Que. Max 0.00 0.00 0.1157 0.2071 0.2667 1.000
  • 94. 25 0.08
  • 95. 14 0.17
  • 96. 10 0.68
  • 97. 4 1
  • 99. 2hop Social Graph • Edge • ( ) • Vertex • ( ) • Gephi http://gephi.org/
  • 101. Social Graph • • GraphDB • Neo4j • R • Visualization

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. &amp;#x30FB;TC&amp;#x3082;mysql&amp;#x3082;&amp;#x73FE;&amp;#x5F79;&amp;#x3060;&amp;#x3057;&amp;#x3001;&amp;#x5927;&amp;#x597D;&amp;#x304D;\n
  41. &amp;#x30FB;TC&amp;#x3082;mysql&amp;#x3082;&amp;#x73FE;&amp;#x5F79;&amp;#x3060;&amp;#x3057;&amp;#x3001;&amp;#x5927;&amp;#x597D;&amp;#x304D;\n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. \n
  63. \n
  64. \n
  65. \n
  66. \n
  67. \n
  68. \n
  69. \n
  70. \n
  71. \n
  72. \n
  73. \n
  74. \n
  75. \n
  76. \n
  77. \n
  78. \n
  79. \n
  80. \n
  81. \n
  82. \n
  83. \n
  84. \n
  85. \n
  86. \n
  87. \n
  88. \n
  89. \n
  90. \n
  91. \n
  92. \n
  93. \n
  94. \n
  95. \n
  96. \n
  97. \n
  98. \n
  99. \n
  100. \n
  101. \n
  102. \n
  103. \n
  104. \n
  105. \n
  106. \n
  107. \n
  108. \n
  109. \n
  110. \n
  111. \n
  112. \n
  113. \n
  114. \n
  115. \n
  116. \n
  117. \n
  118. \n
  119. \n
  120. \n
  121. \n
  122. \n
  123. \n
  124. \n
  125. \n
  126. \n
  127. \n
  128. \n
  129. \n
  130. \n
  131. \n
  132. \n