Downloaded 177 times
More Related Content
Similar to Interactive visualization and exploration of network data with gephi
![Social Network Analysis [1994]](https://cdn.slidesharecdn.com/ss_thumbnails/socialnetworkanalysis1994-160617072245-thumbnail.jpg?width=640&height=640&fit=bounds)
Interactive visualization and exploration of network data with gephi
- 1. Interactive visualization andexploration of network data with gephi Bernhard Rieder Universiteit van Amsterdam Mediastudies Department and some conceptual context
- 2. Context Terms like "bigdata", "computational social science", "digital humanities", "digital methods", etc. are receiving a lot of attention. They point to a set of practices of knowledge production: data analysis, visualization, modeling, etc. Instead of a totalizing search for a "logic" of data analysis, we could inquire into the vocabulary of concepts and analytical gestures that constitute the practice of data analysis. A twofold approach to methods: ☉ Engagement, development, application => digital methods ☉ Conceptual, historical, and political analysis and critique => software studies
- 3. This workshop How dowe talk about data? How do we analyze them? What is our frame of thought? How do we go further in terms of imagination, expressivity? ☉ Introduction ☉ A bit of math ☉ Two kinds of mathematics ☉ Concepts and techniques from graph theory ☉ Working with gephi Engage the theory of knowledge (epistemology) mobilized in data analysis, but through the actual techniques and not generalizing concepts.
- 4. Basic ideas Why? Why donetwork analysis and visualization? Which arguments are put forward? ☉ New media: technical and conceptual structures modeled as networks ☉ Calculative capacities: powerful techniques and tools ☉ Visualization: the network diagram, "visual analytics" ☉ Logistics: data and software are available ☉ Methodology: dissatisfaction with statistics (SNA) ☉ Society: diversification, problems with demographics / statistics / theory
- 5. Platforms like Twitter boostopportunities for connectivity between various types of actors.
- 6. At the sametime, they produce detailed data traces that are highly centralized and searchable. Much of these data can be analyzed as graphs.
- 7. What styles ofreasoning? Hacking (1991) building the concept of "style of reasoning" on A. C. Crombie’s (1994) "styles of scientific thinking": ☉ postulation and deduction ☉ experiment and empirical research ☉ reasoning by analogy ☉ ordering by comparison and taxonomy ☉ statistical analysis of regularities and probabilities ☉ genetic development What kind of reasoning are we mobilizing in data analysis? Is it one type of reasoning or many? Are we "positivists" when we do data analysis? Reductionists?
- 8. Quality / quantity "Oneof my favorite fantasies is a dialogue between Mills and Lazarsfeld in which the former reads to the latter the first sentence of The Sociological Imagination: 'Nowadays men often feel that their private lives are a series of traps.' Lazarsfeld immediately replies: 'How many men, which men, how long have they felt this way, which aspects of their private lives bother them, do their public lives bother them, when do they feel free rather than trapped, what kinds of traps do they experience, etc., etc., etc.' If Mills succumbed, the two of them would have to apply to the National Institute of Mental Health for a million-dollar grant to check out and elaborate that first sentence. They would need a staff of hundreds, and when finished they would have written Americans View Their Mental Health rather than The Sociological Imagination, provided that they finished at all, and provided that either of them cared enough at the end to bother writing anything." (Maurice Stein, cit. in Gitlin 1978) Theory vs. empiricism, macro vs. micro, qualitative vs. quantitative, inductive vs. deductive, associative vs. formalistic, etc. The promise of data analysis tools, applied to exhaustive (and cheap) data, is to bridge the gap, to allow zooming, "quali-quanti" (Latour 2010).
- 9. Two kinds ofmathematics Can there be data analysis without math? No. Does this imply epistemological commitments? Yes. But there are choices, e.g. between: ☉ Confirmatory data analysis => deductive ☉ Exploratory data analysis (Tukey 1962) => inductive There is a fast growing variety of formal analytical gestures relying on mathematical modeling and computation.
- 10. Two kinds ofmathematics Statistics Observed: objects and properties Inferred: social forces Data representation: the table Visual representation: quantity charts Grouping: "class" (similar properties) Graph-theory Observed: objects and relations Inferred: structure Data representation: the matrix Visual representation: network diagrams Grouping: "clique" (dense relations)
- 11. Graph theory Leonhard Euler,"Seven Bridges of Königsberg", 1735 Introducing the "point and line" model
- 12. Graph theory Develops overthe 20th century, in particular the second half. Integrates branches of mathematics (topology, geometry, statistics, etc.). Graph theory is "the mathematics of structure" (Harary 1965), "a mathematical model for any system involving a binary relation" (Harary 1969); it makes relational structure calculable. "Perhaps even more than to the contact between mankind and nature, graph theory owes to the contact of human beings between each other." (König 1936)
- 13. Basic ideas Moreno 1934 Graphtheory developed in exchange with sociometry, small-group research and (later) social exchange theory. Starting point: "the sociometric test" (experimental definition of "relation")
- 14.
- 15. Forsythe and Katz,1946, "adjacency matrix"
- 16.
- 17. Basic ideas The late1990s The network "singularity": ☉ The network imaginary, a "new science of networks" (Watts 2005) ☉ Computational capacities (memory, speed, interfaces, etc.) ☉ New platforms and datasets ☉ Packaged tools Different traditions conflate to form network analysis: ☉ Social network analysis and sociometrics ☉ Scientometrics / science and technology studies ☉ Mathematics / physics / computer science ☉ Information and data visualization ☉ Digital sociology / new media studies
- 18. Basic ideas Adamic andGlance, "Divided They Blog", 2005
- 19. Formalization "As we haveseen, the basic terms of digraph theory are point and line. Thus, if an appropriate coordination is made so that each entity of an empirical system is identified with a point and each relationship is identified with a line, then for all true statements about structural properties of the obtained digraph there are corresponding true statements about structural properties of the empirical system." (Harary et al. 1965) There is always an epistemological commitment! => What can "carry" the reductionism and formalization? => What types of analytical gestures?
- 20. Facebook Page "ElShaheeed",June 2010 – June 2011, (Poell / Rieder, forthcoming) 7K posts, 700K users, 3.6M comments, 10M likes (tool: netvizz), work in progress!
- 21. Facebook Page "ElShaheeed",June 2010 – June 2011: comment timescatter, log10 y scale, likes on
- 22. Facebook Page "ElShaheeed",June 2010 – June 2011: scatterplot comments / likes, per post type
- 23. Facebook Page "ElShaheeed" 700Knodes, 11M connections Color: type
- 24. Facebook Page "ElShaheeed" 700Knodes, 11M connections Color: outdegree
- 25. Basic ideas What Kindof Phenomena/Data? Interactive networks (Watts 2004): link encodes tangible interaction ☉ social network ☉ citation networks ☉ hypertext networks Symbolic networks (Watts 2004): link is conceptual ☉ co-presence (Tracker Tracker, IMDB, etc.) ☉ co-word ☉ any kind of "structure" that can be as point and line => do all kinds of analysis (SNA, transportation, text mining, etc.) => analyze structure in various ways
- 26. Basic ideas What isa graph? An abstract representation of nodes connected by links. Two ways of dealing with graphs: ☉ mathematical analysis (graph statistics, structural measures, etc.) ☉ visualization (network diagram, matrix, arc diagram, etc.)
- 27. Three different force-basedlayouts of my FB profile OpenOrd, ForceAtlas, Fruchterman-Reingold
- 28. Non force-based layouts Circlediagram, parallel bubble lines, arc diagram
- 29. Network statistics betweenness centrality degree Relationalelements of graphs can be represented as tables (nodes have properties) and analyzed through statistics. Network statistics bridge the gap between individual units and the structural forms they are embedded in. This is currently an extremely prolific field of research.
- 30.
- 31. Basic ideas What isa graph? Vertices and edges! Nodes and lines! Two main types: Directed (e.g. Twitter) Undirected (e.g. Facebook) Properties of nodes: degree, centrality, etc. Properties of edges: weight, direction, etc. Properties of the graph: averages, diameter, communities, etc.
- 32.
- 33. Basic ideas Wikipedia: Glossaryof graph theory Tools are easy, concepts are hard
- 34. Basic ideas Interactive visualanalytics Bringing structure to the surface (gephi panel: "layout") ☉ different spatializations (force, geometry, etc.) Projecting variables into the diagram (gephi panel: "ranking") ☉ Size (nodes, edges, labels, etc.) ☉ Color (nodes, edges, labels, etc.) Deriving measures (gephi panel: "statistics") ☉ Properties of nodes, edges, structure => new variables Analysis: e.g. correlation between spatial layout and variables?
- 35.
- 36.
- 37. Basic ideas Twitter #owsdataset, co-hashtag analysis Strong topic clustering
- 38. Twitter 1% sample,co-hashtag analysis 227,029 unique hashtags, 1627 displayed (freq >= 50) Size: frequency Color: modularity
- 39. Size: frequency Color: userdiversity Twitter 1% sample, co-hashtag analysis 227,029 unique hashtags, 1627 displayed (freq >= 50)
- 40. Size: frequency Color: degree Twitter1% sample, co-hashtag analysis 227,029 unique hashtags, 1627 displayed (freq >= 50)
- 41. Twitter 1% sample Co-hashtaganalysis Degree vs. wordFrequency
- 42. Degree vs. userDiversity Twitter1% sample Co-hashtag analysis
- 43. FB group "Islamis dangerous" Friendship network, color: betweenness centrality 2.339 members Average degree of 39.69 81.7% have at least one friend in the group 55.4% five or more 37.2% have 20 or more founder and admin has 609 friends
- 44. FB group "Islamis dangerous" Friendship network, color: Interface language en_us, de, en_uk, it dominate
- 45. Mapping European Extremism Friendshiprelations of 18 extreme-right groups
- 46. FB page "Educatechildren about the evils of Islam" Links have more comments, photos more likes.
- 47. FB page "Stopthe Islamization of the World" Number of posts and reactions
- 48. FB page "Stopthe Islamization of the World"
- 49. Basic ideas Interactive visualanalytics Bringing structure to the surface (gephi panel: "layout") ☉ different spatializations (force, geometry, etc.) Projecting variables into the diagram (gephi panel: "ranking") ☉ Size (nodes, edges, labels, etc.) ☉ Color (nodes, edges, labels, etc.) Deriving measures (gephi panel: "statistics") ☉ Properties of nodes, edges, structure => new variables Analysis: e.g. correlation between spatial layout and variables?
- 50.
- 51. Nine measures ofcentrality (Freeman 1979)
- 52. Label PR α=0.85PR α=0.7 PR α=0.55 PR α=0.4 In-Degree Out-Degree Degree n34 0.0944 0.0743 0.0584 0.0460 4 1 5 n1 0.0867 0.0617 0.0450 0.0345 1 2 3 n17 0.0668 0.0521 0.0423 0.0355 2 1 3 n39 0.0663 0.0541 0.0453 0.0388 5 1 6 n22 0.0619 0.0506 0.0441 0.0393 5 1 6 n27 0.0591 0.0451 0.0371 0.0318 1 0 1 n38 0.0522 0.0561 0.0542 0.0486 6 0 6 n11 0.0492 0.0372 0.0306 0.0274 3 1 4
- 53.
- 54. Thank You rieder@uva.nl https://www.digitalmethods.net http://thepoliticsofsystems.net "Far betteran approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise. Data analysis must progress by approximate answers, at best, since its knowledge of what the problem really is will at best be approximate." (Tukey 1962)
Editor's Notes
- #6 Anatomy of a tweet. https://twitter.com/ICIJorg/status/321585235491962880https://api.twitter.com/1/statuses/show/321585235491962880.json
- #7 Very large scale systems on the one side, but highly concentrated data repositories on the other.The promise of data analysis is, of course, to use that data to make sense of all the complexity.
- #8 Reduction to the point and line model requires purging of context, etc. but provides considerable computational capacities. No free lunch, as it were.
- #9 Gitlin, The Dominant Paradigm, 1978Many people argue that we no longer need that grant, we already have the data.
- #10 Tukey, The Future of Data Analysis, 1962
- #11 Allows for all kinds of folding, combinations, etc. – Math is not homogeneous, but sprawling!Different forms of reasoning, different modes of aggregation.These are already analytical frameworks, different ways of formalizing.Statistics: atomism, structure is implicit ("hidden forces", "social forces" cf. Durhkeim) => groups are abstractions, constituted by socioeconomic similaritySocial Network Analysis: atomism, structure is explicit ("dyadic forces") => groups are concrete, constituted by social exchange
- #16 Now we can calculate (in particular via matrix algebra).
- #17 Handbooks on graph theory are full of exhaustive discussions of basic graph types. Loads of vocabulary and analytical approaches.
- #21 http://www.facebook.com/ElShaheeed (Created by WaelGhonim, considered to be a central place for the sparking of the Egyptian Revolution)http://apps.facebook.com/netvizz/ (tool used for extraction)
- #22 Classic, simple statistics are often very useful.
- #23 It's easy and interesting to produce correlations.
- #24 A very different concept of "relation" – no longer derived and probabilistic (correlation) but explicit.Formalization: here: posts and users, but we could do many other things, e.g. users linked by co-reaction, posts linked by co-reaction, we could look at relationships between words, etc.How do we interpret this: understand the platform, understand the context of the phenomenon, understand the algorithm, etc.
- #25 How do we interpret something like this?
- #28 Visualization is, again, one type of analysis.Which properties of the network are "made salient" by an algorithm?http://thepoliticsofsystems.net/2010/10/one-network-and-four-algorithms/Models behind: spring simulation, simulated annealing (http://wiki.cns.iu.edu/pages/viewpage.action?pageId=1704113)
- #29 Non force-based layouts can be extremely useful. Gephi can produce those as well
- #39 Extend word lists (what am I missing?), account for refraction. Rieder & Gerlitz 2013: http://journal.media-culture.org.au/index.php/mcjournal/article/viewArticle/620Rieder 2012: http://firstmonday.org/ojs/index.php/fm/article/view/4199/3359
- #40 Project variables into the graph User diversity = no of unique users of a hashtag divided by hashtag frequency
- #41 Larger roles of hashtags, not all are issue markers!
- #42 There is no need to analyze and visualize a graph as a network.Characterize hashtags in relation to a whole. (their role beyond a particular topic sample), better understand our "fishing pole" (the sample technique) and the weight it carries.Tbt: throwback thursday
- #44 From DMI workshop on anti-Islamism and right-wing extremism.We can also look at interaction patters: activity structure, held together by leaders?
- #45 From DMI workshop on anti-Islamism and right-wing extremism.Netvizz also allows for looking into interaction patterns in groups..
- #46 Unique user id allows for large scale analysis.The connectors are often the admins.See: https://wiki.digitalmethods.net/CounterJihadism/ProjectGroup7
- #47 We can also look at interaction patters: activity structure, held together by leaders?
- #48 Combination of methods is most interesting.The move to posting a larger number of photos is highly successful for this page.What is happening in April 2012?For more details see: https://wiki.digitalmethods.net/CounterJihadism/ProjectGroup3Network diagrams are not so good with time.
- #49 Simply looking at "images" in quantitative terms is not enough, here the qualitative part begins and netvizz makes it easier to take that step:Extracting photo URLsExtracting commentsStudying most "successful" tropes in depth, etc.For more details see: https://wiki.digitalmethods.net/CounterJihadism/ProjectGroup3
- #52 Network analysis has produced a large number of calculated metrics that take into account the structure of the network."All in all, this process resulted in the specification of nine centrality measures based on three conceptual foundations. Three are based on the degrees of points and are indexes of communication activity. Three are based on the betweenness of points and are indexes of potential for control of communication. And three are based on closeness and are indexes either of independence or efficiency." (Freeman 1979)What concepts are these metricsbased on?
- #53 Network metrics are highly dependent on individual variables. Here: the same network with PageRank with four different values for the dampening parameter alpha. (red=highest PR value, yellow=second highest, turquoise=third highest)See Rieder 2012: http://computationalculture.net/article/what_is_in_pagerank