Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 1
T...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 2
T...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 3
K...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 4
T...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 5
S...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 6
A...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 7
B...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 8
S...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 9
E...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 10
...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 11
...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 12
...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 13
...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 14
...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 15
...
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 16
...
Upcoming SlideShare
Loading in …5
×

Towards Validating Social Network Simulations

547 views

Published on

A paper presented at ESSA 2013, Warsaw. Abstract: We consider the problem of finding suitable measures to validate simulated networks as outcome of (agent-based) social simulations. A number of techniques from computer science and social sciences are reviewed in this paper, which tries to compare and ‘fit’ various simulated networks to the avail-able data by using network measures. We look at several social network analy-sis measures but then turn our focus to techniques that not only consider the po-sition of the nodes but also their characteristics and their tendency to cluster with other nodes in the network – subgroup identification. We discuss how stat-ic and dynamic nature of networks may be compared. We conclude by urging a more comprehensive, transparent and rigorous approach to comparing simula-tion-generated networks against the available data.

Published in: Education, Travel, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
547
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Towards Validating Social Network Simulations

  1. 1. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 1 Towards Validating Social Network Simulations SMA Abbas1, Shah Jamal Alam2 and Bruce Edmonds1 1Centre for Policy Modelling, Manchester Metropolitan University 2School of Geosciences, University of Edinburgh
  2. 2. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 2 The Situation A Simulation Social “System” Generates Measured Are these “essentially” the same? A Class of Networks Another Class of Networks ?
  3. 3. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 3 Key questions • What properties of the synthetic networks, one would expect to observe given how the model has been constructed • Which of these properties are „significant‟ in terms of the intended processes in the model • Which class of target networks one might expect to observe if one could “re-run” reality under the same basic conditions as assumed in the model • Do these classes match in important respects • How do we know they do given we only have samples of synthetic and target networks
  4. 4. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 4 The Problem
  5. 5. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 5 Summary of Issue • The space of possible networks is vast • But many networks will look similar to us, because our brains can not deal with them but automatically simplifies them as part of perception • We are not dealing with single networks but classes of networks… • …though these classes are often implicit when a single network stands for that class (somehow) • However, in principle, if synthetic and target networks do match (in some way) then this is potentially a strong validation
  6. 6. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 6 A Cautionary Tale – comparing two network models Papadopoulos et al. (2012) Popularity versus similarity in growing networks. Nature, 489:537-540.
  7. 7. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 7 But when compared in a different way…
  8. 8. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 8 Some Network Comparison Approaches Different kinds of things to compare: • Network Measures • Network Distributions • Eigenvalue/Eigenvectors • Subgroup Identification • Functional Comparison • Likelihood of being described by an Exponential Random Graph Model • Motif Prevalence
  9. 9. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 9 Examples that follow are from work of S.M.A. Abbas (see papers at https://sites.google.com/site/maliabbas) An Example of Validating Synthetic vs. Target Networks
  10. 10. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 10 An example of comparing measures
  11. 11. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 11 An example of comparing distributions 1 5 10 50 100 500 1000 5e-045e-035e-025e-01 Log-log Plot of Degree Distribution Degree CumulativeFrequency Reference Random FAOF Party Hybrid
  12. 12. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 12 Silo Index Comparison Correlation 0.83 -1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -1.00-0.85-0.70 Dorm Silo Index Reference Dorm HybridDorm Correlation 0.93 -1.00 -0.90 -0.80 -0.70 -1.00-0.90-0.80 Major Silo Index Reference Major HybridMajorCorrelation 0.84 -1.0 -0.5 0.0 0.5 -1.0-0.8-0.6 Year Silo Index Reference Year HybridYear Correlation 0.29 -1.0 -0.8 -0.6 -1.00-0.90 High School Silo Index Reference High School HybridHighSchool Reference vs. Hybrid Mode Silo Indices
  13. 13. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 13 Assortativity Mixing
  14. 14. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 14 Problems in the Literature Authors are often not clear about: • Precisely what the links in a synthetic network are supposed to represent (in terms that would allow an in principle measurement of observed actors) • Which aspects of the target network are subject to measurement error (or otherwise judged not to be significant) and which should be reproduced by a synthetic network • Which aspects of the synthetic network are significant in terms of the generating process (and which are essentially accidental) Readers often cannot judge the extent or meaning of the match/mismatch between synthetic and target networks
  15. 15. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 15 Conclusions • Many social simulation models assume stereotypic networks (e.g. Watts-Strogatz) • It is increasingly clear that the exact network structure matters (e.g. Holzhauer ESSA 2013) • No single „Golden Bullet‟ technique • More thought needed about what is significant about the synthetic and target class networks • Multiple approaches needed to show that classes of networks are similar – a few 1D measures is not enough to show this • Validating networks could be quite a strong validation of our models… • …but much more work is needed in this area!
  16. 16. Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 16 Thanks! SMA Abbas https://sites.google.com/site/maliabbas Shah Jamal Alam https://sites.google.com/site/jamialam Bruce Edmonds http://bruce.edmonds.name Slides at: http://www.slideshare.net/BruceEdmonds

×