Richárd O. Legéndi, László Gulyás, Yuri Mansury      Eötvös Loránd University, AITIA International, Inc., Cornell Universi...
Layout  Motivation and Background     Replication? Why care?  Case Studies and Results     ABM approach for the New Ec...
2012.09.03.   ECPAM 2012 - Replication case studies   3
Replication? Why care?  Replication of experiments, validation of results are   essential  „Simulations as experiments” ...
Validation?  Docking – alingment of different models     Different computational models for the same      phenomenon  R...
An Agent-Based Adaptation of the New Economic Geography2012.09.03.     ECPAM 2012 - Replication case studies      6
New Economic Geography                                             Paul Krugman’s city                                   ...
Model Structure2012.09.03.   ECPAM 2012 - Replication case studies   8
Zipf’s Law in City Formation        City     Population         Rank                   (2010)  New York           8,175,13...
Motivation  Previous works explains Zipf’s law successfully     But lacks micro-foundations  We extended th FKV model  ...
Why the Agent-Based approach?  Introduce heterogeneity     Noise     Agent-specific migration thresholds  Enables migr...
Results  We proposed a spatial AB version of FKV  Applied an inherently different approach  Retains the key features of...
Tomahawk-diagram                                             Population migration (λ)                                    ...
Replication Results                                             Simulations replicates                                   ...
Replication of the Macroeconomics from the Bottom-up2012.09.03.      ECPAM 2012 - Replication case studies   15
Macroeconomics from the Bottom-up                                             Agent-based macro model                    ...
Model Structure              Source: Domenico Delli Gatti, personal communications2012.09.03.        ECPAM 2012 - Replicat...
Agents  Households     Supply labor     Buy consumption goods     Hold deposits  Firms     Demand labor     Produce...
Market Processes I 1. Fims compute net worth, production/price and    labour demand 2. Credit market:       1.          Ba...
Market Processes II 4. Consumption goods market:   1. Workers get wages and compute consumption budget   2. Firms post the...
Why to replicate? Parameter sweeps „[...] suppose that in a model there are just 10 relevant parameters, and that each par...
Why to replicate?  In a different environment?     Matlab  Java/Mason  Efficiency     Reduce required time for a sing...
Background “The CRISIS project addresses building a next generation macroeconomic and financial system policymaking model:...
Replicated           Model                 Modelling                                    Economic Simulator                ...
Results I - Benchmarking2012.09.03.   ECPAM 2012 - Replication case studies   25
Result II – Verification  Scaled agents (w/o changing overall ratio)     Up to 7500 agents  Avg’d 40 runs  t = 1000 ti...
2012.09.03.   ECPAM 2012 - Replication case studies   27
2012.09.03.   ECPAM 2012 - Replication case studies   28
Summary: Case Study 1  We created a replication of the FKV by using a     different approach        Retains hallmark of ...
Project Info                                          http://emergingcities.aitia.ai2012.09.03.   ECPAM 2012 - Replication...
Summary: Case Study 2  We created a replication of the MacroABM model in a     different environment        Identic outp...
Download!              http://www.crisis-economics.eu/jmark-i-build-report2012.09.03.       ECPAM 2012 - Replication case ...
Richard O. Legendi                                    Mail: rlegendi@aitia.ai                                   Twitter: @...
Upcoming SlideShare
Loading in …5
×

Model Replication in the Context of Agent-based Simulation

1,726 views
1,677 views

Published on

My presentation held at the 1st European Conference on Political Attitudes and Mentalities (ECPAM 2012) conference, Bucharest, Romania, September 3-5, 2012.

Electronic paper link:
http://mass.aitia.ai/images/publikaciok/2012-ecpam-replication_case_studies-camera_ready.pdf

Abstract: This paper examines model replication in the context of agent-based simulation through two case studies. Replication of a computational model and validation of its results is an essential tool for scientific researchers, but it is rarely used by modelers. In our work we address the question of validating and verifying simulations in general, and summarize our experience in approaching different models through replication with different motivations. Two models are discussed in details. The first one is an agent-based spatial adaptation of a numerical model, while the second experiment addresses the exact replication of an existing economic model.

Published in: Technology, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,726
On SlideShare
0
From Embeds
0
Number of Embeds
1,035
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Earnings after interest payments and dividends
  • Harmonized library, Game,policymaking model
  • The benefits we gained is on one hand, technical. Over 5 magnitudes (20 min. vs. 3 min.)
  • Work in Progress
  • Model Replication in the Context of Agent-based Simulation

    1. 1. Richárd O. Legéndi, László Gulyás, Yuri Mansury Eötvös Loránd University, AITIA International, Inc., Cornell University rlegendi@aitia.ai, lgulyas@aitia.ai, ysm3@cornell.eduThis work was partially supported by the Hungarian Government (KMOP-1.1.2-08/1-2008-0002), the European UnionSeventh Framework Programme FP7/2007-2013 under grant agreement CRISIS-ICT-2011-288501 (CRISIS –Complexity Research Initiative for Systemic InstabilitieS) and mOSAIC 2011-256910 (Open-Source API and Platformfor Multiple Clouds). These supports are gratefully acknowledged. 1st European Conference on Political Attitude and Mentality ECPAM 2012, Bucharest, September 3-5, 2012
    2. 2. Layout  Motivation and Background  Replication? Why care?  Case Studies and Results  ABM approach for the New Economic Geography  Replication of the Bottom-up Adaptive Macroeconomics  Summary2012.09.03. ECPAM 2012 - Replication case studies 2
    3. 3. 2012.09.03. ECPAM 2012 - Replication case studies 3
    4. 4. Replication? Why care?  Replication of experiments, validation of results are essential  „Simulations as experiments”  If cannot be reproduced, its scientific value is in question  Models never replicated - except a few classical ones  Helps us get a deeper understanding  Of relevant properties, key issues  Deploy simulation as a research tool2012.09.03. ECPAM 2012 - Replication case studies 4
    5. 5. Validation?  Docking – alingment of different models  Different computational models for the same phenomenon  Replication  W/o being able to replicate results of an artificial model, how to target real-world systems?  Several problems, e.g. ambiguity  Different approaches exist (AgentUML, ODD, etc.)  But there’s no consensus on using them...2012.09.03. ECPAM 2012 - Replication case studies 5
    6. 6. An Agent-Based Adaptation of the New Economic Geography2012.09.03. ECPAM 2012 - Replication case studies 6
    7. 7. New Economic Geography  Paul Krugman’s city formation model  Originally a numerical model  Applied agent-based approach Masahisa Fujita, Paul Krugman, Anthony J. Venables: „The Spatial Economy.” MIT Press, Cambridge, MA, 1999.2012.09.03. ECPAM 2012 - Replication case studies 7
    8. 8. Model Structure2012.09.03. ECPAM 2012 - Replication case studies 8
    9. 9. Zipf’s Law in City Formation City Population Rank (2010) New York 8,175,133 1 Los Angeles 3,792,621 2 Chicago 2,695,598 3 Houston 2,099,451 4 Philadelphia 1,526,006 5 Phoenix 1,445,632 6 San Antonio 1,327,407 7 San Diego 1,307,402 8 Dallas 1,197,816 9 San Jose, CA 945,942 102012.09.03. ECPAM 2012 - Replication case studies 9
    10. 10. Motivation  Previous works explains Zipf’s law successfully  But lacks micro-foundations  We extended th FKV model  General-equilibrium model  Excellent micro-foundations  But cannot generate a hierarchical system of cities2012.09.03. ECPAM 2012 - Replication case studies 10
    11. 11. Why the Agent-Based approach?  Introduce heterogeneity  Noise  Agent-specific migration thresholds  Enables migration to proceed in a non ad-hoc way  Extensibility2012.09.03. ECPAM 2012 - Replication case studies 11
    12. 12. Results  We proposed a spatial AB version of FKV  Applied an inherently different approach  Retains the key features of the original model  Including consumers’ love for varieties  Increasing returns in production  Tension between centripetal (agglomeration) and centrifugal (dispersion) forces2012.09.03. ECPAM 2012 - Replication case studies 12
    13. 13. Tomahawk-diagram  Population migration (λ) vs. „freeness” of trade (φ)  Break and sustain point  φB and φS  Closed-form solution and implicit function to evaluate2012.09.03. ECPAM 2012 - Replication case studies 13
    14. 14. Replication Results  Simulations replicates expected results  t = 2000 / 5000 time steps  φB and φS verified2012.09.03. ECPAM 2012 - Replication case studies 14
    15. 15. Replication of the Macroeconomics from the Bottom-up2012.09.03. ECPAM 2012 - Replication case studies 15
    16. 16. Macroeconomics from the Bottom-up  Agent-based macro model  Empirical external validation  Using real-world data  Replication of the same model  In a different environment Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati: Macroeconomics from the Bottom-up. 1st ed. Springer, 2011.2012.09.03. ECPAM 2012 - Replication case studies 16
    17. 17. Model Structure Source: Domenico Delli Gatti, personal communications2012.09.03. ECPAM 2012 - Replication case studies 17
    18. 18. Agents  Households  Supply labor  Buy consumption goods  Hold deposits  Firms  Demand labor  Produce and sell consumption goods  Bank  Receive deposits from households  Extend loans to firms2012.09.03. ECPAM 2012 - Replication case studies 18
    19. 19. Market Processes I 1. Fims compute net worth, production/price and labour demand 2. Credit market: 1. Bank decides credit conditions 2. Firms decide to whether take loan or not 3. Job market: 1. Firms redefine labour demand, publish vacancies: 1. Excess workforce: fire workers 2. Insufficient workforce: hire if possible2012.09.03. ECPAM 2012 - Replication case studies 19
    20. 20. Market Processes II 4. Consumption goods market: 1. Workers get wages and compute consumption budget 2. Firms post their price 3. Consumers contact z firms randomly  Ordered by price 4. Unspent money  Involuntary savings 5. Unsold goods  Sold at zero cost (non-durable) 5. Accounting 1. Firms calculate profits 2. Earnings are retained profits  Used to update net worth.2012.09.03. ECPAM 2012 - Replication case studies 20
    21. 21. Why to replicate? Parameter sweeps „[...] suppose that in a model there are just 10 relevant parameters, and that each parameter can assume 10 different values (a rather simplifying assumption). As a result, one obtains that the constellation of the parameter space is given by 10^10 vectors. If we perform 20 different runs for each one of them to take into account the possible effects of changing the random seeds, the total number of simulations would amount to 2*10^11!” Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati: Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)2012.09.03. ECPAM 2012 - Replication case studies 21
    22. 22. Why to replicate?  In a different environment?  Matlab  Java/Mason  Efficiency  Reduce required time for a single simulation run  Tool support: MEME  Parameter sweep exploration  Being Strong  Exploiting Grid/Cloud systems  Being Smart  Design of Experiments2012.09.03. ECPAM 2012 - Replication case studies 22
    23. 23. Background “The CRISIS project addresses building a next generation macroeconomic and financial system policymaking model: a bottom-up agent-based simulation that fully accounts for the heterogeneity of households, firms, and government actors. The model will incorporate the latest evidence from behavioral economics in portraying agent behavior, and the CRISIS team will also collect new data on agent decision making using experimental economics techniques. While any model must make simplifying assumptions about human behavior, the CRISIS model will be significantly more realistic in its portrayal of relevant agent behavior than the current generation of policymaking models.” Crisis project description: https://www.crisis-economics.eu/2012.09.03. ECPAM 2012 - Replication case studies 23
    24. 24. Replicated Model Modelling Economic Simulator Framework (Cloud-Based Parameter Sweep Execution) Models Web-based Game (Participatory Experiments)2012.09.03. ECPAM 2012 - Replication case studies 24
    25. 25. Results I - Benchmarking2012.09.03. ECPAM 2012 - Replication case studies 25
    26. 26. Result II – Verification  Scaled agents (w/o changing overall ratio)  Up to 7500 agents  Avg’d 40 runs  t = 1000 time steps  Included initial state  High oscillations  Until spontaneous order emerges („equilibrium”)2012.09.03. ECPAM 2012 - Replication case studies 26
    27. 27. 2012.09.03. ECPAM 2012 - Replication case studies 27
    28. 28. 2012.09.03. ECPAM 2012 - Replication case studies 28
    29. 29. Summary: Case Study 1  We created a replication of the FKV by using a different approach  Retains hallmark of the original model  Introduced heterogeneity at several levels  Allows further studies  With different activation regimes  N-cities model2012.09.03. ECPAM 2012 - Replication case studies 29
    30. 30. Project Info http://emergingcities.aitia.ai2012.09.03. ECPAM 2012 - Replication case studies 30
    31. 31. Summary: Case Study 2  We created a replication of the MacroABM model in a different environment  Identic output  Results are platform, environment-independent  Opens up the window of standardized simulation tools  Extensive parameter space explorations (MEME)  Performance speedup  By the factor 5x-10x  On the other hand, code length is increased similarly:  Matlab: ~300 LoC  Java: 1500 + 1000 LoC2012.09.03. ECPAM 2012 - Replication case studies 31
    32. 32. Download! http://www.crisis-economics.eu/jmark-i-build-report2012.09.03. ECPAM 2012 - Replication case studies 32
    33. 33. Richard O. Legendi Mail: rlegendi@aitia.ai Twitter: @legendi_ELTE Blog: http://xcafebabe.blogspot.com Web: http://people.inf.elte.hu/legendi/ September 3., 2012.Emerging Cities Website: http://emergingcities.aitia.ai/ Crsisis Website: http://www.crisis-economics.eu/

    ×