Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Model Replication in the Context of Agent-based Simulation

1,976 views

Published on

My presentation held at the 1st European Conference on Political Attitudes and Mentalities (ECPAM 2012) conference, Bucharest, Romania, September 3-5, 2012.

Electronic paper link:
http://mass.aitia.ai/images/publikaciok/2012-ecpam-replication_case_studies-camera_ready.pdf

Abstract: This paper examines model replication in the context of agent-based simulation through two case studies. Replication of a computational model and validation of its results is an essential tool for scientific researchers, but it is rarely used by modelers. In our work we address the question of validating and verifying simulations in general, and summarize our experience in approaching different models through replication with different motivations. Two models are discussed in details. The first one is an agent-based spatial adaptation of a numerical model, while the second experiment addresses the exact replication of an existing economic model.

Published in: Technology, Economy & Finance
  • Be the first to comment

  • Be the first to like this

Model Replication in the Context of Agent-based Simulation

  1. 1. Richárd O. Legéndi, László Gulyás, Yuri Mansury Eötvös Loránd University, AITIA International, Inc., Cornell University rlegendi@aitia.ai, lgulyas@aitia.ai, ysm3@cornell.eduThis work was partially supported by the Hungarian Government (KMOP-1.1.2-08/1-2008-0002), the European UnionSeventh Framework Programme FP7/2007-2013 under grant agreement CRISIS-ICT-2011-288501 (CRISIS –Complexity Research Initiative for Systemic InstabilitieS) and mOSAIC 2011-256910 (Open-Source API and Platformfor Multiple Clouds). These supports are gratefully acknowledged. 1st European Conference on Political Attitude and Mentality ECPAM 2012, Bucharest, September 3-5, 2012
  2. 2. Layout  Motivation and Background  Replication? Why care?  Case Studies and Results  ABM approach for the New Economic Geography  Replication of the Bottom-up Adaptive Macroeconomics  Summary2012.09.03. ECPAM 2012 - Replication case studies 2
  3. 3. 2012.09.03. ECPAM 2012 - Replication case studies 3
  4. 4. Replication? Why care?  Replication of experiments, validation of results are essential  „Simulations as experiments”  If cannot be reproduced, its scientific value is in question  Models never replicated - except a few classical ones  Helps us get a deeper understanding  Of relevant properties, key issues  Deploy simulation as a research tool2012.09.03. ECPAM 2012 - Replication case studies 4
  5. 5. Validation?  Docking – alingment of different models  Different computational models for the same phenomenon  Replication  W/o being able to replicate results of an artificial model, how to target real-world systems?  Several problems, e.g. ambiguity  Different approaches exist (AgentUML, ODD, etc.)  But there’s no consensus on using them...2012.09.03. ECPAM 2012 - Replication case studies 5
  6. 6. An Agent-Based Adaptation of the New Economic Geography2012.09.03. ECPAM 2012 - Replication case studies 6
  7. 7. New Economic Geography  Paul Krugman’s city formation model  Originally a numerical model  Applied agent-based approach Masahisa Fujita, Paul Krugman, Anthony J. Venables: „The Spatial Economy.” MIT Press, Cambridge, MA, 1999.2012.09.03. ECPAM 2012 - Replication case studies 7
  8. 8. Model Structure2012.09.03. ECPAM 2012 - Replication case studies 8
  9. 9. Zipf’s Law in City Formation City Population Rank (2010) New York 8,175,133 1 Los Angeles 3,792,621 2 Chicago 2,695,598 3 Houston 2,099,451 4 Philadelphia 1,526,006 5 Phoenix 1,445,632 6 San Antonio 1,327,407 7 San Diego 1,307,402 8 Dallas 1,197,816 9 San Jose, CA 945,942 102012.09.03. ECPAM 2012 - Replication case studies 9
  10. 10. Motivation  Previous works explains Zipf’s law successfully  But lacks micro-foundations  We extended th FKV model  General-equilibrium model  Excellent micro-foundations  But cannot generate a hierarchical system of cities2012.09.03. ECPAM 2012 - Replication case studies 10
  11. 11. Why the Agent-Based approach?  Introduce heterogeneity  Noise  Agent-specific migration thresholds  Enables migration to proceed in a non ad-hoc way  Extensibility2012.09.03. ECPAM 2012 - Replication case studies 11
  12. 12. Results  We proposed a spatial AB version of FKV  Applied an inherently different approach  Retains the key features of the original model  Including consumers’ love for varieties  Increasing returns in production  Tension between centripetal (agglomeration) and centrifugal (dispersion) forces2012.09.03. ECPAM 2012 - Replication case studies 12
  13. 13. Tomahawk-diagram  Population migration (λ) vs. „freeness” of trade (φ)  Break and sustain point  φB and φS  Closed-form solution and implicit function to evaluate2012.09.03. ECPAM 2012 - Replication case studies 13
  14. 14. Replication Results  Simulations replicates expected results  t = 2000 / 5000 time steps  φB and φS verified2012.09.03. ECPAM 2012 - Replication case studies 14
  15. 15. Replication of the Macroeconomics from the Bottom-up2012.09.03. ECPAM 2012 - Replication case studies 15
  16. 16. Macroeconomics from the Bottom-up  Agent-based macro model  Empirical external validation  Using real-world data  Replication of the same model  In a different environment Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati: Macroeconomics from the Bottom-up. 1st ed. Springer, 2011.2012.09.03. ECPAM 2012 - Replication case studies 16
  17. 17. Model Structure Source: Domenico Delli Gatti, personal communications2012.09.03. ECPAM 2012 - Replication case studies 17
  18. 18. Agents  Households  Supply labor  Buy consumption goods  Hold deposits  Firms  Demand labor  Produce and sell consumption goods  Bank  Receive deposits from households  Extend loans to firms2012.09.03. ECPAM 2012 - Replication case studies 18
  19. 19. Market Processes I 1. Fims compute net worth, production/price and labour demand 2. Credit market: 1. Bank decides credit conditions 2. Firms decide to whether take loan or not 3. Job market: 1. Firms redefine labour demand, publish vacancies: 1. Excess workforce: fire workers 2. Insufficient workforce: hire if possible2012.09.03. ECPAM 2012 - Replication case studies 19
  20. 20. Market Processes II 4. Consumption goods market: 1. Workers get wages and compute consumption budget 2. Firms post their price 3. Consumers contact z firms randomly  Ordered by price 4. Unspent money  Involuntary savings 5. Unsold goods  Sold at zero cost (non-durable) 5. Accounting 1. Firms calculate profits 2. Earnings are retained profits  Used to update net worth.2012.09.03. ECPAM 2012 - Replication case studies 20
  21. 21. Why to replicate? Parameter sweeps „[...] suppose that in a model there are just 10 relevant parameters, and that each parameter can assume 10 different values (a rather simplifying assumption). As a result, one obtains that the constellation of the parameter space is given by 10^10 vectors. If we perform 20 different runs for each one of them to take into account the possible effects of changing the random seeds, the total number of simulations would amount to 2*10^11!” Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati: Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)2012.09.03. ECPAM 2012 - Replication case studies 21
  22. 22. Why to replicate?  In a different environment?  Matlab  Java/Mason  Efficiency  Reduce required time for a single simulation run  Tool support: MEME  Parameter sweep exploration  Being Strong  Exploiting Grid/Cloud systems  Being Smart  Design of Experiments2012.09.03. ECPAM 2012 - Replication case studies 22
  23. 23. Background “The CRISIS project addresses building a next generation macroeconomic and financial system policymaking model: a bottom-up agent-based simulation that fully accounts for the heterogeneity of households, firms, and government actors. The model will incorporate the latest evidence from behavioral economics in portraying agent behavior, and the CRISIS team will also collect new data on agent decision making using experimental economics techniques. While any model must make simplifying assumptions about human behavior, the CRISIS model will be significantly more realistic in its portrayal of relevant agent behavior than the current generation of policymaking models.” Crisis project description: https://www.crisis-economics.eu/2012.09.03. ECPAM 2012 - Replication case studies 23
  24. 24. Replicated Model Modelling Economic Simulator Framework (Cloud-Based Parameter Sweep Execution) Models Web-based Game (Participatory Experiments)2012.09.03. ECPAM 2012 - Replication case studies 24
  25. 25. Results I - Benchmarking2012.09.03. ECPAM 2012 - Replication case studies 25
  26. 26. Result II – Verification  Scaled agents (w/o changing overall ratio)  Up to 7500 agents  Avg’d 40 runs  t = 1000 time steps  Included initial state  High oscillations  Until spontaneous order emerges („equilibrium”)2012.09.03. ECPAM 2012 - Replication case studies 26
  27. 27. 2012.09.03. ECPAM 2012 - Replication case studies 27
  28. 28. 2012.09.03. ECPAM 2012 - Replication case studies 28
  29. 29. Summary: Case Study 1  We created a replication of the FKV by using a different approach  Retains hallmark of the original model  Introduced heterogeneity at several levels  Allows further studies  With different activation regimes  N-cities model2012.09.03. ECPAM 2012 - Replication case studies 29
  30. 30. Project Info http://emergingcities.aitia.ai2012.09.03. ECPAM 2012 - Replication case studies 30
  31. 31. Summary: Case Study 2  We created a replication of the MacroABM model in a different environment  Identic output  Results are platform, environment-independent  Opens up the window of standardized simulation tools  Extensive parameter space explorations (MEME)  Performance speedup  By the factor 5x-10x  On the other hand, code length is increased similarly:  Matlab: ~300 LoC  Java: 1500 + 1000 LoC2012.09.03. ECPAM 2012 - Replication case studies 31
  32. 32. Download! http://www.crisis-economics.eu/jmark-i-build-report2012.09.03. ECPAM 2012 - Replication case studies 32
  33. 33. Richard O. Legendi Mail: rlegendi@aitia.ai Twitter: @legendi_ELTE Blog: http://xcafebabe.blogspot.com Web: http://people.inf.elte.hu/legendi/ September 3., 2012.Emerging Cities Website: http://emergingcities.aitia.ai/ Crsisis Website: http://www.crisis-economics.eu/

×