This document outlines tools for statistical and behavioral analysis in agent-based modeling, particularly focusing on the El Farol Bar problem and the Crisis project, aimed at enhancing macroeconomic policymaking through realistic simulations. It discusses various tools like PET v2.0 and MEME for conducting participatory experiments and parameter sweeps, while highlighting the need for modernized infrastructure in agent-based modeling. The presentation includes details about future work, existing resources, and collaboration opportunities in the field of agent-based economics.

1. TOOLS FOR STATISTICAL AND BEHAVIORAL
ANALYSIS OF MASON MODELS
László Gulyás, Richard O. Legendi, Tamás Máhr
Rajmund Bocsi, Vilmos Kozma, Peter Rieger
{lgulyas, rlegendi, tmahr, rbocsi, vkozma, prieger}@aitia.ai
AITIA International, Inc. – Intelligent Applications and Web Services
NSF MASON Workshop
George Mason University
15-16 June, 2013, Fairfax, USA
http://cs.gmu.edu/~eclab/projects/mason/workshop/pmwiki.php
The research leading to these results has received funding from the
European Union, Seventh Framework Programme FP7/2007-2013
under grant agreement no. CRISIS-ICT-2011-288501.

2. DEMO
2013.06.15. NSF MASON Workshop 2

3. The El Farol Bar Problem
• Agents are researchers (N=100)
• They visit a popular but small bar in Santa Fé
• If attendance > 60   (overcrowded)
• Who hasn’t come  
• If attendance <= 60  
• Who hasn’t come  
• Each day agents decide individually and in the same time
W. Brian Arthur, “Inductive Reasoning and Bounded Rationality”, American Economic Review
(Papers and Proceedings), 84,406–411, 1994.
2013.06.15. NSF MASON Workshop 3

4. The El Farol Bar Problem
• MASON implementation of a NetLogo variant
• Artifical agents:
• ARMA-based prediction with history
• Players have two actions: No go / Go
• If   +1 Score!
• Goal: get max score
Rand, W. and Wilensky, U. (2007). NetLogo El Farol model. Center for Connected Learning and
Computer-Based Modeling, Northwestern Institute on Complex Systems, Northwestern
University, Evanston, IL. http://ccl.northwestern.edu/netlogo/models/ElFarol
2013.06.15. NSF MASON Workshop 4

5. Let’s play!
http://demo1.aitia.ai/
 Simulations
2013.06.15. NSF MASON Workshop 5

6. 2013.06.15. NSF MASON Workshop 6
Model List
Create
Experiments
Additional Info

7. Admin Page (Admins are subjects too)
2013.06.15. NSF MASON Workshop 7
Subject list
(joined)
Admin tools

8. In-experiment Admin Page
2013.06.15. NSF MASON Workshop 8
Player status
(moved/waiting)

9. In-experiment Subject Page
2013.06.15. NSF MASON Workshop 9
Status messages

10. Post-experiment Scores
2013.06.15. NSF MASON Workshop 10

11. Outline
• Background
• The CRISIS project
• CRISIS Simulation Library
• CRISIS Game & PET v2.0
• MEME
• Future works
2013.06.15. NSF MASON Workshop 11

12. Background: The CRISIS Project
“The CRISIS project addresses building a next generation
macroeconomic and financial system policymaking model:
a bottom-up agent-based simulation that fully accounts for the
heterogeneity of households, firms, and government actors. The
model will incorporate the latest evidence from behavioral
economics in portraying agent behavior, and the CRISIS team
will also collect new data on agent decision making using
experimental economics techniques. While any model must
make simplifying assumptions about human behavior, the
CRISIS model will be significantly more realistic in its portrayal of
relevant agent behavior than the current generation of
policymaking models.”
Crisis project description: https://www.crisis-economics.eu/
2013.06.15. NSF MASON Workshop 12

13. CRISIS Simulation Library
• Olaf covered that 
2013.06.15. NSF MASON Workshop 13

14. 2013.06.15. NSF MASON Workshop 14
Web-based Game
(Participatory Experiments)
Economic Simulator
(Cloud-Based Parameter
Sweep Execution)
Modelling
Framework
Models

15. 2013.06.15. NSF MASON Workshop 15
Web-based Game
(Participatory Experiments)
Economic Simulator
(Cloud-Based Parameter
Sweep Execution)

16. CRISIS Game
2013.06.15. NSF MASON Workshop 16
(Planned design)
http://game.crisis-economics.eu

17. The Model Under the CRISIS Game
2013.06.15. NSF MASON Workshop 17
Source: Domenico Delli Gatti, personal communications
(Single player mode)
Domenico Delli Gatti, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati.
Macroeconomics from the Bottom-up. 1st ed. Springer, 2011.

18. Architecture
2013.06.15. NSF MASON Workshop 18

19. PET v2.0
• Idea came from AITIA's PET Software
• Participatory Extension
• Generalized components behind the CRISIS Game
• Modern, scalable infrastructure
• Previous version is 5+ years old
• Components may run on (several) different machines
• Software already available for any experiment
2013.06.15. NSF MASON Workshop 19

20. PET v2.0
• Can be used:
• Locally
• Laboratory experiments
• Policy makers (scenario analysis with a proper model)
• On any webserver to run constantly
• Gather data (scores from model and all user actions  replay)
• Dissemination
• Questionnaire module
• Verify if subjects understood the rules
• Software is already in use by Universiteit van Amsterdam
2013.06.15. NSF MASON Workshop 20

21. What is required to use?
1. An ABM model
• PET is a generic framework
• Currently Java and MASON models are supported
• (But! No serialization is required )
2013.06.15. NSF MASON Workshop 21

22. What is required to use?
1. An ABM model
• PET is a generic framework
• Currently Java and MASON models are supported
• (But! No serialization is required )
2. Development of the web interface
• No restriction on tools (HTML5, GWT, ...)
• Communication to server by standardized tools (AJAX)
• Config files to map incoming messages to model actions
2013.06.15. NSF MASON Workshop 22

23. MEME
2013.06.15. NSF MASON Workshop 23

24. Parameter sweeps?
„[...] suppose that in a model there are just 10 relevant
parameters, and that each parameter can assume 10
different values (a rather simplifying assumption). As a
result, one obtains that the constellation of the parameter
space is given by 10^10 vectors. If we perform 20
different runs for each one of them to take into account
the possible effects of changing the random seeds, the
total number of simulations would amount to 2*10^11!”
2012.06.22. WEHIA 2012 - Replication of MacroABM 24
Domenico Delli Gatti, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:
Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)

25. Parameter sweeps?
„[...] suppose that in a model there are just 10 relevant
parameters, and that each parameter can assume 10
different values (a rather simplifying assumption). As a
result, one obtains that the constellation of the parameter
space is given by 10^10 vectors. If we perform 20
different runs for each one of them to take into account
the possible effects of changing the random seeds, the
total number of simulations would amount to 2*10^11!”
2012.06.22. WEHIA 2012 - Replication of MacroABM 25
Domenico Delli Gatti, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:
Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)
Even if 1 run / second 
would take ~6 585 000 years

26. MEME – Model Exploration Module
• MEME is the tool for the job
• Standalone application
• Parameter sweep exploration
• Being Strong
• Exploiting Grid/Cloud systems
(LAN, Amazon, etc.)
• Being Smart
• Design of Experiments
2013.06.15. NSF MASON Workshop 26

27. MEME – Model Exploration Module
• Generic tool
• Supports various platforms (NetLogo/Repast J/MASON/...)
• Easy-to-use graphical interface
• Setting the parameter tree
• Recording
• External tool support (R/Matlab/Octave/...)
• Recently declared an open source project
2013.06.15. NSF MASON Workshop 27

28. Design of Experiments
• Reduce # of runs
• Ignore unimportant
cases „smartly”
• Factorial Designs (2/3 lev.)
• Latin Hypercube,
• Box-Behnken,
• Box-Wilson Central
Composite
• Genetic Algorithm Designs,
• Iterative Uniform
Interpolation
• Adaptive Nonlinear Tests
2013.06.15. NSF MASON Workshop 28

29. Demo
2013.06.15. NSF MASON Workshop 29

30. FUTURE WORKS
2013.06.15. NSF MASON Workshop 30

31. Future Works
• Pet v2.0
• Tools to support front-end creation
• JavaScript library (WIP)
• User-friendly Eclipse editor plug-ins (flexible, easy to use)
• Both for the UI, questionnaire
• Tutorials, docs to be developed
• Additional platforms
• MEME
• Publish open source project, documentation...
• Cloud-based economic simulator
• +Your ideas? 
2013.06.15. NSF MASON Workshop 31

32. Interested?
• „A New Set of Tools Supporting Agent-Based Economic
Modeling”, EPIA 2013 (under publication, ~Sept.)
• Statistical and Behavioral Model Analysis Tools.
• Hands-on workshop @ ESSA 2013
• CRISIS Website: http://www.crisis-economics.eu/
2013.06.15. NSF MASON Workshop 32

33. THANK YOU!
• Questions?
• Will be available soon from http://iaws.aitia.ai/
• Download the CRISIS Simulation Library:
• http://www.crisis-economics.eu/software
• Try the CRISIS Game
• http://game.crisis-economics.eu
• Take a look on the El Farol model:
• http://github.com/rlegendi/mason-examples
• Download MEME:
• http://meme.aitia.ai Richard O. Legendi
rlegendi@aitia.ai
http://people.inf.elte.hu/legendi/
@legendi_ELTE
June 15, 2013.
2013.06.15. NSF MASON Workshop 33
The research leading to these results has received funding from the
European Union, Seventh Framework Programme FP7/2007-2013
under grant agreement no. CRISIS-ICT-2011-288501.

34. „Visionary things”
• Usability:
• API level: making it easier to use (annotations, generics)
• step(Object) vs. step(Model)
• DevTools:
• Maven (Graddle, Ivy, Sbt)
• Recording and parameterizing simulations
• Distribution
• Networks
• MASON Simulation Library for the community
2013.06.15. NSF MASON Workshop 34