1. Introduction to Computing
for Complex Systems
(Lab Sessions 7 + 8)

2. Goals For Today
Simple Birth Rates Revisited
Automation III ---
The Behavior Space
Multiple Variable Models

3. Multiple Variable Models
• We’ve seen that Netlogo provides several
ways to output data and run the model so
that we can get statistical information.

4. Multiple Variable Models
• What if there are several variables in a
model? Will this increase the time it would
take to test the parameter space?

5. • Open the Simple Birth rate model
• There are three variables, carrying
capacity, blue fertility rate, and red fertility
rate.
Simple Birth Rates

6. • The Blues and Reds reproduce according
to their fertility rate and the entire
population is limited by the carrying
capacity.
Simple Birth Rates

7. • There is already an output window on the
bottom of the screen that will show how
long it took for one of the colors to go
extinct.
• Use the “run experiment” button to start
the model. Run the model a few times,
varying the fertility rates.
Simple Birth Rates

9. Simple Birth Rates
• The output is relatively simple, but it would
probably take a long time to move through
each combination of blue and red fertility
rates, especially if you wanted to run the
model more than once for each
combination.

10. • We could also vary the carrying capacity,
which would also increase the amount of
time it would take to run these models by
hand.
Simple Birth Rates

11. Automated Analysis
of Model Runs
Part III
T-X

12. The Behavior Space

13. What is BehaviorSpace?
• “BehaviorSpace is a software tool
integrated with NetLogo that allows you to
perform experiments with models.”
• http://ccl.northwestern.edu/netlogo/docs/
behaviorspace.html

14. What is BehaviorSpace?
• “Behavior Space runs a model many
times, systematically varying the model's
settings and recording the results of each
model run. This process is sometimes
called "parameter sweeping”

15. What is BehaviorSpace?
• It lets you explore the model's "space" of
possible behaviors and determine which
combinations of settings cause the
behaviors of interest.”

16. Why Behavior Space is Useful
• BehaviorSpace automates the movement
of the model runs through each
combination of variables, allowing us to
move quickly across the parameters we
wish to test.

17. Why Behavior Space is Useful
• It also outputs the information into a .csv
file so that we can use statistical software
to analyze the data.

18. • So if you have a lot of model runs, multiple
variables, or want to analyze model data
with statistical software, then
BehaviorSpace may be of use to you!
Why Behavior Space is Useful

19. • Let’s see how BehaviorSpace works.
• Go to the Tools scroll down menu and
click BehaviorSpace.
Why Behavior Space
is Useful

21. Working with the
Behavior Space
• Now you should have the experiment
page.
• You will see the options New, Edit,
Duplicate, Delete, and Run.

23. • The New button creates a new
experiment, and brings you to the page
where you will be able to specify the
parameters of the model you wish to test
and how many times you wish the model
to run per combination.
Working with the
Behavior Space

24. • The Edit button allows you to edit any
experiment that is saved in the
BehaviorSpace experiment list.
Working with the
Behavior Space

25. • If you save your Netlogo program, it will
also save your experiments, so you do not
have to write out the parameters you wish
to test each time you turn off Netlogo.
Working with the
Behavior Space

26. • The Duplicate button allows you to create
another copy of an experiment that will be
saved in the experiments box of
BehaviorSpace.
Working with the
Behavior Space

27. • The Delete button permanently deletes an
experiment from the experiments box.
Working with the
Behavior Space

28. • The Run button begins the process of
running your experiment and outputting
the data into a file.
Working with the
Behavior Space

29. • Let’s go to the experimental setup
• Click the ‘New’ button.
• You should now see a screen that will allow
you to detail the variables you wish to test
and how the program should run the
experiment.
Experiment Setup

31. • The Experimental setup screen has a title
box, a variables box, an area that allows
you to control how many times the model
should run per combination, and boxes
that specify any commands you wish the
program to run when the setup or go
button is pressed.
Setting up an Experiment

32. • There are also boxes that allow you to
detail any conditions that should stop the
running of the program and any
commands that should go into effect at the
end of the model run.
• It also includes the ability to determine a
limit to how many steps the model will run
for.
Setting up an Experiment

33. • Let’s take an in depth look at each of
these features.
• I will give the title “Experiment #1: Blue &
Red Fertility [0 1 10]” to my experiment
and move on to the variables box.
Setting up an Experiment

35. Variables Box
• The variables box includes variables that
come from sliders, switches, and choosers
(draw down menus) on the interface. It can
also include variables found within the
program code.

36. Variables Box
• The user is able to specify the boundaries
of the parameter space they wish
BehaviorSpace to “sweep” through. This is
done by writing in the variables and the
values the user wishes to test.

37. Variables Box
• BehaviorSpace keeps variables higher up
in the box constant as it cycles through the
lower variables’ value settings, only
moving the higher variables to their next
setting after finishing a complete cycle
through the possible alternatives in the
variables below it.

38. Variables Box
• If the runs are taking place in parallel, the
output may not exactly mirror this process.
• Regardless of the case, no matter where
you locate a variable, the entire parameter
space for all the variables you define in the
box will be tested.

40. How to’s
• Assigning a value to a variable:
• Ex: ["blue-fertility" 10] This will give the
variable blue-fertility the value of 10 in all
of the model runs.

42. How to’s
• Assigning two values to a variable (listing
values)
• Ex: ["blue-fertility" 1 2] gives the variable
blue-fertility the value of 1 and runs
through all other combinations (if there are
other variables), and then moves on to 2
and runs through all the other possible
combinations with red-fertility and carrying
capacity.

44. How to’s
• Assigning multiple values to a variable
(listing values)
• Ex: ["blue-fertility" 1 2 4 7] Runs through 1
and 2 and also does 4 and then 7.

46. How to’s
• Using an interval to assign multiple values
• The interval must be inside a set of
brackets

47. How to’s
• Ex: ["blue-fertility" [1 1 3]] This example
runs through the values 1 through 3,
moving at an increment of 1. Therefore, it
will assign the value 1, 2, and 3 to the
variable as it moves through the different
combinations.

48. How to’s
• When assigning values to a variable with an
interval, be careful to only include the
numbers you wish to test. The interval is
inclusive and will test all the numbers you
specify.
• Ex: ["blue-fertility" [0 1 4]] This will give the variable the
values 0, 1, 2, 3, and 4.

49. How to’s
• Behavior Space is entirely inclusive of the
specified range
• In other words, Inclusive will execute all
Parameter Combos in the Range

51. • Repetitions: The user can select how
many times Netlogo should run a given
combination of variables
How to’s

53. • Measure runs using these reporters: This
is what the model will measure and output
in the data.
• Measure runs at every step: Checking this
means the data will include output for each
step in the model
How to’s

55. • Setup Commands: The user can include
commands that will be executed in
addition to calling the ‘to setup’ procedure
when the setup button is hit.
How to’s

56. • Go Commands: The user can include
commands that will be executed in
addition to calling the ‘to go’ procedure
when the go button is hit.
How to’s

58. • Stop Conditions: This box allows the user
to specify conditions that would end the
model run if they were met.
How to’s

59. • While there may be many reasons a
modeler would want to do this, one reason
could be to prevent illogical or undesirable
combinations of variables from occurring
while the program sweeps through
parameter space.
How to’s

61. • Final Commands: The user can specify
commands to run at the end of a model
run.
How to’s

63. • Time Limit: The user can specify how
many ticks a given combination of
variables will be allowed to run before
stopping.
• In certain instances model run will go on
forever … need to have a time limit in that
case
How to’s

65. Simple Birth Rates in the
Behavior Space
• OK, now let’s run the model. We will put a
time limit of 100 ticks for each model run.

66. Enter
these
Values
As
shown

67. Running BehaviorSpace
• Select the experiment we just created and
hit “Run”.
• You will now be given the option to select
if your data will be put in a spreadsheet or
in a table.
• The data file will be saved as a comma
separated values (.csv) file.

68. Parallelization
• You will also be given the option to choose
how many experiments should run in
parallel.
• By default, Netlogo will run one
experiment for each core processor in
your computer.

69. Parallelization
• Only one experiment will be shown on
screen, the others will be run in the
background.
• You should individually test what your
computer can handle, since many parallel
runs can slow down your computer.

71. • After making those choices the model will
begin to run.
• You will have the option to turn off viewing
the plot and updating the visualization,
plots, and monitors. This will help increase
the speed of the runs.
Increasing the Speed

72. • You can also use the speed slider, which
determines how
many times the image should be updated.
Increasing the Speed

74. Output
• Spreadsheet data
• At the top, the data sheet will include the
model and experiment name, and the size
of the world.
• The commas will allow you to demarcate
where the columns should be in the
program you will use to analyze the data.

76. Output
• Tables (might be easier to work with in outside
statistical software)
• At the top, the data sheet will include the
model and experiment name, and the size of
the world.
• The commas will allow you to demarcate
where the columns should be in the program
you will use to analyze the data.

78. • You will probably want to edit the file
before sorting it. The information at the top
can be copied and pasted elsewhere if you
need it.
• The actual output from the model runs will
be below.
Output

80. • Once you have edited your file, you will be
able to sort the data as you wish.
Example in Excel Data --> TexttoColumns
Then select delimited and by comma
• As we saw before, the output may not be in
the order, since many runs may have been
running in parallel.
Output

81. • Once your data is in a useful format, you can
begin to analyze it!
Output

83. • We have seen that BehaviorSpace can
run through the parameter space much
faster than having a human go by hand
through all the different variable
combinations.
In Summary

84. • The previous data file took a little more
than 5 minutes to create and output.
In Summary

85. • But it moved in increments of 1.0 and the
carrying capacity remained constant
across the different combinations. Most
importantly, the model was limited to 100
ticks, which may not be enough time to
witness the dynamics the model is
supposed to display.
There are Limitations

86. • By adding these other features we will see
how quickly the length of time required to
run the models would grow if we added
smaller increments of movement across
the parameter space and the extra
carrying capacity variable.
There are Limitations

87. Just
40,814,201
Runs to Go

88. Limitations
• 40,814,201 runs is a lot
• Let’s assume (probably incorrectly) that it
takes 1 sec / run (Including all the steps in
one run of the model)
• 40814201 / 60 secs / 60 mins / 24 hours /
7 days = 67.484 weeks, 1.298 years

89. • That’s not including parallel runs, but even
if you could run 100 in parallel, it would still
take a considerable amount of time
• Thus, full parameter sweep Probably Not
feasible in Netlogo
Limitations

90. Limitations
• One option would be to limit the parameter
space to an area of specific interest.
• For example, the dynamics of two
competing groups with varying fertility
rates is not particularly interesting when
the carrying capacity is zero from the
outset.

91. Limitations
• Netlogo was mainly designed for
visualizations and as an introductory
language.
• If you write good modular code, you will be
able to convert your model into another
language that can perform massively
parallel runs more efficiently than Netlogo
(e.g. Python or C++)