Introduction to Computing
for Complex Systems
(Lab Sessions 7 + 8)
Goals For Today
Simple Birth Rates Revisited
Automation III ---
The Behavior Space
Multiple Variable Models
Multiple Variable Models
•  We’ve seen that Netlogo provides several
ways to output data and run the model so
that we can ...
Multiple Variable Models
•  What if there are several variables in a
model? Will this increase the time it would
take to t...
•  Open the Simple Birth rate model
•  There are three variables, carrying
capacity, blue fertility rate, and red fertilit...
•  The Blues and Reds reproduce according
to their fertility rate and the entire
population is limited by the carrying
cap...
•  There is already an output window on the
bottom of the screen that will show how
long it took for one of the colors to ...
Simple Birth Rates
•  The output is relatively simple, but it would
probably take a long time to move through
each combina...
•  We could also vary the carrying capacity,
which would also increase the amount of
time it would take to run these model...
Automated Analysis
of Model Runs
Part III
T-X
The Behavior Space
What is BehaviorSpace?
•  “BehaviorSpace is a software tool
integrated with NetLogo that allows you to
perform experiments...
What is BehaviorSpace?
•  “Behavior Space runs a model many
times, systematically varying the model's
settings and recordi...
What is BehaviorSpace?
•  It lets you explore the model's "space" of
possible behaviors and determine which
combinations o...
Why Behavior Space is Useful
•  BehaviorSpace automates the movement
of the model runs through each
combination of variabl...
Why Behavior Space is Useful
•  It also outputs the information into a .csv
file so that we can use statistical software
t...
•  So if you have a lot of model runs, multiple
variables, or want to analyze model data
with statistical software, then
B...
•  Let’s see how BehaviorSpace works.
•  Go to the Tools scroll down menu and
click BehaviorSpace.
Why Behavior Space
is U...
Working with the
Behavior Space
•  Now you should have the experiment
page.
•  You will see the options New, Edit,
Duplica...
•  The New button creates a new
experiment, and brings you to the page
where you will be able to specify the
parameters of...
•  The Edit button allows you to edit any
experiment that is saved in the
BehaviorSpace experiment list.
Working with the
...
•  If you save your Netlogo program, it will
also save your experiments, so you do not
have to write out the parameters yo...
•  The Duplicate button allows you to create
another copy of an experiment that will be
saved in the experiments box of
Be...
•  The Delete button permanently deletes an
experiment from the experiments box.
Working with the
Behavior Space
•  The Run button begins the process of
running your experiment and outputting
the data into a file.
Working with the
Beha...
•  Let’s go to the experimental setup
•  Click the ‘New’ button.
•  You should now see a screen that will allow
you to det...
•  The Experimental setup screen has a title
box, a variables box, an area that allows
you to control how many times the m...
•  There are also boxes that allow you to
detail any conditions that should stop the
running of the program and any
comman...
•  Let’s take an in depth look at each of
these features.
•  I will give the title “Experiment #1: Blue &
Red Fertility [0...
Variables Box
•  The variables box includes variables that
come from sliders, switches, and choosers
(draw down menus) on ...
Variables Box
•  The user is able to specify the boundaries
of the parameter space they wish
BehaviorSpace to “sweep” thro...
Variables Box
•  BehaviorSpace keeps variables higher up
in the box constant as it cycles through the
lower variables’ val...
Variables Box
•  If the runs are taking place in parallel, the
output may not exactly mirror this process.
•  Regardless o...
How to’s
•  Assigning a value to a variable:
•  Ex: ["blue-fertility" 10] This will give the
variable blue-fertility the v...
How to’s
•  Assigning two values to a variable (listing
values)
•  Ex: ["blue-fertility" 1 2] gives the variable
blue-fert...
How to’s
•  Assigning multiple values to a variable
(listing values)
•  Ex: ["blue-fertility" 1 2 4 7] Runs through 1
and ...
How to’s
•  Using an interval to assign multiple values
•  The interval must be inside a set of
brackets
How to’s
•  Ex: ["blue-fertility" [1 1 3]] This example
runs through the values 1 through 3,
moving at an increment of 1. ...
How to’s
•  When assigning values to a variable with an
interval, be careful to only include the
numbers you wish to test....
How to’s
•  Behavior Space is entirely inclusive of the
specified range
•  In other words, Inclusive will execute all
Para...
•  Repetitions: The user can select how
many times Netlogo should run a given
combination of variables
How to’s
•  Measure runs using these reporters: This
is what the model will measure and output
in the data.
•  Measure runs at ever...
•  Setup Commands: The user can include
commands that will be executed in
addition to calling the ‘to setup’ procedure
whe...
•  Go Commands: The user can include
commands that will be executed in
addition to calling the ‘to go’ procedure
when the ...
•  Stop Conditions: This box allows the user
to specify conditions that would end the
model run if they were met.
How to’s
•  While there may be many reasons a
modeler would want to do this, one reason
could be to prevent illogical or undesirabl...
•  Final Commands: The user can specify
commands to run at the end of a model
run.
How to’s
•  Time Limit: The user can specify how
many ticks a given combination of
variables will be allowed to run before
stopping...
Simple Birth Rates in the
Behavior Space
•  OK, now let’s run the model. We will put a
time limit of 100 ticks for each mo...
Enter
these
Values
As
shown
Running BehaviorSpace
•  Select the experiment we just created and
hit “Run”.
•  You will now be given the option to selec...
Parallelization
•  You will also be given the option to choose
how many experiments should run in
parallel.
•  By default,...
Parallelization
•  Only one experiment will be shown on
screen, the others will be run in the
background.
•  You should in...
•  After making those choices the model will
begin to run.
•  You will have the option to turn off viewing
the plot and up...
•  You can also use the speed slider, which
determines how
many times the image should be updated.
Increasing the Speed
Output
•  Spreadsheet data
•  At the top, the data sheet will include the
model and experiment name, and the size
of the w...
Output
•  Tables (might be easier to work with in outside
statistical software)
•  At the top, the data sheet will include...
•  You will probably want to edit the file
before sorting it. The information at the top
can be copied and pasted elsewher...
•  Once you have edited your file, you will be
able to sort the data as you wish.
Example in Excel Data --> TexttoColumns
...
•  Once your data is in a useful format, you can
begin to analyze it!
Output
•  We have seen that BehaviorSpace can
run through the parameter space much
faster than having a human go by hand
through ...
•  The previous data file took a little more
than 5 minutes to create and output.
In Summary
•  But it moved in increments of 1.0 and the
carrying capacity remained constant
across the different combinations. Most
i...
•  By adding these other features we will see
how quickly the length of time required to
run the models would grow if we a...
Just
40,814,201
Runs to Go
Limitations
•  40,814,201 runs is a lot
•  Let’s assume (probably incorrectly) that it
takes 1 sec / run (Including all th...
•  That’s not including parallel runs, but even
if you could run 100 in parallel, it would still
take a considerable amoun...
Limitations
•  One option would be to limit the parameter
space to an area of specific interest.
•  For example, the dynam...
Limitations
•  Netlogo was mainly designed for
visualizations and as an introductory
language.
•  If you write good modula...
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz
Upcoming SlideShare
Loading in …5
×

ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz

757 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
757
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
83
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ICPSR - Complex Systems Models in the Social Sciences - Lab Session 7, 8 - Professor Daniel Martin Katz

  1. 1. Introduction to Computing for Complex Systems (Lab Sessions 7 + 8)
  2. 2. Goals For Today Simple Birth Rates Revisited Automation III --- The Behavior Space Multiple Variable Models
  3. 3. Multiple Variable Models •  We’ve seen that Netlogo provides several ways to output data and run the model so that we can get statistical information.
  4. 4. Multiple Variable Models •  What if there are several variables in a model? Will this increase the time it would take to test the parameter space?
  5. 5. •  Open the Simple Birth rate model •  There are three variables, carrying capacity, blue fertility rate, and red fertility rate. Simple Birth Rates
  6. 6. •  The Blues and Reds reproduce according to their fertility rate and the entire population is limited by the carrying capacity. Simple Birth Rates
  7. 7. •  There is already an output window on the bottom of the screen that will show how long it took for one of the colors to go extinct. •  Use the “run experiment” button to start the model. Run the model a few times, varying the fertility rates. Simple Birth Rates
  8. 8. Simple Birth Rates •  The output is relatively simple, but it would probably take a long time to move through each combination of blue and red fertility rates, especially if you wanted to run the model more than once for each combination.
  9. 9. •  We could also vary the carrying capacity, which would also increase the amount of time it would take to run these models by hand. Simple Birth Rates
  10. 10. Automated Analysis of Model Runs Part III T-X
  11. 11. The Behavior Space
  12. 12. What is BehaviorSpace? •  “BehaviorSpace is a software tool integrated with NetLogo that allows you to perform experiments with models.” •  http://ccl.northwestern.edu/netlogo/docs/ behaviorspace.html
  13. 13. What is BehaviorSpace? •  “Behavior Space runs a model many times, systematically varying the model's settings and recording the results of each model run. This process is sometimes called "parameter sweeping”
  14. 14. What is BehaviorSpace? •  It lets you explore the model's "space" of possible behaviors and determine which combinations of settings cause the behaviors of interest.”
  15. 15. Why Behavior Space is Useful •  BehaviorSpace automates the movement of the model runs through each combination of variables, allowing us to move quickly across the parameters we wish to test.
  16. 16. Why Behavior Space is Useful •  It also outputs the information into a .csv file so that we can use statistical software to analyze the data.
  17. 17. •  So if you have a lot of model runs, multiple variables, or want to analyze model data with statistical software, then BehaviorSpace may be of use to you! Why Behavior Space is Useful
  18. 18. •  Let’s see how BehaviorSpace works. •  Go to the Tools scroll down menu and click BehaviorSpace. Why Behavior Space is Useful
  19. 19. Working with the Behavior Space •  Now you should have the experiment page. •  You will see the options New, Edit, Duplicate, Delete, and Run.
  20. 20. •  The New button creates a new experiment, and brings you to the page where you will be able to specify the parameters of the model you wish to test and how many times you wish the model to run per combination. Working with the Behavior Space
  21. 21. •  The Edit button allows you to edit any experiment that is saved in the BehaviorSpace experiment list. Working with the Behavior Space
  22. 22. •  If you save your Netlogo program, it will also save your experiments, so you do not have to write out the parameters you wish to test each time you turn off Netlogo. Working with the Behavior Space
  23. 23. •  The Duplicate button allows you to create another copy of an experiment that will be saved in the experiments box of BehaviorSpace. Working with the Behavior Space
  24. 24. •  The Delete button permanently deletes an experiment from the experiments box. Working with the Behavior Space
  25. 25. •  The Run button begins the process of running your experiment and outputting the data into a file. Working with the Behavior Space
  26. 26. •  Let’s go to the experimental setup •  Click the ‘New’ button. •  You should now see a screen that will allow you to detail the variables you wish to test and how the program should run the experiment. Experiment Setup
  27. 27. •  The Experimental setup screen has a title box, a variables box, an area that allows you to control how many times the model should run per combination, and boxes that specify any commands you wish the program to run when the setup or go button is pressed. Setting up an Experiment
  28. 28. •  There are also boxes that allow you to detail any conditions that should stop the running of the program and any commands that should go into effect at the end of the model run. •  It also includes the ability to determine a limit to how many steps the model will run for. Setting up an Experiment
  29. 29. •  Let’s take an in depth look at each of these features. •  I will give the title “Experiment #1: Blue & Red Fertility [0 1 10]” to my experiment and move on to the variables box. Setting up an Experiment
  30. 30. Variables Box •  The variables box includes variables that come from sliders, switches, and choosers (draw down menus) on the interface. It can also include variables found within the program code.
  31. 31. Variables Box •  The user is able to specify the boundaries of the parameter space they wish BehaviorSpace to “sweep” through. This is done by writing in the variables and the values the user wishes to test.
  32. 32. Variables Box •  BehaviorSpace keeps variables higher up in the box constant as it cycles through the lower variables’ value settings, only moving the higher variables to their next setting after finishing a complete cycle through the possible alternatives in the variables below it.
  33. 33. Variables Box •  If the runs are taking place in parallel, the output may not exactly mirror this process. •  Regardless of the case, no matter where you locate a variable, the entire parameter space for all the variables you define in the box will be tested.
  34. 34. How to’s •  Assigning a value to a variable: •  Ex: ["blue-fertility" 10] This will give the variable blue-fertility the value of 10 in all of the model runs.
  35. 35. How to’s •  Assigning two values to a variable (listing values) •  Ex: ["blue-fertility" 1 2] gives the variable blue-fertility the value of 1 and runs through all other combinations (if there are other variables), and then moves on to 2 and runs through all the other possible combinations with red-fertility and carrying capacity.
  36. 36. How to’s •  Assigning multiple values to a variable (listing values) •  Ex: ["blue-fertility" 1 2 4 7] Runs through 1 and 2 and also does 4 and then 7.
  37. 37. How to’s •  Using an interval to assign multiple values •  The interval must be inside a set of brackets
  38. 38. How to’s •  Ex: ["blue-fertility" [1 1 3]] This example runs through the values 1 through 3, moving at an increment of 1. Therefore, it will assign the value 1, 2, and 3 to the variable as it moves through the different combinations.
  39. 39. How to’s •  When assigning values to a variable with an interval, be careful to only include the numbers you wish to test. The interval is inclusive and will test all the numbers you specify. •  Ex: ["blue-fertility" [0 1 4]] This will give the variable the values 0, 1, 2, 3, and 4.
  40. 40. How to’s •  Behavior Space is entirely inclusive of the specified range •  In other words, Inclusive will execute all Parameter Combos in the Range
  41. 41. •  Repetitions: The user can select how many times Netlogo should run a given combination of variables How to’s
  42. 42. •  Measure runs using these reporters: This is what the model will measure and output in the data. •  Measure runs at every step: Checking this means the data will include output for each step in the model How to’s
  43. 43. •  Setup Commands: The user can include commands that will be executed in addition to calling the ‘to setup’ procedure when the setup button is hit. How to’s
  44. 44. •  Go Commands: The user can include commands that will be executed in addition to calling the ‘to go’ procedure when the go button is hit. How to’s
  45. 45. •  Stop Conditions: This box allows the user to specify conditions that would end the model run if they were met. How to’s
  46. 46. •  While there may be many reasons a modeler would want to do this, one reason could be to prevent illogical or undesirable combinations of variables from occurring while the program sweeps through parameter space. How to’s
  47. 47. •  Final Commands: The user can specify commands to run at the end of a model run. How to’s
  48. 48. •  Time Limit: The user can specify how many ticks a given combination of variables will be allowed to run before stopping. •  In certain instances model run will go on forever … need to have a time limit in that case How to’s
  49. 49. Simple Birth Rates in the Behavior Space •  OK, now let’s run the model. We will put a time limit of 100 ticks for each model run.
  50. 50. Enter these Values As shown
  51. 51. Running BehaviorSpace •  Select the experiment we just created and hit “Run”. •  You will now be given the option to select if your data will be put in a spreadsheet or in a table. •  The data file will be saved as a comma separated values (.csv) file.
  52. 52. Parallelization •  You will also be given the option to choose how many experiments should run in parallel. •  By default, Netlogo will run one experiment for each core processor in your computer.
  53. 53. Parallelization •  Only one experiment will be shown on screen, the others will be run in the background. •  You should individually test what your computer can handle, since many parallel runs can slow down your computer.
  54. 54. •  After making those choices the model will begin to run. •  You will have the option to turn off viewing the plot and updating the visualization, plots, and monitors. This will help increase the speed of the runs. Increasing the Speed
  55. 55. •  You can also use the speed slider, which determines how many times the image should be updated. Increasing the Speed
  56. 56. Output •  Spreadsheet data •  At the top, the data sheet will include the model and experiment name, and the size of the world. •  The commas will allow you to demarcate where the columns should be in the program you will use to analyze the data.
  57. 57. Output •  Tables (might be easier to work with in outside statistical software) •  At the top, the data sheet will include the model and experiment name, and the size of the world. •  The commas will allow you to demarcate where the columns should be in the program you will use to analyze the data.
  58. 58. •  You will probably want to edit the file before sorting it. The information at the top can be copied and pasted elsewhere if you need it. •  The actual output from the model runs will be below. Output
  59. 59. •  Once you have edited your file, you will be able to sort the data as you wish. Example in Excel Data --> TexttoColumns Then select delimited and by comma •  As we saw before, the output may not be in the order, since many runs may have been running in parallel. Output
  60. 60. •  Once your data is in a useful format, you can begin to analyze it! Output
  61. 61. •  We have seen that BehaviorSpace can run through the parameter space much faster than having a human go by hand through all the different variable combinations. In Summary
  62. 62. •  The previous data file took a little more than 5 minutes to create and output. In Summary
  63. 63. •  But it moved in increments of 1.0 and the carrying capacity remained constant across the different combinations. Most importantly, the model was limited to 100 ticks, which may not be enough time to witness the dynamics the model is supposed to display. There are Limitations
  64. 64. •  By adding these other features we will see how quickly the length of time required to run the models would grow if we added smaller increments of movement across the parameter space and the extra carrying capacity variable. There are Limitations
  65. 65. Just 40,814,201 Runs to Go
  66. 66. Limitations •  40,814,201 runs is a lot •  Let’s assume (probably incorrectly) that it takes 1 sec / run (Including all the steps in one run of the model) •  40814201 / 60 secs / 60 mins / 24 hours / 7 days = 67.484 weeks, 1.298 years
  67. 67. •  That’s not including parallel runs, but even if you could run 100 in parallel, it would still take a considerable amount of time •  Thus, full parameter sweep Probably Not feasible in Netlogo Limitations
  68. 68. Limitations •  One option would be to limit the parameter space to an area of specific interest. •  For example, the dynamics of two competing groups with varying fertility rates is not particularly interesting when the carrying capacity is zero from the outset.
  69. 69. Limitations •  Netlogo was mainly designed for visualizations and as an introductory language. •  If you write good modular code, you will be able to convert your model into another language that can perform massively parallel runs more efficiently than Netlogo (e.g. Python or C++)

×