• Save
MEME – An Integrated Tool For Advanced Computational Experiments
Upcoming SlideShare
Loading in...5

MEME – An Integrated Tool For Advanced Computational Experiments






Total Views
Views on SlideShare
Embed Views



1 Embed 1

http://www.slideshare.net 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

MEME – An Integrated Tool For Advanced Computational Experiments MEME – An Integrated Tool For Advanced Computational Experiments Presentation Transcript

  • MEME – An Integrated Tool For Advanced Computational Experiments Rajmund Bocsi 1 , Gabor Ferschl 1 , László Gulyás 1,2 , Attila Szabó 1,2 1 AITIA International Inc. 2 Lorand Eotvos University, Budapest
  • Overview
    • Agent-Based Modeling and Computer Simulations
    • Exploration of Model Response
      • Problems and Solutions
    • The Model Exploration Module (MEME)
    • Future Works
      • Dynamic Methods
    • Summary
  • Introduction
  • Agent-Based Modeling (ABM)
    • Aims at creating complex (social) behavior “from the bottom up” .
      • Complex interactions of a high number of
      • (complex) individuals .
      • A generative and mostly theoretical approach:
        • Computational “thought experiments” ,
        • Existence proofs, etc.
  • Validation and Verification
    • Validation
      • To do the right thing.
    • Verification
      • To do it right.
  • So, how to do it right?
    • … Methodology!
    • However, methodology is often constrained by the available / used tools .
  • Why Tools?
    • One can grind her own lens and build her own microscopes…
    • ... but standardized tools make routine tasks much easier and safer (easier to be correct).
      • Moreover, validation is easier. (No need to report on the lens grinding process, methods, etc.)
    • Also applies to very general tools
      • C, Java, etc.
      • Mathematica
  • The General Approach
    • Computer simulations are experiments
      • Where the experimenter tries to determine
      • How the systems response (output) depends
      • On controllable factors (parameters)
      • One may also want to do replicates (cf. RNG seeds)
    System (p 1 , p 2 , p 3 , p 4 , …) (r 1 , r 2 , r 3 , r 4 , …)
  • Exploration of the Model Response (Parameter Sweeps, Batch Runs)
    • Parameter-Response Analysis
    • Stochastic components
      • Very common in modeling: concepts not modeled explicitly
      • In our example:
        • Initialization
        • Execution order
    • Sensitivity Analysis
      • Dependence on parameter values
  • Problems
  • The Problem of Parameter Sweeps
    • The required number of runs may be extremely large.
    • Reaching into the tens or hundreds of thousands.
  • The Simplest Solution to the High Computational Demand: Distribution
    • Simulation runs with different parameter combinations can be run
      • In parallel,
      • On different computers.
    • Due to the independence of runs.
  • Advanced Distributed Solutions
    • Distribution can be done on clusters (or multiple cores in the same CPU)
      • But one needs to own all the computers for that.
    • Or, distribution can be done on “grids”
      • Grids are “cluster of clusters”
        • A loosely assembled pool of computers in different “administration domains”
        • Utility versus Desktop grids
  • A single cluster
  • A Utility Grid
  • An Example: the QosCosGrid
    • Quasi-Opportunistic Supercomputing for Complex Systems Simulations on the Grid
  • A Desktop Grid Grid Server
  • An Alternative / Additional Solution
  • An Alternative / Additional Solution
    • So far, we adopted a “brute force” approach
      • Even in the most advanced distributed case.
    • We explored all combinations
      • Of all values for
      • All parameters (parameter space dimensions)
  • The Challenge
    • One cannot observe/measure the behavior of the system at all points in the parameter space
      • We can only take samples .
    • In most practical cases, we can’t even explore all possible/meaningful/relevant values of single parameters.
  • An Approach Smarter than “Brute Force”
    • In many disciplines, experiments may be very costly or hard (e.g., living animals)
    • Design of Experiments (DoE) introduced by R. Fisher (1920s)
      • methods that reduce the number of experiments and still yield statistically significant results.
  • Tools: The Model Exploration Module (MEME)
  • The Model Exploration Module
    • Results Collection, Maintenance and Analysis
      • Filtering, Variable Selection, Aggregation and Statistics
      • Scripting for advanced users
      • Numerous charting and exporting options
    • Parameter Space Explorations Intelligently and Efficiently
      • On a single computer
      • On local clusters
      • On grid middleware
    • Design of Experiment (DoE) Support
      • Period2 activities
  • Param Sweeps Param Sweeps Param Sweeps Param Sweeps Results DB Charts Versioning and Merging Filtering, Processing, Restructuring Views Export (Excell, SPSS, etc.) Import (txt, csv, Excell, etc.)
  • Steps of Experiments with MEME
    • Import results & analyze data
      • Create view tables and Charts
      • (Potentially automatically or ‘batched’)
    • Potentially export the results of your analysis
      • (View) Tables or Charts
    • Load the model (automatic)
    • Set the parameter tree
      • manually,
      • (or by configuring automated sweep methods)
    • Set recordable values and statistics
    • Run the experiment
  • MEME Functions, Part 1
    • The user friendly wizard for
      • Manual setting up the parameter tree
      • Specifying recordable elements
    • Simulation execution
      • On single computers
      • Local clusters
      • And possibly on grids
    • Result analysis
  • MEME Functions, Part 2
    • The user friendly wizard for
      • Automatic setting up the parameter tree
        • Design of Experiments plugins
        • Dynamic IntelliSweep methods
      • Specifying recordable elements
        • Together with advanced statistics and on-the-fly code generation
    • Simulation execution
    • Result analysis
      • Advanced export options
  • Example: 2-Level Fractional Factorial Experiments with MEME
  • Example: 2-Level Fractional Factorial Experiments with MEME
  • Example: Latin Hypercube Experiments with MEME
  • Example: Latin Hypercube Experiments with MEME
  • Future Works: “IntelliSweep” Methods
  • Dynamic “IntelliSweep”methods
    • So far, the entire design was fixed before starting the experiment
      • There was no feedback from the measured responses to the design
    • Various (optimization) methods exist that use a different strategy
      • Hill climbing, simulated annealing, genetic algorithms, etc.
  • Iterative Uniform Interpolation 1
    • IUI is a response analysis method
      • Refines the parameter domain between iterations to achieve better interpolation (of the response value)
    • Examines “interesting” subintervals by dividing them further
      • Deviation from the previously observed (assumed) gradient spans new measurements.
  • Iterative Uniform Interpolation 2
  • Genetic Algorithm Driven Methods O ptimization
    • Genetic algorithm (GA) is a heuristic optimization method
      • F( o 1 , …, o n ) -> max
    • Can be directly used for response analysis
      • If we are not interested in the entire response surface, but only in high response values.
  • Genetic Algorithm Driven Methods Active Non-linear Tests, 1
    • Active Non-linear Tests (ANTs) were proposed by John H. Miller of CMU and SFI
    • A response analysis method that uses GA to disprove (prove) a user-defined thesis
      • Thesis: P -> {(o 1 , …, o n ) | <condition>}
      • Sample by GA: P -> {(o 1 , …, o n ) | (p 1 , …, p m )  P m }
  • Genetic Algorithm Driven Methods Active Non-linear Tests, 2
    • Measure “fit” of the sample to the thesis
      • Breed parameter combinations that are farthest from the thesis
    • In the end, the farthest sample provides a level of falsification
      • Notice that an effective non-linear optimization method is being used to falsify the thesis!
  • Multi-Platform Support
    • MEME imports results from virtually any format
    • MEME can master computational experiments of simulations written in
      • Repast J
      • FABLES
      • NetLogo
      • Custom Java
      • Repast Simphony*
  • Summary
    • Correct methodology is vital for computational experiments
    • Methodology is often dependent on the availability of tools
    • The Model Exploration Module is a free tool to support computational modelers
      • In methodologically exploring the response of their computational models
  • Acknowledgements
    • This work benefited from the partial support of the following grants
      • The ELTE Informatics Cooperative Research and Education Center (GVOP-3.2.2-2004.07-005/3.0) by the Hungarian Government;
      • The FP6 STREP project QosCosGrid (contract #033883) by the European Commission;
      • The FP6 STREP project Emergence in the Loop (EMIL) (contract #033841) by the European Commission.
  • Availability of Tools
    • http://mass.aitia.ai/
        • http://fables.aitia.ai/
        • http://meme.aitia.ai/
    • Downloadable free of charge
      • With full documentation