Validation and verification

6,559 views

Published on

This presentation talks about the importance and techniques associated to validating and verifying models, more specifically simulation models.

Published in: Education, Technology, Design
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,559
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
255
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Validation and verification

  1. 1. Validation and Verification Adapted from Jerry Banks
  2. 2. Verification  Concerned with building the model right  Comparison of conceptual model and computer representation  Is the model implemented correctly in the computer?  Are the inputs and logical parameters represented properly?
  3. 3. Validation  Concerned with building the right model  Accurate representation of the real system  This is achieved through the calibration of the model  Iterative process until accuracy is acceptable
  4. 4. Model Building, Verification, and Validation REAL SYSTEM Conceptual Model 1Assumptions on system components 2Structural Assumptions (defines the interactions between the system components) 3Input parameters and data assumptions Operational Model (Computer Representation) Conceptual Validation Model Verification Calibration and Validation
  5. 5. Common sense suggestions for verification  Have someone check the computerized model  Make a flow diagram (with logical actions for each possible event)  Examine model output for reasonableness  Print the input parameters at the end of the simulation
  6. 6. Common sense suggestions for verification  Make the computerized representation as self documenting as possible  If animated, verify what is seen  Use IRC or debuggers  Use graphical interface
  7. 7. Three Classes of Techniques for Verification  Common sense techniques  Thorough documentation  Traces
  8. 8. Calibration and Validation  Validation is the overall process of comparing the model and its behavior to the real system and its behavior  Calibration is the iterative process of comparing the model to the real system and making adjustments to the model, and so on.
  9. 9. Iterative Process of Calibration REAL SYSTEM Initial Model Second Revision of Model First Revision of Model Compare Model to Reality Compare Revised Model to Reality Compare second Revised Model to Reality
  10. 10. 3 Step Approach by Naylor and Finger (1967)  Build a model with high face validity  Validate model assumptions  Compare the model input-output transformations to corresponding input-output transformations of the real system
  11. 11. Possible validation techniques in order of increasing cost-value ratio by Van Horn (1971)  High face validity. Use previous research/ studies/observation/experience  Conduct statistical test for data homogeneity, randomness, and goodness of fit test  Conduct Turing test. Have a group of experts compare model output versus system output and detect the difference  Compare model output to system output using statistical tests
  12. 12. Possible validation techniques in order of increasing cost-value ratio by Van Horn (1971)  After model development, collect new data and apply previous 3 tests  Build a new system or redesign the old one based on simulation results and use this data to validate the model  Do little or no validation. Implement results without validating

×