The document emphasizes the importance of modeling frameworks in making evidence-based decisions for complex systems, highlighting that models serve to predict behavior and support agency mandates. It outlines the necessity of using models correctly to avoid resource wastage and errors while providing a structured approach for decision-making processes. Additionally, it includes concepts related to data management, model applicability, and the integration of various stakeholders' perspectives in the modeling process.

1. ModSimWorld Montreal, Quebec June 8-9, 2009 A Modelling Framework: Supporting Evidence-Based Decisions

2. Main Messages Models are needed to understand and predict the behavior of complex systems. Models are needed to fulfill an agency’s mandate and support its core business. Inadequate or incorrect use of models wastes resources, results in errors, and exposes an agency to liability. Models should be used wisely

3. Outline I. Underlying Concepts Scientific underpinning II. Decision Guide Decision making III. Glossary Common understanding

4. About Models What are they? Simplified representations of reality. Transform data, information, and knowledge into outputs. Why do we use them? Reality is too complex Experiments are infeasible Predict consequences Increase understanding Nonaka (2000) Concepts ..

5. What is a Framework? “ Structural outline of the components of an organization, system, or process and the relationships among them.” Understanding Knowledge Services NRCan (2006) Concepts

6. Framework Objectives Support needs-driven and science-driven analysis. Promote dialogue among modelers, managers, & users. Reduce wasted time, effort, & money. Provide a basis for planning and action. Document and justify decisions. Concepts

7. Framework Design Reflect modelling, management, and user perspectives. Balance efficiency and effectiveness with cost and effort. Applicable to both demand and supply approaches to modelling. Applicable to both logical and computational models Concepts

8. Different Perspectives Concepts What developers proposed What managers funded What stakeholders wanted What users needed

9. Supply & Demand Concepts Supply : I have a model that solves your problem. Demand : I have a problem that needs a model.

10. Modelling System Concepts External Models Develop Nature, Society Internal Models Use Manage Lost Models Share Preserve Knowledge Management

11. Modelling Process Modelling combines science & computers; judgement & experience; insight & intuition. Principles: effort, simple, data, knowledge, transparent, understandable. Complexity: Modelling is a dynamic feedback process with delays and uncertainty. Development : techniques are well-understood; management less understood and practiced. Use: Decision making under uncertainty, unknown elements, outcome probabilities. Concepts

12. Systems Hierarchy Concepts Data Models Decision Support Information Knowledge Management Management Mandate Business Policies Processes

13. Data A model and its data are inseparable; they succeed or fail as one. Data Needs: Situation may involve nature, the system, and/or intervention. Sampling: Statistics are essential to determine how much data is needed. Source: Ownership? Use rights? Privacy & security concerns? Scale : Time, space, and process scale must match the situation. Quality: Level of accuracy, detail, and completeness are needed? Concepts

14. Information System Audience Use Media Concepts Outputs Acquisition Processing Storage Knowledge Organization Society Environment Events Economy Channels Access Search Retrieval Interface Processing Database Inputs Interface System Data Model Outputs Interoperability Integration Availability Utility

15. Models and Knowledge System: Behavior: Approach: Model: Decision: Basis: Concepts 1. Common Flow-through (-) Fixed (1:1) Planning Mechanistic Automated Data, facts 2. Complicated Feedback Linear (1:n) Mathematics Deterministic Certainty Explicit knowledge 3. Complex Predictive feedback (+) Non-linear (1:?) Simulation Stochastic Uncertainty Tacit knowledge 4. Chaotic Emergent Disorganized Scenario analysis Mental Reaction Intuition

16. Outline I. Underlying Concepts Scientific underpinning II. Decision Guide Decision making III. Glossary Common understanding

17. Decision Guide - Hierarchy Phase: (3) demand , supply, project Stage: (7) approach , design, establish, develop, evaluate, implement, conclude Step: (34) screening , problem definition, suitability, knowledge, data Consideration (132): recurrence, importance, problem space, existence

18. Decision Guide - Stages Guide Design Acquire Data Generate Knowledge Approach Out Evaluation Issue Start: ( Demand ) Implementation Outputs Conclusion End Development (Modeller) ( Manager ) ( User ) D1 D2 3 Applicability S2 Model Start ( Supply ) identification S1 5 7 6 Establishment 4 ( Manager ) (All)

19. Decision Guide Phases Guide Issue Model Demand Supply Project End

20. Supply & Demand Guide Start (use) Demand-driven backward chaining, closed question Model Supply-driven forward chaining, open question Start (model) Uses

21. Demand Phase Guide Issue D1 . Approach D2 . Design Development Acquire Data Generate Knowledge Out

22. Guide Approach Stage Issue Initial Screening Problem Definition Suitability Knowledge Evaluation Data Availability Design Recurrence Importance Problem space Existence Business Function Intended use Time available Situation Needs Existing Gap Needs Attributes Accessibility Processing Continue Continue Continue Continue Continue Below threshold Can’t define Unsuitable Excess gap Inadequate Generate ? Acquire ? Yes Yes Out No No D1.1 D1.2 D1.3 D1.4 D1.5

23. Decision Guide Considerations Explains the question. Classify a situation or write a short description. Complete a statement template. Decide where to go next. Not a cookbook to be followed without interpretation. Compliments experience & judgement; doesn’t replace them. Guide

24. Supply Phase Guide Development Model S1. Identification S2. Applicability Evaluation Conclusion

25. Applicability Stage Guide Existing Model Knowledge base Suitability Evaluation Specification criteria (4) Knowledge criteria (3) Business line Function Development S2.2 S2.3 S2.4 Identification Search Description S1 modify Data Availability Data criteria (5) Data Acquisition inaccessible Conclusion unsuitable unjustified Specifications S2.1 Development criteria (3) Development S2.5

26. Project Phase Guide End Design Applicability 4. Development 5. Evaluation 6. Implementation 7. Project Conclusion Out 3. Project Establishment Situation

27. Development Stage Guide Project Establishment Hierarchy Relationships Indicators Review Logic Computation Debugging Review Attributes Consistency Review Uncertainty Representation Review Conceptualization Construction Evaluation Verification Validation Inventory exit exit exit exit 4.2 4.3 4.4 4.5 continue continue continue continue Interaction Awareness Understanding Consensus 4.1

28. Outline I. Underlying Concepts Scientific underpinning II. Decision Guide Decision making III. Glossary Common understanding

29. Glossary Background (introduction, methods, references). Taxonomy (organization, nature, risk analysis, content, modelling, concepts). Definitions - 650 terms from six sources. Links to taxonomy and related terms. Glossary

30. Sample Definition Model: Abstract and simplified construct or representation of reality in the form of a pattern, description, or definition that shows the essential structure, relationships, and workings of a concept, process, or system. (see modelling approach , function, modelling methods, process, relationship, representation, system) Glossary

31. Modelling Framework: Supports an agency’s business Facilitates horizontal integration Minimizes waste & inefficiency Maximizes likely success Documents & justifies decisions “ Using a clear blueprint first prevents chaos latter.” Carla O’Dell (1998)