A Mo de lling
   Frame wo rk:
S uppo rting Evide nc e -
   Bas e d De c is io ns
     ModSimWorld
    Montreal, Quebec
   ...
Main
                           Messages

•   Models are needed to understand and
    predict the behavior of complex syst...
Outline


I. Underlying Concepts
      Scientific underpinning
II. Decision Guide
      Decision making
III. Glossary
    ...
Concepts

                                    About
                                    Models
• What are they?
  – Simpli...
Concepts

                                    What is a
                                    Framework?

“Structural outlin...
Concepts

                         Framework
                         Objectives
• Support needs-driven and science-
  dri...
Concepts

                        Framework
                        Design
• Reflect modelling, management, and
  user per...
Concepts

                            Different
                            Perspectives




 What developers proposed   W...
Concepts


                     Supply &
                     Demand
De mand: I have a
problem that needs
am odel.




   ...
Concepts


                                   Modelling
                                   System
                     Man...
Concepts

                              Modelling
                              Process
    Modelling combines science & c...
Concepts

                                                    Systems
                                                    ...
Concepts



                                      Data

    A model and its data are inseparable;
    they succeed or fail...
Concepts

                                                          Information
                                          ...
Concepts

                                         Models and
                                         Knowledge
         ...
Outline


I. Underlying Concepts
      Scientific underpinning
II. Decision Guide
      Decision making
III. Glossary
    ...
Decision Guide -
                      Hierarchy

•   Phase: (3) demand, supply, project
•   Stage: (7) approach, design, ...
Decision Guide -
Guide


          Stages
         Start: (Demand) Issue                                       Outputs
   ...
Guide

                   Decision
                   Guide Phases

        Issue              Model



    Demand        ...
Guide

                                                 Supply &
                                                 Demand

...
Guide

                          Demand
                          Phase

               Issue



             D1. Approach...
Approach
Guide


                                                 Stage
                            Issue
                ...
Guide

                   Decision Guide
                   Considerations
•   Explains the question.
•   Classify a situa...
Guide

                                Supply
                                Phase
                     Model



        ...
Applicability
Guide

                                Existing
                                Model
                      ...
Guide

                                 Project
                                 Phase
                    Design



Appli...
Developmen
Guide
                                   Project
                                Establishment

        Awarene...
Outline


I. Underlying Concepts
      Scientific underpinning
II. Decision Guide
      Decision making
III. Glossary
    ...
Glossary



                              Glossary


• Background (introduction, methods,
  references).
• Taxonomy (organ...
Glossary

                                      Sample
                                      Definition

Mo de l: Abstract...
Mo de lling Frame wo rk:



•Supports an agency’ business
                   s
•Facilitates horizontal integration
•Minimi...
Upcoming SlideShare
Loading in...5
×

Modeling Framework to Support Evidence-Based Decisions

1,371

Published on

Describes a framework for modelling in a regulatory environment founded on sound scientific and knowledge management concepts. It includes 1) demand (isue-driven) and supply (model driven) approaches to modelling, 2) balancing modeler, manager, and user perspectives, 3) documentation to demonstrate due diligence, and a 700-term glossary.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,371
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Modeling Framework to Support Evidence-Based Decisions

    1. 1. A Mo de lling Frame wo rk: S uppo rting Evide nc e - Bas e d De c is io ns ModSimWorld Montreal, Quebec June 8-9, 2009
    2. 2. Main Messages • Models are needed to understand and predict the behavior of complex systems. • Models are needed to fulfill an agency’s mandate and support its core business. • Inadequate or incorrect use of models wastes resources, results in errors, and exposes an agency to liability. Mo de ls s ho uld be used wisely 2
    3. 3. Outline I. Underlying Concepts Scientific underpinning II. Decision Guide Decision making III. Glossary Common understanding 3
    4. 4. Concepts About Models • What are they? – Simplified representations of reality. – Transform data, information, and knowledge into outputs. • Why do we use them? – Reality is too complex – Experiments are infeasible .. – Predict consequences – Increase understanding Nonaka (2000) 4
    5. 5. Concepts What is a Framework? “Structural outline of the components of an organization, system, or process and the relationships among them.” Understanding Knowledge Services NRCan (2006) 5
    6. 6. Concepts Framework Objectives • Support needs-driven and science- driven analysis. • Promote dialogue among modelers, managers, & users. • Reduce wasted time, effort, & money. • Provide a basis for planning and action. • Document and justify decisions. 6
    7. 7. Concepts Framework Design • Reflect modelling, management, and user perspectives. • Balance efficiency and effectiveness with cost and effort. • Applicable to both demand and supply approaches to modelling. • Applicable to both logical and computational models 7
    8. 8. Concepts Different Perspectives What developers proposed What managers funded What stakeholders wanted What users needed 8
    9. 9. Concepts Supply & Demand De mand: I have a problem that needs am odel. S upply: I have a m odel that solves your problem . 9
    10. 10. Concepts Modelling System Manage Use Nature, Internal External Society Models Models Develop Share Preserve Lost Knowledge Models Management 10
    11. 11. Concepts Modelling Process Modelling combines science & computers; judgement & experience; insight & intuition. • Principles: effort, simple, data, knowledge, transparent, understandable. • Complexity: Modelling is a dynamic feedback process with delays and uncertainty. • Development: techniques are well-understood; management less understood and practiced. • Use: Decision making under uncertainty, unknown elements, outcome probabilities. 11
    12. 12. Concepts Systems Hierarchy Ma ge na Mandate led Ma na me ge Business now io n ge nt K at Su me rm nt I nf o isi on pp ort Dec Models Data Policies Processes 12
    13. 13. Concepts Data A model and its data are inseparable; they succeed or fail as one. – Data Needs: Situation may involve nature, the system, and/or intervention. – Sampling: Statistics are essential to determine how much data is needed. – Source: Ownership? Use rights? Privacy & security concerns? – Scale: Time, space, and process scale must match the situation. – Quality: Level of accuracy, detail, and completeness are needed? 13
    14. 14. Concepts Information System Acquisition Storage Processing Outputs Environment Organization Knowledge Interface Access Interface Inputs Audience Channel System Outputs Search s Processing Use Data Retrieval Database Media Model Events Society Economy Availability Interoperability Integration Utility 14
    15. 15. Concepts Models and Knowledge 3. Complex 2. Complicated System: •Predictive feedback (+) •Feedback Behavior: •Non-linear (1:?) •Linear (1:n) Approach: •Simulation •Mathematics Model: •Stochastic •Deterministic Decision: •Uncertainty •Certainty Basis: •Tacit knowledge •Explicit knowledge •Emergent •Flow-through (-) •Disorganized •Fixed (1:1) •Scenario analysis •Planning •Mental •Mechanistic •Reaction •Automated •Intuition •Data, facts 4. Chaotic 1. Common 15
    16. 16. Outline I. Underlying Concepts Scientific underpinning II. Decision Guide Decision making III. Glossary Common understanding 16
    17. 17. Decision Guide - Hierarchy • Phase: (3) demand, supply, project • Stage: (7) approach, design, establish, develop, evaluate, implement, conclude • Step: (34) screening, problem definition, suitability, knowledge, data • Consideration (132): recurrence, importance, problem space, existence 17
    18. 18. Decision Guide - Guide Stages Start: (Demand) Issue Outputs End D1 7 (Manager) Approach Conclusion 6 Acquire Implementation Data Out (User) D2 5 Design Evaluation Generate 3 (Manager) Knowledge Establishment 4 S2 Development Applicability (Modeller) S1 (All) Start (Supply) Model identification 18
    19. 19. Guide Decision Guide Phases Issue Model Demand Supply Project End 19
    20. 20. Guide Supply & Demand Demand-driven Supply-driven backward chaining, closed question forward chaining, open question Start Start (model) Uses (use) Model 20
    21. 21. Guide Demand Phase Issue D1. Approach Generate Acquire Out Knowledge Data D2. Design Development 21
    22. 22. Approach Guide Stage Issue D1.1 Recurrence Below threshold Importance Initial Problem space Screening Existence Continue D1.2 Business Can’t define Function Problem Intended use Definition Continue D1.3 Time available Unsuitable Suitability Out Situation Continue D1.4 Needs Knowledge Excess gap Existing Generate Gap Evaluation ? No Continue Yes D1.5 Needs Attributes Data Inadequate Acquire Accessibility Availability ? No Processing Continue Yes Design 22
    23. 23. Guide Decision Guide Considerations • Explains the question. • Classify a situation or write a short description. • Complete a statement template. • Decide where to go next. • Not a cookbook to be followed without interpretation. • Compliments experience & judgement; doesn’t replace them. 23
    24. 24. Guide Supply Phase Model S1. Identification Development S2. Applicability Conclusion Evaluation 24
    25. 25. Applicability Guide Existing Model Stage S1 Search Description Identification S2.1 Specification criteria (4) Specifications S2.2 Development criteria (3) Development S2.3 Data inaccessible Data criteria (5) Data Availability Acquisition S2.4 Knowledge unjustified Knowledge base Conclusion criteria (3) S2.5 unsuitable Business line Function Suitability Development modify Evaluation 25
    26. 26. Guide Project Phase Design Applicability 3. Project Out Establishment 4. Development 5. Evaluation Situation 6. Implementation 7. Project End Conclusion 26
    27. 27. Developmen Guide Project Establishment Awareness 4.1 t Stage Understanding Consensus Interaction Hierarchy 4.2 Relationships exit Indicators Conceptualization Review continue Logic 4.3 Computation exit Debugging Construction Review continue Inventory Attributes 4.4 Consistency exit Review Verification continue 4.5 Uncertainty exit Representation Validation Review continue Evaluation 27
    28. 28. Outline I. Underlying Concepts Scientific underpinning II. Decision Guide Decision making III. Glossary Common understanding 28
    29. 29. Glossary Glossary • Background (introduction, methods, references). • Taxonomy (organization, nature, risk analysis, content, modelling, concepts). • Definitions - 650 terms from six sources. • Links to taxonomy and related terms. 29
    30. 30. Glossary Sample Definition Mo de l: Abstract and sim plified construct or representation of reality in the form of a pattern, description, or definition that show the essential s structure, relationships, and w orkings of a concept, process, or system . (s e e modelling approach, function, mode lling me thods , proce s s , re lations hip, re pre s e ntation, s y s te m) 30
    31. 31. Mo de lling Frame wo rk: •Supports an agency’ business s •Facilitates horizontal integration •Minimizes waste & inefficiency •Maximizes likely success •Documents & justifies decisions “ Using a clear blueprint first prevents chaos latter.” 31 Carla O’ (1998) Dell
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×