Advanced Level Course of Program Evaluation in Health Professions Education (The Aga Khan University)

1. Decision-Oriented Evaluation
Approaches
JIBRAN MOHSIN
Advanced Level Course on Program Evaluation in Health Professions Education (HPE)
Department of Educational Development (DED)
The Aga Khan University (AKU)

2. Introduction
• Rationale
• Evaluation being ignored in 1970’s
• Effective: Providing information for decision making by serving decision makers
• Approaches
• CIPP (Context-Input-Process-Product) Model (system, stage wise approach)
• UFE (Utilization-Focused Evaluation) Model (personal approach)
• Evaluability assessment / Performance monitoring

3. CIPP Evaluation Model
• IMPLEMENTIN
G decisions
• RECYCLIN
G
• decisions
• STRUCTIN
G decisions
• PLANNING
• decisions
Context
evaluation
Input
evaluation
Process
evaluation
Product
evaluation

4. CIPP Evaluation Model
• Evaluation Role
• Formative (Guidance, Prospective)
• Summative (Comparison, Retrospective)
• Steps
• Focus
• Information (collection, Organization, Analyses, Report)
• Administration
• Core values

5. Utilization-Focused Evaluation (UFE)
• Assumptions
• Primary purpose: inform decision
• Identify Primary/key user(s) – interested (“Personal factor”) + Authoritative
• Defined as
• Evaluation done for and with specific intended primary users for specific
intended uses”
• Stages
• Identification of intended user, conduct of study and result interpretation /
decision making

6. Evaluability assessment / Performance monitoring
• Evaluability assessment
• Program ready for evaluation?
• Program purpose defined?
• Activities matches goals?
• Managers interested?
• Performance Monitoring
• Performance indicators
• Accountability

7. Application to own educational practices
• Performance monitoring is commonly used to facilitate decision
making on outcome-based information.
• CIPP model can be used to evaluate the undergraduate program at
Medical College.
• Personal factor directed evaluation is another method of evaluating
the internship program at Medical College.

8. Application to own educational practices /
External Reading
• Study conducted by International Islamic University (IIU), Islamabad
• Objective:
• To evaluate the quality of education at school level by assessing the CIPP of two
schools falling under a Welfare School System in Rawalpindi.
• Conclusion:
• Welfare School System under study, barring a few shortcomings, is trying to
maintain quality and
• taking further steps for the improvement of quality of education.
Aziz S, Mahmood M, Rehman Zahra. Implementation of CIPP Model for Quality Evaluation at School Level: A Case Study. Journal of Education and Educational
Development. Jun 2018. 5(1);189-206.

9. Constructive Critique
• Strength
• Oldest approaches, still frequently used (Longevity)
• Help system /people in making decisions
• Weakness
• Social equity and equality being ignored (“Hired gun”)
• Lack of decisive leadership
• Inability to respond to critical questions not matching with concerns of
primary user
• Dynamical organizational environment not addressed

10. Key Messages
• CIPP Model describes 4 program stages and types of decisions
policymaker can face at each stage.
• UFE approach attempts to produce evaluations that improve
decisions by using personal factor (intended user with specific
intended use)
• Evaluability assessment designed to influence decision whether to
conduct evaluation, whereas performance monitoring provide
ongoing outcome information to allow data-based decisions.

11. THANK YOU