Chandigarh Call Girls Service ❤️🍑 9809698092 👄🫦Independent Escort Service Cha...
Tools in Measuring Program and Learning Outcomes - A Guide in Designing a Master Blueprint
1. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes
of Medical Students
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Subtitle: Tools in Measuring Knowledge, Skill and Attitude of Medical
Students - A Guide in Designing a Master Blueprint (Part 1)
Recommended Prior Study: Assessment and Evaluation of Medical
Students – ROJoson TPORs
2. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Notes on plans of ROJ-OCIL on Medical Education on the topic: Tools in
Measuring Program and Learning Outcomes of Medical Students
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Design and Development of Tools to Measure Students’ Learning and
Performance
Construction (Part 2)
Scoring / Grading (Part 3)
3. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Identify the various types of objectives and outcomes of a medical
curriculum that must be considered in order to serve as a guide to
both faculty and students in teaching and learning and in the global
or overall assessment and evaluation of students’ learning and
performance.
Define program educational objectives; program outcomes; course-
level learning outcomes; and session-level learning objectives.
Differentiate learning outcomes from learning objectives.
4. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe the program educational objectives expected by the
Commission on Higher Education (CHED) of a Philippine medical
school.
Enumerate the program outcomes required by CHED of a Philippine
medical school.
Describe the policies of CHED on the use of tools to assess and
evaluate medical students and use of performance indicators.
5. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Reproduce the framework recommended by ROJoson in the design
and development of tools to measure the global or overall
accomplishment of the students’ learning and performance in a
medical school.
Explain the factors to consider in the selection of tools to measure
students’ learning and performance vis-à-vis the various categories
of objectives and outcomes in a medical curriculum.
6. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe the three (3) domains of learning that may be used to
guide in the selection of tools to measure students’ learning and
performance on the various categories of objectives and outcomes
in a medical curriculum.
Describe Bloom’s taxonomy on cognitive domain that can be used in
the selection of tools to measure students’ learning and
performance.
7. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe Bloom’s taxonomy on affective domain that can be used in
the selection of tools to measure students’ learning and
performance.
Describe Bloom’s taxonomy on psychomotor domain that can be
used in the selection of tools to measure students’ learning and
performance.
8. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe the 3 general categories of tools to measure students’
learning and performance as recommended by ROJoson and give
examples of specific tools.
Given the domain of learning, give an example of a tool that may be
used to measure students’ learning and performance.
Given an action verb in a learning objective or outcome, give an
example of a tool that may be used to measure students’ learning
and performance.
9. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Make a draft of a master blueprint of tools to be used in the entire
medical school to measure global accomplishment of students’
learning and performance.
10. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
In the curriculum of a medical school, the
-PROGRAM EDUCATIONAL OBJECTIVES;
-PROGRAM OUTCOMES;
-COURSE-LEVEL LEARNING OUTCOMES; and
-SESSION-LEVEL LEARNING OBJECTIVES
should be spelled out to serve as a guide to both faculty and students
in teaching-learning and in the overall assessment and evaluation of
students’ learning and performance.
11. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM EDUCATIONAL OBJECTIVES
Program educational objectives are the broad statements
that describe the CAREER AND PROFESSIONAL ACCOMPLISHMENTS
that the program is preparing graduates to achieve.
12. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM EDUCATIONAL OBJECTIVES
Case in point: the CHED 2016 guidelines for Philippine medical schools
state the following Program Educational Objectives:
The main goal of the Doctor of Medicine program is to develop
professional physicians for the Philippine healthcare system. The
graduate of the Doctor of Medicine program is a primary care
physician (who can pursue general private medical practice) after
passing the Physician Licensure Examination (PLE).
13. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM OUTCOMES or PROGRAM LEARNING OUTCOMES
Program outcomes or program learning outcomes are the broad
statements
that describe the COMPETENCIES
that the program is preparing graduates to achieve.
14. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM OUTCOMES or PROGRAM LEARNING OUTCOMES
Case in point: the CHED 2016 guidelines for Philippine medical schools
state the following Program Outcomes:
15. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM OUTCOMES or PROGRAM LEARNING OUTCOMES
Program Outcomes Operational Definition of Program
Outcomes
Demonstrate clinical
competence
Completely manage clinical conditions
of all patients in various settings
Communicate effectively Convey information, in written and
oral formats, across all types of
audiences, venues and media in a
manner that can be easily understood
16. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM OUTCOMES or PROGRAM LEARNING OUTCOMES
Program Outcomes Operational Definition of Program
Outcomes
Lead and manage health
care teams
Initiate planning, organizing,
implementation and evaluation of
programs and health facilities.
Provide clear direction, inspiration
and motivation to the healthcare
team/community
Engage in research activities Utilize current research evidence in
decision making as practitioner,
educator or researcher. Participate in
research activities.
17. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM OUTCOMES or PROGRAM LEARNING OUTCOMES
Program Outcomes Operational Definition of Program
Outcomes
Collaborate within
interprofessional teams
Effectively work in teams in managing
patients, institutions, projects and
similar situations
Utilize system-based
approach to healthcare
Utilize systems-based approach in
actual delivery of care; network with
relevant partners in solving general
health problems
Engage in continuing
personal and professional
development
Update oneself through a variety of
avenues for personal and professional
growth to ensure quality healthcare
and patient safety
18. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
PROGRAM OUTCOMES or PROGRAM LEARNING OUTCOMES
Program Outcomes Operational Definition of Program
Outcomes
Adhere to ethical,
professional, and legal
standards
Adhere to national and international
codes of conduct and legal standards
to govern the profession
Demonstrate nationalism,
internationalism and
dedication to service
Demonstrate love for one’s national
heritage, respect for other cultures
and commitment to service
Practice the principles of
social accountability
Adhere to the principles of relevance,
equity, quality and cost effectiveness
in the delivery of healthcare to
patients, families and communities
19. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
COURSE-LEVEL LEARNING OUTCOMES
Course-level learning outcomes are specific statements of learning,
performance and attitude that the students need to achieve at the end
of a particular course or study.
The course-level learning outcomes contribute to the achievement of
program outcomes and eventually the program educational objectives.
20. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
SESSION-LEVEL LEARNING OBJECTIVES
Session-level learning objectives are specific statements of learning,
performance and attitude that the students need to achieve after an
educational activity within a course.
The session-level learning objectives contribute to the achievement to
course-level learning outcomes.
21. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
LEARNING OUTCOMES vs LEARNING OBJECTIVES
A note on learning outcomes vs learning objectives, terms commonly
encountered in education and commonly used interchangeably.
Learning outcomes describe in observable and measurable terms what
students should be able to do after completing a learning experience.
Learning objectives usually describe, also in observable and
measurable terms, what a faculty aims to do through organizing
specific topics or individual learning activities to achieve the learning
outcomes. Students may be allowed to contribute to the formulation
of learning objectives of an educational session.
22. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools with Performance Indicators in Measuring Program and Learning
Outcomes of Medical Students
In the curriculum of a medical school, there shall be tools with
performance indicators to measure the degree of accomplishment of
the students at ALL levels, namely, PROGRAM EDUCATIONAL
OBJECTIVES; PROGRAM OUTCOMES; COURSE-LEVEL LEARNING
OUTCOMES; and SESSION-LEVEL LEARNING OBJECTIVES.
The performance indicators will serve as a basis for assessment and
evaluation of student achievement through different points in the
curriculum, from start to finish.
CHED POLICIES
23. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools with Performance Indicators in Measuring Program and Learning
Outcomes of Medical Students
ROJoson Observation 1:
A lot of times, a lot of medical schools utilize written tests ONLY to
assess and evaluate students. There should be other tools considering
there are other domains of learning to assess and evaluate such as the
affective and psychomotor domains.
Written tests are generally used for cognitive domain and not for
affective and psychomotor domains. Within the cognitive domain,
there are also different levels of learning (see Bloom’s taxonomy of
cognitive domain) that must be assessed and evaluated which require
tools other than written tests.
24. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools with Performance Indicators in Measuring Program and Learning
Outcomes of Medical Students
ROJoson Observation 2:
The Physician Licensure Examination evaluates graduates on the
cognitive domains ONLY, none on the affective and psychomotor
domains.
Ideally, not enough. However, it has been the practice since time
immemorial.
25. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
ROJoson recommended framework
for the design and development of tools to measure the global
accomplishment of the student learning and performance
Elements to
Measure
Tools to Use
to Measure
Performance
Indicators
Assessment
and
Evaluation
Program Educational
Objectives
Program Outcomes
Course-level Learning
Outcomes
Session-level Learning
Objectives
26. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The choice of what tools to use to measure will first depend on the
elements to be measured. These elements should be listed down on
the column under “Elements to Measure.”
Then, depending on the elements to measure, one or several
appropriate tools will be selected and later implemented.
27. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The factors that govern the selection of measuring tools based on the
elements to measure consist of the following:
• Types of learning domain – cognitive, affective and psychomotor
• Levels of learning in each domain – Bloom’s taxonomy
• Behavioral verbs in the learning objectives and outcomes
Other factors that govern the selection of tools:
• Validity
• Reliability
• Usability These will be expounded one by one.
28. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The factors that govern the selection of measuring tools based on the
elements to measure consist of the following:
• Types of learning domain – cognitive, affective and psychomotor
• Levels of learning in each domain – Bloom’s taxonomy
• Behavioral verbs in the learning objectives and outcomes
Other factors that govern the selection of tools:
• Validity
• Reliability
• Usability These will be expounded one by one.
29. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
To facilitate the selection of the tools, the “Elements to Measure” can
be classified into 3 domains for learning:
cognitive, psychomotor and affective
or into the 3 aspects of learning for competence:
knowledge, skills and attitude.
30. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
3 domains of learning relevant to medical education / medical
students:
• Cognitive domain (KNOWING; KNOWLEDGE; mental skills)
• Psychomotor domain (DOING; SKILLS; manual or physical skills)
• Affective domain (FEELING; ATTITUDE; growth in feelings or
emotional areas)
31. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
3 domains of learning relevant to medical education / medical
students:
COGNITIVE domain (KNOWING; KNOWLEDGE; mental skills)
Focusing on knowledge acquisition and intellectual skills and
abilities
(e.g. the diagnosis of disease, strategizing treatment options)
32. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
3 domains of learning relevant to medical education / medical
students:
PSYCHOMOTOR domain (DOING; SKILLS; manual or physical skills)
• Relating to skills that require varying levels of well-coordinated
physical activity and precise manipulative procedures
• (e.g. doing physical examination, performing a needle aspiration,
suturing of an open wound)
33. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
3 domains of learning relevant to medical education / medical
students:
AFFECTIVE domain (FEELING; ATTITUDE; growth in feelings or
emotional areas)
• Dealing with feelings, emotions, mindsets and values, including the
nurturing of desirable attitudes for personal and professional
development
• (e.g. allaying the concerns and fears of patients, displaying empathy
for the relatives of a patient who has just died, displaying mutual
trust and respect in working with members of the healthcare team,
upholding high ethical standards in practice)
34. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The factors that govern the selection of measuring tools based on the
elements to measure consist of the following:
• Types of learning domain – cognitive, affective and psychomotor
• Levels of learning in each domain – Bloom’s taxonomy
• Behavioral verbs in the learning objectives and outcomes
Other factors that govern the selection of tools:
• Validity
• Reliability
• Usability These will be expounded one by one.
35. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Under each domain of learning, there are different levels of
complexity and specificity that are described and being advocated by
educators.
Bloom’s taxonomy of learning is most popular and widely used
globally.
• Bloom’s Taxonomy of Cognitive Domain
• Bloom’s Taxonomy of Affective Domain
• Bloom’s Taxonomy of Psychomotor Domain
36. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Bloom’s Taxonomy of Cognitive, Affective and Psychomotor Domain
Under each domain of learning, there are different levels of
complexity and specificity. These should serve as a guide in the
selection of measuring tools.
Cognitive
Domain
Affective
Domain
Psychomotor
Domain
Lower/est level Remembering Receiving Imitating
Understanding Responding Manipulating
Applying Valuing Precising
Analyzing Organizing Articulating
Evaluating Characterizing Naturalizing
Higher/est level Creating
37. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Before continuing on with the discussion on the factors that govern
the selection of measuring tools, presentation of the different
categories of tools is in order.
These tools can generally be categorized into
1) test;
2) observation or observation technique; and
3) project or project-based technique.
Under each category are many more specific tools. This will be
expounded in another lecture or talk. Some examples are given below
just to give an idea what can fall under each general category of tools.
38. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Examples of common TESTS are written tests (paper-pencil or
digital/online) and oral exams (face-to-face or tele/online).
Examples of common procedures under OBSERVATION are the
practical exams; the Objective Structured Clinical Examination (OSCE)
and observation of performance skills and attitude using checklists
and rating scales; anecdotal records; and critical incident technique.
Examples of PROJECTS to measure students’ learning are case study
and discussion; management of actual problem-solving situations;
research projects; and learning and reflection portfolios; etc.
39. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
*Note: Multiple tools may be selected / or necessary to capture all or
most aspects of competency if any single method may not be sufficient
to do the job.
**Tests are weakest on assessing / evaluating psychomotor or
affective learning domains.
***Observation may be used to assess/evaluate multiple learning
domains simultaneously, psychomotor, cognitive and affective. Direct
observation are stronger tools than indirect observation.
40. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Below is a table of tools commonly and can be used to measure
students’ learning and performance based on the learning domains
implied in the learning objectives and outcomes.
Learning Domains
Implied in learning objectives
and outcomes
Tools that can be used to measure learning /
performance*
Cognitive domain Tests; observation; project
Affective domain Observation; project
- anecdotal records; portfolios; critical incident
technique
Psychomotor domain Observation – practical exams (such as objective
structured clinical exam); project
41. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
*Note: Multiple tools may be selected / or necessary to capture all or most aspects
of competency if any single method may not be sufficient to do the job.
** Observation may be used to assess/evaluate multiple learning domains
simultaneously, psychomotor, cognitive and affective. Direct observation are
stronger tools than indirect observation.
Learning Domains
Implied in learning objectives
and outcomes
Tools that can be used to measure learning /
performance*
Cognitive domain Tests; observation; project
Affective domain Observation; project
- anecdotal records; portfolios; critical incident
technique
Psychomotor domain Observation – practical exams (such as objective
structured clinical exam); project
42. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Below is a table of tools commonly and can be used to measure
students’ learning and performance based on the level of learning in
each domain (Bloom’s taxonomy).
Cognitive
Domain
Examples of tools to measure
(may have several options and combination)
REMEMBERING Tests
UNDERSTANDING Tests
APPLYING Tests, observation
ANALYZING Tests, observation, projects
EVALUATING Tests, observation, projects
CREATING Projects
43. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Below is a table of tools commonly and can be used to measure
students’ learning and performance based on the level of learning in
each domain (Bloom’s taxonomy).
Affective
Domain
Examples of tools to measure
(may have several options and combination)
Receiving Observation
Responding Observation
Valuing Observation, project
Organizing Observation, project
Characterizing Observation, project
44. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Below is a table of tools commonly and can be used to measure
students’ learning and performance based on the level of learning in
each domain (Bloom’s taxonomy).
Psychomotor Domain Examples of measuring tools
(may have several options and
combination)
Imitating Observation
Manipulating Observation
Precising Observation, project
Articulating Observation, project
Naturalizing Observation, project
45. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The factors that govern the selection of measuring tools based on the
elements to measure consist of the following:
• Types of learning domain – cognitive, affective and psychomotor
• Levels of learning in each domain – Bloom’s taxonomy
• Behavioral verbs in the learning objectives and outcomes
Other factors that govern the selection of tools:
• Validity
• Reliability
• Usability These will be expounded one by one.
46. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Behavioral verbs in the learning objective and outcomes should also
serve as a guide in the selection of measuring tools.
USE BEHAVIORAL VERBS TO COME OUT WITH BEHAVIORAL
OBJECTIVES.
Behavioral objectives describe what the student will be able to do
after a learning activity, something that is observable and measurable.
Each objective should begin with a verb that describes an observable
behavior, such as “describe, summarize, demonstrate, compare, plan,
score”, etc.
One can observe the student and/or measure how well the objective
was met.
47. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
USE BEHAVIORAL VERBS TO COME OUT WITH BEHAVIORAL
OBJECTIVES.
Below is a table of tools commonly and can be used to measure
students’ learning and performance based on the behavioral verbs in
the learning objectives and outcomes.
48. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Behavioral verbs in learning objectives
and outcomes
Tools that can be used to measure
learning / performance*
Define Tests (written or oral)
Discuss / explain Tests (written – essay or oral)
Apply Tests; observation; project
Evaluate Tests; observation; project
Create Project
Demonstrate a medical procedural skill Observation
Demonstrate professionalism in
patient care
Observation (direct / indirect) – portfolio;
critical incident technique; anecdotal
reports
Demonstrate oral communication skills Observation
Demonstrate a behavior / attitude Observation (direct / indirect) – portfolio;
critical incident technique; anecdotal
reports
Critique a paper Project-based technique
49. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Behavioral verbs in learning objectives
and outcomes
Tools that can be used to measure
learning / performance*
Define Tests (written or oral)
Discuss / explain Tests (written – essay or oral)
Apply Tests; observation; project
Evaluate Tests; observation; project
Create Project
Demonstrate a medical procedural skill Observation
Demonstrate professionalism in
patient care
Observation (direct / indirect) – portfolio;
critical incident technique; anecdotal
reports
Demonstrate oral communication skills Observation
Demonstrate a behavior / attitude Observation (direct / indirect) – portfolio;
critical incident technique; anecdotal
reports
Critique a paper Project-based technique
*Note: Multiple tools may be selected / or necessary to capture
all or most aspects of competency if any single method may not
be sufficient to do the job.
**Observation may be used to assess/evaluate multiple
learning domains simultaneously, psychomotor, cognitive and
affective. Direct observation are stronger tools than indirect
observation.
50. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The factors that govern the selection of measuring tools based on the
elements to measure consist of the following:
• Types of learning domain – cognitive, affective and psychomotor
• Levels of learning in each domain – Bloom’s taxonomy
• Behavioral verbs in the learning objectives and outcomes
Other factors that govern the selection of tools:
• Validity
• Reliability
• Usability
51. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The specific tool chosen for “Element to Measure” must be
appropriate or valid; reliable and usable or practical.
Validity refers to the appropriateness, adequacy, accuracy and
effectiveness of the measuring tools (they measure what they are
designed to measure).
Example of non-validity: using a tool that measures a medical
student’s technical proficiency is not valid for predicting bedside
manner.
Another example of non-validity: using written tests to
assess/evaluate a medical student’s skill is not valid.
52. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Reliability refers to the consistency of measuring tools in producing
results. Consistent results mean that a student being assessed and
evaluated gets the same score or grade no matter how, where or when
they are subjected to the measuring tools.
Put in another way, if similar scores are obtained when the same
measuring tool is used on the same students at two different
occasions, it can be concluded that the results have a high degree of
reliability from one occasion to another.
Similarly, if different faculty mark students’ answer on the same
measuring tool and obtain quite similar results, it can also be
concluded that the results have a high degree of reliability.
53. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Usability refers to the practicality of the measuring tools. It should be
economical in terms of time and money, easily administered and
scored, and produce results that can be interpreted and applied by the
medical school.
54. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
SUMMARY OF WHAT HAVE BEEN PRESENTED
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Subtitle: Tools in Measuring Knowledge, Skill and Attitude of Medical
Students - A Guide in Designing a Master Blueprint (Part 1)
55. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
ROJoson recommended framework
for the design and development of tools to measure the global
accomplishment of the student learning and performance
Elements to
Measure
Tools to Use
to Measure
Performance
Indicators
Assessment
and
Evaluation
Program Educational
Objectives
Program Outcomes
Course-level Learning
Outcomes
Session-level Learning
Objectives
56. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Elements to measure
Analyze the statements on program and learning objectives and
outcomes
- Identify the domains of learning – cognitive, psychomotor, affective
- Identify the levels of learning – Bloom’s taxonomy
- Identify the behavioral verbs used
57. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
The factors that govern the selection of measuring tools based on the
elements to measure consist of the following:
• Types of learning domain – cognitive, affective and psychomotor
• Levels of learning in each domain – Bloom’s taxonomy
• Behavioral verbs in the learning objectives and outcomes
Other factors that govern the selection of tools:
• Validity
• Reliability
• Usability
58. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools to Use to Measure
3 general categories of tools to select from: TEST, OBSERVATION,
PROJECTS
Select one or a combination of tools which are VALID, RELIABLE and
USABLE for a particular program or learning objective or outcome.
59. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Performance Indicators
Decide on:
• PASS / FAIL
• SATISFACTORY / UNSATISFACTORY / NEEDS IMPROVEMENT
60. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Assessment and Evaluation
Decide whether and which tools will be used for:
• FORMATIVE ASSESSMENT
• SUMMATIVE ASSESSMENT (EVALUATION)
61. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
FOR ENHANCEMENT OF LEARNING
FOR DEVELOPMENT OF THE MEDICAL SCHOOL
MAKE A MASTER BLUEPRINT OF TOOLS IN MEASURING PROGRAM
AND LEARNING OUTCOMES OF MEDICAL STUDENTS
(PART OF QUALITY MANAGEMENT SYSTEM)
ASSIGNMENT
62. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Assignment:
Make a draft of a master blueprint of tools to be used in the
entire medical school to measure global accomplishment of
students’ learning and performance.
63. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Assignment Instructions:
Suggest utilize the framework recommended by ROJoson in the
design and development of tools to measure the global or overall
accomplishment of the students’ learning and performance in a
medical school.
64. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Assignment Instructions:
Elements to
Measure
Tools to Use
to Measure
Performance
Indicators
Assessment
and
Evaluation
Program Educational
Objectives
Program Outcomes
Course-level Learning
Outcomes
Session-level Learning
Objectives
65. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Assignment Instructions:
For the Program Educational Objectives and Program Outcomes, use
those mentioned in CHED 2016 guidelines for medical school.
For the Course-level Learning Outcomes, use a course one is
presently handling. Specific the name of the course.
For the Session-level Learning Objectives, use a session of learning
one is presently handling or has handled. Specific the name or title
of the learning session.
Place the name of the medical school.
Place the name of the faculty accomplishing this assignment.
66. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Assignment Outcome
Target:
To synthesize and integrate all inputs of several faculty to become the
master blueprint for the entire medical school (if there is no existing
one right now).
Tools in Measuring Program and Learning Outcomes of Medical
Students - A Master Blueprint
Name of Medical School
67. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Assignment Outcome
Target:
To synthesize and integrate all inputs of several faculty to become the
master blueprint for the entire medical school (if there is no existing
one right now).
Tools in Measuring Program and Learning Outcomes of Medical
Students - A Master Blueprint
Name of Medical School
ROJoson will help start the ball rolling by giving
inputs particularly on the program educational
objectives and program outcomes.
68. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to Measure Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Program
Educational
Objectives
Formative
assessment
Summative
Assessment
(Evaluation)
To develop professional
primary care physicians
for the Philippine
healthcare system
(Assumption: there is a
medical curriculum
designed and developed
towards this objective)
All the measuring
tools (tests,
observation,
project) planned
out / included in
all the courses of
the curriculum
(see master
blueprint of
measuring tools)
Passing grades in
all the courses in
the medical
curriculum
Pass
Fail
69. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to Measure Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Program
Educational
Objectives
Formative
assessment
Summative
Assessment
(Evaluation)
To qualify for the
Philippine Licensure
Examination (PLE)
[Assumption: graduating
from a CHED-approved
medical curriculum /
school qualifies one to
take PLE]
All the measuring
tools (tests,
observation,
project) planned
out / included in
all the courses of
the curriculum
(see master
blueprint of
measuring tools)
Passing grades in
all the courses in
the medical
curriculum
Pass
Fail
70. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to Measure Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Program
Educational
Objectives
Formative
assessment
Summative
Assessment
(Evaluation)
To enable graduates to pass
the Philippine Licensure
Examination (PLE)
[Assumption: graduating
from a CHED-approved
medical curriculum should
automatically enable one to
pass the PLE; however,
contents of PLE may not be
completely covered in the
medical curriculum / school.
Also, there may be
differences between PLE
examiners and faculty in
answering questions in PLE.]
Written
examinations
patterned after PLE
(multiple choice
questions – 4
options)
and inclusion of
common questions
in PLE
(school will have a
collection of
common PLE
questions)
Passing grade Pass
Fail
71. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to
Measure
Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Program Outcomes See below See below Formative
assessment
Summative
Assessment
(Evaluation)
Demonstrate clinical
competence
Satisfactory
Unsatisfactory
Pass
Fail
Communicate effectively Satisfactory
Unsatisfactory
Pass
Fail
Lead and manage health
care teams
Satisfactory
Unsatisfactory
Pass
Fail
Engage in research activities Satisfactory
Unsatisfactory
Pass
Fail
72. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to
Measure
Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Program Outcomes See below See below Formative
assessment
Summative
Assessment
(Evaluation)
Collaborate with
interprofessional teams
Satisfactory
Unsatisfactory
Pass
Fail
Utilize systems-based
approach to healthcare
Satisfactory
Unsatisfactory
Pass
Fail
Engage in continuing
personal and professional
development
Satisfactory
Unsatisfactory
Pass
Fail
Adhere to ethical,
professional and legal
standards
Satisfactory
Unsatisfactory
Pass
Fail
73. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to
Measure
Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Program Outcomes See below See below Formative
assessment
Summative
Assessment
(Evaluation)
Demonstrate nationalism,
internationalism and
dedication to service
Satisfactory
Unsatisfactory
Pass
Fail
Practice the principles of
social accountability
Satisfactory
Unsatisfactory
Pass
Fail
74. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Program Outcomes Operational Definition of Program
Outcomes
Tools to Use to
Measure
*Demonstrate clinical
competence
Completely manage clinical
conditions of all patients in various
settings
Combination of
Tests,
Observation and
Project
*Communicate effectively Convey information, in written and
oral formats, across all types of
audiences, venues and media in a
manner that can be easily
understood
Observation
Note1: Those with (*) are more of learning in the psychomotor domain whereas those with
(**) are more of affective domain of learning.
Note2: Combination of tools are recommended to be used.
75. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Program Outcomes Operational Definition of Program
Outcomes
Tools to Use to
Measure
*Lead and manage health
care teams
Initiate planning, organizing,
implementation and evaluation of
programs and health facilities.
Provide clear direction, inspiration
and motivation to the healthcare
team/community
Observation and
Project
*Engage in research
activities
Utilize current research evidence in
decision making as practitioner,
educator or researcher.
Participate in research activities.
Observation and
Project
76. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Program Outcomes Operational Definition of Program
Outcomes
Tools to Use to
Measure
*Collaborate within
interprofessional teams
Effectively work in teams in
managing patients, institutions,
projects and similar situations
Observation and
Project
*Utilize system-based
approach to healthcare
Utilize systems-based approach in
actual delivery of care
Network with relevant partners in
solving general health problems
Combination of
Tests,
Observation and
Project
Note1: Those with (*) are more of learning in the psychomotor domain whereas
those with (**) are more of affective domain of learning.
Note2: Combination of tools are recommended to be used.
77. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Program Outcomes Operational Definition of Program
Outcomes
Tools to Use to
Measure
**Engage in continuing
personal and professional
development
Update oneself through a variety
of avenues for personal and
professional growth to ensure
quality healthcare and patient
safety
Observation and
Project
**Adhere to ethical,
professional, and legal
standards
Adhere to national and
international codes of conduct and
legal standards to govern the
profession
Observation and
Project
Note1: Those with (*) are more of learning in the psychomotor domain whereas those with (**) are
more of affective domain of learning.
Note2: Combination of tools are recommended to be used.
78. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Program Outcomes Operational Definition of Program
Outcomes
Tools to Use to
Measure
**Demonstrate
nationalism,
internationalism and
dedication to service
Demonstrate love for one’s
national heritage, respect for other
cultures and commitment to
service
Observation and
Project
**Practice the principles of
social accountability
Adhere to the principles of
relevance, equity, quality and cost
effectiveness in the delivery of
healthcare to patients, families
and communities
Observation and
Project
Note1: Those with (*) are more of learning in the psychomotor domain whereas those with (**) are
more of affective domain of learning.
Note2: Combination of tools are recommended to be used.
79. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to
Measure
Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Course-level Learning
Outcomes
Session-level
Learning Objectives
Cognitive
domain
(knowledge) -
with different
hierarchical
levels (see
Bloom’s
taxonomy)
May have
several options
but choose one
or combination
that are most
appropriate
guided by
behavioral
verbs used
Pass
Fail
Formative
assessment
Summative
Assessment
(Evaluation)
80. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to
Measure
Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Course-level Learning
Outcomes
Session-level
Learning Objectives
Affective
domain
(knowledge) -
with different
hierarchical
levels (see
Bloom’s
taxonomy)
May have
several options
but choose one
or combination
that are most
appropriate
guided by
behavioral
verbs used
Pass
Fail
Formative
assessment
Summative
Assessment
(Evaluation)
81. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Master Blueprint of Tools in Measuring Program and Learning Outcomes of Medical Students
Name of School
Elements to
Measure
Tools to Use to
Measure
Performance
Indicators
Assessment and
Evaluation
Course-level Learning
Outcomes
Session-level
Learning Objectives
Psychomotor
domain
(knowledge) -
with different
hierarchical
levels (see
Bloom’s
taxonomy)
May have
several options
but choose one
or combination
that are most
appropriate
guided by
behavioral
verbs used
Pass
Fail
Formative
assessment
Summative
Assessment
(Evaluation)
82. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Identify the various types of objectives and outcomes of a medical
curriculum that must be considered in order to serve as a guide to
both faculty and students in teaching and learning and in the global
or overall assessment and evaluation of students’ learning and
performance.
Define program educational objectives; program outcomes; course-
level learning outcomes; and session-level learning objectives.
Differentiate learning outcomes from learning objectives.
83. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe the program educational objectives expected by the
Commission on Higher Education (CHED) of a Philippine medical
school.
Enumerate the program outcomes required by CHED of a Philippine
medical school.
Describe the policies of CHED on the use of tools to assess and
evaluate medical students and use of performance indicators.
84. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Reproduce the framework recommended by ROJoson in the design
and development of tools to measure the global or overall
accomplishment of the students’ learning and performance in a
medical school.
Explain the factors to consider in the selection of tools to measure
students’ learning and performance vis-à-vis the various categories
of objectives and outcomes in a medical curriculum.
85. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe the three (3) domains of learning that may be used to
guide in the selection of tools to measure students’ learning and
performance on the various categories of objectives and outcomes
in a medical curriculum.
Describe Bloom’s taxonomy on cognitive domain that can be used in
the selection of tools to measure students’ learning and
performance.
86. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe Bloom’s taxonomy on affective domain that can be used in
the selection of tools to measure students’ learning and
performance.
Describe Bloom’s taxonomy on psychomotor domain that can be
used in the selection of tools to measure students’ learning and
performance.
87. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Describe the 3 general categories of tools to measure students’
learning and performance as recommended by ROJoson and give
examples of specific tools.
Given the domain of learning, give an example of a tool that may be
used to measure students’ learning and performance.
Given an action verb in a learning objective or outcome, give an
example of a tool that may be used to measure students’ learning
and performance.
88. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Learning objectives:
At the end of the session, the student must be able to:
Make a draft of a master blueprint of tools to be used in the entire
medical school to measure global accomplishment of students’
learning and performance.
89. ROJoson OCIL on Medical Education
OCIL = Online Collaborative and Interactive
Learning
Tools in Measuring Program and Learning Outcomes
of Medical Students
Tools in Measuring Program and Learning Outcomes of Medical
Students – A Guide in Designing a Master Blueprint (Part 1)
Subtitle: Tools in Measuring Knowledge, Skill and Attitude of Medical
Students - A Guide in Designing a Master Blueprint (Part 1)
Editor's Notes
This talk is entitled Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Designing a Master Blueprint. This will be Part 1 of the Tools in Measuring Program and Learning Outcomes. There will be subsequent parts in this big topic.
The subtitle of this talk is Tools in Measuring Knowledge, Skills and Attitude of Medical Students – A Guide in Designing a Master Blueprint.
Recommended prior study is Assessment and Evaluation of Medical Students – ROJoson TPORs.
Notes on plans of ROJ-OCIL on Medical Education on the topic: Tools in Measuring Program and Learning Outcomes of Medical Students
Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Designing a Master Blueprint (Part 1)
Design and Development of Tools to Measure Students’ Learning and Performance
Construction (Part 2)
Scoring / Grading (Part 3)