• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Blueprint of exam questions
 

Blueprint of exam questions

on

  • 650 views

Often a recommendation of the external examiners and the accreditation board is the blueprinting of undergraduate examination questions. A blueprint of examination on individual assessment tool ...

Often a recommendation of the external examiners and the accreditation board is the blueprinting of undergraduate examination questions. A blueprint of examination on individual assessment tool reports the students’ learning outcome. It produces a link between test items and students learning objective. It is one of the most important instruments on outcome assessment essentials that ensure the desired coverage of content and objectives at different level of training.

Statistics

Views

Total Views
650
Views on SlideShare
650
Embed Views
0

Actions

Likes
0
Downloads
20
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Blueprint of exam questions Blueprint of exam questions Presentation Transcript

    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Prof. Dr. Shahid Hassan MHPE H/O Unit of Medical Education Faculty of Medicine and Health Sciences Universiti Putr Malaysia
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Definition: A blueprint of examination also known as the test specification provides examination strategy of an institution at a glance. •  Blueprint is the matrix or chart reporting the number and type of test questions represented across the topics in content area, consistent with learning objective and relative weight on test given to each topic. •  A blueprint also identifies the % weighting of cognitive dimensions as the level of competence tested in each konowledge domain.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Purpose of Blueprints: The purpose of constructing examination using a blueprint is to provide a conceptual map of examination format and the content area represented in assessment. Blueprint of examination is the widely used assessment tool that provides list of information about: 1. Type of measurement tools and proportion of each question format in assessment shown in respective weighting colunm. 2. Topics and the level of training for each topic and the relevant learning objectives.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Purpose of Blueprints: 3. Topics and the relative weighting given to each topic in content area. 4. Competencies as unidimesional cognitive process tested in knowledge and skills domains of Bloom’s taxonomy of educatioal objective and Miller’s pyramid of competence learning respectively. 5. Knowledge as multidimensional process of learning proposed as taxonomy table by Anderson including factual knowledge, conceptual knowledge and procedural knowledge.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Bloom’s Unidimensional Taxonomy of Educational Objective   Learning Objective/Outcome Cognitive dimension Knowledge Comprehension Analysis Application Synthesis Evaluation 1 2 3 4 5 6
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Anderson’s Multidimensional Taxonomy Table   Learning Domains è Knowledge Dimension ê Factual Knowledge Conceptual Knowledge Procedural Knowledge Metacognitiv e Knowledge The Cognitive Process Dimension Recall Understand Apply Analyze Evaluate Create
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Who will Use the Blueprint: The blueprint is a globally accepted tool used to construct examination questions for ready reference of stakeholders engaged in assessment process. Following stakeholders can use the blueprints readily available. 1. Candidates to use examination blueprints to ensure prpeparation of assessment for its time management and strategy to achieve the desired outcom of passing the examination.  
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Who will Use the Blueprint: Eevery student however, must be informed that:   •  All topics in blueprint may not be listed and a curriculum detail on content should always be sought by students. •  Topics not listed in blueprints are still part of content, which can be tested through assignments, student centered learning (SCL) and continuous (formative) assessment. •  Students must take the responsibility of acquiring a broadbased knowledge of all those topics not listed in blueprint.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Who will Use the Blueprints: 2. Teaching faculty inclusive of package and module coordinators to ensure the validity of assessment in term of assessment questions being representative of content area documented in curriculum. 3. External examiners to know the assessment specification with optimal information in given time to help facilitate the assessment and review the learning process.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Who will Use the Blueprint: 4. Members of vetting committee to ensure specification of questions being consistent with konowledge and skills domain versus cognitive dimension as per the learning objectives of each module. •  Blueprint also provides a readily available opportunity to vetting members to ensure that the essential topics are not overlooked.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Essentials of Blueprints: The objective of examination at different level of training will vary from topic to topic. A blueprint of examination should not only be consistent with those objectives listed for each topic but should also reflect to achieving them for reasons: 1.  Candidats’ are required to attain the competencies listed in assessment in order to be be promoted to next level in training. •  A blueprint therefore must reflect that a candiadte has acquired the competencies to be able to advance to next level.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Essentials of Blueprints: 2. Blueprint is seen as readily available document, which contains the list of topics covered under each module with its identified learning objectives as blueprint. •  However, to develop the blueprint of examination questions in MD program, we take overall objectives of individual modules in each package as the prelude for listing the outcome compentency of each objective. .
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Essentials of Blueprints: 3. Learning outcome is viewed as module competencies essential for each module and a blueprint provide the eveidence of as many modules as possible reperesented in percentage weighting. •  This will reflect on content validity of assessment in blueprint. •  A blueprint representating the percentage weighting of content across assessment is one of the essentials of assessment validity.
    • 1st Blueprint of Examination Questions Professional Examination of MD Program in FMHS at UPM Guideline to Prepare Blueprints: Blueprint of examination is produced as three separate tables: Table 1: A table reflecting the type of questions and their weighting respectively. •  Ideally each measurement tool and items in a mesure should also have percentage weighting depending on cognitive demand of question. •  In fact time allocation should also vary between questions of same construct such as MTF vs. SBAQ.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Guideline to Prepare Blueprints: •  Currently all question regardless of cognitive demand carried the same marks weighting. We however, follow the practice of marks weighting between the tools but not between the items (all MTF/SBAQ questions carry equal marks). •  We have allocated equal marks to each item in MCQ irrespective of complexity of cognitive demand it is intended to test. •  The current practice somehow sends a wrong message to students that questions with factual recall is as mportant as applications and analysis of knowledge.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Guideline to Prepare Blueprints: Table 2: A series of tables showing the contents with its objectives and percentage weighting and preferably the level of training for each topic/module and this may need several tables for each topic or module in a package
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Guideline to Prepare Blueprints: Table 3: A table potraying the ultimate blueprint developed as a chart representing each module. Total number of questions with its percentage weighting and number of questions weighted against their ability to test the levels of cognitive dimension described by Bloom’s taxonomy of knowledge is the characterstics of blueprint
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Structural Components of Blueprint: Table 1. Type of Measurement Tools in Assessment: •  This table comprises of examination question guided by type of assessment tools and their approximate weighting. This table is the first step towards developing the actual blueprint (See Table 1)
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Structural Components of Blueprint: Table 2: Topics with objectives and learning outcome: •  The next table comprised of a series of matrix containing the code and the title of course content (topics/module), learning objectives, learning outcome (module competencies), weighting and preferably the level of training (See Table 2) •  This may help student and faculty to know about competencies to be learned in the course, highlighted in blueprint.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Structural Components of Blueprint: Table 2: Topics with objectives and learning outcome: •  A summary of detail weighting of content area from each module in a package currently practiced in 1st professional eaxamination is developed with % weighting is calculated. (See Table 3)
    • Table 3: Summary of weighting of each package and their respective modules calculated for number of weeks of study (content area) per module versus weighting currently practiced in 1st Professional Exmination.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Structural Components of Blueprint: Table 2: Topics with objectives and learning outcome: •  It can be ensured that the weighting of assessment in term of content area calculated from number of weeks allocated to deliver that module is accordingly represented in assessment. •  An obvious impression from eyeball rolling over this blueprint clearly shows that the content is not evenly distributed. Some of the modules are not even given a place in weighting whilw other are overrepresented in assessment of professional examoination.    
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Structural Components of Blueprint:   Table 3: Number and Weighting of Questions in Assessment: •  After knowing the learning objectives of each contentent areas as per their modules in preclinical phase of training (see table 2) and the type of measurement tools in 1st professional examination (see table 1) it is easy to finally create the examination blueprint.       (See Table 4-6)
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Table 4: Template of examination blueprint showing number of questions (%) determined by content area (weeks of study) as defined for each packages/modules and their learning domain (cognitive dimension) determined by their respective lerning objectives in 60 questions MCQ/SBAQ (30/30) of 5 marks each from Paper I/II in 1st professional examination of MD course. *   Indicates recommended percent wise representation of questions with respect to their level of competence (cognitive dimension) in different modules however, exact number may not follow this strickly and can be determined in vetting
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Table 5: Template of examination blueprint showing number of questions (%) determined by content area (weeks of study) as defined for each packages/modules and their learning domain (cognitive dimension) determined by their lerning objectives in 12 questions SEQ/MEQ (8/4) of 15 marks each from Paper I/II in 1st professional examination. Of MD course *   Indicates recommended percent wise representation of questions with respect to their level of competence (cognitive dimension) in different modules however, exact number may not follow this strickly and can be determined in vetting
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Table 6: Template of examination blueprint representing the number of station demarcated into packages and modules with skills domains (level of competence) determined by their lerning objectives in 15 stations OSPE and 5 stations OSCE of 5 marks each in 1st professional examination of MD course. *   Indicates recommended percent wise representation of questions with respect to their level of competence (cognitive dimension) in different modules however, exact number may not follow this strickly and can be determined in vetting
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Structural Components of Blueprint:   Table 3: Number and Weighting of Questions in Assessment: •  Examination blueprint is tabulated as chart representing the total number of questions within each module and their distribution as per level of competence (cognitive dimension) in term of knowledge and skills domains respectively. •  This will ensure the desired coverage of topics in term of modules and their relative importance per their lerning objectives and content area in the curriculum.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Rationale of Blueprints: 1. Assessment is a complex and multifaceted aspect of learning and teaching, which may be viewed differently by different users. However, a valid assessment determins the acquired competencies identified as learning outcome in curriculum. 2. The document that provides an overview to assessment and can be readily used on stakeholders (particularly the external examiners, MQA, MMC and Senate) demand of quality assessment evident from its blueprint.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Rationale of Blueprints: 3. Another important aspect of assessment, the feedback on students’ progress and teachers’ delivering of curriculum is less often practiced simply for not well understood assessment process by faculty. •  A well design assessment documented as blueprint provides an opportunity and confidence to faculty to give feedback with realtive importance to measures and their weighting.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Rationale of Blueprints: 4. From student’s point of view assessment always define curriculum. •  What is to be learnt (content area in assessment) and how to be learnt (superficial to deep learning approaches) will depend how assessment is designed and how it is being made available (blueprint) to students and assessors alike.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Rationale of Blueprints: 5. We all agree that most students demonstrate a preference for learning approach. •      In this perspective teaching and learning context determined as a blueprint of examination question in assessment can influence the students’ actual approach in practice.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Rationale of Blueprints: 6. We in FMHS would like to develop a blueprint of examination question not because it is demanded by the external examiners but to guide our students to adopt a learning approach, moving from superficial to deeper level of learning domain. •  This will enable students’ in handling the learning task that our assessment approach not only tends to present but now it is also provded as a blueprint of examination questions.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Rationale of Blueprints:     7. Besides the external pressure to develop blueprint of exam questions, assessment imperatives such as validity, reliability and managebility takes the cetral position in idea behind bluepriting. •  Bloom’s taxonomy of knowledge as test of cognitive dimension is primarily utilized to develop the entire written and some aspect of practical questions. •  Miller’s pyramid is used for judgment of practical and clinical skills of competency to test students’ affective and psychomotor abilities ensured in blueprint.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Systemic Reporting of Questions for Blueprint: •  Further to blueprint a guideline for faculty member to categorize each question into relevant columns of cognitive dimension that the question intend to test will be requested. •  It will be mandatory for each question setter to fill up the proposed table while writing and submitting the examination questions. (See Table 7)
    • Table 7: Guidelines for questions writer to categorize each question to a relevant column og cognitive dimension that the question is set to test.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM Systemic Reporting of Questions for Blueprint: •  All the questions categorized to one or more level of competence refering to cognitive dimension will also be vetted by the vetting committee in their routine meetings. •  This practice will ensure that the examination questions are consistent with target set for weghting of question in blueprint. •  This will also help the academic office to prepare the blueprint of examination questions well on time for ready referal of faculty and external examiners.
    • Blueprint of Examination Questions 1st Professional Examination of MD Program in FMHS at UPM THANK YOU