The document discusses using item analysis to improve the reliability of an internal medicine undergraduate OSCE (objective structured clinical examination). The researchers performed a generalizability study and item analysis on OSCE scores from 8 cohorts of medical students totaling 435 students. They found the median reliability (G-coefficient) was 0.62. Item analysis identified a median of 3.5 "problem stations" per OSCE that negatively impacted reliability. When these stations were removed, the median G-coefficient increased to 0.72, demonstrating item analysis can help identify and remove poor quality stations to improve an OSCE's reliability.
This document summarizes a study that assessed pharmacy students' perceptions of a redesigned Objective Structured Clinical Examination (OSCE) used to evaluate core competency domains outlined by accreditation standards. The exam was administered to second and third year pharmacy students. A survey found most students had positive perceptions but second years were less positive. Many third years did not feel the exam prepared them for advanced pharmacy practice experiences, suggesting curriculum gaps. The OSCE assessed competencies across 11 domains with most stations covering over 75% of domains. Student feedback will be used to improve domain coverage and exam structure in the future.
This document provides information about conducting and appraising a meta-analysis on the use of prophylactic antibiotics for pancreatic necrosis. It outlines the steps of formulating the clinical question using PICO, acquiring relevant studies through database searches and hand searches, appraising study quality, collecting and recording study data, analyzing results using both individual and pooled treatment effects, and reporting findings in a forest plot. Key aspects of meta-analysis methodology are discussed including biases that can affect results.
This document contains a series of exercises on study design for epidemiological research. It provides examples of different types of studies including cohort studies, case-control studies, cross-sectional studies, and longitudinal studies. For each proposed research question, it identifies the most appropriate study design and describes how that design could be implemented to study the relationship of interest. The document also tests understanding of key characteristics and methodologies for different epidemiological study designs.
This document provides an overview of critical appraisal of randomized controlled trials (RCTs). It defines critical appraisal as carefully examining research to assess its trustworthiness and relevance. RCTs are described as the gold standard for clinical trials, where participants are randomly allocated to groups that receive either a treatment or a control. Key factors to examine in appraising an RCT are described, including sample size, eligibility criteria, baseline characteristics, randomization, blinding, follow-up of participants, data collection, presentation of results, and applicability to local populations. Advantages of critical appraisal and RCTs include providing a systematic way to assess research validity and improving practice, while disadvantages include taking time and not always finding clear answers.
The article summarizes a presentation reviewing the top medical education articles from 2016. It discusses how OSCE design factors like the number of competencies raters assess and station order can impact reliability. It also explores how narrative comments from standardized patients can identify problematic student behaviors not captured by checklists. A non-binary checklist was found to provide flexibility in scoring OSCE progress tests and allow feedback on partially completed tasks. The presentation aims to help educators incorporate innovations from the literature into their practice to improve assessment and training.
This document discusses the objective structured clinical examination (OSCE) as a tool for assessing clinical competence. It defines the OSCE as a multidimensional practical exam of clinical skills. The purposes of the OSCE are to identify objective performance criteria for clinical skills, structure these criteria in checklists, and meet accreditation requirements. Examples of skills assessed include history taking, physical exams, clinical problem solving, and procedures. The document also outlines how to organize an OSCE and provides examples of exam stations. Both advantages like reliability and ability to test a wide range of skills as well as disadvantages like time demands are discussed.
This document summarizes a study that assessed pharmacy students' perceptions of a redesigned Objective Structured Clinical Examination (OSCE) used to evaluate core competency domains outlined by accreditation standards. The exam was administered to second and third year pharmacy students. A survey found most students had positive perceptions but second years were less positive. Many third years did not feel the exam prepared them for advanced pharmacy practice experiences, suggesting curriculum gaps. The OSCE assessed competencies across 11 domains with most stations covering over 75% of domains. Student feedback will be used to improve domain coverage and exam structure in the future.
This document provides information about conducting and appraising a meta-analysis on the use of prophylactic antibiotics for pancreatic necrosis. It outlines the steps of formulating the clinical question using PICO, acquiring relevant studies through database searches and hand searches, appraising study quality, collecting and recording study data, analyzing results using both individual and pooled treatment effects, and reporting findings in a forest plot. Key aspects of meta-analysis methodology are discussed including biases that can affect results.
This document contains a series of exercises on study design for epidemiological research. It provides examples of different types of studies including cohort studies, case-control studies, cross-sectional studies, and longitudinal studies. For each proposed research question, it identifies the most appropriate study design and describes how that design could be implemented to study the relationship of interest. The document also tests understanding of key characteristics and methodologies for different epidemiological study designs.
This document provides an overview of critical appraisal of randomized controlled trials (RCTs). It defines critical appraisal as carefully examining research to assess its trustworthiness and relevance. RCTs are described as the gold standard for clinical trials, where participants are randomly allocated to groups that receive either a treatment or a control. Key factors to examine in appraising an RCT are described, including sample size, eligibility criteria, baseline characteristics, randomization, blinding, follow-up of participants, data collection, presentation of results, and applicability to local populations. Advantages of critical appraisal and RCTs include providing a systematic way to assess research validity and improving practice, while disadvantages include taking time and not always finding clear answers.
The article summarizes a presentation reviewing the top medical education articles from 2016. It discusses how OSCE design factors like the number of competencies raters assess and station order can impact reliability. It also explores how narrative comments from standardized patients can identify problematic student behaviors not captured by checklists. A non-binary checklist was found to provide flexibility in scoring OSCE progress tests and allow feedback on partially completed tasks. The presentation aims to help educators incorporate innovations from the literature into their practice to improve assessment and training.
This document discusses the objective structured clinical examination (OSCE) as a tool for assessing clinical competence. It defines the OSCE as a multidimensional practical exam of clinical skills. The purposes of the OSCE are to identify objective performance criteria for clinical skills, structure these criteria in checklists, and meet accreditation requirements. Examples of skills assessed include history taking, physical exams, clinical problem solving, and procedures. The document also outlines how to organize an OSCE and provides examples of exam stations. Both advantages like reliability and ability to test a wide range of skills as well as disadvantages like time demands are discussed.
This document provides an overview of evidence-based medicine and how to critically appraise clinical papers. It discusses how evidence-based medicine involves using both clinical expertise and the best available external evidence in decision making. The origins of evidence-based medicine in the 1970s and 1990s are also reviewed. The document then focuses on how to critically read clinical papers, including the key things to assess for diagnostic tests, clinical course/prognosis, causation, and therapy papers. It provides guidance on an appraisal format and emphasizes the need to both evaluate the study and summarize what it was about. Evidence-based medicine is positioned as an important guide but not a replacement for clinical expertise and judgment.
This document summarizes current recommendations and gaps regarding extrapolation of time-to-event outcomes from clinical trials. It reviewed 11 methodological papers and 5 guidelines on extrapolating survival data. The guidelines, particularly from NICE, provide a detailed process for extrapolation including testing different survival models, validating the best fitting model, and using external data for validation. However, the guidelines need updating to apply to more disease areas beyond oncology and different time-to-event outcomes.
This systematic review aimed to evaluate the efficacy of barrier membranes for vertical and horizontal bone regeneration compared to no membrane controls. The review included human and animal studies evaluating various membrane types. Meta-analyses found a statistically significant benefit of membranes for vertical bone gain in animal studies, but human studies lacked quantitative data. The review concluded that while membranes may aid vertical bone regeneration, more high-quality human RCTs are needed due to heterogeneity between studies.
This document provides an introduction to evidence-based medicine (EBM) and principles of critical appraisal for first year psychiatry residents. It defines EBM as a systematic approach to obtaining clinically relevant information about treatment. Critical appraisal involves carefully examining research to assess validity, results, and relevance. The document outlines the five steps of EBM, different study designs commonly used in EBM like randomized controlled trials and systematic reviews, and factors to consider when critically appraising research like potential biases. Key aspects of randomized trials and systematic reviews are described, along with guidelines to standardize research reporting.
1) The document discusses a lecture on evidence-based medicine (EBM) and critical appraisal.
2) EBM involves integrating the best available research evidence with clinical expertise and patient values. It includes formulating clinical questions, searching for evidence, appraising research, and applying the evidence to patient care.
3) The lecture reviews the principles of EBM and critical appraisal, including how to formulate answerable clinical questions using the PICO framework, search for evidence, and appraise different types of research studies.
This document discusses randomised controlled trials (RCTs) and their use in evaluating health care interventions. It provides background on James Lind, who in 1747 conducted one of the earliest known clinical trials to test treatments for scurvy. The document outlines key aspects of RCTs such as defining the research question, population, interventions, and outcomes, as well as the importance of randomization and blinding to reduce bias. It distinguishes between explanatory and pragmatic trials and notes RCTs are generally considered the gold standard for evaluating health care technologies.
Efficacy of Information interventions in reducing transfer anxiety from a cri...Ambika Rai
Efficacy of Information interventions in reducing transfer anxiety from a critical care setting to a general ward: A systematic review and a meta-analysis
Evidence-based Medicine (EBM) Applicationsuzairktk
In 1996 David Sackett wrote that "Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”
This document provides an agenda and background materials for a workshop on observational studies in a learning health system. The workshop will explore the role of observational studies in generating evidence to guide clinical decisions and health policy, considering issues such as bias, generalizing randomized controlled trial results, detecting treatment heterogeneity, and predicting individual responses. Over two days, the workshop will include presentations and panel discussions on these topics from experts in various disciplines. The goal is to identify strategies to advance the appropriate use of observational studies for evidence generation in a learning health system.
This document appears to be a methodology checklist for assessing controlled trials. It includes sections for evaluating the internal validity of a study, the overall assessment of the study, and a description of study details. The checklist addresses factors like randomization, blinding, similarity of groups, outcome measurement, analysis of results, applicability to relevant patient populations, and assessment of bias. It is intended to help appraise trials and determine the strength and validity of evidence presented.
This document appears to be a methodology checklist for assessing controlled trials. It includes sections for evaluating the internal validity of a study, the overall assessment of the study, and a description of study details. The checklist addresses factors like randomization, blinding, similarity of groups, outcome measurement, analysis of results, applicability to relevant patient populations, and assessment of bias. It is intended to help appraise trials and determine the strength and validity of evidence presented.
Meta-analysis in Epidemiology is:
Useful tool for epidemiological studies which investigates the relationships between certain risk factors and disease.
Useful tool to improve animal well-being and productivity
Despite of a wealth of suitable studies it is relatively underutilized in animal and veterinary science.
Meta-analysis can provide reliable results about diseases occurrence, pattern and impact in livestock.
It is utmost essential to take benefit of this statistical tool for produce. more reliable estimates of concern effects in animal and veterinary science data.
This systematic review analyzed 11 studies with 1,194 participants to understand professionals' views of fetal monitoring during labor. Four key themes emerged: 1) Reassurance - professionals viewed monitoring as reassuring patients and improving outcomes. 2) Technology - monitoring was seen as both empowering through knowledge but also limiting mobility and natural birth. 3) Communication/Education - monitoring impacted communication and some felt more education was needed. 4) Midwife by proxy - some saw monitoring as a way to practice without midwifery skills or undermining their roles. The review provided insight but also identified continued use of electronic fetal monitoring contrary to evidence and called for further research.
VAP is one of the most significant causes of morbidity and mortality in mechanically ventilated critically ill patients. Lack of knowledge on VAP is highly influenced on incidence of VAP. Objective: The objective of the study was to assess the level of knowledge regarding VAP among the staff nurses working in the intensive care unit. Methods: Non-experimental cross sectional research design was used with 50 samples at selected hospital. The tool used for the study to collect the data was demographic variables and structured multiple choice questionnaires regarding VAP. Data was collected by self-administered method. Results: Out of 50 samples, 38 (76%) had adequate knowledge, seven (14%) had moderately adequate knowledge and five (10%) had inadequate knowledge and there was no significant association with selected demographic variables. Conclusion: The findings of the present study concluded that there was no cent percent adequate knowledge on VAP. Organize regular and periodical continuing nursing education or on the job training on ventilator associated pneumonia for Staff nurses working in the intensive care unit to impart knowledge.
Critical appraisal is the process of carefully and systematically analyze the research paper to judge its trustworthiness, its value and relevance in a particular context. (Amanda Burls 2009)
A critical review must identify the strengths and limitations in a research paper and this should be carried out in a systematic manner.
The Critical Appraisal helps in developing the necessary skills to make sense of scientific evidence, based on validity, results and relevance.
Levels of evidence, recommendations & phases ofsanyal1981
This document discusses levels of evidence and phases of clinical trials. It defines evidence-based medicine as using current best evidence from systematic research to make decisions about patient care. Levels of evidence are ranked from 1A to 5 based on study design, with systematic reviews and randomized controlled trials ranked highest. Clinical trials progress through four phases to test safety, efficacy, and optimal use of new drugs or devices. Phase 1 assesses safety, phase 2 establishes efficacy, phase 3 confirms safety and efficacy in larger groups, and phase 4 studies monitor risks and benefits after marketing.
Advantages and Disadvantages of the Objective Structured Clinical Examination...ijtsrd
This document summarizes a literature review on the advantages and disadvantages of using Objective Structured Clinical Examinations (OSCEs) in nursing education. The review found that OSCEs are an effective way to evaluate students' clinical skills and have grown in use over the last decade. However, OSCEs also have several disadvantages, such as being resource-intensive, expensive, and difficult to implement. While OSCEs provide benefits like objectivity and preparing students for practice, there are obstacles that limit their applicability in some contexts. Overall, the literature indicates that OSCEs are useful for nursing education when implemented appropriately, but other assessment methods may need to be used in combination due to the challenges of OSCEs
1) OSCE was introduced in 1975 as a standardized tool to objectively assess clinical competencies such as history taking, physical examination, communication skills, and data interpretation. It consists of stations that evaluate specific competencies using checklists.
2) The article discusses issues with OSCE including concerns about validity, objectivity, reliability, and lack of clear passing standards. It also notes that Indian experiences with OSCE are limited.
3) The authors argue for a cautious approach when considering OSCE as a supplementary assessment tool in India, given the need to address its limitations and sensitize faculty and students. Standard setting and determining minimum passing scores are highlighted as particular challenges.
This document provides an overview of evidence-based medicine and how to critically appraise clinical papers. It discusses how evidence-based medicine involves using both clinical expertise and the best available external evidence in decision making. The origins of evidence-based medicine in the 1970s and 1990s are also reviewed. The document then focuses on how to critically read clinical papers, including the key things to assess for diagnostic tests, clinical course/prognosis, causation, and therapy papers. It provides guidance on an appraisal format and emphasizes the need to both evaluate the study and summarize what it was about. Evidence-based medicine is positioned as an important guide but not a replacement for clinical expertise and judgment.
This document summarizes current recommendations and gaps regarding extrapolation of time-to-event outcomes from clinical trials. It reviewed 11 methodological papers and 5 guidelines on extrapolating survival data. The guidelines, particularly from NICE, provide a detailed process for extrapolation including testing different survival models, validating the best fitting model, and using external data for validation. However, the guidelines need updating to apply to more disease areas beyond oncology and different time-to-event outcomes.
This systematic review aimed to evaluate the efficacy of barrier membranes for vertical and horizontal bone regeneration compared to no membrane controls. The review included human and animal studies evaluating various membrane types. Meta-analyses found a statistically significant benefit of membranes for vertical bone gain in animal studies, but human studies lacked quantitative data. The review concluded that while membranes may aid vertical bone regeneration, more high-quality human RCTs are needed due to heterogeneity between studies.
This document provides an introduction to evidence-based medicine (EBM) and principles of critical appraisal for first year psychiatry residents. It defines EBM as a systematic approach to obtaining clinically relevant information about treatment. Critical appraisal involves carefully examining research to assess validity, results, and relevance. The document outlines the five steps of EBM, different study designs commonly used in EBM like randomized controlled trials and systematic reviews, and factors to consider when critically appraising research like potential biases. Key aspects of randomized trials and systematic reviews are described, along with guidelines to standardize research reporting.
1) The document discusses a lecture on evidence-based medicine (EBM) and critical appraisal.
2) EBM involves integrating the best available research evidence with clinical expertise and patient values. It includes formulating clinical questions, searching for evidence, appraising research, and applying the evidence to patient care.
3) The lecture reviews the principles of EBM and critical appraisal, including how to formulate answerable clinical questions using the PICO framework, search for evidence, and appraise different types of research studies.
This document discusses randomised controlled trials (RCTs) and their use in evaluating health care interventions. It provides background on James Lind, who in 1747 conducted one of the earliest known clinical trials to test treatments for scurvy. The document outlines key aspects of RCTs such as defining the research question, population, interventions, and outcomes, as well as the importance of randomization and blinding to reduce bias. It distinguishes between explanatory and pragmatic trials and notes RCTs are generally considered the gold standard for evaluating health care technologies.
Efficacy of Information interventions in reducing transfer anxiety from a cri...Ambika Rai
Efficacy of Information interventions in reducing transfer anxiety from a critical care setting to a general ward: A systematic review and a meta-analysis
Evidence-based Medicine (EBM) Applicationsuzairktk
In 1996 David Sackett wrote that "Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”
This document provides an agenda and background materials for a workshop on observational studies in a learning health system. The workshop will explore the role of observational studies in generating evidence to guide clinical decisions and health policy, considering issues such as bias, generalizing randomized controlled trial results, detecting treatment heterogeneity, and predicting individual responses. Over two days, the workshop will include presentations and panel discussions on these topics from experts in various disciplines. The goal is to identify strategies to advance the appropriate use of observational studies for evidence generation in a learning health system.
This document appears to be a methodology checklist for assessing controlled trials. It includes sections for evaluating the internal validity of a study, the overall assessment of the study, and a description of study details. The checklist addresses factors like randomization, blinding, similarity of groups, outcome measurement, analysis of results, applicability to relevant patient populations, and assessment of bias. It is intended to help appraise trials and determine the strength and validity of evidence presented.
This document appears to be a methodology checklist for assessing controlled trials. It includes sections for evaluating the internal validity of a study, the overall assessment of the study, and a description of study details. The checklist addresses factors like randomization, blinding, similarity of groups, outcome measurement, analysis of results, applicability to relevant patient populations, and assessment of bias. It is intended to help appraise trials and determine the strength and validity of evidence presented.
Meta-analysis in Epidemiology is:
Useful tool for epidemiological studies which investigates the relationships between certain risk factors and disease.
Useful tool to improve animal well-being and productivity
Despite of a wealth of suitable studies it is relatively underutilized in animal and veterinary science.
Meta-analysis can provide reliable results about diseases occurrence, pattern and impact in livestock.
It is utmost essential to take benefit of this statistical tool for produce. more reliable estimates of concern effects in animal and veterinary science data.
This systematic review analyzed 11 studies with 1,194 participants to understand professionals' views of fetal monitoring during labor. Four key themes emerged: 1) Reassurance - professionals viewed monitoring as reassuring patients and improving outcomes. 2) Technology - monitoring was seen as both empowering through knowledge but also limiting mobility and natural birth. 3) Communication/Education - monitoring impacted communication and some felt more education was needed. 4) Midwife by proxy - some saw monitoring as a way to practice without midwifery skills or undermining their roles. The review provided insight but also identified continued use of electronic fetal monitoring contrary to evidence and called for further research.
VAP is one of the most significant causes of morbidity and mortality in mechanically ventilated critically ill patients. Lack of knowledge on VAP is highly influenced on incidence of VAP. Objective: The objective of the study was to assess the level of knowledge regarding VAP among the staff nurses working in the intensive care unit. Methods: Non-experimental cross sectional research design was used with 50 samples at selected hospital. The tool used for the study to collect the data was demographic variables and structured multiple choice questionnaires regarding VAP. Data was collected by self-administered method. Results: Out of 50 samples, 38 (76%) had adequate knowledge, seven (14%) had moderately adequate knowledge and five (10%) had inadequate knowledge and there was no significant association with selected demographic variables. Conclusion: The findings of the present study concluded that there was no cent percent adequate knowledge on VAP. Organize regular and periodical continuing nursing education or on the job training on ventilator associated pneumonia for Staff nurses working in the intensive care unit to impart knowledge.
Critical appraisal is the process of carefully and systematically analyze the research paper to judge its trustworthiness, its value and relevance in a particular context. (Amanda Burls 2009)
A critical review must identify the strengths and limitations in a research paper and this should be carried out in a systematic manner.
The Critical Appraisal helps in developing the necessary skills to make sense of scientific evidence, based on validity, results and relevance.
Levels of evidence, recommendations & phases ofsanyal1981
This document discusses levels of evidence and phases of clinical trials. It defines evidence-based medicine as using current best evidence from systematic research to make decisions about patient care. Levels of evidence are ranked from 1A to 5 based on study design, with systematic reviews and randomized controlled trials ranked highest. Clinical trials progress through four phases to test safety, efficacy, and optimal use of new drugs or devices. Phase 1 assesses safety, phase 2 establishes efficacy, phase 3 confirms safety and efficacy in larger groups, and phase 4 studies monitor risks and benefits after marketing.
Advantages and Disadvantages of the Objective Structured Clinical Examination...ijtsrd
This document summarizes a literature review on the advantages and disadvantages of using Objective Structured Clinical Examinations (OSCEs) in nursing education. The review found that OSCEs are an effective way to evaluate students' clinical skills and have grown in use over the last decade. However, OSCEs also have several disadvantages, such as being resource-intensive, expensive, and difficult to implement. While OSCEs provide benefits like objectivity and preparing students for practice, there are obstacles that limit their applicability in some contexts. Overall, the literature indicates that OSCEs are useful for nursing education when implemented appropriately, but other assessment methods may need to be used in combination due to the challenges of OSCEs
1) OSCE was introduced in 1975 as a standardized tool to objectively assess clinical competencies such as history taking, physical examination, communication skills, and data interpretation. It consists of stations that evaluate specific competencies using checklists.
2) The article discusses issues with OSCE including concerns about validity, objectivity, reliability, and lack of clear passing standards. It also notes that Indian experiences with OSCE are limited.
3) The authors argue for a cautious approach when considering OSCE as a supplementary assessment tool in India, given the need to address its limitations and sensitize faculty and students. Standard setting and determining minimum passing scores are highlighted as particular challenges.
An Objective Structured Clinical Examination (OSCE) is a standardized way to assess clinical skills and competence through a circuit of short stations involving simulated patients. Candidates are observed and evaluated as they perform tasks like history taking, physical exams, and treatments. OSCEs aim to be objective by using standardized patients, questions, marking schemes, and timings at each station to reduce variability between candidates. They provide a comprehensive evaluation of a wide range of clinical skills across different specialties through an integrated assessment of both skills and theoretical knowledge.
Introduction of Objective Structured Clinical Examination as assessment tool ...iosrjce
This document describes a study that introduced Objective Structured Clinical Examinations (OSCEs) as an assessment tool in formative examinations for the Dermatology, Venereology and Leprology department at a medical college in India. The study aimed to assess the feasibility and acceptability of OSCEs by students and faculty. Students and faculty were oriented to OSCEs and then 15 stations were used to assess students' clinical skills over 15 days. Feedback found that over 90% of students and faculty found OSCEs acceptable, feasible, improved clinical skills, and were better than conventional assessment methods. The study concluded that introducing OSCEs increased reliable assessment and student confidence in clinical skills.
final project (nursing major) najah universitymahdyvika
The knowledge of nursing toward the role of them in End of life care in Intensive care units and oncology units in Nablus hospitals- Cross sectional study.
OSCE as a Summative Assessment Tool for Undergraduate Students of Surgery—Our...KETAN VAGHOLKAR
The letter discusses the use of Objective Structured Clinical Examinations (OSCE) for assessing undergraduate medical students in India. It notes that OSCEs are well-suited for use in private medical colleges that have limited patient numbers, as they avoid issues with traditional long and short case examinations. However, the letter also points out that conducting high-quality OSCEs requires adequate faculty and training, as multiple examiners are typically needed at each station. The author advocates for the Medical Council of India to develop standardized practices for implementing OSCEs nationally to improve consistency in medical education assessments.
Research critique for published journal.kindly support and do correction if so.
Writing a Critique
describe: give the reader a sense of the writer's overall purpose and intent.
analyze: examine how the structure and language of the text convey its meaning.
interpret: state the significance or importance of each part of the text.
assess: make a judgment of the work's worth or value.
This document describes a study that introduced an OSCE (Objective Structured Clinical Examination) to assess clinical skills of medical students at a traditional Brazilian medical school. Some key findings:
- Students were generally comfortable with the exam structure but nearly half criticized organization aspects in the first year. Most also reported difficulties with time management and stress. Improvements reduced criticism but time and stress issues remained.
- Faculty acknowledged OSCE accuracy but felt it did not assess integrated patient care and was very time-consuming to organize. The educational impact was limited since other faculty did not engage with the results.
- While the OSCE revealed student performance and teaching effectiveness, cultural aspects and lack of support precluded its permanent
Critical appraisal.docx IMPORTAN TO HEALTH SCIENCE STUDENTSMulugetaAbeneh1
Critical appraisal is the process of systematically examining evidence to assess its validity, results, and relevance before using it to inform decisions. This document discusses tools for critically appraising different types of studies, including systematic reviews, guidelines, and primary studies. It provides examples of appraising systematic reviews using the AMSTAR tool and appraising randomized controlled trials using the JBI critical appraisal checklist. The document concludes that plastic wraps effectively prevent hypothermia in preterm and low birth weight infants compared to standard care, as shown in multiple systematic reviews and randomized trials.
Appraisal of weekly conducted clinical pathologic case conference (CPC).pdfLeslie Schulte
- The document discusses a survey evaluating a weekly clinical pathologic case conference (CPC) conducted at the School of Medical Sciences, Universiti Sains Malaysia.
- The survey found that the majority of faculty were unaware of the original/formal CPC format and that many presentations deviated from this format by lacking differential diagnosis discussion.
- However, most faculty expressed willingness to follow guidelines to adapt presentations to the formal CPC format and recognized CPC as an important continuing medical education opportunity.
The document discusses the Objective Structured Clinical Examination (OSCE), which is used to assess clinical skills in health science students. An OSCE involves students rotating through multiple stations that simulate real-world clinical scenarios, where they demonstrate skills like communication, physical exams, and procedures. Examiners use checklists to score students uniformly on competencies. The OSCE aims to make clinical assessments more objective, structured, and practical compared to traditional exams.
presentation on objective structured clinical examinationManimozhi R
The document discusses the Objective Structured Clinical Examination (OSCE) used to assess clinical skills in health sciences. It provides that OSCEs use standardized patients and checklists to evaluate trainees in a structured manner across stations testing various skills. The document outlines the history, definition, purposes, uses, components, organization, advantages and disadvantages of OSCEs. It notes OSCEs aim to objectively assess clinical competence in a valid and reliable way compared to traditional exams.
- Medical students receive little training in otoscopy, averaging only 57 hours. General practitioners also demonstrate limited skills.
- A new web-based otoscopy simulator called OtoTrain was developed and its face validity, content validity, and usefulness for medical education were evaluated through an expert survey.
- The survey found that OtoTrain demonstrated good face validity and content validity for teaching otoscopy skills. Experts also indicated it was generally superior to traditional teaching methods like textbooks and lectures.
This study assessed the effectiveness of a breaking bad news training program for primary care physicians through a pre-post objective structured clinical examination (OSCE). The OSCE consisted of 8 simulated patient encounters requiring physicians to break bad news. Standardized patients rated physicians' communication skills. 17 physicians received the training (intervention group) while 17 others participated in a control group. Results showed the intervention group significantly improved their average OSCE score from pre- to post-test, while the control group showed minimal improvement. This provides evidence the training program improved physicians' ability to break bad news.
This study assessed the effectiveness of a breaking bad news training program for primary care physicians through a pre-post objective structured clinical examination (OSCE). The OSCE consisted of 8 simulated patient encounters requiring physicians to break bad news. Standardized patients evaluated physicians before and after the intervention or a control group using communication rating scales. Results showed the intervention group significantly improved their average post-test grade compared to pre-test, while the control group showed minimal improvement. The OSCE proved to be a reliable tool for assessing physicians' bad news communication skills, and provided evidence the intervention was effective.
1. A study conducted at the School of Medicine, Universitas Muhammadiyah Yogyakarta in Indonesia analyzed whether OSCE mentorship could effectively help students pass the OSCE UKMPPD national examination.
2. The study involved 153 participants who received OSCE mentorship including expert lectures, independent learning, simulated OSCE practice, and motivational sessions.
3. The results showed that the pass rate for students who received OSCE mentorship was significantly higher at 93% compared to 54% for students who did not receive mentorship, demonstrating that OSCE mentorship can effectively improve students' ability to pass the OSCE UKMPPD exam.
Attitudes toward interprofessional education and collaborative practices in M...SubmissionResearchpa
From the perspective of activity theory, it can be argued that the major challenge in relation to implementing interprofessional education (IPE) could be embraced as contradictions that may lead to change. Patients have complex health needs and typically require insight from more than one discipline to address issues regarding their health status (Lumague et al. 2006). The World Health Organization (WHO) recommends that institutions engaged in health professional education and training consider implementing IPE in both undergraduate and postgraduate programs (WHO, 2010). The purpose of this study was to identify the needs of IPECP for health care professionals, faculty members, and students. Methods: The survey instrument contained four scales to evaluate faculty attitudes toward IPE and teamwork, adapted from the methods of Curran et al. (2007). Each scale asked respondents to rate their attitudes toward statements on a 5-point Likert scale (1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree). The initial factor extractions were performed by means of principal components analysis. To define the model structure more clearly, an exploratory factor analysis using varimax rotation was conducted. The level of significance was p<.0001 for all tests. Results: As shown in Table 2, the Kaiser–Meyer-Olkin index was 0.902, indicating sampling adequacy, and the Bartlett Sphericity Chi-Square index was 2246.5 (p <0.0001). Cronbach’s alpha for the 14 items was 0.731, revealing a high rate of internal consistency. The modified Attitude toward health care team scores (ATHCTS) questionnaire was categorized into four factors: “Quality of care,” “Team efficiency,” “Patient-centered care,” and “Negative factors.” Conclusion: Findings suggest that the positive attitude of health care professionals, faculty members and students towards IPE indicates the need for IPE training by Azjargal Baatar, Sumberzul Nyamjav and Oyuntsetseg Sandag 2020. Attitudes toward interprofessional education and collaborative practices in Mongolia. International Journal on Integrated Education. 3, 8 (Aug. 2020), 48-52. DOI:https://doi.org/10.31149/ijie.v3i8.536 https://journals.researchparks.org/index.php/IJIE/article/view/536/512 https://journals.researchparks.org/index.php/IJIE/article/view/536
Similar to Osce item-analysis-to-improve-reliability-for-internal-medicine-auewarakul-downing-praditswuan-jaturatamrong (20)
Paradigmas e tendencias do ensino universitario Mendonca/Lelis/Cotta/CarvalhoJr.PROIDDBahiana
O documento discute os paradigmas e tendências do ensino universitário atual, apontando estratégias de formação docente. Utilizou pesquisa-ação para avaliar capacitações docentes para tutoria em disciplina semipresencial. Os resultados apontaram insatisfação com o modelo tradicional de ensino e importância de professores estarem abertos à inovação, com processos inovadores de ensino sendo estratégia importante para mudança de paradigma educacional.
O documento fornece instruções sobre o uso do portal do professor, incluindo como observar o status do plano de ensino, preencher e atualizar o plano de ensino, importar planos de ensino e aulas, gerenciar frequências e notas dos alunos, e enviar recados.
The document provides an overview of the Bahiana School of Medicine and Public Health (EBMSP), including its founding in 1952, current academic units and courses, leadership structure, and student support services like medical clinics and psychology services. It introduces the mission, vision, and values of EBMSP and discusses strategies around organizational competencies, benefits, and occupational health and safety programs.
O documento fornece um resumo da Bahiana, uma instituição de ensino fundada em 1952. A Bahiana atualmente possui três unidades acadêmicas e aproximadamente 1.300 colaboradores. O documento também lista as lideranças e coordenadores da Bahiana, além de descrever brevemente os serviços e benefícios oferecidos aos funcionários.
Este documento apresenta um código de programação. Contém informações sobre a versão do código, que está na versão 00, indicando que é a primeira versão lançada.
O planejamento-como-necessidade-avaliacao-docentePROIDDBahiana
O documento discute a importância do planejamento docente no ensino superior, definindo-o como uma organização sistemática e sequenciada das ações necessárias para o processo de ensino-aprendizagem. O texto explica que o planejamento inclui planos de ensino, com detalhes sobre disciplinas, objetivos, conteúdos e metodologias, e planos de aula, elaborados pelo professor. Além disso, discute estratégias metodológicas ativas que podem ser usadas para engajar os alunos, como debates, projetos em grupo e
O documento discute os elementos essenciais de um plano de ensino, incluindo o plano de ensino em si, a ementa, competências, metodologia, avaliação da aprendizagem, recursos didáticos e referências bibliográficas. É mediado por Gidélia Alencar e Lígia Vilas Bôas.
O Núcleo de Supervisão Acadêmico-Pedagógica (NUSP) da Bahiana foi criado em 2013 para contribuir com as práticas de ensino e aprendizagem e o projeto político-pedagógico institucional. O conceito surgiu inicialmente em 2001 para apoiar docentes e alunos de Medicina, e foi expandido para outros cursos em 2009. Atualmente, o NUSP planeja e avalia processos de ensino/aprendizagem usando metodologias ativas e instrumentos de avaliação.
O documento discute a avaliação da aprendizagem, incluindo os propósitos da avaliação, o que deve ser avaliado, quem deve avaliar e como avaliar. Ele fornece exemplos de estratégias de avaliação do trabalho do professor e formas de apoiar a aprendizagem dos alunos.
Este documento fornece recomendações técnicas para a elaboração de questões de avaliação. Ele discute a importância da avaliação para o processo de ensino-aprendizagem e fornece diretrizes para a construção de questões objetivas e subjetivas, incluindo os tipos de questões, critérios de elaboração e organização.
Este documento fornece instruções sobre como realizar o cadastro e submissão de projetos de pesquisa envolvendo seres humanos na Plataforma Brasil. Explica os passos para criar uma conta, preencher as informações sobre o projeto como objetivos, metodologia e riscos, e anexar os documentos necessários para a avaliação do Comitê de Ética em Pesquisa.
O documento fornece instruções sobre como preencher o currículo Lattes. Explica que o Lattes é um sistema de informação curricular usado no Brasil e deve ser preenchido por pesquisadores, estudantes e outros atores do sistema de ciência e tecnologia. Detalha os passos para criar um currículo Lattes online, incluindo informações pessoais, formação, experiência profissional, produção acadêmica e orientações.
The document discusses the use of simulated patients and objective structured clinical examinations (OSCEs) in medical education. Some key points:
1) Simulated patients have been used in medical education for over 30 years to teach communication skills, clinical skills, monitor doctor performance, and for use in clinical examinations.
2) OSCEs were first described in 1979 as a timed examination where students interact with simulated patients to assess history taking, physical exams, counseling, and patient management. OSCEs have been shown to be reliable and valid forms of assessment.
3) The use of OSCEs is now widespread in undergraduate and postgraduate medical examinations in the UK and other countries. Several medical licensing bodies incorporate OS
Este documento discute as distorções comuns no processo tutorial da aprendizagem baseada em problemas e fornece sugestões para melhorar a qualidade das tutorias. Algumas das distorções comuns incluem a falta de análise adequada dos problemas, discussões que não integram aspectos biológicos, psicológicos e sociais, e questões de aprendizagem formuladas de forma inadequada. Os autores argumentam que uma melhor compreensão do modelo pedagógico pode ajudar tutores e estudantes a superar inseguranças e conduzir tutorias
El documento trata sobre V. de Gowin. En pocas palabras, el documento se refiere a una persona llamada V. de Gowin, pero no proporciona ningún otro detalle sobre quién es esta persona o sobre qué trata específicamente el documento.
Gender and Mental Health - Counselling and Family Therapy Applications and In...PsychoTech Services
A proprietary approach developed by bringing together the best of learning theories from Psychology, design principles from the world of visualization, and pedagogical methods from over a decade of training experience, that enables you to: Learn better, faster!
How to Setup Default Value for a Field in Odoo 17Celine George
In Odoo, we can set a default value for a field during the creation of a record for a model. We have many methods in odoo for setting a default value to the field.
🔥🔥🔥🔥🔥🔥🔥🔥🔥
إضغ بين إيديكم من أقوى الملازم التي صممتها
ملزمة تشريح الجهاز الهيكلي (نظري 3)
💀💀💀💀💀💀💀💀💀💀
تتميز هذهِ الملزمة بعِدة مُميزات :
1- مُترجمة ترجمة تُناسب جميع المستويات
2- تحتوي على 78 رسم توضيحي لكل كلمة موجودة بالملزمة (لكل كلمة !!!!)
#فهم_ماكو_درخ
3- دقة الكتابة والصور عالية جداً جداً جداً
4- هُنالك بعض المعلومات تم توضيحها بشكل تفصيلي جداً (تُعتبر لدى الطالب أو الطالبة بإنها معلومات مُبهمة ومع ذلك تم توضيح هذهِ المعلومات المُبهمة بشكل تفصيلي جداً
5- الملزمة تشرح نفسها ب نفسها بس تكلك تعال اقراني
6- تحتوي الملزمة في اول سلايد على خارطة تتضمن جميع تفرُعات معلومات الجهاز الهيكلي المذكورة في هذهِ الملزمة
واخيراً هذهِ الملزمة حلالٌ عليكم وإتمنى منكم إن تدعولي بالخير والصحة والعافية فقط
كل التوفيق زملائي وزميلاتي ، زميلكم محمد الذهبي 💊💊
🔥🔥🔥🔥🔥🔥🔥🔥🔥
A Free 200-Page eBook ~ Brain and Mind Exercise.pptxOH TEIK BIN
(A Free eBook comprising 3 Sets of Presentation of a selection of Puzzles, Brain Teasers and Thinking Problems to exercise both the mind and the Right and Left Brain. To help keep the mind and brain fit and healthy. Good for both the young and old alike.
Answers are given for all the puzzles and problems.)
With Metta,
Bro. Oh Teik Bin 🙏🤓🤔🥰
1. Advances in Health Sciences Education (2005) 10:105–113
DOI 10.1007/s10459-005-2315-3
Ó Springer 2005
Item Analysis to Improve Reliability
for an Internal Medicine Undergraduate OSCE
CHIRAYU AUEWARAKUL1,2,4,*, STEVEN M. DOWNING3,
RUNGNIRAND PRADITSUWAN1, and UAPONG JATURATAMRONG2
1
Department of Medicine, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok,
Thailand; 2Office of Medical Education, Faculty of Medicine Siriraj Hospital, Mahidol
University, Bangkok, Thailand; 3Department of Medical Education, University of Illinois at
Chicago, Chicago, USA; 4Director of Medical Education Research Unit, Office of Medical
Education, Faculty of Medicine Siriraj Hospital, 2 Prannok Road, Bangkoknoi, Bangkok, 10700,
Thailand (*Corresponding author: Phone: 662-419-7000 ext. 4448; Fax: 662-418-1602; E-mail:
chirayuaue@yahoo.com; sicaw@mahidol.ac.th)
Received 13 January 2004; accepted 16 February 2005
Abstract. Utilization of objective structured clinical examinations (OSCEs) for final assessment
of medical students in Internal Medicine requires a representative sample of OSCE stations. The
reliability and generalizability of OSCE scores provides validity evidence for OSCE scores and
supports its contribution to the final clinical grade of medical students. The objective of this
study was to perform item analysis using OSCE stations as the unit of analysis and evaluate the
extent to which OSCE score reliability can be improved using item analysis data. OSCE scores
from eight cohorts of fourth-year medical students (n = 435) in a 6-year undergraduate program
were analyzed. Generalizability (G) coefficients of OSCE scores were computed for each cohort.
Item analysis was performed by considering each OSCE station as an item and computing the
corrected item-total correlation. OSCE stations which negatively impacted the reliability were
deleted and the G-coefficient was recalculated. The G-coefficients of OSCE scores from the eight
cohorts ranged from 0.48 to 0.80 (median 0.62). The median number of OSCE stations that
negatively impacted the G-coefficient was 3.5 (out of a median of 25 total stations). When the
‘‘problem stations’’ were deleted, the median G-coefficient across eight cohorts increased to
0.62–0.72. In conclusion, item analysis of OSCE stations is useful and should be performed to
improve the reliability of total OSCE scores. Problem stations can then be identified and improved.
Key words: clinical competence, generalizability, item analysis, OSCE, performance assessment,
reliability, undergraduate medical education
Background
Assessment of the clinical competence of medical students should consist
of multiple testing methods (Wass et al., 2001). Traditionally, written tests
including short-essays and multiple-choice questions are the primary
methods used to assess the knowledge of medical students. Medical
2. 106
CHIRAYU AUEWARAKUL ET AL.
knowledge alone, however, does not always predict clinical competence of
the students (Miller, 1990). The Faculty of Medicine Siriraj Hospital at
the Mahidol University as the oldest medical school in Thailand has
traditionally relied on summative oral examinations in the form of short
cases and long cases at the end of the clinical rotations as a means to
assess clinical performance. This type of assessment, however, was shown
to be unreliable because the pass/fail decision depends on a single judgment of a student by one faculty member on one or two real patient cases
(van der Vleuten, 2000). The objective structured clinical examination
(OSCE) was developed in 1975, as a means of clinical competence
assessment (Harden et al., 1975), and this new performance method was
then introduced in Thailand around 1985. OSCE has gradually replaced or
been used in conjunction with the short- and long-case oral examinations
and presently contributes meaningfully to the final clinical grade of
medical students in several clinical departments at the Faculty of Medicine
Siriraj Hospital, especially in the Department of Medicine.
The undergraduate MD curriculum at the Faculty of Medicine Siriraj
Hospital is a 6-year program. The clinical years include year 4–6 where
students rotate through all major specialties and electives. Students are required to rotate through Internal Medicine every clinical year, starting from
year 4 to 6. At the end of the fourth-year Internal Medicine rotation, students
are required to take an OSCE as part of the final evaluation. The final grade
depends on a composite score, 25-percent of which is derived from the OSCE
scores. Other components include problem-solving and general medical
knowledge multiple-choice questions (MCQ) (25%) and ward, outpatient
clinic, and preceptor evaluation (50%). Since its introduction to the
Department of Medicine, validity evidence and reliability data have never
been evaluated for the OSCEs. Whether it tests what it purports to test is not
known. The reliability and generalizability of OSCE scores should provide
some validity evidence for OSCE scores and their contribution to the final
grade of fourth-year medical students (Crossley et al., 2002; Downing, 2003).
A representative sample of OSCE stations needs to be demonstrated in order
to justify the continued use of OSCE for final grade evaluation (Colliver and
Williams, 1993; van der Vleuten, 2000).
Objective
The purpose of this study was to evaluate the effect on reliability of removing
poorly performing OSCE stations. Although item analysis has routinely been
performed for most knowledge-based MCQ examinations, the utilization of
item analysis for undergraduate Internal Medicine OSCE is less frequently
practiced and reported in the literature (Kassam, 2003; Newble and Swanson,
1988). In this study, we undertook a Generalizability (G) study of 8 cohorts
3. ITEM ANALYSIS FOR AN UNDERGRADUATE OSCE
107
of fourth-year Internal Medicine OSCEs, followed by an item analysis using
OSCE stations as the unit of analysis.
Subjects and Instruments
Students
In each academic year, 4 cohorts of fourth-year medical students in a 6-year
MD program rotate through Internal Medicine wards. At the end of a
9-week rotation, students are required to take OSCEs as part of their final
evaluation in Internal Medicine. In this study, data from 8 cohorts of
fourth-year medical students in the academic year 2001 and 2002 was collected. The total number of fourth-year medical students studied was 435.
OSCE Structure
The OSCE is conducted during the last week of the students’ 9-week rotation.
Junior faculty members in the Department of Medicine are raters. The 20 to
25 four-minute OSCEs stations are designed to test history-taking and
physical examination skills, as well as interpretation of laboratory tests and
procedural skills. Some stations specifically test communication and counseling skills but in all stations, students must demonstrate appropriate skills
and attitudes in approaching the patients. Real volunteer patients and simulated patients are routinely used. All patients are instructed to act in a
standardized manner by the team of faculty members who developed the
questions. During the 2-hour examination, each student meets with the same
rater for each station.
OSCE questions are drawn from a pool of previously used as well as newly
developed questions. Faculty members in each discipline of Internal Medicine
write questions, which are reviewed by the Internal Medicine Undergraduate
Committee. Each OSCE is developed from a blueprint that corresponds to the
expected clinical knowledge and performance of fourth-year students.
Checklists are used for each station with a total score computed for each
checklist. OSCEs account for 25-percent of the final composite scores.
Item Analysis of OSCE
OSCE Content
Each OSCE consists of the questions from all subspecialties in Internal
Medicine. The design of OSCE questions was such that each station assessed
the fourth-year medical students by a multi-disciplinary approach. A single
OSCE station may assess 3–4 subspecialties simultaneously; for example, a
station on ‘‘history-taking of an elderly patient who presents with weight
4. CHIRAYU AUEWARAKUL ET AL.
108
loss’’ has items on the checklist that are related to oncology, hematology,
endocrine, gastroenterology and socioeconomic problems. On average, each
OSCE consists of 6 history-taking stations, 7 physical-examination stations,
11 laboratory-tests and procedural-skills stations and 1 counseling station.
G-Studies of OSCE
A G-study was performed for each student cohort using a random model p x
i design (Brennan, 2001; Colliver et al., 1989; Cronbach et al., 1972). Item
analysis was performed using each OSCE station as an item and computing
the corrected item-total correlation for each station. OSCE stations which
negatively impacted reliability were deleted and the generalizability coefficient was recalculated. The content and nature of each problem station was
also reviewed to determine the possible cause of the problem.
The mean OSCE scores, standard deviations (SD) and G-coefficients of
the eight OSCE cohorts are shown in Table I. The median number of stations
and students per cohort was 25 and 54.5, respectively. The mean OSCE
scores ranged from 58.57% to 61.27% with a median of 60.18%. The
G-coefficients ranged from 0.45 to 0.80 with a median of 0.62. Five of the 8
cohorts had G-coefficients over 0.60.
Identification of ‘‘Problem Stations’’
Table II shows the number of stations per cohort that had negative impact
on the G-coefficients. The median number of ‘‘problem stations’’ per cohort
was 3.5 of 25 stations. The recalculated G coefficients ranged from 0.60 to
0.83 with a median of 0.72, compared to the median G coefficient of 0.62
before deletion of the problem cases.
Table I. G-coefficients of Internal Medicine undergraduate OSCE scores
Cohort
OSCE
Number of Stations
1
2
3
4
5
6
7
8
Median
Number of students
Mean (%)
SD
G-Coefficient
25
21
22
25
24
25
25
25
25
52
51
55
54
57
56
56
54
54.5
59.99
60.95
60.36
58.59
61.27
58.57
61.08
58.64
60.18
3.07
6.68
5.00
5.8
4.66
6.55
5.28
6.04
5.54
0.55
0.80
0.48
0.63
0.45
0.70
0.60
0.68
0.62
5. 109
ITEM ANALYSIS FOR AN UNDERGRADUATE OSCE
Table II. Number of stations deleted and G-coefficient changes in each cohort
Cohort
G-coefficient,
before deletions
G-coefficient,
after deletions
Number of stations
deleted per total station
1
2
3
4
5
6
7
8
Median
0.55
0.80
0.48
0.63
0.45
0.70
0.60
0.68
0.62
0.64
0.83
0.60
0.73
0.62
0.76
0.70
0.74
0.72
6/25
2/21
6/22
4/25
5/24
3/25
3/25
4/25
3.5/25
Table III. Problem stations categorized according to specific skills tested
Clinical skills
Number of problem
stations in each category
Laboratory and procedural skills
Physical examination skills
History-taking skills
Counseling skills
23/33 (70%)
6/33 (18.2%)
3/33 (9.1%)
1/33 (3%)
Upon review of each cohort, the problem stations were identified as shown
in Table III. Of 33 problem stations, 70% were interpretation of laboratory
tests stations (Table IV). The subspecialty with the most frequent negative
impact on generalizability was infectious disease and the stations frequently
found to be problematic were the ones that test the students’ ability to
diagnose infectious organisms under the microscope. The G coefficients of
the stations with laboratory questions were lower than the non-laboratory
stations median of 0.40 and 0.64, respectively). The students’ mean scores on
laboratory stations were also lower than the non-laboratory stations (median
5.27 vs 6.40, respectively). (The medians were computed across cohorts to
simplify the presentation of data.)
Discussion
Since they were developed in 1975, OSCEs have been extensively utilized for
various purposes in medical education (van der Vleuten and Swanson, 1990).
6. CHIRAYU AUEWARAKUL ET AL.
110
Table IV. OSCE station content with negative impact on G-coefficients
Cohort
Specialty
Skill assessed*
1
General medicine
Critical care
General medicine
Rheumatology
Endocrine
Infectious disease
Infectious disease
General medicine
Infectious disease
Pulmonary disease
General medicine
Hematology
Nephrology
Infectious disease
Infectious disease
Rheumatology
Nephrology
Genetics
Neurology
Geriatrics
Nephrology
Critical care
Infectious disease
Infectious disease
Genetics
Critical care
Cardiology
Endocrine
Infectious disease
Neurology
Infectious disease
Endocrine
Hematology
Lab (urine exam)
Procedural skill (oxygen administration)
PE (skin exam)
Lab (synovial fluid exam)
PE (weight loss)
Lab (malaria)
Lab (malaria)
History-taking (hypertension)
Lab (malaria)
PE (lung exam)
Lab (sputum exam)
Lab (blood smear)
Lab (urine exam)
Lab (stool parasite)
Lab (malaria)
History-taking (arthritis)
Lab (urine exam)
Lab (genetic disease)
Lab (spinal fluid exam)
History-taking (social problem in the elderly)
Lab (urine exam)
Procedural skill (central venous pressure evaluation)
Lab (stool parasite)
Lab (sputum exam)
Lab (genetic disease)
Lab (blood gases)
PE (precordium)
PE (weight loss)
Lab (pus exam)
PE (visual field testing)
Lab (sputum exam)
Counseling (diabetic patient)
Lab (blood smear)
2
3
4
5
6
7
8
Abbreviations: PE (Physical Examination), Lab (Laboratory Investigations).
In the teaching and learning arena, OSCEs have an important role in the
clinical learning process by providing exposure of medical students to standardized patients or real patients in various clinically relevant situations
designed by the medical school faculties. OSCEs have also been utilized in
7. ITEM ANALYSIS FOR AN UNDERGRADUATE OSCE
111
many medical schools around the world for formative and summative
assessment, either to give feedback to students at the end of clinical rotations
or as one component of the final composite scores or grades or as a prerequisite for graduation from medical school (Collins and Gamble, 1996; van
der Vleuten, 2000).
In order to use OSCEs as the final summative assessment, the reliability
and generalizability of OSCEs in each local program should be evaluated
(Boulet et al., 2003). Reliability data provides one major source of validity
evidence according to Messick’s unitary concept of construct validity
(Downing, 2003; Messick, 1989). In this study of 8 cohorts of fourth-year
medical students, the G-coefficients were on average over 0.60, with a
range of 0.48–0.80. These results were comparable to other reports of
reliability achieved in locally developed OSCEs (A-Latif, 1992; Kassam,
2003; Matsell et al., 1991; Petrusa et al., 1990; Regehr et al., 1998; Verhoeven et al., 2000). Our OSCE blueprint, however, was quite different
from those in some western medical schools since we put a major
emphasis not only on the data gathering, physical examination and
communication skills, but also on the ability of our students to interpret
laboratory tests. Approximately 40% of our OSCE stations were thus
related to laboratory tests. Since our graduates must work in rural hospitals with limited facilities, where they must be able to supervise the
medical technicians or even perform the laboratory tests themselves, laboratory skills are very important. For example, the laboratory tests in our
OSCE include blood smears for diagnosis of malaria, Dengue hemorrhagic
fever, thalassemia, stool examination for parasites, urine examination for
urinary tract infection, nephrotic syndrome, spinal fluid examination for
meningitis and chest x-rays for tuberculosis and pneumonia.
Using item analysis for the Internal Medicine OSCE has revealed
important and useful information. When we deleted the problem stations,
the G-coefficients improved considerably, so that all 8 cohorts had Gcoefficients greater than 0.60. This suggested that the problem stations
might have tested skills that students did not acquire or were not taught
in the clinical rotation. We examined the content of the problem stations
and found that the majority of these stations identified by item analysis
were laboratory skills related. The improvement in the G-coefficients by
eliminating these stations is likely due to the nature of skills tested in the
laboratory-skills stations, since laboratory skills are obviously different
from data-gathering and physical examination skills. The overall lower
mean scores on the laboratory stations as compared to the non-laboratory
stations could also indicate that the students did not adequately acquire
these particular laboratory skills during the Internal Medicine rotation.
Students who perform well in communication skills may not do as well in
the interpretation of blood smears, urine analysis, and so on. Achieving
8. 112
CHIRAYU AUEWARAKUL ET AL.
competence in one area is thus not necessarily a good predictor of competence in another (van der Vleuten, 2000; Wass et al., 2001).
We subsequently computed the G-coefficients for laboratory skills stations alone and found these G-coefficients to be low, with a median of
0.40 (0.04–0.63), suggesting ‘‘case specificity’’ for students’ laboratory
skills. The reliability of non-laboratory skills stations was better than the
laboratory skills stations as shown by the G coefficients of 0.56–0.87
(median of 0.64). The lower reliability of laboratory stations could be
accounted for by the smaller number of laboratory stations (mean of 9
stations) as compared to non-laboratory stations (mean of 15 stations).
However, we also observed that the inter-station correlations were very
variable among laboratory stations across all cohorts. These results could
be due to the fact that the student’s laboratory skills are case-specific and
the stations independently tested unique and distinct skills or due to the
problem inherent within the station scoring process that needs to be
further studied.
Since students are required to perform and interpret laboratory tests in
order to meet the requirement of our department and the National Medical
Council Standards, the problem stations can not be eliminated, because this
would reduce the validity evidence for the OSCE. Rather, there should be a
remediation of the station and further change in the curriculum to facilitate
students learning these skills. Validity should not be lost at the expense of
reliability (Downing, 2003; Norman et al., 1991; van der Vleuten, 2000).
In conclusion, this Internal Medicine OSCE has been shown to be reliable
and acceptable for local use. Item analysis of OSCE stations is useful and
should be performed to improve the reliability of total OSCE scores. Careful
structuring of OSCE questions and remediation of OSCE problem stations is
crucial to support the continued use of OSCEs for final clinical performance
assessments.
Acknowledgements
We would like to thank the Foundation for Advancement in Medical Education Research (FAIMER) for financial support of Dr. Chirayu Auewarakul through the International Fellowship of Medical Education
(IFME)-2002 award. The Faculty development awards from the Anandamahidol Foundation and Siriraj Chalermprakiat Fund, Thailand are also
appreciated. We thank Ms. Jaree Prasarnkul, Ms. Pichavadee Sae-ung and
Mr. Somkuan Sriyounglek at the Office of Undergraduate Medical Education, Department of Medicine for their excellent work with OSCE administration, score processing and data collection.
9. ITEM ANALYSIS FOR AN UNDERGRADUATE OSCE
113
References
A-Latif, A. (1992). An examination of the examinations: The reliability of the objective structured clinical examination. Medical Teacher 14: 179–183.
Boulet, J.R., McKinley, D.W., Whelan, G.P. & Hambleton, R.K. (2003). Quality assurance
methods for performance-based assessments. Advances in Health Sciences Education 8: 27–47.
Brennan, R.L. (2001). Generalizability Theory. New York, NY: Springer-Verlag.
Collins, J.P. & Gamble, G.D. (1996). A multi-format interdisciplinary final examination. Medical
Education 30: 259–265.
Colliver, J.A & Williams, R.G. (1993). Technical issues: Test application. Academic Medicine 68:
454–458.
Colliver, J.A., Verhulst, S.J., Williams, R.G. & Norcini, J.J. (1989). Reliability of performance
on standardized patient cases: A comparison of consistency measures based on generalizability theory. Teaching and Learning in Medicine 1: 31–37.
Cronbach, L.J., Gleser, G.C., Nanda, H. & Rajaratnam, N. (1972). The dependability of behavioral measurements: Generalizability for scores and profiles. New York: John Wiley and Sons.
Crossley, J., Davies, H., Humphries, G. & Jolly, B. (2002). Generalisability: A key to unlock
professional assessment. Medical Education 36: 972–978.
Downing, S.M. (2003). Validity: On the meaningful interpretation of assessment data. Medical
Education 37: 830–837.
Harden, R., Stevenson, M., Downie, W. & Wilson, G. (1975). Assessment of clinical competence
using objective structured examinations. British Medical Journal 1: 447–451.
Kassam, N. (2003). Some validity evidence of an undergraduate Internal Medicine OSCE.
Masters of Health Professions Education (MHPE) Thesis, University of Illinois at Chicago,
Department of Medical Education, Chicago.
Matsell, D.G., Wolfish, N.M. & Hsu, E. (1991). Reliability and validity of the objective structured clinical examination in pediatrics. Medical Education 25: 293–299.
Messick, S.J. (1989). Validity. In R.L. Linn (ed), Educational Measurement (3rd ed), pp. 13–104.
New York: American Council on Education and Macmillan.
Miller, G.E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine
65: S63–S67.
Newble, D.I. & Swanson, D.B. (1988). Psychometric characteristics of the objective structured
clinical examination. Medical Education 22: 325–334.
Norman, G.R., Van der Vleuten, C.P.M. & De Graff, E. (1991). Pitfalls in the pursuit of
objectivity: issues of validity efficiency and acceptability. Medical Education 25: 119–126.
Petrusa, E.R., Blackwell, T.A. & Ainsworth, M.A. (1990). Reliability and validity of an objective
structured clinical examination for assessing the clinical performance of residents. Archives in
Internal Medicine 150: 573–577.
Regehr, G., MacRae, H., Reznick, R.K. & Szalay, D. (1998). Comparing the psychometric
properties of checklists and global rating scales for assessing performance on an OSCE
format examination. Academic Medicine 73: 993–997.
Van der Vleuten, C. (2000). Validity of final examinations in undergraduate medical training.
British Medical Journal 321: 1217–1219.
Van der Vleuten, C.P.M. & Swanson, D.B. (1990). Assessment of clinical skills with standardized
patients: State of the art. Teaching and Learning in Medicine 2: 58–76.
Verhoeven, BH., Hamers, J., Scherpbier, A.J., Hoogenboom, R.J. & Vleuten, C.P.van der (2000).
The effect on reliability of adding a separate written assessment component to an objective
structured clinical examination. Medical Education 34: 525–529.
Wass, V., Vleuten, C.van der, Shatzer, J. & Jones, R. (2001). Assessment of clinical competence.
Lancet 357: 945–949.