Overview of Institutional Assessment
Upcoming SlideShare
Loading in...5

Like this? Share it with your network

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Institutional Assessment and Educational Initiatives David S. Roffman, PharmD BCPS Associate Dean for Academic Affairs
  • 2. Institutional Assessment
    • Goal : To institute a comprehensive, iterative process of continuous improvement evaluating key issues of critical importance, regardless of a seemingly endless supply of data.
    • Components and Considerations : Institutional assessment must be both qualitative and quantitative , contain longitudinal data to detect trends over time, and incorporate peer comparisons and benchmarks when available.
  • 3. Institutional Assessment
  • 4. Institutional Assessment
  • 5. Institutional Assessment
      • Institutional Level: course evaluations
      • Programmatic Level: curriculum mapping
      • Learning Outcomes Level: OSCEs & portfolios
  • 6. 1. Institutional Level Assessment
    • Standard course evaluation questions: was the syllabus clear, did the coursemaster address class-wide issues, was the textbook helpful, were prerequisite courses adequate preparation for this course…
    • Shady Grove: “ Asynchronous availability of lecture material was beneficial to my formal learning experience in this course .”
    • Student feedback indicated a need to improve on presentation quality of asynchronous lectures delivered via MediaSite, so CNS support staff created “ Building a Better Presentation ,” a workshop for faculty, post-docs and teaching assistants.
  • 7. 2. Programmatic Level Assessment
    • At a programmatic level, the revised PharmD curriculum is being mapped for science content and terminal performance outcomes .
    • Mapped 198 items in 4 areas of science:
        • basic biomedical – immunology, biochemistry…
        • pharmaceutical – pharmacology, biopharmaceutics…
        • social/administrative – ethics, practice management…
        • clinical science – drug information, medication safety…
    • At what level of emphasis – introduced, reinforced, or emphasized
    • Have we covered everything that needs to be covered in a comprehensive PharmD curriculum, and are there any hidden gaps? The faculty will determine if or how those areas should be addressed.
  • 8. 3. Learning Outcomes Assessment
    • Terminal performance outcomes (TPOs) form the “strategic plan” for the PharmD curriculum. These are the 63 “goals” we set for our students to reach by the time they graduate. Each course supports these goals with their own course objectives and assessments.
          • TPOs are aligned with the 3 areas of CAPE* Outcomes :
          • Pharmaceutical care (60% of our TPOs)
          • Health care system (20% of our TPOs)
          • Population (20% of our TPOs)
    • *Center for the Advancement of Pharmacy Education
  • 9. Educational Initiatives - OSCEs
    • Objective Structured Clinical Exams , or OSCEs, are a means of measuring student progress toward the terminal performance outcomes.
    • OSCEs are standardized simulations of clinical experiences using professional actors. They are used in Canadian medical/pharmacy licensing exams and in USMLE.
    • Traditional assessment methods are not as useful for evaluation of clinical skills, including clinical knowledge and interpersonal and problem-solving skills.
  • 10. Educational Initiatives - OSCEs
    • EXAMPLE: P2 Fishbowl Learning Objectives : After completing pre-learning activity assignments and participating in the session, student groups will be able to do the following:
    • After watching a simulated patient counseling encounter, develop an analytical checklist of performance and/or knowledge items that should have been demonstrated by the “student” during the encounter.
    • After re-watching a simulated patient counseling encounter, rate “student” performance using an analytical checklist and global impression scale.
    • Interview a simulated patient to collect a past medical history, medication history, and focused review of systems.
    • Practice and observe a variety of verbal and non-verbal communication behaviors such as active listening, empathy, use of lay terminology, and re-direction of interview.
  • 11. History of OSCE at Maryland
    • Summer 2006 – team sent to Toronto for 3-day training workshop
    • 2006-2007 academic year – OSCE piloted in PCPPM course
    • 2007-2009 – OSCE expanded to years 2-4 of the curriculum, blueprint developed based on program TPOs, and practice opportunities integrated into courses
    • 2009 and beyond – development of interdepartmental committee to direct OSCE activities, further expand blueprint, add additional practice opportunities
  • 12. Proposed OSCE Activities (housed within Abilities Lab sequence)
    • Year 1
      • TOSCE fall semester
      • TOSCE + 3-station OSCE spring semester
    • Year 2
      • TOSCE fall semester
      • TOSCE + 3-station OSCE spring semester
    • Year 3
      • TOSCE fall semester
      • TOSCE + 5-station OSCE spring semester
    • Year 4
      • 5-station OSCE
  • 13. Educational Initiatives - OSCEs
    • Graded OSCE
    • STEP 1
    • Student is presented with a case scenario
  • 14. Educational Initiatives - OSCEs
    • Graded OSCE
    • STEP 2
    • Student meets with standardized patient (professional actor) and student interviews patient, performs task/skill in stations, such as
          • Symptom analysis
          • Medication history
          • Patient interviewing
          • Vitals
  • 15. Educational Initiatives - OSCEs
    • Graded OSCE
    • STEP 3: Patient evaluates student using a standard rubric
  • 16.  
  • 17. Educational Initiatives - Portfolios
    • Electronic student portfolios were established in the Fall 2009 to generate a sense of awareness and self-evaluation that leads to continuous professional development. In the future, preceptors and faculty will also be able to comment and review.
    • Two main components:
    • 1. Students will store meaningful “artifacts” throughout their education, and be able to look back at previous entries and see their growth.
    • 2. Each semester, students self-evaluate their competency on the terminal performance outcomes, identify knowledge gaps and plan for improvement.
  • 18. Educational Initiatives – Contemporary Issues
    • Contemporary Issues is a concept that was identified by the Curriculum Committee during the redesign process: students needed more exposure to relevant current events that affect the profession.
    • Students are asked to attend 2 educational forums and 2 social events each semester, then write up guided reflections which are stored in their electronic portfolios as “artifacts.”
      • Purpose of educational forums is to inspire and intrigue their professional interests with “hot topics” in healthcare (interdisciplinary where possible)
      • Purpose of social events is to explore the more informal side of their pharmacy education, and build important skills such as interpersonal communication, cultural awareness and collegiality