Using ExamSoft Data to Prepare For and Ease the Accreditation Process
•Download as PPTX, PDF•
2 likes•301 views
Presented by Kimberly K. Daugherty, PharmD, BCPS, Professor and Assistant Dean of Academic Affairs and Assessment, Sullivan University College of Pharmacy, and Ashley Castleberry, PharmD, MAEd, Assistant Professor and Director of Assessment, University of Arkansas for Medical Sciences College of Pharmacy, and Leah Simpson, EdD, MPA, Director of Curriculum Development and Assessment, University of Kentucky When preparing for the accreditation process, ExamSoft data can be used to document student mastery of accreditation standards, the student development of self-awareness, as well as provide data for course and curriculum improvement. Data from ExamSoft can also provide students with report cards providing autonomy supportive feedback (“feed up, feed back, feed forward”). In this model, the first step is to establish a clear goal for the students to obtain (feed up). Next, the teacher responds to the work being done (feed back). Finally, students are provided with ways to improve attainment of their goals (feed forward). This process can also be useful for the teacher as it allows for justifiable improvement in teaching and in the course. Student report cards are autonomy supportive, as they can provide students with the goal to obtain (achievement of the program outcomes) and allow the teacher or advisor to comment on things being done well or needing improvement. It is possible to collate student data so that a college can see which outcomes are being taught well in the curriculum and which may need improvement. Finally, report cards can be useful tools to help students develop and practice their metacognitive skills. The University of Arkansas, University of Kentucky, and Sullivan University Colleges of Pharmacy have been using this type of data to improve student learning and performance. The purpose of this session is to describe how ExamSoft longitudinal assessment data can be used to develop a report card for students showing their progression through a curriculum and how this data can be used for accreditation.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Using ExamSoft Data to Prepare For and Ease the Accreditation Process
Similar to Using ExamSoft Data to Prepare For and Ease the Accreditation Process (20)
More from ExamSoft
More from ExamSoft (20)
Recently uploaded
Recently uploaded (20)
Using ExamSoft Data to Prepare For and Ease the Accreditation Process
- 1. 1
- 2. Use of ExamSoft Data to Prepare For and Ease the Accreditation Process Kimberly K. Daugherty, PharmD, BCPS Ashley Castleberry, PharmD, MAEd Leah Simpson, EdD, MPA
- 3. Polling Question: Who is in the audience? • Please select the category that best fits you: 1. Assistant/Associate Dean 2. Director of Assessment 3. Chair of Curriculum and/or Assessment Committee 4. Faculty member 5. Staff Member
- 4. Session Learning Outcomes • Discuss how the autonomy supportive feedback system can be used by students and faculty to enhance student learning and metacognitive development • Demonstrate how longitudinal assessment data run from ExamSoft can be used to provide students with assessment report cards showing their progress across a curriculum • Model how ExamSoft longitudinal data can be aggregated for use in accreditation reports
- 5. Autonomy and ExamSoft • Self-Determination Theory (SDT) states that all humans have three basic psychological needs: autonomy, competence, and relatedness (Deci & Ryan, 1985, 1994, 2000) • Autonomy in learning has been linked to greater student satisfaction, persistence, and deeper learning (Seiver & Troja, 2014; Wielenga- Meijer, Taris, Wigboldus & Kompier, 2012)
- 6. Autonomy-Support • By meeting students’ intrinsic desires, autonomy-support has been shown to increase student satisfaction, persistence, creativity, and curiosity (Patrick & Williams, 2009; Seiver & Troja, 2014) • Perception of autonomy-support leads to greater school achievement (Mih & Mih, 2013) • Feedback can be autonomy-supportive when it Feeds Up, Feeds Back, and Feeds Forward (Hattie & Timperley, 2007)
- 7. University of Kentucky Student End of Exam Report Cards
- 8. Feed Up, Feed Back, and Feed Forward with ExamSoft Reports
- 9. Students LOVE these reports!
- 10. Does your institution use end of exam reports? 1. Yes 2. No
- 11. University of Arkansas for Medical Sciences Longitudinal Reports -Students -Course -Curriculum
- 12. Reports > Longitudinal Analysis
- 13. Student and Course Reports
- 14. Student and Course Reports 64.17%
- 15. Student and Course Reports
- 16. How Longitudinal Reports Fit Autonomy Support Model • Feed Up • Provide students with data needed to formulate goals in specified areas • Feed Back • Provides students with data on achievement of their goals • Feed Forward • Students meet with faculty to discuss data and make plans for improvement
- 17. How Longitudinal Reports Fit Autonomy Support Model • Programmatic Assessment • Data from courses provide aggregate data on course effectiveness • Data from the curriculum provide aggregate data on curriculum effectiveness
- 20. Does your institution use longitudinal reporting at the course level? 1. Yes 2. No
- 21. Sullivan University Comprehensive Student Report Cards
- 22. SUCOP Comprehensive Student Report Card • Background on Sullivan University College of Pharmacy • 3-year year around program • Quarter system • Comprehensive Student Report Cards • Reason for implementation: to provide students and their advisors information on overall GPA, progression on pharmacy knowledge and skills, and progression on program outcome attainment • Data on report cards is aggregated from multiple sources • E-value-experiential software • ExamSoft-didactic testing software • Campus Vue-Student Information System • CHARM: OSCE data
- 23. Student Report Card • Required didactic elements: data is rolled up from exam question results tagged in ExamSoft • Program outcome elements: data rolled up from exam question results tagged in ExamSoft • Report cards are compiled after the Fall and Spring quarters are completed for the first and second professional year classes
- 24. Required Didactic Elements Assessment BS PS SABS CS ExamSoftStudentAverage** 63.92 77.73 77.55 76.41 NumberofQuestions 888 1748 674 1291 ClassAverage ClassSD 75.29 12.07 86.27 11.43 85.69 17.34 84.98 13.86 PCOAAug2015StudentAverag 248 277 263 246 SUCOPAvg-NationalAvg 328-350 324-334 327-339 318-326
- 25. Program Outcomes CommunityIPPE*** 3 3.00 3.00 3 NA 3.00 3.00 HospitalIPPE*** 3 3 3 NA 3 3 NA ExamSoftStudentAverage** 75.87 78.52 71.4 82.67 76.37 81.63 67.65 #ofQuestions 3063 933 222 70 125 686 34 ClassAverage 85.46 83.66 85.55 84.39 82.92 89.06 84.88 ClassSD 6.41 9.06 9.41 12.69 8.61 11.6 14.96 AssessmentAreas 1.1 2.1 2.2 2.3 2.4 3.1 3.2
- 26. How Report Cards Fit Autonomy Support Model • Feed Up • Provide students with data needed to be able to formulate goals in each of the specified areas • Feed Back • Provides students with data on achievement of their goals • Feed Forward • Students meet with their advisors to discuss the data and make plans for improvement • Programmatic assessment • Data from the individual report cards is “rolled-up” to provide aggregate data on element attainment
- 27. How data can be combined with other data to tell accreditation story PY1 Didactic PY1 and PY2 Didactic EO # Questions Mean SD # Questions Mean SD 1.1 1820 84.5% 0.1 4896 85.9% 0.1 2.1 300 78.5% 0.1 809 86.7% 0.1 2.2 270 86.5% 0.1 270 86.6% 0.1 2.3 46 77.9% 0.2 98 85.8% 0.1 2.4 14 78.2% 0.1 212 85.7% 0.1 3.1 293 90.2% 0.1 871 84.3% 0.1 3.2 34 93.0% 0.1 62 92.7% 0.1 3.3 90 85.1% 0.2 97 81.4% 0.1 3.4 8 93.7% 0.1 9 89.6% 0.1 3.5 3 77.0% 0.2 7 90.2% 0.1 3.6 149 87.0% 0.1 176 90.4% 0.1 4.1 25 87.2% 0.1 25 92.7% 0.1 4.2 5 76.1% 0.2 6 87.9% 0.1 4.3 NA NA NA NA NA NA 4.4 79 85.9% 0.1 577 87.8% 0.1 JCPP-PPCP % of Exam Mean SD % of Exam Mean SD C NA NA NA 2.6% 36.0% 0.2 H NA NA NA 55.7% 80.1% 0.1 A NA NA NA 17% 93.8% 0.1 R NA NA NA 19% 80.3% 0.2 M NA NA NA 5% 61.5% 0.2 Blocks 1 and 2* Blocks 3 and 4** EO Mean SD Mean SD 1.1 3.20 0.58 3.63 1.6 2.1 3.07 0.73 4.04 1.19 2.2 2.91 1.32 3.84 1.54 2.3 NA NA 3.21 2.04 2.4 2.64 1.5 3.41 1.82 3.1 3.35 0.51 4.18 0.86 3.2 2.98 0.8 3.64 1.6 3.3 NA NA 2.84 2.11 3.4 3.72 0.47 4.45 0.73 3.5 NA NA 3.69 1.62 3.6 2.97 0.73 3.98 1.29 4.1 3.61 0.61 4.37 0.73 4.2 3.29 0.58 4.01 1.32 4.3 NA NA 3.12 1.93 4.4 3.40 0.52 4.53 0.66 JCPP-PCPP Mean SD Mean SD C NA NA 4.12 1.21 H NA NA 4.11 1.17 A NA NA 4.02 1.14 R NA NA 4.02 1.17 M NA NA 3.92 1.25 Blocks 1 and 2: 4-point Likert scale (0=NA-4=Exceed); Blocks 3 and 4: 5-point Likert scale (0=NA-1=Beyond entry level)
- 28. Does your institution use longitudinal reporting at the program level? 1. Yes 2. No
- 29. QUESTIONS
Editor's Notes
- Although it isn’t really possible to see intelligence or “grit,” we often know many of the students that will be most successful in our course – even on the first day of class. The students that are excited to be in your class, give a little fist pump when you mention the type of assignments required, and look at you with rapt attention when you talk about all the topics you will cover over the semester – you know those students will do just fine in your course? Why? It’s probably because they are most motivated. Self-Determination Theory states that in order for a person to be highly motivated, they must have their three, basic psychological needs met. Every person needs to feel competent, experience relatedness, and have autonomy. When we have these three needs met, we are highly motivated. The students that I just mentioned – they have those three needs met. Although all the needs are very important, researchers have come to believe that, at least for education purposes, autonomy may be the most important. Students that have autonomy for learning want to be in the course. They would have signed up for the class, even if it wasn’t required. These students tend to experience those positive educational outcomes (persistence, deeper learning, greater satisfaction) that we desire. But what about those students that aren’t that excited? The ones that are in your course because it’s a means to an end; can we get them to experience those same positive outcomes?
- Yes, if we support their autonomy. Autonomy-support can come in a variety of forms: autonomy-supportive speech (asking rather than telling), providing rationale for activities in the course, helping students understand the relevance of the learning activities, or lettings students provide feedback about the course. When students experience these supportive activities, they often experience the same positive educational outcomes as students with naturally high levels of autonomy for learning. In short, we can improve student achievement through autonomy-support. One of the best ways to support student-autonomy is through feedback. Not all feedback is autonomy-supportive. In order to meet that standard, feedback must meet three criteria: Feed Up, Feed Back, and Feed Forward. Feed Up: Tell students where they are going. “Here’s what you will be able to do or will know by the end of this unit.” Feed Back: Tell students how they are going. “Here’s how you have performed thus far for each thing we’re trying to accomplish.” Feed Forward: Tell students how to move ahead. “Here’s what you need to do in order to improve.”
- At the University of Kentucky College of Pharmacy, we encourage faculty to think logically about providing student feedback about the most pertinent and obvious topics that are covered in the curriculum. Faculty members create their hierarchy of outcomes and topics and align questions/rubrics to the items on the hierarchy. Most faculty align to the topics of the course, though some now align to the CHARM process, as well, so that students can get feedback not only regarding their knowledge and application for specific subject areas, but also on the patient care process. After assessments, faculty send reports to students that mirror the one on the slide. It is important to note that UK has been slowly rolling out ExamSoft usage throughout the curriculum. Though many faculty have begun to use the alignment feature, not all have. However, more and more use it because they see value in the feedback. In addition, students have made it clear that this is the best part of ExamSoft. The feedback is good. If, for some reason, a students report has a glitch so that they can only see their grade, they request to be resent the report so that it contains the more detailed feedback.
- Foundational Knowledge Essentials for Practice and Care Approach to Practice and Care (3.2=Educator) Personal and Professional Development