1. 2011
ASSESSMENT INSTITUTE
OCTOBER 30-NOVEMBER 1
INDIANAPOLIS, IN
A my Sl o a n e
A s s e ssmen t
C o o rdi nator
Of fi ce o f
A s s essme nt
U N C E s h e lman
Sc h o o l o f
P h a rm ac y
2. 7 COMMON MISPERCEPTIONS
ABOUT ASSESSMENT
1. We’re doing just fine without it.
2. We’re already doing it.
3. We’re far too busy to do it.
4. The most important things can’t/shouldn’t be
measured.
5. We’d need more staff and lots more money.
6. They’ll use the results against us.
7. No one will care about or use what we find.
3. 7 COMMON MISPERCEPTIONS
ABOUT ASSESSMENT
AND 7 REASONABLE RESPONSES
1. We’re doing just fine without it.
OK, then let’s use assessment to find out what works, and to help
us document and build on our successes.
2. We’re already doing it.
OK, then let’s audit all the assessments we already do to discover
what we know and what we don’t.
3. We’re far too busy to do it.
OK, then let’s audit all the assessments we already do to discover
what we know and what we don’t.
4. The most important things can’t/shouldn’t be
measured.
And not everything measurable should be measured, but let’s see
if we can agree on how we can tell when we’re succeeding in these
most important things.
4. 7 COMMON MISPERCEPTIONS
ABOUT ASSESSMENT
AND 7 REASONABLE RESPONSES
5. We’d need more staff and lots more money.
Since we're unlikely to get more resources: how, what, and where
can we piggyback, embed, and substitute?
6. They’ll use the results against us.
They might. So, let’s build in strong safeguards against misuse
before we agree to assess.
7. No one will care about or use what we find .
To avoid that, let's agree not to do any assessments without a firm
commitment from stakeholders to use the results.
5. PRE-INSTITUTE WORKSHOP:
ASSESSMENT 101
Arizona State University’s Assessment Handbook:
5 Steps to Effective Program Assessment
6. PRE-INSTITUTE WORKSHOP:
ASSESSMENT 101
Planning Assessment:
Consider the institutional, school, program mission statements
Consider program goals
Consider program learning outcomes
Identify measures for each outcome
Identify performance criteria for each measure
Identify a sampling strategy for each measure
7. SESSIONS
Assessment & the Promise of the Degree Profile
Assessment & Improvement: Rethinking Faculty Involvement
Doing Assessment as if Learning Matter s Most: Engaging Faculty &
Students with Simple, Practical, Power ful Techniques
Institutional Transparency of Student Learning Outcome
Assessment: A Framework
Valuing People Through an Improved Process for Student Evaluation
of Teaching
The Measuring Quality Inventor y
Building a Culture of Improved Learning, Student Engagement,
Communication, and Assessment Using Rubrics
Assessing Assessment: Taking Stock of Your Own Campus
8. KEY TAKE HOME POINTS/IDEAS
Create a culture of assessment based on evidence
within the UNC Eshelman School of Pharmacy
Provide institutional leadership
Engage stakeholders, faculty, staff, and students
Develop and share a common assessment language
Establish trust
Build shared and valued goals and motivation
9. KEY TAKE HOME POINTS/IDEAS
Of fice of Assessment and Center for Educational Excellence
in Pharmacy should collaborate to provide opportunities for
faculty to connect assessment to teaching and learning
Program Assessment Handbook and “The 5 Steps to Ef fective
Program Assessment” will be a valuable resource to us in
curricular transformation ef forts
Create useable and transparent program learning outcomes;
consider rubrics and/or degree profiling tools; post on Of fice
of Assessment website
Use and communicate assessment results faculty and
students to enhance teaching and learning experiences
20th Anniversary of conferenceDesigned for faculty, student affairs professionals, and administrators who have an interest in or responsibility for assessmentLearn about innovations in assessment; best practices, new techniques, approaches, and ideas in a variety of outcomes assessment areasSessions are presented by leaders at universities, organizations, and businessesOpportunity for networking with other assessment professionals (1000+ attendees)My goal for attending:Dedicated time to learn about assessment resources and techniquesNetwork with other higher ed assessment professionalsMy masters program is in Training and Development includes learning about assessment but it is not the focus
This list of misperceptions was presented at one of the sessions I attended, and I thought you would appreciate it. Common comments we hear from faculty.
The next time you’re talking to a colleague about assessment, and your colleague is grumbling about the practice, and utters on one of these misperceptions, you have a reasonable response.
Workshop name: Assessment 1016-hour workshop Led by Wanda Baker, Associate Director for Assessment and Information Technology, in the University Office of Evaluation and Educational Effectivenessat Arizona State University.We received a copy of ASU’s Assessment Handbook: 5 Steps to Effective Program Assessment. (For our purposes, “program” can refer to the PharmD program or PhD program.)We worked through “Step 1: Planning assessment” during the workshop. The handbook includes all 5 steps; however, Step 1 takes the most amount of work and time, and therefore takes up most of the handbook. I have a paper copy of the handbook that I’ll pass around the room. For those of you interested in looking at it more in depth, I also have a PDF that I can email to you.
In the workshop, we walked through the components of “planning assessment”. We worked individually with a program of our choice. (I worked with our PharmD program.)Then, we divided into small groups to review the work we had done individually. Lastly, all participants came together to discuss the process and ask questions.Consider the institutional, school, program mission statementsInstitutional mission is the foundation of everything we doSchool and program missions should flow from and directly support the institutional missionAssessment outcomes must be directly related to program mission (and, by extension, those of the school and institution) *I reviewed UNC-Chapel Hill’s mission statement, the School of Pharmacy’s mission statement, and the PharmD mission statement.Consider program goalsWhat hopes and aspirations do program faculty have for program graduates 3-5 yrs after graduation?Focus on a few goals; not all possible goals.Do the goals support the program mission statement? *I picked a program goal from our existing ‘Ability-Based Outcomes for the Doctor of Pharmacy Curriculum’ document. ”Provide leadership in advancing the standard of health care and the profession of pharmacy”Consider program learning outcomesWhat should program graduates know and be able to do?This is the most difficult and time consuming part of the process.Focus on 3-5 learning outcomesWe discussed different ways to review/monitor outcomes (p. 13)We discussed important guidelines to consider when writing program outcomes (p. 14)E.g., don’t include multiple outcomes in a single statementOur exercise was to develop a single program outcome in support of the long term goal we developed for our program in prior exercise. * I reviewed the existing outcomes in the ‘Ability –Based Outcomes’ document, and picked one that I thought aligned with the program goal I selected. PharmD graduates will be able to…“Participate in the development of drug use policy and pharmacy benefits” (2.3)Identify measures (direct and indirect) for each outcomeIdentify 2 measures for each outcome. The first must be a direct measure. The second can be direct or indirect.We reviewed examples of direct and indirect measures.We reviewed guidelines to consider when identifying appropriate measures for our outcomes.E.g., course grades are not appropriate measures of student learning. It is appropriate to use the grade of a specific exam or project that specifically measures student learning on the outcome.*For direct measure, I identified “practical clinical assessment”*For indirect measure, I identified, “Employer or Alumni Survey”Identify performance criteria for each measureIdentify a threshold above which you will be satisfied that students who graduate from the program possess the knowledge or skill specified in the outcome.Think of a reasonable standard. Not 100%.We reviewed guidelines to consider when identifying performance criteria for each measure.E.g., course grades are not appropriate for use with performance criteria. *I wrote: 90% of students will earn a grade of 85 or higher on the practical clinical assessment.
These are the sessions I attended. Many of the sessions were interactive and great networking opportunities.There was also a social hour sponsored by AACP where I had the opportunity to meet assessment leaders from others schools of pharmacy (Purdue, Northern Ohio U, Pacific University in Oregon)