Judging the Relevance and worth of ideas part 2.pptx
Georgia International Information Literacy Conference Closing the Loop Assessment Data
1. Georgia International Information
Literacy Conference
September 21, 2012
Closing the Loop: Using
Assessment Data to Improve
Library Instruction Programs
Adrianna Lancaster, Dean
Chelsea Baker, Instruction & Circulation Librarian
East Central University Linscheid Library
10. Our Data
Peer Evaluations
Survey Questions
1. Does the librarian seem approachable and generally
aware of the reference floor while at the reference desk?
2. Does the librarian greet people in a courteous manner
as they approach the reference desk?
3. Does the librarian understand the user's needs and
respond accordingly?
4. Does the librarian make full use of the available
resources to meet the user’s needs?
5. Does the librarian seem at ease and take steps to make
the user comfortable?
6. Does the librarian ask follow-up questions to ensure the
user's needs are met?
7. Does the librarian close the transaction tactfully and
make the user aware they are available should they
require further assistance?
11. Our Data
Does the librarian make full use of the available resources to meet the user's
needs?
2008 2009 2010
Does Not Meet Expectations (1) - - 5 18% 1 3%
Meets Expectations (2) 13 57% 7 25% 14 45%
Exceeds Expectations (3) 10 43% 16 57% 16 52%
Does the librarian seem at ease and take steps to make the user
comfortable?
2008 2009 2010
Does Not Meet Expectations (1) 2 9% 1 4% -- 0%
Meets Expectations (2) 15 65% 20 71% 25 81%
Exceeds Expectations (3) 6 26% 7 25% 6 19%
12. Closing the Loop Process
1. Brainstorm possible problems.
2. Select the problem to address.
3. Brainstorm many possible solutions
to the problem, and the resources
needed for each solution.
4. Select a strategy or strategies to
implement.
13. Our Data
Does the librarian make full use of the available resources to meet the user's
needs?
2008 2009 2010
Does Not Meet Expectations (1) - - 5 18% 1 3%
Meets Expectations (2) 13 57% 7 25% 14 45%
Exceeds Expectations (3) 10 43% 16 57% 16 52%
Does the librarian seem at ease and take steps to make the user
comfortable?
2008 2009 2010
Does Not Meet Expectations (1) 2 9% 1 4% -- 0%
Meets Expectations (2) 15 65% 20 71% 25 81%
Exceeds Expectations (3) 6 26% 7 25% 6 19%
15. Our Data
Does the librarian make full use of the available resources to meet the user's
needs?
2008 2009 2010
Does Not Meet Expectations (1) - - 5 18% 1 3%
Meets Expectations (2) 13 57% 7 25% 14 45%
Exceeds Expectations (3) 10 43% 16 57% 16 52%
Does the librarian seem at ease and take steps to make the user
comfortable?
2008 2009 2010
Does Not Meet Expectations (1) 2 9% 1 4% -- 0%
Meets Expectations (2) 15 65% 20 71% 25 81%
Exceeds Expectations (3) 6 26% 7 25% 6 19%
17. The Closed Loop
Does the librarian make full use of the available resources to meet the user's
needs?
2008 2009 2010 2011
Does Not Meet Expectations (1) - 0% 5 18% 1 3% 1 3%
Meets Expectations (2) 13 57% 7 25% 14 45% 21 53%
Exceeds Expectations (3) 10 43% 16 57% 16 52% 18 45%
Does the librarian seem at ease and take steps to make the user comfortable?
2008 2009 2010 2011
Does Not Meet Expectations (1) 2 9% 1 4% - 0% - 0%
Meets Expectations (2) 15 65% 20 71% 25 81% 24 60%
Exceeds Expectations (3) 6 26% 7 25% 6 19% 16 40%
Sticky notes—What specific questions are you hoping that this session will answer? If you don’t have any specific questions, what are you hoping to get out of this session?
CB To give you an idea where we’re coming from…We are from East Central University in Ada, Oklahoma. Ada has a population of about 16,000 and is located 90 miles from Oklahoma City. East Central is a regional public university offering Bachelor’s and Master’s degrees, with an enrollment of about 4400 students. East Central University’s Linscheid Library has 5 full-time librarian positions, in addition to the dean, and offers 75 hours of operation each week during the regular semesters.
AL talks about steps 1 and 2 briefly
CB ( 10 minutes ) One kind of instruction assessment involves student learning outcomes. This is a direction that we are moving toward, by looking at SAILs test scores and other measurements. Until recently, we’ve been focused on rather traditional forms of program assessment, like surveys, usage data, and evaluation forms, rather than on student learning. Currently and historically, we’ve collected traditional data in three areas for our program assessment: reference, instruction, and resources.
For reference program assessment, we have two measurements: evaluation forms which are completed by the other librarians, and reference assessment survey cards which patrons are invited to fill out after reference interactions.
For instruction program assessment, we have three data points: peer evaluations which are completed by the other librarians, faculty or instructor evaluations which are completed by the instructor of the course for which we are providing library instruction, and student evaluations which are completed by the students in the course.
Now that you know what kinds of data we collect, we’d like to know…what kinds of data do you collect?
CB talks about step 4 briefly, then AL talks about Step 5: Closing the Loop CB: Once you’ve collected your data, don’t just put it on a shelf. You need to spend some time analyzing and interpreting the data. This is when you would apply any statistical analyses that you may want to do. This step is really about identifying the highs, lows, and mid-points of your data. Where are you doing well? What area needs the most help? And so on…Once you’ve identified these areas, then you can move on to your final step of using the data to close the loop.
AL
AL explains data, then we tag team the modeling session. CB hands out data sheet while AL explains the data
Tag team
CB 1. Here we mean the causes for the low points in your data. Sometimes there are multiple causes for one poor rating. Sometimes there is one root cause underlying multiple low scores…So try to keep this in mind and cast your net widely while you’re brainstorming possible problems. 2. Select the problem to address. Which problem seems to be the likeliest or the most pressing? 3. Brainstorm many possible solutions to the problem, and the resources needed for each solution. Again, you want to brainstorm as many solutions as possible because if your initial intervention doesn’t work you might be coming back to this list. And, of course, we need to take into account the resources needed for each solution at this point. Sometimes the best solution just might not be feasible once you start looking at the resources. We all have constraints that we have to work within. 4. Finally, select a strategy that seems feasible and promising, and implement it. Now, let’s apply this process to the dataset that we’ve been looking at.
Tag team
CB (AL hands out worksheets) Now, it’s your turn! Let’s break into groups of 3-5 people. (even # of groups) (Once we’ve gotten the groups organized…) Let’s start by having you all introduce yourselves to your group. Maybe say your name, where you’re from, and what position you hold at your institution. Ok, now we are going to hand out some data sets and you are going to go through the closing the loop process using the worksheet on the back of your data set. Two groups will look at each set of data, so we’ll be interested to see what different strategies are selected. Once everyone is done, we’ll come back together, and each group will have a chance to present about their process. Adrianna and I will be roaming around to help answer any questions… (We’d like 6 or 9 groups so that each dataset is assigned to at least two groups) (After 20 minutes , I’ll bring the groups back together, and then they will present for 20 minutes .)