Knowing what we don't know rodrigues-apqla2013

KNOWING WHAT WE DON'T KNOW:
STUDENT SELF-ASSESSMENT AND LIBRARY USE
Denyse Rodrigues
Liaison Librarian, E-learning Librarian
Mount Saint Vincent University Library
Thursday May 9, 2013
81st ABQLA Annual Conference (2013)
Assessment: Gaining insight, improving services
Évaluer : mieux connaître pour mieux servir

The Context
Mount Saint Vincent University
• Located in Halifax, NS
• Approximately 4000 students, primarily undergraduate
• Local students: 75% from NS
• 80% female student body

What we know so far
(from the literature & observation)
• Students rate their own research skill quite high (Raven, 2012)
• Faculty and librarians rate students as having low
research skills (Raven, 2012)
• To support students, the Mount Library provides a
combination of:
• help guides/online videos via the library website
• reference assistance & research appointments
• one-shot workshops in collaboration with faculty
• credit course
LIBR 2100: Introduction to Research Skill in the Information Age

What I want to know
• A bench mark of students information literacy/research
readiness skills using an assessment tool
• If there is a gap between actual skill and perception of
skill, would knowing about the gap motivate students to
utilize library supports.

Method
1. Self assessment: undergraduate students invited to complete an
online self assessment at the beginning of the term.
• Group 1
• Invited to completed the survey prior to or during a one-shot workshop. (with option to opt out of
research project)
• 126 invited - 102 self assessments
• Group 2
• Invited to complete the survey separate from the offering or provision of a workshop
• 307 invited - 58 self assessments
• Response rate: .36
2. Semi-structured interview: At the end of the term, participants invited
to a semi structured interview regarding their research skills and the
library services and resources made available to them during the term.
• 8 interviews

Participating Students
• 93% female
• 42% first-year;
37% second-year;
21% third-year plus
• 60% age 20 or less;
27% age 21-30;
13% age 30+
• 46% only campus
34% blended campus and distance courses
20% only distance
• All disciplines represented

Research Readiness Self Assessment
• Developed by Casey & Ivanitskaya at Central Michigan University, RRSA
consists of ~50 quiz type questions to objectively measure information
literacy competencies
• Assess student performance in the knowledge and skill areas of the ACRL
Informational Literacy Competency Standards for Higher Education
• Obtaining Information
• Evaluating Information
• Understanding Plagiarism
• Survey student beliefs and behaviors pertaining to information seeking
activities
• Research experience
• Perceived skills
• Browsing the Internet
• Provides direct feedback to students with suggestions for improvement of
skills

Sample feedback
The following categories measure your experience, attitudes and
beliefs:
Research and library experience - 17 / 34 - 50%
Browsing the Internet - 25 / 50 - 50%
Perceived research skills - 20 / 40 - 50%
The following categories measure your knowledge and skills:
Obtaining information - 15/ 30 - 50%
Evaluating information - 6 / 12 - 50%
Understanding of plagiarism - 7 / 14 - 50%
Overall score, an indicator of your research readiness - 28 / 56 - 50%

Sample feedback
Your score has indicated that you tend to use general search engines on
the Internet for information. Surfing the Web can sometimes give you
relevant information but should not be trusted for the majority of research you
do because it is not always accurate or reliable.
The Mount Library provide access to books and articles through licensed
databases. These items are paid for by the Library and are available to
students, faculty, and staff. Licensed databases offer access to scholarly
journal articles and other sources of information that students should be
reading when working on university level papers.
* Visit our subject guides for databases in your area of study -
http://bit.ly/msvulibraryguides
* For more information on evaluating websites, please see
http://bit.ly/web-site-evaluation

Undergraduates
Number of Completed Records: 160
95% of students above 50% in skill levels
Category Min Score Max Score
Average
Score
Points
Possible
%
Obtaining
information
5 24 15.78 24 66
Evaluating
information
1 10 6.13 12 51
Understanding of
plagiarism
6 16 13.67 18 76
Browsing the
Internet
0 45.2 18.5 50 37
Perceived research
skills
8 35 25.32 35 72
Research and
library experience
0 31 15.56 40 39

First year students
Number of Completed
Records: 67
90% of students above 50% in skill levels
Average
Score
Points
Possible
%
Obtaining
information
7 23 15.48 24 64
Evaluating
information
1 10 5.78 12 48
Understanding of
plagiarism
6 16 13.58 18 75
Browsing the
Internet
0 42 18.39 50 37
Perceived research
skills
8 32 25.32 35 72
Research and
library experience
0 25 12.85 40 33

• One can retrieve the most documents in an online library
catalog by searching for:
• Training and turnover
• Training not turnover
• Training or turnover
• Not training turnover
• Or turnover and training
69%
31%
Chart Title
incorrect correct

Faculty feedback
• WOMS 1110 – 60 students
• Feedback on bibliographies compared to previous years
• Decreased use of open Internet websites
• Room for improvement with evaluating information
sources
• Citation and bibliographies completed but contained
errors in APA style

Interviews
• Research Readiness Self Assessment
• Did you read the feedback that was sent to you by the self assessment
earlier in the term?
• Did you follow any of the links?
• Have you ever watched any of the help videos on the Library website?
• Did you make any changes to the way you conduct research after
receiving the self assessment report?
• Were the suggested resources helpful?
• Research What are your research strategies?
• What are the challenges you encounter when conducting research for
your course assignments?
• Library
• How can the Library better support you with your research?

Interview group
4 on campus/ 4 distance
Average
Score
Points
Possible
%
Obtaining
information
10 22 17.63 24 73
Evaluating
information
2 9 6.63 12 55
Understanding of
plagiarism
8 16 13.38 18 74
Browsing the
Internet
2.58 38.38 22.22 50 44
Perceived research
skills
8 32.88 24.8 35 62
Research and
library experience
3 24 11 40 31

Undergraduates
Average
Score
Points
Possible
%
Obtaining
information
5 24 15.78 24 66
Evaluating
information
1 10 6.13 12 51
Understanding of
plagiarism
6 16 13.67 18 76
Browsing the
Internet
0 45.2 18.5 50 37
Perceived research
skills
8 35 25.32 35 72
Research and
library experience
0 31 15.56 40 39

RRSA feedback
• Students recalled receiving the feedback but no one
reported following any of the links for further information in
the RRSA feedback

I vaguely remember it. I know I did receive it and I did read
it over, but it’s been a couple months so it is not fresh in my
mind, but I do remember and I remember I don’t think I
scored too poorly on it. I haven’t really ventured out to use
any of the tips to help me with my research.
“I remember I checked it and I looked at it and I kind of like
went through it really fast but I didn’t really do any more . . .
There was something I did absolutely terrible on but I can’t
even remember what it was [laughs] That’s not much help I
guess. What was it? [uhhhhhm]. I don’t know but I got a 1
out of however much.

• “Not really, that was a long time. I remember it was very
big. I remember receiving some criticism on one section .
. . and I knew it was true anyway. . . and I can’t remember
what it was.”
• “it was... was a really long email I scanned it and moved
on. I like just one or two lines. That’s it. I really need to
know the gist of what I’m reading in one or two lines at the
top. I do not like to be inundated with information. Unless I
go searching for it I do not want to be confronted by it. “

Challenges: research strategy
• Finding Articles/Narrowing Results
“Finding articles. Usually that’s kind of the part that I am kind of
not very good at. So usually like if there is a huge number of
articles and I find something that I like I usually look under those
words, you know at what words they show or whatever, and then
I’ll type those in so I can get smaller results.”
• Evaluating information was not mentioned explicitly
“Well, … a lot of the time … the articles are really, really long and I
don’t have the time to like read the whole thing so ones that are
kind of shorter I tend to like go towards I guess, and the ones that
seem more relevant to the topic that I want to focus on really, I
guess.”

Positive Faculty Influence
• My psychology professor is very specific. . . there these
certain articles that she wants us to actually find on the
database so she actually tells me that this is where and
what I want you look up on the library database and do
research on it so [laughs] that’s really it.
• My history professor suggested about going on and
finding the particular reading that we needed for an
assignment and that is when I basically discovered the
library. So, I’ve used it probably, if not every day then
every other day I’ve been on the website.

Challenges - citation
• Yeah I don’t know how to cite properly I don’t think.
• I cite my sources but with a less formal setup, like I would
just list them at the end.

Challenges: school/work/life balance
• I am usually working and going to school and all of this
homework and assignments you have to do and then
doing the research on top of it is sometimes unbearable. I
keep a calendar and I budget my time. If I allow myself 2
hours this afternoon to get materials I need, then I have to
stick within that time to ensure that I use those hours
efficiently to get what I need for my assignments. I
wouldn’t discount looking further but I would definitely
would choose the quicker options when it comes to
searching.

Promotion of the library to distance
students
• Distance students less aware of library resources and
services
“I did not realize what the library had to offer until I did this
[self-assessment and interview]. Good to know that it is just
not a ‘resource’ but that it is actually going to help me
succeed”

Conclusions
• Students rate their research skills higher than they
actually are, but most possess basic research skills
• The evaluation of information needs to be better
integrated into research instruction.
• Self-assessment was most useful during in-class
instruction for student engagement.
• The personalized self-assessment information does not
appear to have motivated any of the undergraduate
students to explore the Library’s help resources.

Food for thought
Students do not see research as a process but think of it as
the tools they use to find articles.
Resnis, E., Gibson, K., Hartsell-Gundy, A., & Misco, M. (2010). Information literacy assessment: A
case study at Miami university. New Library World, 111, 7, 287-301.
There are discrepancies between what students know and
what they do regarding source selection.
Kim, K.-S., & Sin, S.-C. (2011). Selecting quality sources: Bridging the gap between the perception
and use of information sources. Journal of Information Science,37, 2, 178-188.

Knowing what we don't know rodrigues-apqla2013

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Knowing what we don't know rodrigues-apqla2013

Similar to Knowing what we don't know rodrigues-apqla2013 (20)

More from ABQLA2013

More from ABQLA2013 (8)

Recently uploaded

Recently uploaded (20)

Knowing what we don't know rodrigues-apqla2013