KNOWING WHAT WE DON'T KNOW:
STUDENT SELF-ASSESSMENT AND LIBRARY USE
Denyse Rodrigues
Liaison Librarian, E-learning Librari...
The Context
Mount Saint Vincent University
• Located in Halifax, NS
• Approximately 4000 students, primarily undergraduate...
What we know so far
(from the literature & observation)
• Students rate their own research skill quite high (Raven, 2012)
...
What I want to know
• A bench mark of students information literacy/research
readiness skills using an assessment tool
• I...
Method
1. Self assessment: undergraduate students invited to complete an
online self assessment at the beginning of the te...
Participating Students
• 93% female
• 42% first-year;
37% second-year;
21% third-year plus
• 60% age 20 or less;
27% age 2...
Research Readiness Self Assessment
• Developed by Casey & Ivanitskaya at Central Michigan University, RRSA
consists of ~50...
Sample feedback
The following categories measure your experience, attitudes and
beliefs:
Research and library experience -...
Sample feedback
Your score has indicated that you tend to use general search engines on
the Internet for information. Surf...
Undergraduates
Number of Completed Records: 160
95% of students above 50% in skill levels
Category Min Score Max Score
Ave...
First year students
Number of Completed
Records: 67
90% of students above 50% in skill levels
Category Min Score Max Score...
• One can retrieve the most documents in an online library
catalog by searching for:
• Training and turnover
• Training no...
Faculty feedback
• WOMS 1110 – 60 students
• Feedback on bibliographies compared to previous years
• Decreased use of open...
Interviews
• Research Readiness Self Assessment
• Did you read the feedback that was sent to you by the self assessment
ea...
Interview group
Number of Completed Records: 8
4 on campus/ 4 distance
Category Min Score Max Score
Average
Score
Points
P...
Undergraduates
Number of Completed Records: 160
Category Min Score Max Score
Average
Score
Points
Possible
%
Obtaining
inf...
Interview group
Number of Completed Records: 8
4 on campus/ 4 distance
Category Min Score Max Score
Average
Score
Points
P...
RRSA feedback
• Students recalled receiving the feedback but no one
reported following any of the links for further inform...
I vaguely remember it. I know I did receive it and I did read
it over, but it’s been a couple months so it is not fresh in...
• “Not really, that was a long time. I remember it was very
big. I remember receiving some criticism on one section .
. . ...
Challenges: research strategy
• Finding Articles/Narrowing Results
“Finding articles. Usually that’s kind of the part that...
Positive Faculty Influence
• My psychology professor is very specific. . . there these
certain articles that she wants us ...
Challenges - citation
• Yeah I don’t know how to cite properly I don’t think.
• I cite my sources but with a less formal s...
Challenges: school/work/life balance
• I am usually working and going to school and all of this
homework and assignments y...
Promotion of the library to distance
students
• Distance students less aware of library resources and
services
“I did not ...
Conclusions
• Students rate their research skills higher than they
actually are, but most possess basic research skills
• ...
Food for thought
Students do not see research as a process but think of it as
the tools they use to find articles.
Resnis,...
Upcoming SlideShare
Loading in …5
×

Knowing what we don't know rodrigues-apqla2013

374 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
374
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Knowing what we don't know rodrigues-apqla2013

  1. 1. KNOWING WHAT WE DON'T KNOW: STUDENT SELF-ASSESSMENT AND LIBRARY USE Denyse Rodrigues Liaison Librarian, E-learning Librarian Mount Saint Vincent University Library Thursday May 9, 2013 81st ABQLA Annual Conference (2013) Assessment: Gaining insight, improving services Évaluer : mieux connaître pour mieux servir
  2. 2. The Context Mount Saint Vincent University • Located in Halifax, NS • Approximately 4000 students, primarily undergraduate • Local students: 75% from NS • 80% female student body
  3. 3. What we know so far (from the literature & observation) • Students rate their own research skill quite high (Raven, 2012) • Faculty and librarians rate students as having low research skills (Raven, 2012) • To support students, the Mount Library provides a combination of: • help guides/online videos via the library website • reference assistance & research appointments • one-shot workshops in collaboration with faculty • credit course LIBR 2100: Introduction to Research Skill in the Information Age
  4. 4. What I want to know • A bench mark of students information literacy/research readiness skills using an assessment tool • If there is a gap between actual skill and perception of skill, would knowing about the gap motivate students to utilize library supports.
  5. 5. Method 1. Self assessment: undergraduate students invited to complete an online self assessment at the beginning of the term. • Group 1 • Invited to completed the survey prior to or during a one-shot workshop. (with option to opt out of research project) • 126 invited - 102 self assessments • Group 2 • Invited to complete the survey separate from the offering or provision of a workshop • 307 invited - 58 self assessments • Response rate: .36 2. Semi-structured interview: At the end of the term, participants invited to a semi structured interview regarding their research skills and the library services and resources made available to them during the term. • 8 interviews
  6. 6. Participating Students • 93% female • 42% first-year; 37% second-year; 21% third-year plus • 60% age 20 or less; 27% age 21-30; 13% age 30+ • 46% only campus 34% blended campus and distance courses 20% only distance • All disciplines represented
  7. 7. Research Readiness Self Assessment • Developed by Casey & Ivanitskaya at Central Michigan University, RRSA consists of ~50 quiz type questions to objectively measure information literacy competencies • Assess student performance in the knowledge and skill areas of the ACRL Informational Literacy Competency Standards for Higher Education • Obtaining Information • Evaluating Information • Understanding Plagiarism • Survey student beliefs and behaviors pertaining to information seeking activities • Research experience • Perceived skills • Browsing the Internet • Provides direct feedback to students with suggestions for improvement of skills
  8. 8. Sample feedback The following categories measure your experience, attitudes and beliefs: Research and library experience - 17 / 34 - 50% Browsing the Internet - 25 / 50 - 50% Perceived research skills - 20 / 40 - 50% The following categories measure your knowledge and skills: Obtaining information - 15/ 30 - 50% Evaluating information - 6 / 12 - 50% Understanding of plagiarism - 7 / 14 - 50% Overall score, an indicator of your research readiness - 28 / 56 - 50%
  9. 9. Sample feedback Your score has indicated that you tend to use general search engines on the Internet for information. Surfing the Web can sometimes give you relevant information but should not be trusted for the majority of research you do because it is not always accurate or reliable. The Mount Library provide access to books and articles through licensed databases. These items are paid for by the Library and are available to students, faculty, and staff. Licensed databases offer access to scholarly journal articles and other sources of information that students should be reading when working on university level papers. * Visit our subject guides for databases in your area of study - http://bit.ly/msvulibraryguides * For more information on evaluating websites, please see http://bit.ly/web-site-evaluation
  10. 10. Undergraduates Number of Completed Records: 160 95% of students above 50% in skill levels Category Min Score Max Score Average Score Points Possible % Obtaining information 5 24 15.78 24 66 Evaluating information 1 10 6.13 12 51 Understanding of plagiarism 6 16 13.67 18 76 Browsing the Internet 0 45.2 18.5 50 37 Perceived research skills 8 35 25.32 35 72 Research and library experience 0 31 15.56 40 39
  11. 11. First year students Number of Completed Records: 67 90% of students above 50% in skill levels Category Min Score Max Score Average Score Points Possible % Obtaining information 7 23 15.48 24 64 Evaluating information 1 10 5.78 12 48 Understanding of plagiarism 6 16 13.58 18 75 Browsing the Internet 0 42 18.39 50 37 Perceived research skills 8 32 25.32 35 72 Research and library experience 0 25 12.85 40 33
  12. 12. • One can retrieve the most documents in an online library catalog by searching for: • Training and turnover • Training not turnover • Training or turnover • Not training turnover • Or turnover and training 69% 31% Chart Title incorrect correct
  13. 13. Faculty feedback • WOMS 1110 – 60 students • Feedback on bibliographies compared to previous years • Decreased use of open Internet websites • Room for improvement with evaluating information sources • Citation and bibliographies completed but contained errors in APA style
  14. 14. Interviews • Research Readiness Self Assessment • Did you read the feedback that was sent to you by the self assessment earlier in the term? • Did you follow any of the links? • Have you ever watched any of the help videos on the Library website? • Did you make any changes to the way you conduct research after receiving the self assessment report? • Were the suggested resources helpful? • Research What are your research strategies? • What are the challenges you encounter when conducting research for your course assignments? • Library • How can the Library better support you with your research?
  15. 15. Interview group Number of Completed Records: 8 4 on campus/ 4 distance Category Min Score Max Score Average Score Points Possible % Obtaining information 10 22 17.63 24 73 Evaluating information 2 9 6.63 12 55 Understanding of plagiarism 8 16 13.38 18 74 Browsing the Internet 2.58 38.38 22.22 50 44 Perceived research skills 8 32.88 24.8 35 62 Research and library experience 3 24 11 40 31
  16. 16. Undergraduates Number of Completed Records: 160 Category Min Score Max Score Average Score Points Possible % Obtaining information 5 24 15.78 24 66 Evaluating information 1 10 6.13 12 51 Understanding of plagiarism 6 16 13.67 18 76 Browsing the Internet 0 45.2 18.5 50 37 Perceived research skills 8 35 25.32 35 72 Research and library experience 0 31 15.56 40 39
  17. 17. Interview group Number of Completed Records: 8 4 on campus/ 4 distance Category Min Score Max Score Average Score Points Possible % Obtaining information 10 22 17.63 24 73 Evaluating information 2 9 6.63 12 55 Understanding of plagiarism 8 16 13.38 18 74 Browsing the Internet 2.58 38.38 22.22 50 44 Perceived research skills 8 32.88 24.8 35 62 Research and library experience 3 24 11 40 31
  18. 18. RRSA feedback • Students recalled receiving the feedback but no one reported following any of the links for further information in the RRSA feedback
  19. 19. I vaguely remember it. I know I did receive it and I did read it over, but it’s been a couple months so it is not fresh in my mind, but I do remember and I remember I don’t think I scored too poorly on it. I haven’t really ventured out to use any of the tips to help me with my research. “I remember I checked it and I looked at it and I kind of like went through it really fast but I didn’t really do any more . . . There was something I did absolutely terrible on but I can’t even remember what it was [laughs] That’s not much help I guess. What was it? [uhhhhhm]. I don’t know but I got a 1 out of however much.
  20. 20. • “Not really, that was a long time. I remember it was very big. I remember receiving some criticism on one section . . . and I knew it was true anyway. . . and I can’t remember what it was.” • “it was... was a really long email I scanned it and moved on. I like just one or two lines. That’s it. I really need to know the gist of what I’m reading in one or two lines at the top. I do not like to be inundated with information. Unless I go searching for it I do not want to be confronted by it. “
  21. 21. Challenges: research strategy • Finding Articles/Narrowing Results “Finding articles. Usually that’s kind of the part that I am kind of not very good at. So usually like if there is a huge number of articles and I find something that I like I usually look under those words, you know at what words they show or whatever, and then I’ll type those in so I can get smaller results.” • Evaluating information was not mentioned explicitly “Well, … a lot of the time … the articles are really, really long and I don’t have the time to like read the whole thing so ones that are kind of shorter I tend to like go towards I guess, and the ones that seem more relevant to the topic that I want to focus on really, I guess.”
  22. 22. Positive Faculty Influence • My psychology professor is very specific. . . there these certain articles that she wants us to actually find on the database so she actually tells me that this is where and what I want you look up on the library database and do research on it so [laughs] that’s really it. • My history professor suggested about going on and finding the particular reading that we needed for an assignment and that is when I basically discovered the library. So, I’ve used it probably, if not every day then every other day I’ve been on the website.
  23. 23. Challenges - citation • Yeah I don’t know how to cite properly I don’t think. • I cite my sources but with a less formal setup, like I would just list them at the end.
  24. 24. Challenges: school/work/life balance • I am usually working and going to school and all of this homework and assignments you have to do and then doing the research on top of it is sometimes unbearable. I keep a calendar and I budget my time. If I allow myself 2 hours this afternoon to get materials I need, then I have to stick within that time to ensure that I use those hours efficiently to get what I need for my assignments. I wouldn’t discount looking further but I would definitely would choose the quicker options when it comes to searching.
  25. 25. Promotion of the library to distance students • Distance students less aware of library resources and services “I did not realize what the library had to offer until I did this [self-assessment and interview]. Good to know that it is just not a ‘resource’ but that it is actually going to help me succeed”
  26. 26. Conclusions • Students rate their research skills higher than they actually are, but most possess basic research skills • The evaluation of information needs to be better integrated into research instruction. • Self-assessment was most useful during in-class instruction for student engagement. • The personalized self-assessment information does not appear to have motivated any of the undergraduate students to explore the Library’s help resources.
  27. 27. Food for thought Students do not see research as a process but think of it as the tools they use to find articles. Resnis, E., Gibson, K., Hartsell-Gundy, A., & Misco, M. (2010). Information literacy assessment: A case study at Miami university. New Library World, 111, 7, 287-301. There are discrepancies between what students know and what they do regarding source selection. Kim, K.-S., & Sin, S.-C. (2011). Selecting quality sources: Bridging the gap between the perception and use of information sources. Journal of Information Science,37, 2, 178-188.

×