Preliminary Results Of The Fall 2008 Intd 105 Assessment


Published on

Published in: Technology, Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Preliminary Results Of The Fall 2008 Intd 105 Assessment

  1. 1. Preliminary Results from the Fall 2008 INTD 105 Assessment Pretest vs. Posttest
  2. 2. About these results <ul><li>Preliminary </li></ul><ul><li>No analysis (yet) of wrong answers </li></ul>
  3. 3. Goal 1, Objective 1 <ul><li>Students will be able to demonstrate an understanding of the organization of the library website </li></ul>
  4. 4. Goal 1, Objective 1 <ul><li>Students still unsure of the existence of or how to get to our Resources by Subject pages. </li></ul>Pretest Posttest Correctly found the Business RBS page 40% 46% Found the English RBS page (multichoice) 48% 56% Citing sources page 66% 75%
  5. 5. Goal I, Objective 2 <ul><li>Students will be able to describe the purpose of: </li></ul><ul><ul><li>IDS </li></ul></ul><ul><ul><li>SRC </li></ul></ul><ul><ul><li>Reference Desk </li></ul></ul><ul><ul><li>Circulation Desk </li></ul></ul><ul><ul><li>Citation Style Guides </li></ul></ul>
  6. 6. Goal 1, Objective 2 <ul><li>Students can generally pick out the appropriate service from a list. </li></ul><ul><li>This does not imply that students are already aware that such a service exists. </li></ul><ul><li>We see big increases in the number of students who recognize IDS </li></ul>Pretest Posttest IDS 70% 95% STC 88% 93% Reference Desk 85% 93% Circulation Desk 86% 92%
  7. 7. Goal 2, Objective 1 <ul><li>Students will be able to define GLOCAT and the types of material it contains </li></ul>
  8. 8. Goal 2, Objective 1 <ul><li>Students can figure out pretty easily that they need to use GLOCAT to find books, but they have major problems understanding that you can’t use it to find journal articles. </li></ul>Pretest Posttest What can you find in GLOCAT 11% 12% What database to use to find Harry Potter 76% 92%
  9. 9. Goal 2, Objective 2 <ul><li>Students will be able to find books by topic author or title </li></ul>
  10. 10. Goal 2, Objective 2 <ul><li>In the pretest, 14% of students left the second question blank, as opposed to 4% in the posttest. </li></ul><ul><li>Most of the wrong answers were off topic. They were able to find a book, but it didn’t include one or more aspects of the topic. </li></ul>Pretest Posttest Correct call number 47% 54% Find a book about “running a small business out of your home” 44% 36%
  11. 11. Goal 2, Objective 3 <ul><li>Students will be able to interpret results lists and item records </li></ul><ul><li>9 questions about this on the assessment </li></ul>
  12. 12. Goal 2, Objective 3 <ul><li>In general, students can interpret our GLOCAT item records with ease from the beginning. </li></ul><ul><li>We see a big increase in the awareness that the call number will help them locate the book in Milne library. </li></ul><ul><li>They are not making the connection between the GLOCAT results screen and library floorplans. </li></ul>Pretest Posttest What info do you need to find the book in the library 58% 78% Which item is on the lower level 43% 46% Checked out? 89% 94% ID the author? 99% 98% ID subject? 89% 91% Type of resource 96% 96% Where to get full text? 96% 92%
  13. 13. Goal 2, Objective 4 <ul><li>Students will be able to demonstrate an understanding of where books and journals are physically located in Milne library </li></ul>
  14. 14. Goal 2, Objective 4 <ul><li>I don’t think they are making it to our library floor plans. I think the increase in question 2 comes from students knowledge. </li></ul>Pretest Posttest Given a results screen, which item is on the lower level? 43% 46% Given 4 options, which is NOT upstairs? 59% 77%
  15. 15. Goal 3, Objective 1 <ul><li>Students will be able to select an appropriate periodical database for a given task </li></ul>
  16. 16. Goal 3, Objective 1 <ul><li>Students are aware that Google, Yahoo!, etc. aren’t acceptable for scholarly research, but they don’t seem to know what is appropriate. </li></ul><ul><li>Open ended article question: In the pretest, 35% of students left the question blank, versus 16% in the posttest. </li></ul><ul><li>Students were not making the best choices in their choice of database, although they did slightly better in the posttest. </li></ul>Pretest Posttest Library recommended database for English 48% 56% Which search engine is appropriate to find scholarly sources 90% 91%
  17. 17. Resources Used to find an article Pretest Percent Posttest Percent Multisearch 12% 30% Proquest 14% 27% JSTOR 7% 27% [Blank] 35% 16% WorldCat 1% 10% Health and Wellness Resource Center 5% 8% Pubmed 3% 8% CINAHL Plus with full text 2% 8% Academic Search Premier 1% 6% Google Scholar 3% 4% Journal Finder 1% 4% Ebsco 3% 3% Resources Used to find an article Pretest Percent Posttest Percent Advanced Multisearch 0% 3% ScienceDirect 0% 2% Medline 1% 1% Google 1% 1% Psycinfo 1% 1% Scopus 0% 1% SAGE journals online 2% 0% Search Engine 1% 0% medicine and health database 1% 0% 1% 0% resources from A-Z 1% 0% search 1% 0% Wikipedia 1% 0%
  18. 18. Goal 3, Objective 2 <ul><li>Students will be able to find appropriate articles on a given topic </li></ul><ul><li>Open ended questions </li></ul><ul><ul><li>One asked for an article about the portrayal of Native Americans in film using Proquest </li></ul></ul><ul><ul><li>The other asked for an article about the effectiveness of the HPV vaccine </li></ul></ul>
  19. 19. Goal 3, Objective 2 <ul><li>Find an article about the portrayal of Native Americans in Film. </li></ul><ul><li>While the percent of students who found actual articles went down, the percent who found book reviews went way up. Almost all of the book reviews were on-topic. </li></ul>Reason for result Pretest Percent Posttest Percent Book Review 50% 77% Incomplete Citation 4% 0% OK 5% 0% Good 6% 4% No Answer Given 24% 0% Off Topic 11% 9%
  20. 20. Goal 3, Objective 2 <ul><li>Find a scholarly article about the effectiveness of the new HPV vaccine. </li></ul><ul><li>Total number of students who found something Scholarly went up significantly, as did the number of students who even attempted the question. </li></ul>*Difference in “OK” percent has more to do with my evaluation of whether a source was on topic or not. I need to go back and re-do this. Reasons for result Pretest Percent Posttest Percent Book 1% 2% OK* 1% 26% Incomplete Citation 12% 11% Not Scholarly 14% 12% Scholarly 33% 25% No Answer 35% 9% Government Report 0% 5% Off topic 4% 10%
  21. 21. Goal 3, Objective 3 <ul><li>Students will be able to determine the availability of full text articles online, in print at Milne Library, and through IDS </li></ul><ul><li>Mostly a test of the Journal Finder feature </li></ul>
  22. 22. Goal 3, Objective 3 <ul><li>It seems our Find Journals feature isn’t as straightforward as I originally thought. </li></ul><ul><li>A semester at college seems to significantly increase students’ ability to use the site, but the ability to figure out which online resource to use is still lacking. </li></ul>Pretest (21%) and Posttest (26%) students all want to go to JSTOR to find the online volume, even though it isn’t listed anywhere in the screen shot. Pretest Posttest Print Volume 58% 71% Online Volume 48% 48% IDS 49% 70%
  23. 23. Challenge Pretest Posttest Notes Citation 6% 5% Determining appropriateness of source 12.0% 7.4% Often in conjunction with finding material Evaluation 4.0% 7.4% Sorting through search results, finding &quot;relevant&quot; material Finding articles/books/sources 20.0% 22.1% In the post-test, we are more likely to see students mention finding material &quot;on a particular topic&quot; or &quot;on an obscure topic&quot; Finding help 2.0% 0.5% Online or in person Finding print materials 2.0% 2.1% Sometimes finding print, sometimes finding things &quot;in the library&quot; Getting full text 0.0% 1.1% Frustration at getting full text when the student thinks it is available Information overload 3.0% 2.6% Internet searching 2.0% 0.5% Interpretation 0.0% 4.7% Interpreting the material they find. Finding information in long articles or books Item availability 0.0% 8.9% Frustration at the speed of IDS, lack of knowledge of how it works, inability to find things &quot;here&quot; Laziness 0.0% 2.1% Mostly expressed as a desire to find things quickly, one student admitted laziness No Answer 22.0% 8.4% Search strategy 5.0% 13.7% Mostly expressed as &quot;keywords to use&quot; Time 2.0% 1.1% Topic selection 0.0% 1.1% Using a database 1.0% 2.1% JSTOR and GLOCAT were mentioned specifically Where to start 8.0% 2.1% Which resource to use 11.0% 13.2%