Terra incognita with no GPS: A case study approach to mapping assessment


Published on

Amy Hofer & Margot Hanson. From Loex-of-the-West 2010 conference program: "We need evidence of our value to justify the library budget, gain buy-in from faculty, and please accreditors, but the process of gathering meaningful evidence can be quite a challenge. After all, were you trained to do this kind of research? Neither were we, but the Assessment Coordinator that our university hired in preparation for a WASC visit guided us through our study design. Drawing on this relationship with a neutral expert outside the library, two business librarians at Golden Gate University designed an effective learning assessment of our instruction to our school’s intensive ESL program. We’ll use the valuable lessons we learned as a starting point for small-group exploration of some real-world assessment territory."

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Terra incognita with no GPS: A case study approach to mapping assessment

  1. 1. TERRA INCOGNITA WITH NO GPS: A case study approach to mapping assessment Amy Hofer, Distance Learning Librarian, Portland State University Margot Hanson, Web Services Librarian, Golden Gate University LOEX-of-the-West 2010, Mount Royal University, Calgary, Alberta
  2. 2. ABOUT US! ‣Golden Gate University ‣The IL landscape ‣Our assessment needs
  3. 3. ABOUT YOU! ‣Hello my name is... ‣Your school ‣Pressing assessment issues (past, present, or future)
  4. 4. PROGRAM EVALUATION Study your program to find out whether it’s meeting your goals
  5. 5. LEARNING ASSESSMENT Study whether students gained knowledge, skills, and abilities as a result of your instruction
  6. 6. PLEASE DISCUSS ‣Think about an assessment project at your organization. ‣Learning assessment or program evaluation? ‣Right tool for the job?
  7. 7. LESSONS FROM LISA ‣ For what general purpose is the evaluation being conducted? ‣ What are the specific goals of the study? ‣ Who is the audience for the research findings? ‣ What kinds of information are needed to make the decisions you need to make? ‣ What data collection method will be appropriate?
  8. 8. OUR STUDY
  9. 9. POPULATIONS: FALL 2009 PLUS Undergraduate ‣ English language learners ‣ 43% ELL in our sample ‣ 8 librarian visits ‣ 3 librarian UG class visits out of 61 UG classes offered Fall 09 ‣ 2 librarians per visit (none in our sample) ‣ 18.5 hours total ‣ 30-60 minutes per visit
  10. 10. METHODS PLUS Students Undergraduate Students ‣ 43 completed tests before research instruction (paper) ‣ 51 completed tests (paper and online) ‣ 40 completed tests after research instruction (paper) ‣ Sample drawn from 9 classes with agreeable faculty ‣ 39 writing samples
  11. 11. OUR AREAS OF INQUIRY ‣ A. Dostudents identify appropriate sources for their research need? http://tinyurl.com/lotw2010 of the ‣ B. Do students ask a great question that fits the scope assignment and requires research and analysis to answer? ‣ C. Do students demonstrate academic integrity by correctly citing sources? ‣ D. Do students use the library’s resources and services as part of the research/writing process? ‣ E. Qualitative info about students
  12. 12. Didn't Get It Got It Got It & Ran With It! •Uses a range of formats, including reports, articles and •Uses only one format  •Uses a range of formats, websites including reports, articles and Appropriate •Not relevant or authoritative websites •Includes resources only available Sources •Heavy reliance on one or two •Uses only resources presented from the library, but went beyond You don’t have to read this - sources resources presented in class in class •Sources are relevant and substantive it’s online! •Topic too narrow or too broad •Research question is of •Uses critical thinking and creative approach to topic, which is of appropriate scope for assignment appropriate scope for assignment Topic • Uses report format, as a simple •Includes original analysis •Research question requires Development original analysis statement of facts, missing original •Question can be answered with analysis resources presented in class •Question requires additional sources beyond what was presented in class •Errors in APA formatting •APA correctly and consistently •Incorrect use of paraphrasing, •Consistent APA formatting in- used for in-text and reference list quotations, or summarizing Citation •Quotes are dumped into paper text and reference list citations and not incorporated into analysis • •Citations are distributed evenly Originality report reveals less than 10% exact match throughout paper, contribute to •Originality report reveals analysis, and support conclusions problems with exact matches
  13. 13. FINDINGS
  14. 14. OMG, data!
  15. 15. Most Improved Players PLUS pretest PLUS posttest Undergrad 70% 60% 66% 60% 50% 54% Test Score 40% 30% 20% 10%
  16. 16. What the PLUS work samples told us Got it & ran with it! Got it Didn’t get it Appropriate sources Topic development Citation 39 10 8 8 Student Papers 31 23 13 18 21 16 8 18 11 10
  17. 17. Undergrads: How many times during the semester do you get help from a librarian? 2% 6% 22% More than 5 times 2-5 times 46% One time Never I don’t know 24%
  18. 18. Do you think that the library and its services contribute to your academic success at GGU? Yes Somewhat Maybe No I don’t know Undergrad PLUS 10% 2% 5% 13% 18% 51% 20% 83%
  19. 19. Do students know what they know? 20 15 Students 10 5 0 0 1 2 3 4 5 6 7 8 9 10 Test Scores My skills are excellent My skills are good My skills are fair My skills are poor I don’t know
  20. 20. NEXT STEPS ‣Connections to your own situation or small-group discussions? ‣Strategic outreach - getting a foot in the door? ‣Questions for us?
  21. 21. Image sources: http://www.flickr.com/photos/telmo32/2967379555 http://www.flickr.com/photos/rhiannonstone/3782706061 http://www.wordle.net http://www.flickr.com/photos/princessashley/1546575032 http://www.flickr.com/photos/gadl/320300354 http://www.flickr.com/photos/mcaven/4319012152 http://www.flickr.com/photos/huffstutterrobertl/3269126061/ http://www.flickr.com/photos/quinnanya/3183174451/
  22. 22. Thank you! Amy Hofer Margot Hanson ahofer@pdx.edu mhanson@ggu.edu