Assessing Information Literacy From the Ground Up

Assessing Information Literacy from the Ground Up – Creating a Culture of Assessment
Kristy Motz, Randall Schroeder & Mari Kermit-Canfield
Ferris State University

FLITE Assessment Team

What’s Out There?

Starting Up – Graduate Level Assessments (CMS)
Developed tools two years ago in Blackboard Vista product
Ran assessments through several semesters
Summer 2009: Refined questions and mapped them to Information Literacy Competency Standards for Higher Education

MMBA Pre- and Post-Instruction Assessments
Mixed delivery or blended delivery class
Two Saturdays on-site - remaining course online
Class runs eight weeks = ½ semester
Assessments run in class first and last day
8 questions pre-instruction (first day of class)
5 questions post-instruction (end of class)

Online Assessment in CMS

Pre-Instruction Course Assessment

Post-Instruction Assessment MMBA

MMBA CMS Issues
Surveys or ungraded quiz tools from CMS don’t provide results.
Instructor must allow point credit for results to remain in Gradebook.
Students with first-day user ID/password problems can’t reach the assessment.
Questions must be tweaked to fit CMS question software format.

Fall 2008 MMBA - Fagerman

Summer 2009 MMBA - Hamel

ESPN 505 Special Education
19 students took Pre-Instruction Assessment
15 students took Post-Instruction Assessment
Instructor added 2 point bonus for participation
Instructor used CMS with on-campus course

ESPN 505 PRE AND POST INSTRUCTION ASSESSMENT

NEXT Steps: Real-Time Undergraduate assessments

Vogel Library  FLITE
FLITE’s new department head: Feb 2008
Assessment Team’s new focus on Information Literacy Instrument
Announcement of new James Madison Information Literacy Testing

How To Begin?Vogel Library – WarTburg College

Vogel Library
Seven years of pre-instruction and post-instruction information literacy testing
Based on Information Literacy Competency Standards for Higher Education
Designed question by question to highlight targeted standards

Vogel Library
Developed by
Librarians Randall Schroeder, Jill Gremmels, and Karen Lehman
Dr. Judith Griffith, English Faculty Chair
Led to Faculty Buy-In

Vogel Adaptations to FLITE
FSU Students
FSU Focus and Mission
Information Literacy a new Concept
Implementation and Testing
Faculty Buy-in: 3 volunteers worked with us

Challenges
Conflicts with Scantrons and Testing Center
Need for definitive learning outcomes
Testing on concepts not always covered in classes

What are we Testing?
16. A student who is deciding what topic to research for an assigned paper approaches the reference librarian and says, “This summer was too hot. I wonder if it is because of global warming?”
What might be the thesis statement of this student’s paper?
1. It was too hot this summer.
2. Global warming is causing the mean temperature of the earth to rise, which could have disastrous effects on the ecosystems of the planet.
3. Did global warming cause abnormal heat around the planet?
4. We should buy hybrid vehicles because of global warming.
5. This paper will be about the effects of global warming.

Successes - Survey Questions
Rank the activities on which you spent the most (4) to the least (1) time when you are on the computer.
Searching for specific information related to a school or personal task.
Communicating with individuals through Facebook/My Space, IM, or via e-mail.

Downloading movies, music or graphics.

Surfing the Internet for enjoyment or playing games.

What do we know about our students?

Where Do You Get Your Information?

Which of these have you done within the last year?

How many papers did you write?

Schroeder Words of Wisdom: Hard Lessons Learned
Give yourself permission to fail
Assess to learn things – Don’t just assess to assess
Be sure to have Faculty Buy-In

Starting Again
Involve new staff
Improve librarian buy-In
Imagine: What will happen when developers leave?

What are the Information Literacy Competency Standards?
http://ala.org/ala/mgrps/divs/acrl/standards/standards.pdf

FLITE Information Literacy Assessment Team CREATES LEARNING OUTCOMES
In-depth study of the 5 competency standards
Focus on performance indicators
Curriculum mapping - Group discussion and consensus
7 performance indicators
1-2 indicators from each standard

FLITE Information Literacy Assessment Team CREATES LEARNING OUTCOMES
Group work: Individual questions written for each learning outcome (performance indicator) - 35 questions for assessments
Discussion and consensus: Team selected 34 questions for Question Bank
12 questions are survey questions asked for every assessment
22 questions can be tailored for class

Information Literacy – Round Two Fall 2009
Question Bank built for use with different classes
Individualized assessments built by individual library instructors from Question Bank
Assessments delivered electronically if possible

Survey Monkey
Started in 1999, SurveyMonkey is an online survey tool that enables people of all experience levels to create their own surveys quickly and easily.

FLITE Library’s Assessment Team picked Survey Monkey because:
Seven years of pre-instruction and post-instruction information literacy testing
Based on Information Literacy Competency Standards for Higher Education
Designed question by question to highlight targeted standards

FLITE Library’s Assessment Team picked Survey Monkey because…
History
Team members have previous experience with the program
Easy of use
User needs no statistical background
User needs no knowledge of statistical software
Malleable
Accessible from web; in class or at home
Easily printable for a paper version

Success – Yes, You Can Too
We went beyond the “What if’s” and created a project.
We gave ourselves permission to fail.
We didn’t have to be perfect – we just had to start.
We created a Culture of Assessment: we refined what we did and moved on.

Next Steps
Post-instruction assessment – Dec 2009
Refine questions based on data input
Expand to other faculty and departments
Adapt procedures for Distance Learning
Write new assessments for upper-level classes

Accomplished Major Goal
Closed the loop
Curriculum mapping: using assessment data to revise and improve instruction curriculum
Keep assessing – TracDat

Culture of Assessment
Helps us inform faculty where we want them to go with us
Allows us to create customized instructions across the curriculum
Showcases FLITE – the Library is one of the early reporters of assessment data on campus

Just Start!
Get Faculty Buy-in
Give Yourself Permission to Fail
Take the Plunge and Begin

Questions?
Kristen Motz motzk@ferris.edu
Randall Schroeder randallschroeder@ferris.edu
Mari Kermit-Canfield kermitt@ferris.edu

Assessing Information Literacy From the Ground Up

by freeweb on Nov 03, 2009

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 6

Statistics

Assessing Information Literacy From the Ground Up — Presentation Transcript