Assessing Information Literacy from the Ground Up – Creating a Culture of Assessment Kristy Motz, Randall Schroeder & Mari Kermit-CanfieldFerris State University

FLITE Assessment Team

What’s Out There?

Starting Up – Graduate Level Assessments (CMS)Developed tools two years ago in Blackboard Vista productRan assessments through several semestersSummer 2009: Refined questions and mapped them to Information Literacy Competency Standards for Higher Education

Developed tools two years ago in Blackboard Vista product

Ran assessments through several semesters

Summer 2009: Refined questions and mapped them to Information Literacy Competency Standards for Higher Education

MMBA Pre- and Post-Instruction AssessmentsMixed delivery or blended delivery classTwo Saturdays on-site - remaining course onlineClass runs eight weeks = ½ semesterAssessments run in class first and last day8 questions pre-instruction (first day of class)5 questions post-instruction (end of class)

Mixed delivery or blended delivery class

Two Saturdays on-site - remaining course online

Class runs eight weeks = ½ semester

Assessments run in class first and last day

8 questions pre-instruction (first day of class)

5 questions post-instruction (end of class)

Online Assessment in CMS

Pre-Instruction Course Assessment

Post-Instruction Assessment MMBA

MMBA CMS IssuesSurveys or ungraded quiz tools from CMS don’t provide results.Instructor must allow point credit for results to remain in Gradebook.Students with first-day user ID/password problems can’t reach the assessment.Questions must be tweaked to fit CMS question software format.

Surveys or ungraded quiz tools from CMS don’t provide results.

Instructor must allow point credit for results to remain in Gradebook.

Students with first-day user ID/password problems can’t reach the assessment.

Questions must be tweaked to fit CMS question software format.

Fall 2008 MMBA - Fagerman

Summer 2009 MMBA - Hamel

ESPN 505 Special Education 19 students took Pre-Instruction Assessment15 students took Post-Instruction AssessmentInstructor added 2 point bonus for participationInstructor used CMS with on-campus course

19 students took Pre-Instruction Assessment

15 students took Post-Instruction Assessment

Instructor added 2 point bonus for participation

Instructor used CMS with on-campus course

ESPN 505 PRE AND POST INSTRUCTION ASSESSMENT

NEXT Steps: Real-Time Undergraduate assessments

Vogel Library  FLITE FLITE’s new department head: Feb 2008Assessment Team’s new focus on Information Literacy InstrumentAnnouncement of new James Madison Information Literacy Testing

FLITE’s new department head: Feb 2008

Assessment Team’s new focus on Information Literacy Instrument

Announcement of new James Madison Information Literacy Testing

How To Begin?Vogel Library – WarTburg College

Vogel LibrarySeven years of pre-instruction and post-instruction information literacy testingBased on Information Literacy Competency Standards for Higher EducationDesigned question by question to highlight targeted standards

Seven years of pre-instruction and post-instruction information literacy testing

Based on Information Literacy Competency Standards for Higher Education

Designed question by question to highlight targeted standards

Vogel LibraryDeveloped by Librarians Randall Schroeder, Jill Gremmels, and Karen LehmanDr. Judith Griffith, English Faculty ChairLed to Faculty Buy-In

Developed by

Librarians Randall Schroeder, Jill Gremmels, and Karen Lehman

Dr. Judith Griffith, English Faculty Chair

Led to Faculty Buy-In

Vogel Adaptations to FLITEFSU StudentsFSU Focus and MissionInformation Literacy a new ConceptImplementation and TestingFaculty Buy-in: 3 volunteers worked with us

FSU Students

FSU Focus and Mission

Information Literacy a new Concept

Implementation and Testing

Faculty Buy-in: 3 volunteers worked with us

ChallengesConflicts with Scantrons and Testing Center Need for definitive learning outcomesTesting on concepts not always covered in classes

Conflicts with Scantrons and Testing Center

Need for definitive learning outcomes

Testing on concepts not always covered in classes

What are we Testing?16. A student who is deciding what topic to research for an assigned paper approaches the reference librarian and says, “This summer was too hot. I wonder if it is because of global warming?” What might be the thesis statement of this student’s paper?1. It was too hot this summer.2. Global warming is causing the mean temperature of the earth to rise, which could have disastrous effects on the ecosystems of the planet.3. Did global warming cause abnormal heat around the planet?4. We should buy hybrid vehicles because of global warming.5. This paper will be about the effects of global warming.

Successes - Survey Questions Rank the activities on which you spent the most (4) to the least (1) time when you are on the computer. Searching for specific information related to a school or personal task.Communicating with individuals through Facebook/My Space, IM, or via e-mail. Downloading movies, music or graphics. Surfing the Internet for enjoyment or playing games.

Searching for specific information related to a school or personal task.

Communicating with individuals through Facebook/My Space, IM, or via e-mail.

Downloading movies, music or graphics.

Surfing the Internet for enjoyment or playing games.

What do we know about our students?

Where Do You Get Your Information?

Which of these have you done within the last year?

How many papers did you write?

Schroeder Words of Wisdom: Hard Lessons LearnedGive yourself permission to failAssess to learn things – Don’t just assess to assessBe sure to have Faculty Buy-In

Give yourself permission to fail

Assess to learn things – Don’t just assess to assess

Be sure to have Faculty Buy-In

Starting AgainInvolve new staffImprove librarian buy-InImagine: What will happen when developers leave?

Involve new staff

Improve librarian buy-In

Imagine: What will happen when developers leave?

What are the Information Literacy Competency Standards?http://ala.org/ala/mgrps/divs/acrl/standards/standards.pdf

FLITE Information Literacy Assessment Team CREATES LEARNING OUTCOMESIn-depth study of the 5 competency standardsFocus on performance indicatorsCurriculum mapping - Group discussion and consensus 7 performance indicators1-2 indicators from each standard

In-depth study of the 5 competency standards

Focus on performance indicators

Curriculum mapping - Group discussion and consensus

7 performance indicators

1-2 indicators from each standard

FLITE Information Literacy Assessment Team CREATES LEARNING OUTCOMESGroup work: Individual questions written for each learning outcome (performance indicator) - 35 questions for assessmentsDiscussion and consensus: Team selected 34 questions for Question Bank12 questions are survey questions asked for every assessment22 questions can be tailored for class

Group work: Individual questions written for each learning outcome (performance indicator) - 35 questions for assessments

Discussion and consensus: Team selected 34 questions for Question Bank

12 questions are survey questions asked for every assessment

22 questions can be tailored for class

Information Literacy – Round Two Fall 2009Question Bank built for use with different classesIndividualized assessments built by individual library instructors from Question BankAssessments delivered electronically if possible

Question Bank built for use with different classes

Individualized assessments built by individual library instructors from Question Bank

Assessments delivered electronically if possible

Survey MonkeyStarted in 1999, SurveyMonkey is an online survey tool that enables people of all experience levels to create their own surveys quickly and easily.

FLITE Library’s Assessment Team picked Survey Monkey because:Seven years of pre-instruction and post-instruction information literacy testingBased on Information Literacy Competency Standards for Higher EducationDesigned question by question to highlight targeted standards

Seven years of pre-instruction and post-instruction information literacy testing

Based on Information Literacy Competency Standards for Higher Education

Designed question by question to highlight targeted standards

FLITE Library’s Assessment Team picked Survey Monkey because…HistoryTeam members have previous experience with the programEasy of useUser needs no statistical backgroundUser needs no knowledge of statistical softwareMalleableAccessible from web; in class or at homeEasily printable for a paper version

Success – Yes, You Can TooWe went beyond the “What if’s” and created a project.We gave ourselves permission to fail.We didn’t have to be perfect – we just had to start.We created a Culture of Assessment: we refined what we did and moved on.

We went beyond the “What if’s” and created a project.

We gave ourselves permission to fail.

We didn’t have to be perfect – we just had to start.

We created a Culture of Assessment: we refined what we did and moved on.

Next StepsPost-instruction assessment – Dec 2009Refine questions based on data inputExpand to other faculty and departments Adapt procedures for Distance LearningWrite new assessments for upper-level classes

Accomplished Major GoalClosed the loopCurriculum mapping: using assessment data to revise and improve instruction curriculumKeep assessing – TracDat

Closed the loop

Curriculum mapping: using assessment data to revise and improve instruction curriculum

Keep assessing – TracDat

Culture of AssessmentHelps us inform faculty where we want them to go with usAllows us to create customized instructions across the curriculumShowcases FLITE – the Library is one of the early reporters of assessment data on campus

Helps us inform faculty where we want them to go with us

Allows us to create customized instructions across the curriculum

Showcases FLITE – the Library is one of the early reporters of assessment data on campus

Just Start!Get Faculty Buy-inGive Yourself Permission to FailTake the Plunge and Begin

Get Faculty Buy-in

Give Yourself Permission to Fail

Take the Plunge and Begin

Questions?Kristen Motz motzk@ferris.eduRandall Schroeder randallschroeder@ferris.eduMari Kermit-Canfield kermitt@ferris.edu