Overview of Library Instruction Assessment

ASSESSMENT OF LIBRARY INSTRUCTION @ Randall Library

What is Assessment?  “Assessment is an ongoing process aimed at understanding and improving student learning.”1 Main purposes: 2  Increase student learning  Strengthen instructional program  Answer calls for accountability http://www.acrl.org/ala/mgrps/divs/acrl/issues/infolit/resources/assess/issues.cfm (1) Oakleaf & Kaske, 2009 (2)

Why Assess Library Instruction?  To improve instruction  To increase student learning  To inform future assessment efforts  To provide data for accountability measures  To collaborate with faculty  To collaborate with other librarians  To argue for increased resources http://libraryassessment.org/bm~doc/oakleaf_assessment_cycle.pps Forward to Slide 9

Why Assess Library Instruction?  “Well done assessment is essential to documenting the effects of information literacy programs and the degree to which they are meeting their objectives. The assessment process can also assist anyone involved in information literacy instruction by getting them actively involved in the process and engaged in classroom activities, by giving them feedback about students‟ thinking and learning, and by helping them focus on goals and objectives …” Assessing Student Learning Outcomes for Information Literacy Instruction in Academic Institutions

Resistance and Misperceptions  “_____ is making us do this.”  “_____ is trying to tell us how to teach.”  “We have to get this over with.” (Once and done)  “I don‟t have time for this.”  “I have to develop a pre-test and a post-test.”  “Assessment is a fad.”

What Needs to be Assessed?  Macro to micro ◦ UNCW curriculum as a whole ◦ All UNCW programs / majors ◦ All UNCW courses ◦ Current “Basic Studies” program / New “University Studies” program (General Education)  Information Literacy (IL) component of University Studies curriculum (as a whole)  “Information Literacy courses” within IL component  Library instruction sessions within each course

Past Library Instruction Assessment  Library Instruction Sessions ◦ Primary focus on teaching evaluation  Online student evaluation forms  Peer (librarian) observations  Faculty evaluations of teaching (Attending instructor)  Focus has been on us ◦ Perceptions of teaching ◦ In general, little measure of what students are actually learning / can do  Some measure of what they feel they are learning  Indirect assessment

Moving Forward  Comprehensive library instruction assessment plan ◦ Instruction program (holistically) ◦ Student learning outcomes for individual instruction sessions  Ongoing, continuous process

Strategic Plan for Assessment  Determine purpose(s) of assessment  Examine mission of the university, library, and instruction program  Examine structure of library instruction program  Examine program goals  Examine program resources  Collect and examine relevant data / statistics  Examine / establish collaborations with other faculty  Establish aggressive yet attainable timeline

Purpose(s)  Why are we engaging in assessment?  What do we hope to gain from assessment efforts? Main purposes:  Increase student learning  Strengthen instructional program  Answer calls for accountability  See slide #3

Other Purposes  Connection between library instruction assessment and UNCW assessment ◦ Only area offering courses not represented in UNCW assessment efforts (?) ◦ No representation on Learning Assessment Council ◦ Instruction sessions a component of UNCW courses  Promotion of library instruction program and information literacy  Internally-focused purposes: ◦ Improve workload, workflow, organizational structures in library, etc.

Driving Forces (Randall Library)  Formation of Information Literacy Strategic Planning Team (Fall 2008)  Instructional Services Coordinator training: ◦ ACRL Immersion Program (Summer 2008) ◦ Master of Science in Instructional Technology (MIT)  MIT 531 (Assessment of Learning Outcomes) (Summer 2009) ◦ LOEX Conference (May 2010) ◦ NCLA Bibliographic Instruction Group: Assessment & Student Learning Outcomes Workshop (May 2010) ◦ Georgia Conference on Information Literacy (October 2010)

ILSPT  Mission Statement: “The Information Literacy Strategic Planning Team will lead long-term, strategic planning to help transform the current set of instructional services activities into a well conceived, planned program that is integrated into both the basic studies curriculum and into each major. An information literacy program is one that utilizes well placed instruction sessions and tutorials/online modules to integrate information literacy skills into the curriculum of courses and builds skills progressively through a student‟s college career.”

Driving Forces (RL & UNCW)  UNCW SACS accreditation  New/revised “University Studies” program which includes Information Literacy Component  Need for library to demonstrate importance, effectiveness, need for resources, etc.

Driving Forces (UNCW)  Highest priority: Student learning  UNCW shift to culture of assessment ◦ 2005 campus assessment workshops (Susan Hatfield, consultant) ◦ Learning Assessment Council ◦ Institutional Research & Assessment ◦ Assessment directors ◦ Assessment of Basic Studies  National Survey of Student Engagement (NSSE)  Collegiate Learning Assessment (CLA)  General Education Assessment Committee (Report)

Assessment so Far (UNCW)  UNCW Learning Goals  General Education Report (March 2009)  On Information Literacy ◦ No librarians were consulted in the construction of this process / report ◦ Instructional Services Coordinator contacted the Director of University Planning and sent the Information Literacy Competency Standards to Director through campus mail http://uncw.edu/assessment/Documents/General%20Education/GenEdAssessmentCommitteeReportMarc ◦ Communication was not returned h2009.pdf

General Education Report  “All UNCW freshman seminar participants undergo instruction by the Randall Library in locating and using information, although there is no current assessment of student‟s information literacy proficiency either before or after that instruction.”

General Education Report  “Recent data indicate that college students have difficulty developing a topic question and are often not aware of informational resources available to them. Even when aware of appropriate information resources, they may lack the ability to adequately evaluate and synthesize information. This can result in classroom work based on haphazard discussion groups or personal opinion.” Albitz, R.S. (2007) The what and who of information literacy and critical thinking in higher education. Libraries and the Academy. 7, 1, 97‐109

General Education Report  “Of 37 common learning outcomes established for the Basic Studies categories, only two (CMP3 and IP1A) fit into the category of information literacy. Perhaps there is not sufficient emphasis in the spectrum of Basic Studies courses on information literacy in a broad range of disciplines, or perhaps additional common learning outcomes should be developed to encompass efforts that are currently being made.” CMP3: “Employ a writing process that includes finding, evaluating, analyzing, and synthesizing appropriate primary and secondary sources, and successfully integrating personal knowledge with source material.” IP 1A: “Access and analyze information about topics or issues that are interdisciplinary or transdisciplinary.”

National Survey of Student Engagement  “First-year students were asked in NSSE about the frequency with which they „worked on a paper or project that required integrating ideas or information from various sources,‟ a component of information literacy. UNCW first-year students reported a frequency that was statistically significantly below that reported by our selected peers, significantly below that reported by national master‟s universities, and significantly below that reported by all NSSE 2007 institutional participants. This information led to the development of a rubric-based assessment plan for information literacy to be implemented with the comprehensive assessment of Basic Studies beginning Fall 2009.”  Assessment tool selected: ◦ The American Association of Colleges and Universities‟ (AAC&U) “Information Literacy Metarubric” http://uncw.edu/assessment/BasicStudiesAssessment.html http://uncw.edu/assessment/Documents/General%20Education/2004-2005NSSE- BasicStudies.pdf

Merging Efforts Information Literate UNCW Graduates UNCW (University Randall Library (Library Studies Assessment) Instruction Assessment)

On the Horizon  University Studies (Fall 2011)  Students who complete the University Studies program will: ◦ Goal 3. Locate, evaluate, and effectively use information by applying a variety of academic and technological skills (Information Literacy) http://www.uncw.edu/universitystudies/documents/Univ.StudiesCurriculumReport.pdf

Information Literacy Curriculum  Component of University Studies curriculum  “Designed to foster the development of students‟ information literacy skills by requiring three information literacy intensive courses: The First Year Experience and two additional information literacy intensive courses, with at least one in the major field of study.”

Information Literacy Curriculum  Based on the Information Literacy Competency Standards for Higher Education  The information literate student … 1. Determines the nature and extent of the information needed. 2. Accesses needed information effectively and efficiently. 3. Evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system. 4. Individually or as a member of a group, uses information effectively to accomplish a specific purpose. 5. Understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.

Information Literacy Curriculum  University Studies Information Literacy Student Learning Outcomes (adopted by Faculty Senate November 2009) The student will:  IL 1. Be able to determine the nature and extent of information needed to solve a problem.  IL 2. Access information effectively and efficiently from a variety of sources.  IL 3. Evaluate information critically and incorporate appropriate information into his or her knowledge base.  IL 4. Individually, or as a member of a group, use information effectively to accomplish a specific purpose.  IL 5. Understand many of the economic, legal, and social issues surrounding the use of information and access and use information ethically and legally.

Information Literacy Curriculum Course in the Major (Research Methods or other)(3 hours) 9 hours of Library instruction sessions focus on IL Comp courses Standards 2 (Accessing info through discipline specific resources) and 5 (Ethical use: Discipline (“Double specific citations and writing) dipping” from other English Composition or other (e.g. LIB 103) components) (3 hours) Library instruction sessions focus on IL Comp Content Standards 1 (Determining info needed), 2 (Accessing info), and 4 (Using info effectively) reinforced throughout Freshman Seminar (3 hours) curriculum Library instruction sessions focus on IL Comp Standards 3 (Evaluating information) and 5 (Ethical use: Plagiarism)

What‟s Different for Us?  Every major must have an “information literacy intensive course”  Faculty must supply information literacy student learning outcomes for that course  Information literacy on the “radar” in a new way ◦ “Just send „em to the library” does not apply ◦ Potential for collaborations that did not exist before  Information Literacy is a goal of the entire University Studies program ◦ The university will assess this

Library Instruction Program Shift  From bibliographic instruction sessions to information literacy ◦ Moving from scattered “one shot” instruction sessions to integration of information literacy skills instruction into UNCW curriculum  Defining the “Instruction Program” ◦ Information literacy curriculum (University Studies)  Freshman Seminar instruction  English composition instruction (ENG 103 and 201)  Instruction in each major ◦ Workshops & tours ◦ Online tutorials ◦ LIB Courses ◦ Regional engagement / instruction (high school student instruction, Pathways workshops, etc.)

Current Library Instruction  AY 2008-2009  Course-related instruction sessions, workshops, and tours ◦ 587 sessions ◦ 678 contact hours ◦ Teaching 10,750 users PLUS  Thirty four hours of credit courses offered ◦ LIB 101 (4 sections, 1credit hour, 19 students in each) ◦ LIB 103 (9 sections, 3 credit hours, 19 students in each) ◦ LIB 104 (1 section, 3 credit hours, 19 students in each) https://share.uncw.edu/sites/aa/library/pubsvcs/Shared%20Documents/Annual%20Reports/annrept0 809.docx

Moving to Assessment: A System  Part 1: Expected Outcomes (Part of Assessment Plan)  Part 2: Assessment Methods and Tools (Part of Assessment Plan)  Part 3: Implementation (Part of Assessment Plan)  Part 4: Summary of Findings (Part of Assessment Report)  Part 5: Actions Taken (Part of Assessment Report) http://www.uncw.edu/cas/assessmentplanandreport.html

Steps of Assessment 1. Determine what we need to assess by creating Student Learning Outcomes for:  Freshman Seminar library instruction sessions  English Composition (ENG 103 and 201) library instruction sessions  Instruction sessions in each major (as applicable)  LIB Courses  ILSPT has reviewed ACRL Standards: Good starting point 2. Determine how we are going to assess  Means of Assessment (what method or tool we will use)  Rubric used to score worksheet or other student products, “5 minute paper,” tests, etc. 3. Determine what and when we will assess 4. Review assessment information 5. Make changes based on assessment information Repeat all steps continuously

Assessment Cycle Implement Changes Propose Formulate Improvements Outcomes to Program Library Instruction Program Develop Interpret Means of Results Assessment Gather & Analyze Data

Assessment Checklist 1. What are our research questions? (What are we trying to discover about student skills, knowledge, abilities, etc.; and what evidence do we have already?) 2. What is the expected level of performance? 3. When in the students‟ career do we assess this outcome? (entry, end of sophomore year, senior, etc.) 4. In which course(s) or venue? 5. What student work/artifacts are collected? 6. How is the student work evaluated? (criteria/rubric) 7. Who evaluates the student work? 8. Who analyzes the results? 9. Where do recommendations for action go? 10. Who takes action? (And how do we ensure changes are evidence‐based and data‐driven?) 11. How is the process documented? 12. Where is the documentation kept? 13. What is the timetable/schedule for determining which outcomes are assessed Developed by the General Education Assessment Committee for designing assessment of a learning when? outcome.

Who Is Going to Do This?  All Instruction Librarians ◦ Comprehensive plan coordinated by Instructional Services Coordinator  Supported by Information Literacy Strategic Planning Team ◦ Assessment at micro level (instruction sessions)  Done by individual librarians with assistance from Instructional Services Coordinator  Will seek guidance from campus experts (e.g. Dr. Linda Seifert, Director of Assessment, College of Arts and Sciences)

First Steps  Completion of assessment structure and plan outline (Summer 2010)  LIB 103 assessment planning (Begin Summer 2010)  Freshman Seminar assessment planning ◦ SLOs and means of assessment (Begin Fall 2010)  English Composition (ENG 103 and 201) assessment planning ◦ SLOs and means of assessment (Begin Spring 2011)  Assessment planning for the majors ◦ SLOs and means of assessment (Begin Fall

Other Assessment & Evaluation  SWOT of library instruction  Continued annual peer program (Fall 2010) assessment of teaching  Self analysis of program  Continued use of SPOTS for based on “Characteristics of LIB courses Programs of Information Literacy that Illustrate Best  Future implementation of Practices: A Guideline” (Fall faculty surveys and 2010) evaluations  Self analysis of librarians  Review benchmarking data based on “Association of (Collected in fall 2008 and College and Research spring 2009) Libraries Standards for Proficiencies for Instruction Librarians and Coordinators”  Review of LibQual+ Survey (Fall 2010) results (Administered in 2008)  Continuous use of online student evaluation forms

To Explore in the Future  Large Scale Information Literacy Assessment Tools for UNCW: ◦ Project SAILS (Standardized Assessment of Information Literacy Skills)  Sample ◦ iCritical Thinking™ Certification (ETS)  Replaced the ETS iSkills Test  Graduate Courses

Data Management  WASSAIL (Open source)  “WASSAIL is a database-driven, web-based application employing PHP, MySQL, and Javascript/AJAX technologies. It was created to manage question and response data from the Augustana Library's library instruction sessions, pre- and post-tests from credit bearing information literacy (IL) courses, and user surveys. It has now expanded beyond its original function and is being used to manage question and response data from a variety of settings.”  http://www.library.ualberta.ca/augustana/infolit/wassail/

Helpful Resources  ACRL Information Literacy website (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm)  ACRL Institute for Information Literacy (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm)  ARL Library Assessment Conference (http://libraryassessment.org)  “A Twenty Year Path: Learning about Assessment; Learning from Assessment” (http://www.comminfolit.org/index.php/cil/article/view/Vol3- 2009PER1/97)  Information Literacy in the Disciplines (http://wikis.ala.org/acrl/index.php/Information_literacy_in_the_disciplines)

Helpful Resources  Internet Resources for Higher Education Outcomes Assessment (http://www2.acs.ncsu.edu/UPA/assmt/resource.htm)  Megan Oakleaf‟s Presentations and Publications (http://meganoakleaf.info/presentations.html)  National Institute for Learning Outcomes Assessment (http://learningoutcomesassessment.org/)  Task Force on Academic Library Outcomes Assessment Report (http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/taskforceacademic.cfm)  Utah State University Library Instruction Assessment (http://library.usu.edu/instruct/assessment/index.php)  “Writing Information Literacy Assessment Plans: A Guide to Best Practice” (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER2/98)

Library Instruction Assessment Website Anne Pemberton Instructional Services Coordinator Chair, Information Literacy Strategic Planning Team pembertona@uncw.edu http://library.uncw.edu/staff/anne_pemberton

Overview of Library Instruction Assessment

by Randall_Library

Accessibility

Upload Details

Usage Rights

Statistics

Overview of Library Instruction Assessment Presentation Transcript