Assessing Information Literacy Online Internet Librarian October 2008 Kevin Roddy and Margot Hanson University of Hawaii

Purpose & Description LILO (Learning Information Literacy Online) is a project of the UH Libraries Information Literacy Committee (UHLILC) for the UH System. LILO is a productivity application that guides students’ research process using library and Web resources. Students create and add to an online Research Journal, documenting their research on a topic over a period of weeks.

LILO (Learning Information Literacy Online) is a project of the UH Libraries Information Literacy Committee (UHLILC) for the UH System. LILO is a productivity application that guides students’ research process using library and Web resources. Students create and add to an online Research Journal, documenting their research on a topic over a period of weeks.

The origin of LILO The UH library system obtained the code to LOBO, the Library Online Basic Orientation at NC State, through an End-User License Agreement, in Spring 2004. The tutorial has been completely redesigned and is regularly updated by UH Libraries Information Literacy committee (LILC) members. Instructional faculty, librarian, and student input is very important.

The UH library system obtained the code to LOBO, the Library Online Basic Orientation at NC State, through an End-User License Agreement, in Spring 2004. The tutorial has been completely redesigned and is regularly updated by UH Libraries Information Literacy committee (LILC) members. Instructional faculty, librarian, and student input is very important.

LILO today Tutorial Research journal IM chat help Email access to librarians and instructors Videos Links to interactive elements

Tutorial

Research journal

IM chat help

Email access to librarians and instructors

Videos

Links to interactive elements

Unique needs of Hawaii High level of demand for local topics research materials Low level of Hawaii-topic indexing in national/international databases

High level of demand for local topics research materials

Low level of Hawaii-topic indexing in national/international databases

Changing times and capturing the interest of students Embedded and linked interactive material from various sources

Embedded and linked interactive material from various sources

Tutorial Six modules General to specific

Six modules

General to specific

My LILO Journal Research journal Students record research process

Research journal

Students record research process

Chat assistance 24-hour chat reference service

24-hour chat reference service

Email access to librarians and instructors

Videos YouTube Shared content by other libraries Create our own

YouTube

Shared content by other libraries

Create our own

Links to interactive elements

Student Feedback Students in Leeward CC instructor Donna Matsumoto’s classes said: “LILO improved my research dramatically. It has everything you need and journals to keep you updated on where you are on your research. Without LILO, I would've procrastinated on this assignment.

Students in Leeward CC instructor Donna Matsumoto’s classes said: “LILO improved my research dramatically. It has everything you need and journals to keep you updated on where you are on your research. Without LILO, I would've procrastinated on this assignment.

Written Survey Results 95% of students (36 of 38) found that LILO is helpful in their research, and would recommend LILO to their friends. The Research Journal, Citation Builder, and Assignment Calculator were identified as the most useful LILO tools.

95% of students (36 of 38) found that LILO is helpful in their research, and would recommend LILO to their friends.

The Research Journal, Citation Builder, and Assignment Calculator were identified as the most useful LILO tools.

Future enhancements Rate usefulness of video, Web, and text content Course syllabus integration Pre-test evaluation and push WASC – Western Association of Schools and Colleges (4-year+) ACCJC – Accrediting Commission for Community and Junior Colleges

Rate usefulness of video, Web, and text content

Course syllabus integration

Pre-test evaluation and push

Assessment Assessment!

Assessment Challenges Meeting accrediting body (WASC, ACCJC)*, individual campus/system, ALA expectations Faculty Buy-In Data Analysis WASC – Western Association of Schools and Colleges (four year/graduate) ACCJC – Accrediting Commission for Community and Junior Colleges

Meeting accrediting body (WASC, ACCJC)*, individual campus/system, ALA expectations

Faculty Buy-In

Data Analysis

1. Accreditation Directives (WASC) Western Association of Schools and Colleges (WASC) Standard 2: “ Baccalaureate programs engage students an integrated course of study of sufficient breadth and depth to prepare them for work, citizenship, and a fulfilling life . These programs also ensure the development of core learning abilities and competencies including, but not limited to, college-level written and oral communication; college-level written and oral communication; college-level quantitative skills; information literacy ; and the habit of critical analysis of data and argument.” WASC 2001 Accreditation Handbook, Standard 2 (Achieving Educational Objectives Through Core Functions): Criteria for Review.

Western Association of Schools and Colleges (WASC) Standard 2:

“ Baccalaureate programs engage students an integrated course of study of sufficient breadth and depth to prepare them for work, citizenship, and a fulfilling life . These programs also ensure the development of core learning abilities and competencies including, but not limited to, college-level written and oral communication; college-level written and oral communication; college-level quantitative skills; information literacy ; and the habit of critical analysis of data and argument.”

WASC 2001 Accreditation Handbook, Standard 2 (Achieving Educational Objectives Through Core Functions): Criteria for Review.

1. Accreditation Directives - ACCJC Standard II. A. 3. (Instruction - General Education requirements) A.3.b. A capability to be a productive individual and life long learner: skills include oral and written communication, information competency , computer literacy, scientific and quantitative reasoning, critical analysis/logical thinking, and the ability to acquire knowledge through a variety of means. Standard II.C.1.b. (Library and Learning Support Services) The institution provides ongoing instruction for users of library and other learning support services so that students are able to develop skills in information competency .

Standard II. A. 3. (Instruction - General Education requirements)

A.3.b. A capability to be a productive individual and life long learner: skills include oral and written communication, information competency , computer literacy, scientific and quantitative reasoning, critical analysis/logical thinking, and the ability to acquire knowledge through a variety of means.

Standard II.C.1.b. (Library and Learning Support Services)

The institution provides ongoing instruction for users of library and other learning support services so that students are able to develop skills in information competency .

1. UH General Education “Hallmark” Directives

1. ALA IL Competency Standards Standards, Performance Indicators, Outcomes (“SPIOs”) Standard One The information literate student determines the nature and extent of the information needed. Performance Indicator 2 The information literate student identifies a variety of types and formats of potential sources for information Outcome d Identifies the purpose and audience of potential resources (e.g., popular vs. scholarly, current vs. historical)

Standards, Performance Indicators, Outcomes (“SPIOs”)

Standard One The information literate student determines the nature and extent of the information needed. Performance Indicator 2 The information literate student identifies a variety of types and formats of potential sources for information Outcome d Identifies the purpose and audience of potential resources (e.g., popular vs. scholarly, current vs. historical)

1. Educational Testing Service (ETS) and Information Literacy ETS investment in IL: Information and Communication Technology (ICT) Literacy Assessment iSkills assessment

ETS investment in IL:

Information and Communication Technology (ICT) Literacy Assessment

iSkills assessment

2. Faculty Buy-in Librarian/Instructor Collaborative Model IL competency is now an institutional requirement Collaboration facilitates “total student learning” Good evidence for tenure/promotion and self-study documents Promotion Print format (e.g., club cards) Department, writing intensive, new faculty meeting Campus Involvement Developing student learning outcomes (SLOs) for courses Service on the campus governance and policy committees

Librarian/Instructor Collaborative Model

IL competency is now an institutional requirement

Collaboration facilitates “total student learning”

Good evidence for tenure/promotion and self-study documents

Promotion

Print format (e.g., club cards)

Department, writing intensive, new faculty meeting

Campus Involvement

Developing student learning outcomes (SLOs) for courses

Service on the campus governance and policy committees

2. Assessing IL Online LILO shows Instructors: Accreditation, UH System, ALA expectations How to integrate LILO into course How to obtain IL competency reports LILO Statistics module http://www.hawaii.edu/lilo/0908/lilostats_2008_2009.php

LILO shows Instructors:

Accreditation, UH System, ALA expectations

How to integrate LILO into course

How to obtain IL competency reports

LILO Statistics module

http://www.hawaii.edu/lilo/0908/lilostats_2008_2009.php

3. Data Analysis Response data in student journals LILO delivers response data to instructors to evaluate separately if desired LILO Librarians evaluate response data and send to Instructors Librarians and Instructors use response data to improve content and questions

Response data in student journals

LILO delivers response data to instructors to evaluate separately if desired

LILO Librarians evaluate response data and send to Instructors

Librarians and Instructors use response data to improve content and questions

Evaluation Cycle 1. Accreditation, University And ALA Directives 2. LILO Content and Questions 3. Student Responses 4. Score with Rubrics 5. Response Data to Faculty as a report

Question/Rubric/SPIO example http://www.hawaii.edu/lilo/ http://www.hawaii.edu/lilo/0908/eval_web_purpose.php

http://www.hawaii.edu/lilo/

http://www.hawaii.edu/lilo/0908/eval_web_purpose.php

Scoring Module LILO librarians rate student responses online UH SLIS graduate students participate as service learning

LILO librarians rate student responses online

UH SLIS graduate students participate as service learning

LILO Software/Hardware PHPMyAdmin 2.8.0.2 MYSQL 4.1.20 - 29 tables Dreamweaver 8/CS3 UH System-Administered Server Programming team of 1 No previous programming background Learned mostly from books (and lynda.com)

PHPMyAdmin 2.8.0.2

MYSQL 4.1.20 - 29 tables

Dreamweaver 8/CS3

UH System-Administered Server

Programming team of 1

No previous programming background

Learned mostly from books (and lynda.com)

IL Reports show… Evidence of IL instruction and Assessment at UH Evidence of Librarian/Instructor Collaborative Effort Invaluable data for Institutional Self-Studies!

Evidence of IL instruction and Assessment at UH

Evidence of Librarian/Instructor Collaborative Effort

Invaluable data for Institutional Self-Studies!

. Thanks from Margot and Kevin! [email_address] [email_address]