Phase I: Overview
Upcoming SlideShare
Loading in...5
×
 

Phase I: Overview

on

  • 298 views

 

Statistics

Views

Total Views
298
Views on SlideShare
294
Embed Views
4

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 4

http://www.sjsu.edu 4

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Phase I: Overview Phase I: Overview Presentation Transcript

  • A Standardized Approach to Assessment: Results from a Pilot Study of Library Instruction Sessions at San José State University Shannon M. Staley Social Sciences Librarian
  • Purpose of Study
    • To Develop and Test an assessment tool that:
    • Facilitates the development of pre and post achievement tests
    • Links survey questions to learning outcomes and ACRL standards
    • Delivers real time reports and statistical analysis
  • Timeline: An Overview
    • Review of the Literature – Summer 2008
    • Create assessment tool – Fall 2008
    • Garner input from library faculty / Refine tool - 12/08
    • Develop multiple choice questions – 12/08
    • Standardize questions – 1/09
    • Pilot test assessment tool – 1/09
    • Enact actual study – Spring 2009
  • Methodology: Develop Multiple-Choice Questions
    • Investigated questions from already existing tutorials / survey instruments for ideas
    • Created 20 multiple-choice questions based on social sciences 100W learning outcomes
    • Each question presented four answers, with only one correct answer
    • Each question had an added option for “not sure”
  • Methodology: Standardization of Survey Multiple Choice Questions
    • Multiple choice questions were emphasized to allow for immediate computation of results
    • The co-occurrence of LibQual during Spring 2009 provides for complimentary qualitative data
    • Questions were reviewed by library faculty to ensure they reflected learning outcomes / ACRL standards
    • Questions and learning outcomes were emailed to all teaching faculty in the Psychology department for review
    • Questions were reviewed by experts at the Center for Assessment under the Office of Institutional Research
    • Cognitive interviews with 5 students further addressed the clarity of survey questions
  • Methodology: Pilot Study
    • Conducted with students in a PSYC 139 course to
    • determine:
    • Average length of time to complete surveys
    • Overall impact on what could be covered in a single instruction session
    • Any technical glitches with survey interface, login, usability etc.
    • If survey questions were clear and comprehensible to students
  • Methodology: Pilot Study
    • Conducted with students in PSYC 139 to
    • determine:
    • Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion)
    • Overall impact on what could be covered in a single instruction session
    • Any technical glitches with survey interface, login, usability etc.
    • If survey questions were clear and comprehensible to students
  • Methodology: Pilot Study
    • Conducted with students in PSYC 139 to
    • determine:
    • Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion)
    • Overall impact on what could be covered in a single instruction session - no major impact with tighter planning
    • Any technical glitches with survey interface, login, usability etc.
    • If survey questions were clear and comprehensible to students
  • Methodology: Pilot Study
    • Conducted with students in PSYC 139 to
    • determine:
    • Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion)
    • Overall impact on what could be covered in a single instruction session - no major impact with tighter planning
    • Any technical glitches with survey interface, login, usability etc. - no major problems. Logins for the post test could not contain typos
    • If survey questions were clear and comprehensible to students
  • Methodology: Pilot Study
    • Conducted with students in PSYC 139 to
    • determine:
    • Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion)
    • Overall impact on what could be covered in a single instruction session - no major impact with tighter planning
    • Any technical glitches with survey interface, login, usability etc. - no major problems. Logins for the post test could not contain typos
    • If survey questions were clear and comprehensible to students – no confusion was reported
  • Methodology: Planning and Instruction
    • Coordinated with instructor beforehand in encouraging students to show up early
    • Scripted instructions were read to students prior to taking the pre test survey
    • Lecture with embedded exercises; last 15 minutes reserved for getting started on course assignments
    • Course outline and handouts were distributed to students after completion of their post test surveys
  • Methodology: Actual Study – Spring 2009
    • Conducted with students in 5 PSYC 100W courses (83 students total)
    • Survey participation was voluntary
    • Participants who completed the pre-test survey but failed to complete the post-test survey were automatically eliminated from the data pool
    • 11 multiple-choice survey questions covering 3 ACRL standards and 8 learning objectives
    • 6 background questions on the pre-test survey – they were not repeated on the post-test
  • Methodology: Pre-Test Survey Background Questions
  • Methodology: PSYC 100W Multiple-Choice Survey Questions
  • Data Analysis A Bird’s Eye View
  • Data Analysis A Bird’s Eye View
  • Data Analysis A Bird’s Eye View
  • Data Analysis Statistical Significance
    • Paired TTEST of student mean scores
    • Excel formula computes statistical significance in the report generation feature of the assessment tool
  • Data Analysis Assessment Tool Sample Report
  • Data Analysis Future Enhancements
    • Master spreadsheet that incorporates all reports
    • More granular TTEST analysis of individual questions
    • Statistical analysis of how background factors (class level, research experience, instruction experience) affect scores
  • Benefits
    • Assessment data will inform pedagogical directions and enhance the educational impact on SJSU students
    • Data can be shared with department faculty and campus administrators to demonstrate instructional efficacy
    • Data can also be used to bolster documentation during the accreditation process
    • Librarians can publish findings from their own sessions, particularly with a more rigorous statistical model
    • Librarians can include assessment reports in their dossiers