A Standardized Approach to Assessment: Results from a Pilot Study of Library Instruction Sessions at San José State Univer...
Purpose of Study <ul><li>To Develop and Test an assessment tool that: </li></ul><ul><li>Facilitates the development of pre...
Timeline: An Overview <ul><li>Review of the Literature – Summer 2008 </li></ul><ul><li>Create assessment tool – Fall 2008 ...
Methodology: Develop Multiple-Choice Questions  <ul><li>Investigated questions from already existing tutorials / survey in...
Methodology: Standardization of Survey Multiple Choice Questions  <ul><li>Multiple choice questions were emphasized to all...
Methodology: Pilot Study  <ul><li>Conducted with students in a PSYC 139 course to </li></ul><ul><li>determine: </li></ul><...
Methodology: Pilot Study  <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Av...
Methodology: Pilot Study  <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Av...
Methodology: Pilot Study  <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Av...
Methodology: Pilot Study  <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Av...
Methodology: Planning and Instruction  <ul><li>Coordinated with instructor beforehand in encouraging students to show up e...
Methodology: Actual Study – Spring 2009  <ul><li>Conducted with students in 5 PSYC 100W courses (83 students total) </li><...
Methodology: Pre-Test Survey Background Questions
Methodology: PSYC 100W Multiple-Choice Survey Questions
Data Analysis A Bird’s Eye View
Data Analysis A Bird’s Eye View
Data Analysis A Bird’s Eye View
Data Analysis Statistical Significance  <ul><li>Paired TTEST of student mean scores </li></ul><ul><li>Excel formula comput...
Data Analysis Assessment Tool Report Options
Data Analysis Assessment Tool Sample Report
Data Analysis Future Enhancements <ul><li>Master statistical spreadsheet that incorporates all reports </li></ul><ul><li>T...
Benefits <ul><li>Assessment data will inform pedagogical directions and enhance the educational impact on SJSU students  <...
How Can I Get Involved <ul><li>Attend the InfoLit meeting in July regarding use of the Information Literacy Assessment Too...
Upcoming SlideShare
Loading in …5
×

Standardized Information Literacy Assessment

446
-1

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
446
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Standardized Information Literacy Assessment

  1. 1. A Standardized Approach to Assessment: Results from a Pilot Study of Library Instruction Sessions at San José State University
  2. 2. Purpose of Study <ul><li>To Develop and Test an assessment tool that: </li></ul><ul><li>Facilitates the development of pre and post achievement tests </li></ul><ul><li>Links survey questions to learning outcomes and ACRL standards </li></ul><ul><li>Delivers real time reports and statistical analysis </li></ul>
  3. 3. Timeline: An Overview <ul><li>Review of the Literature – Summer 2008 </li></ul><ul><li>Create assessment tool – Fall 2008 </li></ul><ul><li>Garner input from library faculty / Refine tool - 12/08 </li></ul><ul><li>Develop multiple choice questions – 12/08 </li></ul><ul><li>Standardize questions – 1/09 </li></ul><ul><li>Pilot test assessment tool – 1/09 </li></ul><ul><li>Enact actual study – Spring 2009 </li></ul>
  4. 4. Methodology: Develop Multiple-Choice Questions <ul><li>Investigated questions from already existing tutorials / survey instruments for ideas </li></ul><ul><li>Created 20 multiple-choice questions based on social sciences 100W learning outcomes </li></ul><ul><li>Each question presented four answers, with only one correct answer </li></ul><ul><li>Each question had an added option for “not sure” </li></ul>
  5. 5. Methodology: Standardization of Survey Multiple Choice Questions <ul><li>Multiple choice questions were emphasized to allow for immediate computation of results </li></ul><ul><li>The co-occurrence of LibQual during Spring 2009 provides for complimentary qualitative data </li></ul><ul><li>Questions were reviewed by library faculty to ensure they reflected learning outcomes / ACRL standards </li></ul><ul><li>Questions and learning outcomes were emailed to all teaching faculty in the Psychology department for review </li></ul><ul><li>Questions were reviewed by experts at the Center for Assessment under the Office of Institutional Research </li></ul><ul><li>Cognitive interviews with 5 students further addressed the clarity of survey questions </li></ul>
  6. 6. Methodology: Pilot Study <ul><li>Conducted with students in a PSYC 139 course to </li></ul><ul><li>determine: </li></ul><ul><li>Average length of time to complete surveys </li></ul><ul><li>Overall impact on what could be covered in a single instruction session </li></ul><ul><li>Any technical glitches with survey interface, login, usability etc. </li></ul><ul><li>If survey questions were clear and comprehensible to students </li></ul>
  7. 7. Methodology: Pilot Study <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion) </li></ul><ul><li>Overall impact on what could be covered in a single instruction session </li></ul><ul><li>Any technical glitches with survey interface, login, usability etc. </li></ul><ul><li>If survey questions were clear and comprehensible to students </li></ul>
  8. 8. Methodology: Pilot Study <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion) </li></ul><ul><li>Overall impact on what could be covered in a single instruction session - no major impact with tighter planning </li></ul><ul><li>Any technical glitches with survey interface, login, usability etc. </li></ul><ul><li>If survey questions were clear and comprehensible to students </li></ul>
  9. 9. Methodology: Pilot Study <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion) </li></ul><ul><li>Overall impact on what could be covered in a single instruction session - no major impact with tighter planning </li></ul><ul><li>Any technical glitches with survey interface, login, usability etc. - no major problems. Logins for the post test could not contain typos </li></ul><ul><li>If survey questions were clear and comprehensible to students </li></ul>
  10. 10. Methodology: Pilot Study <ul><li>Conducted with students in PSYC 139 to </li></ul><ul><li>determine: </li></ul><ul><li>Average length of time to complete surveys - 5 minutes (10 minutes total for pre and post survey completion) </li></ul><ul><li>Overall impact on what could be covered in a single instruction session - no major impact with tighter planning </li></ul><ul><li>Any technical glitches with survey interface, login, usability etc. - no major problems. Logins for the post test could not contain typos </li></ul><ul><li>If survey questions were clear and comprehensible to students – no confusion was reported </li></ul>
  11. 11. Methodology: Planning and Instruction <ul><li>Coordinated with instructor beforehand in encouraging students to show up early </li></ul><ul><li>Scripted instructions were read to students prior to taking the pre test survey </li></ul><ul><li>Lecture with embedded exercises; last 15 minutes reserved for getting started on course assignments </li></ul><ul><li>Course outline and handouts were distributed to students after completion of their post test surveys </li></ul>
  12. 12. Methodology: Actual Study – Spring 2009 <ul><li>Conducted with students in 5 PSYC 100W courses (83 students total) </li></ul><ul><li>Survey participation was voluntary </li></ul><ul><li>Participants who completed the pre-test survey but failed to complete the post-test survey were automatically eliminated from the data pool </li></ul><ul><li>11 multiple-choice survey questions covering 3 ACRL standards and 8 learning objectives </li></ul><ul><li>6 background questions on the pre-test survey – they were not repeated on the post-test </li></ul>
  13. 13. Methodology: Pre-Test Survey Background Questions
  14. 14. Methodology: PSYC 100W Multiple-Choice Survey Questions
  15. 15. Data Analysis A Bird’s Eye View
  16. 16. Data Analysis A Bird’s Eye View
  17. 17. Data Analysis A Bird’s Eye View
  18. 18. Data Analysis Statistical Significance <ul><li>Paired TTEST of student mean scores </li></ul><ul><li>Excel formula computes statistical significance in the report generation feature of the assessment tool </li></ul>
  19. 19. Data Analysis Assessment Tool Report Options
  20. 20. Data Analysis Assessment Tool Sample Report
  21. 21. Data Analysis Future Enhancements <ul><li>Master statistical spreadsheet that incorporates all reports </li></ul><ul><li>TTEST analysis of individual questions </li></ul><ul><li>Multivariate statistical analysis of how background factors (class level, research experience, instruction experience) influence overall scores </li></ul>
  22. 22. Benefits <ul><li>Assessment data will inform pedagogical directions and enhance the educational impact on SJSU students </li></ul><ul><li>Data can be shared with department faculty and campus administrators to demonstrate instructional efficacy </li></ul><ul><li>Data can also be used to bolster documentation during the accreditation process </li></ul><ul><li>Librarians can publish findings from their own sessions, particularly with a more rigorous statistical model </li></ul><ul><li>Librarians can include assessment reports in their dossiers </li></ul>
  23. 23. How Can I Get Involved <ul><li>Attend the InfoLit meeting in July regarding use of the Information Literacy Assessment Tool </li></ul><ul><li>Prepare to brainstorm a process for widespread implementation </li></ul><ul><li>Work with Shannon to set up an individual account for using the tool </li></ul><ul><li>Start working with departmental faculty in developing learning objectives within ACRL standards </li></ul><ul><li>Begin to develop survey questions that can be vetted through expert review and student feedback </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×