1. Improving Teaching and Learning:Using Rubrics to Measure Information Literacy Skills QQML Conference 2012 Kathy Crowe Associate Dean for Public Services
2. Information literacy program• Long history of commitment to information literacy• Two positions devoted to information literacy• 650 classes 2010-11• Online tutorial since 2000• Information literacy game 2008
3. Information literacy in the curriculum at UNCG• Librarians serve on UNCG Faculty Senate committees• Information literacy included in one of core General Education learning goals: – Foundational skills: “Think critically, communicate effectively, and develop appropriate fundamental skills in quantitative and information literacies.”
4. Information literacy assessment• Established student learning outcomes 2009• Experimented with a variety of methods – pre and post tests, clickers, one minute papers, worksheets• Administered iSkills test Spring 2010
5. Assessment study• Communication Studies (CST 300)• Four semester study – Pilot Spring 2009 – Fall 2009 & Spring 2010 – Added inter-raters Spring 2011
6. CST 300• CST 300 (Communication Theory) required of all CST majors• Students required to write 8-10 pp paper using primary research articles• Long history of collaboration with the Library – Students attend library instruction session – Librarian prepares Lib Guide – Student complete worksheet
7. CST 300 worksheet• Identify theory and context• Identify keywords and Boolean search• Identify 1 book or chapter• Identify 3 primary source articles• Use APA citation style
8. Five questions (Gilchrist and Zald) ACRL Information Literacy Assessment Immersion What do you want the 1. Outcome student to be able to do? What does the student need 2. Curriculum to know in order to do this well? What type of instruction will 3. Pedagogy best enable the learning? How will the student 4. Assessment demonstrate the learning? How will I know the student 5. Criteria for evaluation has done this well?
9. Information literacy outcomes• Students construct a search strategy using appropriate vocabulary and Boolean operators in order to search for information effectively• Students distinguish primary source journal articles in order to gather appropriate resources for a research paper• Students apply an established citation style in order to document the sources they use appropriately.
10. Course outcome for information literacy• Apply a working knowledge of information literacy as a tool for scholarship in communication studies including APA style for professional writing , library search techniques and use of primary sources (journal articles and other research publications)
11. CST Department learning outcome• 4. Engage communication scholarship using appropriate theory and research methods. 4.1 Complete an independent research project that applies appropriate theory and methodology to a communication problem.
12. Curriculum and Pedagogy• Students attended class • Using encyclopedias and session by a librarian texts to help choose• Completed worksheet relevant vocabulary (revised to match • Using Boolean operators outcomes) • Library catalog • Selecting and using databases • Scholarly vs. popular articles • Primary source articles • Communication Studies journals • APA handout
13. Assessment• Developed a rubric to score library worksheet
14. Outcome #1 (CST 300)Students construct a search strategy using appropriate vocabulary and Boolean operators inorder to search for information effectively. Criteria: Uses two variables in search strategy with “and” operator Uses two topic-related variables with (3) Excellent “and” operator appropriately Uses two variables but 1 term (2) Good doesn’t match topic or uses “and” operator inappropriately Uses one topic-related variable and (1) Acceptable doesn’t use “and” appropriately Missing variables and terms don’t (0) Needs Improvement match topic. Doesn’t use “and” operator
15. Outcome #2Students distinguish primary source research material in order to gather appropriate resourcesfor a research paper. Criteria: Student selects Appropriate Includes theory primary source journals or books and context in research material material All items from All items are primary Communication/Allied Theory and context in (3) Excellent source research all items journals or scholarly books 2 items from Two items are primary Communication/Allied Theory and context in (2) Good source material journals or scholarly 2 items books 1 item from One item is a primary Communication/Allied Theory and context in (1) Acceptable source journals or scholarly 1 item books No items from Does not include (0) Needs No items are primary Communication/Allied theory in context in Improvement research journals or scholarly any items. books
16. Outcome #3Students apply an established citation style in order to document the sources they useappropriately. Criteria – APA format: Student includes all components of a citation in correct order, format and punctuation in accordance with the APA style. Citations include all components with correct order, (3) Excellent format and punctuation Citations are missing one of the following: (2) Good •1 item out of order •1 punctuation item Citations are missing one of the following: •1 component (1) Acceptable •1 item out of order •1 punctuation item Citations are missing two of the following: •1 component (0) Needs Improvement •1 item out of order •1 punctuation item
17. Results -Skills 2009 Skills Spring 20091.61.41.2 10.80.60.40.2 0 Q1 two variables Q2 and operator Q3 theory & context Q4 primary research Q5 appropriate Q6 APA in sources articles journals
19. Recommendations after pilot• Fall 2009 • Spring 2010 – Require students to take – Developed tutorial on selected chapters of primary sources in Libraries tutorial Communication Studies – Delay instruction session so that students have more time to think about their topics – Revise rubric to include four levels – Score the annotated bibliography
23. Conclusions and outcomes• Pilot provided solid data that students not gaining information literacy skills• One-shot lecture and worksheet not adequate• APA performance still a challenge• Closed the loop on assessment• Enhanced collaboration with teaching faculty in CST
24. Outcomes, cont.• CST did faculty survey Fall 2010 to improve delivery of information literacy skills across their curriculum• As a result crafted 8 information literacy learning outcomes for majors• Added SLO to all syllabi: – Apply a working knowledge of information literacy as a tool for scholarship in communication studies including APA style for professional writing, library search techniques, and use of primary sources (journal articles and other research publications).
25. Award!• Project won the 2012 Student Learning Enhancement Award at UNCG!
26. Challenges• Devising rubric takes time• Scoring worksheets takes time (about 7-10 minutes for each)• Gaining evidence far outweighs challenges