Measure for Measure
“Using collaborative assessment to build stronger information literacy skills”
PRESENTED BY: Susan Gar...
• Private Jesuit and Marymount University in Los Angeles, CA
• 6,064 Undergraduates
• 2,189 Graduates
• Library Open 24/5
...
Info. Literacy Learning Outcomes
Info lit flag
Core Structure
Rhetorical Arts Course Criteria
Assign at least 10% of the final course grade on the basis of information
literacy, with a...
Spring 2014
• 1272 first-year students
had a required library visit
• Across 72 sections of
Rhetorical Arts
3 Info Lit Course Assignments
in Common Syllabus
Library Visit Agenda
1. Intro to LibGuide
2. Why use the library?
3. Evaluating sources
(RADAR game)
4. Finding sources
(d...
Library Support Material
Rhetorical Arts 1000 LibGuide
RADAR Game
• Students were paired
up and given a source
to evaluate
• Using the resources in
the course LibGuide
the stude...
Gamification
• Incorporating Friendly
Competition (points)
• Motivational Feedback
(stars)
• Measure Progress
(showing sta...
Research Strategies Tutorial
(Recommended Homework)
Research
Questions
• How effective was the
face-to-face instruction?
• How effective were the
course-integrated
assignment...
Methodology
Direct Measures
Overall Averages:
• Student scores across 100
sampled annotated
bibliographies. Scored with a
calibrated r...
Indirect Measures
Survey:
• Stratified random sample of 300
students (48% response rate)
• All instructors teaching a sect...
Comments
Survey comments
were analyzed using
NVivo and coded
Major Findings
& Applications
High-Scoring Areas:
o citing sources correctly
o identifying the main
purpose of a source
Low-Scoring Areas:
o identifying...
3.54
4
0
1
2
3
4
5
Helpfulness/Evaluating Evidence
Students
Instructors
Annotated Bibliography
Comment No. of Times
Not helpful for writing/research process 7
Too many steps/make simpler 3
Annotated Bibliography/Comme...
Comment No. of Times
Fun/Engaging 21
Helpful 15
Competition 8
Interactive 7
Liked partner work 6
Not fun/engaging 6
Too ea...
Students who listed a
major in the College of
Science & Engineering
liked the RADAR game
more than the average
student
Sta...
Annotated Bibliography Revised
Added Hints Added More Discussion
RADAR Game Revised
Pre-Search Worksheet
56%22%
22%
Instructors (Mean = 3.44 / 5)
Agree
Disagree
Neutral
51%31%
18%
Students (Mean = 3.17 / 5)
Agree
Disagree
Neutr...
Comment No. of Times
Too easy/not complex enough 17
Did not integrate with or apply to other projects/assignments 5
Add “w...
Research Diary
3.29
3.71
3.33
3.5
2.77
2.58
0 1 2 3 4 5
Reflecting on search process
Helped collect info
Grading rubric/significant
aspec...
Comment No. of Times
Busy work/not helpful 24
Add section on relevancy of sources/source
reflection/how to use/more source...
Pre-Search Worksheet Revised
Continued…
Library Visits/Library Instruction
75%
17%
8%
Instructors (Mean = 3.92 / 5)
Valuable
Somewhat
Not Very
Valuable
46%
30%
24%
Students (Mean = 3.22 / 5)
Valuab...
4.3
4.32
4
3.88
3.57
3.57
3.55
3.67
3.8
3.8
3.41
3.58
3.5
3.53
0 1 2 3 4 5
Scholarly vs. Popular
Getting Help from Librari...
Comment No. of Times
LibGuide 30
Services Offered by Librarians 29
Research Databases 24
Research Strategies 16
Evaluating...
Comment No. of Times
Fine – no suggestions for improvement 50
Research databases/keyword searching 16
Finding scholarly so...
Revised: Finding Sources Activity
(Team-based Active Learning)
LibGuide Student Ranking
Comment No. of Times
Couldn’t find/unaware of 14
Too complicated/confusing/too many links 9
LibGuide: Student/Instructor C...
Research Strategies Tutorial
29%
50%
21%
Instructors Who
Required Tutorial
Yes
No
Not Sure
55%
45%
Students Who
Completed Tutorial
Yes
No
Research Stra...
Student Ranking of Tutorial
3.71 out of 5
Revised: Tutorial as Mandatory
Homework Assignment
Encourage better integration
of library visit and info lit
assignments by assigning
the same research topic
throughout mul...
Thank You:
William H. Hannon Library
Research Incentive Travel Grant
Susan Gardner Archambault
susan.archambault@lmu.edu
@...
Measure for Measure: Using collaborative assessment to build stronger information literacy skills
Upcoming SlideShare
Loading in …5
×

Measure for Measure: Using collaborative assessment to build stronger information literacy skills

741 views

Published on

Presented at the 2015 Northumbria Conference in Edinburgh, Scotland.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
741
On SlideShare
0
From Embeds
0
Number of Embeds
203
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Pre-Search Worksheet Student/Faculty Comments
  • Measure for Measure: Using collaborative assessment to build stronger information literacy skills

    1. 1. Measure for Measure “Using collaborative assessment to build stronger information literacy skills” PRESENTED BY: Susan Gardner Archambault
    2. 2. • Private Jesuit and Marymount University in Los Angeles, CA • 6,064 Undergraduates • 2,189 Graduates • Library Open 24/5 • New Core Curriculum in Fall of 2013 LMU
    3. 3. Info. Literacy Learning Outcomes
    4. 4. Info lit flag Core Structure
    5. 5. Rhetorical Arts Course Criteria Assign at least 10% of the final course grade on the basis of information literacy, with a librarian-led workshop and one or more course-integrated assignments INFORMATION LITERACY OUTCOMES: 1. Conceptualize an effective research strategy and then collect, interpret, evaluate and cite evidence in written and oral communication 2. Distinguish between types of information resources and how these resources meet the needs of different levels of scholarship and different academic disciplines
    6. 6. Spring 2014 • 1272 first-year students had a required library visit • Across 72 sections of Rhetorical Arts
    7. 7. 3 Info Lit Course Assignments in Common Syllabus
    8. 8. Library Visit Agenda 1. Intro to LibGuide 2. Why use the library? 3. Evaluating sources (RADAR game) 4. Finding sources (demo)
    9. 9. Library Support Material
    10. 10. Rhetorical Arts 1000 LibGuide
    11. 11. RADAR Game • Students were paired up and given a source to evaluate • Using the resources in the course LibGuide the student pairs answered questions about the source they were given • Discussion with the librarian after completing the RADAR Game
    12. 12. Gamification • Incorporating Friendly Competition (points) • Motivational Feedback (stars) • Measure Progress (showing star count) • Reward effort (getting the answer right on the first try earns more points)
    13. 13. Research Strategies Tutorial (Recommended Homework)
    14. 14. Research Questions • How effective was the face-to-face instruction? • How effective were the course-integrated assignments? • How effective was the library support material?
    15. 15. Methodology
    16. 16. Direct Measures Overall Averages: • Student scores across 100 sampled annotated bibliographies. Scored with a calibrated rubric by a group of volunteer R.A. instructors
    17. 17. Indirect Measures Survey: • Stratified random sample of 300 students (48% response rate) • All instructors teaching a section (57% response rate)
    18. 18. Comments Survey comments were analyzed using NVivo and coded
    19. 19. Major Findings & Applications
    20. 20. High-Scoring Areas: o citing sources correctly o identifying the main purpose of a source Low-Scoring Areas: o identifying intended audience o authority of the author o accuracy of the evidence o bias Annotated Bibliography
    21. 21. 3.54 4 0 1 2 3 4 5 Helpfulness/Evaluating Evidence Students Instructors Annotated Bibliography
    22. 22. Comment No. of Times Not helpful for writing/research process 7 Too many steps/make simpler 3 Annotated Bibliography/Comments
    23. 23. Comment No. of Times Fun/Engaging 21 Helpful 15 Competition 8 Interactive 7 Liked partner work 6 Not fun/engaging 6 Too easy 6 Not useful 5 RADAR Game/Student Comments
    24. 24. Students who listed a major in the College of Science & Engineering liked the RADAR game more than the average student Statistically Significant
    25. 25. Annotated Bibliography Revised
    26. 26. Added Hints Added More Discussion RADAR Game Revised
    27. 27. Pre-Search Worksheet
    28. 28. 56%22% 22% Instructors (Mean = 3.44 / 5) Agree Disagree Neutral 51%31% 18% Students (Mean = 3.17 / 5) Agree Disagree Neutral Helpfulness in Developing a Topic
    29. 29. Comment No. of Times Too easy/not complex enough 17 Did not integrate with or apply to other projects/assignments 5 Add “working bibliography” of sources 4 Layout needs more space 4 Add reflective questions re: terminology & inclusion/exclusion choices 3 Add citation tracking element 1 Have instructor approve the topic 1 Allow space to change positions 1 ID stakeholder and their position 1 Pre-Search Worksheet Student/Faculty Comments
    30. 30. Research Diary
    31. 31. 3.29 3.71 3.33 3.5 2.77 2.58 0 1 2 3 4 5 Reflecting on search process Helped collect info Grading rubric/significant aspects of learning Grading rubric/clear criteria Student Instructor Research Diary
    32. 32. Comment No. of Times Busy work/not helpful 24 Add section on relevancy of sources/source reflection/how to use/more source interaction 6 Timing/better integration with other assignments (earlier in semester) 2 Research Diary Student/Faculty Comments
    33. 33. Pre-Search Worksheet Revised
    34. 34. Continued…
    35. 35. Library Visits/Library Instruction
    36. 36. 75% 17% 8% Instructors (Mean = 3.92 / 5) Valuable Somewhat Not Very Valuable 46% 30% 24% Students (Mean = 3.22 / 5) Valuable Somewhat Not Very Valuable How Valuable Are Library Visits?
    37. 37. 4.3 4.32 4 3.88 3.57 3.57 3.55 3.67 3.8 3.8 3.41 3.58 3.5 3.53 0 1 2 3 4 5 Scholarly vs. Popular Getting Help from Librarian Finding Info Finding a Topic Evaluating Appropriateness/ Info Citing Sources Using Evidence/Make Arguments Student Instructor How Helpful Are Library Visits?
    38. 38. Comment No. of Times LibGuide 30 Services Offered by Librarians 29 Research Databases 24 Research Strategies 16 Evaluating Sources 13 What Did Students Find Most Helpful?
    39. 39. Comment No. of Times Fine – no suggestions for improvement 50 Research databases/keyword searching 16 Finding scholarly sources 13 Repetition of FYS or other course 10 Too easy 10 Better relevance or integration with real assignment 8 Citing 8 Navigating library web site or LibGuide 8 What Needs Improvement? Student/Faculty Comments
    40. 40. Revised: Finding Sources Activity (Team-based Active Learning)
    41. 41. LibGuide Student Ranking
    42. 42. Comment No. of Times Couldn’t find/unaware of 14 Too complicated/confusing/too many links 9 LibGuide: Student/Instructor Comments
    43. 43. Research Strategies Tutorial
    44. 44. 29% 50% 21% Instructors Who Required Tutorial Yes No Not Sure 55% 45% Students Who Completed Tutorial Yes No Research Strategies Tutorial: Usage
    45. 45. Student Ranking of Tutorial 3.71 out of 5
    46. 46. Revised: Tutorial as Mandatory Homework Assignment
    47. 47. Encourage better integration of library visit and info lit assignments by assigning the same research topic throughout multiple assignments Additional Improvement
    48. 48. Thank You: William H. Hannon Library Research Incentive Travel Grant Susan Gardner Archambault susan.archambault@lmu.edu @susanarcham

    ×