Peer Review In a Library Research Course
•Download as PPTX, PDF•
1 like•528 views
These are slides for a talk about a peer review assignment I gave in my section of a library research course. The assignment required students to use the citation manager, Flow, to share student bibliographies and annotations. In the talk, I reflect of some of what I learned from the experience and what I may try differently the next time I give this assignment.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Peer Review In a Library Research Course
Similar to Peer Review In a Library Research Course (20)
More from John Pell
More from John Pell (20)
Recently uploaded
Recently uploaded (20)
Peer Review In a Library Research Course
- 1. Peer Review in Library 100- Introduction to Information Research John Pell Assistant Professor Hunter College Libraries
- 2. What We Did • Students reviewed the annotated bibliographies of five fellow students using an assessment form. • The instructor supplied students with their anonymous peer review scores.
- 3. What We Used • Flow Citation Management • Google Forms
- 4. What it Looked Like: Flow Shared Space
- 5. What it Looked Like: Google Form
- 6. Mistakes Were Made • Effort to Make the Peer Review Process Anonymous Failed – However, this resonated with class discussions and assigned readings about the limitations of peer review and possible improvements to the peer review model: • Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D, investigators P. Does masking author identity improve peer review quality?: A randomized controlled trial. Jama. 1998;280(3):240-242.
- 7. What I Will (Maybe) Do Differently • Assign fewer peer reviewers • Use an open peer review model • Have students use the same rubric as me • Share the peer reviewers comments • Require a self-assessment
- 8. Appendix: Library 100 Objectives • Improve student abilities to: – define and critique concepts in scholarly communication such as plagiarism, authority, peer review, media bias, publication bias, conflict of interest, and publication impact – state research questions that reflect awareness of the scope of available information and current research priorities – translate research questions into strategies for retrieving information – use technology to organize and publish an annotated bibliography – cite information sources accurately and ethically according to academic style guidelines.
- 9. Appendix: Annotated Bibliography Grading Rubric
Editor's Notes
- Library 100 is a course offered through Hunter College libraries that was developed to meet ACRL Information Literacy Standards. The complete course objectives are listed at the end of these slides. Suffice to say for now that students in the course are required to understand how peer review is carried out and how to identify peer-reviewed journals. Since the concept of academic peer-review makes up part of the content of the course- it is appropriate for students to apply some form of peer review to their course work.
- Students in my section completed a brief annotated bibliography for their midterm project that I evaluated according to the rubric listed in the “Appendix” section of this slide show. For their final project, they revised this annotated bibliography according to the mid-term feedback that they received, submitted the revised version, and completed a peer-review assessment of five of their classmates using an evaluation form that represented a simplified version of the rubric that I used to evaluate their bibliographies.
- I used two technological tools that are readily available at Hunter College: Flow and Google Forms.
- Flow is a citation manager published by ProQuest that is available to Hunter Affiliates at no additional cost. It allows its users to share bibliographic data and files with other Flow users. Students shared their bibliographies and annotations according to a protocol that was intended to protect anonymity.
- Students used this Google form to submit their peer evaluations. I adapted the evaluation form from the rubric that I used to evaluate their assignments. The form asks them to rate the quality of the bibliography according to criteria that were established in the course and then to explain why they gave that rating. They were asked to do the same for the quality of the annotations. I reported the peer review ratings and summarized student comments in my feedback to students on their final project.
- The assignment included a protocol that was intended to make the peer review process anonymous. This protocol failed due to my flawed understanding of how the shared area in Flow displays information. In order to make a shared collection in Flow anonymous the owner of the collection must make their account user name anonymous- I assumed it would be sufficient to give the collection an anonymous name. I caught this mistake before the students were due to carry out their peer review and gave revised instructions about how to make the collections anonymous, but anonymity had already been compromised. I attempted to turn this into a “teachable moment” by tying the occasion in with some of our readings about how anonymous peer review may fail to live up to its ideals.
- Reflecting on the experience, a few possible modifications occur to me. The first is to assign fewer reviewers. The first time I gave this assignment, I only shared the student’s peer reviewer ratings- not the actual comments. Since I was only delivering a numerical score to the student, I thought five data points would make for a more meaningful median score than three. I also worried about directly sharing the raw student reviewer comments- I thought this might compromise the anonymity of the process. Having reflected on the experience, I am inclined to attempt an open peer review model where the students have access to their reviewer’s comments in their entirety; I think that such a model may provide more meaningful feedback and encourage even more careful and constructive comments (even in the anonymous model, I was pleased with the constructive attention to detail of the student comments.) I gave a simplified evaluation form because I assigned so many reviews. Assigning fewer reviews could make a more complex rubric a more manageable task.