Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Peer feedback on writing: Is more better?
A pilot study in progress
Christina Hendricks & Jeremy Biesanz, University of Br...
Upcoming SlideShare
Loading in …5
×

Peer Feedback On Writing: Is More Better? A Pilot Study in Progress (poster)

711 views

Published on

This is a slide of a poster that was presented at the Society for Teaching and Learning in Higher Education conference in Vancouver, BC, Canada, in June of 2015. Download to view as Power Point slide and enlarge to see it all!

Published in: Education
  • Be the first to comment

  • Be the first to like this

Peer Feedback On Writing: Is More Better? A Pilot Study in Progress (poster)

  1. 1. Peer feedback on writing: Is more better? A pilot study in progress Christina Hendricks & Jeremy Biesanz, University of British Columbia Figure 1: Instructor comments: number of comments rated as “2” in each category Figure 3: Student peer comments: number of comments rated as “2” in each category 1st Method Course: A year-long, writing-intensive course for first-year students at a large university. Students write 10-12 essays over the year, and give/receive peer feedback on each one. Participants: 13 students agreed to participate in this pilot study. We plan to do a larger study in 2015- 2016. Measures: 10 essays from each of 13 participants; all the student peer comments (n=1218) and all the instructor comments on those essays (n=3263). Procedures: All comments were coded according to a rubric with four categories (and subcategories within each): Strength of Argument, Insight, Organization, and Style & Mechanics. Fleiss’ Kappa for coders: 0.61; for the most frequently used categories: 0.8. Essays will be coded with the same rubric. Comments were rated as 1 (very negative), 2 (somewhat negative), or 3 (positive). We will analyze all data according to the design in Figure 5. Introduction There is ample evidence that both giving and receiving peer feedback improves student writing (e.g., Topping, 1998; Cho & MacArthur, 2010; Cho & MacArthur, 2011; Cho & Cho, 2011; Li, Liu & Steckelberg, 2010, Crossman & Kite, 2012). This pilot study addresses two questions for which there isn’t yet a great deal of research evidence: 1. How do students use peer comments given and received for improving later essays rather than on drafts of the same essay? 2. Are students more likely to use peer comments given and received for improving their writing after more than one peer feedback session? How many sessions are optimal? We are also asking: 3. Do students use peer comments on later essays even if the instructor did not make similar comments? 4. Does the quality of student comments improve over time? Contact: c.hendricks@ubc.ca Essay 1 quality Essay 2 quality Essay 3 … n=10 quality Essay 1 comments Essay 2 comments Essay 3 … n=10 comments Figure 5: Cross-lagged panel design with auto-regressive structure. After we code the essays in Summer 2015 we will analyze these relationships for all 10 essays from each student Figure 2: Instructor comments: number of comments rated as “3” in each category Figure 4: Student peer comments: number of comments rated as “3” in each category 0 2 4 6 8 10 12 024681012 Essay Number InstructorNumberofComments Argument Strength Style Insight Organization 0 2 4 6 8 10 12 012345 Essay Number InstructorNumberofComments Argument Strength Style Insight Organization 0 2 4 6 8 10 12 01234 Essay Number StudentNumberofComments Argument Strength Style Insight Organization 0 2 4 6 8 10 12 0.00.51.01.52.02.53.0 Essay Number StudentNumberofComments Argument Strength Style Insight Organization .31*** .11** .08** -.28** -.04* -.16** .19** Results so far We have only analyzed the student and instructor comments so far. Trends over time: There is a strong upward trend in instructor comments rated “3” in each category over time. There is also a significant downward trend in the number of instructor comments rated “2” in Strength and Insight. The student peer comments are more erratic, with only a significant downward trend in comments rated “2” in Style. Student/instructor agreement: The average comment ratings agree strongly between student and instructor: b = .48, t(297) = 5.08, p < .00001. However, this agreement goes down across essays: b = -.04, t(296) = -3.02, p = .003. This is because instructor ratings go up over time, b = .033, t(107) = 5.84, p < .00001, while student ratings increase at only half that rate, b = .016, t(86) = 1.89, p = .06. Auto-regressive analysis of comments from essay 1 to essay 2, essay 2 to essay 3, etc. There are significant effects for instructor comments of “3” in Strength (.33***) and Style (.34****). For student comments, there are significant effects in ratings of “3” in Strength (.20*), Style (.22**), Organization (.20*), and for ratings of “2” in Strength (.23*) and Style (.27**). *p < .05, **p< .01, ***p< .001, ****p < .0001 References Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Science, 39, 629-643. Cho, K., & MacArthur, C. (2010). Student revision with peer and expert reviewing, Learning and Instruction, 20, 328-338. Cho, K. & MacArthur, C. (2011). Learning by reviewing, Journal of Educational Psychology, 103(1), 73-84. Crossman, J. M., & Kite, S. L. (2012). Facilitating improved writing among students through directed peer review, Active Learning in Higher Education, 13, 219-229. Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536. Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68, 249- 276.

×