1. Rotating Summarizing Roles in Online Discussions
Effects on Learners’ Listening Behaviors During
and Subsequent to Role Assignment
Supported by the Social Sciences and Humanities Research Council of Canada
AERA 2013, San Francisco CA
Alyssa Friend Wise
Ming Ming Chiu

2. Overview
• Study investigated effects of summarizing (Synthesizer,
Wrapper) roles on how students attend to the posts of
others in online discussions
• Multilevel, cross-classification modeling of listening
behaviors during in-role and subsequent weeks
• In-role, students increased their breadth of listening,
(Synthesizers also increased depth somewhat), but
effects were not sustained post-role.
• Negative post-role behavior changes suggest unintended
effect of role rotation on abdication of responsibility.

3. Merging Two Lines of Research
Online Listening
• Students attend to others’
posts in multiple distinct
ways (e.g. disregardful,
targeted, social, coverage,
interactive)
• Some naturally engage in
productive behaviors, others
need support
(Wise et al., 2011a; 2011b; 2012)
Roles Assignment
• Students assigned
summarizing roles tend to
make posts in more
advanced phases of
knowledge construction
• Is part of the mechanism
for this due to improved
online listening?
(De Wever et al. 2007; Schellens et
al. 2005; 2007; Wise & Chiu, 2011)
Theoretical Framework

4. Online “Listening” and “Speaking”
• From a social constructivist perspective, the goal
of online discussions is for learners to build
understanding through dialoging with other
• This involves 2 basic interrelated processes:
SPEAKING
Externalizing one’s ideas by contributing posts to an
online discussion
LISTENING
Taking in the externalizations of others by accessing
existing posts
Theoretical Framework
(NOT LURKING!)
(NOT (JUST) READING)

5. Dimensions of Listening Behavior
Dimension Definition
Breadth The quantity of unique posts one views - important in terms
of the diversity of ideas a learner is exposed to.
Depth The length of time spent reading posts – important to allow for
deeper consideration of others’ ideas.
Temporal
Contiguity
The degree to which learners disperse or concentrate their
participation - important for integration, development over time.
Revisitation The extent to which students return to posts made by
themselves and others – important in connecting ideas across
the discussion over time.
Theoretical Framework

6. Role Assignment
• Roles which serve a summarizing function consistently
result in posts in more advanced phases of knowledge
construction
– Wrapper (summarize at end of discussion)
– Synthesiser (summarize midway through discussion)
• Mechanism for achieving advanced KC posts unclear –
might be related to broader and deeper listening
• If listening activity internalized, may be sustained by
students after role is no longer assigned
Theoretical Framework

7. Role Rotation
• Common strategy for assigning roles is to rotate them
across discussions weeks to give benefit to all students
• But there is a danger that when role-work is complete,
students may feel a reduced responsibility to group
• Problem potentially exists any time a role-rotation
strategy is used, but currently uninvestigated
Theoretical Framework

8. Research Questions
1. Does being assigned a summarizing role impact
students’ listening behaviors in an online
discussion during their assigned role week?
2. Are any effects of summarizing roles sustained in
subsequent weeks when the role is no longer
assigned?
3. Are there any other changes in listening
behaviors in weeks subsequent to that of the
role-assignment?
Research Questions

9. Learning Environment
• Fully online course
– Educational Psychology
– Undergraduate students (33/95)
– Participants mostly typical of course, high-achieving
• 6 week-long discussions; groups of ~12 students
– Contrasting perspectives on real world teaching
questions (e.g. ability tracking), must come to consensus
– Require 3 posts on different days, 20% of grade.
– Basic open-source asynchronous threaded forum
Methods

10. Role Assignment
• Each week two students assigned specific roles
to play in the discussion
Synthesizer
– Makes a post around the second day summarizing the
initial ideas contributed and pointing a way forward
Wrapper
– Pulls discussion together at the end with a summary
post of the group’s final response to the debate
Methods

11. Data Extraction and Processing
Date Time Session Action Duration
(min)
Length
(words)
Message #
6/3/2011 23:46 1 Read 44.43 413 447
6/3/2011 23:52 1 Read 1.73 60 455
6/4/2011 00:08 1 Scan 0.23 117 459
6/4/2011 00:09 1 Read 12.51 413 460
6/4/2011 23:49 2 Post 3.18 120 477
• Actions: View (read or scan), Post, Review (and edit)
– Scan vs. Read (6.5 words per second threshold) (Hewitt 2003)
• Session closed when action length > 60 min
• Summary variables calculated per student per week
– 33 students x 6 weeks = 198 student-weeks
Methods

12. Variables Calculated
Dimensions Variables (calc’d per discussion week)
Listening Breadth • Percent of other’s posts read (at least once)
• Percent of other’s posts viewed (at least once)
• Average number of views per session
Listening Depth
• Percent of total views that were reads (not scans)
• Average length of time reading a post
Temporality • Number of sessions
• Average length of session
• Percent of sessions with posts
Reflectivity
• Number of reviews (of one’s own posts)
Posting • Number of posts made
• Average length of posts
• Average time to create posts
Methods

13. Statistical Analysis
• Simultaneously modeled the 12 listening and posting
outcome variables with multilevel cross-classification
models
• Explanatory variables entered in sequential sets by
level of granularity and tested for significance
– Non-significant variables removed immediately to reduce the
likelihood of false positives
– Two-stage linear step-up procedure used to control for the
false discovery rate
– Alpha level of .05
Methods

14. Summary Statistics
• Discussions ranged 30 to 60 posts per week
• Great diversity in individual listening behaviors
– Students logged in between 1 and 30 times
– Sessions ranged from 2 min to 2 hrs in length
– Post every session versus post in few sessions
– Viewed 5-100% of others’ posts; mean of 75%
– Read 3-94% of others’ posts; mean of 45%
– Length of time reading a post 0-15 min; mean of 3.5
– Reviews ranged from 0-29; mean of 2.2
Results

15. Explanatory
Model
# of
sessions
% of
sessions w/
posts
Average # of views per
session
% of
others’
posts
viewed
% of
others’
posts read
% of total
views that
were reads
Average length of
timereading post
# of
reviews
# of posts
made
Average
length of
posts
Average time to create
posts
1st
Week
Group
size
% of women
Mean age
SD age
Mean course grade
Woman
Age
Course
grade
Goal orientation
Synthesizer
Wrapper
Role week
already occurred
Week Group Group-week Student Student-week Dependent variables
(no sig. var.)
–0.29 **
+0.13 **
+0.20 ***
–0.14 **
–0.25 ***
+0.38 ***
+0.53 **
–0.17 **
–0.14 *
–0.14 **
+0.25 ***
+0.16 *
+0.36 **
+0.22 ***
+0.26 ***
+0.34 *
+0.48 ***
+0.60 ***
–0.13 **
+0.18 ***
+0.35 ***
–0.27 ***
+0.56 **
+0.31 ***
–0.84 ***
–0.39 **
+0.86 ***
+1.16 ***
–0.96 ***
–0.66 ***
+0.27 ***
+0.84***
+0.30 ***
+0.21 **

16. Summary of Role Effects
Results
Dimension Variable Synthesizer Wrapper Post-Role
Breadth Percent of others' posts read 0.34 0.48
Percent of others' posts viewed 0.16
Average number of views per session
Depth Percent of total views that were reads 0.56
Average length of time reading a post
Temporality Number of sessions -0.29
Average length of session
Percentage of sessions with posts 0.38
Reflectivity Number of reviews -0.39
Posting
Number of posts made 0.86
Average length of posts 0.84
Average time to create posts

17. Discussion
• Students are more comprehensive in their listening when
performing a summarizing role
– Possible factor contributing to elevated knowledge construction
– Could directly encouraging breadth of listening have simile effect?
– Interpretation of being sustained only superficially post-role
• Limited effects of summarizing roles on listening depth
– Unexpected. Ceiling effect? Artificial smoothing? Quality issues?
• Apparent abdication of responsibility
– Rotation intuitively appealing as equitable, but benefits may be
outweighed by negative post-role effects
– More beneficial to assign roles to only some students for the first
part of discussion?
Discussion

18. Conclusions
• Listening breadth may be part of the
mechanism contributing to the increased
quality of summarizing-role posts
• Additional strategies to encourage sustained
comprehensive student listening in online
discussions are needed
• Urge caution in using role rotation; need to
investigate how to assign roles to maximize
benefits for all students.
Conclusions

19. Thank you! Questions?
Alyssa Friend Wise
Simon Fraser University
alyssa_wise@sfu.ca
Ming Ming Chiu
State University of New York – Buffalo
mingmingchiu@gmail.com