CSCL Tools for Regulating Collaboration & Teamwork


Published on

Mariel Miller & Allyson Hadwin, University of Victoria
Presented at the 2013 conference for the Canadian Society for the Study of Education (CSSE)

Increasing emphasis on collaboration in academic and work contexts means learners are required to develop skills for regulating teamwork. The purpose of this study was to examine scripting and visualization tools for supporting regulation of shared task perceptions during a complex collaborative task. Prior to the task, groups engaged in individual and group planning using either (a) a scripting tool structuring regulation including task analysis, or (b) a scripting tool augmented with visualization of each member’s task perceptions. Findings indicated that, in both groups, shared task perceptions were generally accurate in relation to the instructor’s expectations. However, groups (a) struggled to construct consensus among diverse individual perceptions, (b) demonstrated little active and purposeful construction of shared task perceptions, and (c) encountered planning related challenges during collaboration. Groups showed small improvements across assignments, however, many difficulties re-occurred.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CSCL Tools for Regulating Collaboration & Teamwork

  1. 1.!Negotiating Task Perceptions During Computer-Supported Collaborative Problem SolvingMariel Miller & Allyson HadwinUniversity of Victoria•  Collaboration is a critical 21st century skill in today’s knowledgeeconomy. Unfortunately, groups commonly encounter challengeswith strategic planning, such as different understandings of thetask, derailing their efforts •  While CSCL tools for enhancing collaboration have becomecommon place in classrooms, their potential for supporting sharedregulation & planning has been largely overlooked.IntroductionShared task perceptions are critical for collaborationShared task perceptions in SSRL provide teams with foundational metacognitiveknowledge for collaboration. Regrettably, left to their own devices, groups oftenmisperceive tasks and fail to actively engage in this process. •  Construction of shared task perceptions isa demanding transactive social processin which groups leverage diversity in eachothers’ perceptions to construct a unifiedunderstanding of what the task requires(explicit task perceptions) and why (implicittask perceptions). •  Furthermore, shared task perceptions alsoneed to be accurate with the instructors’expectationsThe purpose of this cross-case comparison was to examine scriptingand visualization tools for supporting groups’ construction of sharedtask perceptions in two complex collaborative assignments. Specifically, we compared one group in each condition to explore:1.  Whether groups constructed shared task perceptions aligned witheach other as well as the instructor2.  How groups engaged in negotiation of shared task perceptions3.  Challenges groups encountered during collaboration4.  Changes in shared task perceptions and challenges across tasksPurposeBodemer, D., & Dehler, J. (2011). Group awareness in CSCL environments. Computers in Human Behavior, 27(3), 1043–1045. doi: 10.1016/j.chb.2010.07.014Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P.A. Kirschner, (Ed.) Three Worlds of CSCL. Can We Support CSCL, pp. 61–91. Heerlen: Open Universiteit Nederland.Hadwin, A. F., Järvelä, S. & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. In B. Zimmerman & D. Schunk (Eds.). Handbook of Self-Regulation of Learning and Performance (pp. 65-84). New York: Routledge. Järvelä, S. & Hadwin, A. F., (2013). New Frontiers: Regulating Learning in CSCL Regulation is a Neglected Area in CSCL. Educational Psychologist.Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in collaborative problem solving. Computers andEducation, 51, 279-296.Miller, M., & Hadwin, A. F. (submitted, 2013) Investigating undergraduate students’ perceptions of academic tasks: Do perceptions of implicit and explicit task information predict task performance?Miller, M., & Hadwin, A. (2012, April). Social aspects of regulation: Measuring socially-shared regulation in collaborative contexts. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, BC, CA. Patton, M. Q. (1990). Qualitative evaluation and research methods. Thousand Oaks, CA: Sage. Soller, A., Martínez-Monés, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15(4), 261-290.Weinberger, A., Ertl, B., & Fischer, F. (2005). Epistemic and social scripts in computer-supported collaborative learning. Instructional Science: An International Journal of Learning and Cognition, 33, 1–30. Weinberger, A., Stegmann, K. & Fischer, F. (2007). Knowledge convergence in collaborative learning: Concepts and assessment. Learning & Instruction, 17, 416-426.Winne, P. H., Hadwin, A. F., & Perry, N. E. (2012). Metacognition and computer supported learning. In C. Chan & C. Hmelo-Silver (Eds.). The international handbook of collaborative learning. New York: Taylor and Francis. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Hillsdale, NJ: Lawrence Erlbaum. 371.References•  CSCL tools may provide effective support for regulation. o  Previous research suggests groups often neglect implicit taskcriteria and focus on shallow explicit task criteria (Miller & Hadwin,2012). However both groups discussed multiple layers of the taskand shared task perceptions were generally accurate.•  Groups may have been unaware of the role of taskperceptions in effective collaboration or lacked skillsneeded for shared task analysis. o  After the shared planning session in both assignments, groupmembers continued to hold different ideas about the task.o  Discussion was shallow and groups demonstrated little effort indiscussing diverse interpretations.o  May have posed issues while working together since both groupsreported encountering numerous challenges related to poorplanning.o  Neither group demonstrated substantial improvements in sharedplanning across tasks despite acknowledging multiple planningchallenges.•  Groups may need more support in order to effectively useCSCL tools for regulation.o  Visualization group did not recognize the purpose of the tool oruse it in Assignment 1. More explanation about how to interpret/use the tool may have been needed. o  Visualization tool format (displaying the frequency of individuals’task perceptions) may have prompted groups to findcommonalities rather than fully leverage each others’ taskperceptions. o  Scripting group had many unacknowledged ideas in chat.Suggests scripting alone may not have adequately helped groupmembers build awareness of the breadth of interpretations thatexisted within the groupConclusions•  Extends knowledge of how groups construct shared taskperceptions & how CSCL tools can be leveraged to supportthis process •  Extends measurement beyond the traditional focus onindividual reports, outcomes, and processes by exploringsocial regulation across the individual and group level usingmultiple data sources •  Informs ways instructors can leverage commonly availabletools, such as Moodle, to support socially shared regulation &planning. Implications•  Future research is needed to: •  Corroborate findings using other measures, such asinterviews, since measurement of shared task perceptionsdepended on individuals’ and groups’ externalization oftask perceptions during task analysis & discussion•  Examine a larger sample of scripting and visualizationcondition groups to compare effectiveness of each tooland inform future design.Considerations & FutureResearchScripting Group Visualization GroupGender 2 M2 F2 M2 FPrior Knowledge Mdn = 71.14 Mdn = 66.67Assignment Grade 56.70 60.00This study was conducted in a first year undergraduate course,“Strategies for University Success” (ED-D 101). The purpose of thecourse was to develop theoretical and practical understandings aboutSRL. Coursework included two collaborative assignments in whichstudents worked in groups of four assigned at the outset of the course.Groups worked online in Moodle via chat to analyze a problem casescenario and identify the root of the problem or suggest solutions.Data included individual & group planning tool responses, chat discussions, log files of tool use, and self-reported planning challenges in solo reflections. To address theresearch questions, group profiles were created to compare scripting and visualization groups across both assignments in terms of the following:Analysis & FindingsPurposeful case sampling was used to select two groups (Patton,1990). The first group received a macro scripting tool guidingregulation of collaboration and a microscripting tool guiding individualand shared task analysis (Scripting Condition). The second groupreceived the same macro & microscript augmented with visualizationof each member’s individual task perceptions (Visualization Condition)ParticipantsNote. M = Male, F= FemaleShared TaskPerceptionsSimilarities and differences in ideas about the task in responses to solo and group planning tools were identified. Discussion of each idea was then identified in chatdiscourse. Ideas about the task were coded as shared when groups (a) adopted individuals’ task perceptions, or (b) generated novel task perceptions duringdiscussion not previously considered by any individual. Ideas were coded as divergent when groups failed to acknowledge or discuss individuals’ ideas about thetask.Accuracy of SharedTask PerceptionsGroups’ shared task perceptions of explicit and implicit task features were compared to those of the course instructor. Accuracy ranged from 0 to 10 with 0indicating none of the groups’ shared task perceptions matched those of the instructor and 10 indicating perfect accuracy.Mode of Negotiation The degree to which groups engaged in dynamic and transactive negotiation of task perceptions was measured by coding discussion of each idea in chat as (a)Quick-consensus (group accepted suggested idea at face value, e.g. ‘okay’); (b) Integrated-oriented consensus (group accepted ideas by rephrasing or adding tothe idea without changing its meaning); (c) Conflict-oriented consensus (suggested idea was modified before being accepted); or (d) not acknowledged’ (groupmembers ignored or failed to respond to a suggested idea) (c.f. Weinberger et al., 2007). Planning Challenges Planning challenges encountered during collaboration were examined using frequency analysis of group members’ responses to self-report items on the soloreflection activityTool Use Moodle log files and chat records were used to examine whether group members used the visualization tool. Theoretical FrameworkSuccessful collaboration requires learners not only to self-regulatetheir cognition, behaviour, and motivation in a collaborative task(SRL), but also socially share regulation (SSRL) as a group.TaskPerceptionsGoals &PlanningStrategicEnactmentLarge ScaleAdaptationSRLShared TaskPerceptionsSharedGoals &PlanningSharedStrategicEnactmentSharedLarge ScaleAdaptationSSRLEach team memberregulates his/herstrategicengagement Collectiveregulation of groupprocesses andsuccessfulcoordination ofstrategiesFailure toNegotiateShared TaskPerceptionsMisalignedstandards andplans for thetaskLimitedopportunitiesfor monitoring,evaluating &adaptingDifficultiesduringcollaboration /Weaker GroupPerformanceCSCL Tools for RegulationTwo types of CSCL technologies offer potential for regulation(Bodemer & Dehler, 2011; Dillenbourg, 2002; Jermann & Dillengbourg, 2003; Soller, et al., 2005; Weinberger et al., 2005)Structuring ToolsMacro & microscripting collaborationby specifying &sequencing activities /discussionVisualization ToolsSupport group awarenessthrough graphicvisualizations of groupprocesses and activities. (Hadwin, Järvelä, Miller, 2011; Jarvela & Hadwin, 2013; Miller et al. 2012; Miller & Hadwin, submitted; Winne et al., 2012; Winne & Hadwin, 1998)MasterConceptsSolo &SharedPlanningIn-ClassCollabProblemSolvingAssignmentSoloReflection onCollaborationGroup Planning Tool for CollaborativeChallengeLast week you created a solo planner. Now create a group plan to helpyou work together during the challenge. Discuss all questions belowwith your group in chat•  Term test of the four topics covered in the assignment.•  Each group member selected one topic and constructed a “cheatsheet” to cue memory of critical information that could be used inthe assignment.•  Scripted individuals to choose thefive correct answers for explicit(What is my group being asked todo ) and implicit (Why is my groupbeing asked to do this) assignmentfeatures•  Parallel microscript completed as agroup at the beginning of theassignment class.Groups completed the synchronous collaborative assignmentvia group chat in Moodle during one 90-minute class.Visualization Condition: Summary of solo planning responses Scripted individuals to reflect on collaboration strengths andweaknesses including the challenges faced during collaboration.Challenge items adapted from Miller & Hadwin, 2012, & Järvenoja &Järvelä, 2009)Both groups reported encountering challenges related topoor planning during collaboration. Type and number ofchallenges were similar across assignments"Research Context & ProcedureResearch funded by a SSHRC Standard Research Grant 435-2012-0529 (A. Hadwin) and SSHRC Doctoral Fellowship (M. Miller)024681012Scripting Group (Assignment1)Visualization Group(Assignment 1)024681012Scripting Group(Assignment 2)Visualization Group(Assignment 2)Quick ConsensusIntegration OrientedConflict OrientedUnacknowledged012345678Scripting Group(Assignment 1)Visualization Group(Assignment 1)012345678ScriptingGroup(Assignment 2)VisualizationGroup(Assignment 2)Differentunderstandings ofwhat we need todoDifferent goals/standards for ourworkDifferent ideasabout how to startDifferent ideasabout how to worktogetherDifferent ideasabout how toorganize our timeAll PlanningChallengesPlanning ChallengesAssignment 1 Assignment 2Negotiation of Shared Task PerceptionsQuick consensus was the most common way both groups decided on task perceptions across both assignments.Groups engaged in slightly more in depth discussion in Assignment 2. Neither group leveraged the CSCL tools toengage in highly transactive discussion of their task perceptions."Assignment 1 Assignment 2Groups’ Shared Task PerceptionsAssignment 1 Assignment 2Scripting groupspontaneously adoptedvisualization tool inAssignment 2Visualization Group - Not allmembers accessed the toolin Assignment 1Log Files of Tool UseMicroscripting Individual & Group Task AnalysisBoth groups’ shared task perceptions were generally accurate (both explicit and implicit task perceptions were aligned with the instructor). However, after the planningsession, members continued to hold different ideas about the task (divergent ideas denoted in light blue). In some cases, shared task perceptions were less accurate thanindividuals’. Both groups improved across tasks, but improvements were small.