Assessing and promoting computer-supported collaborative learning

Assessing and promoting computer-supported collaborative learning Anne Meier University of Freiburg, Institute of Psychology [email_address]

Introduction to CSCL (computer-supported collaborative learning)
Assessing CSCL learning processes
Supporting CSCL learning processes
Example study: adaptive support for knowledge co-construction
Overview

The CSCL community
a short history of CSCL…
“ seeds” in the 1980s, e.g. 1989 NATO-sponsored workshop “computer-supported collaborative learning” (Maratea, Italy)
since 1995: bi-annual CSCL conferences
since 2003: CSCL community part of International Society of the Learning Sciences (ISLS)
own journal: International Journal of CSCL (ijCSCL) published by Springer since 2006
highly interdisciplinary community

Introduction to CSCL
CSCL researchers study:
How people can learn together with the help of computers (Stahl, Koschmann, & Suthers, 2007)
How technology can facilitate the sharing and creation of knowledge and expertise through peer interaction and group learning processes (Restra & Laferrière, 2007)

advantages/strengths
challenges/pitfalls
What is your experience with (computer-supported) collaborative learning?

Neo-Piagetian perspective
learning = cognitive restructuring
resolving socio-cognitive conflict arising from peer collaboration
Cognitive elaboration perspective
learning = elaboration and integration of knowledge
very important: constructing explanations
Neo-Vygotskian perspective
learning = appropriation, internalization
knowledge co-construction; scaffolding and fading
Situated learning perspective
learning = increasingly “central” participation in a community of practice
distributed cognition: persons, tools, symbols, artefacts,…
What makes collaborative learning effective? See for example: Cohen, 1994; Dillenbourg et al., 1995; Fischer, 2002; Webb & Palincsar, 1996)

Motivational process loss (e.g. Salomon & Globerson, 1989)
Free-rider effect (“social loafing”)
Sucker effect
Production blocking
having to wait for others to finish their turn
e.g. in brainstorming (Diehl & Stroebe, 1987)
Biased information sampling (e.g. Brodbeck et al., 2007; Stasser & Titus, 1985))
neglecting individuals’ unique knowledge
striving for consensus rather than understanding
 Putting people in a (computer-supported) group does not mean that they will collaborate well!
Pitfalls of collaborative learning

 What characterizes “good” computer-supported collaborative learning?
Overview

Cognitive, social, and affective aspects of collaboration quality in CSCL
Communication (Clark & Brennan, 1991)
Grounding
adapting utterances to the amount of shared knowledge/ perspective/ experience
establishing referential identity (e.g. of objects in a shared whiteboard, of previous messages/ contributions)
establishing a shared terminology
Conversation management
initiating conversations
managing turn-taking
ensuring that contributions are taken up
For additional literature/ references, please see Meier, Spada, & Rummel, 2007

Joint information-processing
Elaborative information-processing
eliciting and providing elaborated explanations
using the partner as a resource
elaborating on partners’ contributions
Argumentative information-processing
constructing justified arguments and counterarguments
engaging in a critical discussion: avoiding an illusion of consensus

Coordination (explicit or tacit) (e.g. Malone & Crowstone, 1994)
Task division
identifying interdependent subtasks
blending individual and collaborative work
Time management
agreeing on a realistic time schedule
monitoring the remaining time during the work process
Resource management
handling the available tools efficiently
agreeing on who may use a technical feature at what time

Relationship management
maintaining equal participation
symmetric or complementary, depending on role structure
solving conflicts constructively
epistemic vs. social conflicts

Motivation
individual task orientation
keeping up a high level of expended effort
volitional processes: focusing attention, exerting motivation control
mutual self-regulation
mutual encouragement
monitoring performance and giving feedback

Example: Collaboration quality rating-scheme
Development
sample from study on interdisciplinary collaboration : students of psychology and medicine solving complex patient cases (Rummel & Spada, 2005)
Meier, A., Spada, H. & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2 , 63-86.

Example: Collaboration quality rating-scheme Control Room Experimental Room I Experimental Room II

Development
sample from study on interdisciplinary collaboration : students of psychology and medicine solving complex patient case (Rummel & Spada, 2005)
data- and theory-driven analyses  5 aspects/ 9 dimensions
for each dimension:
collaboration “standard” defined and illustrated in rating handbook
collaboration quality rated on 5-point scales

Example: Collaboration quality rating-scheme model / script > control model-plus > model Information pooling Task division Time management Technical coordination model > control > script model-plus > model Individual task orientation Quality of joint solution (Rummel, Spada, & Hauser, 2009) Rummel rummele

adaptation to new CSCL setting (Synergo) (Voyiatzaki et al., 2008)
descriptive framework valid in this setting as well
But: changed operationalization of dimensions and re-anchoring of scales necessary
… .. work in progress:
providing adaptive feedback to students based on ratings of their collaboration quality
(Meier, Voyiatzaki, Kahrimanis, Rummel, Spada, Avouris, 2008)

 How can beneficial collaboration processes be facilitated?
Overview

Earlier approaches: support “around” collaboration
Collaboration scripts: support during collaboration
Adaptivity: from fixed to flexible support

Support prior to collaboration, e.g. training for strategic questioning (King, 1991)
Support after collaboration, e.g. group processing approaches (Yager, Johnson, Johnson, & Snider, 1986)
after: Diziol & Rummel, accepted

provide specific instructions about task-related interaction (Kollar et al., 2006)
Sequencing work phases
Distributing roles
Specifying activities
 goal: enhance cognitive, meta-cognitive and social learning processes

Collaboration Scripts
S plit W here I nteraction S hould H appen (SWISH) ( Dillenbourg & Jermann, 2007)
Schema Split Compensation mutual regulation cognitive & metacognitive tasks (e.g. recaller & detector) Reciprocal argumentation conflicting opinions (e.g. pro & contra-roles) Conflict exchange of information distribution of knowledge (e.g. expert groups & teams) Jigsaw

Adaptivity: from fixed scripts to flexible support
Danger of “overscripting” collaboration (Dillenbourg, 2002); instead: taking into account students’ prior knowledge and “internal collaboration scripts”
realizing flexible, adaptive support:
“ Wizard of Oz” studies
adaptive feedback based on automated analyses of interaction (e.g. Dönmez et al, 2005)

Overview

Example: Supporting Collaborative Inferences F - I - R - E ! Figure from: Bauer, K., & Hesse, F. (2006). Von Kopf zu Kopf. [From head to head]. Gerhirn und Geist [Brain & Mind], 5/2006, 34-39.

Example: Supporting Collaborative Inferences Wolfgang‘s fingerprints are on the gun. Wolfgang showed the guns to his guests in the afternoon. A B

Example: Supporting Collaborative Inferences Wolfgang left his fingerprints on the weapon when he showed it to his guests. A B

Example: Supporting Collaborative Inferences Meier & Spada, 2007 Person B Person A shared individual collaborative Inference type Information distribution ** *

Example: Supporting Collaborative Inferences
Why is it so difficult to draw collaborative inferences?
individual group member holds “unconnected” information
seen as less relevant and therefore less likely to be brought up during discussion (Fraidin, 2004)
inference must be drawn on the basis of newly learned information and recalled information
people tend to focus on old rather than new information (Wittenbaum, Hubbel & Zuckermann, 1999)
recall vulnerable to disruptions in group discussion (Finlay, Hitch & Meudell, 2000)
Training Experiment: train collaboration strategies for
drawing inferences
pooling “unconnected” information
taking up new information

Example: Supporting Collaborative Inferences Meier & Spada, 2008 Test task (murder mystery) No Training (n=9 ) Testing phase collaborative reflection ... with inference tutoring tool discussion & solution Training task (medical diagnosis): individual reading phase read text on collaboration strategies Training phase Training Task + Text + Tutoring (n=9) Training Task+ Text (n=9) Training Task (n=9)

Example: Supporting Collaborative Inferences 6 7 5 8 4 2 ... 3 1 inference patient information disease information New Information! ANJA has matching information.

Example: Supporting Collaborative Inferences 6 7 5 8 4 2 ... 3 1 inference patient information disease information Well done! You have drawn an important inference!

Example: Supporting Collaborative Inferences Meier & Spada, 2008 Test task (murder mystery) No Training (n=9 ) Testing phase collaborative reflection ... with inference tutoring tool discussion & solution Training task (medical diagnosis): individual reading phase read text on collaboration strategies Training phase Training Task + Text + Tutoring (n=9) Training Task+ Text (n=9) Training Task (n=9)

Example: Supporting Collaborative Inferences ** ** performance during testing (without tutoring tool) Meier & Spada, 2008

In a nutshell…
diverse perspectives on collaborative learning within field of CSCL
successful collaboration does not occur spontaneously!
focus here was on processes, rather than outcomes or preconditions
many relevant aspects: communication, information-processing, coordination, relationship management, motivation
collaboration scripts: (computer-)support during collaboration
moving towards more flexible, more adaptive support
collaborative inferences are important but difficult
adaptive support yields best training results

Many thanks to the CoEmCo-Team: Hans Spada, Nikol Rummel Dejana Diziol, Sabine Hauser Eva Zerpies, Malte Jansen This work was funded by Thank you for your attention!

Brodbeck, F., Kerschreiter, R., Mojzisch, A., & Schulz-Hardt, S. (2007). Improving group decision making under conditions of distributed knowledge: The information asymmetries model. Academy of Management Review, 32, 459-479.
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127–148). Washington, DC: American Psychological Association.
Cohen,E.G.:Restructuring the classroom: Conditions for productive small groups. Review of Educational Research,64,1-35.1994
Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology, 53, 497-509
Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL? (pp. 61-91). Heerlen: Open Universiteit Nederland.
Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1995). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.), Learning in humans and machines: Towards an interdisciplinary learning science (pp. 189–211). Oxford: Pergamon.Dillenbourg, P. & Jermann, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 259-288). New York: Springer.
Diziol, D., & Rummel, N. (to appear). How to design support for collaborative e-learning: A framework of relevant dimensions. In: Ertl, B. (Ed.): E-Collaborative knowledge construction: learning from computer-supported and virtual environments.
Dönmez, P., Rose, C., Stegmann, K., Weinberger, A., & Fischer, F. (2005). Supporting CSCL with automated corpus analysis technology. In T. Koschmann, D. Suthers, & T. W. Chan (Eds.), Proceedings of the CSCL 2005 (pp. 125–134). Mahwah, NJ: Lawrence Erlbaum Associates.Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts - a conceptual analysis. Educational Psychology Review. Springer.
Fischer, F. (2002). Gemeinsame Wissenskonstruktion - theoretische und methodologische Aspekte. Psychologische Rundschau, 53 (3), 119-134.
Fraidin, S. N. (2004). When is one head better than two? Interdependent information in group decision making. Organizational Behavior and Human Decision Processes, 93, 102-113.
Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts - a conceptual analysis. Educational Psychology Review, 18, 159-185.
Koschmann, T. (Ed.). (1996). CSCL: Theory and practice of an emerging paradigm. Mahwah, NJ: Lawrence Erlbaum Associates.
Koschmann, T., Hall, R., & Miyake, N. (Eds.). (2002). CSCL 2: Carrying forward the conversation. Mahwah, NJ: Lawrence Erlbaum Associates.
Kraut, R. (2003). Applying social psychological theory to the problems of group work. In: J. M. Carroll (ed.), HCI models, theories, and frameworks. Towards a multidisciplinary science (pp. 325-356). Amsterdam: Morgan Kaufmann.
Malone, T. W., & Crowston, K. (1994). The interdisciplinary study of coordination. ACM Computing
Meier, A., & Spada, H. (2007). Information pooling and processing in group problem solving: analysis and promotion of collaborative inferences from distributed information. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society (pp. 473-479). Austin, TX: Cognitive Science Society.
Meier, A., & Spada, H. (2008). Promoting the drawing of inferences in collaboration: insights from two experimental studies. Proceedings of the International Conference of the Learning Sciences, Utrecht, The Netherlands, 2008 .
Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2 , 63-86.
Meier, A., Voyatzaki, E., Kahrimanis, G., Rummel, N., Spada, H., & Avouris, N. (2008). Teaching students how to improve their collaboration: Assessing collaboration quality and providing adaptive feedback in a CSCL setting. Symposium New Challenges in CSCL: Towards adaptive script support (N. Rummel & A. Weinberger), ICLS 2008, Utrecht, The Netherlands.
Resta, P. & Laferrière, T. (2007). Technology in support of collaborative learning. Educational Psychology Review, 19, 65-83.
Rummel, N., Hauser, S., & Spada, H. (2007). How does net-based interdisciplinary collaboration change with growing domain expertise? In C. A. Chinn, G. Erkens & S. Puntambekar (Eds.), Mice, minds and sociecty. Proceedings of the Computer Supported Collaborative Learning (CSCL) Conference 2007, Vol 8, II (pp. 611-620). International Society of the Learning Sciences, Inc. ISSN 1819-0146
Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem-solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201-241.
Rummel, N., Spada, H., & Hauser, S. (2009). Learning to collaborate while being scripted or by observing a model. International Journal of Computer-Supported Collaborative Learning, 4(1), 69-92.
Salomon, G./Globerson, T.: When teams do not function the way they ought to. Internationnal Journal of Educational Research,11, 623-637. 1989.
Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. Journal of the Learning Sciences, 3(3), 265-283.
Stahl, G., Koschmann, T. & Suthers, D. (2007). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409-426). Cambridge, UK: Cambridge University Press.
Stasser, G. & Titus, W. (1985). Pooling of unshared information in group decision making: Biased information sampling during discussion. Journal of Personality and Social Psychology, 48, 1467-1478.
Voyiatzaki, E., Meier, A., Kahrimanis, G., Rummel, N., Spada, H., & Avouris, N. (2008). Rating the quality of collaboration during networked problem solving activities, Proceedings of the 6th International Conference on Networked Learning, pp. 409-416, Halkidiki, May 2008.
Webb, N.M., & Palincsar, A.S. (1996). Group processes in the classroom. In D. Berliner & R. Calfee (Eds.), Handbook of educational psychology (pp. 841-873). New York: Macmillan
Wittenbaum, G., Hubbell, A, & Zuckermann, C. (1999). Mutual enhancement: Toward an understanding of the collective preference for shared information. Journal of Personality and Social Psychology, 77 (5), 967-978.
References / Readings

Example: Collaboration quality rating-scheme Meier, A., Spada, H. & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2 , 63-86.
Time management
Information pooling
Joint information processing
Reaching consensus
Sustaining mutual understanding
Communication
Task division
Coordination
Technical coordination
Individual task orientation
Motivation
Reciprocal interaction
Relationship management
Dialog management

Outcomes of collaborative learning
When are groups better than individuals? insights from social psychology (Kraut, 2003)
Aggregation: combining the unique resources of individual group members
Making use of members’ complementary knowledge, perspectives, skills etc.
e.g. a cross-functional marketing team making strategic decisions based on members’ complementary expertise
Synergy: going beyond the resources contributed by group members
building on each others’ contributions, creating innovative ideas & solutions
e.g. a product-design team developing a new product
“ assembly bonus”
however: groups tend to neglect members’ unique knowledge and focus instead on shared knowledge (Stasser & Titus, 1985)

Measuring the success of computer-supported collaborative learning
Individual learning
types of knowledge and skills
conceptual vs. procedural
skills: domain, collaboration, self-regulation, computer-literacy
level of evaluation: subjective evaluation - retention – transfer
Group-level learning
transactive memory, shared mental models
 facilitates future collaboration in the same group
Interpersonal and motivational outcomes
trust, liking, friendships,…
self- and group-efficacy
interest
 motivation for future collaboration

Measuring the success of computer-supported collaborative learning
What characterizes “success” in the TEL-environment you study? How do you assess it?

Assessing collaboration quality cognitive, social, and affective processes process gain: assembly bonus, synergy effects process loss: motivation and coordination problems group size & composition task type technical & informational resources goal structure Group learning, e.g. transactive memory, shared mental models Individual learning - domain knowledge and problem-solving skills - collaboration skills - technical skills Interpersonal & motivational outcomes, e.g. trust, liking, self-efficacy, group-efficacy institutional context Input

Assessing and promoting computer-supported collaborative learning

by telss09

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 10

Statistics

Assessing and promoting computer-supported collaborative learning Presentation Transcript