Analysing collaborative interaction and providing feedback to learners

1. New challenges in CSCL:
N h ll i CSCL
Towards adaptive script support
Towards adaptive script support
… a comment
Symposium at ICLS 2008, June 27th, Utrecht
Patrick.Jermann@epfl.ch
ÉCOLE POLYTECHNIQUE
FÉDÉRALE DE LAUSANNE

2. Adaptivity Regulation
Adaptivity = Regulation
Wiener (1948) Carver & Scheier (1998)
Standard
Input
Output
p
Script

3. Regulation hierarchies
Regulation hierarchies
Broadbent(1977) Robertson & Powers’ (1990) Leont’ev (1981) Lord & Levy (1994)
1‐10 minutes 0.1‐1 minute
10‐100 minutes
Activity
y Action Operation
p
Rational
Micro Script User Interface
Macro Script
Sentence openers
Group Mirrors

4. Scaffolding (Christof Wecker & Frank Fisher)
g
Prompts
Productive
“Provide counter-arguments”
argumentation
Peer
Production
Propose
Distributed
monitoring
Debate
Pro Contra
Black Box
Evaluate

5. Scaffolding (Christof Wecker & Frank Fisher)
g
Prompts
Productive
“Provide counter-argument”
argumentation
Peer
6
D e c la ra tiv e k n o w le d g e a b o u t
Production
Propose
Distributed 5
monitoring
a rg u m e n ta tio n
Debate
4
Distributed
monitoring:
3
no yes
Pro Contra
2
1
Peer as « mediators » interpreting prompts to
Black Box
Evaluate
0
monitor someone else
no yes
=> internalization Fading
Fading works … but is not adaptive yet
d k b d
Q: What are the indicators that someone is ready to argue without the prompt ?
• How much would it improve outcomes ?

6. Supporting the diagnosis
(Erin Walker, Ken Koedinger & Nikol Rummel)
Collaboration
Peer
Tutor
Delayed Test Gain
Peer tutor Correlation Tutee Tutor
Incorrect
Attempts ‐.614 .428
Adaptive (Adaptive)
Peer
Tutoring Getting
‐.591 .463
Stuck
Incorrect
Attempts ‐.378 .472
(Fixed)
Fixed Peer
Tutoring
Skipping
Skipping
‐.614 ‐.369
Problems
Peer as « mediator » interprets tutors advice
=> internalization
Q: Peer tutor and AI tutor might have different internal models
Q Peer t tor and AI t tor mi ht ha e different internal models
• Can you simulate the bad student ?
• Skipping problems by discouragement ? Peer tutor does motivational support ?

7. Indicators (Anne Meier & colleagues)
Feedback
Rating
Dimensions Specific / General
Rese
arch
er
0.5 Task 1 Task 2
Offline analysis 0.4
of interaction
of interaction 0.3
03
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4
Researchers rate the interaction controls generic adaptive
feedback feedback
Efficient because relevant and specific indicators
Q: Long delay between analysis and feedback ?
• Why is generic feedback so counter‐productive ?
• Did the adaptive FB also read the generic part ?
• Could a « cockpit » for teachers help ?

8. Analysis & diagnosis (Carolyn Rosé & al.)
y g
Feedback
db k
Accountable talk, Hints / interactive agents
Tutor
transactivity
Machine
M hi
learning
Automatic
dialogue analysis
dialogue analysis
System does diagnose and provides advice
Indicators and standards are key variables of fruitful class discussion
I di t dt d d k i bl f f itf l l di i
Q: How can the AI tutor be more sensitive to students’ relationship ?
• Is it more efficient to internalize the rules or the feedback ?

9. Who is in charge of the diagnosis?
Who is in charge of the diagnosis?
• « Locus of processing »
a) Students become self‐regulators through
Students become self‐regulators through
monitoring and diagnosing, internalization
(Christof, Nikol)
(Christof Nikol)
b) Teachers need indicators + flexible tools
(Anne, Carolyn)
(Anne Carolyn)
c) System poweful to classify input but still
« artificial » intervention
(Carolyn)

10. Orchestration
Scripts
Cognition
http://craft.epfl.ch