Automatic Assessment of Collaborative Chat Conversations with PolyCAFe Traian Rebedea 1 , Mihai Dascalu 1 , Stefan Trausan-Matu 1 , Gillian Armitt 2 & Costin Chiru 1 1 - Politehnica University of Bucharest 2 - University of Manchester
A debate about the best collaboration tool for the web: chat, blog, wiki, forums and Google Wave. Each student shall choose one of the 5 tools and shall present its advantages and the disadvantages of the other tools. Thus, you will act as a " sales person " for your tool and try to convince the others that you have the best offer. You must also defend your product whenever possible and criticize the other products if needed.
VT2: Learners perceive that the feedback received from the system contributes to informing their study activities
Logging: 285 visits to PolyCAFe and 1447 page-views, that results in more than 40 page-views in average per student.
Quality and Consistency of Feedback Validation statement Mean Standard deviation % Agree / Strongly agree The information the system provides me is accurate enough for helping me perform my learning tasks. 3.7 0.52 60% P olyCAFe's feedback is sufficiently accurate to inform my study activities. 3.8 0.88 64% PolyCAFe provides feedback that is useful to my study activities. 3.8 0.85 72% P olyCAFe provides feedback that is relevant to my study activities. 3.9 0.91 72% I trust PolyCAFe to provide helpful feedback. 4.0 0.87 80%
VT3: Learner performance in online discussions is improved in the areas of content coverage and collaboration when using PolyCAFe
Measurements computed for the second chat assignment, by comparing experimental with control groups
Quality of Educational Output Experimental group Control group Improvement over control group Average score for a chat conversation (collaboration + content) 6.80 6.37 6.8% Average importance of the most important 20 concepts 0.194 0.192 1.2% Average number of utterances 351 338 3.8% Average distribution of (implicit and explicit) links between utterances 1.12 0.87 29%