presentation at ICALT2012 of the paper "Falakmasir M., Hsiao I., Mazzola L., Grant N., Brusilovsky P. (2012). The Impact of Social Performance Visualization on Students. Proceedings of the 12th IEEE International Conference on Advanced Learning Technologies and Technology-enhanced Learning, Rome, Italy, July 4-6, 2012, pgg. 565-569"

1. ICALT 2012 – Rome - 05 July 2012
The Impact of Social
Performance Visualization
on Students
M. H. Falakmasir, I-Han Hsiao, L. Mazzola,
N. Grant, P. Brusilovsky
University of Pittsburgh, USA, University of Lugano, CH,
and Community College of Allegheny County, PA, USA
1

2. Agenda
The Context
Our approach
Results of an experiment
Analysis of data collected
Consideration/Conclusions
2

3. Context
Problem: researches reported possbile isolation and
disengagement in online educational exp.
Proposal: recovering some contextual and social
information in web-based Intelligent Tutoring Systems (ITS),
or Learning Management Systems (LMS)
Secondary usage of the already collected data (learner
model), taking some hints from Information Visualization to
limit the informative/cognitive overload problem.
To visualize student learning progress:
targeting for the needs of instructors / academic advisors
(*) present visualizations directly to the student (OSLM)
3

4. Claim
providing students with explorable
social performance visualization
could improve their engagement
in learning and positively impact
their performance in the
educational system.
4

5. Our approach
Used data in the experiment from ADAPT2 ITS:
average mark achieved on self-evaluation questionnaire
total number of attempts done to solve it
basic representation of learner's knowledge
facilitating their meta-cognitive activities
promoting self-reflection
social aspects:
to get a view of their performance in the context of the class
progress, positive sense of competition using a
representation of average class performance
5

6. Infrastructure
Mazzola L., Mazza R. (2011). Visualizing Learner Models through data
aggregation: a test case. Red-conference, rethinking education in the
knowledge society, Monte Verità, Switzerland, March 7-10, pp.372- 380,
ISBN 978-88-6101-010-9 .
Mazzola L., Mazza R. (2010). GVIS: A Facility for Adaptively Mashing Up and Representing Open Learner Models.
"Sustaining TEL: From Innovation to Learning and Practice" Lecture Notes in Computer Science, Volume
6383/2010, pp. 554-559, Proceedings of EC-TEL 2010, DOI: 10.1007/978-3-642-16020-2_53 .
Mazzola L., Eynard D., Mazza R. (2010). GVIS: a framework for graphical mashups of heterogeneous sources to
support data interpretation. 3nd Conference on Human System Interactions, 2010. HSI '10.
6

7. An initial prototype
7

8. The “running” implementation – part 1
8

9. The “running” implementation – part 2
9

10. Some Results
Weekly representation
of usage of the
system (red: WebEx
+ NavEx; blue:
QuizGuide).
On the top the usage
with Visualization,
on the bottom the
one without.
→ .. so a success? Mhhhh, let's go deeper in the data...
10

11. Some Results
Weekly representation
of usage of the
system (red: WebEx
+ NavEx; blue:
QuizGuide).
On the top the usage
with Visualization,
on the bottom the
one without.
Seems to suggest a positive impact, but needs more analysis...
11

12. Some Results
Initial idea suggested by the data:
Learner uses more the system and are more interested in give feedback
Average activity in the system higher but lower on quiz (less try&guess)
The average success rate increases (but the std. dev. is higher)
Needs more deep analysis from statistical point of view...
12

13. Some Results: deeper analysis
Different patterns before and after midterm (the green dashed line)
13

14. More Analysis of the data...
At a p-level of 0.05 not verified an effect of the social visualization tool
(on whole semester, left), but seems to work in involving learner
on the first stages (that means here before midterm, on the right)
14

15. Consideration and Conclusions
We designed and deployed a Social visualization tool
for ITS, devoted to learner's self usage
Tested against usage of the system (contents and self-
evaluation quizzes), with logs.
Seems to produce more engagements and to reduce
the phenomenon of gaming for the evaluation system
The indicated enhancements seems to be even
statistically sustained, at least analyzing the data
before the first midterm.
But require more analysis...
15

16. Falakmasir M., Hsiao I., Mazzola L., Grant N., Brusilovsky P. (2012). The Impact of Social
Performance Visualization on Students. Proceedings of the 12th IEEE International
Conference on Advanced Learning Technologies and Technology-enhanced Learning, Rome,
Italy, July 4-6, 2012, pgg. 565-569
Thank you for the attention...
...any questions?
http://www.slideshare.net/mazzolaluca/icalt2012-presentation-13548761
luca.mazzola@usi.ch
16