This document presents research on a tool called Progressor+ that combines personalized guidance and social visualization for elearning. Progressor+ aims to integrate the benefits of personalized and social learning approaches. It provides navigation support through topic-based and progress-based adaptation within an open social student modeling visualization. An evaluation found that Progressor+ engaged students longer, led them to attempt more assessments and examples, and achieved higher knowledge gain and success rates than other tools. Stronger students also tended to explore content earlier, leaving traces to guide weaker students.

1. School of Information Sciences, University of Pittsburgh
Mo#va#onal
Social
Visualiza#ons
for
Personalized
Elearning
Sharon Hsiao & Peter Brusilovsky
Sept. 2012
1

2. Agenda
• Introduc#on
• Background
• Progressor+:
An
Innova#ve
Tabular
Open
Social
Student
Modeling
Interface
• Evalua#on
&
Results
• Summary
2

3. INTRODUCTION
3

4. Personalized vs. Social?
Personalized Social
Learning Learning
current knowledge level increase motivation
relevant interesting content better performance
increase learning quality development of high level thinking skills
increase learning rate
higher satisfaction
reduce navigational overhead
better retention
increase satisfaction
increase motivation higher self-esteem, attitude

5. What We Did Before
Personalized Learning Social Learning
QuizGuide Knowledge Sea II
(Brusilovsky, Sosnovsky, et al., 2004; Sosnovsky & (Brusilovsky, Chavan, & Farzan, 2004)
Brusilovsky, 2005)

6. Why do We Want a Merge?
Increasing amount of educational resources
– High costs of maintain the associations between
content and domain
– Social technologies provide collective wisdom
that might replace knowledge engineering
Motivation is important
– Even a great guidance will not provide good
impact without motivation

7. Problems illustration
Model complexity
Target area
Model precision
Integrated-approach (Hybrid)
Personalized approach
Social-based approach
7

8. Challenge
How do we introduce personalized guidance to social
technologies and harness the benefits from both
approaches?
• Keep the benefits of personalized guidance
• Increase user motivation

9. This work…
1. Personalized Guidance
– Navigation support: topic-based & progress-based
adaptation
2. Social Visualization
– easy-to-grasp and holistic view of student model &
content model
3. Integration of 1&2 above in Open Student
Modeling visualization
Navigation Support in Open Social Student Modeling Visualization

10. Research Questions
1: What are the design principles (key features) to
implement personalized guidance in open social
student modeling visualizations?
2: Will navigation support combined with open
social student modeling visualization work in
realistic content collections?
3: Will this approach guide students to the right
content at the right time?
4: Will this approach increase students motivation &
engagement?
10

11. Research model
11

12. BACKGROUND
12

13. Supporting Theories
• Self-regulated theory (Zimmerman, 1990)
– High jumpers (students who gained higher conceptual
understandings): good at using effective strategies, creating
sub-goals, monitoring emerging understanding, and
planning their time and effort.
– Low jumpers: did no spend much time monitoring their
learning, tend to engage help seeking behavior.
• Social comparison theory (Festinger, 1954; Dijkstra. et
al., 2008)
– lateral comparison: self-evaluation
– downward comparison: self-enhancement
– upward comparison: self-improvement
13

14. Related work
Adaptive navigation support Social navigation and
in E-Learning visualization for E-Learning
– AHA! (De Bra & Calvi, 1998) – EDUCO (Kurhila, Miettinen,
– ELM-ART (Weber & Nokelainen, & Tirri, 2006)
Brusilovsky, 2001)
– KnowledgeSea II (Brusilovsky,
– KBS-Hyperbook (Henze &
Nejdl, 2001) et al., 2009)
– INSPIRE (Grigoriadou, – AnnotatEd (Farzan &
Papanikolaou, Kornilakis, & Brusilovsky, 2008)
Magoulas, 2001) – Comtella (Vassileva & Sun,
– InterBook (Brusilovsky, 1998) 2007)
– NavEx (Brusilovsky, et al., – OLMlets (Bull & Britland,
2009) 2007)
– ISIS-Tutor (Brusilovsky, – CourseVis (Mazza & Dimitrova,
1994) 2007)
– QuizGuide (Brusilovsky,
Sosnovsky, et al., 2004)
14

15. THE
WAY
TO
PROGRESSOR+
15

16. QuizMap
(EC-­‐TEL
2011)
16

17. Parallel
Introspec#ve
Views
17

18. Progressor
18

19. PROGRESSOR+
19

20. Progressor+ design rationale
• Navigating and comparing segments of pie
graphs in a huge dataset takes longer time for
comprehension (Gillan & Callahan, 2000)
• Interacting and visualizing large data in Table
Lens (Rao & Card, 1994)
• Small multiples principle (Tufte, 1990)
20

21. Progressor+
21

22. 1. Sequence
• provides the direction for the students to
progress through the course
22

23. 2. Identity
• simple rows & columns table representation,
fragments can be easily cohesively shown
23

24. 3. Interactivity
• Direct accessing content, sorting, comparing,
collapse-and-expand
24

25. 4. Comparison
• macro- and micro- comparisons
25

26. 5. Transparency
• Holistic view of all the models
26

27. EVALUATION
&
RESULTS
27

28. Students spent more time in Progressor+
400
Total time spent (minutes)
321.1
350
296.9
300
250
224.7
200
Quiz
150.19
Example
150
121.23
110.66
100
69.52
60.04
50
0
QuizJET
JavaGuide
Progressor
Progressor+
Quiz =: 5 hours
Example : 5 hours 20 mins
28

29. More diversity helped increase
problem solving success
Topic Coverage
14
distinct questions
12.28
12.92
12.2
distinct examples
11.77 11.47
12 80 30 27.37
25.125
61.84
10 9.15
8.48 60 52.7
7.81 46.18 20 17.3
8
40 33.37 Quiz
10.86
6 Example
10
20
4
2 0 0
QuizJET JavaGuide Progressor Progressor+ QuizJET JavaGuide Progressor Progressor+
0
QuizJET JavaGuide Progressor Progressor+
• the more diverse of the questions the students tried, the
higher success rate they obtained (r=0.707, p<.01)
• the more diverse of the example the students studied,
the higher success rate they obtained (r=0.538, p<.01)
29

30. Students achieved higher Success Rate
p<.01
Success Rate
80.00% 71.20%
68.39%
58.31%
60.00%
42.63%
40.00%
20.00%
0.00%
QuizJET JavaGuide Progressor Progressor+
30

31. Impact on Learning – cont.
Time time spent sorted by knowledge gain
1800.00
1600.00
1400.00
1200.00
1000.00
example
800.00
quiz
600.00 Linear
(example)
Linear (quiz)
400.00
200.00
0.00
-200.00 Knowledge Gain
0 0 0 0 0 0 0 0 3 4 5 5 5 6 7 7 7 8 8 8 8 10 11 11 11 11 12 13 13 13 13 13 14 14 14 14 14 14 15 15 15 15 16 16 18 20
• The more time the students spent on the content (quizzes and examples), the higher
the level of knowledge gain they obtained (r=0.563, p<.01; r=0.448, p<.01)
• The more the students studied (more lines), the higher level of knowledge they
gained (r=0.492, p<.01) 31

32. The Mechanism of Social Guidance
stronger students left the traces for weaker
ones to follow
Topics
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
uu
Time
32

33. Strong students lead ahead in Progressor+
Mixed collection
Quizzes Examples
33

34. Students
worked
with
the
systems
during
exam
prepara#on,
especially
in
final
exam
period
Non-adaptive adaptive
Social, adaptive, single content Progressor+
34

35. Non-adaptive adaptive
Strong students were hours ahead of weak ones
100
83.59
80 84.3
60
Progressor
40 Progressor+
17.15
19.63
20
13.39
9.17
0
Easy Moderate Complex
Strong students worked earlier than weaker ones
Progressor Progressor+
35

36. Subjective Evaluation
• Usefulness
• Ease of Use
• Ease of Learning
• Satisfaction
• Privacy & Data Sharing
36

37. Students’ opinions
• Praised Progressor+
• “…it’s a great tool, should be used in other
classes…”
• “… I find the examples and quizzes helped. I
did recommend other students use it…”
37

38. CONCLUSION
38

39. Results Summary
• Engaged longer with Progressor+
• Attempted more self-assessment quizzes
• Explored more annotated examples & lines
• Obtained higher knowledge gain
• Achieved higher Success Rate
• Stronger students left the traces for weaker ones to follow
• Effectively led students to work at the right level of
questions among mixed collections of educational content
• Both strong and weak student had consistent performance
across all different questions’ complexities
39

40. THANK
YOU
J
40