1. http://www.slideshare.net/jlsantoso
Goal-oriented visualizations of activity
tracking: a case study with engineering
students
José Luis Santos, StenGovaerts,
KatrienVerbert, Erik Duval
1

2. Motivation
1. Means: Information
visualization
2. Goals
1. Awareness
2. Self-reflection
3. Sense-making
but even more important…
to engage students on
these processes.
2

3. Related work
LAMS DOKEOS
MOODLE
KHAN ACADEMY 3

4. 1. Upton et al. Narcissus: Group and Individual Models to Support Small Group Work. UMAP’09 http://goo.gl/ZzNbF
2. Derick et al. GLASS: A first look through a Learning Analytics System. LAK’12 http://goo.gl/eY4w2
3. Mazza et al. Exploring usage analysis in learning systems. Workshop at AIED’05 http://goo.gl/ycrFQ 4

5. 1. Santos et al. Visualizing PLE usage. EFEPLE’11 workshop. http://goo.gl/PWbBF
2. Govaerts et al. The student activity meter for awareness and reflection. ACM SIGCHI 2012. http://goo.gl/SGfta
3. Duval et al. Learning dashboard and learnscapes. Workshop at ACM SIGCHI 2012. http://goo.gl/j7NeJ
4. Duval. Attention please! Learning analytics for visualization and recommendation. LAK’11. http://goo.gl/Kfwii 5

7. Design based research
methodology
Design Implementation Deployment
Observation
Requirements Analysis
& Evaluation
7

8. One… two… three… and… four!
1. Paper prototype
– Professors and teacher assistants.
2. Digital prototype
– Professors and teacher assistants.
3. First release
– Students
4. Second release
– Students
8

9. 1st and 2nd iterations
-Generic (CAM)
-Usability
-Think-aloud protocol.
-15-30 minutes
- 1st -> 7 participants
- 2nd -> 5 participants
9

10. Results
• Size of tables
• Redundant information
• Personalization
• Context centralization vs diversification
• Motion chart
10

11. Scenario
Awareness and
reflection
Goal-oriented
How do students spend their time during
the lab sessions?
11

12. Why and what do we track?
• Individual and group spent time
• Type of activity
(development, writing, …)
• Application
• Program code
• Websites
+
12

13. How do we visualize this data?
http://sourceforge.net/apps/mediawiki/role-project
http://role-sandbox.eu/spaces/dashboardkul
or
http://goo.gl/qzP9H
13

14. 14

15. 15

16. 16

17. 17

18. 18

19. 19

20. 20

21. 21

22. First evaluation with real data
• 36 students
• Questionnaire after seeing a demo
– From USE questionnaire
• Perceived usefulness
• Privacy concerns
22

23. Conclusions
23

24. • Useful
• Helpful
• Lab tracking
• Usability?
24

25. Second evaluation with real data
• 10 students
• 4 lab sessions
• 15 minutes
• SUS
• Perceived usefulness
• Tracking/privacy concerns.
• Questionnaire
25

26. Conclusions
• SUS score: 72
• Useful?
• Detection of
certain patterns
26

27. Conclusions
27

28. Final conclusions
• Not included in this paper
• 14 students
• SUS: 66
• Useful for:
– Teachers (6)
– Students (5)
– Both (3)
• Satisfaction:
– 4.4 (over 7)
28

29. Although they consider it
useful, understandable, us
able…
29

30. …students did not use the
tool frequently
WHY?
30

31. One student said:
“We had other priorities”
31

32. How can we engage students
in this process?
32

33. 33

34. Thank you! Any question?
@jlsantoso
@erikduval
@katrien_v
http://jlsantoso.blogspot.com
http://erikduval.wordpress.com
34