This document summarizes a presentation on learner models in online personalized educational experiences. It discusses the context of technology enhanced learning and the problem of disengagement in online learning. A proposed solution called GVIS creates user profiles by aggregating data from multiple sources and presenting it visually to support self-reflection and tutoring. Initial analysis found that simpler, more aggregated visualizations were preferred. GVIS was integrated with Moodle and Adapt2 to test social visualization. Results found an impact on user behavior enhanced by social aspects, and that tutors preferred compact, intuitive information. Further development of GVIS was suggested to better support instructional design.

1. 1
Presented by L. Mazzola
Faculty of Communication Sciences
Institute for Communication Technologies
University of Lugano, CH
Lugano - 23 May 2014
Learner Models in Online Personalized
Educational Experiences: an infrastructure and
some experiments

2. 2
Agenda
● The context
● The problem
● A solution
● The proposal
● Initial analysis in GRAPPLE
● Some testing in and outside GRAPPLE
● Consideration/Conclusions
● Possible next steps

3. 3
The context
● Technology Enhanced Learning (TEL)
– ICT applied to education process
● Availability of connection
● Enhancement in research and science → new knowledge
● Support for individual needs
● Availability of Learning Management and Intelligent Tutoring System
– Possibility of continuous education
● Distance and Blended modalities
● Informal learning
● On-the-job training
→ additional resources and tools to support educational experiences

4. 4
The problem of TEL
●
PROS:
– Decoupling of time and space
– Personal pace
– Asynchronous interaction
●
CONS:
– Disengagement / Drop Out
– Less “social pressure”
– Difficulty in self-regulating
– Depletion of stimulus to active participation
● Needs of tools to support the learning/teaching process

5. 5
A solution, in the literature
● Creating a user profile
– Adoption of content and presentation
– Positive effect of Disclosure (OLM)
– Integration with other sources/external provider (global
and long-run indicators)
– Representation aspect (Information Visualization):
● From text/analytic to graphical/summary
● For Supporting purposes:
– Enhance and stimulating self-reflection / awareness
– Fostering the tutoring process

6. 6
AIMS
(1) Representation aspects:
– How the OLM can be represented fruitfully to learners?
– ...and to teacher/tutors?
(2) Adaptive and social visualization of OLM:
– How they can affect the user experience?

7. 7
OBJECTIVES
(1) Demonstrate that mixing different and
heterogeneous sources can have a meaningful
didactic interpretation
(2) Explore approaches and representational models
considered effective by learners and tutors/teachers
(3) Measure the perceived effect/impact of the
introduction of such a tool

8. 8
The proposal: GVIS
Configurations / Semantics
Data Sources
Processing levels

9. 9
The proposal : GVIS
● PHP code with OO approach
● 3 layers that are specialized in source interfacing,
aggregation of data into information, and presentation
aspects
● Each layer controlled by one or more XML description
of the operation/attribute (didactic semantic)
● AJAX controlled interaction (interactive and
responsive)
● Adaptive segments in the XML configuration

10. 10
The proposal: GVIS

11. 11
UML sequence

12. 12
Representation aspect analysis
On mockups, through online questionnaire (learner & teacher)

13. 13
Analysis of MockUp
● # Users:
– 43 Learners
– 32 Instructor (Tutor/Teacher)
● Results:
– Simpler visualizations preferred
– More complex on user request (exploration)
– Usefulness of filtering capabilities of data presented
– Peers comparisons useful, but only at aggregated level
– Didactic meaningful aggregation for tutors/teacher

14. 14
Widgets for GRAPPLE
Bridge
Bridge

15. 15
Subjective Assessment of Visualisation
Dimensions Learner Teacher/Tutor
Perceived usability/suitability - in terms of:
- suitability for the task XX XX
- self-descriptiveness XX XX
Visualization benefits:
- Meta-cognition XX XX
- Cognitive load XX XX
- Learning effectiveness XX
- Benefits for instructors
(personalised/individualised instruction) XX
- Benefits for peers/collaboration XX
- Acceptance XX

16. 16
Subjective Assessment of Visualisation: result
(+) Suitable for their intended purpose and largely
self-descriptive and understandable
(+) Suitable for getting an overview of the current status
in the learning process
(+) Generally easy to understand and not unnecessarily
complex
(-) Comparison with the class might be problematic and
negatively affect self-worth and collaboration, especially
for underachievers
● Better a comparison with one self own prior
performance

17. 17
GVIS and Moodle

18. 18
GVIS in Moodle: evaluation
Question ++ + 0 -
Easy understandable X
Not unnecessarily complex X
Help instructor to tailor to individual needs X
Suitable for getting an overview of the current status X
Visualization does not provide irrelevant information X
Visualization can help learners to reflect on their learning X
Usefulness of comparison with other peers for reflection X
Expected impact on learners performances X
Promote awareness and understanding of learning progress X
Help teacher in better understand the learners needs X
Visualization able to leverage mental workload X
Risk of hindering the collaboration amongst peers X
Additional cognitive effort on learner to understand it X

19. 19
GVIS and Adapt2: social visualization

20. 20
GVIS and Adapt2: social visualization

21. 21
GVIS and Adapt2: evaluation
MidTerm Final
WITH GVIS
WITHOUT

22. 22
GVIS for User navigation history

23. 23
GVIS for Domain profile

24. 24
Adaption of visualizations
● At two levels: aggregation and building (presentation)
<cond>
+ <op>(v1 AND ((A &gt; 3) OR !(z)))</op> // FIRST LEVEL
| <operands>
| <val id="v1">CourseX.Concepts.list</val>
| <val id="z">CourseX.Student.count</val>
| <val id="A">CourseX.ConceptA.mean.knowledge</val>
| </operands>
+ <true>...</true>
+ <false>
| + <op>(h &lt; t)</op> // SECOND LEVEL CONDITION
| | <operands>
| | <val id="t">CourseX.ConceptA.mean.knowledge</val>
| | <val id="h">CourseX.ConceptA.userH.knowledge</val>
| | </operands>
| + <true>...</true>
| + <false>...</false>
+</false>
</cond>

25. 25
Adaption of visualizations: examples
Graphical format & aggregation
Graphical vs. Textual
Relative vs. Absolute scale

26. 26
Results
● Found an impact on user behaviors, enhanced by
social aspects
● Simpler and immediate presentation correlate with
higher (perceived or measured) effects
● Positive social pressure factor for learners,
improved by the peers comparison functionality
– Sense of community
– Stimulating healthy competition

27. 27
Results
● Tutors/teacher: preferred compact, intuitive, and
just-in-time information (didactic interpretation)
– Clearer picture
– Able to support identifying performances issues
● Possible cognitive overload: needed further studies.
● Sum-up: consider generally useful and enough
flexible to be adapted to different needs and context.
● TinCan API recently solved some of these issues...

28. 28
Possible Next Steps
● A graphical language to specify the pipeline from data
to didactic meaningful information
● An interface/editor for generating the XML
configurations of extractor, aggregator and builder from
the graphical language
● A library of freely available basic didactic components
(common and useful configurations) for reuse
● a set of adaptation templates could simplify the usage
of these capabilities by the Instruction Designers

29. 29
Possible Next Steps
● More filtering and data reordering procedures
through an easy visual interface to facilitate the
exploratory navigation of the information
● A more extensive and structured testing of the tool,
both to understand
– its full potentialities and threats
– to analyse more in depth the impact that a visualisation
(in all its form: adaptive, social and others) can have on
different type of education models, from blended
courses to completely online ones or from single course
to fully online degree.

30. 30
Thanks for the attention... questions?
mazzola.luca@gmail.com
luca.mazzola@usi.ch