Intelligent Interfaces for
Open Social Student Modeling
Peter Brusilovsky
Sharon Hsiao,Tomek Loboda, Julio
Guerra, Jordan ...
Overview
• Goals
– Why we are doing it?
• Open Student Models
– From ANS to OSM
• Open Social Student Models
– QuizMap, Pr...
From Goals to Technologies
• Technologies
–Adaptive Navigation Support
–Open Student Models
–Open Social Student Modeling
...
Targets Engaged
• Adaptive Navigation Support
• Topic-based Adaptation
• Open Student Modeling
• Social Navigation and Com...
Adaptive Link Annotation: InterBook
1. Concept role
2. Current concept state
3. Current section state
4. Linked sections s...
Questions of
the current
quiz, served
by QuizPACK
List of annotated
links to all quizzes
available for a
student in the
cu...
Topic-Based Adaptation
Concept
A
Concept
B
Concept
C
 Each topic is associated with a number of
educational activities to...
QuizGuide: Adaptive Annotations
• Target-arrow abstraction:
– Number of arrows – level of
knowledge for the specific
topic...
QuizGuide: Success Rate
QuizGuide: Motivation
Average activity
0
50
100
150
200
250
300
2002 2003 2004
Average num. of
sessions
0
5
10
15
20
2002 ...
• Topic-Based interface organization is
familiar, matches the course
organization, and provides a
compromise between too-m...
Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
• Social Navigation and Compar...
Social Navigation
• Concept-based and topic-based navigation support
work well to increase success and motivation
• Knowle...
Knowledge Sea – Social Navigation
Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based...
Open Social Student Modeling
• Motivation
– Combine benefits of Open Student Models with social navigation
and social comp...
QuizMap
16
Parallel Introspective Views
17
Progressor
18
• Topic organization should follow the
natural progress or topics in the
course
• Clear comparison between “me” and
“group...
The Value of OSLM
205.73
113.05
80.81
125.5
0
50
100
150
200
250
Attempts
Progressor
QuizJET+IV
QuizJET+Portal
JavaGuide
6...
The Mechanism of Social Guidance
stronger students left the traces for weaker ones to
follow
21Time
Topics




...
The Secret
Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
Social Navigation and Compari...
Progressor+ OSLM for two types of content
• macro- and micro- comparisons (group or peers)
24
Students Spent More Time in Progressor+
Quiz =: 5 hours
Example : 5 hours 20 mins25
60.04
150.19
224.7
296.9
69.52
121.23
...
Students Achieved Higher Success Rate
26
42.63%
58.31%
68.39%
71.20%
0.00%
20.00%
40.00%
60.00%
80.00%
QuizJET JavaGuide P...
Mastery Grids
27
Mastery Grids: Content Access
28
Mastery Grids: Group and Peer OSLM
29
MG flexibility
• Parameters to set the visualization:
– show hide toolbar or any of its elements
– set the (sub) groups: t...
Mastery Grids Engage More
31
0
10
20
30
40
50
60
Problems Solved
0
5
10
15
20
25
30
35
40
45
50
Examples Viewed
And social...
OSSM Engages Persistently
32
10
15
20
25
30
PART 1 PART 2
Activity by Session
OSM OSSM
Step-wise
regression:
being in the
...
OSSM Group Becomes More Effective
• Instructional Effectiveness (Paas & Van Merriënboer, 1993)
Relates performance in prob...
Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
Social Navigation and Compari...
Concept-Based Student Modeling
Example 2 Example M
Example 1
Problem 1
Problem 2 Problem K
Concept 1
Concept 2
Concept 3
C...
These cells (first row) shows your
progress in the topics of the course
This bar chart shows
your progress in the
concepts...
An overlayed pane opens
indicating which topic you
are inspecting (in this case
the topic "Comparisons")
The concepts with...
Mousing over this
activity
Concepts in the selected
activity are highlighted
This gauge estimates the
how much you can lea...
Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
Social Navigation and Compari...
Acknowledgements
• Joint work with
– Sergey Sosnovsky
– Sharon Hsiao
– Julio Guerra
– Jordan Barria-Pineda
• NSF Grants
– ...
Read About It!
• Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009) Addictive
links: The motivational value of adapti...
Upcoming SlideShare
Loading in …5
×

IUI2017 SmartLearn keynote: Intelligent Interfaces for Open Social Student Modeling

135 views

Published on

In this talk I will introduce the emerging technology of
Open Social Student Modeling (OSSM) and review several
projects performed in our research lab to investigate the
potential of OSSM.
OSSM is a recent extension of Open Student Modeling
(OSM), a popular technology in the area of personalized
learning systems. While in traditional personalized systems,
student models were hidden “under the hood” and used to
personalize the educational process; open student modeling
introduced the ability to view and modify the state of
students’ own knowledge to support reflection, selforganized
learning, and system transparency. Open Social
Student Modeling takes this idea one step further by
allowing students to explore each other’s models or an
aggregated model of the class. The idea to make OSM
social was originally suggested and explored by Bull [1; 2].
Over the last few years, our team explored several
approaches to present OSSM in a highly visual form and
evaluated these approaches in a sequence of classroom and
lab studies. I will present a summary of this work
introducing such systems as QuizMap [3], Progressor [4],
and Mastery Grids [5] and reviewing most interesting
research evidence collected by the studies.

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
135
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

IUI2017 SmartLearn keynote: Intelligent Interfaces for Open Social Student Modeling

  1. 1. Intelligent Interfaces for Open Social Student Modeling Peter Brusilovsky Sharon Hsiao,Tomek Loboda, Julio Guerra, Jordan Barria-Pineda PAWS Lab, University of Pittsburgh
  2. 2. Overview • Goals – Why we are doing it? • Open Student Models – From ANS to OSM • Open Social Student Models – QuizMap, Progressor, Progressor+ • Mastery Grids – Topic-level OSLM in Mastery Grids – Concept-level OSLM in Mastery Grids
  3. 3. From Goals to Technologies • Technologies –Adaptive Navigation Support –Open Student Models –Open Social Student Modeling • Why to use it –Increase user performance –Increase motivation and retention
  4. 4. Targets Engaged • Adaptive Navigation Support • Topic-based Adaptation • Open Student Modeling • Social Navigation and Comparison • Open Social Student Modeling • Social Educational Progress Visualization • Multiple Content Types • Open Source • Concept-Based Adaptation
  5. 5. Adaptive Link Annotation: InterBook 1. Concept role 2. Current concept state 3. Current section state 4. Linked sections state 4 3 2 1 √
  6. 6. Questions of the current quiz, served by QuizPACK List of annotated links to all quizzes available for a student in the current course Refresh and help icons QuizGuide = Topic-Based ANS
  7. 7. Topic-Based Adaptation Concept A Concept B Concept C  Each topic is associated with a number of educational activities to learn about this topic  Each activity classified under 1 topic
  8. 8. QuizGuide: Adaptive Annotations • Target-arrow abstraction: – Number of arrows – level of knowledge for the specific topic (from 0 to 3). Individual, event-based adaptation. – Color Intensity – learning goal (current, prerequisite for current, not-relevant, not-ready). Group, time- based adaptation.  Topic–quiz organization:
  9. 9. QuizGuide: Success Rate
  10. 10. QuizGuide: Motivation Average activity 0 50 100 150 200 250 300 2002 2003 2004 Average num. of sessions 0 5 10 15 20 2002 2003 2004 Average course coverage 0% 10% 20% 30% 40% 50% 60% 2002 2003 2004  Within the same class QuizGuide session were much longer than QuizPACK sessions: 24 vs. 14 question attempts at average.  Average Knowledge Gain for the class rose from 5.1 to 6.5
  11. 11. • Topic-Based interface organization is familiar, matches the course organization, and provides a compromise between too-much and too-little • Two-way adaptive navigation support guides to the right topic • Open student model provides clear overview of the progress Topic-Based ANS: Success Recipes
  12. 12. Targets Engaged Adaptive Navigation Support Topic-based Adaptation Open Student Modeling • Social Navigation and Comparison • Open Social Student Modeling • Social Educational Progress Visualization • Multiple Content Types • Open Source • Concept-Based Adaptation
  13. 13. Social Navigation • Concept-based and topic-based navigation support work well to increase success and motivation • Knowledge-based approaches require some knowledge engineering – concept/topic models, prerequisites, time schedule • In our past work we learned that social navigation – “wisdom” extracted from the work of a community of learners – might replace knowledge-based guidance • Social wisdom vs. knowledge engineering
  14. 14. Knowledge Sea – Social Navigation Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling. 10th International User Modeling Conference Lecture Notes in Artificial Intelligence, vol. 3538. Berlin: Springer Ve
  15. 15. Open Social Student Modeling • Motivation – Combine benefits of Open Student Models with social navigation and social comparisons • Key steps – Assume simple topic-based design – Show topic- and content- level knowledge progress of a student in contrast to the progress of the class – The design should guide students to most appropriate topics and content • Main challenge – How to design the interface to show student and class progress over topics? – We went through several attempts…
  16. 16. QuizMap 16
  17. 17. Parallel Introspective Views 17
  18. 18. Progressor 18
  19. 19. • Topic organization should follow the natural progress or topics in the course • Clear comparison between “me” and “group” • Ability to compare with individual peers, not only the group • Privacy management OSLM: Success Recipes
  20. 20. The Value of OSLM 205.73 113.05 80.81 125.5 0 50 100 150 200 250 Attempts Progressor QuizJET+IV QuizJET+Portal JavaGuide 68.39% 71.35% 42.63% 58.31% 0.00% 20.00% 40.00% 60.00% 80.00% Success Rate Progressor QuizJET+IV QuizJET+Portal JavaGuide
  21. 21. The Mechanism of Social Guidance stronger students left the traces for weaker ones to follow 21Time Topics                    
  22. 22. The Secret
  23. 23. Targets Engaged Adaptive Navigation Support Topic-based Adaptation Open Student Modeling Social Navigation and Comparison Open Social Student Modeling Social Educational Progress Visualization • Multiple Content Types • Open Source • Concept-Based Adaptation
  24. 24. Progressor+ OSLM for two types of content • macro- and micro- comparisons (group or peers) 24
  25. 25. Students Spent More Time in Progressor+ Quiz =: 5 hours Example : 5 hours 20 mins25 60.04 150.19 224.7 296.9 69.52 121.23 110.66 321.1 0 50 100 150 200 250 300 350 400 QuizJET JavaGuide Progressor Progressor+ Total time spent (minutes) Quiz Example
  26. 26. Students Achieved Higher Success Rate 26 42.63% 58.31% 68.39% 71.20% 0.00% 20.00% 40.00% 60.00% 80.00% QuizJET JavaGuide Progressor Progressor+ Success Rate p<.01
  27. 27. Mastery Grids 27
  28. 28. Mastery Grids: Content Access 28
  29. 29. Mastery Grids: Group and Peer OSLM 29
  30. 30. MG flexibility • Parameters to set the visualization: – show hide toolbar or any of its elements – set the (sub) groups: top N, other sub groups – preset values (for example load individual view by default) – enable/disable recommendation • Parameters can be specified by group or by user
  31. 31. Mastery Grids Engage More 31 0 10 20 30 40 50 60 Problems Solved 0 5 10 15 20 25 30 35 40 45 50 Examples Viewed And social comparison (OSSM) features strengthen the effect
  32. 32. OSSM Engages Persistently 32 10 15 20 25 30 PART 1 PART 2 Activity by Session OSM OSSM Step-wise regression: being in the OSSM group means an increase of about 30 activities, as compared to being in the OSM group.
  33. 33. OSSM Group Becomes More Effective • Instructional Effectiveness (Paas & Van Merriënboer, 1993) Relates performance in problems and time spent 33 -0.4 -0.2 0 0.2 PART 1 PART 2 Effectiveness Score OSSM OSM
  34. 34. Targets Engaged Adaptive Navigation Support Topic-based Adaptation Open Student Modeling Social Navigation and Comparison Open Social Student Modeling Social Educational Progress Visualization Multiple Content Types Open Source • Concept-Based Adaptation
  35. 35. Concept-Based Student Modeling Example 2 Example M Example 1 Problem 1 Problem 2 Problem K Concept 1 Concept 2 Concept 3 Concept 4 Concept 5 Concept N Examples Problems Concepts
  36. 36. These cells (first row) shows your progress in the topics of the course This bar chart shows your progress in the concepts of the course Each topic has several concepts associated to it. Mouseover a topic to highlight its concepts This bar chart (upside-down) shows the average progress of the rest of the class on the concepts Middle row shows the difference between your progress and the progress of the group Third row shows the progress of the group in blue Concept level OSLM
  37. 37. An overlayed pane opens indicating which topic you are inspecting (in this case the topic "Comparisons") The concepts within the selected topic are highlighted
  38. 38. Mousing over this activity Concepts in the selected activity are highlighted This gauge estimates the how much you can learn in the selected activity. You will probably learn more in activities that have more new concepts See more in IUI 2017 Demo! "Concept-Level Knowledge Visualization for Supporting Self- Regulated Learning"
  39. 39. Targets Engaged Adaptive Navigation Support Topic-based Adaptation Open Student Modeling Social Navigation and Comparison Open Social Student Modeling Social Educational Progress Visualization Multiple Content Types Open Source Concept-Based Adaptation
  40. 40. Acknowledgements • Joint work with – Sergey Sosnovsky – Sharon Hsiao – Julio Guerra – Jordan Barria-Pineda • NSF Grants – EHR 0310576 – IIS 0426021 – CAREER 0447083 • ADL “PAL” grant to build Mastery Grids
  41. 41. Read About It! • Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009) Addictive links: The motivational value of adaptive link annotation. New Review of Hypermedia and Multimedia 15 (1), 97-118. • Brusilovsky, P., Hsiao, I.-H., and Folajimi, Y. (2011) QuizMap: Open Social Student Modeling and Adaptive Navigation Support with TreeMaps. Proceedings of 6th European Conference on Technology Enhanced Learning (ECTEL 2011), pp. 71-82 • Hsiao, I.-H., Bakalov, F., Brusilovsky, P., and König-Ries, B. (2013) Progressor: social navigation support through open social student modeling. New Review of Hypermedia and Multimedia • Brusilovsky, P., Somyurek, S., Guerra, J., Hosseini, R., Zadorozhny, V., and Durlach, P. (2016) Open Social Student Modeling for Personalized Learning. IEEE Transactions on Emerging Topics in Computing 4 (3), 450-461. • Jordan, B.-P., Guerra, J., Huang, Y., and Brusilovsky, P. (2017) Concept-Level Knowledge Visualization for Supporting Self-Regulated Learning. In: Proceedings of Companion of the 22nd International Conference on Intelligent User Interfaces (IUI '17), Limassol, Cyprus, ACM, pp. 141-144 also available at https://doi.org/10.1145/3030024.3038262.

×