Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

EC-TEL 2015

220 views

Published on

The conference presentation slides

Published in: Data & Analytics
  • Be the first to comment

EC-TEL 2015

  1. 1. What Should I Do Next? Adaptive Sequencing in the Context of Open Social Student Modeling Roya Hosseini, I-Han Hsiao, Julio Guerra, Peter Brusilovsky
 PAWS Lab
 University of Pittsburgh
  2. 2. Overview • Motivation – why do we care about guidance? • Past work – how to guide students to the right content? • Current work – adaptive sequencing combined with social guidance – what we learned from the classroom study • Work in progress & future work 2
  3. 3. Motivation Goal – personalized guidance to the most appropriate educational content for each learner ! Why personalized guidance? – helps students acquire knowledge faster – improves learning outcomes – reduces navigational overhead – increases student motivation to work with content 3
  4. 4. Existing Guidance Technologies 1. Knowledge-based approaches • decide the most appropriate content for an individual with respect to the domain model, student model, and course goal • adaptation type: • fine-grained concept-based (ELM-ART, NavEx) • coarse-grained topic-based (QuizGuide) ! 2. Social guidance 4
  5. 5. Concept-Based Adaptation Example 2 Example M Example 1 Problem 1 Problem 2 Problem K Concept 1 Concept 2 Concept 3 Concept 4 Concept 5 Concept N Examples Problems Concepts 5
  6. 6. ELM-ART: Adaptive Link Annotation in LISP 6 green bullet indicates a recommended page red bullet indicates a page user is not ready for G. Weber And P. Brusilovsky, IJAIED 2001. Elm-Art: An Adaptive Versatile System For Web-Based Instruction
  7. 7. NavEx: Concept-Based Adaptive Navigation Support bullet is filled based on progress font style denotes the relevance of example a relevant example with no progress an example not ready to be browsed 7 M. Yudelson And P. Brusilovsky, AIED 2005. Navex: Providing Navigation Support For Adaptive Browsing Of Annotated Code Examples.
  8. 8. Topic-Based Adaptation • each topic is associated with a number of educational activities ! • each activity is classified under 1 topic 8 Topic A Topic B Topic C
  9. 9. QuizGuide : Topic-Based Adaptive Navigation Support Current quiz number of arrows: knowledge in the topic (0-3) color Intensity: learning goal P. Brusilovsky, S. Sosnovsky And O. Shcherbinina, E-Learn 2004. Quizguide: Increasing The Educational Value Of Individualized Self-Assessment Quizzes With Adaptive Navigation Support. 9 current prerequisite not-relevant not-ready
  10. 10. Knowledge Maximizer Paradigm 10 Hosseini, R., Brusilovsky, P., & Guerra, J. (AIED 2013, January). Knowledge Maximizer: Concept-based Adaptive Problem Sequencing for Exam Preparation. Learn maximum knowledge from next 
 activity while controlling prerequisites
  11. 11. Existing Guidance Technologies 1. Knowledge-based approaches 2. Social guidance • uses Open Social Student Modeling (OSSM) • students can view each others’ or class knowledge model • almost as efficient as knowledge-based guidance - higher success rates & engagement - much less knowledge engineering overhead • drawback: make students more conservative in their work ! ! 11
  12. 12. Mastery Grids: Topic-based Navigation Support in OSSM Platform anonymized ranked list of peers and their topic-based progress position of current student in class topic-based progress of student topic-based progress of class Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. (EC-TEL 2014). Mastery Grids: An Open Source Social Educational Progress Visualization. 12
  13. 13. • combines social guidance with knowledge-based guidance • enhances the approach to maximize student knowledge • implements the guidance in context of Mastery Grids OSSM • reports the results from the classroom study Sequencing + Open Social Student Modeling 13
  14. 14. Greedy Sequencing (GS) • aims at maximizing student knowledge in domain concepts • concept-based adaptation: - uses prerequisite and outcome concepts in content items 14 User% Modeling% database% Greedy% Sequencing% Knowledge% Report%Service% Rank%C1% Prerequisites% Outcomes% Content%C1:%Concepts%
  15. 15. Greedy Sequencing: Content Ranking by Knowledge Maximization 15 amount of known prerequisites amount of unknown outcomes rank of the content, [0-1] number of outcomes np:number of prerequisites ki: knowledge of concept i wi: weight of concept i, log(tf-idf value)
  16. 16. 16 • marked top three recommendations generated by GS • size of star shows relative rank of content - bigger star —> higher priority
  17. 17. The Study 143 undergraduates in ASU (Fall 2014) Java Programming & Data Structure course ‣ 111 problems — 103 examples — 19 topics ! Study had 2 main Parts (1) no sequencing (Aug. 21 – Sep. 25) (2) with sequencing (Sep. 26 – Oct. 21) • 86 subjects logged into the system • we considered 53 subjects with problem attempts >= 30 17
  18. 18. Navigational Pattern Analysis GS breaks out the common path of social guidance 0.08 0.08 0.16 0.68 0.06 0.05 0.12 0.78 0.17 0.17 0.2 0.47 Jump−Backward Jump−Forward Next−Topic Within−Topic Part 1 Part 2−N Part 2−R when following GS, “groupthink” stay on the current topic shortens considerably ! students moved to next topic more quickly & expanded their non-sequential navigation
  19. 19. Value of GS on Amount of Learning & Speed 19 Learning gain: • no significant differences in the learning gain - non-followers (M = 0.50, SD = 0.27) - followers (M=0.44, SD=0.23) ! Learning speed: (learning gain/number of problem attempts)×100 ! • speed of learning was higher among the followers - non-followers (M = 0.54%, SD = 0.27%) - followers (M = 0.97%,SD = 0.88%) speed increased about twice - p = .083, using a Welch t-test
  20. 20. Value of GS on Learning & Speed: Weak vs. Strong Students 20 0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.40# 1.60# 1.80# 2.00# Weak#students# Strong#students# %#Learning#speed## Non;followers# Followers# 0" 0.1" 0.2" 0.3" 0.4" 0.5" 0.6" 0.7" 0.8" 0.9" Weak"students" Strong"students" Normalized"learning"gain" Non?followers" Followers" • no significant differences in learning gain • followers with high prior knowledge learn faster (p=.039)
  21. 21. Value of GS on Problem Solving Performance 21 Correctness is more frequent in recommended problems • odds of correct answer in a problem offered by GS was 1.59 (SE = 0.19) times more than a not-recommended problem How: • data collected from part 1 and 2 of study (5760 problem attempts: 5275 not-recommended, 485 offered by GS) • fitted a logistic mixed effects model • fixed effect: attempt type (recommended, not-recommended) • response variable: correctness of attempt (0/1)
  22. 22. Value of GS on Class Performance 22 An attempt on a GS recommendation was associated with higher grade • attempting a recommended content (problem/example) was associated with 0.56 increase in final grade (SE=0.24, p=.017) ~ 9 times greater than the effect of a not-recommended content How: • data of 40 students (had exam score + used system) • fitted regression model to predict exam grade using number of attempts on contents
  23. 23. • 6 questions (5-point Likert scale) • data collected from 51 students (answered questionnaire + used the system) M:4.1 M:3.9 M:3.1 M:3.8 M:4.2M:2.4 Subjective Feedback 23 like star useful clear ! reason distractive
  24. 24. Wrap Up adaptive sequencing + social guidance: ! ✓encouraged non-sequential navigation patterns  ✓increased learning speed of stronger students ‣ more optimal content navigation ✓was positively related to student performance ‣ higher exam score ‣ more success in problems
  25. 25. Work in Progress & Future Work ๏ running study with over 200 students in ASU - GS vs. probabilistic approach based on FAST ! ๏ what is the best way to visualize student/class data? - alternatives to topic-based guidance (2D content maps ) ! ๏ how to increase students’ awareness of recommendations? - adding annotations, …
  26. 26. References Knowledge Maximizer: Hosseini, R., Brusilovsky, P., & Guerra, J. (2013, January). Knowledge Maximizer: Concept-based Adaptive Problem Sequencing for Exam Preparation. In Artificial Intelligence in Education (pp. 848-851). Springer Berlin Heidelberg. ! Mastery Grids: Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. (2014). Mastery Grids: An Open Source Social Educational Progress Visualization. In Open Learning and Teaching in Educational Communities (pp. 235-248). Springer International Publishing. ! QuizGuide: P. Brusilovsky, S. Sosnovsky And O. Shcherbinina, 2004. Quizguide: Increasing The Educational Value Of Individualized Self-Assessment Quizzes With Adaptive Navigation Support. In: J. Nall And R. Robson, Eds., World Conference On Elearning, E-Learn 2004 Aace, Washington, Dc, Usa, 1806-1813. ! NavEx: M. Yudelson And P. Brusilovsky, 2005. Navex: Providing Navigation Support For Adaptive Browsing Of Annotated Code Examples. In: C.-K. Looi, G. Mccalla, B. Bredeweg And J. Breuker, Eds., 12Th International Conference On Artificial Intelligence In Education, Ai-Ed'2005 Ios Press, Amsterdam, The Netherlands, 710-717. ! ELM-ART: G. Weber And P. Brusilovsky, 2001. Elm-Art: An Adaptive Versatile System For Web-Based Instruction. International Journal Of Artificial Intelligence In Education, 12 (4), 351-384 26
  27. 27. Thank You! Intelligent Systems Program Roya Hosseini roh38@pitt.edu Peter Brusilovsky peterb@pitt.edu I-Han (Sharon) Hsiao sharon.hsiao@asu.edu Julio Guerra jdg60@pitt.edu Try it! adapt2.sis.pitt.edu/kt/mg-gs.html https://www.youtube.com/watch?v=Kak8F2y5GkU

×