Successfully reported this slideshow.
Your SlideShare is downloading. ×

Doctoral Consortium UMAP 2020 Jordan Barria-Pineda

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
A Grade Merlin Analysis
A Grade Merlin Analysis
Loading in …3
×

Check these out next

1 of 16 Ad
Advertisement

More Related Content

Similar to Doctoral Consortium UMAP 2020 Jordan Barria-Pineda (20)

Recently uploaded (20)

Advertisement

Doctoral Consortium UMAP 2020 Jordan Barria-Pineda

  1. 1. Exploring the Need forTransparency in Educational Recommender Systems Student: Jordan Barría-Pineda ✉️ jab464@pitt.edu @j_barria Advisor: Dr. Peter Brusilovsky Doctoral Consortium @ UMAP’20
  2. 2. 0 1 2 3 Who am I? Motivation Related Work Progress-to-date4 Proposed Approach DoctoralConsortium@UMAP’20 5 Future Work
  3. 3. 0 DoctoralConsortium@UMAP’20 Who am I? Recommender Systems Visual Analytics User- centered design Technology-Enhanced Learning My research work
  4. 4. 1 Motivation DoctoralConsortium@UMAP’20 Explainable Recommender Systems
  5. 5. 1 Motivation DoctoralConsortium@UMAP’20 Research Gap Explainable Recommender Systems Benefits Domains Online shopping Movies Music Persuasi- veness Trust Efficiency
  6. 6. 1 Motivation DoctoralConsortium@UMAP’20 Research Gap Real-world Utility Explainable Recommender Systems Benefits Domains Online shopping Movies Music Technology- Enhanced Learning (CSEd) Persuasi- veness Trust Efficiency Changing level of knowledge Changing learning goals Measures of effectiveness
  7. 7. 2 Related Work DoctoralConsortium@UMAP’20 Exploring the Need for Explainable Artificial Intelligence (XAI) in Intelligent Tutoring Systems (ITS) (2019) Vanessa Putnam and Cristina Conati at IUI’ 19 Complementing educational recommender systems with open learner models (2020) Solmaz Abdi et. al. at LAK’ 20
  8. 8. 3 Proposed Approach DoctoralConsortium@UMAP’20 • RQ1: What are the most common learning goals that students set to guide their learning in online learning platforms? • RQ2: What type of (a) explanations and (b) presentation formats (e.g. textual, visual) explanatory interfaces would be most appealing and meaningful to students? • RQ3: What are the effects of providing learning content recommendations’ explanations on students’ behavior within an online educational system and their associated perceptions and learning?
  9. 9. 3 Proposed Approach DoctoralConsortium@UMAP’20 • RQ1: What are the most common learning goals that students set to guide their learning in online learning platforms? • RQ2: What type of (a) explanations and (b) presentation formats (e.g. textual, visual) explanatory interfaces would be most appealing and meaningful to students? • RQ3: What are the effects of providing learning content recommendations’ explanations on students’ behavior within an online educational system and their associated perceptions and learning? Iterative User- centered design Learning goals Cognitive processing Student knowledge
  10. 10. 3 Proposed Approach DoctoralConsortium@UMAP’20 Now Sept 2021 Summer 2020 Fall 2020 Spring 2021 Summer 2021 Interviews / focus groups with students Meaningful recommender algorithms Participatory GUI prototyping Transparent (Controllable) GUI designs Controlled user study (e.g. think- aloud protocol) Data Analysis / Thesis writing IRB (ethics office) applic. Final prototype refinement Experimental GUI treatments Classroom study RQ 1 RQ 2 RQ 2-3
  11. 11. 4 Progress-to-date DoctoralConsortium@UMAP’20 3 preliminary studies a) Explanations in context of a Java course (explanations vs no explanations) b) Explanations in context of a SQL course (different explanation formats) c) Explanations in context of a Java course (explanations always vs explanations on-demand) – work in progress 1st study for 1st phase Series of semi-structured interviews with college students
  12. 12. 4 Progress-to-date DoctoralConsortium@UMAP’20 3 preliminary studies a) Explanations in context of a Java course (explanations vs no explanations) b) Explanations in context of a SQL course (different explanation formats) c) Explanations in context of a Java course (explanations always vs explanations on-demand) – work in progress 1st study for 1st phase Series of semi-structured interviews with college students Jordan Barria-Pineda and Peter Brusilovsky. 2019. Making Educational Recommendations Transparent through a Fine-Grained Open Learner Model. In IUI’19 Workshops.
  13. 13. 4 Progress-to-date DoctoralConsortium@UMAP’20 3 preliminary studies a) Explanations in context of a Java course (explanations vs no explanations) b) Explanations in context of a SQL course (different explanation formats) c) Explanations in context of a Java course (explanations always vs explanations on-demand) – work in progress 1st study for 1st phase Series of semi-structured interviews with college students Jordan Barria-Pineda, Kamil Akhuseyinoglu, Peter Brusilovsky, Kerttu Pollari- Malmi, Teemu Sirkiä, Lauri Malmi. 2020. Personalized Remedial Recommendations for SQL Programming Practice System. In APCSE’20 Workshop co-located with UMAP’20.
  14. 14. 4 Progress-to-date DoctoralConsortium@UMAP’20 3 preliminary studies a) Explanations in context of a Java course (explanations vs no explanations) b) Explanations in context of a SQL course (different explanation formats) c) Explanations in context of a Java course (explanations always vs explanations on-demand) – work in progress 1st study for 1st phase Series of semi-structured interviews with college students Learning goals change throughout the term (they are non-static) Close guided support through sequential stages of their learning
  15. 15. 5 Future work DoctoralConsortium@UMAP’20 Target conferences: • UMAP • IUI • AIED Target journals: • UMUAI • ACM TiiS • JAIED or Computers & Education Technology- Enhanced Learning Recommender Systems Intelligent User Interfaces Targets
  16. 16. 5 Future work DoctoralConsortium@UMAP’20 Open Questions Is the scope enough for a PhD thesis? Or maybe too little or too much? Are the proposed research methods appropriate for my RQs? How deep should I go for trying to adapt to students needs? (personalization vs customization)

Editor's Notes

  • Educational Recommender Systems (EdRecSys) are different in nature from conventional Recommender Systems (RecSys) -mostly related to e-commerce- as the main goal of EdRecSys is supporting students learning’ instead of maximizing users’ satisfaction from consuming the recommended items. Thus, research on transparency for traditional RecSys is hard to transfer from e-commerce contexts to educational scenarios, as the level of knowledge of the end-user (i.e. the student) is crucial for generating and evaluating the impact of the recommendations on students’ learning.
  • Educational Recommender Systems (EdRecSys) are different in nature from conventional Recommender Systems (RecSys) -mostly related to e-commerce- as the main goal of EdRecSys is supporting students learning’ instead of maximizing users’ satisfaction from consuming the recommended items. Thus, research on transparency for traditional RecSys is hard to transfer from e-commerce contexts to educational scenarios, as the level of knowledge of the end-user (i.e. the student) is crucial for generating and evaluating the impact of the recommendations on students’ learning.
  • Educational Recommender Systems (EdRecSys) are different in nature from conventional Recommender Systems (RecSys) -mostly related to e-commerce- as the main goal of EdRecSys is supporting students learning’ instead of maximizing users’ satisfaction from consuming the recommended items. Thus, research on transparency for traditional RecSys is hard to transfer from e-commerce contexts to educational scenarios, as the level of knowledge of the end-user (i.e. the student) is crucial for generating and evaluating the impact of the recommendations on students’ learning.
  • • RQ1: What are the most common learning goals that students set to guide their learning in online learning platforms that could be supported by providing learning content recommendations?.

    • RQ2: What type of (a) explanations and (b) presentation formats (e.g. textual, visual) explanatory interfaces would be most appealing and meaningful to students for obtaining information about why specific learning materials were recommended to them?
    – RQ2.1: What information about the recommendation process (e.g., input data, algorithm) should be disclosed?
    – RQ2.2: How should the display format (e.g., visual, textual or a combination) and the interactivity of the explanations?
    – RQ2.3: What are the trade-offs between the amount of information provided and the format/interactivity of the explanations? (in terms of complexity and usefulness for learners).

    • RQ3: What are the effects of providing learning content recommendations’ explanations on students’ behavior within an online educational system and their associated perceptions and learning?
  • • RQ1: What are the most common learning goals that students set to guide their learning in online learning platforms that could be supported by providing learning content recommendations?.

    • RQ2: What type of (a) explanations and (b) presentation formats (e.g. textual, visual) explanatory interfaces would be most appealing and meaningful to students for obtaining information about why specific learning materials were recommended to them?
    – RQ2.1: What information about the recommendation process (e.g., input data, algorithm) should be disclosed?
    – RQ2.2: How should the display format (e.g., visual, textual or a combination) and the interactivity of the explanations?
    – RQ2.3: What are the trade-offs between the amount of information provided and the format/interactivity of the explanations? (in terms of complexity and usefulness for learners).

    • RQ3: What are the effects of providing learning content recommendations’ explanations on students’ behavior within an online educational system and their associated perceptions and learning?
  • - How can you make sure your results are valid?
    1st by involving students throughout the iterations of the design process
    2nd by testing the final system in an ecologically valid context as a programming course
  • A classroom experiment was run where we implemented a Mastery Grids version in an introductory Java course with a rule-based recommendation engine designed for supporting students’ knowledge expansion [1] (see Fig. 1). The group that received explanations exhibited more persuasive recommendations in terms of adopting the suggested material as part of their learning process.
  • In a second related attempt, we developed a Mastery Grids version with remedial recommendations (focused on fixing students’ misconceptions) for an introductory database course. Here, students had access to an explanatory interface which visualized struggling concepts. Four experimental treatments were implemented here regarding the type of recommendation explanation format they had access to (see Fig. 2) [2]. Learners exhibited lower conversion rates for recommended problems than for non-recommended ones when any type of explanations (i.e., visual or textual) were provided. On the subjective side, remedial recommendations were perceived to be of better quality when they were justified by partial recommendation explanation (either only textual or visual) than when received none or complete explanations.
  • Note: I removed one of the systems as wanted to use as study platform

×