Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

LAK18 Reciprocal Peer Recommendation for Learning Purposes

111 views

Published on

Boyd Potts, Hassan Khosravi , Carl Reidsema, Aneesha Bakharia, Mark Belonogof, Melanie Fleming (2018). Proceeding of the 8th International Learning Analytics and Knowledge (LAK) Conference

Published in: Education
  • Be the first to comment

  • Be the first to like this

LAK18 Reciprocal Peer Recommendation for Learning Purposes

  1. 1. Reciprocal Peer Recommendation for Learning Purposes Page 1 Reciprocal Peer Recommendation for Learning Purposes LAK 2018 Carl Reidsema The University of Queensland c.reidsema@uq.edu.au Hassan Khosravi The University of Queensland h.khosravi@uq.edu.au Aneesha Bakharia The University of Queensland a.bakharia1@uq.edu.au Melanie Fleming The University of Queensland melanie.fleming@uq.edu.au Mark Belonogoff The University of Queensland mark.belonogoff@gmail.com Boyd A. Potts The University of Queensland b.potts@uqconnect.edu.au @haskhosravi @ReidsemaC @aneesha
  2. 2. Reciprocal Peer Recommendation for Learning Purposes Page 2 Introduction Related Work The RiPPLE Platform Compatibility Function Reciprocal Peer Recommendation Evaluation and Future Work
  3. 3. Reciprocal Peer Recommendation for Learning Purposes Page 3 Introduction 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Higher education full year student data, commencing students by year (Department of Education and Training, 2017) Both increased student enrolments and availability of MOOCs have resulted in increased student/staff ratios.
  4. 4. Reciprocal Peer Recommendation for Learning Purposes Page 4 Peer Learning • The potential benefits and significance of peer learning have long been recognised (Boud et al., 2014) • Engagement and networks contribute to student success (Wilcox et al., 2005) • Learning communities lead to the development of cognitive, intellectual, communication and professional skills (Falchikov, 2001) • Participation in networks is an important predictor of employability (Van Der Heijden et a., 2019) A beneficial way to address high student/staff ratios is to introduce peer learning and support.
  5. 5. Reciprocal Peer Recommendation for Learning Purposes Page 5 Facilitation of Peer Learning Methods for effective facilitation of peer learning and support present a current challenge. Source: http://www.uft.org/linking-learning/creating-student-tech-team Student learners are hesitant to approach each other
  6. 6. Reciprocal Peer Recommendation for Learning Purposes Page 6 Introduction The RiPPLE Platform Compatibility Function Reciprocal Peer Recommendation Evaluation and Future Work Related Work
  7. 7. Reciprocal Peer Recommendation for Learning Purposes Page 7 Peer Learning and Group Formation • PHeLpS provided students with the means to find peer helpers (Greer et al., 1998). • I-Help focused on just-in-time requests for help in an online environment (Bull et al., 2001) • DEPTHS used participants’ competencies to suggest potential collaborators (Jeremić et al., 2009) • DIANA addressed the formation of small heterogeneous groups for the purposes of collaborative learning. (Moreno et al., 2012)
  8. 8. Reciprocal Peer Recommendation for Learning Purposes Page 8 Recommender Systems for TEL • Much of the primary research is directed at the recommendation of relevant content and resources: – Documents and resources (Mangina and Kilbride, 2008) – Courses (Bousbahi and Chorfi, 2015) – Student authored questions (Khosravi et al, 2017) • Also, used for predicting student performance (Thai-Nghe et al., 2011) (Drachsler et al, 2015) performed an extensive classification of 82 different systems
  9. 9. Reciprocal Peer Recommendation for Learning Purposes Page 9 Reciprocal Recommender Systems • Much of the research in this field has been developed and evaluated in existing social networks and particularly online dating sites (Pizzato et al., 2013). • (Prabhakar et al, 2017) proposed a reciprocal recommender system for learners in MOOCs Reciprocal recommendation seeks to connect two users such that both sets of preferences are satisfied
  10. 10. Reciprocal Peer Recommendation for Learning Purposes Page 10 Introduction Related Work Compatibility Function Reciprocal Peer Recommendation Evaluation and Future Work The RiPPLE Platform
  11. 11. Reciprocal Peer Recommendation for Learning Purposes Page 11 The RiPPLE Platform Platform Description: http://hassan-khosravi.net/publications/khosravi2018ripple.pdf Git Repository: https://github.com/hkhosrav/RiPPLE-Core Demo: https://hkhosrav.github.io/RiPPLE-Core/?demoStudent=true#/question/answer Recommendation in Personalised Peer Learning Environments (RiPPLE) is an open source, adaptive, student- facing learning platform that provides: 1. Co-creation 2. Knowledge tracing 3. Content recommendation 4. Study session recommendation
  12. 12. Reciprocal Peer Recommendation for Learning Purposes Page 12 Study Session Recommendation • Individuals nominate weekly availability and learning support preferences • Indicator of competency are updated with cumulative assessment over learning period)
  13. 13. Reciprocal Peer Recommendation for Learning Purposes Page 13 Problem Formulation Element Form Description Requests RUxLxQ A three-dimensional array where Rulq = 1 indicates that user u has indicated interest in participating in a study session on topic l with role q Availability AUxT A two-dimensional array in which Aut = 1 shows that user u is available at time t Competencies CUxL A two-dimensional array in which Cul shows the competency of user u in topic l on a 100-point scale Preferences PUxQ A two-dimensional array in which Puq shows the competency preference of a user u in role q Output: a list of up to k recommendations for each user, where a recommendation is of the form [u1, u2, [l], [q], t, s] indicating that user u1 receives recommendation to connect with user u2 on a list of topic [l] on a list of roles [q] at time t with a reciprocal score of s.
  14. 14. Reciprocal Peer Recommendation for Learning Purposes Page 14 Compatibility Function • Compatibility between two users is computed using 1. Joint competency threshold 2. Competency preferences – • i.e. how competent do you prefer a partner to be? • Role-driven
  15. 15. Reciprocal Peer Recommendation for Learning Purposes Page 15 Introduction Related Work The RiPPLE Platform Reciprocal Peer Recommendation Evaluation and Future Work Compatibility Function
  16. 16. Reciprocal Peer Recommendation for Learning Purposes Page 16 Joint Competency • Define joint competency as the magnitude of the vector of two competencies in Cartesian space • Propose that peers’ joint competency (J) should be above a certain threshold (τ < J) for effective sessions 0 20 40 60 80 100 0 50 100 Competency, user1 Competency, user2 Topic l user1 user2 J
  17. 17. Reciprocal Peer Recommendation for Learning Purposes Page 17 Joint Competency Threshold • Use joint competency (J) in a logistic function (H) to compute the extent to which a partnership meets the desirable threshold (τ), with leniency parameter (α) -0.3 0.2 0.7 1.2 0 0.5 1 H score Joint competency more strict more lenient τ
  18. 18. Reciprocal Peer Recommendation for Learning Purposes Page 18 Competency Preferences • Users set preferences (puq) for the competencies in their peers • E.g. pu11 = -10 means u1 is comfortable providing support to peers whose competency is 10 points below their own competency • E.g. pu22 = 75 means u2 is seeking support from peers whose competency is 75 points above their own competency
  19. 19. Reciprocal Peer Recommendation for Learning Purposes Page 19 Competency Preferences Model • Compatibility w.r.t. puq is calculated as the height of a Gaussian function (G) with centre Cu1l + pu1q and standard deviation σ • σ models the leniency for matching peers that do not fit their exact preference puq • pu1q = -10 , Cu1l = 75 0 20 40 60 80 100 120 0 20 40 60 80 100 120 G score Competency
  20. 20. Reciprocal Peer Recommendation for Learning Purposes Page 20 Compatibility Function • Putting it together: • The compatibility score (s) between two users is the product of H and G, summed over matched topics l and related role preference q
  21. 21. Reciprocal Peer Recommendation for Learning Purposes Page 21 Introduction Related Work The RiPPLE Platform Compatibility Function Evaluation and Future Work Reciprocal Peer Recommendation
  22. 22. Reciprocal Peer Recommendation for Learning Purposes Page 22 Reciprocal Peer Recommendation • The harmonic mean guarantees to provide smaller reciprocal scores for users whose compatibilities differ considerably, so as to prioritise recommendations that benefit both users 1. Select a user u1, then for each other user (u2) uses A to find a mutually convenient time slot 2. R is used to find a set of matching roles and associated topics 3. Users not satisfying constraints A and R receive score ε 4. Reciprocal score Score[u2] is calculated as the harmonic mean of the compatibilities from u1→u2 and u2→u1
  23. 23. Reciprocal Peer Recommendation for Learning Purposes Page 23 Reciprocal Peer Recommendation • u1 is providing support and prefers users who have competency 10 points lower) • u2 is seeking peer support with a preference for those who have competency 75 points higher • The extent to which both users are recommended to each other is defined by the harmonic mean distribution shown in the third frame Preference of the peer providing supporter Reciprocal score Preference of the peer receiving supporter
  24. 24. Reciprocal Peer Recommendation for Learning Purposes Page 24 Introduction Related Work The RiPPLE Platform Compatibility Function Reciprocal Peer Recommendation Evaluation and Future Work
  25. 25. Reciprocal Peer Recommendation for Learning Purposes Page 25 Experimental Environment Setup • Synthetic data generated for R, C, A, P (see paper for details) • Evaluation Metrics. – Scalability: Based on the time taken for running algorithm 1. – Reciprocality: Based on precision of reciprocal recommender systems as described on the next page. – Coverage: Based on the percentage of users that have been recommended at least once to other users. – Quality: Based on the average joint competency of learners that are recommended to each other across all learners.
  26. 26. Reciprocal Peer Recommendation for Learning Purposes Page 26 Scalability and Reciprocality Scalability Reciprocality Precision: learner u1 is a successful (reciprocal) recommendation (out of the K- total) for learner u2, if and only if u1 is also in the top k recommendations of learner u2 (Prabhakar et al, 2017).
  27. 27. Reciprocal Peer Recommendation for Learning Purposes Page 27 Coverage and Quality Coverage Quality Coverage of the platform as U is increased under different settings for τ Approximating the quality of the recommendations as τ is increased under different settings for U and α
  28. 28. Reciprocal Peer Recommendation for Learning Purposes Page 28 Future Work • Subsequent empirical evaluation – – Designed A/B testing in RiPPLE; evaluate with control group the effectiveness of the recommendations – Behavioural – how learners choose among recommendations, conditions of accepting recommendations • Extend the platform to provide reciprocal content recommendation for peer learning study sessions – Submitted to AIED 2018
  29. 29. Reciprocal Peer Recommendation for Learning Purposes Page 29 References 1. Boud, D., Cohen, R., & Sampson, J. (Eds.). (2014). Peer learning in higher education: Learning from and with each other. Routledge. 2. Bousbahi, F., & Chorfi, H. (2015). MOOC-Rec: a case based recommender system for MOOCs. Procedia-Social and Behavioral Sciences, 195, 1813-1822. 3. Bull, S., Greer, J., McCalla, G., Kettel, L., & Bowes, J. (2001, July). User modelling in i-help: What, why, when and how. In International Conference on User Modeling (pp. 117-126). Springer, Berlin, Heidelberg. 4. Department of Education and Training - Document library, Australian Government - https://docs.education.gov.au/node/45146 5. Drachsler, H., Verbert, K., Santos, O. C., & Manouselis, N. (2015). Panorama of recommender systems to support learning. In Recommender systems handbook (pp. 421-451). Springer, Boston, MA. 6. Falchikov, N. (2001). Learning together: Peer tutoring in higher education. Psychology Press. 7. Greer, J., McCalla, G., Cooke, J., Collins, J., Kumar, V., Bishop, A., & Vassileva, J. (1998, August). The intelligent helpdesk: Supporting peer-help in a university course. In International Conference on Intelligent Tutoring Systems (pp. 494-503). Springer, Berlin, Heidelberg. 8. Jeremić, Z., Jovanović, J., & Gašević, D. (2009, October). Semantic web technologies for the integration of learning tools and context- aware educational services. In International Semantic Web Conference (pp. 860-875). Springer, Berlin, Heidelberg. 9. Khosravi, H., Cooper, K., & Kitto, K. (2017). RiPLE: Recommendation in Peer-Learning Environments Based on Knowledge Gaps and Interests. Journal of Educational Data Mining, 9(1). 10. Mangina, E., & Kilbride, J. (2008). Evaluation of keyphrase extraction algorithm and tiling process for a document/resource recommender within e-learning environments. Computers & Education, 50(3), 807-820. 11. Moreno, J., Ovalle, D. A., & Vicari, R. M. (2012). A genetic algorithm approach for group formation in collaborative learning considering multiple student characteristics. Computers & Education, 58(1), 560-569. 12. Pizzato, L., Rej, T., Akehurst, J., Koprinska, I., Yacef, K., & Kay, J. (2013). Recommending people to people: the nature of reciprocal recommenders with a case study in online dating. User Modeling and User-Adapted Interaction, 23(5), 447-488. 13. Prabhakar, S., Spanakis, G., & Zaiane, O. (2017, September). Reciprocal Recommender System for Learners in Massive Open Online Courses (MOOCs). In International Conference on Web-Based Learning (pp. 157-167). Springer, Cham. 14. Thai-Nghe, N., Drumond, L., Horváth, T., Krohn-Grimberghe, A., Nanopoulos, A., & Schmidt-Thieme, L. (2011). Factorization techniques for predicting student performance. Educational recommender systems and technologies: Practices and challenges, 129-153. 15. Van Der Heijden, B., Boon, J., Van der Klink, M., & Meijs, E. (2009). Employability enhancement through formal and informal learning: an empirical study among Dutch non-academic university staff members. International journal of training and development, 13(1), 19- 37. 16. Wilcox, P., Winn, S., & Fyvie-Gauld, M. (2005). ‘It was nothing to do with the university, it was just the people’: the role of social support in the first-year experience of higher education. Studies in higher education, 30(6), 707-722.
  30. 30. Reciprocal Peer Recommendation for Learning Purposes Page 30 Thank you! Carl Reidsema The University of Queensland c.reidsema@uq.edu.au Hassan Khosravi The University of Queensland h.khosravi@uq.edu.au Aneesha Bakharia The University of Queensland a.bakharia1@uq.edu.au Melanie Fleming The University of Queensland melanie.fleming@uq.edu.au Mark Belonogoff The University of Queensland mark.belonogoff@gmail.com Boyd A. Potts The University of Queensland b.potts@uqconnect.edu.au @haskhosravi @ReidsemaC @aneesha

×