Boyd Potts, Hassan Khosravi , Carl Reidsema, Aneesha Bakharia, Mark Belonogof, Melanie Fleming (2018). Proceeding of the 8th International Learning Analytics and Knowledge (LAK) Conference

1. Reciprocal Peer Recommendation for Learning Purposes Page 1
Reciprocal Peer Recommendation
for Learning Purposes
LAK 2018
Carl Reidsema
The University of Queensland
c.reidsema@uq.edu.au
Hassan Khosravi
The University of Queensland
h.khosravi@uq.edu.au
Aneesha Bakharia
The University of Queensland
a.bakharia1@uq.edu.au
Melanie Fleming
The University of Queensland
melanie.fleming@uq.edu.au
Mark Belonogoff
The University of Queensland
mark.belonogoff@gmail.com
Boyd A. Potts
The University of Queensland
b.potts@uqconnect.edu.au
@haskhosravi @ReidsemaC
@aneesha

2. Reciprocal Peer Recommendation for Learning Purposes Page 2
Introduction
Related Work
The RiPPLE Platform
Compatibility Function
Reciprocal Peer Recommendation
Evaluation and Future Work

3. Reciprocal Peer Recommendation for Learning Purposes Page 3
Introduction
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
Higher education full year student data,
commencing students by year
(Department of Education and Training, 2017)
Both increased student enrolments and availability of
MOOCs have resulted in increased student/staff ratios.

4. Reciprocal Peer Recommendation for Learning Purposes Page 4
Peer Learning
• The potential benefits and significance
of peer learning have long been
recognised (Boud et al., 2014)
• Engagement and networks contribute to
student success (Wilcox et al., 2005)
• Learning communities lead to the
development of cognitive, intellectual,
communication and professional skills
(Falchikov, 2001)
• Participation in networks is an important
predictor of employability (Van Der
Heijden et a., 2019)
A beneficial way to address high student/staff ratios is to
introduce peer learning and support.

5. Reciprocal Peer Recommendation for Learning Purposes Page 5
Facilitation of Peer Learning
Methods for effective facilitation of peer learning and support present
a current challenge.
Source: http://www.uft.org/linking-learning/creating-student-tech-team
Student learners are hesitant to approach each other

6. Reciprocal Peer Recommendation for Learning Purposes Page 6
Introduction
The RiPPLE Platform
Compatibility Function
Reciprocal Peer Recommendation
Evaluation and Future Work
Related Work

7. Reciprocal Peer Recommendation for Learning Purposes Page 7
Peer Learning and Group Formation
• PHeLpS provided students with the means to find peer
helpers (Greer et al., 1998).
• I-Help focused on just-in-time requests for help in an
online environment (Bull et al., 2001)
• DEPTHS used participants’ competencies to suggest
potential collaborators (Jeremić et al., 2009)
• DIANA addressed the formation of small heterogeneous
groups for the purposes of collaborative learning. (Moreno
et al., 2012)

8. Reciprocal Peer Recommendation for Learning Purposes Page 8
Recommender Systems for TEL
• Much of the primary research is directed at the
recommendation of relevant content and resources:
– Documents and resources (Mangina and Kilbride, 2008)
– Courses (Bousbahi and Chorfi, 2015)
– Student authored questions (Khosravi et al, 2017)
• Also, used for predicting student performance (Thai-Nghe
et al., 2011)
(Drachsler et al, 2015) performed an extensive classification of 82
different systems

9. Reciprocal Peer Recommendation for Learning Purposes Page 9
Reciprocal Recommender Systems
• Much of the research in this field has been developed and
evaluated in existing social networks and particularly
online dating sites (Pizzato et al., 2013).
• (Prabhakar et al, 2017) proposed a reciprocal
recommender system for learners in MOOCs
Reciprocal recommendation seeks to connect two users such that both
sets of preferences are satisfied

10. Reciprocal Peer Recommendation for Learning Purposes Page 10
Introduction
Related Work
Compatibility Function
Reciprocal Peer Recommendation
Evaluation and Future Work
The RiPPLE Platform

11. Reciprocal Peer Recommendation for Learning Purposes Page 11
The RiPPLE Platform
Platform Description: http://hassan-khosravi.net/publications/khosravi2018ripple.pdf
Git Repository: https://github.com/hkhosrav/RiPPLE-Core
Demo: https://hkhosrav.github.io/RiPPLE-Core/?demoStudent=true#/question/answer
Recommendation in Personalised Peer Learning
Environments (RiPPLE) is an open source, adaptive, student-
facing learning platform that provides:
1. Co-creation
2. Knowledge tracing
3. Content recommendation
4. Study session recommendation

12. Reciprocal Peer Recommendation for Learning Purposes Page 12
Study Session Recommendation
• Individuals nominate weekly availability and learning support
preferences
• Indicator of competency are updated with cumulative assessment
over learning period)

13. Reciprocal Peer Recommendation for Learning Purposes Page 13
Problem Formulation
Element Form Description
Requests RUxLxQ A three-dimensional array where Rulq = 1 indicates that user u has
indicated interest in participating in a study session on topic l with
role q
Availability AUxT A two-dimensional array in which Aut = 1 shows that user u is
available at time t
Competencies CUxL A two-dimensional array in which Cul shows the competency of user
u in topic l on a 100-point scale
Preferences PUxQ A two-dimensional array in which Puq shows the competency
preference of a user u in role q
Output: a list of up to k recommendations for each user, where a
recommendation is of the form [u1, u2, [l], [q], t, s] indicating that
user u1 receives recommendation to connect with user u2 on a list
of topic [l] on a list of roles [q] at time t with a reciprocal score of s.

14. Reciprocal Peer Recommendation for Learning Purposes Page 14
Compatibility Function
• Compatibility between two users is computed using
1. Joint competency threshold
2. Competency preferences –
• i.e. how competent do you prefer a partner to be?
• Role-driven

15. Reciprocal Peer Recommendation for Learning Purposes Page 15
Introduction
Related Work
The RiPPLE Platform
Reciprocal Peer Recommendation
Evaluation and Future Work
Compatibility Function

16. Reciprocal Peer Recommendation for Learning Purposes Page 16
Joint Competency
• Define joint competency as the magnitude of the vector of
two competencies in Cartesian space
• Propose that peers’ joint competency (J) should be above a
certain threshold (τ < J) for effective sessions
0
20
40
60
80
100
0 50 100
Competency, user1
Competency, user2
Topic l
user1 user2 J

17. Reciprocal Peer Recommendation for Learning Purposes Page 17
Joint Competency Threshold
• Use joint competency (J) in a logistic function (H) to
compute the extent to which a partnership meets the
desirable threshold (τ), with leniency parameter (α)
-0.3
0.2
0.7
1.2
0 0.5 1
H score
Joint competency
more strict more lenient
τ

18. Reciprocal Peer Recommendation for Learning Purposes Page 18
Competency Preferences
• Users set preferences (puq) for the competencies in
their peers
• E.g. pu11 = -10 means u1 is comfortable providing
support to peers whose competency is 10 points below
their own competency
• E.g. pu22 = 75 means u2 is seeking support from peers
whose competency is 75 points above their own
competency

19. Reciprocal Peer Recommendation for Learning Purposes Page 19
Competency Preferences Model
• Compatibility w.r.t. puq is calculated as the height of a
Gaussian function (G) with centre Cu1l + pu1q and standard
deviation σ
• σ models the leniency for matching peers that do not fit
their exact preference puq
• pu1q = -10 , Cu1l = 75
0
20
40
60
80
100
120
0 20 40 60 80 100 120
G score
Competency

20. Reciprocal Peer Recommendation for Learning Purposes Page 20
Compatibility Function
• Putting it together:
• The compatibility score (s) between two users is the
product of H and G, summed over matched topics l and
related role preference q

21. Reciprocal Peer Recommendation for Learning Purposes Page 21
Introduction
Related Work
The RiPPLE Platform
Compatibility Function
Evaluation and Future Work
Reciprocal Peer Recommendation

22. Reciprocal Peer Recommendation for Learning Purposes Page 22
Reciprocal Peer Recommendation
• The harmonic mean guarantees to provide smaller reciprocal
scores for users whose compatibilities differ considerably, so
as to prioritise recommendations that benefit both users
1. Select a user u1, then for each other user (u2) uses A to find a mutually convenient
time slot
2. R is used to find a set of matching roles and associated topics
3. Users not satisfying constraints A and R receive score ε
4. Reciprocal score Score[u2] is calculated as the harmonic mean of the
compatibilities from u1→u2 and u2→u1

23. Reciprocal Peer Recommendation for Learning Purposes Page 23
Reciprocal Peer Recommendation
• u1 is providing support and prefers users who have competency 10
points lower)
• u2 is seeking peer support with a preference for those who have
competency 75 points higher
• The extent to which both users are recommended to each other is
defined by the harmonic mean distribution shown in the third frame
Preference of the peer
providing supporter
Reciprocal score
Preference of the peer
receiving supporter

24. Reciprocal Peer Recommendation for Learning Purposes Page 24
Introduction
Related Work
The RiPPLE Platform
Compatibility Function
Reciprocal Peer Recommendation
Evaluation and Future Work

25. Reciprocal Peer Recommendation for Learning Purposes Page 25
Experimental Environment Setup
• Synthetic data generated for R, C, A, P (see paper
for details)
• Evaluation Metrics.
– Scalability: Based on the time taken for running
algorithm 1.
– Reciprocality: Based on precision of reciprocal
recommender systems as described on the next page.
– Coverage: Based on the percentage of users that have
been recommended at least once to other users.
– Quality: Based on the average joint competency of
learners that are recommended to each other across all
learners.

26. Reciprocal Peer Recommendation for Learning Purposes Page 26
Scalability and Reciprocality
Scalability Reciprocality
Precision: learner u1 is a successful (reciprocal) recommendation (out of the K-
total) for learner u2, if and only if u1 is also in the top k recommendations of
learner u2 (Prabhakar et al, 2017).

27. Reciprocal Peer Recommendation for Learning Purposes Page 27
Coverage and Quality
Coverage Quality
Coverage of the platform as U is
increased under different settings for τ
Approximating the quality of the
recommendations as τ is increased
under different settings for U and α

28. Reciprocal Peer Recommendation for Learning Purposes Page 28
Future Work
• Subsequent empirical evaluation –
– Designed A/B testing in RiPPLE; evaluate with control
group the effectiveness of the recommendations
– Behavioural – how learners choose among
recommendations, conditions of accepting
recommendations
• Extend the platform to provide reciprocal content
recommendation for peer learning study sessions
– Submitted to AIED 2018

29. Reciprocal Peer Recommendation for Learning Purposes Page 29
References
1. Boud, D., Cohen, R., & Sampson, J. (Eds.). (2014). Peer learning in higher education: Learning from and with each other. Routledge.
2. Bousbahi, F., & Chorfi, H. (2015). MOOC-Rec: a case based recommender system for MOOCs. Procedia-Social and Behavioral
Sciences, 195, 1813-1822.
3. Bull, S., Greer, J., McCalla, G., Kettel, L., & Bowes, J. (2001, July). User modelling in i-help: What, why, when and how.
In International Conference on User Modeling (pp. 117-126). Springer, Berlin, Heidelberg.
4. Department of Education and Training - Document library, Australian Government - https://docs.education.gov.au/node/45146
5. Drachsler, H., Verbert, K., Santos, O. C., & Manouselis, N. (2015). Panorama of recommender systems to support learning.
In Recommender systems handbook (pp. 421-451). Springer, Boston, MA.
6. Falchikov, N. (2001). Learning together: Peer tutoring in higher education. Psychology Press.
7. Greer, J., McCalla, G., Cooke, J., Collins, J., Kumar, V., Bishop, A., & Vassileva, J. (1998, August). The intelligent helpdesk: Supporting
peer-help in a university course. In International Conference on Intelligent Tutoring Systems (pp. 494-503). Springer, Berlin, Heidelberg.
8. Jeremić, Z., Jovanović, J., & Gašević, D. (2009, October). Semantic web technologies for the integration of learning tools and context-
aware educational services. In International Semantic Web Conference (pp. 860-875). Springer, Berlin, Heidelberg.
9. Khosravi, H., Cooper, K., & Kitto, K. (2017). RiPLE: Recommendation in Peer-Learning Environments Based on Knowledge Gaps and
Interests. Journal of Educational Data Mining, 9(1).
10. Mangina, E., & Kilbride, J. (2008). Evaluation of keyphrase extraction algorithm and tiling process for a document/resource
recommender within e-learning environments. Computers & Education, 50(3), 807-820.
11. Moreno, J., Ovalle, D. A., & Vicari, R. M. (2012). A genetic algorithm approach for group formation in collaborative learning
considering multiple student characteristics. Computers & Education, 58(1), 560-569.
12. Pizzato, L., Rej, T., Akehurst, J., Koprinska, I., Yacef, K., & Kay, J. (2013). Recommending people to people: the nature of reciprocal
recommenders with a case study in online dating. User Modeling and User-Adapted Interaction, 23(5), 447-488.
13. Prabhakar, S., Spanakis, G., & Zaiane, O. (2017, September). Reciprocal Recommender System for Learners in Massive Open Online
Courses (MOOCs). In International Conference on Web-Based Learning (pp. 157-167). Springer, Cham.
14. Thai-Nghe, N., Drumond, L., Horváth, T., Krohn-Grimberghe, A., Nanopoulos, A., & Schmidt-Thieme, L. (2011). Factorization
techniques for predicting student performance. Educational recommender systems and technologies: Practices and challenges, 129-153.
15. Van Der Heijden, B., Boon, J., Van der Klink, M., & Meijs, E. (2009). Employability enhancement through formal and informal learning:
an empirical study among Dutch non-academic university staff members. International journal of training and development, 13(1), 19-
37.
16. Wilcox, P., Winn, S., & Fyvie-Gauld, M. (2005). ‘It was nothing to do with the university, it was just the people’: the role of social
support in the first-year experience of higher education. Studies in higher education, 30(6), 707-722.

30. Reciprocal Peer Recommendation for Learning Purposes Page 30
Thank you!
Carl Reidsema
The University of Queensland
c.reidsema@uq.edu.au
Hassan Khosravi
The University of Queensland
h.khosravi@uq.edu.au
Aneesha Bakharia
The University of Queensland
a.bakharia1@uq.edu.au
Melanie Fleming
The University of Queensland
melanie.fleming@uq.edu.au
Mark Belonogoff
The University of Queensland
mark.belonogoff@gmail.com
Boyd A. Potts
The University of Queensland
b.potts@uqconnect.edu.au
@haskhosravi @ReidsemaC
@aneesha