1) The document summarizes a proposed social semantic recommender system for learning that would provide recommendations to users in an online learning environment based on their social connections and activities. 2) An empirical study evaluated different recommender algorithms on several datasets to determine the most accurate algorithm, and found the extended T-index algorithm performed best. 3) Further proposed research includes a user evaluation study and pilot study to test the system with users and continue improving the recommender system and evolving the social network.

1. A Social Semantic Recommender
for Learning
Soude Fazeli, PhD candidate
Dr. Hendrik Drachsler
Prof. Dr. Peter Sloep
page 1

2. The doctoral study is funded by
• NELLL
(Netherlands Laboratory for Lifelong Learning at the OUNL)
• Open Discovery Space (ODS)
A socially-powered, multilingual
open learning infrastructure
to boost the adaptation of
eLearning Resources in Europe
Run-time: 2011-2015
page 2

3. A social space for learning
page 3

4. page 4

5. Recommender systems?
page 5

6. Link to Learning Analytics (LA)
• Duval (2011) introduced recommenders as a solution
• To deal with the “paradox of choice”
• To turn the abundance from a problem into an asset for
learning
• Several domains try to find patterns in a large amount of data
• Educational data mining, Big Data, and Web analytics
• Recommender systems and personalization as an important part
of LA research, Greller and Drachsler (2012)
page 6

7. A proposed recommender system for learning
Based on the framework
proposed by Manouselis &
Costopoulou (2007)
For more details, please
refer to Fazeli, S., Drachsler,
H., Brouns, F. and Sloep, P.
(2012)
page 7
!

8. Similarity
Sparsity!
page 8

9. State-of-the-art educational recommenders
• Manouselis et al. (2010)
•
Testing multi-attribute recommenders within Learning Resource Exchange
(http://lreforschools.eun.org)
• Cechinel et al. (2012)
•
Several memory-based collaborative filtering algorithms on the MERLOT
repository (http://www.merlot.org)
• Koukourikos et al. (2012)
•
Using sentiment analysis techniques to enhance collaborative filtering
algorithms within MERLOT dataset
• Sparsity!
• Verbert et al. (2011)
• Different algorithms on several datasets: MACE, Travel well, MovieLens
• Manouselis et al. (2012)
• Organic.Edunet (http://portal.organic-edunet.eu/) and a synthetic dataset
page 9
including the real data plus some simulated data

10. (Golbeck, 2009; Kamvar et al., 2003; Ziegler & Golbeck, 2007;
Massa & Avesani, 2004; Lathia et al., 2008; Fazeli et al., 2010)
page 10

11. A social recommender system:
T-index approach
(Fazeli et al., 2010)
• Creates trust relationships between users
• Based on the ratings information
• Proposes T-index concept
• To measure trustworthiness of users
• To improve the process of finding the nearest neighbours
• Inspired by H-index
• Used to evaluate the publications of an author
• Based on results, T-index improves
• Prediction accuracy of generated recommendations
• Structure of trust networks of users
page 11

12. Trust in recommender systems
• Trustworthy users == like-minded users
• A new trust relationship between two thus far unconnected users
is inferred if and only if:
• Condition 1:
•
page 12
• mutual trust value between intermediate users is higher than a
certain threshold
Condition 2:
• The number of intermediate users is lower than an upper bound;
in this study the upper bound is 2

13. Social trust in recommender systems
rated
rated
Bob
Alice
Carol
page 13
rated
rated
if A trusts B and B trusts C, then A trusts C if and only if
condition 1 is met
and
condition 2 is met

14. Social data
page 14

15. • RQ1: How to generate more accurate and thus,
more relevant recommendations by using the
social data originating from social activities of
users within an online environment?
• RQ2: Can the use of the inter-user trust
relationships that originally come from the social
activities of users within an online environment,
help user networks evolve?
page 15

16. Proposed research
1. Requirement analysis
• Literature review
• Interview study
2. Data-driven study
3. User evaluation study
4. Pilot study
page 16

17. 1. Requirement analysis
• Goal
•
Investigating main needs and requirements of users in an online social environment
• Method
•
•
A summer school for European teachers in Greece, July 2012
Asking the participants to fill in a questionnaire regarding
• The importance or usefulness of the activities within an online social environment
• The use of recommender systems.
• Description
•
•
33 teachers participated from 14 countries (Portugal, Germany, France, Finland,
Greece, Austria, Poland, Lithuania, Spain, Hungary, Romania, Cyprus, Ireland, Serbia
and the US)
“sharing content on Facebook, Twitter, etc. or by email” important, useful or not
• Expected outcomes
•
A list of the most important needs and requirements of teachers within an online social
environment like the ODS portal
page 17

18. 1. Requirement analysis
1.1. Use case diagram
page 18
!

19. 1. Requirement analysis
1.2. Results
How much the teachers find the online social
activities important/useful
How much teachers find the detailed requirements important/
page 19
useful
!

20. Proposed research
1. Requirement analysis
• Literature review
• Interview study
2. Data-driven study
3. User evaluation study
4. Pilot study
page 20

21. 2. Data-driven study
• Goal
•
To find out the most suitable recommender system algorithm for a social
online platform like ODS platform
• Method
•
•
An offline empirical study of candidate algorithms including the extended Tindex algorithm
Datasets:
• TravelWell, Mace, OpenScout, MovieLens (as a standard dataset for comparison)
• Mendeley, MERLOT
• Variables to be measured
• Performance: Precision accuracy, recall, F-measure (F1)
• Network analysis: degree centrality
• Expected outcomes
• Which of the recommender algorithms best performs and thus, is suitable for
social online platforms like ODS platform
page 21

22. 2. Data-driven study
2.1. F1 result
0.1"
0.09"
0.08"
0.07"
0.06"
0.05"
0.04"
0.03"
0.02"
0.01"
0"
OpenScout%
0.14"
0.12"
0.1"
0.08"
Tanimoto3Jaccard"(CF1)"
0.06"
Loglikelihood"(CF2)"
Euclidean"(CF3)"
0.04"
Euclidean"(CF3)"
Graph4based"(CF4)"
0.02"
Graph3based"(CF4)"
Tanimoto4Jaccard"(CF1)"
Loglikelihood"(CF2)"
F1@10%
F1@10%
MACE%
0"
3"
5"
7"
3"
10"
5"
Travel%well%
10"
MovieLens%
0.25"
0.08"
0.2"
0.06"
Tanimoto3Jaccard"(CF1)"
0.04"
Loglikelihood"(CF2)"
Euclidean"(CF3)"
0.02"
Graph3based"(CF4)"
0"
3"
5"
7"
size%of%neighborhood%(n)%
page 22
10"
F1@10%
0.1"
F1@10%
7"
size%of%neighborhood%(n)%
size%of%neighborhood%(n)%
Tanimoto0Jaccard"(CF1)"
0.15"
Loglikelihood"(CF2)"
0.1"
Euclidean"(CF3)"
0.05"
Graph0based"(CF4)"
0"
3"
5"
7"
10"
size%of%neighborhood%(n)%
F1 of the extended T-index and Tanimoto algorithms for different
datasets, based on the size of neighborhood

23. 2. Data-driven study
2.2. user network
page 23

24. 2. Data-driven study
2.3. Degree centrality
250"
200"
150"
MovieLens"
degree%
OpenScout"
100"
MACE"
Travel"well"
50"
0"
u1"
u2"
u3"
u4"
u5"
u6"
u7"
u8"
u9"
u10"
Top)10%central%users%
Degree distribution of top-10 central users for
different datasets
page 24

25. Proposed research
1. Requirement analysis
• Literature review
• Interview study
2. Data-driven study
3. User evaluation study
4. Pilot study
page 25

26. 3. User evaluation study
• Goal
•
To study usability of developed prototype by evaluating
users’ satisfaction
• Method
•
•
Questionnaire
Adapting the user-centric evaluation proposed by Pu et al.
(2011) in the context of recommender systems
• Variables to be measured
•
Quality of recommendations based on accuracy, novelty,
and usefulness
• Expected outcomes
•
page 26
Initial feedback by end-users on users’ satisfaction as an
input for pilot study

27. Proposed research
1. Requirement analysis
• Literature review
• Interview study
2. Data-driven study
3. User evaluation study
4. Pilot study
page 27

28. 4. Pilot study
• Goal
• To deploy the final release
• To test it under realistic operational conditions with the end-users
• Method
• Evaluating performance of the designed recommender system algorithm
• Study the structure of the built users network
• Variables to be measured
• Prediction precision and recall, and F-measure (F1)
• Effectiveness in terms of total number of visited, bookmarked, or rated
•
learning objects for two groups of users (pre and post study)
Degree centrality distribution to study how the structure of users network
changes
• Expected outcomes
•
•
page 28
Empirical data on performance of the used recommender algorithm
The visualization of teachers’ networks

29. Conclusion
• The aim is to support user in social platforms to
find the most suitable content or people
• Recommender systems as a solution
• How to deal with the sparsity problem by use of
social data of users
page 29

30. Ongoing and Further work
• Data set study (May 2013)
•
•
Testing more datasets (Mendeley, MERLOT)
Testing other recommender algorithms (loglikelihood for implicit indicators,
Pearson, Euclidian for explicit indicators)
• Go online with the ODS platform (June 2013)
• User evaluation study (September 2013)
page 30

31. Soude
Fazeli
PhD
candidate
Open
University
of
the
Netherlands
Centre
for
Learning
Sciences
and
Technologies
(CELSTEC)
PO-­‐Box
2960
6401
DL
Heerlen,
The
Netherlands
email:
soude.fazeli@ou.nl
page 31