Revisiting the Multi-Criteria Recommender System of a Learning Portal
Upcoming SlideShare
Loading in...5
×
 

Revisiting the Multi-Criteria Recommender System of a Learning Portal

on

  • 711 views

Presentation of paper for Recommender Systems in Technology Enhanced Learning (RecSysTEL) workshop, ECTEL'12, Saarbruecken, Germany

Presentation of paper for Recommender Systems in Technology Enhanced Learning (RecSysTEL) workshop, ECTEL'12, Saarbruecken, Germany

Statistics

Views

Total Views
711
Views on SlideShare
689
Embed Views
22

Actions

Likes
0
Downloads
11
Comments
0

2 Embeds 22

http://www.agroknow.gr 12
http://agroknow.redpanda.gr 10

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Revisiting the Multi-Criteria Recommender System of a Learning Portal Revisiting the Multi-Criteria Recommender System of a Learning Portal Presentation Transcript

  • Revisiting the Multi-CriteriaRecommender System of a Learning Portal Nikos Manouselis1, Giorgos Kyrgiazos2, Giannis Stoitsis1 Agro-Know Technologies, 2CTI 1 @RecSysTEL’12, Saarbruecken, 19/9/12
  • our nice portal
  • our nice portal
  • collected data•id user item•Name* URL•Email* tags Organic.Edunet •Value social data schema •Date •Value •Dimension •Date reviews ratings •Value
  • current service• recommendation of potentially interesting learning resources to users – not very “loud”• one recommendation algorithm based on collaborative filtering – rating history – neighborhood-based – multi-attribute over 3 criteria [Subject Relevance, Educational Usefulness, Metadata] – parameters defined & hard-coded
  • issues• lots of parameters could be different – selected recommendation methods – neighborhood size – similarity measures• parameterization took place using a similar dataset [but not the same] – EUN’s Learning Resource Exchange (MELT) multi- attribute ratings dump• Organic.Edunet’s user/content base continuously evolves
  • in the year 2007…
  • in the year 2007…
  • problem outline• How do we know that the selected algorithm is still(?) good for the given portal? – specific rating dimensions (criteria) – selected parameterization – alternative algorithms – specific dataset & its expected evolution
  • experiment
  • approach• carry out same experiment: simulation of how multi-attribute collaborative filtering algorithms perform – real data from Organic.Edunet users – simulated/synthetic data from expected future scenario (when more ratings will be provided) – base algorithms from 2007 vs. additional/alternative algorithms
  • real data from Organic.Edunet• 477 ratings – 99 users (only 0.02% of registered ones) – 345 items (only 0.03% of indexed resources)
  • simulated/synthetic data• used Monte Carlo simulator to generate more ratings of the same users – 1,280 ratings
  • 2007 base algorithms• Manouselis & Costopoulou (2006;2007)• classic neighborhood-based collaborative filtering – extended for multi-criteria ratings – prediction per criterion (PG) – many parameters open for tweaking/experimentation • different algorithm variations
  • additional/alternative algorithms• Adomavicius & Kwon (2007)• similar approach, neighborhood-based collaborative filtering extended for multi- criteria ratings – weights prediction based with average (AS) or minimum (WS) similarities per criterion – same parameters open for tweaking/experimentation • different algorithm variations
  • overall experiment setting• 18 variations of each examined algorithm (PG, AW, WS) – plus some base non-personalised ones• various values for parameters defining the neighborhood size-> over 1,080 algorithmic variations executed and compared over each dataset
  • results: real dataset
  • results: synthetic dataset
  • best over both Algorithm Similarity Normalization method AVG Coverage AVG MAE MNN variations PG Cosine Deviation-from-Mean 61.33% 0.8855 PG Euclidian Simple Mean 61.33% 0.8626 CWT variations PG Cosine Deviation-from-Mean 57.91% 0.8908 PG Cosine Simple Mean 57.91% 0.86732007:
  • implementation implications• based on existing dataset and the foreseen future scenario – keep same algorithm (PG) for recommendation service – adapt selection of options and their parameterization – “actual” performance (vs. 2007) is probably worse
  • conclusions
  • lessons learnt• after 2 years of service operation – tried to repeat an offline experimental simulation – candidate multi-criteria recommendation algorithms – data from real usage vs. synthetic data• feeling better about algorithm choice – some insight into expected performance – not real impact into the actual service
  • to explore• would be interesting to experiment with more future scenarios – make various estimations/projections about dataset size and sparseness – execute algorithms over synthetic datasets simulating these projections• would be interesting to make a service that is really used – get more ratings, on more items – provide visible recommendations – measure impact to search/discovery behaviour
  • up & beyond
  • experiments beyond a single dataset • combining data from various sources to boost the way recommenders work • design algorithms that could provide cross- border recommendations • provide many parallel/cascading/competing options for recommendation algorithms • not really care about data size & storage
  • a social data infrastructure for learning …portals… Meta Social Meta Social Meta Social Social data data Data Data data Data Data API API API API Federated Recommendation Aggregation of metadata, social and usage data Services Resolution services Social Metadata Data per URIwww.opendiscoveryspace.eu Anonymised
  • challenges• define common metadata schema(s)• aggregate (e.g. harvest/crawl) social data• transform each social data schema• URI resolution• scalability• anonymised approach• …
  • thank you! nikosm@ieee.org http://wiki.agroknow.grhttp://www.organic-edunet.eu