• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Information Retrieval and User-centric Recommender System Evaluation
 

Information Retrieval and User-centric Recommender System Evaluation

on

  • 479 views

Poster describing the ERCIM-funded project on IR- and user-centric recommender system evaluation currently being undertaken in the Information Access group at CWI.

Poster describing the ERCIM-funded project on IR- and user-centric recommender system evaluation currently being undertaken in the Information Access group at CWI.
Presented at UMAP 2013.

Statistics

Views

Total Views
479
Views on SlideShare
453
Embed Views
26

Actions

Likes
2
Downloads
5
Comments
0

1 Embed 26

https://twitter.com 26

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Information Retrieval and User-centric Recommender System Evaluation Information Retrieval and User-centric Recommender System Evaluation Document Transcript

    • ERCIM  research  project  at  CWI    Focused  on  novel  aspects  of  recommender  systems  evalua3on  from  an  informa3on  retrieval  and  a  user-­‐centric  perspec3ve.    Dura3on  and  personnel  §  January  2013  –  March  2014  §  24  person  months  §  2  postdocs    Centrum  Wiskunde  &  Informa:ca                                                      www.cwi.nl  Alan  Said,  Alejandro  Bellogín,  Arjen  De  Vries,  Benjamin  Kille  {alan.said,  alejandro.bellogin,  arjen.de.vries}@cwi.nl,  kille@dai-­‐lab.de  Informa3on  Retrieval  and  User-­‐Centric  Recommender  System  Evalua3on    UMAP  2013  –  Rome,  Italy  Acknowledgement    This  work  is  carried  out  during  the  tenure  of  an  ERCIM  “Alain  Bensoussan”  Fellowship  Programme.  The  research  leading  to  these  results  has  received  funding  from  the  European  Union  Seventh  Framework  Programme  (FP7/2007-­‐2013)  under  grant  agreement  no.246016.    User-­‐centric  Evalua:on    A  personalized  offline  evalua:on  protocol  mimicking  real-­‐life  deployed  recommender  systems.  •  Each  algorithm  is  trained  once  per  user  based  on  a    personalized  training  and  test  split.  •  Candidate  test  items  are  selected  based  on  a  personal  ra:ng    value  threshold  (e.g.  mean).    Diversity-­‐oriented  evalua:on  methods  focusing  on:  •  non-­‐accuracy-­‐based  evalua3on  metrics  •  item  feature-­‐centric  evalua3on  •  novelty  IR-­‐based  Evalua:on    A  coherence-­‐based  func:on  that  is  able  to  predict  the  user  performance  •  Correlated  with  the  “magic  barrier”  •  Improves  the  total  performance  of  the  system  when  used  to  group  users  into  difficult  and  easy  ones    Explore  correla:ons  between  metrics  typically  used  in  offline  experiments  (precision)  and  in  real  applica3ons  (e.g.,  CTR).    Perform  an  analysis  of  how  evalua3on  in  recommenda3on  should  be  performed  to  obtain  interpretable  and  comparable  results.    Related  Events    Workshop  on  Reproducibility  and  Replica1on  in  Recommender  Systems  Evalua1on.  In  conjunc3on  with  RecSys  2013      Workshop  on  Benchmarking  Adap1ve  Retrieval  and  Recommender  Systems.  In  conjunc3on  with  SIGIR  2013      ACM  TIST  Special  Issue  on  Recommender  System  Benchmarking  Further  Reading    Ar1st  popularity:  do  web  and  social  music  services  agree?    A.  Bellogín  et  al.  In  ICWSM  2013        User-­‐Centric  Evalua1on  of  a  K-­‐Furthest  Neighbor  Collabora1ve  Filtering    Recommender  Algorithm    A.  Said  et  al.  in  CSCW  2013    Poster  Abstract  Informa1on  Retrieval  and  User-­‐Centric  Recommender    System  Evalua1on  A.  Said,  A.  Bellogín  et  al.  in  UMAP  2013