Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
RISE User EvaluationLiz Malletthttp://www.open.ac.uk/blogs/rise
Recommendations Improve theSearch Experience? “That recommender systems   can enhance the studentexperience in new generat...
Evaluation                    Online Survey         Face to Face interviews         Review of web analyticshttp://www.flic...
Survey results 1Related to records you have viewed                 Very useful           Not sure                    45%  ...
Survey results 2People on your course viewed                                 Very useful         Not                      ...
Survey results 3Search terms                            Very useful                               47%                     ...
Survey results 4          How relevant where the recommendations?                                          Very relevant  ...
Survey results
Focus Groups     Undergraduates                   PostgraduatesLike ratings and reviews from   Citation as a recommendatio...
Face to face interviews First impressions of recommendations (course-related) Asked to enter a search term. Results and re...
Should we have a recommender    system?  “I think it would be a very good useful feature. It would be definitely very  ver...
Should we have a recommender  system?“Im afraid my first reaction is to be a bit sceptical - it presumably doesnt tellyou ...
Why they prefer course-related    recommendations“I can’t be bothered with knowing what everybody else is interested in. I...
Suggestions for improvement“Maybe include a date. It would be interesting to know when a resourcewas last looked at” Postg...
Recommendations usage             Relationship                 24%        Search                             40%          ...
Recommendations usage  2000  1800  1600  1400  1200                            Relationship  1000                      Cou...
Findings and lessons learnt  • Users like recommendations „in    principle‟  • Recommendations provenance  • Interest in t...
Blog: www.open.ac.uk/blogs/RISE            Code: http://code.google.com/p/rise-            project/source/browse/trunk/ris...
RISE User EvaluationLiz Malletthttp://www.open.ac.uk/blogs/rise
Upcoming SlideShare
Loading in …5
×

Rise presentation-users-2012-01

5,431 views

Published on

Presentation on user reactions to RISE given at RISE celebration event on 24 January 2012 by Elizabeth Mallett

  • Be the first to comment

  • Be the first to like this

Rise presentation-users-2012-01

  1. 1. RISE User EvaluationLiz Malletthttp://www.open.ac.uk/blogs/rise
  2. 2. Recommendations Improve theSearch Experience? “That recommender systems can enhance the studentexperience in new generation e- resource discovery services”
  3. 3. Evaluation Online Survey Face to Face interviews Review of web analyticshttp://www.flickr.com/photos/42505898@N00/305205950/sizes/m/in/photostream/
  4. 4. Survey results 1Related to records you have viewed Very useful Not sure 45% 11% Not useful 22% Quite useful 22% Slightly useful 0%
  5. 5. Survey results 2People on your course viewed Very useful Not 31% sure 0% Quite useful 15% Slightly useful 15% Not useful 39%
  6. 6. Survey results 3Search terms Very useful 47% Not sure 0% Quite useful Not useful 20% 33% Slightly useful 0%
  7. 7. Survey results 4 How relevant where the recommendations? Very relevant Not 17% used Quite relevant 4% 31% Not sure 0% Not relevant 35% Slightly relevant 13%
  8. 8. Survey results
  9. 9. Focus Groups Undergraduates PostgraduatesLike ratings and reviews from Citation as a recommendation other students „other people‟s experiences Wary of provenance valuable‟ Feed to module website Which module studied? Want synonyms How high a mark? Trust repository
  10. 10. Face to face interviews First impressions of recommendations (course-related) Asked to enter a search term. Results and recommendations explored. Asked about relevance Asked about preference for type of recommendation
  11. 11. Should we have a recommender system? “I think it would be a very good useful feature. It would be definitely very very useful” postgraduate Maths student“So it would be interesting to see what other people are looking at. Yes, I woulddefinitely use that because my limited knowledge of the library might mean thatother people were using slightly different ways of searching and getting differentresults.” undergraduate English Literature student I have just had a go, it was good with suggested papers that I had already found (which shows potential in my view) through Google.
  12. 12. Should we have a recommender system?“Im afraid my first reaction is to be a bit sceptical - it presumably doesnt tellyou if fellow students found the information/article useful or relevant to whatthey were looking for. I would hate to waste time following unproductivelinks laid down by others who might be failing students or think that any"lazy" students might develop poor practice by relying on what others hadlooked at. It sounds like a good idea but I think caution needs to beexercised. ”
  13. 13. Why they prefer course-related recommendations“I can’t be bothered with knowing what everybody else is interested in. Itake a really operational view you know, I’m on here, I want to get thereferences for this particular piece of work, and those are the people thatare most likely to be doing a similar thing that I can use.” H800 student“I suppose if I wasn’t so sure on an assignment it would perhaps be quiteuseful to see what other people were looking at to know if I was thinkingalong the right lines.” - Undergrad literature student
  14. 14. Suggestions for improvement“Maybe include a date. It would be interesting to know when a resourcewas last looked at” Postgraduate political philosophy student“If somebody used similar search but three years ago, is that going to carrythe same weight?” Postgraduate maths student Include course drop-down choice. “I would be looking at that and saying “which of my courses does it refer to?”
  15. 15. Recommendations usage Relationship 24% Search 40% Course 36%
  16. 16. Recommendations usage 2000 1800 1600 1400 1200 Relationship 1000 Course Search 800 600 400 200 0 1 2 3 4
  17. 17. Findings and lessons learnt • Users like recommendations „in principle‟ • Recommendations provenance • Interest in the search tools
  18. 18. Blog: www.open.ac.uk/blogs/RISE Code: http://code.google.com/p/rise- project/source/browse/trunk/rise/ Questions?http://www.flickr.com/photos/rmgimages/4660272978/in/photostream/
  19. 19. RISE User EvaluationLiz Malletthttp://www.open.ac.uk/blogs/rise

×