Your SlideShare is downloading. ×
Relevance Ranking of Learning Objects
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Relevance Ranking of Learning Objects

1,116

Published on

Presentation at EC-TEL 2007 (Crete). The need for relevance rank is presented and basic metrics are presented and evaluated.

Presentation at EC-TEL 2007 (Crete). The need for relevance rank is presented and basic metrics are presented and evaluated.

Published in: Economy & Finance, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,116
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Relevance Ranking Metrics for Learning Objects Xavier Ochoa, ESPOL, Ecuador Erik Duval, KULeuven, Belgium
  • 2. Agenda
    • Why Relevance Ranking?
    • What is Relevance?
    • Relevance Ranking Metrics
    • Do they work?
    • What is next?
  • 3. Economy of Abundance
  • 4. Economy of Abundance Put your LMS here!
  • 5. Why Relevance Ranking?
    • Abundance
    • =
    • Difficult to find most relevant
  • 6. Why Relevance Ranking?
    • Abundance
    • =
    • Difficult to find most relevant
    Solution: Do it the Google way
  • 7. Why Relevance Ranking?
    • Abundance
    • =
    • Difficult to find most relevant
    Relevance Ranking MEANINGFUL & SCALABLE
  • 8. How we do it now?
    • Manual rating:
      • Meaningful but not Scalable
    • Text based Ranking Algorithms:
      • Scalable but not Meaningful
  • 9. What is Relevance?
  • 10. Relevance Ranking Metrics Relevance Ranking Metric B Metric A Metric C Context
  • 11. Relevance Ranking Metrics Relevance Ranking Meaningful Metric B Metric A Metric C Context
  • 12. Relevance Ranking Metrics Relevance Ranking Scalable Metric B Metric A Metric C Context
  • 13. Topical Relevance
  • 14. Personal Relevance
  • 15. Situational Relevance
  • 16. Do they work?
    • Exploratory Study
      • 10 users (8 Prof. and 2 R.A.)
      • MIT OCW learning objects (34,640 LO)
      • Search and Select Objects to create 10 lessons (10 different topics on CS)
      • Web application was used
  • 17.  
  • 18.  
  • 19.  
  • 20. Ranking the Rankings
    • Baseline Rank: TF-IDF similarity algorithm
    • Re-rank according to Basic Metrics
    • All rankings compared with manual relevance ranking
    • Kendall Tau was used to measure similaritiy between lists
  • 21. Results
  • 22. Basic Topical Relevance Metric Trees
  • 23. Basic Personal Relevance Metric Net.
  • 24. Basic Situational Relevance Metric XML
  • 25. Linear Combination
  • 26. Linear Combination
  • 27. Results
    • Even basic metrics improve the ranking
    • Linear Combination should be learnt
    • Limited (and Synthetic) Study!
  • 28. What is next?
    • Implementation in ARIADNE NEXT
    • Capture user behavior and learn from it for 3 months
    • Natural (Real) evaluation of the metrics
    • Develop BETTER metrics
  • 29. Conclusions
    • Meaningful and Scalable metrics are possible
    • Cheap to implement
    • Provide significant improvement
    • While not optimal, they could be use as a base-line for further research
  • 30. Thank You! Dank U! Gracias!
    • Questions, Comments, Critics… are all welcome!!
    Xavier Ochoa [email_address] http://www.cti.espol.edu.ec/xavier Erik Duval Erik.Duval@cs.kuleuven.be http://www.cs.kuleuven.ac.be/~erikd

×