The current oer search dilemma


Published on

This presentation discusses the current dilemma with respect to Open Educational Resources (OER) search. It introduces existing OER search methodologies and highlights their weaknesses. The Desirability framework for parametrically measuring the usefulness of an OER is also discussed. The desirability framework uses the D-index to measure the openness, accessibility and relevance of an OER. OERScout, a text mining based faceted search engine is introduced for improved OER search. It uses autonomously identified domain specific keywords, the D-index and faceted search to allow focused OER search.

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The current oer search dilemma

  1. 1. The Current OER Search DilemmaGuest Lecture24th and 27th June 2013Sukhothai Thammathirat Open University, ThailandIshan AbeywardenaMSc, MSc (Brunel), BSc (Bangalore), MIEEE, MBCS, MIET, MTASenior Lecturer, School of Science and TechnologyWawasan Open UniversityPenang, Malaysia
  2. 2. AcknowledgementI express my gratitude to:– Assoc. Prof Dr. Chailerd Pichitpornchai forextending me this kind invitation;– Ms. Rattip Phukkeson and all other colleagues atSTOU for having me here.
  3. 3. Lecture Outline• Review of the current OER search dilemma.• Parametrically measuring the Desirability ofOER using D-index.• OERScout Technology Framework: A NovelApproach to OER Search.
  4. 4. The Current OER Search Dilemma
  5. 5. Main ReferenceAbeywardena, I.S., & Chan, C.S. (2013).Review of the Current OER SearchDilemma. Proceedings of the 57th WorldAssembly of International Council onEducation for Teaching (ICET 2013),Nonthaburi, Thailand.Available at:
  6. 6. Talking Points• Current State of OER• The Dilemma• Some Existing Solutions– Pearson’s Project Blue Sky– GLOBE– LRMI– OERScout
  7. 7. Current State of OER
  8. 8. The DilemmaSo…how do I find the material I need for myteaching
  9. 9. Literature• ...The problem is in finding the resources, and more correctly finding the “right”resources. Using a regular search engine like Google to find content is not always aviable option as it will generate too many answers. There is, hence, a need to easilyfind relevant content...” (Hatakka, 2009)• “searching this way (using existing search engines such as Google) might be a longand painful process as most of the results are not usable for educational purposes”(Pirkkalainen & Pawlowski, 2010)• No single search engine is still able to locate resources from all the OERrepositories (West & Victor, 2011)• One of the major barriers to the use and re-use of OER is the difficulty of findingquality OER matching a specific context (Dichev & Dicheva, 2012)• “…the problem with open content is not the lack of available resources on theInternet but the inability to locate suitable resources for academic use” (Unwin,2005).
  10. 10. Some Existing Solutions• Google: “…searching this way might be a long andpainful process as most of the results are not usable foreducational purposes” (Pirkkalainen & Pawlowski, 2010).• Federated Search: BRENHET2; OpeScout; GlobalLearning Object Brokered Exchange (GLOBE); andPearson’s Project Blue Sky.• Semantic Search: OER-CC ontology; the “Assistant”prototype; the “Folksemantic” project; and “Agrotags”.Pirkkalainen, H., Pawlowski, J. (2010). Open Educational Resources and Social Software in Global E-Learning Settings. In Yliluoma, P. (Ed.)Sosiaalinen Verkko-oppiminen. IMDL, Naantali, 23–40.
  11. 11. The DilemmaYeah…but which one do I choose
  12. 12. Pearson’s Project Blue Sky
  13. 13. GLOBE
  14. 14. LRMI
  15. 15. OERScout
  16. 16. Questions
  17. 17. Desirability of OER using D-index
  18. 18. Main ReferenceAbeywardena, I.S., Raviraja, R., & Tham, C.Y.(2012). Conceptual Framework forParametrically Measuring the Desirability ofOpen Educational Resources using D-index.International Review of Research in Open andDistance Learning, 13(2), 104-121 (ISI-indexed publication).Available at:
  19. 19. Talking Points• Usefulness of OER• The Desirability of OER• Measuring the Desirability using D-index
  20. 20. What is the Usefulness of an OER?• The usefulness of an OER for aparticular teaching or learning needcan only be accurately assessed byreading through the content.• This aspect of use and re-use ofOER will remain a human functionregardless of the improvements intechnology.
  21. 21. Measuring the Usefulness OERHowever, there are other aspects of a resource which arefundamental to the usefulness of that particular resource and can beparametrically identified by a software based mechanism:– Whether a resource is relevant to a user’s needs;– Whether the resource is open enough for using,reusing, remixing and redistributing;– Whether the resource is accessible with respectto technology.
  22. 22. What is Desirability?• If the user cannot easily use,reuse and remix a resourcewith available technology, theresource becomes less usefulto the user.• Less useful resources are lessdesirable for teaching andlearning needs….
  23. 23. Measuring the Desirability of OERWithin the requirement of being able to use and reuse aparticular OER, the three parameters of Desirability can bedefined as:– level of openness: the permission to use and reusethe resource;– level of access: the technical keys required to unlockthe resource;– relevance: the level of match between the resourceand the needs of the user.
  24. 24. i.e. DesirabilityD-index = (level of access x level of openness x relevance) / 256
  25. 25. OpennessPermission ValueReuse 1Redistribute 2Revise 3Remix 4The level of openness based on the four R’s of opennessMapping the CC licenses to the 4 R’sPermission Creative Commons (CC) licence ValueReuse None 1Redistribute Attribution-NonCommercial-NoDerivs (CC BY-NC-ND)Attribution-NoDerivs (CC BY-ND)2Revise Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)Attribution-ShareAlike (CC BY-SA)3Remix Attribution-NonCommercial (CC BY-NC)Attribution (CC BY)4
  26. 26. AccessAccess to editingtoolsLevel of expertise required torevise or remixMeaningfullyeditableSource-file access ValueLOW HIGH NO NO 01LOW HIGH NO YES 02LOW HIGH YES NO 03LOW HIGH YES YES 04LOW LOW NO NO 05LOW LOW NO YES 06LOW LOW YES NO 07LOW LOW YES YES 08HIGH HIGH NO NO 09HIGH HIGH NO YES 10HIGH HIGH YES NO 11HIGH HIGH YES YES 12HIGH LOW NO NO 13HIGH LOW NO YES 14HIGH LOW YES NO 15HIGH LOW YES YES 16ThelevelofaccessbasedontheALMSanalysis
  27. 27. RelevanceSearch rank ValueBelow the top 30 ranks of the search results 1Within the top 21-30 ranks of the search results 2Within the top 11-20 ranks of the search results 3Within the top 10 ranks of the search results 4The level of relevance based on search rank (Vaughan, 2004)• Users will only consider the top ten ranked results for a particular searchas the most relevant;• Users will ignore the results below the top 30 ranks.Vaughan, L. (2004). New measurements for search engine evaluation proposed and tested. Information Processing and Management 40, 677–691.
  28. 28. CalculationOriginal search results (OER Commons example)Calculation of D-indexAfter application of D-index
  29. 29. Sample SearchSearch Rank Title CC Lisence File Type1 18.01 Single Variable Calculus CC BY-NC-SA PDF2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text3 18.01 Single Variable Calculus CC BY-NC-SA PDF4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text5 18.02 Multivariable Calculus CC BY-NC-SA PDF6 Single Variable Calculus CC BY-NC-SA PDF7 Calculus Online Textbook CC BY-NC-SA PDF8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text9 18.075 Advanced Calculus for Engineers CC BY-NC-SA PDF10 MATH 140 - Calculus I, Summer 2007 CC BY-NC-SA ProtectedTop 10 search results returned by MERLOT for the keyword “calculus”
  30. 30. Original Search Results• The original top ten search results only containresources which are released under the CC BY-NC-SA license.• 6/10 resources returned are in PDF format whichmake them difficult to reuse and remix.• Resource ranked as number ten is a protectedresource which requires a specific username andpassword to access.
  31. 31. Application of D-indexRank AfterApplying D-indexOriginalSearchRank Title CC Lisence File Type D-index1 2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.752 4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text 0.753 8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.754 14 Multivariable Calculus CC BY HTML/Text 0.755 19MATH 10250 - Elements of CalculusI, Fall 2008 CC BY-NC-SA HTML/Text 0.566 20 18.022 Calculus CC BY-NC-SA PDF 0.567 22 Single-Variable Calculus I CC BY HTML/Text 0.508 25 Single-Variable Calculus II CC BY HTML/Text 0.509 15 Highlights of Calculus CC BY-NC-SA Video 0.4210 21 Calculus I CC BY HTML/Text 0.38Top 10 results when D-index is applied to the results returned by MERLOT
  32. 32. Results After Applying D-index• 8/10 resources are in HTML/Text formatswhich are the most accessible in terms ofreuse.• 4/10 resources are available under the CC BYlicence which make them the most openresources in the list.
  33. 33. Benefits of the D-indexThe application of the D-indexwould greatly improve theeffectiveness of the search withrespect to locating the mostsuitable resources for use andreuse.
  34. 34. Let’s discuss more over tea?
  35. 35. OERScout Technology Framework
  36. 36. Main ReferenceAbeywardena, I.S., Chan, C.S., & Tham, C.Y.(2013). OERScout Technology Framework: ANovel Approach to Open EducationalResources Search. International Review ofResearch in Open and Distance Learning, Inpress (ISI-indexed publication).Available at:Currently in press. Will be available at
  37. 37. Talking Points• Why not Google?• Why not native search engines ofrepositories?• What is OERScout?• Why use OERScout?
  38. 38. Google“AdvancedSearch”resultsforOERonChemistry(24thMay2012)
  39. 39. Why Not Repositories?Identify which material to look for(e.g. integration, C++ programming)Identify the search queries(e.g. “undergraduate mathematics”)Locate repository(word of mouth, some linksomewhere, go to the more popularrepositories)Run multiple queries to find resourcesRead each resource to identify the usefulness(openness, access, relevance)Identify useful resourcesRepeat steps 3-6 on multiple repositories(hundreds to thousands…..)
  40. 40. The Declarationi. Facilitate finding, retrieving and sharing of OER.Encourage the development of user-friendlytools to locate and retrieve OER that arespecific and relevant to particular needs.(UNESCO Paris OER Declaration, 2012)UNESCO. (2012). Paris OER Declaration, Retrieved September18, 2012 from
  41. 41. Specific & Relevant• Specific: the suitability of an OER for a particularteaching need.For example, an OER on physics from the finalyear syllabus of a physics degree would not besuitable for a high school physics class.• Relevant: the match between the content of theOER and the content needed for a particularteaching need.For example, physical chemistry is not relevant fora teaching need in organic chemistry.
  42. 42. OERScoutA Text Mining Algorithm which–READS text based OER;–LEARNS which academic domain(s) andsub-domain(s) they belonged to;–RECCOMENDS Desirable material for aparticular academic need.
  43. 43. Keyword-Document Matrix (KDM)
  44. 44. Advantages of OERScout• Using the KDM, the system generates ranked listsof relevant OER from heterogeneousrepositories to suit a given search query;• Incorporates the Desirability framework torecommend useful resources for academicpurposes;• Uses a faceted search approach to allow users toquickly zero-in on the resources they are after.
  45. 45. Benefits to Content Creators• No need to manually define metadata;• No need to publicise the availability of material;• No need to build custom search mechanisms forrepositories;• More visibility and reach of material to a wideraudience.
  46. 46. Benefits to Users• Provides a central location for findingresources scattered across the globe hidden inhigh volume repositories;• Locate only the most relevant resources.
  47. 47. ConclusionThe ultimate benefit of OERScoutis that both content creators andusers now only need toconcentrate on the actualcontent and not the searchingand location of relevant OER.
  48. 48. Questions
  49. 49. About…Ishan AbeywardenaSenior Lecturer, School of Science and Technology, Wawasan Open University, Penang, Malaysia• MSc in Wireless Enterprise Business Systems, Brunel University, UK.• MSc in Engineering Management, Brunel University, UK.• BSc in Computer Science, Bangalore University, India.• PhD Candidate in Computer Science, University Malaya, Malaysia. Areas of specialisation: text mining,metadata, faceted searchProfessional Member of– Institute of Electrical and Electronic Engineers (MIEEE)– British Computer Society (MBCS)– Institution of Engineering and Technology (MIET)– Microsoft Technology Associate (MTA)Official Profile: Profile: Profile: Blog: http://www.ishantalks.comE-mail:
  50. 50. References• Dichev, C., & Dicheva, D. (2012). Open Educational Resources in Computer Science Teaching. SIGCSE’11, February 29–March3, 2012, Raleigh, NC, USA.• Caswell, T., Henson, S., Jenson, M., & Wiley, D. (2008). Open Educational Resources: Enabling universal education.International Review of Research in Open and Distance Learning 9(1), 1-11.• Hatakka, M. (2009). Build It and They Will Come? – Inhibiting Factors for Reuse of Open Content in Developing Countries,EJISDC 37(5), 1-16.• Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The four R‘s of openness and ALMS Analysis: Frameworks for openeducational resources. Open Learning: The Journal of Open and Distance Learning, 25(1), 37-44.• Pirkkalainen, H., Pawlowski, J. (2010). Open Educational Resources and Social Software in Global E-Learning Settings. InYliluoma, P. (Ed.) Sosiaalinen Verkko-oppiminen. IMDL, Naantali, 23–40.• Unwin, T. (2005). Towards a Framework for the Use of ICT in Teacher Training in Africa. Open Learning 20, 113-130.• West, P., & Victor, L. (2011). Background and action paper on OER. Report prepared for The William and Flora HewlettFoundation.