This document discusses metadata quality in learning repositories. It summarizes a study analyzing metadata completeness and quality in the Organic.Edunet learning repository before and after implementing a metadata quality assurance mechanism. The mechanism involved experts reviewing metadata records and providing feedback. Analysis found completeness improved for many mandatory, recommended, and optional metadata elements after implementation, especially for educational elements. However, further work is still needed to improve completeness of optional elements and additional quality metrics.

1. Metadata quality
in learning repositories:
Issues & considerations
Nikos Palavitsinis, Nikos Manouselis,
Salvador-Sanchez Alonso
ED-MEDIA 2011, 27/7-1/7, Lisbon, Portugal

2. Key Concepts
• Learning Repositories
• Resources
• Metadata

3. Quantity is NOT the problem!
• MERLOT, ARIADNE, LRE for schools, MACE, etc
• Millions of resources on various
topics/areas/fields
Image taken from: http://farm1.static.flickr.com/223/475156707_d56e38f251.jpg

4. Quality?
• Publishers of content are under pressure to
ensure that in the plethora of information that
is available in today’s web, resources are
highly discoverable and highly credible
Walker, 2010
Quality of metadata
Quality of content

5. Truth or…
“Isn't it true, that only librarians like to
search? Everyone else likes to find”
Roy Tennant
Images taken from: http://gigglebot.net/wp-content/uploads/wpsc/product_images/librarian.png
& http://www.ala.org/img/alonline/computer%20guy.jpg

6. …dare?

7. Context of the study

8. about Organic.Edunet
• Completed eContentPlus project (10/2007 –
10/2010) funded with 2.5M €
• Main objective: Make learning content on
Organic Agriculture and Agro-ecology
available online, for a range of stakeholders.
• And: Develop educational scenarios to be used
within schools & universities

10. Stats
• 1,5 year of operation
• 53.859 visits
• 208.963 page views
• 41.804 unique visitors
• 166 countries
• 2.041 registered users
• 10.967 resources

11. Multilingualism
• Multilingual interface (17 languages)
– Text-based search
– Browsing
– Tag-based search
– Semantic search
• Multilingual resources (10 languages)
– Images, videos, articles, scenarios, etc
• Multilingual metadata (17 languages)
– IEEE LOM Application Profile

12. How did we get there?
Started on June of 2008…

13. Process (1/2)
• Developed an Application Profile of IEEE LOM
to be used to describe resources on Organic
Agriculture & Agroecology

14. Organic.Edunet
IEEE LOM AP
Definition of own requirements
Selection of LOM elements
Semantics Refinement
Multiplicity constraints and values
Relationships and dependencies
Required extensions
Application Profile Binding
Evaluation of AP
Evaluation phase
Results’ analysis
AP modifications

15. Process (2/2)
• Developed a Repository Management Tool
where resources would be described with IEEE
LOM metadata (Confolio – http://oe.confolio.org)

16. May 2009
• 6.600 resources uploaded in Confolio
– 1.100 fully annotated by partners
– 5.500 harvested and manually enriched records
• Time to start playing with the data…

17. First Analysis
• Measured metadata completeness for all IEEE
LOM AP elements
• Usage of metadata elements was not as high
as desired, but was as low as anticipated!

18. Results

19. Improving Metadata Quality
Through a Metadata Quality
Assurance Mechanism

20. Step 1
• Review of metadata records by experts
(9/2009)
– Direct feedback through e-mail &
– Good & Bad metadata practices

21. Step 2
• Review of metadata records by subject-matter
experts (12/2009)
– Direct feedback through e-mail
– General statistics

22. January 2010

23. Time to measure again…

24. Second Analysis
• Improved completeness for…
– Mandatory elements: -0.3% to 82,4%
• Rights.Copyrights and Other Restrictions (82.4%)
• Rights.Cost (64%)
– Recommended elements: 47,4% to 82,9%
• Rights.Description (82.9%)
• General.Keyword (78.1%)
– Optional elements: 13,2% to 79,9%
• General.Coverage (79.9%)

25. What about Educational elements?
• Educational…
– Intended End User Role: +66.1% (78.9%)
– Typical Age Range: +57.7% (61.5%)
– Context: +47.4% (57.6%)
– Language: +50.9% (51.2%)
– Difficulty: +36.1% (36.2%)
– Semantic Density: +35.9% (36.1%)
– Interactivity Level: +35.8% (36.1%)
– Interactivity Type: +35.6% (35.9%)
– Description: +13.2% (14.7%)

26. Overview
-20,00%
0,00%
20,00%
40,00%
60,00%
80,00%
100,00%
120,00%
Metadata elements
Completeness
After
Before

27. Conclusions
• Training experts on a specific field on
metadata concepts & principles, improved
resulting metadata completeness
• Manual effort is really important
• Further effort for optional elements needed

28. Limitations
Improvement coming from familiarizing with the
annotation process
VS
Improvement coming from the mechanisms put
into place
Is it possible to measure separately?

29. Future Research
• Insert more similar mechanisms throughout
the learning resource/repository lifecycle
• Focus more on additional metadata quality
metrics on top of completeness
• Extend this framework to other types of
resources, i.e. cultural collections, etc.

30. Metadata quality
in learning repositories:
Issues & considerations
Thank you!
palavitsinis@grnet.gr
ED-MEDIA 2011, 27/7-1/7, Lisbon, Portugal