JCDL 2011: Semantically Augmented Annotations in Digitized Map Collections

1,304 views
1,242 views

Published on

Historic maps are valuable scholarly resources that record information often retained by no other written source. With the YUMA Map Annotation Tool we want to facilitate collaborative annotation for scholars studying historic maps, and allow for semantic augmentation of annotations with structured, contextually relevant information retrieved from Linked Open Data sources. We believe that the integration of Web resource linkage into the scholarly annotation process is not only relevant for collaborative research, but can also be exploited to improve search and retrieval. In this paper, we introduce the COMPASS Experiment, an ongoing crowdsourcing eff ort in which we are collecting data that can serve as a basis for evaluating our assumption. We discuss the scope and setup of the experiment framework and
report on lessons learned from the data collected so far.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,304
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 1/1
  • 1/1
  • 1/1
  • JCDL 2011: Semantically Augmented Annotations in Digitized Map Collections

    1. 1. Semantically Augmented Annotations in Digitized Map Collections Rainer Simon Austrian Institute of Technology Bernhard Haslhofer Cornell University Werner Robitza, Elaheh Momeni University of Vienna ACM/IEEE Joint Conference on Digital Libraries | June 13-17, 2011, Ottawa, Canada
    2. 2. Part of Europeana Connect EU Project Core Components for the European Digital Library www.europeanaconnect.eu EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    3. 3. Europeana Connect | Annotation & Scholarship <ul><li>“ Annotation” is a fundamental scholarly practice common across disciplines </li></ul><ul><li>Scholars share and exchange knowledge through annotations </li></ul><ul><li>Scholars manage , interpret and coordinate sources through annotations </li></ul>EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus by Romana Klee CC BY-SA 2.0
    4. 4. History Art Technology Geography
    5. 5. YUMA Map | Y UMA U niversal M edia A nnotator <ul><li>Viewing of high-resolution map scans as zoomable images </li></ul><ul><li>Free-text annotation </li></ul><ul><li>Reply functionality & RSS feeds (on items, annotations, users) to support collaboration </li></ul><ul><li>Map geo-referencing </li></ul><ul><ul><li>Data overlay – modern-day country borders, coastlines, etc. </li></ul></ul><ul><ul><li>Annotation export to KML – viewing in Google Earth </li></ul></ul><ul><ul><li>Place search </li></ul></ul><ul><li>Semantic tagging – linking free text with contextual Linked Data resources! </li></ul><ul><ul><li>Named entity recognition & link discovery via DBpedia Spotlight 1 </li></ul></ul><ul><ul><li>Geographical features in the annotated area via Geonames </li></ul></ul><ul><ul><li>Semi-automatic approach: human-verified through the Context Tag Cloud </li></ul></ul>1 http://dbpedia.org/spotlight , http://wiki.dbpedia.org/spotlight/knownuses EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    6. 6. YUMA Map | Annotation & Semantic Tagging (Screencast) http://vimeo.com/21798530 EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    7. 7. The Assumption... “ A map retrieval system that indexes annotations with user-verified links to Linked Data resources can be more effective with regard to retrieval than systems that index only metadata or purely textual annotations.” EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    8. 8. The COMPASS 1 Crowdsourcing Experiment 1 C ollection O f M a P s, A nnotations & S emantic Link S <ul><li>Test data kindly provided by the Library of Congress </li></ul><ul><ul><li>130,935 user search queries collected over two years </li></ul></ul><ul><ul><li>6,306 high-resolution digitized map images </li></ul></ul><ul><ul><li>Descriptive metadata </li></ul></ul><ul><ul><li>Pooling to reduce amount of queries for Step 1 </li></ul></ul><ul><li>Step 1: Users are shown a map/query pair </li></ul><ul><ul><li>Relevant or Not Relevant ? </li></ul></ul><ul><li>Step 2: Users are invited to annotate maps with YUMA Map </li></ul><ul><li>Step 3: precision and recall analysis for different retrieval approaches </li></ul><ul><ul><li>Metadata only vs. Annotated (plain text) vs. Annotated (Linked Data context) </li></ul></ul>EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    9. 9. COMPASS | Screenshot EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    10. 10. Work in Progress... | Preliminary Results <ul><li>90+ Users from at least 12 countries </li></ul><ul><li>~60% of all judgments produced by the top 10 contributors </li></ul><ul><li>~25% “Lurkers” – submitted no judgement at all </li></ul><ul><li>1/3 of active participants declared themselves experts: </li></ul><ul><ul><li>General library domain (~15%) </li></ul></ul><ul><ul><li>Map library domain (~15%) </li></ul></ul><ul><ul><li>Fields related to the scope of COMPASS, e.g. GIS, cartography (~4%) </li></ul></ul><ul><li>Level of involvement higher among experts : </li></ul><ul><ul><li>6 experts among top ten contributors </li></ul></ul><ul><ul><li>~59% of all submitted judgments </li></ul></ul>EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    11. 11. Work in Progress... | Next Steps <ul><li>Finish COMPASS Step 1 </li></ul><ul><ul><li>45% of required amount for a 400-query “ground truth” reached </li></ul></ul><ul><li>Start COMPASS Step 2 – Integration COMPASS/YUMA </li></ul><ul><ul><li>Development of a “COMPASS Portal” currently in progress </li></ul></ul><ul><ul><li>Lower the entry barrier! Simplify tasks, provide “guided tour” mode </li></ul></ul><ul><ul><li>Raise motivation: social interactions, rewards (game mechanics), rating & reputation scores (gaining authority in the community) </li></ul></ul><ul><ul><li>Enable users to “do something” with the content: e.g. embedding of annotated maps </li></ul></ul><ul><li>In general: encourage feedback & investigate users’ motivations </li></ul><ul><li>Dissemination! </li></ul>EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus
    12. 12. Thank you for your attention Questions! Map Sources Library of Congress – Discovery and Exploration Maps Wikimedia Commons http://dme.ait.ac.at/annotation (take the tour) http://github.com/yuma-annotation (get the code) http://compass.cs.univie.ac.at (help us evaluate!) EuropeanaConnect is coordinated by the Austrian National Library Co-funded by the European Community Programme eContent plus

    ×