Petr Přidal
                                                petr.pridal@mzk.cz




                          GEOREFERENCER:
COLLABORATIVE ONLINE GEOREFERENCING TOOL FOR
               SCANNED MAPS

                  Moravian Library Brno: OldMapsOnline.org & TEMAP
               R&D grant of the Czech Ministry of Culture (DC08P02OUK006)
     International Cartographic Association: 6th International Workshop on Digital Approaches in Cartographic Heritage
                                          The Hague, Netherlands, 7-8 April 2011
INTRODUCTION

Large collections of old maps are scanned and published
online in digital repositories. But ...



MAPS ARE NOT ONLY IMAGES!

... once they are georeferenced, they are much more
attractive and useful: visualizations, comparing, location
context, annotations, geographical search...
PROBLEMS
• Hand made georeferencing is time consuming process
• The scanned collections are two large (thousands of
  maps to handle)
• There are more technical approaches how to
  georeference old maps, not always clear workflow
  especially for pre-1850 - you need to keep this “how”
• Disk space: georeferenced copy next to the original scan
• GIS software and knowledge necessary
• Workers need to have access to large files (JPEG/TIFF/...)
GEOREFERENCER AS A SOLUTION
• User-friendly online tool: runs in a web browser
• Simply visit a website, log-in and you can start
• Easy and enjoyable process
• Open for online visitors, organized volunteers, crowdsourcing
• No GIS software necessary, no local files to handle
• GIS-ready outputs possible - you can easily create GeoTIFF/GeoJP2 if you
  have access to the high-resolution images
• Service for EU libraries, archives & citizens
• Developed in a library - for libraries and other culture heritage institutions

GEOREFERENCER: TURN AN IMAGE INTO A MAP.
MOTIVATION FOR GEOREFERENCER

  INSTITUTIONS:                   INDIVIDUALS:
• Enrichment of metadata        • Georeference any online
  (MARC, DC, ISO 19139)           image on the Internet
• Engaging visitors             • Combine images from
• Attractive 3D visualization     different institutions
  (Google Earth)                • Create your own virtual
• GIS services (WMS, WMTS/        collection
  TMS, ...)                     • Conduct research online,
• Statistics & crediting          compare, analyze, share, ...
INSTITUTIONS: PILOT APPLICATIONS

• Moravian Library, Czech Republic (students)
• Nationaal Archief, Netherlands (organized volunteers)
• National Library of Scotland (online visitors, anybody)


          TRY THE GEOREFERENCER PILOT:
         HTTP://NLS.GEOREFERENCER.COM/

     OR THE PUBLIC SERVICE FOR INDIVIDUALS:
        HTTP://WWW.GEOREFERENCER.ORG/
http://nls.georeferencer.com/
GEOREFERENCER PILOTS
Pilot: Nationaal Archief. 13 volunteers. Great feedback!
GEOREFERENCER PILOTS
Result: 650 maps georeferenced in less then one month!
                 Metadata enrichment.
GEOREFERENCER PILOTS
Crowdsourcing: Retired volunteers. Excellent local knowledge.
             Competition & a public winner.
GEOREFERENCER PILOTS
First public pilot: the National Library of Scotland.
GEOREFERENCER PILOTS
Anybody can participate. It is easy & fun.
 http://geo.nls.uk/maps/georeferencer/
ADVANCED TECHNOLOGY
  • Unique scalable architecture. One centralized service
    for all libraries and maps. Upgrades and new
    functionality deployed automatically.
  • Runs “in the cloud” - Google AppEngine / Amazon.
  • Server side based on open-source GIS stack (Proj4,
    GDAL, MapSever). Closure Library on the client side.



        MINIMAL EFFORT TO PARTICIPATE.
IMAGES ARE USED DIRECTLY FROM EXISTING ONLINE
                REPOSITORIES!
NEXT STEPS?
GEOREFERENCER: NEW DESIGN
SPATIAL REFERENCE SYSTEMS

Coordinate systems with map projection, for example
 OSGB or S-JTSK and in general any reference system
    from EPSG database usable in Georeferencer
IMPROVED BASE MAPS
  Out-of-copyright base maps, agreements with local
   authorities for using official mapping data (WMS).
Fast and visually appealing version of OpenStreetMap as
       an alternative - probably from MapQuest.
                     OpenAerialMap.
HIGH QUALITY GEOCODING
Fast and reliable search in gazeteer database based on
        OpenStreetMap or GeoNames.org data.
 Online service, without license restrictions of Google
   GeoCoding API - usable both in GeoReferencer a
                   GeoParser services.
API FOR DATA INTERCHANGE
           AND 3RD PARTY FUNCTIONALITY
 REST API based on JSON for exchange of data between
      Georeferencer and third-party applications.
JavaScript API for easy integration into library websites.
COLLABORATIVE EDITING

 Technical solution of the problem of concurent team
 editing of map georeference by serveral people in the
same moment - locking and communication a la online
        chat in a web browser (via HTTP Comet)
CROWDSOURCING
                  Social aspects:
Motivation. Team. Competition. Guiding. Rewarding.
DUPLICATES +
           AUTOGEOREFERENCING
Research and development
    of algorithms and
  application for raster
     image analysis.

Target: detection of scans
   of the same map (via
    “image hash”) and
  following automatical
  assignment of ground
       control points.
NEATLINE DETECTION
Automatic detection of map sheet borders based on the
             image analysis algorithms.
GEOREFERENCING OF MAP SERIES
Database of Czech map series
with GIS-ready map indexes.

       Integration with
Georeferencer: georeferencing
with the help of existing map
           indexes.

Authorities for cataloguing of
         map series.

    Maybe a new project:
    OpenMapIndexes.org
WEBGL EARTH
Google Earth-like visualization of georeferenced maps in
          a web browser or on mobile devices.
100% open-source, no license restrictions, runs without
a need to install any extra software. Under development.

           http://www.webglearth.com/
http://www.webglearth.com/
COOPERATE. CONTACT US.
• Institutions interested in participation
• Share expectations - ideas for improvement?
• Sources of support on international level: development of
  a service for EU institutions and citizens?
• We are seeking research partners: image processing, ...

       HTTP://NLS.GEOREFERENCER.COM/
       HTTP://WWW.GEOREFERENCER.ORG/

                PETR.PRIDAL@MZK.CZ
Petr Přidal
                                       petr.pridal@klokantech.com




                       KLOKAN TECHNOLOGIES - Museo 300                            KLOKAN TECHNOLOGIES - Museo Sans 500
                       KLOKAN TECHNOLOGIES - Museo 500                            KLOKAN TECHNOLOGIES - Museo Sans 500
                       KLOKAN TECHNOLOGIES - Museo 700                            KLOKAN TECHNOLOGIES - Museo Sans 700




INTUITIVE GEOGRAPHICAL SEARCHING IN MAP
  COLLECTIONS AND METADATA CATALOGS

  International Cartographic Association: 6th International Workshop on Digital Approaches in Cartographic Heritage
                                       The Hague, Netherlands, 7-8 April 2011
GEOGRAPHICAL SEARCHING
    • Research, scientific articles, work on Ph.D. thesis (Petr Přidal),
      Computer Science + Geodesy and Cartography
    • Alternative projects tested in OldMapsOnline.org (R&D project in MZK):
       • Lucene with GeoTemporal Ext. (LGTE) from DigMap.eu
       • Alexandria Digital Library
       • TimeMap.net (ECAI.org)
       • Google GeoSearch API
    • Knowledge of traditional GIS search methods (PostGIS, ...)

Result: Development of a new geographical search engine
                 for map collections.
Explore library catalog by browsing a map.
CORE FEATURES

• Map-driven interface: Simple. Intuitive. Web 2.0.
  Users love it!
• Combined queries: Where, When, Who & What
• Results sorted geographically with an advanced
  spatial ranking algorithm.
• Instant search. Immediate delivery of the results by
  moving the map or typing.
• It is fast. Even for large databases (million+ records).
METADATA & CUSTOMIZATION
• Indexing metadata in MARC, DublinCore, ISO
  19139 formats, ...
• Batch import or harvesting: OAI-PMH or OGC CWS
• Numerical geographical boundaries necessary!
  (MARC 034, DublinCore DCMI) - enrichment with
  Geoparser or Georeferencer tools
• Possible customization of the user interface,
  integration into products or websites
• For developers: REST API (JSON format), queries
  following OpenSearch.org
Try it at: http://demo.mapranksearch.com/
Try it at: http://rumsey.mapranksearch.com/
Try it at: http://mapy.mzk.cz/
TOUCH INTERFACE
MAPRANK RANKING &
               SCALABILITY
• MapRank ranking: an advanced algorithm calculating
  similarity of geographical area of query and area covered
  by every document and combining it with map scale, time
  range, predefined categories and traditional full text
  search.
• Data indexing implemented specifically to provide ranked
  real-time results even for large databases. Traditional R-
  Tree indexing or Lucene are failing.
• Server component: FastCGI, C++
• Default user interface developed with Closure JavaScript
  Library + Closure Compiler.
ENRICHMENT OF METADATA

Geoparser
Text parsing and semi-automatic assignment of
bounding boxes to map records with detectable
geonames (gazetteer), known physical size and map
scale.

Georeferencer
Crowdsourced georeferencing. Higher accuracy, more
outputs beside geometadata such as visualization, 3D
view, comparing of maps, OGC WMS for use in GIS.
NEXT STEPS?
IMPROVEMENT OF THE SYSTEM

• The ranking: new requirements in Switzerland, union
  catalog. Boosting some maps, etc.
• Improvements related to map series - visualization of
  map indexes in the search interface
• Faceted search - refine the query: suggested authors
• Advanced search options: alternative sorting, ...


  IDEA: FEDERATED MAP SEARCH SYSTEM
CONTACT & LINKS
           Try the applications:
       http://www.kartenportal.ch/
    http://rumsey.mapranksearch.com/
           http://mapy.mzk.cz/

        More details and enquiries:

http://www.mapranksearch.com/
       INFO@KLOKANTECH.COM

CartoHeritage 2011: Georeferencer & MapRank Search

  • 1.
    Petr Přidal petr.pridal@mzk.cz GEOREFERENCER: COLLABORATIVE ONLINE GEOREFERENCING TOOL FOR SCANNED MAPS Moravian Library Brno: OldMapsOnline.org & TEMAP R&D grant of the Czech Ministry of Culture (DC08P02OUK006) International Cartographic Association: 6th International Workshop on Digital Approaches in Cartographic Heritage The Hague, Netherlands, 7-8 April 2011
  • 2.
    INTRODUCTION Large collections ofold maps are scanned and published online in digital repositories. But ... MAPS ARE NOT ONLY IMAGES! ... once they are georeferenced, they are much more attractive and useful: visualizations, comparing, location context, annotations, geographical search...
  • 3.
    PROBLEMS • Hand madegeoreferencing is time consuming process • The scanned collections are two large (thousands of maps to handle) • There are more technical approaches how to georeference old maps, not always clear workflow especially for pre-1850 - you need to keep this “how” • Disk space: georeferenced copy next to the original scan • GIS software and knowledge necessary • Workers need to have access to large files (JPEG/TIFF/...)
  • 4.
    GEOREFERENCER AS ASOLUTION • User-friendly online tool: runs in a web browser • Simply visit a website, log-in and you can start • Easy and enjoyable process • Open for online visitors, organized volunteers, crowdsourcing • No GIS software necessary, no local files to handle • GIS-ready outputs possible - you can easily create GeoTIFF/GeoJP2 if you have access to the high-resolution images • Service for EU libraries, archives & citizens • Developed in a library - for libraries and other culture heritage institutions GEOREFERENCER: TURN AN IMAGE INTO A MAP.
  • 8.
    MOTIVATION FOR GEOREFERENCER INSTITUTIONS: INDIVIDUALS: • Enrichment of metadata • Georeference any online (MARC, DC, ISO 19139) image on the Internet • Engaging visitors • Combine images from • Attractive 3D visualization different institutions (Google Earth) • Create your own virtual • GIS services (WMS, WMTS/ collection TMS, ...) • Conduct research online, • Statistics & crediting compare, analyze, share, ...
  • 9.
    INSTITUTIONS: PILOT APPLICATIONS •Moravian Library, Czech Republic (students) • Nationaal Archief, Netherlands (organized volunteers) • National Library of Scotland (online visitors, anybody) TRY THE GEOREFERENCER PILOT: HTTP://NLS.GEOREFERENCER.COM/ OR THE PUBLIC SERVICE FOR INDIVIDUALS: HTTP://WWW.GEOREFERENCER.ORG/
  • 10.
  • 11.
    GEOREFERENCER PILOTS Pilot: NationaalArchief. 13 volunteers. Great feedback!
  • 12.
    GEOREFERENCER PILOTS Result: 650maps georeferenced in less then one month! Metadata enrichment.
  • 13.
    GEOREFERENCER PILOTS Crowdsourcing: Retiredvolunteers. Excellent local knowledge. Competition & a public winner.
  • 14.
    GEOREFERENCER PILOTS First publicpilot: the National Library of Scotland.
  • 15.
    GEOREFERENCER PILOTS Anybody canparticipate. It is easy & fun. http://geo.nls.uk/maps/georeferencer/
  • 16.
    ADVANCED TECHNOLOGY • Unique scalable architecture. One centralized service for all libraries and maps. Upgrades and new functionality deployed automatically. • Runs “in the cloud” - Google AppEngine / Amazon. • Server side based on open-source GIS stack (Proj4, GDAL, MapSever). Closure Library on the client side. MINIMAL EFFORT TO PARTICIPATE. IMAGES ARE USED DIRECTLY FROM EXISTING ONLINE REPOSITORIES!
  • 17.
  • 18.
  • 19.
    SPATIAL REFERENCE SYSTEMS Coordinatesystems with map projection, for example OSGB or S-JTSK and in general any reference system from EPSG database usable in Georeferencer
  • 20.
    IMPROVED BASE MAPS Out-of-copyright base maps, agreements with local authorities for using official mapping data (WMS). Fast and visually appealing version of OpenStreetMap as an alternative - probably from MapQuest. OpenAerialMap.
  • 21.
    HIGH QUALITY GEOCODING Fastand reliable search in gazeteer database based on OpenStreetMap or GeoNames.org data. Online service, without license restrictions of Google GeoCoding API - usable both in GeoReferencer a GeoParser services.
  • 22.
    API FOR DATAINTERCHANGE AND 3RD PARTY FUNCTIONALITY REST API based on JSON for exchange of data between Georeferencer and third-party applications. JavaScript API for easy integration into library websites.
  • 23.
    COLLABORATIVE EDITING Technicalsolution of the problem of concurent team editing of map georeference by serveral people in the same moment - locking and communication a la online chat in a web browser (via HTTP Comet)
  • 24.
    CROWDSOURCING Social aspects: Motivation. Team. Competition. Guiding. Rewarding.
  • 25.
    DUPLICATES + AUTOGEOREFERENCING Research and development of algorithms and application for raster image analysis. Target: detection of scans of the same map (via “image hash”) and following automatical assignment of ground control points.
  • 26.
    NEATLINE DETECTION Automatic detectionof map sheet borders based on the image analysis algorithms.
  • 27.
    GEOREFERENCING OF MAPSERIES Database of Czech map series with GIS-ready map indexes. Integration with Georeferencer: georeferencing with the help of existing map indexes. Authorities for cataloguing of map series. Maybe a new project: OpenMapIndexes.org
  • 28.
    WEBGL EARTH Google Earth-likevisualization of georeferenced maps in a web browser or on mobile devices. 100% open-source, no license restrictions, runs without a need to install any extra software. Under development. http://www.webglearth.com/
  • 29.
  • 30.
    COOPERATE. CONTACT US. •Institutions interested in participation • Share expectations - ideas for improvement? • Sources of support on international level: development of a service for EU institutions and citizens? • We are seeking research partners: image processing, ... HTTP://NLS.GEOREFERENCER.COM/ HTTP://WWW.GEOREFERENCER.ORG/ PETR.PRIDAL@MZK.CZ
  • 31.
    Petr Přidal petr.pridal@klokantech.com KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo Sans 500 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo Sans 500 KLOKAN TECHNOLOGIES - Museo 700 KLOKAN TECHNOLOGIES - Museo Sans 700 INTUITIVE GEOGRAPHICAL SEARCHING IN MAP COLLECTIONS AND METADATA CATALOGS International Cartographic Association: 6th International Workshop on Digital Approaches in Cartographic Heritage The Hague, Netherlands, 7-8 April 2011
  • 32.
    GEOGRAPHICAL SEARCHING • Research, scientific articles, work on Ph.D. thesis (Petr Přidal), Computer Science + Geodesy and Cartography • Alternative projects tested in OldMapsOnline.org (R&D project in MZK): • Lucene with GeoTemporal Ext. (LGTE) from DigMap.eu • Alexandria Digital Library • TimeMap.net (ECAI.org) • Google GeoSearch API • Knowledge of traditional GIS search methods (PostGIS, ...) Result: Development of a new geographical search engine for map collections.
  • 33.
    Explore library catalogby browsing a map.
  • 34.
    CORE FEATURES • Map-driveninterface: Simple. Intuitive. Web 2.0. Users love it! • Combined queries: Where, When, Who & What • Results sorted geographically with an advanced spatial ranking algorithm. • Instant search. Immediate delivery of the results by moving the map or typing. • It is fast. Even for large databases (million+ records).
  • 35.
    METADATA & CUSTOMIZATION •Indexing metadata in MARC, DublinCore, ISO 19139 formats, ... • Batch import or harvesting: OAI-PMH or OGC CWS • Numerical geographical boundaries necessary! (MARC 034, DublinCore DCMI) - enrichment with Geoparser or Georeferencer tools • Possible customization of the user interface, integration into products or websites • For developers: REST API (JSON format), queries following OpenSearch.org
  • 36.
    Try it at:http://demo.mapranksearch.com/
  • 37.
    Try it at:http://rumsey.mapranksearch.com/
  • 38.
    Try it at:http://mapy.mzk.cz/
  • 39.
  • 40.
    MAPRANK RANKING & SCALABILITY • MapRank ranking: an advanced algorithm calculating similarity of geographical area of query and area covered by every document and combining it with map scale, time range, predefined categories and traditional full text search. • Data indexing implemented specifically to provide ranked real-time results even for large databases. Traditional R- Tree indexing or Lucene are failing. • Server component: FastCGI, C++ • Default user interface developed with Closure JavaScript Library + Closure Compiler.
  • 41.
    ENRICHMENT OF METADATA Geoparser Textparsing and semi-automatic assignment of bounding boxes to map records with detectable geonames (gazetteer), known physical size and map scale. Georeferencer Crowdsourced georeferencing. Higher accuracy, more outputs beside geometadata such as visualization, 3D view, comparing of maps, OGC WMS for use in GIS.
  • 44.
  • 45.
    IMPROVEMENT OF THESYSTEM • The ranking: new requirements in Switzerland, union catalog. Boosting some maps, etc. • Improvements related to map series - visualization of map indexes in the search interface • Faceted search - refine the query: suggested authors • Advanced search options: alternative sorting, ... IDEA: FEDERATED MAP SEARCH SYSTEM
  • 46.
    CONTACT & LINKS Try the applications: http://www.kartenportal.ch/ http://rumsey.mapranksearch.com/ http://mapy.mzk.cz/ More details and enquiries: http://www.mapranksearch.com/ INFO@KLOKANTECH.COM