UK Digitisation programme Developing Community Content strand of the JISC Digitisation and e-Content programme Welsh Voices of the Great War in Wales – Cardiff University
Online engagement tool based on web 2.0 principles Galaxy Zoo is an online astronomy project which invites members of the public to assist in classifying over sixty million galaxies Old Weather is a web-based effort to transcribe weather observations made by Royal Navy ships around the time of World War I The Great War Archive , was a 2008 project led by the University of Oxford that asked members of the general public to digitise any artefacts they held relating to the First World War and upload them to a purpose built website.
Bank directory listing banks and banking companies Educational directory listing educational institutions and teachers by their subject Law directory listing juridical institutions and practitioners Medical directory listing medical and surgical institutions and practitioners Insurance directory listing insurance companies Rich source of adverts which give an idea as to lifestyles, spending habits, Of interest to genealogists, local or family historians, academic researchers
44,000 historical maps of Scotland – county maps, town plans, admiralty charts (coastline), military maps, Historic OS series Plus 600 of Edinburgh and its environs Images, OCR text Creative Commons licences - IPR free - Attribution-NonCommercial-ShareAlike 2.5 UK: Scotland Internet Archive team based at the National Library of Scotland for scanning the Scottish Post office Directories used in the project.
Registered users Google Geocoding API assigns a georeference with scales of accuracy – from town to street to intersection to building
Identify and fix line returns, identify which fields belong to which column, Fix OCR errors – list of search patters and their replace strings (for names, professions, addresses XML files) Name stop words to remove commercial entries
POI’s in this case are POD entries – namely Address, Name and profession
Critical Mass – it could be argued that the geographical & temporal coverage provided by AH doesn’t provide the critical mass of content required to attract and engage ‘the crowd’? This is borne out in our usage stats (and registered users) – which whilst not small were modest
AddressingHistory: Lessons and Messages
AddressingHistory – Lessons and Messages Stuart Macdonald Associate Data Librarian EDINA & Data Library University of Edinburgh email@example.comAssociation of American Geographers Annual Meeting - Working Digitally with Historical Maps, New York Public Library, 25 Feb. 2012
Phase 1JISC-funded Community Contentproject6 months (April 2010 – September2010)Partner with National Library ofScotlandAdvisory Board
To create an online crowdsourcing tool which will combinedata from digitised historical Scottish Post OfficeDirectories (PODs) with contemporaneous historical mapsSimilar to Australian Historic Newspapers projectprovided by National Library of Australia where membersof the public correct and improve OCR’d text of oldnewspapers - http://www.nla.gov.au/ndp/project_details/JISC-funded Great War Archive (Univ. Oxford) that askedmembers of the general public to digitise any First WorldWar artefacts and upload them to a purpose built website.
PODs offer a fine-grained spatialand temporal view on social,economic and demographiccircumstancesThey provide residential names,occupations, and addresses.Each contain 3 sub-directories:general, street, and tradesMay also contain misc. tradedirectories e.g. banking,education, law, insurance,medical
Phase 1 focused on 3 vols. ofEdinburgh PODs: 1784-5; 1865;1905-6Historic Scottish maps geo-referenced by NLSPODs digitised by NLS inconjunction with the InternetArchivec.700 PODs (1773 to 1911)covering 28 of Scotlands townsand counties now onlinePublic domain (CC BY-NC-SA 2.5)
Using Open Layers as web-based mapping clientTool allows ‘the crowd’ togeoreference a POD entry bymoving a ‘map pin’ on adigitised map thus facilitatingthe addition of an grid referenceto the OCR’d POD held in XMLformat in a database structure(PostgreSQL)API available allowing webdevelopers access to the rawdata in multiple output formats(JSON, XML, CSV)Geo-coding of POD addressesparsed against Googlegeocoder
Interface had to be easy-to-use for arange of usersRobust and scalable to accommodatec.700 digitised Scottish PODsMechanism to check user-generatedcontent such as geo-references,name or address edits/annotationsCrowdsourcing of geo-coded gridreferencesView original scanned directory pageAmplification of tool and API viaSocial Media Channels – Facebook,Twitter, Blog, Flickr, YouTube
Search people, place, profession Historic Map overlay selected Record edits by the ‘crowd’ View originalSearchresults Download options
Phase 2 sought to develop functionality to resonate with JISC’svision to build sustainable and durable deliverables and tocompliment phase 1 by broadening both geographic and temporalcoverageFeb. – Sept. 2011 (EDINASustainability Funding)New content (Aberdeen, Glasgow,Edinburgh for 1881 & 1891Re-evaluate (and enhance) parsingtool performanceOld parser :•Exact geotag – 60%•Professions – 25%New parser (no configuration file):•Exact geotag – 72%•Professions – 76%New parser (with configuration file)•Exact geotag – 88%•Professions – 82%
Phase 2Other additional features include: • Spatial searching (bounding box) • Associate map pin with search results • Search across multiple address • Aid searching by applying Standard Industrial Classification (SIC) codes to Professions
Augmented RealityAn AddressingHistory layer hasbeen created and published foruse with the ‘Layar’ Applicationfor either iPhone or AndroidGeo-referenced Points ofInterest (POIs) are uploadedinto the BuildAR CMSPOIs (e.g. each profession orSIC Code) have an imageassociated with itThe App allows users to compare their current location (from phone)with the geo-referenced AH records in order to establish which namesand professions are located in the local vicinity
Lessons LearnedCritical mass – does geographic & temporal coverage attract andengage the crowd?Separate out parsing from interface and back endstorage - to allow any refinements to be implemented withoutimpacting on tool and APIExternalise ‘configuration’ files – editable XML-based filesthat accommodate repeated OCR and content inconsistencies –these are run in conjunction with the POD parser to refine the parsedcontent henceimproved searchingParsing and refining process is almost unending -Identify what is realistically achievable with available resourcesand time constraints- i.e. perform proper requirements analysisConsult with others - involved in digitising and parsingcity/town/post office directories e.g. Richard Marciano(UNC Chapel-Hill), Matt Knutzen (NYPL)
SustainabilityGiven the broad applicability of theresource a range of communities may beinterested in the longer term curation ofthe project tools e.g. the Open Street Mapcommunity, NLSEvaluation of possible business modelsfor sustainability:• revenue generation via online donations• subscription model (e.g. per annum, permonth, per use)• ‘freemium model’ (e.g. free APIdownload of a certain number of recordswith payment for further downloads)• academic advertising.
Second last slide…New content and features to be made available start ofMarch 2012Gauging the success of the project goes beyond thedelivery of engaging and innovative online tools. It willbe ultimately be measured by continual and extendeduse within the wider community.
Website:http://addressinghistory.edina.ac.uk/ THANKING YOU! Credits: Image by aroid - http://www.flickr.com/photos/selago/34843234/ - CC BY 2.0 Image by konqui - http://www.flickr.com/photos/konqui/2301314089/ - CC BY-NC 2.0 Image by mosilager - http://www.flickr.com/photos/mosilager/2260598271/ - CC BY-NC-SA 2.0 Image by racoles - http://www.flickr.com/photos/racoles/5719938981/ - CC BY-NC 2.0 Image by James Bowe - http://www.flickr.com/photos/jamesrbowe/3351247547/ (CC BY 2.0) Image by yelnoc - http://www.flickr.com/photos/yelnoc/361303918/ - CC BY-NC-SA 2.0 Image by epSos.de - http://www.flickr.com/photos/epsos/3384297473/ - CC BY 2.0 Image by bek30 - http://www.flickr.com/photos/bek30/6107854810/ - CC BY-NC 2.0 Image by karen horton - http://www.flickr.com/photos/karenhorton/3261277303/ - CC BY-NC 2.0 Image by lofaesofa - http://www.flickr.com/photos/lofaesofa/227019975/ - CC BY 2.0 Image by Psycho Delia - http://www.flickr.com/photos/24557420@N05/5588473657/ - CC BY-NC 2.0 Image by wdj(0) - http://www.flickr .com/photos/davidjoyner/534893725/ - CC BY-SA 2.0 Image by Symic - http://www.flickr.com/photos/symic/2870349309/ - CC BY-SA 2.0 Image by ~milj - http://www.flickr.com/photos/21989292@N07/4938052014/ - CC BY-NC-SA 2.0 Acknowledgements: JISC - http://www.jisc.ac.uk/ NLS Geo-referenced maps and applications - http://geo.nls.uk/ Visualising Urban Geographies (VUG) project – http://geo.nls.uk/urbhist/ Edinburgh City Libraries – http://www.edinburgh.gov.uk/libraries/
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.