Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Eric Meyer, profile picture
Uploaded byEric Meyer
PPTX, PDF877 views

tidsrdhoxss2012

The document discusses impact assessment of digital resources. It defines impact as reaching intended audiences, enabling new research questions, and enabling new approaches to education. It recommends identifying key stakeholders and goals to ensure impact. Metrics for measuring impact include usage analytics, citations, and embedding in other resources. The TIDSR project studied five digital resources and tested impact assessment methods. Interviews found resources transformed access and usage patterns, allowing more comprehensive searches and serendipitous findings. Resources are now part of broader transformations in research workflows, allowing more efficient discovery of sources.

Education◦Technology◦
Download to read offline
Impact as a process: considering the reach of resources from the start Eric T. Meyer & Kathryn Eccles Oxford Internet Institute, University of Oxford @etmeyer Digital Humanities@Oxford Summer School #tidsr #dhoxss 5th July 2012 Slides at: http://www.slideshare.net/etmeyer/tidsrdhoxss2012
What is impact and why consider it? What do we mean by impact? •Reaching intended audience •Reaching new audiences •Attracting users •Attracting new usage •Enabling new research questions •Enabling new approaches to education
Where to begin? • Identify your audience and key stakeholders • Set your goals. What types of impact do you envisage your resource having? • What steps are you taking to ensure these types of impact? • Identify connections • What resources do you see as successful in terms of audience and impact? • Do you see your resource as part of a network of connected resources?
Digitisation and Impact
Measuring usage and impact What to measure? • Users • Types of use • Awareness • Citation practices • Marketing strategies • Embedding
TIDSR: The first usage and impact study • JISC funded project • July 2008-April 2009 • Looked at five specific JISC-funded resources • Designed to test the TIDSR methods and review them for the TIDSR toolkit
Methods Quantitative methods Qualitative methods • Webometrics • Interviews • Web Analytics • Focus groups • Log file analysis • User feedback • Scientometrics / • Referrer analysis bibliometrics • Content Analysis • Content Analysis
100% 2% 2% 1% 2% 3% 7% 7% 7% 7% 90% 7% Use it regularly or 18% frequently 80% 22% 27% 28% 70% 60% Use it on occasion 35% 50% 88% 40% 69% Have seen it, but don't 30% 63% 63% use it 20% 39% 10% I haven't heard of it 0% HistPop BOPCRIS BL News BL Sounds Med Backfiles
100% 96% 90% 90% 86% 84% 82% 79% 80% 76% 71% 72% 69% 70% 60% 61% 60% Important to my research 51% 49% 50% Important to my teaching 40% 40% 35% 36% Important to field 34% 33% 29% 30% Would recommend 20% 10% 0% HistPop BOPCRIS BL News BL Sounds Med Backfiles
 Top Search Phrases: Histpop
 Perception: Specific niche community  Well known by target audience  Transforming access and usage patterns  User surveys:  Embedded in educational resources  Enhanced access to primary sources ▪ ‘Histpop made it possible to do a completely different project’  Continuing education, online resources, non- traditional learners
Have you ever published a piece If so, how did you cite the collection? based on your work in this collection? Original version Original + URL Online version Other 46% HistPop 9% 55% 36% 0% 29% BOPCRIS 36% 36% 21% 7% 35% BL News 53% 38% 6% 2% BL 20% 38% 50% 13% 0% Sounds Med 43% 43% 48% 10% 0% Backfiles 60% 40% 20% 0% 0% 20% 40% 60% 80% 100%
• Highest numbers for original British Library resource (analogue) • 19th Century British Library Newspapers registers strong links for a project page • Note: Importance of comparator sites when using webometrics
 Time intensive, but productive if you are careful about what you ask!  Different stakeholders:  Project team: Positive view of the work only  Broader stakeholders: While the digital project was good, it also introduced tensions in the broader setting of the library  New kinds of serendipity, wide range of users
Engagement officer
• Some resources are available through multiple outlets • Webometrics can capture comparative awareness • These results show how powerful known resources and/or publishers can be
 Wellcome Medical Journals Backfiles project page records strong links, links to Pub Med for WMJB material impossible to trace
 Historians? (would be looking at older articles)  Not typical PubMed users  Search interface issues / limited search  Clinicians? (would be looking at newer articles)  Not typically reading 100 year old articles  Other users?  Paths of discovery?
 Majority of downloads targeted more recent material – opening up of new resources to clinicians  More thorough and comprehensive searches  Historians reported more comprehensive search results (quantitative results)  Also reported increased browsing, greater serendipity, due to time saved finding articles
House of Commons Parliamentary Papers 48% 83% British Library Newspapers 50% 77% British Periodicals 56% 71% Old Bailey Online 36% 62% Imperial War Museum 32% 58% Archival Sounds 24% 54% BOPCRIS 27% 51% Internet Lib of Early Journals 38% 47% Historical Directories 32% 45% Wellcome Medical Journal Backfiles 29% 34% Fine Rolls 18% 31% Chronicling America 30% 17% Histpop 9% Non-UK Awareness 16% Sciper 11% 12% UK Awareness Cornell Animal Sounds 13% 7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
50% 45% 40% HistPop 35% BOPCRIS 30% 25% BL News 20% BL Sounds 15% Med Backfiles 10% 5% 0%
http://microsites.oii.ox.ac.uk/tidsr/
Coming this autumn…. Project page: Impact of Digitised Resources http://www.kdcs.kcl.ac.uk/innovation/impact.html Simon Tanner’s blog post about the project http://simon-tanner.blogspot.co.uk/2012/03/new-approach-to- measuring-impact-for.html
University of Oxford Podcasts
Proceedings of the Old Bailey Online
British History Online
Siobhan Davies RePlay
http://www.rin.ac.uk/humanities-case-studies Bulger, M., Meyer, E.T., de la Flor, G., Terras, M., Wyatt, S., Jirotka, M., Eccles, K., Madsen, C.
The Case Studies
Browsing and Searching 79% Google 66% Google Scholar Libraries 59% Visit the library 55% Browse library materials online 62% Search library materials online 83% Citation chaining Journals 48% Browse printed journals 76% Browse online journals Peers 95% Consult peers and experts
Reconfiguring Resources
“ Old Bailey Online hasn’t replaced anything for me or displaced anything for me, but it is part of this general transformation of how I do what I do.
“ The amount of time I now spend doing the very mechanical, laborious, time- consuming work is much smaller. You can now do things in 5 seconds which it took you 3 months to do a few years ago.
Transformations in Use
“ It’s a huge change. You can do things much more quickly, read much more widely, find connections…it’s very, very important.
“ With something like the Burney Collection, 5 years ago for writing an article I would need to review the newspapers, I would have gone into the British Library and done it on microfilm. 20 years ago, I would have gone into the British Library and done it with the actual paper in front of me. Now I sit at home and I do a keyword search.
“ Asking new questions?
“ I’m not sure all of this raises the quality of anybody’s work. I think it would be quite daft to pretend that all of this makes us better scholars, or makes our books or papers of higher quality. I don’t know if that is true by any means, but it certainly makes it easier and I suppose makes the quantity of stuff that you can produce greater.
“ What might take you several months if not years of research, you could do in hours, days, a week. So I think that means that it makes the nature of your research different because it allows you quantitative information much more quickly, which then allows you to maybe think about how you might use that information differently, because you’ve got so much more time.
Slides at: http://www.slideshare.net/etmeyer/tidsrdhoxss2012 Eric T. Meyer Kathryn Eccles eric.meyer@oii.ox.ac.uk kathryn.eccles@oii.ox.ac.uk http://www.oii.ox.ac.uk/people/?id=120 http://www.oii.ox.ac.uk/people/?id=138 Oxford e-Social Science Project Project work funded by: