TIDSR

Slides from Humanities on the Web: Is it working? Date: Thursday, 19 March 2009, 10-4 Location: Oxford University, Oxford, UK Webcast URL: http://webcast.oii.ox.ac.uk/?view=Webcast&ID=20090319_274 Slide URL: http://www.slideshare.net/etmeyer/TIDSR Morning Event: 10:00 – 11:15: Toolkits for e-Humanities Overview of Phase 1 Digitisation Projects (Dr. Kathryn Eccles) Quantitative Tools (Prof. Mike Thelwall & Dr. Kathryn Eccles) Qualitative Tools (Dr. Eric Meyer & Dr. Kathryn Eccles) Survey Data (Christine Madsen & Dr. Eric Meyer) Discussion and questions Full details: http://www.oii.ox.ac.uk/events/details.cfm?id=238

Toolkit for the Impact of Digitised Scholarly Resources: Introduction to JISC Phase One Digitisation projects Kathryn Eccles Oxford Internet Institute University of Oxford

Project 1 – Online Historical Population Reports (OHPR/Histpop)

Project 1 – Online Historical Population Reports (Histpop)

Project 2 – British Library 19th Century Newspapers

Project 3 – British Library Archival Sounds

Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834

Project 5 – Wellcome Medical Journals: the backfiles project

Find out more:
Find out more at the Project websites:
Histpop: http://www.histpop.org
British Library C19th Newspapers: http://www.bl.uk/reshelp/findhelprestype/news/newspdigproj/paperdigit.html
British Library Archival Sounds: http://sounds.bl.uk/
BOPCRIS: http://www.parl18c.soton.ac.uk/parl18c/digbib/home
Medical backfiles: http://library.wellcome.ac.uk/backfiles/

Find out more:
More about the JISC Digitisation Programme:
http://www.jisc.ac.uk/whatwedo/programmes/digitisation.aspx
Visit the toolkit at:
http://microsites.oii.ox.ac.uk/tidsr/

Oxford Internet Institute University of Oxford Kathryn Eccles [email_address] http://www.oii.ox.ac.uk/people/faculty.cfm?id=138

Toolkit for the Impact of Digitised Scholarly Resources: Quantitative Methods Dr Kathryn Eccles Oxford Internet Institute, University of Oxford & Professor Mike Thelwall University of Wolverhampton

Overview Quantitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources: • Webometrics • (Google) Analytics • Log file analysis • Scientometrics / bibliometrics

Webometrics
What is webometrics?
Gathering, processing and analysing large scale data from the web (web pages, hyperlinks, blogs, Web 2.0) for many purposes that include online communication, although primarily for scientific communication
What can webometrics tell us about the Impact of Digitised Scholarly Resources ?
Counting pages that link to scholarly resource archives can indicate the impact of the archives
More links suggests more users and higher impact
Assessing the origins of links gives insights into users of the scholarly resources by finding out why the links were created
E.g., universities, schools, bloggers, countries
Comparing the results with similar archives
Indicates relative impact & potentially missed user groups

Webometrics
Using webometrics in the TIDSR project
Comparator sites
Running the analysis – problems
Results:
Histpop performs well against its comparator projects

Webometrics
British Library C19th Newspapers records a large number of links for an information page
British Library Archival Sounds performs well; contrasts heavily with British Library Sound Archive
BOPCRIS C18th Parliamentary Papers records fewer links than to BOPCRIS homepage, but higher than to its commercial partner at ProQuest
Wellcome Medical Backfiles project page records strong links, links to PubMed for WMB material impossible to trace

Analytics
What are (Google) analytics?
Data about a web site’s visitors typically derived from tracking code hidden in each of the web site’s pages
Google Analytics provides a free service and extensive summary statistics about visitors
Commercial analytics can provide more in-depth and customisable statistics – probably most useful for large-scale commercial websites

What can analytics tell us?
Google Analytics can be extremely effective in providing information about (Fang, 2007) :
The usage of a website
Visitor behaviors
The efficiency of the website's menu system
Suggestions for improving user experiences
The most effective redesign of the website
The geographic origins of visitors
Search queries sending visitors to the site

Log File analysis
What is Log File Analysis?
(Web server) log files are simple plain text files that record who accesses the pages and other resources in a web site
Each access is recorded on a separate line of the file
Who accessed each page is recorded by IP address
Log file analysis programs read in the raw log files and produce useful summary statistics (similar to analytics), e.g.
Daily or hourly number of visitors
Most popular pages
Average time on site of visitors
Search engine queries driving visitors to site
Geographic origin of visitors (from decoded IP addresses)

Log File analysis
Using log file analysis in the TIDSR project
Collecting web statistics from log file data
Comparing:
- What kind of data was collected
- How often this was consulted
Collecting raw log files
Looking at:
- What search terms were used to find the resource
- The top referrer sites
- Access statistics

Log File Analysis
Top Search Phrases: Histpop

Scientometrics/Bibliometrics
Bibliometrics are typically measurements of document collections and are traditionally mainly derived from scientific databases
A common (partial) indicator of the importance of a document is how often it has been cited by other documents
Often-cited documents tend to be thought of as more valuable to the scientific community
Bibliometrics are less relevant outside science in the humanities and social sciences and are not reliable for individual articles
Bibliometric indicators can help to evaluate the impact of a collection of documents

How useful were Scientometrics/Bibliometrics in the TIDSR project?
Google Scholar, Scopus, ISI Web of Knowledge/Web of Science
Limited time scale
Examples from projects (and many were about the projects, rather than links to project materials)
Histpop: 11
BL 19 th Century Newspapers: 4
BL Archival Sounds: 9
BOPCRIS: 8
Medical Journals: 17
Importance of trackable citation styles

For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/

Toolkit for the Impact of Digitised Scholarly Resources: Qualitative Methods Dr Eric Meyer & Dr Kathryn Eccles Oxford Internet Institute, University of Oxford

Overview
Qualitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources:
• Interviews
Focus groups
User feedback
Referrer analysis

Interviews
Interviews
Key project personnel
Related but non included institutional personnel
End-user communities
Subject specialists
Other stakeholders
Funding personnel

Interviews
Project impacts revealed through interviews
Quality of undergraduate dissertation work improved by early contact with digitised primary sources
‘ Histpop made it possible to do a completely different project’
Type of research projects being presented at conferences increasingly quantitative
‘ You do feel the ground shifting...’
New possibilities for serendipitous research
‘ What I’ve found most enjoyable... is the serendipity’

Focus groups
Focus groups
Themes we covered:
General attitude to and knowledge of digital resources
Methods of searching for digital resources
Citation of digital resources
Issues of trust in discovering and using digital resources
Training and support in using digital resources
Resource specific questions – findability and usability

User feedback
Using User feedback within your own resource:
Keep a record of all the correspondence you receive from users
Periodically review the content of your user feedback (classify each response as negative, positive, both or neutral
Use contact with users to recruit focus group and survey participants
Use contact with users to gather information about how your resources is being used (Distance learning courses, undergraduate dissertations)
Use this extra feedback to enhance the impact of the resource on these communities

User feedback
User feedback in the TIDSR project
Projects were reticent about providing email content from users
We were only able to gather anecdotal evidence from most resources
User feedback from Histpop:
We were allowed to send an email to be forwarded to users who had contacted Histpop
This allowed us to contact a range of users and ask them detailed questions about the resource
This yielded further information about the ways in which the resource was being used, particularly in the teaching and learning environments

Referrer analysis
Referrer Analysis
Similar to the quantitative webometrics in approach, but requires a much more qualitative look at the resulting pages to try to find educational referrers (such as libraries and course reading lists)

TIDSR Survey Data Eric Meyer Christine Madsen OII 19 March 2008

Ave. time spent filling out survey: 17.5 minutes
83% completed the survey in under 20 min
714 started
550 completed (77%)
80% agreed to continue on to further questions

Demographics

Demographics Average Age 45.3 Age Range 20-83

Degrees & Field

Awareness of Resources by Country

Awareness of Resource by Country

Have you ever published a piece based on your work in this collection? If so, how did you cite the collection?

How did you find this resource?

How did you find this resource? Histpop BOPCRIS BL News BL Sounds Med Backfiles Google 33% 33% 25% 19% 37% Prof society, discussion list or conference 25% 22% 26% 43% 27% Library 4% 22% 32% 10% 27% Colleague 25% 18% 11% 24% 6% Other 13% 4% 7% 5% 2% Student 0% 0% 0% 0% 0% Campus IT 0% 0% 0% 0% 0%

How do you use this resource?

How do you use this resource? Histpop BOPCRIS BL News BL Sounds Med Backfiles Online reference source 78% 81% 74% 66% 78% For personal interests 35% 49% 50% 46% 27% Download to use offline 26% 36% 40% 29% 57% Teaching resource 30% 30% 29% 39% 27% Manual analysis 17% 32% 38% 10% 39% Find materials to consult in person 4% 23% 18% 12% 39% Computational analysis 30% 6% 4% 2% 10% Reuse, remix or edit 17% 11% 9% 7% 0% Other 4% 0% 3% 5% 6%

Attitudes towards Digitisation

TIDSR

by Eric Meyer on Mar 27, 2009

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 2

Statistics

TIDSR Presentation Transcript