Slides from Humanities on the Web: Is it working? Date: Thursday, 19 March 2009, 10-4 Location: Oxford University, Oxford,...
Toolkit for the Impact of Digitised Scholarly Resources: Introduction to JISC Phase One Digitisation projects Kathryn Eccl...
Project 1 – Online Historical Population Reports (OHPR/Histpop)
Project 1 – Online Historical Population Reports (OHPR/Histpop)
Project 1 – Online Historical Population Reports (Histpop)
Project 2 – British Library 19th Century Newspapers
Project 2 – British Library 19th Century Newspapers
Project 2 – British Library 19th Century Newspapers
Project 2 – British Library 19th Century Newspapers
Project 3 – British Library Archival Sounds
Project 3 – British Library Archival Sounds
Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parlia...
 
Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parlia...
Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parlia...
Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parlia...
Project 5 – Wellcome Medical Journals: the backfiles project
Project 5 – Wellcome Medical Journals: the backfiles project
Project 5 – Wellcome Medical Journals: the backfiles project
Find out more: <ul><li>Find out more at the Project websites: </li></ul><ul><li>Histpop:  http://www.histpop.org   </li></...
Find out more: <ul><li>More about the JISC Digitisation Programme: </li></ul><ul><li>http://www.jisc.ac.uk/whatwedo/progra...
Oxford Internet Institute University of Oxford Kathryn Eccles [email_address] http://www.oii.ox.ac.uk/people/faculty.cfm?i...
Toolkit for the Impact of Digitised Scholarly Resources: Quantitative Methods Dr Kathryn Eccles Oxford Internet Institute,...
Overview Quantitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources: •  Webometrics •  (Goog...
Webometrics <ul><li>What is webometrics? </li></ul><ul><li>Gathering, processing and analysing large scale data from the w...
Webometrics <ul><li>Using webometrics in the TIDSR project </li></ul><ul><li>Comparator sites </li></ul><ul><li>Running th...
Webometrics <ul><ul><li>British Library C19th Newspapers records a large number of links for an information page </li></ul...
Analytics <ul><li>What are (Google) analytics? </li></ul><ul><li>Data about a web site’s visitors typically derived from t...
<ul><li>What can analytics tell us? </li></ul><ul><li>Google Analytics can be extremely effective in providing information...
Log File analysis <ul><li>What is Log File Analysis? </li></ul><ul><li>(Web server)   log files  are simple plain text fil...
Log File analysis <ul><li>Using log file analysis in the TIDSR project </li></ul><ul><li>Collecting web statistics from lo...
Log File Analysis <ul><li>Top Search Phrases: Histpop </li></ul>
Scientometrics/Bibliometrics <ul><li>Scientometrics/Bibliometrics </li></ul><ul><li>Bibliometrics are typically measuremen...
Scientometrics/Bibliometrics <ul><li>How useful were Scientometrics/Bibliometrics in the TIDSR project? </li></ul><ul><li>...
For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/
Toolkit for the Impact of Digitised Scholarly Resources: Qualitative Methods Dr Eric Meyer  & Dr Kathryn Eccles Oxford Int...
Overview <ul><li>Qualitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources: </li></ul><ul><l...
Interviews <ul><li>Interviews </li></ul><ul><li>Key project personnel </li></ul><ul><li>Related but non included instituti...
Interviews <ul><li>Project impacts revealed through interviews </li></ul><ul><li>Quality of undergraduate dissertation wor...
Focus groups <ul><li>Focus groups </li></ul><ul><li>Themes we covered: </li></ul><ul><li>General attitude to and knowledge...
User feedback <ul><li>Using User feedback within your own resource: </li></ul><ul><li>Keep a record of all the corresponde...
User feedback <ul><li>User feedback in the TIDSR project </li></ul><ul><li>Projects were reticent about providing email co...
Referrer analysis <ul><li>Referrer Analysis </li></ul><ul><li>Similar to the quantitative webometrics in approach, but req...
For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/
TIDSR Survey Data Eric Meyer Christine Madsen OII 19 March 2008
<ul><li>Ave. time spent filling out survey: 17.5 minutes </li></ul><ul><li>83% completed the survey in under 20 min </li><...
Demographics
Demographics Average Age 45.3 Age Range 20-83
Degrees & Field
 
 
Awareness of Resources by Country
Awareness of Resource by Country
 
 
 
 
Have you ever published a piece based on your work in this collection?  If so, how did you cite the collection?
 
How did you find this resource?
How did you find this resource? Histpop BOPCRIS BL News BL Sounds Med Backfiles Google 33% 33% 25% 19% 37% Prof society, d...
How do you use this resource?
How do you use this resource? Histpop BOPCRIS BL News BL Sounds Med Backfiles Online reference source 78% 81% 74% 66% 78% ...
 
 
 
Attitudes towards Digitisation
 
 
 
For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/
Upcoming SlideShare
Loading in...5
×

TIDSR

2,013

Published on

Combined set of presentations on ways of measuring the impact of digitised scholarly resources.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,013
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Transcript of "TIDSR"

    1. 1. Slides from Humanities on the Web: Is it working? Date: Thursday, 19 March 2009, 10-4 Location: Oxford University, Oxford, UK Webcast URL: http://webcast.oii.ox.ac.uk/?view=Webcast&ID=20090319_274 Slide URL: http://www.slideshare.net/etmeyer/TIDSR Morning Event: 10:00 – 11:15: Toolkits for e-Humanities Overview of Phase 1 Digitisation Projects (Dr. Kathryn Eccles) Quantitative Tools (Prof. Mike Thelwall & Dr. Kathryn Eccles) Qualitative Tools (Dr. Eric Meyer & Dr. Kathryn Eccles) Survey Data (Christine Madsen & Dr. Eric Meyer) Discussion and questions Full details: http://www.oii.ox.ac.uk/events/details.cfm?id=238
    2. 2. Toolkit for the Impact of Digitised Scholarly Resources: Introduction to JISC Phase One Digitisation projects Kathryn Eccles Oxford Internet Institute University of Oxford
    3. 3. Project 1 – Online Historical Population Reports (OHPR/Histpop)
    4. 4. Project 1 – Online Historical Population Reports (OHPR/Histpop)
    5. 5. Project 1 – Online Historical Population Reports (Histpop)
    6. 6. Project 2 – British Library 19th Century Newspapers
    7. 7. Project 2 – British Library 19th Century Newspapers
    8. 8. Project 2 – British Library 19th Century Newspapers
    9. 9. Project 2 – British Library 19th Century Newspapers
    10. 10. Project 3 – British Library Archival Sounds
    11. 11. Project 3 – British Library Archival Sounds
    12. 12. Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834
    13. 14. Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834
    14. 15. Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834
    15. 16. Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834
    16. 17. Project 5 – Wellcome Medical Journals: the backfiles project
    17. 18. Project 5 – Wellcome Medical Journals: the backfiles project
    18. 19. Project 5 – Wellcome Medical Journals: the backfiles project
    19. 20. Find out more: <ul><li>Find out more at the Project websites: </li></ul><ul><li>Histpop: http://www.histpop.org </li></ul><ul><li>British Library C19th Newspapers: http://www.bl.uk/reshelp/findhelprestype/news/newspdigproj/paperdigit.html </li></ul><ul><li>British Library Archival Sounds: http://sounds.bl.uk/ </li></ul><ul><li>BOPCRIS: http://www.parl18c.soton.ac.uk/parl18c/digbib/home </li></ul><ul><li>Medical backfiles: http://library.wellcome.ac.uk/backfiles/ </li></ul>
    20. 21. Find out more: <ul><li>More about the JISC Digitisation Programme: </li></ul><ul><li>http://www.jisc.ac.uk/whatwedo/programmes/digitisation.aspx </li></ul><ul><li>Visit the toolkit at: </li></ul><ul><li>http://microsites.oii.ox.ac.uk/tidsr/ </li></ul>
    21. 22. Oxford Internet Institute University of Oxford Kathryn Eccles [email_address] http://www.oii.ox.ac.uk/people/faculty.cfm?id=138
    22. 23. Toolkit for the Impact of Digitised Scholarly Resources: Quantitative Methods Dr Kathryn Eccles Oxford Internet Institute, University of Oxford & Professor Mike Thelwall University of Wolverhampton
    23. 24. Overview Quantitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources: • Webometrics • (Google) Analytics • Log file analysis • Scientometrics / bibliometrics
    24. 25. Webometrics <ul><li>What is webometrics? </li></ul><ul><li>Gathering, processing and analysing large scale data from the web (web pages, hyperlinks, blogs, Web 2.0) for many purposes that include online communication, although primarily for scientific communication </li></ul><ul><li>What can webometrics tell us about the Impact of Digitised Scholarly Resources ? </li></ul><ul><li>Counting pages that link to scholarly resource archives can indicate the impact of the archives </li></ul><ul><ul><li>More links suggests more users and higher impact </li></ul></ul><ul><li>Assessing the origins of links gives insights into users of the scholarly resources by finding out why the links were created </li></ul><ul><ul><li>E.g., universities, schools, bloggers, countries </li></ul></ul><ul><li>Comparing the results with similar archives </li></ul><ul><ul><li>Indicates relative impact & potentially missed user groups </li></ul></ul>
    25. 26. Webometrics <ul><li>Using webometrics in the TIDSR project </li></ul><ul><li>Comparator sites </li></ul><ul><li>Running the analysis – problems </li></ul><ul><li>Results: </li></ul><ul><ul><li>Histpop performs well against its comparator projects </li></ul></ul>
    26. 27. Webometrics <ul><ul><li>British Library C19th Newspapers records a large number of links for an information page </li></ul></ul><ul><ul><li>British Library Archival Sounds performs well; contrasts heavily with British Library Sound Archive </li></ul></ul><ul><ul><li>BOPCRIS C18th Parliamentary Papers records fewer links than to BOPCRIS homepage, but higher than to its commercial partner at ProQuest </li></ul></ul><ul><ul><li>Wellcome Medical Backfiles project page records strong links, links to PubMed for WMB material impossible to trace </li></ul></ul>
    27. 28. Analytics <ul><li>What are (Google) analytics? </li></ul><ul><li>Data about a web site’s visitors typically derived from tracking code hidden in each of the web site’s pages </li></ul><ul><li>Google Analytics provides a free service and extensive summary statistics about visitors </li></ul><ul><li>Commercial analytics can provide more in-depth and customisable statistics – probably most useful for large-scale commercial websites </li></ul>
    28. 29. <ul><li>What can analytics tell us? </li></ul><ul><li>Google Analytics can be extremely effective in providing information about (Fang, 2007) : </li></ul><ul><li>The usage of a website </li></ul><ul><li>Visitor behaviors </li></ul><ul><li>The efficiency of the website's menu system </li></ul><ul><li>Suggestions for improving user experiences </li></ul><ul><li>The most effective redesign of the website </li></ul><ul><li>The geographic origins of visitors </li></ul><ul><li>Search queries sending visitors to the site </li></ul>
    29. 30. Log File analysis <ul><li>What is Log File Analysis? </li></ul><ul><li>(Web server) log files are simple plain text files that record who accesses the pages and other resources in a web site </li></ul><ul><ul><li>Each access is recorded on a separate line of the file </li></ul></ul><ul><ul><li>Who accessed each page is recorded by IP address </li></ul></ul><ul><li>Log file analysis programs read in the raw log files and produce useful summary statistics (similar to analytics), e.g. </li></ul><ul><ul><li>Daily or hourly number of visitors </li></ul></ul><ul><ul><li>Most popular pages </li></ul></ul><ul><ul><li>Average time on site of visitors </li></ul></ul><ul><ul><li>Search engine queries driving visitors to site </li></ul></ul><ul><ul><li>Geographic origin of visitors (from decoded IP addresses) </li></ul></ul>
    30. 31. Log File analysis <ul><li>Using log file analysis in the TIDSR project </li></ul><ul><li>Collecting web statistics from log file data </li></ul><ul><ul><li>Comparing: </li></ul></ul><ul><ul><li>- What kind of data was collected </li></ul></ul><ul><ul><li>- How often this was consulted </li></ul></ul><ul><li>Collecting raw log files </li></ul><ul><ul><li>Looking at: </li></ul></ul><ul><ul><li>- What search terms were used to find the resource </li></ul></ul><ul><ul><li>- The top referrer sites </li></ul></ul><ul><ul><li>- Access statistics </li></ul></ul>
    31. 32. Log File Analysis <ul><li>Top Search Phrases: Histpop </li></ul>
    32. 33. Scientometrics/Bibliometrics <ul><li>Scientometrics/Bibliometrics </li></ul><ul><li>Bibliometrics are typically measurements of document collections and are traditionally mainly derived from scientific databases </li></ul><ul><li>A common (partial) indicator of the importance of a document is how often it has been cited by other documents </li></ul><ul><li>Often-cited documents tend to be thought of as more valuable to the scientific community </li></ul><ul><li>Bibliometrics are less relevant outside science in the humanities and social sciences and are not reliable for individual articles </li></ul><ul><li>Bibliometric indicators can help to evaluate the impact of a collection of documents </li></ul>
    33. 34. Scientometrics/Bibliometrics <ul><li>How useful were Scientometrics/Bibliometrics in the TIDSR project? </li></ul><ul><li>Google Scholar, Scopus, ISI Web of Knowledge/Web of Science </li></ul><ul><li>Limited time scale </li></ul><ul><li>Examples from projects (and many were about the projects, rather than links to project materials) </li></ul><ul><ul><li>Histpop: 11 </li></ul></ul><ul><ul><li>BL 19 th Century Newspapers: 4 </li></ul></ul><ul><ul><li>BL Archival Sounds: 9 </li></ul></ul><ul><ul><li>BOPCRIS: 8 </li></ul></ul><ul><ul><li>Medical Journals: 17 </li></ul></ul><ul><li>Importance of trackable citation styles </li></ul>
    34. 35. For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/
    35. 36. Toolkit for the Impact of Digitised Scholarly Resources: Qualitative Methods Dr Eric Meyer & Dr Kathryn Eccles Oxford Internet Institute, University of Oxford
    36. 37. Overview <ul><li>Qualitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources: </li></ul><ul><li>• Interviews </li></ul><ul><li>Focus groups </li></ul><ul><li>User feedback </li></ul><ul><li>Referrer analysis </li></ul>
    37. 38. Interviews <ul><li>Interviews </li></ul><ul><li>Key project personnel </li></ul><ul><li>Related but non included institutional personnel </li></ul><ul><li>End-user communities </li></ul><ul><li>Subject specialists </li></ul><ul><li>Other stakeholders </li></ul><ul><li>Funding personnel </li></ul>
    38. 39. Interviews <ul><li>Project impacts revealed through interviews </li></ul><ul><li>Quality of undergraduate dissertation work improved by early contact with digitised primary sources </li></ul><ul><li>‘ Histpop made it possible to do a completely different project’ </li></ul><ul><li>Type of research projects being presented at conferences increasingly quantitative </li></ul><ul><li>‘ You do feel the ground shifting...’ </li></ul><ul><li>New possibilities for serendipitous research </li></ul><ul><li>‘ What I’ve found most enjoyable... is the serendipity’ </li></ul>
    39. 40. Focus groups <ul><li>Focus groups </li></ul><ul><li>Themes we covered: </li></ul><ul><li>General attitude to and knowledge of digital resources </li></ul><ul><li>Methods of searching for digital resources </li></ul><ul><li>Citation of digital resources </li></ul><ul><li>Issues of trust in discovering and using digital resources </li></ul><ul><li>Training and support in using digital resources </li></ul><ul><li>Resource specific questions – findability and usability   </li></ul>
    40. 41. User feedback <ul><li>Using User feedback within your own resource: </li></ul><ul><li>Keep a record of all the correspondence you receive from users </li></ul><ul><li>Periodically review the content of your user feedback (classify each response as negative, positive, both or neutral </li></ul><ul><li>Use contact with users to recruit focus group and survey participants </li></ul><ul><li>Use contact with users to gather information about how your resources is being used (Distance learning courses, undergraduate dissertations) </li></ul><ul><li>Use this extra feedback to enhance the impact of the resource on these communities </li></ul>
    41. 42. User feedback <ul><li>User feedback in the TIDSR project </li></ul><ul><li>Projects were reticent about providing email content from users </li></ul><ul><li>We were only able to gather anecdotal evidence from most resources </li></ul><ul><li>User feedback from Histpop: </li></ul><ul><li>We were allowed to send an email to be forwarded to users who had contacted Histpop </li></ul><ul><li>This allowed us to contact a range of users and ask them detailed questions about the resource </li></ul><ul><li>This yielded further information about the ways in which the resource was being used, particularly in the teaching and learning environments </li></ul>
    42. 43. Referrer analysis <ul><li>Referrer Analysis </li></ul><ul><li>Similar to the quantitative webometrics in approach, but requires a much more qualitative look at the resulting pages to try to find educational referrers (such as libraries and course reading lists) </li></ul>
    43. 44. For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/
    44. 45. TIDSR Survey Data Eric Meyer Christine Madsen OII 19 March 2008
    45. 46. <ul><li>Ave. time spent filling out survey: 17.5 minutes </li></ul><ul><li>83% completed the survey in under 20 min </li></ul><ul><li>714 started </li></ul><ul><li>550 completed (77%) </li></ul><ul><li>80% agreed to continue on to further questions  </li></ul>
    46. 47. Demographics
    47. 48. Demographics Average Age 45.3 Age Range 20-83
    48. 49. Degrees & Field
    49. 52. Awareness of Resources by Country
    50. 53. Awareness of Resource by Country
    51. 58. Have you ever published a piece based on your work in this collection? If so, how did you cite the collection?
    52. 60. How did you find this resource?
    53. 61. How did you find this resource? Histpop BOPCRIS BL News BL Sounds Med Backfiles Google 33% 33% 25% 19% 37% Prof society, discussion list or conference 25% 22% 26% 43% 27% Library 4% 22% 32% 10% 27% Colleague 25% 18% 11% 24% 6% Other 13% 4% 7% 5% 2% Student 0% 0% 0% 0% 0% Campus IT 0% 0% 0% 0% 0%
    54. 62. How do you use this resource?
    55. 63. How do you use this resource? Histpop BOPCRIS BL News BL Sounds Med Backfiles Online reference source 78% 81% 74% 66% 78% For personal interests 35% 49% 50% 46% 27% Download to use offline 26% 36% 40% 29% 57% Teaching resource 30% 30% 29% 39% 27% Manual analysis 17% 32% 38% 10% 39% Find materials to consult in person 4% 23% 18% 12% 39% Computational analysis 30% 6% 4% 2% 10% Reuse, remix or edit 17% 11% 9% 7% 0% Other 4% 0% 3% 5% 6%
    56. 67. Attitudes towards Digitisation
    57. 71. For more information, visit the toolkit at: http://microsites.oii.ox.ac.uk/tidsr/
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×