Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Julie Fukuyama & Simon Tanner: Developing impact assessment indicators – making a proposal for the UK Web Archive

317 views

Published on

Presented at the IIPC Web Archiving Conference, 6-7th June 2019, Zagreb, Croatia.
http://netpreserve.org/ga2019/programme/wac/

This paper presents the results of a study to examine, determine and propose the optimal approach to develop impact assessment indicators for the UK Web Archive (UKWA). In the United Kingdom, legal deposit libraries collaboratively operate a nationwide web archiving project, the UKWA, which has collected over 500 TB of data and is growing by approximately 60–70 TB a year. At the same time, UK publicly funded organisations face reduced funding and the challenge of convincing funders to finance their archival function by undergoing evaluations of their services’ values.

Under such circumstances, a proper assessment of the values and impacts of web archiving is a point of discussion for cultural heritage organisations. To the best of the authors’ knowledge, there has not yet been a comprehensive assessment or evaluation of the UKWA conducted. Thus, this paper seeks to answer the research question: “What would the indicators of impact assessment for the UKWA be?” As a result, we propose a set of impact assessment indicators for the UKWA (and web archiving in general) with broad strategic perspectives including social, cultural, educational and economic impact.

This study examines and proposes the optimal approach to develop impact assessment indicators for the UKWA. The research began by analysing the literature of impact assessment frameworks for digital resources and the types of impact in related fields. Primarily drawing from Simon Tanner’s Balanced Value Impact Model (BVI Model), this research then proposes impact indicators for the UKWA and develops an impact assessment plan consisting of three stages: context setting, indicator development, and indicator evaluation.

This paper will present the method and results of the study. Firstly, it identified the UKWA’s foundational context, the mission, the principal values and the key stakeholder groups. The research project prioritised focal areas for the archive that seem most advantageous for stakeholders and aligned with Tanner’s Value Lenses. Secondly, we proposed the UKWA impact assessment indicators; scrutinising existing indicators and various evidence collection methods. In the third stage, the developed indicators’ functionality was checked against set quality criteria and then tested through semi-structured interviews and survey submissions with 8 UKWA staff members.

Finally, the paper presents the thirteen potential indicators for the UKWA. Based on the lessons learned, presenters will also make recommendations for organisations which recognise the necessity of undertaking impact assessments of their web archives.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Julie Fukuyama & Simon Tanner: Developing impact assessment indicators – making a proposal for the UK Web Archive

  1. 1. Simon Tanner King’s College London simon.tanner@kcl.ac.uk @SimonTanner Julie Fukuyama National Diet Library ju-fukuy@ndl.go.jp IIPC Web Archiving Conference 2019, Zagreb, Croatia, 7 June 2019 & Developing impact assessment indicators: Making a proposal for the UK Web Archive
  2. 2. Background
  3. 3. What would the indicators of impact assessment for the UKWA be? Research Question
  4. 4. The Balanced Value Impact Model http://simon-tanner.blogspot.com/2017/10/BVI-Model-V2.html Approach to develop the UKWA IA indicators
  5. 5. Approach to develop the UKWA IA indicators
  6. 6. The UKWA’s mission gathered from the website, the government’s guidance on the regulations, and the Agency for the Legal Deposit Libraries (ALDL) Sorted out mission of the UKWA Activity Objects How For whom UKWA - Collect - Capture - Save - Preserve - Give permanent access - All of the UK Web Space - Key UK websites - Assets: html pages, images, pdf’s, video’s - Information that is only available on the Web - Significant witness to our times - Highly ephemeral: altered, disappear - Annually - Comprehensively - Future generations - Current & future researchers (academic or otherwise) Guidance - Archive - Preserve - National collection of non-print publications - Efficient - Future generations Agency - Collect - Preserve - Make it available - Nation’s published output - Nation’s intellectual record - Future published heritage - Systematically - Future generations - Readers within the deposit libraries
  7. 7. User segments and their benefits
  8. 8. User segments and their benefits
  9. 9. Balancing perspectives and value lenses for the UKWA
  10. 10. Focus areas of the UKWA: areas to be assessed
  11. 11. Approach to develop the UKWA IA indicators
  12. 12. Focus areas and elements of success criteria
  13. 13. Focus areas and elements of success criteria
  14. 14. Elements of success criteria Success criteria Effectiveness/ease of information search Users find it effective/easy to search Collection coverage of existing websites The UKWA collects the whole UK websites Usability of datasets Users find the datasets as usable/meaningful Trustworthiness of archived websites Archived websites prove to be unchanged/secure Meaningful/interesting collections Users find them meaningful/interesting collections Outcomes of R&D Researchers: find research questions, use data, publish the results Developers: find issues, develop tools, tools become widespread Economic benefits The UKWA becomes seeds of grants, produces direct/indirect outcome/benefits Influence on web archiving community Stakeholders adopt/apply/base the UKWA technologies/research results Detailed success criteria of the UKWA
  15. 15. Questions for the UKWA
  16. 16. List of the impact assessment indicators
  17. 17. Test of the quality of the impact assessment indicators
  18. 18. Test of the quality of the impact assessment indicators
  19. 19. Feedback collection
  20. 20. Feedback from UKWA staffs
  21. 21. Feedback from UKWA staffs
  22. 22. Feedback from UKWA staffs 2,50 1,06
  23. 23. Feedback from UKWA staffs
  24. 24. The final impact assessment indicators list
  25. 25.  To adopt our indicators as a starting point to consider impact indicators  To discuss among partners and with stakeholders  To focus upon measuring what you can measure by conducting small scale internal pilot impact assessments Recommendations
  26. 26. Thanks to: • Nicola Bingham (The British Library) • Helena Byrne (The British Library) • The staff of the UK Web Archive Acknowledgements

×