The document details the development of the First World War Poetry Digital Archive, aimed at making primary source materials accessible for research and education while engaging diverse user groups. It outlines key phases of user engagement, including outreach, focus groups, and user testing to gather feedback for site design enhancements. The project successfully collected over 6,500 items from public contributors and received extensive positive responses from educators and users, highlighting its comprehensive nature as an archival resource.

1. Web usability testing in practice: a case study from the First World War Poetry Digital Archive Kate Lindsay, Project Manager Oxford University

2. Overview Background User Engagement User Testing The Great War Archive Initiative: engaging non web-savvy users Lessons Learned

3. Background 1996: JISC funded the Virtual Seminars Project Drew together primary materials on Wilfred Owen scattered across a range of archives and an array of contextual resources (WOMDA) Web based tutorials to advance the possibilities of traditional teaching One of the first multimedia collections designed specifically as a teaching resource -more than 1 million hits. Funding received for Apr 07 - Mar 09 and Oct 08 - Sept 09 to expand and enhance the archive (JISC Digitisation Programme).

4. Goals of the First World War Poetry Digital Archive To make primary source material available to researchers and students which would otherwise be difficult to access To place the material in context thus widening the site’s appeal to include history, military history, women studies, and media studies To add value by providing tools and resources for research and education Proof of concept: The Great War Archive

5. The Archive Collections c. 4000 digital images of primary source material (manuscripts, letters, service records) from Edward Thomas, Robert Graves, Isaac Rosenberg, Vera Brittain and Roland Leighton on launch. Online corpora of the full-texts of the poems. c. 500 Multimedia objects (photographs, audio and video) from the IWM. Publications of War (recruitment posters, trench papers etc.) c. 7000 items submitted by the general public of items originating from the Great War.

6. User Engagement and User Testing The story of successful impact To ensure that the archive had an impact on the research, teaching and study of the First World War and the literature that it inspired we needed answers to the following questions: Who would be the archive’s key users? What are the requirements of a web archive for these user groups? How are the users likely to find out about the archive? How would they want to engage with it?

7. Defining User Groups Researchers HE Lecturers Teachers The General Public UG Students (Who have stuff from WW1!!!) Military Historians School Students Parents Family Historians

8. April ‘07: Building Communities AIM: To develop a user base early on to provide requirements and feedback throughout the development lifecycle. Enlisted ‘Friends of the Project’ who would disseminate our work To raise public awareness produced podcasts with key experts and public figures on the First World War and released these into itunes. Discussion groups, Facebook, mailing lists set up early on

9. April ‘07: The Steering Group Formed a user-based steering group made up of researchers, lecturers and teachers, and technical experts. We gained an in depth knowledge of current research and teaching processes for the subject area and in doing so derived requirements and user feedback in a focus group fashion. Act as brokers and champions for the target user groups

10. Nov ‘07: Teaching WW1 Literature Workshop AIM: To understand how teachers' experience and perceive teaching, learning of the subject, and online archives / ICTs Run in November 2007, before development had commenced on the web interface or the production of educational resources. Focus was on literature not technology Collected scenarios of how First World War Literature is currently taught and from this emerged the foundations for the design of the archive and the types of resources that were to be developed.

11. May ‘08 and Sep ‘08: User Testing AIM: To collect specific feedback on the online archive interface and functionality and act upon it. Phase 1: Focused on 7 researchers and looked at the more advanced search and viewing functionalities Implications for metadata Implications for system development Phase 2: Focused on 10 teachers and students and looked at the browsing functionality and educational materials Implications for web site design and structure Implications for content, especially educational resources

12. Methodology Scenario-based online survey (in surveymonkey.com) guiding participants through a series of tasks in the archive. Participants went through the survey with two researchers and encouraged to ‘talk-aloud’ Additional notes taken by the researcher Screen capture using Adobe Captivate Participants drawn from our communities, interviews lasted c. 2 hrs and took place in the participants office / home. All data collected and the resulting actions documented in a public wiki http: //wiki . oucs .ox.ac.uk/ltg-public/WW1projectUserFeedback

13. Run between March 08 and June 08. A ‘Community Collection’ to harvest digital versions of items originating from the Great War held by members of the general public. Aim: To create a digital collection of worth at low cost by negating the need for institutional digitisation and metadata creation. Targeted specifically at genealogists, military collectors and enthusiasts, and the elderly. The Great War Archive Initiative

14. Looked at case studies of similar initiatives. Simple submission process, no need for registration. Contributors asked to agree to basic terms and conditions of the JISC/HEFCE Model license. Enter basic metadata and attach files / enter a story . http://www.oucs.ox.ac.uk/ww1lit Building the submission web site

15. Reading non-technical users: submission days A roadshow to offer on the spot digitisation and advice. Held at libraries, museums and archives across the country. A submission day pack was made available for organisations who wished to perform their own event.

16. Over 6500 items collected. Only 1 item rejected. Costs came in at c. £3.50 per image BUT we did perform digitisation on items posted to us (we didn’t have the heart to say no!). Highly commended for the THE Awards and UCISA Award for Excellence. Submission software (CoCoCo) released to the open source community. A Flickr Group continues to collect items. The Great War Archive Initiative: Outputs

17. November 11th 2009: Launch Reported in over 30 publications inc. The Telegraph, the Times Higher, BBC Online and Radio, Guardian Books. Overall, this is one of the most comprehensive (if not the most comprehensive) archival sites on the web. It is also one of the best attempts to navigate the museum/archive/website divide that I have seen. I didn't expect to get so engaged with this tutorial but found it fascinating and learned a lot myself. This illustrates well how adding learning activities to content (however rich) can increase engagement at a deep level. I will try to spend more time with this tool at a later date as I intend to use this collection to introduce the war poets to my home educated son. That's how impressed I am with this work. Well done to the team – the amount of work and passion is very evident. Reviews in History, Jan ‘09 JISC Educational Resources Review Over 20,000 visits Podcasts: 20,000 downloads

18. User Feedback I love the site and my students love using it - only today I had Year 13 group using it to research particular forms of writing for our own war wiki on our VLE. A couple of my students became totally addicted to deciphering Vera Brittain's handwriting and finding out what was next in her letter to Leighton and downloaded the letter to take home! Go and explore The First World War Digital Poetry Archive and The Great War Archive, both based at the University of Oxford. Go even if you don’t care about the First World War, just to revel in the high quality of the thought that has gone into creating such a wonderful resource. Go to have a look at the more than 6500 artefacts submitted by members of the public which are all now freely accessible and searchable. This is the thing that has made me happiest this week. Natalie Usher, Teacher Dan Todman, Historian The students I demonstrated the Archive to, in a lecture in February, were very interested in it. Most students seem to find online material far more appealing than printed material, but the content of web sites is often less than academic. It's very good to be able to refer to students to a web site of such quality from a sound academic source. Andrea Peterson, Lecturer

19. March 09 - Sept 09: Continuing User Engagement Use Google Stats to analyse site traffic and act upon it! Two workshops with teachers and lecturers in 2009 to develop and share educational materials using the archive’s resources. Collect further feedback Establish a strong community Use of social media software to take the archive to the users and drive traffic to the site. Twitter Facebook fan page Culture 24 page Flickr Group

20. Outcomes and Lessons Users played a valuable part in the design of the archive’s interface. User engagement activities need to be built into schedules right from the start. Early requirements gathering activities will mean less time spend on development and alterations later on. Many users do not know how a technology can benefit them until they can see its possibilities firsthand (e.g. Web 2.0) - Show them rather than ask them! Think of social media as a way to take you site to users as well as pull users to your site.

21. Further Information Kate Lindsay, Project Manager [email_address] http://www. oucs .ox.ac.uk/ww1lit http: //wiki . oucs .ox.ac.uk/ltg-public/WW1projectUserFeedback