Testing a archival DL prototype asking to users: the case of "una città per gli archivi"


Published on

Feliciati contribution to "User studies for Digital Libraries development" tutorial in JCDL2012 conference, Washington June 10th 2012

Published in: Technology, Education
  • Be the first to comment

Testing a archival DL prototype asking to users: the case of "una città per gli archivi"

  1. 1. TESTING OF AN ARCHIVAL DL PROTOTYPEASKING TO USERS:THE CASE OF “UNA CITTÀ PER GLI ARCHIVI” Pierluigi Feliciati University of Macerata (Italy) JCDL – Washington (USA) 2012.06.10
  2. 2. INTRODUCING THISDISTANT PRESENTATION (1) I am an ICT senior researcher in Macerata University (Italy), but my background covers archaeology, historical archives management and distance services development. I’m working especially on Web services quality Rarely archives on line (archival DLs) are approached from the point of view of final Feliciati - JCDL 2012 users, and input and output activities and environments are often mixed in an unique layer Users are probably not satisfied (but who knows it 2 for sure?)
  3. 3. INTRODUCING THISDISTANT PRESENTATION (2) User studies in the prototypal stage of a DL life- cycle are rare but crucial, to finalise the quality of services. The case study I will present is probably one of hugest projects in this sense in terms of methods and number of people involved This video presentation will ask you 15-20 Feliciati - JCDL 2012 minutes of kind attention, then Milena and Nicci will be happy to ask to all of your questions instead of me (unabled to move to Washington) 3
  4. 4. MAIN TOPICS1. Archives on line and Users: a museum of broken relationships2. Testing a prototype: Why? What? How? Results implementation policy3. The Città per gli Archivi (Italy) project4. Asking to users 1: internal staff5. Asking to users 2: target users / focus groups Feliciati - JCDL 20126. Asking to users 3: expert evaluation7. First conclusions and lessons learnt 4
  5. 5. ARCHIVES ON LINE AND USERS  Without the expert mediation of archivists what is based on the quality of access/use? Feliciati - JCDL 2012  A lesson to be learnt: Input is not output!  Archival standards create encoded contents, not automathically good displays  Available very few (American) studies on Archives on line quality (from the final users satisfaction point of view) 5
  6. 6. ARCHIVES ON LINE AND USERS Some typical issues:  Terminology: use of jargon Feliciati - JCDL 2012  Archival hierarchical structure: browsing is guided just by titles  Contents: too encoded  Searching: no intuitive tools with no subjects  Coverage: what could I find here? 6
  7. 7. TESTING A PROTOTYPE: WHY AND WHAT? User analysis could be very useful in the early stages of a project (prototype) Evaluation is called formative, when conducted while the DL is still under development Part of the data gathered during this stage will focus on the content and the functionalities of the DL and how these correspond to the users’ needs Feliciati - JCDL 2012 User needs could be checked asking directly to users, who can provide feedback that helps in the recognition of flaws in the interaction design. 7
  8. 8. TESTING A PROTOTYPE: WHY AND WHAT? Morever, expert opinions can provide assistance when the DL prototype has been refined, giving their insights on the quality features of the system Feliciati - JCDL 2012 8
  9. 9. TESTING A PROTOTYPE: HOW? To directly involve users in the prototype step, we could launch a survey (e.g. using on-line questionnaires), if we have already a designed community to address On the other hand, we have to organize as more focus groups as possible trying to address user panels who could be interested to the future DL we are working on Feliciati - JCDL 2012 Finally, the expert study could involve both experts on content and functionality, to gather advanced qualitative data on performance, usability and 9 expected success of our project
  10. 10. TESTING A PROTOTYPE: DEFINING A RESULTSIMPLEMENTATION POLICY During formative evaluations, the researcher may return to previous stages of development and adopt their findings, thus altering the status of the DL. Feliciati - JCDL 2012 10
  11. 11. THE CITTÀ PER GLI ARCHIVI PROJECTUna città per gli archivi (A City for Archives CPA) is a project supported by two banks foundations, with the collaboration of other institutions (archival and city administration)Its mission is to improve the preservation, valorization and full access for the most important Archives related to the contemporary history of the city of Bologna (Italy) Feliciati - JCDL 2012The project focused not only on institutional archival records but as well on audiovisual and photographic archives 11This huge cultural project started in 2007
  12. 12. THE CITTÀ PER GLI ARCHIVI PROJECT More then 200 Archives have been preserved and described, made of 300.000 records, paper documents, photographic and graphic materials, sound and audiovisual recordings 70 archivists were involved 200.000 digital objects created A Web portal (an archival DL…) was developed to enable users to access the described archives and the digital reproductions Feliciati - JCDL 2012 Export of metadata towards Italian National Archival System, Italian Cultural Portal and Europeana was scheduled 12
  13. 13. THE CITTÀ PER GLI ARCHIVI PROJECT A wide program of restoration and sorting out of Archives at risk of loss, redaction of guidelines for descriptions and authority control Feliciati - JCDL 2012Thus, during theprototypal step of theproject, a user andfunctionality study 13was launched
  14. 14. CPA portal prototype a snapshot Feliciati - JCDL 2012 1414
  15. 15. CPA USER STUDYASKING TO USERS 1: INTERNAL STAFF For the quality archival descriptions, the competence of archivists is crucial Rarely those input content providers are directly involved in the development of output environments The CPA users and functionalites study planned 2 special focus groups, inviting together archival staff and DL developers (beginning and end of the study) Those meetings were moderated according to a Feliciati - JCDL 2012 special protocol, mixing typical focus groups method with that of a moderated brainstorming (archivists vs engineers) 15
  16. 16. CPA USER STUDYASKING TO USERS 2: FOCUS GROUPS5 focus groups were organized to collect precious data about potential target users needs and satisfaction against CPA portal  1 with high school students (16 pople, aged 16- 19)  2 with university students (total 20 people, aged 23-29) Feliciati - JCDL 2012  1 with general public (16 people, aged 39-63)  1 with archivists and cultural heritage (LIS and museums) professionals (16 people, aged 29- 16 54)
  17. 17. CPA USER STUDYASKING TO USERS 2: FOCUS GROUPS All the focus groups were based on the same detailed protocol, asking 100 minutes and including 6 sections: 1. Demographic and ICT skills questionnaire 2. CPA prototype presentation 3. First impressions and discussion 4. 5 tasks, whose results were recorded asking Feliciati - JCDL 2012 3-4 questions for each task 5. Advanced impressions and discussion 6. Final discussion 17
  18. 18. CPA USER STUDYASKING TO USERS 3: EXPERT EVALUATION Finallyan expert study activity was scheduled for next September, when the prototype will be almost ready The 7 experts involved are: 3 archivists (2 university professors and a City Archive director), 3 DL and metadata experts, 1 ICT university professor Feliciati - JCDL 2012 The protocol adopted is focused on performance, functionality, usability issues, terminology, quality of content 18
  19. 19. FIRST CONCLUSIONS(AND LESSONS LEARNT) Archives on line are special DLs: they open peculiar issues to face users expectations and guarantee satisfaction Users study are crucial, to finalise the DLs and ensure a (really) good quality The prototypal stage of the project, when possible, offers a good occasion for evaluation and assessment activitis based on users involvment The Italian CPA project shows that such activities Feliciati - JCDL 2012 are not that expansive, they need an expert (and neutral) coordination, a good organizational staff and a network of people and institutions to be involved 19
  20. 20. THANK YOU FOR YOUR ATTENTION,WISH YOU ALL A GREAT WORK TOGETHER!pierluigi.feliciati@unimc.itThis presentation is licensed under: CC-BY-SA