Your SlideShare is downloading. ×
Use of a follow-up survey for improvement of a digital library
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Use of a follow-up survey for improvement of a digital library

123
views

Published on

This paper will begin with an overview of digital libraries and usability and a brief discussion of recent trends in usability testing of digital libraries. That is followed by a brief discussion of …

This paper will begin with an overview of digital libraries and usability and a brief discussion of recent trends in usability testing of digital libraries. That is followed by a brief discussion of continuous quality improvement of a digital library, specifically the implementation of a follow-up survey to be completed by the general population who visit a publicly accessed digital library. Then a follow-up instrument is introduced.


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
123
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Kathryn Brockmeier kathryn.brockmeier@nebraska.gov Lincoln, Nebraska Use of a follow-up survey for improvement of a digital library This paper will begin with an overview of digital libraries and usability and a brief discussion of recent trends in usability testing of digital libraries. That is followed by a brief discussion of continuous quality improvement of a digital library, specifically the implementation of a follow- up survey to be completed by the general population who visit a publicly accessed digital library. Then a follow-up instrument is introduced. Definition of a digital library A digital library is a repository of digital content. Digital libraries are typically accessed online, using computers or other electronic devices. Librarians and information technology staff work together to create an environment where the user, whether it be the public, researchers, or employees, can easily search for. Definitions of a digital library abound. According to the Digital Library Federation (1998): Digital libraries are organizations that provide the resources, including the specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities. Definition of usability Usability testing is formal testing of a website whereby participants complete tasks and may be observed by the test administrator or tracked electronically. Users may be asked to respond to questions about these tasks. One of the most widely cited definitions of usability is found in the International Standards Organization’s (ISO) 9241-11, which purports that usability is “the 1
  • 2. extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (p. 2) There is a body of literature which studies usability of library websites (for example, Cervone, 2014; Letnikova, 2008). Since then, a growing number of studies have examined the usability of digital libraries (Comeaux, 2008; Dickson, 2006; Jeng, 2005; Jeng, 2006; Joo and Lee, 2010; Kahl and Williams, 2006; Kani-Zabihi, Ghinea, and Chen, 2006; Koohang, 2004; Koohang, 2004; Zimmerman and Paschal, 2009). Much of the literature is dedicated to usability testing of academic digital libraries. One exception is the Zimmerman and Paschal (2009) study, which included testing of a non-academic digital library. Users and subjects In many instances, the subjects were students or faculty at an academic institution, and the digital library was of an academic nature. Very little literature is devoted to the usability of digital libraries of cultural heritage institutions or non-academic libraries. Dickson (2006), however, did include members of the general public in that case study. Continuous quality improvement Usability testing and instrument creation in the literature have overwhelmingly focused on stages of creation of the digital library. But there is a need to monitor the digital library for continuous quality improvement. In the National Information Standards Organization’s (NISO) (2007) paper A framework of guidance for building good digital collections, Initiatives Principle 4 states that “a good digital initiative has an evaluation component.” (p. 91) Specifically: While output measures such as the number of items digitized can be useful, recent emphasis is on outcome assessment, which is concerned with how people, collections, organizations, and systems have been affected by the project. The 2
  • 3. evaluation plan should emphasize the importance of an ongoing two-way dialogue with key stakeholder communities. (p. 91) As Cervone (2014) states, “It is not possible to know if a website is truly meeting the needs of patrons if there is no on-going program of assessment.” (p. 11) Cervone (2014) adds, “As a mechanism for assessment and quality improvement, evidence-based practice is critical.” (p.12) Surveys Surveys, also known as questionnaires, are one way to capture the impressions of those using a digital library. A questionnaire can serve to gather data on several dimensions of use. There are various good reasons conducting a survey: “Although survey methods might not produce detailed strategies directly applicable to system improvements, they are nevertheless useful in gathering data from relatively large samples and diagnosing the current status of usability in general.” (Joo and Lee, 2010, p. 534) NISO (2007) states that surveys are a good way to measure inputs and outputs. A questionnaire is not necessarily usability testing. It is more a method to gather empirical data about a user’s impressions of the digital library interface and content. Most of these studies focus on testing via observable tasks and answers to questions related to those tasks. Some studies include a follow-up survey to gauge perceptions of the digital library. Several assess the methodology of usability testing on digital collections, including the creation of a standardized instrument grounded in empirical research. Some questions were open-ended while others were built on a Likert scale. There are several frameworks that propose underlying attributes, criteria, or constructs which contribute to ascertaining usability. Nielsen’s (1993) framework is one of the most widely cited usability models. It introduces five attributes—learnability, efficiency, memorability, errors, and 3
  • 4. satisfaction—as subordinate elements of usability. Frameworks that specify the usefulness of a digital library include: • Kani-Zabihi, Ghinea, and Chen (2006): Functionality, interface/usability, and content • Dickson (2006): Usefulness, effectiveness, learnability, and likability • Jeng (2006): Satisfaction, broken down into the areas of ease of use, organization of information, terminology and labeling, visual appearance, content, and mistake recovery • Koohang (2004): Simplicity, comfort, user friendliness, control, readability, information adequacy/task match, navigability, recognition, access time, relevancy, consistency, and visual presentation • Joo and Lee (2010): Efficiency, effectiveness, satisfaction, learnability Survey instrument questions The questions in the proposed instrument are drawn and modified from previous studies as well as generated to address several underlying constructs of usability: efficiency, effectiveness, satisfaction, and learnability. Wording and terminology are selected to be understandable by the lifelong learner, as opposed to the academician or college student. This question bank could be tested for validity and reliability and the list of questions narrowed down. 1. Overall, how satisfied are you with your use of this digital library today? • Very satisfied • Satisfied • Neither satisfied nor dissatisfied • Dissatisfied • Very dissatisfied 2. How did your use of this digital library meet your expectations? • Better than I expected • Just as I expected • Worse than I expected 3. How easy or difficult was it for you to navigate this digital library? • Very easy • Easy • Difficult • Very difficult 4
  • 5. 4. Were you able to find what you were looking for today? • Yes • No 5. Did you ever feel like you got lost on this site? • Yes • No 6. Did you browse the collection by subject? • Yes • No • I don’t know 7. How helpful were the search feature? • Very helpful • Helpful • Not very helpful • Not at all helpful • I didn’t use this feature 8. How useful were the help features? • Very helpful • Helpful • Not very helpful • Not at all helpful • I didn’t use this feature 9. Are you a (check all that apply): • Teacher • Student • Researcher • Lifelong learner • Other (please describe) ________________________________ 10. Comments about your visit today: ______________________________ In summary, there is a need for institutions to make an effort toward planning for evaluation of their digital libraries. While academic libraries have made strides in this area, there is room for cultural heritage and information service institutions to do the same. 5
  • 6. References Cervone, F. (2014). Evidence-based practice and web usability assessment. OCLC Systems & Services, 30(1). 11-14. Comeaux, D. J. (2008). Usability studies and user-centered design in digital libraries. Journal of Web Librarianship, 2(2/3), 457-475. Dickson, M. (2006). CONTENTdm digital collection management software and end-user efficacy. Journal of Web Librarianship, 2(2-3), 339-379. Digital Library Federation. (1998). A working definition of digital library [1998]. Retrieved from http://old.diglib.org/about/dldefinition.htm International Standard Organization. (1997). ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs), Part 11: Guidance on usability specification and measures. Jeng, J. (2005). What is usability in the context of the digital library and how can it be measured? Information Technology & Libraries, 24(2), 47-56. Jeng, J. (2006). Usability of the digital library: An evaluation model (Unpublished doctoral dissertation). Rutgers University, Brunswick, N.J. Retrieved from ProQuest. Joo, S. & Lee, J. (2010). Measuring the usability of academic digital libraries: Instrument development and validation. The Electronic Library, 29(4), 523-537. Kahl, C. M. & Williams, S. C. (2006). Accessing digital libraries: A study of ARL members’ digital projects. Journal of Academic Librarianship, 32(4), 364-369. Kani-Zabihi, E., Ghinea, G. & Chen, S. Y. (2006). Digital libraries: what do users want? Online Information Review, 30(4), 395-412. Koohang, A. (2004). Development and validation of an instrument for assessing users’ views about the usability of digital libraries. Issues in Informing Science & Information Technology, 1, 55-63. Koohang, A. (2004). Students’ perceptions toward the use of the digital library in weekly web- based distance learning assignments portion of a hybrid programme. British Journal of Educational Technology, 35(5), 617-626. Letnikova, G. (2008). Developing a standardized list of questions for the usability testing of an academic library web site. Journal of Web Librarianship, 2(2/3), 381-415. National Information Standards Organization (NISO). (2007). A framework of guidance for building good digital collections. Retrieved from http://www.niso.org/publications/rp/framework3.pdf Nielsen, J. (1993). Usability engineering. Boston: Academic Press. Zimmerman, D. & Paschal, D. (2009). An exploratory usability evaluation of Colorado State University Libraries’ digital collections and the Western Waters Digital Library web sites. Journal of Academic Librarianship, 35(3), 227-240. 6
  • 7. References Cervone, F. (2014). Evidence-based practice and web usability assessment. OCLC Systems & Services, 30(1). 11-14. Comeaux, D. J. (2008). Usability studies and user-centered design in digital libraries. Journal of Web Librarianship, 2(2/3), 457-475. Dickson, M. (2006). CONTENTdm digital collection management software and end-user efficacy. Journal of Web Librarianship, 2(2-3), 339-379. Digital Library Federation. (1998). A working definition of digital library [1998]. Retrieved from http://old.diglib.org/about/dldefinition.htm International Standard Organization. (1997). ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs), Part 11: Guidance on usability specification and measures. Jeng, J. (2005). What is usability in the context of the digital library and how can it be measured? Information Technology & Libraries, 24(2), 47-56. Jeng, J. (2006). Usability of the digital library: An evaluation model (Unpublished doctoral dissertation). Rutgers University, Brunswick, N.J. Retrieved from ProQuest. Joo, S. & Lee, J. (2010). Measuring the usability of academic digital libraries: Instrument development and validation. The Electronic Library, 29(4), 523-537. Kahl, C. M. & Williams, S. C. (2006). Accessing digital libraries: A study of ARL members’ digital projects. Journal of Academic Librarianship, 32(4), 364-369. Kani-Zabihi, E., Ghinea, G. & Chen, S. Y. (2006). Digital libraries: what do users want? Online Information Review, 30(4), 395-412. Koohang, A. (2004). Development and validation of an instrument for assessing users’ views about the usability of digital libraries. Issues in Informing Science & Information Technology, 1, 55-63. Koohang, A. (2004). Students’ perceptions toward the use of the digital library in weekly web- based distance learning assignments portion of a hybrid programme. British Journal of Educational Technology, 35(5), 617-626. Letnikova, G. (2008). Developing a standardized list of questions for the usability testing of an academic library web site. Journal of Web Librarianship, 2(2/3), 381-415. National Information Standards Organization (NISO). (2007). A framework of guidance for building good digital collections. Retrieved from http://www.niso.org/publications/rp/framework3.pdf Nielsen, J. (1993). Usability engineering. Boston: Academic Press. Zimmerman, D. & Paschal, D. (2009). An exploratory usability evaluation of Colorado State University Libraries’ digital collections and the Western Waters Digital Library web sites. Journal of Academic Librarianship, 35(3), 227-240. 6