Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serving researchers in a self service world

Presentation for Museums and the Web 2018 (Vancouver)
April 19, 2018

In a world in which a family historian can type her grandfather's name into to start building a family tree, and a journalist can Google to download public domain images, where do the collections searches, online tools, and APIs that museums and archives provide fit in? This paper outlines strategies for better serving people who are looking for the knowledge and expertise within your collections and staff. At the National Archives and Records Administration of the United States, we undertook a significant user experience (UX) research project to better understand the online experiences of professional researchers, family historians, and history enthusiasts. Research methods included audits of existing user data (e.g., Google Analytics, survey data) as well as new user interviews, usability testing, a survey, and a landscape analysis. Key findings include the fact that researchers struggle to complete their tasks using existing online tools; people researching family history are particularly unsatisfied and in need of better support; and all audiences require just-in-time help and appropriate orientation to archival research. A major challenge highlighted by this research is how to meet user expectations for item-level records while providing access to digitized records at massive scale.

Read full paper:

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Serving researchers in a self service world

  1. 1. Serving researchers in a self-service world Dana Allen-Greil @danamuses Museums and the Web - April 29, 2018
  2. 2. In the next 15 minutes 1. How and why the National Archives is investing in UX research 2. Key findings that might apply to similar organizations 3. Areas for further research and collaboration Paper:
  3. 3. About the National Archives The National Archives (NARA) is America’s record keeper. We hold hundreds of miles of film and tape, 14 million still photographs, and billions of letters, drawings, maps, treaties, posters, and other documents that tell the stories of America’s history as a nation. Workers unloading Veteran’s Bureau Records (June 1936)
  4. 4. The Problem. ● Digital products developed over time without a strong (or unifying) user-centered framework ● Low customer satisfaction overall ● Lots of assumptions about researchers A frustrated driver sits through a traffic jam in Herald Square (May 1973)
  5. 5. Satisfaction scores by audience Average Public service gap D
  6. 6. Most users are unable to accomplish their task How might we we help more people accomplish their tasks? …(Yes and) how might we help people better define research tasks so they can be accomplished? Did you accomplish what you wanted to do today on the site?
  7. 7. Core products for researchers
  8. 8.
  9. 9. Catalog
  10. 10. Consistently declining customer satisfaction Scores for the Catalog fell 16 points in 7 months. What happened?
  11. 11. 11 History Hub Learn more about how you can get involved:
  12. 12. Why is this issue urgent now? Our strategic plan dictates: ★ By FY 2024, NARA will digitize 500 million pages of records and make them available online to the public through the National Archives Catalog. ★ By FY 2025, NARA will provide digital, next-generation finding aids to 95 percent of the holdings described in the National Archives Catalog. ★ By FY 2025, NARA will have 1 million records enhanced by citizen contributions to the National Archives Catalog.
  13. 13. Why invest in UX? ❏ In the past, NARA designed and developed products without a user-centered framework. These legacy products are now very difficult to change due to rigid procurement and development processes. ❏ We must figure out not only how to provide access to a massive number of archival resources but also how to do it in a way that is usable and useful at scale. ❏ Our research is designed to help us use our limited resources most efficiently and effectively to support successful user-centered products.
  14. 14. UX add to strategic plan! Added “Build capabilities in design and user experience, to better understand user needs.”
  15. 15. Research Goals To better understand today's online marketplace for researchers and how we can best serve their needs. 1. Assess the current level of audience engagement 2. Understand NARA’s high-value audiences 3. Identify new, aspirational audiences 4. Review peer organizations for best practices, features, and trends 5. Recommend ways to improve user experience on current digital platforms 6. Identify new opportunities to engage audiences
  16. 16. Research Methods 1 Usability testing/ observation 9 remote users were given task-based questions to carry out on, Catalog, and History Hub, and asked to explain their choices 3 Comparative analysis Reviewed 12 websites to compare features, functionality, and content to understand the wider online research services landscape 5 Review of existing data -Customer satisfaction survey - Google Analytics data - Search keyword metrics - Social media analytics -Textual analysis of Contact Us emails 2 User interviews 19 interviews (30-60 min) with genealogists, in-person researchers, online researchers, electronic records experts, and citizen archivists 4 New survey of researchers 1,847 responses
  17. 17. Personas
  18. 18. Audiences vs. Tasks “Online researcher” is a diverse category. We should optimize digital services to meet the needs of specific user tasks rather than broad audience categories.
  19. 19. Online Researchers Expect Online Records First ● Only 25% of online researchers have ever visited a NARA research facility. ● 86% said online research (as opposed to in-person research) is important or very important. ● Most researchers self-identified as experienced/very experienced. World War I poster (1943-1945)
  20. 20. Not just a “learning curve” Returning visitors are no more satisfied than new visitors. ➔ Suggests that no new knowledge is gained in the first visit that makes subsequent visits easier to execute. ➔ Solution can’t just be about “better orientation” or more research guides (although just-in-time support would help). ➔ Need to fix fundamental UX problems so our digital services are usable and useful.
  21. 21. Thanks, Google. Keyword topics and names of people provide relevant results.
  22. 22. People researching family history are particularly unsatisfied ● At least 1/3 of our online researcher audience ● Only 24% of this audience was able to complete their task ● Many visit or similar sites first, where they are accustomed to user-friendly features like person name search and highlighted results ○ Popularity of DNA heritage tests is driving growth of this audience ● Most do not identify with the term “genealogist” or even
  23. 23. Orientation Even experienced researchers need help orienting to your collection: ● scope and size ○ What is an archive? ○ What’s in YOUR archive? ● how is it structured ● what is digitized and what isn’t Recommended reading: Whitelaw, M. “Generous Interfaces for Digital Cultural Collections”
  24. 24. Wizards
  25. 25. Just-in-time help Guidance (in a PPT file!) on a different website is not helpful in the middle of a search How can we help people when they get stuck? Image:
  26. 26. How might AI help us support research? 1. Item-level expectations - Our systems and processes are structured around descriptions of records groups and series. While we have millions of digitized items, we don’t have good metadata to support their discoverability (e.g., by topic, by person, by what is viewable in an image). How might AI (artificial intelligence) help bridge this gap? 2. Finding needles in the haystack - How might machine learning make search results more relevant and get smarter over time based on use? (This is especially interesting for thinking about the tsunami of email records coming soon.) 3. Providing in-line support - Can chatbots provide useful customer service for common researcher requests? How can we re-purpose knowledgebases like History Hub to scale up personalized support?
  27. 27. Next steps Front end 1. Consolidate products (distinction between websites unclear to users) 2. Show digitized items first by default (DUH!) 3. Build streamlined interfaces to guide users in specific research tasks 4. Orient audiences who are underprepared for archival research, including a new visual overview of holdings 5. Offer in-line help and support Back end 1. Transform our procurement and development to more agile methodologies with UX work baked in to the process 2. Leverage AI and machine learning to support discoverability and tailored support 3. Expand and improve API (including faster performance, better image delivery, and more coherent data structure)
  28. 28. How can we better leverage our API? To kick the tires on our API and uncover limitations (performance, data structure, documentation), we have developed 6 prototypes: 1. Treemap/overview of holdings, a “generous interface” that communicates scope and breadth 2. Facebook Messenger chatbot which encourages tagging of records 3. "When Am I?" game for engaging with records by time period (and adding tags) 4. Voice-activated Alexa skill which enables exploration of recipes 5. Drupal image selection and cropping tool (open source module) 6. Drupal CAPTCHA module that facilitates tagging records (open source module)
  29. 29. Facebook Messenger Chatbot How might we massively scale up crowdsourcing of metadata? (Our API is read/write to include citizen-contributed tags, transcriptions, and comments.)
  30. 30. Let’s talk! Dana Allen-Greil @danamuses