Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MCN2017 | From Research to Action: Translating User Feedback into Digital Products v2

231 views

Published on

As museums and cultural institutions work to create more user-friendly products, we conduct research that sometimes ends up buried as we talk more about what was created. Whether it’s prototyping and testing along the way, or a summative evaluation of one project leading to changes for the next one, how do we learn from these findings?

Desi Gonzalez, The Andy Warhol Museum, Pittsburgh
Jenn Phillips-Bacher, Wellcome Collection, London
Susan Wigodner, The Field Museum, Chicago

Published in: Technology
  • Be the first to comment

  • Be the first to like this

MCN2017 | From Research to Action: Translating User Feedback into Digital Products v2

  1. 1. From Research to Action: Translating User Feedback into Digital Products #MCN2017-T4 Desi Gonzalez, Jenn Phillips-Bacher, and Susan Wigodner November 9, 2017
  2. 2. From Research to Action #MCN2017-T4 2
  3. 3. Planning to Redesign Fieldmuseum.org #MCN2017-T4 Susan Wigodner @swigs27 The Field Museum, Chicago @FieldMuseum MCN 201703
  4. 4. 4 https://digitalstories.wellcomecollection.org
  5. 5. 5 ● Web & Digital Communications Director ● Project Manager ● Content Developer ● Project/Content Coordinator ● Social Media Manager ● Front-End Developer (currently vacant) ● UX/UI Designer (currently part-time) ● Media Producer Web & Digital Communications Marketing & Public Relations Information Technology Exhibitions Institutional Advancement Science & Education Website (Product) Digital Content Social Media Microsites (On Hold) + Audience Insights and Research Manager
  6. 6. Upfront User Research 6 Worked with Expose Your Museum from October 2016 - January 2017 on: ● Website pop-up surveys (20K respondents, 10 days) ● Analytics review ● Staff discussion groups (8 groups, 8-12 people each, 90 minutes long) ● User testing (12 sessions, 45 minutes long) Shared research reports via All Staff email and Intranet in Spring 2017
  7. 7. Setting Priorities 7 Fieldmuseum.org Redesign #MCN2017-T4
  8. 8. 8 Google Analytics Jan - Dec 2016 Fieldmuseum.org Pageviews by Section
  9. 9. 9
  10. 10. 10
  11. 11. 11
  12. 12. 12 Survicate Dec 2016 Fieldmuseum.org Pop-up Surveys
  13. 13. Setting Priorities 13 Rebranded site needs to launch Spring 2018 Phase 1: General Visitors (visit planning, programs and events, “About” section) Phases 2+ (Priority Order, Schedule Not Yet Set) ● Supporters (donors and involvement opportunities) ● Revenue and Museum Promotion (press, events, traveling exhibitions) ● Educators ● Researchers & Scientists
  14. 14. Setting Goals 14 Fieldmuseum.org Redesign #MCN2017-T4
  15. 15. What Are We Working Towards? 15 User Testing / Pop-Up Survey Takeaways Selected Goals In testing, users: ● displayed an over-reliance on search ● took navigation language very literally ● often struggled to complete defined tasks Develop user-focused site architecture and content 60+% of users classify reason for visiting as “Planning a Visit” In testing, users: ● requested clearer information about timing and pricing for exhibitions and events ● struggled to find information about upcoming exhibitions Prioritize easy access to visit-related content 25% of users classify reason for visiting as “Personal Interest” or “Browsing” In testing, users: ● appeared interested in the collections, but expressed a desire to have more generalist access Encourage discovery about science and natural history
  16. 16. Staff Discussion Groups 72 staff members from across all departments 16 Agreement On: ● Lack of brand and content consistency ● Challenge of varied and overlapping audiences ● Need for more defined publishing process ● Staff hear that people can’t find what they need; rely heavily on search ● Concerns about accessibility ● Ticket options confuse visitors, online purchase platform is difficult to use Disagreement On: ● Priorities, particularly with how homepage is used (visit information versus highlighting science content) ● Who should be able to publish website content ● When content can be retired ● Use and utility of media ...and on...
  17. 17. Guiding Principle Heard from Staff Design Integrity “...no one has this unified view of how the Museum should be doing things” “I have heard a lot of people say that there are places where it's very well done but you have to pick through the rest of the bodies to get to those.” Ease of Use “I'm on the phone a lot with clients and I really have to walk them to the exact page I want them to see and it's time consuming for them.” “...oftentimes it is content that actually is on the site, but it's too many clicks away, or too many clicks deep for them to get to it.” Consistency “I've noticed that [on] certain pages, someone will add content, and that same content will be on an entirely separate page, but then when you go back to update it you may not think to update the second page that has that same content.” “We have a lot of different people doing a lot of different things, and at some level, from my perspective, and I know this leads to chaos, but that's what we ought to be celebrating with respect to the website. There ought to be a ton of content.” Communication “We do need to have some high tiers on the site that are very professionally done, but also we need content. Up to the minute content is going to maybe not be ideally perfect.” “another [thing] I think that we struggle with ...is the way that we see [ourselves] organized, and our work, and maybe a slightly different way that we would want to project and share that with partners.” Accessibility and Inclusivity “...we've thought some about that, how to make this accessible to people who may need a wheelchair, may need some other help when they come.” “It feels like we're looking at it from our perspective of, "Here's everything we do," rather than looking at it from the outside in, going, ‘Who's the audience that's looking in?’" How We Want to Work 17
  18. 18. Digital Stories: Using Research to Iterate #MCN2017-T4 Jenn Phillips-Bacher @MrsAudiac Wellcome Collection, London @ExploreWellcome MCN 2017018
  19. 19. 19 https://digitalstories.wellcomecollection.org Wellcome Collection
  20. 20. 20 Digital Content Digital Platform Digital Production Photography Collections & Research Live Programme Exhibitions Publishing Marketing & Communications The Product Team ● Full-Stack Developer ● 2 Front-End Developers ● UX Designer ● User Analyst ● Visual Designer ● Product Manager
  21. 21. 21
  22. 22. 22
  23. 23. 23 Digital Stories, Version 1: The Approach Big design upfront No user research Agency and freelance contractors Summative evaluation at end
  24. 24. Summative Evaluation Findings: Usability 24 Finding: ‘Find out more’ is a distraction
  25. 25. Summative Evaluation Findings: Usability 25 Finding: Source material doesn’t enhance the narrative
  26. 26. Summative Evaluation Findings: Usability 26 Finding: Support wayfinding Offer way to learn about new stories
  27. 27. Summative Evaluation Findings: Analytics 27 69% visit 1 or 2 chapters only <5% explore an entire story in one session
  28. 28. Summative research is a to-do list in waiting.
  29. 29. Digital Stories: The Return 29 What’s different this time? Multidisciplinary team with permission to make product decisions Continuous discovery Small batch and large batch testing Small numbers for initial concept testing and usability (guerrilla research) Large numbers for data-driven experiments (AB testing, prototypes, remote surveys) Rethink how we produce, publish and position Digital Stories
  30. 30. 30 A/B testing and prototypes help us decide
  31. 31. Chapter Navigation Prototype: Drop Down 31
  32. 32. Chapter Navigation Prototype: Horizontal Scroll 32 Action: We built this version
  33. 33. Sticky Navigation 33 Default state Sticky state
  34. 34. Sticky Navigation 34 Action: Non-sticky Retest on widescreens
  35. 35. Email Signup Experiment 35 ● Mobile visitors only ● Ran experiment for 3 weeks ● Used Mailchimp’s free account to collect email ● More staff than public signed up ● Open rate: 62.5%-75% ● Click rate: 0%-29% Action: We parked email
  36. 36. 36 Guerrilla testing and interviews help us understand and refine
  37. 37. Perception of Content and Usability 37 Action: Changed titles on next story (The Outsiders) “What’s an analog story?” “I’m confused by the title’s meaning” “I would like some sort of summary at the start that told me what I could expect from being on this page.”
  38. 38. 38 Make decisions better. Make better decisions.
  39. 39. User Research at The Warhol #MCN2017-T4 Desi Gonzalez @desigonz The Andy Warhol Museum, Pittsburgh @thewarholmuseum MCN 2017039
  40. 40. 40
  41. 41. 41 ● Manager of Digital Engagement Digital Engagement Marketing & Communications IT (shared by four museums) Learning & Public Engagement Strategic Initiatives Rights & Digital Imaging
  42. 42. Out Loud The Warhol’s Inclusive Audio Guide #MCN2017-T4 MCN 2017042
  43. 43. 43 Out Loud The Warhol’s Inclusive Audio Guide
  44. 44. User Research: Who 44 ● 3 user-experts ● Advisory team ● 20 Accessibility Meetup members ● 3 user-experts ● Advisory team ● 20 Accessibility Meetup members
  45. 45. User Research: How 45 ● Interviews ● Paper and interactive prototypes ● Usability tests
  46. 46. From Interviews to Action 46 Quote Design principle Design decision “I want a little more independence” during a visit Provide a path with a choice Split up audio content into smaller, modular stops and allow visitors to dive deeper if they’d like to learn more “I usually go to museums with other people; it’s a social thing.” Build as a social experience Develop audio content that sparks discussion Provide headphone splitters for visitors to share devices “I expect a back button in the top left corner, the navigation on the bottom, and [for audio or video] a play/pause button in the center of the screen.” Use industry standards for interactions Support VoiceOver functionality on iOS Find inspiration in simple, everyday apps, such as Apple’s Podcasts Provide audio transcripts “Because of my cane, I like things that I can do with one hand.” Keep physical and sound interactions to a minimum Lanyards for devices Avoid playing audio automatically “I like hearing about the history around a work of art more than how it’s made.” “I want to know the details—the medium, the style, the process to make it.” Provide a mix of voices and tones Include various staff voices and perspectives Conversational, podcast-like tones
  47. 47. From Interviews to Action 47 Quote Design principle Design decision “I want a little more independence” during a visit Provide a path with a choice Split up audio content into smaller, modular stops and allow visitors to dive deeper if they’d like to learn more “I usually go to museums with other people; it’s a social thing.” Build as a social experience Develop audio content that sparks discussion Provide headphone splitters for visitors to share devices “I expect a back button in the top left corner, the navigation on the bottom, and [for audio or video] a play/pause button in the center of the screen.” Use industry standards for interactions Support VoiceOver functionality on iOS Find inspiration in simple, everyday apps, such as Apple’s Podcasts Provide audio transcripts “Because of my cane, I like things that I can do with one hand.” Keep physical and sound interactions to a minimum Lanyards for devices Avoid playing audio automatically “I like hearing about the history around a work of art more than how it’s made.” “I want to know the details—the medium, the style, the process to make it.” Provide a mix of voices and tones Include various staff voices and perspectives Conversational, podcast-like tones
  48. 48. Delivering Modular Audio: Early Prototypes 48 Quartz app Prototype 2Prototype 1
  49. 49. Delivering Modular Audio: Final Design 49
  50. 50. Redesigning warhol.org #MCN2017-T4 MCN 2017050
  51. 51. 51 Before After
  52. 52. User Research: How 52 ● Usertesting.com tests ● Personas + journey maps ● Usability testing
  53. 53. User Personas 53 TeacherTourist Access Researcher Family Teen Young Professional
  54. 54. User Personas 54 Teen Alex A teen who identifies as gender non-binary has learned about Warhol in art classes in high school. Their teacher mentions the opportunities for teens in class one day. They are a junior who goes to public school on the North Side. They don’t see themselves reflected in their social circles and are looking to connect to people and places that reflect them. They are thinking about college, but undecided on the way forward. Looking for experiences to guide their thinking.
  55. 55. Journey Mapping 55
  56. 56. Usability Tests: Calendar 56 Observations Learning Design decision Many used the homepage as a way to get an overview of events and exhibitions at the museum People want to search by audience type, but often this segmentation is found in a “Programs”/“Learn” section of a site Provide many different ways into museum events Dynamically pull in calendar events on our more static Programs pages Include pathways for specific audiences to find information tailored to them Several users wondered what day of the week certain events were so they could plan around their schedules “I love that it says ‘Closing soon’ right at the top” Give the basic info, and make it clear Design of event pages: highlights the important logistic information Dynamically change language for exhibition dates: "Opens November 15, 2017" if two weeks before an exhibition opens Terms like “lectures” and “seminars” can be off-putting to art newbies Both tourist user profiles assumed “Film screenings” would be under “Watch” on the Whitney’s website Language is everything Carefully crafted nomenclature to guide our events taxonomy; on calendar, surface events by type and audience, but not by series brand Several users noted the tension between wanting both a broad overview of all events at the museum and needing to drill down to a specific day or time span—they wanted both! Users both want to browse generally and find specific information Toggle between 1, 7, and 30 day views One tourist was mesmerized by the CMOA’s Iris van Herpen image A parent used images to decide if an exhibition would be age-appropriate for her kids Images are powerful Developed web photography guidelines that favor strong images of people interacting with each other
  57. 57. Usability Tests: Calendar 57 Observations Learning Design decision Many used the homepage as a way to get an overview of events and exhibitions at the museum People want to search by audience type, but often this segmentation is found in a “Programs”/“Learn” section of a site Provide diverse ways into museum events Dynamically pull in calendar events on our more static Programs pages Include pathways for specific audiences to find information tailored to them Several users wondered what day of the week certain events were so they could plan around their schedules “I love that it says ‘Closing soon’ right at the top” Give the basic info, and make it clear Design of event pages: highlights the important logistic information Dynamically change language for exhibition dates: "Opens November 15, 2017" if two weeks before an exhibition opens Terms like “lectures” and “seminars” can be off-putting to art newbies Both tourist user profiles assumed “Film screenings” would be under “Watch” on the Whitney’s website Language is everything Carefully crafted nomenclature to guide our events taxonomy; on calendar, surface events by type and audience, but not by series brand Several users noted the tension between wanting both a broad overview of all events at the museum and needing to drill down to a specific day or time span—they wanted both! Users both want to browse generally and find specific information Toggle between 1, 7, and 30 day views One tourist was mesmerized by the CMOA’s Iris van Herpen image A parent used images to decide if an exhibition would be age-appropriate for her kids Images are powerful Developed web photography guidelines that favor strong images of people interacting with each other
  58. 58. Research to Action 58
  59. 59. Conversation and Questions #MCN2017-T4 59 Desi Gonzalez Manager of Digital Engagement The Andy Warhol Museum @desigonz Jenn Phillips-Bacher Product Manager, Digital Experience Wellcome Collection @MrsAudiac Susan Wigodner Web & Digital Project Manager The Field Museum @swigs27
  60. 60. Thank You! MCN 2017060
  61. 61. 61 Links Field Museum ● Fieldmuseum.org Wellcome Collection ● Wellcomecollection.org ● Digital Stories: Mindcraft, The Collectors, Electric Sublime, The Outsiders The Warhol ● Warhol.org ● Out Loud

×