Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Evaluating Digital Technologies
Elizabeth Bolander
Meghan Stockdale
Research Questions
• What do visitors think about Gallery One and
ArtLens?
• How does Gallery One and ArtLens affect the
m...
Process
• Literature review conducted
• Iterative process throughout the planning and
development
– User testing (usabilit...
Mixed Methods Approach
• Usability tests of all interactives
– 30 tests per lens, Collection Wall, and ArtLens
• Observati...
Mixed Methods Approach
• Comprehensive in-depth analysis
– Visitor groups placed into one of four test groups
– Pre- and p...
Mixed Methods Approach
• User data from Lens stations and ArtLens
tracked via Google Analytics
– More difficult to track t...
Research in Action
• Visitor wayfinding concerns
Research in Action
• App labels on walls
Research in Action
• Off-site versus on-site needs
– On-site has the benefit of the actual space
– Off-site users looking ...
Strengths and Challenges
• Significant amount of deep, nuanced data
– Learning more than just the impact of
technologies, ...
Let’s Chat!
• What technologies do your museums use?
How do you evaluate them?
• What challenges have you experienced? How...
Upcoming SlideShare
Loading in …5
×

Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate

1,142 views

Published on

Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate
Museums and the Web at the Cleveland Museum of Art
Take a Deep Dive into Gallery One at the Cleveland Museum of Art

Published in: Technology, Business
  • Be the first to comment

Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate

  1. 1. Evaluating Digital Technologies Elizabeth Bolander Meghan Stockdale
  2. 2. Research Questions • What do visitors think about Gallery One and ArtLens? • How does Gallery One and ArtLens affect the museum visit experience?
  3. 3. Process • Literature review conducted • Iterative process throughout the planning and development – User testing (usability, content, tone) – Audience panels • Ultimately realized we had many different needs that required different approaches
  4. 4. Mixed Methods Approach • Usability tests of all interactives – 30 tests per lens, Collection Wall, and ArtLens • Observations – Timing and tracking maps recorded for 200 visitors • Skype conversations with off-site users of ArtLens
  5. 5. Mixed Methods Approach • Comprehensive in-depth analysis – Visitor groups placed into one of four test groups – Pre- and post-visit interviews • Range of topics explored – Visit videotaped with flip cameras • Usually about 2 hours of video was recorded per visit – Phone interviews completed 2-3 months later • Testing re-use, memory, perception changes, etc.
  6. 6. Mixed Methods Approach • User data from Lens stations and ArtLens tracked via Google Analytics – More difficult to track the Lenses than ArtLens • Very valuable counterpoint to qualitative data
  7. 7. Research in Action • Visitor wayfinding concerns
  8. 8. Research in Action • App labels on walls
  9. 9. Research in Action • Off-site versus on-site needs – On-site has the benefit of the actual space – Off-site users looking for additional contextual information
  10. 10. Strengths and Challenges • Significant amount of deep, nuanced data – Learning more than just the impact of technologies, but a comprehensive look at the entire experience • Data tracking and management can be difficult • Analysis of overall project and preparing reports for a diverse array of internal teams with different priorities
  11. 11. Let’s Chat! • What technologies do your museums use? How do you evaluate them? • What challenges have you experienced? How did you handle them? • What can we do together to create a community of practice around these types of projects?

×