Evaluating Digital Technologies
Elizabeth Bolander
Meghan Stockdale
Research Questions
• What do visitors think about Gallery One and
ArtLens?
• How does Gallery One and ArtLens affect the
m...
Process
• Literature review conducted
• Iterative process throughout the planning and
development
– User testing (usabilit...
Mixed Methods Approach
• Usability tests of all interactives
– 30 tests per lens, Collection Wall, and ArtLens
• Observati...
Mixed Methods Approach
• Comprehensive in-depth analysis
– Visitor groups placed into one of four test groups
– Pre- and p...
Mixed Methods Approach
• User data from Lens stations and ArtLens
tracked via Google Analytics
– More difficult to track t...
Research in Action
• Visitor wayfinding concerns
Research in Action
• App labels on walls
Research in Action
• Off-site versus on-site needs
– On-site has the benefit of the actual space
– Off-site users looking ...
Strengths and Challenges
• Significant amount of deep, nuanced data
– Learning more than just the impact of
technologies, ...
Let’s Chat!
• What technologies do your museums use?
How do you evaluate them?
• What challenges have you experienced? How...
Upcoming SlideShare
Loading in …5
×

Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate

993 views

Published on

Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate
Museums and the Web at the Cleveland Museum of Art
Take a Deep Dive into Gallery One at the Cleveland Museum of Art

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
993
On SlideShare
0
From Embeds
0
Number of Embeds
492
Actions
Shares
0
Downloads
11
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate

  1. 1. Evaluating Digital Technologies Elizabeth Bolander Meghan Stockdale
  2. 2. Research Questions • What do visitors think about Gallery One and ArtLens? • How does Gallery One and ArtLens affect the museum visit experience?
  3. 3. Process • Literature review conducted • Iterative process throughout the planning and development – User testing (usability, content, tone) – Audience panels • Ultimately realized we had many different needs that required different approaches
  4. 4. Mixed Methods Approach • Usability tests of all interactives – 30 tests per lens, Collection Wall, and ArtLens • Observations – Timing and tracking maps recorded for 200 visitors • Skype conversations with off-site users of ArtLens
  5. 5. Mixed Methods Approach • Comprehensive in-depth analysis – Visitor groups placed into one of four test groups – Pre- and post-visit interviews • Range of topics explored – Visit videotaped with flip cameras • Usually about 2 hours of video was recorded per visit – Phone interviews completed 2-3 months later • Testing re-use, memory, perception changes, etc.
  6. 6. Mixed Methods Approach • User data from Lens stations and ArtLens tracked via Google Analytics – More difficult to track the Lenses than ArtLens • Very valuable counterpoint to qualitative data
  7. 7. Research in Action • Visitor wayfinding concerns
  8. 8. Research in Action • App labels on walls
  9. 9. Research in Action • Off-site versus on-site needs – On-site has the benefit of the actual space – Off-site users looking for additional contextual information
  10. 10. Strengths and Challenges • Significant amount of deep, nuanced data – Learning more than just the impact of technologies, but a comprehensive look at the entire experience • Data tracking and management can be difficult • Analysis of overall project and preparing reports for a diverse array of internal teams with different priorities
  11. 11. Let’s Chat! • What technologies do your museums use? How do you evaluate them? • What challenges have you experienced? How did you handle them? • What can we do together to create a community of practice around these types of projects?

×