• Save
Evaluating Getty content on Whyville
Upcoming SlideShare
Loading in...5
×
 

Evaluating Getty content on Whyville

on

  • 1,046 views

Presentation given at 2008 annual meeting of Museum Computer Network (MCN) November 13, 2008

Presentation given at 2008 annual meeting of Museum Computer Network (MCN) November 13, 2008

Statistics

Views

Total Views
1,046
Views on SlideShare
1,045
Embed Views
1

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 2005/06 data from a project launched in 2004. Thoughts in retrospect about measuring impact that may help evaluate projects on third-party social networking sites that didn’t even exist back then.

Evaluating Getty content on Whyville Evaluating Getty content on Whyville Presentation Transcript

  • +
  • Where Do Education and Marketing Meet?
    • Introduce Whyville project
    • Measuring exposure
    • Measuring learning
    • Lessons
  • What is Whyville? online virtual environment for kids since 1999
  • Who are Whyvillians?
    • More than 2 million Whyvillians
    • Kids, ages 8-14
    • 2/3 of Whyvillians are girls
    • Mostly from U.S. and Canada, but many are from U.K., Australia, and even non-English speaking countries like Japan
  • Audience
    • They have the audience, we don't
    • They know how to engage the audience
    Why a Getty Museum on Whyville?
    • Primary Goals
    • Raise awareness: museums and the Getty
    • Test lab for game development
    • Secondary Goals
    • Basic looking skills
    • Art history terminology
    • Research skills
    • Primary Goals (Marketing/PR)
    • Raise awareness: museums and the Getty
    • Test lab for game development
    • Secondary Goals (Education)
    • Basic looking skills
    • Art history terminology
    • Research skills
  • ArtSets
  • Getty Art Treasure Hunt
  •  
    • Planning: Educational goals
    • Measurement: Marketing tools
    A problem
  • Focused Study
    • February–September 2005
    • visit data
    • online focus group
    • September 2005–February 2006
    • online survey
    • http://www.getty.edu/museum/research/metrics_evaluations/downloads/whyville_assessment_2007.pdf
  • Measuring Traffic = Eyeballs Tears , Man Ray, 1930–1932. © Man Ray Trust ARS-ADAGP
  • 2 Sets of Traffic
    • 1. Traffic within Whyville
    • 2. Traffic driven to g
  • Fitting Square Pegs into Circles
    • virtual world vs. museum Web site
    • Different goals, audiences, stakeholders, and strategies
    • How many eyeballs
    • see Getty content on Whyville ?
    • 21,000 unique users / month
    • 61,000 unique visits / month
    • 4.18% of all visitors to Whyville
    • 7.6 minutes = average visit
    • How many eyeballs
    • see Getty content on Whyville ?
    • 21,000 unique users / month
    • They can authenticate, we can’t
    • 61,000 unique visits / month
    • 4.18% of all visitors to Whyville
    • 7.6 minutes = average visit
    • How many eyeballs
    • see Getty content on Whyville ?
    • 21,000 unique users / month
    • 61,000 unique visits / month
    • > 20 minutes = new visit
    • 4.18% of all visitors to Whyville
    • 7.6 minutes = average visit
    • How many eyeballs
    • see Getty content on Whyville ?
    • 21,000 unique users / month
    • 61,000 unique visits / month
    • 4.18% of all visitors to Whyville
    • Cool! (but what does it mean? !?)
    • 7.6 minutes = average visit
    • How many eyeballs
    • see Getty content on Whyville ?
    • 21,000 unique users / month
    • 61,000 unique visits / month
    • 4.18% of all visitors to Whyville
    • 7.6 minutes = average visit
    • For one week in June 2005
    • How many eyeballs
    • migrate to getty.edu?
    • 606 unique visits / month
    • 5.9 minutes = average visit
    • How many eyeballs
    • migrate to getty.edu?
    • 606 unique visits / month
    • 1% from Whyville
    • 5.9 minutes = average visit
    • How many eyeballs
    • migrate to getty.edu?
    • 606 unique visits / month
    • 5.9 minutes = average visit
    • Impressive! (based on?)
    • How many eyeballs
    • migrate to getty.edu?
    • 606 unique visits / month
    • 5.9 minutes = average visit
    • 1% conversion + high visit duration
    • by advertising standards, impressive
  • What does it all mean?
    • On Whyville
    • Compare to other areas
    • Compare games
    • On getty.edu
    • ?
    • Whyville comparison
    • WASA Ion Engine Lab
    • Sponsored by NASA
  • Getty Museum vs. WASA
    • Average visits per month:
    • 0.46% visited WASA Ion Engine Lab
    • 4.31% visited Getty Museum
    • 10x more visits
  • We got Eyeballs
    • What about learning?
    • Can we get to that through statistics?
    • Tracking patterns of activity
    • by individual users
    • vs.
    • Tracking interaction
    • between users
  • Track the Sharing of Content
    • Track discussions:
      • chat logs
      • – keywords
      • – discussion threads
    • Track content sharing:
      • Assets exchanged/bought/traded
  • Traffic Statistics - Lessons
    • Plan ahead:
    • Get traffic stats – in your own language
    • Identify areas for comparison/context
    • Identify stats that can get at “the Social”
  • Measuring Learning = Brains Did we convert eyeballs into brains?
  • Measuring attitudes and learning
    • Focus Groups
    • Survey
    • Focus group to test questions for survey about changed attitudes.
    • Primary Goals (Marketing/PR)
    • Raise awareness: museums and the Getty
    • Test lab for game development
    • Secondary Goals (Education)
    • Basic looking skills
    • Art history terminology
    • Research skills
    • Primary Goals (Marketing/PR)
    • Raise awareness: museums and the Getty
    • Did they learn what the Getty is?
    • Did their attitudes about museums change?
    • Did their attitudes about the Getty change?
    • Secondary Goals (Education)
    • Basic looking skills
      • Do they recognize Getty artworks?
      • Can they distinguish medium and subject types?
    • Art history terminology
      • Can they use new terms?
    • Research skills
      • No direct testing.
  • Online Focus Group
  • 1) Have you been to the Getty Museum on Whyville? 2) After seeing the Getty on Whyville, do you like museums more or less? 3) Have you been to a real museum this summer? 4) Did the Getty Museum on Whyville make you like art more? 5) Have you played ArtSets or the Art Treasure Hunt? 6) Where is the real Getty Museum? 7) Have you visited the Getty web site? Poll Questions
  • “ Getty Art Hour”
    • We had a conversation
    • with open-ended questions:
    • What do you like in museums?
    • Compare real & virtual museums.
    • Why do you like the virtual/real better?
    • Why is art important?
    • Do you recognize these artworks?
  • Poll Questions XWhyDivaX: bridge in america!!! gold837: brooklyn bridge evilworm: isnt it the uh golden gate bridge tokyo1: This is the brooklyn bridge Actingup9: yes on artsets XWhyDivaX : roman plate thing holiday50 : Black Fihure KYlix Sugar7000 : on the treasure hunt biddygurl : Treasure hunt, round four I think? Rose775 : Greece? holiday50 : That one took me hours to find
  • Trick question: Have you seen this? XWhyDivaX : thought i havent seen it before... Sugar7000 : person minerals : its a painting of a women 123kw: Painting of a person minerals : with a sword Girlfreak: but don't know what it is i just know it's a painting 2 supercow9 : PORTRAIT OF A HALBERDIER!
  • Focus Group Issues
    • CONS
    • Self-selected participants
    • Doesn’t test “conversion”
    • Anecdotal information
    • PROS
    • Conversation
    • Opportunity for two-way learning
    • “ Getty Art Hour”
  • Social Learning
  • Role of the Social
    • Education Goals
    • A community of learners reinforces learning
    • Learning is social
      • Learning continues beyond the learning environment (Falk and Dierking)
      • We form ideas by testing them, sharing them with others (Wittgenstein)
  • Role of the Social
    • Marketing / PR Goals
    • Word-of-mouth / viral communication
    • Value created through sharing
    • “Networks of interest”
  • Survey
    • Testing change in attitudes/knowledge
    • Issues:
    • High turnover: Longitudinal study not possible
    • Multiple user accounts
    • Sculpting questions: getting at attitudinal change
  • Longitudinal Kluge
    • group A – had not visited Getty content
    • group B – had visited Getty content
    • #5 Where is the Getty Museum located?
    • Correct answers:
    • 7.5% who had not visited Getty content
    • 22.9% who had visited Getty content
    • #6 Have you visited the Getty Web site?
    • ‘Yes’ answers:
    • 3.5% who had not visited Getty content
    • 36.5% who had visited Getty content
  • Success?
    • Raised awareness of Getty Museum
      • Name recognition
      • Location
  • Insignificant Change
    • All other questions!
    • Attitudinal questions on a scale
    • strongly agree <–> strongly disagree
    • example:
    • The Getty Museum on Whyville makes me interested in visiting a real art museum.
    • Subject matter questions
    • very small margin of increased recognition
    • (most could be reasonably guessed)
  • ?
  • Lessons
    • The goal: Converting eyeballs into brains
    • Keep it simple – tie it to goals
    • Where do education and marketing meet?
    • How to measure &quot;conversion&quot;?
    • The role of the Social
      • Know that engaging in conversations can help. Make it part of the plan. It’s time-consuming, but worth it.
  • Unexpected outcomes
    • Value of conversations
      • Learn from your audience
      • Inspires staff
      • Reinforces learning for audience
    • Benefits of partnerships
      • Reaching new audiences
      • Audience can incorporate you into their community more easily
      • Staff learn new skills and gain expertise
    • Susan Edwards
    • Senior Writer/Editor
    • Web Group
    • J. Paul Getty Trust
    • [email_address]
    • http://whyville.net
    http://www.getty.edu/museum/research/metrics_evaluations/downloads/whyville_assessment_2007.pdf