+
Where Do  Education and Marketing  Meet? <ul><li>Introduce Whyville project </li></ul><ul><li>Measuring exposure </li></ul...
What is Whyville?   online virtual environment for kids  since 1999
Who are Whyvillians? <ul><li>More than 2 million Whyvillians </li></ul><ul><li>Kids, ages 8-14 </li></ul><ul><li>2/3 of Wh...
Audience <ul><li>They  have  the audience, we don't </li></ul><ul><li>They know how to  engage  the audience </li></ul>Why...
<ul><li>Primary Goals </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Test lab for game develop...
<ul><li>Primary Goals (Marketing/PR) </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Test lab f...
ArtSets
Getty Art Treasure Hunt
 
<ul><li>Planning: Educational goals </li></ul><ul><li>Measurement: Marketing tools </li></ul>A problem
Focused Study <ul><li>February–September 2005 </li></ul><ul><li>visit data </li></ul><ul><li>online focus group  </li></ul...
Measuring Traffic = Eyeballs Tears , Man Ray, 1930–1932.  © Man Ray Trust ARS-ADAGP
2 Sets of Traffic <ul><li>1. Traffic within Whyville </li></ul><ul><li>2. Traffic driven to  g  </li></ul>
Fitting Square Pegs into Circles <ul><li>virtual world vs. museum Web site </li></ul><ul><li>Different goals, audiences, s...
<ul><li>How many eyeballs  </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month...
<ul><li>How many eyeballs  </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month...
<ul><li>How many eyeballs  </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month...
<ul><li>How many eyeballs  </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month...
<ul><li>How many eyeballs  </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month...
<ul><li>How many eyeballs  </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month  </li></ul>...
<ul><li>How many eyeballs  </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month  </li></ul>...
<ul><li>How many eyeballs  </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month  </li></ul>...
<ul><li>How many eyeballs  </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month  </li></ul>...
What does it all mean? <ul><li>On Whyville </li></ul><ul><li>Compare to other areas </li></ul><ul><li>Compare games </li><...
<ul><li>Whyville comparison  </li></ul><ul><li>WASA Ion Engine Lab </li></ul><ul><li>Sponsored by NASA </li></ul>
Getty Museum vs. WASA <ul><li>Average visits per month: </li></ul><ul><li>0.46% visited WASA Ion Engine Lab </li></ul><ul>...
We got Eyeballs <ul><li>What about learning? </li></ul><ul><li>Can we get to that through statistics? </li></ul>
<ul><li>Tracking patterns of activity </li></ul><ul><li>by individual users </li></ul><ul><li>vs. </li></ul><ul><li>Tracki...
Track the Sharing of Content <ul><li>Track discussions:  </li></ul><ul><ul><li>chat logs  </li></ul></ul><ul><ul><li>– key...
Traffic Statistics - Lessons <ul><li>Plan ahead: </li></ul><ul><li>Get traffic stats – in your own language </li></ul><ul>...
Measuring Learning = Brains Did we convert eyeballs into brains?
Measuring attitudes and learning <ul><li>Focus Groups </li></ul><ul><li>Survey </li></ul><ul><li>Focus group to test quest...
<ul><li>Primary Goals (Marketing/PR) </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Test lab f...
<ul><li>Primary Goals (Marketing/PR) </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Did they l...
<ul><li>Secondary Goals (Education) </li></ul><ul><li>Basic looking skills </li></ul><ul><ul><li>Do they recognize Getty a...
Online Focus Group
1) Have you been to the Getty Museum on Whyville?  2) After seeing the Getty on Whyville, do you like museums more or less...
“ Getty Art Hour” <ul><li>We had a conversation </li></ul><ul><li>with open-ended questions: </li></ul><ul><li>What do you...
Poll Questions XWhyDivaX: bridge in america!!!  gold837: brooklyn bridge  evilworm: isnt it the uh golden gate bridge  tok...
Trick question: Have you seen this?   XWhyDivaX : thought i havent seen it before...  Sugar7000 : person  minerals : its a...
Focus Group Issues <ul><li>CONS </li></ul><ul><li>Self-selected participants </li></ul><ul><li>Doesn’t test “conversion” <...
Social Learning
Role of the Social <ul><li>Education Goals </li></ul><ul><li>A community of learners reinforces learning </li></ul><ul><li...
Role of the Social <ul><li>Marketing / PR Goals </li></ul><ul><li>Word-of-mouth / viral communication </li></ul><ul><li>Va...
Survey <ul><li>Testing change in attitudes/knowledge </li></ul><ul><li>Issues: </li></ul><ul><li>High turnover:  Longitudi...
Longitudinal Kluge <ul><li>group A – had not visited Getty content </li></ul><ul><li>group B – had visited Getty content <...
<ul><li>#5 Where is the Getty Museum located? </li></ul><ul><li>Correct answers:  </li></ul><ul><li>  7.5%   who  had not ...
Success? <ul><li>Raised awareness of Getty Museum </li></ul><ul><ul><li>Name recognition </li></ul></ul><ul><ul><li>Locati...
Insignificant Change <ul><li>All other questions! </li></ul><ul><li>Attitudinal questions on a scale </li></ul><ul><li>str...
?
Lessons <ul><li>The goal: Converting eyeballs into brains </li></ul><ul><li>Keep it simple – tie it to goals </li></ul><ul...
Unexpected outcomes <ul><li>Value of conversations </li></ul><ul><ul><li>Learn from your audience </li></ul></ul><ul><ul><...
<ul><li>Susan Edwards </li></ul><ul><li>Senior Writer/Editor   </li></ul><ul><li>Web Group </li></ul><ul><li>J. Paul Getty...
Upcoming SlideShare
Loading in …5
×

Evaluating Getty content on Whyville

749
-1

Published on

Presentation given at 2008 annual meeting of Museum Computer Network (MCN) November 13, 2008

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
749
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 2005/06 data from a project launched in 2004. Thoughts in retrospect about measuring impact that may help evaluate projects on third-party social networking sites that didn’t even exist back then.
  • Evaluating Getty content on Whyville

    1. 1. +
    2. 2. Where Do Education and Marketing Meet? <ul><li>Introduce Whyville project </li></ul><ul><li>Measuring exposure </li></ul><ul><li>Measuring learning </li></ul><ul><li>Lessons </li></ul>
    3. 3. What is Whyville? online virtual environment for kids since 1999
    4. 4. Who are Whyvillians? <ul><li>More than 2 million Whyvillians </li></ul><ul><li>Kids, ages 8-14 </li></ul><ul><li>2/3 of Whyvillians are girls </li></ul><ul><li>Mostly from U.S. and Canada, but many are from U.K., Australia, and even non-English speaking countries like Japan </li></ul>
    5. 5. Audience <ul><li>They have the audience, we don't </li></ul><ul><li>They know how to engage the audience </li></ul>Why a Getty Museum on Whyville?
    6. 6. <ul><li>Primary Goals </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Test lab for game development </li></ul><ul><li>Secondary Goals </li></ul><ul><li>Basic looking skills </li></ul><ul><li>Art history terminology </li></ul><ul><li>Research skills </li></ul>
    7. 7. <ul><li>Primary Goals (Marketing/PR) </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Test lab for game development </li></ul><ul><li>Secondary Goals (Education) </li></ul><ul><li>Basic looking skills </li></ul><ul><li>Art history terminology </li></ul><ul><li>Research skills </li></ul>
    8. 8. ArtSets
    9. 9. Getty Art Treasure Hunt
    10. 11. <ul><li>Planning: Educational goals </li></ul><ul><li>Measurement: Marketing tools </li></ul>A problem
    11. 12. Focused Study <ul><li>February–September 2005 </li></ul><ul><li>visit data </li></ul><ul><li>online focus group </li></ul><ul><li>September 2005–February 2006 </li></ul><ul><li>online survey </li></ul><ul><li>http://www.getty.edu/museum/research/metrics_evaluations/downloads/whyville_assessment_2007.pdf </li></ul>
    12. 13. Measuring Traffic = Eyeballs Tears , Man Ray, 1930–1932. © Man Ray Trust ARS-ADAGP
    13. 14. 2 Sets of Traffic <ul><li>1. Traffic within Whyville </li></ul><ul><li>2. Traffic driven to g </li></ul>
    14. 15. Fitting Square Pegs into Circles <ul><li>virtual world vs. museum Web site </li></ul><ul><li>Different goals, audiences, stakeholders, and strategies </li></ul>
    15. 16. <ul><li>How many eyeballs </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month </li></ul><ul><li>61,000 unique visits / month </li></ul><ul><li>4.18% of all visitors to Whyville </li></ul><ul><li>7.6 minutes = average visit </li></ul>
    16. 17. <ul><li>How many eyeballs </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month </li></ul><ul><li>They can authenticate, we can’t </li></ul><ul><li>61,000 unique visits / month </li></ul><ul><li>4.18% of all visitors to Whyville </li></ul><ul><li>7.6 minutes = average visit </li></ul>
    17. 18. <ul><li>How many eyeballs </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month </li></ul><ul><li>61,000 unique visits / month </li></ul><ul><li>> 20 minutes = new visit </li></ul><ul><li>4.18% of all visitors to Whyville </li></ul><ul><li>7.6 minutes = average visit </li></ul>
    18. 19. <ul><li>How many eyeballs </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month </li></ul><ul><li>61,000 unique visits / month </li></ul><ul><li>4.18% of all visitors to Whyville </li></ul><ul><li>Cool! (but what does it mean? !?) </li></ul><ul><li>7.6 minutes = average visit </li></ul>
    19. 20. <ul><li>How many eyeballs </li></ul><ul><li>see Getty content on Whyville ? </li></ul><ul><li>21,000 unique users / month </li></ul><ul><li>61,000 unique visits / month </li></ul><ul><li>4.18% of all visitors to Whyville </li></ul><ul><li>7.6 minutes = average visit </li></ul><ul><li>For one week in June 2005 </li></ul>
    20. 21. <ul><li>How many eyeballs </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month </li></ul><ul><li>5.9 minutes = average visit </li></ul>
    21. 22. <ul><li>How many eyeballs </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month </li></ul><ul><li>1% from Whyville </li></ul><ul><li>5.9 minutes = average visit </li></ul>
    22. 23. <ul><li>How many eyeballs </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month </li></ul><ul><li>5.9 minutes = average visit </li></ul><ul><li>Impressive! (based on?) </li></ul>
    23. 24. <ul><li>How many eyeballs </li></ul><ul><li>migrate to getty.edu? </li></ul><ul><li>606 unique visits / month </li></ul><ul><li>5.9 minutes = average visit </li></ul><ul><li>1% conversion + high visit duration </li></ul><ul><li>by advertising standards, impressive </li></ul>
    24. 25. What does it all mean? <ul><li>On Whyville </li></ul><ul><li>Compare to other areas </li></ul><ul><li>Compare games </li></ul><ul><li>On getty.edu </li></ul><ul><li>? </li></ul>
    25. 26. <ul><li>Whyville comparison </li></ul><ul><li>WASA Ion Engine Lab </li></ul><ul><li>Sponsored by NASA </li></ul>
    26. 27. Getty Museum vs. WASA <ul><li>Average visits per month: </li></ul><ul><li>0.46% visited WASA Ion Engine Lab </li></ul><ul><li>4.31% visited Getty Museum </li></ul><ul><li>10x more visits </li></ul>
    27. 28. We got Eyeballs <ul><li>What about learning? </li></ul><ul><li>Can we get to that through statistics? </li></ul>
    28. 29. <ul><li>Tracking patterns of activity </li></ul><ul><li>by individual users </li></ul><ul><li>vs. </li></ul><ul><li>Tracking interaction </li></ul><ul><li>between users </li></ul>
    29. 30. Track the Sharing of Content <ul><li>Track discussions: </li></ul><ul><ul><li>chat logs </li></ul></ul><ul><ul><li>– keywords </li></ul></ul><ul><ul><li>– discussion threads </li></ul></ul><ul><li>Track content sharing: </li></ul><ul><ul><li>Assets exchanged/bought/traded </li></ul></ul>
    30. 31. Traffic Statistics - Lessons <ul><li>Plan ahead: </li></ul><ul><li>Get traffic stats – in your own language </li></ul><ul><li>Identify areas for comparison/context </li></ul><ul><li>Identify stats that can get at “the Social” </li></ul>
    31. 32. Measuring Learning = Brains Did we convert eyeballs into brains?
    32. 33. Measuring attitudes and learning <ul><li>Focus Groups </li></ul><ul><li>Survey </li></ul><ul><li>Focus group to test questions for survey about changed attitudes. </li></ul>
    33. 34. <ul><li>Primary Goals (Marketing/PR) </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Test lab for game development </li></ul><ul><li>Secondary Goals (Education) </li></ul><ul><li>Basic looking skills </li></ul><ul><li>Art history terminology </li></ul><ul><li>Research skills </li></ul>
    34. 35. <ul><li>Primary Goals (Marketing/PR) </li></ul><ul><li>Raise awareness: museums and the Getty </li></ul><ul><li>Did they learn what the Getty is? </li></ul><ul><li>Did their attitudes about museums change? </li></ul><ul><li>Did their attitudes about the Getty change? </li></ul>
    35. 36. <ul><li>Secondary Goals (Education) </li></ul><ul><li>Basic looking skills </li></ul><ul><ul><li>Do they recognize Getty artworks? </li></ul></ul><ul><ul><li>Can they distinguish medium and subject types? </li></ul></ul><ul><li>Art history terminology </li></ul><ul><ul><li>Can they use new terms? </li></ul></ul><ul><li>Research skills </li></ul><ul><ul><li>No direct testing. </li></ul></ul>
    36. 37. Online Focus Group
    37. 38. 1) Have you been to the Getty Museum on Whyville? 2) After seeing the Getty on Whyville, do you like museums more or less? 3) Have you been to a real museum this summer? 4) Did the Getty Museum on Whyville make you like art more? 5) Have you played ArtSets or the Art Treasure Hunt? 6) Where is the real Getty Museum? 7) Have you visited the Getty web site? Poll Questions
    38. 39. “ Getty Art Hour” <ul><li>We had a conversation </li></ul><ul><li>with open-ended questions: </li></ul><ul><li>What do you like in museums? </li></ul><ul><li>Compare real & virtual museums. </li></ul><ul><li>Why do you like the virtual/real better? </li></ul><ul><li>Why is art important? </li></ul><ul><li>Do you recognize these artworks? </li></ul>
    39. 40. Poll Questions XWhyDivaX: bridge in america!!! gold837: brooklyn bridge evilworm: isnt it the uh golden gate bridge tokyo1: This is the brooklyn bridge Actingup9: yes on artsets XWhyDivaX : roman plate thing holiday50 : Black Fihure KYlix Sugar7000 : on the treasure hunt biddygurl : Treasure hunt, round four I think? Rose775 : Greece? holiday50 : That one took me hours to find
    40. 41. Trick question: Have you seen this? XWhyDivaX : thought i havent seen it before... Sugar7000 : person minerals : its a painting of a women 123kw: Painting of a person minerals : with a sword Girlfreak: but don't know what it is i just know it's a painting 2 supercow9 : PORTRAIT OF A HALBERDIER!
    41. 42. Focus Group Issues <ul><li>CONS </li></ul><ul><li>Self-selected participants </li></ul><ul><li>Doesn’t test “conversion” </li></ul><ul><li>Anecdotal information </li></ul><ul><li>PROS </li></ul><ul><li>Conversation </li></ul><ul><li>Opportunity for two-way learning </li></ul><ul><li>“ Getty Art Hour” </li></ul>
    42. 43. Social Learning
    43. 44. Role of the Social <ul><li>Education Goals </li></ul><ul><li>A community of learners reinforces learning </li></ul><ul><li>Learning is social </li></ul><ul><ul><li>Learning continues beyond the learning environment (Falk and Dierking) </li></ul></ul><ul><ul><li>We form ideas by testing them, sharing them with others (Wittgenstein) </li></ul></ul>
    44. 45. Role of the Social <ul><li>Marketing / PR Goals </li></ul><ul><li>Word-of-mouth / viral communication </li></ul><ul><li>Value created through sharing </li></ul><ul><li> “Networks of interest” </li></ul>
    45. 46. Survey <ul><li>Testing change in attitudes/knowledge </li></ul><ul><li>Issues: </li></ul><ul><li>High turnover: Longitudinal study not possible </li></ul><ul><li>Multiple user accounts </li></ul><ul><li>Sculpting questions: getting at attitudinal change </li></ul>
    46. 47. Longitudinal Kluge <ul><li>group A – had not visited Getty content </li></ul><ul><li>group B – had visited Getty content </li></ul>
    47. 48. <ul><li>#5 Where is the Getty Museum located? </li></ul><ul><li>Correct answers: </li></ul><ul><li> 7.5% who had not visited Getty content </li></ul><ul><li>22.9% who had visited Getty content </li></ul><ul><li>#6 Have you visited the Getty Web site? </li></ul><ul><li>‘Yes’ answers: </li></ul><ul><li> 3.5% who had not visited Getty content </li></ul><ul><li>36.5% who had visited Getty content </li></ul>
    48. 49. Success? <ul><li>Raised awareness of Getty Museum </li></ul><ul><ul><li>Name recognition </li></ul></ul><ul><ul><li>Location </li></ul></ul>
    49. 50. Insignificant Change <ul><li>All other questions! </li></ul><ul><li>Attitudinal questions on a scale </li></ul><ul><li>strongly agree <–> strongly disagree </li></ul><ul><li>example: </li></ul><ul><li>The Getty Museum on Whyville makes me interested in visiting a real art museum. </li></ul><ul><li>Subject matter questions </li></ul><ul><li>very small margin of increased recognition </li></ul><ul><li>(most could be reasonably guessed) </li></ul>
    50. 51. ?
    51. 52. Lessons <ul><li>The goal: Converting eyeballs into brains </li></ul><ul><li>Keep it simple – tie it to goals </li></ul><ul><li>Where do education and marketing meet? </li></ul><ul><li>How to measure &quot;conversion&quot;? </li></ul><ul><li>The role of the Social </li></ul><ul><ul><li>Know that engaging in conversations can help. Make it part of the plan. It’s time-consuming, but worth it. </li></ul></ul>
    52. 53. Unexpected outcomes <ul><li>Value of conversations </li></ul><ul><ul><li>Learn from your audience </li></ul></ul><ul><ul><li>Inspires staff </li></ul></ul><ul><ul><li>Reinforces learning for audience </li></ul></ul><ul><li>Benefits of partnerships </li></ul><ul><ul><li>Reaching new audiences </li></ul></ul><ul><ul><li>Audience can incorporate you into their community more easily </li></ul></ul><ul><ul><li>Staff learn new skills and gain expertise </li></ul></ul>
    53. 54. <ul><li>Susan Edwards </li></ul><ul><li>Senior Writer/Editor </li></ul><ul><li>Web Group </li></ul><ul><li>J. Paul Getty Trust </li></ul><ul><li>[email_address] </li></ul><ul><li>http://whyville.net </li></ul>http://www.getty.edu/museum/research/metrics_evaluations/downloads/whyville_assessment_2007.pdf

    ×