Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Michael Edson @ J. Boye 2011: Jedi Mind Tricks for Measuring and Optimizing Lofty Goals (updated from e-Metrics Summit)

on

  • 4,265 views

Updated version of a talk first given at the e-Metrics Marketing and Optimization Summit, Washington, D.C., October 5, 2010. This version is for the J. Boye conference, Philadelphia, PA, 5-4-2011. ...

Updated version of a talk first given at the e-Metrics Marketing and Optimization Summit, Washington, D.C., October 5, 2010. This version is for the J. Boye conference, Philadelphia, PA, 5-4-2011.

The talk spells out the lofty goals of the Smithsonian Institution and the Smithsonian Commons project, and then describes some of the unconventional measurement methodologies we're using to figure out what to do and how to do it.

J. Boye Conference page: http://jboye.com/conferences/philadelphia11/program/speakers/michael-edson/#presentation

Statistics

Views

Total Views
4,265
Views on SlideShare
4,261
Embed Views
4

Actions

Likes
7
Downloads
26
Comments
0

3 Embeds 4

http://paper.li 2
http://jboye.com 1
http://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • http://flickr.com/photos/dotcommunist/355530076/
  • http://flickr.com/photos/dotcommunist/355530076/
  • http://flickr.com/photos/jay_que/2747368547/
  • http://flickr.com/photos/jay_que/2747368547/
  • http://flickr.com/photos/daveseven/2534461433/
  • http://flickr.com/photos/daveseven/2534461433/
  • http://flickr.com/photos/daveseven/2534461433/
  • http://www.youtube.com/watch?v=FNXahIoXMw8&feature=related
  • http://www.pewinternet.org/PPF/r/107/presentation_display.asp

Michael Edson @ J. Boye 2011: Jedi Mind Tricks for Measuring and Optimizing Lofty Goals (updated from e-Metrics Summit) Michael Edson @ J. Boye 2011: Jedi Mind Tricks for Measuring and Optimizing Lofty Goals (updated from e-Metrics Summit) Presentation Transcript

  • UpdatedJedi Mind Tricks for Less fat : more fiberMeasuring and Optimizing Lofty Goals Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • Preamble@mpedson Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • Preamble @mpedson slideshare.net/edsonmsmithsonian-webstrategy.wikispaces.com Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • Preamble @mpedson slideshare.net/edsonmsmithsonian-webstrategy.wikispaces.com Get the full story here Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • Preamble @mpedson slideshare.net/edsonmsmithsonian-webstrategy.wikispaces.com …and here Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • PreambleThe Smithsonian Commons www.si.edu/commons Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • Preamble He’s not a spokesman! And he doesn’t set policy Michael Edson Director, Web and New Media Strategy Smithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson
  • Before we talk about measuring and optimizing,we need to talk about the lofty goals…
  • Smithsonian Strategic Plan http://www.si.edu/about/ Four Grand ChallengesSmithsonian Secretary G. Wayne Clough
  • Smithsonian Strategic Plan http://www.si.edu/about/ Four Grand Challenges 1. Unlocking the Mysteries of the Universe 2. Understanding and Sustaining a Biodiverse Planet 3. Valuing World Cultures 4. Understanding the American Experience
  • Smithsonian Strategic Plan http://www.si.edu/about/ Four Grand Challenges 1. Unlocking the Mysteries of the Universe 2. Understanding and Sustaining a Biodiverse Planet 3. Valuing World Cultures 4. Understanding the American Experience
  • Smithsonian Strategic Plan http://www.si.edu/about/ Four Grand Challenges 1. Unlocking the Mysteries of the Universe 2. Understanding and Sustaining a Biodiverse Planet 3. Valuing World Cultures 4. Understanding the American Experience
  • Smithsonian Strategic Plan http://www.si.edu/about/ Four Grand Challenges 1. Unlocking the Mysteries of the Universe 2. Understanding and Sustaining a Biodiverse Planet 3. Valuing World Cultures 4. Understanding the American Experience
  • Smithsonian Strategic Plan Strategy Schmatergy! We’ve seen strategy Four Grand Challenges before! 1. Unlocking the Mysteries of the Universe 2. Understanding and Sustaining a Biodiverse Planet 3. Valuing World Cultures 4. Understanding the American Experience
  • Smithsonian Strategic Plan Four Grand Challenges No! This is about solving big hairy 1. Unlocking the Mysteries of problems— “work the Universe that matters” 2. Understanding and (via @timoreilly) Sustaining a Biodiversehttp://radar.oreilly.com/2009/01/work-on-stuff-that- matters-fir.html Planet 3. Valuing World Cultures 4. Understanding the American Experience
  • c
  • c This is big, audacious, importa nt stuff.This is work! This is relevance earned through a job well done.
  • First order questions
  • First order questions• Where is this work going to take place?• What kind of organization, infrastructure, platforms will be needed to support it?• What is the organizational change model? How will change happen?• Who will be the innovators? The connectors? The drivers of change?• …and how do you measure it?
  • Web andNew Media Strategy
  • http://smithsonian-webstrategy.wikispaces.com Web and New Media Strategy More in Fast, Open, and Transparent: developing the Smithsonian’s Web and New Media Strategy http://www.slideshare.net/edsonm/michael-edson-the-smithsonian- web-and-new-media-strategy-what-it-is-how-we-made-it-and-why-it- makes-a-difference-3656578 And http://www.archimuse.com/mw2010/papers/edson/edson.html
  • http://smithsonian-webstrategy.wikispaces.com/
  • The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons— a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.http://smithsonian-webstrategy.wikispaces.com/
  • The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons— a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.http://smithsonian-webstrategy.wikispaces.com/
  • The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons— a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.http://smithsonian-webstrategy.wikispaces.com/
  • The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons— a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.Old Learning Model New Learning Model http://smithsonian-webstrategy.wikispaces.com/
  • Balancing autonomy and control within the Smithsonian.rt of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.http://smithsonian-webstrategy.wikispaces.com/
  • Balancing autonomy and control within the Smithsonian.rt of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities. Innovation at the Edges ---A commons in the middle http://smithsonian-webstrategy.wikispaces.com/
  • …and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.http://smithsonian-webstrategy.wikispaces.com/
  • …and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.http://smithsonian-webstrategy.wikispaces.com/
  • What is a Commons?
  • What is a Commons?A set of resources maintained in the publicsphere for the use and benefit of everyone
  • What is a Commons?
  • What is a Commons?The Anti-Commons…
  • What is a Commons?An organized workshop where raw materials can be found and assembled into new things.
  • What is a Commons?Imagining a Smithsonian Commons• http://www.slideshare.net/edsonm/cil-2009-michael-edson-text-version Museum Commons: A professional interaction http://www.slideshare.net/edsonm/museum-commons-a-professional- interaction-museums-and-the-web-2010-michael-edson-and-rich-cherryWhat is a Commons?
  • What is a Commons?Imagining a Smithsonian Commons• http://www.slideshare.net/edsonm/cil-2009-michael-edson-text-version Also… Museum Commons: A professional interaction http://www.slideshare.net/edsonm/museum-commons-a-professional- interaction-museums-and-the-web-2010-michael-edson-and-rich-cherry A 15-minute talk at the Walker Art CenterWhat is a Commons? Text/footnotes: http://www.slideshare.net/edsonm/m-4402558 Video (starts at minute 12): http://channel.walkerart.org/play/opening-the-field/ Updated 6/21/2010
  • So now we’re at a moment…The Grand ChallengesThe Smithsonian Commons How do you talk about these things in measurable business terms?
  • So now we’re at a moment…The Grand ChallengesThe Smithsonian Commons How do you use “information” to persuade, guide, inform ?
  • So now we’re at a moment…The Grand ChallengesThe Smithsonian Commons How do you measure things that are… difficult to measure?
  • UpdatedJedi Mind Tricks for Less fat : more fiberMeasuring and Optimizing Lofty Goals
  • Jedi Mind Trick #1Show your users to management
  • RelevanceQ: Have you ever visited a Smithsonian Web site? http://www.youtube.com/watch?v=N5x4Sga0d1s
  • Jedi Mind Trick #2Use Organic Search and Alexa
  • Relevance Unexpected Rivals in Google Search Google Images Wikipedia Ocean.comDiscoveryeducation.com NASAEnchantedlearning.com
  • RelevanceUnexpected Rivals in Reach Enchantedlearning.com Google Images Wikipedia si.edu Ocean.com …so much more reach than SI discoveryeducation.com that we don’t even show up on the graph… ocean.com
  • RelevanceUnexpected Rivals in Reach (July – Sept, 2009) Enchantedlearning.com si.edu discoveryeducation.com ocean.com
  • RelevanceUnexpected Rivals in Reach (April, 2011) si.edu discoveryeducation.com ocean.com
  • Enchantedlearning.com Relevance is a two person team, with online reach that’s challenging theUnexpected Rivals in Reach (July – Sept, 2009) world’s largest museum and research complex! Enchantedlearning.com si.edu discoveryeducation.com ocean.com
  • RelevanceTraffic relative to peers si.edu + 2% reach Wikipedia.org + 6% reach MoMA.org + 10% reach npr.org + 7% reach
  • Jedi Mind Trick #3Humble yourself with brandtags.net
  • RelevanceBrand IdentityBrandtags.netWe are the 560th of 928 brands
  • Jedi Mind Trick #4Highlight your competitors
  • RelevanceWe’re competing with… everybody!
  • RelevanceWe’re competing with… everybody!
  • RelevanceWe’re competing with… everybody!
  • Jedi Mind Trick #5Use 3rd party research
  • RelevanceThe Demographic Tsunami Online Content Creation by Age 90 80 70 Percentage 60 50 40 30 20 10 0 Ages 12- Ages 18- Ages 30- Ages 39- Ages 49- Ages 61- Ages 70+ 17 29 38 48 60 69 Internet users November 2007 data: Pew Internet and American Life Project
  • RelevanceThe Demographic Tsunami“Everything we hear from people we interview is that today’s consumers draw no distinctions between an organization’s Web site and their traditional bricks-and-mortar presence: both must be excellent for either to be excellent.” Lee Rainie Pew Internet and American Life Project
  • Jedi Mind Trick #6Demonstrable feedback from users
  • 1,187 positive votes/comments on prototype (out of 1,225 total votes)
  • It’s just Google1,187 positive votes/comments on prototype up Spreadsheet hooked (out of 1,225 total votes)to a form.
  • It’s just Google1,187 positive votes/comments on prototype up Spreadsheet hooked (out of 1,225 total votes)to a form.See all responses on our wiki athttp://smithsonian-webstrategy.wikispaces.com/Public+Comments+on+Smithsonian+Commons+Prototype+0001-0200
  • “Now over 90, age limitations anddistance make physical visiting almostimpossible. The Smithsonian Commonsnow opens wonderful vistas.”Donor/member/supporterComment #785, 8/4/2010
  • Jedi Mind Trick #7Show validation from VIP’s
  • Jedi Mind Trick #8Measure end-user satisfaction
  • Jedi Mind Trick #9Bake measurement into the platform from the start
  • Jedi Mind Trick #9Bake measurement into the platform from the start Smithsonian Office of the CIO is studying its as-is measurement/analysis program
  • Survey Highlights Are there web-user activities NOT being measuredWhich web analytics tools are you using? that you WANT to measure? 10% NO 17% YES 29% 83% 61% What are the some of the most valuable behaviors a user can perform on your site? 80
  • The importance of measurement/analysis and UX is laced throughout the strategyhttp://smithsonian-webstrategy.wikispaces.com/
  • Project Charter (draft)http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
  • Project Charter (draft) and Measurement analysis resources in the project plan from the very starthttp://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
  • Project Charter (draft) and Measurement analysis resources in the project plan from the very startEight measurable goals describedhttp://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
  • Project Charter (draft) This is what the charter says…
  • Project Charter (draft)Measurable Project GoalsThe goal of the Smithsonian Commons project isto stimulate learning, creation, and innovationthrough open access to Smithsonian research,collections and communities.
  • Project Charter (draft)Measurable Project GoalsPrecise, measurable goals are a hallmark ofsuccessful project governance, but these kindsof outcomes are notoriously hard to define andmeasure in practice: seeking to define them tooprecisely or in too nuanced a way early in theproject can be as counterproductive as notmeasuring them at all.
  • Project Charter (draft)Measurable Project GoalsPrecise, measurable goals are a hallmark ofsuccessful project governance, but these kindsof outcomes are notoriously hard to define andmeasure in practice: seeking to define them tooprecisely or in too nuanced a way early in theproject can be as counterproductive as notmeasuring them at all.
  • Project Charter (draft)Measurable Project GoalsTherefore, the project will initially focus onthree measurements: the amount ofinformation in the Smithsonian Commons, theamount of activity on the SmithsonianCommons, and the satisfaction of people whouse the Smithsonian Commons.
  • Project Charter (draft)3. Measurable Project GoalsTherefore, the project will initially focus onthree measurements: the amount ofinformation in the Smithsonian Commons, theamount of activity on the SmithsonianCommons, and the satisfaction of people whouse the Smithsonian Commons.
  • Project Charter (draft)3. Measurable Project GoalsTherefore, the project will initially focus onthree measurements: the amount ofinformation in the Smithsonian Commons, theamount of activity on the SmithsonianCommons, and the satisfaction of people whouse the Smithsonian Commons.
  • Project Charter (draft)Measurable Project GoalsTherefore, the project will initially focus onthree measurements: the amount ofinformation in the Smithsonian Commons, theamount of activity on the SmithsonianCommons, and the satisfaction of people whouse the Smithsonian Commons.
  • Project Charter (draft)The specific, measurable 5-year goals1) 100 million items in the commons…
  • Project Charter (draft)The specific, measurable 5-year goals1) 100 million items in the commons…2) 100 million transactions a year…
  • Project Charter (draft)The specific, measurable 5-year goals1) 100 million items in the commons…2) 100 million transactions a year…3) High end-user satisfaction, both internallyand internally, as measured through the ForeseeResults measurement tool or equivalent
  • Project Charter (draft)The specific, measurable 5-year goals1) 100 million items in the commons…2) 100 million transactions a year…3) High end-user satisfaction, both internally “Bits in: bits out”and internally, as measured(via Brewster Foresee through the Kahle)Results measurement tool or equivalent
  • Project Charter (draft)Five additional measurable goals guide theproject. The Smithsonian Commons will,4) Increase reuse of Smithsonian resources5) Be family-safe and school-safe6) Demonstrate new revenue models7) Drive traffic to unit websites8) Help eliminate cost and work at the unit level
  • Jedi Mind Trick #10Measurement is only one piece of the equation
  • Jedi Mind Trick #10Measurement is only one piece of the equation It’s about understanding and improving user experience
  • Jedi Mind Trick #10Measurement is only one piece of the equation You need a team
  • Smithsonian Commons Prototype So let’s see what that’s like!
  • Thanks!!! Michael Edson Director, Web and New Media StrategySmithsonian Institution, Office of the CIO edsonm@si.edu | @mpedson