Know Thyself: How Suffering Through An Existential Crisis Will Help you Plan Your Metrics
Report
Share
•2 likes•1,853 views
Know Thyself: How Suffering Through An Existential Crisis Will Help you Plan Your Metrics
•2 likes•1,853 views
Report
Share
Download to read offline
A presentation on coming up with institutional strategy before planning for metrics. Given as part of the "Clicks that Count: Achieve and Measure Online Success with Google Analytics & Adwords" workshop at Museums and the Web 2012, San Diego, CA, USA. museumsandtheweb.com Details the process through which the Smithsonian Institution Archives eventually decided on their measurements for a metrics dashboard, using case studies illustrating how the Archives came up with the goals, strategies, tactics, and finally, measurements that drive their dashboard. Consideration was also given to a broader analytics approach, using other techniques to gain insight pertaining into other, non-web program elements.
Recommended
More Related Content
What's hot
What's hot(20)
![Social Media 101 for Not-for-Profits [Webinar]](https://cdn.slidesharecdn.com/ss_thumbnails/socialmedia101-webinar2012-120420013953-phpapp02-thumbnail.jpg?width=768&fit=bounds)
Similar to Know Thyself: How Suffering Through An Existential Crisis Will Help you Plan Your Metrics
Similar to Know Thyself: How Suffering Through An Existential Crisis Will Help you Plan Your Metrics(20)
More from Smithsonian Institution Archives
More from Smithsonian Institution Archives(12)
Recently uploaded
Recently uploaded(20)
Know Thyself: How Suffering Through An Existential Crisis Will Help you Plan Your Metrics
- 1. Know Thyself: How Suffering Through an Existential Crisis Will Help You Create Your Metrics Plan Catherine Shteynberg Web, New Media, & Outreach Coordinator Museums and the Web 2012
- 2. Questions to answer: •What does the planning process look like? •What are some tools that are helpful in planning? •What can measuring do for me? Image: Smithsonian Institution Archives, SIA2009-0711
- 3. Overall Strategy Plan Self Evaluate • What do we think we do now? Listen • What do others think we do? What are others doing? • Who do we interact with? Audiences • Who do we want to interact with? Goals • What is your museum trying to accomplish? • Strategies: the plans you make to achieve your goals Strategy & Tactics • Tactics: the things you do to advance the strategy Measure & Retool • Develop GA/other metrics, Observe, Reevaluate, Repeat
- 4. Step 1: Self-Evaluation Complete an Audit Your Do you have a mission Activities statement? MISSION SO WHAT? WHO CARES?
- 5. Why Should We Listen? •Our best insights often come from without rather than within •Listening is another word for research •Is audience research just pandering to the public? Image: Smithsonian Institution Archives, MNH-1261B
- 6. Step 2: Listening Tools/Methods- What to track- Come up with Organization Keywords Keywords Twitter hashtags and lists Set up Google Reader Industry Keywords Google Alerts URLs of site, blog (on Twitter Search RSS Twitter, Google Alerts, Other Tracking? Wikipedia) Facebook Open Search Events, exhibitions, Twitter lists, individuals services, program names Other Free Monitoring Names of people in org. Tools? (e.g. SocialMention, Related organization blogs BackType, Technorati) Social Media blogs Feed it all into your Reader Influencers on social media channels Archive & Track Your evangelists mentions with Bookmarking
- 7. Listening: Do the Work for Others
- 8. What Does This Listening Achieve? Who’s talking? What are they saying? What do you do differently/successfully? What content is successful? Possible collaboration? Trends? Scan Quickly, Pick Out Patterns Image: Smithsonian Institution Archives, SIA2008-0888
- 9. Steps 3 and 4: Audiences & Goals Choose top Audiences (types, not demographics) & prioritize HINT:“General public” is not a legit audience Goals (2-3) K.I.S.S.
- 10. Case Study: Smithsonian Archives AUDIENCES & GOALS Our initial goals and audiences were too broad (“general public” and “increase influence”) Are your goals what you thought they were?
- 11. Bringing it all together Steps 4 and 5:Develop your strategies and tactics Step 6: Bring it all together to develop your measurements! Audiences & Strategies & Self-evaluation Listening Measurements Goals Tactics
- 12. What is a “money” measurement? Poor measurements vs. “money” measurements -# of views of photos -Percentage of images identified on Flickr Commons on the Flickr Commons per crowdsourcing campaign What’s your $$Money$$ if you’re not actually raising money? $ = specific goals for specific audiences, or a conversion Monitoring for success/failure of content vs. conversions
- 13. Get Inspired Case Study: Outcomes at the Archives Donation of Scopes Trial photographs Meet your grandmother in our collections People identify our Collections A Wikipedian-in-Residence who helps give millions more people access to our collections Our staff have become empowered to create content Images: DC church identification
- 14. And Finally: Do it all over again Self-Evaluate Measure & Listen Refine Strategy & Audiences Tactics Goals
- 15. Resources Reading: Templates: Wearemedia.org Social Media Audit Template, The Other Media "Mission Accomplished?“ by Linda Norris SMART Chart, Spitfire Strategies “Good Research Isn’t About Asking Audiences What Creating Your Organization's Social Media Strategy they Want,” asking audiences Map, Beth Kanter “Five Cool Twitter Search Tricks,” MakeUseOf We Are Media, Social Media Strategy Worksheet and “Spreadsheet Aerobics,” by Beth Kanter AIDS.gov New Media Strategy Map based on We Are Media Worksheet. “Actionable Listening,” by Beth Kanter “The Listening Primer” Social Media Listening Wiki, by Beth Kanter Tools: Google Keyword Tool Google Reader Google Alerts Sociable, Twitter RSS Feed Creator Diigo, or another bookmarking tool List of Social Media Listening, Monitoring, Measuring, and Management Tools, Social Media Listening Wiki, by Beth Kanter
Editor's Notes
- Introduce: also, work for the Archives—we preserve the history of the Smithsonian Institution itself—so strong collections in the history of science, history of museums, etc.Brian talked about how:-organizations can come up with goals, strategies, and tactics and how these help you come up with measurements-how to set up your GA dashboard-using your measurements to come up with benchmarks and to put the measurements into action by making changes based on the data you get back.I wanted to back up a step to give some insight into how setting up our metrics at the Archives gave us a chance to really solidify the Archives’ goals and think critically about who we are, what we want to do, and who are audiences are.And before I move on, may I say that Brian Alpert was indispensablethroughout our planning process, and I also want to add that much of this presentation was influenced by Beth Kanter’s writings on actionable measurement and creating social media strategy, and I’ll include a list of resources mentioned in this presentation, including some of her readings, at the very end.
- I wanted to address a few questions in this presentation:-What does the planning process look like? Hopefully, sharing our planning process will help you. The part before the metrics is in many ways the most difficult part. When we were going through our planning process it sometimes felt like an existential crisis because you’re trying to answer the question “Who am I?” and “Who do I want to become?” Very tough questions.-What are some helpful tools? I will also cover some tools and techniques that were helpful for us in our planning process.-Why should I measure? And finally, why should you spend your time measuring? Presumably, if you’re here you don’t need this question answered, but it’s worth mentioning that in addition to potentially leading to positive institutional change, metrics can be 1) an incredible motivator and 2) they can prove that the money/hours/manpower being spent on outreach or projects is worth it, or perhaps not worth it, so you can move onto better things.
- Here is a basic layout of what our metrics planning process looked like. I’ll just read through the elements of our planning process, some of which Brian touched upon earlier: Read through the above.Brian talked about goals, objectives, strategies, and tactics, but I wanted to speak about the part of the process that comes before that and how evaluating and listening can help you define your audiences and goals, which in turn will lead to the rest of your metrics plan.What I hope this starts to illustrate is that the purpose of planning is not to write a plan that gets buried on your shared drive, or to copy and paste your mission statement from 15 years ago while planning for your metrics. Planning is about revisiting your identity and having that existential crisis so that in the end you have truly identified what your organization wants to spend your time doing. Instead of assuming that you automatically understand what your mission is, you need to back up a few steps and evaluate why you’re doing what you’re doing, and if you’re indeed spending the time and effort on things that further your ability to actually serve your mission and the audiences you want to walk into your virtual or actual front door.Because the thing is, if your plan is bad, if it’s meaningless, then your measurements and your metrics dashboard are going to be meaningless.
- Complete a Social Media or Outreach AuditCompleting a social media audit will not only allow you to understand what kinds of outreach you’re already doing, but it will also clue you in to some of the measurements you might want to think through: how will you judge the success of your Facebook page?, is your tweeting actually helping more people learn about your exhibition?There are a lot of templates out there, but the example above happens to be a social media audit example created by The Other Media.What are you already doing? What do you think is working or not working? If you already have metrics, what are they telling you?Do you have a mission statement?Why do mission statements matter? You can’t move ahead with evaluation if you don’t know what your purpose is supposed to be in the first place. A good mission statement will create a compass for all of your work.Linda Norris of Uncatalogued Museum blog “if your mission is the frame for your work, or the sieve through which all your activities must pass, it just makes sense to have a mission that really matters.”To come up with a good mission statement, ask yourself SO WHAT?Via Norris: we were encouraged “to keep asking why or so what? Why do you collect things? Because no one else does. So what? Because they're disappearing. So what? The goal is to push, push, push until you really figure out the why--the meaning; who it's for; and what your lasting impact will be.”
- People usually stop after self-evaluation, but a lot of your best insights are going to come from outside your organization—from your audiences that you can listen to online and others in your field.What questions should you be asking while you’re listeningWhat do people think of your organization?What do others think is working or not working? What are the positive and negative sentiments?In looking at other organizations and what they’re saying and doing, do you see a niche emerging for yourself?Why do we listen?to be able to better serve your target audience by knowing what they're saying to others and to you.to be able to respond to and/or engage fans and criticsstay abreast of the latest development in your area of workSo much of our success depends on making great content. To get ideas for new content—you have to do research!To figure out if you’re doing what you said you would do!Is audience research a tool for pandering? There was a great blog post recently on “asking audiences,” an arts, museum, and education blog about this debate. I agree that if we asked our audiences what exhibits they wanted to see, there would be a lot of mummies and Monet, and we’d never learn anything new. But the audience research and the listening that I’m proposing above is something different. The article made the point:Research isn’t about “searching for direction from audiences” or asking what they want to see. It about trying to understand your organization and its vision in relation to the people it serves, and on a foundation of empathy with the lives they lead, the cares they have, and the things they’re inspired by.
- So, how do we actually do this listening and research? Reading up on the web can be overwhelming and a huge time suck if you don’t have things set up efficiently.Come up with keywords:Mentions of organization, director, your specialties, your URLs—I have a list on the right side of the screen (read through the list)Google Keyword, other keyword tools, tool to figure out other relevant industry keywords.Set up Google Reader, populating it with Google Alerts for keywords and with Twitter Search RSS feedsSet up other tracking to feed into your ReaderOther Free Monitoring tools include: BackType, Yahoo Pipes, HowSociable,SocialMention, etc.Services out there to help you find relevant Twitter lists, hashtags, and individual influencers in your field so you can have the latest news about what everyone is up to in your field.Track your mentions Tracking mentions on a social bookmarking tool like Diigo is great. It may sound like a bit much, but I bookmark the URLs of all significant press that our organization, our blog posts, our projects get. I can tag them by project, like this is blog press, or this is someone talking about our new online exhibition on the history of architecture at the Smithsonian. I also track social media research on best practices and techniques so I can pass this along as a resource to my co-workers.Tracking allows you to Feed articles to others.:it allows you to send all of those mentions out via RSS feed to other key individuals in your organization who don’t have time to do the listening. Tracking social media research by bookmarking and tagging good articles also allows others in your organization to keep on top of key research and trends without having to filter through the avalanche of information and it allows you to go back to that information at a later date as needed. If your director or manager can see the feed of this stuff, they too will start to understand.
- Who is talking and what are they saying? Discover key influencers, find out what they care about, and seek to influence them in turn. And what audiences are you reaching? The ones that you hoped? Are you tapping into audiences that you didn’t realize you had in the first place?So, I’ll give you the example of the Flickr Commons. Along with many other organizations around the world, the Smithsonian Institution Archives participates in the Flickr Commons. As part of this, we run the Smithsonian Flickr Commons and post no-known copyright images from around the Smithsonian to the Commons. In their own words, “The key goals of The Commons on Flickr are to firstly show you hidden treasures in the world’s public photography archives, and secondly to show you how your input and knowledge can help make these collections richer.” So, we have various sets of images from our collections there. Not too long ago, while doing my daily listening in my RSS feed, I came across the following tweet. I noticed this tweet where she’s linking to a set of the Archives’ images on the Flickr Commons. Ended up developing a relationship with this writing group and featuring them on our blog.Other questions to ask? And where are you having successes—on Facebook, Twitter, Foursquare?What are you doing differently? Do you have a niche that other similar museums don’t? How can you capitalize on it? And what are you doing well? Which content has traction, and which content is bombing? This one is more difficult to evaluate until you have some kind of measurement n place, but nevertheless most organizations have a sense of this or qualitative evidence of what is doing well.What trends are happening right now? E.g.: everyone is talking about Pinterest right now. Whether or not your organization is on there or not, it’s good to know that many museums are, or to see what people are pinning from your collections. The more you do this, the easier it will be to scan your daily listening quickly and see what patterns are emerging—what content is working, who you are interacting with the most, what trends are emerging in your field.By understanding the influence that various audiences or networks are having on your organization or how they’re interacting with your content—all of this might help you decide what you want your goals to be moving forward.
- With all of the listening you’ve been doing in the earlier slides, you should have a great sense of who your audiences are and what kind of content resonates with them. Are you going to expand those audiences? Do you need to add different audiences? What are you going to do to make that happen? As tempting as it may be, “general public” is not an audience. Even if you want to reach everyone, it’s simply not possible. You’ll have better success with something more specific. Similarly, 25-40 year olds in the Washington, DC area is not going to be helpful for us—think about the persona types of people that you want. For example, at the Archives, one of our primary audiences is researchers, ranging from PhD students who come into our reading room to do research, to Wikipedians and photo researchers on the Flickr Commons that we know we already engage.Do you have too many goals, are they too abstract? We’ve all heard those missions statements that read like: “To bring appreciation of the arts to everyone worldwide…”So, your goals need to tie directly into your audiences, and they need to be specific.You have to keep it simple—you can’t take on too many goals and audiences, or you’ll be overwhelmed and you won’t be able to measure anything. KISS—keep it simple stupid.
- Our own audiences were too broad. I mentioned the “general public” example on the last slide because that was a wish by our own leadership initially—but how do you measure that? It’s impossible both in the metrics dashboard and even in my daily listening.Our own goals were too broad. Increase influence is too broad, but increasing influence by becoming the definitive source on Smithsonian history will work.Web Team was new to the organization and we didn’t know what the Archives’ priorities were, so this was a great opportunity to get to know our organization better and to clarify—”definitive source on SI history,” and real focus on researchers.At the same time it’s difficult to track “researchers” online—technically anyone who comes to our site and does a search is a researcher, so it was important to come up with profiles of important researcher groups. For us, one of these important research subsections turned out to be Wikipedians and Flickr Commons folks. Flickr Commons was already a strong presence with lots of potential; Wikipedia was driving a ton of traffic to our site. So, now we had something specific to work with.
- Brian talked about developing strategies and tactics pretty extensively in his presentation, and I gave our own example from the Archives, so I won’t talk about that part extensively. But:Strategies: the plans you make to achieve your goalsTactics: the things you do to advance the strategyBut once you’ve developed those, you’re ready to develop your measurements. So, all of these steps that we’ve now talked through, they each inform one another, and they all come together in the end to give you the measurements.
- So, there’s one more point that I want to make when it comes to measurements: what is a poor measurement versus a money measurement?So many social media measurement articles and tools talk about driving sales, making money. If you’re a museum and a non-profit, and in our case, you’re not focused on fundraising, what is your money?I’ll give you an example from our own experience. Look at the difference between measurements in the example above—it’s the difference in knowing what our popular images are on the Flickr Commons versus fulfilling our goal of increasing the understanding of the diversity and relevance of resources available at the Archives by helping to create and share more user-generated content and to increase the public’s deep interaction with our collections.You can look at # of views a day, but you’ll have no idea who comes, how often, why are they looking. By going through all of the planning steps will help you come to your money measurement. Steps 1, 2, 3, and 4 help you come up with your money measurements, which are basically specific goals for specific audiences. Also want to elaborate on measurements and the difference between “conversions” and seeing which content is doing well, which is a good thing to measure, but is different than a conversion.It’s okay to do some quantity of stuff measurements, as they help us gauge general performance and spot trends (for example, is your website traffic at about normal? What are your traffic ebbs and flows throughout the year? Are your Facebook likes going up, or did you lose a bunch of fans recently? However, these measurements aren’t conversions—these visitors aren’t doing something specific that you want them to, and that shows serious dedication.There’s a difference between the everyday listening that we need to do to gauge how we’re doing, basically what you could call your qualitative metrics; and your quantitative metrics that Brian mentioned. So, to revisit the idea of knowing which content is doing well, you can use your metrics to understand some of your stuff that’s getting traction (i.e. Twitter visitors that click through to your blog and then go deeper than x # of pages), but a lot of that information is going to come via your informal monitoring. That’s why you don’t want to have an intern doing all of your listening—because you’re going to lose a lot of invaluable information and a huge knowledge base about your audiences and your content.So, though coming up with these money measurements is key to setting up your metrics plan and dashboard and to making sure that you’re converting your visitors into big supporters of your organization, both quantitative and qualitative measurements are necessary. The information that can come out of informal monitoring and the case studies that can be developed out of your qualitative metrics can be extremely powerful, which is what I want to show you next.
- So, I wanted to give you some powerful examples of actual outcomes that have come out of our listening, and our informal, qualitative measurement, and using that information to inform our content. I’ll focus on two of the audiences that I’ve been talking about, and that we wanted to target—Flickr Commons enthusiasts and Wikipedians:Scopes Trial Example. I’m sure all of you are familiar with the infamous 1925 Scopes Trial that debated the teaching of evolution in schools. We have some photos from the trial on the Flickr Commons –informal snapshots that were discovered a few years ago in our collections by a researcher. These were quite popular on the Commons, and so we did more outreach with them. The new press, led to a woman seeing them on the Flickr Commons and then contacting us to donation of new images from a woman whose father attended the trial in Tennessee—her only stipulation? That we digitize them and make them available to the world.Women in Science IDs- As part of a Flickr Commons outreach, we’ve been posting women scientists and from our Science Service collections to the Flickr Commons and blogging about them/having crowdsourcing campaigns during Women’s History Month. A woman was id’d and that was confirmed by her granddaughter who encountered her on the Commons. As a result, she came into the Archives and gave us more information—in return she recently received a tray painted by her grandmother. Elizabeth Sabin Goodwin's granddaughter, Linda Goodwin EisenstadtLocation IDs of DC Photos- Gruber photo IDs of Washington DC—within a few days, we had locations of almost all of these locations from local experts and enthusiasts.Wikipedian in Residence- Sarah StierchOur archivists, who weren’t traditionally content creators, became excited about making content—good metrics make your workflow easier!These things wouldn’t have come without engagement with our audiences and with listening. These stories motivate us to go and do more. We are literally taking the research community and not only connecting to them and making them advocates, but we’re co-opting them into our organization. They work with us, and our audiences have become a part of our team in a way—they’ve become the co-producers of our mission.
- So, again, to reiterate, you go through the process, see what works, and then, you start all over again, refining as your audiences change and the goals of your audience change until you get the outcomes that you desire. We still need to see what kinds of refinement can come out of our more sophisticated dashboard—which will allow us to much more quickly understand who our engaged audiences are.