The Engagement Metric: Identifying and measuring audience engagement efforts

2,759 views

Published on

Journalists have a lot to learn from other disciplines about tracking what works. We're not used to gauging our success in ways more sophisticated than ratings or circulation numbers, and we're behind the measurement curve. But these days, it's hard to value what you can't measure. And as newsrooms grapple with how to make room in tight budgets for audience engagement, it's natural that they'd also wonder what the return on that investment might be.

Published in: Business, Technology, Education

The Engagement Metric: Identifying and measuring audience engagement efforts

  1. 1. A resource for newsrooms:Identifying and measuring audience engagement effortsOrganized and edited by Joy Mayer and Reuben Stern of ideas. Presented here is an edited version of that document. Its not intended as a comprehensive guide to engagement, but instead as aJournalists have a lot to learn from other disciplines about tracking what sampling of practical ways to be strategic in our efforts.works. We’re not used to gauging our success in ways more sophisticated Many of the participants have jobs that involve audience outreach andthan ratings or circulation numbers, and we’re behind the measurement engagement, and they talked about their need to be able to show thecurve. But these days, it’s hard to value what you can’t measure. And as concrete results of their work. They also felt the need to persuade theirnewsrooms grapple with how to make room in tight budgets for audience colleagues and their bosses to value engagement. Metrics are key. And byengagement, it’s natural that they’d also wonder what the return on that metrics, we dont just mean web analytics. The number of people who showinvestment might be. up for a newsroom tour is a metric. The number of people who “Like” aWith these issues in mind, a group of journalists interested in audience Facebook page is a metric. The number of story ideas or new sourcesengagement gathered in early May 2011 at the Reynolds Journalism resulting from a community conversation is metric. Academic research alsoInstitute in Columbia, Mo., to talk specifically about measurement. Some of has a lot to teach us about measurement beyond counting. A contentthe participants were widely recognized experts. All were working to effect analysis, for example, can help us track the civility of conversations or thechange in their traditional newsrooms or products. They came because diversity of sources.they believe that as news organizations fight for survival, a more connected And if its information we seek, dont overlook what newsrooms alreadyrelationship with their communities should be valued, and therefore know how to do: report. If we want to know if our work is making ameasured. They were joined by smart folks from other disciplines who difference in the community, what if we assigned reporters to find out,shared their time to help guide the discussions and share their expertise. whether we publish it or not?Our multi-disciplinary group (participant bios are online at As you read this document, we encourage you to keep a particularhttp://www.rjionline.org/events/engagement-metric-bios) focused our question in mind: What do you wish you knew about your relationship withconversations around specific strategies for audience engagement, what your community?their value is to the news organizations, and how the success of the effortscan be evaluated. We spent a full day creating a communal spreadsheetTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 1 of 1
  2. 2. IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSA FEW NOTES ABOUT THIS REPORT PLANNING FOR ENGAGEMENT 1. This is not a guide to engagement possibilities. Many ideas we discussed In moving forward with any engagement efforts, it might help to considerdidnt make the cut, and many more didnt find their way into the these suggestions:conversation. This is a list of strategies we discussed that we thought might • Develop a written strategy for the organizations efforts.be measurable. • Start every project with a measurable goal. In 12 months, six months, one 2. Theres no one-size-fits-all solution. What works in one organization might week or one day, what would success look like?not work in another. Tactics, and therefore metrics, must be specific to yournewsroom, and perhaps to individual projects. • Consider whether your goals align with any other community organizations goals, and consider partnerships. 3. What success looks like in engagement is continually evolving. Onceyouve figured out Twitter, the world will be on to something else. When • Build capacity within the organization (hire people who knowyouve made inroads with a specific group of people in your community, engagement, assign specific duties, train staff, etc.).youll be starting over with another. The industry is changing rapidly, as is ourculture. Journalists need to learn to adapt more quickly, to be comfortable • If engagement is a priority, it should be valued in the newsroom. Considerwith shifting definitions of success and to invest in continued strategic including engagement strategies in the performance evaluation ofplanning. individual journalists, and of departments. Consider hiring people (perhaps even non-journalists) with different expertise, such as analytics, marketing or 4. Some of our measurement suggestions are easy to implement now. research. Embed them in the newsroom, and hold them accountable for(Example: Use outbound links, not just incoming links, to track whether were newsroom training and culture changes. Consider allocating staff time forthe first stop for someone online or the last). Others would take some audience interaction and understanding, and for thoughtful strategizing, assophisticated knowledge or software. (Example: What if our metadata had Google does with its 20 percent innovation time.fields for things like story idea genesis, so we could track the benefits oflistening to diverse voices?) Others are more like dreams than suggestions • Consider learning from disciplines like marketing, analytics, nonprofits,— things we wish we could measure but need help figuring out how. anthropology, social media, civic activism, etc. Whether you want to(Example: Making a network map of our newsrooms community launch a community project, learn about your community, increase theconnections and relationships.) communitys investment in your work or change the perception of your brand, there are folks who know more about how to do it than you do. 5. This is a working document. If you have ideas youd like to see reflected,resources that should be shared or clarifications to make, please let us • Identify specific communities in which to make inroads. Consider a long-know. Email Joy Mayer at mayerj@missouri.edu. term investment in a specific community (community of interest or geography or socio-economic level or education or any other unifyingThis report presents some general suggestions for getting started and being characteristic), with a goal of addressing that communitys needs, or ofstrategic about engagement. Then it moves to the specific engagement transitioning individuals from non-participants to collaborators.ideas, broken into three broad categories. The categories were developedby Joy Mayer after dozens of interviews conducted during her fellowship at • Build community profiles of target communities, like the CIA Worldthe Reynolds Journalism Institute over the 2010-11 academic year. Joy Factbook reports on different countries.asked professionals about audience engagement — what it meanttheoretically, and also practically. She then took the strategies and tips andgrouped them using more specific terminology. There is undoubtedlycrossover among the three broad categories, but our hope is that they canserve as a starting point for talking about the purpose of engagement. Thepages that follow present a summary of what each category means alongwith concrete examples of what each might look like in action.THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 2 of 2
  3. 3. IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSLEVEL 1: COMMUNITY OUTREACHOutreach includes efforts to share ourselves, our expertise and our content with our community. It involves: Taking the content tothe audience, rather than hoping theyll find us. Identifying information needs, catering our products to meet them and distributingthem in a way that makes sense. Being willing to participate in the community as individuals, building connections andpersonalizing our brand. Inviting the community to get to know our people and our processes. Enriching our community, sharingour own knowledge and supporting other community enrichment efforts.STRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMBe visible in the • Spend time being • Build awareness and value of your • Overall web traffic numbers (specifically is percentage of traffic coming from sitescommunity visible/available in the real-world brand other than search engines increasing?) community (coffee shop hours, • Make your organization more • Number of stories generated from being out in the community (metatag on such co-working spaces, libraries, etc.) approachable stories to give credit to process of reader input as well as the journalist) • Consider the relationship that • Number of new sources gained/quoted in stories (content analysis) • Make your organization more relevant the editor of a small-town weekly to community newspaper might have with his • Breadth of range of sources in news stories (content analysis) community, and imagine building • Increase audience/community loyalty • Number of contacts listed in newsroom database (e.g., Public Insight Network or those kinds of connections. • Improve trust/credibility, especially other tracking tool) • Create a street team within niche communities • Benchmark reporters awareness of whom to call on specific issues (pre- and specifically to be out in • Lay a foundation for conversation and post-testing) targeted/underrepresented parts collaboration of community introducing your • Are we getting more PR notices, invites from new communities/organizations, organization and registering • Reduce risk that labor-intensive stories etc.? This indicates increased awareness and/or relevance (track incoming people for your site might be missed by key target notices) audiences, or by the people who would • Are reporters getting called back by sources more frequently/quickly? Are • Attend social events with no most benefit from the content journalistic agenda but instead reporters having easier time finding real people to go on record for stories? just to meet people and be • Help reporters do their jobs more (ongoing newsroom tracking/survey) present (Tweet-ups, topical efficiently by increasing likelihood of • Number of touchpoints, e.g., # of people stopping by coffee shop open hours events, community gatherings, callbacks from sources and making over time; # of commenters on web site, etc. etc.) citizens more likely to go on record • Spread of content via referrals (track # of retweets; traffic from Facebook and • Show presence in online • Increase job satisfaction among blogs, number of people emailing links to stories, etc.) communities where people are reporters engaging by commenting on • Changes in participation and tone of chats (textual analysis of chat transcripts) discussion threads, adding to • Can you measure truth and authenticity of a story to the people who are Twitter hashtag streams, etc. and covered in the story? (survey) thereby help connect people with information/resources the • How do you measure actual impact, for example did someone go to jail? news organization knows about. change a policy? receive attention by the group-in-charge? community problem get fixed? (follow-up reporting after stories publish) • Reach out specifically to the • Has volume of conversation about an issue changed since the story appeared? people who your stories apply to. (content analysis, pre- and post-publication online search analysis, survey research, For example, find the Facebook follow-up research, track movement of stories through social networks like New pages of the people/groups your York Times tool being developed) stories apply to, or find the listserve people use to discuss the • Do stories sourced in the community outperform other stories? (compare web issue. Dont just be on a platform, traffic, referral numbers) be THE place on the platform for • Number of new advertisers (simple count, along with qualitative assessment from your story. ad sales team about what they are hearing while on new sales calls) • Map issue network and centrality of publication to the issueTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 3 of 3
  4. 4. LEVEL 1: OUTREACH (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSSTRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMActively make • Meet with key connectors in the • Improve knowledge of audience • Overall web traffic numbers (specifically is percentage of traffic coming from sitescontact with community(ies) to seek feedback needs so you can deliver more relevant other than search engines increasing?)key on coverage and community materials • Number of stories generated from being out in the community (meta-tag on suchstakeholders/ needs, as well as spread the word stories to give credit to process of reader input as well as the individual journalist?) • Gain insight/connection topotential that you are trying to reach out community/issues/trendspartners and be more inclusive • Number of new sources gained/quoted in stories (content analysis) • Get more and better story ideas • Number of connections made (size of rolodex) • Ask specifically what about the Build relationships/loyalty/stickiness with news coverage is not meeting community influencers • Breadth of range of sources in news stories (content analysis) their specific needs, not including their points of view and/or not • Identify new sources of expertise • Number of contacts listed in newsroom database (e.g., Public Insight Network or accurately representing their other tracking tool) • Expand range of voices present in your communities. content/coverage • Benchmark reporters awareness of whom to call on specific issues (pre- and • Get the word out to and/or post-testing) • Become catalyst for community meet with other content improvement • Are we getting more PR notices, invites from new communities/organizations, producers (bloggers, etc.) as well etc.? This indicates increased awareness and/or relevance (track incoming as other groups working to help • Build community investment in the notices) community (nonprofits, public success of your news organization agencies, etc.) • Are reporters getting called back by sources more frequently/quickly? Are • Help strengthen civic engagement in reporters having easier time finding real people to go on record for stories? • Reach out to communities issues important to community by getting (ongoing newsroom tracking/survey) online and solicit their help in more people involved in the community directing content to the people dialogue • Number of touchpoints, e.g., # of people stopping by coffee shop open hours in their networks who might over time; # of commenters on web site, etc. • Get content out to new want/need it. audiences/attract new customers (this • Spread of content via referrals (track # of retweets; traffic from Facebook and may lead to attracting advertisers who blogs, number of people emailing links to stories, etc.) want to reach these target communities) • Changes in participation and tone of chats over time (textual analysis of chat transcripts) • Can you measure truth and authenticity of a story to the people who are covered in the story? (survey) • How do you measure actual impact, for example did someone go to jail? change a policy? receive attention by the group-in-charge? community problem get fixed? (follow-up reporting after stories publish) • Has volume of conversation about an issue changed since the story appeared? (content analysis, pre- and post-publication online search analysis, survey research, follow-up research, track movement of stories through social networks like New York Times tool being developed) • Do stories sourced in the community outperform other stories? (compare web traffic, referral numbers) • Number of new advertisers (simple count, along with qualitative assessment from advertising sales team about what they are hearing while on new sales calls) • Map issue network and centrality of publication to the issueTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 4 of 4
  5. 5. LEVEL 1: OUTREACH (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSSTRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMInvite people • Solicit user feedback via story • Increase range of content, voices, • Increase in number of comments, submissions and/or chat room commentsinto the commenting, chat rooms, ideas in the coverage/content (count)process/ submissions, etc. • Validate views and opinions of • Increase in number of different people commenting, submitting, or joining chatworkflow • Include links to other credible individuals by helping them be heard rooms (count) sources of information they are (thus increasing users perception of your • Improvement in quality or tone of comments or chat room discussions (textual helping inform your audience organizations value) analysis) • Organize contests for • Increase appreciation/under- • Track where money/advertising/etc. comes from - did they come from an submissions of specific types standing/acceptance of what your aligned outreach effort. organization does • Making your office a place • Measure if levels of awareness/trust/positive attitude increase (survey) where people can and want to • Increase willingness of community to Measure levels of engagement the audience members take (track activity on site come to accomplish their own collaborate with your organization along with attendance at events, etc. Could use some sort of CRM tool to help goals (meeting space? coffee? with this) • Make it easy for people to see how internet cafe? library of some they could get involved. sort?) • Publish a ladder of participation, or suggestions for all the ways people can interact with your organization, with varying levels of commitment or investment required.Sponsor or • Community events sponsorship • Gain exposure to potential new • Measure attendance and empty beer bottles (the group was note really kiddingorganize audience on this one!) -partnering with acommunity nonprofit/cause to support an • Deepen relationship with audience • Measure revenueevents existing event (e.g. donating • Exit surveys (did you have fun, how was the food, what is your opinion of us, etc) • Garner goodwill from other money to and sponsoring and organizations in the community (could • Measure chatter/buzz - tweets/flickr photos. attending local Relay for Life lead to raising positive attitudes about event) • "How did you find out about us" - survey. your brand as well as cooperation with -organizing events of your own, news sourcing and coverage) • Measure who came to your content after the party, i.e. sign people up at event some around journalism and and then track their activity on site after the fact. • Have fun and create a shared positive some just for brand awareness experience and money-making (like singles • Specific url or QR code: Tracks traffic elsewhere based off that initial landing events, community-wide games, • Establish self as vital community page. hacks/hackers) institution • Measure levels of engagement the audience members take (track activity on • Generate revenue (using either site along with attendance at events, etc. Could use some sort of CRM tool to help freemium or flat charge model) with this) • Make organization easier to approach (could lead to more willingness to suggest ideas, be part of news coverage, etc.) • Generate new sources • Improve coverage of event(s) • Reengage journalists as part of community rather than separate from it • Improve quality of life in community by adding things of interest for people to doTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 5 of 5
  6. 6. LEVEL 1: OUTREACH (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSSTRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMShare expertise • Community events sponsorship • Position organization as an entity that • Number of people who sign up and percentage who subsequently participate,with audience cares about improving the community and how often (counts of attendees to trainings, along with some sort of tracking) -partnering with amembers and nonprofit/cause to support an • Build trust, credibility, other positive • Can you track the usage habits on the website of participants who do somethingwith groups affect towards your organization beyond passive reading? (require registration and track use profiles) existing event (e.g. donatingthat have money to and sponsoring andsimilar mission • Gain added validation of your • How many reader bylines appear on the site? How much content from trainees attending local Relay for Life appears elsewhere on the web? (count)of creating/ organizations value by association with event)supporting other trusted/respected organizations in • People may pay for training workshops through fee-for-service on narrativecommunity -organizing events of your own the community storytelling/master technique/topic classes, so income could be a metric • Develop sense of gratitude and desire • Number of partnerships and/or number of members of other organizations to positively reciprocate from people reached via new partnerships (count) you assist • Track referral link traffic by providing different link URLs to different partners to • Increase civic engagement which forward to their constituencies and then seeing how much traffic comes through leads to increased need for the each of those forwarded URLs. information contained in your journalism • Level of credibility, trust, etc. (survey) • Increase civic capacity by helping make other community institutions • Amount of staff time invested in individual relationship with a partner organization stronger (should diminish over time as relationship becomes more solid)Amount of activity related to a partnership (should increase over time) • Expand the network of contacts for your journalistic staff • Number or complexity of issues or problems "solved" or addressed via a partnership. • Expand the audience for your journalism • Growth in audience for partner organizations communications (count) Assess community response to collaborative reporting on gaps in services or other • Asking for ideas and suggestions in (follow-up reporting) non-emergency situations increases likelihood of getting information quickly in breaking news circumstances • Helps news organization live up to a mission of making sure community members get all the information they need to fully participate in the democracy • Build news organizations understand of the information needs of the community; also helps news org better understand where these organizations are coming from which could improve the accuracy of coverage • Foster goodwill all aroundTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 6 of 6
  7. 7. LEVEL 1: OUTREACH (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSSTRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMCreate and • Tailor and develop offerings • Develop trust and credibility, along • Measure trust, credibility, loyalty, etc. (survey)promote based on community feedback with the loyalty associated with a sense • Measure traffic/audience for new initiativescontent/solutio to better address the specific that the news organization cares aboutns that meet needs of individual constituencies what the audience cares about • Measure response to new initiatives (survey, focus group)specific needs or the entire community. This is • Increase publics appetite for andidentified by not just simply about volunteering reliance on news coveragespecific ideas for the community tocommunity choose from, but really listening • Increase use of news outlets offeringsand/or to what they say when you by audience (reading, sharing, linking,audience actively make contact with them, responding) and then developing solutions to • Clarify work priorities by better get them the information and/or understanding what parts of news news-related resources they organizations efforts are delivering value need to become more informed to the audience and what parts are not; and/or engaged in the news this can steer news organization as it landscape attempts to allocate limited resources!THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 7 of 7
  8. 8. IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSLEVEL 2: CONVERSATIONBeing in conversation with our community means listening as well as talking, and adjusting what we do and cover based on whatwe hear. It involves: Hosting discussions in person and online on topics that matter to the community. Participating in conversationswere not hosting, both in person and online. Valuing how a continuing dialogue can make us better journalists and improves thejournalism. Using web analytics to better understand what people are showing us they value in what we do, and basing at leastsome of our decisions about content and staff resources based on what were seeing. Recognizing that journalism is a process, notjust a product, and involving more voices in the process means more diverse journalism.!STRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMMake it a • Ask questions at the end of • Increases journalistic connection to the • Analytics such as time on site, return visits, tweets, Facebook likes. Another goodpriority to posts. community, staying involved. (Dont file one to use for measuring whether traffic drives engagement: ratio of page viewstransform web and run!) to comments. • If comments dont happentraffic into • Staff comments as a percentage of overall comments. naturally, encourage staff to post • Invites users to spend more timeconversation. the first comment. processing and responding to content. • Speed of response on the part of the staff or individual journalist. • Assign staff time to reading and • Offers chances to be transparent • Sentiment analysis: Do users notice/appreciate/respond to involvement of responding to comments within about who journalists are, how journalism journalists? (survey) the first 20 comments (or gets made, etc. whatever makes sense for your • Number of comments, and number of comments shared. • Makes the community feel heard, organization). • Number of people involved in rating or voting on comments (its likely that many which encourages future interaction. • Assign staff to stay involved in people would make themselves heard with a vote but wouldnt comment • Lifts the relevance, substance and themselves) the content they have a hand in, civility of the online conversation. taking note in and, where • Civility of conversations (content analysis) appropriate, responding within 24 • Rewards good behavior, encouraging hours to user reactions and further interaction and loyalty. questions. • Saves staff time by having the crowd • Create tiered comment system help monitor. Also, users who help so the community can help monitor reinforce the culture were trying reward productive, useful, to create. enlightening comments. • Makes the news more social, and • Allow for easy online sharing of encourages sharing of content and individual comments, not just comments. posts. • Share user comments as unique content (as in on the home page), not just as a reaction to journalists content.THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 8 of 8
  9. 9. LEVEL 2: CONVERSATION (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSEncourage and • Allocate staff time for • Leads to better understanding of the • Perceived value on the part of the staff (internal survey).reward community conversations that community, and deeper connections, • Reach into the community — number of conversations journalists wouldnt haveconversation as might not yield immediate which could pay off in all kinds of ways. otherwise had.a goal, not just productivity (consider Googles Serves as an early detection/warninga means to an "20 percent time" for innovation) system for community problems and • Strategic analysis of which (content) areas journalists want to know more about,end. issues. and have they spent time learning about them? • Create a culture of two-way communication, not just • Sends a message to staff that being in • Percentage of the community that has talked to journalists or seen/heard them broadcasting. touch and involved is truly important, around town or online (external survey). and that you trust them to do that. • Number and diversity of unique contributors, and commitment of repeat • Attend community gatherings not to cover them, just to listen. • Allows for the exploration of ideas and contributors projects that could pay off in the long • Reward frequent term. commenters/contributors with recognition, invitations to happy • Sends a message to the community hour or lunch, branded that their voice is desired, appreciated merchandise, credits to and equal in importance. advertiserss businesses, etc. • Serves as a feedback loop for • Reward staff who use community input. conversation threads online to • Offers a window into how the staff uses encourage more debate, not to feedback from users, along with prove themselves right or share validation that the voices mattered. what they know. • Report to the community what journalists hear (in person, analytics, etc.) about whats important to them, and what the follow-up plan is.Have a • Decide what tone and voice • Relationships are formed through • Experiment with different tones and track what drives conversation (similar to A-consistent your news organization should consistent conversation with users, B testing)social media have on social media, and put it making them feel comfortable. • Track consistency of the brands voice (content analysis)voice and in writing in a company-wide • Staff time is used efficiently, with lessaction plan plan. • Staff acceptance of and adherence to guidelines (internal survey) guesswork. • Consider the communication • In case of problematic posts, track speed of response time on the part of the • Setting the tone for the conversation is staff. norms of the platforms youre done in-house, rather than letting the using, and consider whats audience set the tone. appropriate for different kinds of content. • Sets clear, acceptable boundaries for staff, minimizing problematic interactions • Develop a plan for dealing with and facilitating consistent brand inappropriate comments or perception. communication from users, or inappropriate messages linked to your brand (from internal OR external sources)THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 9 of 9
  10. 10. LEVEL 2: CONVERSATION (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSBe consistent • Have regular times and places • Makes it easy for the community to • Log interactions. Track both stickiness (how often the same people come back)with journalists to meet, online and in person know where to go and who to talk to, and reach (the number of new people who join). Both are valuable for differentinvitations to (examples: reader panels, Twitter encouraging ideas and interaction reasons.conversations, hashtags, community coffee • Help users feel like theyre part of a • Think of a Twitter hashtag as a place, and track how crowded it is during a setand with their shop hours) valued relationship. chat time, and how often it is mentioned at other times.participation in • Decide that no message from acommunity • Conversations can continue where • Ask the people who participate (online or in person) why theyre there and if its reader/source/member of the they left off. a good time/place for them.conversations public will go unanswered,theyre not including tweets that mention the • Each group reached will likely include • Track story ideas that emerge. How many stories are enriched or suggested byhosting. news organization. people not in your existing audience, conversations with users? offering a chance for recruitment and • Number of events involved in, people attending, social media impressions. • Join or collaborate with solicitation of coverage ideas. community conversations • Track whether involvement in an issue or event drives traffic to those content already going on, both formally • Increased visibility among non-users areas. (partnering with a nonprofit to offers promotional value and increased sponsor a discussion, partnering social capital. • Keep a log of names/groups of those reached, and follow up to see if those with a blogger) or informally people register on the website, or if their visits change after the interaction, or if • Opportunities for crowdsourcing they invite you back to their conversations. (sitting in the audience at a among non-users. community event, commenting • Track incoming links from sites and conversations journalists are participating in, on a Facebook thread) • A window into brand perception and and retweets and other mentions from those involved. One way to do that is with community impact — positive, negative unique bit.ly links for each site. • As part of covering a beat, or neutral — among non-users, and a make it a priority for a journalist to chance for journalists to (non- comment on outside sites and defensively) ask follow-up questions and blogs that cover their topics, to invite more feedback. add context, share links and be a present, consistent, authoritative • Positive interactions lead to third-party voice. Post on discussion boards. promoters, which is earned (free) Share a link on a groups marketing Facebook wall. • Ask a regular (daily?) question, inviting responses and highlighting the best of them (based on journalists views or user votes) • Make it easy for people to contribute in whatever way they feel most comfortable. Take news tips, photos and other contributions via text message.THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 10 of 10
  11. 11. IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSLEVEL 3: COLLABORATIONCollaborating with our communities, the highest form of engagement, means we have a shared investment in and influence overour journalism. It involves: Soliciting and relying on user contributions. Soliciting and using user input about what we should coverand how we should allocate our resources. Valuing the role the users play in reacting to and sharing our content. Recognizing thatwe can accomplish things with the cooperation of the community that we could not do alone.!STRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMWork with Examples: • Get stories that more authentically • How much content is contributed, and the traffic to that content.specific represent the community — that are OF -Allow nonprofits or advocacy • Outbound traffic to those organizations — sometimes its better to be the firstcommunities or something, rather than ABOUT stop, or the hub of information, than the last stop. groups to post their own contentgroups something. and get the word out about their • Flow of traffic between news organization and community partner, as page own activities. • Expand your source base, building views or social media chatter. connections in an underserved group. -Identify interests that are shared • Number of partnerships or active projects. with a community group like a • Gather story ideas. school, university or historical • Number of sources we wouldnt have reached otherwise (would require • Build connections, and less separation qualitative reporting from the staff). organization. as the media elite. -Identify specific geographic • Does audience increase (registrations, page views, followers) from those groups • Foster a sense of shared purpose, or partnerships? Can be tracked with unique urls, or with geocoding and IP communities, for increased community and goodwill, on behalf of address tracking. Could make a game with different partner groups to see who coverage or collaborative the group and for the benefit of the can drive the most participation or traffic. projects. news organization. • Sentiments of contributors — do they feel it made a difference in overall • Encourage solutions to community coverage and in representation of their community (survey). problems, and suggested actions users can take, to bubble up from the group • Ideas that emerge that we wouldnt have had otherwise. Could be tracked with new metadata field or other internal process. • Ratio of partnerships we propose to those that are pitched to us, over time • Community goodwill and brand perception (survey). • Impact in the community — did we help shape debate or policy? (follow-up reporting, or survey) • Newsroom time spent on organization. Does it decrease over time, increasing the return on investment?Make Examples: • Profit-sharing, on a book or on ideas • Profit from sales.collaborative that bubble up from developers. -Give a church access to our • Households reached by news organizations brand through sales.niche products. archival content for the purposes • Demonstrating that we value a • Possibility of revenue streams from collaboration with developers, who often of a historical book. community, and our shared history or have great ideas about how to package content in ways journalists might not interests. consider. -Release our API to a community of developers. • Expanding our role and reach. • Demonstrating our relevance.THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 11 of 11
  12. 12. LEVEL 3: COLLABORATION (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSSTRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMHarness the Examples: • Get a breadth of information we could • Number of participants.power of the never gather on our own, and the whole -Add a collaboration layer to big • Range of geography or other diversity, compared to other projects (contentcrowd. is often greater than the sum of the parts. stories like data reports or analysis). citywide mapping projects. • Get a sense of trends that would be • Traffic to collaborative projects, compared to traditionally produced projects. less likely to surface through more -Share our digital expertise with • Ratio of active versus passive traffic to collaborative projects (looking at sharing, isolated reporting. contributors. recommending, contributing as compared to overall visits) -Use distributed reporting to share • Foster an increased sense of loyalty and investment on the part of the • Time on site statistics for traffic to collaborative projects. the workload. contributors. • New unique visitors to collaborative projects. -Learn to watch for opportunities • Invest in the contributors ability to • Time spent in newsroom to organize collaborative projects, when compared to where the community will be provide us with quality content. benefits gained. naturally making and sharing their own media. Figure out if you • Invest in community media literacy. • Sharing statistics (email, social media, etc.) for collaborative projects, as have a role in collecting and percentage of overall visits, compared to whats normal for the site. • Harness wide, diverse, distributed sharing it. resources in a time of newsroom Satisfaction survey for contributors — does it make you feel more connected, cutbacks. more valued, better represented? • Broaden the definition — and increase the relevance — of journalism. • A built-in network for sharing the content produced.Use Examples: • Establishes us as being in touch with • Sponsorships sold for specific databases or projects.collaboration -Connect people who need their what community needs are, and • New unique visitors.to meet driveways shoveled with willing invested in helping solve them, therebyspecific building trust and loyalty. • Visitors to these projects who become more invested, going on to other pages shovelers.community or contributing to reporting. • Brings users to the site for a specificneeds. -Focus on services, enabling • Community brand perception after using these services (survey) reason. people looking for a piano teacher or plumber to seek • Were seen as a community resource • Increase in awareness of media brand in overall community (survey) recommendations from the for practical needs, not just the source of wisdom of the crowd. news and stories. -Offer a database of volunteer • Opportunity to connect these projects opportunities, sortable by to related stories, to share published expertise needed, time stories. commitment, skills required, etc. • Opportunity to post queries and -Publicize resources for timely crowdsourcing opportunities connected needs, such as experts willing to to these projects, using a shared interest help with foreclosure, or heating to diversify sources. bills, or medical questions. • Potential to find new compelling stories about the community interactions to report out to the rest of the community as part of our journalistic offeringsTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 12 of 12
  13. 13. LEVEL 3: COLLABORATION (cont.) IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSSTRATEGY SOME SUGGESTED TACTICS WHY WE VALUE THEM HOW WE COULD MEASURE THEMInvite the Examples: • Be seen as working with the • Number of specific requests or questions that are answered by a communitycommunity to community, recognizing and rewarding expert, and traffic to that content. -Become a localized newshelp shape expertise. question & answer source, • Sponsorships sold for specific collaborations or content (as with a Q&A/expertyour agenda project) tapping community experts and • Increase a sense of communityand resources letting the questions drive the investment, in the organization and in • Amount of feedback on story ideas, and how responsive the newsroom is to agenda. specific projects. that feedback. -Publish a running list of projects • Make decisions based on actual • Traffic to stories with the most interest, and whether they do better than the ones being explored or reported, and community information needs, not just that didnt get as much interest. invite feedback on their value. the journalists perspectives. • If funding is part of the collaborative process, how many stories get funded? -Give people a more direct seat How diverse are they? at the editorial table, determining what projects get funded or • If one goal is to increase participation, measure active contributions compared pursued. to overall visits. -Allow individuals to fund specific projects.Become an Examples: • Be seen as a good steward of • Inbound and outbound traffic to other media sites and blogs.active part of a -Embrace and amplify content community information and needs, with • How often our content is shared, and how its used on other sites. (For example,diverse media from other community voices — a focus on collaboration, not a wrapper on text, similar to embed code from YouTube, would let us track howlandscape. competition. our products are consumed when they leave us.) other media outlets, bloggers, community groups. • A more diverse collective product. -Create tools for post-production • A sense of our investment in the collaboration, such as digital community, and the other medias wrappers that track activity on investment in us. other sites, or embeddable • Being able to track how our content widgets for content partners. moves, and know its reach and impact. • Being able to more specifically and strategically push content out, based on what has worked before.THE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 13 of 13
  14. 14. IDENTIFYING AND MEASURING AUDIENCE ENGAGEMENT EFFORTSWHAT WE WISH WE KNEW HOW TO DO OR MEASURE RESOURCES• Make a network map of the staffs professional and personal connections, A few specific websites and articles came up as we were talking, asto evaluate our community connections and reach. examples of how journalists can use metrics, or find help using them:• Make a network map of community and stakeholder connections. How • How Postrank measures engagement:are people connected to each other, regardless of how theyre connected https://analytics.postrank.com/docs/engagementto us? • How Philly.com measures engagement (from Nieman): • Use analytics in the sophisticated way other industries do. A few of us are http://www.niemanlab.org/2010/10/getting-beyond-just-pageviews-philly-good at this, but its shocking how many newsrooms still rely on a few basic coms-seven-part-equation-for-measuring-online-engagement/numbers like page views. There is huge opportunity for a company to offersoftware customized for news organizations, and help journalists figure out • The transcript of the #wjchat on social media metrics:how to strategize around what they know about their audiences. http://wjchat.webjournalist.org/2011/03/chat-3-30-11-social-media-metrics/• Track our interactions with our audiences. We need a real Customer • Gawkers approach to measuring "branded traffic" (from Nieman):Relationship Management tool that makes it easy to record how http://www.niemanlab.org/2010/03/a-reader-affection-formula-gawker-community members interact with us, on the phone, in person, on social creates-a-metric-for-branded-traffic/media or on our website. We want to be able to see a snapshot of how • Confusion Online: Faulty Metrics and the Future of Digital Journalism:what we do shapes those interactions, how people find us, whether they http://www.journalism.columbia.edu/page/633/437come back, and whether they donate or contribute content. All of it. Withas much automated as possible. This would also help us identify the • Information about Joy Mayer’s fellowship work at RJI:stumbling blocks that may get in the way of readers spending more time http://www.rjionline.org/blog/highlights-joy-mayers-community-with our content online. engagement-fellowship-blog • Links Joy has saved on metrics: http://www.delicious.com/mayerjoy/metricsTHE ENGAGEMENT METRIC • REYNOLDS JOURNALISM INSTITUTE • SPRING 2011 Page 14 of 14

×