Using analytics to evaluate digital communications strategies
Using analytics to evaluatedigital communication strategiesDana Chinn1. Developing measurement models2.2 Basic site metrics3. Some thoughts about external ratings and metrics4. Social media metricsPUBD 526 – Public Diplomacy EvaluationApril 2012
Improving digital communication channels start with identifying what needs to change Start here not here 2
Our site has 5,000 monthly unique visitors.Last Tuesday that story got 20,000 pageviews. Our iPhone app was downloaded 10 000 times. 10,000 times We have 2,000 Likes on Facebook. We have 5,000 Twitter followers 5 000 followers. 3
Questions for a e-commerce company Who came to our site? e.g., previous vs. new; high vs. low potential How did they get here? What did they look at? Were they successful in getting what they wanted? A simple e-commerce data story “Current and potential customers who typed in “t-shirts” in Google arrived on our t-shirts landing page. 1.5% of them made a purchase.” 4-- Corey Koberg, Cardinal Path
The questions for a content- or mission-based site are the same… Who came to our site? e.g., previous vs. new; high vs. low potential How did they get here? What did they look at? Were they successful in getting what they wanted?…so why i the typical story usually something lik this? h is h i l ll hi like hi Our site has 5,000 monthly unique visitors. Last Tuesday that story got 20,000 page views. y yg , p g The average time spent on our site last week was 24 minutes. Our iPhone app was downloaded 10,000 times. We have 2,000 fans on our Facebook page. We have 5,000 Twitter followers 5 000 followers. 5
ExamplePurpose of a community news program Increase civic engagement in a community by bringing stakeholders together through their shared need for community news and information 6
Using data for decision-making is dependent first on a clearly defined target audience…Alhambra Boyle Heights “South LA” City of Alhambra City of Los Angeles LA Times …and then on an equally well-defined multichannel strategy 7
Each channel reaches different audiences and hasunique functionality, so each needs its own measurement model.A channel’s model – and metrics – should be developedbased on its role vs. other channels. Goal: What the org wants the channel to do Key Performance Indicator: A metric crucial to the org’s survival Target: The value of the KPI that will indicate success or failure Segment: A group of visits or visitors, categorized by type and/or behavior, that is essential to reach the target 8
Sample measurement model for a website (eek!) 9
1. Establish program objectives. What does your org want to do through all of its channels? 10
2. Establish – exactly – how each channel will contribute toeach program objective. What are the goals of your site? Why does it exist? 11
2. Establish – exactly – how each channel will contribute toeach program objective. What are the goals of your site? Why does it exist?Getting a name and e-mail address is thefirst indicatorsomeone is engagingwith you. Audience info, obtained with permission, is perhaps the most important function of a site. 12
2. Establish – exactly – how each channel will contribute toeach program objective. What are the goals of your site? Why does it exist? 13
3. Decide which metrics will be the Key Performance Indicatorsfor each channel goal. Establish the targets that will indicatesuccess – or f il failure. Analyze audience segments: What type of audience behavior is yp affecting the KPI? Example: Site visitors who entered through search engines visited an average of once a week. Is this good? Yes – Our target was two times a month No – Our target was twice a week! We just added additional resources to put new content up daily! 14
Maybe thM b the content you’re putting up on the site t t ’ tti th itisn’t the content that people want! Type of analysis used by J. Paul Getty Trust from “Web analytics success for government websites,” by Avinash Kaushik, Oct. 12, 2009
3. Decide which metrics will be the Key Performance Indicatorsfor each channel goal. Establish the targets that will indicatesuccess – or f il failure. 16
3. Decide which metrics will be the Key Performance Indicatorsfor each channel goal. Establish the targets that will indicatesuccess – or f il failure. Nonprofits need KPIs need to be to decide specific to what whether the the channel can number of actually do. individuals is more of a Example: A priority than the $20,000 total amount of sponsorship money package will be sold through nondigital fundraising methods. 17
Sample measurement model for a website… …repeat for e-mail, Facebook, Twitter...eek! 18
Web analytics is the analysis of data “to drive a continual improvement of the online experience… which translates into your desired outcomes.” y Just one part of web analytics 19from Web Analytics 2.0 by Avinash Kaushik
Step 1: Understand the clickstream, or every action relevant to site goalsBehavioral research What people did when they came to your site, h h as captured by an action taken on a keyboard or mouse 20
What actions indicate engagement? Visit Vi it , regularly l l Read/view content, a l R d/ i lotInteract,Interact often -- rate, print, vote, take a poll, click on an ad -- share, e-mail, comment, contribute 21
Basic site metrics Unique visitors U i i it visit sites and generate page views 22
Total visits: One indicator of overall site performance A visit i counted i it is o ted every time someone comes to a siteVisits: the strongest metric availableAn increase in visits is always good. -- More people are coming to your site. -- Returning people are coming more often.A decrease in visits? Always bad. 23
Strong vs. weak metrics g Strong metrics are useful tools that give clear indications of what’s successful or notc. Kyle Taylor Weak metrics… -- are conceptually flawed “so what?” counts of things -- are technically flawed metrics calculated by web analytics systems c. Kyle Taylor in ways that give unclear indications …could be so misleading they could lead to bad decisions 24
Really weak metric #1: Unique visitorsA unique visitor is really a unique computer. Unique visitors are either over-counted… …or under-counted. library, school, Internet I t t cafe You’ll never know which or by how much.** It doesn’t matter anyway….better to measure outcomes (did people do what you wanted?) thanthe number of people who came to your site. 25
Really weak metric #2: Page views An increase in page views can be good - or bad * bad.* Bad design navigation site architecture? design, navigation, Lots of page views, annoyed users A redesign improved usability? ? Fewer page views happier users views, Content that should be there but isn’t? Lots of page views, annoyed users Dynamic content? Fewer page views, happier users (probably) * It doesn’t matter anyway….better to measure outcomes (d d people d what you wanted?) than d ’ b (did l do h d ) h the number of pages people went to when they came to your site. 26
Really weak metric #3: Time spent on site An increase in average time spent on site can be good – or bad.* Bad design, navigation, site architecture? g , g , ? Lots of time spent, annoyed users A redesign improved usability? Less time spent, happier users p , pp Technically flawed: Time spent is either over- counted or undercounted * It doesn’t matter anyway….better to measure outcomes (d d people d what you wanted?) than d ’ b (did l do h d ) h how much time people spent on your site. 27
When audiences - new and returning - Wh di d t i come, are they staying? Key Performance Indicator Bounce rate percent of the landing page where most visits start “I came. I saw. I puked.” -- Avinash Kaushik on bounce rateA bounce: a visit with only one page view 28
The bounce rate of a landing page is much more actionable than thebounce rate of the entire site Start by looking at the top landing page, or the page where most visits start 100% 51% 8,331 Home page bounce rate: 43% 16,304 visits visits started on content pages 49% 7,973 57% 43% left the site 4,547 3,426 without going visits went to started at least to another on the one page other home page page Action: Let’s try changing the home page 29
Three types of site metrics that can be usedto segment visitors by behavior1. Visitor acquisition: How did people get to the site? Is your marketing working? Traffic sources: direct, referrals, search engines, campaigns (e.g., e-mail newsletters, ads)2. Site behavior: What did they do once they got to the site? Bounce rate of a landing page – did people leave after seeing only one page? Visits that included internal search Visits that went to a particular type of content3. Outcomes: Did people take the actions essential to the organization’s success? Visit frequency and recency Sign-up for an e-mail newsletter Buy a benefit dinner ticket Donate; sign up for membership
Internal metrics External metrics for for Strategic Planning Marketing, Advertising• Census data • Panel, toolbar data 100% of all visitors, visits, page Activity from a sample of self- views, etc. in a site selected people. Usually not relevant for small sites.• Analysis, decisions, • Marketing, trending, actions, evaluation competitive analysis• Omniture • comScore Google Analytics Nielsen WebTrends Compete etc. etc etc.• Digital Analytics • Interactive Advertising Association Bureau 31
Understand all of the implications Increasing the number andof mapping metrics to goals percent of students with high SAT scores… …leads to a higher ranking by US News & World Report… …which helps with donations, partnerships, faculty recruitment, etc. Action: Centralize UG admissions decisions. One result: Fewer minority journalism majors coming from high school
Measurement models need to be reviewed regularly Charities now need to optimize both “Financial Health” and “Accountability and Transparency” y p y
Is Liberty Hill’s site successful? “No more newsletters mailed to you at home….please register on h l i t the website, even if you’ve been a supporter for years ” years. KPI: Percent of print newsletter subscribers who register Target: 100% Action needed based on analysis of data: We need more [Eastside/media/women] to register. Let’s try a [follow-up postcard/raffle/event].
Is Liberty Hill’s site successful? Look at the site traffic trends after the flyer is mailed to each audience type. -- home page bounce rate -- pop-up bounce rate -- sign-up p g p process Maybe you’re getting people to come to the site, but the site is losing them.
Is Liberty Hill’s site successful? Is our site selling the number of tickets tickets, sponsorships, ads, etc. that we want? Do other channels work better for some items? Analysis needed (multiple data sources needed): -- Total sold last year from all sources, by time period , y p -- Tickets (premier, standard); sponsorships (by type); ads (by type) -- P Percent of registered people t f i t d l (by type) who buy tickets, sponsorships, ads
Is Liberty Hill’s site successful? “Purchase your sponsorship, ticket and ads…online. Go to www.LibertyHill.org/dinner.” Have different direct mail and e-mail campaigns for each audience segment; have a different landing page with a unique URL (e.g., /campaign1; /campaign2) for each. Sending people just to www.libertyhill.org is a wasted tracking opportunity! Sample e-mail newsletter metrics KPI: No. of tickets sold by campaign Week of Jan 26, 2012 Jan. 26 Delivered/sent: 970/1,269 (76% Actions needed based on e- delivery rate) mail KPIs and ticket sales: Target: 100% (indicates list quality) Clicks/delivered: 36/970 (3.7% click- We need to clean up our list. to-delivery rate) Indicates the relevancy of the e-mail content) Let’s try a different message E-mail newsletter bounce rate: 78%* for [environmentalists/past *Estimated - links need to be tagged to track all traffic from a newsletter to the site program advertisers/individual seats].
Is Liberty Hill’s site successful? Where are we losing people in the purchase process? KPI: Percent and no. of visits that started with the dinner overview page and completed the four-step p y payment process p
SocialmediaNot only are the technologies new,but the metrics are as well. --Online Media and Marketing Association Metrics and Measurement program, June 2009 41
We know about “spray and pray” business models… The social media ‘provide and pray’ model Not having a purpose for social media efforts “….often leads to a worst practice we call ‘provide and pray.’ Leaders and managers provide access to a social technology, and then pray that a community forms and that community interactions somehow lead to business value. In most cases, adoption never really materializes; communities may form, but their activity is not considered valuable to the organization.”“Social Media Success Is About Purpose (Not Technology),” by Anthony J. Bradley and Mark P. McDonald, Harvard Business Review,Nov. 1, 2011
Why should organizations… …have a Facebook page? …tweet?“Effectively measuring social media,” Susan Etlinger, Altimeter Group webinar, April 2011 How important are either in achieving their goals? Are either of them essential, given --a large part of the target audience may not be on Facebook and/or Twitter? --extremely limited resources? Social media metrics are just as important as site metrics.
Typical (and rarely explicitlyexpressed) social media objectives d) i l di bj tifor a news or nonprofit organization 1. Drive traffic to the website Social S i l media as b di broadcast advertising mechanism d t d ti i h i 2. 2 Engage -- current audiences -- new audiences it s not getting elsewhere it’s Is this realistic? Using a measurement model is even more important for social media!
“What matters is everything that happens after you post / tweet /participate….The ‘so what’ matters!” 1. Conversation: “Social means talk and listen and discuss. So why not measure that?” 2. Amplification: “The rate at which your followers take your co te t a d s a e t t oug t e content and share it through their network.” 3. 3 Applause: “What does your audience like? What like?”“Best Social Media Metrics,” by Avinash Kaushik, Oct. 10, 2011. Chart designed by Erik Ohlen
Understand how to measure Twitter,and you’ll understand how to measure social media Content Followers not demographics or other typical mass media audience metrics 46
Twitalyzer – user centric TweetReach – tweet-centric Overall impact of an account Influence Looks at tweets for Reach – potential keywords Reach – effective Total reach = Number of unique people following people who Number of: generated one or more --Followers tweets that contained the --Followers of followers keyword y --References to the account by followers f h b f ll --Lists that include the account Total exposure = Number tweets of people (duplicated) --Times the account is replied to --Times the account is retweeted Times Number of Tweets with the keyword Contributors List of users who generated the highest number of impressions (calculated with tweets, retweets, followers of the people who tweeted and followers of the people who retweeted“Twitalyzer and TweetReach – A Symbiotic Pairing for Twitter Analysis,” by Tim Wilson, March 8, 2011
Measurable tweets have have…1. A call to action Go here…look…tell me2.2 A link that you track with link (e g bit y) (e.g., bit.y) and web analytics tools RT - retweet MT – modified tweet3. #Hashtags and/or keywords Via or HT – heard through Favorite Lists4. Topic or person-specific handles …120 or fewer characters, not 140! 120 characters 48
Follower metrics – Avinash Kaushik “Retention is the key to growth.” DRAFT Metric Tool Time period Analysis Follower growth Twitter Counter Over last three Compare rate against rate months two others. Relevant follower Twitalyzer, others Over last three Estimate; sample growth rate months No. of unique, GraphEdge (paid) Over last three Compare against two legitimate months others. followers; percent of overlap Churn ate Ch n rate: Gain GraphEdge; others G hEd th 30-day churn rate 30 d h t Also Al analyze number l b in followers vs. no. of relevant followers of unfollows Follower retweet Klout (velocity) Over last three percent months Unique senders q Klout (engagement) ( g g )Chart derived from “Actionable Twitter Analytics” Market Motive video
Interaction metrics – Avinash Kaushik DRAFT Metric Tool Time period Analysis Click percent, by URL shorteners – Weekly Put in context vs. topic; top bit.ly, tr.im news cycle. referrers Analyze tweet volume. No. of retweets, Twitalyzer, Weekly by topic Retweetrank, Klout (reach), others No. of retweets Twitalyzer, Weekly Compare against per thousand Retweetrank, others two others. followers, by topic Replies sent per Twitter-friends Daily Are you having day conversations? Replies received Twitter-friends Daily Are people per day communicating with you? Inbound metrics I b d t i Klout (engagement) Kl t ( t) per outbound messageChart derived from “Actionable Twitter Analytics” Market Motive video
Outcome metrics – Avinash Kaushik DRAFT Metric Tool Time period Analysis Visits and other Kl.am; Google Need to determine site by Analytics campaign campaign “campaign” tracking parameters. Set up custom GA report. t Sentiment or TweetPsych 23 dimensions such Emotion – What as learning, is your Twitter thinking, positive. account saying Is your account about you? what you want for your brand and what you are trying to do? Sentiment or StatsIt Emotion – What are other people saying about you?? Chart derived from “Actionable Twitter Analytics” Market Motive video
Twitter metrics investment levels – Avinash Kaushik From “Actionable Twitter Analytics” Market Motive video
Facebook Insights Analyze trends in • Posts • People are Talking About This • Weekly Total Reach
“…it is worse to post something that people do notreact to, than to post nothing at all.” “…a completely flat level of weekly reach” “Each post is not really making any difference one way or another. The another Th number of people who are talking about this brand is dropping. b f l h lki b hi b d i d i “This is an indication that you are boring. This brand is likely doing the same few things over and over again, and people are getting bored with it…. “You are slowly turning yourself into a commodity. It is just something people can follow every day, but you are not motivating j gp p y y, y g your audience to act. You are not changing anything.”“Beyond Facebook Analytics,” by Thomas Baekdal, Nov. 8, 2011
Be honest with metricsDo 538 peopleREALLY “Like”this? Or do h O d they jjust want another sweepstakes entry? 55
Audiences and actions differ by channel… …so there are completely different metrics for each! And you need to report them all separately – you can’t add them up to get a total audience number SOCIAL SITES MEDIA MOBILE Totals1. Who? How many? In target audience? ? ? ? ? ? ?2. No. of visits?2 f i i ? How often? ? ? ? ? ? ?3. What did they see? ? ? ? ? ? ? Did they get want they wanted?4. Did they interact? y ? ? ? ? ? ? What did they do? How much? 56
Evaluating effectiveness 1. Define a measurable audience. 2. Set specific goals across all channels; Measure map metrics to goals. Segment.Optimize Act Analytics Report 3. Set up each channel to measure specific actions that indicate engagement and lead to outcomes Analyze Don’t forget about Voice of Customer 57
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.