This talk is called Developments and Challenges in Social Media Measurement. While the traditional format is I speak, you listen or go through emails, and there are questions at the end, I do like a good discussion on this topic. The area of social media measurement is still under construction, so if you want to chime in – by all means feel free. I’m going to talk about my journey through this space, and a solution I developed … but I am happy to provide suggestions for folks who are just starting out in trying to solve these problems at the end of the discussion. One thing I would like to mention as well, is that text analytics can solve more problems than just how people are engaging with your company’s brands or products. They can be useful for mining insights in product development, customer care, and some other interesting applications, and what we’ll be talking about for the next 30 minutes or has a relationship to these other areas as well.
AgendaWho is this guy?Déjà vu all over againA game of Chutes and LaddersLight at the end of the tunnel?
Who is this guy?• 20 years research/analytics experience• Focus on media: Turner Networks, MySpace,Yahoo, media/ad agencies• Quantitatively focused:• MMMs• Segmentation Analysis• Campaign Attribution• Behavioral Targeting• Fan/Follower Valuation
Who is this guy?• The Public Relationsdiscipline took hold of socialmarketing• Porter Novelli’s client baseis global, which leads tosome interesting socialmedia analyticsopportunities
Déjà vu all over again• Dirty data in the social space• Inappropriate methodologies• Vendors that do not care about dataquality• No industry standards
Déjà vu all over again• Data is spam laden• All tweets are not created equal• Interactions across social channelsmean something different• Does an emoji connote sentiment?Does it generate influence? Howmuch influence does it generate?• What is influence worth? What isreputation worth?
Déjà vu all over again• Because of the sheer volume of data,trying to make sense of this has led somefirms down very strange roads• A common approach is to sample thesocial conversation, and infer quantitativeconclusions• This is in defiance of the Central LimitTheorem
Déjà vu all over again• On my arrival into the public relationsindustry, I took as many vendor meetingsas I could. My findings:• All data vendors have the “best”sentiment scoring engine … though thecriteria for this claim is unknown• Vendor-side spam filtering is ineffective• The interest across vendors is creatingprettier charts with vibrant colors, ratherthan data quality“magic beans”
Déjà vu all over again• There are several groups trying to developsome industry standards around social mediameasurement, but as of now, there are noaccepted standards• The best we have at the moment are theBarcelona Principles• Will social media ever get to the same level ofstandards as the IAB/WAA on online mediameasurement?
Chutes and Ladders• “Every thing is measurable”• The reason that standards weredeveloped on the web analytics side wasdue to the investment• Public relations wants more marketingdollars• Standards are coming out, but are theystrong enough?Where:E = excused from flyingI = insanityR = requests an evaluation
Chutes and Ladders• Is the objective of the social analytics qualitative insights mining, measurement, orboth?• If sampling leads to inappropriate or insufficient conclusions what are themeasurement options?• In the web analytics world, we take spam filtration for granted; in social, relevance iseverything.• Every social analytics program is going to have error … some known and someunknown.
Light at the end of the tunnel?• There are platforms that allow a fullanalysis of text … some are robustand offer easy ways to integratetext and other data into onereporting platform• The solution that we havedeveloped is using an open sourcetext analytics platform, so weeffectively built our own solution
Light at the end of the tunnel?• People talk about brands, productsand services using a specificontology• “Sick” connotes “good” for somecategories, “bad” for others• Most vendors who providesentiment scoring across the entireuniverse of conversation are notable to account for thesedifferences
Light at the end of the tunnel?Process:• Pull in data from multiple sources• Build dictionary and grammar rules• Categorize text by conversationcategory and sentiment based on rules(human and machine learningalgorithms)• Human scoring and validation• Dump results to UI
Best Practices• Any vendor who talks about “best” sentiment engine – based on what?• Know your data• Get as close to the source as you can• Solutions custom to your needs are always better than out-of-the-box• Beware of pretty Uis• Good governance of data and analytics