Open Analytics NYC – 11/08/2012Building Effective Frameworks for Social Media Analysis
Agenda • Social Media: An Intelligence perspective • Common Analytic Pitfalls • An Analytic Framework • Case Study: Brand Management – Problem Definition – Source Selection – Data Capture – Data Reporting – Data Analysis • Ways Forward, Future Analysis • Questions?
Intelligence • Intelligence is information that has been transformed to meet an operational need Data Intelligence Operational Lens
Intelligence Cycle • No matter what methodology you use… Collect Distribute Store Analyze intelligence analysis is an iterative process.
Social Media: Intelligence Perspective • Social Media Intelligence is a combination of the best and worst features of: – HUMINT – OSINT – SIGINT HUMINT OSINT SIGINT
Social Media Analysis Goals • Provide value to the organization – turn data into intelligence using an “operational lens” • Ensure cyclical feedback occurs during collection, processing, analysis, and consumption • Validate that a particular network is the right source of data for the questions you need answered
Common Misconceptions • Social media is not a panacea – Not everyone uses social media – Users of social media use it unevenly – User behavior changes based on situations • Just because people can talk about anything does not mean they talk about everything all the time.
Common Pitfalls • Analyzing What Instead of Why: The important thing is often not what people are saying… but why they are saying it. • Using the Wrong Analysis Tools: Reporting tools rarely help dig into the why. Many common tools, reports, and metrics are actually misleading: – Word clouds atomize message context – Sentiment metrics are often highly inaccurate – Information in aggregate hides more than it reveals
Dangers of Disintegration Source: Matthew Auer, Policy Studies Journal, Volume 39, Issue 4, pages 709–736, Nov 2011
Analytic Framework • Data Capture (DC) Capture • Data Reporting (DR) • Data Analysis (DA) – What to measure Analyze Report – What the data is saying – What should be done based on the data Source: Avinash Kaushik, Occam’s Razor Blog http://www.kaushik.net/avinash/web-analytics-consulting- framework-smarter-decisions/
Choosing a Platform • Social media, and the ways that it is used, is relatively new and evolving rapidly: – Static approaches to social media are flawed from the outset – No one metric or set of metrics will always let you know what is happening • Platforms need to be open and highly adaptable to facilitate data capture, reporting, and analysis
Case Study: Brand Management • Industry: Gaming – Experiencing 10% growth annually – Overall revenue expected to exceed $80 billion by 2014 • In May, Zenimax Online Studios announced Elder Scrolls Online – Elder Scrolls V: Skyrim 2nd largest game of 2011
Problem Definition • Question: How can brand managers use social media to track and understand public attitudes toward a product? • Challenge: Capture relevant information for social media sources. – Query too large = false positives – Query too small = miss potential information
Twitter • Twitter has excellent analytical potential: – Enormous volume, 400 million+ tweets per day – Large user base, 140 million+ accounts – Open API • But its not without its limitations: – 140 characters – Limited historical (lookback) capacity without using a 3rd party provider like DataSift or GNIP
Data Capture: Initial Query • Twitter search for “Elder Scrolls Online” – Simplest possible way to access information – RSS feed for 10 days (Jun 27 – July 6 2012)
Data Capture: Entities & Associations Hashtag TwitterHandle URL Unstructured Keywords Time / Date Stamp Who What When Where TwitterHandle Hashtags, Keywords, Time, Date Geo (if Available) URLs
Data Analysis • Analysis needs to be rooted in the operational need: “How can I use social media to track and understand public attitudes toward my product” • Emphasis on hypothesis generation, testing, and experimentation
Data Analysis: Hashtags • Top hashtags were almost all generic or abstract – Undermines tracking and understanding – Top hashtags tied to franchise, not to the game Hashtags #ElderScrolls #concept #games #nerd #online #geek #MMO #gamer #skyrim #ScreenShot
Data Analysis: Expanding the Query • Hash tags from an initial subset of Tweets fed back into the initial query Initial Query Expanded Query Results Results Twitter Stream
Data Analysis: Sentiment • Sentiment analysis on small snippets of text like Tweets is generally poor • Follow and convert linked URLs into derivative sources • Larger text sources offer potential value with sentiment analysis that tweets alone cannot offer
Data Analysis: Sentiment • Top negative and positive sentiment scores can provide a glimpse into aggregate attitudes • Provide starting points for additional analysis
Next Steps: Shape the Conversation • Create and promote hashtags that help shape the conversation and make it easier to collect and analyze the Twitter stream
Next Steps: Segment the Data • Segment, or cluster, your data by: – User name or handle – Hashtags – Keywords – Geographic region to explore patterns and trends at the micro level versus the entire dataset
Lessons Learned • Don’t: – Try drinking from a fire hose, sometimes less really is more; – Use metrics you can’t tie to actions; – Use visualizations or reports that strip the data from its context.
Lessons Learned • Do: – Segment data rather than attempting to work in the aggregate; – Look for the why behind the message; – Always return to the source material; – Explore alternative explanations; – Always consider the ultimate goal.