Open Analytics NYC – 11/08/2012Building Effective Frameworks  for Social Media Analysis
Agenda •   Social Media: An Intelligence perspective •   Common Analytic Pitfalls •   An Analytic Framework •   Case Study...
Intelligence • Intelligence is information that has been   transformed to meet an operational need       Data             ...
Intelligence Cycle • No matter what methodology you use…                         Collect              Distribute          ...
Social Media: Intelligence Perspective • Social Media Intelligence is a combination of the   best and worst features of:  ...
Social Media Analysis Goals • Provide value to the organization – turn data into   intelligence using an “operational lens...
Common Misconceptions • Social media is not a panacea    – Not everyone uses social media    – Users of social media use i...
Common Pitfalls • Analyzing What Instead of Why:   The important thing is often not what people are   saying… but why they...
Pitfalls: An Example of the Challenge
Pitfalls: An Example of the Challenge
Dangers of Disintegration                    Source: Matthew Auer, Policy Studies Journal,                    Volume 39, I...
Analytic Framework • Data Capture (DC)                                                    Capture • Data Reporting (DR) • ...
Choosing a Platform • Social media, and the ways that it is used, is   relatively new and evolving rapidly:   – Static app...
Case Study: Brand Management • Industry: Gaming   – Experiencing 10% growth annually   – Overall revenue expected to excee...
Problem Definition • Question: How can brand managers use social   media to track and understand public   attitudes toward...
Twitter • Twitter has excellent analytical potential:   – Enormous volume, 400 million+ tweets per day   – Large user base...
Data Capture: Initial Query • Twitter search for “Elder Scrolls Online”   – Simplest possible way to access information   ...
Data Capture: Entities & Associations                     Hashtag             TwitterHandle         URL                   ...
Data Reporting
Data Reporting
Data Analysis • Analysis needs to be rooted in the operational   need:    “How can I use social media to track and    unde...
Data Analysis: Hashtags • Top hashtags were almost all generic or   abstract    – Undermines tracking and understanding   ...
Data Analysis: Expanding the Query • Hash tags from an initial subset of Tweets fed   back into the initial query         ...
Data Analysis: Sentiment • Sentiment analysis on small snippets of text   like Tweets is generally poor • Follow and conve...
Data Analysis: Sentiment • Top negative and positive sentiment scores   can provide a glimpse into aggregate attitudes • P...
Next Steps: Shape the Conversation • Create and promote hashtags that help shape   the conversation and make it easier to ...
Next Steps: Segment the Data • Segment, or cluster, your data by:   – User name or handle   – Hashtags   – Keywords   – Ge...
Next Steps: Segment the Data
Next Steps: Graph Analysis
Lessons Learned • Don’t:   – Try drinking from a fire hose, sometimes less     really is more;   – Use metrics you can’t t...
Lessons Learned • Do:   – Segment data rather than attempting to work in     the aggregate;   – Look for the why behind th...
Thank You!                Craig Vitter              www.ikanow.com             cvitter@ikanow.com             github.com/i...
Upcoming SlideShare
Loading in …5
×

Open Analytics: Building Effective Frameworks for Social Media Analysis

861
-1

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
861
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
27
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Introduction and Topic
  • Agenda:Social Media: An Intelligence perspectiveCommon Analytic PitfallsAn Analytic FrameworkCase Study: Brand ManagementWays Forward, Future AnalysisQuestions?
  • Intelligence is information (data) that has been transformed to meet an operational need.There are a lot of ways to move from raw data to usable intelligence.
  • No matter what methodology you use…intelligence analysis is an iterative processYou Collect the data, Store it, Analyze it, and Distribute the end results to your organization in some usable format.
  • HUMINT, Human Intelligence: intelligence gathering by means of interpersonal contact. Pros: Can reveal intentions Cons: Can be unreliableOSINT, Open Source Intelligence: intelligence collected from publicly available sources. Pros: Fast and accessible Cons: NoiseSIGINT, Signals Intelligence: intelligence-gathering by interception of signals. Pros: High volume Cons: Noise
  • My mom does not Tweet or have a FaceBook profile. It only seems like your friends post or tweet every 30 seconds.For example, people use different networks for different reasons so tracking individuals consistently can be difficult
  • Why is someone tweeting or posting? If some checks in from a store is it really because the store is so incredible that they need to share that information or is because they are trying to form an impression about their lifestyle (i.e. image shaping)?Why is much harder than What.
  • http://apps.washingtonpost.com/politics/transcripts/2012/presidential/live/737/Washington Post and Votertide collaboration to analyze how viewers reacted to Clinton’s speech at the DNC Convention a few months ago.They captured 496,222 tweets and generated what amounts to a very basic word cloud that really provides limited value from an analysis perspective.
  • What can you learn from this type of experiment with the wrong tools?A lot of people were tweeting when Clinton was speaking but not many were really tweeting about what Clinton was saying.People like funny tweets
  • Word clouds can tell you something about the language used but not the meaning behind the language. What you see in the cloud is what but not why.
  • So how do you avoid some of these pitfalls and get useful intelligence from social media? The answer to that question, or partial answer at least is the focus of the remainder of the presentation.The key ingredients for a framework include Data Capture, Data Reporting and Data Analysis components. All of which are important but the Data Analysis components are the most interesting. :>
  • There are a lot of platforms that you could use to do social analysis but a few key issues to consider before making a commitment:
  • This is where I repeat that I shamelessly stole most of this presentation from a coworker Andrew who is a huge gaming fan as well as being a super bright analyst.
  • Almost all NLP/text extraction/unstructured data analysis tools perform poorly on small blobs of text
  • Negative: Users weren’t impressed by the game’s teaser and graphics suggesting that the trailer hadn’t been well received.Positive: Other hashtags showed that fans still had positive sentiment towards the Elder Scrolls franchise in general.
  • Use Graph Analysis to explore the links between entities extracted from your data, for example:Identify Key InfluencersView links between tweets, websites, and blogs
  • Open Analytics: Building Effective Frameworks for Social Media Analysis

    1. 1. Open Analytics NYC – 11/08/2012Building Effective Frameworks for Social Media Analysis
    2. 2. Agenda • Social Media: An Intelligence perspective • Common Analytic Pitfalls • An Analytic Framework • Case Study: Brand Management – Problem Definition – Source Selection – Data Capture – Data Reporting – Data Analysis • Ways Forward, Future Analysis • Questions?
    3. 3. Intelligence • Intelligence is information that has been transformed to meet an operational need Data Intelligence Operational Lens
    4. 4. Intelligence Cycle • No matter what methodology you use… Collect Distribute Store Analyze intelligence analysis is an iterative process.
    5. 5. Social Media: Intelligence Perspective • Social Media Intelligence is a combination of the best and worst features of: – HUMINT – OSINT – SIGINT HUMINT OSINT SIGINT
    6. 6. Social Media Analysis Goals • Provide value to the organization – turn data into intelligence using an “operational lens” • Ensure cyclical feedback occurs during collection, processing, analysis, and consumption • Validate that a particular network is the right source of data for the questions you need answered
    7. 7. Common Misconceptions • Social media is not a panacea – Not everyone uses social media – Users of social media use it unevenly – User behavior changes based on situations • Just because people can talk about anything does not mean they talk about everything all the time.
    8. 8. Common Pitfalls • Analyzing What Instead of Why: The important thing is often not what people are saying… but why they are saying it. • Using the Wrong Analysis Tools: Reporting tools rarely help dig into the why. Many common tools, reports, and metrics are actually misleading: – Word clouds atomize message context – Sentiment metrics are often highly inaccurate – Information in aggregate hides more than it reveals
    9. 9. Pitfalls: An Example of the Challenge
    10. 10. Pitfalls: An Example of the Challenge
    11. 11. Dangers of Disintegration Source: Matthew Auer, Policy Studies Journal, Volume 39, Issue 4, pages 709–736, Nov 2011
    12. 12. Analytic Framework • Data Capture (DC) Capture • Data Reporting (DR) • Data Analysis (DA) – What to measure Analyze Report – What the data is saying – What should be done based on the data Source: Avinash Kaushik, Occam’s Razor Blog http://www.kaushik.net/avinash/web-analytics-consulting- framework-smarter-decisions/
    13. 13. Choosing a Platform • Social media, and the ways that it is used, is relatively new and evolving rapidly: – Static approaches to social media are flawed from the outset – No one metric or set of metrics will always let you know what is happening • Platforms need to be open and highly adaptable to facilitate data capture, reporting, and analysis
    14. 14. Case Study: Brand Management • Industry: Gaming – Experiencing 10% growth annually – Overall revenue expected to exceed $80 billion by 2014 • In May, Zenimax Online Studios announced Elder Scrolls Online – Elder Scrolls V: Skyrim 2nd largest game of 2011
    15. 15. Problem Definition • Question: How can brand managers use social media to track and understand public attitudes toward a product? • Challenge: Capture relevant information for social media sources. – Query too large = false positives – Query too small = miss potential information
    16. 16. Twitter • Twitter has excellent analytical potential: – Enormous volume, 400 million+ tweets per day – Large user base, 140 million+ accounts – Open API • But its not without its limitations: – 140 characters – Limited historical (lookback) capacity without using a 3rd party provider like DataSift or GNIP
    17. 17. Data Capture: Initial Query • Twitter search for “Elder Scrolls Online” – Simplest possible way to access information – RSS feed for 10 days (Jun 27 – July 6 2012)
    18. 18. Data Capture: Entities & Associations Hashtag TwitterHandle URL Unstructured Keywords Time / Date Stamp Who What When Where TwitterHandle Hashtags, Keywords, Time, Date Geo (if Available) URLs
    19. 19. Data Reporting
    20. 20. Data Reporting
    21. 21. Data Analysis • Analysis needs to be rooted in the operational need: “How can I use social media to track and understand public attitudes toward my product” • Emphasis on hypothesis generation, testing, and experimentation
    22. 22. Data Analysis: Hashtags • Top hashtags were almost all generic or abstract – Undermines tracking and understanding – Top hashtags tied to franchise, not to the game Hashtags #ElderScrolls #concept #games #nerd #online #geek #MMO #gamer #skyrim #ScreenShot
    23. 23. Data Analysis: Expanding the Query • Hash tags from an initial subset of Tweets fed back into the initial query Initial Query Expanded Query Results Results Twitter Stream
    24. 24. Data Analysis: Sentiment • Sentiment analysis on small snippets of text like Tweets is generally poor • Follow and convert linked URLs into derivative sources • Larger text sources offer potential value with sentiment analysis that tweets alone cannot offer
    25. 25. Data Analysis: Sentiment • Top negative and positive sentiment scores can provide a glimpse into aggregate attitudes • Provide starting points for additional analysis
    26. 26. Next Steps: Shape the Conversation • Create and promote hashtags that help shape the conversation and make it easier to collect and analyze the Twitter stream
    27. 27. Next Steps: Segment the Data • Segment, or cluster, your data by: – User name or handle – Hashtags – Keywords – Geographic region to explore patterns and trends at the micro level versus the entire dataset
    28. 28. Next Steps: Segment the Data
    29. 29. Next Steps: Graph Analysis
    30. 30. Lessons Learned • Don’t: – Try drinking from a fire hose, sometimes less really is more; – Use metrics you can’t tie to actions; – Use visualizations or reports that strip the data from its context.
    31. 31. Lessons Learned • Do: – Segment data rather than attempting to work in the aggregate; – Look for the why behind the message; – Always return to the source material; – Explore alternative explanations; – Always consider the ultimate goal.
    32. 32. Thank You! Craig Vitter www.ikanow.com cvitter@ikanow.com github.com/ikanow/Infinit.e

    ×