Analyzing social conversation: a guide to data mining and data visualization

on

  • 492 views

These slides were presented by Mick Conroy of Tempero and Jonathan Stray of Associated Press/Overview Project as part of Social Media Week New York #smwnyc

These slides were presented by Mick Conroy of Tempero and Jonathan Stray of Associated Press/Overview Project as part of Social Media Week New York #smwnyc

Statistics

Views

Total Views
492
Views on SlideShare
464
Embed Views
28

Actions

Likes
0
Downloads
1
Comments
0

2 Embeds 28

http://eventifier.co 15
https://twitter.com 13

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Analyzing social conversation: a guide to data mining and data visualization Analyzing social conversation: a guide to data mining and data visualization Presentation Transcript

  • Are you an NYC influencer? Find out @TemperoUK
  • Analyzing social conversation:a guide to data mining anddata visualization#SMWDataGeeksMichael Conroy Jonathan StraySenior Analyst, Tempero Project Lead, Overview@mickyconroy @jonathanstray@TemperoUK @overviewproject
  • WHO ARE WE? Mick Conroy Jonathan Stray Senior Social Media Analyst Project Lead, Overview Tempero Associated Press
  • WE’RE DROWNING IN TEXT!
  • THESE ONLY GO SO FAR…
  • Word clouds? Every time I see a word cloud presented as insight, I die a little inside. -JACOB HARRIS New York Times Data Journalism Team
  • IT’S NOT JUST ‘US’ Journalists! How do you deal with WikiLeaks? Freedom of Information Requests? The problems are very similar
  • An overview of OverviewJonathan StrayOverview Project Lead
  • structured data
  • unstructured data
  • more video on YouTube than produced by TV networks during entire 20th century
  • 10,000 legally-required reports filed by U.S. public companies every day
  • All New York Times articles ever = 0.06 terabytes (13 million stories, 5k per story)
  • Computer-assisted unstructured text analysis buzzwords! text analysis semantic visualization automatic classification
  • Document Clustering = about cars = not about cars
  • We look at every word in the documentsfind the words that make some documents different from all the others
  • Unstructured textNot tables of numbers, or stacks of fill-in-the-blank forms, but: Emails, meeting minutes, social media posts, event reports, news articles, blog posts, tweets, contracts, archives, press releases, instruction manuals, research articles, transcripts…
  • Open source project of the Associated Press overviewproject.org
  • Overview and Social MediaMick ConroySenior Analyst, Tempero
  • OVERVIEW AND SOCIAL? Understanding ‘how many’ is easy Understanding what is being said - conversation analysis - is hard
  • HUMANS AND ALGORITHMS
  • Exploring social data withOverview
  • AN INTRODUCTION TO DRONESHUMANS AND ALGORITHMS
  • AN INTRODUCTION TO DRONESHUMANS AND ALGORITHMS
  • 7 STAGES OF DATA ANALYSIS 1. Acquire 2. Parse/Filter 3. Mine 4. Represent 5. Refine 6. Interact
  • 1. ACQUIRE
  • 1. ACQUIRE
  • 2. PARSE/FILTER interaction.link text interaction.content url RT @Mikoshoes17: majority of the deaths ... http://twitter.com/Proud_Libtard/statuse ... https://twitter.com/Proud_Libtard/statuse RT @Mikoshoes17: majority of the deaths ... nicki minaj on american idol, even more ... http://twitter.com/FOXXXI/statuses/24734 ... https://twitter.com/FOXXXI/statuses/24734 nicki minaj on american idol, even more ... ... @BarackObama U say ur foreign policy is http://twitter.com/TFitz62/statuses/2473 ... https://twitter.com/TFitz62/statuses/2473 ... @BarackObama U say ur foreign policy is ... Sept 28 we find out which arm company lo ... http://twitter.com/NoDronesCanada/status... https://twitter.com/NoDronesCanada/status Sept 28 we find out which arm company lo ...... So GOP solutions to all foreign policy p ... http://twitter.com/ownersmag/statuses/24 ... https://twitter.com/ownersmag/statuses/24 ... So GOP solutions to all foreign policy p ... @KanwalZaidi @SenRehmanMalik The stubbor http://twitter.com/AymenAhmad1/statuses/ ... ... https://twitter.com/AymenAhmad1/statuses/ ... @KanwalZaidi @SenRehmanMalik The stubbor ... RT @OwenJones84: Other than propping ... http://twitter.com/KCruickshank1/statuseup ... https://twitter.com/KCruickshank1/statuse ... RT @OwenJones84: Other than propping up ... RT @BuzzFeedAndrew: RT @BuzzFeedPol: ... http://twitter.com/svdaveo/statuses/2473 Wha ... https://twitter.com/svdaveo/statuses/2473 ... RT @BuzzFeedAndrew: RT @BuzzFeedPol: Wha ... Marcus Aurelius: CIA Scrambles to Rush S http://twitter.com/PacketknifeToo/status ... https://twitter.com/PacketknifeToo/status ... Marcus Aurelius: CIA Scrambles to Rush S ... Ellip6: from military drones to the Worl ... http://twitter.com/RSIClub/statuses/2473 ... https://twitter.com/RSIClub/statuses/2473 Ellip6: from military drones to the Worl ... ... Parrot Ar.Drone 2.0 - полетееел http://twitter.com/Gertsevoa/statuses/24 ... ... https://twitter.com/Gertsevoa/statuses/24 ... Parrot Ar.Drone 2.0 - полетееел ... RT @OwenJones84: Other than propping up http://twitter.com/NooseCorpSE/statuses/ ... ... https://twitter.com/NooseCorpSE/statuses/ ... RT @OwenJones84: Other than propping up...
  • 3. MINE
  • The end result
  • 4. REPRESENT
  • 5. REFINE
  • 5. REFINE
  • 5. REFINE
  • 6. MAKE IT INTERACTIVE Add methods for people to manipulate the data, or control what features are visible More importantly, it empowers people to explore your data! Also good for showing off
  • MAKE DECISIONS Do something with the data If you don’t, you’ve just made expensive graphs What do you know now that you didn’t know before?
  • WHAT ELSE COULD YOU DO? Look through the rest of Datasift’s data Make ever more beautiful (and useful!) visualizations
  • OR GET IN TOUCH WITH US!HUMANS AND ALGORITHMS
  • WE WORK WITH EVERYTHINGHUMANS AND ALGORITHMS
  • Are you an NYC influencer? Find out @TemperoUK
  • Thanks for not falling asleep.Any questions?#SMWDataGeeksoverviewproject.orgMichael Conroy Jonathan StraySenior Analyst, Tempero Project Lead, Overview@mickyconroy @jonathanstray@TemperoUK @overviewproject