SXSWi '11: Beyond Wordclouds: Analyzing Trends with Social Media APIs

Beyond Wordclouds
Chris Busse
Partner, Fahrenheit Technology
Emerging Media Group
Richmond, VA #RVA
Work: FahrenheitTechnology.com
Blog: ChrisBusse.com
Twitter: @busse
Analyzing Trends with Social Media APIs
SXSW 2011
#beyondwc
SXSW 2011 #beyondwc

This talk will cover:
Some things I’ve learned from building systems to monitor online communications for brands
An analysis of 100K Tweets from 48 hours of monitoring “coffee” on Twitter on Thursday and Friday of SXSW
Q & A
SXSW 2011 #beyondwc

API: Application Programming Interface
An application programming interface (API) is a particular set of rules and specifications that a software program can follow to access and make use of the services and resources provided by another particular software program that implements that API.
It serves as an interface between different software programs and facilitates their interaction, similar to the way the user interface facilitates interaction between humans and computers.
http://en.wikipedia.org/wiki/Application_programming_interface
SXSW 2011 #beyondwc

SXSW 2011 #beyondwc

<entry>
<id>tag:search.twitter.com,2005:47520097484550144</id>
<published>2011-03-15T04:50:48Z</published>
<link type="text/html" href="http://twitter.com/wingztw33t/statuses/47520097484550144" rel="alternate"/>
<title>I'm at Sin Hoi How Coffee Shop (JalanTun H S Lee, Kuala Lumpur) http://4sq.com/ebaDo2</title>
<content type="html">I'm at Sin Hoi How <b>Coffee</b> Shop (JalanTun H S Lee, Kuala Lumpur) <ahref="http://4sq.com/ebaDo2">http://4sq.com/ebaDo2</a></content>
<updated>2011-03-15T04:50:48Z</updated>
<link type="image/png" href="http://a1.twimg.com/profile_images/658397775/zoom_resize_normal.jpg" rel="image"/>
<twitter:geo> </twitter:geo>
<twitter:metadata>
<twitter:result_type>recent</twitter:result_type> </twitter:metadata>
<twitter:source><ahref="http://foursquare.com" rel="nofollow">foursquare</a></twitter:source> <twitter:lang>id</twitter:lang>
<author> <name>wingztw33t (TeikWingChoo)</name> <uri>http://twitter.com/wingztw33t</uri>
</author>
</entry>
SXSW 2011 #beyondwc

APIs are the Bridge Between Systems
API
Customer
Service
API
SXSW 2011 #beyondwc

APIs are the Bridge Between Systems
API
SXSW 2011 #beyondwc

A Problem I Faced
Need to monitor a large volume of SM data around a brand, its products, and competitors
Need to identifytrends in that data that might not be readily apparent
Need to manage the communications activities of a team of online “Digital Advocates”
Need to link this data to other internal software/ database systems (order processing, support center)
SXSW 2011 #beyondwc

The Starting Point
Wanted custom reporting, but commercially available tools didn’t seem like a good fit
UI/UX frustrations
Cost for large volume of data
Extensibility / deep customization
Felt there were limitations in the Twitter Search API
Historical depth
Query flexibility
Needed to mix in more data (corporate mashups)
I wanted control (choosing the wrong vendor partner jeopardizes my client relationships)
I wanted to work with the development tools I’m familiar with
SXSW 2011 #beyondwc

Buy vs Build?: Reasons to Build
Standard question for “the Enterprise”
Get data in to other sources, no silos
Custom workflows
Social CRM
Start an archive now, you never know when you’ll need it
Business Intelligence (BI) staff have a favored toolset
Enterprise data security: SaaS might be a NoNo
Enterprise platform standards: .NET, Java, LAMP, PowerPoint
SXSW 2011 #beyondwc

Build to be minimally relianton platform APIs
Get the data out and move on

<entry>
<id>tag:search.twitter.com,2005:47520097484550144</id>
<published>2011-03-15T04:50:48Z</published>
<link type="text/html" href="http://twitter.com/wingztw33t/statuses/47520097484550144" rel="alternate"/>
<title>I'm at Sin Hoi How Coffee Shop (JalanTun H S Lee, Kuala Lumpur) http://4sq.com/ebaDo2</title>
<content type="html">I'm at Sin Hoi How <b>Coffee</b> Shop (JalanTun H S Lee, Kuala Lumpur) <ahref="http://4sq.com/ebaDo2">http://4sq.com/ebaDo2</a></content>
<updated>2011-03-15T04:50:48Z</updated>
<link type="image/png" href="http://a1.twimg.com/profile_images/658397775/zoom_resize_normal.jpg" rel="image"/>
<twitter:geo> </twitter:geo>
<twitter:metadata>
<twitter:result_type>recent</twitter:result_type> </twitter:metadata>
<twitter:source><ahref="http://foursquare.com" rel="nofollow">foursquare</a></twitter:source> <twitter:lang>id</twitter:lang>
<author> <name>wingztw33t (TeikWingChoo)</name> <uri>http://twitter.com/wingztw33t</uri>
</author>
</entry>
SXSW 2011 #beyondwc

Data Fields
Id
Published
Link (to Tweet)
Title
Content
Updated
Link (to Profile Image)
Twitter:Geo
Twitter:Metadata
Twitter:Result Type
Twitter:Source
Twitter:Lang
Author
Name
URI
SXSW 2011 #beyondwc

Automate
SXSW 2011 #beyondwc

Tweets + Your CRM Data = Social CRM
SXSW 2011 #beyondwc

Automate More!
SXSW 2011 #beyondwc

Customer support issues?
I thought this was about “analyzing trends” ?

Measuring ROI in Social Media
That’s like, the Holy Grail, right?

Measuring ROI
SXSW 2011 #beyondwc

Always Go Back to Your Goals and Objectives … as quickly as possible
SXSW 2011 #beyondwc

Coffee
100K Tweets on
SXSW 2011 #beyondwc

Collecting Tweets with “coffee”
Start: 3/11/11 17:24:01 GMT
End: 3/13/11 15:14:53 GMT
# Tweets: 101,487
SXSW 2011 #beyondwc

“coffee” + _______
SXSW 2011 #beyondwc

“coffee” + _______
1,377
SXSW 2011 #beyondwc

“coffee” + (“4sq” | “gowal”)
SXSW 2011 #beyondwc

Most Popular Coffee in Austin?
SXSW 2011 #beyondwc

Most Popular Coffee in the World?
. . . . . . . . . . . . . . . . .
SXSW 2011 #beyondwc

Worldwide Long Tail % of Check-in
SXSW 2011 #beyondwc

When does Austin get coffee?
| Thursday | Friday | Saturday
SXSW 2011 #beyondwc

SXSW 2011 #beyondwc

Did everyone leave New York?
SXSW 2011 #beyondwc

What’s in a word?
140,295 words in 100K tweets

How to Turn Tweets in to Words
Loop through all your collected Tweets
Split() them on “ ‘
Strip out , . ! ? ( ) … and also “: “
Escape the single quotes
Strip out/exclude common words like “the”, “for”, “you”, …
Record the Word back to a database table
Perhaps add counts to that word, and segment by a time span such as day or hour
SXSW 2011 #beyondwc

General Word Clouds Lack Relevance
Source:http://tweetcloud.com/search/term/coffee
Source: custom script/db
via Twitter Search API
SXSW 2011 #beyondwc

SXSW 2011 #beyondwc

Decaf is more popular?!
SXSW 2011 #beyondwc

…or maybe just linkbait
SXSW 2011 #beyondwc

Speaking of Links
SXSW 2011 #beyondwc

SXSW 2011 #beyondwc

Look at More Data(there’s an API for this too)
SXSW 2011 #beyondwc

Other things to explore
Better location-based analysis
Words over time span, compared to previous time span
Add the people who tweeted about this back in to the search strings, capture their tweets for analysis
Find even more data to add in to the mix
Begin to identify real influencers in the context of your strategic goals and objectives – this might not be as simple as a “score”
SXSW 2011 #beyondwc

The Future
Open APIs vs. Closed APIs
Cost / Terms of Service starts to close down what was once open (or perhaps ensures its continued availability?)
Platform commitment
Traditional software package vendors working with platform/service APIs intelligently
Smart people solving this problem for the Enterprise (and smaller orgs) in a way that fits strategy, not forcing strategy to fit a tool
SXSW 2011 #beyondwc

Q & A
Contact Info:
Chris Busse
@busse
cbusse@fahrenheittechnology.com
http://FahrenheitTechnology.com
http://ChrisBusse.com
Thank You:
@jboogie& @mr_sterling
@mavnet & @VCU_Brandcenter
@cdharrison & @krusk & @akuzemchak
@willowtree84
@IconBuffet
The entire @FahrenheitTech / @FahrenheitEM Team
All who helped promote this talk & all the SXSW staff & volunteers!
SXSW 2011 #beyondwc

http://chrisbusse.com	258
http://paper.li	5
http://webcache.googleusercontent.com	1
http://www.linkedin.com	1

SXSWi '11: Beyond Wordclouds: Analyzing Trends with Social Media APIs

by Chris Busse

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 265

Statistics