Methodological
Challenges
IR14 Pre-Conference Workshop Denver,CO
dp.woodford@qut.edu.au
@dpwoodford
Wednesday, 23 October 13

PLATFORMS
• Twitter: Open API, Limited on Volume, Real-Time unless
paying for API; good data on user profiles.
• Facebook: Multiple levels of access; public pages easily
accessible, historical data easy to obtain (unless deleted),
but less precise (e.g. likes not timestamped unless page
owner).
• Weibo? Vkontake? LinkedIn?
• Cost:Benefit analysis for developing infrastructure,
Wednesday, 23 October 13

TWITTER
• Streaming API is limited to ~1% of total tweets per second
& Firehose access is expensive.
• Large data sets are not easily malleable, or visually
analyzed (e.g. with Tableau):
– Our database of Twitter users is ~3.7TB, and growing.
– A weeks worth of selected TV data (current US shows) in JSON
format is 750MB, and 600MB in TSV (selected fields). And
millions of rows.
• Analyzing large data sets is slow, if it’s even possible =>
“Usable Data”
Wednesday, 23 October 13

WHAT DOES 1% LOOK LIKE DURING
SCANDAL?
Wednesday, 23 October 13

RANDOM SAMPLING VS FULL SAMPLE
ANALYSIS
Source:
Tony
Hirst
(Open
University
UK)
Wednesday, 23 October 13

FULL SAMPLE, REPEATED CAPTURE
Source:
Bruns
/
Woodford
[Mapping
Online
Publics]
Wednesday, 23 October 13

ANALYSIS TOOLS: DATA ANALYTICS
• Historically @ CCI / Mapping Online Publics: .csv files,
Gawk Scripts, Excel.
• Ideally: Custom Tools, R, Large Sample analysis.
• Reality: We’ve compromised on Tableau. A great option for
easy data analysis, and reasonably large samples:
– But still, large flat files => memory leaks (e.g. Accession charts
for Barack Obama).
– Currently investigating database solutions, with mySQL as a
stop-gap.
Wednesday, 23 October 13

ANALYSIS TOOLS: CONTENT ANALYSIS
Source:
Woodford
/
Prowd
[Fan
Cultures
and
Hatred
in
Big
Brother
15:
Race
Rows,
EliMsm
&
SporMng
Tribalism
-­‐-­‐
Forthcoming]
• Lots of advanced tools, but sometimes they overcomplicate things; simple visuals are good for
communicating to your audience..
Wednesday, 23 October 13

ANALYSIS TOOLS: WORDLE SOMETIMES
WORKS
Source:
Woodford
/
Prowd
[Fan
Cultures
and
Hatred
in
Big
Brother
15:
Race
Rows,
EliMsm
&
SporMng
Tribalism
-­‐-­‐
Forthcoming]
Wednesday, 23 October 13

ANALYSIS TOOLS: WORDLE SOMETIMES
WORKS
Source:
Woodford
/
Prowd
[Fan
Cultures
and
Hatred
in
Big
Brother
15:
Race
Rows,
EliMsm
&
SporMng
Tribalism
-­‐-­‐
Forthcoming]
Wednesday, 23 October 13

WHO’S THE AUDIENCE?
• Who you want to connect with is important in choosing
methods and presentations.
• My commentary / analysis on Big Brother 15 (US) across
my blog, MOP and SocialMedia@QUT was read many
more times than ANY academic article I’ve ever written.
• And was picked up by industry..
• Who should we be writing for?
Wednesday, 23 October 13

ACKNOWLEDGEMENTS
• ARC Centre for Excellence in Creative Industries and
Innovation (CCI) - http://www.cci.edu.au & http://
www.mappingonlinepublics.net
• Social Media Research Group -- http://
socialmedia.qut.edu.au
• Queensland University of Technology
Wednesday, 23 October 13