#FAIL! Things that didn't work out in social media research - and what we can learn from them. #fail2015a Workshop at Web Science Conference 2015, Oxford, June 2015.

1. #FAIL!
THINGS THAT DIDN‘T WORK OUT IN SOCIAL
MEDIA RESEARCH
- AND WHAT WE CAN LEARN FROM THEM
Workshop at Web Science Conference, Oxford, June 29, 2015.

2. • Workshop hashtag: #fail2015a
• Conference hashtag: #websci15
• Workshop website:
http://failworkshops.wordpress.com
• Etherpad: https://pad.okfn.org/p/fail
WELCOME
Luca Rossi
@LR
Taha Yasseri
@TahaYasseri
Katrin Weller
@kwelle

3. 0
100
200
300
400
500
600
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Twitter
Facebook
YouTube
Blogs
Wikis
Foursquare
LinkedIn
MySpace
Number of publications per year, which mention the respective social media platform‘s name in their title. Scopus
Title Search. For details: http://kwelle.wordpress.com/2014/04/07/bibliometric-analysis-of-social-media-research/
CURRENT SOCIAL MEDIA RESEARCH

4. 2008-2013 papers on Twitter and elections: data sources
Weller, K. (2014). Twitter und Wahlen: Zwischen 140 Zeichen und Milliarden von Tweets. In: R. Reichert (Ed.), Big
Data: Analysen zum digitalen Wandel von Wissen, Macht und Ökonomie (pp. 239-257). Bielefeld: transcript.
4
Data source number
No information 11
Collected manually from Twitter website (Copy-Paste /
Screenshot)
6
Twitter API (no further information) 8
Twitter Search API 3
Twitter Streaming API 1
Twitter Rest API 1
Twitter API user timeline 1
Own program for accessing Twitter APIs 4
Twitter Gardenhose 1
Official Reseller (Gnip, DataSift) 3
YourTwapperKeeper 3
Other tools (e.g. Topsy) 6
Received from colleagues 1
CURRENT SOCIAL MEDIA RESEARCH

5. • Social media platforms and users – a moving
target…
• Best practices and pitfalls in social media
research are mainly discussed informally. Few
possibilities to share unsuccessful approaches.
• Researchers with lots of different disciplinary
backgrounds enter the field. Different fields of
expertise, few interdisciplinary exchange of
approaches.
• Limited possibilities for data sharing / validation
and reproduction of results.
CURRENT PROBLEMS

6. OUR SOLUTION: #FAIL! WORKSHOPS
• First workshop at WebSci15
• Traveling on to different conferences. Up next
is #ir16 (Internet Research, Phoenix, October
2015).
• Collect various examples for things that can go
wrong and share them with different
communities  learn from experiences.

7. WHAT COULD GO WRONG?
Upcoming examples from project “THE HIDDEN DATA OF SOCIAL MEDIA RESEARCH”, more information:
Kinder-Kurlanda, K., & Weller, K. (2014). “I always feel it must be great to be a hacker!” The role of interdisciplinary work in social media
research. Proceedings of the ACM Web Science Conference 2014, Bloomington, USA 2014.
Weller, K., & Kinder-Kurlanda, K. (2015). Uncovering the Challenges in Collection, Sharing and Documentation: The Hidden Data of Social Media
Research? In Standards and Practices in Large-Scale Social Media Research: Papers from the 2015 ICWSM Workshop. Proceedings Ninth
International AAAI Conference on Web and Social Media Oxford University, May 26, 2015 – May 29, 2015, 28-37. Ann Arbor, MI: AAAI Press.

8. research design I
“I mean we’ve looked at the Arab spring […].
And we were just tracking #Egypt and #Lybia as
keywords, as hashtags. What was quite obvious
early on was that we missed a lot of the early
tweets around Egypt because they were still
using #25jan or #jan25 as the hashtag around
the first big popular demonstration on the 25th
of January […] we didn’t really realize that.”

9. research design II
“My questions are limited
to what I can do.”

10. dependencies
• Platforms and APIs
• Other software

11. death by opportunities
“To be fair, I have collected a lot of data that I
haven't done anything with. I thought it would
be interesting, and it would be interesting, just I
haven't had time to look at everything. […] so I
have lots of datasets still sitting on my computer
waiting for me to actually do something with.”

12. traditional approaches meet new data
• Platforms and APIs
• Other software

13. ethics
“the quantity of the data we’re analyzing
doesn’t really allow us to use the same
approaches that have been developed over the
last centuries for social sciences. We cannot ask
for permission”

14. OUR AIM
• Categorization: Identify different reasons for
failed approaches in social media research 
guidebook?
Today:
- 4 presentations
- Think about your own experiences!
- … in connection to each presentation
- … in general

15. 2:00 pm Welcome and introduction
2:15 pm Presentation by Taha Yasseri:
“The double-edged sword of statistical significance”
2:45 pm Presentation by Michael Bossetta and Anamaria Dutceac Segesten:
“Tracing Eurosceptic Party Networks via Hyperlink Network Analysis and
#FAIL!ng: Can Web Crawlers Keep up with Web Design?”
3:15 pm break
3:30 pm Presentation by Elodie Crespel:
“Extending data collection with web browser extension”
4:00 pm Presentation by Marie Van Cranenbroeck: “
Managing and Using Unstable Data in a Social Science Research about
Museums and Audiences on Social Media”
4:30 pm: Discussion and conclusions
PROGRAM

16. • Other experiences?
• Main categories of #fail cases?
• Top 3 take away messages for next workshop?
DISCUSSION

17. WHERE TO GO FROM HERE?
• Next steps – lessons learnt for future
workshop organisation
• Which additional conferences?
• Publication?

18. THANKS!
To our
• authors
• participants
• reviewers