University of Southampton, profile picture
Uploaded byUniversity of Southampton
PPTX, PDF563 views

Online data sources and information exposure

The document discusses various online data sources relevant to social science research, focusing on data collection methods from social media and web browsing. It highlights the significance of analyzing user-generated content and clickstream data for understanding public opinion and information exposure, particularly in the context of ideological segregation. Additionally, it outlines processing techniques for extracting and analyzing data, including web scraping and text analysis, while providing statistical insights from a survey and detailing the methodologies for cleaning and organizing the data collected.

Embed presentation

Download to read offline
Online Data Sources: Linking Old and New, Big and Small Susan Banducci and Iulia Cioroianu Exeter Q-Step Centre
• Many questions of interest to social scientists: candidate messages, events data, public opinion… What can social media data tell us? • highly segregated partisan structure … limited connectivity..." (Conover et al. 2011)​ e.g. ideological polarisation online (Twitter) • "ideological segregation … is low in absolute terms” (Gentzkow & Shapiro 2011) But what happens when we look at other data sources?
Information Exposure • Platforms (e.g. Twitter) for sharing user- generated content, often including links to other content (e.g news stories) Social media • Visits sites by directly clicking, from aggregators or from search engines (clickstream data) Online browsing • Asking people what they saw, read or watched onlineSurveys
What we will cover: • Sources of data • Extracting data • Processing raw data • Analysis – comparing sources Computational methods for studying social phenomena: Comparing data sources
• 3 waves: February, April, June 2016 • 1154 respondents Survey (ICM) • 900 respondents with clickstream/browsing history • 2 weeks per wave Browsing History (Clickstream) • Underrepresents retired respondents (15% to 30%), younger (avg age < 4 yrs), similar on women/men, regionally representative. Representativeness (BES f2f)
Where do we get data on what people read online? Web browsing data • Our data • ICM Reflected Life panel. • Other methods • Bing browser extension • Other apps (CosComm) Individual level
Cleaning web history data https://mvt.api.bbc.com/buckets?activate=false http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bmost-popular%5D%5Bid%5D=comp-most-popular&batch%5Bmost- popular%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bmost-popular%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=4&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=true&batch%5Bmost-popular%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=secondary_column http://www.bbc.co.uk/news/uk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-leave.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 86457556&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-remain.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 68662470&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-turnout.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 3191489&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-winners.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 36415436&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14368/content/iframe/english/index.html?v=0.2.35&initialWidth=592&childId=responsive-iframe- 72190634&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 https://ssl.bbc.co.uk/idcta/init?policy=notification&buttonColour=white&ptrt=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bfrom-other-news-sites%5D%5Bid%5D=comp-from-other-news-sites&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bconditions%5D%5B%5D=is_local_page&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Basset_id%5D=uk-politics- 36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=7&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=false&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=primary_column&batch%5Bfrom-other-news-sites%5D%5Btemplate%5D=%2Fcomponent%2Ffrom-other-news- sites
Cleaning web history data https://mvt.api.bbc.com/buckets?activate=false http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bmost-popular%5D%5Bid%5D=comp-most-popular&batch%5Bmost- popular%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bmost-popular%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=4&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=true&batch%5Bmost-popular%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=secondary_column http://www.bbc.co.uk/news/uk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-leave.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 86457556&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-remain.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 68662470&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-turnout.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 3191489&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-winners.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 36415436&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14368/content/iframe/english/index.html?v=0.2.35&initialWidth=592&childId=responsive-iframe- 72190634&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 https://ssl.bbc.co.uk/idcta/init?policy=notification&buttonColour=white&ptrt=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bfrom-other-news-sites%5D%5Bid%5D=comp-from-other-news-sites&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bconditions%5D%5B%5D=is_local_page&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Basset_id%5D=uk-politics- 36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=7&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=false&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=primary_column&batch%5Bfrom-other-news-sites%5D%5Btemplate%5D=%2Fcomponent%2Ffrom-other-news- sites http://www.bbc.co.uk/news/uk-politics-36616028
How do we identify news page urls? Cleaning clickstream data List of UK news domains Amazon Alexa Top Sites (alternative: SimilarWeb) •Expert coders checked the list. Restrict clickstream links to news domains •460 news domains Identify news article pages on these domains (vs. ads, widgets, images, videos, etc.) Regular expressions •http://www.bbc.co.uk/news/.+- d{8}$ •Alternative: machine learning. Restrict data to news articles 26.781 unique news urls in the clickstream data
Collecting Twitter data The method you choose depends on: • Your programming skills or willingness to develop these skills • The characteristics of the data you want to collect • Your budget Twitter APIs • Some programming skills required • Many available packages in Python (Yweepy) and R (TwitteR) • Free scripts and tutorials: ExpoNET tools • Flexible but constraints on data collected. Web scraping • More advanced programming • More flexible, you can get more data • Still have to follow Twitter rules of service Commercial or free software • Chorus, NodeXL, Voson, etc. • Easy access • Some include data analysis options • Less flexible • Same constraints as the API Purchasing data from Twitter • Convenient but very expensive
Processing social media data • Hierarchical data. • More efficient. • Scales up. Working with .json files. • Relational databases • MySQL • Non-relational databases • MongoDB Data storage • Text • User information • Lists of friends and followers • Links shared Extracting relevant information { "created_at": "Thu Apr 06 15:24:15 +0000 2016", "id": 850006145121695744, "id_str": "850006145121695744", "text": "Support our campaign! nhttps://t.co/XweGngmxlP", "user": { "id": 2244994945, "id_str": "2244994945", "screen_name": "Campaigner", "followers_count": 47684, "friends_count": 1524, "favourites_count": 251, "statuses_count": 3121 }, "entities": { "urls": [{ "indices": [ 32, 52 ], "url": "http://t.co/IOwBrTZR", "display_url": "youtube.com/watch?v=oHg5SJ…", "expanded_url": "http://www.campaign.com" }] } } 1.6 million tweets with links to news domains
Extracting text, title, author, etc. from article pages Web scraping • Python Scrapy, BeautifulSoup, Selenium Boilerplate/content extraction tools • Diffbot, Dragnet, Skin Tech Resolving urls These are the same: • Goo.gl/C8Snv6 • http://www.bbc.co.uk/news/politics/uk_ leaves_the_eu Python requests, urllib, urlclean Clickstream Twitter
Extracting information from text From words to numbers. Text processing • Standard text cleaning and NLP methods • Lower case, removing punctuation and stop words, stemming, tokenization • Part of speech tagging, n-grams Counting keywords Measuring distances/similarities Topic extraction
Topic models Uncovering hidden thematic structures in documents. Latent Dirichlet Allocation (LDA) • Documents are a mixture of topics. • Topics generate words based on their probability distribution. • Algorithm: • Determine number of words in document. • Determine mixture of topics in document. • Based on the topics' multinomial distribution, assign words to documents. Mallet, Python Gensim, R Quanteda.
id label top probability words 0 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand"
id label top probability words 0 ? 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 ? 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 ? 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 ? 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 ? 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand"
id label top probability words 0 economy 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 education 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 immigration 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 defence 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 housing 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand"
id label top probability words 0 economy 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 education 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 immigration 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 defence 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 housing 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand" • Topic probability in each document. • For each topic, average probabilities across articles extracted from clickstream and twitter data.
Building networks • Explicitly model interdependencies between observations. • Patterns of information exposure • Detect communities/echo chambers Why model it like this? • Mathematical construct of nodes and edges. • Information (attributes) on the nodes as well as the edges. What is a network? • News domains are nodes • Edges are formed between two domains if a user shares/reads an article on both domains. In our case:
Data linking Multiple levels. • User level Survey and clickstream data • Domain level • Article level Clickstream and social media data • Domain level • Article level Survey, clickstream and social media data
Following slides show results from: Survey and links/URLs (comparing users) Topics from news stories and topic modelling (comparing topics) Networks for shared stories (comparing networks)
Comparing Users: Clickstream Data After Processing 56,289 data points Create Quantities of Interest: Total URLs visited, text analysis of URLs, Left/Right URLs, 0 5,000 10,000 15,000 dailypost telegraph metro express theguardian thesun mirror dailymail manchestereveningnews bbc
Comparing Users: Reported Use & URL visits Clickstream processed URLs Quantity of interest – count of visited news URLs per user about EU Survey Data Reported online news exposure Over the past 7 days, how often have you done the following with respect to the EU Referendum…? 0246810 ReadOnlineNewspaperStoriesaboutEU Not at All Once or Twice Most Days Every Day Days Last Week on News About EU
Comparing users: Brexit preference and news exposure. Bars indicate mean number of views per user during fieldwork period (n = 513).
Topics in our corpus
Comparing Content What do we learn about information exposure by comparing content/topics from news stories shared on Twitter to news stories viewed online? Content
Comparing networks What do we learn about information exposure by comparing news exposure from browsing histories to social media data? Networks
Methodologica l challenges in using online data. Can we conclude that social media leads to segregation in news exposure based on 40million Tweets? Challenges: Representativeness, Measurement error & causal inference Multiple sources of linked data as a possible solution.
Additional resources • Annotated bibliography, notes from presentation: https://mediaeffectsresearch.wordpress.com/ • Jupyter notebooks in our GitHub repository: https://github.com/EXPONetProject/tools/
Additional resources • Annotated bibliography, notes from presentation: https://mediaeffectsresearch.wordpress.com/ • Jupyter notebooks in our GitHub repository: https://github.com/EXPONetProject/tools/

More Related Content

PPT
Information Retrieval and Social Media
byArjen de Vries
 
PPTX
Web Archives and the dream of the Personal Search Engine
byArjen de Vries
 
PDF
Using social media to address professional issues in LIS
byMmIT - Multimedia Information Technology Group for CILIP
 
PPTX
Exploring the Networks in Open Public Data
byUldis Bojars
 
PPTX
Research into Practice case study 2: Library linked data implementations an...
byHazel Hall
 
PPTX
Big data
byStar Queens
 
PPTX
Linked open data project
byFaathima Fayaza
 
PPT
Envisioning Social Applications of Library Linked Data
byUldis Bojars
 
Information Retrieval and Social Media
byArjen de Vries
 
Web Archives and the dream of the Personal Search Engine
byArjen de Vries
 
Using social media to address professional issues in LIS
byMmIT - Multimedia Information Technology Group for CILIP
 
Exploring the Networks in Open Public Data
byUldis Bojars
 
Research into Practice case study 2: Library linked data implementations an...
byHazel Hall
 
Big data
byStar Queens
 
Linked open data project
byFaathima Fayaza
 
Envisioning Social Applications of Library Linked Data
byUldis Bojars
 

What's hot

PDF
Collecting Twitter Data
byCornelius Puschmann
 
PPTX
New trends and skill in library automation: impact of Artificial Intelligence...
byMokhtar Ben Henda
 
PPTX
The Power of Open Data!
byRenaine Julian
 
PPTX
Jussi Nuorteva - Power of Open Data in Archives
byICARUS - International Centre for Archival Research
 
PDF
Sirris innovate2011 - Smart Products with smart data - introduction, Dr. Elen...
bySirris
 
PDF
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...
byShalin Hai-Jew
 
PPTX
E-challenges11 WeGov Workshop
byWeGov project
 
PDF
Rogers digitalmethods 4nov2010
byDigital Methods Initiative
 
PPTX
WAPWG 16 Jan Thomson holdslide
bySara Day Thomson
 
PDF
Tut mathematics and hypermedia research seminar 2011 11-11
byYleisradio
 
PDF
Towards Cognitive Agents for BigData Discovery
byJack Park
 
PDF
Measuring System Performance in Cultural Heritage Systems
byToine Bogers
 
PDF
Designing a second generation of open data platforms
byYannis Charalabidis
 
PDF
Tagging: Can User-Generated Content Improve Our Services?
byKatja Šnuderl
 
PDF
TING.concept ELAG conference presentation 2010-06-09
byhernvall
 
PPT
Evolution of e-Content Distribution: Ad Hoc to Standardization
byNational Information Standards Organization (NISO)
 
PDF
Nilges Making The Metadata Work NISO Virtual Conference Ebooks
byNational Information Standards Organization (NISO)
 
Collecting Twitter Data
byCornelius Puschmann
 
New trends and skill in library automation: impact of Artificial Intelligence...
byMokhtar Ben Henda
 
The Power of Open Data!
byRenaine Julian
 
Jussi Nuorteva - Power of Open Data in Archives
byICARUS - International Centre for Archival Research
 
Sirris innovate2011 - Smart Products with smart data - introduction, Dr. Elen...
bySirris
 
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...
byShalin Hai-Jew
 
E-challenges11 WeGov Workshop
byWeGov project
 
Rogers digitalmethods 4nov2010
byDigital Methods Initiative
 
WAPWG 16 Jan Thomson holdslide
bySara Day Thomson
 
Tut mathematics and hypermedia research seminar 2011 11-11
byYleisradio
 
Towards Cognitive Agents for BigData Discovery
byJack Park
 
Measuring System Performance in Cultural Heritage Systems
byToine Bogers
 
Designing a second generation of open data platforms
byYannis Charalabidis
 
Tagging: Can User-Generated Content Improve Our Services?
byKatja Šnuderl
 
TING.concept ELAG conference presentation 2010-06-09
byhernvall
 
Evolution of e-Content Distribution: Ad Hoc to Standardization
byNational Information Standards Organization (NISO)
 
Nilges Making The Metadata Work NISO Virtual Conference Ebooks
byNational Information Standards Organization (NISO)
 

Similar to Online data sources and information exposure

PPTX
Social Media Data Collection & Analysis
byScott Sanders
 
PPT
The evolution of research on social media
byFarida Vis
 
PDF
s00146-014-0549-4.pdf
byEngrAliSarfrazSiddiq
 
PPTX
CSE509 Lecture 5
byWeb Science Research Group at Institute of Business Administration, Karachi, Pakistan
 
PDF
Unleashing twitter data for fun and insight
byDigital Reasoning
 
PDF
Social Media As Social Science Data Steven Lloyd Wilson
byhursebonieyi
 
PDF
Big Data Factories Collaborative Approaches 1st Edition Sorin Adam Matei
byfahqcyzr907
 
PDF
Online text data for machine learning, data science, and research - Who can p...
byFredrik Olsson
 
PDF
Challenges in-archiving-twitter
byKatrin Weller
 
PDF
The data we want
byElena Simperl
 
PPTX
Ralph schroeder and eric meyer
byoiisdp
 
PDF
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
byStefan Dietze
 
PDF
Weller social media as research data_psm15
byKatrin Weller
 
PDF
Processing Large Complex Data
byYiannis Kompatsiaris
 
PDF
Why Twitter Is All The Rage: A Data Miner's Perspective (PyTN 2014)
byMatthew Russell
 
PDF
Unleashing Twitter Data for Fun and Insight
byMatthew Russell
 
PDF
Big data tutorial_part4
byheyramzz
 
PDF
Mining Social Web APIs with IPython Notebook (Strata 2013)
byMatthew Russell
 
PPTX
Accessing and analysing your own social media data.pptx
byLadduAnanu
 
PDF
591 Final Report - Team 7 - Political Issues
byTim Sawicki
 
Social Media Data Collection & Analysis
byScott Sanders
 
The evolution of research on social media
byFarida Vis
 
s00146-014-0549-4.pdf
byEngrAliSarfrazSiddiq
 
CSE509 Lecture 5
byWeb Science Research Group at Institute of Business Administration, Karachi, Pakistan
 
Unleashing twitter data for fun and insight
byDigital Reasoning
 
Social Media As Social Science Data Steven Lloyd Wilson
byhursebonieyi
 
Big Data Factories Collaborative Approaches 1st Edition Sorin Adam Matei
byfahqcyzr907
 
Online text data for machine learning, data science, and research - Who can p...
byFredrik Olsson
 
Challenges in-archiving-twitter
byKatrin Weller
 
The data we want
byElena Simperl
 
Ralph schroeder and eric meyer
byoiisdp
 
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
byStefan Dietze
 
Weller social media as research data_psm15
byKatrin Weller
 
Processing Large Complex Data
byYiannis Kompatsiaris
 
Why Twitter Is All The Rage: A Data Miner's Perspective (PyTN 2014)
byMatthew Russell
 
Unleashing Twitter Data for Fun and Insight
byMatthew Russell
 
Big data tutorial_part4
byheyramzz
 
Mining Social Web APIs with IPython Notebook (Strata 2013)
byMatthew Russell
 
Accessing and analysing your own social media data.pptx
byLadduAnanu
 
591 Final Report - Team 7 - Political Issues
byTim Sawicki
 

More from University of Southampton

PPTX
Data quality: total survey error
byUniversity of Southampton
 
PPTX
Introduction to Survey Data Quality
byUniversity of Southampton
 
PPTX
Teaching research methods: pedagogy of methods learning
byUniversity of Southampton
 
PPTX
Generating SPSS training materials in StatJR
byUniversity of Southampton
 
PDF
Introduction to spatial interaction modelling
byUniversity of Southampton
 
PPT
Introduction to multilevel modelling | Ian Brunton-Smith
byUniversity of Southampton
 
PPTX
Cognitive interviewing
byUniversity of Southampton
 
PPTX
Survey questions and measurement error
byUniversity of Southampton
 
PPTX
Biosocial research:How to use biological data in social science research?
byUniversity of Southampton
 
PPT
Multi level modelling - random intercept models | Ian Brunton Smith
byUniversity of Southampton
 
PPT
Multi level modelling- random coefficient models | Ian Brunton-Smith
byUniversity of Southampton
 
PPTX
Multilevel models:random coefficient models
byUniversity of Southampton
 
PPTX
Multilevel models:random intercept models
byUniversity of Southampton
 
PPTX
Introduction to multilevel modelling
byUniversity of Southampton
 
PPTX
How to write about research methods
byUniversity of Southampton
 
PPTX
Teaching research methods: pedagogy hooks
byUniversity of Southampton
 
PPT
Integrating biological and social research data - Michaela Benzeval
byUniversity of Southampton
 
PPTX
Introduction to the Stat-JR software package
byUniversity of Southampton
 
PPTX
better off living with parents
byUniversity of Southampton
 
PPTX
How to write about research methods
byUniversity of Southampton
 
Data quality: total survey error
byUniversity of Southampton
 
Introduction to Survey Data Quality
byUniversity of Southampton
 
Teaching research methods: pedagogy of methods learning
byUniversity of Southampton
 
Generating SPSS training materials in StatJR
byUniversity of Southampton
 
Introduction to spatial interaction modelling
byUniversity of Southampton
 
Introduction to multilevel modelling | Ian Brunton-Smith
byUniversity of Southampton
 
Cognitive interviewing
byUniversity of Southampton
 
Survey questions and measurement error
byUniversity of Southampton
 
Biosocial research:How to use biological data in social science research?
byUniversity of Southampton
 
Multi level modelling - random intercept models | Ian Brunton Smith
byUniversity of Southampton
 
Multi level modelling- random coefficient models | Ian Brunton-Smith
byUniversity of Southampton
 
Multilevel models:random coefficient models
byUniversity of Southampton
 
Multilevel models:random intercept models
byUniversity of Southampton
 
Introduction to multilevel modelling
byUniversity of Southampton
 
How to write about research methods
byUniversity of Southampton
 
Teaching research methods: pedagogy hooks
byUniversity of Southampton
 
Integrating biological and social research data - Michaela Benzeval
byUniversity of Southampton
 
Introduction to the Stat-JR software package
byUniversity of Southampton
 
better off living with parents
byUniversity of Southampton
 
How to write about research methods
byUniversity of Southampton
 

Recently uploaded

PDF
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
PPTX
Pig- piggy bank in Big Data Analytics.ppt.pptx
byVarshini M
 
PPTX
Repeat Orders_ Use Odoo Blanket Agreement
byCeline George
 
PPTX
How to Configure Push & Pull Rule in Odoo 18 Inventory
byCeline George
 
PPTX
AN EXTREMELY BORING GENERAL QUIZ FOR UG.pptx
bySriramG47
 
PPTX
UNIT 1: COMMUNICATION SKILLS, BARRIERS TO COMMUNICATION, PERSPECTIVES IN COMM...
byPOOJA KARVANDE
 
PPTX
Planning Assessment for Outcome-based Education
byAdri Jovin
 
PDF
IMANI Africa files RTI request seeking full disclosure on 2026 SIM registrati...
bynservice241
 
PPTX
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 
PPTX
How to Create Lead_Opportunity From Odoo 18 Website
byCeline George
 
PDF
Most Imp Chapters & Weightage for Boards 2025.pdf
byTamannaSagar
 
PPTX
The Cell & Cell Cycle-detailed structure and function of organelles.pptx
byAshish Umale
 
PDF
Biology Practical Class 12th 2025 PDF(2)-2-52.pdf
bySCPRPWomenCollegeChi
 
PDF
DHA/HAAD/MOH/DOH OPTOMETRY MCQ PYQ. .pdf
byAnmol Singh
 
PPTX
Introduction-to-Anatomy-and-Physiology.pptx
byAshish Umale
 
PDF
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
PPTX
ATTENTION -PART 2.pptx Shilpa Hotakar for I semester BSc students
bySHILPA HOTAKAR
 
PDF
Types of eggs and Egg membranes (Developmental Zoology)
bySudhandraKarthi
 
PPTX
Partial Correlation - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂ r₁₂, r₁₃ and r₂₃
bySundar B N
 
PPTX
REVISED DEFENSE MECHANISM / ADJUSTMENT MECHANISM-FINAL
byShimla
 
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
Pig- piggy bank in Big Data Analytics.ppt.pptx
byVarshini M
 
Repeat Orders_ Use Odoo Blanket Agreement
byCeline George
 
How to Configure Push & Pull Rule in Odoo 18 Inventory
byCeline George
 
AN EXTREMELY BORING GENERAL QUIZ FOR UG.pptx
bySriramG47
 
UNIT 1: COMMUNICATION SKILLS, BARRIERS TO COMMUNICATION, PERSPECTIVES IN COMM...
byPOOJA KARVANDE
 
Planning Assessment for Outcome-based Education
byAdri Jovin
 
IMANI Africa files RTI request seeking full disclosure on 2026 SIM registrati...
bynservice241
 
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 
How to Create Lead_Opportunity From Odoo 18 Website
byCeline George
 
Most Imp Chapters & Weightage for Boards 2025.pdf
byTamannaSagar
 
The Cell & Cell Cycle-detailed structure and function of organelles.pptx
byAshish Umale
 
Biology Practical Class 12th 2025 PDF(2)-2-52.pdf
bySCPRPWomenCollegeChi
 
DHA/HAAD/MOH/DOH OPTOMETRY MCQ PYQ. .pdf
byAnmol Singh
 
Introduction-to-Anatomy-and-Physiology.pptx
byAshish Umale
 
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
ATTENTION -PART 2.pptx Shilpa Hotakar for I semester BSc students
bySHILPA HOTAKAR
 
Types of eggs and Egg membranes (Developmental Zoology)
bySudhandraKarthi
 
Partial Correlation - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂ r₁₂, r₁₃ and r₂₃
bySundar B N
 
REVISED DEFENSE MECHANISM / ADJUSTMENT MECHANISM-FINAL
byShimla
 

Online data sources and information exposure

  • 1.
    Online Data Sources:Linking Old and New, Big and Small Susan Banducci and Iulia Cioroianu Exeter Q-Step Centre
  • 2.
    • Many questionsof interest to social scientists: candidate messages, events data, public opinion… What can social media data tell us? • highly segregated partisan structure … limited connectivity..." (Conover et al. 2011)​ e.g. ideological polarisation online (Twitter) • "ideological segregation … is low in absolute terms” (Gentzkow & Shapiro 2011) But what happens when we look at other data sources?
  • 3.
    Information Exposure • Platforms(e.g. Twitter) for sharing user- generated content, often including links to other content (e.g news stories) Social media • Visits sites by directly clicking, from aggregators or from search engines (clickstream data) Online browsing • Asking people what they saw, read or watched onlineSurveys
  • 4.
    What we willcover: • Sources of data • Extracting data • Processing raw data • Analysis – comparing sources Computational methods for studying social phenomena: Comparing data sources
  • 7.
    • 3 waves:February, April, June 2016 • 1154 respondents Survey (ICM) • 900 respondents with clickstream/browsing history • 2 weeks per wave Browsing History (Clickstream) • Underrepresents retired respondents (15% to 30%), younger (avg age < 4 yrs), similar on women/men, regionally representative. Representativeness (BES f2f)
  • 8.
    Where do weget data on what people read online? Web browsing data • Our data • ICM Reflected Life panel. • Other methods • Bing browser extension • Other apps (CosComm) Individual level
  • 10.
    Cleaning web historydata https://mvt.api.bbc.com/buckets?activate=false http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bmost-popular%5D%5Bid%5D=comp-most-popular&batch%5Bmost- popular%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bmost-popular%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=4&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=true&batch%5Bmost-popular%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=secondary_column http://www.bbc.co.uk/news/uk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-leave.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 86457556&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-remain.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 68662470&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-turnout.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 3191489&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-winners.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 36415436&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14368/content/iframe/english/index.html?v=0.2.35&initialWidth=592&childId=responsive-iframe- 72190634&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 https://ssl.bbc.co.uk/idcta/init?policy=notification&buttonColour=white&ptrt=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bfrom-other-news-sites%5D%5Bid%5D=comp-from-other-news-sites&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bconditions%5D%5B%5D=is_local_page&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Basset_id%5D=uk-politics- 36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=7&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=false&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=primary_column&batch%5Bfrom-other-news-sites%5D%5Btemplate%5D=%2Fcomponent%2Ffrom-other-news- sites
  • 12.
    Cleaning web historydata https://mvt.api.bbc.com/buckets?activate=false http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bmost-popular%5D%5Bid%5D=comp-most-popular&batch%5Bmost- popular%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bmost-popular%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=4&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bmost- popular%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=true&batch%5Bmost-popular%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=secondary_column http://www.bbc.co.uk/news/uk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-leave.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 86457556&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-parties-remain.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 68662470&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-turnout.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 3191489&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14367/content/iframe/english/uk-winners.html?v=0.5.55&initialWidth=592&childId=responsive-iframe- 36415436&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/special/2016/newsspec_14368/content/iframe/english/index.html?v=0.2.35&initialWidth=592&childId=responsive-iframe- 72190634&parentUrl=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 https://ssl.bbc.co.uk/idcta/init?policy=notification&buttonColour=white&ptrt=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fuk-politics-36616028 http://www.bbc.co.uk/news/components?alternativeJsLoading=true&batch%5Bfrom-other-news-sites%5D%5Bid%5D=comp-from-other-news-sites&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5BassetId%5D=36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bconditions%5D%5B%5D=is_local_page&batch%5Bfrom-other- news-sites%5D%5Bopts%5D%5Bloading_strategy%5D=include_content&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Basset_id%5D=uk-politics- 36616028&batch%5Bfrom-other-news-sites%5D%5Bopts%5D%5Bposition_info%5D%5BinstanceNo%5D=1&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BpositionInRegion%5D=7&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastInRegion%5D=true&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5BlastOnPage%5D=false&batch%5Bfrom-other-news- sites%5D%5Bopts%5D%5Bposition_info%5D%5Bcolumn%5D=primary_column&batch%5Bfrom-other-news-sites%5D%5Btemplate%5D=%2Fcomponent%2Ffrom-other-news- sites http://www.bbc.co.uk/news/uk-politics-36616028
  • 13.
    How do weidentify news page urls? Cleaning clickstream data List of UK news domains Amazon Alexa Top Sites (alternative: SimilarWeb) •Expert coders checked the list. Restrict clickstream links to news domains •460 news domains Identify news article pages on these domains (vs. ads, widgets, images, videos, etc.) Regular expressions •http://www.bbc.co.uk/news/.+- d{8}$ •Alternative: machine learning. Restrict data to news articles 26.781 unique news urls in the clickstream data
  • 15.
    Collecting Twitter data The methodyou choose depends on: • Your programming skills or willingness to develop these skills • The characteristics of the data you want to collect • Your budget Twitter APIs • Some programming skills required • Many available packages in Python (Yweepy) and R (TwitteR) • Free scripts and tutorials: ExpoNET tools • Flexible but constraints on data collected. Web scraping • More advanced programming • More flexible, you can get more data • Still have to follow Twitter rules of service Commercial or free software • Chorus, NodeXL, Voson, etc. • Easy access • Some include data analysis options • Less flexible • Same constraints as the API Purchasing data from Twitter • Convenient but very expensive
  • 16.
    Processing social media data • Hierarchicaldata. • More efficient. • Scales up. Working with .json files. • Relational databases • MySQL • Non-relational databases • MongoDB Data storage • Text • User information • Lists of friends and followers • Links shared Extracting relevant information { "created_at": "Thu Apr 06 15:24:15 +0000 2016", "id": 850006145121695744, "id_str": "850006145121695744", "text": "Support our campaign! nhttps://t.co/XweGngmxlP", "user": { "id": 2244994945, "id_str": "2244994945", "screen_name": "Campaigner", "followers_count": 47684, "friends_count": 1524, "favourites_count": 251, "statuses_count": 3121 }, "entities": { "urls": [{ "indices": [ 32, 52 ], "url": "http://t.co/IOwBrTZR", "display_url": "youtube.com/watch?v=oHg5SJ…", "expanded_url": "http://www.campaign.com" }] } } 1.6 million tweets with links to news domains
  • 18.
    Extracting text, title,author, etc. from article pages Web scraping • Python Scrapy, BeautifulSoup, Selenium Boilerplate/content extraction tools • Diffbot, Dragnet, Skin Tech Resolving urls These are the same: • Goo.gl/C8Snv6 • http://www.bbc.co.uk/news/politics/uk_ leaves_the_eu Python requests, urllib, urlclean Clickstream Twitter
  • 20.
    Extracting information from text From wordsto numbers. Text processing • Standard text cleaning and NLP methods • Lower case, removing punctuation and stop words, stemming, tokenization • Part of speech tagging, n-grams Counting keywords Measuring distances/similarities Topic extraction
  • 21.
    Topic models Uncovering hidden thematic structures in documents. LatentDirichlet Allocation (LDA) • Documents are a mixture of topics. • Topics generate words based on their probability distribution. • Algorithm: • Determine number of words in document. • Determine mixture of topics in document. • Based on the topics' multinomial distribution, assign words to documents. Mallet, Python Gensim, R Quanteda.
  • 22.
    id label topprobability words 0 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand"
  • 23.
    id label topprobability words 0 ? 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 ? 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 ? 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 ? 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 ? 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand"
  • 24.
    id label topprobability words 0 economy 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 education 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 immigration 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 defence 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 housing 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand"
  • 25.
    id label topprobability words 0 economy 0.016*"uk"+0.015*"growth"+0.014*"brexit"+0.012*"year"+0.010*"economy"+0.0 10*"uncertainty"+0.009*"referendum"+0.008*"term"+0.008*"economic"+0.008*" may"+0.007*"investment"+0.007*"cent" 1 education 0.032*"young"+0.029*"university"+0.026*"people"+0.020*"women"+0.019*"stud ents"+0.015*"research"+0.014*"professor"+0.013*"school"+0.011*"education"+0 .011*"eu"+0.010*"student"+0.010*"science"+0.009*"study" 2 immigration 0.036*"eu"+0.027*"immigration"+0.025*"uk"+0.020*"ireland"+0.015*"migration" +0.014*"people"+0.011*"citizens"+0.011*"british"+0.011*"work"+0.011*"migrants "+0.011*"irish"+0.010*"northern"+0.009*"living" 3 defence 0.027*"security"+0.021*"russia"+0.021*"nato"+0.017*"military"+0.015*"eu"+0.01 4*"defence"+0.014*"putin"+0.013*"russian"+0.011*"foreign"+0.010*"former"+0.0 10*"army"+0.009*"china"+0.008*"us"+0.008*"world" 4 housing 0.044*"prices"+0.044*"house"+0.034*"property"+0.032*"london"+0.031*"housin g"+0.026*"market"+0.023*"price"+0.019*"homes"+0.017*"mortgage"+0.016*"bu yers"+0.013*"average"+0.013*"home"+0.012*"new"+0.012*"demand" • Topic probability in each document. • For each topic, average probabilities across articles extracted from clickstream and twitter data.
  • 27.
    Building networks • Explicitlymodel interdependencies between observations. • Patterns of information exposure • Detect communities/echo chambers Why model it like this? • Mathematical construct of nodes and edges. • Information (attributes) on the nodes as well as the edges. What is a network? • News domains are nodes • Edges are formed between two domains if a user shares/reads an article on both domains. In our case:
  • 28.
    Data linking Multiple levels. • Userlevel Survey and clickstream data • Domain level • Article level Clickstream and social media data • Domain level • Article level Survey, clickstream and social media data
  • 29.
    Following slides show results from: Surveyand links/URLs (comparing users) Topics from news stories and topic modelling (comparing topics) Networks for shared stories (comparing networks)
  • 31.
    Comparing Users: ClickstreamData After Processing 56,289 data points Create Quantities of Interest: Total URLs visited, text analysis of URLs, Left/Right URLs, 0 5,000 10,000 15,000 dailypost telegraph metro express theguardian thesun mirror dailymail manchestereveningnews bbc
  • 32.
    Comparing Users: Reported Use & URL visits Clickstreamprocessed URLs Quantity of interest – count of visited news URLs per user about EU Survey Data Reported online news exposure Over the past 7 days, how often have you done the following with respect to the EU Referendum…? 0246810 ReadOnlineNewspaperStoriesaboutEU Not at All Once or Twice Most Days Every Day Days Last Week on News About EU
  • 33.
    Comparing users: Brexit preference and news exposure. Barsindicate mean number of views per user during fieldwork period (n = 513).
  • 35.
  • 36.
    Comparing Content What do welearn about information exposure by comparing content/topics from news stories shared on Twitter to news stories viewed online? Content
  • 38.
    Comparing networks What do welearn about information exposure by comparing news exposure from browsing histories to social media data? Networks
  • 39.
    Methodologica l challenges in usingonline data. Can we conclude that social media leads to segregation in news exposure based on 40million Tweets? Challenges: Representativeness, Measurement error & causal inference Multiple sources of linked data as a possible solution.
  • 40.
    Additional resources • Annotatedbibliography, notes from presentation: https://mediaeffectsresearch.wordpress.com/ • Jupyter notebooks in our GitHub repository: https://github.com/EXPONetProject/tools/
  • 41.
    Additional resources • Annotatedbibliography, notes from presentation: https://mediaeffectsresearch.wordpress.com/ • Jupyter notebooks in our GitHub repository: https://github.com/EXPONetProject/tools/

Editor's Notes

  • #2 Michael Alvaerz – Introuction to COmputational Social Science -  "the new data sets and analytical opportunities allow researchers to examine questions that could not be easily studied before and to conduct interesting and important research in cost-effective ways."  In other words, the chapters in this book all hinge on innovative computational tools used to answer lasting and important questions in social science and public policy, and thus I’ve titled the book Computational Social Science. Again, it’s not the data, itself nor the “size” of the data that is critical to the innovations in this volume; the emphasis rather is on the cross-disciplinary applications of statistical, computational, and machine learning tools to the new types of data, at a larger scale, to learn more about politics and policy.
  • #3  "Compared with algorithmic ranking, individuals’ choices played a stronger role in limiting exposure to cross-cutting content." (Bakshy et al. 2015)  Ideological Segregation Online and Offline * | The Quarterly Journal of Economics | Oxford Academic. (n.d.). Retrieved November 20, 2017, from https://academic.oup.com/qje/article/126/4/1799/1924154 We find that the growth in polarization in recent years is largest for the demographic groups least likely to use the internet and social media. https://www.brown.edu/Research/Shapiro/pdfs/age-polars.pdf We find that ideological segregation of onlinenews consumption is low in absolute terms, higher than the segregation of most offline news consumption, and significantly lower than the segregation of face-to-face interactions with neighbors, co-workers, or family members.
  • #4  "Compared with algorithmic ranking, individuals’ choices played a stronger role in limiting exposure to cross-cutting content." (Bakshy et al. 2015)  Ideological Segregation Online and Offline * | The Quarterly Journal of Economics | Oxford Academic. (n.d.). Retrieved November 20, 2017, from https://academic.oup.com/qje/article/126/4/1799/1924154 We find that the growth in polarization in recent years is largest for the demographic groups least likely to use the internet and social media. https://www.brown.edu/Research/Shapiro/pdfs/age-polars.pdf We find that ideological segregation of onlinenews consumption is low in absolute terms, higher than the segregation of most offline news consumption, and significantly lower than the segregation of face-to-face interactions with neighbors, co-workers, or family members.
  • #5 Everyone wants to use social media in research, but what does it tell us, and how does it compare to offline data, or data which is not generated and shared on a social network? Applied to multiple areas, …. We are interested in what it tells us about information exposure, and we will use this example in the presentation, but you can use the main tools and methods presented to study other topics as well.
  • #8  "Compared with algorithmic ranking, individuals’ choices played a stronger role in limiting exposure to cross-cutting content." (Bakshy et al. 2015)  Ideological Segregation Online and Offline * | The Quarterly Journal of Economics | Oxford Academic. (n.d.). Retrieved November 20, 2017, from https://academic.oup.com/qje/article/126/4/1799/1924154 We find that the growth in polarization in recent years is largest for the demographic groups least likely to use the internet and social media. https://www.brown.edu/Research/Shapiro/pdfs/age-polars.pdf We find that ideological segregation of onlinenews consumption is low in absolute terms, higher than the segregation of most offline news consumption, and significantly lower than the segregation of face-to-face interactions with neighbors, co-workers, or family members.
  • #16  Turn these into a pipeline. Adapt Exponet one. 
  • #21 Diagrams of what the text (of the tweet, of the links in the CS data, of the links in the twitter data) and networks are.  Example of link and story
  • #30 Present statistics on the number of people, number of domains, number of links, etc.
  • #32 Not sure what goes in this box
  • #39 Add network graph.
  • #40 Discuss them in relation to each type of data.  Usualy study a sample A sample is “a smaller (but hopefully representative) collection of units from a population used to determine truths about that population” (Field, 2005) factors that influence sample representativeness •Sampling procedure •Sample size •Participation (response) Sampling frame and coverage Can switch order of sthis slide and the one above.