SlideShare a Scribd company logo
The Dynamics of Topical Conversations
in Microblogging
Victoria Lai and William Rand
SocialCom 2013
September 11, 2013
 Keyword and trend identification
 Background noise
 Importance of time, subject,
geography?
2
 Twitter’s trending topics
 Real-time trend identification
◦ Mathioudakis and Koudas (2010)
◦ Cataldi, Di Caro, and Schifanella (2010)
 Other trend detection methods
◦ Benhardus and Kalita (2013)
 Trends in a geographic area
◦ Wilkinson and Thelwall (2012)
3
 Daily Twitter REST API queries, 47 weeks
 Topics
◦ 11 keywords/phrases chosen for time, subject, and
geographic variety (baseball, economy, global
warming, love, London riots)
◦ 5 control (I, the, and, a, of)
 Geography
◦ Global
◦ 9 cities (Boston, D.C.,
London, Sydney)
4
 Term frequency
 Inverse document frequency
 TF-IDF
 Keyword lists as target/control sets vary
 Time, subject, geography
 Tf-idf and rank correlation
𝑓𝑓 𝑇𝑇𝑡𝑡∗, 𝑠𝑠∗, 𝑔𝑔∗ 𝐶𝐶𝑡𝑡, 𝑠𝑠, 𝑔𝑔 )
6
…
tf phrase 1 #
phrase 2 #
phrase 3 #C0, ∀sc, ∀g
C1, ∀sc, ∀g
C2, ∀sc, ∀g
C3, ∀sc, ∀g
idf
phrase 1 #
phrase 2 #
phrase 3 #
phrase 1 #
phrase 2 #
phrase 3 #
…
T0, s*, ∀g
* SC = {I, the, and, a, of }
 Control set variation by time
◦ Single week
◦ Cumulative week
 Control set variation by geography
 Target set variation by time
8
Topic matters, time doesn’t.
t
9
Top keywords are very
consistent, particularly
for subjective topics.
10
The correlation decays as we add tweets
from another week back in time.
11
12
For trends relative to the current background
noise, we only need a few weeks of control data.
13
Trends are independent of local background noise
and more dependent on global background noise.
14
Some topics are more similar to background
conversations than others.15
Subjective topics exhibit less vocabulary
change over time.
16
The results of this analysis provide suggestions
for how to build a trend monitoring tool.
These recommendations help define and
understand:
 Frequency of collection
 Short-term vs. long-term collections
 Local collections vs. global collections
 Keyword selection for best results
17
 Subject variation
 Other methods of comparison
 Blogging data
18
19

More Related Content

Similar to Does Love Change on Twitter? The Dynamics of Topical Conversations in Microblogging

Gsdrc Helpdesk Connecting Knowledges
Gsdrc Helpdesk   Connecting KnowledgesGsdrc Helpdesk   Connecting Knowledges
Gsdrc Helpdesk Connecting Knowledges
powerinbetween
 
ReflectiveDispositional Assignment Reflecting on and Documentin.docx
ReflectiveDispositional Assignment Reflecting on and Documentin.docxReflectiveDispositional Assignment Reflecting on and Documentin.docx
ReflectiveDispositional Assignment Reflecting on and Documentin.docx
carlt3
 
Australia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityAustralia's Environmental Predictive Capability
Australia's Environmental Predictive Capability
TERN Australia
 
Mid-Sweden University/SNIA Conference 13 October 2008
Mid-Sweden University/SNIA Conference 13 October 2008Mid-Sweden University/SNIA Conference 13 October 2008
Mid-Sweden University/SNIA Conference 13 October 2008
Mark Conrad
 
PERICLES presentation on Appraisal - IDCC15 workshop
PERICLES presentation on Appraisal - IDCC15 workshopPERICLES presentation on Appraisal - IDCC15 workshop
PERICLES presentation on Appraisal - IDCC15 workshop
PERICLES_FP7
 
Meeting the NSF DMP Requirement: March 7, 2012
Meeting the NSF DMP Requirement: March 7, 2012Meeting the NSF DMP Requirement: March 7, 2012
Meeting the NSF DMP Requirement: March 7, 2012
IUPUI
 
RDA Work Groups Outputs and Adoption - Early WG Report back session
RDA Work Groups Outputs and Adoption - Early WG Report back sessionRDA Work Groups Outputs and Adoption - Early WG Report back session
RDA Work Groups Outputs and Adoption - Early WG Report back session
Research Data Alliance
 
Data matters-bournemouth-2015
Data matters-bournemouth-2015Data matters-bournemouth-2015
Data matters-bournemouth-2015
Alan Dix
 
RDC Jane Fry, Chantal Ripp - Data Interoperability I
RDC Jane Fry, Chantal Ripp - Data Interoperability IRDC Jane Fry, Chantal Ripp - Data Interoperability I
RDC Jane Fry, Chantal Ripp - Data Interoperability I
CASRAI
 
Öppen data och forskningens genomslag
Öppen data och forskningens genomslagÖppen data och forskningens genomslag
Öppen data och forskningens genomslag
Kungliga biblioteket National Library of Sweden
 
Digging into Data Funders Forum
Digging into Data Funders ForumDigging into Data Funders Forum
Digging into Data Funders Forum
Strategic Content Alliance
 
School of Science, Technology, Engineering and MathDep.docx
School of Science, Technology, Engineering and MathDep.docxSchool of Science, Technology, Engineering and MathDep.docx
School of Science, Technology, Engineering and MathDep.docx
anhlodge
 
Open Data is not Enough
Open Data is not EnoughOpen Data is not Enough
Open Data is not Enough
Research Data Alliance
 
Concepts and Challenges of Text Retrieval for Search Engine
Concepts and Challenges of Text Retrieval for Search EngineConcepts and Challenges of Text Retrieval for Search Engine
Concepts and Challenges of Text Retrieval for Search Engine
Gan Keng Hoon
 
From Research to Applications: What Can We Extract with Social Media Sensing?
From Research to Applications: What Can We Extract with Social Media Sensing?From Research to Applications: What Can We Extract with Social Media Sensing?
From Research to Applications: What Can We Extract with Social Media Sensing?
Yiannis Kompatsiaris
 
Meeting the NSF DMP Requirement June 13, 2012
Meeting the NSF DMP Requirement June 13, 2012Meeting the NSF DMP Requirement June 13, 2012
Meeting the NSF DMP Requirement June 13, 2012
IUPUI
 
An Ensemble Model for Cross-Domain Polarity Classification on Twitter
An Ensemble Model for Cross-Domain Polarity Classification on TwitterAn Ensemble Model for Cross-Domain Polarity Classification on Twitter
An Ensemble Model for Cross-Domain Polarity Classification on Twitter
Symeon Papadopoulos
 
Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...
Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...
Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...
Digitalmikkeli
 
Lecture 7B Panel Econometrics I 2011
Lecture 7B Panel Econometrics I 2011Lecture 7B Panel Econometrics I 2011
Lecture 7B Panel Econometrics I 2011
Moses sichei
 
Open Access to Research Data: Challenges and Solutions
Open Access to Research Data: Challenges and SolutionsOpen Access to Research Data: Challenges and Solutions
Open Access to Research Data: Challenges and Solutions
Martin Donnelly
 

Similar to Does Love Change on Twitter? The Dynamics of Topical Conversations in Microblogging (20)

Gsdrc Helpdesk Connecting Knowledges
Gsdrc Helpdesk   Connecting KnowledgesGsdrc Helpdesk   Connecting Knowledges
Gsdrc Helpdesk Connecting Knowledges
 
ReflectiveDispositional Assignment Reflecting on and Documentin.docx
ReflectiveDispositional Assignment Reflecting on and Documentin.docxReflectiveDispositional Assignment Reflecting on and Documentin.docx
ReflectiveDispositional Assignment Reflecting on and Documentin.docx
 
Australia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityAustralia's Environmental Predictive Capability
Australia's Environmental Predictive Capability
 
Mid-Sweden University/SNIA Conference 13 October 2008
Mid-Sweden University/SNIA Conference 13 October 2008Mid-Sweden University/SNIA Conference 13 October 2008
Mid-Sweden University/SNIA Conference 13 October 2008
 
PERICLES presentation on Appraisal - IDCC15 workshop
PERICLES presentation on Appraisal - IDCC15 workshopPERICLES presentation on Appraisal - IDCC15 workshop
PERICLES presentation on Appraisal - IDCC15 workshop
 
Meeting the NSF DMP Requirement: March 7, 2012
Meeting the NSF DMP Requirement: March 7, 2012Meeting the NSF DMP Requirement: March 7, 2012
Meeting the NSF DMP Requirement: March 7, 2012
 
RDA Work Groups Outputs and Adoption - Early WG Report back session
RDA Work Groups Outputs and Adoption - Early WG Report back sessionRDA Work Groups Outputs and Adoption - Early WG Report back session
RDA Work Groups Outputs and Adoption - Early WG Report back session
 
Data matters-bournemouth-2015
Data matters-bournemouth-2015Data matters-bournemouth-2015
Data matters-bournemouth-2015
 
RDC Jane Fry, Chantal Ripp - Data Interoperability I
RDC Jane Fry, Chantal Ripp - Data Interoperability IRDC Jane Fry, Chantal Ripp - Data Interoperability I
RDC Jane Fry, Chantal Ripp - Data Interoperability I
 
Öppen data och forskningens genomslag
Öppen data och forskningens genomslagÖppen data och forskningens genomslag
Öppen data och forskningens genomslag
 
Digging into Data Funders Forum
Digging into Data Funders ForumDigging into Data Funders Forum
Digging into Data Funders Forum
 
School of Science, Technology, Engineering and MathDep.docx
School of Science, Technology, Engineering and MathDep.docxSchool of Science, Technology, Engineering and MathDep.docx
School of Science, Technology, Engineering and MathDep.docx
 
Open Data is not Enough
Open Data is not EnoughOpen Data is not Enough
Open Data is not Enough
 
Concepts and Challenges of Text Retrieval for Search Engine
Concepts and Challenges of Text Retrieval for Search EngineConcepts and Challenges of Text Retrieval for Search Engine
Concepts and Challenges of Text Retrieval for Search Engine
 
From Research to Applications: What Can We Extract with Social Media Sensing?
From Research to Applications: What Can We Extract with Social Media Sensing?From Research to Applications: What Can We Extract with Social Media Sensing?
From Research to Applications: What Can We Extract with Social Media Sensing?
 
Meeting the NSF DMP Requirement June 13, 2012
Meeting the NSF DMP Requirement June 13, 2012Meeting the NSF DMP Requirement June 13, 2012
Meeting the NSF DMP Requirement June 13, 2012
 
An Ensemble Model for Cross-Domain Polarity Classification on Twitter
An Ensemble Model for Cross-Domain Polarity Classification on TwitterAn Ensemble Model for Cross-Domain Polarity Classification on Twitter
An Ensemble Model for Cross-Domain Polarity Classification on Twitter
 
Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...
Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...
Datajalostamo-seminaari 5.6.2014: Tutkimusdatan avoimuus – globaalit tutkimus...
 
Lecture 7B Panel Econometrics I 2011
Lecture 7B Panel Econometrics I 2011Lecture 7B Panel Econometrics I 2011
Lecture 7B Panel Econometrics I 2011
 
Open Access to Research Data: Challenges and Solutions
Open Access to Research Data: Challenges and SolutionsOpen Access to Research Data: Challenges and Solutions
Open Access to Research Data: Challenges and Solutions
 

Recently uploaded

Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Operating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptxOperating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptx
Pravash Chandra Das
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
flufftailshop
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 

Recently uploaded (20)

Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Operating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptxOperating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptx
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 

Does Love Change on Twitter? The Dynamics of Topical Conversations in Microblogging

  • 1. The Dynamics of Topical Conversations in Microblogging Victoria Lai and William Rand SocialCom 2013 September 11, 2013
  • 2.  Keyword and trend identification  Background noise  Importance of time, subject, geography? 2
  • 3.  Twitter’s trending topics  Real-time trend identification ◦ Mathioudakis and Koudas (2010) ◦ Cataldi, Di Caro, and Schifanella (2010)  Other trend detection methods ◦ Benhardus and Kalita (2013)  Trends in a geographic area ◦ Wilkinson and Thelwall (2012) 3
  • 4.  Daily Twitter REST API queries, 47 weeks  Topics ◦ 11 keywords/phrases chosen for time, subject, and geographic variety (baseball, economy, global warming, love, London riots) ◦ 5 control (I, the, and, a, of)  Geography ◦ Global ◦ 9 cities (Boston, D.C., London, Sydney) 4
  • 5.  Term frequency  Inverse document frequency  TF-IDF
  • 6.  Keyword lists as target/control sets vary  Time, subject, geography  Tf-idf and rank correlation 𝑓𝑓 𝑇𝑇𝑡𝑡∗, 𝑠𝑠∗, 𝑔𝑔∗ 𝐶𝐶𝑡𝑡, 𝑠𝑠, 𝑔𝑔 ) 6
  • 7. … tf phrase 1 # phrase 2 # phrase 3 #C0, ∀sc, ∀g C1, ∀sc, ∀g C2, ∀sc, ∀g C3, ∀sc, ∀g idf phrase 1 # phrase 2 # phrase 3 # phrase 1 # phrase 2 # phrase 3 # … T0, s*, ∀g * SC = {I, the, and, a, of }
  • 8.  Control set variation by time ◦ Single week ◦ Cumulative week  Control set variation by geography  Target set variation by time 8
  • 9. Topic matters, time doesn’t. t 9
  • 10. Top keywords are very consistent, particularly for subjective topics. 10
  • 11. The correlation decays as we add tweets from another week back in time. 11
  • 12. 12
  • 13. For trends relative to the current background noise, we only need a few weeks of control data. 13
  • 14. Trends are independent of local background noise and more dependent on global background noise. 14
  • 15. Some topics are more similar to background conversations than others.15
  • 16. Subjective topics exhibit less vocabulary change over time. 16
  • 17. The results of this analysis provide suggestions for how to build a trend monitoring tool. These recommendations help define and understand:  Frequency of collection  Short-term vs. long-term collections  Local collections vs. global collections  Keyword selection for best results 17
  • 18.  Subject variation  Other methods of comparison  Blogging data 18
  • 19. 19