SlideShare a Scribd company logo
Validating search protocols
for mining of health and disease events
on Twitter
Aditya Lia Ramadona1,2*, Lutfan Lazuardi3, Sulistyawati1,4,
Anwar Dwi Cahyono5, Åsa Holmner6, Hari Kusnanto3, Joacim Rocklöv1
The International Conference on Public Health (ICPH)
Solo, Indonesia; September 14-15, 2016
https://arxiv.org/abs/1608.05910
Introduction Twitter
• free social networking and micro-
blogging service
• 140-character: news, events,
personal feeling and experiences, …
• May 2016: 24.34 million Indonesian
active users ~ 10% (Statista, 2016)
Twitter offers streams of the public
data flowing
• might contain health-related
information
• can be explored for public health
monitoring and surveillance
purposes (Paul et al. 2016)
Indonesia Social Media Trend (Jakpat, 2016)
Introduction
Previous studies
• Signorini et al. 2011: track levels of disease activity
• Eichstaedt et al. 2015: predicts heart disease mortality
• Strom et al. 2013: measuring health-related quality of life
• many more…
Methodological challenges
• data and language processing
• model development
www.bahasakita.com
Subjects and Methods
Develop groups of words and phrases relevant to disease symptoms
and health outcomes in the Bahasa Indonesia
historical Twitter
Twitter stream14d
real-time
Subjects and Methods
Sentiment analysis
• examining a tweet from Twitter feeds
• the decisions were made by people with expert knowledge
millions of tweets: time-consuming and inefficient
Replicating expert assessment
• develop a model, interpret results and adjust the model
• make predictions
Results: text analysis
Historical Twitter feeds: 390 tweets
• "rumah OR sakit OR rawat OR inap OR demam OR panas -cuaca OR berdarah
OR pendarahan OR tombosit OR badan OR muntah OR badan OR tua OR ':('"
Preprocessing
• removing retweets and eliminate some noise
• removing punctuation, numbers, capitalization, and the Bahasa stop-words
(e.g. kamu and aja)
[107] "@XYZ kamu izin aja, bilang kamu sakit :(("
[107] "xyz izin bilang sakit"
Results: text analysis
1,632 words
• the most highly correlate
words: sakit (sick, ill, pain)
hati (0.23) ~ shame, broken heart, …
rasa (0.13) ~ pain
perut (0.12) ~ stomach ache
Figure 1. Words that appear more than 10 times
Results: model development
Predictors
• highest words frequencies (22)
• counting the number of the predictor words in a tweet
Classification and Regression Trees model
(Breiman et al. 1983)
• rpart package (Therneau et al. 2015)
Results: model development
390 tweets
historical Twitter feeds
• 273 tweets (70%): training
• 117 tweets (30%): validating
1,145,649 tweets
Twitter stream feeds: testing
Indonesia: between 11°S and 6°N and 95°E and 141°E,
7 days: 26th July – 1st August 2016
• 100 from 6,109 TRUE results
• 100 from 1,139,540 FALSE result
Results: model development
Results
Results
Model Performance Validation Testing
AUC 0.82 0.70
Sensitivity 80.0 42.0
Specifity 84.6 98.0
Positive Predictive Value 86.7 95.5
Negative Predictive Value 77.2 62.8
Limitations + Challenges = Future Work
team member involved
• academics, health workers
Twitter users
• telecommunications infrastructure
• characteristics of people
methods
• data: streaming (Indonesia, 7d/24h ~ 1.5GB in csv format)
• model: CART, RandomForest, GBM, …
Summary
Monitoring of public sentiment on Twitter + contextual knowledge
• a nearly real-time proxy for health-related indicators
Models do not replace expert judgment
• accurately analyze small amounts of information (tweets)
• improve and refine the model
• bias and emotion: integrate assessments of many experts
Summary
1 Department of Public Health and Clinical Medicine, Epidemiology and Global Health, Umeå University
2 Center for Environmental Studies, Universitas Gadjah Mada
3 Department of Public Health, Faculty of Medicine, Universitas Gadjah Mada
4 Department of Public Health, Universitas Ahmad Dahlan
5 District Health Office, Yogyakarta
6 Department of Radiation Sciences, Umeå University
*alramadona@ugm.ac.id
www.themexpert.com/images/easyblog_articles/270/twitter_cover.jpg

More Related Content

Similar to Validating search protocols for mining of health and disease events on Twitter

Introduction to IHME
Introduction to IHMEIntroduction to IHME
maternal mortality sri lanka intro_murray_11012010_ihme
maternal mortality sri lanka intro_murray_11012010_ihmematernal mortality sri lanka intro_murray_11012010_ihme
maternal mortality sri lanka intro_murray_11012010_ihme
Institute for Health Metrics and Evaluation - University of Washington
 
Electronic cigarettes for smoking cessation: What's the evidence?
Electronic cigarettes for smoking cessation: What's the evidence?Electronic cigarettes for smoking cessation: What's the evidence?
Electronic cigarettes for smoking cessation: What's the evidence?
Health Evidence™
 
Social Media Research and Practice in the Health Domain - Tutorial, Part II
Social Media Research and Practice in the Health Domain - Tutorial, Part IISocial Media Research and Practice in the Health Domain - Tutorial, Part II
Social Media Research and Practice in the Health Domain - Tutorial, Part II
Ingmar Weber
 
Ppr r101-session-3-smart
Ppr r101-session-3-smartPpr r101-session-3-smart
Ppr r101-session-3-smart
SharmiBhat
 
OPERATIONAL RESEARCH (OR)
OPERATIONAL RESEARCH (OR)OPERATIONAL RESEARCH (OR)
OPERATIONAL RESEARCH (OR)
Bikash Debbarma
 
Interventions with potential to reduce sedentary time in adults: What's the e...
Interventions with potential to reduce sedentary time in adults: What's the e...Interventions with potential to reduce sedentary time in adults: What's the e...
Interventions with potential to reduce sedentary time in adults: What's the e...
Health Evidence™
 
Leveraging Text Classification Strategies for Clinical and Public Health Appl...
Leveraging Text Classification Strategies for Clinical and Public Health Appl...Leveraging Text Classification Strategies for Clinical and Public Health Appl...
Leveraging Text Classification Strategies for Clinical and Public Health Appl...
Karin Verspoor
 
Access to primary health care DA 20.10.2015
Access to primary health care DA 20.10.2015Access to primary health care DA 20.10.2015
Access to primary health care DA 20.10.2015
Deena Alasfoor
 
Co-Creating Health through Digital Technology
Co-Creating Health through Digital TechnologyCo-Creating Health through Digital Technology
Co-Creating Health through Digital Technology
Sarah Amani
 
Future of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive TrendsFuture of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive Trends
Sean Koon, MD, MS
 
Project ECHO QI: Communicating and Advocating Using Data June 29, 2016
Project ECHO QI: Communicating and Advocating Using Data June 29, 2016Project ECHO QI: Communicating and Advocating Using Data June 29, 2016
Project ECHO QI: Communicating and Advocating Using Data June 29, 2016
CHC Connecticut
 
Introduction to Health Research
Introduction to Health ResearchIntroduction to Health Research
Introduction to Health Research
Dr Ghaiath Hussein
 
Shu ti (sintesi)
Shu ti (sintesi)Shu ti (sintesi)
Shu ti (sintesi)
Carlo Favaretti
 
Early benefits and impacts of EPR implementation
Early benefits and impacts of EPR implementationEarly benefits and impacts of EPR implementation
Early benefits and impacts of EPR implementation
Health Informatics New Zealand
 
Facilitating cross-talk in mHealth intervention development
Facilitating cross-talk in mHealth intervention developmentFacilitating cross-talk in mHealth intervention development
Facilitating cross-talk in mHealth intervention development
Megan Ranney
 
Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...
Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...
Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...
Patient-Centered Outcomes Research Institute
 
Jokers to the left. clowns to the right
Jokers to the left. clowns to the rightJokers to the left. clowns to the right
Jokers to the left. clowns to the right
Shane Fast
 
F3 patient engagement-2015philadelphia
F3 patient engagement-2015philadelphiaF3 patient engagement-2015philadelphia
F3 patient engagement-2015philadelphia
Todd Berner MD
 
Engaging Patients in Research: Does algorithmically created content have a ro...
Engaging Patients in Research: Does algorithmically created content have a ro...Engaging Patients in Research: Does algorithmically created content have a ro...
Engaging Patients in Research: Does algorithmically created content have a ro...
Katja Reuter, PhD
 

Similar to Validating search protocols for mining of health and disease events on Twitter (20)

Introduction to IHME
Introduction to IHMEIntroduction to IHME
Introduction to IHME
 
maternal mortality sri lanka intro_murray_11012010_ihme
maternal mortality sri lanka intro_murray_11012010_ihmematernal mortality sri lanka intro_murray_11012010_ihme
maternal mortality sri lanka intro_murray_11012010_ihme
 
Electronic cigarettes for smoking cessation: What's the evidence?
Electronic cigarettes for smoking cessation: What's the evidence?Electronic cigarettes for smoking cessation: What's the evidence?
Electronic cigarettes for smoking cessation: What's the evidence?
 
Social Media Research and Practice in the Health Domain - Tutorial, Part II
Social Media Research and Practice in the Health Domain - Tutorial, Part IISocial Media Research and Practice in the Health Domain - Tutorial, Part II
Social Media Research and Practice in the Health Domain - Tutorial, Part II
 
Ppr r101-session-3-smart
Ppr r101-session-3-smartPpr r101-session-3-smart
Ppr r101-session-3-smart
 
OPERATIONAL RESEARCH (OR)
OPERATIONAL RESEARCH (OR)OPERATIONAL RESEARCH (OR)
OPERATIONAL RESEARCH (OR)
 
Interventions with potential to reduce sedentary time in adults: What's the e...
Interventions with potential to reduce sedentary time in adults: What's the e...Interventions with potential to reduce sedentary time in adults: What's the e...
Interventions with potential to reduce sedentary time in adults: What's the e...
 
Leveraging Text Classification Strategies for Clinical and Public Health Appl...
Leveraging Text Classification Strategies for Clinical and Public Health Appl...Leveraging Text Classification Strategies for Clinical and Public Health Appl...
Leveraging Text Classification Strategies for Clinical and Public Health Appl...
 
Access to primary health care DA 20.10.2015
Access to primary health care DA 20.10.2015Access to primary health care DA 20.10.2015
Access to primary health care DA 20.10.2015
 
Co-Creating Health through Digital Technology
Co-Creating Health through Digital TechnologyCo-Creating Health through Digital Technology
Co-Creating Health through Digital Technology
 
Future of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive TrendsFuture of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive Trends
 
Project ECHO QI: Communicating and Advocating Using Data June 29, 2016
Project ECHO QI: Communicating and Advocating Using Data June 29, 2016Project ECHO QI: Communicating and Advocating Using Data June 29, 2016
Project ECHO QI: Communicating and Advocating Using Data June 29, 2016
 
Introduction to Health Research
Introduction to Health ResearchIntroduction to Health Research
Introduction to Health Research
 
Shu ti (sintesi)
Shu ti (sintesi)Shu ti (sintesi)
Shu ti (sintesi)
 
Early benefits and impacts of EPR implementation
Early benefits and impacts of EPR implementationEarly benefits and impacts of EPR implementation
Early benefits and impacts of EPR implementation
 
Facilitating cross-talk in mHealth intervention development
Facilitating cross-talk in mHealth intervention developmentFacilitating cross-talk in mHealth intervention development
Facilitating cross-talk in mHealth intervention development
 
Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...
Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...
Seeking Input on Future PROMIS® Research: Educating Patients and Stakeholders...
 
Jokers to the left. clowns to the right
Jokers to the left. clowns to the rightJokers to the left. clowns to the right
Jokers to the left. clowns to the right
 
F3 patient engagement-2015philadelphia
F3 patient engagement-2015philadelphiaF3 patient engagement-2015philadelphia
F3 patient engagement-2015philadelphia
 
Engaging Patients in Research: Does algorithmically created content have a ro...
Engaging Patients in Research: Does algorithmically created content have a ro...Engaging Patients in Research: Does algorithmically created content have a ro...
Engaging Patients in Research: Does algorithmically created content have a ro...
 

Recently uploaded

一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
dwreak4tg
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
rwarrenll
 
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
zsjl4mimo
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
roli9797
 
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfEnhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
GetInData
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
Bill641377
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
Natural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptxNatural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptx
fkyes25
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
g4dpvqap0
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
slg6lamcq
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
74nqk8xf
 
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
u86oixdj
 
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
74nqk8xf
 
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
mzpolocfi
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
Sachin Paul
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
soxrziqu
 

Recently uploaded (20)

一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
 
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(Harvard毕业证书)哈佛大学毕业证如何办理
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
 
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfEnhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
Natural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptxNatural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptx
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
 
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
原版制作(swinburne毕业证书)斯威本科技大学毕业证毕业完成信一模一样
 
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
 
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
 

Validating search protocols for mining of health and disease events on Twitter

  • 1. Validating search protocols for mining of health and disease events on Twitter Aditya Lia Ramadona1,2*, Lutfan Lazuardi3, Sulistyawati1,4, Anwar Dwi Cahyono5, Åsa Holmner6, Hari Kusnanto3, Joacim Rocklöv1 The International Conference on Public Health (ICPH) Solo, Indonesia; September 14-15, 2016 https://arxiv.org/abs/1608.05910
  • 2.
  • 3. Introduction Twitter • free social networking and micro- blogging service • 140-character: news, events, personal feeling and experiences, … • May 2016: 24.34 million Indonesian active users ~ 10% (Statista, 2016) Twitter offers streams of the public data flowing • might contain health-related information • can be explored for public health monitoring and surveillance purposes (Paul et al. 2016) Indonesia Social Media Trend (Jakpat, 2016)
  • 4. Introduction Previous studies • Signorini et al. 2011: track levels of disease activity • Eichstaedt et al. 2015: predicts heart disease mortality • Strom et al. 2013: measuring health-related quality of life • many more… Methodological challenges • data and language processing • model development www.bahasakita.com
  • 5. Subjects and Methods Develop groups of words and phrases relevant to disease symptoms and health outcomes in the Bahasa Indonesia historical Twitter Twitter stream14d real-time
  • 6. Subjects and Methods Sentiment analysis • examining a tweet from Twitter feeds • the decisions were made by people with expert knowledge millions of tweets: time-consuming and inefficient Replicating expert assessment • develop a model, interpret results and adjust the model • make predictions
  • 7. Results: text analysis Historical Twitter feeds: 390 tweets • "rumah OR sakit OR rawat OR inap OR demam OR panas -cuaca OR berdarah OR pendarahan OR tombosit OR badan OR muntah OR badan OR tua OR ':('" Preprocessing • removing retweets and eliminate some noise • removing punctuation, numbers, capitalization, and the Bahasa stop-words (e.g. kamu and aja) [107] "@XYZ kamu izin aja, bilang kamu sakit :((" [107] "xyz izin bilang sakit"
  • 8. Results: text analysis 1,632 words • the most highly correlate words: sakit (sick, ill, pain) hati (0.23) ~ shame, broken heart, … rasa (0.13) ~ pain perut (0.12) ~ stomach ache Figure 1. Words that appear more than 10 times
  • 9. Results: model development Predictors • highest words frequencies (22) • counting the number of the predictor words in a tweet Classification and Regression Trees model (Breiman et al. 1983) • rpart package (Therneau et al. 2015)
  • 10. Results: model development 390 tweets historical Twitter feeds • 273 tweets (70%): training • 117 tweets (30%): validating 1,145,649 tweets Twitter stream feeds: testing Indonesia: between 11°S and 6°N and 95°E and 141°E, 7 days: 26th July – 1st August 2016 • 100 from 6,109 TRUE results • 100 from 1,139,540 FALSE result
  • 13. Results Model Performance Validation Testing AUC 0.82 0.70 Sensitivity 80.0 42.0 Specifity 84.6 98.0 Positive Predictive Value 86.7 95.5 Negative Predictive Value 77.2 62.8
  • 14. Limitations + Challenges = Future Work team member involved • academics, health workers Twitter users • telecommunications infrastructure • characteristics of people methods • data: streaming (Indonesia, 7d/24h ~ 1.5GB in csv format) • model: CART, RandomForest, GBM, …
  • 15. Summary Monitoring of public sentiment on Twitter + contextual knowledge • a nearly real-time proxy for health-related indicators Models do not replace expert judgment • accurately analyze small amounts of information (tweets) • improve and refine the model • bias and emotion: integrate assessments of many experts
  • 17. 1 Department of Public Health and Clinical Medicine, Epidemiology and Global Health, Umeå University 2 Center for Environmental Studies, Universitas Gadjah Mada 3 Department of Public Health, Faculty of Medicine, Universitas Gadjah Mada 4 Department of Public Health, Universitas Ahmad Dahlan 5 District Health Office, Yogyakarta 6 Department of Radiation Sciences, Umeå University *alramadona@ugm.ac.id www.themexpert.com/images/easyblog_articles/270/twitter_cover.jpg