Summarizing Situational Tweets in Crisis Scenario
Report
Muhammad ImranFollow
Aug. 16, 2016•0 likes•595 views
Summarizing Situational Tweets in Crisis Scenario
Aug. 16, 2016•0 likes•595 views
Download to read offline
Report
Data & Analytics
During mass convergence events such as natural disasters, microblogging platforms like Twitter are widely used by affected people to post situational awareness messages. These crisis related messages disperse among multiple categories like infrastructure damage, information about missing, injured, and dead people etc. The challenge here is to extract important situational updates from these messages, assign them appropriate informational categories, and finally summarize big trove of information in each category. In this paper, we propose a novel framework which first assigns tweets into different situational classes and then summarize those tweets. In the summarization phase, we propose a two stage summarization framework which first extracts a set of important tweets from the whole set of information through an Integer-linear programming (ILP) based optimization technique and then follows a word graph and content word based abstractive summarization technique to produce the final summary. Our method is time and memory efficient and outperforms the baseline in terms of quality, coverage of events, locations et al., effectiveness, and utility in disaster scenarios.
Muhammad ImranFollow
More Related Content
Viewers also liked
#Membership renewal new logopatcalhoun
ISCRAM 2013: Extracting Information Nuggets from Disaster-Related Messages i...ISCRAM Events
01.사람의 이해와 UX/UI(시스템과 유저 인터페이스)Billy Choi
Laundry Management Systemcetrosoft
영화 미디어에 투영된 미래 분석 및 미래가전 시사점 퍼셉션PERCEPTION
Concepto e importancia de la mercadotecniaNombre Apellidos
Similar to Summarizing Situational Tweets in Crisis Scenario
Processing Social Media Messages in Mass Emergency: A SurveyMuhammad Imran
OCHA Think Brief - Hashtag Standards for emergenciesJan Husar
Data Analytics and Industry-Academic Partnerships: An Irish PerspectiveJohn Breslin
Open analytics social media frameworkOpen Analytics
Twitter as a personalizable information service iiKan-Han (John) Lu
Using Twitter as a Postgraduate ResearcherSimon Bishop
More from Muhammad Imran
Damage Assessment from Social Media Imagery Data During DisastersMuhammad Imran
Image4Act: Online Social Media Image Processing for Disaster ResponseMuhammad Imran
Real-Time Processing of Social Media Content for Social GoodMuhammad Imran
AIDR Tutorial (Artificial Intelligence for Disaster Response)Muhammad Imran
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...Muhammad Imran
The Role of Social Media and Artificial Intelligence for Disaster ResponseMuhammad Imran
Recently uploaded
Colaboration Eco-system MNC Bank.pptxrezairsyadul
Dunning - SIGMOD - Data Economy.pptxTed Dunning
Data collection.pdfMuthuLakshmi124949
Data Warehouse with Fabric on data lakehouseMarco Pozzan
International Observe the Moon Night 2023VICTOR MAESTRE RAMIREZ
Essential numpy before you start your Machine Learning journey in python.pdfSmrati Kumar Katiyar
Summarizing Situational Tweets in Crisis Scenario
- 1. Summarizing Situational Tweets in Crisis Scenario Imran Muhammad Qatar Computing Research Institute, HBKU Doha, Qatar Co-authors: Koustav Rudra, Niloy Ganguly, Pawan Goyal (CSE, IIT Kharagpur, India) Siddhartha Banerjee (Pennsylvania State University, USA) Prasenjit Mitra (Qatar Computing Research Institute)
- 2. Aid Needs and Information Needs Info. Info. Info. Disaster event Urgent needs of affected people Information gathering Humanitarian organizations and local administration Information gathering, especially in real-time, is the most challenging part Relief operations - Food, water - Shelter - Medical emergences - Donations - …
- 3. Information: A Lifeline During Disasters The opaqueness induced by disasters is overwhelming People need information as much as water, food, medicine or shelter Lack of information can make people victims of disaster and targets of aid
- 4. Twitter: A Useful Information Source • Provide active communication channels during crises • Useful information: reports of casualties, damages, donation offers and requests • Quicker than traditional channels (e.g. first tweet about Westgate Mall attack reported within a minute)
- 5. Opportunities • Rapid crisis response • Time-critical situational awareness • Access to actionable information • … • But, it requires real-time data processing • Categorizations of each incoming item should be done as soon as it arrives • Rapid automatic summaries generation
- 6. Enhanced Situational Awareness Time-critical situational awareness by generating automatic summaries • We use AIDR (Artificial Intelligence for Disaster Response) system for: – real-time data processing – categorizations of tweets • We proposed a novel framework for summarization of informative tweets
- 7. Summarization of Tweets Example Dharara Tower built in 1832 collapses in Kathmandu during earthquake Historic Dharara Tower Collapses in Kathmandu After 7.9 Earthquake Dharara tower built in 1832 collapses in Kathmandu after 7.9 earthquake.
- 8. Key Characteristics and Objectives • Information coverage – Capture most situational updates from data. The summary should be rich in terms of information coverage • Less redundant information – Messages on Twitter contain duplicate information. We aim for summaries with less redundancy while keeping important updates • Readability – Twitter messages are often noisy, informal, and full of grammatical mistakes. We aim to produce more readable summaries • Real-time – The system should not be heavily overloaded with computations such that by the time the summary is produced, the utility of that information is marginal
- 9. High-level Approach Automatic Classification and Summarization
- 10. Datasets • Nepal earthquake tweets from 25th to 27th April 2015 • AIDR classified tweets to the following categories: – Missing trapped or found people (10,751 tweets) – Infrastructure and utilities damage (16,842 tweets) – Shelter and supplies (19,006 tweets)
- 11. The Role of Content Words in Extractive Summarization • Studies show the significance of content words to capture important events – Nouns (e.g. hospitals, buildings, bridges names) – Numerals (e.g. number of casualties) – Main verbs (e.g. collapsed, destroyed, killed)
- 12. Abstractive Summarization • We generate a word graph where nodes are bigrams • We generate sentences from the word graph Challenge: Maintaining informativeness and readability – Covering important content words – Favoring more informative paths – Maintaining linguistic quality ILP model combining the above factors
- 13. Bi-gram Based Word Graph • Word graph: nodes represent bi-grams (along with their POS-tags) • An edge represents consecutive words • Nodes of two tweets with same bi-gram and POS- tags are merged
- 14. ILP Based Formulation Parameters • Score of sentences/generated paths (CW(s)) – Centroid score • Linguistic quality(LQ(s)) – Trigram language model – LQ(s) = 1/(1-ll(w1,w2,…,wq)) – ll(w1,w2,…,wq) = 1/Llog2∏q t =3 P(wt|wt-2wt-1)
- 15. ILP Based Solution max(∑i=1…n CW(i)*LQ(i)*xi + ∑j=1…m yj) xi, yj binary variable xi tweet indicator, yj content word indicator CW(i) = tweet i centroid score LQ(i) = Linguistic score of tweet i Constraints ∑i=1…n xi * Length(i) ≤ L Length(i) = number of words in tweet i L = required summary word length ∑i ∈ Tj xi ≥ yj j = [1 … m] Tj = set of tweets where content word j is present If yj is selected then at least one tweet covering that word is also selected ∑j ∈ Ci yj ≥|Ci| * xi i = [1 … n] Ci = set of content words present in tweet i If tweet i is selected then all the content words of that tweet are also selected
- 16. Baselines and Gold Standard • COWTS: content word based extractive tweet stream summarization algorithm [CIKM 2015] • APSAL: affinity clustering based supervised disaster related news articles summarizer [ACL 2015] • TOWGS: runtime bigram based abstractive summarization [EACL 2014] Gold Standard: We used summaries generated by UN OCHA experts during Nepal earthquake
- 17. Summarization Results . If number of clusters increases, determining importance of different clusters becomes difficult [APSAL] COWTS tries to maximize the coverage of content words but can’t combine information from related tweets TOWGS didn’t consider content words into account COWABS tries to combine similar information from related tweets as well as maximizing coverage of content words
- 18. Performance Variation with Summary Length If summary length increases COWABS still performs better than the baselines
- 19. Performance Comparison (User Studies)
- 20. Performance Comparison (User Studies)
- 21. Summarization Quality (Location) COWABS captures information at more granular level with location specific information
- 22. Summarization Quality (Event) We extract event phrases using the method proposed by Ritter et al [EMNLP 2011] COWABS captures more event specific information
- 23. Summarization Quality (Numeral) COWABS captures more numerical information which includes information about victims, helpline numbers etc.
- 24. Conclusions • Rapid situational awareness is necessary for effective relief operations • Twitter as useful information source during emergencies • Automatic classification and summarization approach • Propose approach outperforms all baselines and deemed effective –learned from user studies
- 25. Thank you! Imran Muhammad Twitter: @mimran15
Editor's Notes
- For an effective planning, we need as complete information as possible. Looking back over the events of 2004 and before, it is striking how many of the year’s disasters, deaths, medical emergencies etc. could have been avoided with better information and communication with the affected people. The opaqueness induced by disasters is overwhelming. Information could be a lifeline for everybody involved in a disasters situation. People need information as much as water, food, medicine or shelter. Lack of information make people victims of disasters and targets of aid. http://www.ifrc.org/Global/Publications/disasters/WDR/69001-WDR2005-english-LR.pdf
- A number of research studies reveal that social media platforms such as twitter can be very useful sources of information during disasters. For example, Twitter provide active communciation channels during crises. Affected people post sitatuational awareness and actionable information. Studies have seen reports of casualities, damages, donations offers and requests… We have also seen that Twitter for example, can be quicker than traditional channels in terms of reporting first hand information
- Social media data can be useful for rapid crisis response But, it requires real-time data analysis So the categorizations of each incoming item into different user-defined classes should be as soon as they For example, infrastructure damage, urgent needs
- Social media data can be useful for rapid crisis response But, it requires real-time data analysis So the categorizations of each incoming item into different user-defined classes should be as soon as they For example, infrastructure damage, urgent needs
- We use a trigram-based log-likelihood score using a language model as a dummy representative of the Readability of the generated content.
- For extractive summarization, we mainly focus on content words. For example, nouns, numerals, main verbs Studies show that content words play significant role to capture important events. Now the question is, how much of the content words we can capture after processing N number of words. For this purpose, we randomly sample same number of tweets for each of the three classes for Nepal EQ and other two events black-Friday and Chelsea match. It was observed that number of content words are very small in case of disaster events. BLF = blackfriday CHL = chelsea match Distinct content words posted during disaster are very less in number compared to generic events Within 1000 words limit, we can cover more than 80% of the content words in a disaster event.
- Challenges in abstractive summarization
- Given this graph, in a real-scenario, the number of generated tweet-paths can be several thousands, because there can be multiple points of merging across several tweets. Our goal is to generate sentences by selecting the best paths from these generated paths with the objective of generating a readable and informative summary. We set a minimum (10 words) and maximum (16 words) length for a sentence to be generated. We apply such constraints to avoid very long sentences that might be grammatically ill-formed and very short sentences that are often incomplete.
- The ILP-based technique optimizes based upon three factors - (i) Presence of content words (ii) Informativeness of a path i.e. importance of a path, and (iii) Linguistic Quality Score that captures the readability of a path using a trigram confidence score. Each sentence is represented as a TF-IDF vector. The centroid is basically the mean of the TF-IDF vectors of all the sentences.
- Optimization problem and constraints An integer programming problem is a mathematical optimization program in which some or all of the variables are restricted to be integers and the objective function and the constraints (other than the integer constraints) are linear. Integer programming is NP-hard.
- Comparison of different methods and their drawbacks, our improvements
- Comparison with baselines when summary length increases