When crises hit, many flog to social media to share or consume information related to the event. Social media posts during crises tend to provide valuable reports on affected people, donation offers, help requests, advice provision, etc. Automatically identifying the category of information (e.g., reports on affected individuals, donations and volunteers) contained in these posts is vital for their efficient handling and consumption by effected communities and concerned organisations. In this paper, we introduce Sem-CNN; a wide and deep Convolutional Neural Network (CNN) model designed for identifying the category of information contained in crisis-related social media content. Unlike previous models, which mainly rely on the lexical representations of words in the text, the proposed model integrates an additional layer of semantics that represents the named entities in the text, into a wide and deep CNN network. Results show that the Sem-CNN model consistently outperforms the baselines which consist of statistical and non-semantic deep learning models. Paper access: http://oro.open.ac.uk/51726/

1. GRÉGOIRE BUREL, HASSAN SAIF, HARITH ALANI
Knowledge Media Institute, The Open University, Milton Keynes, UK.
ISWC’17, Vienna, Austria.
21-25 October 2017.
Semantic Wide and Deep Learning for
Detecting Crisis-Information
Categories on Social Media

2. Event Detection and Crisis Situations
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
2
Event detection is “the task of automatically
identifying certain clues in texts that denote a
specific event type or theme”.
- Help identifying/responding to events.
- Organise relevant information during
crises.
Twitter:
~200 million active users.
~400 million tweets a day.
Twitter usage during crises:
1. During the 2011 Japan earthquake, 177
million tweets related to the event were
sent in one day.
2. The news about the Boston bombings
first appeared on Twitter.

3. Crisis-Related Event Detection Tasks
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
Publications
3
Crisis-related event detection is often divided into three main tasks (Olteanu et al.
2015):
Crisis Related /
Unrelated
Crisis
Type
Information
Categories
Task 1
Identify the
different types
of crises the
message is
related to.
Differentiate the
type of information
contained in the
message.
e.g., shooting,
explosion, building
collapse, fires, floods,
meteorite fall, etc.
e.g., affected individuals,
infrastructures and
utilities, donations and
volunteer, caution and
advice, etc.
Granularity
Differentiate the
posts that are
related or unrelated
to crises.
Task 2 Task 3

4. Crisis-Related Event Detection Tasks
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
Publications
4
Crisis-related event detection is often divided into three main tasks (Olteanu et al.
2015):
Crisis Related /
Unrelated
Crisis
Type
Information
Categories
Task 1
Identify the
different types
of crises the
message is
related to.
Differentiate the
type of information
contained in the
message.
e.g., shooting,
explosion, building
collapse, fires, floods,
meteorite fall, etc.
e.g., affected individuals,
infrastructures and
utilities, donations and
volunteer, caution and
advice, etc.
Granularity
Differentiate the
posts that are
related or unrelated
to crises.
Task 2 Task 3

5. ‘Traditional’ ML vs. Deep Learning
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
5
Deep Learning
- Artificial neural networks.
- Minimum feature engineering
- Word embeddings (Bengio et
al., 2013).
‘Traditional’ ML
- Standard classifiers (e.g., SVM,
J48…).
- Feature engineering (e.g.,
lemmatisation, TF-IDF…).
- Bag of words.

6. Text vs. Semantics – Document Contextualisation
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
6
Obama attends vigil for Boston Marathon bombing victims
Politician /
Person
Sports Event / Social
Event / Event
Disaster / Event
Incorporating Semantics into ML Classification Methods for
contextualising documents:
- Approach 1: Traditional ML Classifiers
- Approach 2: Deep Learning

7. CNN for Sentence Classification (Kim et al., 2014)
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
7

8. Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
8
CNN for Sentence Classification Dual-CNN (Semantic Channel)
CNN for Sentence Classification (Kim et al., 2014)
+ Competitive results for text classification tasks.
+ No or Little Feature Engineering required.
+ Relatively good at taking local textual relations
within short documents.
- No ‘native’ semantic context.
Dual-CNN (Burel et al., 2017)
+ Text CNN
+ Aligned Semantic channel
- Concept extraction.
- Semantics vocabulary (4000) <<
Words vocabulary (60000)

9. Wide and Deep Learning (Cheng et al., 2016)
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
9

10. Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
10
Wide and Deep Learning Sem-CNN (W-D-CNN)
Wide and Deep Learning (Cheng et al., 2016)
+ Efficiently Deal with ‘sparse’ and ‘dense’
inputs.
- Not very efficient for modelling text relations.
- No ‘native’ semantic context.
Sem-CNN (W-D-CNN)
+ Text CNN / Wide and Deep Models
+ Deep Shallow Word Embeddings
+ Wide Deep Semantics
- Requires semantic extraction.

11. Wide and Deep Semantic CNN (Sem-CNN)
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
11

12. Sem-CNN – Experimental Setup
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
Dataset - T26 (28,000 annotated tweets)
- 12 Crisis types (shooting, explosion, building collapse, fires, floods,
meteorite fall, haze, bombing, typhoon, crash, earthquake, and
derailment).
- 6 Information categories (affected individuals, infrastructures and
utilities, donations and volunteer, caution and advice, sympathy and
emotional support, and other useful information)
Semantic Extraction -
- Extracted Entities/Concepts: 65% dataset coverage.
Concept Vectors Initialisation
- Concept Labels: Obama → dbo:Obama
- Concept Abstracts: Obama → dbo:Obama → ‘Barack Hussein
Obama II; born August 4, 1961) is an American politician…’

13. Sem-CNN – Experimental Setup
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
13
Dataset versions
- Full Dataset: 28,000 tweets.
- Balanced Dataset (BD1): 9100 tweets (32.6%).
- Semantically Balanced Dataset (>2 entities/concepts, BD2): 1194 tweets
(4.3%).
Baselines
- SVM (TF-IDF): Linear SVM using the words’ TF-IDF vectors extracted
from our dataset.
- SVM (Word2Vec): Linear SVM using the Google pre-trained 300-
dimensional word embeddings.
Evaluation
- 5-folds cross validation.
- Sem-CNN: 300-dim embeddings, Fn = 128 convolutional filter of sizes
Fs = [3,4,5], 0.5 dropout and ADAM.
- Evaluation Measures: P, R and F1.
?

14. Results
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
14

15. Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
Publications
15
+ - Sem-CNN significantly outperforms the baselines
(p < 0.001)
- More semantics leads to better results.
- Sem-CNN appears to perform better than Dual-
CNN (up to +4% F1) with F1 up to 64%.
- Abstract outperform the Concept vectors but it is
not always significant (i.e., on the full dataset).
- Consider more complex deep learning models
such as Recurrent Neural Networks (RNN) or
Hierarchical Attention Networks (HAN).
- Initialise with different embeddings (e.g., Twitter)
and perform parameter optimisation.
- Investigate other methods for integrating
semantics (e.g., extended concept graphs).
-
Results and Future Work
CREES
Crisis Event Extraction
Service
?

16. Questions
@
Email: g.burel@open.ac.uk
Twitter: @evhart
CREES: https://github.com/evhart/crees
COMRADES: http://comrades-project.eu
Semantic Wide and Deep Learning for Detecting Crisis-Information Categories on Social Media
16