Summarizing news articles is becoming crucial for allowing quick and concise access to information about daily events. This task can be challenging when the same event is reported with various levels of detail or is subject to diverse view points. A well established technique in the area of news summarization consists in modeling events as a set of semantic triples. These triples are weighted, mainly based on their frequencies, and then fused to build summaries. Typically, triples are extracted from main clauses, which might lead to information loss. Moreover, some crucial facets of news, such as reasons or consequences, are mostly reported in subordinate clauses and thus they are not properly handled. In this paper, we focus on an existing work that uses a graph structure to model sentences allowing the access to any triple independently from the clause it belongs to. Summary sentences are then generated by taking the top ranked paths that contain many triples and show grammatical correctness. We further provide several improvements to that approach. First, we leverage node degrees for finding the most important triples and facets shared among sentences. Second, we enhance the process of triple fusion by providing more effective similarity measures that exploit entity linking and predicate similarity. We performed extensive experiments using the DUC’04 and DUC’07 datasets showing that our approach outperforms baseline approaches by a large margin in terms of ROUGE and PYRAMID scores.

1. 0
Modeling and Summarizing News Events
Using Semantic Triples
ESWC’2018,Heraklion|Crete,Greece
Radityo Eko Prasojo, Mouna Kacimi, Werner Nutt
5 June 2018

2. Eve the News Reader
1
… the deal is unfair for US
Industries … his desire to overturn
anything that has Obama’s stamp
of approval on it …
and 1000++ more
… Hurricane Nate slammed
Louisiana … The hurricane
killed 2 people … The pope
sent his prayer …
and 1000++ more
… Three people sustained
injuries … politicians are
corrupt … police staged
mass raids …
and 1000++ more
THIS IS
TIME
CONSUMING

3. Wall-e promises…
2
Don’t worry Eve! I’ll
build a summary
for you!
!

4. Extractive vs Abstractive
3

5. The Chemistry of Abstractive Summarization
“… Hurricane Nate
slammed Louisiana …
Nate struck The State
of Louisiana … The
hurricane in Louisiana
killed 2 people …”
“Hurricane” “Nate” “slammed” “Louisiana”
“Nate” “struck” “the” “State” “of” “Louisiana”
“The” “hurricane” “killed” “2” “people”
Information
Extraction
Fact fusion or merging
Extractive summary:
“Hurricane Nate slammed Louisiana.”
“The hurricane killed 2 people.”
Abstractive summary, with conjoined facts:
“Hurricane Nate slammed Louisiana, killing 2 people.”
Open Information Extraction
Entity Linking
Verb Linking 4

6. Abstractive News Summarization Approaches
5
Phrase-selection based
(PSB)
• Tentatively pairs
subject and verb phrases
from different sentences
• Checks for compatibility
Pattern-graph Fusion
(PGF)
• Looks for similar tokens
in different sentences
• Fuses the tokens,
thus forming a graph

7. PSB Example
“Hurricane” “Nate” “slammed” “Louisiana”
“It” “killed” “2” “people”
compatible
compatible
similar
“Hurricane Nate slammed Louisiana, killed 2 people”
“Nate” “struck” “the” “State” “of” “Louisiana”
compatible
similar
similar
6

8. PSB Limitation
7
Dilemma: losing information or being redundant

9. PSB Limitation: Subclause/Adverbial Fact
“Hurricane” “Nate” “slammed” “Louisiana” “after leaving Central America”
“Nate” “slammed Louisiana with 85mph winds”
similar?
Incomplete case: “Hurricane Nate slammed Louisiana after leaving Central America.”
Redundant case: “Hurricane Nate slammed Louisiana after leaving Central America,
slammed Louisiana with 85mph winds.”
8

10. Pattern Graph Fusion (PGF)
<people; killed by; hurricane>
<people; died in; Louisiana>
Extraction
OLLIE
9
Stanford
SEMAFOR
<PERSON; killed by; PROTAGONIST>
<PERSON; died in; LOCATION>
Typing
PERSON
killed by died in
PROTAG LOC
Graph fusion
*Li et al. Weakly Supervised Natural Language Processing Framework
for Abstractive Multi-Document Summarization (CIKM’15)

11. Pattern Graph Fusion: Extraction
“… Hurricane Nate slammed Louisiana with 85 mph winds …”
“… On Saturday, Nate struck Louisiana …”
“… In Louisiana, the hurricane killed 2 people …”
“Hurricane” “Nate” “slammed” “Louisiana” “with” “85” “mph” “winds”
“On” “Saturday,” “Nate” “struck” “Louisiana”
“The” “hurricane” “in” “Louisiana” “killed” “2” “people”
OLLIE
10
Extraction Typing Fusion

12. TypingExtraction
Pattern Graph Fusion: Typing
Stanford NLP
& SEMAFOR
“Hurricane” PERSON “slammed” LOCATION “with” “85” “mph” “winds”
“On” DATE “,” PERSON “struck” LOCATION
“The” KILLER “in” “Louisiana” “killed” “2” VICTIM
“Hurricane” “Nate” “slammed” “Louisiana” “with” “85” “mph” “winds”
“On” “Saturday,” “Nate” “struck” “Louisiana”
“The” “hurricane” “in” “Louisiana” “killed” “2” “people”
11
Extraction Typing Fusion

13. Fusion of Sentences along Types
Hurricane PERSON slammed LOCATION with 85 mph
winds
On DATE , struck
The KILLER killed 2 VICTIM
“On Saturday, Nate slammed Louisiana with 85 mph winds”
in
12
Extraction Typing Fusion
PERSON slammed LOCATION with 85 mph
winds
On DATE ,

14. Pattern Graph Fusion: Limitations #1
Hurricane PERSON slammed LOCATION with 85 mph
winds
On DATE , struck
The KILLER killed 2 VICTIM
“On Saturday, Nate sent his prayer for Louisiana with 85 mph winds”
in
Correctness: Typing might lead to incorrect result
Pope sent his RITE for
13
PERSON LOCATION with 85 mph
winds
On DATE ,
sent his RITE for

15. Pattern Graph Fusion: Limitations #2
• Coverage: Merging misses semantically similar verbs
• “withdraw” vs “pull out”
• Grammaticality: Merging leads to ungrammatical sentences
• “The US pulled out from the Paris Agreement caused
disappointment among environmentalist.”
14

16. Our Summarization
We improve
• Correctness and coverage by Entity Linking
• Coverage by Verb Linking
• Grammaticality by Grammatical Fixing
15

17. Clustering
Fusion, Ranking
and Generation
Our Summarization: Pipeline
Dataset Triple Extraction
Entity
Linking (EL)
Verb Linking
(VL)
Fusion, Ranking
and Generation
Summary Typing
16

18. Pipeline: Entity Linking
17
• Named entity recognition: DBpedia Spotlight
• Name normalization: DBpedia
• Coreference resolution: DBpedia
Stanford Coref
Hurricane Nate slammed Louisiana with 85 winds
The hurricane in The Creole State killed 2 people
It struck Central America with 90
dbp:type hurricane dbo:WikiPageRedirectsOf
winds
mph
mph

19. Pipeline: Verb Linking
18
• WordNet Similarity > 0.9
It struck Central America on Monday
Hurricane Nate slammed Louisiana with 85 windsmph

20. Pipeline: Fusion and Ranking
19
#1
• What should be merged?
• Entities, Verbs
• Types of non-entity and non-verb tokens, if exist
• Non-stopword tokens
• The fused graph when merging entities and verbs:
killed 2 people
Hurricane Nate slammed Louisiana with 85 windsmph
Central America with 90 windsmph
in
2

21. • Path collection
• Path ranking criteria
• Path coverage
• Node degree
• “Hurricane Nate in Louisiana with 85 mph winds”
• Path coverage = 2, Avg node degree = 15/7 = 2.14
• “Hurricane Nate slammed Louisiana killed 2 people”
• Path coverage = 2, Avg node degree = 15/6 = 2.5
Pipeline: Fusion and Ranking
20
#2
killed 2 people
2 4 4 2 2 1
2 2 1
2 2 2 1
Hurricane Nate slammed Louisiana with 85 windsmph
2
Central America with 90 windsmph
2
in
2
2

22. Pipeline: Summary Generation
21
• Entity: the canonicalized form of DBpedia
• Verb: the form most frequently used in the cluster
Hurricane Nate slammed Louisiana
killed 2 people
2
It
The hurricane struck The Creole State

23. Pipeline: Summary Generation
22
Grammatical fixing for dangling verb
• Dangling verb identification (e.g, subject presence check)
• Verb active/passive type check
 Fix verb (e.g., transform to participle)
killed 2 peopleHurricane Nate slammed Louisiana
subj
subj?
dobj
killing

24. Evaluation
23
• DUC 2004 and DUC 2007 Dataset
• 95 news topics in total
• For each topic:
• 10 news articles
• 4 human summaries as the gold standard
• (for DUC’07) semantic gold standard
• Settings: different merging strategies to test
• entities, verbs, typings, non-stopwords
• Metrics:
• ROUGE: n-gram overlaps with the gold standard
• PYRAMID: semantic comparison with the semantic gold standard
• Human assessment to measure coherence (grammaticality) and
correctness

25. Results: Rouge
24
EL + T + NS
VL + T + NS
Our (VL)EL + VL + T + NS
EL + VL + NS
EL + VL
EL + VL + T
Baseline (T + NS)
Baseline=Li et al. Weakly Supervised Natural Language Processing Framework for Abstractive Multi-Document Summarization (CIKM’15)
EL= entity linking, VL=verb linking, T=typing, NS=non-stopwords
Rouge-1 Rouge-2 Rouge-1 Rouge-2
Recall Recall Recall Recall

26. Results: Pyramid
25
EL+VL+T
Baseline (T + NS)
Manual: Humans judge quality on a
5-grade Likert scale
Pyramid: Semantic similarity with
gold standard, checked using low-
dimensional latent word vector*
EL + T + NS
VL + T + NS
Our (VL)EL + VL + T + NS
EL + VL + NS
EL + VL
EL + VL + T
Baseline (T + NS)
Recall
Baseline=Li et al. Weakly Supervised Natural Language Processing Framework for Abstractive Multi-Document Summarization (CIKM’15)
EL= entity linking, VL=verb linking, T=typing, NS=non-stopwords
*Guo & Diab, ACL’12

27. • PGF is best-fit for abstractive summarization
• We enriched PGF by semantic annotations from kbs:
• entity and verb linking
• path ranking leveraging node degrees
+ added grammatical fixes
• Experiments show our enrichments outperform the baseline
26
Conclusions

28. • More fine-grained representation of facts
• Semantic ranking
• Fluency
27
Future work
Hurricane Nate slammed Louisiana
killing …
causing …
On Saturday
After leaving…
With winds of …

29. From Information Extraction to Abstractive Summarization 28
Thank you!
Wall-e summarizes: I found that between extractive and abstractive
summarization, the latter is what we want. Then, I found that pattern graph
fusion is best-fit for abstractive summarization due to the dilemma of phrase
selection-based approaches. I improved PGF approach by introducing semantic
enrichments that are entity linking, verb linking, and path ranking leveraging
node degrees, on top of adding some grammatical fixes. My experiment shows
that our enrichments outperform the baseline.