The application of Temporal Random Indexing (TRI) to the annotated New York Times corpus is used to identify the semantic shift in the context of named entities.

1. Forward to the Past
Annalina Caputo, PhD
EDGE COFUND MSCA Research Fellow
The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.

2. www.adaptcentre.ie
2008 2010 20152007 2012
1
m
illion
7
m
illion
15
m
illion
20
m
illion
25
m
illion
Google Books titles

3. www.adaptcentre.ie
2008 2010 20152007 2012
1
m
illion
7
m
illion
15
m
illion
20
m
illion
25
m
illion
130 million
Google Books titles

4. Click to edit Master title style
Photo by Victoria Kure-Wu on Unsplash
Close Reading

5. www.adaptcentre.ieDistant Reading
graph generated with http://alpha.huviz.dev.nooron.com/

6. www.adaptcentre.ieCulturomics
A form of computational lexicology that
studies human behavior and cultural
trends through the quantitative analysis of
digitized texts.
“
J.-B. Michel et al., Quantitative Analysis of Culture Using Millions of Digitized Books, Science, 2011

7. www.adaptcentre.ieGrammar Evolution

8. www.adaptcentre.iePopularity
Google Books Ngram Viewer: word frequencies

9. www.adaptcentre.ieEvents
Russian Flu
Spanish Flu
Asian Flu
Google Books Ngram Viewer: influenza

10. www.adaptcentre.ieLimitations: no semantics
full of awe terrible
Google Books Ngram Viewer: awful

11. www.adaptcentre.ieLimitations: no context
Google Books Ngram Viewer: no context

12. www.adaptcentre.ieDistributional Semantic Models
You shall know a word by
the company it keeps!
John Rupert Firth
https://goo.gl/nY4els
Meaning of a word is
determined by its
usage.
LudwigWittgenstein
https://goo.gl/mD1oKn
Distributional structure
Mathematical
structures of language
Zellig Harris
https://goo.gl/b3sMtC

13. www.adaptcentre.ieDistributional Semantic Models
• Analysis of word-usage
statistics over huge
corpora
• Geometric space of
concepts
• Similar words are
represented close in the
space
computer
animal

14. www.adaptcentre.ieDistributional Semantic Models
A WordSpace is a
snapshot of a specific
corpus it does not take
into account temporal
information
“
Photo by Victoria Kure-Wu on Unsplash

15. www.adaptcentre.ieDiachronic Analysis
Temporal
WordSpace
Time
Series
Change Point
Detection
Build a WordSpace
for each year
Provide a time series
for each word
Detect significant
changes in the time
series
Kulkarni et al., Statistically Significant Detection of Linguistic Change, WWW, 2015

16. www.adaptcentre.ieTemporal WordSpace: Temporal Random Indexing
Corpus with temporal
information: split in several
time periods
Build a WordSpace for each
time period
Words in different
WordSpaces are comparable!WordSpace
year 1
WordSpace
year 2
WordSpace
year n
Corpus
year 1
Corpus
year 2
Corpus
year n
…

17. www.adaptcentre.ieTime Series: Cosine Similarity
!"##" = −0.04, −0.15, 0.04, 0.00, 0.01, …
!"##- = 0.00, −0.22,0.05, −0.01, −0.03, …
!"##"
!"##-

18. www.adaptcentre.ieChange point detection: Mean shift model
diff max-min of
bootstrap series
diff max-min
CUMBSUM of
similarity series

19. www.adaptcentre.ieSemantic change: cell
GOOGLE
NGRAM
TRI

20. www.adaptcentre.ieWord-word similarity: tweet

21. www.adaptcentre.ieTemporal Entity Aspects
Source https://en.wikipedia.org

22. www.adaptcentre.ieTemporal Entity Aspects
Temporal
WordSpace
Time
Series
Change Point
Detection
Build a WordSpace for
each year
Provide a time series for
each word/entity
Detect significant
changes in the time
series
Entity
Linking
Recognition and
linking of named
entities

23. www.adaptcentre.ieEntity Linking
Task of linking entity mentions to entries in a knowledge base (DBpedia)
Asked to name the leader of the Democratic Party, Mr. Lieberman did
not immediately mention Mr. Gore, the standard bearer from 2000 ,
who beat George W. Bush in the popular vote .

24. www.adaptcentre.ieEntity Linking
Task of linking entity mentions to entries in a knowledge base (DBpedia)
Asked to name the leader of the
[dbp:Democratic_Party_(United_States)], Mr. [dbp:Joe_Lieberman] did
not immediately mention Mr. [dbp:Al_Gore], the standard bearer from
2000 , who beat [dbp:George_W._Bush] in the popular vote .
[1] https://upload.wikimedia.org/wikipedia/commons/thumb/7/73/US_Democratic_Party_Logo.svg/300px-US_Democratic_Party_Logo.svg.png
[2] https://en.wikipedia.org/wiki/File:George-W-Bush.jpeg
[3] https://en.wikipedia.org/wiki/File:Al_Gore,_Vice_President_of_the_United_States,_official_portrait_1994.jpg
[4] https://upload.wikimedia.org/wikipedia/commons/thumb/6/62/Joe_Lieberman_official_portrait_2.jpg

25. www.adaptcentre.ieThe Corpus
• 1.8 million articles
• 1987 to 2007
• 5,268,315 recognised entities
• 22,738 entities appear in every year

26. www.adaptcentre.ieWhite House
Similar entities in 2000
not present in 2001
Similar entities in 2001
not present in 2000

27. www.adaptcentre.ieRepublican Party (US)
Similar entities in 2000
not present in 2001
Similar entities in 2001
not present in 2000

28. www.adaptcentre.ieGeorge H. W. Bush
Similar entities in 2000
not present in 2001
Similar entities in 2001
not present in 2000

29. www.adaptcentre.ieSimilarity vs Frequentist approach

30. www.adaptcentre.ieLimitations

31. www.adaptcentre.ieSomething went wrong!
The
Band!

32. www.adaptcentre.ieLimitations
George W. Bush
presidency

33. Click to edit Master title style
That's not fair.
That's not fair at all.
There was time now.
There was—
was all the time I needed…!
It's not fair!
It's not fair!
“
https://tinyurl.com/yb954482
Time Enough at Last