Two Approaches to Factor Time into Word and Entity Representations Learned fr...Federico Bianchi
Time is a crucial factor when dealing with distributional models of language and knowledge. For example, tracking word meaning shift and entity evolution can have several applications and time may sneak into similarity as computed with these models in a way that may be difficult to control. In this presentation, we discuss two novel approaches to factor time into word and knowledge representations learned from text: explicit, with representations of temporal references (e.g., years, days, etc.), and implicit, with time-dependent representations of words and entities (e.g., amazon_1975 vs. amazon_2012). Finally, being this an emerging field of research, we will discuss several open topics in this research domain.
FBK, Trento, 10/5/2019
Type Vector Representations from Text. DL4KGS@ESWC 2018Federico Bianchi
Type Vector Representations from Text: An Empirical Analysis. For the Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies (DL4KGS).
Held in conjunction with ESWC 18 in June 2018 in Crete, Greece.
Type Embeddings, Ontology Matching, Type Similarity
Evaluation Initiatives for Entity-oriented Searchkrisztianbalog
This document discusses evaluation initiatives for entity-oriented search tasks. It describes several shared tasks and evaluation campaigns that have been held at conferences like TREC, CLEF, and INEX to provide standardized test collections, gold standard annotations, and evaluation metrics for tasks like ad-hoc entity retrieval from knowledge bases. Examples of test collections created for entity retrieval include datasets using Wikipedia, DBpedia, and a web crawl. The document also discusses related entity finding, entity linking, and understanding keyword queries by associating terms with entities.
The document discusses relation extraction and information extraction tasks. It describes named entity extraction, binary and n-ary relation extraction, and other tasks like event extraction. It also discusses evaluation of information extraction systems through conferences like MUC which focused on extracting templates of entities and relations from texts.
The document discusses relation extraction and information extraction tasks. It describes named entity extraction, binary and n-ary relation extraction, and other tasks like event extraction. It also discusses early information extraction evaluations like MUC which focused on filling templates and recognized named entities, relations, and coreference resolution. The document outlines the components of a semantic model used for information extraction, including entities, relations, attributes, and events.
Temporal models for mining, ranking and recommendation in the WebTu Nguyen
This document discusses temporal models for mining, ranking, and recommendation on the web through the lens of time. It outlines research questions about how relevant aspects of entity-centric queries change around associated event times, and how to form ranked lists of documents to maximize coverage for ambiguous queries. The document motivates these questions using examples and discusses approaches that include time and type classification, joint learning in a cascaded manner, and multi-criteria learning models. It describes datasets, comparison methods, experiments analyzing feature performance for different event types and times, and lessons learned.
This document summarizes a study analyzing the content and connections between 1,509 top U.S. political blogs between 2012 and 2016. The study used text mining techniques like removing stop words, tokenizing words, creating n-grams, and applying topic models like LDA to analyze common topics and identify cliques of blogs discussing similar issues. For example, analysis of blogs discussing the Trayvon Martin case found topics related to the court case, political aspects, social functions of blogging, facts of the story, and racism. Topic weights were used to predict link formation and spread of ideas between blogs over time.
Two Approaches to Factor Time into Word and Entity Representations Learned fr...Federico Bianchi
Time is a crucial factor when dealing with distributional models of language and knowledge. For example, tracking word meaning shift and entity evolution can have several applications and time may sneak into similarity as computed with these models in a way that may be difficult to control. In this presentation, we discuss two novel approaches to factor time into word and knowledge representations learned from text: explicit, with representations of temporal references (e.g., years, days, etc.), and implicit, with time-dependent representations of words and entities (e.g., amazon_1975 vs. amazon_2012). Finally, being this an emerging field of research, we will discuss several open topics in this research domain.
FBK, Trento, 10/5/2019
Type Vector Representations from Text. DL4KGS@ESWC 2018Federico Bianchi
Type Vector Representations from Text: An Empirical Analysis. For the Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies (DL4KGS).
Held in conjunction with ESWC 18 in June 2018 in Crete, Greece.
Type Embeddings, Ontology Matching, Type Similarity
Evaluation Initiatives for Entity-oriented Searchkrisztianbalog
This document discusses evaluation initiatives for entity-oriented search tasks. It describes several shared tasks and evaluation campaigns that have been held at conferences like TREC, CLEF, and INEX to provide standardized test collections, gold standard annotations, and evaluation metrics for tasks like ad-hoc entity retrieval from knowledge bases. Examples of test collections created for entity retrieval include datasets using Wikipedia, DBpedia, and a web crawl. The document also discusses related entity finding, entity linking, and understanding keyword queries by associating terms with entities.
The document discusses relation extraction and information extraction tasks. It describes named entity extraction, binary and n-ary relation extraction, and other tasks like event extraction. It also discusses evaluation of information extraction systems through conferences like MUC which focused on extracting templates of entities and relations from texts.
The document discusses relation extraction and information extraction tasks. It describes named entity extraction, binary and n-ary relation extraction, and other tasks like event extraction. It also discusses early information extraction evaluations like MUC which focused on filling templates and recognized named entities, relations, and coreference resolution. The document outlines the components of a semantic model used for information extraction, including entities, relations, attributes, and events.
Temporal models for mining, ranking and recommendation in the WebTu Nguyen
This document discusses temporal models for mining, ranking, and recommendation on the web through the lens of time. It outlines research questions about how relevant aspects of entity-centric queries change around associated event times, and how to form ranked lists of documents to maximize coverage for ambiguous queries. The document motivates these questions using examples and discusses approaches that include time and type classification, joint learning in a cascaded manner, and multi-criteria learning models. It describes datasets, comparison methods, experiments analyzing feature performance for different event types and times, and lessons learned.
This document summarizes a study analyzing the content and connections between 1,509 top U.S. political blogs between 2012 and 2016. The study used text mining techniques like removing stop words, tokenizing words, creating n-grams, and applying topic models like LDA to analyze common topics and identify cliques of blogs discussing similar issues. For example, analysis of blogs discussing the Trayvon Martin case found topics related to the court case, political aspects, social functions of blogging, facts of the story, and racism. Topic weights were used to predict link formation and spread of ideas between blogs over time.
A presentation on the value and the risks of identifying, mining, and visualizing data. All this is described in a big-data-aware setting. The presentation is meant for a wide audience, not requiring deep technical background.
The original presentation was done within the KAS Seminar on Data Journalism in Dec 2017.
This document summarizes a presentation on analyzing political communication data from Twitter. It discusses analyzing the structure of Twitter data by examining things like retweet networks and interaction patterns, versus analyzing the content of tweets by looking at topics, sentiments, and word frequencies. It provides examples of studies that take both structural and content-based approaches. Specifically, it examines studies that analyzed how Twitter discussions relate to televised political debates and who engages in uncivil language online. The presentation concludes that the most insightful approach is often to combine structural and content-based analyses.
We present a framework that combines machine learnt classifiers and taxonomies of topics to enable a more conceptual analysis of a corpus than can be accomplished using Vector Space Models and Latent Dirichlet Allocation based topic models which represent documents purely
in terms of words. Given a corpus and a taxonomy of topics, we learn a classifier per topic and annotate each document with the topics covered by it. The distribution of topics in the corpus can then be visualized as a function of the attributes of the documents. We apply this framework to the US State of the Union and presidential election speeches to observe how topics such as jobs and employment have evolved from being relatively unimportant to being the most discussed topic. We show that our framework is better than Vector Space Models and an Latent Dirichlet Allocation based topic model for performing certain kinds of analysis.
Persuasive Essay On Capital Punishment. Essay on Capital Punishment Internat...Monica Clark
007 Persuasive Essay About Death Penalty Capital Punishment L ~ Thatsnotus. Persuasive Writing - Capital Punishment. - GCSE Religious Studies .... Capital Punishment Persuasive Speech | PDF | Punishments | Capital .... Capital Punishment (Essay) | Capital Punishment | Hanging.
Visualising data: Seeing is Believing - CS Forum 2012Richard Ingram
When patterns and connections are revealed between numbers, content and people that might otherwise be too abstract or scattered to be grasped, we’re able to make better sense of where we are, what it might mean and what needs to be done.
Surfacing Real-World Event Content on TwitterHila Becker
The document describes a method for identifying real-world events from Twitter data in real-time using an unsupervised machine learning approach. Tweets are clustered based on textual similarity and metadata like time and location. Cluster-level features are then extracted and used to train a classifier to identify event clusters. Key steps include content representation, incremental clustering of tweets, feature extraction from clusters, building an event classifier, and selecting tweets to display for each identified event. The goal is to surface real-world event content from Twitter in an unsupervised and automated fashion.
Tutorial semantic wikis and applicationsMark Greaves
This document outlines an agenda for a tutorial on semantic wikis and applications. The tutorial will include introductions to Semantic MediaWiki, diving deeper into its features, applications of semantic wikis, extensions for Semantic MediaWiki developed by various contributors, connecting Semantic MediaWiki with MS Office, augmenting it with a triple store, discussing future development, and concluding with a question and answer session, followed by a 30 minute break.
The document discusses knowledge graphs and their future directions. It summarizes a panel discussion on knowledge graphs at ESWC 2020 and references several papers on industry-scale knowledge graphs, weak supervision for knowledge graph construction, and representing entities and identities in knowledge bases. It concludes that knowledge graph construction involves complex pipelines with many components and calls for an updated theory of knowledge engineering to address the demands of modern knowledge graphs at large scale and with continuous changes.
Mining Social Media with Linked Open Data, Entity Recognition, and Event Extr...Leon Derczynski
Presented at the 4th DEOS workshop, http://diadem.cs.ox.ac.uk/deos13/
Social media presents itself as a context-rich source of big data, readily exhibiting volume, velocity and variety. Mining information from microblogs and other social media is a challenging, emerging research area. Unlike carefully authored news text and other longer content, social media text poses a number of new challenges, due to the short, noisy, context-dependent, and dynamic nature.
This talk will discuss firstly how Linked Open Data (LOD) vocabularies (namely DBpedia and YAGO) have been used to help entity recognition and disambiguation in such content. We will introduce LODIE, the LOD-based extension of the widely used ANNIE open-source entity recognition system. LODIE includes also entity disambiguation (including products, as well as names of persons, locations, and organisations) and has been developed as part of the TrendMiner and uComp projects. Quantitative evaluation results will be shown, including a comparison against other state-of-the-art methods and an analysis of how errors in upstream linguistic pre-processing (i.e. tokenisation and POS tagging) can affect disambiguation performance. Our results demonstrate the importance of adjusting approaches for this genre.
The second half of the talk will focus on fine-grained events in tweets. Awareness of temporal context in social media enables many interesting applications. We identify events using the TimeML schema, focusing on occurrences and actions. Challenges of event annotation will be discussed, as well as the development of a supervised event extractor specifically for social media. We evaluate this against traditional event annotation approaches (e.g. Evita, TIPSem).
The document discusses the evolution of the internet and communication technologies over time. It describes how the cost of information has decreased with new mediums like the internet and how business structures have shifted from hierarchies to networks. Recent developments discussed include the growth of e-commerce, rise of user-generated content, social media platforms, and mobile internet connectivity.
The document discusses key concepts of relational databases including:
1. Relational databases organize data into tables with records and fields and allow for defining relationships between tables.
2. Tables represent relations with rows as tuples and columns as attributes.
3. Common operations on relations include select, project, join, union, intersection and difference which allow querying and manipulating the data.
4. The document provides examples of designing database tables to model real-world entities and relationships. Primary keys are used to uniquely identify rows.
Invited Keynote by Professor Noah A Smith (University of Washington) for ACL 2017. 1 Aug 2017. Vancouver, Canada.
Also available at https://homes.cs.washington.edu/~nasmith/slides/acl-8-1-17.pdf
License: CC BY 4.0
The document summarizes an entity extraction and typing framework proposed by the author. The framework constructs a heterogeneous graph connecting entity mentions, surface names, and relation phrases extracted from documents. It then performs joint type propagation and relation phrase clustering on the graph to infer types for entity mentions. Evaluation on news, tweets and reviews shows the framework outperforms existing methods in recognizing new types and domains without extensive feature engineering or human supervision. It obtains improvements by modeling each mention individually and addressing data sparsity through relation phrase clustering.
Measuring the Complexity of the Law: The United States Code ( Slides by Danie...Daniel Katz
This document presents a framework for measuring the complexity of the United States Code using computational methods. It represents the Code as a mathematical object with dimensions including its hierarchical structure, citation network, and linguistic content. Complexity is measured across individual titles using weighted ranks that composite scores across these dimensions. The results provide insights into the relative complexity of different areas of law and how this may relate to the complexity of the topics regulated. The authors aim to advance empirical study of legal complexity and demonstrate the potential of computational analysis of large legal corpora.
1) The document examines differences in how the 2010 BP oil spill crisis was framed in US and UK news media versus BP's press releases over time.
2) It finds BP sought to distance itself from responsibility for the cause of the spill while presenting itself as able to solve the crisis.
3) Political actors and complex connections between issues were more emphasized in news coverage compared to BP's statements, and frames simplified over the course of the crisis.
From text to entities: Information Extraction in the Era of Knowledge GraphsGraphRM
Incontro del 23/07/2018
In recent years there has been a proliferation of free and commercial "knowledge graphs" (KGs), which represent real-world entities together with their semantic relationships in a graphical form. Those are becoming a powerful asset both for tech giants, with Google Knowledge Graph, IBM’s Watson QA system and Facebook’s Open Graph, as well as for startups that are developing AI products, such as, semantic search, data analytics, recommender systems. While KGs provide a structured access to a large amount of knowledge, a vast majority of the information available on the Web is still inaccessible because encoded only in the form of natural-language text. The talk will present an overview of public available KGs and the main techniques used to bridge unstructured text with them, enabling a wide variety of knowledge-based applications.
Speaker: Matteo Cannaviccio
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
A presentation on the value and the risks of identifying, mining, and visualizing data. All this is described in a big-data-aware setting. The presentation is meant for a wide audience, not requiring deep technical background.
The original presentation was done within the KAS Seminar on Data Journalism in Dec 2017.
This document summarizes a presentation on analyzing political communication data from Twitter. It discusses analyzing the structure of Twitter data by examining things like retweet networks and interaction patterns, versus analyzing the content of tweets by looking at topics, sentiments, and word frequencies. It provides examples of studies that take both structural and content-based approaches. Specifically, it examines studies that analyzed how Twitter discussions relate to televised political debates and who engages in uncivil language online. The presentation concludes that the most insightful approach is often to combine structural and content-based analyses.
We present a framework that combines machine learnt classifiers and taxonomies of topics to enable a more conceptual analysis of a corpus than can be accomplished using Vector Space Models and Latent Dirichlet Allocation based topic models which represent documents purely
in terms of words. Given a corpus and a taxonomy of topics, we learn a classifier per topic and annotate each document with the topics covered by it. The distribution of topics in the corpus can then be visualized as a function of the attributes of the documents. We apply this framework to the US State of the Union and presidential election speeches to observe how topics such as jobs and employment have evolved from being relatively unimportant to being the most discussed topic. We show that our framework is better than Vector Space Models and an Latent Dirichlet Allocation based topic model for performing certain kinds of analysis.
Persuasive Essay On Capital Punishment. Essay on Capital Punishment Internat...Monica Clark
007 Persuasive Essay About Death Penalty Capital Punishment L ~ Thatsnotus. Persuasive Writing - Capital Punishment. - GCSE Religious Studies .... Capital Punishment Persuasive Speech | PDF | Punishments | Capital .... Capital Punishment (Essay) | Capital Punishment | Hanging.
Visualising data: Seeing is Believing - CS Forum 2012Richard Ingram
When patterns and connections are revealed between numbers, content and people that might otherwise be too abstract or scattered to be grasped, we’re able to make better sense of where we are, what it might mean and what needs to be done.
Surfacing Real-World Event Content on TwitterHila Becker
The document describes a method for identifying real-world events from Twitter data in real-time using an unsupervised machine learning approach. Tweets are clustered based on textual similarity and metadata like time and location. Cluster-level features are then extracted and used to train a classifier to identify event clusters. Key steps include content representation, incremental clustering of tweets, feature extraction from clusters, building an event classifier, and selecting tweets to display for each identified event. The goal is to surface real-world event content from Twitter in an unsupervised and automated fashion.
Tutorial semantic wikis and applicationsMark Greaves
This document outlines an agenda for a tutorial on semantic wikis and applications. The tutorial will include introductions to Semantic MediaWiki, diving deeper into its features, applications of semantic wikis, extensions for Semantic MediaWiki developed by various contributors, connecting Semantic MediaWiki with MS Office, augmenting it with a triple store, discussing future development, and concluding with a question and answer session, followed by a 30 minute break.
The document discusses knowledge graphs and their future directions. It summarizes a panel discussion on knowledge graphs at ESWC 2020 and references several papers on industry-scale knowledge graphs, weak supervision for knowledge graph construction, and representing entities and identities in knowledge bases. It concludes that knowledge graph construction involves complex pipelines with many components and calls for an updated theory of knowledge engineering to address the demands of modern knowledge graphs at large scale and with continuous changes.
Mining Social Media with Linked Open Data, Entity Recognition, and Event Extr...Leon Derczynski
Presented at the 4th DEOS workshop, http://diadem.cs.ox.ac.uk/deos13/
Social media presents itself as a context-rich source of big data, readily exhibiting volume, velocity and variety. Mining information from microblogs and other social media is a challenging, emerging research area. Unlike carefully authored news text and other longer content, social media text poses a number of new challenges, due to the short, noisy, context-dependent, and dynamic nature.
This talk will discuss firstly how Linked Open Data (LOD) vocabularies (namely DBpedia and YAGO) have been used to help entity recognition and disambiguation in such content. We will introduce LODIE, the LOD-based extension of the widely used ANNIE open-source entity recognition system. LODIE includes also entity disambiguation (including products, as well as names of persons, locations, and organisations) and has been developed as part of the TrendMiner and uComp projects. Quantitative evaluation results will be shown, including a comparison against other state-of-the-art methods and an analysis of how errors in upstream linguistic pre-processing (i.e. tokenisation and POS tagging) can affect disambiguation performance. Our results demonstrate the importance of adjusting approaches for this genre.
The second half of the talk will focus on fine-grained events in tweets. Awareness of temporal context in social media enables many interesting applications. We identify events using the TimeML schema, focusing on occurrences and actions. Challenges of event annotation will be discussed, as well as the development of a supervised event extractor specifically for social media. We evaluate this against traditional event annotation approaches (e.g. Evita, TIPSem).
The document discusses the evolution of the internet and communication technologies over time. It describes how the cost of information has decreased with new mediums like the internet and how business structures have shifted from hierarchies to networks. Recent developments discussed include the growth of e-commerce, rise of user-generated content, social media platforms, and mobile internet connectivity.
The document discusses key concepts of relational databases including:
1. Relational databases organize data into tables with records and fields and allow for defining relationships between tables.
2. Tables represent relations with rows as tuples and columns as attributes.
3. Common operations on relations include select, project, join, union, intersection and difference which allow querying and manipulating the data.
4. The document provides examples of designing database tables to model real-world entities and relationships. Primary keys are used to uniquely identify rows.
Invited Keynote by Professor Noah A Smith (University of Washington) for ACL 2017. 1 Aug 2017. Vancouver, Canada.
Also available at https://homes.cs.washington.edu/~nasmith/slides/acl-8-1-17.pdf
License: CC BY 4.0
The document summarizes an entity extraction and typing framework proposed by the author. The framework constructs a heterogeneous graph connecting entity mentions, surface names, and relation phrases extracted from documents. It then performs joint type propagation and relation phrase clustering on the graph to infer types for entity mentions. Evaluation on news, tweets and reviews shows the framework outperforms existing methods in recognizing new types and domains without extensive feature engineering or human supervision. It obtains improvements by modeling each mention individually and addressing data sparsity through relation phrase clustering.
Measuring the Complexity of the Law: The United States Code ( Slides by Danie...Daniel Katz
This document presents a framework for measuring the complexity of the United States Code using computational methods. It represents the Code as a mathematical object with dimensions including its hierarchical structure, citation network, and linguistic content. Complexity is measured across individual titles using weighted ranks that composite scores across these dimensions. The results provide insights into the relative complexity of different areas of law and how this may relate to the complexity of the topics regulated. The authors aim to advance empirical study of legal complexity and demonstrate the potential of computational analysis of large legal corpora.
1) The document examines differences in how the 2010 BP oil spill crisis was framed in US and UK news media versus BP's press releases over time.
2) It finds BP sought to distance itself from responsibility for the cause of the spill while presenting itself as able to solve the crisis.
3) Political actors and complex connections between issues were more emphasized in news coverage compared to BP's statements, and frames simplified over the course of the crisis.
From text to entities: Information Extraction in the Era of Knowledge GraphsGraphRM
Incontro del 23/07/2018
In recent years there has been a proliferation of free and commercial "knowledge graphs" (KGs), which represent real-world entities together with their semantic relationships in a graphical form. Those are becoming a powerful asset both for tech giants, with Google Knowledge Graph, IBM’s Watson QA system and Facebook’s Open Graph, as well as for startups that are developing AI products, such as, semantic search, data analytics, recommender systems. While KGs provide a structured access to a large amount of knowledge, a vast majority of the information available on the Web is still inaccessible because encoded only in the form of natural-language text. The talk will present an overview of public available KGs and the main techniques used to bridge unstructured text with them, enabling a wide variety of knowledge-based applications.
Speaker: Matteo Cannaviccio
Similar to Towards Encoding Time in Text-Based Entity Embeddings (20)
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
Open Source Contributions to Postgres: The Basics POSETTE 2024ElizabethGarrettChri
Postgres is the most advanced open-source database in the world and it's supported by a community, not a single company. So how does this work? How does code actually get into Postgres? I recently had a patch submitted and committed and I want to share what I learned in that process. I’ll give you an overview of Postgres versions and how the underlying project codebase functions. I’ll also show you the process for submitting a patch and getting that tested and committed.
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
We are pleased to share with you the latest VCOSA statistical report on the cotton and yarn industry for the month of March 2024.
Starting from January 2024, the full weekly and monthly reports will only be available for free to VCOSA members. To access the complete weekly report with figures, charts, and detailed analysis of the cotton fiber market in the past week, interested parties are kindly requested to contact VCOSA to subscribe to the newsletter.
Towards Encoding Time in Text-Based Entity Embeddings
1. Towards Encoding Time in
Text-Based Entity Embeddings
Federico Bianchi, Matteo Palmonari and Debora Nozza
University of Milano-Bicocca
INSID&S Lab
Interaction and Semantics for
Innovation with Data & Services
International Semantic Web Conference, Monterey, California. 2018
MIND Lab
Models in Decision making
and data analysis
2. Knowledge Graphs
Large knowledge bases
Entities classified using types
Types organized in sub-types graphs
Binary relationships between entities
Semantics and inference via
rules/axioms
Semantic similarity with lexical,
topological and other feature-based
approaches
A.S.
Roma
Kostas
Manolas
team
Soccer
Player
Soccer
Club
Athlete
Thing
Person
Sports
Club
Garry
Kasparov
Chess
Player
Real
Madrid
Organis.
3. Knowledge Graphs Embeddings
Generate vector representations of entities and relationships
A.S.
Roma
Kostas
Manolas
team 2
5
6
2
6
4
2
12
5
2
Kostas
Manolas
A.S.
Roma
4
2
12
5
2
team
Given in input a KG
Generate vector
representations
Embedding
Algorithm
Why should we embed?
● Latent components (e.g., → link prediction)
● Features generation (e.g., → entity linking)
● Fast and intuitive way to compute similarity
4. From Word Embeddings to Text-based Entity Embeddings
- Word embeddings (e.g., [Mikolov+, 2013])
- Text-based Entity Embeddings
- Text as main source vs. Graph as main source [Bordes+,2013][Trouillon+,2016]
- Typed Entity Embeddings (TEE): use word embeddings algorithms on documents where entities and
types replace words (next slide :) )
- Pros: good for similarity evaluation
- Cons: no embedding of relations, just entity
corpus
cat
black
eats
dog
similar words corresponds
to similar vectors
C
W
The big black cat eats its food.
My little black cat sleeps all day.
Sometimes my cat eats too much!
5. TEE: Typed Entity Embeddings from Text
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
[Bianchi+,2017b]
[Bianchi+, 2018a]
Wikipedia’s abstracts
6. TEE: Typed Entity Embeddings from Text
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
“dbr:Rome dbr:Italy
dbr:Rome dbr:Lazio …”“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
Link to DBpedia
entities via named
entity linking tools
[Bianchi+,2017b]
[Bianchi+, 2018a]
Wikipedia’s abstracts
7. TEE: Typed Entity Embeddings from Text
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
“dbr:Rome dbr:Italy
dbr:Rome dbr:Lazio …”
“dbo:City dbo:Country City
dbo:Administrative_Region …”
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
Link to DBpedia
entities via named
entity linking tools
Replace
entities
with their most
specific types
[Bianchi+,2017b]
[Bianchi+, 2018a]
Wikipedia’s abstracts
8. TEE: Typed Entity Embeddings from Text
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
“dbr:Rome dbr:Italy
dbr:Rome dbr:Lazio …”
“dbo:City dbo:Country City
dbo:Administrative_Region …”
Generate Type
Vectors
From Text
Generate Entity
Vectors
From Text“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
Link to DBpedia
entities via named
entity linking tools
Replace
entities
with their most
specific types
[Bianchi+,2017b]
[Bianchi+, 2018a]
Wikipedia’s abstracts
9. TEE: Typed Entity Embeddings from Text
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
“dbr:Rome dbr:Italy
dbr:Rome dbr:Lazio …”
“dbo:City dbo:Country City
dbo:Administrative_Region …”
Generate Type
Vectors
From Text
Generate Entity
Vectors
From Text
Concatenate
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
Link to DBpedia
entities via named
entity linking tools
Replace
entities
with their most
specific types
[Bianchi+,2017b]
[Bianchi+, 2018a]
Wikipedia’s abstracts
10. TEE: Typed Entity Embeddings from Text
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
“dbr:Rome dbr:Italy
dbr:Rome dbr:Lazio …”
“dbo:City dbo:Country City
dbo:Administrative_Region …”
Generate Type
Vectors
From Text
Generate Entity
Vectors
From Text
Concatenate
“Rome is the capital of
Italy and a special
comune (named
Comune di Roma
Capitale). Rome also
serves as the capital of
the Lazio region.”
Link to DBpedia
entities via named
entity linking tools
Replace
entities
with their most
specific types
[Bianchi+,2017b]
[Bianchi+, 2018a]
1 3 6 3 19 5 6
v(Rome)v(City)
Wikipedia’s abstracts
11. Why Time?
● To the best of our knowledge this is the first approach to explicitly encode time periods into entity
embeddings
● We expect that when we evaluate similarity between entities time is important:
○ Entities are similar when they co-occur frequently, entities that share a time period co-occur
Most similar entities to “Winston Churchill” are his contemporary politicians
● In this paper we try to provide an approach to explicitly encode time in such a way that we can use
those representation to control the similarity with respect to time
Winston Churchill Harold Macmillan
13. Textual Descriptions of Time Periods via Events
“The succession of events is an inherent property of our time
perception. Memory is necessary, and the order of these
events is fundamental”
Snaider&al. 2012, Cognitive Systems Research
14. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description
15. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description
16. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description
Adolf Hitler
Nazi Germany
World War II
4 3 6 2 3
5 1 2 9 2
1 2 8 4 1
17. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description
Adolf Hitler 4 3 6 2 3
Nazi Germany 5 1 2 9 2
World War II 1 2 8 4 1
1941
9 2 3 5 5
AVG
18. Towards Time Aware Similarity
Time flattened similarity: to reduce the impact of time in the similarity.
E.g., make US presidents similar independently from their temporal context.
Time boosted similarity: to boost the impact of time in the similarity.
E.g., make politicians that share temporal contexts more similar
20. Time Flattened Similarity
Extract the embeddings for the two entities
What’s the time flattened similarity between
Barack Obama and Bill Clinton?
21. Time Flattened Similarity
1999 2003
Find the closest year vectors to the two entity
embeddings (e.g., the entity vector of Barack
Obama is close to the vector of the year
2003).
What’s the time flattened similarity between
Barack Obama and Bill Clinton?
23. Time Flattened Similarity
1999 2003
𝝍( , ) = η( , )
Cosine similarity
What’s the time flattened similarity between
Barack Obama and Bill Clinton?
24. Time Flattened Similarity
1999 2003
𝝍( , ) = η( , ) - ηn
( , )1990 2003
Normalized cosine similarity
What’s the time flattened similarity between
Barack Obama and Bill Clinton?
25. Time Flattened Similarity
1999 2003
𝝍( , ) = ⍺η( , ) - (1 - ⍺) ηn
( , )1999 2003
⍺ to control the weight of the time factor
What’s the time flattened similarity between
Barack Obama and Bill Clinton?
26. Experiments: Research Questions
1. Quality: properties of the year embeddings
2. Similarity and Time:
a. Time Bias in TEE and EE: Effect of time in entity embeddings from text
i. Adherence to Natural Time Order
ii. Clustering WWI and WWII Battles
iii. Relative Ordering of Entities
b. Controlling Time Bias: handling the effect of time
27. Embedded Representations vs. Natural Time Flow
191X
years
201X
years
PCA in 1D vs. natural order of years: Kendall τ = 0.80 and Spearman Rank correlation coefficient = 0.94
Good resemblance of natural time flow!
2D projection (PCA)
1D projection (PCA)
28. Time Bias: Adherence to Natural Time Order
Task: count number of entities shared by sequences of 2-3 contiguous years vs
number of entities shared in non contiguous years (randomly sampled):
● (e.g, 1991-1992 vs 1934-1992)
Dataset: two and three contiguous years and non contiguous years (1931-1991).
Results: contiguous years share an higher amount of entities than non contiguous
years.
29. Time Bias: Clustering Battles with EE
Task: classify battles as belonging to WWI or WWII.
Dataset: 152 resource identifier of WWI (63) and WWII (89) battles from Wikipedia.
Method: K-means clustering (K=2) on the vector representation in the entity
embedding space.
Results: 95% accuracy. Centroids of the two groups are close to WWI years and
WWII years respectively.
30. Controlling Time Bias: Flattened Similarity
Task: find similar entities to a given input entity but that are far in time
Barack
Obama
31. Controlling Time Bias: Flattened Similarity
Task: find similar entities to a given input entity but that are far in time. E.g., find
past president given one
Ford
Coolidge
Hoover
T. Kennedy
Truman
Barack
Obama
32. Controlling Time Bias: Flattened Similarity
Task: find similar entities to a given input entity but that are far in time. E.g., find
past president given one
Ford
Coolidge
Hoover
T. Kennedy
Truman
Barack
Obama
Correct
Correct
Correct
Correct
Wrong
33. Controlling Time Bias: Flattened Similarity
Dataset: US presidents entities and British Prime ministers entities (19 and 19)
Method: start with the 6 most recent presidents for each group. For each entity
compute the number of older presidents that are in the ranked list created by the
similarity measures.
Time flattened reorders top-100 results from cosine similarity
Algorithms:
● Time-aware Similarity TEE (TATEE), with time-flattened similarity;
● Similarity TEE (STEE) (standard neighborhood with cosine);
● Time-Aware Similarity EE (TAEE), with time-flattened similarity;
● Similarity EE (SEE) (standard neighborhood with cosine);
● Time-flattened similarity Wiki2Vec (Baseline).
34. Controlling Time Bias: Flattened Similarity
Results: time-flattened similarity on TATEE seems able to get the best results. This
is also due to the fact that TATEE considers type representations and thus it can
easily retrieve entities sharing types.
35. Controlling Time Bias: Qualitative Analysis
Clinton
Reagan
G. Bush
Carter
Al Gore
Nixon
J. Kerry
D. Cheney
McCain
Biden
The most
similar
entities to
Barack
Obama using
cosine
similarity in
TEE
36. Controlling Time Bias: Qualitative Analysis
Clinton
Reagan
G. Bush
Carter
Al Gore
Nixon
J. Kerry
D. Cheney
McCain
Biden
The most
similar
entities to
Barack
Obama using
cosine
similarity in
TEE
Clinton
Reagan
G. Bush
Carter
Al Gore
Nixon
Ford
Coolidge
T. Kennedy
Hoover
Time flattened
similarity to
reorder the
top-100 most
similar
alpha = 0.7
New
New
New
New
37. Controlling Time Bias: Qualitative Analysis
Clinton
Reagan
G. Bush
Carter
Al Gore
Nixon
J. Kerry
D. Cheney
McCain
Biden
The most
similar
entities to
Barack
Obama using
cosine
similarity in
TEE
Time flattened
similarity to
reorder the
top-100 most
similar
alpha = 0.1
New
New
New
New
Ford
Coolidge
Hoover
Truman
Roosevelt
Wilson
E. Roosevelt
Harding
Cleveland
Eisenhower
New
New
New
New
New
New
38. Conclusions and Future Work
Conclusions
● Time can be represented in the vector space using events descriptions
● Time sneaks into entity similarity (time bias)
● Time bias can be controlled by considering explicit representations of
time periods
Future Work
● Study compositionality of time periods representations
● Comparison with Doc2Vec
● Improve time-aware similarity measure
● Comparison with other KG embeddings models
39. References
Snaider, J., McCall, R., & Franklin, S. (2012). Time production and representation in a conceptual and computational cognitive
model. Cognitive Systems Research, 13(1), 59-71.
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling
multi-relational data. In Advances in neural information processing systems (pp. 2787-2795).
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., & Bouchard, G. (2016, June). Complex embeddings for simple link prediction. In
International Conference on Machine Learning (pp. 2071-2080).
Tran, N. K., Tran, T., & Niederée, C. (2017, May). Beyond time: Dynamic context-aware entity recommendation. In European
Semantic Web Conference (pp. 353-368). Springer, Cham.
Bianchi, F., Soto, M., Palmonari, M., & Cutrona, V. (2018). Type vector representations from text: An empirical analysis. In Deep
Learning for Knowledge Graphs and Semantic Technologies Workshop, co-located with the Extended Semantic Web
Conference, Crete.
Bianchi, F., Palmonari, M., & Nozza, D. (2018), “Towards Encoding Time in Text-Based Entity Embeddings” in International
Semantic Web Conference (to appear), Monterey, California.
40. References
Bianchi, F., Palmonari, M., Cremaschi, M., & Fersini, E. (2017, May). Actively learning to rank semantic associations for
personalized contextual exploration of knowledge graphs. In European Semantic Web Conference (pp. 120-135). Springer,
Cham.
Bianchi, F., & Palmonari, M. (2017). Joint learning of entity and type embeddings for analogical reasoning with entities. In In
Proceedings of the NL4AI Workshop, co-located with the International Conference of the Italian Association for Artificial
Intelligence (AI* IA).
42. Qualitative Evaluation of Time Flattened Similarity
Winston Churchill Harold Macmillan
Tony Blair
Gordon Brown
Most similar 49th in
the list of
most
similars
41st in
the list of
most
similars
Method: Cosine similarity
Input: Winston Churchill
43. Qualitative Evaluation of Time Flattened Similarity
Winston Churchill Margaret Thatcher
Tony Blair
Gordon Brown
Most similar 16th in
the list of
most
similars
14th in
the list of
most
similars
Method: Time-flattened Similarity
Input: Winston Churchill