Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning

•

0 likes•1,088 views

Poster for our EMNLP paper on extracting non-standard relations from the Web with distant supervision and imitation learning. Read the full paper here: https://aclweb.org/anthology/D/D15/D15-1086.pdf

Technology

Isabelle Augenstein, Andreas
Vlachos, Diana Maynard
Extracting Relations between Non-Standard Entities using
Distant Supervision and Imitation Learning
Approach Overview
Musical Artist Album
Michael Jackson Music & Me
The Beatles ?
Stealers Wheel ?
Sentences
Preview songs from Forever, Michael by Michael Jackson on the iTunes Store.
The only Beatles album to occasion negative, even hostile reviews, there are
few other rock records as controversial as Let It Be.
The Beatles recorded ten songs during a single studio session for their debut
LP, Please Please Me.
Stealers Wheel are a Scottish folk rock band formed in Paisley, Renfrewshire.
Web Search
Sentences
Music & Me is the third studio album by
American singer Michael Jackson.
Stealers Wheel are a Scottish folk rock
band formed in Paisley, Renfrewshire.
NER
Training Instances
Label Subject Object
true Michael Jackson Forever, Michael
Sentence
Preview songs from
false Michael Jackson iTunes Store Preview songs from
Testing Instances
true The Beatles Let It Be The only Beatles album to
false The Beatles LP The Beatles recorded ten
true The Beatles Please Please Me The Beatles recorded ten
false Stealers Wheel Scottish Stealers Wheel are a
false Stealers Wheel Paisley, Renfrewshire Stealers Wheel are a
Distant
Supervision
NE Classifier
Relation Extractor
Train
Predict
Output
Subject Object
The Beatles Let It Be
The Beatles Please Please Me
Stealers Wheel
Distant Supervision
Train relation extractors without manually labeled
data, using a knowledge base
and unlabeled text
How can we recognise arguments of
relations?
Named Entity Recognition (NER) and Named Entity Classification
(NEC) are typically used as part of preprocessing using tools such
as Stanford NER or FIGER
Is off-the-shelf NER good
enough?
•  Experiments with 16 relations (e.g. album,
character, record label, author, origin)
Recall of Stanford NER compared to simple
POS-based heuristics
Improving NEC for RE: Imitation Learning
•  Simple solution (OS): adding NEC features to RE
•  Problem: NEC features (e.g. mention, mention context)
overpower RE features (e.g. path between s and o)
Ø  OS would incorrectly predict Steven Spielberg,
because context is stronger
•  Solution (IL): Decomposing the learning task into series of
actions: NEC, then RE if NEC prediction is positive
•  Classifiers are trained iteratively with imitation learning
algorithm DAGGER (Ross et al., 2011)
•  NEC stage is fairly permissive and enhances RE
Ø  NEC prediction for both candidates is positive
Ø  RE correctly predicts Alfred Hitchcock
Improving NEC for RE: Web Features
Results
0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

P-‐avg

Rel
only

Stanf

FIGER

OS

IL

One of director Steven Spielberg’s greatest heroes was
Alfred Hitchcock, the mastermind behind Psycho.
Candidates for director relation with subject Psycho:
Steven Spielberg, Alfred Hitchcock
Arctic Monkeys
Arctic Monkeys are a rock band from Sheffield,
famous for albums such as AM.
Albums:
- Whatever People Say I Am, That's What I'm Not
- AM
header
link
bold
list
0
0.5
1

PER

LOC

ORG

MISC

Stanford
NER

HeurisHc

Conclusions
•  Imitation learning approach outperforms baselines with supervised NEC
(Stanford NER and FIGER) by 10 points in average precision
•  Web features such as appearance in lists or links to other Web improve
average precision by 7 points
•  Sparse, high-precision features (such as parse) outperform high-recall low-
precision features (such as BOW features)
References
Stéphane Ross, Geoffrey J. Gordon, and Drew Bagnell.
2011. A Reduction of Imitation Learning and Structured
Prediction to No-Regret Online Learning. JMLR.

More from Isabelle Augenstein

Most work on scholarly document processing assumes that the information processed is trustworthy and factually correct. However, this is not always the case. There are two core challenges, which should be addressed: 1) ensuring that scientific publications are credible -- e.g. that claims are not made without supporting evidence, and that all relevant supporting evidence is provided; and 2) that scientific findings are not misrepresented, distorted or outright misreported when communicated by journalists or the general public. I will present some first steps towards addressing these problems and outline remaining challenges.

Determining the Credibility of Science Communication

Isabelle Augenstein

Outline: - Fact checking – what is it and why do we need it? - False information online - Content-based automatic fact checking - Explainability – what is it and why do we need it? - Making the right predictions for the right reasons - Model training pipeline - Explainable fact checking – some first solutions - Rationale selection - Generating free-text explanations - Wrap-up

Towards Explainable Fact Checking (DIKU Business Club presentation)

Isabelle Augenstein

Tutorial on 'Explainability for NLP' given at the first ALPS (Advanced Language Processing) winter school: http://lig-alps.imag.fr/index.php/schedule/ The talk introduces the concepts of 'model understanding' as well as 'decision understanding' and provides examples of approaches from the areas of fact checking and text classification. Exercises to go with the tutorial are available here: https://github.com/copenlu/ALPS_2021

Explainability for NLP

Isabelle Augenstein

Automatic fact checking is one of the more involved NLP tasks currently researched: not only does it require sentence understanding, but also an understanding of how claims relate to evidence documents and world knowledge. Moreover, there is still no common understanding in the automatic fact checking community of how the subtasks of fact checking — claim check-worthiness detection, evidence retrieval, veracity prediction — should be framed. This is partly owing to the complexity of the task, despite efforts to formalise the task of fact checking through the development of benchmark datasets. The first part of the talk will be on automatically generating textual explanations for fact checking, thereby exposing some of the reasoning processes these models follow. The second part of the talk will be on re-examining how claim check-worthiness is defined, and how check-worthy claims can be detected; followed by how to automatically generate claims which are hard to fact-check automatically.

Towards Explainable Fact Checking

Isabelle Augenstein

Talk on 'Tracking False Information Online' at W-NUT workshop at EMNLP 2019. ========= Digital media enables fast sharing of information and discussions among users. While this comes with many benefits to today’s society, such as broadening information access, the manner in which information is disseminated also has obvious downsides. Since fast access to information is expected by many users and news outlets are often under financial pressure, speedy access often comes at the expense of accuracy, which leads to misinformation. Moreover, digital media can be misused by campaigns to intentionally spread false information, i.e. disinformation, about events, individuals or governments. In this talk, I will present on different ways false information is spread online, including misinformation and disinformation. I will then report findings from our recent and ongoing work on automatic fact checking, stance detection and framing attitudes.

Tracking False Information Online

Isabelle Augenstein

One of the core challenges in typology is to record properties of languages in a structured way. As a result of manual efforts, typological knowledge bases have emerged, which contains information about languages’ phonological, morphological and syntactic properties; as well as information about language families. Ideally, such typological knowledge bases would provide useful information for multilingual NLP models to learn how to selectively share parameters. A related area of research suggests a different way of encoding properties of languages, namely to learn language representation vectors directly from text documents. In this talk, I will analyse and contrast these two ways of encoding linguistic properties, as well as present research on how the two can benefit one another.

What can typological knowledge bases and language representations tell us abo...

Isabelle Augenstein

Paper presented at NAACL 2018. Link: https://arxiv.org/abs/1802.09913 Abstract: ============ We combine multi-task learning and semi-supervised learning by inducing a joint embedding space between disparate label spaces and learning transfer functions between label embeddings, enabling us to jointly leverage unlabelled data and auxiliary, annotated datasets. We evaluate our approach on a variety of sequence classification tasks with disparate label spaces. We outperform strong single and multi-task baselines and achieve a new state-of-the-art for topic-based sentiment analysis.

Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate ...

Isabelle Augenstein

When labelled training data for certain NLP tasks or languages is not readily available, different approaches exist to leverage other resources for the training of machine learning models. Those are commonly either instances from a related task or unlabelled data. An approach that has been found to work particularly well when only limited training data is available is multi-task learning. There, a model learns from examples of multiple related tasks at the same time by sharing hidden layers between tasks, and can therefore benefit from a larger overall number of training instances and extend the models' generalisation performance. In the related paradigm of semi-supervised learning, unlabelled data as well as labelled data for related tasks can be easily utilised by transferring labels from labelled instances to unlabelled ones in order to essentially extend the training dataset. In this talk, I will present my recent and ongoing work in the space of learning with limited labelled data in NLP, including our NAACL 2018 papers 'Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spaces [1] and 'From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings’ [2]. [1] https://t.co/A5jHhFWrdw [2] https://arxiv.org/abs/1802.09375 ========== Bio from my website http://isabelleaugenstein.github.io/index.html: I am a tenure-track assistant professor at the University of Copenhagen, Department of Computer Science since July 2017, affiliated with the CoAStAL NLP group and work in the general areas of Statistical Natural Language Processing and Machine Learning. My main research interests are weakly supervised and low-resource learning with applications including information extraction, machine reading and fact checking. Before starting a faculty position, I was a postdoctoral research associate in Sebastian Riedel's UCL Machine Reading group, mainly investigating machine reading from scientific articles. Prior to that, I was a Research Associate in the Sheffield NLP group, a PhD Student in the University of Sheffield Computer Science department, a Research Assistant at AIFB, Karlsruhe Institute of Technology and a Computational Linguistics undergraduate student at the Department of Computational Linguistics, Heidelberg University.

Learning with limited labelled data in NLP: multi-task learning and beyond

Isabelle Augenstein

Spreading of mis- and disinformation is growing and is having a big impact on interpersonal communications, politics and even science. Traditional methods, e.g. manual fact-checking by reporters cannot keep up with the growth of information. On the other hand, there has been much progress in natural language processing recently, partly due to the resurgence of neural methods. How can natural language processing methods fill this gap and help to automatically check facts? This talk will explore different ways to frame fact checking and detail our ongoing work on learning to encode documents for automated fact checking, as well as describe future challenges.

Learning to read for automated fact checking

Isabelle Augenstein

Shared task summary for SemEval 2017 Task 10: ScienceIE – Extracting Keyphrases and Relations from Scientific Publications Paper: https://arxiv.org/abs/1704.02853 Abstract: We describe the SemEval task of extracting keyphrases and relations between them from scientific documents, which is crucial for understanding which publications describe which processes, tasks and materials. Although this was a new task, we had a total of 26 submissions across 3 evaluation scenarios. We expect the task and the findings reported in this paper to be relevant for researchers working on understanding scientific content, as well as the broader knowledge base population and information extraction communities.

SemEval 2017 Task 10: ScienceIE – Extracting Keyphrases and Relations from Sc...

Isabelle Augenstein

1st Workshop for Women and Underrepresented Minorities (WiNLP) at ACL 2017 - ...

Isabelle Augenstein

1st Workshop for Women and Underrepresented Minorities (WiNLP) at ACL 2017 - ...

Isabelle Augenstein

Machine Reading Using Neural Machines (talk at Microsoft Research Faculty Sum...

Isabelle Augenstein

Presentation of work that will be published at EMNLP 2016. Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak, Sebastian Riedel. emoji2vec: Learning Emoji Representations from their Description. SocialNLP at EMNLP 2016. https://arxiv.org/abs/1609.08359 Georgios Spithourakis, Isabelle Augenstein, Sebastian Riedel. Numerically Grounded Language Models for Semantic Error Correction. EMNLP 2016. https://arxiv.org/abs/1608.04147 Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, Kalina Bontcheva. Stance Detection with Bidirectional Conditional Encoding. EMNLP 2016. https://arxiv.org/abs/1606.05464

Weakly Supervised Machine Reading

Isabelle Augenstein

This paper describes the University of Sheffield's submission to the SemEval 2016 Twitter Stance Detection weakly supervised task (SemEval 2016 Task 6, Subtask B). In stance detection, the goal is to classify the stance of a tweet towards a target as "favor", "against", or "none". In Subtask B, the targets in the test data are different from the targets in the training data, thus rendering the task more challenging but also more realistic. To address the lack of target-specific training data, we use a large set of unlabelled tweets containing all targets and train a bag-of-words autoencoder to learn how to produce feature representations of tweets. These feature representations are then used to train a logistic regression classifier on labelled tweets, with additional features such as an indicator of whether the target is contained in the tweet. Our submitted run on the test data achieved an F1 of 0.3270. Paper: http://isabelleaugenstein.github.io/papers/SemEval2016-Stance.pdf

USFD at SemEval-2016 - Stance Detection on Twitter with Autoencoders

Isabelle Augenstein

Invited talk in Heriot-Watt University Computer Science seminar series about our EMNLP paper on extracting non-standard relations from the Web with Distant Supervision and Imitation Learning. Read the full paper here: https://aclweb.org/anthology/D/D15/D15-1086.pdf View the EMNLP poster here: http://www.slideshare.net/isabelleaugenstein/extracting-relations-between-nonstandard-entities-using-distant-supervision-and-imitation-learning

Distant Supervision with Imitation Learning

Isabelle Augenstein

Information Extraction with Linked Data

Isabelle Augenstein

Lodifier: Generating Linked Data from Unstructured Text

Isabelle Augenstein

Relation Extraction from the Web using Distant Supervision

Isabelle Augenstein

Natural Language Processing for the Semantic Web

Isabelle Augenstein

More from Isabelle Augenstein (20)

Determining the Credibility of Science Communication

Towards Explainable Fact Checking (DIKU Business Club presentation)

Explainability for NLP

Towards Explainable Fact Checking

Tracking False Information Online

What can typological knowledge bases and language representations tell us abo...

Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate ...

Learning with limited labelled data in NLP: multi-task learning and beyond

Learning to read for automated fact checking

SemEval 2017 Task 10: ScienceIE – Extracting Keyphrases and Relations from Sc...

1st Workshop for Women and Underrepresented Minorities (WiNLP) at ACL 2017 - ...

Machine Reading Using Neural Machines (talk at Microsoft Research Faculty Sum...

Weakly Supervised Machine Reading

USFD at SemEval-2016 - Stance Detection on Twitter with Autoencoders

Distant Supervision with Imitation Learning

Information Extraction with Linked Data

Lodifier: Generating Linked Data from Unstructured Text

Relation Extraction from the Web using Distant Supervision

Natural Language Processing for the Semantic Web

Recently uploaded

Exploring Multimodal Embeddings with Milvus

Zilliz

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Orbitshub

In the ever-evolving landscape of data management, Zero-ETL is an approach that is reshaping how businesses handle and integrate their data. This webinar explores Zero-ETL, a paradigm shift from the traditional Extract, Transform, Load (ETL) process, offering a more streamlined, efficient, and real-time data integration method. We will begin with an introduction to the concept of Zero-ETL, including how it allows direct access to data in its native environment and real-time data transformation, providing up-to-date information with significantly reduced data redundancy. Next, we'll take you through several demonstrations showing how Zero-ETL can deliver real-time data and enable the free movement of data between systems. We will also discuss the various tools that support all aspects of Zero-ETL, providing attendees with an understanding of how they can adopt this innovative approach in their organizations. Lastly, the session will conclude with an interactive Q&A segment, allowing participants to gain deeper insights into how Zero-ETL can be tailored to their specific business needs and how they can get started today. Join us to discover how Zero-ETL can elevate your organization's data strategy.

The Zero-ETL Approach: Enhancing Data Agility and Insight

Safe Software

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

How to Check CNIC Information Online with Pakdata cf

danishmna97

The integration landscape is changing rapidly with the introduction of technologies like GraphQL, gRPC, stream processing, iPaaS, and platformless. However, not all existing applications and industries can keep up with these new technologies. Certain industries, like manufacturing, logistics, and finance, still rely on well-established EDI-based message formats. Some applications use XML or CSV with file-based communications, while others have strict on premises deployment requirements. This talk focuses on how Ballerina's built-in integration capabilities can bridge the gap between "old" and "new" technologies, modernizing enterprise applications without disrupting business operations.

Modernizing Legacy Systems Using Ballerina

WSO2

Stronger Together: Developing an Organizational Strategy for Accessible Desig...

caitlingebhard1

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

Dubai, known for its towering skyscrapers, luxurious lifestyle, and relentless pursuit of innovation, often finds itself in the global spotlight. However, amidst the glitz and glamour, the emirate faces its own set of challenges, including the occasional threat of flooding. In recent years, Dubai has experienced sporadic but significant floods, disrupting normalcy and posing unique challenges to its infrastructure. Among the critical nodes in this bustling metropolis is the Dubai International Airport, a vital hub connecting the world. This article delves into the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Orbitshub

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

In this session, we will discuss the journey of API governance from its initial, ungoverned state to the development of sophisticated models that tackle contemporary challenges. We'll explore how APIs have become essential in the intersection of business and technology, adapting to advancements and evolving needs. We'll focus on how organizations have moved from launching to monetizing APIs, using models like pay-per-use and subscriptions, and finding the right balance between technical implementation and business strategy. We'll also highlight the impact of governance on monetization strategies, especially how data security, compliance, and service quality influence pricing. Real-world examples will demonstrate the effective integration of governance with monetization, including AI's role in dynamic pricing. Looking ahead, we'll share insights into future trends in API governance and monetization, emphasizing the importance of adaptability, continuous learning, and innovation.

API Governance and Monetization - The evolution of API governance

WSO2

In the dynamic field of DevOps, the quest for efficiency and productivity is endless. This talk introduces a revolutionary toolkit: Large Language Models (LLMs), including ChatGPT, Gemini, and Claude, extending far beyond traditional coding assistance. We'll explore how LLMs can automate not just code generation, but also transform day-to-day operations such as crafting compelling cover letters for TPS reports, streamlining client communications, and architecting innovative DevOps solutions. Attendees will learn effective prompting strategies and examine real-life use cases, demonstrating LLMs' potential to redefine productivity in the DevOps landscape. Join us to discover how to harness the power of LLMs for a comprehensive productivity boost across your DevOps activities.

ChatGPT and Beyond - Elevating DevOps Productivity

VictorSzoltysek

In this keynote, Asanka Abeysinghe, CTO,WSO2 will explore the shift towards platformless technology ecosystems and their importance in driving digital adaptability and innovation. We will discuss strategies for leveraging decentralized architectures and integrating diverse technologies, with a focus on building resilient, flexible, and future-ready IT infrastructures. We will also highlight WSO2's roadmap, emphasizing our commitment to supporting this transformative journey with our evolving product suite.

Platformless Horizons for Digital Adaptability

WSO2

DBX First Quarter 2024 Investor Presentation

Dropbox

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard37

Recently uploaded (20)

Exploring Multimodal Embeddings with Milvus

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

The Zero-ETL Approach: Enhancing Data Agility and Insight

AWS Community Day CPH - Three problems of Terraform

How to Check CNIC Information Online with Pakdata cf

Modernizing Legacy Systems Using Ballerina

Stronger Together: Developing an Organizational Strategy for Accessible Desig...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Introduction to Multilingual Retrieval Augmented Generation (RAG)

WSO2's API Vision: Unifying Control, Empowering Developers

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Corporate and higher education May webinar.pptx

API Governance and Monetization - The evolution of API governance

ChatGPT and Beyond - Elevating DevOps Productivity

Platformless Horizons for Digital Adaptability

DBX First Quarter 2024 Investor Presentation

JohnPollard-hybrid-app-RailsConf2024.pptx

Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning

1. Isabelle Augenstein, Andreas Vlachos, Diana Maynard Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning Approach Overview Musical Artist Album Michael Jackson Music & Me The Beatles ? Stealers Wheel ? Sentences Preview songs from Forever, Michael by Michael Jackson on the iTunes Store. The only Beatles album to occasion negative, even hostile reviews, there are few other rock records as controversial as Let It Be. The Beatles recorded ten songs during a single studio session for their debut LP, Please Please Me. Stealers Wheel are a Scottish folk rock band formed in Paisley, Renfrewshire. Web Search Sentences Music & Me is the third studio album by American singer Michael Jackson. Stealers Wheel are a Scottish folk rock band formed in Paisley, Renfrewshire. NER Training Instances Label Subject Object true Michael Jackson Forever, Michael Sentence Preview songs from false Michael Jackson iTunes Store Preview songs from Testing Instances true The Beatles Let It Be The only Beatles album to false The Beatles LP The Beatles recorded ten true The Beatles Please Please Me The Beatles recorded ten false Stealers Wheel Scottish Stealers Wheel are a false Stealers Wheel Paisley, Renfrewshire Stealers Wheel are a Distant Supervision NE Classifier Relation Extractor Train Predict Output Subject Object The Beatles Let It Be The Beatles Please Please Me Stealers Wheel Distant Supervision Train relation extractors without manually labeled data, using a knowledge base and unlabeled text How can we recognise arguments of relations? Named Entity Recognition (NER) and Named Entity Classification (NEC) are typically used as part of preprocessing using tools such as Stanford NER or FIGER Is off-the-shelf NER good enough? •  Experiments with 16 relations (e.g. album, character, record label, author, origin) Recall of Stanford NER compared to simple POS-based heuristics Improving NEC for RE: Imitation Learning •  Simple solution (OS): adding NEC features to RE •  Problem: NEC features (e.g. mention, mention context) overpower RE features (e.g. path between s and o) Ø  OS would incorrectly predict Steven Spielberg, because context is stronger •  Solution (IL): Decomposing the learning task into series of actions: NEC, then RE if NEC prediction is positive •  Classifiers are trained iteratively with imitation learning algorithm DAGGER (Ross et al., 2011) •  NEC stage is fairly permissive and enhances RE Ø  NEC prediction for both candidates is positive Ø  RE correctly predicts Alfred Hitchcock Improving NEC for RE: Web Features Results 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 P-‐avg Rel only Stanf FIGER OS IL One of director Steven Spielberg’s greatest heroes was Alfred Hitchcock, the mastermind behind Psycho. Candidates for director relation with subject Psycho: Steven Spielberg, Alfred Hitchcock Arctic Monkeys Arctic Monkeys are a rock band from Sheffield, famous for albums such as AM. Albums: - Whatever People Say I Am, That's What I'm Not - AM header link bold list 0 0.5 1 PER LOC ORG MISC Stanford NER HeurisHc Conclusions •  Imitation learning approach outperforms baselines with supervised NEC (Stanford NER and FIGER) by 10 points in average precision •  Web features such as appearance in lists or links to other Web improve average precision by 7 points •  Sparse, high-precision features (such as parse) outperform high-recall low- precision features (such as BOW features) References Stéphane Ross, Geoffrey J. Gordon, and Drew Bagnell. 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. JMLR.

Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning

Recommended

Recommended

More Related Content

More from Isabelle Augenstein

More from Isabelle Augenstein (20)

Recently uploaded

Recently uploaded (20)

Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning