Distant supervision for relation extraction without labeled data Mike Mintz, Steven Bills, Rion Snow, Dan Jurafsky ACL 2009 I introduced this paper at NAIST Machine Translation Study Group.

1. Distant supervision
for relation extraction
without labeled data
Mike Mintz, Steven Bills, Rion Snow, Dan Jurafsky
ACL 2009
Introduced by Makoto Morishita

2. Contribution of this paper
• Proposed “distant supervision” for the first
time.
• By using distant supervision,
we can extract the relation between entities
from the sentences without annotation work.
2

3. Current training methods
• Supervised learning
• Unsupervised learning
• Self-training
• Active learning
3

4. Supervised learning
• Use only annotated data to train a model.
• Need a heavy cost to make the data.
4
Annotated data

5. Unsupervised learning
5
• Use only unannotated data.
• The result may not be suitable for some purposes.
Unannotated
data

6. Self-training
6
• Use annotated data for the seed of training model, then annotate
the unlabeled data by myself.
• It may be low precision and have a bias from the annotated data.
Unannotated data
Annotated
data

7. Active learning
7
• Use existing model to evaluate what data we
want to next, then annotate the selected data.
Unannotated data
Annotated
data
Evaluate
Annotate

8. Distant supervision
8
• We use existing database and unannotated data
to train classifier, then annotate the new data.
Unannotated
data
Classifier
Unannotated
data
Existing database
train
train
annotate

9. In this paper…

10. 10

11. 11

12. What we want to do
• Extract the relation between entities from
sentences.
• e.g.
sentence: Kyoto, the famous place in Japan.
entity: Japan, Kyoto
relation: location-contains <Japan, Kyoto>
12

13. In this work…
13
• Freebase: 102 relations, 940k entities,
1.8M instances.
Unannotated
data
Classifier
Unannotated
data
Freebase
train
train
annotate
Wikipedia
Multiclass logistic
regression classifier
Wikipedia

14. Freebase
14

15. Training
• Find the sentence that contains two entities.
- This sentence tends to express the relation.
- Entities are found by a named entity tagger.
• Train classifier.
- I will explain the features later.
15

16. Example
• Known relation:
location-contains <Virginia, Richmond>
location-contains <France, Nantes>
• We found the sentences like:
- Richmond, the capital of Virginia.
- Henry’s Edict of Nantes helped the
Protestants of France.
• Train the classified using these sentences.
16

17. Testing
• Find the sentence that contains two entities.
- This sentence tends to express the relation.
- Entities are found by a named entity tagger.
• Using trained classifier, we can know these
entities have a relation.
17

18. Features
• Lexical features:
- specific words between and surrounding
the two entities in the sentence.
• Syntactic features:
- dependency path
18

19. Lexical features
• The sequence of words between the two entities.
• The part-of-speech tags of these words.
• A flag indication which entity came first in the sentence.
• A window of k words to the left of Entity 1 and their part-of-speech tags.
• A window of k words to the right of Entity 2 and their part-of-speech tags.
19
Astronomer Edwin Hubble was born in Marshfield, Missouri.

20. Syntactic features
20
• A dependency path between the two entities.
• For each entity, one “window” node that is not part of the dependency path.

21. Result

22. Trained features
22

23. Automatic evaluation
23

24. Human evaluation
24

25. Conclusion
• By using this method, we can extract the
relation from unlabeled texts.
• By using database, the label is suit for the
current database.
• Extracted relations are seemed to be
accurate.
25

26. Example usage of distant supervision
26
Existing database Target annotation
Freebase
(relation between entities)
Wikipedia sentences
(find new relations)
Emoticon
Tweet
(annotate positive, negative)
Dependency parse tree,
knowledge base
semantic parser

27. Comments
• Distant supervision can be useful for other
tasks.
- Currently, this method is used mainly for
relation extraction task.
• However, it supposes that we already have a
large database.
27

28. END