Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules
•
2 likes•547 views
Authors: Isabel Segura Bedmar, Doaa Samy, José L. Martínez Fernández, Paloma Martínez SemEval 2007, 4th International Workshop on Semantic Evaluations, (2007) Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (13)
Similar to Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules
Similar to Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules (20)
More from Grupo HULAT
More from Grupo HULAT (20)
Recently uploaded
Recently uploaded (20)
Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules
- 1. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 1 Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules Isabel Segura Bedmar, Doaa Samy, José L. Martínez Fernández, Paloma Martínez Universidad Carlos III de Madrid
- 2. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 2 Agenda Motivation Task 4 of SEMEVAL 2007: Classification of Semantic Relations Between Nominales The Automatic System The Set of Linguistic Rules Results Conclusions Future Work
- 3. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 3 Motivation Ontology Development to support Semantic Web has lead to the improvement of techniques capable of managing underlying semantic relations between concepts. Detecting semantic relations is a key issue in automating the ontology learning process. In Question Answering System, detecting semantic relations could help to find the exact answers for questions such as: What are the chemical components of aspirin? (Part-Whole). List of all causes from asthma (Cause-Effect) List of all country producer of olive oil” (Product-Producer)
- 4. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 4 SEMEVAL 2007 (ACL 2007). Task: Classification of Semantic Relations between Nominals Seven semantic relations: Cause-Effect (e.g., virus-flu) Instrument-User (e.g., laser-print) Product-Producer (e.g., honey-bee) Origin-Entity (e.g., rye-whiskey) Purpose-Tool (e.g., soup-pot) Part-Whole (e.g., wheel-car) Content-Container (e.g., apple-basket) For the relation Part-Whole, our automatic system was the first in this SEMEVAL task.
- 5. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 5 Training and Testing Data Put tea in a heat-resistant jug Put <e1>tea</e1> in a <e2>heat-resistant jug</e2> Query=“* in a *” WordNet(e1) = "tea%1:13:00::“ WordNet(e2) = "jug%1:06:00::“ Training sentences: Content-Container(e1, e2)=true Testing sentences: Content-Container(e1, e2)=?
- 6. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 6 The Automatic System
- 7. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 7 Features The set of features used for the classification of semantic relations includes information from different levels: Lexical Features Nominals (blue room) and their heads (room) The two word tokens occurring before, after the nominals and the list of words between both nominals Information regarding Preposition and Verb Query POS Features POS tags of tokens in the lexical features Path: concatenated POS tags of wordlist between nominals Semantic Features Lexical File numbers: act, animal, person, location, etc Synset number WordNet Vector: select the synsets of the third level of depth. If the synset is ancestor of the nominal, it is assigned the value 1, else it is assigned the value 0.
- 8. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 8 Rules The rules do not pretend to achieve a total coverage of the whole dataset, but to improve the results of the learning algorithm in cases where it is obvius to decide if the relation is true o false according to simple semantic and knwoledge constraints. The rules are based on: WordNet Information POS Lexical
- 9. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 9 Classification of Rules: Semantic rules Lexico-semantic rules Morphosyntactic rules Complex rules combining two or more of the above mentioned criteria.
- 10. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 10 Example of Rule for the Relation Cause-Effect IF (Cause = TIME OR Cause = LOCATION OR Cause = PERSON OR Cause = ANIMAL OR Cause = GROUP OR Cause = COMMUNICATIVE_ACT) THEN Relation(Cause- Effect)=False
- 11. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 11 Example of Rule for Product- Producer Relation Pattern: <PRODUCER> verb <PRODUCT> IF (verb (active voice) = {make| produce| manufacture|fabricate} {cultivate| grow| farm for} {develop| compose|prepare} {create| procreate} {come from} {provide} {emit} AND PRODUCT!=STATE AND PRODUCT!=ACTIVITY AND PRODUCT!=TIME Then Relation(Product-Producer)=TRUE
- 12. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 12 Coverage and Accuracy Relations Rules (num) Coverage Train Coverag e Test Acc. SVM Acc. SVM+ Rules Improvement Cause-Effect 4 25% 27% 56,2% 60,75% 4,5% Inst-Agency 3 8% 8% 69,2% 70,5% 1,3% Prod-Producer 7 50% 48% 60,2% 75% 15% Origin-Entity 15 36% 38% 65,4% 70,5% 5% Theme-Tool 6 38% 25% 53,5% 60% 6,7% Part-Whole 45 62% 36% 70,8% 76,38% 5.58% C-Container 15 16% 18% 60.8% 55.40% -5.4%
- 13. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 13
- 14. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 14 Conclusion Results obtained demonstrate that applying a hybrid approach on a supervised system achieves good results in a semantic classification task between nominals.
- 15. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 15 Future Work Wider use of semantic information such as hypernym for each synset or information regarding the entity features. Introducing more syntactic information through syntactic chunking could affect significantly the results. In the rule-based module, we consider increasing the set of rules for a broader coverage. At present, we use this approach in the biomedicine domain for detecting relations between proteins, genes, drugs, etc.
- 16. Detecting Semantic Relations between Nominals using Support Vector Machines and Linguistic-Based Rules 16 Thank you for you attention
Editor's Notes
- I’m talking about the improvement of an automatic system for detecting semantic relations between nominals by the use of linguistically motivated knowledge combined with machine learning techniques. I apologize because I’m not confident in English. May, I can’t understand you, when you ask me. Please, be patient /’peisant/ with me. I’m talking about a hybrid supervised system that combines SVM and a set of linguistic rules for detecting /ditekting/ semantic relations between nominals.
- Firstly, I’m talking about our motivation for detecting semantic relations. Secondly, I’m descrabing /discraibing/ the task Classification of Semantic Relations Between Nominals. This task was proposed by the International Workshop on Semantic Evaluation, SEMEVAL. And, I’m showing some examples of training and testing data for this task. After, I’m descrabing the automatic system that we developed [dI’velapt] for this task and I’m talking about the main features used for the classification. Then, I’m explaining / Iks’pleining/ the linguistic rules that we used to improve the performance of automatic system. And Finally, I’m showing the improvements / [Im’pruuvments] /and some conclusions and future work.
- We have two motivations: firstly, we think that detecting semantic relations is a crucial part in the automatic ontology learning. In addition, we think that the semantic relations could help to answer complex questions such as: What are the chemical /kemical/ components of aspirin Semantic classification proposed by this system can be integrated in a ontology building or in a Question Answering system . For QA systems, to answer complex questions, a deeper understanding of the question is necessary. For example, to answer questions such as “list of all causes from asthma” (cause-effect), “list of all country producer of olive oil” (product-producer) or “what are the chemical components of aspirin? (Part-Whole). Detecting these types of semantic relations could help to find the exact answer.
- The International Workshop on Semantic Evaluations, SEMEVAL , proposed the evaluation task Classification of Semantic Relations between nominals. SEMEVAL provided training and testing data for this task. The dataset included seven semantic relatons: Cause-Effect, Instrument-User, Product-Producer, Origin-Entity, Purpose-Tool, Part-Whole, Content-Container We participated [paatisipeitid] with a previous [‘priviuus] version using a support vector machines classifier. And our system was the best in the relation Part-Whole. SEMEVAL 2007, created a benchmark dataset and evaluation task, Classification of Semantic Relations between Nominals, enabling researches to compare their algorithms. In this task, a nominal is a noun, excluding named entities, or base noun phrase,. A base noun phrase is a noun and its premodifiers (nouns, adjetives, determiners). . In this task, a nominal is a noun, excluding named entities, or base noun phrase,. A base noun phrase is a noun and its premodifiers (nouns, adjetives, determiners).
- In this task, a nominal is a noun, excluding named entities, or base noun phrase,. A base noun phrase is a noun and its premodifiers (nouns, adjetives, determiners). Each sentence was manually annotated including the following information: (1) the nominal boundaries (2) the used query to search the sentence in the web (e.g., “* in a *”) (3) Wordnet sense key for each nominal (4) And in the case of training sentences, the result of relation: true o false. In this case, the result is true. For each relation, 140 training sentences, 70 testing sentences and a precise definition of the relation were provided by the organizers.
- We decided to use Support Vector Machine (SVM), as one of the most successful Machine Learning techniques, achieving the best performances for many classification tasks. Prior to the classification of semantic relations, characteristics of each sentence are automatically extracted using GATE. GATE is an infrastructure for developing and deploying software components for Language Engineering. We used the following GATE components: English Tokenizer, Part-Of-Speech (POS) tagger and Morphological analyser. The set of features used for the classification of semantic relations includes information from different levels: word tokens, POS tags, verb lemmas, semantic information from WordNet, etc. POS tags and lemmas are automatically extracted using the using GATE. Lexical features are related to both the nominals and the contextual information (two words before the first nominal, the two words after the second nominal and the word list in-between) At the semantic level, we used synset number and the lexical file number for each nominal. For the Part-Whole relation, we use an additional feature indicating the meronymy relation between the two nominals. The set of WordNet lexical files can help to determine if the nominals satisfy the restrictions for each relation. For example, in the relation Theme-Tool, the theme should be an object, an event, a state of being, an agent, or a substance. Else, it is possible to affirm that the relation is false.
- We trained support vector machines with the training data, adn we used the following feautures: In the lexical level, we used the nominals and their heds.
- Parallel to the work on training the machine learning model, a set of linguistic and knowledge based constraint rules were manually developed. the constraint rules do not pretend to achieve a total coverage of the whole dataset, but to improve the results of the learning algorithm in cases where it is obvious to decide if the relation is true or false according to simple semantic and knowledge constraints We counted on three basic sources of information, considered as basic criteria:- WordNet Information concerning synsets and lexical file numbers of each nominalin the WordNet structure.- POS information mainly lemmas, grammatical categories and lemmas, providedfrom the basic processing module.- Lexical information regarding the word tokens occurring in the sentences are alsoprovided in the basic processing module.It is important to highlight the fact that we did not use a syntactic analyzer or achunker. Thus, the syntactic information, included in the rules, consists basically ofsimple sequences of POS tags.
- Rules developed can be divided into four basic classes: 1) semantic rules dealing with the most abstract level. These rules depend mainly on the lexical file numbers. 2) Lexico-semantic rules where special lemmas and tokens of the contextual information are taken into account. 3) morphosyntactic rules where special patterns consisting of sequences of POS are considered.4) Complex rules combining two or more of the above mentioned criteria or addressing special grammatical phenomena such as the passive voice.Rules developed can be divided into four basic classes: 1) semantic rules dealing with the most abstract level. These rules depend mainly on the lexical file numbers. 2) Lexico-semantic rules where special lemmas and tokens of the contextual information are taken into account. 3) morphosyntactic rules where special patterns consisting of sequences of POS are considered.4) Complex rules combining two or more of the above mentioned criteria or addressing special grammatical phenomena such as the passive voice.
- An simple rule
- this relationship is frequently represented in English language through the use of transitive verbs indicating actions like: {make, produce, manufacture}, {develop, compose, prepare} or {create, procreate}. A complex rule is developed (combining lexical, semantical and syntactic through a sequence of POS tags) using morpho-syntactic and lexical information together with semantic information regarding the synsets of these verbs and simple semantic constraints indicating that PRODUCT cannot be TIME, STATE or ACTIVITY.
- Best results were obtained in the PRODUCT-PRODUCER relation where the improvement reached almost 15% raising the overall accuracy from 60.8% to 75%. This is due to the relatively high coverage of the rules (48%) and their high precision. In CAUSE-EFFECT, rules improved the results approximately 4.5% raising the overall accuracy from 56.2% to 60.75%. Again, the rules proved a high precision. For the INSTRUMENT-AGENCY, improvement was trivial (only 1.3%) and, thus, we believe that more rules should be implemented. Results for ORIGIN-ENTITY relation was improved by a percentage of 5% reaching an overall accuracy of 70.5% in comparison with 65.4% achieved by the machine learning algorithm alone. For the THEME-TOOL relation, enhancement obtained is almost 6% improving the total accuracy from 54.2% to 60%. It is also important to point out that the precision of the rules in this case was optimum 100% over the rule. Finally in the PART-WHOLE relation, rules covered 36% of the test data. The improvement obtained reaches 6% enhancing the total accuracy to 76.38% after 70.8% obtained by the machine learning module (for this relation, our system was the first in the SEMEVAL task#4). Results obtained demonstrate that applying a hybrid approach on a supervised system achieves good results in a semantic classification task between nominals. Improvement obtained varies from 1.2% till 15% over the total accuracy scores obtained by the support vector machine algorithm, except for the relation CONTENT-CONTAINER.
- Results obtained demonstrate that applying a hybrid approach on a supervised system achieves good results in a semantic classification task between nominals. Improvement obtained varies from 1.2% till 15% over the total accuracy scores obtained by the support vector machine algorithm, except for the relation CONTENT-CONTAINER.
- We want to introduce further training and enhancement. Improving results across the whole dataset requires wider use of semantic information such as hypernym for each synset or information regarding the entity features. Also, introducing more syntactic information through syntactic chunking could affect significantly the results. In the rule-based module, we consider increasing the set of rules for a broader coverage. Semantic classification proposed by this system can be integrated in a Question Answering system or ontology building. For QA systems, to answer complex questions, a deeper understanding of the question is necessary. For example, to answer questions such as “list of all causes from asthma” (cause-effect), “list of all country producer of olive oil” (product-producer) or “what are the chemical components of aspirin? (Part-Whole). Detecting these types of semantic relations could help to find the exact answer.