Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Detecting, Analyzing and Representing Vagueness in 
Ontologies for Facilitating Data Reuse 
Dr. Panos Alexopoulos 
Semanti...
What will I talk about 
Vagueness and Ontologies 
Research Questions and (some) Answers 
Ongoing & Future Work
What will I talk about 
Vagueness and Ontologies 
Research Questions and (some) Answers 
Ongoing & Future Work
Vagueness 
„Vagueness is a semantic phenomenon where predicates admit 
borderline cases, namely cases where it is not dete...
Where does vagueness appear in an ontology? 
Vague Classes 
●A class is vague if it admits 
borderline cases, i.e. if ther...
Vagueness in human communication 
I am telling you this is a 
strategic client for the firm 
with large-budget projects! 
...
When is vagueness in an ontology a problem? 
● Defining instances: Vagueness will cause disagreement among experts 
when d...
What will I talk about 
Vagueness and Ontologies 
Research Questions and (some) Answers 
Ongoing & Future Work
Research Questions 
Can we automatically detect vague ontological 
definitions? 
Vagueness 
Detection 
Vagueness 
Measurem...
Q1: Vagueness Detection 
Problem Definition 
● Can we automatically determine whether an ontology entity (class, 
relation...
Q1: Vagueness Detection 
Approach 
● We train a supervised classifier that may distinguish between vague 
and non-vague wo...
Q1: Vagueness Detection 
Data 
● 2,000 adjective senses from WordNet. 
● 1,000 vague 
● 1,000 non-vague 
Vague Senses Non ...
Q1: Vagueness Detection 
Training & Evaluation 
● 80% of the data used to train a multinomial Naive Bayes classifier. 
● W...
Q1: Vagueness Detection 
Comparison with Subjectivity Analyzer 
● We also used a subjective sense classifier to classify t...
Q1: Vagueness Detection 
Detecting Vagueness in CiTO Ontology 
● As an ontology use case we considered CiTO, an ontology t...
Q1: Vagueness Detection 
Detecting Vagueness in CiTO Ontology 
Vague Relations Non Vague Relations 
• plagiarizes: A prope...
Q1: Vagueness Detection 
Detecting Vagueness in CiTO Ontology 
● Classification Results: 
● 82% relations were correctly c...
Q2: Vagueness Measurement 
Problem Definition & Proposal 
● Can we use metrics to quantify the existence and importance of...
Q2: Vagueness Measurement 
Vagueness Spread 
● The ratio of the number of ontological elements (classes, relations and 
da...
Q2: Vagueness Measurement 
Vagueness Intensity 
● The degree to which the ontology's users disagree on the validity of the...
Q3: Vagueness Explicitness 
Problem Definition 
● Can we make the intended meaning of vague elements in ontologies 
more e...
Q3: Vagueness Explicitness 
Our Proposal: Vagueness Aware Ontologies 
Vagueness-aware ontologies are “ontologies 
whose va...
Q3: Vagueness Explicitness 
What should be made explicit? 
Vagueness Existence E.g. “Tall Person” is vague. 
E.g. “Low Bud...
Q3: Vagueness Explicitness 
The Vagueness Ontology
Vagueness Ontology 
Supported Competency Questions 
● Q1: What entities have been explicitly defined either as vague or no...
Vagueness Ontology 
Example Scenario 
● The object property isExpertInResearchArea is considered 
vague by John Doe in the...
Vagueness Ontology 
Example Scenario
Vagueness Ontology 
Intended Usage 
● Vagueness Ontology is meant to be used by both producers and 
consumers of ontologie...
Vagueness Ontology 
Usage and Benefits 
Vagueness Ontology Usage Expected Benefits 
Structuring Data 
with a Vague 
Ontolo...
Vagueness Ontology 
Documentation and Resources 
Documentation: http://www.essepuntato.it/2013/10/vagueness/documentation....
What will I talk about 
Vagueness and Ontologies 
Research Questions and (some) Answers 
Ongoing & Future Work
Ongoing and Future Work 
Creation of Vagueness-Aware Ontologies 
● Develop ontology authoring tool that: 
1. Takes as inpu...
Ongoing and Future Work 
Vagueness-Based Evaluation of Ontologies 
● Refine/expand/evaluate the current vagueness metrics ...
Thank you for your attention! 
Dr. Panos Alexopoulos 
Semantic Applications Research Manager 
Email: palexopoulos@isoco.co...
Upcoming SlideShare
Loading in …5
×

Detecting, Measuring and Representing Vagueness in Ontologies

631 views

Published on

Detecting, Measuring and Representing Vagueness in Ontologies

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Detecting, Measuring and Representing Vagueness in Ontologies

  1. 1. Detecting, Analyzing and Representing Vagueness in Ontologies for Facilitating Data Reuse Dr. Panos Alexopoulos Semantic Applications Research Manager iSOCO S.A. University of Aberdeen 1/4/2014 1 l S eptember 10, 2014
  2. 2. What will I talk about Vagueness and Ontologies Research Questions and (some) Answers Ongoing & Future Work
  3. 3. What will I talk about Vagueness and Ontologies Research Questions and (some) Answers Ongoing & Future Work
  4. 4. Vagueness „Vagueness is a semantic phenomenon where predicates admit borderline cases, namely cases where it is not determinately true that the predicate applies or not“ [Shapiro, 2006] This happens when predicates have blurred boundaries: •What’s the threshold number of years separating old and not old films? •What are the exact criteria that distinguish modern restaurants from non-modern?
  5. 5. Where does vagueness appear in an ontology? Vague Classes ●A class is vague if it admits borderline cases, i.e. if there are (or could be) individuals for which it is indeterminate whether they instantiate the class. ● “TallPerson”, “StrategicClient” Vague Datatypes Vague Relations ●A relation is vague if there are (or could be) pairs of individuals for which it is indeterminate whether they stand in the relation. ● “isExpertInSubject”, “belongsToGenre” ●Datatypes whose value range consists of a set of vague terms. ● E.g. “RestaurantPriceRange” when this comprises the terms “cheap”, “moderate” and “expensive”.
  6. 6. Vagueness in human communication I am telling you this is a strategic client for the firm with large-budget projects! Come on, $300,000 in two years is hardly a large budget!
  7. 7. When is vagueness in an ontology a problem? ● Defining instances: Vagueness will cause disagreement among experts when defining class and relation instances. ● Utilizing Vague Facts in Ontology-Based Systems: Reasoning results might not meet users’ expectations. ● Integrating Vague Semantic Information: The merging of particular vague elements can lead to data that will not be valid for all its users. ● Reusing Vague Datasets: The intended meaning of the dataset’s vague elements might not be compatible to the one needed in a particular application context.
  8. 8. What will I talk about Vagueness and Ontologies Research Questions and (some) Answers Ongoing & Future Work
  9. 9. Research Questions Can we automatically detect vague ontological definitions? Vagueness Detection Vagueness Measurement Vagueness Explicitness Can we quantify the amount and importance of vagueness in an ontology and/or a dataset? Can we make the intended meaning of vague elements in ontologies more explicit than it typically is?
  10. 10. Q1: Vagueness Detection Problem Definition ● Can we automatically determine whether an ontology entity (class, relation etc.) is vague or not? ● For example, “StrategicClient” as “A client that has a high value for the company” is vague! ● “AmericanCompany” as “A company that has legal status in the Unites States” is not!
  11. 11. Q1: Vagueness Detection Approach ● We train a supervised classifier that may distinguish between vague and non-vague word senses, using corresponding examples from Wordnet. ● We use this classifier to determine whether a given ontology element definition is vague or not.
  12. 12. Q1: Vagueness Detection Data ● 2,000 adjective senses from WordNet. ● 1,000 vague ● 1,000 non-vague Vague Senses Non Vague Senses • Abnormal: not normal, not typical or usual or regularor conforming to a norm ● Inter-agreement of vague/non-vague annotation among 3 human judges was 0.64 (Cohen’s Kappa) • Compound: composed of more than one part • Impenitent: impervious to moral persuasion • Biweekly: occurring every two weeks. • Notorious: known widely and usually unfavorably • Irregular: falling below the manufacturer's standard • Aroused: emotionally aroused • Outermost: situated at the farthest possible point from a center.
  13. 13. Q1: Vagueness Detection Training & Evaluation ● 80% of the data used to train a multinomial Naive Bayes classifier. ● We removed stop words and we used the bag of words assumption to represent each instance. ● The remaining 20% of the data was used as a test set. ● Classification accuracy was 84%!
  14. 14. Q1: Vagueness Detection Comparison with Subjectivity Analyzer ● We also used a subjective sense classifier to classify the dataset’s senses as subjective or objective. ● From the 1000 vague senses, only 167 were classified as subjective while from the 1000 non-vague ones 993. ● This shows that treating vagueness in the same way as subjectiveness is not really effective.
  15. 15. Q1: Vagueness Detection Detecting Vagueness in CiTO Ontology ● As an ontology use case we considered CiTO, an ontology that enables characterization of the nature or type of citations. ● CiTO consists primarily of relations, many of which are vague (e.g., plagiarizes). ● We selected 44 relations and we had 3 human judges manually classify them as vague or not. ● Then we applied the Wordnet-trained vagueness classifier on the textual definitions of the same relations.
  16. 16. Q1: Vagueness Detection Detecting Vagueness in CiTO Ontology Vague Relations Non Vague Relations • plagiarizes: A property indicating that the author of the citing entity plagiarizes the cited entity, by including textual or other elements from the cited entity without formal acknowledgement of their source • sharesAuthorInstitutionWith: Each entity has at least one author that shares a common institutional affiliation with an author of the other entity • citesAsAuthority: The citing entity cites the cited entity as one that provides an authoritative description or definition of the subject under discussion. • providesDataFor: The cited entity presents data that are used in work described in the citing entity.
  17. 17. Q1: Vagueness Detection Detecting Vagueness in CiTO Ontology ● Classification Results: ● 82% relations were correctly classified as vague/non-vague ● 94% accuracy for non-vague relations. ● 74% accuracy for vague relations. ● Again, we classified the same relations with the subjectivity classifier: ● 40% of vague/non-vague relations were classified as subjective/objective respectively. ● 94% of non-vague were classified as objective. ● 7% of vague relations were classified as subjective.
  18. 18. Q2: Vagueness Measurement Problem Definition & Proposal ● Can we use metrics to quantify the existence and importance of vagueness in ontologies and semantic data? ● E.g. can we somehow claim that ontology A is more vague than ontology B and have a number to support this claim? ● Some initial metrics we have thought: ● Vagueness Spread ● Vagueness Intensity
  19. 19. Q2: Vagueness Measurement Vagueness Spread ● The ratio of the number of ontological elements (classes, relations and datatypes) that are vague, divided by the total number of elements. ● This metric reflects the extent to which vagueness is present in the ontology. ● It also provides an indication of the ontology's potential comprehensibility and shareability: ● An ontology with a high value of vagueness spread is less explicit and shareable than an ontology with a low value. ● Calculation of this metric can be done semi-automatically using the vagueness detector we have described.
  20. 20. Q2: Vagueness Measurement Vagueness Intensity ● The degree to which the ontology's users disagree on the validity of the (potential) instances of the ontology elements. ● The higher this disagreement is for an element, the more problems the element is likely to cause. ● Metric Calculation: ● Consider a sample set of vague element instances, ● Have potential ontology users denote whether and to what extent they believe these instances are valid ● Measure the inter-agreement between users (e.g. by using Cohen’s kappa)
  21. 21. Q3: Vagueness Explicitness Problem Definition ● Can we make the intended meaning of vague elements in ontologies more explicit than it typically is? ● For example, ontologies typically do not explicitly state which of their elements are vague and which are not. ● Why is this a problem? ● “Fat Person” as “A person weighing many kilos” is vague! ● “Fat Person” as “A person with a BMI greater than 25” is not! ● What are the odds that a typical ontology user will immediately see this distinction without prior notice?
  22. 22. Q3: Vagueness Explicitness Our Proposal: Vagueness Aware Ontologies Vagueness-aware ontologies are “ontologies whose vague entities are accompanied by meta-information that describes the nature and characteristics of their vagueness”
  23. 23. Q3: Vagueness Explicitness What should be made explicit? Vagueness Existence E.g. “Tall Person” is vague. E.g. “Low Budget” has quantitative vagueness while “Expert Consultant” qualitative. “E.g. “Strategic Client" is vague in the dimension of the generated revenue” “E.g. “Strategic Client" is vague in the dimension of the generated revenue in the context of Financial Reporting” E.g. “Strategic Client" is vague in the dimension of the generated revenue according to the Financial Manager. Vagueness Type Vagueness dimensions Applicability Context Entity Creator
  24. 24. Q3: Vagueness Explicitness The Vagueness Ontology
  25. 25. Vagueness Ontology Supported Competency Questions ● Q1: What entities have been explicitly defined either as vague or non-vague? ● Q2: What entities are vague, in what contexts and according to whom? ● Q3: What entities have been defined both as vague and non-vague at the same time and why? ● Q4: What entities of a specific type (e.g., classes) have been defined either as vague or non-vague? ● Q5: What entities are characterised by a specific vagueness type? ● Q6: What entities have quantitative vagueness, in what dimensions and what is the context of their dimensions (if any)?
  26. 26. Vagueness Ontology Example Scenario ● The object property isExpertInResearchArea is considered vague by John Doe in the context of researcher hiring. ● Moreover, it is quantitatively vague since, for him, expertise is judged by the number of publications and projects. ● These two different dimensions he thinks are applicable in different contexts: ● Number of relevant publications in Academia ● Number of relevant projects in Industry.
  27. 27. Vagueness Ontology Example Scenario
  28. 28. Vagueness Ontology Intended Usage ● Vagueness Ontology is meant to be used by both producers and consumers of ontologies and semantic data. ● The former to annotate the vague part of their produced ontologies with relevant vagueness metainformation ● The latter to query this metainformation and use it to make a better use of the data. ● The annotation task should ideally take place in the course of the ontology's construction and evolution process.
  29. 29. Vagueness Ontology Usage and Benefits Vagueness Ontology Usage Expected Benefits Structuring Data with a Vague Ontology • Communicate the meaning of the vague elements to the domain experts. • Use the metamodel to characterize the created data's vagueness. • Make the job of the experts easier and faster and reduce disagreements among them. Utilizing Vague Semantic Data in an Ontology-Based System • Check which data is vague. • Use the properties of the vague elements to provide vagueness-related explanations to the users. • Know a priori which data may affect the system's effectiveness. Integrating Vague Semantic Datasets • Compare same vague elements across datasets according to their vagueness type and dimensions • Know a priori which data may affect the system's effectiveness. Evaluating Vague Semantic Datasets for Reuse • Query the metamodel to check the vagueness compatibility of the dataset with the intended domain and application scenario. • Avoid re-using (parts of) datasets that are not compatible to own interpretation of vagueness.
  30. 30. Vagueness Ontology Documentation and Resources Documentation: http://www.essepuntato.it/2013/10/vagueness/documentation.html Examples: http://www.essepuntato.it/2013/10/vagueness/examples.html
  31. 31. What will I talk about Vagueness and Ontologies Research Questions and (some) Answers Ongoing & Future Work
  32. 32. Ongoing and Future Work Creation of Vagueness-Aware Ontologies ● Develop ontology authoring tool that: 1. Takes as input an ontology. 2. Detects automatically vague entities. 3. Guides the user into annotating them with the Vagueness Ontology in a Q&A manner. 4. Gives as output a vagueness annotation for the given ontology.
  33. 33. Ongoing and Future Work Vagueness-Based Evaluation of Ontologies ● Refine/expand/evaluate the current vagueness metrics we have defined. ● Devise methods and tools for their effective and efficient calculation. Reasoning with Vagueness-Aware Ontologies ● Identify vagueness-related reasoning tasks that are useful when working with vague ontologies. ● Expand/revise the Vagueness Ontology accordingly.
  34. 34. Thank you for your attention! Dr. Panos Alexopoulos Semantic Applications Research Manager Email: palexopoulos@isoco.com Web: www.panosalexopoulos.com LinkedIn: www.linkedin.com/in/panosalexopoulos Twitter: @PAlexop

×