A Topic map-based ontology IR system versus Clustering-based IR System: A Comparative Study in Security Domain Myongho Yi ...
Agenda 1. Related Works 2. Research Questions 3. Research Designs 4. Research Results  5. Conclusion
Background <ul><li>Many information organization approaches such as taxonomy, thesaurus, classification, and ontology have...
Three Information Org. Approaches Term Lists: Synonym Rings* Authority Files* Glossaries/Dictionaries Gazetteers* Natural ...
Clustering 2.0  <ul><li>Classification of data into different subtopic categories.  </li></ul><ul><li>Clustering shows rel...
Works in Medical Domain <ul><li>Less Polysemes </li></ul><ul><li>Mainly Hierarchical Relationships </li></ul><ul><li>Cance...
Norwegian Electronic Health Library United States Nat’l Lib of Medicine
How about Other Domains? <ul><li>Social Sciences </li></ul><ul><li>Humanities </li></ul><ul><li>Polysemes </li></ul><ul><l...
Clustering - Limitations <ul><li>Relevant results ? </li></ul><ul><ul><li>Loosely related associative relationships </li><...
Purpose of Study <ul><li>To measure the efficiency on representations of associative relationships </li></ul><ul><li>To co...
Related Works <ul><li>Yi (2008) </li></ul><ul><ul><li>Compared on Ontology based System to thesaurus based system </li></u...
Research Questions <ul><li>Are there recall/precision differences between TMIR and CIR?  </li></ul><ul><li>Are there searc...
Research Design <ul><li>Subjects </li></ul><ul><ul><li>Information Technology Major Students </li></ul></ul><ul><li>Data C...
Research Variables Independent Variables Dependent Variables Two Retrieval Systems Topic Map-Based Ontology  Information R...
Search Task Types Task # Degree of  Relationships Task 1 Simple Task List all the security software  2 Complex Task Name a...
Ontology Development Process <ul><li>Code using XML Topic Maps  (XTM) </li></ul><ul><li>Identify Equivalent Relationships ...
Ontology Modeling <ul><li>Associations </li></ul><ul><ul><li>Works for </li></ul></ul><ul><ul><li>Maintains </li></ul></ul...
Embed in <ul><li>Cryptography embed in Hardware </li></ul>
Two Retrieval Systems Compared <ul><li>Search for “Firewall” </li></ul><ul><li>Clustering-based Information Retrieval Syst...
Two Retrieval Systems Compared <ul><li>Clustering-based Information Retrieval System </li></ul>Firewall Software Listed No...
Two Retrieval Systems Compared <ul><li>Converted to the Identical Interface using Omnigator </li></ul>
Two Retrieval Systems Compared <ul><li>Search for “Firewall” </li></ul><ul><li>Topic Maps-based Information Retrieval Syst...
Two Retrieval Systems Compared <ul><li>Search for “Firewall” </li></ul><ul><li>Topic Maps-based Information Retrieval Syst...
Research Results <ul><li>There was a significant difference in recall between the two groups.  </li></ul><ul><li>The estim...
Discussion <ul><li>There were significant differences between the two groups and in terms of recall, precision, search tim...
Conclusion <ul><li>This study illustrates that the positive influences of a Topic map-based ontology IR system are improve...
Q & A <ul><li>Myongho Yi </li></ul><ul><li>Texas Woman’s University, TX, USA, myi@twu.edu </li></ul><ul><li>Sam Gyun Oh </...
Upcoming SlideShare
Loading in …5
×

A Topic map-based ontology IR system versus Clustering-based IR System: A Comparative Study in Security Domain

2,267 views

Published on

Due to the increasing amount and complexity of digital resources, there are several critical issues that arise in digital environments such as ill-structured and poor management of digital information. Different information organization approaches have been used to address these issues. In particular, Semantic Web has been explored for 10 years; however there are not many practical applications. This is in part due to the fact that much attention has been given to the creation rather than the migration of existing data. In addition, the lack of guidelines for choosing the right migration approach, whether Topic Maps or Resource Description Framework (RDF), needs to be addressed. This paper presents a comparison of Semantic Web Data Models (Topic Maps and RDF), followed by an example of migration of existing metadata into ontology-based data for Semantic Web.

Published in: Technology, Education
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
2,267
On SlideShare
0
From Embeds
0
Number of Embeds
72
Actions
Shares
0
Downloads
75
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

A Topic map-based ontology IR system versus Clustering-based IR System: A Comparative Study in Security Domain

  1. 1. A Topic map-based ontology IR system versus Clustering-based IR System: A Comparative Study in Security Domain Myongho Yi Texas Woman’s University, TX, USA, myi@twu.edu Sam Gyun Oh SungKyunKwan University, Seoul, Korea, samoh@skku.edu
  2. 2. Agenda 1. Related Works 2. Research Questions 3. Research Designs 4. Research Results 5. Conclusion
  3. 3. Background <ul><li>Many information organization approaches such as taxonomy, thesaurus, classification, and ontology have been attempted to provide effective searching. </li></ul><ul><li>Among them, clustering and ontology approaches have received much attention. However, there have not been many studies which compare in terms of user performance. </li></ul>
  4. 4. Three Information Org. Approaches Term Lists: Synonym Rings* Authority Files* Glossaries/Dictionaries Gazetteers* Natural language Controlled language Weakly-structured Strongly-structured Classification & Categorization: Subject Headings Classification schemes* Taxonomies* Categorization schemes Relationship Groups : Ontologies* Semantic networks* Concept maps* Thesauri* Pick lists* (Zeng, 2005)
  5. 5. Clustering 2.0 <ul><li>Classification of data into different subtopic categories. </li></ul><ul><li>Clustering shows related items according to their similarity. </li></ul><ul><li>Classify related search results into topic folders </li></ul><ul><li>Clustering 2.0 </li></ul><ul><ul><li>Remix clustering  </li></ul></ul><ul><ul><ul><li>Shows other subtle topics </li></ul></ul></ul>
  6. 6. Works in Medical Domain <ul><li>Less Polysemes </li></ul><ul><li>Mainly Hierarchical Relationships </li></ul><ul><li>Cancer </li></ul><ul><ul><li>Breast Cancer </li></ul></ul><ul><ul><li>Prostate Cancer </li></ul></ul><ul><ul><li>Colon Cancer </li></ul></ul><ul><ul><li>Lung Cancer </li></ul></ul><ul><ul><li>Skin Cancer </li></ul></ul><ul><ul><li>… . </li></ul></ul>
  7. 7. Norwegian Electronic Health Library United States Nat’l Lib of Medicine
  8. 8. How about Other Domains? <ul><li>Social Sciences </li></ul><ul><li>Humanities </li></ul><ul><li>Polysemes </li></ul><ul><li>Bank </li></ul><ul><ul><li>Financial institution </li></ul></ul><ul><ul><li>Rely on </li></ul></ul>
  9. 9. Clustering - Limitations <ul><li>Relevant results ? </li></ul><ul><ul><li>Loosely related associative relationships </li></ul></ul><ul><ul><ul><li>Same / Different category </li></ul></ul></ul><ul><ul><li>Examples </li></ul></ul><ul><ul><ul><li>Security </li></ul></ul></ul><ul><ul><ul><ul><li>Information Security </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Network Security </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>PGP </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Customers (?) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Valuable (?) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Other Topics (?) </li></ul></ul></ul></ul></ul>No classified? Loosely related terms? Term Lists?
  10. 10. Purpose of Study <ul><li>To measure the efficiency on representations of associative relationships </li></ul><ul><li>To compare the user performance of our Topic Maps-based method with the Clustering-based method. </li></ul>
  11. 11. Related Works <ul><li>Yi (2008) </li></ul><ul><ul><li>Compared on Ontology based System to thesaurus based system </li></ul></ul><ul><ul><ul><li>40 subjects, 8 queries, 2 types of queries, search time and recall </li></ul></ul></ul><ul><ul><ul><li>An ontology system showed a better recall and search time for relationship based queries </li></ul></ul></ul><ul><li>Oh (2006) </li></ul><ul><ul><li>Compared on Topic Map-Based Korean Folk Music (Pansori) Retrieval System (TMPRS) to Current Pansori Retrieval System (CPRS) </li></ul></ul><ul><ul><ul><li>Twenty LIS Students in Korea, 7 different search tasks and own query </li></ul></ul></ul><ul><ul><ul><li>TMPRS showed higher performance for objective and subjective measurements in general </li></ul></ul></ul><ul><li>E.K.F. Dang, Luk, Ho, Chan, & Lee, (2008) </li></ul><ul><ul><li>Clustering algorithms </li></ul></ul><ul><ul><li>Partitioning and hierarchical </li></ul></ul>
  12. 12. Research Questions <ul><li>Are there recall/precision differences between TMIR and CIR? </li></ul><ul><li>Are there search time differences between TMIR and CIR? </li></ul><ul><li>Are there search steps differences between TMIR and CIR? </li></ul>
  13. 13. Research Design <ul><li>Subjects </li></ul><ul><ul><li>Information Technology Major Students </li></ul></ul><ul><li>Data Collection </li></ul><ul><ul><li>Questionnaire </li></ul></ul><ul><ul><li>Screen Recording </li></ul></ul><ul><li>TMIR and CIR </li></ul><ul><ul><li>Topic Maps-based Security Information Retrieval (TMIR) system and Clustering-based Security Information Retrieval (CIR) system. </li></ul></ul>
  14. 14. Research Variables Independent Variables Dependent Variables Two Retrieval Systems Topic Map-Based Ontology Information Retrieval System Clustering-Based Information Retrieval System Quantitative Measurement Search steps, Search Time
  15. 15. Search Task Types Task # Degree of Relationships Task 1 Simple Task List all the security software 2 Complex Task Name all the Security engineers who work for Cisco 3 Complex Task Find Vendors providing security training service 4 Association and Cross Reference Related Task List all the security hardware supported by IBM Consultants 5 Association and Cross Reference Related Task List all software using RSA cryptography and find engineers who specialize in these software packages. 6 Association and Cross Reference Related Task Find security system engineers who specialize in firewall and their supervisor and sale representatives 7 Association and Cross Reference Related Task Assume that your organization is interested in security training. Who will be the right people to contact? Please provide their e-mail addresses
  16. 16. Ontology Development Process <ul><li>Code using XML Topic Maps (XTM) </li></ul><ul><li>Identify Equivalent Relationships </li></ul><ul><li>Identify Hierarchical Relationships </li></ul><ul><li>Identify Associative Relationships </li></ul><ul><li>- Same categories </li></ul><ul><li>- Different categories </li></ul>List Terms by Ontology Engineer Classify/Categorize by Ontology Engineer Add Semantic Relationships by Ontology Engineer Normalize by Domain Expert Implemented by Programmer <ul><li>Domain Experts </li></ul><ul><li>Identify index terms </li></ul><ul><li>List the index terms </li></ul><ul><li>Do not distinguish between preferred and non-preferred terms </li></ul><ul><li>Classify terms </li></ul><ul><li>Categorize terms </li></ul><ul><li>One term can be in multiple categories </li></ul><ul><li>Verify three relationships </li></ul><ul><li>Add additional relationships </li></ul>
  17. 17. Ontology Modeling <ul><li>Associations </li></ul><ul><ul><li>Works for </li></ul></ul><ul><ul><li>Maintains </li></ul></ul><ul><ul><li>Applied to </li></ul></ul><ul><ul><li>Embed in </li></ul></ul><ul><ul><li>Provides </li></ul></ul><ul><ul><li>Complies with </li></ul></ul><ul><ul><li>Designs </li></ul></ul><ul><ul><li>Makes </li></ul></ul><ul><ul><li>Provides </li></ul></ul>
  18. 18. Embed in <ul><li>Cryptography embed in Hardware </li></ul>
  19. 19. Two Retrieval Systems Compared <ul><li>Search for “Firewall” </li></ul><ul><li>Clustering-based Information Retrieval System </li></ul>Show Related Terms
  20. 20. Two Retrieval Systems Compared <ul><li>Clustering-based Information Retrieval System </li></ul>Firewall Software Listed No Related Information Provided Such as Vendor, Engineers for Firewall
  21. 21. Two Retrieval Systems Compared <ul><li>Converted to the Identical Interface using Omnigator </li></ul>
  22. 22. Two Retrieval Systems Compared <ul><li>Search for “Firewall” </li></ul><ul><li>Topic Maps-based Information Retrieval System </li></ul>Show Topic Types and Associative Relationships
  23. 23. Two Retrieval Systems Compared <ul><li>Search for “Firewall” </li></ul><ul><li>Topic Maps-based Information Retrieval System </li></ul>Shows the type of information and related information such as developers and sales person
  24. 24. Research Results <ul><li>There was a significant difference in recall between the two groups. </li></ul><ul><li>The estimate value shows the recall on TMIR was higher than CIR. </li></ul><ul><li>The estimate value also has shown that the search time/search steps in the experimental group was less than in the control group. </li></ul>
  25. 25. Discussion <ul><li>There were significant differences between the two groups and in terms of recall, precision, search time, and search steps. </li></ul><ul><li>Overall, recall was higher when performing simple task than when performing complex tasks. </li></ul><ul><li>Performing complex-tasks took more search time than performing simple tasks across the two groups. The control group took more total search time than the experimental group. </li></ul>
  26. 26. Conclusion <ul><li>This study illustrates that the positive influences of a Topic map-based ontology IR system are improved recall/precision, shorter search time and search steps for given search tasks than the clustering-based IR system. </li></ul><ul><li>The results of this study attest to the potential of Topic Maps-based ontology to improve information retrieval system performance through better support for associative relationships between terms belonging to different hierarchies by providing explicit relationships among resources. </li></ul>
  27. 27. Q & A <ul><li>Myongho Yi </li></ul><ul><li>Texas Woman’s University, TX, USA, myi@twu.edu </li></ul><ul><li>Sam Gyun Oh </li></ul><ul><li>SungKyunKwan University, Seoul, Korea, samoh@skku.edu </li></ul>

×