SlideShare a Scribd company logo
Measuring the usefulness of
Knowledge Organization Systems
in
Information Retrieval applications
Philipp Mayr
Observatory for Knowledge Organisation
Systems KNOWeSCAPE workshop, Valletta,
Malta
February 01, 2017
GESIS
• We are developing interactive information retrieval systems for
searching indexed literature and data sets
• We follow the principle „research-based service“; develop research
prototypes, test and evaluate them and implement the features which
are working for the end users
2
Intro
• Typical difficulties in searching digital libraries (DL)
– Vagueness between search and indexing terms
– How to support searchers with controlled vocabulary?
• Assumption: a user’s search (experience) should
improve by using Knowledge Organization Systems
(KOS):
– Vague search tasks
– Unfamiliar fields
– Cross domain searches
• Case studies to demonstrate the effectiveness of
KOS in different search scenarios 3
Case Study 1: Information retrieval
experiment
• Intra- and interdisciplinary cross-
concordances in the project
KoMoHe
– Social Sciences-SocSci; SocSci-
Economics; SocSci-Psychology; Politics-
Economics; Medicine-Psychology, …
• Information retrieval evaluation of the
mappings (effectiveness of
intellectual mapping)
4Controlled terms
Case Study 1
• How effective are the mappings in an actual search?
Does the application of term mappings (TT) improve
search over a non-transformed subject (i.e. controlled
vocabulary) search (CT)?
• Real queries, only equivalence relations, 13 thesaurus
mappings
5
Mayr/Petras 2008
• Overlap and more
identical terms in
intradisciplinary
mappings
• Interdisciplinary
mappings made the
strongest effect
Case Study 2: Information retrieval
experiment
• Discipline-specific Search-Term-
Recommendation (STR) Services in IRM
project
• Are recommendations from discipline
specific STRs better suited for query
expansion than general ones?
• Co-occurence of terms
in title/description and
assigned controlled
terms
• 17 STR services
– 16 discipline-spec.
– 1 global
6Lüke et al. 2012
Case Study 2
• Are recommendations from discipline specific STRs better suited for
query expansion (QE) than general ones?
• 100 topics from the GIRT corpus, top 4 recommendations to expand the
original query
• gSTR = global STR; tSTR = topical STR; bSTR = best-performing STR
7
Lüke et al. 2012
• QE with specific STRs leads to significantly better results than QE with a
general STR
• Selecting the best matching specific STR in an automatic way is a major
Case Study 3: Interactive IR
experiment
• Measuring the utility and
performance of Search Term
Recommendation (STR) Services in
AMUR project
• Logfile-based evaluation of STR
usage and later search session
success
• We defined positive signals (export,
save, email, full text …) in the
system
enter_search_term→select_term_from_reco
mmender→search→view_record_1→view_r
ecord_2→view_record_3→export_record
• Analysis of one year of log data
8
Hienert/Mutschke 2016
Case Study 3
• Usage of the STR significantly often implicates the
occurrence of positive signals during the following session
steps
9
Hienert/Mutschke 2016
Conclusions
• Information retrieval and
interactice IR settings are able to
demonstrate the utility of KOS
usage (usefulness)
– In experimental settings
– In user evaluations
• Each methodology has pros and
cons
– Effort and significance in small user
studies
– Too controlled, system-based,
without real users
• Terminology mapping projects 10
IR
Interactive
IR
Availability of corpora high low
Reproducibility high low
Control high low
Measures medium medium
Effort low high
Significance medium medium
Generalisability medium medium
Realistic Scenario no high
Outlook
• Integrate different recommender systems in
real retrieval tasks (search sessions)
• Use and evaluate recommenders for query
expansion and as dynamic features in IR, in
the retrieval process (AMUR project)
• Develop new measures of utility of
recommender systems
– E.g. measure task completion rates or goal
satisfaction
11
References
• Hienert, D. & Mutschke, P. (2016). A Usefulness-based Approach for
Measuring the Local and Global Effect of IIR Services. In Proceedings of
the 2016 ACM on Conference on Human Information Interaction and
Retrieval (CHIIR '16). ACM, New York, NY, USA, 153-162.
http://dx.doi.org/10.1145/2854946.2854962
• Lüke, T., Schaer, P., & Mayr, P. (2012). Improving Retrieval Results with
discipline-specific Query Expansion. In International Conference on Theory
and Practice of Digital Libraries (TPDL 2012) (pp. 408–413). Paphos,
Cyprus: Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-642-
33290-6_44
• Mayr, P., & Petras, V. (2008). Cross-concordances: terminology mapping
and its effectiveness for information retrieval. In 74th IFLA World Library and
Information Congress. Québec, Canada: IFLA. Retrieved from
http://www.ifla.org/IV/ifla74/papers/129-Mayr_Petras-en.pdf
12
Thank you
Contact:
Dr Philipp Mayr
GESIS - Leibniz Institute for the Social Sciences,
Germany
Email: philipp.mayr@gesis.org
Twitter: @philipp_mayr
13

More Related Content

What's hot

Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14
Pedro Bonifácio Costa
 
Peter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016IIIPeter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016III
Peter Sung
 
Data and Data collection
Data and Data collection Data and Data collection
Data and Data collection
Joan Cahuich Perez
 
Ms 66 marketing research
Ms 66 marketing researchMs 66 marketing research
Ms 66 marketing research
smumbahelp
 
Viva
VivaViva
C017510717
C017510717C017510717
C017510717
IOSR Journals
 
Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure
petrknoth
 
Literature review
Literature reviewLiterature review
Literature review
Darshit Kanziya
 
environmental scanning
environmental scanningenvironmental scanning
environmental scanning
sona44dg
 
Crowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral OutcomesCrowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral Outcomes
Alekya Yermal
 
Glossary
GlossaryGlossary
Glossary
asfawm
 
Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...
niloysarkar
 
Replicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearchReplicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearch
Andrea Wiggins
 
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
National Information Standards Organization (NISO)
 
KREAM@ICCS2013
KREAM@ICCS2013KREAM@ICCS2013
KREAM@ICCS2013
Jaakko Lappalainen
 
an empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniquesan empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniques
swathi78
 
Strasser "Effective data management and its role in open research"
Strasser "Effective data management and its role in open research"Strasser "Effective data management and its role in open research"
Strasser "Effective data management and its role in open research"
National Information Standards Organization (NISO)
 
Multi-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information SystemMulti-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information System
tulipbiru64
 
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
petrknoth
 
Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...
Stats Statswork
 

What's hot (20)

Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14
 
Peter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016IIIPeter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016III
 
Data and Data collection
Data and Data collection Data and Data collection
Data and Data collection
 
Ms 66 marketing research
Ms 66 marketing researchMs 66 marketing research
Ms 66 marketing research
 
Viva
VivaViva
Viva
 
C017510717
C017510717C017510717
C017510717
 
Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure
 
Literature review
Literature reviewLiterature review
Literature review
 
environmental scanning
environmental scanningenvironmental scanning
environmental scanning
 
Crowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral OutcomesCrowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral Outcomes
 
Glossary
GlossaryGlossary
Glossary
 
Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...
 
Replicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearchReplicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearch
 
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
 
KREAM@ICCS2013
KREAM@ICCS2013KREAM@ICCS2013
KREAM@ICCS2013
 
an empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniquesan empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniques
 
Strasser "Effective data management and its role in open research"
Strasser "Effective data management and its role in open research"Strasser "Effective data management and its role in open research"
Strasser "Effective data management and its role in open research"
 
Multi-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information SystemMulti-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information System
 
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
 
Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...
 

Viewers also liked

Recent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information RetrievalRecent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information Retrieval
GESIS
 
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
GESIS
 
Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...
Charles Seger
 
Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...
GESIS
 
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
GESIS
 
Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)
GESIS
 
Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...
GESIS
 
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
GESIS
 
Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...
GESIS
 
Opening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social SciencesOpening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social Sciences
GESIS
 
Past, present and future of scientific information
Past, present and future of scientific informationPast, present and future of scientific information
Past, present and future of scientific information
GESIS
 
Demonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations SystemsDemonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations Systems
GESIS
 
Recent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization SystemsRecent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization Systems
GESIS
 
Pennants for Descriptors
Pennants for DescriptorsPennants for Descriptors
Pennants for Descriptors
GESIS
 
Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016
GESIS
 
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshopIntroduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
GESIS
 
Assessing a human mediated current awareness service
Assessing a human mediated current awareness serviceAssessing a human mediated current awareness service
Assessing a human mediated current awareness service
GESIS
 
Using co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguationUsing co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguation
GESIS
 
Towards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social SciencesTowards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social Sciences
GESIS
 
How to build your own citation index
How to build your own citation indexHow to build your own citation index
How to build your own citation index
GESIS
 

Viewers also liked (20)

Recent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information RetrievalRecent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information Retrieval
 
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
 
Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...
 
Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...
 
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
 
Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)
 
Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...
 
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
 
Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...
 
Opening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social SciencesOpening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social Sciences
 
Past, present and future of scientific information
Past, present and future of scientific informationPast, present and future of scientific information
Past, present and future of scientific information
 
Demonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations SystemsDemonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations Systems
 
Recent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization SystemsRecent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization Systems
 
Pennants for Descriptors
Pennants for DescriptorsPennants for Descriptors
Pennants for Descriptors
 
Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016
 
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshopIntroduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
 
Assessing a human mediated current awareness service
Assessing a human mediated current awareness serviceAssessing a human mediated current awareness service
Assessing a human mediated current awareness service
 
Using co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguationUsing co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguation
 
Towards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social SciencesTowards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social Sciences
 
How to build your own citation index
How to build your own citation indexHow to build your own citation index
How to build your own citation index
 

Similar to Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications

Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement
Roi Blanco
 
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Platforma Otwartej Nauki
 
Managing Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research MethodsManaging Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research Methods
Rebecca Grant
 
Unit 3- Theories and approaches in Monitoring and Evaluation.ppt
Unit 3- Theories and approaches in Monitoring and Evaluation.pptUnit 3- Theories and approaches in Monitoring and Evaluation.ppt
Unit 3- Theories and approaches in Monitoring and Evaluation.ppt
tibulyam
 
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
William Gunn
 
Ignacio panach ormeño et-al_caise2013
Ignacio panach   ormeño et-al_caise2013Ignacio panach   ormeño et-al_caise2013
Ignacio panach ormeño et-al_caise2013
caise2013vlc
 
Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision Making
Katrien Verbert
 
Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...
ResearchSpace
 
Paving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflowsPaving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflows
The University of Edinburgh
 
Personality Assessment using Twitter Tweets
Personality Assessment using Twitter TweetsPersonality Assessment using Twitter Tweets
Personality Assessment using Twitter Tweets
Nadeem Ahmad Ch
 
RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...
ASIS&T
 
UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...
Jisc
 
Modern elicitation trends asma & ayesha paper presentation
Modern elicitation trends  asma & ayesha paper presentationModern elicitation trends  asma & ayesha paper presentation
Modern elicitation trends asma & ayesha paper presentation
Asma Sajid
 
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Scott Hutcheson, Ph.D.
 
The overlaps between Action Research and Design Research
The overlaps between Action Research and Design ResearchThe overlaps between Action Research and Design Research
The overlaps between Action Research and Design Research
Sandeep Purao
 
Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...
UoLResearchSupport
 
Introduction to Systematic Literature Review method
Introduction to Systematic Literature Review methodIntroduction to Systematic Literature Review method
Introduction to Systematic Literature Review method
Norsaremah Salleh
 
1.1 business research class discussions
1.1 business research class discussions1.1 business research class discussions
On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...
Grial - University of Salamanca
 
Öppen data och forskningens genomslag
Öppen data och forskningens genomslagÖppen data och forskningens genomslag
Öppen data och forskningens genomslag
Kungliga biblioteket National Library of Sweden
 

Similar to Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications (20)

Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement
 
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
 
Managing Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research MethodsManaging Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research Methods
 
Unit 3- Theories and approaches in Monitoring and Evaluation.ppt
Unit 3- Theories and approaches in Monitoring and Evaluation.pptUnit 3- Theories and approaches in Monitoring and Evaluation.ppt
Unit 3- Theories and approaches in Monitoring and Evaluation.ppt
 
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
 
Ignacio panach ormeño et-al_caise2013
Ignacio panach   ormeño et-al_caise2013Ignacio panach   ormeño et-al_caise2013
Ignacio panach ormeño et-al_caise2013
 
Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision Making
 
Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...
 
Paving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflowsPaving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflows
 
Personality Assessment using Twitter Tweets
Personality Assessment using Twitter TweetsPersonality Assessment using Twitter Tweets
Personality Assessment using Twitter Tweets
 
RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...
 
UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...
 
Modern elicitation trends asma & ayesha paper presentation
Modern elicitation trends  asma & ayesha paper presentationModern elicitation trends  asma & ayesha paper presentation
Modern elicitation trends asma & ayesha paper presentation
 
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
 
The overlaps between Action Research and Design Research
The overlaps between Action Research and Design ResearchThe overlaps between Action Research and Design Research
The overlaps between Action Research and Design Research
 
Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...
 
Introduction to Systematic Literature Review method
Introduction to Systematic Literature Review methodIntroduction to Systematic Literature Review method
Introduction to Systematic Literature Review method
 
1.1 business research class discussions
1.1 business research class discussions1.1 business research class discussions
1.1 business research class discussions
 
On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...
 
Öppen data och forskningens genomslag
Öppen data och forskningens genomslagÖppen data och forskningens genomslag
Öppen data och forskningens genomslag
 

More from GESIS

10th BIR Workshop @ECIR 2020: introduction
10th  BIR Workshop @ECIR 2020: introduction10th  BIR Workshop @ECIR 2020: introduction
10th BIR Workshop @ECIR 2020: introduction
GESIS
 
From closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journalsFrom closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journals
GESIS
 
Highly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeHighly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over time
GESIS
 
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
GESIS
 
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with BibliometricsBibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
GESIS
 
Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”
GESIS
 
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
GESIS
 
Searching beyond datasets in the Social Sciences
Searching beyond datasets in the Social SciencesSearching beyond datasets in the Social Sciences
Searching beyond datasets in the Social Sciences
GESIS
 
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der SozialwissenschaftenBedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
GESIS
 
Contextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living LabContextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living Lab
GESIS
 
41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)
GESIS
 
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
GESIS
 
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
GESIS
 
Challenges in Extracting and Managing References
Challenges in Extracting and Managing ReferencesChallenges in Extracting and Managing References
Challenges in Extracting and Managing References
GESIS
 
Einführung in das Vektorraummodell
Einführung in das VektorraummodellEinführung in das Vektorraummodell
Einführung in das Vektorraummodell
GESIS
 
Industrie 4.0
Industrie 4.0Industrie 4.0
Industrie 4.0
GESIS
 

More from GESIS (16)

10th BIR Workshop @ECIR 2020: introduction
10th  BIR Workshop @ECIR 2020: introduction10th  BIR Workshop @ECIR 2020: introduction
10th BIR Workshop @ECIR 2020: introduction
 
From closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journalsFrom closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journals
 
Highly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeHighly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over time
 
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
 
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with BibliometricsBibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
 
Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”
 
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
 
Searching beyond datasets in the Social Sciences
Searching beyond datasets in the Social SciencesSearching beyond datasets in the Social Sciences
Searching beyond datasets in the Social Sciences
 
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der SozialwissenschaftenBedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
 
Contextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living LabContextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living Lab
 
41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)
 
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
 
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
 
Challenges in Extracting and Managing References
Challenges in Extracting and Managing ReferencesChallenges in Extracting and Managing References
Challenges in Extracting and Managing References
 
Einführung in das Vektorraummodell
Einführung in das VektorraummodellEinführung in das Vektorraummodell
Einführung in das Vektorraummodell
 
Industrie 4.0
Industrie 4.0Industrie 4.0
Industrie 4.0
 

Recently uploaded

Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...
Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...
Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...
Sérgio Sacani
 
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
Sérgio Sacani
 
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpectiveGasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Recupera
 
poikilocytosis 23765437865210857453257844.pptx
poikilocytosis 23765437865210857453257844.pptxpoikilocytosis 23765437865210857453257844.pptx
poikilocytosis 23765437865210857453257844.pptx
muralinath2
 
Complementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotailComplementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotail
Sérgio Sacani
 
Summer program introduction in Yunnan university
Summer program introduction in Yunnan universitySummer program introduction in Yunnan university
Summer program introduction in Yunnan university
Hayato Shimabukuro
 
Simulations of pulsed overpressure jets: formation of bellows and ripples in ...
Simulations of pulsed overpressure jets: formation of bellows and ripples in ...Simulations of pulsed overpressure jets: formation of bellows and ripples in ...
Simulations of pulsed overpressure jets: formation of bellows and ripples in ...
Sérgio Sacani
 
poikilocytosis 237654378658585210854.pptx
poikilocytosis 237654378658585210854.pptxpoikilocytosis 237654378658585210854.pptx
poikilocytosis 237654378658585210854.pptx
muralinath2
 
Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...
Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...
Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...
University of Maribor
 
MARIGREEN PROJECT - overview, Oana Cristina Pârvulescu
MARIGREEN PROJECT - overview, Oana Cristina PârvulescuMARIGREEN PROJECT - overview, Oana Cristina Pârvulescu
MARIGREEN PROJECT - overview, Oana Cristina Pârvulescu
Faculty of Applied Chemistry and Materials Science
 
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
Sérgio Sacani
 
Hydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanet
Hydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanetHydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanet
Hydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanet
Sérgio Sacani
 
seed drying lecture, different types of dryers
seed drying lecture, different types of dryersseed drying lecture, different types of dryers
seed drying lecture, different types of dryers
Rammehargahlot1
 
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary TrackThe Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
Sérgio Sacani
 
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
James AH Campbell
 
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptxAN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
kalpnayadav03021986
 
MCQ in Electrostatics. for class XII pptx
MCQ in Electrostatics. for class XII  pptxMCQ in Electrostatics. for class XII  pptx
MCQ in Electrostatics. for class XII pptx
ArunachalamM22
 
Introduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdfIntroduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdf
kaavyashreegoskula
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Hossein Fani
 
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Dr. sreeremya S
 

Recently uploaded (20)

Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...
Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...
Transmission Spectroscopy of the Habitable Zone Exoplanet LHS 1140 b with JWS...
 
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
 
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpectiveGasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
 
poikilocytosis 23765437865210857453257844.pptx
poikilocytosis 23765437865210857453257844.pptxpoikilocytosis 23765437865210857453257844.pptx
poikilocytosis 23765437865210857453257844.pptx
 
Complementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotailComplementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotail
 
Summer program introduction in Yunnan university
Summer program introduction in Yunnan universitySummer program introduction in Yunnan university
Summer program introduction in Yunnan university
 
Simulations of pulsed overpressure jets: formation of bellows and ripples in ...
Simulations of pulsed overpressure jets: formation of bellows and ripples in ...Simulations of pulsed overpressure jets: formation of bellows and ripples in ...
Simulations of pulsed overpressure jets: formation of bellows and ripples in ...
 
poikilocytosis 237654378658585210854.pptx
poikilocytosis 237654378658585210854.pptxpoikilocytosis 237654378658585210854.pptx
poikilocytosis 237654378658585210854.pptx
 
Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...
Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...
Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking ...
 
MARIGREEN PROJECT - overview, Oana Cristina Pârvulescu
MARIGREEN PROJECT - overview, Oana Cristina PârvulescuMARIGREEN PROJECT - overview, Oana Cristina Pârvulescu
MARIGREEN PROJECT - overview, Oana Cristina Pârvulescu
 
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
 
Hydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanet
Hydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanetHydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanet
Hydrogen sulfide and metal-enriched atmosphere for a Jupiter-mass exoplanet
 
seed drying lecture, different types of dryers
seed drying lecture, different types of dryersseed drying lecture, different types of dryers
seed drying lecture, different types of dryers
 
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary TrackThe Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
The Dynamical Origins of the Dark Comets and a Proposed Evolutionary Track
 
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
Probing the northern Kaapvaal craton root with mantle-derived xenocrysts from...
 
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptxAN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
 
MCQ in Electrostatics. for class XII pptx
MCQ in Electrostatics. for class XII  pptxMCQ in Electrostatics. for class XII  pptx
MCQ in Electrostatics. for class XII pptx
 
Introduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdfIntroduction to Artificial Intelligence.pdf
Introduction to Artificial Intelligence.pdf
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
 
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
 

Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications

  • 1. Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications Philipp Mayr Observatory for Knowledge Organisation Systems KNOWeSCAPE workshop, Valletta, Malta February 01, 2017
  • 2. GESIS • We are developing interactive information retrieval systems for searching indexed literature and data sets • We follow the principle „research-based service“; develop research prototypes, test and evaluate them and implement the features which are working for the end users 2
  • 3. Intro • Typical difficulties in searching digital libraries (DL) – Vagueness between search and indexing terms – How to support searchers with controlled vocabulary? • Assumption: a user’s search (experience) should improve by using Knowledge Organization Systems (KOS): – Vague search tasks – Unfamiliar fields – Cross domain searches • Case studies to demonstrate the effectiveness of KOS in different search scenarios 3
  • 4. Case Study 1: Information retrieval experiment • Intra- and interdisciplinary cross- concordances in the project KoMoHe – Social Sciences-SocSci; SocSci- Economics; SocSci-Psychology; Politics- Economics; Medicine-Psychology, … • Information retrieval evaluation of the mappings (effectiveness of intellectual mapping) 4Controlled terms
  • 5. Case Study 1 • How effective are the mappings in an actual search? Does the application of term mappings (TT) improve search over a non-transformed subject (i.e. controlled vocabulary) search (CT)? • Real queries, only equivalence relations, 13 thesaurus mappings 5 Mayr/Petras 2008 • Overlap and more identical terms in intradisciplinary mappings • Interdisciplinary mappings made the strongest effect
  • 6. Case Study 2: Information retrieval experiment • Discipline-specific Search-Term- Recommendation (STR) Services in IRM project • Are recommendations from discipline specific STRs better suited for query expansion than general ones? • Co-occurence of terms in title/description and assigned controlled terms • 17 STR services – 16 discipline-spec. – 1 global 6Lüke et al. 2012
  • 7. Case Study 2 • Are recommendations from discipline specific STRs better suited for query expansion (QE) than general ones? • 100 topics from the GIRT corpus, top 4 recommendations to expand the original query • gSTR = global STR; tSTR = topical STR; bSTR = best-performing STR 7 Lüke et al. 2012 • QE with specific STRs leads to significantly better results than QE with a general STR • Selecting the best matching specific STR in an automatic way is a major
  • 8. Case Study 3: Interactive IR experiment • Measuring the utility and performance of Search Term Recommendation (STR) Services in AMUR project • Logfile-based evaluation of STR usage and later search session success • We defined positive signals (export, save, email, full text …) in the system enter_search_term→select_term_from_reco mmender→search→view_record_1→view_r ecord_2→view_record_3→export_record • Analysis of one year of log data 8 Hienert/Mutschke 2016
  • 9. Case Study 3 • Usage of the STR significantly often implicates the occurrence of positive signals during the following session steps 9 Hienert/Mutschke 2016
  • 10. Conclusions • Information retrieval and interactice IR settings are able to demonstrate the utility of KOS usage (usefulness) – In experimental settings – In user evaluations • Each methodology has pros and cons – Effort and significance in small user studies – Too controlled, system-based, without real users • Terminology mapping projects 10 IR Interactive IR Availability of corpora high low Reproducibility high low Control high low Measures medium medium Effort low high Significance medium medium Generalisability medium medium Realistic Scenario no high
  • 11. Outlook • Integrate different recommender systems in real retrieval tasks (search sessions) • Use and evaluate recommenders for query expansion and as dynamic features in IR, in the retrieval process (AMUR project) • Develop new measures of utility of recommender systems – E.g. measure task completion rates or goal satisfaction 11
  • 12. References • Hienert, D. & Mutschke, P. (2016). A Usefulness-based Approach for Measuring the Local and Global Effect of IIR Services. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR '16). ACM, New York, NY, USA, 153-162. http://dx.doi.org/10.1145/2854946.2854962 • Lüke, T., Schaer, P., & Mayr, P. (2012). Improving Retrieval Results with discipline-specific Query Expansion. In International Conference on Theory and Practice of Digital Libraries (TPDL 2012) (pp. 408–413). Paphos, Cyprus: Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-642- 33290-6_44 • Mayr, P., & Petras, V. (2008). Cross-concordances: terminology mapping and its effectiveness for information retrieval. In 74th IFLA World Library and Information Congress. Québec, Canada: IFLA. Retrieved from http://www.ifla.org/IV/ifla74/papers/129-Mayr_Petras-en.pdf 12
  • 13. Thank you Contact: Dr Philipp Mayr GESIS - Leibniz Institute for the Social Sciences, Germany Email: philipp.mayr@gesis.org Twitter: @philipp_mayr 13