SlideShare a Scribd company logo
1 of 13
Download to read offline
Measuring the usefulness of
Knowledge Organization Systems
in
Information Retrieval applications
Philipp Mayr
Observatory for Knowledge Organisation
Systems KNOWeSCAPE workshop, Valletta,
Malta
February 01, 2017
GESIS
• We are developing interactive information retrieval systems for
searching indexed literature and data sets
• We follow the principle „research-based service“; develop research
prototypes, test and evaluate them and implement the features which
are working for the end users
2
Intro
• Typical difficulties in searching digital libraries (DL)
– Vagueness between search and indexing terms
– How to support searchers with controlled vocabulary?
• Assumption: a user’s search (experience) should
improve by using Knowledge Organization Systems
(KOS):
– Vague search tasks
– Unfamiliar fields
– Cross domain searches
• Case studies to demonstrate the effectiveness of
KOS in different search scenarios 3
Case Study 1: Information retrieval
experiment
• Intra- and interdisciplinary cross-
concordances in the project
KoMoHe
– Social Sciences-SocSci; SocSci-
Economics; SocSci-Psychology; Politics-
Economics; Medicine-Psychology, …
• Information retrieval evaluation of the
mappings (effectiveness of
intellectual mapping)
4Controlled terms
Case Study 1
• How effective are the mappings in an actual search?
Does the application of term mappings (TT) improve
search over a non-transformed subject (i.e. controlled
vocabulary) search (CT)?
• Real queries, only equivalence relations, 13 thesaurus
mappings
5
Mayr/Petras 2008
• Overlap and more
identical terms in
intradisciplinary
mappings
• Interdisciplinary
mappings made the
strongest effect
Case Study 2: Information retrieval
experiment
• Discipline-specific Search-Term-
Recommendation (STR) Services in IRM
project
• Are recommendations from discipline
specific STRs better suited for query
expansion than general ones?
• Co-occurence of terms
in title/description and
assigned controlled
terms
• 17 STR services
– 16 discipline-spec.
– 1 global
6Lüke et al. 2012
Case Study 2
• Are recommendations from discipline specific STRs better suited for
query expansion (QE) than general ones?
• 100 topics from the GIRT corpus, top 4 recommendations to expand the
original query
• gSTR = global STR; tSTR = topical STR; bSTR = best-performing STR
7
Lüke et al. 2012
• QE with specific STRs leads to significantly better results than QE with a
general STR
• Selecting the best matching specific STR in an automatic way is a major
Case Study 3: Interactive IR
experiment
• Measuring the utility and
performance of Search Term
Recommendation (STR) Services in
AMUR project
• Logfile-based evaluation of STR
usage and later search session
success
• We defined positive signals (export,
save, email, full text …) in the
system
enter_search_term→select_term_from_reco
mmender→search→view_record_1→view_r
ecord_2→view_record_3→export_record
• Analysis of one year of log data
8
Hienert/Mutschke 2016
Case Study 3
• Usage of the STR significantly often implicates the
occurrence of positive signals during the following session
steps
9
Hienert/Mutschke 2016
Conclusions
• Information retrieval and
interactice IR settings are able to
demonstrate the utility of KOS
usage (usefulness)
– In experimental settings
– In user evaluations
• Each methodology has pros and
cons
– Effort and significance in small user
studies
– Too controlled, system-based,
without real users
• Terminology mapping projects 10
IR
Interactive
IR
Availability of corpora high low
Reproducibility high low
Control high low
Measures medium medium
Effort low high
Significance medium medium
Generalisability medium medium
Realistic Scenario no high
Outlook
• Integrate different recommender systems in
real retrieval tasks (search sessions)
• Use and evaluate recommenders for query
expansion and as dynamic features in IR, in
the retrieval process (AMUR project)
• Develop new measures of utility of
recommender systems
– E.g. measure task completion rates or goal
satisfaction
11
References
• Hienert, D. & Mutschke, P. (2016). A Usefulness-based Approach for
Measuring the Local and Global Effect of IIR Services. In Proceedings of
the 2016 ACM on Conference on Human Information Interaction and
Retrieval (CHIIR '16). ACM, New York, NY, USA, 153-162.
http://dx.doi.org/10.1145/2854946.2854962
• Lüke, T., Schaer, P., & Mayr, P. (2012). Improving Retrieval Results with
discipline-specific Query Expansion. In International Conference on Theory
and Practice of Digital Libraries (TPDL 2012) (pp. 408–413). Paphos,
Cyprus: Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-642-
33290-6_44
• Mayr, P., & Petras, V. (2008). Cross-concordances: terminology mapping
and its effectiveness for information retrieval. In 74th IFLA World Library and
Information Congress. Québec, Canada: IFLA. Retrieved from
http://www.ifla.org/IV/ifla74/papers/129-Mayr_Petras-en.pdf
12
Thank you
Contact:
Dr Philipp Mayr
GESIS - Leibniz Institute for the Social Sciences,
Germany
Email: philipp.mayr@gesis.org
Twitter: @philipp_mayr
13

More Related Content

What's hot

Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14Pedro Bonifácio Costa
 
Peter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016IIIPeter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016IIIPeter Sung
 
Ms 66 marketing research
Ms 66 marketing researchMs 66 marketing research
Ms 66 marketing researchsmumbahelp
 
Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure petrknoth
 
environmental scanning
environmental scanningenvironmental scanning
environmental scanningsona44dg
 
Crowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral OutcomesCrowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral OutcomesAlekya Yermal
 
Glossary
GlossaryGlossary
Glossaryasfawm
 
Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...niloysarkar
 
Replicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearchReplicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearchAndrea Wiggins
 
an empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniquesan empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniquesswathi78
 
Multi-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information SystemMulti-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information Systemtulipbiru64
 
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...petrknoth
 
Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...Stats Statswork
 

What's hot (20)

Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14Profiling Users' Preferences with Text Mining '14
Profiling Users' Preferences with Text Mining '14
 
Peter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016IIIPeter (Yun-shao) Sung's Resume 2016III
Peter (Yun-shao) Sung's Resume 2016III
 
Data and Data collection
Data and Data collection Data and Data collection
Data and Data collection
 
Ms 66 marketing research
Ms 66 marketing researchMs 66 marketing research
Ms 66 marketing research
 
Viva
VivaViva
Viva
 
C017510717
C017510717C017510717
C017510717
 
Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure Integrating research indicators for use in the repositories infrastructure
Integrating research indicators for use in the repositories infrastructure
 
Literature review
Literature reviewLiterature review
Literature review
 
environmental scanning
environmental scanningenvironmental scanning
environmental scanning
 
Crowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral OutcomesCrowdsourcing Predictors of Behavioral Outcomes
Crowdsourcing Predictors of Behavioral Outcomes
 
Glossary
GlossaryGlossary
Glossary
 
Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...Introduction to Research methodology: Orientation for Doctoral Program Course...
Introduction to Research methodology: Orientation for Doctoral Program Course...
 
Replicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearchReplicating FLOSS Research as eResearch
Replicating FLOSS Research as eResearch
 
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
Llebot "Research Data Support for Researchers: Metadata, Challenges, and Oppo...
 
KREAM@ICCS2013
KREAM@ICCS2013KREAM@ICCS2013
KREAM@ICCS2013
 
an empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniquesan empirical performance evaluation of relational keyword search techniques
an empirical performance evaluation of relational keyword search techniques
 
Strasser "Effective data management and its role in open research"
Strasser "Effective data management and its role in open research"Strasser "Effective data management and its role in open research"
Strasser "Effective data management and its role in open research"
 
Multi-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information SystemMulti-factor Information Security Risk in Information System
Multi-factor Information Security Risk in Information System
 
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...COAR Next Generation Repositories WG - Text mining and Recommender system sto...
COAR Next Generation Repositories WG - Text mining and Recommender system sto...
 
Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...Research design decisions and be competent in the process of reliable data co...
Research design decisions and be competent in the process of reliable data co...
 

Viewers also liked

Recent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information RetrievalRecent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information RetrievalGESIS
 
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...GESIS
 
Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...Charles Seger
 
Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...GESIS
 
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...GESIS
 
Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)GESIS
 
Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...GESIS
 
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...GESIS
 
Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...GESIS
 
Opening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social SciencesOpening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social SciencesGESIS
 
Past, present and future of scientific information
Past, present and future of scientific informationPast, present and future of scientific information
Past, present and future of scientific informationGESIS
 
Demonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations SystemsDemonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations SystemsGESIS
 
Recent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization SystemsRecent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization SystemsGESIS
 
Pennants for Descriptors
Pennants for DescriptorsPennants for Descriptors
Pennants for DescriptorsGESIS
 
Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016GESIS
 
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshopIntroduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshopGESIS
 
Assessing a human mediated current awareness service
Assessing a human mediated current awareness serviceAssessing a human mediated current awareness service
Assessing a human mediated current awareness serviceGESIS
 
Using co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguationUsing co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguationGESIS
 
Towards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social SciencesTowards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social SciencesGESIS
 
How to build your own citation index
How to build your own citation indexHow to build your own citation index
How to build your own citation indexGESIS
 

Viewers also liked (20)

Recent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information RetrievalRecent Advances in Bibliometric-Enhanced Information Retrieval
Recent Advances in Bibliometric-Enhanced Information Retrieval
 
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
Opening Scholarly Communication in Social Sciences by Connecting Collaborativ...
 
Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...Adaptive Design implications for Knowledge Organization and Information Retri...
Adaptive Design implications for Knowledge Organization and Information Retri...
 
Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...Establishing an Online Access Panel for Interactive Information Retrieval Res...
Establishing an Online Access Panel for Interactive Information Retrieval Res...
 
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
PEP-TF: Social Media Monitoring of the Campaigns for the 2013 German Bundesta...
 
Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)Opening Scholarly Communication in Social Sciences (OSCOSS)
Opening Scholarly Communication in Social Sciences (OSCOSS)
 
Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...Are topic-specific search term, journal name and author name recommendations ...
Are topic-specific search term, journal name and author name recommendations ...
 
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
Introduction of the 3rd International Workshop on Bibliometric-enhanced Infor...
 
Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...Analyzing the research output presented at European Networked Knowledge Organ...
Analyzing the research output presented at European Networked Knowledge Organ...
 
Opening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social SciencesOpening Scholarly Communication in the Social Sciences
Opening Scholarly Communication in the Social Sciences
 
Past, present and future of scientific information
Past, present and future of scientific informationPast, present and future of scientific information
Past, present and future of scientific information
 
Demonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations SystemsDemonstrating a Framework for KOS-based Recommendations Systems
Demonstrating a Framework for KOS-based Recommendations Systems
 
Recent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization SystemsRecent applications of Knowledge Organization Systems
Recent applications of Knowledge Organization Systems
 
Pennants for Descriptors
Pennants for DescriptorsPennants for Descriptors
Pennants for Descriptors
 
Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016Introduction to the 15th NKOS workshop @TPDL2016
Introduction to the 15th NKOS workshop @TPDL2016
 
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshopIntroduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
Introduction of the Bibliometric-enhanced Information Retrieval (BIR) workshop
 
Assessing a human mediated current awareness service
Assessing a human mediated current awareness serviceAssessing a human mediated current awareness service
Assessing a human mediated current awareness service
 
Using co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguationUsing co-authorship networks for author name disambiguation
Using co-authorship networks for author name disambiguation
 
Towards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social SciencesTowards a Semantic Citation Index for the German Social Sciences
Towards a Semantic Citation Index for the German Social Sciences
 
How to build your own citation index
How to build your own citation indexHow to build your own citation index
How to build your own citation index
 

Similar to Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications

Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Roi Blanco
 
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...Platforma Otwartej Nauki
 
Managing Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research MethodsManaging Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research MethodsRebecca Grant
 
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...William Gunn
 
Ignacio panach ormeño et-al_caise2013
Ignacio panach   ormeño et-al_caise2013Ignacio panach   ormeño et-al_caise2013
Ignacio panach ormeño et-al_caise2013caise2013vlc
 
Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingKatrien Verbert
 
Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...ResearchSpace
 
Paving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflowsPaving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflowsThe University of Edinburgh
 
Personality Assessment using Twitter Tweets
Personality Assessment using Twitter TweetsPersonality Assessment using Twitter Tweets
Personality Assessment using Twitter TweetsNadeem Ahmad Ch
 
RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...ASIS&T
 
UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...Jisc
 
Modern elicitation trends asma & ayesha paper presentation
Modern elicitation trends  asma & ayesha paper presentationModern elicitation trends  asma & ayesha paper presentation
Modern elicitation trends asma & ayesha paper presentationAsma Sajid
 
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...Scott Hutcheson, Ph.D.
 
The overlaps between Action Research and Design Research
The overlaps between Action Research and Design ResearchThe overlaps between Action Research and Design Research
The overlaps between Action Research and Design ResearchSandeep Purao
 
Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...UoLResearchSupport
 
Introduction to Systematic Literature Review method
Introduction to Systematic Literature Review methodIntroduction to Systematic Literature Review method
Introduction to Systematic Literature Review methodNorsaremah Salleh
 
On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...Grial - University of Salamanca
 
Australia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityAustralia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityTERN Australia
 

Similar to Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications (20)

Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement
 
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
Linking Heterogeneous Scholarly Data Sources in an Interoperable Setting: the...
 
Managing Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research MethodsManaging Ireland's Research Data - 3 Research Methods
Managing Ireland's Research Data - 3 Research Methods
 
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
Sci Know Mine 2013: What can we learn from topic modeling on 350M academic do...
 
Ignacio panach ormeño et-al_caise2013
Ignacio panach   ormeño et-al_caise2013Ignacio panach   ormeño et-al_caise2013
Ignacio panach ormeño et-al_caise2013
 
Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision Making
 
Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...Paving the way to open and interoperable research data service workflows Prog...
Paving the way to open and interoperable research data service workflows Prog...
 
Paving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflowsPaving the way to open and interoperable research data service workflows
Paving the way to open and interoperable research data service workflows
 
Personality Assessment using Twitter Tweets
Personality Assessment using Twitter TweetsPersonality Assessment using Twitter Tweets
Personality Assessment using Twitter Tweets
 
RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...RDAP14 Poster: Evaluation of research data services: What things should we ev...
RDAP14 Poster: Evaluation of research data services: What things should we ev...
 
UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...UCL’s research IT management systems architecture review aligned with Open Sc...
UCL’s research IT management systems architecture review aligned with Open Sc...
 
Modern elicitation trends asma & ayesha paper presentation
Modern elicitation trends  asma & ayesha paper presentationModern elicitation trends  asma & ayesha paper presentation
Modern elicitation trends asma & ayesha paper presentation
 
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...Getting from Here to There: Eight Characteristics of Effective Economic & Com...
Getting from Here to There: Eight Characteristics of Effective Economic & Com...
 
The overlaps between Action Research and Design Research
The overlaps between Action Research and Design ResearchThe overlaps between Action Research and Design Research
The overlaps between Action Research and Design Research
 
Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...Open from beginning to end: addressing barriers to open research - a personal...
Open from beginning to end: addressing barriers to open research - a personal...
 
Introduction to Systematic Literature Review method
Introduction to Systematic Literature Review methodIntroduction to Systematic Literature Review method
Introduction to Systematic Literature Review method
 
1.1 business research class discussions
1.1 business research class discussions1.1 business research class discussions
1.1 business research class discussions
 
On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...
 
Öppen data och forskningens genomslag
Öppen data och forskningens genomslagÖppen data och forskningens genomslag
Öppen data och forskningens genomslag
 
Australia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityAustralia's Environmental Predictive Capability
Australia's Environmental Predictive Capability
 

More from GESIS

10th BIR Workshop @ECIR 2020: introduction
10th  BIR Workshop @ECIR 2020: introduction10th  BIR Workshop @ECIR 2020: introduction
10th BIR Workshop @ECIR 2020: introductionGESIS
 
From closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journalsFrom closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journalsGESIS
 
Highly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeHighly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeGESIS
 
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...GESIS
 
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with BibliometricsBibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with BibliometricsGESIS
 
Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”GESIS
 
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...GESIS
 
Searching beyond datasets in the Social Sciences
Searching beyond datasets in the Social SciencesSearching beyond datasets in the Social Sciences
Searching beyond datasets in the Social SciencesGESIS
 
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der SozialwissenschaftenBedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der SozialwissenschaftenGESIS
 
Contextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living LabContextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living LabGESIS
 
41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)GESIS
 
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...GESIS
 
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...GESIS
 
Challenges in Extracting and Managing References
Challenges in Extracting and Managing ReferencesChallenges in Extracting and Managing References
Challenges in Extracting and Managing ReferencesGESIS
 
Einführung in das Vektorraummodell
Einführung in das VektorraummodellEinführung in das Vektorraummodell
Einführung in das VektorraummodellGESIS
 
Industrie 4.0
Industrie 4.0Industrie 4.0
Industrie 4.0GESIS
 

More from GESIS (16)

10th BIR Workshop @ECIR 2020: introduction
10th  BIR Workshop @ECIR 2020: introduction10th  BIR Workshop @ECIR 2020: introduction
10th BIR Workshop @ECIR 2020: introduction
 
From closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journalsFrom closed to open access: A case study of flipped journals
From closed to open access: A case study of flipped journals
 
Highly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeHighly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over time
 
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural...
 
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with BibliometricsBibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
Bibliometric-enhanced Information Retrieval: Connecting IR with Bibliometrics
 
Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”Analyzing the network structure and gender differences of the “NKOS community”
Analyzing the network structure and gender differences of the “NKOS community”
 
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
Recent advances in the project EXCITE – Extraction of Citations from PDF Docu...
 
Searching beyond datasets in the Social Sciences
Searching beyond datasets in the Social SciencesSearching beyond datasets in the Social Sciences
Searching beyond datasets in the Social Sciences
 
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der SozialwissenschaftenBedeutung von Text Mining am Beispiel der Sozialwissenschaften
Bedeutung von Text Mining am Beispiel der Sozialwissenschaften
 
Contextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living LabContextualised Browsing in a Digital Library’s Living Lab
Contextualised Browsing in a Digital Library’s Living Lab
 
41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)41st European Conference on Information Retrieval (ECIR 2019)
41st European Conference on Information Retrieval (ECIR 2019)
 
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
Offenes kollaboratives Schreiben: Eine „Open Science“-Infrastruktur am Beispi...
 
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
A Complete Year of User Retrieval Sessions in a Social Sciences Academic Sear...
 
Challenges in Extracting and Managing References
Challenges in Extracting and Managing ReferencesChallenges in Extracting and Managing References
Challenges in Extracting and Managing References
 
Einführung in das Vektorraummodell
Einführung in das VektorraummodellEinführung in das Vektorraummodell
Einführung in das Vektorraummodell
 
Industrie 4.0
Industrie 4.0Industrie 4.0
Industrie 4.0
 

Recently uploaded

Pests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPRPests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPRPirithiRaju
 
THE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptx
THE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptxTHE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptx
THE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptxAkinrotimiOluwadunsi
 
Main Exam Applied biochemistry final year
Main Exam Applied biochemistry final yearMain Exam Applied biochemistry final year
Main Exam Applied biochemistry final yearmarwaahmad357
 
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrashi Coaching
 
soft skills question paper set for bba ca
soft skills question paper set for bba casoft skills question paper set for bba ca
soft skills question paper set for bba caohsadfeeling
 
Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...Sérgio Sacani
 
biosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibioticsbiosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibioticsSafaFallah
 
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...Sérgio Sacani
 
CW marking grid Analytical BS - M Ahmad.docx
CW  marking grid Analytical BS - M Ahmad.docxCW  marking grid Analytical BS - M Ahmad.docx
CW marking grid Analytical BS - M Ahmad.docxmarwaahmad357
 
Controlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform EnvironmentControlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform EnvironmentRahulVishwakarma71547
 
Shiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky Way
Shiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky WayShiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky Way
Shiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky WaySérgio Sacani
 
Excavation Methods in Archaeological Research & Studies
Excavation Methods in Archaeological Research &  StudiesExcavation Methods in Archaeological Research &  Studies
Excavation Methods in Archaeological Research & StudiesPrachya Adhyayan
 
Intensive Housing systems for Poultry.pptx
Intensive Housing systems for Poultry.pptxIntensive Housing systems for Poultry.pptx
Intensive Housing systems for Poultry.pptxHarshiniAlapati
 
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...PirithiRaju
 
World Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlabWorld Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlabkiyorndlab
 
Human brain.. It's parts and function.
Human brain.. It's parts and function. Human brain.. It's parts and function.
Human brain.. It's parts and function. MUKTA MANJARI SAHOO
 
TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)chatterjeesoumili50
 
Pests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdf
Pests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdfPests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdf
Pests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdfPirithiRaju
 
Pests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPRPests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPRPirithiRaju
 
Bureau of Indian Standards Specification of Shampoo.pptx
Bureau of Indian Standards Specification of Shampoo.pptxBureau of Indian Standards Specification of Shampoo.pptx
Bureau of Indian Standards Specification of Shampoo.pptxkastureyashashree
 

Recently uploaded (20)

Pests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPRPests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPR
 
THE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptx
THE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptxTHE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptx
THE HISTOLOGY OF THE CARDIOVASCULAR SYSTEM 2024.pptx
 
Main Exam Applied biochemistry final year
Main Exam Applied biochemistry final yearMain Exam Applied biochemistry final year
Main Exam Applied biochemistry final year
 
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
 
soft skills question paper set for bba ca
soft skills question paper set for bba casoft skills question paper set for bba ca
soft skills question paper set for bba ca
 
Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...
 
biosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibioticsbiosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibiotics
 
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
 
CW marking grid Analytical BS - M Ahmad.docx
CW  marking grid Analytical BS - M Ahmad.docxCW  marking grid Analytical BS - M Ahmad.docx
CW marking grid Analytical BS - M Ahmad.docx
 
Controlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform EnvironmentControlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform Environment
 
Shiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky Way
Shiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky WayShiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky Way
Shiva and Shakti: Presumed Proto-Galactic Fragments in the Inner Milky Way
 
Excavation Methods in Archaeological Research & Studies
Excavation Methods in Archaeological Research &  StudiesExcavation Methods in Archaeological Research &  Studies
Excavation Methods in Archaeological Research & Studies
 
Intensive Housing systems for Poultry.pptx
Intensive Housing systems for Poultry.pptxIntensive Housing systems for Poultry.pptx
Intensive Housing systems for Poultry.pptx
 
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
 
World Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlabWorld Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlab
 
Human brain.. It's parts and function.
Human brain.. It's parts and function. Human brain.. It's parts and function.
Human brain.. It's parts and function.
 
TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)
 
Pests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdf
Pests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdfPests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdf
Pests of wheat_Identification, Bionomics, Damage symptoms, IPM_Dr.UPR.pdf
 
Pests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPRPests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPR
 
Bureau of Indian Standards Specification of Shampoo.pptx
Bureau of Indian Standards Specification of Shampoo.pptxBureau of Indian Standards Specification of Shampoo.pptx
Bureau of Indian Standards Specification of Shampoo.pptx
 

Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications

  • 1. Measuring the usefulness of Knowledge Organization Systems in Information Retrieval applications Philipp Mayr Observatory for Knowledge Organisation Systems KNOWeSCAPE workshop, Valletta, Malta February 01, 2017
  • 2. GESIS • We are developing interactive information retrieval systems for searching indexed literature and data sets • We follow the principle „research-based service“; develop research prototypes, test and evaluate them and implement the features which are working for the end users 2
  • 3. Intro • Typical difficulties in searching digital libraries (DL) – Vagueness between search and indexing terms – How to support searchers with controlled vocabulary? • Assumption: a user’s search (experience) should improve by using Knowledge Organization Systems (KOS): – Vague search tasks – Unfamiliar fields – Cross domain searches • Case studies to demonstrate the effectiveness of KOS in different search scenarios 3
  • 4. Case Study 1: Information retrieval experiment • Intra- and interdisciplinary cross- concordances in the project KoMoHe – Social Sciences-SocSci; SocSci- Economics; SocSci-Psychology; Politics- Economics; Medicine-Psychology, … • Information retrieval evaluation of the mappings (effectiveness of intellectual mapping) 4Controlled terms
  • 5. Case Study 1 • How effective are the mappings in an actual search? Does the application of term mappings (TT) improve search over a non-transformed subject (i.e. controlled vocabulary) search (CT)? • Real queries, only equivalence relations, 13 thesaurus mappings 5 Mayr/Petras 2008 • Overlap and more identical terms in intradisciplinary mappings • Interdisciplinary mappings made the strongest effect
  • 6. Case Study 2: Information retrieval experiment • Discipline-specific Search-Term- Recommendation (STR) Services in IRM project • Are recommendations from discipline specific STRs better suited for query expansion than general ones? • Co-occurence of terms in title/description and assigned controlled terms • 17 STR services – 16 discipline-spec. – 1 global 6Lüke et al. 2012
  • 7. Case Study 2 • Are recommendations from discipline specific STRs better suited for query expansion (QE) than general ones? • 100 topics from the GIRT corpus, top 4 recommendations to expand the original query • gSTR = global STR; tSTR = topical STR; bSTR = best-performing STR 7 Lüke et al. 2012 • QE with specific STRs leads to significantly better results than QE with a general STR • Selecting the best matching specific STR in an automatic way is a major
  • 8. Case Study 3: Interactive IR experiment • Measuring the utility and performance of Search Term Recommendation (STR) Services in AMUR project • Logfile-based evaluation of STR usage and later search session success • We defined positive signals (export, save, email, full text …) in the system enter_search_term→select_term_from_reco mmender→search→view_record_1→view_r ecord_2→view_record_3→export_record • Analysis of one year of log data 8 Hienert/Mutschke 2016
  • 9. Case Study 3 • Usage of the STR significantly often implicates the occurrence of positive signals during the following session steps 9 Hienert/Mutschke 2016
  • 10. Conclusions • Information retrieval and interactice IR settings are able to demonstrate the utility of KOS usage (usefulness) – In experimental settings – In user evaluations • Each methodology has pros and cons – Effort and significance in small user studies – Too controlled, system-based, without real users • Terminology mapping projects 10 IR Interactive IR Availability of corpora high low Reproducibility high low Control high low Measures medium medium Effort low high Significance medium medium Generalisability medium medium Realistic Scenario no high
  • 11. Outlook • Integrate different recommender systems in real retrieval tasks (search sessions) • Use and evaluate recommenders for query expansion and as dynamic features in IR, in the retrieval process (AMUR project) • Develop new measures of utility of recommender systems – E.g. measure task completion rates or goal satisfaction 11
  • 12. References • Hienert, D. & Mutschke, P. (2016). A Usefulness-based Approach for Measuring the Local and Global Effect of IIR Services. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR '16). ACM, New York, NY, USA, 153-162. http://dx.doi.org/10.1145/2854946.2854962 • Lüke, T., Schaer, P., & Mayr, P. (2012). Improving Retrieval Results with discipline-specific Query Expansion. In International Conference on Theory and Practice of Digital Libraries (TPDL 2012) (pp. 408–413). Paphos, Cyprus: Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-642- 33290-6_44 • Mayr, P., & Petras, V. (2008). Cross-concordances: terminology mapping and its effectiveness for information retrieval. In 74th IFLA World Library and Information Congress. Québec, Canada: IFLA. Retrieved from http://www.ifla.org/IV/ifla74/papers/129-Mayr_Petras-en.pdf 12
  • 13. Thank you Contact: Dr Philipp Mayr GESIS - Leibniz Institute for the Social Sciences, Germany Email: philipp.mayr@gesis.org Twitter: @philipp_mayr 13