Be the first to like this
22 августа, семинар "RUSSIR Summer School Best Practices"
Ben Carterette "Advances in Information Retrieval Evaluation"
There is great interest in producing effectiveness measures that model user behavior in order to better model the utility of a system to its users. These measures are often formulated as a sum over the product of a discount function of ranks and a gain function mapping relevance assessments to numeric utility values. We develop a conceptual framework for analyzing such effectiveness measures based on classifying members of this broad family of measures into four distinct families, each of which reflects a different notion of system utility. This is a theory of model-based measures within which we can hypothesize about the properties that such a measure should have and test those hypotheses against user and system data.