Lipani, A; Carterette, B; Yilmaz, E; (2019) From a User Model for Query Sessions to Session Rank Biased Precision (sRBP). In: The 5th ACM SIGIR International Conference on the Theory of Information Retrieval. (pp. pp. 109-116). ACM: Santa Clara, CA, USA.

Preview

Text From_a_User_Model_for_Query_Sessions_to_Session_Rank_Biased_Precision__sRBP_ (1).pdf - Accepted Version Download (896kB) | Preview

Abstract

To satisfy their information needs, users usually carry out searches on retrieval systems by continuously trading off between the examination of search results retrieved by under-specified queries and the refinement of these queries through reformulation. In Information Retrieval (IR), a series of query reformulations is known as a query-session. Research in IR evaluation has traditionally been focused on the development of measures for the ad hoc task, for which a retrieval system aims to retrieve the best documents for a single query. Thus, most IR evaluation measures, with a few exceptions, are not suitable to evaluate retrieval scenarios that call for multiple refinements over a query-session. In this paper, by formally modeling a user’s expected behaviour over query-sessions, we derive a session-based evaluation measure, which results in a generalization of the evaluation measure Rank Biased Precision (RBP). We demonstrate the quality of this new session-based evaluation measure, named Session RBP (sRBP), by evaluating its user model against the observed user behaviour over the query-sessions of the 2014 TREC Session track

Download activity - last month

Export as XMLJSONCSV

Download activity - last 12 months

Export as CSVJSONXML

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item