Yu, Youjing; Shi, Zhengxiang; Lipani, Aldo; (2024) Understanding Users’ Confidence in Spoken Queries for Conversational Search Systems. In: Iliadis, Lazaros S and Maglogiannis, Ilias and Papaleonidas, Antonios and Pimenidis, Elias and Jayne, Chrisina, (eds.) Communications in Computer and Information Science. (pp. pp. 405-418). Springer Nature

Text Youjing__EANN___Understanding_User_Confidence_in_Conversational_Search (2).pdf - Accepted Version Access restricted to UCL open access staff until 23 June 2025. Download (413kB)

Abstract

The confidence level in users’ speech has long been recognised as an important signal in traditional dialogue systems. In this work, we highlight the importance of user confidence detection in queries in conversational search systems (CSSs). Accurately estimating a user’s confidence level in CSSs is important because it enables the CSSs to infer the degree of competency of a user on the queried topic and subsequently tailor its responses appropriately. This is especially important in CSSs since their responses need to be concise and precise. However, few prior works have evaluated user confidence in CSSs due to a lack of available datasets. We present a novel speech-based dataset named UNderstanding Spoken qUeRiEs (UNSURE) (Code and instructions on how to obtain this dataset is available at https://github.com/YoujingYu99/confidence_css), which contains confidence grading annotations of user queries in natural language conversations. Based on this dataset, we propose a multimodal approach to infer users’ confidence in spoken queries as a baseline model. Preliminary experimental results demonstrate that our proposed fusion model is capable of achieving near human-level performance.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item