UCL Discovery Stage
UCL home » Library Services » Electronic resources » UCL Discovery Stage

Multi-Rater Consensus Learning for Modeling Multiple Sparse Ratings of Affective Behaviour

Romeo, Luca; Olugbade, Temitayo; Pontil, Massimiliano; Berthouze, Nadia; (2023) Multi-Rater Consensus Learning for Modeling Multiple Sparse Ratings of Affective Behaviour. IEEE Transactions on Affective Computing (In press). Green open access

[thumbnail of TAC_MRCL_MutliTask-1.pdf]
Preview
Text
TAC_MRCL_MutliTask-1.pdf - Accepted Version

Download (2MB) | Preview

Abstract

The use of multiple raters to label datasets is an established practice in affective computing. The principal goal is to reduce unwanted subjective bias in the labelling process. Unfortunately, this leads to the key problem of identifying a ground truth for training the affect recognition system. This problem becomes more relevant in a sparsely-crossed annotation where each rater only labels a portion of the full dataset to ensure a manageable workload per rater. In this paper, we introduce a Multi-Rater Consensus Learning (MRCL) method which learns a representative affect recognition model that accounts for each rater’s agreement with the other raters. MRCL combines a multitask learning (MTL) regularizer and a consensus loss. Unlike standard MTL, this approach allows the model to learn to predict each rater’s label while explicitly accounting for the consensus among raters. We evaluated our approach on two different datasets based on spontaneous affective body movement expressions for pain behaviour detection and laughter type recognition respectively. The two naturalistic datasets were chosen for the different forms of labelling (different in affect, observation stimuli, and raters) that they together offer for evaluating our approach. Empirical results demonstrate that MRCL is effective for modelling affect from datasets with sparsely-crossed multi-rater annotation.

Type: Article
Title: Multi-Rater Consensus Learning for Modeling Multiple Sparse Ratings of Affective Behaviour
Open access status: An open access version is available from UCL Discovery
Publisher version: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?pu...
Language: English
Additional information: This version is the author-accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Multiple labels, multitask learning, sparse raters ratings, body expressions, pain behaviour, protective behaviour, laughter types.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
URI: https://discovery-pp.ucl.ac.uk/id/eprint/10173489
Downloads since deposit
3,773Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item