Romeo, Luca;
Olugbade, Temitayo;
Pontil, Massimiliano;
Berthouze, Nadia;
(2023)
Multi-Rater Consensus Learning for Modeling Multiple Sparse Ratings of Affective Behaviour.
IEEE Transactions on Affective Computing
(In press).
Preview |
Text
TAC_MRCL_MutliTask-1.pdf - Accepted Version Download (2MB) | Preview |
Abstract
The use of multiple raters to label datasets is an established practice in affective computing. The principal goal is to reduce unwanted subjective bias in the labelling process. Unfortunately, this leads to the key problem of identifying a ground truth for training the affect recognition system. This problem becomes more relevant in a sparsely-crossed annotation where each rater only labels a portion of the full dataset to ensure a manageable workload per rater. In this paper, we introduce a Multi-Rater Consensus Learning (MRCL) method which learns a representative affect recognition model that accounts for each rater’s agreement with the other raters. MRCL combines a multitask learning (MTL) regularizer and a consensus loss. Unlike standard MTL, this approach allows the model to learn to predict each rater’s label while explicitly accounting for the consensus among raters. We evaluated our approach on two different datasets based on spontaneous affective body movement expressions for pain behaviour detection and laughter type recognition respectively. The two naturalistic datasets were chosen for the different forms of labelling (different in affect, observation stimuli, and raters) that they together offer for evaluating our approach. Empirical results demonstrate that MRCL is effective for modelling affect from datasets with sparsely-crossed multi-rater annotation.
Type: | Article |
---|---|
Title: | Multi-Rater Consensus Learning for Modeling Multiple Sparse Ratings of Affective Behaviour |
Open access status: | An open access version is available from UCL Discovery |
Publisher version: | https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?pu... |
Language: | English |
Additional information: | This version is the author-accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
Keywords: | Multiple labels, multitask learning, sparse raters ratings, body expressions, pain behaviour, protective behaviour, laughter types. |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre |
URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10173489 |
Archive Staff Only
View Item |