UCL Discovery Stage
UCL home » Library Services » Electronic resources » UCL Discovery Stage

Teardrops on My Face: Automatic Weeping Detection from Nonverbal Behavior

Kuster, D; Steinert, L; Baker, M; Bhardwaj, N; Krumhuber, EG; (2022) Teardrops on My Face: Automatic Weeping Detection from Nonverbal Behavior. IEEE Transactions on Affective Computing 10.1109/TAFFC.2022.3228749. (In press). Green open access

[thumbnail of Teardrops_on_My_Face_Automatic_Weeping_Detection_from_Nonverbal_Behavior.pdf]
Preview
Text
Teardrops_on_My_Face_Automatic_Weeping_Detection_from_Nonverbal_Behavior.pdf - Accepted Version

Download (13MB) | Preview

Abstract

Human emotional tears are a powerful socio-emotional signal. Yet, they have received relatively little attention in empirical research compared to facial expressions or body posture. While humans are highly sensitive to others' tears, to date, no automatic means exist for detecting spontaneous weeping. This paper employed facial and postural features extracted using four pre-trained classifiers (FACET, Affdex, OpenFace, OpenPose) to train a Support Vector Machine (SVM) to distinguish spontaneous weepers from non-weepers. Results showed that weeping can be accurately inferred from nonverbal behavior. Importantly, this distinction can be made before the appearance of visible tears on the face. However, features from at least two classifiers need to be combined, with the best models blending three or four classifiers to achieve near-perfect performance (97% accuracy). We discuss how direct and indirect tear detection methods may help to yield important new insights into the antecedents and consequences of emotional tears and how affective computing could benefit from the ability to recognize and respond to this uniquely human signal.

Type: Article
Title: Teardrops on My Face: Automatic Weeping Detection from Nonverbal Behavior
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/TAFFC.2022.3228749
Publisher version: https://ieeexplore.ieee.org/document/9984983
Language: English
Additional information: This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Keywords: Face recognition, Feature extraction, Videos, Affective computing, Emotion recognition, Training data, Psychology
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > Experimental Psychology
URI: https://discovery-pp.ucl.ac.uk/id/eprint/10162563
Downloads since deposit
900Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item