Bell, Lauren;
Garnett, C;
Bao, Y;
Cheng, Z;
Qian, T;
Perski, Olga;
Potts, Henry;
(2023)
How notifications affect engagement with a behaviour change app: Results from a micro-randomised trial.
JMIR mHealth and uHealth
, 11
, Article e38342. 10.2196/38342.
Preview |
PDF
PDF (3).pdf - Published Version Download (195kB) | Preview |
Abstract
Background: Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to “Please complete your drinks and mood diary,” yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users’ reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement. Objective: Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy. Methods: We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement. Results: Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different. Conclusions: We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase “in-the-moment” engagement. Further optimization is required to improve the long-term engagement. International Registered Report Identifier (IRRID): RR2-10.2196/18690
Type: | Article |
---|---|
Title: | How notifications affect engagement with a behaviour change app: Results from a micro-randomised trial |
Open access status: | An open access version is available from UCL Discovery |
DOI: | 10.2196/38342 |
Publisher version: | https://doi.org/10.2196/38342 |
Language: | English |
Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
Keywords: | Mobile health; mHealth; digital health; behavior change; behavior change; digital behavior change; engagement; micro-randomized trial; randomized trial; randomization; just-in-time adaptive intervention; adaptive intervention; push notification; notification; excessive alcohol consumption; smartphone app; alcohol; drinking; drinker; mobile phone |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Epidemiology and Health UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Epidemiology and Health > Behavioural Science and Health UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics > CHIME |
URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10171904 |
Archive Staff Only
View Item |