JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Original Paper A Mobile Phone–Based Intervention to Reduce Mental Health Problems in Health Care Workers During the COVID-19 Pandemic (PsyCovidApp): Randomized Controlled Trial Maria Antònia Fiol-DeRoque1,2*, PhD; Maria Jesús Serrano-Ripoll1,2,3*, PhD; Rafael Jiménez1,3, PhD; Rocío Zamanillo-Campos1,2,4, PhD; Aina María Yáñez-Juan1,5, PhD; Miquel Bennasar-Veny1,6, PhD; Alfonso Leiva1,2,7, PhD; Elena Gervilla1,3, PhD; M Esther García-Buades3, PhD; Mauro García-Toro1, PhD, MD; Pablo Alonso-Coello8,9, PhD, MD; Guadalupe Pastor-Moreno8,10, PhD; Isabel Ruiz-Pérez8,10, PhD, MD; Carolina Sitges1,3,11, PhD; Javier García-Campayo7,12, PhD, MD; Joan Llobera-Cánaves1,2,7, PhD, MD; Ignacio Ricci-Cabello1,2,8, PhD 1 Health Research Institute of the Balearic Islands, Palma de Mallorca, Spain 2 Primary Care Research Unit of Mallorca, Balearic Islands Health Services, Palma de Mallorca, Spain 3 Department of Psychology, University of the Balearic Islands (UIB), Palma de Mallorca, Spain 4 Department of Health, Valencian International University (VIU), Valencia, Spain 5 Research Group on Global Health & Human Development, University of the Balearic Islands, Palma de Mallorca, Spain 6 Department of Nursing and Physiotherapy, University of the Balearic Islands, Palma de Mallorca, Spain 7 Primary Care Prevention and Health Promotion Research Network, Madrid, Spain 8 CIBER de Epidemiología y Salud Pública, Madrid, Spain 9 Department of Clinical Epidemiology and Public Health, Biomedical Research Institute Sant Pau, Iberoamerican Cochrane Centre, Barcelona, Spain 10 Andalusian School of Public Health, Granada, Spain 11 Department of Psychology, Research Institute of Health Sciences, Palma de Mallorca, Spain 12 Aragon Institute for Health Research, Miguel Servet University Hospital, Zaragoza, Spain * these authors contributed equally Corresponding Author: Maria Jesús Serrano-Ripoll, PhD Health Research Institute of the Balearic Islands Edificio S, Hospital Universitario Son Espases Carretera de Valldemossa Palma de Mallorca, 07120 Spain Phone: 34 +34610753696 Email: mariajesus.serranoripoll@ssib.es Abstract Background: The global health emergency generated by the COVID-19 pandemic is posing an unprecedented challenge to health care workers, who are facing heavy workloads under psychologically difficult situations. Mental mobile Health (mHealth) interventions are now being widely deployed due to their attractive implementation features, despite the lack of evidence about their efficacy in this specific population and context. Objective: The aim of this trial is to evaluate the effectiveness of a psychoeducational, mindfulness-based mHealth intervention to reduce mental health problems in health care workers during the COVID-19 pandemic. Methods: We conducted a blinded, parallel-group, controlled trial in Spain. Health care workers providing face-to-face health care to patients with COVID-19 were randomly assigned (1:1) to receive the PsyCovidApp intervention (an app targeting emotional skills, healthy lifestyle behavior, burnout, and social support) or a control app (general recommendations about mental health care) for 2 weeks. The participants were blinded to their group allocation. Data were collected telephonically at baseline and after 2 weeks by trained health psychologists. The primary outcome was a composite of depression, anxiety, and stress (overall score on the Depression Anxiety Stress Scale-21 [DASS-21]). Secondary outcomes were insomnia (Insomnia Severity Index), burnout (Maslach Burnout Inventory Human Services Survey), posttraumatic stress (Davidson Trauma Scale), self-efficacy (General https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 1 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Self-Efficacy Scale), and DASS-21 individual scale scores. Differences between groups were analyzed using general linear modeling according to an intention-to-treat protocol. Additionally, we measured the usability of the PsyCovidApp (System Usability Scale). The outcome data collectors and trial statisticians were unaware of the treatment allocation. Results: Between May 14 and July 25, 2020, 482 health care workers were recruited and randomly assigned to PsyCovidApp (n=248) or the control app (n=234). At 2 weeks, complete outcome data were available for 436/482 participants (90.5%). No significant differences were observed between the groups at 2 weeks in the primary outcome (standardized mean difference –0.04; 95% CI –0.11 to 0.04; P=.15) or in the other outcomes. In our prespecified subgroup analyses, we observed significant improvements among health care workers consuming psychotropic medications (n=79) in the primary outcome (–0.29; 95% CI –0.48 to –0.09; P=.004), and in posttraumatic stress, insomnia, anxiety, and stress. Similarly, among health care workers receiving psychotherapy (n=43), we observed improvements in the primary outcome (–0.25; 95% CI –0.49 to –0.02; P=.02), and in insomnia, anxiety, and stress. The mean usability score of PsyCovidApp was high (87.21/100, SD 12.65). After the trial, 208/221 participants in the intervention group (94.1%) asked to regain access to PsyCovidApp, indicating high acceptability. Conclusions: In health care workers assisting patients with COVID-19 in Spain, PsyCovidApp, compared with a control app, reduced mental health problems at 2 weeks only among health care workers receiving psychotherapy or psychotropic medications. Trial Registration: ClinicalTrials.gov NCT04393818; https://clinicaltrials.gov/ct2/show/NCT04393818. (JMIR Mhealth Uhealth 2021;9(5):e27039) doi: 10.2196/27039 KEYWORDS COVID-19; randomized controlled trial; mental health; health care workers; mHealth; app interventions is supported by the positive results of acceptance Introduction rates [9] and sustainability [10] observed in other contexts and Since the declaration of the COVID-19 pandemic, the disease populations. Recent trials have examined the efficacy of has spread globally, with almost 55 million known cases and a mHealth interventions addressing mental health problems, death toll of over 1.3 million people [1]. Early anecdotal including depression [11,12], suicide [13], schizophrenia [14], evidence from Wuhan showed how this unprecedented situation substance use disorders [15], and psychosis [16], among others impacted the mental health of frontline health care workers [2]. [17]. Recent systematic reviews investigating the efficacy of This was confirmed by subsequent systematic reviews [3,4], in smartphone apps for mental health show that these apps can which a remarkably high prevalence of acute stress (40%), produce significant reductions in anxiety [18] and depression anxiety (30%), burnout (28%), depression (24%), and [19]. However, the effectiveness of mHealth interventions in posttraumatic stress disorder (13%) was observed among health care workers in the COVID-19 pandemic context is frontline health care workers. Working in an environment with largely unknown, as observed by recent reviews that highlighted a high risk of infection, job-related stress, heavy workloads, the lack of evaluations of client-relevant outcomes [20] and the and lack of protective equipment were found to significantly lack of both quantitative and qualitative evidence to inform the exacerbate these psychological problems [3]. On May 8, 2020, selection of interventions that are beneficial to the mental health Spain reported the highest cumulative number of COVID-19 of frontline health care workers [21]. Hence, robust, large-scale infections among health care workers worldwide (30,663 trials are urgently needed to determine the extent to which infections, accounting for 20% of all infections in health care mHealth interventions can improve the mental health of frontline workers worldwide) [5]. Not surprisingly, the psychological health care workers. consequences disproportionally affected the mental health of This blinded, individually randomized, parallel-group, controlled Spanish health care workers, with approximately 57% of them trial aimed to evaluate the effectiveness of PsyCovidApp (a presenting symptoms of posttraumatic stress disorder, 59% self-managed and self-guided psychoeducational mobile-based presenting symptoms of anxiety disorder, 46% presenting intervention with no therapist support) to reduce symptoms of symptoms of depressive disorder, and 41% feeling emotionally depression, anxiety, stress and other mental health problems in drained [6]. health care workers during the COVID-19 pandemic in Spain. Health services worldwide are being urged to implement strategies to mitigate the severe psychological consequences Methods experienced by health care workers. Among the different types of strategies considered, mobile health (mHealth) interventions Design and Setting are receiving special attention [7] not only because of their We conducted a blinded, individually randomized, attractive implementation features but also because they can be parallel-group, controlled trial in Spain. Because the ultimate delivered in the absence of face-to-face interactions, reducing goal of the study was to inform decisions about rolling up the the risk of infection with SARS-CoV-2. Further, they can intervention to make it available to all health care workers in address non–treatment-seeking behavior (a common issue Spain, we used a pragmatic approach. Pragmatic trials are ideally among health care workers [8]), as they provide the opportunity suited to inform choices between treatments because as opposed to engage individuals in need of treatment in a timely and to exploratory trials (which typically examine treatment benefits anonymous fashion. This growing interest in mHealth under ideal conditions using carefully defined subjects), https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 2 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al pragmatic trials enable measurement of the effectiveness of of the PsyCovidApp intervention or the control contents. Within interventions under real conditions in a sample of participants 48 hours after participants successfully downloaded and to whom the treatment would be applied [22]. Participants were activated the app (user activation was used as a checkpoint to individually randomized with an allocation ratio of 1:1 to receive ensure the participants could successfully access the app), a either the PsyCovidApp intervention or the control app over 2 member of our research team loaded the contents to the app weeks. Ethical approval was obtained by the Research Ethics according to the group the participants had been allocated to. Committee of the Balearic Islands (IB 4216/20 PI). The study During the following 14 days, all health care workers had access protocol and statistical analysis plan have been published to the content of their assigned group (PsyCovidApp previously (ClinicalTrials.gov NCT04393818) [23]. intervention or control). The PsyCovidApp intervention was developed by a group of psychologists (MJSR, EG, CS, RJ, Participants MEGB), psychiatrists (JGC, MGT), and experts in healthy The target population was male and female health care workers lifestyle promotion (AMYJ, MBV), informed by findings from aged >18 years who had provided health care to patients with an exploratory qualitative study involving in-depth interviews COVID-19 during the viral outbreak in Spain. We included with 9 health care workers seeking psychological support as a health care workers from any medical specialty (pneumology, result of their professional activity during the COVID-19 internal medicine, emergency, primary care, etc) and role pandemic (unpublished results). PsyCovidApp was specifically (physicians, nurses, nurse assistants, etc) with access to a designed to prevent and mitigate the most frequent mental smartphone. We included health care workers who had provided problems suffered by health care workers who are dealing with direct, face-to-face health care to patients with a diagnosis of the COVID-19 emergency (depression, anxiety, posttraumatic infection with COVID-19. We excluded health care workers stress, and burnout). A detailed description of the intervention who were not able to download and activate the app used to is available elsewhere [23]. In short, the self-managed deliver the intervention during the next 10 days following the psychoeducational intervention, based on cognitive-behavioral baseline assessment. therapy and mindfulness approaches, included written and Following a purposive sampling method, we sent invitations to audiovisual content targeting four areas: emotional skills, health care workers to participate in the trial through direct healthy lifestyle behavior, work stress and burnout, and social contact via email and telephone to key stakeholders (115 hospital support. Additionally, the intervention included daily prompts and home care centers, 138 professional associations, and 8 (notifications) that included brief questionnaires to monitor scientific societies and trade unions) and by social media. Health mental health status, followed by short messages offering care workers who were willing to participate registered their tailored information and resources based on the participants´ interest by completing a web-based questionnaire, consenting responses. The full content of the intervention is available in to be contacted telephonically. A team of 23 health psychologists Multimedia Appendix 1. who had previously received a 2-hour training session (to ensure Participants in the Control App group had access through the homogeneity in recruitment, questionnaire administration, and Clinicovery app to brief written information about the mental data entry methods) contacted the registered health care workers health care of health care workers during the COVID-19 by telephone to confirm the eligibility criteria and obtain pandemic (adapted from a set of materials developed by the informed consent (audio-recorded). The recruitment period Spanish Society of Psychiatry; the contents are available in spanned 10 weeks, from May 14 to July 25, 2020. Participants Multimedia Appendix 2). were enrolled on a rolling “first-come-first-served” basis until target sample sizes were met. To incentivize trial participation, After 2 weeks, the apps in both groups were disabled, and a we offered participation certificates to all health care workers postintervention psychological assessment was undertaken. The completing the postintervention assessment. follow-up was undertaken via telephone between 24 hours and 10 days after the intervention concluded, and it included the Randomization and Masking same questionnaires used in the first evaluation and the System Participants were randomly assigned (1:1) to receive the Usability Scale (SUS) [24]. Once the postintervention PsyCovidApp intervention or the control app over 2 weeks by assessment had finished, all participants were offered free, a designated researcher (MAF, who was not involved in data unrestricted access to PsyCovidApp. collection or analysis) using a computer-generated sequence of Outcome Measures random numbers created by internet relay chat (IRC). Randomization was not stratified. Health care workers were The primary outcome was an overall index of depression, blinded to group allocation (as both groups received an app). anxiety, and stress (overall score of the Spanish version of the The outcome data collectors and trial statisticians were unaware Depression, Anxiety, and Stress Scale [DASS-21] instrument of the treatment allocation. [25]) assessed at 2 weeks. The score ranges from 0 (best outcome) to 21 (worst outcome). The instrument contains three Procedures 7-item subscales assessing the presence and intensity of Immediately after obtaining informed consent, a team of depression, anxiety, and stress. The items are based on a Likert psychologists conducted a psychological (preintervention) scale ranging from 0 to 3 points. The instrument shows adequate evaluation via telephone interview and instructed participants internal consistency (Cronbach α=.91) and construct validity on how to download the Clinicovery App (Apploading, Inc). (3-factor structure identified after exploratory factor analysis, Clinicovery is the app that was used to deliver either the contents explaining 50% of the total variance) [26]. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 3 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Secondary outcome measures were the difference between the at least an effect size of 0.25 (standardized between-group mean intervention and control groups in the mean scores of the difference) on DASS-21 with 80% power and 5% α (one-sided). following instruments: The analyses followed the agreed statistical analysis plan, • Davidson Trauma Scale (DTS) [27]. The DTS is a 17-item published before database lock [23]. Descriptive statistics self-report Likert scale instrument that assesses the 17 summarizing prerandomization variables, and outcome measures symptoms of posttraumatic stress disorder in the Diagnostic at 2 weeks were reported by treatment group and overall. and Statistical Manual of Mental Disorders, Fourth Edition. Differences between groups of primary and secondary outcomes Both a frequency and a severity score can be determined. were analyzed using general linear modelling (analysis of The DTS yields a frequency score (ranging from 0 to 68), covariance) for continuous variables, adjusted by baseline score. severity score (ranging from 0 to 68), and total score We report standardized between-group differences in primary (ranging from 0 to 136). Higher scores are indicative of a and secondary outcome measures at 2 weeks. In those outcomes worse outcome. The Spanish version of the instrument that were interpreted as being in favor of PsyCovidApp if the shows adequate reliability (test-retest intraclass standardized group difference was more than 0, the estimated correlation=0.87; α=.90) as well as high discriminant effect was reflected (ie, multiplied by –1) so that outcomes for validity [28]. For this study, the instrument was adapted to which the standardized group difference was lower than 0 could measure posttraumatic stress disorder since the onset of the be homogenously interpreted as being in favor of the COVID-19 pandemic. This adaptation consisted of intervention. In the primary statistical analysis, all health care reformulating the stem of all the items, replacing “during workers who agreed to participate were included in the analysis the last week” with “since the onset of the current according to the group to which they were assigned. We used COVID-19 pandemic”. multiple imputation by chained equations to fill in missing • Maslach Burnout Inventory - Human Services Survey values (50 imputation sets) [38]. In Multimedia Appendix 3, (MBI-HSS) [29], which yields specific scores for each of we report unstandardized between-group differences. In the its three subscales (emotional exhaustion, α=.85; analysis, P values and CIs were not corrected for multiple depersonalization, α=.58; and professional accomplishment, secondary outcome comparisons. α=.71). The Spanish version of the instrument shows We conducted three prespecified subgroup analyses to examine adequate internal consistency (except for the the impact of the PsyCovidApp intervention on the primary and depersonalization subscale) and adequate factorial validity secondary outcomes based on the following baseline [30]. characteristics: use of psychotropic medications (yes vs no), • Insomnia Severity Index (ISI) [31]. The ISI is a 7-item, use of psychotherapy (yes vs no), and symptomatology of self-reported Likert scale instrument assessing the severity depression, anxiety, and stress (yes vs no, based on baseline of both nighttime and daytime components of insomnia. DASS-21 median overall score). We conducted statistical tests Scores range from 0 (best outcome) to 28 (worst outcome). for interaction (including an interaction term in the models) to In the Spanish version, principal component analysis determine whether chance was an unlikely explanation for the showed only one factor explaining 69% of the total apparent subgroup effects identified. We used the Instrument variance, with an internal consistency reliability of 0.91. to Assess the Credibility of Effect Modification Analyses Regarding its validity, the ISI shows statistically significant (ICEMAN) [39] to assess the credibility of our subgroup positive correlations with the Athens Insomnia Scale-5 analyses. As a sensitivity analysis, we reanalyzed all outcomes (r=0.93) and negative correlations with the Mini Mental on a complete case basis (ie, without imputation or adjustment State Examination (r=–0.15) [32]. for baseline predictors of missingness). We used SPSS, version • General Self-Efficacy Scale (GSE) [33]. The GSE is a 25 (IBM Corporation) and Stata, version 13 (StataCorp LLC) 10-item, Likert-scale, self-reported instrument that assesses to conduct the statistical analyses. optimistic self-beliefs to cope with a variety of difficult demands in life. Scores range from 10 (worst outcome) to Role of the Funding Source 40 (best outcome). The Spanish version shows adequate The funder of the study had no role in study design, data internal consistency (α=.85) and construct validity [34]. collection, data analysis, data interpretation, or writing of the • Individual subscales of the DASS-21 instrument: depression report. All authors had full access to all the data in the study (α=.90), anxiety (α=.88), and stress (α=.88) [35,36]. and had final responsibility for the decision to submit for • Usability of PsyCovidApp at postintervention: SUS [24]. publication. Higher scores are indicative of higher usability, and scores above the 68-point threshold can be considered to indicate Results high usability. The Spanish version shows adequate internal consistency (α=.80) and concurrent validity (significant Between May 14 and July 25, 2020, 684 health care workers correlation with the Adjective Rating Scale; r=0.56) [37]. submitted an expression of interest in enrolling in the Statistical Analysis PsyCovidApp trial. 482 eligible participants provided informed consent and were randomly assigned to the PsyCovidApp The sample size and power calculations have been described intervention group (n=248) or the Control App group (n=234; previously [23]. We estimated that 440 participants (220 per Figure 1). Recruitment by region is shown in Multimedia group, allowing for 10% attrition) would be required to detect https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 4 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Appendix 4. Participants included health care workers from all (31.7%) were physicians, and 147 (30.5%) were nurse assistants. regions in Spain except the Canary Islands and Cantabria. Most worked in the hospital setting: 98/482 (20.3%) in internal medicine, 81/482 (16.8%) in intensive care units, 79/482 The baseline characteristics were balanced between the groups (16.4%) in hospital emergency units, 31/482 (6.4%) in infection (Table 1). Most participants were women (401/482; 83.2%), units, and 103/482 (21.4%) in other hospital units. Of the 482 and the median age was 42 years (IQR 33-49). Approximately participants, 61 (12.7%) worked in primary care and 29 (6%) one-third (161, 33.4%) of the 482 participants were nurses, 153 in home-care settings. Figure 1. Trial profile. Multiple imputation was used to facilitate the overall sample analysis; all randomized participants contributed to the statistical analysis. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 5 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Table 1. Baseline demographic characteristics of the intention-to-treat population (N=482). Characteristic PsyCovidApp group (n=248) Control App group (n=234) Total (N=482) Age, years Mean (SD) 42.07 (11.0) 40.62 (9.6) 41.37 (10.4) Median (IQR; range) 42 (34-51; 23-63) 41 (32-47; 23-61) 41.5 (33-49; 23-63) <36, n (%) 75 (30.2) 82 (35) 157 (32.6) 36-45, n (%) 79 (31.9) 81 (34.6) 160 (33.2) 46-55, n (%) 60 (24.2) 52 (22.2) 112 (23.2) >55, n (%) 34 (13.7) 19 (8.1) 53 (11) Gender, n (%) Male 38 (15.3) 43 (18.4) 81 (16.8) Female 210 (84.7) 191 (81.6) 401 (83.2) Occupational role, n (%) Physician 76 (30.6) 77 (32.9) 153 (31.7) Nurse 87 (35.1) 74 (31.6) 161 (33.4) Nurse assistant 77 (31) 70 (29.9) 147 (30.5) Other 8 (3.2) 13 (5.6) 22 (4.6) Setting, n (%) Primary care 35 (14.1) 26 (11.1) 61 (12.7) Internal medicine 48 (19.4) 50 (21.4) 98 (20.3) Intensive care unit 40 (16.1) 41 (17.5) 81 (16.8) Hospital emergencies unit 31 (12.5) 48 (20.5) 79 (16.4) Home care 19 (7.7) 10 (4.3) 29 (6) Infection unit 16 (6.5) 15 (6.4) 31 (6.4) Other hospital unit 59 (23.8) 44 (18.8) 103 (21.4) Time working with patients with COVID-19 (weeks), n (%) <2 9 (3.6) 4 (1.7) 13 (2.7) 2-4 20 (8.1) 14 (6) 34 (7.1) >4 219 (88.3) 216 (92.3) 435 (90.2) Infected with COVID-19, n (%) Yes 31 (12.5) 34 (14.5) 65 (13.5) No 214 (86.3) 195 (83.3) 409 (84.9) Unknown 3 (1.2) 5 (2.1) 8 (1.7) Perception about the adequacy of available measures to protect health care workers from COVID-19, n (%) Inadequate measures 84 (33.9) 77 (32.9) 161 (33.4) Adequate measures 163 (65.7) 157 (67.1) 320 (66.4) Perception about the information about the procedures to provide health care to patients with COVID-19, n (%) Inadequate information 101 (40.7) 116 (49.6) 217 (45) Adequate information 147 (59.3) 117 (50) 264 (54.8) Currently consuming psychotropic medication, n (%) No 207 (83.5) 196 (83.8) 403 (83.6) Yes 41 (16.5) 38 (16.2) 79 (16.4) Currently receiving psychotherapy, n (%) No 227 (91.5) 212 (90.6) 439 (91.1) https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 6 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Characteristic PsyCovidApp group (n=248) Control App group (n=234) Total (N=482) Yes 21 (8.5) 22 (9.4) 43 (8.9) By the time of recruitment, most participants (435/482, 90.2%) In relation to their mental health, 206 of the 482 participants had been providing health care to patients with COVID-19 for (42.7%) presented symptoms of depression, 250 (51.9%) had more than 4 weeks, and 65/482 (13.5%) had received a diagnosis symptoms of anxiety, 292 (60.6%) had symptoms of stress, 194 of COVID-19 infection. Approximately one-third (161/482, (40.2%) had symptoms of posttraumatic stress, and 128 (26.6%) 33.4%) perceived that the measures offered to protect them from had symptoms of insomnia (Table 2). Concerning burnout, COVID-19 had been inadequate, and 217/482 (45%) perceived 282/482 participants (58.5%) presented emotional exhaustion, that they had received inadequate information about the 165/482 (34.2%) presented emotional depersonalization, and procedures to provide health care to patients with COVID-19. 203/482 (42.1%) presented moderate or low professional Of the 482 participants, 79 (16.4%) were using psychotropic accomplishment. The mean self-efficacy score was 32.2 out of medications, and 43 (8.9%) were receiving psychotherapy. 40 (SD 4.7), indicating a high level of self-efficacy. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 7 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Table 2. Baseline clinical characteristics of the intention-to-treat population (N=482). Characteristic PsyCovidApp group (n=248) Control App group (n=234) Total (N=482) Depression, anxiety, and stress DASS-21a overall score, mean (SD) 5.8 (3.9) 6.1 (3.8) 6.0 (3.8) Depression (DASS-21 subscale), n (%) No symptoms (<5 points) 143 (57.7) 133 (56.8) 276 (57.3) Mild (5-6 points) 26 (10.5) 32 (13.7) 58 (12) Moderate (7-10 points) 55 (22.2) 48 (20.5) 103 (21.4) Severe (11-13 points) 15 (6) 11 (4.7) 26 (5.4) Extremely severe (>13 points) 9 (3.6) 10 (4.3) 19 (3.9) Anxiety (DASS-21 subscale), n (%) No symptoms (<4 points) 121 (48.8) 111 (47.4) 232 (48.1) Mild (4 points) 32 (12.9) 25 (10.7) 57 (11.8) Moderate (5-7 points) 40 (16.1) 42 (17.9) 82 (17) Severe (8-9 points) 24 (9.7) 20 (8.5) 44 (9.1) Extremely severe (>9 points) 31 (12.5) 36 (15.4) 67 (13.9) Stress (DASS-21 subscale), n (%) No symptoms (<8 points) 104 (41.9) 86 (36.8) 190 (39.4) Mild (8-9 points) 28 (11.3) 32 (13.7) 60 (12.4) Moderate (10-12 points) 56 (22.6) 58 (24.8) 114 (23.7) Severe (13-16 points) 43 (17.3) 45 (19.2) 88 (18.3) Extremely severe (>16 points) 17 (6.9) 13 (5.6) 30 (6.2) Posttraumatic stress (DTSb), n (%) No (<40 points) 150 (60.5) 138 (59) 288 (59.8) Yes (≥40 points) 98 (39.5) 96 (41) 194 (40.2) Burnout (MBI-HSSc), n (%) Emotional exhaustion Low (0-16 points) 95 (38.3) 89 (38) 184 (38.2) Moderate (17-26 points) 61 (24.6) 50 (21.4) 111 (23) High (>27 points) 92 (37.1) 95 (40.6) 187 (38.8) Professional accomplishment subscale High (>39 points) 144 (58.1) 135 (57.8) 279 (57.9) Moderate (32-38 points) 65 (26.2) 54 (23.1) 119 (24.7) Low (0-31 points) 39 (15.7) 45 (19.2) 84 (17.4) Depersonalization subscale Low (0-6 points) 163 (65.7) 154 (65.8) 317 (65.8) Moderate (17-12 points) 50 (20.2) 31 (13.2) 81 (16.8) High (>13 points) 35 (14.1) 49 (20.9) 84 (17.4) Insomnia (ISId), n (%) Not clinically significant (0-7 points) 102 (41.1) 87 (37.2) 189 (39.2) Subthreshold insomnia (8-14 points) 89 (35.9) 76 (32.5) 165 (34.2) Clinical insomnia (moderate severity) (15-21 49 (19.8) 61 (26.1) 110 (22.8) points) https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 8 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Characteristic PsyCovidApp group (n=248) Control App group (n=234) Total (N=482) Clinical insomnia (severe) (22–28) 8 (3.2) 10 (4.3) 18 (3.7) Self-efficacy (GSEe) Mean score out of 40 points (SD) 32.4 (4.7) 32 (4.7) 32 (4.7) a DASS-21: Depression, Anxiety, and Stress Scale-21. b DTS, Davidson Trauma Scale. c MBI-HSS: Maslach Burnout Inventory - Human Services Survey. d ISI: Insomnia Severity Index. e GSE: General Self-Efficacy Scale. At 2 weeks, 27 of the 248 participants (10.9%) in the Primary and secondary outcome data were available for 436 of PsyCovidApp group and 19 of the 234 participants (8.1%) in the 482 participants (90.5%): 221 of 248 (89.1%) health care the Control App group were lost to follow-up because they workers in the PsyCovidApp intervention group versus 215 of decided to withdraw from the study at the time of the 234 (91.9%) in the Control App group. For the primary outcome, postintervention psychological assessment (6 in the intervention scale scores were lower at 2 weeks than at baseline in the group and 6 in the control group) or because we were unable PsyCovidApp and Control App groups (Figure 2). Similar to reach them for the telephonic postintervention psychological reductions were observed at 2 weeks in both groups for all the assessment (21 in the intervention group and 13 in the control secondary outcomes except for self-efficacy and group). None of the participants were deemed to be associated depersonalization. with reported adverse events or death. Figure 2. Changes in median DASS-21 scores over time, with the raw data plot of the median DASS-21 scores. Baseline scores were recorded before randomization. The effect sizes for all outcomes are shown in Table 3, Table difference –0.04; 95% CI –0.11 to 0.04; P=.15). Similarly, none 4, and Multimedia Appendix 5. For the primary outcome, no of the secondary outcomes significantly differed between groups significant differences in the DASS-21 overall score were at 2 weeks (all P>.05). identified between the groups at 2 weeks (standardized mean https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 9 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Table 3. Descriptive summaries of the primary and secondary outcome measures at baseline and 2 weeks for the PsyCovidApp and Control App groups. Measure At baseline At 2 weeks PsyCovidApp Control App group Overall (N=482) PsyCovidApp Control App group Completers at fol- group (n=248) (n=234) group (n=221) (n=215) low-up (n=436) Mean Range Mean Range Mean Range Mean Range Mean Range Mean Range (SD) (SD) (SD) (SD) (SD) (SD) Primary outcome DASS-21a overall 5.84 0- 6.14 0- 5.99 0- 3.83 0- 4.27 0- 4.05 0- score (3.85) 17.33 (3.77) 16.33 (3.81) 17.33 (3.21) 16.33 (3.47) 15.33 (3.35) 16.33 Secondary outcomes DASS-21 depression 4.46 0-18 4.58 0-15 4.51 0-18 2.97 0-17 3.05 0-18 3.01 0-18 subscale (4.13) (4.02) (4.07) (3.49) (3.65) (3.56) DASS-21 anxiety 4.35 0-16 4.70 0-18 4.52 0-18 2.21 0-15 2.84 0-17 2.64 0-17 subscale (3.86) (4.25) (4.06) (2.43) (3.36) (3.13) DASS-21 stress sub- 8.75 0-21 9.15 0-19 8.94 0-21 6.11 0-21 6.94 0-21 6.51 0-21 scale (5.07) (4.63) (4.86) (4.50) (4.68) (4.60) Posttraumatic stress 34.57 0-117 36.91 0-100 35.71 0-117 24.91 0-96 26.36 0-91 25.62 0-96 (DTSb) (23.47) (23.18) (23.33) (20.41) (21.02) (20.70) Burnout (MBI- 23.27 0-54 23.57 0-54 23.41 0-54 19.43 0-51 19.67 0-54 19.54 0-54 HSSc) emotional ex- (12.20) (12.34) (12.26) (12.25) (12.91) (12.57) haustion subscale Burnout (MBI-HSS) 39.69 10-48 39.59 15-48 39.64 10-48 40.33 13-48 39.54 15-48 39.94 13-48 professional accom- (6.43) (6.62) (6.52) (6.31) (6.93) (6.63) plishment subscaled Burnout (MBI-HSS) 4.69 0-29 5.24 0-23 4.95 0-29 4.51 0-29 4.78 0-23 4.64 0-29 depersonalization (5.08) (5.41) (5.25) (4.96) (5.25) (5.10) subscale Insomnia (ISIe) 9.80 0-26 10.16 0-27 9.98 0-27 8.07 0-28 8.44 0-23 8.25 0-28 (6.19) (6.53) (6.36) (6.18) (6.68) (6.43) Self-efficacy 32.42 19-40 32.00 18-40 32.21 18-40 33.22 18-40 32.54 17-40 32.88 17-40 f d (4.71) (4.73) (4.72) (4.65) (4.88) (4.77) (GSE ) a DASS-21: Depression, Anxiety, and Stress Scale-21. b DTS: Davidson Trauma Scale. c MBI-HSS: Maslach Burnout Inventory - Human Services Survey. d Scale scores are reversed to homogeneously convey a similar treatment effect. e ISI: Insomnia Severity Index. f GSE: General Self-Efficacy Scale. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 10 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Table 4. Comparison of outcome measures between the PsyCovidApp and the Control App groups at 2 weeks. Data are adjusted standardized between-group mean differences with 95% CIs in parentheses. P values are not adjusted for multiple testing. Measure Sample of completers at follow-up (n=436) Overall samplea (N=482) Adjusted standardized between-group P value Adjusted standardized between-group P value mean differences (95% CI) mean differences (95% CI) DASS-21b overall score –0.04 (-0.11 to 0.04) .16 –0.04 (–0.11 to 0.04) .15 DASS-21 depression subscale 0.00 (–0.07 to 0.08) .47 0.00 (–0.07 to 0.08) .47 DASS-21 anxiety subscale –0.04 (–0.12 to 0.04) .15 –0.04 (–0.12 to 0.04) .17 DASS-21 stress subscale –0.06 (–0.14 to 0.01) .06 –0.06 (–0.14 to 0.01) .05 Posttraumatic stress (DTSc) 0.00 (–0.06 to 0.06) .47 0.00 (–0.06 to 0.07) .47 Burnout (MBI-HSSd) emotional exhaustion 0.01 (–0.06 to 0.08) .38 0.01 (–0.06 to 0.08) .39 subscale Burnout (MBI-HSS) professional accom- –0.04 (–0.12 to 0.03) .13 –0.05 (–0.12 to 0.03) .12 plishment subscalee Burnout (MBI-HSS) depersonalization 0.01 (–0.06 to 0.09) .36 0.01 (–0.06 to 0.09) .36 subscale Insomnia (ISIf) 0.01 (–0.05 to 0.07) .38 0.01 (–0.05 to 0.07) .38 Self-efficacy (GSEg)e –0.02 (–0.10 to 0.05) .26 –0.02 (–0.01 to 0.05) .27 a Overall sample, derived by multiple imputation (50 imputations). b DASS-21: Depression, Anxiety, and Stress Scale-21. c DTS: Davidson Trauma Scale. d MBI-HSS, Maslach Burnout Inventory - Human Services Survey. e Scale scores reversed to homogeneously convey a similar treatment effect. f ISI: Insomnia Severity Index. g GSE: General Self-Efficacy Scale. The impact of the intervention on prespecified subgroups of In the subgroup of participants receiving psychotherapy (n=43) health care workers is presented in Figures 3 and 4 and (Figure 4), the PsyCovidApp group presented significantly Multimedia Appendix 3. In the subgroup of health care workers lower DASS-21 overall scores (suggesting improved mental consuming psychotropic medications (n=79) (Figure 3), the health) at 2 weeks than the Control App group; adjusted PsyCovidApp group presented significantly lower DASS-21 standardized mean difference –0.25; 95% CI –0.49 to –0.02; overall scores (suggesting improved mental health) at 2 weeks P=.02). Compared to the control app, the PsyCovidApp than the Control App group (adjusted standardized mean intervention significantly improved symptoms of anxiety (–0.24; difference –0.29; 95% CI –0.48 to –0.09; P=.004). Compared 95% CI –0.48 to 0.00; P=.02), stress (–0.27; 95% CI –0.55 to to the control app, the PsyCovidApp intervention significantly 0.001; P=.02), and insomnia (–0.20; 95% CI –0.42 to 0.02; improved symptoms of anxiety (–0.26; 95% CI –0.45 to –0.08; P=.03); meanwhile. no statistically significant differences were P=.004), stress (–0.30; 95% CI –0.50 to –0.09; P=.003), observed for symptoms of depression, posttraumatic stress posttraumatic stress (–0.20; 95% CI –0.37 to –0.03; P=.01), and disorder, emotional exhaustion, professional accomplishment, insomnia (–0.16; 95% CI –0.30 to –0.02; P=.01); meanwhile, depersonalization, or self-efficacy (P>.05). No statistically no differences were observed for symptoms of depression, significant differences were observed in any of the outcomes emotional exhaustion, professional accomplishment, in the group of health care workers not receiving psychotherapy depersonalization, or self-efficacy (all P>.05). No significant (n=439). The interaction P values for anxiety, stress, and differences were observed in any of the outcomes in the group insomnia were <.05. of health care workers not consuming psychotropic medications No statistically significant differences (P>.05) were observed (n=403). The interaction P values for anxiety, stress, in the primary outcome or in any of the secondary outcomes posttraumatic stress, and insomnia were all <.05, suggesting examined in the subgroups of health care workers with higher that the apparent interaction was not a chance finding. and lower baseline DASS-21 scores (based on baseline DASS-21 median overall score). https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 11 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Figure 3. Standardized mean differences for primary and secondary outcomes in healthcare workers reporting the use of psychotropic medications at baseline. Forest plot of standardized group differences between PsyCovidApp and Control App groups for all outcomes, whereby an effect lower than 0 favored the PsyCovidApp group. Error bars show 95% Confidence Intervals (CIs). DASS-21, Depression, Anxiety, and Stress Scale. *P<.05. Figure 4. Standardized mean differences for primary and secondary outcomes in health care workers reporting the use of psychotherapy at baseline. Forest plot of standardized group differences between the PsyCovidApp and Control App groups for all outcomes, whereby an effect lower than 0 favored the PsyCovidApp group. Error bars show 95% CIs. DASS-21: Depression, Anxiety, and Stress Scale. *P<.05. The usability of the PsyCovidApp intervention is described in 12.65). After the trial, 208 of the 221 participants in the Table 5. In general, participants perceived that PsyCovidApp intervention group (94.1%) asked the research team if they could was highly usable (mean overall usability score 87.21/100; SD regain access to the PsyCovidApp intervention. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 12 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Table 5. Usability of the PsyCovidApp intervention by the PsyCovidApp group (n=221) measured with the System Usability Scale (the theoretical score range is 0-4 for single items; higher scores are indicative of higher usability). Item Mean score (SD) I would like to use this App frequently 3.00 (1.02) The App is not unnecessarily complex 3.39 (1.03) The App is easy to use 3.56 (0.88) No need for the support of a technical person to use this App 3.73 (0.81) The various functions in this App are well integrated 3.40 (0.83) Not too much inconsistency in this App 3.61 (0.81) Most people would learn to use this App very quickly 3.43 (0.82) The App is not very cumbersome to use 3.70 (0.70) Confidence using the App 3.35 (0.95) No need to learn a lot of things to use the App 3.69 (0.84) Overall usability scorea 87.21 (12.65) a Overall System Usability Scale theoretical score range: 0-100 (higher scores are indicative of higher usability). The sensitivity analysis of all outcomes on a complete case basis the control group (which consisted of a similar app but with (ie, without imputation or adjustment for baseline predictors of access limited to general information and contents) also had a missingness) is shown in Table 4. The results of these sensitivity positive effect. The impact of the control app may have been analyses had little effect on the results, with similar findings enhanced by the Hawthorne effect [40] because the stimulus for the primary outcome and the secondary outcomes when introduced by the control app could have induced a positive compared with the results of the main analyses. behavioral change due to awareness of being observed. Therefore, we cannot rule out the possibility that the use of a Discussion passive comparator (eg, waiting list) in our trial may have resulted in a different outcome for PsyCovidApp. Principal Results It is plausible that the intervention did not produce the desired The global health emergency generated by the COVID-19 effects because of the short trial duration (ie, too short for the pandemic is posing an unprecedented challenge to frontline intervention to produce the intended benefits). During the time health care workers, who are facing high levels of workload of the study, health care workers in Spain were overwhelmed under psychologically difficult situations with scarce resources with heavy workloads, and it is likely that a large proportion and support. To our knowledge, this is the first randomized struggled to find time to use PsyCovidApp during the 2-week controlled trial to date to assess the efficacy of a mental mHealth intervention period. Suboptimal use of mental health apps is intervention for frontline health care workers fighting the health indeed a widely acknowledged challenge: user retention rate emergency generated by the COVID-19 pandemic. Our analysis for smartphone apps in the general population is low, and showed that at 2 weeks, PsyCovidApp only produced significant approximately 25% of users abandon apps after one use [41]. improvements in the primary and secondary outcomes of health As pointed out by a recent systematic review of apps for care workers who were receiving psychotherapy or psychotropic depression and anxiety, more than 70% of users stopped using medications. mental health apps after 6 weeks [42]. Unfortunately, we were Implications for Clinical Practice not able to register the time of use, and we were therefore not able to explore a potential dose-effect relationship. Although the results of our trial indicate that the PsyCovidApp intervention was not effective in comparison with the control In any case, as it stands, the trial showed that the PsyCovidApp app in the overall health care worker population, we cannot rule intervention did not produce significant improvements in the out the possibility that the intervention produced beneficial primary and secondary outcomes in the overall population when effects that our trial was not able to detect for various reasons, compared with a control app. It could be interpreted that the including the choice of an active comparator and the level of PsyCovidApp intervention was not effective in improving use of the intervention. Concerning the choice of an active mental health outcomes in the short term in this specific comparator, most of the outcomes at 2 weeks improved similarly population and context. It could be argued that considering all in both the intervention and control group. The improvements the issues health care workers are required to deal with, in the Control App group could be attributed to the natural providing psychological aid only through a mHealth intervention progression of the disease in a context of decreasing levels of may not be sufficient to produce significant improvements. external stressors (as the impact of the first wave of COVID-19 In our subgroup analyses, we observed that the intervention did in Spain was starting to decrease by the time the trial was not produce significant effects among those health care workers initiated). However, it is also plausible that the intervention in using the intervention in absence of additional mental help. This https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 13 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al finding is consistent with findings from a recent systematic been widely recognized as key factors to enhance the acceptance review, which showed a lack of effect of mental mHealth of information and communication technologies tools. interventions when used as a standalone therapy [43]. However, According to the technology acceptance model, the intention PsyCovidApp was effective in improving the primary outcome to use a product in the future is strongly correlated with its ease and some secondary outcomes when used in conjunction with of use [46]. We observed that more than 90% of the participants evidence-based treatments (such as psychotherapy or asked to regain access to the PsyCovidApp intervention after psychotropic medications). This result supports findings from the trial. This finding supports the correlation of ease of use and a recent trial, which showed that a web-based psychoeducation intention to use, and at the same time, it suggests that the approach was not sufficiently effective in improving depressive intervention was acceptable and perceived as useful. symptoms in a general population of workers but was effective In terms of future research needs, it is worth noting that for workers who had recently sought help for mental health PsyCovidApp was based on a group of software features related [11]. It also supports findings from another recent trial, which to the intervention (eg, learning and in situ use) and reported that a web-based psychoeducational intervention communication (eg, prompting) deployed in smartphone produced a significantly greater reduction in depression severity interventions that mostly mimic more traditional mobile phone among participants who had undergone psychotherapy before and mHealth solutions. More innovative use of the capabilities enrolling in the study [12]. According to the ICEMAN criteria of smartphones, such as sensing, alternative delivery paradigms, (Multimedia Appendix 6), the results of our subgroup are highly and advanced analytics, could have produced a more beneficial credible because they were correctly hypothesized a priori, are effect. The possibilities of current smartphone technology have supported by prior evidence [11,12], are supported by tests of only just been tapped, and further research is needed to explore interaction, are the result of analyzing a small number of effect them fully, as are studies to rigorously analyze the empirical modifiers, and show consistent effect modification across related effectiveness of these systems. outcomes. We hypothesize that the level of motivation to engage with the PsyCovidApp intervention may account for some of A process evaluation is now underway, which will shed light the differences in the effects observed among health care on the mechanisms and contexts in which the intervention did workers using and not using psychotherapy or psychotropic or did not work. In this process evaluation, we will medications. Motivation has indeed been identified as a key retrospectively investigate the “reach” of interventions (the element in behavior change interventions. Following the extent to which the study participants came into contact with transtheoretical model of behavior change [44], health care the intervention and how they did so). Although a recent workers using psychotherapy and psychotropic medications meta-analysis of a range of mental health apps concluded that may be more likely in the “action” stage (as they had already age does not impact treatment effect [19], we cannot rule out made specific overt modifications in acquiring new healthy the possibility that in our trial, older professionals experienced behaviors) and therefore may have been more motivated to more problems engaging with PsyCovidApp than younger engage with PsyCovidApp, whereas the rest of the health care participants. Therefore, as part of the process evaluation, we workers may be more likely in the “precontemplation” or will specifically examine the extent to which intervention “contemplation” stages (ie, not intending to take action in the engagement differed across age groups. We will also use foreseeable future) and therefore may have been less motivated qualitative research methods to gain a deeper understanding of to engage with and follow the techniques recommended by implementation barriers and facilitators and to identify PsyCovidApp. Future studies are needed to analyze this suggestions about how to improve the intervention to maximize hypothesis. However, altogether, our results suggest that its effects on a broader range of health care workers. Once although the mHealth interventions may not be effective as a PsyCovidApp becomes publicly available, we will prospectively standalone strategy, one possibility to benefit from apps could follow up with new users to identify patterns of use associated be to integrate them into a clinical setting and use them in with higher intervention benefits. The findings of this process conjunction with other evidence-based treatments. This is a evaluation will inform future developments of the PsyCovidApp relevant field that is still not well understood and should be intervention. further investigated. Limitations The fact that female health care workers were overrepresented The study has several limitations. First, the 2-week follow-up in our trial (83% in our trial vs 68% overall in Spain [45]) could period may not have been sufficient to detect clinically have biased our results if the intervention had produced a meaningful differences in mental health. A longer period of differential effect by gender. However, as far as we know, there time may be needed to produce the desired positive effects. is no evidence in the literature that such a difference exists. In There were two main reasons for this short follow-up period: our postprotocol subgroup analyses, we did not find gender (1) according to available literature [42], the use of mental health differences in any of the outcomes considered (results not apps substantially decreases after the initial weeks, and (2) the shown); therefore, this overrepresentation of female participants short follow-up allowed us to obtain evidence in a timely is unlikely to have substantially influenced the results of our manner, which was critical to inform decisions about scaling trial. up the intervention in a time when health care workers in Spain PsyCovidApp presented a high usability level, with an overall were experiencing remarkably high prevalence rates of stress, score of 87.2 points—clearly above the threshold of 68 points anxiety, posttraumatic stress disorder, and insomnia. Second, used to determine high usability [24]. Usability factors have the mental health of the participants was not evaluated through https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 14 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al a diagnostic clinical interview but rather using instruments participants, outcome assessors, and data analysts of the research indicated for symptomatology assessment rather than for clinical were blinded to the intervention allocation to reduce biases in diagnosis. Third, we did not restrict our sample to health care the evaluation of the effects of the intervention. A common workers with mental health problems at baseline. Including a limitation of previous mHealth trials is that researchers do not large proportion of participants with no (or minor) mental health have control of the proportion of participants having actual problems in our study may have limited our ability to observe access to their interventions. In our trial, we ensured that all mental health improvements. Fourth, we did not include a participants successfully downloaded and activated the app waiting list or treatment-as-usual control group; thus, we are before their enrollment in the trial, which is a novel and unable to determine whether the apparent reduction in mental important strength. health symptoms in both groups at 2 weeks represents equal effectiveness of the treatment allocations or the natural Conclusion progression of the symptoms. Fifth, it was technically unfeasible For the first time, the PsyCovidApp trial studied the impact of to monitor use of the intervention, which prevented us to explore a cognitive behavioral therapy and mindfulness-based mHealth a dose-response effect. Finally, our study was remarkably intervention specifically designed to protect the mental health specific to the current pandemic context, which limits the of health care workers fighting on the front lines of the external validity of our results. The trial population was COVID-19 pandemic. No significant differences were observed restricted to health care workers who had provided direct health between the intervention and control groups at 2 weeks in the care to patients with COVID-19. The intervention was primary outcome and in the rest of the outcomes. However, specifically designed to address the most common mental health significant improvements were observed among health care problems experienced under these special circumstances, workers who were consuming psychotropic medications or included specific content acknowledging the key challenges receiving psychotherapy in the primary outcome, as well as in health care workers face during the COVID-19 situation, and posttraumatic stress, insomnia, anxiety, and stress. PsyCovidApp provided recommendations about how to overcome them. may therefore improve mental health among health care workers Therefore, and according to best practice guidelines [47], for who are already using other effective interventions, such as the investigation of the impact of mobile health interventions psychotherapy or pharmacological treatments. in health care workers in a broader, nonpandemic context, the Data Sharing findings of our trial should only be taken into consideration as indirect evidence. Deidentified data collected for the study, including individual participant data and a data dictionary defining each field in the Strengths set, will be made available to others upon request to the The strengths of this study include the pragmatic design, large corresponding author, following a signed data access agreement. sample size, and high follow-up rates. Moreover, the trial Acknowledgments This study was funded by the Balearic Islands Health Research Institute (IdISBa) through the Regional Government of the Balearic Islands, Spain (grant code: COVID-19/06). IRC was supported by a Miguel Servet Fellowship (CP17/00019) from the Instituto de Salud Carlos III (Spanish Ministry of Sciences, Innovation and Universities). MJSR was supported by a FOLIUM - FUTURMed Fellowship from IdISBa, co-founded by ITS-2017 and PO FSE 2014-2020 from the Balearic Islands. We thank our psychologist team (Marian Pérez, Elena Bonet, Toni Riera, Andrea Seguí, Beatriz González, Yasmina Castaño, María Barceló, Catalina Calafat, Miriam Kefauver, Desirée Guillem, Omar Vial, Adela Marmy, Inés Forteza-Rey, Patricia López, Ana Serapio, Catalina Moragues, Alberto Ávila and Ana Martí) for conducting the psychological assessments of the health care workers. We also thank Cristian Sánchez and Daniel Gallego for their support with data management and with technical assistance for health care workers who experienced difficulties in setting up the study Apps. We also thank Massimo de Faveri, Mercedes Martínez, and Alfonso Morillas, from Apploading, Inc, for making the Clinicovery app available for use in this study. Finally, we thank all the health care workers who participated in this study. Authors' Contributions IRC, MJSR, MAF, and JGC designed the study. MJSR, AMYJ, MBV, MEGB, MGT, CS, RJ, EG, and JGC developed the contents of the PsyCovidApp intervention under the coordination of MJSR. MAF transferred the intervention contents to the Clinicovery app. MJSR and the psychologist team (see Acknowledgments) collected the data. MAF conducted data management. RJ, EG, AYMJ, and AL analyzed the data. All authors interpreted the data. IRC wrote the report. All authors critically revised and approved the final version of the manuscript. IRC and MJSR were the principal co-investigators. Conflicts of Interest MJSR, AMYJ, MBV, MEGB, MGT, CS, RJ, EG, and JGC developed the contents of the intervention. All other authors have no conflicts to declare. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 15 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Multimedia Appendix 1 Content of the PsyCovidApp intervention. [PDF File (Adobe PDF File), 777 KB-Multimedia Appendix 1] Multimedia Appendix 2 Content of the control app. [PDF File (Adobe PDF File), 110 KB-Multimedia Appendix 2] Multimedia Appendix 3 Subgroup analyses: comparison of outcome measures between the PsyCovidApp and the Control App groups at 2 weeks in prespecified subgroups (high DASS-21 baseline scores; users of psychotherapy; consumers of psychotropic medications). [PDF File (Adobe PDF File), 202 KB-Multimedia Appendix 3] Multimedia Appendix 4 Number of health care workers recruited by Spanish region. [PDF File (Adobe PDF File), 96 KB-Multimedia Appendix 4] Multimedia Appendix 5 Trial results reported in terms of adjusted mean between-group differences (overall sample). [PDF File (Adobe PDF File), 133 KB-Multimedia Appendix 5] Multimedia Appendix 6 Instrument to Assess the Credibility of Effect Modification Analyses (ICEMAN) used in the study. [PDF File (Adobe PDF File), 160 KB-Multimedia Appendix 6] Multimedia Appendix 7 CONSORT-EHEALTH checklist (v 1.6.1). [PDF File (Adobe PDF File), 1102 KB-Multimedia Appendix 7] References 1. COVID-19 Weekly Epidemiological Update. World Health Organization. 2020 Nov 15. URL: https://www.who.int/ publications/m/item/weekly-epidemiological-update---17-november-2020 [accessed 2021-05-11] 2. Kang L, Li Y, Hu S, Chen M, Yang C, Yang BX, et al. The mental health of medical workers in Wuhan, China dealing with the 2019 novel coronavirus. Lancet Psychiatry 2020 Mar;7(3):e14 [FREE Full text] [doi: 10.1016/S2215-0366(20)30047-X] [Medline: 32035030] 3. Serrano-Ripoll MJ, Meneses-Echavez JF, Ricci-Cabello I, Fraile-Navarro D, Fiol-deRoque MA, Pastor-Moreno G, et al. Impact of viral epidemic outbreaks on mental health of healthcare workers: a rapid systematic review and meta-analysis. J Affect Disord 2020 Dec 01;277:347-357 [FREE Full text] [doi: 10.1016/j.jad.2020.08.034] [Medline: 32861835] 4. Kisely S, Warren N, McMahon L, Dalais C, Henry I, Siskind D. Occurrence, prevention, and management of the psychological effects of emerging virus outbreaks on healthcare workers: rapid review and meta-analysis. BMJ 2020 May 05;369:m1642 [FREE Full text] [doi: 10.1136/bmj.m1642] [Medline: 32371466] 5. Bandyopadhyay S, Baticulon RE, Kadhum M, Alser M, Ojuka DK, Badereddin Y, et al. Infection and mortality of healthcare workers worldwide from COVID-19: a systematic review. BMJ Glob Health 2020 Dec;5(12) [FREE Full text] [doi: 10.1136/bmjgh-2020-003097] [Medline: 33277297] 6. Luceño-Moreno L, Talavera-Velasco B, García-Albuerne Y, Martín-García J. Symptoms of posttraumatic stress, anxiety, depression, levels of resilience and burnout in Spanish health personnel during the COVID-19 pandemic. Int J Environ Res Public Health 2020 Jul 30;17(15) [FREE Full text] [doi: 10.3390/ijerph17155514] [Medline: 32751624] 7. Alexopoulos AR, Hudson JG, Otenigbagbe O. The use of digital applications and COVID-19. Community Ment Health J 2020 Oct;56(7):1202-1203 [FREE Full text] [doi: 10.1007/s10597-020-00689-2] [Medline: 32734311] 8. Davidson SK, Schattner PL. Doctors' health-seeking behaviour: a questionnaire survey. Med J Aust 2003 Sep 15;179(6):302-305. [doi: 10.5694/j.1326-5377.2003.tb05552.x] [Medline: 12964913] 9. Proudfoot J, Parker G, Hadzi Pavlovic D, Manicavasagar V, Adler E, Whitton A. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. J Med Internet Res 2010 Dec 19;12(5):e64 [FREE Full text] [doi: 10.2196/jmir.1475] [Medline: 21169174] https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 16 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al 10. Andersson E, Steneby S, Karlsson K, Ljótsson B, Hedman E, Enander J, et al. Long-term efficacy of Internet-based cognitive behavior therapy for obsessive-compulsive disorder with or without booster: a randomized controlled trial. Psychol Med 2014 Oct;44(13):2877-2887. [doi: 10.1017/S0033291714000543] [Medline: 25066102] 11. Imamura K, Kawakami N, Tsuno K, Tsuchiya M, Shimada K, Namba K. Effects of web-based stress and depression literacy intervention on improving symptoms and knowledge of depression among workers: A randomized controlled trial. J Affect Disord 2016 Oct;203:30-37. [doi: 10.1016/j.jad.2016.05.045] [Medline: 27280960] 12. Reins JA, Boß L, Lehr D, Berking M, Ebert DD. The more I got, the less I need? Efficacy of Internet-based guided self-help compared to online psychoeducation for major depressive disorder. J Affect Disord 2019 Mar 01;246:695-705. [doi: 10.1016/j.jad.2018.12.065] [Medline: 30611913] 13. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS One 2016;11(4):e0152285 [FREE Full text] [doi: 10.1371/journal.pone.0152285] [Medline: 27073900] 14. Firth J, Torous J. Smartphone apps for schizophrenia: a systematic review. JMIR mHealth uHealth 2015 Nov 06;3(4):e102 [FREE Full text] [doi: 10.2196/mhealth.4930] [Medline: 26546039] 15. Steinkamp JM, Goldblatt N, Borodovsky JT, LaVertu A, Kronish IM, Marsch LA, et al. Technological interventions for medication adherence in adult mental health and substance use disorders: a systematic review. JMIR Ment Health 2019 Mar 12;6(3):e12493 [FREE Full text] [doi: 10.2196/12493] [Medline: 30860493] 16. Killikelly C, He Z, Reeder C, Wykes T. Improving adherence to web-based and mobile technologies for people with psychosis: systematic review of new potential predictors of adherence. JMIR mHealth uHealth 2017 Jul 20;5(7):e94 [FREE Full text] [doi: 10.2196/mhealth.7088] [Medline: 28729235] 17. Miralles I, Granell C, Díaz-Sanahuja L, Van Woensel W, Bretón-López J, Mira A, et al. Smartphone apps for the treatment of mental disorders: systematic review. JMIR mHealth uHealth 2020 Apr 02;8(4):e14897 [FREE Full text] [doi: 10.2196/14897] [Medline: 32238332] 18. Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [doi: 10.1016/j.jad.2017.04.046] [Medline: 28456072] 19. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [doi: 10.1002/wps.20472] [Medline: 28941113] 20. Drissi N, Ouhbi S, Marques G, de la Torre Díez I, Ghogho M, Janati Idrissi MA. A systematic literature review on e-mental health solutions to assist health care workers during COVID-19. Telemed J E Health 2020 Sep 22. [doi: 10.1089/tmj.2020.0287] [Medline: 32970532] 21. Pollock A, Campbell P, Cheyne J, Cowie J, Davis B, McCallum J, et al. Interventions to support the resilience and mental health of frontline health and social care professionals during and after a disease outbreak, epidemic or pandemic: a mixed methods systematic review. Cochrane Database Syst Rev 2020 Nov 05;11:CD013779. [doi: 10.1002/14651858.CD013779] [Medline: 33150970] 22. Roland M, Torgerson DJ. What are pragmatic trials? BMJ 1998 Jan 24;316(7127):285-285 [FREE Full text] [doi: 10.1136/bmj.316.7127.285] [Medline: 9472515] 23. Serrano-Ripoll MJ, Ricci-Cabello I, Jiménez R, Zamanillo-Campos R, Yañez-Juan AM, Bennasar-Veny M, et al. Effect of a mobile-based intervention on mental health in frontline healthcare workers against COVID-19: Protocol for a randomized controlled trial. J Adv Nurs 2021 Mar 06. [doi: 10.1111/jan.14813] [Medline: 33675247] 24. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum-Comput Int 2008 Jul 30;24(6):574-594. [doi: 10.1080/10447310802205776] 25. Antony MM, Bieling PJ, Cox BJ, Enns MW, Swinson RP. Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychol Assess 1998 Jun;10(2):176-181. [doi: 10.1037/1040-3590.10.2.176] 26. Antúnez Z, Vinet E. Escalas de Depresión, Ansiedad y Estrés (DASS - 21): Validación de la Versión abreviada en Estudiantes Universitarios Chilenos. Ter Psicol 2012 Dec;30(3):49-55. [doi: 10.4067/S0718-48082012000300005] 27. Bobes J, González M, Sáiz P, Bascarán M, Bousoño M. Instrumentos básicos para la práctica de la psiquiatría clínica. Barcelona, Spain: Psiquiatría Editores, SL; 2004. 28. Bobes J, Calcedo-Barba A, García M, François M, Rico-Villademoros F, González MP, Grupo Espanol de Trabajo para el Estudio del Trastornostres Postraumático. Evaluation of the psychometric properties of the Spanish version of 5 questionnaires for the evaluation of post-traumatic stress syndrome. Article in Spanish. Actas Esp Psiquiatr 2000;28(4):207-218. [Medline: 11116791] 29. Gil-Monte PR. Factorial validity of the Maslach Burnout Inventory (MBI-HSS) among Spanish professionals. Rev Saude Publica 2005 Feb;39(1):1-8 [FREE Full text] [doi: 10.1590/s0034-89102005000100001] [Medline: 15654454] 30. Olivares-Faúndez V, Mena-Miranda L, Macía-Sepúlveda F, Jélvez-Wilke C. Validez factorial del Maslach Burnout Inventory Human Services (MBI-HSS) en profesionales chilenos. Univ Psychol 2014 May 01;13(1). [doi: 10.11144/javeriana.upsy13-1.vfmb] https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 17 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al 31. Bastien CH, Vallières A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med 2001 Jul;2(4):297-307. [doi: 10.1016/s1389-9457(00)00065-4] [Medline: 11438246] 32. Sierra JC, Guillén-Serrano V, Santos-Iglesias P. Insomnia Severity Index: some indicators about its reliability and validity on an older adults sample. Article in Spanish. Rev Neurol 2008;47(11):566-570 [FREE Full text] [Medline: 19048535] 33. Herrero R, Espinoza M, Molinari G, Etchemendy E, Garcia-Palacios A, Botella C, et al. Psychometric properties of the General Self Efficacy-12 Scale in Spanish: general and clinical population samples. Compr Psychiatry 2014 Oct;55(7):1738-1743. [doi: 10.1016/j.comppsych.2014.05.015] [Medline: 24973225] 34. Soler Sánchez MI, Meseguer de Pedro M, García Izquierdo M. Propiedades psicométricas de la versión española de la escala de resiliencia de 10 ítems de Connor-Davidson (CD-RISC 10) en una muestra multiocupacional. Revista Latinoamericana de Psicología 2016 Sep;48(3):159-166. [doi: 10.1016/j.rlp.2015.09.002] 35. Orellana C, Orellana L. Propiedades psicométricas de la Escala abreviada de Depresión, Ansiedad y Estrés (DASS-21) en adultos salvadoreños. Teoría y Praxis 2021;19(38):38-54 [FREE Full text] 36. Bados A, Solanas A, Andrés R. Psychometric properties of the Spanish version of Depression, Anxiety and Stress Scales (DASS). Psicothema 2005;17(4):679-683 [FREE Full text] 37. Gao M. Multi-Cultural Usability Assessment with System Usability Scale. Thesis. Rice University. 2019. URL: https://hdl. handle.net/1911/106168 [accessed 2021-05-11] 38. Royston P, White I. Multiple imputation by chained equations (MICE): implementation in Stata. J Stat Soft 2011;45(4). [doi: 10.18637/jss.v045.i04] 39. Schandelmaier S, Briel M, Varadhan R, Schmid CH, Devasenapathy N, Hayward RA, et al. Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ 2020 Aug 10;192(32):E901-E906 [FREE Full text] [doi: 10.1503/cmaj.200077] [Medline: 32778601] 40. Wickström G, Bendix T. The "Hawthorne effect"—what did the original Hawthorne studies actually show? Scand J Work Environ Health 2000 Aug;26(4):363-367 [FREE Full text] [Medline: 10994804] 41. Rodde T. 25% of users abandon an app after one use. Upland. 2019. URL: http://info.localytics.com/blog/ 25-of-users-abandon-apps-after-one-use [accessed 2021-05-11] 42. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res 2018 Jun 06;20(6):e199 [FREE Full text] [doi: 10.2196/jmir.9275] [Medline: 29875089] 43. Weisel KK, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, Ebert DD. Standalone smartphone apps for mental health-a systematic review and meta-analysis. NPJ Digit Med 2019;2:118 [FREE Full text] [doi: 10.1038/s41746-019-0188-8] [Medline: 31815193] 44. Prochaska JO, DiClemente CC. Transtheoretical therapy: toward a more integrative model of change. Psychol Psychother 1982;19(3):276-288. [doi: 10.1037/h0088437] 45. Profesionales sanitarios colegiados. Instituto Nacional de Estadística. 2019. URL: https://www.ine.es/dyngs/INEbase/es/ operacion.htm?c=Estadistica_C&cid=1254736176781&menu=ultiDatos&idp=1254735573175 [accessed 2021-05-11] 46. Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Management Science 2000 Feb;46(2):186-204. [doi: 10.1287/mnsc.46.2.186.11926] 47. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [doi: 10.2196/jmir.1923] [Medline: 22209829] Abbreviations DASS-21: Depression, Anxiety, and Stress Scale DTS: Davidson Trauma Scale GSE: General Self-Efficacy Scale IdISBa: Balearic Islands Health Research Institute ISI: Insomnia Severity Index MBI-HSS: Maslach Burnout Inventory - Human Services Survey mHealth: mobile health SUS: System Usability Scale https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 18 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Fiol-DeRoque et al Edited by L Buis; submitted 11.01.21; peer-reviewed by N Kawakami, T Alessa; comments to author 16.02.21; revised version received 08.03.21; accepted 31.03.21; published 18.05.21 Please cite as: Fiol-DeRoque MA, Serrano-Ripoll MJ, Jiménez R, Zamanillo-Campos R, Yáñez-Juan AM, Bennasar-Veny M, Leiva A, Gervilla E, García-Buades ME, García-Toro M, Alonso-Coello P, Pastor-Moreno G, Ruiz-Pérez I, Sitges C, García-Campayo J, Llobera-Cánaves J, Ricci-Cabello I A Mobile Phone–Based Intervention to Reduce Mental Health Problems in Health Care Workers During the COVID-19 Pandemic (PsyCovidApp): Randomized Controlled Trial JMIR Mhealth Uhealth 2021;9(5):e27039 URL: https://mhealth.jmir.org/2021/5/e27039 doi: 10.2196/27039 PMID: 33909587 ©Maria Antònia Fiol-DeRoque, Maria Jesús Serrano-Ripoll, Rafael Jiménez, Rocío Zamanillo-Campos, Aina María Yáñez-Juan, Miquel Bennasar-Veny, Alfonso Leiva, Elena Gervilla, M Esther García-Buades, Mauro García-Toro, Pablo Alonso-Coello, Guadalupe Pastor-Moreno, Isabel Ruiz-Pérez, Carolina Sitges, Javier García-Campayo, Joan Llobera-Cánaves, Ignacio Ricci-Cabello. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 18.05.2021. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included. https://mhealth.jmir.org/2021/5/e27039 JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 5 | e27039 | p. 19 (page number not for citation purposes) XSL• FO RenderX