JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Original Paper Real-world Implementation of a Smartphone-Based Psychoeducation Program for Bipolar Disorder: Observational Ecological Study Aitana García-Estela1,2, MSc; Jordi Cantillo, BSc, MSc; Natalia Angarita-Osorio1,2, MSc; Estanislao Mur-Milà1,2,3, MD; Gerard Anmella4,5, MD; Víctor Pérez1,2,3,5, MD, PhD; Eduard Vieta4,5, MD, PhD; Diego Hidalgo-Mazzei4,5,6, MD, PhD; Francesc Colom1,3,5,7, MSc, PhD 1 Mental Health Research Group, Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain 2 Department of Psychiatry and Forensic Medicine, School of Medicine, Universitat Autònoma de Barcelona, Barcelona, Spain 3 Institute of Neuropsychiatry and Addictions, Hospital del Mar, Parc de Salut Mar, Barcelona, Spain 4 Bipolar and Depressive Disorders Unit, Department of Psychiatry and Psychology, Institute of Neuroscience, Hospital Clínic, Universitat de Barcelona, Barcelona, Spain 5 Centre for Biomedical Research in Mental Health Network (CIBERSAM), Madrid, Spain 6 Centre for Affective Disorders, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom 7 Department of Basic, Evolutive and Education Psychology, Faculty of Psychology, Universitat Autònoma de Barcelona, Barcelona, Spain Corresponding Author: Francesc Colom, MSc, PhD Mental Health Research Group Hospital del Mar Medical Research Institute (IMIM) Office 202, PRBB Building Doctor Aiguader, 88 Barcelona, 08003 Spain Phone: 34 933160400 ext 1493 Email: fcolom@imim.es Abstract Background: SIMPLe is an internet‐delivered self‐management mobile app for bipolar disorder (BD) designed to combine technology with evidence-based interventions and facilitate access to psychoeducational content. The SIMPLe app was launched to the real world to make it available worldwide within the context of BD treatment. Objective: The main aims of this study are as follows: to describe app use, engagement, and retention rates based on server data; to identify patterns of user retention over the first 6-month follow-up of use; and to explore potential factors contributing to discontinuation of app use. Methods: This was an observational ecological study in which we pooled available data from a real-world implementation of the SIMPLe app. Participation was open on the project website, and the data-collection sources were a web-based questionnaire on clinical data and treatment history administered at inclusion and at 6 months, subjective data gathered through continuous app use, and the use patterns captured by the app server. Characteristics and engagement of regular users, occasional users, and no users were compared using 2-tailed t tests or analysis of variance or their nonparametric equivalent. Survival analysis and risk functions were applied to regular users’ data to examine and compare use and user retention. In addition, a user evaluation analysis was performed based on satisfaction, perceived usefulness, and reasons to discontinue app use. Results: We included 503 participants with data collected between 2016 and 2018, of whom 77.5% (n=390) used the app. Among the app users, 44.4% (173/390) completed the follow-up assessment, and data from these participants were used in our analyses. Engagement declined gradually over the first 6 months of use. The probability of retention of the regular users after 1 month of app use was 67.4% (263/390; 95% CI 62.7%-72.4%). Age (P=.002), time passed since illness onset (P<.001), and years since diagnosis of BD (P=.048) correlate with retention duration. In addition, participants who had been diagnosed with BD for longer used the app on more days (mean 97.73, SD 69.15 days; P=.002) than those who had had a more recent onset (mean 66.49, SD 66.18 days; P=.002) or those who had been diagnosed more recently (mean 73.45, SD 66 days; P=.01). https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 1 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Conclusions: The user retention rate of the app decreased rapidly after each month until reaching only one-third of the users at 6 months. There exists a strong association between age and app engagement of individuals with BD. Other variables such as years lived with BD, diagnosis of an anxiety disorder, and taking antipsychotics seem relevant as well. Understanding these associations can help in the definition of the most suitable user profiles for predicting trends of engagement, optimization of app prescription, and management. (J Med Internet Res 2022;24(2):e31565) doi: 10.2196/31565 KEYWORDS bipolar disorder; psychoeducation; smartphone; app; SIMPLe; Intervention; mobile phone evidence-based–practice frames that, simultaneously, have been Introduction subjected to efficacy tests [16,17]. Background Within this framework, the SIMPLe project was designed to Globally, an estimated 46 million people have been diagnosed leverage the potential of combining technology with with bipolar disorder (BD) [1]. Besides behavioral changes evidence-based interventions by developing and evaluating an occurring during mood episodes, BD has a serious impact on internet‐delivered self‐management mobile app (SIMPLe psychosocial functioning, cognition, quality of life, and survival 1.5) for BD in addition to standard treatment. The SIMPLe 1.5 rate of individuals with this condition [2,3]. Although ranges app collects information about potential symptoms, with the vary dramatically because of methodological differences among advantage of providing users with personalized studies, there is agreement that people with BD are between 9 psychoeducational messages and alerts that are tailored to and 30 times more likely to die from suicide than someone specific needs. The app is based on group psychoeducation, a without this condition [4,5]. well-established and evidence-based care-focused psychological intervention that addresses relevant issues of self-management Although the fundamental treatment for BD relies basically on for BD, such as identification and management of early warning psychopharmacology, some adjuvant psychological interventions signs, lifestyle, and treatment adherence [18,19]. have been shown to improve the long-term outcomes of this disease [6]. Among psychological interventions, Up until now, the SIMPLe app has proved acceptable to users psychoeducation programs have proven to be a cost-effective and has shown interesting and optimistic results: a high retention approach to prevent episodes by helping patients to improve rate was attained in a 3-month feasibility study and positive adherence, embrace healthy habits, and learn to recognize the outcomes regarding satisfaction were found in a naturalistic prodromes and symptoms of upcoming episodes. It is obvious implementation feasibility study [20-22]. In addition, some that efficacious treatments only work for patients who can potential improvements in terms of biological rhythms and receive them [7]; yet, these psychological interventions are medication adherence were suggested by post hoc analyses [23]. scarce and difficult to access for most patients [8]. It is worth mentioning that pharmacological treatment adherence is a particularly complex issue in BD because more than 50% e-Mental health—the delivery of mental health–related tools of the patients are estimated to be nonadherent fully to the through the internet and related technologies [9]—is one of the prescribed doses of medication [24]. most promising strategies to address this access gap, relieving overburdened mental health services and increasing the services’ Considering that the ultimate aim of the SIMPLe project is to cost-efficiency while maintaining its quality. Although extend and facilitate access to psychoeducational content internet-based interventions for mental health are still relatively through the SIMPLe app to all potential users, wide and free new, the number of solutions such as web-based platforms, access to the SIMPLe 1.5 app around the Spanish-speaking smartphone apps, and wearables is rapidly growing globally world was offered. This way, we had the opportunity to routinely [10], with their actual acceptability by patients tending to be collect implementation data on use in a real-world setting and increasingly higher [11]. naturalistic condition. However, the strong desire to proliferate e-mental health A previous report on the OpenSIMPLe study presented partial solutions has not been translated into a transformation in the data demonstrating the high dropout rates when a delivery of mental health care because there is little available psychoeducation smartphone-based intervention for individuals evidence of uptake of mental health apps [12]. Moreover, despite with BD is offered openly [22]. We hereby present the results the increasing number of e-mental health apps available in app of the whole sample of SIMPLe users throughout the study. stores, few are properly validated in a scientific process; this This research differs from our previous partial sample in terms lack of validation could jeopardize the safety and health of of both sample size and follow-up time frame. Statistical potential users [13,14]. In addition, the quality of the content analyses of the previous partial sample only included descriptive, of the apps for BD in the app stores does not live up to pre–post, and logistic regression tests without considering the expectations because most of them do not rely on the available temporal retention period as we did in this paper. best practice clinical guidelines [15]. Thus, it remains a In this paper, we focus on exploratory analyses that aim to challenge to implement platforms developed within investigate in depth the relationships among variables that may predict overall engagement as well as retention rates mainly by https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 2 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al means of survival analyses. More specifically, we intend to shed pharmacological treatment for BD provided by a psychiatrist; some light on the ways in which the SIMPLe app engagement (4) owning and using daily an Android or iOS smartphone; (5) and user retention patterns are influenced by individual fluency in Spanish; (6) an active email account; and (7) standard variables, including sociodemographic and clinical data. cutoff score of 7, co-occurrence of at least two symptoms, and moderate or severe impairment on a modified version of the This study is based on an ecological experimental Mood Disorder Questionnaire (MDQ) [26,27]. The participants’ implementation of an e-mental health resource. Thus, the reader answers were reviewed by a psychiatrist to assess consistency, may miss the usual randomized controlled studies’ constraints after which they were informed whether they were eligible for such as the lack of sample size calculation (which, by definition, the study. would be the bigger the better) or some control measures. However, this study reflects the real day-to-day problems faced To prevent duplicate use and potential misuse, the possibility by a mental health app when launched to the app stores to be of completing the questionnaire multiple times from the same used by the target population. IP address was blocked. Web-based technical support was provided to the app users, if needed, through email. Objectives The main aims of this study are to (1) describe app use, Data from the sample were drawn from 503 SIMPLe users who engagement, and retention rates based on server data; (2) had provided informed consent and completed the app’s identify patterns of user retention over the first 6-month onboarding questionnaire between May 2016 and June 2018. follow-up of use; and (3) explore potential factors contributing Of the 503 participants, 390 (77.5%) used the app, and data to the risk of discontinuing app use. from these participants were used in the analyses. The expectations are that these exploratory analyses will help All procedures contributing to this work complied with the to confirm preliminary use data of the SIMPLe app and ethical standards of national and international guidelines and understand user retention rates as well as the ways in which the basic principles of protection of dignity and human rights, users self-manage BD in a real-world setting. In addition, we as stated in the Declaration of Helsinki (64th General Assembly, hope that the results will provide our research colleagues with Fortaleza, Brazil, October 2013), and were conducted according relevant insights into the interplay, dynamics, and predictive to current regulations. The ethical committees from both factors of user engagement with mental health apps at the time Hospital Clínic of Barcelona (HCB/2016/0403) and the Hospital of their implementation in real-world conditions. del Mar Medical Research Institute (2016/6764/I) approved the protocol. Methods Assessments Procedure and Participants In all, two sources of data were used: (1) a web-based form administered at program inclusion and at 6 months and (2) Participation availability was open on the project website [25] subjective data gathered through continuous app use and the to anyone with BD who was fluent in Spanish. Specific app use patterns captured by the app server. characteristics and functionality offered for the SIMPLe app have been published elsewhere [20,21]. In brief, the app Psychometric External Assessments consisted of a daily graphic 5-item screening test (mood, energy, Sociodemographic data as well as illness and treatment history sleep time, medication adherence, and irritability) and a weekly, were collected at program inclusion through a web-based form more comprehensive Yes or No test, considering all Diagnostic from participants who had provided informed consent. The and Statistical Manual of Mental Disorders, Fifth Edition, number of hospitalizations and suicide attempts as well as criteria for manic and depressive episodes. The resulting mood treatment history during the past 6 months were also collected assessments were displayed in a chart on the home screen. On 6 months after inclusion. the basis of the information collected, a daily pop-up notification prompted the user to read a short psychoeducational message The Spanish validated version of the 5-item World Health providing brief information or advice about how to deal with Organization Well-being Index [28] was used to assess mental specific situations to avoid relapses. Each message was extracted well-being at baseline and 6 months later at the final follow-up. from a library of more than 500 messages categorized according App-Derived Assessments to different clinical situations based on a published psychoeducation manual and lay-language books on BD written Subjective Information by 2 of the authors (FC and EV). Additional optional modules The subjective information assessed was as follows: available in the app were personalized medication reminders, 1. Self-reported mood, sleep, medication adherence, and prodromal symptoms, and mood-chart sharing. energy: The app prompts users to answer 5 daily slider Following a real-world naturalistic approach, no active screening tests on mood, energy, sleep time, irritability, recruitment strategy or advertisement was used. Potential and drug-treatment adherence. The daily scores appearing participants approached the study voluntarily through the in the chart are the results of an algorithm, which was website [25] (more detailed information can be found elsewhere previously tested during the development phase [20]. In [22]). Potential participants needed to meet the following addition, a more comprehensive test, considering Diagnostic inclusion criteria: (1) aged ≥18 years; (2) having a and Statistical Manual of Mental Disorders, Fifth Edition, psychiatrist-confirmed diagnosis of BD before entry; (3) https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 3 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al criteria for manic and depressive episodes, including suicide Initially, basic descriptive statistics of sociodemographic and thoughts, was prompted weekly. clinical variables were run, including age, sex, marital status, 2. Self-reported usability: The app was evaluated by users family psychiatric history, follow-up time, number of episodes, who made it through to complete follow-up assessment. substance abuse, and comorbid medical and psychiatric The users’ perception of the usability of the app was diagnoses. Continuous variables have been described based on measured through the System Usability Scale (SUS) [29], the mean and SD; the median and the 25th and 75th percentiles a 10-item questionnaire with 5 response options (from have also been used in comparative analyses of the time spent strongly agree to strongly disagree) that allows evaluation using the SIMPLe 1.5 app. We defined categorical variables in of products and services. Interpretation of the raw scores terms of the number and percentage of users per response was achieved by converting them into percentile ranks [30] category. In addition, statistical techniques were used to confirm and associating them with adjectives [31]. assumptions of the statistical tests before carrying out parametric 3. Satisfaction and perceived helpfulness: Satisfaction and tests to compare means and proportions. When the assumptions perceived helpfulness of each subcomponent of the app of parametric tests were violated, nonparametric tests were used. were assessed in the follow-up questionnaire through Likert We performed a comparative analysis of the variables among scales after 6 months of program initiation. Suggestions groups using a 2-tailed t test or analysis of variance on and comments regarding the app were also registered. continuous variables or their nonparametric equivalent; the Mobile Terminal–Mined Information Wilcoxon test or the Kruskal–Wallis test was performed User retention, app use, and engagement data were constantly depending on the inherent characteristics of the variables recorded at the servers over the study duration, reflecting analyzed; the Mann–Whitney U test was carried out when continuously and in detail app use and engagement. User dependent variables were ordinal; and the chi-square test was retention was defined as the proportion of participants who used performed when analyzing categorical variables. the app for the entire duration of the study and completed the The Spearman correlation coefficient (ρ) was used to examine 6-month follow-up assessment. The SIMPLe app has multiple correlations between SIMPLe 1.5 app time use and the different components, three of which we consider the core active continuous variables. ingredients: the daily and weekly tests and the psychoeducational messages. To determine retention, we A subsample of users with engagement ≥12% was selected for considered the user to be active if we registered data in the the analysis of the use time of the SIMPLe 1.5 app to determine servers from these 3 interactions, and we considered the user a minimum frequency of use and avoid overestimating the use to have discontinued participation if there was a lack of data time. A homogeneity analysis was performed between the from these variables in the server for >1 month. selected subsample and the rest of the participants using the chi-square test for categorical variables and the Engagement with mobile apps is considered a multidimensional Kolmogorov–Smirnov Z test for continuous variables. construct, and different definitions can be used or combined to measure it. In this study, engagement was understood as the Survival analysis was used on the selected subsample to examine ability of an app to engage users and sustain user interactions the time spent using the SIMPLe 1.5 app because the length of and it was assessed through indicators such as usability, the follow-up period was variable and there were participants acceptability, and feasibility [32]. In this case, engagement was who did not experience the event quit using the app during the calculated based on the weekly percentage of completed tasks 6-month follow-up. Estimates of survival and risk functions of (ie, answering daily and weekly tests and reading the daily the time use of the app were calculated by applying the psychoeducational messages). Kaplan–Meier method. We used the log-rank test to compare various survival distributions and the Cox proportional-hazards Design model validated by Schoenfeld residual analysis to assess risks This is an observational ecological study in which we pooled related to the survival of the app users. A sensitivity analysis available data from a real-world implementation of the SIMPLe was performed by repeating the survival analysis with all users app. to evaluate the effect of the selection of the subsample in the study of the use time. Results were considered significant with The rationale for the OpenSIMPLe study and detailed methods 2-sided P values <.05. have been published elsewhere [22]; the methods are briefly outlined here. Results Statistical Analysis Descriptive Analysis Smartphone app data (ie, participants’ mood ratings, manic or hypomanic and depressive symptoms, and details of their use Figure 1 depicts the number of participants registered on the of the app) were downloaded directly from the servers. Likewise, OpenSIMPLe website who were initially interested in using the the users’ baseline and follow-up responses at both baseline app alongside those included in the statistical analysis. The and follow-up web-based questionnaires were retrieved from reasons reported by users who did not want to participate in the the servers. All analyses were run using SPSS software (version study are also listed. Finally, potential participants who were 26.0; IBM Corp) and R statistical package for Windows (version excluded are described, as well as the number of participants 4.0.2; The R Foundation for Statistical Computing). who actually used the app and those who responded to the 6-month follow-up assessment. https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 4 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Figure 1. Flowchart of participants included in the statistical analysis. MDQ: Mood Disorder Questionnaire. Regarding the clinical variables, a mean disorder duration of Sociodemographic and Clinical Characteristics 13.23 (SD 9.97) years was identified; 49.7% (194/390) of the The mean age of users was 34.74 (SD 10.48) years, and most users stated that they had experienced ≥10 depressive episodes; (264/390, 67.7%) of them were women. The most frequent and 33.3% (130/390) reported ≥10 manic or hypomanic ethnicity was Latin American (266/390, 68.2%), with high episodes. Most of the participants who used the app were education levels (241/390, 61.8%). A high percentage of the receiving treatment with at least one mood stabilizer (353/390, sample was employed at the time of study entry (156/390, 40%), 90.5%) and at least one antipsychotic (252/390, 64.6%), whereas whereas only 17.7% (69/390) were either on temporary or almost half (193/390, 49.5%) of the participants were receiving permanent disability leave. Regarding housing conditions, 33.8% at least one antidepressant. Furthermore, 71.5% (279/390) of (132/390) of the participants lived in the parental household the participants were receiving some kind of psychological and more than half of the sample reported living independently, treatment. The clinical variables collected at baseline are either owning (117/390, 30%) or renting their current house or described in Table 2. flat (90/390, 23.1%). Sociodemographic variables of the SIMPLe app users are described in Table 1. https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 5 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Table 1. Baseline sociodemographic characteristics of participants (N=390). Characteristic Value Gender, female, n (%) 264 (67.7) Ethnicity, n (%) African 2 (0.5) White 119 (30.5) Latin American 266 (68.2) Asian 2 (0.5) Other 1 (0.3) Age (years), mean (SD) 34.74 (10.48) Marital status, n (%) Single 192 (49.2) Married 81 (20.8) Cohabitation 50 (12.8) Divorced or separated 54 (13.8) Widowed 1 (0.3) Other 12 (3.1) Housing status, n (%) Shared home 43 (11) Tenant 90 (23.1) Homeowner 117 (30) Parental home 132 (33.8) Residence or institution 8 (2.1) Completed studies, n (%) None 1 (0.3) Primary education 8 (2.1) Secondary education 67 (17.2) A-level or general certificate of education 73 (18.7) Vocational education and training or certificate of higher education or higher national 95 (24.4) diploma Bachelor’s degree 101 (25.9) Graduate certificate or postgraduate diploma or master’s degree 45 (11.5) Employment status, n (%) Unemployed 78 (20) Student 81 (20.8) Employed 156 (40) Retired 10 (2.6) Temporary disability leave 35 (9) Permanent disability leave 30 (7.7) Country, n (%) Spain 130 (33.3) Chile 76 (19.5) Argentina 66 (16.9) Mexico 25 (6.4) https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 6 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Characteristic Value Colombia 23 (5.9) Guatemala 12 (3.1) Brazil 9 (2.3) Other 49 (12.6) https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 7 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Table 2. Baseline clinical variables of app users (N=390). Illness course Value Years since onset, mean (SD) 13.23 (9.97) Years since diagnosis of bipolar disorder, mean (SD) 6.4 (6.55) Depressive episodes, n (%) 0-4 110 (28.2) 5-9 86 (22.1) ≥10 194 (49.7) Manic or hypomanic episodes, n (%) 0-4 143 (36.7) 5-9 117 (30) ≥10 130 (33.3) Previous hospital admissions because of an episode, n (%) None 185 (47.4) 1-2 135 (34.6) ≥2 70 (17.9) Suicide attempts, n (%) None 156 (40) 1-2 142 (36.4) ≥2 92 (23.6) Treatment setting, n (%) Public health network 145 (37.2) Private health network 184 (47.2) Both 61 (15.6) Past psychological treatment, n (%) None 39 (10) Yes, individual psychotherapy 260 (66.7) Yes, group psychotherapy 9 (2.3) Yes, individual and group psychotherapy 82 (21) Current psychological treatment, n (%) None 111 (28.5) Individual psychotherapy 230 (59) Group psychotherapy 15 (3.8) Individual and group psychotherapy 34 (8.7) Current pharmacological treatment, n (%) Mood stabilizer 353 (90.5) Antipsychotic 252 (64.6) Antidepressant 193 (49.5) Anxiolytic 183 (46.9) were no users, 357 (70.9%) were regular users, and 33 (6.6%) Engagement were occasional users. In addition, among the participants who The 503 participants included in the study can be divided into used the app, 44.4% (173/390) completed the follow-up three broad categories based on their app use: no users (never assessment too. used the app), occasional users (engagement <12%), and regular users (engagement ≥12%). Of the 503 participants, 113 (22.5%) We analyzed the number of days containing any kind of record in the app from users over the 6-month follow-up period. https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 8 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Monthly progress of regular users’ engagement declined estimations of retention duration. The results showed a 7-day gradually over the first 6 months. The highest engagement was increase in the median survival time when we included observed in the first month (mean 0.74, SD 0.20); in the second occasional users in the analysis, whereas the probability of month, it dropped sharply. At 6 months, the users had a mean survival only increased by a score of 0.027 (SD 0.007) on engagement of 0.39 (SD 0.34). average. Therefore, we are confident that selection bias did not occur and that the subsets were representative of the data of the Occasional users used the app a mean of 139.06 (SD 56.80) sample. days, with a use frequency of 2.05 (SD 0.87) days per month, whereas regular users used the app 83.98 (SD 69.95) days, with Use and Retention Duration a use frequency of 19.42 (SD 7.76) days per month. The group We analyzed the number of days with records in the app from of occasional users rarely used the app over long periods of users over the 6-month follow-up period. It turned out that only time, which could overestimate use time rates, for example, a 13.8% (54/390) of the users used the SIMPLe app for >100 participant who used the app at the 3-month follow-up for the days. The mean survival time of regular app users was 87.95 last time but only used the app 4 times overall. The estimate (SD 72.08; 95% CI 80.48-95.43) days. that this participant used the app for 3 months may lead us to a use bias. As the SIMPLe app was developed for daily use, we The probability of survival for the 357 participants under could consider that the interaction of this kind of user with the consideration for these analyses after 1 month of app use was app is low enough to overestimate the time of use in the 67.4% (95% CI 62.7%-72.4%); at 3 months, the probability of statistical analysis. To avoid this bias, we used the variable survival was 43% (95% CI 38.1%-48.5%); and at the 6-month overall engagement—the percentage of tasks completed follow-up assessment, the probability of survival was 28% (95% compared with those expected to be completed during the time CI 23.6%-33.2%). The risk of discontinuing app use increased in which they used the app—to identify users who may make as the days passed: at 3 months, the cumulative risk of us enhance the time-of-use overestimation. Occasional users discontinuing app use was 83.7%; however, at 6 months, this were ruled out in the survival analysis. rose to a cumulative risk of 126.3%. Sensitivity and homogeneity analyses show homogeneous The correlations between retention duration and the baseline variables across subsets of participants defined by sociodemographic and clinical variables of users at baseline retention duration, with the exception of the cocaine use were analyzed. A direct correlation between age and engagement variable, which was significantly greater among the group of was observed (ρ=0.168; P=.002); older users (aged ≥46 years) no users than among the occasional and regular users. In had higher total app use (mean 109.78, SD 71.42; P=.005) than addition, we performed survival analysis again with all users younger users (aged 18-23 years; mean 63.12, SD to assess the effect of the selection of regular users in the 63.94; P=.005; Figure 2). Figure 2. Plot of Kaplan–Meier age estimates of survival of participants using the SIMPLe app. The horizontal axis represents the survival time (in days) with records in the app (6 months maximum). https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 9 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Time passed since illness onset was strongly associated with risk of discontinuing app use (Table 3). Variables with a time of use (ρ=0.194; P<.001) and years since diagnosis of BD significant contribution were years since onset (χ22=11.7; (ρ=0.106; P=.048). In addition, participants who had been P=.003), years with BD diagnosis (χ21=8.9; P=.003), opiate use diagnosed with BD for longer (>17 years) used the app for more days (mean 97.73, SD 69.15; P=.002) than those who had had (χ21=7.9; P=.005), age (χ23=12.3; P=.006), housing status a more recent onset (mean 66.49, SD 66.18; P=.002) or those (χ24=12.3; P=.006), employment status (χ25=13.5; P=.02), and who had been diagnosed <6.5 years ago (mean 73.45, SD 66; antipsychotic use (χ21=4.9; P=.03). P=.01). We performed log-rank (Mantel–Cox) test analysis to compare Survival-curve plots for the variables of interest were produced survival curves and detect potential factors contributing to the over the 6-month-long follow-up period and are described in the following paragraphs (Table 4). Table 3. Log-rank test for overall survivala. Variables Log rank (Mantel–Cox) Chi-square (df) P value Clinical variables Comorbid psychiatric disorder 2.5 (1) .11 Anxiety disorder 3.9 (1) .04 Personality disorder 2.4 (1) .12 Substance abuse disorder 0.5 (1) .46 Eating disorder 2.0 (1) .15 PTSDb 1.1 (1) .30 Other comorbid psychiatric disorders 2.2 (1) .13 c 2.9 (2) .23 WHO-5 Illness course Years since onset of first episode 11.7 (2) .003 Years diagnosed with bipolar disorder 8.9 (1) .003 Depressive episodes 4.2 (2) .12 Manic or hypomanic episodes 4.5 (2) .10 Hospitalizations because of an episode 2.1 (2) .35 Suicide attempts 1.9 (2) .36 Treatment Psychotherapy 1.1 (3) .77 Mood stabilizer 0.2 (1) .65 Antipsychotic 4.9 (1) .02 Antidepressant 0.0 (1) .84 Anxiolytic 0.1 (1) .73 Electroconvulsive therapy 1.8 (1) .17 a Users with engagement ≥12%. b PTSD: posttraumatic stress disorder. c WHO-5: 5-item World Health Organization Well-being Index. https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 10 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Table 4. Comparing survival curves. Characteristic Regular usersa Mean, estimate (SE; 95% CI) Median, estimate (SE; 95% CI) Age (years) 18-23 65.72 (8.60; 48.85-82.58) 41.00 (9.90; 21.59-60.40) 24-29 87.11 (8.69; 70.06-104.16) 70.00 (23.12; 24.67-115.32) 30-45 86.26 (5.54; 75.40-97.12) 65.00 (7.28; 50.72-79.27) ≥46 114.83 (9.17; 96.85-132.80) 134.00 (22.24; 90.39-177.60) Housing status Shared home 62.65 (9.91; 43.21-82.08) 50.00 (8.51; 33.31-66.68) Tenant 98.03 (7.71; 82.91-113.15) 83.00 (16.02; 51.58-114.41) Homeowner 99.95 (7.34; 85.56-114.34) 109.00 (26.39; 57.26-160.73) Parental house 77.52 (6.35; 65.05-89.98) 56.00 (10.86; 34.70-77.29) Residence or institution 96.62 (20.35; 56.72-136.52) 62.00 (60.81; 0.00-181.19) Employment status Unemployed 72.87 (7.74; 57.69-88.05) 55.00 (12.96; 29.58-80.41) Student 75.02 (7.85; 59.62-90.42) 57.00 (14.14; 29.28-84.71) Employed 97.40 (6.09; 85.45-109.36) 80.00 (12.27; 55.9-104.06) Retired 109.40 (24.72; 60.93-157.86) 84.00b Temporary disability leave 101.96 (13.21; 76.06-127.86) 104.00 (61.23; 0.00-224.01) Permanent disability leave 85.14 (14.47; 56.77-113.52) 69.00 (24.71; 20.55-117.44) Diagnostic time (years) 0-6.5 81.14 (4.79; 71.74-90.54) 61.00 (7.45; 46.38-75.61) >6.5 98.15 (6.36; 85.67-110.63) 94.00 (15.98; 62.67-125.32) Anxiety disorder No 95.07 (4.91; 85.43-104.71) 83.00 (9.17; 65.01-100.98) Yes 78.19 (5.93; 66.56-89.83) 46.00 (7.96; 30.39-61.61) a Users with engagement ≥12%. b SE and 95% CI are not available. The app use survivorship of the oldest participants (aged ≥46 Being unemployed seemed to worsen app use survivorship years) seems greater than that of the youngest group of users pretty much at all time points because the survival likelihood (aged 18-23 years) because the estimated mean was 114.83 mean estimation of unemployed participants was 72.87 (95% (95% CI 96.85-132.80) days for users in the former age range, CI 57.69-88.05) days, that is, lower than any other employment whereas the use mean was 65.72 (95% CI 48.85-82.58) days status. At 60 days, the highest survival cumulative risk was that for users in the latter group. At 60 days, the probability of of unemployed participants (0.737), followed by students survival of the youngest users was 38.9% (95% CI 28%-53.9%); (0.734), individuals on permanent (0.596) and temporary this likelihood increased to 69.4% for the oldest group of disability leave (0.573), employers (0.523), and retired people participants, with a cumulative risk of 0.923 and 0.36, (0.336). respectively. In addition, app use declined faster among participants who had Regarding housing status, we observed that the mean estimation been recently diagnosed (<6.5 years) compared with users who of app use survival of people sharing a house or flat was 62.65 had been diagnosed for a longer period of time; at 60 days, the (95% CI 43.21-82.08) days, that is, between 15 and 37 days cumulative risk of app use discontinuation among people who fewer than users with other housing statuses, suggesting a had a recent diagnosis was 0.681, whereas this risk was lower survival disadvantage. At 60 days, the cumulative incidence for those who had been diagnosed earlier (0.518). The mean risk estimates among users who shared a house were 0.880, app use estimation of individuals with a more recent diagnosis whereas they were 0.435 for individuals who lived in residences, of BD was 8.14 (95% CI 71.74-90.54) days, whereas it increased 0.488 for tenants, 0.509 for homeowners, and 0.734 for those to 98.15 (95% CI 85.67-110.63) days for people with an earlier who lived in the parental home. diagnosis. https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 11 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al The survival time of patients with comorbid anxiety disorder anxiolytics use and self-reported anxiety disorder (P<.001), and diverged from those who did not have symptoms of it over time, 59.6% (99/166) of the users who reported having an anxiety with a cumulative risk of use discontinuation of 0.789 and 0.500 disorder did use them, whereas anxiolytics use decreased to at 60 days, respectively. Relatively few patients continued to 37.5% (84/224) among the participants who did not report an use the app after the very first month overall, but among those anxiety disorder. As antidepressant drugs can also be used to who were still using it, participants with anxiety disorder treat a number of other conditions, including anxiety disorders, continued to show a survival disadvantage over those who did we also analyzed association of self-reported anxiety disorder not experience it. The mean estimation of app use of the latter with anxiolytics, along with the most widely prescribed group was 95.07 (95% CI 85.43-104.71) days, whereas that of antidepressants for anxiety. It turned out that 74.7% (124/166) participants with an anxiety disorder was 78.19 (95% CI of the users who reported having an anxiety disorder did use 66.56-89.83) days; therefore, having an anxiety disorder these medications, despite the guideline recommendations to significantly influenced app use. Nevertheless, anxiety disorder avoid antidepressants in BD. was self-reported based on what users consider anxiety; hence, Furthermore, we performed a Cox (proportional hazards) we tried to see if there was homogeneity between self-reports regression analysis to estimate the hazard of discontinuing app and treatment prescriptions at study initiation. Analysis showed use for regular users, given their prognostic variables. The that there is a statistically significant relationship between results of the Cox model analysis are presented in Table 5. Table 5. Cox regression model analysis of user survival using the SIMPLe app 1.5a. Characteristics Coefficient Exp (coefficient; 95% CI) P value Concordance, mean (SE) Age –0.016 0.984 (0.971-0.998) .02 0.589 (–0.019) Country N/Ab Spain —c — — Chile –0.143 0.867 (0.588-1.278) .47 Argentina 0.091 1.096 (0.748-1.605) .63 Mexico –0.121 0.886 (0.495-1.585) .68 Colombia 0.318 1.375 (0.797-2.37) .25 Other 0.141 1.152 (0.779-1.703) .47 Anxiety disorder Yes 0.233 1.262 (0.975-1.634) .07 Antipsychotic Yes 0.334 1.396 (1.058-1.843) .02 a Schoenfeld residuals to check the proportional-hazards assumption: age (χ21=0.006; P=.94), anxiety disorder (χ21=2.7; P=.10), and antipsychotic (χ21=0.1; P=.72). b N/A: not applicable. c Our Cox model analyzed the risk of discontinuing the app use that participants from different nationalities had in comparison with Spanish participants. This row was maintained in the table to make clear that Spain was not included in the category Other. The variables age, anxiety disorder, antipsychotic, and country P=.02), which suggests that this variable is a risk factor and that were explored (Table 5). In this model, age is suggested as a users who take antipsychotics have a 33% hazard for protective factor for app use discontinuation, that is, the older discontinuing app use. the individual, the lower the risk of discontinuing app use. The regression coefficient for age was –0.016 (P=.02), which would Usability, Satisfaction, and Perceived Usefulness imply a better engagement for older individuals. We calculated The analysis of user evaluation of the SIMPLe app contained the percentage change in hazard rate for years’ increase in age in this section was exclusively performed with data of the 173 using the formula 100 × (e(–0.016×10) – 1) = –14.8, which allows participants who used the app and completed the follow-up us to estimate that a user older by 10 years would have a 14.8% assessment too. reduction in their hazard compared with a user younger by 10 The mean raw SUS score was 77.05 (SD 17.21), which is above years. average at the 75th percentile. As shown in Figure 3, 74% We did not observe statistically significant differences among (128/173) of the users were highly or completely satisfied with countries. Participants from the countries analyzed did not have the speed and discretion of the app and 75.7% (131/173) of the a significantly different risk of discontinuing app use compared users were satisfied with the ease of daily use. A high or with Spanish users. complete level of overall satisfaction was rated by 65.3% (113/173) of the users. The regression coefficient for taking antipsychotics is statistically significant (coefficient=0.334, 95% CI 1.058-1.843; https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 12 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Figure 3. Satisfaction with the SIMPLe app. The bars denote the percentage of satisfaction of users who responded to the follow-up questionnaire (n=173) after having experienced using the app. With regard to usefulness, most users found somewhat useful interventions and improvement in the real-world uptake of or very useful the following features and functions: daily test self-management apps for BD, as well as in clinical benefits (125/173, 72.3%), mood chart (127/173, 73.4%), personalized and associated outcomes [33]. psychoeducational messages (117/173, 67.6%), weekly test (117/173, 67.6%), stressful events record (108/173, 62.4%), Comparison With Prior Work emergency alert notifications (103/173, 59.6%), the enabled Survival analysis is not a new idea in statistics, and it is option to share the mood chart (94/173, 54.3%), and the frequently used in several medical fields; in fact, it was prodromes module (87/173, 50.3%). considered the main outcome measure in the seminal works by Colom [34], who first showed the efficacy of group Among the users who registered the reason for discontinuing psychoeducation of patients in participants with BD. However, using the app (91/173, 52.6%) by answering a multiple-choice this method is a novel approach to quantify user retention and question, 28.6% (26/91) found it very repetitive, 23.1% (21/91) engagement of mobile apps that few studies have previously had technical issues, 17.6% (16/91) did not find it useful, 16.5% applied in nonpsychiatric populations [35,36]. Interestingly, (15/91) stated that it was an undesired daily reminder of their Chien et al [37] used machine learning techniques to identify condition, 14.3% (13/91) stated that it affected smartphone heterogeneity in patient engagement with internet-based performance, 13.2% (12/91) gave other reasons, 7.7% (7/91) cognitive behavioral therapy (CBT) for symptoms of depression were concerned about the stigma attached to having it installed and anxiety and found that patterns of patient behavior may on their smartphone, 4.4% (4/91) relapsed, and 3.3% (3/91) reveal different modalities of engagement. found it difficult to use. The aforementioned results are in line with one of the largest Users suggested app improvements by responding to a data sets on engagement in remote digital health compiled to multiple-choice question. The improvements more frequently date [38] (there is significant attrition in remote research). In suggested were enabling a personalized plan to follow when addition, Pratap et al [38] observed indicators of retention in potential relapse symptoms appear (122/173, 70.5%), remote digital health studies through survival analysis, which personalization of stressful events (103/173, 59.5%), and a wider revealed the factors associated with an increase in participant variety of psychoeducational messages (99/173, 57.2%). retention time, including older age (an increase of 4 days). In contrast, our study revealed that years since onset and years Discussion since diagnosis of BD (2 variables highly related to age) had a significant impact on app use; mobile app median use of After proving positive outcomes regarding satisfaction, usability, participants with earlier disease onset and diagnosis increased and helpfulness in previous research, the SIMPLe app was by 32 days and 29 days compared with individuals with a more launched to the real world to make it available worldwide within recent onset and more recent diagnosis, respectively. Similarly, the context of BD treatment. when comparing potential predictors of traditional group Principal Findings psychoeducation in BD in a digital format, Reinares et al [39] The outcomes of this real-world study represent the first attempt identified that receiving an early diagnosis of BD may indicate to evaluate, by means of survival analysis, use, retention a better response to face-to-face group psychoeducation. Other patterns, and engagement of a large-scale wide-reaching factors with a significant contribution to the risk of discontinuing app-based intervention providing psychoeducational content to app use (that are also highly associated with age) were housing patients with BD. and employment status. Retention duration of unemployed participants and those who shared a house was lower than that One of the most important advantages of the data collected of users with other housing statuses. It is obvious that having a through smartphone apps in clinical studies is the continuous recent onset and diagnosis, sharing a house, and unemployment and granular characteristics of the data registered at servers. are more common in younger populations, suggesting a Using detailed log and use data to examine predictive factors disadvantage regarding app use. allows an understanding of the engagement and its underlying mechanisms. It will also aid optimization of smartphone-based https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 13 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Our findings are consistent with previously collected preliminary brief psychoeducation in BD. At 12 weeks, the retention reported use data on the SIMPLe app [22]. We previously observed that was 93%; however, it was only operationalized based on the older age was a predictor significantly associated with higher percentage of participants who returned the borrowed mobile odds of retention. In addition, the attrition rate of the program devices of the study. was still high, but this time our research focused on retention At odds with the aforementioned studies, the SIMPLe app was factors. Furthermore, overall satisfaction of the participants was designed as an independent self-management method targeting quite positive because 65.3% (113/173) were highly or relapse prevention. For research purposes, the study team helped completely satisfied with SIMPLe in the context of low (remotely) participants to install the app and log into the system, retention. The satisfaction, usability, and helpfulness outcomes as well as provided a brief explanation. Users were provided are in line with our previous results deriving from an with a telephone number to contact the research team for further implementation feasibility study of SIMPLe. In terms of assistance in case they experienced technical issues. The usefulness, the best-rated features and functions of the app were retention rate of the original SIMPLe study was 74% (36/51) daily and weekly tests, mood charts, and personalized after 3 months of app use [21]. psychoeducational messages. The last-named is a unique and differentiating feature that other mobile-based platforms for BD The OpenSIMPLe study differs from the others in that it is the do not offer [40]. On the basis of the information collected only modality that does not involve some contact with a person through screening tests on mood, energy, sleep time, medication providing support or human interaction. It is reasonable to adherence, and irritability, the SIMPLe app pushes daily pop-up assume that the lower retention rates of the OpenSIMPLe study notifications with a short psychoeducational message containing may have been influenced by the absence of human support in brief information (usually fewer than 50 words) on how to deal comparison with the other studies assessing smartphone-based with specific situations to avoid relapses. Psychoeducational platforms; the latter were more demanding in terms of time and short messages carefully provided to cover detected specific staff resources. Besides the notifications systems recalling user needs are the closest way to feature the human touch, and adherence in the aforementioned studies, the fact that they fulfill quite convincingly our initial intention to partly participants established an alliance with clinical study staff and replicate the successful Barcelona Bipolar Psychoeducation received previous face-to-face intervention or continuous model [18] in an app so that it could de-escalate treatment costs telephone-delivered psychoeducation has influenced retention and make combined therapy (psychoeducation and for certain [44]. psychopharmacology) available to the greatest number of It has been suggested that a positive relationship between app affected individuals. engagement and improvements in therapeutic outcomes in In contrast, Faurholt-Jepsen et al [41] investigated the effect of mental health and well-being may indicate the effectiveness of smartphone-based patient-reported and objective monitoring, internet-based interventions [45]. Moreover, the type of including a mood prediction system. The collected objective engagement in terms of intervention use has demonstrated smartphone data included phone use, social activity, and different relationships with outcomes [46]. However, there is a mobility. Patients with BD were randomized to the Monsenso general tendency among mental health apps toward low retention monitoring system (Monsenso A/S) or to standard treatment. and engagement rates [32,47]. At the same time, detailed Clinical feedback was established for patients in the intervention engagement and retention rates are rarely reported in group in the case of signs of impairment (eg, lack of smartphone-based clinical trials but are of crucial importance self-monitoring data). Overall, there was no effect of to understand the underpinning of their effects. Those studies smartphone-based monitoring on symptoms compared with the reporting and analyzing them frequently do so with control group. The intervention group adhered to the daily heterogeneous and nonstandardized methodologies. self-monitoring 72.6% (196/270) of the days over the 9-month It is worth mentioning that we are comparing our retention, app study period. use, and engagement data with other studies that used different Yet another project [42] evaluated 2 mobile phone–augmented parameters to measure these indicators for mental health apps interventions: an in-person session of CBT combined with and even used different criteria to assess them. Results from a automated thought-challenging and adaptive behavior delivered systematic review conducted by Ng et al [32] indicated high through mobile devices and an in-person session of variability across studies regarding the operationalization of the psychoeducation with mobile interaction involving indicators of engagement, as well as highlighted the need to self-monitoring of symptoms. The retention rates were 77% for using objective criteria when assessing them. To date, the lack the self-monitoring group and 91% for the CBT condition at 6 of clear consensus on the definition and standardization of these months. Follow-up in both active conditions consisted of parameters represents a big obstacle to understanding feasibility telephone calls by the study therapist to remind participants of and comparing results in the field of mobile-based apps and assessment appointments and encourage adherence. However, digital therapeutics. The real effectiveness of every intervention these outcomes are difficult to extrapolate and compare because can be hardly estimated without quantifying the exact dosage the cited Californian study included patients with schizophrenia of intervention that individuals receive, and engagement is not as well as patients with BD. Previously, Depp et al [43] had usually considered or it is evaluated erratically through effect explored a similar approach integrating a mobile size estimation in randomized controlled trials. device–delivered intervention linking patient-reported mood states with self-management strategies, preceded by face-to-face https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 14 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al Limitations the time range of app use between patients who used the Some limitations of this study should be noted, and caution SIMPLe app (n=390) and those who used it and completed the should be exercised in generalizing results. First, our analyses follow-up assessment (n=173); the former group used it a mean relied on a rather heterogeneous sample, where participants of 88.64 (SD 70.56) days, whereas this mean increased to 119.64 differed in terms of sociodemographic, clinical, and (SD 69.9) days in the latter group. However, the former group psychological characteristics. This can be explained by the fact used the app slightly more every day on average (mean 0.92, that we opened the platform to a real-world setting without SD 0.73) than the latter group (mean 0.83, SD 0.51). Therefore, considering inclusion and exclusion criteria that were too these differences in terms of app use between the groups suggest restrictive. However, the participants were a good representation that the users may have different profiles. of an unselected real-world population. As this work was an exploratory study (ie, a flexible rather than All measures were administered using exclusively self-reported structured approach to data collection was considered useful), web-based methods that did not allow us to get back in touch there was no control group or alternative intervention for with participants who dropped out to collect feedback on the comparison of effects because the study was not designed to reasons for attrition. In addition, we did not have access to either test the efficacy of the SIMPLe app. For the same reason and medical records or passive data to validate the accuracy and to avoid unlimited assessments that would probably result in reliability of the information provided, which may have the attrition rate soaring to unacceptable levels, we decided to influenced our sample and outcomes. As shown in Figure 1, keep control and covariate data to a minimum, which obviously there were only 2 participants excluded by the MDQ at represents at the same time something gained and something screening. The MDQ is a popular, simple, and sensible screening lost. instrument for the detection of BD. However, this tool is far All the participants included came from Latin American or from perfect [48]; it has low specificity, and it is likely that it Spanish populations. The cultural characteristics of these origins did not discriminate among participants with a range of disorders may be difficult to generalize, but little is known about app such as borderline personality disorder. This is a common adherence (or even drug or psychotherapy adherence) across disadvantage in studies that screen participants through cultures. This may become an exciting topic awaiting proper web-based methods exclusively. exploration. A weakness of this study is that we limited use and retention Finally, it should be considered that the outcomes of this study analysis to the regular users of the SIMPLe 1.5 app and did not deal with a high level of missing data derived from highly analyze other data from the occasional users, who did not use variable retention rates and lack of adherence after a few weeks the app consistently. However, the aim of limiting these analyses of use among users of mental health apps, which is a common to data provided by regular users was to avoid overestimation hindrance in internet-based research [12,49] that we tried to of use time and retention. In addition, sensitivity and handle in an honest and rigorous manner. homogeneity analysis confirmed that the data were coherent when we repeated the survival analysis with the whole sample Conclusions of users; this showed that the selection of regular users in the The user retention rate of the app decreased at a rapid rate after study of app use prevented an overestimation of it, whereas the each month until reaching only one-third of the users at 6 effect of the selection on survival probability is small. months. There exists a strong association between age and app engagement of individuals with BD. Other variables such as Furthermore, only 44.4% (173/390) of the users completed the years lived with BD, diagnosis of an anxiety disorder, and taking follow-up assessment, which implies some bias in the data antipsychotics seem to play a relevant role as well. We believe collected regarding evaluation of the app because the variables that an understanding of these associations will help clinicians measured at follow-up (including SUS, perceived usefulness, in the definition of the most suitable user profiles for predicting and satisfaction) were exclusively assessed by these users. An trends of engagement, optimization of app prescription, and example that indicates this bias is that we found differences in management. Acknowledgments The authors of this manuscript were indirectly supported by research grants from the Spanish Ministry of Economy and Competitiveness PI15/00588 and PI19/00009 (to FC) and FI20/00008 (to AGE); Instituto de Salud Carlos III, Subdirección General de Evaluación y Fomento de la Investigación; and Fondo Europeo de Desarrollo Regional, Unión Europea, Una manera de hacer Europa. Other sources of indirect support are a Juan Rodés (JR18/00021) contract from Instituto de Salud Carlos III (to DH) as well as Secretaria d′Universitats I Recerca del Departament d′Economia I Coneixement (2017 SGR 1365 to EV, August Pi i Sunyer Biomedical Research Institute, and 2017 SGR 134 to FC, Hospital del Mar Medical Research Institute). GA is supported by a Pons Bartran 2020 grant (PI046549). EV’s research is supported by the Spanish Ministry of Science and Innovation integrated into the Plan Nacional de I+D+I and cofinanced by the Instituto de Salud Carlos III (Subdirección General de Evaluación) and the Fondo Europeo de Desarrollo Regional; the ISCIII; CIBER Mental Health; the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365), the Centres de Recerca de Catalunya program, and the Departament de Salut de la Generalitat de Catalunya. The funding sources did not have any involvement in study design, collection, analysis, https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 15 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al interpretation of data, or the writing of this manuscript. The authors are very grateful to all the users of the SIMPLe app for their collaboration in this project. The technical development of the app software was commissioned to Sodep SA (Asunción, Paraguay). Authors' Contributions This study was conceived by FC, DH, and EV. DH and FC developed and maintained the project’s website. FC, DH, and AGE were responsible for the methodology. JC assisted with data cleaning, statistical analysis, and interpretation of the results. Regarding the manuscript, the original draft was prepared by AGE, GA, and DH; review and editing were carried out by NAO, GA, EMM, FC, and VP; and supervision was by FC. EV and FC were responsible for funding acquisition. All authors have read and approved the final version of the manuscript. Conflicts of Interest DH, EV, and FC designed the SIMPLe smartphone app mentioned in this study. The authors do not have any economic interests in the SIMPLe app, its use, or copyrights. EV has received grants and served as consultant, advisor or CME speaker for the following entities (unrelated to the present work): AB-Biotics, Abbott, Abbvie, Aimentia, Angelini, Biogen, Boehringer -Ingelheim, Casen-Recordati, Celon, Dainippon Sumitomo Pharma, Ferrer, Gedeon Richter, GH Research, Glaxo Smith-Kline, Janssen, Lundbeck, Organon, Otsuka, Sage, Sanofi-Aventis, Sunovion, Takeda, and Viatris. GA has received CME-related honoraria, or consulting fees from Janssen-Cilag, Lundbeck, and Angelini with no financial or other relationship relevant to the subject of this article. References 1. GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018 Dec 10;392(10159):1789-1858 [FREE Full text] [doi: 10.1016/S0140-6736(18)32279-7] [Medline: 30496104] 2. MacQueen G, Young LT, Joffe R. A review of psychosocial outcome in patients with bipolar disorder. Acta Psychiatr Scand 2001 Mar;103(3):163-170 [FREE Full text] [doi: 10.1034/j.1600-0447.2001.00059.x] [Medline: 11240572] 3. Michalak EE, Yatham LN, Lam RW. Quality of life in bipolar disorder: a review of the literature. Health Qual Life Outcomes 2005 Nov 15;3:72 [FREE Full text] [doi: 10.1186/1477-7525-3-72] [Medline: 16288650] 4. Dutta R, Boydell J, Kennedy N, Van Os J, Fearon P, Murray RM. Suicide and other causes of mortality in bipolar disorder: a longitudinal study. Psychol Med 2007 Mar 12;37(6):839-847 [FREE Full text] [doi: 10.1017/s0033291707000347] [Medline: 17349107] 5. Pompili M, Gonda X, Serafini G, Innamorati M, Sher L, Amore M, et al. Epidemiology of suicide in bipolar disorders: a systematic review of the literature. Bipolar Disord 2013 Aug 12;15(5):457-490 [FREE Full text] [doi: 10.1111/bdi.12087] [Medline: 23755739] 6. National Collaborating Centre for Mental Health (UK). Bipolar disorder: the NICE guideline on the assessment and management of bipolar disorder in adults, children and young people in primary and secondary care. In: National Institute for Health and Care Excellence: Clinical Guidelines. London, UK: The British Psychological Society and The Royal College of Psychiatrists; 2014. 7. Scott J, Colom F, Popova E, Benabarre A, Cruz N, Valenti M, et al. Long-term mental health resource utilization and cost of care following group psychoeducation or unstructured group support for bipolar disorders: a cost-benefit analysis. J Clin Psychiatry 2009 Mar 10;70(3):378-386 [FREE Full text] [doi: 10.4088/jcp.08m04333] [Medline: 19284929] 8. Saxena S, Thornicroft G, Knapp M, Whiteford H. Resources for mental health: scarcity, inequity, and inefficiency. Lancet 2007 Sep;370(9590):878-889 [FREE Full text] [doi: 10.1016/s0140-6736(07)61239-2] [Medline: 17804062] 9. Christensen H, Griffiths K, Evans K. E-Mental Health in Australia?: Implications of the Internet and Related Technologies for Policy. Canberra: Commonwealth Dept. of Health and Ageing; 2002. 10. World Health Organization. Global Diffusion of eHealth: Making Universal Health Coverage Achievable. Report of the Third Global Survey on eHealth. Geneva: World Health Organization; 2016. 11. Berry N, Lobban F, Emsley R, Bucci S. Acceptability of Interventions Delivered Online and Through Mobile Phones for People Who Experience Severe Mental Health Problems: A Systematic Review. J Med Internet Res 2016 May 31;18(5):e121 [FREE Full text] [doi: 10.2196/jmir.5250] [Medline: 27245693] 12. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health 2018 Aug 05;21(3):116-119 [FREE Full text] [doi: 10.1136/eb-2018-102891] [Medline: 29871870] 13. Lui JH, Marcus DK, Barry CT. Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof Psychol Res Pract 2017 Jun;48(3):199-210. [doi: 10.1037/pro0000122] 14. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS One 2016 Apr 13;11(4):e0152285 [FREE Full text] [doi: 10.1371/journal.pone.0152285] [Medline: 27073900] https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 16 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al 15. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res 2015 Aug 17;17(8):e198 [FREE Full text] [doi: 10.2196/jmir.4581] [Medline: 26283290] 16. Meurk C, Leung J, Hall W, Head BW, Whiteford H. Establishing and governing e-mental health care in australia: a systematic review of challenges and a call for policy-focussed research. J Med Internet Res 2016 Jan 13;18(1):e10 [FREE Full text] [doi: 10.2196/jmir.4827] [Medline: 26764181] 17. Wozney L, Newton AS, Gehring ND, Bennett K, Huguet A, Hartling L, et al. Implementation of eMental health care: viewpoints from key informants from organizations and agencies with eHealth mandates. BMC Med Inform Decis Mak 2017 Jun 02;17(1):78 [FREE Full text] [doi: 10.1186/s12911-017-0474-9] [Medline: 28577543] 18. Colom F, Vieta E. Psychoeducation Manual for Bipolar Disorder. Cambridge, United Kingdom: Cambridge University Press; 2006. 19. Colom F, Vieta E, Sánchez-Moreno J, Palomino-Otiniano R, Reinares M, Goikolea JM, et al. Group psychoeducation for stabilised bipolar disorders: 5-year outcome of a randomised clinical trial. Br J Psychiatry 2009 Mar 02;194(3):260-265 [FREE Full text] [doi: 10.1192/bjp.bp.107.040485] [Medline: 19252157] 20. Hidalgo-Mazzei D, Mateu A, Reinares M, Undurraga J, Bonnín CM, Sánchez-Moreno J, et al. Self-monitoring and psychoeducation in bipolar patients with a smart-phone application (SIMPLe) project: design, development and studies protocols. BMC Psychiatry 2015 Mar 20;15(1):52 [FREE Full text] [doi: 10.1186/s12888-015-0437-6] [Medline: 25884824] 21. Hidalgo-Mazzei D, Mateu A, Reinares M, Murru A, del Mar Bonnín C, Varo C, et al. Psychoeducation in bipolar disorder with a SIMPLe smartphone application: feasibility, acceptability and satisfaction. J Affect Disord 2016 Aug;200:58-66 [FREE Full text] [doi: 10.1016/j.jad.2016.04.042] [Medline: 27128358] 22. Hidalgo-Mazzei D, Reinares M, Mateu A, Nikolova VL, Bonnín CM, Samalin L, et al. OpenSIMPLe: A real-world implementation feasibility study of a smartphone-based psychoeducation programme for bipolar disorder. J Affect Disord 2018 Dec 01;241:436-445 [FREE Full text] [doi: 10.1016/j.jad.2018.08.048] [Medline: 30145515] 23. Hidalgo-Mazzei D, Reinares M, Mateu A, Juruena MF, Young AH, Pérez-Sola V, et al. Is a SIMPLe smartphone application capable of improving biological rhythms in bipolar disorder? J Affect Disord 2017 Dec 01;223:10-16 [FREE Full text] [doi: 10.1016/j.jad.2017.07.028] [Medline: 28711743] 24. Levin JB, Krivenko A, Howland M, Schlachet R, Sajatovic M. Medication adherence in patients with bipolar disorder: a comprehensive review. CNS Drugs 2016 Sep 19;30(9):819-835 [FREE Full text] [doi: 10.1007/s40263-016-0368-x] [Medline: 27435356] 25. Proyecto SIMPLe. Bipolar Disorders Program Barcelona. URL: https://simplebipolarproject.org/ [accessed 2021-06-22] 26. Hirschfeld RM, Holzer C, Calabrese JR, Weissman M, Reed M, Davies M, et al. Validity of the mood disorder questionnaire: a general population study. Am J Psychiatry 2003 Jan;160(1):178-180 [FREE Full text] [doi: 10.1176/appi.ajp.160.1.178] [Medline: 12505821] 27. Vieta E, Sánchez-Moreno J, Bulbena A, Chamorro L, Ramos J, Artal J, EDHIPO (Hypomania Detection Study) Group. Cross validation with the mood disorder questionnaire (MDQ) of an instrument for the detection of hypomania in Spanish: the 32 item hypomania symptom check list (HCL-32). J Affect Disord 2007 Aug;101(1-3):43-55 [FREE Full text] [doi: 10.1016/j.jad.2006.09.040] [Medline: 17189651] 28. Bonnín CM, Yatham L, Michalak E, Martínez-Arán A, Dhanoa T, Torres I, et al. Psychometric properties of the well-being index (WHO-5) Spanish version in a sample of euthymic patients with bipolar disorder. J Affect Disord 2018 Mar 01;228:153-159 [FREE Full text] [doi: 10.1016/j.jad.2017.12.006] [Medline: 29248821] 29. Brooke J. Smart phone applications for people with brain injury: the project SUS - a quick and dirty usability scale usability and context. In: Usability Evaluation In Industry. London: CRC Press; 1996. 30. 5 ways to interpret a SUS score. Measuring U. URL: https://measuringu.com/interpret-sus-score/ [accessed 2021-05-05] 31. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009;4(3):114-123 [FREE Full text] [doi: 10.5555/2835587.2835589] 32. Ng MM, Firth J, Minen M, Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Pyschiatr Serv 2019 Jul 01;70(7):538-544 [FREE Full text] [doi: 10.1176/appi.ps.201800519] [Medline: 30914003] 33. Scherer EA, Ben-Zeev D, Li Z, Kane JM. Analyzing mHealth engagement: joint models for intensively collected user engagement data. JMIR Mhealth Uhealth 2017 Jan 12;5(1):e1 [FREE Full text] [doi: 10.2196/mhealth.6474] [Medline: 28082257] 34. Colom F, Vieta E, Reinares M, Martínez-Arán A, Torrent C, Goikolea JM, et al. Psychoeducation efficacy in bipolar disorders: beyond compliance enhancement. J Clin Psychiatry 2003 Sep 15;64(9):1101-1105 [FREE Full text] [doi: 10.4088/jcp.v64n0917] [Medline: 14628987] 35. Edney S, Ryan JC, Olds T, Monroe C, Fraysse F, Vandelanotte C, et al. User engagement and attrition in an app-based physical activity intervention: secondary analysis of a randomized controlled trial. J Med Internet Res 2019 Nov 27;21(11):e14645 [FREE Full text] [doi: 10.2196/14645] [Medline: 31774402] 36. Lin Y, Chen S, Lin P, Tai A, Pan Y, Hsieh C, et al. Assessing user retention of a mobile app: survival analysis. JMIR Mhealth Uhealth 2020 Nov 26;8(11):e16309 [FREE Full text] [doi: 10.2196/16309] [Medline: 33242023] https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 17 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al 37. Chien I, Enrique A, Palacios J, Regan T, Keegan D, Carter D, et al. A machine learning approach to understanding patterns of engagement with internet-delivered mental health interventions. JAMA Netw Open 2020 Jul 01;3(7):e2010791 [FREE Full text] [doi: 10.1001/jamanetworkopen.2020.10791] [Medline: 32678450] 38. Pratap A, Neto EC, Snyder P, Stepnowsky C, Elhadad N, Grant D, et al. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. NPJ Digit Med 2020 Feb 17;3(1):21 [FREE Full text] [doi: 10.1038/s41746-020-0224-8] [Medline: 32128451] 39. Reinares M, Pacchiarotti I, Solé B, García-Estela A, Rosa AR, Bonnín CM, et al. A prospective longitudinal study searching for predictors of response to group psychoeducation in bipolar disorder. J Affect Disord 2020 Sep 01;274:1113-1121 [FREE Full text] [doi: 10.1016/j.jad.2020.02.047] [Medline: 32663939] 40. Lagan S, Ramakrishnan A, Lamont E, Ramakrishnan A, Frye M, Torous J. Digital health developments and drawbacks: a review and analysis of top-returned apps for bipolar disorder. Int J Bipolar Disord 2020 Dec 01;8(1):39 [FREE Full text] [doi: 10.1186/s40345-020-00202-4] [Medline: 33259047] 41. Faurholt-Jepsen M, Frost M, Christensen EM, Bardram JE, Vinberg M, Kessing LV. The effect of smartphone-based monitoring on illness activity in bipolar disorder: the MONARCA II randomized controlled single-blinded trial. Psychol Med 2019 Apr 04;50(5):838-848 [FREE Full text] [doi: 10.1017/S0033291719000710] [Medline: 30944054] 42. Depp C, Perivoliotis D, Holden J, Dorr J, Granholm E. Single-session mobile-augmented intervention in serious mental illness: a three-arm randomized controlled trial. Schizophr Bull 2019 Jun 18;45(4):752-762 [FREE Full text] [doi: 10.1093/schbul/sby135] [Medline: 30281086] 43. Depp CA, Ceglowski J, Wang VC, Yaghouti F, Mausbach BT, Thompson WK, et al. Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord 2015 Mar 15;174:23-30 [FREE Full text] [doi: 10.1016/j.jad.2014.10.053] [Medline: 25479050] 44. Baumel A, Kane JM. Examining predictors of real-world user engagement with self-guided ehealth interventions: analysis of mobile apps and websites using a novel dataset. J Med Internet Res 2018 Dec 14;20(12):e11491 [FREE Full text] [doi: 10.2196/11491] [Medline: 30552077] 45. Carpenter J, Crutchley P, Zilca RD, Schwartz HA, Smith LK, Cobb AM, et al. Seeing the "Big" picture: big data methods for exploring relationships between usage, language, and outcome in internet intervention data. J Med Internet Res 2016 Aug 31;18(8):e241 [FREE Full text] [doi: 10.2196/jmir.5725] [Medline: 27580524] 46. Short CE, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, et al. Measuring engagement in ehealth and mhealth behavior change interventions: viewpoint of methodologies. J Med Internet Res 2018 Nov 16;20(11):e292 [FREE Full text] [doi: 10.2196/jmir.9397] [Medline: 30446482] 47. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [doi: 10.2196/14567] [Medline: 31573916] 48. Zimmerman M, Galione JN, Ruggero CJ, Chelminski I, Dalrymple K, Young D. Are screening scales for bipolar disorder good enough to be used in clinical practice? Compr Psychiatry 2011 Nov;52(6):600-606 [FREE Full text] [doi: 10.1016/j.comppsych.2011.01.004] [Medline: 21406301] 49. Arean PA, Hallgren KA, Jordan JT, Gazzaley A, Atkins DC, Heagerty PJ, et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res 2016 Dec 20;18(12):e330 [FREE Full text] [doi: 10.2196/jmir.6482] [Medline: 27998876] Abbreviations BD: bipolar disorder CBT: cognitive behavioral therapy MDQ: Mood Disorder Questionnaire SUS: System Usability Scale Edited by R Kukafka; submitted 25.06.21; peer-reviewed by M Elhadi, E Chan, E Morton; comments to author 23.07.21; revised version received 16.09.21; accepted 29.10.21; published 02.02.22 Please cite as: García-Estela A, Cantillo J, Angarita-Osorio N, Mur-Milà E, Anmella G, Pérez V, Vieta E, Hidalgo-Mazzei D, Colom F Real-world Implementation of a Smartphone-Based Psychoeducation Program for Bipolar Disorder: Observational Ecological Study J Med Internet Res 2022;24(2):e31565 URL: https://www.jmir.org/2022/2/e31565 doi: 10.2196/31565 PMID: https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 18 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH García-Estela et al ©Aitana García-Estela, Jordi Cantillo, Natalia Angarita-Osorio, Estanislao Mur-Milà, Gerard Anmella, Víctor Pérez, Eduard Vieta, Diego Hidalgo-Mazzei, Francesc Colom. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 02.02.2022. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included. https://www.jmir.org/2022/2/e31565 J Med Internet Res 2022 | vol. 24 | iss. 2 | e31565 | p. 19 (page number not for citation purposes) XSL• FO RenderX