JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Original Paper A Mobile App (mHeart) to Detect Medication Nonadherence in the Heart Transplant Population: Validation Study Mar Gomis-Pastor1,2, DPharm; Eulalia Roig2,3*, MD, PhD; Sonia Mirabet2*, MD, PhD; Jan T De Pourcq4, DPharm; Irene Conejo4, DPharm; Anna Feliu4, PhD, DPharm; Vicens Brossa2, MD; Laura Lopez2, MD; Andreu Ferrero-Gregori5, BSc; Anna Barata6, DClinPsych; M Antonia Mangues4, PhD, DPharm 1 Pharmacy Department, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain 2 Heart Failure and Heart Transplant Unit, Cardiology Department, Hospital de la Santa Creu i Sant Pau, Barcelona, Catalonia, Spain 3 Medicine Department, Autonomous University of Barcelona (UAB), Barcelona, Catalonia, Spain 4 Pharmacy Department, Hospital de la Santa Creu i Sant Pau, Barcelona, Catalonia, Spain 5 Institute of Biomedical Research IIB Sant Pau, Biomedical Research Networking Center on Cardiovascular Diseases (CIBERCV), Universitat Autonoma de Barcelona, Barcelona, Catalonia, Spain 6 Moffitt Cancer Center, Florida, FL, United States * these authors contributed equally Corresponding Author: Mar Gomis-Pastor, DPharm Pharmacy Department Hospital de la Santa Creu i Sant Pau San Quintin nº89 PC 08041 Barcelona, 08026 Spain Phone: 34 636060364 Email: mar@margomis.com Abstract Background: Medication nonadherence in heart transplant recipients (HTxR) is related to graft loss and death. mHeart is a mobile app that uses electronic patient-reported outcome measures (ePROMs) to identify and manage medication nonadherence in the outpatient heart transplant (HTx) population. Objective: The study primarily aimed to validate mHeart to measure medication nonadherence in early stage HTxR by assessing the psychometric properties of ePROMs. The secondary aims were to (1) measure patient satisfaction with the mHeart tool and its usability and (2) explore the impact of a theory-based treatment on medication nonadherence rates to determine its scalability to larger research. Methods: A prospective study was conducted in the outpatient clinic of a tertiary hospital. All consecutive early stage HTxR (<1.5 years from HTx) were included. The ePROM psychometric properties assessed were validity, reliability, responsiveness, interpretability, and burden. ePROMs comprised the 4-item Morisky-Green-Levine questionnaire and an adapted version of the Haynes-Sackett questionnaire. The Simplified Medication Adherence Questionnaire (SMAQ) was also applied on-site. Three consecutive medication nonadherence assessments were performed by a transplant pharmacist. To improve medication nonadherence, theory-based interventions were delivered in a 1-month period. Patient satisfaction was assessed by a semiquantitative Web-based survey at the end of the study. Results: We included 31 early stage HTxR (age: mean 54 years, SD 12 years), and 71% (22/31) of them were men. The HTxR were taking a mean 13 (SD 4; range 7-18) drugs per day. A total of 42% (13/31) of patients were unaware of the consequences of medication nonadherence, and 39% (12/31) of patients were nonadherent to immunosuppressive treatment. The content validity measure showed excellent levels of expert panel agreement for the Haynes-Sacket (14/14, 100%) and Morisky-Green-Levine (13/14, 93%) questionnaires. SMAQ and Morisky-Green-Levine ePROMs showed similar measurement domains (convergent validity, phi=0.6, P<.001), which, as expected, differed from Haynes-Sackett ePROMs (divergent validity, phi=0.3, P=.12). Reliability assessment revealed a very strong association between ePROM and on-site PROMs (phi>0.7, P<.001). Reproducibility was moderate (Haynes-Sackett κ=0.6, P<.002) or poor (Morisky-Green-Levine κ=0.3, P=.11) because of unexpected improved medication adherence rates during the test-retest period. According to responsiveness, the theory-based multifaceted intervention https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 1 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al program improved medication nonadherence by 16% to 26% (P<.05). A burden analysis showed that ePROMs could potentially overcome traditional on-site limitations (eg, automatic recording of ePROM responses in the hospital information system). The mean score for overall patient satisfaction with the mHeart approach was 9 (SD 2; score range: 0-10). All 100% (29/29) of patients surveyed reported that they would recommend the mHeart platform to other HTxR. Conclusions: ePROMs adhered to the quality standards and successfully identified medication nonadherence in the HTx population, supporting their widespread use. The theory-based intervention program showed a promising improvement in medication adherence rates and produced excellent patient satisfaction and usability scores in HTxR. (JMIR Mhealth Uhealth 2020;8(2):e15957) doi: 10.2196/15957 KEYWORDS self-report; patient-reported outcome measures; behavioral sciences; treatment adherence and compliance; transplantation; early medical intervention; telemedicine; mobile health; validation studies; patient satisfaction Behavior-based theories with demonstrated effectiveness in Introduction reducing medication nonadherence are recommended to be Background combined when a new intervention program is designed [21,22]. Moreover, motivational interviewing is a useful tool for Heart transplant recipients (HTxR) require lifelong professionals to deliver such theory-based interventions in the immunosuppressive therapy to prevent rejection episodes. The transplant population [23,24]. Although these behavior change estimated percentage of medication nonadherence to techniques are increasingly used, there is a lack of studies immunosuppressive treatment after heart transplant (HTx) applying them to internet interventions in the HTx population ranges from 15% to 30% [1]. These rates are worrisome as [2,25,26]. medication nonadherence impairs quality of life, increases health costs, and is a direct cause of graft loss and death after HTx Previous Work [1-6]. The mHeart medical device is a mobile app that is primarily Medication nonadherence in the HTx population is a dynamic intended to measure and manage medication adherence in behavior influenced by multilevel patient, provider, and health outpatient HTxR. mHeart was designed to use ePROMs to system factors [7]. To improve medication adherence, it is identify medication nonadherence in the home setting and essential to frequently monitor medication nonadherence and facilitate behavior change interventions to improve medication identify modifiable risk factors for medication nonadherence, nonadherence rates. According to the directions for the such as high therapeutic complexity, weak professional-patient International Society for Research on Internet Interventions relationship, and lack of patient motivation [4,5,7,8]. (ISRII) [13], a prerequisite before recommending the widespread use of internet delivery is to demonstrate the accuracy of Subjective methods to evaluate medication adherence, such as ePROM scores and their relationship with traditional in-clinic self-reporting, are widely used [9] and well correlated with methods [27]. Moreover, the workflow of any new electronic objective methods (eg, immunosuppressive drug level assay or behavior-based intervention designed to manage medication electronic monitoring systems) [5,10]. Indeed, self-reporting is nonadherence should be tested before scaling it to larger research considered the best method to capture patient experiences and [28]. individual risk factors for medication nonadherence, such as patients’ medication beliefs [11]. However, this method involves Study Objectives in-clinic facilities and requires patients to travel to the clinic The primary aim of this study was to validate mHeart to measure [5]. medication nonadherence in early stage HTxR in the home setting. To do this, we sought to identify the quality of the Emerging research indicates that patient-reported adherence psychometric properties of the ePROMs reported as being through mobile devices produces data of similar quality to those critical in electronic health behavior change instruments [29,30], provided by traditional in-clinic methods [12]. Therefore, the that is, validity, reliability, responsiveness, interpretability, and use of electronic patient-reported measures (ePROMs) to detect burden [31]. nonadherent HTxR could help increase the feasibility of self-reporting and overcome current in-clinic limitations [13,14]. The secondary aims were (1) to measure patient satisfaction with mHeart and its usability and (2) to explore the impact of Medication adherence ePROMs could also provide valuable theory-based interventions on medication nonadherence rates information to care providers to implement early and among HTxR to determine the hypothetical scalability of the personalized interventions through mobile health technology. treatment within the context of a larger research study. Indeed, internet interventions (ie, “treatments, typically behaviorally based, that are operationalized and transformed for delivery via the Internet”) [13,15-17] show a promising Methods impact on prompting changes in health behaviors such as Study Design and Setting medication adherence [15,16,18-20]. This prospective research study was conducted in the ambulatory setting of a Heart Failure and Transplant Unit of a tertiary https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 2 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al university hospital from July 15, 2016, to December 1, 2016. [17] were followed to report the internet-based intervention The study was approved by the institutional review board of program. The Theory Coding Scheme (TCS) [32] provided a the hospital (IIBSP-MHE-2014-55). The participants were reliable method to describe the theory underpinning the informed of the study purposes, the length of the follow-up, all interventions. the procedures, and the research team behind the study. Written In addition, the Checklist for Reporting Results of Internet informed consent was obtained from all participants. E-Surveys [33] was applied to ensure the quality of reporting Study Reporting Guidelines of the Web-based satisfaction survey. The psychometric quality of the ePROMs was based on the Sample Scientific Advisory Committee of the Medical Outcomes Trust Enrollment was conducted from July 21, 2016, to October 26, (SAC-MOS) [31] and the COnsensus-based Standards for the 2016, in the Cardiology Outpatient Clinic by transplant selection of health Measurement INstruments (COSMIN) physicians during routine in-clinic appointments. All consecutive consensus guideline [30]. The quality of the results obtained adult, early stage HTxR (less than 1.5 years from HTx) owning was compared with the International Society for Quality of Life a smartphone and with no cognitive impairment were included. Research (ISOQOL) standards [11]. Cognitive impairment was defined as any condition limiting We followed the European Society for Patient Adherence, patients’ ability, including memory and thinking skills, to use COMpliance, and Persistence Medication Adherence Reporting the mHeart system and complete the questionnaires. No previous Guideline (EMERGE) [28] recommended criteria for transparent computer or smartphone knowledge was required. HTxR did and accurate medication adherence reporting data. The directions not receive any financial compensation, a phone, or wearables for the ISRII [13] and the Consolidated Standards of Reporting for their participation. The patient flowchart is shown in Figure Trials of Electronic and Mobile Health Applications and OnLine 1. TeleHealth (CONSORT-EHEALTH) guidelines (section 5) Figure 1. Val-mHeart study patient flowchart. Early-stage: <1.5 years from HTx; ePROMs: electronic patient-reported outcome measures; HTx: heart transplant. assessed for eligibility (ie, the same day as enrollment by the Study Procedures physicians), and they completed a baseline face-to-face visit The algorithm summarizing the procedures is shown in Figure with the transplant pharmacist, followed by an initial mHeart 2. After signing the informed consent form, all patients were training session. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 3 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Figure 2. Intervention algorithm summarizing the procedures performed throughout the study period. ePROM: electronic patient-reported outcome measure; HTx: heart transplant; PROMs: patient-reported outcome measures. The interview with the pharmacist lasted approximately 45 min. Thereafter, a technical mHeart initial setup was provided by the Sociodemographic and clinical data were extracted from mHeart Help Center of the private firm developing the patients’ electronic health records. The data were collected in technology. This session was conducted by telephone and lasted a form provided in Multimedia Appendices 1 and 2. At the end at least 15 min to enable at-home monitoring, that is, (1) of the visit, the pharmacist registered the new patient’s profile downloading the app from the app store, (2) guiding the first in the mHeart system. Patient access was facilitated by an access, and (3) providing training on the functionalities of the automated message sent to the patient’s phone with a username mHeart platform. This service was also responsible for query and password. resolution and user assistance throughout the study. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 4 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al As soon as the HTxR had received training, the transplant interview to perform the second medication adherence PROMs pharmacist sent them a welcome message through mHeart, assessment. On the same day, the HTxR were also asked to requesting the patients’ response to confirm their activation in complete the ePROMs in the home setting. the mHeart follow-up. Once the patients had responded to this Thereafter, to allow the test-retest reliability analysis, the message, three consecutive assessments were scheduled. The patients used mHeart for 7 days without any additional assessment procedures are described below, and these were interventions by the pharmacist or contact with the HTx team. conducted to measure the validity properties of the ePROMs. At the end of the reproducibility time interval, the patients were Assessment 1 telephoned by the pharmacist to confirm clinical and therapeutic After the baseline visit (ie, on the same day), medication stability. adherence was measured by the pharmacist using in-clinic Assessment 3 patient-reported outcome measures (PROMs; Multimedia After the test-retest reliability analysis, HTxR were asked to Appendix 2). No other interventions were performed to manage electronically complete the mHeart ePROMs and the satisfaction medication adherence during this in-clinic interview. On the and usability survey. same day, using the mHeart tool, patients were asked to complete the same ePROMs in the home setting. mHeart Features Used During the Study During the 1-month period between assessments 1 and 2, The mHeart medical device is a home-based mobile phone app multifaceted theory-based interventions were provided through complemented by a website [38]. From a technical point of mHeart to optimize adherence management [34]. The electronic view, access to the tool is multiplatform (ie, smartphone, tablet, interventions were interactive, with additional human support or computer), and it can be downloaded for free from app stores from the transplant pharmacist through the mHeart platform. [39,40]. mHeart is bidirectionally integrated with the hospital The interventions were individually tailored, based on electronic information system (HIS) using encrypted data. This integration patient-reported data. Several behavior change techniques between the two systems allows mHeart to directly obtain [21,22] were used based on those with the strongest evidence sociodemographic data from the HIS. In addition, mHeart base in medication adherence, such as social cognitive theory, uploads a weekly clinical report to the HIS, including all the the health belief model, transtheoretical model, and data reported by the patients on the platform. The general layout self-regulation model. Among others, less often reported but is represented in Figure 3. An in-depth description of the also used are the information-behavior-skill model, technical specification of the system and the source code are self-management theory, behavior modification theory, and provided in the online Mendely (Dataset) [41]. The version problem-solving theory [22]. The techniques were based on number of the app used was 3.0.2, and the content was frozen Michie’s taxonomy [35] and were delivered using motivational during the study. interviewing [23,24] as a common practice pattern to improve From a clinical point of view, the mHeart tool was designed to posttransplant medication adherence in HTx centers [9]. primarily manage medication nonadherence using several Interactive elements were also used as digital triggers to counter features (Table 1). In addition, three of the subfunctionalities the law of attrition: alerts, prompts, reminders, notifications, of the platform were to (1) resolve patients’ queries about their messages, logs, reports, visualizations, and video calls [36,37]. treatment and health condition, (2) empower patients in terms The theoretical framework, the behavior change intervention of self-care, and (3) facilitate professionals’ interventions based techniques used, and the intervention workflow are fully on patient-reported outcomes (ie, symptoms and adverse effects described in Multimedia Appendix 3. of drugs, heart rate, glycemia, weight, and blood pressure). A Assessment 2 detailed demonstration of the clinical use of mHeart in HTxR can be found in a video format in Multimedia Appendix 4. More Once the intervention program period finished, at least 30 days details about functionalities are also provided in the online after assessment 1, the pharmacist conducted an in-clinic Mendely (Dataset) [41]. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 5 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Figure 3. The mHeart functional layer and cloud architecture. AWS: Amazon Web Services; BI: business intelligence; HIS: hospital information system; HL7: high level-7; LOPD: the Spanish Organic Data Protection Law; WS: Web server. Table 1. mHeart platform features related to medication adherence management. Features Descriptions Patient drug intakes Push text reminds patients of medication intakes on their mobile phone. Patients can accept or reject the intakes scheduled. If a patient cancels a dose, they are asked to specify their reason for doing so on a checklist. Doses taken versus the total number of doses prescribed can be tracked. • A traffic light system warns the professional of a decrease in the patient’s weekly adherence. • Detailed data are presented for patients and professionals in tables or graphs, including reasons for not taking medication. Medication adherence ePROMsa The ePROMs included to detect MNAb are the 1-item Haynes-Sackett questionnaire [42,43] adapted to the mHeart platform and the 4-item Morisky-Green-Levine questionnaire [44]. The professional sets up the frequency of the electronic questionnaire on the patient’s diary. Push text alerts on the phone remind the patient to perform the programmed task. Test results are shown in tables and graphs to patients and professionals directly from the HISc or the mHeart platform website. a ePROM: electronic patient-reported outcome measure. b MNA: medication nonadherence. c HIS: hospital information system. reduction or increase in dosing). Poor regularity of intakes refers Measurement Variables to delays up to 2 hours in the transplant population [45,46]. Medication Adherence Measures Medication nonadherence measured by the questionnaires below was defined as any response to items with an answer indicating On the basis of the Ascertaining Barriers to Compliance nonadherence. taxonomy, medication adherence is divided into three phases: initiation, implementation, and persistence [34]. In this study, The ePROM validity study was based on two questionnaires we focused on assessing the implementation phase of medication implemented in the mHeart tool. First, the nonadherence by using self-reported instruments. Medication Morisky-Green-Levine questionnaire is a 4-item scale [44] nonadherence implementation is defined as “the extent to which assessing patients’ medication-taking habits. In transferring the a patient’s actual dosing corresponds to the prescribed dosing questionnaire to an electronic format, we implemented an exact regimen” (ie, omitting single or consecutive doses, delays in copy of the Spanish validated version [47]. Second, the medication intakes, or self-initiated dose changes such as a Haynes-Sackett questionnaire [42,43] is a 1-item scale asking https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 6 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al patients whether they have any difficulty with their treatment. For the convergent and discriminant validity assessment, we In transferring this questionnaire to an electronic format, we used the Simplified Medication Adherence Questionnaire implemented the Spanish version [48] and added six (SMAQ) Spanish version as the standard instrument. This multiple-choice responses on patients’ difficulties with questionnaire is a 6-item scale validated in the transplant medication [49] to improve providers’ understanding of population receiving immunosuppressive treatment [50]. To nonadherence (Figure 4 and Multimedia Appendix 5). In both identify medication nonadherence risk factors [10], patients mHeart questionnaires, the items can be answered using Yes or were also asked about (1) the knowledge of their regimen, (2) No checkboxes. their opinion of the inconvenience of their medication regimens, (3) the importance of the immunosuppressive treatment, and (4) the adverse effects (Multimedia Appendix 3). Figure 4. Electronic version of the Haynes-Sackett questionnaire, including 6 additional responses by patients to aid provider understanding of their difficulties with medication adapted for use with the mHeart platform. The score is based on the item 1 response: No (adherent) or Yes (nonadherent). were able to review and change their answers before submitting Patient Satisfaction and Usability their responses. Patient satisfaction with the mHeart intervention program and the usability of the tool were assessed by a Web-based The survey was closed to the study participants. The participants nonvalidated survey created for the study using the Google were sent an mHeart message by a clinical pharmacist who was Forms tool. The survey items comprised 8 qualitative and 17 different from the transplant pharmacist in charge of the semiquantitative (score range: 0-10) questions (original version follow-up. The patients had no previous interaction with this in Multimedia Appendix 6). No personal information was provider. The message content comprised an invitation to collected. Adaptive questioning was used to reduce the complete the opinion survey to help the team and developers complexity of the survey. In addition, all items had a no improve the usability and clinical use of the tool. The patients response option, and no blank items were allowed. Respondents were assigned a random number from 1 to 31. The survey was voluntary, and no incentives were offered for participation. The https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 7 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al patients had 1 week to complete the survey before it was closed [33] were analyzed in depth. The completion time by to new responses. A reminder was sent to all the HTxR 3 days participants was not determined. after the invitation was issued. HTxR accessed the survey through a link uploaded to their mHeart personal profile. Survey Psychometric Variables to Assess Electronic completion was permitted by the Google Form tool when Patient-Reported Outcome Measure Validity participants provided their identification number to avoid The psychometric quality of the ePROMs was assessed in terms multiple entries. of validity, reliability, responsiveness, interpretability, and burden [30,31]. The validation measures and methodology are The responses and the survey completion rate (ie, the ratio of detailed in Multimedia Appendix 7 and briefly described in users who finished the survey/users who agreed to participate) Table 2. Table 2. Brief description of the validity properties assessed for the mHeart medication adherence electronic patient-reported outcome measures. Validity properties Descriptiona Content validity The interrater agreement among an expert panel was performed to assess the following three content validity aspects. The expert panel comprised 14 health professionals, including 3 nurses, 7 cardiologists, and 4 clinical pharmacists. • The suitability of the questionnaires proposed for inclusion in the mHeart app. The discussion was verbal, and voting was by hand. • The suitability of the ePROMsb compared with the traditional in-clinic version. After written records were taken, a verbal discussion was held. • The suitability of the six medication difficulties added to the electronic version of the Haynes- Sacket questionnaire. After written records were taken, a verbal discussion was held. Convergent and discriminant validity Convergent and discriminant validity were assessed using the following aspects: • The correlation between the ePROM rates and a standard questionnaire was assessed. • The complementarity of the adherence to medication domains of the ePROMs included in the mHeart system was measured. Reliability (reproducibility) Reliability and reproducibility were assessed using two methods with different purposes: • The equivalent forms reliability method was used to assess the adequate association between the ePROMs scores and the in-clinic scores. With this aim, the PROMs were assessed in the same group of patients and on the same day. • The test-retest reliability method was used to assess the stability of the ePROM scores during a short time period (7 days) in clinically stable patients. Responsiveness (sensitivity to change) Change over time in medication adherence was measured by the difference in ePROM scores while a theory-based intervention program was performed. A 1-month interval was considered adequate to measure the validity of an indirect smartphone measure [51,52]. Interpretability Three aspects of the interpretability property were analyzed and discussed: • The interpretation of the ePROM scores. • The meaningful change detected. • The scores obtained versus those published by other authors. Respondent and administrative burden Several criteriaa were assessed regarding the time, effort, and other criteria of the ePROMs, depending on the respondents’ and administrative points of view. a Full details on validity properties assessed are provided in Multimedia Appendix 7. b ePROM: electronic patient-reported outcome measure. version 22.0 (IBM, Armonk, New York) and the R version 3.5.1 Statistical Analyses (R Project for Statistical Computing, Vienna, Austria). Descriptive Analysis Validity Analysis Categorical variables are expressed as the number of cases and The statistical methods used in the validation study are fully their percentages, whereas quantitative variables are expressed detailed in Multimedia Appendix 7. To estimate the interrater as mean and SD. Ordinal and quantitative variables not showing agreement measures, an agreement >75% of the expert panel normal distribution are expressed as the median and quartiles. was considered adequate [53]. The one-sample proportion test McNemar test was used on paired nominal data to determine with continuity correction was applied. Association was whether the row and column marginal frequencies were equal. measured by the phi coefficient (values range from 1 to +1). Phi The level of significance was <5% (alpha<.05), bilateral values above 0.7 are interpreted as showing a very strong approximation. All analyses were performed using the SPSS association, from 0.4 to 0.69 are interpreted as strong, from 0.3 https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 8 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al to 0.39 are interpreted as moderate, from 0.2 to 0.29 are useful (22/31, 71%) or very useful (4/31, 13%). One-third of interpreted as weak, and from <0.19 to <0.001 are interpreted the patients (9/31, 29%) reported that they needed personal as showing no association [53,54]. Agreement was assessed by assistance to get started with using the mHeart platform. the kappa coefficient (values range from 1 to +1). Kappa values >0.75 are interpreted as strong agreement, from 0.4 to 0.75 Polypharmacy and Determinants of Medication indicate moderate agreement, and <0.40 indicate poor agreement Nonadherence [53,55]. In general, values of reliability coefficients >0.80 Polypharmacy was common; the mean total medication count indicate excellent agreement [56]. was 13 (SD 4; range 7-18), exceeding 14 drugs per day in 36% (11/31) of patients. Patients reported a mean number of 6 (SD Sample Size 4) adverse effects. As many as 61% (19/31) of them reported In this finite population of early stage HTxR, we used a 5 being self-reliant for medication management. subject-to-variable ratio rule [57]. Therefore, a sample size greater than or equal to 25 participants for a total of 5 items Medication-related inconvenience was moderate to high (>6 of (1-item Haynes-Sackett and 4-item Morisky-Green-Levine 10) in 25% (8/31) of HTxR. As many as 74% (23/31) of them questionnaire) was considered the minimum sample required. believed they were taking excessive medication. The danger of sometimes not taking immunosuppressive drugs was understood To assess validity, reliability (equivalent forms method), by 42% (13/31) of recipients. Furthermore, 32% (10/31) of responsiveness, interpretability, and burden, we included the recipients were unaware of the consequences of completely entire sample in the analysis. For the test-retest reproducibility abandoning antirejection therapy. More details are provided in study, we included HTxR who remained stable for 7 days [31]. Multimedia Appendix 9. Stability was defined as the absence of need for medication changes or health center consultation and the absence of any Validity Measures symptoms different from those present at the last clinical Content Validity evaluation. Regarding the adequate representability and relevance of the ePROMs to be included in the mHeart system, the Results Haynes-Sackett and the Morisky-Green-Levine questionnaires Participant Characteristics showed excellent agreement (>85%), whereas the SMAQ showed poor agreement (<75%; Table 3). A total of 31 early stage HTxR were included (age: mean 54 years, SD 12 years) and analyzed, and no attrition was observed The suitability of the medication difficulties to support its (Figure 1). In all, 71% (22/31) of participants were men. The addition to the Haynes-Sackett electronic version was excellent mean follow-up was 2.3 (SD 0.9) months. The mean time (>80%). Item agreement is detailed in Table 4. between HTx and the study was 1.2 years (SD 0.8 years). The The overall agreement between the ePROMs and the on-site patients’ demographic and clinical characteristics are detailed PROMs was strong for the Haynes-Sackett (κ=0.826, P<.001) in Multimedia Appendix 8. and for the Morisky-Green-Levine (κ=1, P<.001) questionnaires. At baseline, 71% (22/31) of patients used technologies Item agreement is detailed in Table 5. frequently. Most of the patients reported that mHeart could be Table 3. Expert panel interrater agreement on the most suitable questionnaires to measure medication adherence using the mHeart platform, measured by the group consensus method. Round & adherence electronic patient-reported outcome Agreementa, n (%) P valueb Inclusion in mHeart measure Round 1 Haynes-Sackett 13 (93) .11 N/Ac Morisky-Green-Levine 12 (86) .27 N/A SMAQd 10 (71) .50 N/A Round 2 Haynes-Sackett 14 (100) .03 Included Morisky-Green-Levine 13 (93) .11 Included SMAQ 6 (43) .99 Nonincluded a Percentages of agreement. An agreement >75% of the expert panel was considered adequate. b P value was one-sided to test whether P was greater than .75 (75%). c N/A: not applicable. d SMAQ: Simplified Medication Adherence Questionnaire validated in Spanish transplant population. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 9 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Table 4. Expert panel interrater agreement on several criteria for the six reasons for medication nonadherence Haynes-Sackett electronic patient-reported outcome measure, measured by the nominal group consensus method. Reasons for medication nonadherence Intuitivea, n (%) Easya, n Briefa, n Usefula, n Percentage of overall P valuec b (%) (%) (%) agreement , n (%) I sometimes forget to take my medication 14 (100) 14 (100) 14 (100) 14 (100) 14 (100) <.001 I lack information on medication and/or the dis- 14 (100) 13 (93) 14 (100) 14 (100) 13.8 (98) <.001 ease I feel demotivated about taking my medication 12 (86) 11 (79) 14 (100) 13 (93) 12.5 (89) .01 Because of side effects or fear of having them 13 (93) 13 (93) 14 (100) 13 (93) 13.3 (95) <.001 Because of complex regimens and/or inconvenient 13 (93) 8 (57) 14 (100) 14 (100) 12.3 (88) .02 regimens Because of other reasons 12 (86) 14 (100) 13 (93) 12 (86) 12.8 (91) .004 a Item criteria full description: true to the original in-clinic test, useful to evaluate medication adherence construct, intuitive, brief or fast to complete, and easy-to-understand language. b Percentages of agreement. An agreement >75% of the expert panel was considered adequate. c P value was one-sided to test whether P is greater than .75 (75%). Table 5. Expert panel agreement on item characteristics of electronic patient-reported outcome measures compared with on-site patient-reported outcome measures, measured by the nominal group consensus method. Patient-reported outcome measure item Truea Usefula Intuitivea Briefa Easya Kappa P value Kappa P value Kappa P value Kappa P value Kappa P value value value value value value Item 1 MGLb 1 <.001 1 <.001 1 <.001 1 <.001 1 <.001 Item 2 MGL 1 <.001 1 <.001 1 <.001 1 <.001 1 <.001 Item 3 MGL 1 <.001 1 <.001 1 <.001 1 <.001 1 <.001 Item 4 MGL 1 <.001 1 <.001 1 <.001 1 <.001 1 <.001 Item 1 HSc 1 <.001 0.6 <.01 0.4 .04 1 <.001 1 <.001 a Item characteristics’ full description: true to the original in-clinic test, useful to evaluate medication adherence construct, intuitive, brief or fast to complete, and easy-to-understand language. b MGL: Morisky-Green-Levine 4-item questionnaire. c HS: Haynes-Sackett questionnaire. Convergent and Discriminant Validity The correlation of adherence to medication domains of the PROMs compared with the SMAQ is shown in Table 6. Table 6. Convergent and discriminant validity assessed by the correlation of medication adherence patient-reported outcome measures with the Simplified Medication Adherence Questionnaire. Validity property Adherence to medication Electronic In clinic Interpretation PROMsa Phi coefficient P value Phi coefficient P value Convergent Morisky-Green-Levine ver- 0.6 <.001 0.9 <.001 Strong correlation; measures similar sus SMAQb adherence domains Divergent Haynes-Sackett versus 0.3 .12 0.4 .04 Weak correlation; measures differ- SMAQ ent adherence domains a PROMs: patient-reported outcome measures to assess medication adherence. b SMAQ: Simplified Medication Adherence Questionnaire validated in Spanish transplant population. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 10 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Reproducibility For the test-retest reliability method, all participants remained The equivalent forms reliability method showed a very strong stable between assessments. Low reproducibility was observed, association between the scores obtained using the ePROMs and whereas medication adherence improved during this interval on-site PROMs (phi>0.7, P<.001; Table 7). according to both types of ePROM (Table 8). Table 7. Reliability of medication adherence electronic patient-reported outcome measures compared with on-site patient-reported outcome measures using the equivalent forms reliability method. Adherence to medication PROMsa Phi coefficient P value HSb overall 0.8 <.001 MGLc overall 0.7 <.001 MGL item 1 0.7 <.001 MGL item 2 0.7 <.001 MGL item 3 0.6 <.001 MGL item 4 1 <.001 a PROMs: patient-reported outcome measures to assess medication adherence. b HS: Haynes-Sackett questionnaire. c MGL: Morisky-Green-Levine 4-item questionnaire. Table 8. Test-retest reliability method to measure stability of medication adherence electronic patient-reported outcome measure scores over time. ePROMsa Assessment 2 Assessment 3 Kappa P value Interpretation value Adherent, n (%) Nonadherent, n (%) Adherent, n (%) Nonadherent, n (%) b 29 (94) 2 (7) 31 (100) 0 (0) 0.6 .002 Moderate stability HS MGLc 28 (90) 3 (10) 30 (97) 1 (3) 0.3 .11 Poor stability a ePROMs: electronic patient-reported outcome measures to assess medication adherence. b HS: Haynes-Sackett questionnaire. c MGL: Morisky-Green-Levine 4-item questionnaire. not at assessment 2. Almost all the patients were adherent Responsiveness or Sensitivity to Change according to the ePROMs at assessment 3. The baseline overall According to the change in medication adherence over time, in-clinic medication nonadherence rate was 32% (10/31), as similar rates were obtained in assessment 2 between ePROMs measured by the Morisky-Green-Levine PROMs. According and PROMs (Table 9). Details for each item are provided in to the SMAQ, 39% (12/31) of HTxR were nonadherent to Multimedia Appendix 10. immunosuppressive treatment. The theory-based multifaceted intervention program showed significant (P<.05) improvements Interpretability in medication nonadherence, ranging from 16% to 26%, The ePROM scores showed a nonsignificant underestimation depending on the questionnaire used (Table 9). (P>.05) of medication nonadherence rates at assessment 1 but https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 11 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Table 9. Medication adherence rates and improvement between study assessments. Measure Adherence to medication ratesa P value Assessment 1b Assessment 2b Assessment 3b Adherent, n Nonadherent, n (%) Adherent, n Nonadherent, n (%) Adherent, n Nonadherent, n (%) A1 vs A1 vs (%) (%) (%) A2 A3 ePROMsc HSd 25 (81) 6 (19) 29 (94) 2 (7) 31 (100) 0 (0) .10 .01 MGLe 25 (81) 6 (19) 28 (90) 3 (10) 30 (97) 1 (3) .26 .06 In-clinic PROMs HS 22 (71) 9 (29) 28 (90) 3 (10) N/Af N/A .03 N/A MGL 21 (68) 10 (32) 27 (87) 4 (13) N/A N/A .06 N/A SMAQg 19 (61) 12 (39) 27 (87) 4 (13) N/A N/A .005 N/A a Medication adherence in the implementation phase is expressed as a binary variable: adherent or nonadherent. The Haynes-Sackett and Morisky-Green-Levine questionnaires measures adherence to overall medication. The Simplified Medication Adherence Questionnaire validated in the Spanish transplant population measures adherence to immunosuppression. b The behavior-based interventional program established by the pharmacist was performed between assessments 1 and 2 (1 month at least). There was a 7-day gap between assessments 2 and 3 to allow the reproducibility test retest study without provider interactions. Only the electronic questionnaires were administered in Assessment 3. c ePROM: electronic patient-reported outcome measures to assess medication adherence. d HS: Haynes-Sackett. e MGL: Morisky-Green-Levine. f N/A: not applicable. g SMAQ: Simplified Medication Adherence Questionnaire validated in the Spanish transplant population. Improvements were implemented based on patient feedback, Burden for example, (1) to avoid patient recall bias, the order of the Regarding the criteria of respondent burden, 81% (25/31) of ePROM items was designed to automatically change whenever patients reported spending 1 to 2 min while completing the the test is completed; (2) patients could graphically consult any ePROMs, whereas the mean time for in-clinic PROMs was 6 values they recorded in mHeart (eg, blood pressure); (3) pop-up min (SD 2 min; range 3-9). All patients were able to learn the alerts were established to let patients know that a new text basic digital competencies needed to complete the ePROMs. message from the provider had arrived; and (4) diverse actions No missing values were found using the two methods. were implemented to decrease telephone use by patients to Regarding administrative burden, the total mean time spent per inquire about the compatibility of new therapies: the usability day by the pharmacist on mHeart was 33 min (SD 6 min; range of the mHeart function to inquire about new therapies was 21-44). This time allowed follow-up of all the patients. The improved, and text messages were sent to the patients, on-site PROMs required an office to be available and an average explaining how to use this function. of 45 min for each individual assessment. Both methods required The details of each survey item score, patient suggestions, and the professional to be trained in motivational interviewing, the subsequent improvements are provided in Multimedia medication management, and transplant basics. Appendix 11. Patient Satisfaction and Usability Survey Discussion The completion rate was 93% (29/31 patients). The reasons for no response to the survey are detailed in Figure 1. HTxR Principal Findings reported no inconvenience because of the mHeart intervention In this study, the first challenge was to validate mHeart ePROMs approach employed. The mean ePROM appropriateness score to detect early stage HTxR at risk for medication nonadherence was 8 (SD 2; score range: 0-10). The mean score for overall in the home setting. With this aim in mind, the COSMIN [30] satisfaction with the mHeart approach was 9 (SD 2; score range: and SAC-MOS [31] standards provided a solid framework to 0-10). All 100% (29/29) of the patients would recommend the support the quality of the assessed ePROM psychometric mHeart platform to other recipients. Regarding patient variables. As the mHeart ePROMs meet the minimum standards suggestions for improving the platform, 24% (7/29) of HTxR set by the ISOQOL [11], they can be used in clinical practice made eight suggestions and 76% (22/29) of them responded, and comparative effectiveness research. “No, I like it just as it is.” The excellent agreement observed between ePROMs and on-site PROMs confirmed that the mHeart electronic approach was as https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 12 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al effective as the traditional on-site method in identifying personalized communication, support from professionals, and medication nonadherence. The ePROMs used in mHeart showed self-empowerment, which were the most relevant criteria used multiple advantages over on-site PROMS, such as eliminating to design the mHeart intervention program. potential professional interpretation of ambiguous responses that could affect medication adherence rates. Furthermore, the Limitations electronic approach required fewer in-clinic facilities than the Our study includes a limited but representative sample traditional method of assessing medication nonadherence. The comprising 86% of all early stage HTxR in our center. This integration of the mHeart data with the HIS also reduced the characteristic is common in transplant population, as the time required to record ePROM responses in patients’ medical prevalence is limited [61]. We did not enroll chronic-stage records. These advantages reduced burden and enabled the recipients for the following reasons: (1) early posttransplant pharmacists to focus on clinical tasks. This is clinically medication nonadherence is a high-risk behavior with a huge significant, as a pharmacist intervention is associated with better impact on survival [1], (2) we wanted to avoid wide use of evidence-based therapies, reducing medication errors heterogeneity in chronic-stage providers and treatments, and and emergency department visits while increasing patient (3) we wanted to avoid chronic-stage recipients having to travel satisfaction [58]. to the clinic for the study. In addition, although early stage recipients are typically better adherers [3,62], this did not Regarding the baseline medication nonadherence values, the prevent us from observing an effect in the highest risk period percentage of HTxR nonadherent to overall medication in this after transplant. study was worrisome according to the Morisky-Green-Levine PROMs (32%) but similar to that observed in another series The interval between medication nonadherence assessments (33%) [59]. According to the SMAQ, the percentage (39%) of may have led to recall bias. Although this bias could have medication nonadherence to immunosuppressive treatment was influenced the electronic score, this limitation is intrinsically slightly higher than the overall percentage reported in the HTx related to the validation methodology to ensure that the population (34%) [7] and considerably higher than that reported electronic and traditional methods are performed in similar in a meta-analysis of the solid organ transplant population (25%) conditions and in patients with similar psychological and [4,25]. These results suggest that the ePROMs used in our study functional status. Moreover, the short study periods used were have a synergistic effect in identifying nonadherent recipients. methodologically grounded according to the main study aim of However, they also highlight the need for intervention programs validating the ePROMs. In sensitivity to change measures, a to improve medication nonadherence, as almost half of the 1-month interval is considered an adequate interval to measure HTxR were unaware of the consequences of medication the validity of an indirect smartphone health measure [51,52]. nonadherence. Nevertheless, comparisons among medication Moreover, fortnightly assessments are sufficient to identify nonadherence rates in the field of transplantation should be additional medication nonadherence in the transplant population interpreted with caution as studies use different populations and [46]. In reproducibility measures, intervals of 1 to 2 weeks are methodologies [4]. common [63]. Therefore, a 7-day interval was selected to minimize the effect of possible confounding variables [31] The exploratory intervention program established to deal with related to the multifaceted factors affecting posttransplant this problem showed promising results. Immunosuppressive medication nonadherence [10,46]. treatment adherence rates significantly improved in one-third of the recipients, according to the SMAQ. This figure is higher Long-Term Workflow and Clinical Applications than that reported by most studies showing low or medium Adherence monitoring is recognized as a standard quality effect sizes of around 10% to 20% in medication adherence practice in transplant centers [7]. In the past few decades, there improvement [10,45]. In contrast to these studies, our program has been a growing interest in improving the screening was designed to deliver personalized, internet-based multilevel opportunities of medication nonadherence without increasing interventions based on behavioral theories [7,21,22]. Indeed, the in-clinic burden. The quality of the ePROM psychometric human support and tailored interventions have been shown to variables and the patient satisfaction reported in this study be a requisite to improve medication nonadherence rates support the scalability of the mHeart ePROM for use in clinical throughout eHealth [10,18]. Moreover, our exploratory study practice and research [11]. The results obtained indicate that meets 72% of the TCS criteria (ie, items 1-11), indicating that the electronic self-reporting approach provides a highly sensitive the interventional study design complies with the theoretical medication nonadherence measure in the transplant population basis of the intervention [32]. This is important as interventions to complement traditional and more time-consuming methods, meeting a minimum of 60% of the TCS criteria have been found such as blood tests or medication refills [25,46]. to be highly effective [60]. Furthermore, given that medication nonadherence behavior in Nonelectronic theory-based interventions have been considered the transplant population is influenced by several factors [7], highly effective when improvement was >20% [45]. Therefore, optimal daily adherence is a real challenge for recipients [64]. as the improvement in medication nonadherence in our Consequently, feasible holistic strategies are needed to help exploratory study was higher (30%), the strategies applied recipients reduce the negative impact of medication proved to be synergistic and to enhance the effectiveness of the nonadherence on health outcomes [20]. The exploratory results program [21,32]. Equally important, patients adhered well to of this behavioral theory–based intervention on medication the study protocol and provided excellent feedback. Among the nonadherence rates are encouraging. Future studies will benefits of the mHeart approach, patients highlighted https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 13 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al determine the intervention’s effectiveness on clinical outcomes HTx population. ePROMs demonstrated their potential to [13]. With this aim in mind, the EMERGE [28], TCS [32], and overcome the limitations of traditional on-site methods. The CONSORT-EHEALTH reporting criteria [17] standards were ePROMs’ quality properties supported their widespread use in followed to support the scalability of the intervention research and clinical practice. The theory-based intervention methodology used in larger research. For now, the feasibility program showed an encouraging improvement in medication and effectiveness found in this study encourage following this adherence rates, with excellent patient satisfaction and usability path to curb the widespread problem of medication scores. Therefore, the mHeart program resulted in significant nonadherence. benefits in estimating medication nonadherence in the HTx population and showed promise in guiding professionals’ Conclusions interventions with the potential to optimize HTx outcomes. The electronic method implemented in the mHeart medical device successfully identified medication nonadherence in the Acknowledgments This study was supported by the General Pharmaceutical Council of Barcelona (a scientific research grant, 2016) and Astellas Pharma SL (a contribution with the institution). These contributions aimed to reinforce the study resources and were partly used to fund MG’s time on the study and the technical development of the mHeart platform. This study would not have been possible without the patients’ contributions and willingness to participate in the study and use the mHeart platform. The authors wish to thank the following experts involved in the mHeart implementation: J Caparrus and N Alberich for help with data protection and confidentiality policy; V Robert and the Information Systems Department for conducting the platform interoperability; O Pozas and F Carbajo for their support from the Legal department; D Gallardo for expert advice on Intellectual Property; and M Ors, B Monsalve, and N Ichaso for their expertise on transference and innovation from the Hospital de la Santa Creu i Sant Pau, Barcelona, Spain (HSCSP) Research Institute. The authors would also like to thank the multidisciplinary team of the HSCSP Heart Failure and Heart Transplant Department for their contribution to the study (D Gil, E Galvez, S Ros, M Pirla, N Mesado, E Sole, J Alvarez, and M Ribas). Authors' Contributions All the authors made substantial contributions to specific areas, revised the work for important intellectual content, approved the final version to be published, and agreed to be accountable for all aspects of the work, ensuring that questions related to the accuracy or integrity of any part of the work were appropriately investigated and resolved. Conflicts of Interest The technological development of the mHeart platform lasted from March 15, 2015, to June 2, 2016, and was carried out by a private Spanish firm specializing in health care system apps (Trilema Salud). Most of the authors of this study participated as consultants, and MG, the transplant pharmacist, was assigned the role of the Project Coordinator. Neither the abovementioned funding entities nor the technical developers played any role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Multimedia Appendix 1 Questionnaire designed for completion in face-to-face interviews: sociodemographic, clinical, and technology acceptance among the heart transplant recipients included in the Val-mHeart study. [PDF File (Adobe PDF File), 52 KB-Multimedia Appendix 1] Multimedia Appendix 2 Questionnaire designed to be administered in face-to-face interviews: patient-reported outcomes related to the treatment regimen. [PDF File (Adobe PDF File), 46 KB-Multimedia Appendix 2] Multimedia Appendix 3 Behavior change techniques selected for use during the behavior-based intervention program and workflow adapted for delivery using the mHeart platform, and a description of the mHeart-based treatment designed to improve medication adherence in the Val-mHeart study. [PDF File (Adobe PDF File), 93 KB-Multimedia Appendix 3] Multimedia Appendix 4 Sample video of the mHeart mobile app. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 14 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al [MP4 File (MP4 Video), 90609 KB-Multimedia Appendix 4] Multimedia Appendix 5 Electronic version of the Haynes-Sackett questionnaire, including 6 additional responses by patients to improve provider understanding of their difficulties with medication adapted for use with the mHeart platform. [PDF File (Adobe PDF File), 33 KB-Multimedia Appendix 5] Multimedia Appendix 6 Patient Satisfaction and Usability Survey Original Spanish Version (the results and the items' translation to English language are provided in Multimedia Appendix 11). [PDF File (Adobe PDF File), 270 KB-Multimedia Appendix 6] Multimedia Appendix 7 Methodology used to measure validity properties of the electronic patient reported outcome measures to assess medication adherence in the at-home setting using the mHeart platform in heart transplant recipients (The Val-mHeart Study). [PDF File (Adobe PDF File), 80 KB-Multimedia Appendix 7] Multimedia Appendix 8 Demographic and clinical characteristics of the early-stage heart transplant recipients included in the Val-mHeart Study. [PDF File (Adobe PDF File), 44 KB-Multimedia Appendix 8] Multimedia Appendix 9 Heart transplant recipients’ therapeutic characteristics and treatment-related patient-reported outcomes. [PDF File (Adobe PDF File), 66 KB-Multimedia Appendix 9] Multimedia Appendix 10 Nonadherence rates in early-stage heart transplant recipients listed by item in the study period. [PDF File (Adobe PDF File), 38 KB-Multimedia Appendix 10] Multimedia Appendix 11 Results of the Web-based survey on usability and satisfaction with the mHeart platform and intervention implemented during the study period in heart transplant recipients. Table 11A shows the categorical variables and Table 11B shows the quantitative variables. [PDF File (Adobe PDF File), 96 KB-Multimedia Appendix 11] References 1. Dobbels F, de Geest S, van Cleemput J, Droogne W, Vanhaecke J. Effect of late medication non-compliance on outcome after heart transplantation: a 5-year follow-up. J Heart Lung Transplant 2004 Nov;23(11):1245-1251. [doi: 10.1016/j.healun.2003.09.016] [Medline: 15539122] 2. Brocks Y, Zittermann A, Grisse D, Schmid-Ott G, Stock-Gießendanner S, Schulz U, et al. Adherence of heart transplant recipients to prescribed medication and recommended lifestyle habits. Prog Transplant 2017 Jun;27(2):160-166. [doi: 10.1177/1526924817699959] [Medline: 28617170] 3. Korb-Savoldelli V, Sabatier B, Gillaizeau F, Guillemain R, Prognon P, Bégué D, et al. Non-adherence with drug treatment after heart or lung transplantation in adults: a systematic review. Patient Educ Couns 2010 Nov;81(2):148-154. [doi: 10.1016/j.pec.2010.04.013] [Medline: 20627643] 4. Dew MA, DiMartini AF, Dabbs AV, Myaskovsky L, Steel J, Unruh M, et al. Rates and risk factors for nonadherence to the medical regimen after adult solid organ transplantation. Transplantation 2007 Apr 15;83(7):858-873. [doi: 10.1097/01.tp.0000258599.65257.a6] [Medline: 17460556] 5. Hansen R, Seifeldin R, Noe L. Medication adherence in chronic disease: issues in posttransplant immunosuppression. Transplant Proc 2007 Jun;39(5):1287-1300. [doi: 10.1016/j.transproceed.2007.02.074] [Medline: 17580126] 6. Hugon A, Roustit M, Lehmann A, Saint-Raymond C, Borrel E, Hilleret M, et al. Influence of intention to adhere, beliefs and satisfaction about medicines on adherence in solid organ transplant recipients. Transplantation 2014 Jul 27;98(2):222-228. [doi: 10.1097/TP.0000000000000221] [Medline: 24926826] https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 15 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al 7. Denhaerynck K, Berben L, Dobbels F, Russell CL, Crespo-Leiro MG, Poncelet AJ, BRIGHT study team. Multilevel factors are associated with immunosuppressant nonadherence in heart transplant recipients: the international BRIGHT study. Am J Transplant 2018 Jun;18(6):1447-1460 [FREE Full text] [doi: 10.1111/ajt.14611] [Medline: 29205855] 8. Gomis-Pastor M, Mingell ER, Perez SM, Loidi VB, Lopez LL, Bassons AD, et al. Multimorbidity and medication complexity: new challenges in heart transplantation. Clin Transplant 2019 Oct;33(10):e13682. [doi: 10.1111/ctr.13682] [Medline: 31368585] 9. Senft Y, Kirsch M, Denhaerynck K, Dobbels F, Helmy R, Russell CL, BRIGHT study team. Practice patterns to improve pre and post-transplant medication adherence in heart transplant centres: a secondary data analysis of the international BRIGHT study. Eur J Cardiovasc Nurs 2017 Dec 1:1474515117747577. [doi: 10.1177/1474515117747577] [Medline: 29231747] 10. Kini V, Ho PM. Interventions to improve medication adherence: a review. J Am Med Assoc 2018 Dec 18;320(23):2461-2473. [doi: 10.1001/jama.2018.19271] [Medline: 30561486] 11. Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res 2013 Oct;22(8):1889-1905. [doi: 10.1007/s11136-012-0344-y] [Medline: 23288613] 12. Sarzynski E, Decker B, Thul A, Weismantel D, Melaragni R, Cholakis E, et al. Beta testing a novel smartphone application to improve medication adherence. Telemed J E Health 2017 Apr;23(4):339-348. [doi: 10.1089/tmj.2016.0100] [Medline: 27564971] 13. Ritterband LM, Andersson G, Christensen HM, Carlbring P, Cuijpers P. Directions for the International Society for Research on Internet Interventions (ISRII). J Med Internet Res 2006 Sep 29;8(3):e23. [doi: 10.2196/jmir.8.3.e23] [Medline: 17032639] 14. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res 2006 Jun 23;8(2):e10. [doi: 10.2196/jmir.8.2.e10] [Medline: 16867965] 15. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Feb;15(Pt 1):1-39 [FREE Full text] [doi: 10.1348/135910709X466063] [Medline: 19646331] 16. Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [doi: 10.1371/journal.pmed.1001362] [Medline: 23349621] 17. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [doi: 10.2196/jmir.1923] [Medline: 22209829] 18. Lancaster K, Abuzour A, Khaira M, Mathers A, Chan A, Bui V, et al. The use and effects of electronic health tools for patient self-monitoring and reporting of outcomes following medication use: systematic review. J Med Internet Res 2018 Dec 18;20(12):e294 [FREE Full text] [doi: 10.2196/jmir.9284] [Medline: 30563822] 19. Mira JJ, Navarro I, Botella F, Borrás F, Nuño-Solinís R, Orozco D, et al. A Spanish pillbox app for elderly patients taking multiple medications: randomized controlled trial. J Med Internet Res 2014 Apr 4;16(4):e99 [FREE Full text] [doi: 10.2196/jmir.3269] [Medline: 24705022] 20. Fleming JN, Taber DJ, McElligott J, McGillicuddy JW, Treiber F. Mobile health in solid organ transplant: the time is now. Am J Transplant 2017 Sep;17(9):2263-2276 [FREE Full text] [doi: 10.1111/ajt.14225] [Medline: 28188681] 21. Davis R, Campbell R, Hildon Z, Hobbs L, Michie S. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev 2015;9(3):323-344 [FREE Full text] [doi: 10.1080/17437199.2014.941722] [Medline: 25104107] 22. Conn VS, Enriquez M, Ruppar TM, Chan KC. Meta-analyses of theory use in medication adherence intervention research. Am J Health Behav 2016 Mar;40(2):155-171 [FREE Full text] [doi: 10.5993/AJHB.40.2.1] [Medline: 26931748] 23. Salvo MC, Cannon-Breland ML. Motivational interviewing for medication adherence. J Am Pharm Assoc (2003) 2015;55(4):e354-61; quiz e362. [doi: 10.1331/JAPhA.2015.15532] [Medline: 26161493] 24. Miller WR, Rose GS. Toward a theory of motivational interviewing. Am Psychol 2009 Sep;64(6):527-537 [FREE Full text] [doi: 10.1037/a0016830] [Medline: 19739882] 25. Fine RN, Becker Y, de Geest S, Eisen H, Ettenger R, Evans R, et al. Nonadherence consensus conference summary report. Am J Transplant 2009 Jan;9(1):35-41 [FREE Full text] [doi: 10.1111/j.1600-6143.2008.02495.x] [Medline: 19133930] 26. Vitinius F, Ziemke M, Albert W. Adherence with immunosuppression in heart transplant recipients. Curr Opin Organ Transplant 2015 Apr;20(2):193-197. [doi: 10.1097/MOT.0000000000000166] [Medline: 25856181] 27. Froud R, Fawkes C, Foss J, Underwood M, Carnes D. Responsiveness, reliability, and minimally important and minimal detectable changes of 3 electronic patient-reported outcome measures for low back pain: validation study. J Med Internet Res 2018 Oct 24;20(10):e272 [FREE Full text] [doi: 10.2196/jmir.9828] [Medline: 30355556] 28. de Geest S, Zullig LL, Dunbar-Jacob J, Helmy R, Hughes DA, Wilson IB, et al. ESPACOMP Medication Adherence Reporting Guideline (EMERGE). Ann Intern Med 2018 Jul 3;169(1):30-35. [doi: 10.7326/M18-0543] [Medline: 29946690] https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 16 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al 29. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010 May;19(4):539-549 [FREE Full text] [doi: 10.1007/s11136-010-9606-8] [Medline: 20169472] 30. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010 Jul;63(7):737-745. [doi: 10.1016/j.jclinepi.2010.02.006] [Medline: 20494804] 31. Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002 May;11(3):193-205. [doi: 10.1023/a:1015291021312] [Medline: 12074258] 32. Michie S, Prestwich A. Are interventions theory-based? Development of a theory coding scheme. Health Psychol 2010 Jan;29(1):1-8. [doi: 10.1037/a0016939] [Medline: 20063930] 33. Eysenbach G. Improving the quality of web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [doi: 10.2196/jmir.6.3.e34] [Medline: 15471760] 34. Vrijens B, de Geest S, Hughes DA, Przemyslaw K, Demonceau J, Ruppar T, et al. A new taxonomy for describing and defining adherence to medications. Br J Clin Pharmacol 2012 May;73(5):691-705 [FREE Full text] [doi: 10.1111/j.1365-2125.2012.04167.x] [Medline: 22486599] 35. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008 May;27(3):379-387. [doi: 10.1037/0278-6133.27.3.379] [Medline: 18624603] 36. Baumel A, Kane JM. Examining predictors of real-world user engagement with self-guided eHealth interventions: analysis of mobile apps and websites using a novel dataset. J Med Internet Res 2018 Dec 14;20(12):e11491 [FREE Full text] [doi: 10.2196/11491] [Medline: 30552077] 37. Muench F, Baumel A. More than a text message: dismantling digital triggers to curate behavior change in patient-centered health interventions. J Med Internet Res 2017 May 26;19(5):e147 [FREE Full text] [doi: 10.2196/jmir.7463] [Medline: 28550001] 38. Trilema Salud. mHeart Website URL: https://salud.trilema.es/ [accessed 2019-07-20] 39. Socioemprende SL. Google Play. mHeart URL: https://play.google.com/store/apps/details?id=nabelia.mheart&gl=ES [accessed 2019-07-20] 40. Socioemprende. App Store - Apple. mHeart URL: https://apps.apple.com/us/app/mheart/id1449696988 [accessed 2019-07-20] 41. Gomis-Pastor M, Mangues M, Pellicer V, Ors M. Mendeley Data. 2019. mHeart - mHealthCare Platform Adapted to the Heart Transplant Population - Technical Specifications and Software Source Code URL: https://data.mendeley.com/datasets/ yf2dgcfmmh/4 [accessed 2020-01-10] 42. Sackett DL, Snow JC. The magnitude of compliance and non-compliance. In: Haynes RB, Taylor DW, Sackett DL, editors. Compliance in Health Care. Baltimore: Johns Hopkins University Press; 1979:11-22. 43. Sackett DL, Haynes RB, Guyat GH. Ayudar a los pacientes a cumplir los tratamientos. In: Haynes RB, Sackett DL, Guyatt GH, Tugwell P, editors. Epidemiol clínica, Cienc básica para la Med clínica 2a ed. Madrid, Spain: Editorial Médica Panamericana SA; 1994:249-278. 44. Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care 1986 Jan;24(1):67-74. [doi: 10.1097/00005650-198601000-00007] [Medline: 3945130] 45. Dobbels F, de Bleser L, Berben L, Kristanto P, Dupont L, Nevens F, et al. Efficacy of a medication adherence enhancing intervention in transplantation: The MAESTRO-Tx trial. J Heart Lung Transplant 2017 May;36(5):499-508. [doi: 10.1016/j.healun.2017.01.007] [Medline: 28162931] 46. Gustavsen MT, Midtvedt K, Lønning K, Jacobsen T, Reisaeter AV, de Geest S, et al. Evaluation of tools for annual capture of adherence to immunosuppressive medications after renal transplantation - a single-centre open prospective trial. Transpl Int 2019 Jun;32(6):614-625. [doi: 10.1111/tri.13412] [Medline: 30770608] 47. Val JA, Amorós BG, Martínez PF. Estudio descriptivo del cumplimiento del tratamiento farmacológico antihipertensivo y validación del test de Morisky y Green. In: Estudio descriptivo del cumplimiento del tratamiento farmacológico antihipertensivo y validación del test de Morisky y Green. Spain: Atención primaria; 1992:767-770. 48. Orozco-Beltrán D, Carratalá-Munuera C, Gil-Guillén V. Improving treatment adherence: one of the most effective ways of increasing patient survival in secondary prevention. Rev Esp Cardiol Supl 2015;15(5):12-18. [doi: 10.1016/s1131-3587(16)30004-8] 49. Conthe P, Tejerina F. Heart failure treatment adherence and quality of life. Rev Esp Cardiol Supl 2007;7(6):57F-66F. [doi: 10.1016/s1131-3587(07)75256-1] 50. Suárez FJ, Plumed JS, Valentín MA, Palomo PP, Cepeda MA, Aguiar DL, Grupo de Estudio Vatren. Validation on the simplified medication adherence questionnaire (SMAQ) in renal transplant patients on tacrolimus. Nefrologia 2011;31(6):690-696 [FREE Full text] [doi: 10.3265/Nefrologia.pre2011.Aug.10973] [Medline: 22130285] 51. Conthe P, Contreras EM, Pérez AA, García BB, Martín MN, Jurado MG, et al. Treatment compliance in chronic illness: current situation and future perspectives. Rev Clin Esp 2014;214(6):336-344. [doi: 10.1016/j.rce.2014.03.008] [Medline: 24816042] https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 17 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al 52. Pan YC, Lin HH, Chiu YC, Lin SH, Lin YH. Temporal stability of smartphone use data: determining fundamental time unit and independent cycle. JMIR Mhealth Uhealth 2019 Mar 26;7(3):e12171 [FREE Full text] [doi: 10.2196/12171] [Medline: 30912751] 53. Viladrich C, Doval E. Medición: Fiabilidad y Validez. Bellaterra, Barcelona: Laboratori d'estadística Aplicada i Modelització, Universitat Autónoma de Barcelona (UAB); 2011. 54. Akoglu H. User's guide to correlation coefficients. Turk J Emerg Med 2018 Sep;18(3):91-93 [FREE Full text] [doi: 10.1016/j.tjem.2018.08.001] [Medline: 30191186] 55. Fleiss JL, Levin B, Myunghee CP. Statistical Methods for Rates & Proportions. Third Edition. Hoboken: John Wiley Sons, Inc; 1981. 56. Hogan TP, Benjamin AM, Brezinski KL. Reliability methods: a note on the frequency of use of various types. Educ Psychol Meas 2000;60(4):523-531. [doi: 10.1177/00131640021970691] 57. Gorsuch RL. Factor Analysis. Second Edition. Hillsdale: NJ Lawrence Erlbaum Assoc; 1983. 58. Milfred-LaForest SK, Chow SL, DiDomenico RJ, Dracup K, Ensor CR, Gattis-Stough W, et al. Clinical pharmacy services in heart failure: an opinion paper from the Heart Failure Society of America and American College of Clinical Pharmacy Cardiology Practice and Research Network. Pharmacotherapy 2013 May;33(5):529-548. [doi: 10.1002/phar.1295] [Medline: 23649813] 59. Pérez AB, Suárez AL, Rodríguez JR, Márquez JM, Gallé EL. Medication adherence in patients who undergo cardiac transplantation. Transplant Proc 2013;45(10):3662-3664. [doi: 10.1016/j.transproceed.2013.11.006] [Medline: 24314989] 60. Lycett HJ, Raebel EM, Wildman EK, Guitart J, Kenny T, Sherlock J, et al. Theory-based digital interventions to improve asthma self-management outcomes: systematic review. J Med Internet Res 2018 Dec 12;20(12):e293 [FREE Full text] [doi: 10.2196/jmir.9666] [Medline: 30541741] 61. de Bleser L, Matteson M, Dobbels F, Russell C, de Geest S. Interventions to improve medication-adherence after transplantation: a systematic review. Transpl Int 2009 Aug;22(8):780-797 [FREE Full text] [doi: 10.1111/j.1432-2277.2009.00881.x] [Medline: 19386076] 62. de Geest S, Burkhalter H, Bogert L, Berben L, Glass TR, Denhaerynck K, Psychosocial Interest Group, Swiss Transplant Cohort Study. Describing the evolution of medication nonadherence from pretransplant until 3 years post-transplant and determining pretransplant medication nonadherence as risk factor for post-transplant nonadherence to immunosuppressives: the Swiss Transplant Cohort Study. Transpl Int 2014 Jul;27(7):657-666 [FREE Full text] [doi: 10.1111/tri.12312] [Medline: 24628915] 63. Polit DF. Getting serious about test-retest reliability: a critique of retest research and some recommendations. Qual Life Res 2014 Aug;23(6):1713-1720. [doi: 10.1007/s11136-014-0632-9] [Medline: 24504622] 64. Vanhoof JM, Vandenberghe B, Geerts D, Philippaerts P, de Mazière P, Dabbs AD, PICASSO-Tx consortium. Shedding light on an unknown reality in solid organ transplant patients' self-management: a contextual inquiry study. Clin Transplant 2018 Aug;32(8):e13314. [doi: 10.1111/ctr.13314] [Medline: 29897621] Abbreviations CONSORT-EHEALTH: Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and OnLine TeleHealth COSMIN: COnsensus-based Standards for the selection of health Measurement Instruments EMERGE: European Society for Patient Adherence, COMpliance, and Persistence Medication Adherence Reporting Guideline ePROM: electronic patient-reported outcome measure HIS: hospital information system HSCSP: Hospital de la Santa Creu i Sant Pau, Barcelona, Spain HTx: heart transplant HTxR: heart transplant recipients ISOQOL: International Society for Quality of Life Research ISRII: International Society for Research on Internet Interventions MNA: medication nonadherence SAC-MOS: Scientific Advisory Committee of the Medical Outcomes Trust SMAQ: Simplified Medication Adherence Questionnaire TCS: Theory Coding Scheme https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 18 (page number not for citation purposes) XSL• FO RenderX JMIR MHEALTH AND UHEALTH Gomis-Pastor et al Edited by G Eysenbach; submitted 21.08.19; peer-reviewed by H Mehdizadeh, S Weinland, T Gal, R Leek, S Hamel, JB Park; comments to author 18.09.19; revised version received 18.10.19; accepted 16.12.19; published 04.02.20 Please cite as: Gomis-Pastor M, Roig E, Mirabet S, T De Pourcq J, Conejo I, Feliu A, Brossa V, Lopez L, Ferrero-Gregori A, Barata A, Mangues MA A Mobile App (mHeart) to Detect Medication Nonadherence in the Heart Transplant Population: Validation Study JMIR Mhealth Uhealth 2020;8(2):e15957 URL: https://mhealth.jmir.org/2020/2/e15957 doi: 10.2196/15957 PMID: 32014839 ©Mar Gomis-Pastor, Eulalia Roig, Sonia Mirabet, Jan T De Pourcq, Irene Conejo, Anna Feliu, Vicens Brossa, Laura Lopez, Andreu Ferrero-Gregori, Anna Barata, M Antonia Mangues. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 04.02.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included. https://mhealth.jmir.org/2020/2/e15957 JMIR Mhealth Uhealth 2020 | vol. 8 | iss. 2 | e15957 | p. 19 (page number not for citation purposes) XSL• FO RenderX