JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Original Paper Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People With Multiple Sclerosis: Nonrandomized Interventional Feasibility Study Luciana Midaglia1,2, MD; Patricia Mulero1, MD; Xavier Montalban1,3, MD, PhD; Jennifer Graves4, MAS, MD, PhD; Stephen L Hauser5, MD; Laura Julian6, PhD; Michael Baker7, MSc; Jan Schadrack7, MD; Christian Gossens7, MBA, PhD; Alf Scotland7, MSc; Florian Lipsmeier7, PhD; Johan van Beek7, PhD; Corrado Bernasconi7, MD, PhD; Shibeshih Belachew7, MD, PhD; Michael Lindemann7,8, MBA, PhD 1 Department of Neurology-Neuroimmunology, Multiple Sclerosis Centre of Catalonia, Vall d’Hebron University Hospital, Barcelona, Spain 2 Department of Medicine, Autonomous University of Barcelona, Barcelona, Spain 3 Division of Neurology, University of Toronto, Toronto, ON, Canada 4 Department of Neurology, University of California, San Diego, San Diego, CA, United States 5 Department of Neurology, University of California, San Francisco, San Francisco, CA, United States 6 Genentech Inc, South San Francisco, CA, United States 7 F Hoffmann–La Roche Ltd, Basel, Switzerland 8 Department of Economics, Baden-Wuerttemberg Cooperative State University, Loerrach, Germany Corresponding Author: Christian Gossens, MBA, PhD F Hoffmann–La Roche Ltd 124 Grenzacherstrasse Basel, Switzerland Phone: 41 61 687 5113 Fax: 41 61 691 9391 Email: christian.gossens@roche.com Related Article: This is a corrected version. See correction statement in: https://www.jmir.org/2019/10/e16287 Abstract Background: Current clinical assessments of people with multiple sclerosis are episodic and may miss critical features of functional fluctuations between visits. Objective: The goal of the research was to assess the feasibility of remote active testing and passive monitoring using smartphones and smartwatch technology in people with multiple sclerosis with respect to adherence and satisfaction with the FLOODLIGHT test battery. Methods: People with multiple sclerosis (aged 20 to 57 years; Expanded Disability Status Scale 0-5.5; n=76) and healthy controls (n=25) performed the FLOODLIGHT test battery, comprising active tests (daily, weekly, every two weeks, or on demand) and passive monitoring (sensor-based gait and mobility) for 24 weeks using a smartphone and smartwatch. The primary analysis assessed adherence (proportion of weeks with at least 3 days of completed testing and 4 hours per day passive monitoring) and questionnaire-based satisfaction. In-clinic assessments (clinical and magnetic resonance imaging) were performed. Results: People with multiple sclerosis showed 70% (16.68/24 weeks) adherence to active tests and 79% (18.89/24 weeks) to passive monitoring; satisfaction score was on average 73.7 out of 100. Neither adherence nor satisfaction was associated with specific population characteristics. Test-battery assessments had an at least acceptable impact on daily activities in over 80% (61/72) of people with multiple sclerosis. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 1 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Conclusions: People with multiple sclerosis were engaged and satisfied with the FLOODLIGHT test battery. FLOODLIGHT sensor-based measures may enable continuous assessment of multiple sclerosis disease in clinical trials and real-world settings. Trial Registration: ClinicalTrials.gov: NCT02952911; https://clinicaltrials.gov/ct2/show/NCT02952911 (J Med Internet Res 2019;21(8):e14863) doi: 10.2196/14863 KEYWORDS multiple sclerosis; patient adherence; patient satisfaction; smartphone; wearable electronic devices; mobile phone smartphone and smartwatch schedule of assessments and its Introduction impact on their daily activities using a patient satisfaction Disease progression throughout the clinical course of multiple questionnaire. Other objectives of the study, which will be sclerosis (MS) is measured using clinician-reported outcomes, addressed in subsequent publications, are to determine the most commonly the Expanded Disability Status Scale (EDSS) association between exploratory sensor-based outcomes derived [1], magnetic resonance imaging (MRI), and patient-reported from the respective components of the FLOODLIGHT test outcomes (PROs). Conventional assessment of the clinical battery and conventional MS clinical outcomes and explore course of MS relies on relapse-associated and periodic in-clinic whether the FLOODLIGHT test battery can differentiate visits. However, the current intermittently conducted between participants with and without MS. clinic-based outcome measures in MS have limitations in studying the insidiously subtle progression in MS and may fail Methods to comprehensively capture transient symptomatic and performance fluctuations that affect people with multiple Trial Design and Participants sclerosis. After providing written informed consent, people with multiple sclerosis and HCs, preferentially partners or cohabitants, were The ability of consumer wearable technology to measure evaluated for eligibility for enrollment in the FLOODLIGHT functional impairment associated with various neurological study. Eligibility criteria for people with multiple sclerosis disease symptoms through smartphone-based assessments is an included the ability to comply with the study protocol, age 18 important area of research [2-5]. Smartphones as vehicles for to 55 years, diagnosis of MS (2010 revised McDonald criteria, sensor-based technologies offer the potential for enhanced active treated or untreated) [14], EDSS score 0 to 5.5 (inclusive), and and passive real-time data capture that may fundamentally shift weight from 99 to 243 lbs (45 to 110 kg). An EDSS score of traditional paradigms of clinical monitoring [6-9]. A recent 5.5 as a maximum limit was meant to ensure any patient with large-scale study demonstrated that more than 95% of people relapsing or progressive MS would not have any significant with multiple sclerosis have access to a mobile device and most difficulty in participating in the proposed testing as per study use it routinely [10]. Recently published studies have described protocol. Further details on eligibility criteria are provided in the use of technologies in developing tools to assess people with Multimedia Appendix 1. multiple sclerosis [7,9,11-13]. In a recent study, it was reported that both healthy participants and people with multiple sclerosis This study was conducted at two sites in two countries with a were capable of completing daily tasks on a smartphone for 1 total of 76 people with multiple sclerosis and 25 HCs; 60 people year [7]. Collection of data on a variety of cognitive and motor with multiple sclerosis and 20 HCs were recruited from the tests via the smartphone may represent a feasible way to gather Multiple Sclerosis Centre of Catalonia, Vall d’Hebron highly granular data to accurately describe the MS disease University Hospital, Barcelona, Spain, and 16 people with course outside of the clinic [7]. multiple sclerosis and 5 HCs were recruited from the University of California, San Francisco, California. The FLOODLIGHT study [NCT02952911] was a prospective pilot study to assess the feasibility of remote measurements The protocol, informed consent forms, any information given using smartphones and smartwatches in people with multiple to the participants, and relevant supporting information were sclerosis and healthy controls (HCs). The smartwatch and reviewed and approved by the institutional review board/ethics smartphone contained apps that prompted the user to perform committee before the study was initiated. Confidentiality was various assessments and protocols, referred to as active tests. maintained by assigning each participant enrolled in the study The app also passively recorded sensor data during daily life, a unique identification number. referred to as passive monitoring. The novel smartphone- and Study Design smartwatch-based FLOODLIGHT active tests were developed to be self-administered by people with multiple sclerosis to The FLOODLIGHT study combines continuous sensor data capture MS symptoms, including hand motor function, gait and capture with smartphones and smartwatches and standard posture, mood, and cognitive impairment. clinical outcome measures. Eligible people with multiple sclerosis and HCs were enrolled in the study and assessed The primary objectives of the FLOODLIGHT study, which clinically at the enrollment visit, week 12, and termination visit began in November 2016, were to evaluate participant adherence (week 24). In addition, participants were asked to perform a set to smartphone- and smartwatch-based assessments and collect of daily active tests and contribute sensor data via passive feedback from people with multiple sclerosis and HCs on the http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 2 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al monitoring with smartphone and smartwatches over a period of 24 weeks (Figure 1). Figure 1. FLOODLIGHT study design. PRO: patient-reported outcome. "a" indicates that active tests were administered weekly or every two weeks (see next figure for schedule). "b" indicates that brain magnetic resonance imaging was performed in people with multiple sclerosis. a smartphone and smartwatch preconfigured so participants In-Clinic Assessments could only run the FLOODLIGHT software. A belt bag was At each scheduled in-clinic visit (enrollment, week 12, and also provided for participants to carry their smartphone in an week 24), the following reference clinical tests were performed anterior medial position. The smartphone and smartwatch pair for all participants: 9-Hole Peg Test (9HPT), oral version of contained preinstalled apps that prompted the user to perform Symbol Digit Modalities Test (SDMT) [15-17], Timed 25-Foot various assessments, referred to as active tests. The apps also Walk (T25FW) test, Berg Balance Scale (BBS) [18], Fatigue passively recorded sensor data, referred to as passive monitoring. Scale for Motor and Cognitive Functions (FSMC) [19], and At the enrollment visit, participants received training on the use Patient Health Questionnaire–9 (PHQ-9) [20]. For people with of the smartphone and smartwatch and were provided with multiple sclerosis only, disability was measured by EDSS [1], supporting content to help them complete the tests successfully. Patient Determined Disease Steps (PDDS) [21], and Multiple Participants were instructed to complete the active tests at Sclerosis Impact Scale–29 (MSIS-29; version 2) [22-24]. While approximately the same time each day and carry the smartphone performing some of the in-clinic tests, people with multiple and smartwatch throughout the day, recharging the devices sclerosis and HCs were asked to carry or wear the smartphone overnight. Data transfer from the smartphone and smartwatch and smartwatch to collect sensor data alongside the in-clinic is described in Multimedia Appendix 1. measures. On the scheduled in-clinic visits, the smartphone and smartwatch FLOODLIGHT active tests were performed under FLOODLIGHT Active Tests investigator supervision. The satisfaction questionnaire People with multiple sclerosis and HCs were asked to perform (Multimedia Appendix 2) assessed people with multiple various active tests (daily, weekly, every two weeks, or on sclerosis’ and HCs’ experience regarding smartphone and demand) via the smartphone (Figure 2 and Table 1). These novel smartwatch use and its impact on their daily activities at the active tests were developed to be self-administered by people week 12 visit and at the study termination/early discontinuation with multiple sclerosis to capture MS symptoms. A range of visit. Brain MRI was performed in people with multiple sclerosis clinical and sensor-based assessments were chosen to capture at the enrollment visit and at week 24. the most prominent symptoms of MS from a broad spectrum of symptoms. People with multiple sclerosis and HCs were Smartphone and Smartwatch Testing required to wear the smartwatch throughout the active tests. At the enrollment visit, people with multiple sclerosis and HCs were provided with the FLOODLIGHT solution that included http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 3 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Figure 2. FLOODLIGHT active tests and their schedule frequency. DMQ: Daily Mood Question; MSIS-29: Multiple Sclerosis Impact Scale–29; SBT: Static Balance Test; SDMT: Symbol Digit Modalities Test; ST: Symptom Tracker; 2MWT: Two-Minute Walk Test; 5UTT: 5 U-Turn Test. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 4 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Table 1. FLOODLIGHT active tests. Domain and test Short description Daily hand motor function testsa Draw a Shape (DaS) Test The aim of the DaS Test is to assess fine finger/manual dexterity while the participants are instructed to hold the mobile device in the untested hand and draw on the smartphone touchscreen six prewritten alternating shapes of increasing complexity (linear, rectangular, circular, sinusoidal, and spiral) with the second finger of the tested hand as fast and as accurately as possible within a maximum time (30 seconds for each of the two attempts per shape). Pinching Test The aim of the Pinching Test is to assess fine pinching/grasping dexterity while the participants are instructed to hold the mobile device in the untested hand and touch the screen with two fingers from the tested hand (thumb + second or thumb + third finger preferred) to squeeze/pinch as many round shapes (ie, tomatoes) as they can during 30 seconds. Daily gait testsb Two-Minute Walk Test Participants are instructed to walk as fast and as long as they can for 2 minutes but walk safely. The 2MWT is a (2MWT) simple test that is required to be performed on an even ground in a place where participants have identified they could walk straight for as far as ≥200 meters without U-turns. Participants are allowed to wear regular footwear and an assistive device and/or orthotic as needed. 5 U-Turn Test (5UTT) The aim of this test is to assess difficulties or unusual patterns in performing U-turns while walking on a short distance at comfortable pace. The 5UTT can be performed indoors or outdoors, on an even ground where participants are instructed to walk safely and perform five successive U-turns going back and forward between two points a few meters apart for 1 minute. Participants are allowed to wear regular footwear and an assistive device and/or orthotic as needed. Static Balance Test (SBT) Participants are asked to stand still unsupported for 30 seconds with relaxed arms straight alongside the body if possible. Weekly cognitive test Electronic version of the The aim of SDMT testing is to detect impairment of key neurocognitive functions that underlie many substitution Symbol Digit Modalities tasks. Test (SDMT) [15-17] Patient-reported outcomes (PROs) Daily Mood Question This test represents an assessment of participants’ perceived overall state by responding daily to the question “How (DMQ) do you feel now?” on a 5-item Likert scale, ranging from excellent to horrible. Electronic version of the This questionnaire measures the physical and psychological impact of multiple sclerosis. Multiple Sclerosis Impact Scale–29, version 2 (MSIS- 29) [22-24]; people with multiple sclerosis only Multiple Sclerosis Symptom Patients are asked if they experienced new or significantly worsening symptoms during the last 2 weeks. If yes, Tracker (MSST); people onset of the symptoms and the patients’ perception to whether they think they experienced a relapse (yes, no, or with multiple sclerosis only unsure) are recorded. a Tests alternatingly performed with right and left hand; users are instructed on daily alternation. b Recommended position of smartphone in an anterior medial position in the belt bag. withdrew from the study for any reason were still included in FLOODLIGHT Passive Monitoring the FAS. Supportive analyses of selected variables were carried Passive monitoring collected metrics on gait and mobility out in the per-protocol population, which included people with throughout the day in a continuous and unobtrusive manner. multiple sclerosis who completed at least 1 week in the study Participants were instructed to carry their smartphone preferably and did not discontinue due to an adverse event or a reason in an anterior medial position in a belt bag or, alternatively, in unrelated to the use of the FLOODLIGHT solution (Multimedia their pocket and wear the smartwatch all day as they went about Appendix 3). their daily routine until the devices ran out of charge. Adherence was evaluated for the following tests and test groups: Statistical Analyses all FLOODLIGHT active tests, Two-Minute Walk Test The analyses of the primary objectives of this study were (2MWT), all active tests except 2MWT, smartphone use, descriptive. Statistical tests were exploratory and conducted at smartwatch use, and for the per-protocol population. the two-sided 5% significance level without adjustment for Adherence to active tests was measured as the proportion of multiple comparisons. The analyses were based on all enrolled study weeks with at least 3 days of completed testing (study patients (full analysis set [FAS]). Patients who prematurely co-primary endpoint). Adherence to sensor-based passive http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 5 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al monitoring was measured as the proportion of study weeks with items are reported, along with covariate analyses of at least 3 days of passive monitoring for at least 4 hours per day demographics and disease state. Analysis of the change in while the devices were worn by the participant [25] (study satisfaction score between week 12 and week 24 are reported co-primary endpoint). Descriptive statistics of calculated using the Wilcoxon signed-rank test. adherence were reported for all active tests, all active tests Patient baseline characteristics incorporated as covariates in the except 2MWT, 2MWT, smartphone use, and smartwatch use. analysis of correlation with FLOODLIGHT adherence and Categorical and numeric variables with fewer than five values satisfaction outcomes were age, gender, body mass index, time were tested for association with adherence using the since first MS symptom onset, EDSS, T25FW time, 9HPT time, Kruskal-Wallis test. The association between continuous and the oral SDMT correct responses. A descriptive analysis of variables was assessed using the Spearman rank correlation. safety variables, including adverse events and serious adverse Participant complete abandoning of active testing and passive events, was carried out in the FAS. monitoring was also investigated in a time-to-event survival analysis based on the Kaplan-Meier method along the Results FLOODLIGHT study. The abandoning event was defined as the last week in which the participant was adherent according Baseline Demographics and Characteristics to the definitions above for active tests and passive monitoring. Participants’ baseline demographics for the FAS are described Active tests performed on days of in-clinic visits were not in Table 2. There was an expected greater proportion of females considered in the adherence calculation to focus the abandoning among the people with multiple sclerosis compared with HCs. analysis on the remote use. Participants leaving the study before The majority (69/76, 91%) of people with multiple sclerosis the terminal visit were considered as censored. The impact of had relapsing-remitting MS (RRMS), with a mild EDSS score different characteristics on adherence was assessed using Cox (mean 2.4) and presumably “normal” hand/arm function based regression. on an upper limit of normal range defined as the average 9HPT A satisfaction score was developed from the satisfaction time for HCs plus two standard deviations [26] (Table 2). questionnaire (Multimedia Appendix 2) that sums the individual Overall, 92% (70/76) of people with multiple sclerosis and 64% answers to questions 1-7 and 10-12 rescaled to 0-100 from their (16/25) of HCs who enrolled in the FLOODLIGHT study original Likert scale. An interquestion correlation analysis was reached the week 24 visit. Reasons for discontinuation from the performed to ensure questions are equally correlated and can study are described in Multimedia Appendix 3. be combined. Descriptive statistics of satisfaction score and http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 6 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Table 2. Demographics and characteristics of people with multiple sclerosis and healthy controls (HCs) at baseline. Parameter People with multiple sclerosis (n=76) HCs (n=25) Age (years), mean (SD) 39.5 (7.9) 34.9 (9.3) Female, n (%) 53 (70) 7 (28) Multiple sclerosis (MS) diagnosis, n (%) Primary progressive multiple sclerosis 3 (4) —a Secondary progressive multiple sclerosis 4 (5) — Relapsing-remitting multiple sclerosis 69 (91) — Time since MS symptom onset (years), mean (SD) 11.3 (7.0)b — Proportion of people with multiple sclerosis with ≥1 relapse in the past year, n (%) 18 (24) — Expanded Disability Status Scale, mean (SD) 2.4 (1.4) — Proportion of people with multiple sclerosis with ≥1 T1 Gdc-enhancing lesion, n (%) 2 (3)d — Total FLAIRe lesion volume (mL), mean (SD) 6.3 (7.5)f — 9-Hole Peg Test (seconds), mean (SD) Dominant hand 22.1 (4.6)g 18.9 (2.1) Nondominant hand 22.8 (4.9)h 19.5 (2.0) Timed 25-Foot Walk (seconds), mean (SD) 6.0 (2.1)b 5.0 (1.0) Symbol Digit Modalities Test (correct responses), mean (SD) 53.8 (11.8)b 63.8 (10.0) Berg Balance Scale, mean (SD) 52.5 (5.7)i 56.0 (0)j Patient Determined Disease Steps, mean (SD) 1.5 (1.6) — Fatigue Scale for Motor and Cognitive Functions (total score), mean (SD) 59.1 (22.7)g 25.5 (6.0) Patient Health Questionnaire–9, mean (SD) 8.3 (6.1)k 2.4 (2.9)l Participants with any previous medications, n (%) 46 (61) 6 (24) Previous disease-modifying treatmentm, n (%) Daclizumab (Zinbryta) 0 (0) — Glatiramer acetate (Copaxone) 12 (16) — Glatiramer acetate (Glatopa) 1 (1) — IFNn β-1a IMo (Avonex) 4 (5) — IFN β-1a SCp (Rebif) 5 (7) — IFN β-1b SC (Betaseron/Betaferon) 6 (8) — IFN β-1b SC (Extavia) 1 (1) — Pegylated IFN β-1a (Plegridy) 2 (3) — Dimethyl fumarate (Tecfidera) 9 (12) — Fingolimod (Gilenya) 9 (12) — Teriflunomide (Aubagio) 3 (4) — Alemtuzumab (Lemtrada) 2 (3) — Mitoxantrone (Novantrone) 1 (1) — Natalizumab (Tysabri) 19 (25) — Otherq 5 (7) — a Not applicable. b n=75. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 7 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al c Gd: gadolinium. d n=68. e FLAIR: fluid-attenuated inversion recovery. f n=70. g n=73. h n=74. i n=71. j n=22. k n=60. l n=20. m Total baseline disease-modifying treatment history. n IFN: interferon. o IM: intramuscular. p SC: subcutaneous. q Hidroferol; Radiance study (RPC1063 versus IFN β-1a); Rituximab (Rituxan). completed all active tests on average 4 out of 7 days per week Adherence (Figure 4), and recorded at least 4 hours of data via passive Over a period of 18 months (November 2016-April 2018), more monitoring on average 4 out of 7 days per week (Figure 5). The than 6 terabytes of raw data were collected from 76 people with lowest average adherence over 24 weeks was observed for active multiple sclerosis and 25 HCs. Participants performed 67,544 tests including the 2MWT and the 2MWT only, with participants active tests, of which 9787 were the 2MWT, and recorded showing highest average adherence for passive monitoring 200,171 hours of passive monitoring, of which 113,165 hours (Figure 6). A total of 70% (16.68/24 weeks) of participants were were captured with the smartwatch. Over 24 weeks, most adherent to all active tests, 75% (17.95/24 weeks) to all active participants performed 5 to 7 active tests per week, including tests except 2MWT, 71% (17.13/24 weeks) to 2MWT, 79% the 2MWT (Figure 3). Adherence of people with multiple (18.89/24 weeks) to smartphone- or smartwatch-based passive sclerosis to completing active tests and passive monitoring was monitoring, 66% (15.74/24 weeks) to smartphone-based passive good and remained stable over time after week 6 (Figures 4 and monitoring, and 74% (17.69/24 weeks) to smartwatch-based 5). Even in the last week of the 24-week study, participants passive monitoring. Figure 3. Adherence of people with multiple sclerosis to active tests for individual participants: number of performed active tests per week [level of activity (light green: high; dark green/grey: low] over individual study weeks [columns]). http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 8 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Figure 4. Adherence of people with multiple sclerosis to active tests. 2MWT: Two-Minute Walk Test. Figure 5. Adherence of people with multiple sclerosis to smartphone and smartwatch passive monitoring. Days with more than 4 hours of passive monitoring on a device are considered as adherent. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 9 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Figure 6. Adherence of people with multiple sclerosis to active tests and passive monitoring. The results of the time-to-event survival analysis based on the Kaplan–Meier method along the FLOODLIGHT study. The abandoning event was defined as the last week in which the participant was adherent according to the definitions for active tests and passive monitoring. Active tests performed on days of in-clinic visits were not considered in the adherence calculation, to focus the abandoning analysis on the remote use. Participants leaving the study before the terminal visit were considered as censored. 2MWT: Two-Minute Walk Test. Correlation was explored between adherence measures and Implications for the use of the FLOODLIGHT test battery in people with multiple sclerosis population characteristics. Only people with multiple sclerosis were assessed from individual disease duration showed significant small negative correlation questions from the patient satisfaction questionnaire. When with measures of adherence (Spearman rank correlations; asked to rate the impact of the smartphone, smartwatch, and 2MWT adherence: –0.42, P<.001; smartphone passive active tests on daily living, more than 80% (61/72) of people monitoring adherence: –0.29, P=.02; all active tests except with multiple sclerosis perceived the FLOODLIGHT test battery 2MWT adherence: –0.37, P=.003; and smartwatch passive to have at least acceptable impact on daily activities (Figure 7). monitoring adherence: –0.27, P=.04), indicating that disease Nearly 50% (32/71) of participants had no issue with any of the severity and demographics did not appear to play a significant active tests, and only one-third would prefer to avoid the 2MWT, role in adherence. most likely due to increased burden from execution—for example, having to find a place to perform the test or not Patient Satisfaction wanting to go outside in bad weather (Figure 8). Without The average overall satisfaction score among people with providing any data feedback to the people with multiple sclerosis multiple sclerosis who completed the study at week 12 (n=64) throughout the study, more than 60% (46/72) of participants was 74.1 out of a possible 100 and remained stable at week 24 would have liked to continue using FLOODLIGHT “to (study termination/early discontinuation visit [n=68]) with 73.7 understand my MS better and improve my disease management” out of 100 (Wilcoxon signed-rank test P=.71). There was one (Figure 9). Approximately 90% (65/72) of people with multiple significant association between overall satisfaction score and sclerosis indicated their interest to see the results of the tests, gender (P=.04). Individual questions from the satisfaction which will be addressed in future Roche-sponsored studies using questionnaire (Multimedia Appendix 2) were analyzed for their FLOODLIGHT (CONSONANCE [NCT03523858] and association with people with multiple sclerosis population FLOODLIGHT Open [floodlightopen.com]; Figure 10). characteristics, described in Multimedia Appendix 1. Analysis of patient responses to the satisfaction questionnaire is described in Multimedia Appendix 1. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 10 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al Figure 7. Implications of FLOODLIGHT in people with multiple sclerosis for “impact on daily activities” from the patient satisfaction questionnaire. Figure 8. Implications of FLOODLIGHT in people with multiple sclerosis for “avoiding one component of FLOODLIGHT" from the patient satisfaction questionnaire. 2MWT: Two-Minute Walk Test. Figure 9. Implications of FLOODLIGHT in people with multiple sclerosis for “desire to continue using the FLOODLIGHT app” from the patient satisfaction questionnaire. Figure 10. Implications of FLOODLIGHT in people with multiple sclerosis for “prefer to see results immediately to monitor” from the patient satisfaction questionnaire. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 11 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al primary progressive MS (PPMS) differentially impacts Discussion adherence compared with people with relapsing MS (RMS) Principal Findings cannot be gleaned from the current data set due to the low numbers of advanced patients enrolled in the study; however, This study demonstrates that the use of smartphones and this important question warrants future research exploring smartwatches for remote daily active testing and continuous remote monitoring in patients with more advanced MS. The passive monitoring is feasible over 6 months and provides importance of continuous monitoring in RMS should also not further support to earlier studies, which have shown that healthy be overlooked, as the sensitivity of this novel approach aiming participants and people with multiple sclerosis were capable of at detecting progression in a real-world setting may provide an completing daily tasks on a smartphone [7]. This study provides earlier window into disease progression outside of the clinic. further evidence for the use of digital technology, including smartphones, for data collection. Other studies in MS have used Comparison With Prior Work smartphone apps to (1) assess steps when walking on a treadmill A recent study assessing the feasibility of the MS TeleCoach, [9]; (2) assess pain, fatigue, anxiety, and quality of life [13]; a novel intervention offering telemonitoring of fatigue and and (3) assess the feasibility of gathering passive and active telecoaching of physical activity in people with multiple performance data [7]. Together with the current analyses, these sclerosis, showed that participants were highly engaged, with studies document the focus toward developing digital measures 76% (57/75) of participants completing the study, and 91% to continuously monitor and assess the MS disease. (21/23) of a subset of completers showing a median of quite In this protocol, the FLOODLIGHT solution collected metrics satisfied in the patient satisfaction questionnaire [27]. During on cognition, mood, upper extremity function, and gait and the 12-week study period, use of the MS TeleCoach improved posture by instructing participants to perform a set of daily fatigue levels in people with multiple sclerosis with moderate active tests, which should take approximately 5 minutes in total to severe fatigue, suggesting that implementation of digital to complete and capture activity data via passive monitoring technologies can enhance patient performance. Together with over a period of 24 weeks. A previous study has shown that the data presented here, these results indicate that the use of 51% of participants (22/38 of people with multiple sclerosis consumer devices by people with multiple sclerosis for sensor and 17/38 of healthy participants) completed 12 months of daily data capture fulfills the prerequisites of people with multiple data collection, where participants were prompted to complete sclerosis satisfaction and acceptable adherence to daily active one assigned test [7]. In the context of the FLOODLIGHT study, tests and passive monitoring for potential integration in we observed that overall adherence to active tests was 70% long-term clinical trials and treatment monitoring. Regarding (16.68/24 weeks), which appears to be higher than the adherence future studies, for example FLOODLIGHT Open, attempts will of participants to 12 months of daily data collection (39/76, be made to improve participant adherence throughout the study 51%) from Bove et al [7]. However, comparisons between the by introducing controlled app variations, such as reminders, studies are limited, as the study design and burden of testing types of achievements, and fun metrics. are different—for example, the app from Bove et al contained Conclusions 19 different tests, of which participants were prompted to In summary, these analyses showed that people with multiple complete one each day. As the FLOODLIGHT app was sclerosis are highly engaged with performing active tests and integrated into standalone devices in this study, deployment of capturing continuous data via passive monitoring and are the app on participants’ own mobile devices may increase satisfied with the FLOODLIGHT test battery. Neither adherence because it removes the need to carry a separate, satisfaction nor adherence showed strong correlation with study dedicated device and decreases burden on the individual. population characteristics. More than 60% (46/72) of people Limitations with multiple sclerosis indicated their interest to continue to As this study remains a pilot investigation designed to collect use FLOODLIGHT, and approximately 90% (65/72) wanted first experiences from continuous sensor data capture, the main to see the results of their tests in real time as biofeedback, which limitation is the small sample size and short duration of was implemented in future studies using the FLOODLIGHT follow-up. Future ongoing FLOODLIGHT studies solution. These findings indicate that smartphone-based (CONSONANCE and FLOODLIGHT Open) will collect longer FLOODLIGHT outcomes may represent a promising avenue term data on smartphone-based sensor data capture in a larger to enable a more accurate and continuous assessment of MS number of participants from a broader disability spectrum. disease in clinical trials and real-world practice settings and Additionally, whether physical and cognitive limitations in may eventually also contribute to informing and guiding clinical people with secondary progressive MS (SPMS) and people with research and clinical practice in the future. Acknowledgments We would like to thank all patients, their families, and the investigators who participated in this trial. This research was funded by F Hoffmann–La Roche Ltd, Basel, Switzerland. We would like to thank the following employees from F Hoffmann–La Roche Ltd who supported and contributed to the study: Atieh Bamdadian, Alessandro Barbato, Jan Beckmann, Sandro Fritz, Nicholas Pierce Heinemeier, Timothy Kilchenmann, Lito Kriara, Bernd Laub, Grégoire Pointeau, Caroline Polakowska, Marcin Puhacz, Cedric Simillion, Jens Schjodt-Eriksen, Jöerg Sprengel, Ralf Stubner, and Krzysztof Trybus. Writing and editorial assistance for http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 12 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al this manuscript was provided by Heather Latimer from Articulate Science, United Kingdom, and funded by F Hoffmann–La Roche Ltd. Qualified researchers may request access to individual patient-level data through the clinical study data request platform (https://clinicalstudydatarequest.com). Further details on Roche’s criteria for eligible studies are available at https://clinicalstudydatarequest.com/Study-Sponsors/Study-Sponsors-Roche.aspx. For further details on Roche’s Global Policy on the Sharing of Clinical Information and how to request access to related clinical study documents, see https://www.roche.com/research_and_development/who_we_are_how_we_work/clinical_trials/our_commitment_to_data_sharing.htm. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: F Hoffmann–La Roche Ltd, Basel, Switzerland, provided financial support for the study and publication of this manuscript. Conflicts of Interest XM has received speaker honoraria and travel expense reimbursement for participation in scientific meetings, been a steering committee member of clinical trials, or served on advisory boards of clinical trials for Actelion, Biogen, Celgene, Merck, Novartis, Oryzon, Roche, Sanofi Genzyme, and Teva Pharmaceutical. JG has received grants or research support from Biogen, Genentech Inc, and S3 Group and has received compensation for a nonbranded resident and fellow education seminar supported by Biogen. SLH serves on the scientific advisory boards for Annexon, Symbiotix, Bionure, and Molecular Stethoscope, is on the board of trustees for Neurona Therapeutics, and has received travel reimbursement and writing assistance from F Hoffmann–La Roche Ltd for CD20-related meetings and presentations. LJ is an employee of Genentech Inc and a shareholder of F Hoffmann–La Roche Ltd. MB, JS, and CG are employees and shareholders of F Hoffmann–La Roche Ltd. AS, FL, and JvB are employees of F Hoffmann–La Roche Ltd. CB and ML are contractors for F Hoffmann–La Roche Ltd. SB was an employee of F Hoffmann–La Roche Ltd during the completion of the work related to this manuscript. SB is now an employee of Biogen (Cambridge, MA), which was not in any way associated with this study. LM and PM have nothing to disclose. Multimedia Appendix 1 Additional trial information. [PDF File (Adobe PDF File), 278KB-Multimedia Appendix 1] Multimedia Appendix 2 Satisfaction questionnaire. [PDF File (Adobe PDF File), 238KB-Multimedia Appendix 2] Multimedia Appendix 3 Participant flow table. [PDF File (Adobe PDF File), 138KB-Multimedia Appendix 3] References 1. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 1983 Nov;33(11):1444-1452. [Medline: 6685237] 2. Bradshaw MJ, Farrow S, Motl RW, Chitnis T. Wearable biosensors to monitor disability in multiple sclerosis. Neurol Clin Pract 2017 Aug;7(4):354-362. [doi: 10.1212/CPJ.0000000000000382] [Medline: 29185551] 3. Lavorgna L, Brigo F, Moccia M, Leocani L, Lanzillo R, Clerico M, et al. e-Health and multiple sclerosis: An update. Mult Scler 2018 Nov;24(13):1657-1664. [doi: 10.1177/1352458518799629] [Medline: 30231004] 4. Busis N. Mobile phones to improve the practice of neurology. Neurol Clin 2010 May;28(2):395-410. [doi: 10.1016/j.ncl.2009.11.001] [Medline: 20202500] 5. Lipsmeier F, Taylor KI, Kilchenmann T, Wolf D, Scotland A, Schjodt-Eriksen J, et al. Evaluation of smartphone-based testing to generate exploratory outcome measures in a phase 1 Parkinson's disease clinical trial. Mov Disord 2018 Dec;33(8):1287-1297 [FREE Full text] [doi: 10.1002/mds.27376] [Medline: 29701258] 6. Prasad S, Ramachandran R, Jennings C. Development of smartphone technology to monitor disease progression in multiple sclerosis. Neurology 2012 Apr 22;78(Meeting Abstracts 1):P01.144. [doi: 10.1212/WNL.78.1_MeetingAbstracts.P01.144] 7. Bove R, White CC, Giovannoni G, Glanz B, Golubchikov V, Hujol J, et al. Evaluating more naturalistic outcome measures: a 1-year smartphone study in multiple sclerosis. Neurol Neuroimmunol Neuroinflamm 2015 Dec;2(6):e162 [FREE Full text] [doi: 10.1212/NXI.0000000000000162] [Medline: 26516627] 8. Boukhvalova AK, Kowalczyk E, Harris T, Kosa P, Wichman A, Sandford MA, et al. Identifying and quantifying neurological disability via smartphone. Front Neurol 2018;9:740 [FREE Full text] [doi: 10.3389/fneur.2018.00740] [Medline: 30233487] http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 13 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al 9. Balto JM, Kinnett-Hopkins DL, Motl RW. Accuracy and precision of smartphone applications and commercially available motion sensors in multiple sclerosis. Mult Scler J Exp Transl Clin 2016;2:2055217316634754 [FREE Full text] [doi: 10.1177/2055217316634754] [Medline: 28607720] 10. Haase R, Schultheiss T, Kempcke R, Thomas K, Ziemssen T. Modern communication technology skills of patients with multiple sclerosis. Mult Scler 2013 Aug;19(9):1240-1241. [doi: 10.1177/1352458512471882] [Medline: 23388164] 11. Rudick RA, Miller D, Bethoux F, Rao SM, Lee J, Stough D, et al. The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool. J Vis Exp 2014 Jun 30(88):e51318 [FREE Full text] [doi: 10.3791/51318] [Medline: 25046650] 12. Gholami F, Trojan DA, Kovecses J, Haddad WM, Gholami B. A Microsoft Kinect-based point-of-care gait assessment framework for multiple sclerosis patients. IEEE J Biomed Health Inform 2017 Dec;21(5):1376-1385. [doi: 10.1109/JBHI.2016.2593692] [Medline: 27455529] 13. Kos D, Raeymaekers J, Van Remoortel A, D'hooghe MB, Nagels G, D'Haeseleer M, et al. Electronic visual analogue scales for pain, fatigue, anxiety and quality of life in people with multiple sclerosis using smartphone and tablet: a reliability and feasibility study. Clin Rehabil 2017 Sep;31(9):1215-1225. [doi: 10.1177/0269215517692641] [Medline: 28786335] 14. Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol 2011 Feb;69(2):292-302 [FREE Full text] [doi: 10.1002/ana.22366] [Medline: 21387374] 15. Smith A. The symbol-digit modalities test: A neuropsychologic test of learning and other cerebral disorders. Seattle: Helmuth J, editor. Learning disorders. Seattle: Special Child Publications; 1968:83-91. 16. Smith A. Symbol Digits Modalities Test: Manual. Los Angeles: Western Psychological Services; 1982. 17. Rao SM, Losinski G, Mourany L, Schindler D, Mamone B, Reece C, et al. Processing speed test: validation of a self-administered, iPad-based tool for screening cognitive dysfunction in a clinic setting. Mult Scler 2017 Dec;23(14):1929-1937. [doi: 10.1177/1352458516688955] [Medline: 28080262] 18. Berg KO, Wood-Dauphinee SL, Williams JI, Maki B. Measuring balance in the elderly: validation of an instrument. Can J Public Health 1992;83 Suppl 2:S7-S11. [Medline: 1468055] 19. Penner IK, Raselli C, Stöcklin M, Opwis K, Kappos L, Calabrese P. The Fatigue Scale for Motor and Cognitive Functions (FSMC): validation of a new instrument to assess multiple sclerosis-related fatigue. Mult Scler 2009 Dec;15(12):1509-1517. [doi: 10.1177/1352458509348519] [Medline: 19995840] 20. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline: 11556941] 21. Learmonth YC, Motl RW, Sandroff BM, Pula JH, Cadavid D. Validation of patient determined disease steps (PDDS) scale scores in persons with multiple sclerosis. BMC Neurol 2013 Apr 25;13:37 [FREE Full text] [doi: 10.1186/1471-2377-13-37] [Medline: 23617555] 22. Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technol Assess 2009 Feb;13(12):1-177 [FREE Full text] [doi: 10.3310/hta13120] [Medline: 19216837] 23. Hobart J, Lamping D, Fitzpatrick R, Riazi A, Thompson A. The Multiple Sclerosis Impact Scale (MSIS-29): a new patient-based outcome measure. Brain 2001 May;124(Pt 5):962-973. [doi: 10.1093/brain/124.5.962] [Medline: 11335698] 24. Riazi A, Hobart JC, Lamping DL, Fitzpatrick R, Thompson AJ. Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry 2002 Dec;73(6):701-704 [FREE Full text] [doi: 10.1136/jnnp.73.6.701] [Medline: 12438473] 25. Cheng W, Scotland A, Lipsmeier F, Kilchenmann T, Jin L, Schjodt-Eriksen J, et al. Human activity recognition from sensor-based large-scale continuous monitoring of Parkinson's disease patients. 2017 Presented at: Proceedings of the 2nd IEEE/ACM International Conference on Connected Health; July 17-19, 2017; Philadelphia. [doi: 10.1109/CHASE.2017.87] 26. Wang Y, Bohannon RW, Kapellusch J, Garg A, Gershon RC. Dexterity as measured with the 9-Hole Peg Test (9-HPT) across the age span. J Hand Ther 2015;28(1):53-59. [doi: 10.1016/j.jht.2014.09.002] [Medline: 25449717] 27. D'hooghe M, Van Gassen G, Kos D, Bouquiaux O, Cambron M, Decoo D, et al. Improving fatigue in multiple sclerosis by smartphone-supported energy management: the MS TeleCoach feasibility study. Mult Scler Relat Disord 2018 May;22:90-96 [FREE Full text] [doi: 10.1016/j.msard.2018.03.020] [Medline: 29649789] Abbreviations 2MWT: Two-Minute Walk Test 5UTT: 5 U-Turn Test 9HPT: 9-Hole Peg Test BBS: Berg Balance Scale DMQ: Daily Mood Question EDSS: Expanded Disability Status Scale FAS: full analysis set FLAIR: fluid-attenuated inversion recovery http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 14 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Midaglia et al FSMC: Fatigue Scale for Motor and Cognitive Functions HCs: healthy controls MRI: magnetic resonance imaging MS: multiple sclerosis MSIS-29: Multiple Sclerosis Impact Scale–29 MSST: Multiple Sclerosis Symptom Tracker PDDS: Patient Determined Disease Steps PHQ-9: Patient Health Questionnaire–9 PPMS: primary progressive multiple sclerosis PRO: patient-reported outcome RMS: relapsing multiple sclerosis RRMS: relapsing-remitting multiple sclerosis SBT: Static Balance Test SDMT: Symbol Digit Modalities Test SPMS: secondary progressive multiple sclerosis T25FW: Timed 25-Foot Walk Edited by G Eysenbach; submitted 11.06.19; peer-reviewed by A Nguyen, L Visser, L Lavorgna, T Clavier; comments to author 29.06.19; revised version received 11.07.19; accepted 19.07.19; published 30.08.19 Please cite as: Midaglia L, Mulero P, Montalban X, Graves J, Hauser SL, Julian L, Baker M, Schadrack J, Gossens C, Scotland A, Lipsmeier F, van Beek J, Bernasconi C, Belachew S, Lindemann M Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People With Multiple Sclerosis: Nonrandomized Interventional Feasibility Study J Med Internet Res 2019;21(8):e14863 URL: http://www.jmir.org/2019/8/e14863/ doi: 10.2196/14863 PMID: 31471961 ©Luciana Midaglia, Patricia Mulero, Xavier Montalban, Jennifer Graves, Stephen L Hauser, Laura Julian, Michael Baker, Jan Schadrack, Christian Gossens, Alf Scotland, Florian Lipsmeier, Johan van Beek, Corrado Bernasconi, Shibeshih Belachew, Michael Lindemann. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 30.08.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. http://www.jmir.org/2019/8/e14863/ J Med Internet Res 2019 | vol. 21 | iss. 8 | e14863 | p. 15 (page number not for citation purposes) XSL• FO RenderX