23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License RESEARCH ARTICLE Preliminary validity of the Draw a Shape Test for upper extremity assessment in multiple sclerosis Jennifer S. Graves1 , Marco Ganzetti2 , Frank Dondelinger2,† , Florian Lipsmeier2 , Shibeshih Belachew2 , Corrado Bernasconi2 , Xavier Montalban3 , Johan van Beek2,‡ , Michael Baker2, Christian Gossens2 & Michael Lindemann2 1 Department of Neurosciences, University of California San Diego, San Diego, California, USA 2 F. Hoffmann-La Roche Ltd, Basel, Switzerland 3 Department of Neurology-Neuroimmunology, Centre d’Esclerosi Mu ltiple de Catalunya (Cemcat), Hospital Universitari Vall d’Hebron, Barcelona, Spain Correspondence Abstract Florian Lipsmeier, F. Hoffmann-La Roche Ltd, Grenzacherstrasse 124, 4070 Basel, Objective: To validate the smartphone sensor-based Draw a Shape Test – a Switzerland. Tel: +41616877909; part of the Floodlight Proof-of-Concept app for remotely assessing multiple E-mail: florian.lipsmeier@roche.com sclerosis-related upper extremity impairment by tracing six different shapes. Methods: People with multiple sclerosis, classified functionally normal/abnor- Funding Information mal via their Nine-Hole Peg Test time, and healthy controls participated in a This research was funded by F. Hoffmann-La 24-week, nonrandomized study. Spatial (trace accuracy), temporal (mean and Roche Ltd, Basel, Switzerland. variability in linear, angular, and radial drawing velocities, and dwell time Received: 23 June 2022; Revised: 15 October ratio), and spatiotemporal features (trace celerity) were cross-sectionally ana- 2022; Accepted: 5 November 2022 lyzed for correlation with standard clinical and brain magnetic resonance imag- ing (normalized brain volume and total lesion volume) disease burden measures, and for capacity to differentiate people with multiple sclerosis from healthy controls. Results: Data from 69 people with multiple sclerosis and 18 healthy controls were analyzed. Trace accuracy (all shapes), linear velocity vari- doi: 10.1002/acn3.51705 ability (circle, figure-of-8, spiral shapes), and radial velocity variability (spiral † At the time of writing author was an shape) had a mostly fair/moderate-to-good correlation (|r| = 0.14–0.66) with all employee of F. Hoffmann-La Roche Ltd; disease burden measures. Trace celerity also had mostly fair/moderate-to-good current affiliation is Novartis Institutes for correlation (|r| = 0.18–0.41) with Nine-Hole Peg Test performance, cerebellar Biomedical Research, Basel, Switzerland. functional system score, and brain magnetic resonance imaging. Furthermore, partial correlation analysis related these results to motor impairment. People ‡ At the time of writing author was an with multiple sclerosis showed greater drawing velocity variability, though employee of F. Hoffmann-La Roche Ltd; slower mean velocity, than healthy controls. Linear velocity (spiral shape) and current affiliation is Biogen, Cambridge, MA, USA. angular velocity (circle shape) potentially differentiate functionally normal peo- ple with multiple sclerosis from healthy controls. Interpretation: The Draw a Shape Test objectively assesses upper extremity impairment and correlates with all disease burden measures, thus aiding multiple sclerosis-related upper extremity impairment characterization. with MS (PwMS) experience or show signs of impaired Introduction upper extremity function.5–7 Multiple sclerosis (MS) is a chronic inflammatory, Upper extremity dysfunction can impact quality of life autoimmune, demyelinating disease of the central nervous and daily activities including, but not limited to, grasping system, which can cause impairment to several functional and manipulating objects, typing, and handwriting.4,8–10 domains.1 One commonly affected domain is upper Notwithstanding, functional ability, including upper extremity function, in which impairment arises from extremity function, is not routinely assessed in clinical either ataxia, motor dysfunction, or sensory dysfunc- practice with disability scales.11 Currently, despite the tion.2–4 Reports have suggested that 60–76% of people availability of other tools, the Nine-Hole Peg Test ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 1 This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. (9HPT) is the only well-accepted standardized test assess- upper extremity function. Using this expanded feature set, ing upper extremity function.12 The relationships between we assess the DaS Test’s agreement with the clinical and 9HPT and MRI have been previously explored in PwMS, brain MRI measures of MS disease state, and its ability to yielding substantial trends and correlations.13 A 20% differentiate PwMS, including those with normal 9HPT increase in the 9HPT time is commonly used to define a times, from healthy controls (HC). We also analyze all fea- clinically meaningful worsening.12 However, the 9HPT tures applicable to each shape with detailed comparisons. has several key limitations, such as the need for patient supervision and pegboard, in addition to the low granu- Methods larity and functionality of the summary score of overall performance, limited to reporting the time per hand. Study design and participants Developing accessible, objective measures that capture upper extremity dysfunction and progression in MS is an This 24-week, nonrandomized study assessed the feasibil- unmet need. Early evidence has shown the potential of ity of remotely monitoring PwMS with the Floodlight novel, smartphone sensor-based, visually guided tasks to PoC app on a provisioned smartphone. The full study objectively assess upper extremity function with high design, and inclusion and exclusion criteria have been granularity in MS,3,14–17 Parkinson’s disease,18,19 Hunting- previously reported.22 Seventy-six PwMS and 25 HC aged ton’s disease,20 and essential tremor.21 18–55 years were enrolled. The HC were often the part- The study “Monitoring of Multiple Sclerosis Partici- ners of PwMS, and while not matched, there were only pants With the Use of Digital Technology (Smartphones small differences in age. Seventy PwMS were considered and Smartwatches) – A Feasibility Study” was the first adequate to detect a linear correlation coefficient of 0.33 clinical trial to implement the Draw a Shape (DaS) Test with >80% power. PwMS were diagnosed using the 2010 from the Floodlight Proof-of-Concept (PoC) app.22 The revised McDonald criteria26 and had a baseline Expanded DaS Test has also been used as a remote, smartphone Disability Status Scale (EDSS) score27 of 0.0–5.5. Within sensor-based assessment of upper extremity function in the PwMS group, participants were further subclassified Huntington’s disease20 and Parkinson’s disease.18 We pre- as PwMS with normal (PwMS-Normal) and abnormal viously showed that two test features, overall mean trace 9HPT times at baseline (PwMS-Abnormal); an abnormal accuracy and overall mean trace celerity, had good test– 9HPT time was greater than the mean plus two standard retest reliability in PwMS as measured by intraclass coeffi- deviations of the normative data of HC derived from cient correlation (second model, first class)23 (ICC[2,1]: Erasmus et al.3 Thresholds were 22.58 sec for both hands 0.85 [0.79–0.90] and 0.81 [0.73–0.87], respectively) and combined (mean of both hands), 22.15 sec for the domi- showed fair correlations with 9HPT (r = 0.48 and nant hand, and 23.01 sec for the nondominant hand. r = 0.40; p < 0.001) and T2-weighted fluid-attenuated At every scheduled clinical visit (baseline, week 12, and inversion recovery (T2-FLAIR) lesion volume (both week 24), both PwMS and HC were clinically evaluated r = 0.26; p <0.05).16 Notably, the DaS Test offers a mul- with the 9HPT and the oral Symbol Digit Modalities Test tidimensional feature space (i.e., a set of measures derived (SDMT; Fig. 1). Additionally, PwMS were assessed by the from the DaS Test that capture distinctly different aspects 29-item Multiple Sclerosis Impact Scale (MSIS-29)28 and of upper extremity function, e.g., spatial, temporal, and EDSS at all three clinical visits, and by MRI at baseline spatiotemporal aspects [mathematical features of the raw and week 24. All Floodlight tests (except the electronic sensor data waveform]), which may address one of SDMT [e-SDMT]) were performed daily22 to improve 9HPT’s key limitations by providing a more granular data granularity. The e-SDMT was administered weekly assessment of upper extremity function. Additionally, to reduce the impact of practice effects commonly recent qualitative data suggest face and ecological validity observed on the SDMT.29,30 At the baseline visit, all par- (i.e., the degree to which the test achieves its intended ticipants received a Samsung Galaxy S7 smartphone with aims and reflects the level of performance of daily activi- the Floodlight PoC app pre-installed. The app instructed ties), with PwMS who performed the DaS Test identifying the participants to perform daily and weekly smartphone dexterity, hand steadiness, and focus as key skills required sensor-based active tests to assess their upper extremity for test performance and daily activities (e.g., writing, function, cognition, gait, and balance. coloring, drawing) that are affected by MS.24,25 Here we present our secondary, exploratory analyses that Standard protocol approvals, registrations, expand on prior work by increasing the feature space to and patient consents include spatial, spatiotemporal, and temporal features, including shape-specific features, and investigate the use of All participants provided informed consent, and ethical the DaS Test as a convenient and objective assessment of approval was obtained from the ethics committee of the 2 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results Smartphone-based tests 76 PwMS (performed daily or weekly) Clinical Clinical Clinical • EDSS: 0.0–5.5 • Age: 18–55 years MRI MRI Baseline Week 12 Week 24 25 HC Clinical Clinical Clinical Smartphone-based tests (performed daily or weekly) • Age: 18–55 years Figure 1. Study design. The Floodlight PoC app prompted both PwMS and HC to perform daily and weekly smartphone-based tests on a provi- sioned Samsung Galaxy S7 smartphone. This included the daily Draw a Shape Test. Additionally, PwMS and HC were clinically assessed at the three clinic visits at baseline, week 12, and week 24 (end of study). These assessments included, among others, the 9HPT, oral SDMT, MSIS-29 (administered in PwMS only), and EDSS (administered in PwMS only). Brain MRI was collected from PwMS at baseline and week 24. 9HPT, Nine- Hole Peg Test; EDSS, Expanded Disability Status Scale; HC, healthy controls; MSIS-29, 29-item Multiple Sclerosis Impact Scale; PoC, Proof-of- Concept; PwMS, people with multiple sclerosis; SDMT, Symbol Digit Modalities Test. Hospital Universitari Vall d’Hebron, Barcelona, Spain the second attempt was displayed only if the first attempt (study number: PR(AG)300/2016) and the Institutional to draw the shape was not successful. Review Board of the University of California San Francisco, San Francisco, CA, USA (reference number: 175728) prior Feature extraction to study initiation. The study was registered on clinicaltrials.gov (NCT02952911). Spatial, temporal, and spatiotemporal waveform features were extracted for each shape (Fig. 2, Fig. S1A).14,16 Spatial features included trace accuracy, defined as the proportion The Draw a Shape Test of sample points that overlapped with the reference shape, The DaS Test is a remotely performed, smartphone sensor- which was computed for all six shapes. Temporal perfor- based, active test of upper extremity function, which is mance was assessed by features capturing drawing velocity included in the Floodlight PoC app and is performed and swiftness in directional challenge. Velocity-based fea- daily.14 To perform the DaS Test, all study participants held tures included the mean and coefficient of variation (CV) their smartphones in the untested hand and traced six of linear drawing velocity (Fig. S1B). These features were increasingly complex shapes on the touchscreen (two diag- computed for all shapes. Additionally, shape-specific fea- onal lines [one from the top right corner to the bottom left tures were computed for both the circle and spiral (mean corner and one from the bottom left corner to the top right and CV angular velocity [i.e., how fast the fingertip rotates corner], a square, a circle, a figure-of-8, and a spiral, all with respect to the center of the shape]; mean and CV drawn clockwise) with the index finger of the tested hand radial velocity [i.e., the rate of change at which the fingertip “as fast and as accurately as possible” (Fig. 2). To draw a moves directly away from the center of the shape]; shape successfully, the participants had to continuously Fig. S1C) as well as the square (dwell time ratio; Fig. S1D). slide their finger on the touchscreen and connect indicated The dwell time ratio describes the time spent at the three start and endpoints while passing through all indicated corners when drawing the square relative to the total draw- checkpoints within 30 sec. The dominant and nondomi- ing duration. Finally, spatiotemporal features included nant hands were tested on alternate test runs. Each partici- trace celerity, which was computed for all shapes by divid- pant had two attempts per shape for successful completion; ing trace accuracy by the total drawing duration. ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 3 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. (A) Square Circle Figure-of-8 Spiral (B) Feature Target shape Larger values mean Drawing accuracy Trace accuracy All shapes Increased accuracy Drawing speed Temporal Mean linear drawing velocity All shapes Faster drawing speed Drawing speed Temporal Mean angular drawing velocity Circle, spiral Faster drawing speed Drawing speed Temporal Mean radial drawing velocity Circle, spiral Faster drawing speed Smoothness of movement Temporal CV velocity All shapes Smoother movement Smoothness of movement Temporal CV angular velocity Circle, spiral Smoother movement Smoothness of movement Temporal CV radial velocity Circle, spiral Smoother movement Temporal Square Trace celerity All shapes Figure 2. The Draw a Shape Test. (A) Participants were instructed to draw six different shapes of increasing complexity by tracing the reference shape with their finger on the smartphone display as fast and as accurately as possible. (B) Spatial, temporal, and spatiotemporal features were computed for each of the six shapes. In addition, shape-specific features were derived from the circle and spiral (mean and CV angular velocity; mean and CV radial velocity) as well as the square (dwell time ratio). CV, coefficient of variation. <14 valid tests may not be indicative of the true func- Data processing tional ability of PwMS, whereas including >14 valid tests As the DaS Test was performed unsupervised, data quality may be biased towards a less impaired sample. For the measures were implemented to ensure test performance correlation analysis, the DaS Test features were correlated was representative of upper extremity function. Invalid against the 9HPT, oral SDMT, MSIS-29 (hand items 2, test runs were identified by quality control flags, which 3, 8–10, 15), EDSS (including cerebellar functional sys- detected test runs that were executed without any screen tem and pyramidal functional system), and MRI mea- interactions or test runs characterized by “play-to-quit” sures (normalized brain volume and T2-FLAIR lesion attempts (defined as test runs having both short drawing volume). The strength of the correlation was considered duration [<0.5 sec] and low trace accuracy [<10% overlap not correlated (|r| < 0.25), fair (|r| = 0.25–0.49), with the reference shape]) and were subsequently disre- moderate-to-good (|r| = 0.50–0.75), or good-to-excellent garded from the analyses. All remaining test runs, that is (|r| > 0.75).31 Additionally, a partial correlation analysis all valid test runs, contributed by a participant were was conducted by correlating each DaS Test feature aggregated by taking the median across the entire study against 9HPT while adjusting for oral SDMT and against duration as in Montalban et al.16 Additionally, the mean oral SDMT while adjusting for 9HPT. A separate analysis of the three in-clinic assessments (or mean of two examined the DaS Test’s ability to differentiate among assessments for MRI) was computed for the correlation PwMS, PwMS-Normal, PwMS-Abnormal, and HC. Dif- analysis. ferences between the groups were assessed for statistical significance by Mann–Whitney U test; statistical signifi- cance was set at p < 0.05 without correction for multiple Statistical analysis comparisons, as the analyses were exploratory rather All participants with at least 14 valid test runs were than confirmatory. Data from the first attempt were used included in the analyses. Based on a characteristics com- to compare like with like in all analyses. Furthermore, all parison between the analyzed population (see Table 1) analyses were adjusted for age and sex with a robust lin- versus those of the total population reported by Midaglia ear model,32 and were conducted for both hands com- et al.,22 14 valid test runs were calculated as optimal for bined (mean of both hands) and for the individual a stable cross-sectional analysis. This is because including hands. 4 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results Table 1. Baseline demographics and disease characteristics. Assessed study population (n = 87)1 Variable HC (n = 18) PwMS (n = 69) Female, n (%) 6 (33) 47 (68) Age, years, mean  SD 35.0  8.9 39.4  7.8 Height, cm, mean  SD 174.0  7.7 169.1  9.0 Diagnosis, n (%) Relapsing–remitting multiple sclerosis – 62 (90) Primary progressive multiple sclerosis – 3 (4) Secondary progressive multiple sclerosis – 4 (6) Disease onset from baseline, years, mean  SD – 9.1  6.5 Oral SDMT, mean  SD (range) 64.6  8.4 (52.0–77.0) 53.7  12.0 (26.0–77.0) Expanded Disability Status Scale, mean  SD (range) Score – 2.43  1.36 (0.0–5.5) Cerebellar Functional System – 0.78  0.92 (0.0–3.0) Pyramidal Functional System – 1.36  1.06 (0.0–3.0) Nine-Hole Peg Test time, sec, mean  SD Both hands combined 18.83  1.72 22.29  4.15 Dominant hand 18.66  1.95 22.13  4.70 Nondominant hand 19.01  1.79 22.46  4.38 29-item Multiple Sclerosis Impact Scale hand items, mean  SD2 – 24.56  24.58 Total brain volume, mL, mean  SD – 1,470.9  76.2 T2-FLAIR lesion volume, mL, mean  SD – 6.3  7.6 FLAIR, fluid-attenuated inversion recovery; HC, healthy control; PwMS, people with multiple sclerosis; SDMT, Symbol Digit Modalities Test; T2, T2-weighted. 1 All study participants who fulfilled the quality control criteria were included in the assessed study population. 2 The hand-related components include items 2, 3, 8, 9, 10, and 15. |r| = 0.32–0.66, oral SDMT: |r| = 0.25–0.48, MSIS-29 hand Results items: |r| = 0.27–0.50, EDSS: |r| = 0.25–0.53, cerebellar In total, 76 PwMS and 25 HC were enrolled between functional system: |r| = 0.24–0.47, pyramidal functional November 28, 2016 and May 4, 2018 (end of study). Fol- system: |r| = 0.31–0.46, total brain volume: |r| = 0.24–0.46, lowing the quality control, valid test runs were available T2 lesion volume: |r| = 0.24–0.38, for p < 0.05) are sum- for 69 (91%) PwMS and 18 (72%) HC. Baseline demo- marized in Figure 3. Worse performance captured by spa- graphics and disease characteristics of the assessed study tial features was associated with worse outcomes on population (Table 1) were similar to those previously standard clinical scales and MRI outcomes, showing fair reported by Midaglia et al.22 for the entire study popula- correlations with 9HPT (r = 0.33 to 0.47), oral SDMT tion. PwMS included in the analyses mostly presented (r = 0.25–0.35), MSIS-29 hand items (r = 0.36 to mild disease characteristics with limited upper extremity 0.44), EDSS (r = 0.36 to 0.43), pyramidal functional dysfunction, having a mean EDSS score of 2.43 (range: system (r = 0.32 to 0.42), and total brain volume 0.0–5.5), and a mean 9HPT time of 22.29 sec (SD: 4.15). (r = 0.28–0.39). For most shapes, spatial features also Furthermore, changes observed for the 9HPT, EDSS, and showed mostly fair correlations with cerebellar functional MRI (T2-FLAIR lesion volume) measures between the system score (r = 0.24 to 0.28) and T2 lesion volume baseline and the end of study (week 24) clinical visit were (r = 0.24 to 0.29), for p < 0.05. In the temporal mostly within one standard deviation of the baseline domain, greater variability (CV) in drawing velocity, but assessments with no change over time observed (Fig. S2). not slower mean drawing velocity, was generally related to worse outcomes on all standard clinical scales and MRI measures when considering the complex, round Correlation with standard clinical scales and shapes. For most of these features, correlations with brain MRI measures of MS disease state 9HPT (r = 0.41–0.66), MSIS-29 hand items (r = 0.27– The correlations between digital features obtained for 0.50), EDSS (r = 0.25–0.53) were fair or moderate-to- both hands combined and standard clinical scales as well good in strength, and correlations with oral SDMT as brain MRI measures of MS disease state (9HPT: (r = 0.26 to 0.48), cerebellar function system ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 5 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. Spearman’s 9HPT 1.00 -0.58 0.48 0.54 0.63 0.43 -0.44 0.25 1.00 0.01 Line B2T trace accuracy -0.42 0.34 -0.36 -0.40 -0.24 -0.36 0.38 -0.29 -0.21 0.19 Line T2B trace accuracy -0.33 0.25 -0.37 -0.42 -0.26 -0.42 0.36 -0.21 -0.20 0.09 Square trace accuracy -0.39 0.33 -0.40 -0.38 -0.28 -0.33 0.39 -0.22 -0.15 0.19 Circle trace accuracy -0.38 0.30 -0.40 -0.38 -0.22 -0.37 0.28 -0.19 -0.18 0.20 Figure-of-8 trace accuracy -0.47 0.35 -0.40 -0.36 -0.25 -0.33 0.31 -0.24 -0.28 0.21 Spiral trace accuracy -0.45 0.31 -0.44 -0.40 -0.24 -0.32 0.36 -0.26 -0.25 0.17 Overall mean trace accuracy -0.44 0.35 -0.42 -0.43 -0.28 -0.40 0.38 -0.28 -0.22 0.21 Line B2T mean linear vel. -0.13 0.20 0.21 0.16 -0.11 0.17 0.07 -0.12 -0.10 0.12 Line T2B mean linear vel. -0.20 0.22 0.13 0.05 -0.18 0.07 0.10 -0.13 -0.16 0.13 Square mean linear vel. -0.18 0.22 0.14 0.10 -0.12 0.11 0.08 -0.12 -0.16 0.11 Circle mean linear vel. -0.12 0.17 0.17 0.12 -0.14 0.13 0.06 -0.04 -0.15 0.09 Figure-of-8 mean linear vel. -0.13 0.20 0.17 0.11 -0.15 0.12 0.09 -0.04 -0.11 0.12 Spiral mean linear vel. -0.15 0.16 0.18 0.14 -0.15 0.12 0.07 -0.04 -0.15 0.06 Line B2T CV linear vel. -0.04 0.05 -0.01 0.05 -0.07 0.11 -0.12 0.11 0.01 0.08 Line T2B CV linear vel. 0.10 0.07 0.07 0.13 0.06 0.15 -0.17 0.09 0.17 0.14 Temporal features Square CV linear vel. 0.22 -0.10 0.13 0.16 0.13 0.15 -0.25 0.25 0.19 0.03 Circle CV linear vel. 0.43 -0.39 0.14 0.25 0.36 0.31 -0.46 0.32 0.39 -0.12 Figure-of-8 CV linear vel. 0.57 -0.48 0.27 0.38 0.42 0.44 -0.41 0.38 0.47 -0.20 Spiral CV linear vel. 0.47 -0.33 0.31 0.27 0.39 0.21 -0.29 0.37 0.43 0.02 Circle mean angular vel. -0.13 0.18 0.16 0.13 -0.15 0.12 0.06 -0.04 -0.15 0.09 Spiral CV mean angular vel. -0.16 0.20 0.16 0.12 -0.17 0.10 0.10 -0.07 -0.15 0.07 Circle CV angular vel. 0.43 -0.38 0.11 0.18 0.34 0.22 -0.45 0.32 0.41 -0.13 Spiral CV angular vel. 0.41 -0.26 0.14 0.23 0.28 0.20 -0.19 0.26 0.29 -0.04 Circle mean radial vel. 0.06 -0.20 -0.01 0.03 0.02 -0.01 -0.17 0.04 -0.10 -0.21 Spiral mean radial vel. -0.19 0.22 0.15 0.11 -0.16 0.09 0.11 -0.05 -0.16 0.09 Circle CV radial vel. 0.12 -0.14 -0.05 0.03 0.09 -0.05 -0.22 0.08 0.09 -0.10 Spiral CV radial vel. 0.66 -0.44 0.50 0.53 0.47 0.46 -0.40 0.31 0.48 -0.04 0.21 -0.13 0.29 0.23 0.11 0.21 -0.23 0.28 0.10 -0.01 Line B2T trace celerity -0.32 0.39 0.09 -0.02 -0.23 0.00 0.20 -0.24 -0.21 0.27 Line T2B trace celerity -0.41 0.46 -0.04 -0.17 -0.31 -0.12 0.31 -0.25 -0.28 0.33 features Square trace celerity -0.34 0.38 0.03 -0.04 -0.23 -0.01 0.24 -0.27 -0.28 0.22 Circle trace celerity -0.32 0.38 0.04 -0.03 -0.23 -0.03 0.18 -0.19 -0.24 0.23 Figure-of-8 trace celerity -0.40 0.44 -0.02 -0.09 -0.27 -0.05 0.29 -0.19 -0.26 0.27 Spiral trace celerity -0.38 0.39 -0.01 -0.08 -0.30 -0.06 0.29 -0.23 -0.26 0.22 Overall mean trace celerity -0.38 0.42 0.03 -0.09 -0.27 -0.05 0.28 -0.26 -0.30 0.29 EDSS pyramidal FS Total brain vol. T2-FLAIR lesion vol. 9HPT | Oral SDMT EDSS cerebellar FS Oral SDMT | 9HPT Oral SDMT MSIS hand EDSS 9HPT -1.00 -0.75 -0.50 -0.25 0.00 0.25 0.50 0.75 1.00 Figure 3. Spearman’s rank correlations and partial Spearman’s correlations of the DaS Test (for both hands combined) with standard clinical and MRI measures in PwMS. Correlation coefficients are shown. Statistically significant positive correlations are highlighted in red, and statistically significant negative correlations are in blue. Partial correlations included correlations between the DaS Test and 9HPT after adjusting for oral SDMT (9HPT | oral SDMT) and between the DaS Test and oral SDMT after adjusting for 9HPT (Oral SDMT | 9HPT). The strengths of the correlations were either fair (|r| = 0.25–0.49) or moderate-to-good (|r| = 0.50–0.75). Higher values indicated better performance on features assessing trace accuracy, mean velocity, and trace celerity. Lower values indicated better performance on features assessing CV velocity and dwell time ratio. 9HPT, Nine-Hole Peg Test; B2T, bottom to top; CV, coefficient of variation; DaS, Draw a Shape; EDSS, Expanded Disability Status Scale; FLAIR, fluid-attenuated inversion recovery; FS, functional system; MSIS, Multiple Sclerosis Impact Scale; PwMS, people with multiple sclerosis; SDMT, Symbol Digit Modalities Test; T2, T2-weighted; T2B, top to bottom; vel., drawing velocity. 6 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results (r = 0.28–0.47), pyramidal functional system (r = 0.31– velocity and increased variability in linear velocity (all 0.46), total brain volume (r = 0.25 to 0.46), and T2 p < 0.05 except for mean linear velocity on the circle). lesion volume (r = 0.25–0.38) reached fair strength, for PwMS also performed worse than HC on spatiotemporal p < 0.05. Worse spatiotemporal performance was associ- features on both simple, linear shapes, and complex, ated with worse outcomes on the 9HPT (r = 0.32 to round shapes (all p < 0.05 except line top to bottom). –0.41) and oral SDMT (r = 0.38–0.46) and, on most Additionally, statistically significant differences were shapes, with cerebellar functional system score (r = 0.27 observed between PwMS-Abnormal and either HC or to 0.31), total brain volume (r = 0.24–0.31), and T2 PwMS-Normal on features of all three categories (spatial, lesion volume (r = 0.25 to 0.27), for p < 0.05. These spatiotemporal, and temporal). correlations were all mostly fair in strength. The analysis of shape-specific features revealed that Few differences between shape-specific features were PwMS drew the spiral with slower angular and radial noted. While CV angular velocity on both the circle and velocity than HC (both p < 0.05). Drawings of the spiral spiral correlated with 9HPT (r = 0.43 and r = 0.41, by PwMS-Abnormal were also characterized by increased respectively), cerebellar functional system score (r = 0.34 variability in angular velocity (compared with PwMS- and r = 0.28, respectively), and T2 lesion volume Normal; p < 0.05) and by increased variability in radial (r = 0.32 and r = 0.26, respectively), CV radial velocity velocity (compared with either HC or PwMS-Normal; on the spiral, but not on the circle, showed moderate-to- both p < 0.001). Similarly on the circle, mean angular good correlations with the 9HPT (r = 0.66), MSIS-29 velocity differentiated between PwMS-Abnormal and hand items (r = 0.50), and EDSS (r = 0.53), and fair cor- either HC (p < 0.001) or PwMS-Normal (p < 0.01), while relations with all other clinical scales and MRI outcome variability in angular velocity differentiated between (oral SDMT: r = 0.44, cerebellar functional system PwMS and HC (p < 0.01). Dwell time ratio did not dif- score: r = 0.47, pyramidal functional system score: ferentiate between groups (all p > 0.05). r = 0.46, total brain volume: r = 0.40, T2 lesion volume: Of note, CV linear velocity on the spiral (p < 0.01) and r = 0.31). Furthermore, a higher square dwell time ratio CV angular velocity on the circle (p < 0.05) were both was associated with worse outcomes on the MSIS-29 hand increased in PwMS-Normal versus HC despite PwMS- items (r = 0.29; fair strength) and with larger T2 lesion Normal having no apparent upper extremity function volume (r = 0.28; fair strength). impairment as measured by the 9HPT. Having observed The partial correlation analysis revealed that correla- the good performance of these variability-based velocity tions with 9HPT remained statistically significant after features, we further visualized them by comparing the adjusting for oral SDMT for CV linear velocity on the representative tests from people with increasing levels of round shapes (r = 0.39–0.47), CV angular velocity on the impairment. These visualizations depict that increasing circle (r = 0.41), and CV radial velocity on the spiral levels of impairment were associated with increasing vari- (r = 0.48). However, the correlations between these fea- ability in linear velocity on the spiral (Fig. 4), angular tures and oral SDMT were no longer statistically signifi- velocity on the circle (Fig. 5), and radial velocity on the cant after adjusting for 9HPT (all p > 0.05). spiral (Fig. S3). Furthermore, the circle angular velocity In an analysis of individual hand performance was characterized by both high-speed and low-speed (Table S1), results were predominantly comparable for drawing sections. While spiral radial velocity showed a spatial and temporal features. However, for more complex similar pattern, the interpretation of these features differs spatiotemporal features, correlations were associated with as described in the Discussion. hand dominance. Correlations between these features with The ability of the temporal features to differentiate cerebellar functional system score, brain volume, or lesion PwMS from HC was similar for the individual hands and volume were stronger on the dominant hand. By compar- both hands combined. However, the ability of spatial fea- ison, correlations between these spatiotemporal features tures to differentiate between PwMS-Abnormal and either and 9HPT were generally stronger on the nondominant HC or PwMS-Normal tended to be greater on the non- hand. dominant hand than on the dominant hand, particularly on the simpler, linear shapes (Table S2). Ability to differentiate between HC and PwMS Discussion The comparison between PwMS, PwMS-Abnormal, and Our results suggest that the remotely performed, smart- PwMS-Normal subgroups, and HC for both hands com- phone sensor-based DaS Test objectively assesses upper bined are summarized in Table 2. Compared with HC, extremity function and may help accurately detect MS- PwMS drew the complex, round shapes with slower linear related functional impairment and reduce patient burden. ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 7 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. Table 2. Descriptive statistics by subject group (both hands combined). Mean  SD p1 PwMS- PwMS- HC vs. HC vs. PwMS Normal Abnormal HC vs. PwMS- PwMS- PwMS-Normal vs. Feature HC (n = 18) (n = 69) (n = 51) (n = 18) PwMS Normal Abnormal PwMS-Abnormal Spatial features Trace accuracy Line bottom to 0.93  0.08 0.90  0.10 0.92  0.09 0.85  0.10 0.457 0.924 0.014 0.007 top Line top to 0.88  0.05 0.85  0.10 0.87  0.09 0.79  0.10 0.258 0.848 0.003 0.006 bottom Square 0.92  0.05 0.89  0.09 0.91  0.09 0.86  0.08 0.457 0.924 0.043 0.028 Circle 0.93  0.06 0.90  0.10 0.91  0.09 0.86  0.10 0.396 0.870 0.029 0.022 Figure-of-8 0.93  0.07 0.91  0.10 0.93  0.09 0.85  0.10 0.463 0.891 0.011 0.005 Spiral 0.89  0.06 0.86  0.09 0.88  0.08 0.81  0.10 0.340 0.924 0.008 0.008 Mean overall 0.89  0.06 0.86  0.09 0.88  0.09 0.81  0.09 0.285 0.827 0.007 0.007 trace accuracy Temporal features Mean linear velocity, mm/sec Line bottom to 49.5  16.4 44.0  32.3 47.9  34.8 33.0  20.9 0.088 0.280 0.008 0.016 top Line top to 48.9  14.6 45.3  29.2 49.1  30.1 34.3  24.0 0.180 0.652 0.003 0.007 bottom Square 50.4  7.3 44.0  19.1 47.3  19.4 34.7  15.3 0.031 0.163 0.001 0.005 Circle 42.0  8.5 37.9  24.2 41.1  25.7 28.7  16.7 0.059 0.274 0.002 0.011 Figure-of-8 34.4  7.1 29.5  20.7 32.7  22.1 20.3  12.3 0.015 0.140 <0.001 0.003 Spiral 40.1  7.5 36.1  22.2 39.4  23.4 26.7  15.1 0.032 0.234 <0.001 0.003 CV linear velocity Line bottom to 0.46  0.06 0.49  0.08 0.49  0.08 0.48  0.08 0.330 0.318 0.527 0.816 top Line top to 0.49  0.07 0.51  0.09 0.51  0.09 0.53  0.08 0.489 0.623 0.343 0.682 bottom Square 0.53  0.05 0.55  0.09 0.54  0.08 0.60  0.11 0.438 0.827 0.066 0.048 Circle 0.33  0.02 0.36  0.04 0.34  0.03 0.40  0.05 0.004 0.065 <0.001 <0.001 Figure-of-8 0.35  0.02 0.37  0.04 0.36  0.03 0.42  0.05 0.027 0.374 <0.001 <0.001 Spiral 0.35  0.03 0.39  0.05 0.38  0.03 0.44  0.07 <0.001 0.009 <0.001 <0.001 Mean angular velocity, rad/sec Circle 1.55  0.33 1.40  0.92 1.53  0.98 1.05  0.62 0.058 0.299 <0.001 0.007 Spiral 2.44  0.45 2.20  1.34 2.41  1.42 1.60  0.88 0.024 0.204 <0.001 0.002 CV angular velocity Circle 0.32  0.02 0.36  0.05 0.34  0.03 0.40  0.06 0.002 0.043 <0.001 <0.001 Spiral 0.55  0.04 0.54  0.06 0.53  0.05 0.57  0.07 0.593 0.190 0.155 0.010 Mean radial velocity, mm/sec Circle 0.10  0.25 0.14  0.30 0.14  0.33 0.14  0.16 0.645 0.604 0.849 0.692 Spiral 3.53  0.59 3.15  1.78 3.46  1.87 2.28  1.13 0.024 0.229 <0.001 <0.001 CV radial velocity Circle 9.48  8.32 8.62  9.36 8.06  9.58 10.20  8.78 0.867 0.733 0.776 0.512 Spiral 1.96  0.19 2.13  0.45 1.97  0.29 2.56  0.55 0.300 0.672 <0.001 <0.001 Dwell time ratio Square 0.38  0.04 0.38  0.06 0.37  0.05 0.40  0.07 0.652 0.382 0.506 0.204 Spatiotemporal features (Continued) 8 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results Table 2 Continued. Mean  SD p1 PwMS- PwMS- HC vs. HC vs. PwMS Normal Abnormal HC vs. PwMS- PwMS- PwMS-Normal vs. Feature HC (n = 18) (n = 69) (n = 51) (n = 18) PwMS Normal Abnormal PwMS-Abnormal Trace celerity, 1/sec Line bottom to 0.66  0.17 0.56  0.30 0.62  0.31 0.38  0.18 0.014 0.129 <0.001 <0.001 top Line top to 0.57  0.13 0.51  0.27 0.58  0.27 0.31  0.16 0.078 0.642 <0.001 <0.001 bottom Square 0.22  0.03 0.18  0.05 0.20  0.05 0.14  0.04 0.004 0.058 <0.001 <0.001 Circle 0.22  0.03 0.19  0.06 0.21  0.06 0.14  0.05 0.007 0.095 <0.001 <0.001 Figure-of-8 0.23  0.03 0.19  0.07 0.22  0.07 0.13  0.05 0.003 0.061 <0.001 <0.001 Spiral 0.13  0.02 0.11  0.04 0.13  0.04 0.08  0.03 0.007 0.123 <0.001 <0.001 Mean overall 0.36  0.06 0.31  0.12 0.34  0.12 0.22  0.08 0.013 0.155 <0.001 <0.001 trace celerity Higher values indicated better performance on features assessing trace accuracy, mean velocity, and trace celerity. Lower values indicated better performance on features assessing CV velocity and dwell time ratio. Statistically significant p values (<0.05) are in bold. CV, coefficient of variation; HC, healthy control; PwMS, people with multiple sclerosis. 1 Mann–Whitney U test. The strengths of the DaS Test include its multidimen- pyramidal function system was associated with spatial test sional feature characteristics, capturing spatial, temporal, features across all shapes, including simple, linear shapes. and spatiotemporal information across six increasingly Contrastingly, cerebellar ataxia, measured by the cerebel- complex shapes. We previously showed that such multidi- lar functional system score of the EDSS, was generally mensionality is necessary to predict 9HPT time with a associated with spatial, temporal, and spatiotemporal fea- mean absolute error comparable to the variability of the tures. Previous studies have linked cerebellar ataxia to actual 9HPT times.14 Here, we identified features that impaired motor coordination and increased variability in showed fair, reaching moderate-to-good, correlations with motor performance.33,34 standard clinical scales and brain MRI measures (normal- Correlations with standard clinical scales and brain ized brain volume and total lesion volume). MRI measures tended to be stronger for variability-based Overall mean trace celerity was designed to measure drawing velocity features than for mean drawing velocity. how accurately and quickly PwMS can draw the six In PwMS, keeping a low variability in drawing velocity shapes. This feature showed fair correlations with the features could rely more strongly on sensory feedback 9HPT and differentiated both between HC and PwMS, compared with the actual drawing velocity, which could and between PwMS-Normal and PwMS-Abnormal. Simi- potentially explicate the stronger association of the vari- lar results were obtained when assessing trace celerity on ability with impairment.35 the individual shapes, thereby corroborating these find- Moreover, drawing variability features have been pro- ings. This suggests that overall mean trace celerity fulfills posed as a measure of fine motor control.35 For a perfect two important criteria of an ideal DaS feature: (1) at least spiral, the expected radial drawing velocity would be a a fair agreement with standard clinical scales of upper positive value, as radial velocity measures the movement extremity function and (2) the ability to differentiate outward from the center. Hence, negative values indicate between groups with different levels of upper extremity an acceleration in the wrong direction, and large positive impairment. Additionally, we previously reported good values indicate rapid movement outwards. Thus, minute test–retest reliability for this feature.16 corrections that happen due to deviations from the refer- The strongest associations with MS disease burden as ence trace of the spiral, for example resulting from subtle measured by brain MRI measures and standard clinical degradation of fine motor control, will result in increased scales of upper extremity function, cognitive information variability in radial drawing velocity. processing speed, and overall disease severity were The notion that variability-based features capture fine observed for the spatial test features for all shapes and the motor control is further supported by the partial correla- variability-based temporal features for the most complex tions analysis. The statistically significant correlations with shapes (circle, figure-of-8, and spiral). Notably, the 9HPT after adjusting for oral SDMT and lack of ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 9 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. PwMS-Normal PwMS-Abnormal Male, 46 years Male, 38 years HC Dominant hand Dominant hand Male, 29 years Diagnosis: RRMS Diagnosis: RRMS Dominant hand Avg. baseline 9HPT: 20.7 s Avg. baseline 9HPT: 33.45 s Avg. baseline 9HPT: 17.1 s EDSS: 1.5 EDSS: 3.0 Spiral trace accuracy: 0.79 Spiral trace accuracy: 0.74 Spiral trace accuracy: 0.76 Spiral trace celerity: 0.18 Spiral trace celerity: 0.14 Spiral trace celerity: 0.08 Spiral CV linear velocity: 0.25 Spiral CV linear velocity: 0.43 Spiral CV linear velocity: 0.56 0 120 0 120 0 120 20 100 20 100 20 100 80 80 80 40 40 40 mm/s mm/s mm/s mm mm mm 60 60 60 60 60 60 40 40 40 80 80 80 20 20 20 100 100 100 0 0 0 0 10 20 30 40 50 60 0 10 20 30 40 50 60 0 10 20 30 40 50 60 mm mm mm 100 40 50 80 40 30 60 Counts Counts Counts 30 20 40 20 10 20 10 0 0 0 0 50 100 0 50 100 0 50 100 Linear velocity (mm/s) Linear velocity (mm/s) Linear velocity (mm/s) 120 120 120 100 100 Linear velocity (mm/s) Linear velocity (mm/s) 100 Linear velocity (mm/s) 80 80 80 60 60 60 40 40 40 20 20 20 0 0 0 14:23:59 14:24:01 14:24:04 17:20:39 17:20:41 17:20:44 15:26:11 15:26:15 15:26:20 Time Time Time Figure 4. Increased variability in spiral linear velocity with higher levels of MS-related impairment. The top row shows the drawn traces of the spi- ral shape for an HC (left), a PwMS-Normal (middle), and a PwMS-Abnormal (right), with the color bar indicating the linear velocity. The corre- sponding histograms and time series of the spiral linear velocity are shown in the middle and bottom rows. The x-axis in the bottom row displays the time of day (h:min:sec). 9HPT, Nine-Hole Peg Test; avg., average; CV, coefficient of variation; EDSS, Expanded Disability Status Scale; HC, healthy control; MS, multiple sclerosis; PwMS, people with multiple sclerosis; RRMS, relapsing–remitting multiple sclerosis; s, seconds. 10 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results PwMS-Normal PwMS-Abnormal Male, 30 years Female, 32 years HC Dominant hand Dominant hand Male, 29 years Diagnosis: RRMS Diagnosis: PPMS Dominant hand Avg. baseline 9HPT: 18.2 s Avg. baseline 9HPT: 25.9 s Avg. baseline 9HPT: 17.1 s EDSS: 2.0 EDSS: 4.5 Circle trace accuracy: 0.99 Circle trace accuracy: 0.83 Circle trace accuracy: 0.23 Circle trace celerity: 0.41 Circle trace celerity: 0.34 Circle trace celerity: 0.09 Circle CV angular velocity: 0.25 Circle CV angular velocity: 0.40 Circle CV angular velocity: 0.56 0 6 0 6 0 6 20 5 20 5 20 5 4 4 4 40 40 40 rad/s rad/s rad/s mm mm mm 3 3 3 60 60 60 2 2 2 80 80 80 1 1 1 100 100 100 0 0 0 0 10 20 30 40 50 60 0 10 20 30 40 50 60 0 10 20 30 40 50 60 mm mm mm 25 30 40 20 20 30 15 Counts Counts Counts 20 10 10 10 5 0 0 0 0 2 4 6 0 2 4 6 0 2 4 6 Angular velocity (rad/s) Angular velocity (rad/s) Angular velocity (rad/s) 6 6 6 Angular velocity (rad/s) Angular velocity (rad/s) Angular velocity (rad/s) 5 5 5 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0 14:40:51 14:40:52 14:40:53 20:18:43 20:18:44 20:18:45 20:14:44 20:14:45 20:14:46 Time Time Time Figure 5. Increased variability in circle angular velocity with higher levels of MS-related impairment. The top row shows the drawn traces of the circle shape for a HC (left), a PwMS-Normal (middle), and a PwMS-Abnormal (right), with the color bar indicating the angular velocity. The corre- sponding histograms and time series of the circle angular velocity are shown in the middle and bottom rows. The x-axis in the bottom row dis- plays the time of day (h:min:sec). 9HPT, Nine-Hole Peg Test; avg., average; CV, coefficient of variation; EDSS, Expanded Disability Status Scale; HC, healthy control; MS, multiple sclerosis; PPMS, primary progressive multiple sclerosis; PwMS, people with multiple sclerosis; RRMS, relapsing– remitting multiple sclerosis; s, seconds. ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 11 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. correlation with oral SDMT after adjusting for 9HPT sug- A few limitations and potential future study objectives gest that CV linear velocity of round shapes, CV angular are noted. First, PwMS enrolled in this study had mild, velocity on the circle, and CV radial velocity on the spiral clinically stable disease in terms of both upper extremity are primarily driven by a motor component. dysfunction and overall disability, limiting the generaliz- The DaS Test differentiated PwMS from HC, with tem- ability of the results. While the results are promising, poral and spatiotemporal features showing a stronger abil- including more PwMS-Abnormal participants could help ity to differentiate PwMS from HC compared with spatial further validate the DaS Test. Second, our analyses were features. PwMS drew with increased drawing velocity vari- cross-sectional given the relatively short study duration of ability and with reduced velocity and celerity than HC. 24 weeks and the stability of the standard clinical scales Two features, CV linear velocity on the spiral and CV and MRI outcomes over the study period. This limited angular velocity on the circle, demonstrated the capacity the ability to assess the DaS Test’s sensitivity with regards to differentiate HC from PwMS with apparent normal to disease worsening or the utility to detect MS progres- upper extremity function (PwMS-Normal). However, the sion. Additionally, potentially relevant clinical correlates, lack of age- and sex-controlled matching in this study lim- for example, monofilament and grip strength testing were ited our ability to adequately compare the HC and PwMS not investigated in this study. Future and ongoing studies, subgroups. We, therefore, used a global threshold to clas- including the CONSONANCE and Floodlight MS – sify PwMS as normal or abnormal with respect to their TONiC studies, shall clarify the use of the DaS Test in a upper extremity function. This threshold was derived from larger, broader patient population (including people with a normative population with a similar age and sex distri- more advanced disease), examine the test performance bution as our PwMS cohort.3 An alternative approach to over time, derive composite measures from data, and out- classify PwMS is to apply adaptive thresholds based on line the mapping of the features to a specific motor and their age and sex. This methodology, however, is better sensory symptoms. Foundational work on establishing a suited for larger studies that enroll more diverse popula- composite, functional upper extremity score based on tions. Larger ongoing and future studies, including the data derived from the DaS Test and other Floodlight tests CONSONANCE study (NCT03523858) and FloodlightTM has already been undertaken, showing good correlations MS – TONiC (ISRCTN11088592), will, therefore, explore with the 9HPT.37 These studies will also help to define the use of such adaptive thresholds. normative bands for different levels of impairment and We also noted differences in both analyses (correlations support the development of a digital upper extremity with standard clinical scales and brain MRI measures, and function score for easier interpretation of the perfor- ability to differentiate between HC and PwMS) between mance on the DaS Test in clinical practice. Such a score individual shapes when considering the variability-based could also be used as an endpoint in clinical research, for velocity features. Complex, round shapes outperformed example, for drug development. Studies comprehensively simple, linear shapes in both analyses. We suggest two assessing the correlation of multiple regional MRI out- possible explanations. First, it may become increasingly comes with Floodlight test data and standard clinical challenging for a demyelinated brain to quickly assemble measures are also underway. the required motor plan as the complexity of the shapes In conclusion, the DaS Test provides an objective and increases. Second, complex, round shapes are character- self-administered assessment of upper extremity function, ized by a continuous and smooth change in trajectory, which highlights its potential use in clinical practice and which may be challenging to complete for some PwMS. in clinical trials. We identified specific features derived Drawing them may, therefore, rely more strongly on sen- from the DaS Test that correlated with measures of upper sory feedback. Such feedback has been previously shown extremity dysfunction, the patient’s perspective on the to play a role in the execution of visually guided tasks.36 impact of the disease, and overall disease disability and Although complex shapes demonstrated the greatest sensi- severity, and that differentiated HC from PwMS. Further tivity in this analysis, it is possible that simple shapes may research is warranted in test-specific symptomatology and become more relevant in patient populations with more in characterizing the DaS Test in wider patient popula- significant impairment, especially if the patients struggle tions and in other disease areas. with drawing the complex shapes or have other comor- bidities. Thus, including all shapes may enhance the test’s Author Contributions versatility and comprehensiveness in assessing upper extremity function. Future analyses in a broader patient S. B., C. B., M. B., C. G., and M. L. designed and concep- population, including in patients with progressive disease, tualized the study. J. S. G. and X. M. contributed to par- will provide additional insights into the characteristics of ticipant enrollment and data collection. M. G. statistically each shape. analyzed the data. J. S. G., M. G., F. D., F. L., S. B., C. B., 12 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results X. M., C. G., and M. L. interpreted the data. M. B. Actelion, Alexion, Bayer, Biogen, Celgene, EMD Serono, acquired funding for the study. All authors critically Genzyme, Immunic, MedDay, Merck, Mylan, Nervgen, reviewed and approved the final version of the manu- Novartis, Roche, Sanofi-Genzyme, Teva Pharmaceuticals, script. TG Therapeutics, Excemed, MSIF, and NMSS. J. van Beek was an employee of F. Hoffmann-La Roche Ltd during the completion of the work related to this manuscript Acknowledgments and is now an employee of Biogen (Cambridge, MA), The authors would like to thank all study participants, which is not in any way associated with this study. M. their families, and the investigators who participated in Baker and C. Gossens are employees of and shareholders this trial, including the following coinvestigators who in F. Hoffmann-La Roche Ltd. M. Lindemann is a consul- contributed to data collection: Stephen L. Hauser, MD, tant to F. Hoffmann-La Roche Ltd via Inovigate. from the UCSF Weill Institute of Neurosciences and Department of Neurology, University of California San Data Availability Statement Francisco, San Francisco, CA, USA; Luciana Midaglia, MD, and Patricia Mulero, MD, from the Department of For up-to-date details on Roche’s Global Policy on Shar- Neurology-Neuroimmunology, Centre d’Esclerosi M ulti- ing of Clinical Study Information and how to request ple de Catalunya (Cemcat), Hospital Universitari Vall access to related clinical study documents, see https://go. d’Hebron, Barcelona, Spain. They also thank the follow- roche.com/data_sharing. Request for the data underlying ing colleagues at F. Hoffmann-La Roche Ltd for their this publication requires a detailed, hypothesis-driven sta- contributions and support to the study: Jan Beckmann, tistical analysis plan that is collaboratively developed by Sandro Fritz, Nicholas Pierce Heinemeier, Sven Holm, the requestor and company subject matter experts. Such Timothy Kilchenmann, Lito Kriara, Gregoire Pointeau, requests should be directed to dbm.datarequest@roche.- Cedric Simillion, Jens Schjodt-Eriksen, J€ org Sprengel, and com for consideration. Anonymized records for individual Mattia Zanon. Writing and editorial assistance for this patients across more than one data source external to manuscript was provided by Frank Biegun, MSc, from Roche cannot, and should not, be linked due to a poten- Articulate Science, United Kingdom, and funded by F. tial increase in the risk of patient re-identification. Hoffmann-La Roche Ltd. References Conflicts of Interest 1. Reich DS, Lucchinetti CF, Calabresi PA. Multiple sclerosis. N Engl J Med. 2018;378:169-180. The authors disclosed receipt of the following financial 2. Poser S. Multiple sclerosis: an analysis of 812 cases by support for the research, authorship, and/or publication means of electronic data processing. Schriftenr Neurol. of this article: F. Hoffmann-La Roche Ltd, Basel, Switzer- 1978;20:1-93. land, provided financial support for the study and publi- 3. Erasmus L-P, Sarno S, Albrecht H, Schwecht M, P€ ollmann cation of this manuscript. J. S. Graves has received W, K€ onig N. Measurement of ataxic symptoms with a research support from Biogen, EMD Serono, Novartis, graphic tablet: standard values in controls and validity in and Sanofi; has received speaking honoraria from Bristol multiple sclerosis patients. J Neurosci Methods. Myers Squibb, Bayer, and Alexion; and served on advisory 2001;108:25-37. boards for Genentech and Bayer. M. Ganzetti is a con- 4. Yozbatiran N, Baskurt F, Baskurt Z, et al. Motor tractor for F. Hoffmann-La Roche Ltd. F. Dondelinger assessment of upper extremity function and its relation was an employee of and is a shareholder in F. Hoffmann- with fatigue, cognitive function and quality of life in La Roche Ltd; he is currently employed by Novartis Insti- multiple sclerosis patients. J Neurol Sci. 2006;246:117-122. tutes for Biomedical Research. F. Lipsmeier is an 5. Johansson S, Ytterberg C, Claesson IM, et al. High employee of F. Hoffmann-La Roche Ltd. S. Belachew was concurrent presence of disability in multiple sclerosis. an employee of F. Hoffmann-La Roche Ltd during the Associations with perceived health. J Neurol. 2007;254:767- completion of the work related to this manuscript and is 773. now an employee of Biogen (Cambridge, MA), which is 6. Kister I, Bacon TE, Chamot E, et al. Natural history of not in any way associated with this study. C. Bernasconi multiple sclerosis symptoms. Int J MS Care. 2013;15:146- is a contractor for F. Hoffmann-La Roche Ltd. X. Montal- 158. ban has received speaking honoraria and travel expenses 7. Bertoni R, Lamers I, Chen CC, Feys P, Cattaneo D. for participation in scientific meetings; has been a steering Unilateral and bilateral upper limb dysfunction at body committee member of clinical trials or participated in functions, activity and participation levels in people with advisory boards of clinical trials in the past years with multiple sclerosis. Mult Scler. 2015;21:1566-1574. ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 13 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Draw a Shape Test Validity – Preliminary Results J. S. Graves et al. 8. Poole JL, Nakamoto T, McNulty T, et al. Dexterity, visual 22. Midaglia L, Mulero P, Montalban X, et al. Adherence and perception, and activities of daily living in persons with satisfaction of smartphone- and smartwatch-based remote multiple sclerosis. Occup Ther Health Care. 2010;24:159-170. active testing and passive monitoring in people with 9. Lam KH, Meijer KA, Loonstra FC, et al. Real-world multiple sclerosis: nonrandomized interventional feasibility keystroke dynamics are a potentially valid biomarker for study. J Med Internet Res. 2019;21:e14863. clinical disability in multiple sclerosis. Mult Scler. 23. Koo TK, Li MY. A guideline of selecting and reporting 2020;27:1421-1431. intraclass correlation coefficients for reliability research. J 10. Bisio A, Pedulla L, Bonzano L, Tacchino A, Brichetto G, Chiropr Med. 2016;15(2):155-163. Bove M. The kinematics of handwriting movements as 24. Clinch S, McDougall F, Barrett A, et al. A patient-focused expression of cognitive and sensorimotor impairments in qualitative study to support content validity of digital people with multiple sclerosis. Sci Rep. 2017;7:17730. performance assessments in MS. Mult Scler J. 2021;27 11. Rae-Grant A, Bennett A, Sanders AE, Phipps M, Cheng E, (2S):134-740 (P309). Bever C. Quality improvement in neurology: multiple 25. Ram N, Brinberg M, Pincus AL, Conroy DE. The sclerosis quality measures: executive summary. Neurology. questionable ecological validity of ecological momentary 2015;85:1904-1908. assessment: considerations for design and analysis. Res 12. Feys P, Lamers I, Francis G, et al. The Nine-Hole Peg Test Hum Dev. 2017;14(3):253-270. as a manual dexterity performance measure for multiple 26. Polman CH, Reingold SC, Banwell B, et al. Diagnostic sclerosis. Mult Scler. 2017;23:711-720. criteria for multiple sclerosis: 2010 revisions to the 13. Sbardella E, Petsas N, Tona F, et al. Assessing the McDonald criteria. Ann Neurol. 2011;69:292-302. correlation between grey and white matter damage with 27. Kurtzke JF. Rating neurologic impairment in multiple motor and cognitive impairment in multiple sclerosis sclerosis: an Expanded Disability Status Scale (EDSS). patients. PLoS One. 2013;8(5):e63250. Neurology. 1983;33:1444-1452. 14. Creagh A, Simillion C, Scotland A, et al. Smartphone- 28. Hobart J, Lamping D, Fitzpatrick R, Riazi A, Thompson based remote assessment of upper extremity function for A. The Multiple Sclerosis Impact Scale (MSIS-29): a new multiple sclerosis using the Draw a Shape Test. Physiol patient-based outcome measure. Brain. 2001;124:962-973. Meas. 2020;41:054002. 29. Koh CL, Lu WS, Chen HC, Hsueh IP, Hsieh JJ, Hsieh CL. 15. DelMastro HM, Ruiz JA, Gromisch ES, et al. Test-retest reliability and practice effect of the oral-format Quantification characteristics of digital spiral analysis for Symbol Digit Modalities Test in patients with stroke. Arch understanding the relationship among tremor and clinical Clin Neuropsychol. 2011;26(4):356-363. measures in persons with multiple sclerosis. J Neurosci 30. Holm SP, Wolfer AM, Pointeau GHS, Lipsmeier F, Methods. 2018;307:254-259. Lindemann M. Practice effects in performance outcome 16. Montalban X, Graves J, Midaglia L, et al. A smartphone measures in patients living with neurologic disorders — a sensor-based digital outcome assessment of multiple systematic review. Heliyon. 2022;8(8):E10259. sclerosis. Mult Scler. 2022;28(4):654-664. 31. Portney LG, Watkins MP. Foundations of Clinical 17. Messan KS, Pham L, Harris T, et al. Assessment of Research: Applications to Practice. Pearson/Prentice Hall; smartphone-based spiral tracing in multiple sclerosis 2009. reveals intra-individual reproducibility as a major 32. Huber PJ. Robust Statistics. John Wiley and Sons, Inc.; determinant of the clinical utility of the digital test. Front 1981. Med Technol. 2022;3:17. 33. Bastian AJ. Learning to predict the future: the cerebellum 18. Taylor K, Postuma R, Boess F, et al. A comprehensive adapts feedforward movement control. Curr Opin digital biomarker active testing and passive monitoring Neurobiol. 2006;16:645-649. suite for the remote and frequent assessment of motor 34. Wilkins A. Cerebellar dysfunction in multiple sclerosis. symptom progression in Parkinson’s disease. Mov Disord. Front Neurol. 2017;8:312. 2018;33(suppl 2):Abstract 1427. 35. Longstaff MG, Heath RA. Spiral drawing performance as 19. Aghanavesi S, Nyholm D, Senek M, Bergquist F, Memedi M. A an indicator of fine motor function in people with smartphone-based system to quantify dexterity in Parkinson’s multiple sclerosis. Hum Mov Sci. 2006;25:474-491. disease patients. Inform Med Unlocked. 2017;9:11-17. 36. Feys P, Helsen WF, Liu X, et al. Effect of visual 20. Tortelli R, Simillion C, Lipsmeier F, et al. The Digital-HD information on step-tracking movements in patients with study: smartphone-based remote testing to assess cognitive intention tremor due to multiple sclerosis. Mult Scler. and motor symptoms in Huntington’s disease. Neurology. 2003;9:492-502. 2020;94(suppl 15):Abstract 1816. 37. Rukina D, Bogaarts G, Dondelinger F, et al. Novel 21. Haubenberger D, Kalowitz D, Nahab FB, et al. Validation smartphone sensor-based scores for remote measurement of digital spiral analysis as outcome parameter for clinical of gait and hand function impairment in people with MS. trials in essential tremor. Mov Disord. 2011;26:2073-2080. Mult Scler J. 2021;27(2S):134-740 (P306). 14 ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 23289503, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/acn3.51705 by Readcube (Labtiva Inc.), Wiley Online Library on [30/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License J. S. Graves et al. Draw a Shape Test Validity – Preliminary Results Figure S3. Increased variability in spiral radial velocity Supporting Information with higher levels of MS-related impairment. Additional supporting information may be found online Table S1. Spearman’s rank correlations with standard in the Supporting Information section at the end of the clinical scales and MRI measures in PwMS by handed- article. ness. Table S2. Descriptive statistics by subject group and Figure S1. The Draw a Shape Test features. handedness. Figure S2. Clinical and MRI measures from baseline to week 24 (end of study). ª 2022 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals LLC on behalf of American Neurological Association. 15