Journal of Neurology (2020) 267:3541–3554 https://doi.org/10.1007/s00415-020-10023-1 ORIGINAL COMMUNICATION Reduced accuracy of MRI deep grey matter segmentation in multiple sclerosis: an evaluation of four automated methods against manual reference segmentations in a multi‑center cohort Alexandra de Sitter1 · Tom Verhoeven1 · Jessica Burggraaff2 · Yaou Liu1 · Jorge Simoes1 · Serena Ruggieri3,4 · Miklos Palotai5 · Iman Brouwer1 · Adriaan Versteeg1 · Viktor Wottschel1 · Stefan Ropele6 · Mara A. Rocca7,8 · Claudio Gasperini4 · Antonio Gallo9 · Marios C. Yiannakas10 · Alex Rovira11 · Christian Enzinger12 · Massimo Filippi7,8,13,14 · Nicola De Stefano15 · Ludwig Kappos16 · Jette L. Frederiksen17 · Bernard M. J. Uitdehaag2 · Frederik Barkhof1,18 · Charles R. G. Guttmann5 · Hugo Vrenken1 · the MAGNIMS Study Group Received: 4 May 2020 / Revised: 22 June 2020 / Accepted: 23 June 2020 / Published online: 3 July 2020 © The Author(s) 2020 Abstract Background Deep grey matter (DGM) atrophy in multiple sclerosis (MS) and its relation to cognitive and clinical decline requires accurate measurements. MS pathology may deteriorate the performance of automated segmentation methods. Accu- racy of DGM segmentation methods is compared between MS and controls, and the relation of performance with lesions and atrophy is studied. Methods On images of 21 MS subjects and 11 controls, three raters manually outlined caudate nucleus, putamen and thala- mus; outlines were combined by majority voting. FSL-FIRST, FreeSurfer, Geodesic Information Flow and volBrain were evaluated. Performance was evaluated volumetrically (intra-class correlation coefficient (ICC)) and spatially (Dice similar- ity coefficient (DSC)). Spearman’s correlations of DSC with global and local lesion volume, structure of interest volume (ROIV), and normalized brain volume (NBV) were assessed. Results ICC with manual volumes was mostly good and spatial agreement was high. MS exhibited significantly lower DSC than controls for thalamus and putamen. For some combinations of structure and method, DSC correlated negatively with lesion volume or positively with NBV or ROIV. Lesion-filling did not substantially change segmentations. Conclusions Automated methods have impaired performance in patients. Performance generally deteriorated with higher lesion volume and lower NBV and ROIV, suggesting that these may contribute to the impaired performance. Keywords Multiple sclerosis · Deep grey matter · Atrophy · Automated segmentation methods Introduction impairment [1–4]. Accurate segmentations of these struc- tures from structural MRI are key to understanding these In multiple sclerosis (MS), atrophy of deep grey matter atrophic processes and their role in MS. (DGM) structures like the caudate nucleus (caudate), puta- However, it is unclear whether DGM segmentation men and thalamus is associated with cognitive and clinical using state-of-the-art automated methods is as accurate in MS cases as in healthy controls. Since studies have shown that white matter (WM) lesions and atrophy could affect Alexandra de Sitter and Tom Verhoeven contributed equally. measures such as whole-brain grey matter (GM) volume, it could be expected that such pathology also affects DGM Electronic supplementary material The online version of this segmentation [1, 5–9]. article (https​://doi.org/10.1007/s0041​5-020-10023​-1) contains A direct comparison of automated methods to expert supplementary material, which is available to authorized users. manual (reference) segmentation was performed by Dera- * Alexandra de Sitter khshan et al. (2010) in a small dataset containing 3 slices A.deSitter@amsterdamumc.nl each of 3 MS patients [1]. Although that paper provided Extended author information available on the last page of the article insights into the spatial overlap between automated and 13 Vol.:(0123456789) 3542 Journal of Neurology (2020) 267:3541–3554 manual segmentations, with 3 slices per subject no volu- Acquisition metric analysis was possible. Moreover, the small number of subjects did not allow any analysis of relations between An overview of acquisition parameters for each site is given segmentation performance and MS-related pathological in Supplementary Tables 1 and 2. Briefly, MRI data were changes. obtained using magnets operating at 3 T for all cases with Therefore, this study quantitatively investigated auto- three vendors (Siemens, Philips and GE). One of the two mated segmentation performance in a whole-brain dataset following imaging protocols was used: (1) 3D T1‐weighted of 32 subjects including MS patients and healthy controls. scan (different pulse sequences for different venders) and Four publicly available segmentation method packages a dual-echo spin echo scan with both 2D T2-weighted and (FSL-FIRST [10], FreeSurfer [11], Geodesic Informa- 2D proton density (PD) weighted; or (2) 3D T1-weighted tion Flow (GIF) [12] and volBrain [13]) were evaluated magnetization prepared rapid gradient echo (MPRAGE) in terms of volumetric and spatial agreement with manual scan and 2D fluid-attenuated inversion recovery (FLAIR) segmentations created by combining manual outlines of T2-weighted fast spin-echo sequence. three trained raters by majority voting. Moreover, the relation of segmentation accuracy with total and regional Manual segmentation of DGM structures lesion load, whole-brain volume, and volume of the struc- ture of interest was assessed to determine possible con- Manual segmentation of three DGM structures was per- founding disease relations factors. formed using the SPINE online environment for collabo- rative research (https​://spine​virtu​allab​.org). The caudate nucleus, putamen and thalamus were all manually seg- mented on the full 3D T1-weighted images in each subject Methods by each rater. Four scans were outlined a second time by each rater to examine intra-rater variability. A summary of Subjects the segmentation protocol is added to the supplementary data (Supplementary Protocol 1). In total 21 MS patients and 11 healthy controls subjects The manual segmentations of the three raters were com- were retrospectively selected from two of the multi-center bined into a reference using majority voting: i.e., a voxel was studies of the MAGNIMS Study Group (www.magni​ classified as part of a structure if at least 2 of the 3 raters ms.eu) [14, 15]. Demographic details of the subjects are assigned it to that structure. listed in Table 1. Subjects had been recruited at nine Euro- pean centers, see Supplementary data A for the centers. The selection of the cases was based on maximizing the Lesion segmentation number of scanners and the number of secondary progres- sive MS (SPMS) and primary progressive MS (PPMS) Lesion segmentation was also performed manually by one cases while considering the workload for the three raters. expert rater, on the FLAIR scan or on the PD scan. The All patients and controls had given informed consent for lesion segmentation was performed using the Medical Image the use of their brain MRI-scans for research within the Processing, Analysis, and Visualization (MIPAV) software original study. environment whereby only lesions of at least three voxels were included. Table 1  Demographics of the subjects Lesion filling Disease Number of Average age Median Aver- status cases (male/ in years ± std EDSS score age DD Lesion-filling is a common pre-processing step in patient female) (range) year ± std scans, in which the intensities of voxels identified as being part of WM lesions are replaced by intensities similar to HC 11 (3/8) 37.6 ± 8.2 n.a n.a normal-appearing white matter. In this study, lesion-filling MS 21 (9/12) 43.2 ± 10.1 3.5 (6.0) 9.5 ± 6.9 was applied using two algorithms: lesion segmentation RRMS 10 (4/6 39.8 ± 8.3 2.3 (2.5) 8.0 ± 9.8 toolbox (LST-LF) [16], and LEAP [8], and both versions of SPMS 5 (3/2) 41.3 ± 8.9 4.0 (6.0) 11.0 ± 5.3 lesion-filled images as well as native images were analyzed. PPMS 6 (2/4) 49.4 ± 10.8 3.5 (4.5) 14.0 ± 4.0 The lesion masks were first co-registered from their original EDSS expended disability status scale, DD disease duration, std PD or FLAIR space to 3D-T1 space using FSL-FLIRT with Standard deviation, HC healthy control, RR relapsing remitting, SP tri-linear interpolation and a threshold of 0.5 because this secondary progressive, PP primary progressive 13 Journal of Neurology (2020) 267:3541–3554 3543 was previously found to provide good results for whole-brain images. GIF captures the local variation in morphology and GM volume measurements [6]. The Supplementary data B in standard space locations. With the use of an iterative geo- provides a description of LST-LF and LEAP. In this study, desic minimization algorithm and the manual labels, more lesion-filling was applied using two algorithms: lesion seg- accurate segmentations are expected [12]. mentation toolbox (LST-LF) [16], and LEAP [8], and both VolBrain, version 1.0 is an online pipeline for volumetric versions of lesion-filled images as well as native images brain analysis The proposed pipeline is based on a library were analyzed. The lesion masks were first co-registered of manually labeled atlas cases to perform the segmenta- from their original PD or FLAIR space to 3D-T1 space using tion process, including subcortical structure segmentation FSL-FLIRT with tri-linear interpolation and a threshold of as proposed by Coupé et al. 2011 [13, 17]. 0.5, based on literature [6]. Second, the lesion was filled on the 3D-T1 weighted image with the use of the lesion Brain volume mask in 3D-T1 space. The Supplementary data B provides a description of the two used lesion filling methods (LST-LF The normalized brain volume (NBV) and brain volume (BV) and LEAP). were measured with SIENAX (part of FSL version 5.0) [18] on the lesion filled data. SIENAX is the cross-sectional pipe- Automatic DGM segmentation method line of the SIENA method [19]. Based on voxel intensities it estimates partial volume fractions of GM, WM and cerebro- Within this study four automatic DGM segmentation meth- spinal fluid (CSF) for each voxel. Volumes of GM and WM ods were assessed; FSL-FIRST (https​://fsl.fmrib​.ox.ac.uk/ were added to obtain BV. SIENAX performs normalization fsl/fslwik​ i/FIRST)​ , FreeSurfer (https:​ //surfer​ .nmr.mgh.harva​ of skull size to MNI space to obtain NBV. rd.edu), volBrain (https​://volBr​ain.upv.es) and GIF (https​:// nifty​web.cs.ucl.ac.uk). Relation with MS pathology FSL-FIRST, version 6.0.1, has previously been described by Patenaude et al. (2011). In short, FSL-FIRST finds the The association of automatic segmentation performance, as most plausible outline based on the observed intensities from measured by Dice similarity coefficient (DSC, see statistical the T1-weighted input image using shape and appearance analyses section), with multiple MS-related disease param- models derived from a large training dataset. Surface meshes eters, i.e. WM lesion load, regional lesion load, normalized of the subcortical structures were converted to boundary cor- brain volume (NBV) and DGM structure volume, was inves- rected voxelwise segmentations [10]. tigated. WM lesion load was determined from the manual FreeSurfer, version 6.0.0. is described on the FreeSurfer- lesion outlines. Regional lesion load was evaluated by meas- Wiki page (https​://surfe​r.nmr.mgh.harva​rd.edu/fswik​i/). In uring the lesion load within a pre-defined distance from the short, labels are assigned to each voxel in the subcortical DGM structure (see Fig. 1). Using the distance transform, region (WM + subcortical GM). From these segmentations, the distance of each voxel to the reference of the structure the binary segmentations for the individual structure were in the specific subject under investigation was calculated. extracted [11]. By thresholding of the subject- and structure-specific dis- Geodesic Information Flow (GIF), versions V2.0, uses tance map and masking with the subject-specific WM mask manually created atlases for segmentation of the input obtained with FreeSurfer, for each case, a “surrounding Fig. 1  Method for lesion load calculation within a set border. a a dis- in T1 space. In grey the lesion border is shown, in red the lesions tance field is created around DGM structure, in this case the caudate without overlap with the border and in yellow the lesions with an nucleus. b Distance is set around the DGM structure, seen in grey. c overlap in the border An overlay is created between the DGM border and the lesion mask 13 3544 Journal of Neurology (2020) 267:3541–3554 WM border” region at distances of 0–10 mm was defined. patients. To assess the effect of lesion-filling, two-way Region-specific lesion volumes were obtained by masking ANOVA analysis was performed comparing DSC of native the WM lesion mask in 3D-T1 space with these WM sur- images separately with those from each of the two lesion- rounding WM border. filling methods (LEAP and LST). The relation of disease pathology with spatial perfor- mance was examined using Spearman’s correlation coef- Statistical analyses ficient, in which 0.0 <|r|< 0.2 was considered a weak cor- relation, 0.2 ≤|r|< 0.5 a moderate correlation and |r|≥ 0.5 a Intra-rater agreement was measured between the first and strong correlation [22]. second manual segmentation of the structures with the Dice For all statistical analysis P-values < 0.05 were consid- similarity coefficient (DSC) [20]: ered statistically significant. 2(A ∩ B) DSC(A, B) = A+B Results with A and B defined as the segmentations and where ∩ is the intersection of the two segmentations. Manual segmentation Volumetric analysis was done by comparing the volume of the reference with volumes of the automated segmenta- Volumes of the reference DGM structures are listed in tion. Moreover, the intra-class correlation coefficient (ICC) Table 2. Intra-rater agreement was assessed through DSC for the absolute agreement was calculated [21]. means per structure for three cases (Table 3). Intra-rater Spatial overlap between reference and the automated DSC was consistently high, with DSC ≥ 0.85 for all the method was measured with DSC. Student’s t test was used experts across all six DGM structures. No difference in inter- to compare DSC with the reference between controls and rater DSC was observed between raters or between DGM Table 2  For all structures and hemispheres, first the mean volume ± standard deviation (std) in millimeter of reference and the four automated segmentation software Method Left Caudate Right Caudate N = 32 Volume ICC DSC Volume ICC DSC Reference 3.99 ± 0.64 3.98 ± 0.60 FSL-FIRST 3.46 ± 0.45 0.69 0.84 ± 0.04 3.53 ± 0.49 0.81 0.84 ± 0.04 FreeSurfer 3.55 ± 0.52 0.74 0.76 ± 0.09 3.75 ± 0.58 0.68 0.77 ± 0.09 GIF 3.52 ± 0.42 0.50 0.83 ± 0.04 3.73 ± 0.47 0.60 0.83 ± 0.05 volBrain 3.55 ± 0.55 0.85 0.83 ± 0.07 3.57 ± 0.54 0.86 0.83 ± 0.07 Left Putamen Right Thalamus Volume ICC DSC Volume ICC DSC Reference 4.79 ± 0.79 4.65 ± 0.73 FSL-FIRST 4.85 ± 0.73 0.94 0.88 ± 0.03 4.83 ± 0.79 0.93 0.87 ± 0.04 FreeSurfer 4.60 ± 0.88 0.94 0.81 ± 0.08 4.66 ± 0.88 0.96 0.81 ± 0.07 GIF 4.37 ± 0.70 0.98 0.80 ± 0.03 4.34 ± 0.65 0.92 0.81 ± 0.03 volBrain 4.09 ± 0.65 0.77 0.84 ± 0.06 4.04 ± 0.60 0.80 0.84 ± 0.06 Left Thalamus Right Thalamus Volume ICC DSC Volume ICC DSC Reference 6.83 ± 1.19 6.80 ± 1.21 FSL-FIRST 7.87 ± 1.01 0.76 0.82 ± 0.05 7.69 ± 0.98 0.79 0.83 ± 0.05 FreeSurfer 7.32 ± 1.10 0.80 0.77 ± 0.08 6.93 ± 1.01 0.95 0.78 ± 0.08 GIF 5.81 ± 0.76 0.72 0.77 ± 0.04 5.67 ± 0.68 0.64 0.76 ± 0.04 volBrain 5.47 ± 1.01 0.68 0.80 ± 0.07 5.39 ± 0.98 0.67 0.80 ± 0.07 Second, the intra-class correlation coefficient (ICC) and mean dice similarity coefficient (DSC) ± std between the reference and the segmentation of the automated segmentation software. N = amount of subjects 13 Journal of Neurology (2020) 267:3541–3554 3545 Table 3  For all structures and hemispheres the spatial overlap of intra rater agreement. Spatial overlap is shown with the mean dice similarity coefficient ± standard deviation and is calculated over four subjects Rater Left caudate Right caudate Left putamen Right putamen Left thalamus Right thalamus Expert 1 0.87 ± 0.031 0.87 ± 0.037 0.89 ± 0.047 0.91 ± 0.026 0.87 ± 0.035 0.88 ± 0.007 Expert 2 0.87 ± 0.051 0.88 ± 0.032 0.85 ± 0.039 0.88 ± 0.018 0.89 ± 0.031 0.88 ± 0.022 Expert 3 0.92 ± 0.004 0.92 ± 0.008 0.91 ± 0.022 0.92 ± 0.016 0.89 ± 0.022 0.91 ± 0.008 Table 4  The average (± standard deviation) amount of voxels that method. Over the total dataset (n = 32) automated average were selected by one rater for both healthy control groups (HC) as volumes all differed from reference segmentations: caudate patients (MS) group and putamen volumes were on average underestimated by all Structure HC (n = 11) MS (n = 21) automated method, while thalamus volumes were overesti- mated by FSL-FIRST and FreeSurfer and underestimated Caudate 1356 ± 220 1413 ± 211 by GIF and volBrain (all p < 0.01). Despite these systematic Putamen 1460 ± 290 1536 ± 436 differences, ICC for FSL-FIRST, FreeSurfer and volBrain Thalamus 2126 ± 443 2083 ± 526 varied between good (0.60 ≤ ICC < 0.75) and excellent Volumes differed between the Reference and all four automated meth- (ICC ≥ 0.75) and for GIF from fair (0.40 ≤ ICC < 0.60) to ods for all structures (all p < 0.01), but there were no differences excellent (Table 3). between the volumes for pairs of automated methods Spatial agreement structures. The manual labels were combined by majority The DSC between reference and automatic segmentations voting to create the reference segmentation. An evaluation at were assessed. Figure 4, Table 5 show the DSC for both the voxel level showed similar numbers of voxels segmented controls and patients. For thalamus, all the DSC were signifi- by only a single rater in both MS and controls, indicating cantly lower for patients compared to controls (p < 0.05). For that there was no greater disagreement between the raters for putamen, this was the case for FreeSurfer and GIF, for both MS patients compared to controls, see Table 4. left and right hemisphere. The volumes were, however, dif- ferent in all cases, mostly lower in MS. For the caudate, only Performance of automated methods a significant difference in DSC between controls and patients was found in the right hemisphere for FreeSurfer and Gif. In Volumetric agreement all cases, a large variation was observed for patients com- pared to controls, see Fig. 4. DGM volumes of the reference and automatic segmentations were compared. In Fig. 2, an example T1 image is shown Relation with pathology along with the corresponding segmentation of reference and automated method. Figure 3 and Table 2 show the volumet- Higher WM lesion load was associated with a lower per- ric and spatial agreement between reference and automated formance of the automated method: total WM lesion load Fig. 2  T1 weighted images and segmentation of majority voting, FSL-FIRST, Freesurfer, GIF and volBrain. Segmentations of both left and right hemisphere and for all three structures; caudate, putamen and thalamus 13 3546 Journal of Neurology (2020) 267:3541–3554 13 Journal of Neurology (2020) 267:3541–3554 3547 ◂Fig. 3  Majority voting segmentation volume and volume by auto- DGM segmentation methods perform not as accurate in MS matic segmentation are given for each deep gray matter structure and patients as in controls, as reflected by the lower DSC (over- segmentation method. Volumes are given in milliliters lap) scores. This implies that the results may incorporate increased random variability and bias when applied to MS was negatively correlated with DSC for all method (Table 6 cases and should be interpreted with great caution. In the and Fig. 5). The correlation was moderate to strong for all future, methodological improvements are required to achieve structures and for all methods and both sides (|r|> 0.2), how- better performance in MS. ever, not all were significant, see Table 6. The regional WM Only a limited number of studies directly investigated the lesion load, i.e., that located within 10 mm of the structure, performance of DGM segmentation methods when applied was also negatively correlated with DSC (Table 6), however, to MS. Derakhshan et al. (2010) evaluated six automated these correlations ranged from weak to moderate for puta- segmentation method for GM atrophy on T1 MR images of men and thalamus and for caudate from moderate too strong. three MS patients. They concluded that severe shortcomings Both NBV and the volume of the structure of interest are present in the segmentation of DGM structures [1]. The itself were positively correlated with DSC (Table 6). The current study extends those findings substantially by inves- correlations between NBV and DSC were often not signifi- tigating a multi-center dataset comprising 21 MS subjects cant and ranged between weak and strong for the different and 11 controls using full three-dimensional manual seg- structures, methods and sides. The correlations between the mentations of three DGM structures bilaterally. Importantly, volume of the structure and DSC were often significant. using this dataset we were able to objectively compare sev- Only the correlation of DSC and volume measured on left eral widely applied automated segmentation techniques in a caudate with GIF and on left thalamus with FreeSurfer were multi-center setting. By selecting from previously acquired not significant. The correlations ranged for the putamen, data a subset that maximized the number of scanners and the thalamus and right caudate from moderate too strong and number of progressive patients, we were able to demonstrate for left thalamus from weak too strong. quantitatively that this performance impairment exists in MS To overcome the effect of lesions, two lesion-filling meth- patients with a relatively long disease duration and/or pro- ods (LST-LF and LEAP) were used. Two-way ANOVA gressive course. It would next be important to confirm this analysis per hemisphere, per method and DGM structure independently, as well as to investigate if the effect already showed no significant difference in segmentation volume occurs in early MS or CIS, given that DGM atrophy already and DSC for both lesion filling methods compared to native occurs at those early stages [29]. (non-filled) patients images (see Supplementary Fig. 1 and To obtain insights that could aid in amending the impair- Supplementary Table 5). Moreover, Student’s t test showed ment of segmentation performance, we investigated several no significant difference in segmentation volume or DSC possible causes. One important candidate reason for the for either of the filling methods compared to native patient reduced accuracy of DGM segmentation in MS is formed images (Table 7). by the focal WM lesions. Previous work on whole-brain total GM volume measurement has shown that MS WM lesions affect the GM volume measurement for a number Discussion of different packages [6–9, 30]. Similarly, the presence of local or overall brain atrophy or diffusions damage could Using a systematic and objective evaluation against a con- affect the performance of segmentation methods [31]. The sensus of manual segmentations in a multi-center dataset, precise mechanism behind these deteriorating effects may this study provides evidence that automated DGM seg- differ between packages but could include effects on image mentation methods performed worse on brain scans of MS intensity histograms, image registration and non-brain tissue patients than on those of healthy controls. Higher lesion removal [5]. Therefore, we investigated whether total lesion volumes were associated with poorer DGM segmentation load, regional lesion load, NBV and the volume of the struc- performance. ture itself were related to the performance of the automated The accuracy of DGM segmentations is not an academic DGM segmentation method. The strongest association with question but also has great clinical importance. Clini- poorer accuracy in MS cases compared to healthy controls cal and cognitive deterioration in MS have been linked to was observed for total WM lesion load. Higher regional brain and GM atrophy [5, 23, 24], and several treatments lesions load and lower total and local brain volumes were are now available that are able to reduce brain atrophy rates also associated with poorer performance, but less strongly. in MS [25–28]. Accurate measurement of the volumes of It should be mentioned that the small size of regional lesion DGM structures in MS is becoming especially important, load—which is confined to a narrow region around the because of the strong relation of DGM atrophy with cog- structure of interest—and the relatively small number of nitive impairment (1,5). This study reveals that existing MS patients may have hampered our ability to detect this 13 3548 13 Fig. 4  Dice similarity coefficients between segmentations from majority voting and each automated method per DGM structure for both healthy controls (HC) and patients (green) Journal of Neurology (2020) 267:3541–3554 Journal of Neurology (2020) 267:3541–3554 3549 Table 5  For all structures and hemispheres the spatial overlap between the “Gold standard” and the automated segmentation methods for both control and patients group Method Caudate nucleus Controls (N = 11) Patients (N = 21) Left Right Left Right FSL-FIRST 0.84 ± 0.44 0.87 ± 0.03 0.86 ± 0.24 0.86 ± 0.04 FreeSurfer 0.85 ± 0.02 0.85 ± 0.02 0.82 ± 0.06 0.83 ± 0.05 GIF 0.85 ± 0.02 0.86 ± 0.01 0.83 ± 0.05 0.83 ± 0.06 volBrain 0.88 ± 0.02 0.88 ± 0.02 0.87 ± 0.03 0.87 ± 0.02 Putamen Controls (N = 11) Patients (N = 21) Left Right Left Right FSL-FIRST 0.89 ± 0.03 0.88 ± 0.04 0.88 ± 0.03 0.87 ± 0.04 FreeSurfer 0.89 ± 0.01 0.88 ± 0.01 0.85 ± 0.05 0.86 ± 0.03 GIF 0.82 ± 0.02 0.83 ± 0.01 0.80 ± 0.03 0.81 ± 0.02 volBrain 0.89 ± 0.01 0.89 ± 0.01 0.88 ± 0.02 0.89 ± 0.02 Thalamus Controls (N = 11) Patients (N = 21) Left Right Left Right FSL-FIRST 0.86 ± 0.02 0.86 ± 0.03 0.81 ± 0.05 0.81 ± 0.06 FreeSurfer 0.85 ± 0.02 0.86 ± 0.02 0.83 ± 0.05 0.83 ± 0.05 GIF 0.80 ± 0.01 0.77 ± 0.02 0.77 ± 0.03 0.75 ± 0.04 volBrain 0.87 ± 0.01 0.87 ± 0.01 0.84 ± 0.05 0.84 ± 0.04 The spatial overlap is given as the mean ± standard deviation of the Dice Similarity Coefficient. Values of patients are bold if they are signifi- cantly diferent from those of controls (p-value < 0.05). N = amount of subjects association. Moreover, we should mention that the relation could also be indirectly related to disease effects: due to the between the volume of the structure and the performance anatomical location of the structures, some brain regions are of the automated DGM segmentation methods could also more prone to contain lesions than others or could be more result from the artifact that the performance is dependent affected by regional atrophy (both of which could impair on the volume (greater volume could result in higher DSC). DGM segmentation). A study with more patients and a more Therefore, the positive relationship should be studied in diverse lesion load could give more insight if the automated more detail for a better understanding of this effect. method performs differently on DGM structures. Moreover, While the accuracy of the segmentation was the most a study on spatial patterns on the DGM structures could also important focus of the present work, the accuracy of the give more insight into the performance of the methods. resulting volumes may be considered at least equally impor- It has been suggested that filling lesions increases the tant from a clinical viewpoint. Here to, systematic differ- accuracy of total GM segmentation, and we also expected ences were observed between the automated methods and an improvement of DGM segmentation after lesions fill- the reference measurements. We also saw a difference ing [6, 30, 33]. However, we measured no difference in the between the volumes obtained from different methods for performance of the automated method compared to manual each structure separately. The automated methods underes- segmentation after filling lesions. This is similar to the timated volumes of caudate and putamen while the volume results reported in 2014 by Popescu et al. for filling with of the thalamus was generally overestimated. This difference FLS-lesion filling and LEAP and segmentation with FSL- could be related to the different anatomical definitions used First for multiple DGM structures (e.g. thalamus, putamen, in the manual standard and the automated methods. One spe- caudate nucleus, brainstem) (7). Therefore, it seems that cific example is the question of whether the lateral genicu- lesion filling increases the accuracy of total GM segmenta- late nuclei bodies should be included or excluded when seg- tion, however, it does not increase the accuracy of DGM menting the thalamus [32]. The differences for the structures segmentation. Our hypothesis on this is that an underlying 13 3550 Journal of Neurology (2020) 267:3541–3554 Table 6  Spearman correlation between the dice similarity index and lesion load (LL), regional lesion load (RLL), normalized brain volume (NBV) and volume of region of interest (ROIV) Method Left caudate Right caudate N = 21 LL RLL NBV ROIV LL RLL NBV ROIV FSL-FIRST − 0.31 − 0.52* − 0.88 0.45* − 0.33 − 0.41 0.36 0.53** FreeSurfer − 0.60** − 0.49* 0.20 0.47** − 0.57** − 0.62* 0.25 0.37* GIF − 0.68** − 0.57** 0.25 0.17 − 0.57** − 0.63* 0.34 0.38* volBrain − 0.34 − 0.43 0.17 0.61** − 0.57** − 0.58** 0.18 0.82** Left Putamen Right Putamen LL RLL NBV ROIV LL RLL NBV ROIV FSL-FIRST − 0.56** − 0.30** 0.69** 0.72** − 0.69** − 0.80** 0.43 0.62** FreeSurfer − 0.26 − 0.45 0.08 0.74** − 0.56** − 0.20 0.23 0.37** GIF − 0.26 − 0.25 0.52 0.65** − 0.54* − 0.50 0.40 0.59** volBrain − 0.39 − 0.09 0.43 0.56** − 0.34 − 0.68** 0.44* 0.65** Left Thalamus Right Thalamus LL RLL NBV ROIV LL RLL NBV ROIV FSL-FIRST − 0.46* − 0.16 0.36 0.54** − 0.52* − 0.27 0.29 0.42* FreeSurfer − 0.53* − 0.30 0.30 0.26 − 0.49* − 0.38 0.46* 0.64* GIF − 0.42 − 0.43 0.18 0.52** − 0.23 − 0.29 0.18 0.44* volBrain − 0.30 − 0.16 0.45* 0.48** − 0.31 − 0.26 0.41 0.63** Correlation is measured for all structures, hemispheres and automated segmentation software. With α for * < 0.05 and ** < 0.01 for significant spearman correlation. N = amount of subjects Fig. 5  Dice similarity coefficients versus lesion load, represented per DGM structure and segmentation method and left (blue) and right (green) hemisphere 13 Journal of Neurology (2020) 267:3541–3554 3551 Table 7  For all structures, hemispheres and automated segmentation method the p-value of students t test between dice similarity index of non- filled and filled T1 images before applying automated segmentation method Method Left Caudate Right Caudate N = 21 LEAP LST-filling LEAP LST-filling FSL-FIRST 0.70 0.90 0.79 0.73 FreeSurfer 0.73 0.75 0.16 0.56 GIF 0.84 0.76 0.78 0.54 volBrain 0.75 0.84 0.84 0.79 Left Putamen Right Putamen LEAP LST-filling LEAP LST-filling FSL-FIRST 0.55 0.92 0.91 0.97 FreeSurfer 0.89 0.56 0.89 0.97 GIF 0.36 0.86 0.25 0.90 volBrain 0.86 0.95 0.79 0.51 Left Thalamus Right Thalamus LEAP LST-filling LEAP LST-filling FSL-FIRST 0.56 0.95 0.63 0.95 FreeSurfer 0.85 0.87 0.86 0.98 GIF 0.84 0.96 0.94 0.88 volBrain 0.98 0.98 0.78 0.89 Filling is done with LEAP and LST N = amount of subjects factor such as regional atrophy or GM lesion load or a com- changes, the effect of image processing) should be investi- bination of the pathology aspects (e.g. lesion load, atrophy, gated as well. NAWM, diffusion damage) could be a cause. It should be Furthermore, as it is important to have segmentation with mentioned that the lesions were manually outlined on either accurate spatial level for correct localization and shape, T2/PD images or FLAIR images, potentially leading to dif- future research could take are more in-depth approach ferences in the lesion segmentations that could have affected regarding shape analysis e.g. quantitative vertex displace- our results. However, after dividing the group into two dif- ment analysis [36]. This analysis enables the finding of ferent sets the same effects were visible as in the complete vertices which have a significantly different shape from the group, though less significant, as expected for the smaller reference and would be of added value in unraveling why group sizes. some packages are outperforming others. Moreover, we suggest a study on the effect of WM-GM In conclusion, the performance of four state-of-the-art contrast-to-noise ratio which might cause this effect. As MS automated DGM segmentation method is impaired in MS, pathology affects both the WM and GM, resulting in more which warrants caution in interpreting DGM volumes both variation in the WM and GM signal, it is possible that the in group studies and in individual patients. Poorer accuracy WM-GM contrast ratio is changed. Ratio could be changed was associated with higher WM lesion load and smaller due to iron change or damage of WM and/or GM [34]. West- global-local brain volumes, but the mechanism is not yet lye et al. (2009) showed in Alzheimer’s Disease (AD) that understood. Remarkably, the impaired performance was cortical thickness in subjects with regionally reduced tissue not improved by lesion-filling. More research is needed to contrast was overestimated compared to subjects without understand the underlying causes of reduced accuracy and reduces tissue contrast. They indicate that the overestimation then eliminate their effects. is related to alterations in myelin density and water com- partment close to the WM. Moreover, adjusting for local Acknowledgements A. de Sitter is employed on a project sponsored by a research grant from Teva Pharmaceuticals (grant to H. Vrenken variability in tissue contrast could correct the overestima- and F. Barkhof). T. Verhoeven has no disclosures. Y. Liu has no disclo- tion [35]. Therefore, further studies should investigate this in sures related to this paper. J. Burggraaff has no disclosures. J. Simoes MS and, moreover, other possible causes (e.g. diffuse signal has no disclosures. S. Ruggieri received fees as an invited speaker or travel expenses for attending the meeting from Biogen, Merck- Serono, 13 3552 Journal of Neurology (2020) 267:3541–3554 Teva, Sanofi, Novartis. M. Palotai has no disclosures. I. Brouwer is ed to this paper. CE has no disclosures related to this paper. MF is partly employed on projects sponsored by research grants from Teva Editor-in-Chief of the Journal of Neurology; received compensation Pharmaceuticals and Novartis Pharma (grants to H. Vrenken and F. for consulting services and/or speaking activities from Biogen Idec, Barkhof). A. Versteeg has no disclosures. V. Wottschel has no dis- Merck-Serono, Novartis, Teva Pharmaceutical Industries; and receives closures related to this paper. S. Ropele has no disclosures related research support from Biogen Idec, Merck-Serono, Novartis, Teva to this paper. M.A. Rocca received speakers’ honoraria from Biogen Pharmaceutical Industries, Roche, Italian Ministry of Health, Fondazi- Idec, Novartis, Genzyme, Sanofi-Aventis, Teva, Merck Serono, Roche one Italiana Sclerosi Multipla, and ARiSLA (Fondazione Italiana di and Celgene and receives research support from the Italian Ministry Ricerca per la SLA). NDS has no disclosures related to this paper. of Health and Fondazione Italiana Sclerosi Multipla. C. Gasperini LK has no disclosures related to this paper. JLF declares personal fees received fees as a speaker for Bayer-Schering Pharma, Sanofi-Aventis, from Biogen Idec, Merck Serono and Sanofi-Aventis, participation in Gen- zyme, Biogen, Teva, Novartis, and Merck-Serono, and received scientific advisory boards for Almiral, Genzyme and Novartis, person- a grant for research by Teva. A. Gallo has no disclosures related to this al fees (speaker honoraria) from Biogen, Merck, Santhera and Teva, paper. M.C. Yiannakas has no disclosures. A. Rovira has no disclo- all unrelated to this paper. FB has received compensation for con- sures related to this paper. C. Enzinger has no disclosures related to sulting services and/or speaking activities from Bayer, Biogen Idec, this paper. M. Filippi is Editor-in-Chief of the Journal of Neurology; Merck Serono, Novartis, Roche, Teva, Bracco and IXICO. CRGG has received compensation for consulting services and/or speaking activi- received support from the National Multiple Sclerosis Society, the In- ties from Biogen Idec, Merck-Serono, Novartis, Teva Pharmaceutical ternational Progressive Multiple Sclerosis Alliance, the U.S. Office for Industries; and receives research support from Biogen Idec, Merck- Naval Research, Mobilengine (free use of platform and programming Serono, Novartis, Teva Pharmaceutical Industries, Roche, Italian Min- by Mobilengine Engineers), NIH, as well as travel support from Roche istry of Health, Fondazione Italiana Sclerosi Multipla, and ARiSLA Pharmaceuticals; C.R.G owns stock in Roche, Novartis, GSK, Al- (Fondazione Italiana di Ricerca per la SLA). N. De Stefano has no nylam, Protalix Biotherapeutics, Arrowhead Pharmaceuticals, Cocrys- disclosures related to this paper. L. Kappos has no disclosures related tal Pharma, Sangamo Therapeutics. HV has received research grants to this paper. J. L. Frederiksen declares personal fees from Biogen Idec, from Pfizer, MerckSerono, Novartis and Teva, speaker honoraria from Merck Serono and Sanofi-Aventis, participation in scientific advisory Novartis, and consulting fees from MerckSerono; all funds were paid boards for Almiral, Genzyme and Novartis, personal fees (speaker directly to his Institution. honoraria) from Biogen, Merck, Santhera and Teva, all unrelated to this paper. F. Barkhof has received compensation for consulting Ethical standards The local ethical review boards had approved the services and/or speaking activities from Bayer, Biogen Idec, Merck original study. All patients and controls provided written informed Serono, Novartis, Roche, Teva, Bracco and IXICO. C.R.G. Gutmann consent for participating in the original study and for the use of their has received support from the National Multiple Sclerosis Society, the brain MRI-scans. International Progressive Multiple Sclerosis Alliance, the U.S. Office for Naval Research, Mobilengine (free use of platform and program- ming by Mobilengine Engineers), NIH, as well as travel support from Open Access This article is licensed under a Creative Commons Attri- Roche Pharmaceuticals; C.R.G owns stock in Roche, Novartis, GSK, bution 4.0 International License, which permits use, sharing, adapta- Alnylam, Protalix Biotherapeutics, Arrowhead Pharmaceuticals, tion, distribution and reproduction in any medium or format, as long Cocrystal Pharma, Sangamo Therapeutics. H. Vrenken has received as you give appropriate credit to the original author(s) and the source, research grants from Pfizer, MerckSerono, Novartis and Teva, speaker provide a link to the Creative Commons licence, and indicate if changes honoraria from Novartis, and consulting fees from MerckSerono; all were made. The images or other third party material in this article are funds were paid directly to his Institution. included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not Funding There was no funding specifically for this work. permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a Data availability Data are not available for other research groups, copy of this licence, visit http://creat​iveco​mmons​.org/licen​ses/by/4.0/. because of ethical and privacy issues. Compliance with ethical standards References Conflict of interest AdS is employed on a project sponsored by a re- search grant from Teva Pharmaceuticals (grant to H. Vrenken and F. 1. Derakhshan M, Caramanos Z, Giacomini PS, Narayanan S, Barkhof). TV has no disclosures. YL has no disclosures related to this Maranzano J, Francis SJ et al (2010) Evaluation of automated paper. JB has no disclosures. JS has no disclosures. SR received fees as techniques for the quantification of grey matter atrophy in patients invited speaker or travel expenses for attending meeting from Biogen, with multiple sclerosis. Neuroimage. 52(4):1261–1267. https​:// Merck-Serono, Teva, Sanofi, Novartis. MP has no disclosures. IB is doi.org/10.1016/j.neuro​image​.2010.05.029Epub 2010/05/21 partly employed on projects sponsored by research grants from Teva 2. Eshaghi A, Marinescu RV, Young AL, Firth NC, Prados F, Jorge Pharmaceuticals and Novartis Pharma (grants to HV and FB). AV has Cardoso M et al (2018) Progression of regional grey matter atro- no disclosures. VW has no disclosures related to this paper. SR has no phy in multiple sclerosis. Brain 141(6):1665–1677. https​://doi. disclosures related to this paper. MAR received speakers’ honoraria org/10.1093/brain​/awy08​8Epub 2018/05/10 from Biogen Idec, Novartis, Genzyme, Sanofi-Aventis, Teva, Merck 3. Modica CM, Bergsland N, Dwyer MG, Ramasamy DP, Carl E, Serono, Roche and Celgene and receives research support from the Zivadinov R et al (2016) Cognitive reserve moderates the impact Italian Ministry of Health and Fondazione Italiana Sclerosi Multip- of subcortical gray matter atrophy on neuropsychological sta- la. CG received fees as speaker for Bayer-Schering Pharma, Sanofi- tus in multiple sclerosis. Mult Scler. 22(1):36–42. https​://doi. Aventis, Genzyme, Biogen, Teva, Novartis, and Merck-Serono, and org/10.1177/13524​58515​57944​3Epub 2015/04/30 received a grant for research by Teva. AG has no disclosures related 4. Schoonheim MM, Ciccarelli O (2018) The value of including to this paper. MCY has no disclosures. AR has no disclosures relat- thalamic atrophy as a clinical trial endpoint in multiple sclerosis. 13 Journal of Neurology (2020) 267:3541–3554 3553 Neurology. 90(15):677–678. https​://doi.org/10.1212/WNL.00000​ 20. Zijdenbos AP, Dawant BM, Margolin RA, Palmer AC (1994) 00000​00527​9Epub 2018/03/16 Morphometric analysis of white matter lesions in MR images: 5. Amiri H, de Sitter A, Bendfeldt K, Battaglini M, Gandini Wheeler- method and validation. IEEE Trans Med Imaging. 13(4):716–724. Kingshott CAM, Calabrese M et al (2018) Urgent challenges in https​://doi.org/10.1109/42.36309​6Epub 1994/01/01 quantification and interpretation of brain grey matter atrophy in 21. Tukey J (1952) Statistical Methods for Research Workers. Econo- individual MS patients using MRI. Neuroimage Clin. 19:466–475. metrica 20(3):511–512. https​://doi.org/10.2307/19074​25 https​://doi.org/10.1016/j.nicl.2018.04.023Epub 2018/07/10 22. Corder GW, Foreman DI (2009) Nonparametric statistics for non- 6. Popescu V, Ran NC, Barkhof F, Chard DT, Wheeler-Kingshott statisticians: a step-by-step approach. Wiley, Hoboken CA, Vrenken H (2014) Accurate GM atrophy quantification in MS 23. Benedict RHB, Hulst HE, Bergsland N, Schoonheim MM, Dwyer using lesion-filling with co-registered 2D lesion masks. Neuroim- MG, Weinstock-Guttman B et al (2013) Clinical significance of age Clin 4:366–373. https​://doi.org/10.1016/j.nicl.2014.01.004 atrophy and white matter mean diffusivity within the thalamus of 7. Battaglini M, Jenkinson M, De Stefano N (2012) Evaluating and multiple sclerosis patients. Mult Scler J 19(11):1478–1484. https​ reducing the impact of white matter lesions on brain volume ://doi.org/10.1177/13524​58513​47867​5 measurements. Hum Brain Mapp 33(9):2062–2071. https​://doi. 24. Chard DT, Brex PA, Ciccarelli O, Griffin CM, Parker GJ, Dal- org/10.1002/hbm.21344​Epub 2011/09/02 ton C et al (2003) The longitudinal relation between brain lesion 8. Chard DT, Jackson JS, Miller DH, Wheeler-Kingshott CA (2010) load and atrophy in multiple sclerosis: a 14 year follow up study. Reducing the impact of white matter lesions on automated meas- J Neurol Neurosurg Psychiatry 74(11):1551–1554. https​://doi. ures of brain gray and white matter volumes. J Magn Reson Imag- org/10.1136/jnnp.74.11.1551Epub 2003/11/18 ing. 32(1):223–228. https​://doi.org/10.1002/jmri.22214​Epub 25. Calabresi PA, Radue EW, Goodin D, Jeffery D, Rammohan KW, 2010/06/25 Reder AT et al (2014) Safety and efficacy of fingolimod in patients 9. Nakamura K, Fisher E (2009) Segmentation of brain magnetic with relapsing-remitting multiple sclerosis (FREEDOMS II): a dou- resonance images for measurement of gray matter atrophy in mul- ble-blind, randomised, placebo-controlled, phase 3 trial. Lancet Neu- tiple sclerosis patients. Neuroimage 44(3):769–776. https​://doi. rol 13(6):545–556. https​://doi.org/10.1016/S1474​-4422(14)70049​-3 org/10.1016/j.neuro​image​.2008.09.059Epub 2008/11/15 26. Coles AJ, Twyman CL, Arnold DL, Cohen JA, Confavreux C, 10. Patenaude B, Smith SM, Kennedy DN, Jenkinson M (2011) A Fox EJ et al (2012) Alemtuzumab for patients with relapsing Bayesian model of shape and appearance for subcortical brain seg- multiple sclerosis after disease-modifying therapy: a randomised mentation. Neuroimage 56(3):907–922. https​://doi.org/10.1016/j. controlled phase 3 trial. Lancet 380(9856):1829–1839. https:​ //doi. neuro​image​.2011.02.046Epub 2011/03/01 org/10.1016/S0140​-6736(12)61768​-1Epub 2012/11/06 11. Dale AM, Fischl B, Sereno MI (1999) Cortical surface-based 27. Hauser SL, Bar-Or A, Comi G, Giovannoni G, Hartung HP, analysis I. Segmentation and surface reconstruction. Neuroim- Hemmer B et al (2017) Ocrelizumab versus interferon beta-1a in age 9(2):179–194. https​://doi.org/10.1006/nimg.1998.0395Epub relapsing multiple sclerosis. N Engl J Med 376(3):221–234. https​ 1999/02/05 ://doi.org/10.1056/NEJMo​a1601​277Epub 2016/12/22 12. Cardoso MJ, Modat M, Wolz R, Melbourne A, Cash D, Rueck- 28. La Mantia L, Di Pietrantonj C, Rovaris M, Rigon G, Frau S, ert D et al (2015) Geodesic information flows: spatially-variant Berardo F et al (2016) Interferons-beta versus glatiramer acetate graphs and their application to segmentation and fusion. IEEE for relapsing-remitting multiple sclerosis. Cochrane Database Syst Trans Med Imaging 34(9):1976–1988. https​://doi.org/10.1109/ Rev 11:CD009333. https:​ //doi.org/10.1002/146518​ 58.CD0093​ 33. TMI.2015.24182​98Epub 2015/04/17 pub3Epub 2016/11/24 13. Manjon JV, Coupe P (2016) volBrain: an online MRI brain volu- 29. Eshaghi A, Prados F, Brownlee WJ, Altmann DR, Tur C, Cardoso metry system. Front Neuroinform. 10:30. https​://doi.org/10.3389/ MJ et al (2018) Deep gray matter volume loss drives disability fninf​.2016.00030​Epub 2016/08/12 worsening in multiple sclerosis. Ann Neurol 83(2):210–222. https​ 14. Rocca MA, Valsasina P, Hulst HE, Abdel-Aziz K, Enzinger C, ://doi.org/10.1002/ana.25145​Epub 2018/01/14 Gallo A et al (2014) Functional correlates of cognitive dysfunc- 30. Guo C, Ferreira D, Fink K, Westman E, Granberg T (2019) tion in multiple sclerosis: a multicenter fMRI Study. Hum Brain Repeatability and reproducibility of FreeSurfer, FSL-SIENAX Mapp 35(12):5799–5814. https​://doi.org/10.1002/hbm.22586​ and SPM brain volumetric measurements and the effect of lesion 15. Ropele S, Kilsdonk ID, Wattjes MP, Langkammer C, de Graaf filling in multiple sclerosis. Eur Radiol 29(3):1355–1364. https:​ // WL, Frederiksen JL et al (2014) Determinants of iron accumula- doi.org/10.1007/s0033​0-018-5710-xEpub 2018/09/23 tion in deep grey matter of multiple sclerosis patients. Mult Scler 31. de Bresser J, Portegies MP, Leemans A, Biessels GJ, Kappelle 20(13):1692–1698. https​://doi.org/10.1177/13524​58514​53108​5 LJ, Viergever MA (2011) A comparison of MR based segmen- 16. Schmidt P, Gaser C, Arsic M, Buck D, Forschler A, Berthele tation methods for measuring brain atrophy progression. Neu- A et al (2012) An automated tool for detection of FLAIR- roimage 54(2):760–768. https​://doi.org/10.1016/j.neuro​image​ hyperintense white-matter lesions in multiple sclerosis. Neuro- .2010.09.060Epub 2010/10/05 image 59(4):3774–3783. https​://doi.org/10.1016/j.neuro​image​ 32. Power BD, Wilkes FA, Hunter-Dickson M, van Westen D, Santillo .2011.11.032Epub 2011/11/29 AF, Walterfang M et al (2015) Validation of a protocol for manual 17. Coupe P, Manjon JV, Fonov V, Pruessner J, Robles M, Col- segmentation of the thalamus on magnetic resonance imaging lins DL (2011) Patch-based segmentation using expert priors: scans. Psychiatry Res 232(1):98–105. https​://doi.org/10.1016/j. application to hippocampus and ventricle segmentation. Neu- pscyc​hresn​s.2015.02.001Epub 2015/03/11 roimage 54(2):940–954. https​://doi.org/10.1016/j.neuro​image​ 33. Magon S, Gaetano L, Chakravarty MM, Lerch JP, Naegelin Y, .2010.09.018Epub 2010/09/21 Stippich C et al (2014) White matter lesion filling improves the 18. Smith SM, De Stefano N, Jenkinson M, Matthews PM (2001) accuracy of cortical thickness measurements in multiple sclerosis Normalized accurate measurement of longitudinal brain patients: a longitudinal study. BMC Neurosci 15:106. https​://doi. change. J Comput Assist Tomo 25(3):466–475. https​: //doi. org/10.1186/1471-2202-15-106Epub 2014/09/10 org/10.1097/00004​728-20010​5000-00022​ 34. Yablonskiy DA, Luo J, Sukstanskii AL, Iyer A, Cross AH (2012) 19. Smith SM, Zhang Y, Jenkinson M, Chen J, Matthews PM, Fed- Biophysical mechanisms of MRI signal frequency contrast in mul- erico A et al (2002) Accurate, robust, and automated longitudinal tiple sclerosis. Proc Natl Acad Sci USA 109(35):14212–14217. and cross-sectional brain change analysis. Neuroimage 17(1):479– https​://doi.org/10.1073/pnas.12060​37109​Epub 2012/08/15 489 Epub 2002/12/17 13 3554 Journal of Neurology (2020) 267:3541–3554 35. Westlye LT, Walhovd KB, Dale AM, Espeseth T, Reinvang I, Raz 36. Kim H, Mansi T, Bernasconi A, Bernasconi N (2011) Vertex- N et al (2009) Increased sensitivity to effects of normal aging and wise shape analysis of the hippocampus: disentangling positional Alzheimer’s disease on cortical thickness by adjustment for local differences from volume changes. Med Image Comput Comput variability in gray/white contrast: a multi-sample MRI study. Neu- Assist Interv 14(Pt 2):352–359. https​://doi.org/10.1007/978-3- roimage 47(4):1545–1557. https​://doi.org/10.1016/j.neuro​image​ 642-23629​-7_43 Epub 2011/10/15 .2009.05.084Epub 2009/06/09 Affiliations Alexandra de Sitter1 · Tom Verhoeven1 · Jessica Burggraaff2 · Yaou Liu1 · Jorge Simoes1 · Serena Ruggieri3,4 · Miklos Palotai5 · Iman Brouwer1 · Adriaan Versteeg1 · Viktor Wottschel1 · Stefan Ropele6 · Mara A. Rocca7,8 · Claudio Gasperini4 · Antonio Gallo9 · Marios C. Yiannakas10 · Alex Rovira11 · Christian Enzinger12 · Massimo Filippi7,8,13,14 · Nicola De Stefano15 · Ludwig Kappos16 · Jette L. Frederiksen17 · Bernard M. J. Uitdehaag2 · Frederik Barkhof1,18 · Charles R. G. Guttmann5 · Hugo Vrenken1 · the MAGNIMS Study Group 1 10 Department of Radiology and Nuclear Medicine, MS Center Research Unit, Queen Square MS Centre, Department Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, of Neuroinflammation, UCL Queen Square Institute Location VUmc, De Boelelaan 1117, 1081 HV Amsterdam, of Neurology, University College London, London, UK The Netherlands 11 Unitat de Ressonància Magnètica (Servei de Radiologia), 2 Department of Neurology, MS Center Amsterdam, Hospital Universitari Vall D’Hebron, Autonomous University Amsterdam Neuroscience, Amsterdam UMC, Location of Barcelona, Barcelona, Spain VUmc, Amsterdam, The Netherlands 12 Division of Neuroradiology, Vascular and Interventional 3 Department of Human Neurosciences, “Sapienza” University Radiology, Department of Radiology Medical, University of Rome, Rome, Italy of Graz, Graz, Austria 4 13 Department of Neurosciences, San Camillo Forlanini Neurophysiology Unit, IRCCS San Raffaele Scientific Hospital, Rome, Italy Institute, Milan, Italy 5 14 Center for Neurological Imaging, Department of Radiology, Vita-Salute San Raffaele University, Milan, Italy Brigham and Women’s Hospital, Harvard Medical School, 15 Department of Neurological and Behavioural Sciences, Boston, MA, USA University of Siena, Siena, Italy 6 Department of Neurology, Medical University of Graz, Graz, 16 Department of Neurology, University Hospital, Austria Kantonsspital, Basel, Switzerland 7 Neuroimaging Research Unit, Division of Neuroscience, 17 Department of Neurology, Glostrup University Hospital Institute of Experimental Neurology, Milan, Italy Copenhagen, Copenhagen, Denmark 8 Neurology Unit, IRCCS San Raffaele Scientific Institute, 18 Institutes of Neurology and Healthcare Engineering, UCL Milan, Italy London, London, UK 9 Division of Neurology and MRI Research Center, Department of Medical, Surgical, Neurologic, Metabolic and Aging Sciences, University of Campania “Luigi Vanvitelli”, Naples, Italy 13