Bruin et al. Translational Psychiatry (2020)10:342 https://doi.org/10.1038/s41398-020-01013-y Translational Psychiatry ARTICLE Open Access Structural neuroimaging biomarkers for obsessive-compulsive disorder in the ENIGMA-OCD consortium: medication matters Willem B. Bruin 1, Luke Taylor2, Rajat M. Thomas1, Jonathan P. Shock 3, Paul Zhutovsky 1, Yoshinari Abe 4, Pino Alonso5,6,7, Stephanie H. Ameis8,9, Alan Anticevic10, Paul D. Arnold 11,12, Francesca Assogna13, Francesco Benedetti 14, Jan C. Beucke15,16, Premika S. W. Boedhoe17,18, Irene Bollettini14, Anushree Bose19, Silvia Brem20,21, Brian P. Brennan22, Jan K. Buitelaar 23,24, Rosa Calvo 25,26,27, Yuqi Cheng28, Kang Ik K. Cho29, Sara Dallaspezia14, Damiaan Denys 1,30, Benjamin A. Ely 31, Jamie D. Feusner32, Kate D. Fitzgerald33, Jean-Paul Fouche34, Egill A. Fridgeirsson1, Patricia Gruner10, Deniz A. Gürsel35,36, Tobias U. Hauser 20,37,38, Yoshiyuki Hirano 39, Marcelo Q. Hoexter40, Hao Hu41, Chaim Huyser 42,43, Iliyan Ivanov44, Anthony James 45, Fern Jaspers-Fayer46, Norbert Kathmann15, Christian Kaufmann15, Kathrin Koch35,36, Masaru Kuno39, Gerd Kvale47,48, Jun Soo Kwon 49,50, Yanni Liu33, Christine Lochner51, Luisa Lázaro25,26,27,52, Paulo Marques53,54,55, Rachel Marsh56,57, Ignacio Martínez-Zalacaín5,7, David Mataix-Cols58, José M. Menchón 5,6,7, Luciano Minuzzi59, Pedro S. Moreira53,54,55, Astrid Morer25,26,27,52, Pedro Morgado 53,54,55, Akiko Nakagawa39, Takashi Nakamae 4, Tomohiro Nakao60, Janardhanan C. Narayanaswamy19, Erika L. Nurmi32, Joseph O’Neill61, Jose C. Pariente62, Chris Perriello22,63, John Piacentini 32, Fabrizio Piras 13, Federica Piras13, Y. C. Janardhan Reddy19, Oana G. Rus-Oswald 64,65, Yuki Sakai 4,66, João R. Sato67, Lianne Schmaal 68,69, Eiji Shimizu39,70, H. Blair Simpson56,71, Noam Soreni72,73, Carles Soriano-Mas 5,6,74, Gianfranco Spalletta13,75, Emily R. Stern76,77, Michael C. Stevens 78,79, 1234567890():,; 1234567890():,; 1234567890():,; 1234567890():,; S. Evelyn Stewart46,80,81, Philip R. Szeszko82,83, David F. Tolin84,85, Ganesan Venkatasubramanian 19, Zhen Wang 41,86, Je-Yeon Yun 87,88, Daan van Rooij89, ENIGMA-OCD Working Group, Paul M. Thompson90, Odile A. van den Heuvel17,18, Dan J. Stein 91 and Guido A. van Wingen 1 Abstract No diagnostic biomarkers are available for obsessive-compulsive disorder (OCD). Here, we aimed to identify magnetic resonance imaging (MRI) biomarkers for OCD, using 46 data sets with 2304 OCD patients and 2068 healthy controls from the ENIGMA consortium. We performed machine learning analysis of regional measures of cortical thickness, surface area and subcortical volume and tested classification performance using cross-validation. Classification performance for OCD vs. controls using the complete sample with different classifiers and cross-validation strategies was poor. When models were validated on data from other sites, model performance did not exceed chance-level. In contrast, fair classification performance was achieved when patients were grouped according to their medication status. These results indicate that medication use is associated with substantial differences in brain anatomy that are widely distributed, and indicate that clinical heterogeneity contributes to the poor performance of structural MRI as a disease marker. Correspondence: Willem B. Bruin (willembenjamin@gmail.com) or Guido A. van Wingen (g.a.vanwingen@amsterdamumc.nl) 1 Amsterdam UMC, University of Amsterdam, Department of Psychiatry, Introduction Amsterdam Neuroscience, Amsterdam, Netherlands 2 Obsessive-compulsive disorder (OCD) is a severe and Department of Physiology, Anatomy and Genetics, Oxford, UK Full list of author information is available at the end of the article debilitating condition that occurs in 2–3% of the A list of members and their affiliations are listed at the end of the paper. © The Author(s) 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. Bruin et al. Translational Psychiatry (2020)10:342 Page 2 of 12 population1. It is characterized by recurrent, intrusive, with a full range of technical and clinical heterogeneity. irrational and distressing thoughts (obsessions) and We assessed the ability of MVPA to distinguish OCD repetitive behaviors or mental acts (compulsions)2. So far, patients from healthy controls using structural neuroi- no biomarkers that aid differential diagnosis are available, maging data at the individual subject level. We investi- and diagnosis relies entirely on recognition of character- gated machine learning classification performance in both istic symptoms assessed by clinical interview3. Many single-site and multi-site samples using different valida- neuroimaging studies have provided evidence for tion strategies to assess generalizability. Furthermore, the abnormalities in cortico-striato-thalamo-cortical (CSTC) large sample size enables investigation of the influence of circuits, as well as distributed changes in limbic, parietal clinical heterogeneity by stratification and subsampling, in and cerebellar regions4,5. These findings have recently order to assess the influence of clinical variability on been confirmed by different meta-analyses and mega- classification accuracy. analyses of neuroimaging studies, based on results that were reported in the literature or by using original data Materials and methods within different consortia6–10. However, inference has Study population been at the group-level, and the small effect sizes reported The ENIGMA-OCD working group includes 46 data preclude clinical application. sets from 36 international research institutes, with neu- Analytic tools such as multivariate pattern analysis roimaging and clinical data from adult (≥18 years) and (MVPA) enable inference at the individual-level, which pediatric (<18 years) samples. In total, we analyzed data may result in better discrimination3,11. MVPA techniques from 4372 participants, including 2304 OCD patients (n can be used to develop predictive models that extract = 1801 adult, n = 503 child) and 2068 healthy controls common patterns from neuroimaging data to classify (HC; i.e., free of psychopathology; n = 1629 adult, n = 439 individuals based on their diagnosis. A major advantage of child), with 38 of 46 datasets identical to those described MVPA compared to traditional methods of analysis is its in previous mega-analyses by this working group6,7,22. All ability to use inter-regional correlations to detect subtle participating sites obtained permission from their local and spatially distributed effects4. Therefore, MVPA seems institutional review boards or ethics committees to pro- particularly well suited for neuroimaging analyses in vide anonymized data for analysis, and all study partici- OCD, as abnormalities are typically distributed across the pants provided written informed consent. Demographic brain12,13. Previous MVPA studies have been able to dis- and clinical characteristics of each site are detailed in tinguish OCD patients from controls with accuracies supplementary Table S1. A complete overview of instru- ranging from 66–100%14. Although these results are ments used to obtain diagnosis and clinical information promising, sample sizes have typically been small, limiting can be found elsewhere (Data Supplement 1, Supple- model performance optimization and leading to high mentary Section S1)7. Diagnosis was determined in variance in estimated accuracy, which may result in accordance with DSM2; MINI and SCID were used for reporting optimistic or pessimistic classification rates15. In adult samples and K-SADS, MINI-KIDS and ADIS were addition, most studies have been performed using data used for pediatric samples23–27. from one research center to minimize technical (e.g., scanner hardware, protocols, and diagnostic assessment) MRI processing and clinical (e.g., age, medication status, disease chroni- Structural T1-weighted brain MRI scans were acquired city, and severity) heterogeneity. It is therefore not clear and processed locally at each site. Image acquisition whether the MVPA results obtained from these mono- parameters are listed elsewhere7. Parcellations were per- center studies generalize well to other centers, which formed using FreeSurfer software version 5.3 (http:// would be required for clinical application16–18. Interest- surfer.nmr.mgh.harvard.edu), following standardized ingly, whereas classification accuracies of monocenter ENIGMA protocols to harmonize analyses and quality studies only tend to increase with larger samples19,20, control procedures across multiple sites (see http:// accuracies for multicenter studies in other psychiatric enigma.usc.edu/protocols/imaging-protocols/). Mean disorders such as schizophrenia and autism tend to be values of parcellations of 34 cortical (Desikan-Killiany lower with increasing sample size12,15–17. This paradoxical atlas-based28) and 7 subcortical gray matter structures per effect of lower classification accuracy with larger samples hemisphere, lateral ventricle volumes, two whole- has been attributed to larger sample heterogeneity21, hemisphere measures and total intracranial volume were which inevitably increases when combining data from extracted, visually inspected and statistically evaluated for different centers. Here, we used data from the Enhancing outliers (quality assurance is reported elsewhere7). Brain Neuro-Imaging and Genetics through Meta-Analysis regions (features) used for classification included cortical (ENIGMA) OCD consortium, including 4372 partici- thickness (CT), surface area (SA) and subcortical volumes pants recruited at 36 research institutes around the world, of ROIs, two lateral ventricular and intra-cranial volumes Bruin et al. Translational Psychiatry (2020)10:342 Page 3 of 12 (ICV), and two whole-hemisphere measures for SA test-fold sizes are set to match those of LOSO-CV. This and CT. was done to evaluate whether differences between site- stratified and LOSO-CV performance were due to dif- Multivariate classification and validation ferences in heterogeneity or test-fold size variance. Finally, Participants with >10% missing entries were excluded we also performed single-site predictions using repeated (n = 276), and median imputation was used for missing 5-fold CV (with 10 repeats) to assess classification per- MRI data on the training set. Continuous features were formance for individual sites with reduced heterogeneity. centered around median zero and scaled according to For a schematic representation of LOSO and site- their interquartile range. FreeSurfer variables were com- stratified CV see supplementary Fig. S2. Statistical sig- bined with covariates age, sex, and site by concatenating nificance of model performance and 95% confidence individual feature vectors. Categorical covariates were intervals were assessed through the obtained AUC scores one-hot encoded prior to classification. All analyses were using the Mann–Whitney-U statistic for non-parametric performed separately for pediatric and adult patients, and testing (see Supplement for details)33,35,36. The both groups combined. Common MVPA classifiers were Bonferroni-corrected level of significance was set at alpha applied: support vector machine (SVM) with linear and = 0.05 for the number of classifiers and comparisons (3 non-linear (radial-basis-function (RBF)) kernels, logistic CV types × 10 classifiers for multi-site classifications; regression (LR) with L1 and L2 regularization, Gaussian alpha = 0.05/30). As previous work from ENIGMA-OCD processes classification (GPC) with a linear kernel, and has identified distinct alterations in pediatric and adult two decision-tree based ensemble methods, namely the patients, we performed all multi-site classification ana- random forest classifier (RFC) and the XGBoost (XGB) lyses for pediatric (<=18 years of age) and adult (>18 years algorithm29–32. A neural network was also implemented of age) data separately, as well as both age groups (fully connected; 3 hidden layers with 60, 40, and 20 nodes combined6,7. respectively). SVM and LR classifiers were combined with and without automatic dimensionality reduction via Clinical variables and sensitivity analysis principal component analysis (PCA), using the minimal To explore the effects of clinical heterogeneity on number of components explaining 90% of the variance. classification performance, we selected subgroups with Hyper-parameters for SVM (linear and non-linear), LR particular demographic and clinical characteristics: med- and XGB were optimized using nested cross-validation; ication use, OCD severity, age of onset (AO) and duration RFC and GPC were tuned following recommendations. of illness. Classifications performed were HC vs. low Details on handling missing data, model implementation (YBOCS <= 24; mild-moderate37) and high severity and hyper-parametrization can be found in Supplemen- (YBOCS > 24; moderate-severe) OCD; HC vs. early tary Methods (and supplementary Fig. S1 for assessment (<18 yrs) and late AO (>= 18 yrs) OCD; HC vs. short of different imputation strategies). The primary perfor- (<=7 yrs) and long duration (>7 yrs) OCD; and HC vs. mance metric was the area under the receiver operator unmedicated and medicated OCD. For disease duration curve (AUC) and reported metrics are averaged across CV and severity, median splits were used to define groups; the folds33. Balanced accuracy, sensitivity and specificity are 18 year threshold for AO was chosen in line with prior reported in the supplement. ENIGMA-OCD mega-analyses6,7. Finally, as particular Multi-site classification of OCD patients versus HC was clinical variables can co-occur, we performed a post-hoc assessed using different cross-validation (CV) approaches. sensitivity analysis to investigate the effects of potential First, we assessed multi-site classification using 10-fold clinical covariance for results with AUC ≥ 0.8. First, cor- site-stratified CV to obtain maximally homogeneous relations between all clinical features were computed train-test splits, with approximately the same number of using point-biserial correlations between dichotomous subjects in each fold and the same proportion of samples and continuous variables, phi correlation for dichotomous coming from each site (also referred to as ‘internal vali- variables and Pearson correlation for continuous vari- dation’). Next, we addressed leave-one-site-out (LOSO) ables. Only those clinical features that were significantly CV, in which all but one site were used to train the models correlated (Bonferroni-corrected) were investigated fur- while the left out site was used to assess model perfor- ther by rerunning previously described classifications, but mance (external validation). This procedure is then now using samples further split according to their cor- repeated so that each site is used once as a test set. LOSO- related features (e.g., HC vs. unmedicated, short duration CV may result in large between-sample heterogeneity of OCD patients; etc.). The use of more homogenous sub- training and test sets, resulting in lower classification samples is expected to improve classification perfor- performance34. Because LOSO-CV has different fold sizes, mance, while reducing sample size itself is expected to we additionally performed site-stratified CV with varying decrease performance. If classifications are relatively fold sizes, in which the number of CV folds and respective unaffected by further splitting, the correlated clinical Bruin et al. Translational Psychiatry (2020)10:342 Page 4 of 12 variable did not have a large influence on the classification CV (SD; site-stratified fixed: 0.02–0.04; site-stratified results. variable: 0.05–0.08; LOSO: 0.07–0.11). A complete over- view of classification results is provided in supplementary Feature importance Table S2. Multi-site classification with site-stratified CV To assess which brain regions and clinical variables (with equal fold sizes), performed separately on pediatric contributed most to classification we used feature and adult samples yielded similar results, ranging from importance extracted from RFC combined with a per- 0.55 (95% CI = 0.43–0.67; pcorrected = 1) to 0.62 (95% CI mutation testing framework (see Supplementary = 0.51–0.74; pcorrected = 0.71) and 0.56 (95% CI = Methods)38. 0.5–0.62; pcorrected = 0.69) to 0.61 (95% CI = 0.55–0.67; pcorrected = 0.008) AUC, respectively (see supplementary Results Tables S3-4). As site-stratified CV with equal fold-sizes Multi-site classification resulted in the best performances, we used this strategy Three different CV approaches were used to assess the for further evaluation of intra-site performance and the influence of sample heterogeneity. Results using various influence of clinical variables. RFC classification perfor- classification algorithms are summarized in Fig. 1. Clas- mance is reported here by default, as differences between sification performance (AUC) using site-stratified CV classifiers were minimal and this model was also used to (with training on combined samples and equal fold sizes) extract feature importance. ranged between 0.57 (95% confidence intervals (CI) = 0.51–0.63; pcorrected = 0.19) and 0.62 (95% CI = 0.56–0.67; Single-site classification pcorrected < 0.001) across different classifiers. All models Single-site classification performance with 10-fold CV had statistically significant performance after multiple varied greatly, with AUCs ranging between 0.30–0.89 comparison corrections except for PCA + LR, PCA + across different sites and classifiers (see supplementary SVM and NN classifiers. LOSO-CV led to lower classifi- Table S5). Figure 2 summarizes RFC performances for cation performance; 0.51 (95% CI = 0.4–0.62; pcorrected = each individual site. We assessed the correlation between 1) to 0.54 (95% CI = 0.42–0.65; pcorrected = 1) AUC with the number of participants in each site and its obtained relatively high variance across folds (SD = 0.07–0.11) and classification performance (AUC averaged over CV folds), no classifiers surviving multiple comparison corrections. which was significant (rS = 0.37, p = 0.014). In addition, we AUC values obtained through site-stratified CV with investigated the relationship between single-site classifi- varying fold sizes were similar to site-stratified CV results cation performance and the following clinical variables of with equal fold sizes, ranging between 0.56 (95% CI = interest: mean and standard deviation of AO, duration, 0.45–0.67; pcorrected > 0.99) and 0.62 (95% CI = severity and the proportion of medicated patients and its (0.51–0.73); pcorrected = 0.55). However, variance across standard deviation. None of these clinical variables showed CV-folds was higher and comparable to that from LOSO- a significant correlation with classification performance. Fig. 1 Performance for multi-site classification using different algorithms and cross-validation schemes. Boxplots summarize AUC scores obtained across CV-folds; dashed line represents chance-level performance and asterisks indicate scores significantly different from chance (Mann–Whitney-U statistic; p < 0.05 Bonferroni corrected (10 classifiers × 3 CV types), see Supplement for details). SVM Support Vector Machine, PCA Principal Component Analysis, RBF Radial Basis Function, LR Logistic Regression, GPC Gaussian Processes Classification, RFC Random Forest Classifier, XGB XGBoost, NN Neural Network. Bruin et al. Translational Psychiatry (2020)10:342 Page 5 of 12 Fig. 3 Performance for classification between subgroups of patients based on medication status. Only RFC classifier performance for combined (pediatric and adult) data is shown here; Boxplots summarize AUC scores obtained across CV-folds; dashed line Fig. 2 Scatterplot illustrating relationship between number of represents chance-level performance and asterisks indicate scores participants and classification performance across sites. Only RFC significantly different from chance (Mann–Whitney-U statistic; p < 0.05 classifier performance averaged across CV-folds and repeats are Bonferroni corrected (10 classifiers × 3 CV types), see Supplement for plotted (Spearman correlation; rS = 0.37, p = 0.014). details). unmed unmedicated, med medicated. high severity OCD in 0.58 (95% CI = 0.49–0.66; Clinical variables and sensitivity analysis pcorrected = 1). To assess the influence of different clinical variables on Correlation analysis between medication status and classification performance, we repeated the analysis for other clinical variables only showed a significant associa- specific subgroups split according to medication use, AO, tion with disease duration (r = −0.094; pcorrected < 10−05; disease duration, and severity. A complete overview is Bonferroni-corrected). We therefore performed addi- provided in supplementary Tables S6(a–d), and results tional medication status classifications after splitting using RFC on combined data with age, sex and site as patients for disease duration (e.g., HC vs. medicated + covariates are reported below. Medicated OCD vs. HC short duration OCD; HC vs. unmedicated + short dura- classification resulted in 0.69 AUC (95% CI = 0.63–0.75; tion OCD, etc.). Classifications with or without splitting pcorrected < 0.001), unmedicated OCD vs. HC in 0.60 (95% patients for disease duration were comparable (see sup- CI = 0.54–0.67; pcorrected = 0.03), and medicated vs. plementary Tables S7(a–c) for full overview). unmedicated OCD in 0.78 (95% CI = 0.72–0.85; pcorrected < 0.001) (see Fig. 3). XGB performance was notably higher Influence of covariates for medicated vs. unmedicated OCD classification with an As seen in supplementary Table S1, several sites inclu- AUC of 0.86 (95% CI = 0.78–0.9; pcorrected < 0.001). Early ded only medicated patients whereas other sites only AO OCD vs. HC classification resulted in 0.68 AUC (95% included non-medicated patients. To assess whether the CI = 0.62–0.75; pcorrected < 0.001), late AO OCD vs. HC in high performance reported above for classifying medica- 0.73 (95% CI = 0.67–0.79; pcorrected < 0.001), and early vs. tion status could be explained by site-differences in the late AO in 0.81 (95% CI = 0.74–0.88; pcorrected < 0.001). As covariates (e.g., site ID) rather than neuroimaging data, we no late AO patients were present in pediatric samples, performed the following control experiments. We reran classifications were re-run on adult samples only, result- both medication and main diagnosis classifications with- ing in 0.65 AUC (95% CI = 0.57–0.72;pcorrected = 0.01) for out using covariates (using only brain data), using cov- early AO vs. HC, 0.70 (95% CI = 0.63–0.76; pcorrected < ariates only, and by using neuroimaging data after 0.001) for late AO vs. HC, and 0.73 (95% CI = 0.64–0.82; removing the effect of the covariates using multiple linear pcorrected < 0.001) for early vs. late AO. Classification of regression. To maintain independence between the short disease duration OCD vs. HC resulted in 0.68 AUC training and test sets, regression coefficients were esti- (95% CI = 0.61–0.75;pcorrected < 0.001), long disease dura- mated on the training data and applied to the test data. tion vs. HC in 0.71 (95% CI = 0.65–0.78; pcorrected < Results for the main classifications between OCD 0.001), and short vs. long duration in 0.78 (95% CI = patients and HC using neuroimaging data only resulted in 0.7–0.85; pcorrected < 0.001). Finally, low severity OCD vs. AUC of 0.61 (95% CI = 0.55–0.66; pcorrected = 0.001), HC classification resulted in 0.60 AUC (95% CI = while use of covariates only resulted in AUC of 0.58 (95% 0.53–0.67; pcorrected = 0.15), high severity OCD vs. HC in CI = 0.52–0.64; pcorrected = 0.04), and using neuroimaging 0.61 (95% CI = 0.54–0.68; pcorrected = 0.04), and low vs. data following correction for covariates resulted in AUC Bruin et al. Translational Psychiatry (2020)10:342 Page 6 of 12 of 0.58 (95% CI = 0.52–0.64; pcorrected = 0.02). These in either pediatric, adult or combined samples. However, results show that the covariates contain sufficient infor- for medicated vs. unmedicated OCD classification in mation to distinguish patients from HC, but the results combined samples, 24 significant and consistently selec- were otherwise comparable to the AUCs reported above ted features were found. In addition, 12 features were for classifications using the covariates as features. found for early vs. late AO patients classifications in Classification for medicated patients vs. HC using combined samples. A complete overview of these findings neuroimaging data only resulted in AUC of 0.66 (95% (including features importance for medication classifica- CI = 0.60–0.73; pcorrected < 0.001) and AUC of 0.70 (95% tion in adult samples) can be found in supplementary CI = 0.64–0.76; pcorrected < 0.001) for SVM-RBF. Using Tables S10–11. covariates only resulted in AUC of 0.71 (95% CI = 0.65–0.77; pcorrected < 0.001), and neuroimaging data fol- Discussion lowing correction for covariates resulted in AUC of 0.59 We found that MVPA of parcellated structural neu- (95% CI = 0.52–0.66; pcorrected = 0.053), and higher AUC roimaging data is unable to provide accurate distinction of 0.62 (95% CI = 0.55–0.69; pcorrected = 0.007) for XGB. between OCD cases and HC. Classification of the com- Next, classification of unmedicated patients vs. HC using plete sample using site-stratified CV ranged between an neuroimaging data only resulted in an AUC of 0.58 (95% AUC of 0.57 and 0.61, which is not sufficient for clinical CI = 0.51–0.68; pcorrected = 0.12) with significant classifier application. Differences in performance between classi- performance for PCA + SVM-RBF with AUC of 0.61 (95% fiers were minimal. Similar results were obtained for CI = 0.55–0.68; pcorrected = 0.007). Using covariates only classifications performed separately on pediatric and adult resulted in AUC of 0.64 (95% CI = 0.58–0.73; pcorrected < samples. When validated on completely new data from 0.001), and using neuroimaging data following correction other sites using LOSO-CV, model performance hardly for covariates resulted in AUC of 0.52 (95% CI = exceeded chance-level (0.5 AUC). 0.45–0.59; pcorrected = 1). Finally, classification of medi- Our findings highlight the impact of validation schemes cated vs. unmedicated patients using neuroimaging data on classification performance and suggest poor dis- only resulted in AUC of 0.74 (95% CI = 0.67–0.81; crimination between OCD patients and HC when com- pcorrected < 0.001), using covariates only in AUC of 0.84 bining data from multiple sites. In contrast, (95% CI = 0.78–0.92; pcorrected < 0.001), and using neu- discrimination between subgroups of patients based on roimaging data after correction for covariates resulted in medication status enabled fair individual subject classifi- AUC of 0.59 (95% CI = 0.50–0.67; pcorrected = 0.2). A full cation. However, our control experiments indicated that overview of these control experiment results can be found non-brain covariates such as age, sex and site can heavily in supplementary Tables S8–9. These results show that affect classification performance, dependent on the rela- the covariates contained sufficient information to classify tion between the structural neuroimaging data and those these subgroups, and especially to distinguish between covariates. Yet, even after removal of the covariate effects, medicated vs. unmedicated patients, as subgroups were the results still indicated that medication use is associated already partially defined by site. Nevertheless, the classi- with substantial differences in brain anatomy that are fications using neuroimaging data only were comparable widely distributed, whereas gross gray matter anatomy of to the classifications reported above when covariates were patients with OCD was comparable to that of healthy added as features. However, correcting the neuroimaging controls. At the same time, this also suggests that clinical data for covariates lowered the performance substantially heterogeneity contributes to the poor performance of compared to adding the covariates as features, suggesting structural MRI as a disease biomarker. that neuroimaging data were partially related to the cov- Few diagnostic classifiers have been applied to OCD ariates (e.g., different scanners at different sites). across multiple scanners and sites. Prior studies using structural MRI data to classify OCD using single-site Feature importance samples yielded accuracies ranging from 0.72 to 0.9314. We investigated which brain regions (features) con- The wide range of performances observed in our indivi- tributed most to OCD vs. HC classifications for site- dual site classification is in agreement with the published stratified CV only, using the feature importance values literature. Such a wide range may in part be explained by from RFC and permutation testing, and as we were sample size, as larger samples tended to have higher AUC interested in brain regions rather than the influence of the values16,19,34. However, this relationship does not neces- covariates, we focused on classifications using brain data sarily hold true for large-scale multicenter studies, due to only following multiple linear regression of confounding heterogeneity that arises from pooling samples with dif- factors. No features were selected consistently (survived ferent scanning parameters, processing pipelines, inclu- false discovery rate (FDR) correction in >50% CV-folds) sion criteria, demographic and clinical characteristics14,21. for the main analyses (OCD patients vs. HC classification) All these factors can impact the data and obscure a Bruin et al. Translational Psychiatry (2020)10:342 Page 7 of 12 pattern of abnormalities shared by all patients. Mono- not provide inherent localization information (i.e., attri- center studies that minimize heterogeneity may therefore buting effect sizes to individual ROIs) as all features used yield higher classification performances, but limit the for prediction are considered as a whole. We derived generalizability to new, unseen data and its use in clinical individual feature importance from the RFC classifier practice16,17. Thus, whereas small monocenter studies using permutation-based inference to find brain features focus on answering a specific question about their patient that contributed both significantly and consistently (across population, large multicenter studies assume that a fun- CV folds) to classification performance. Feature impor- damental pattern of the disorder of interest can be tance was derived from classifications using neuroimaging detected despite the presence of heterogeneity, and both data after regression of covariates to avoid any undesirable are geared toward answering complementary questions effects on the interpretation of weights caused by non- about a particular disorder18,39. Our LOSO-CV results imaging features. No feature importance obtained for the demonstrated that structural MRI features do not provide main classifications (OCD patients vs. HC) in either a biomarker that enables generalization to new sites. pediatric, adult or combined samples was statistically sig- Multi-site classification within subgroups, split accord- nificant. This is likely due to the low classification per- ing to medication status, resulted in fair performance even formances obtained, suggesting that the features used are after accounting for correlated clinical variables (i.e., dis- either too noisy or non-informative for main diagnosis ease duration) through additional splits. Evidence from predictions and unable to achieve statistical significance rodent studies suggests that serotonin reuptake inhibitors and consistency across folds. Similarly, no significant fea- (SRIs) mediate neuroplasticity in various cortical and tures were found for medicated OCD vs. HC and unme- subcortical structures through glio-genesis and neuro- dicated OCD vs. HC classifications. On the other hand, genesis40–42. However, little is known about how these significant features were found that enabled multivariate findings might translate to humans and what the effects of classifications for medicated vs. unmedicated OCD in long-term medication use are43. The few longitudinal adult and combined samples. These included widespread studies with small samples suggest that SRI treatment cortical thickness in frontal and temporal regions, normalizes brain volumes. One study reported sig- including the left inferior temporal gyrus, medial orbital nificantly larger thalamic volumes in twenty-one treat- frontal and bilateral transverse temporal cortex, left insula, ment-naïve pediatric patients compared to HC and that and bilateral anterior cingulate cortex, as well as surface these differences decreased following paroxetine treat- area of the right entorhinal, left paracentral and bilateral ment44. Another study reported that smaller putamen temporal cortex, and left thalamus, pallidum, and ventricle volume in treatment-naïve patients was no longer volumes. A full overview of these significant features, as detectable in the thirteen patients that were treated with well as those found for early vs. late onset OCD classifi- fluoxetine45. Nonetheless, it remains unclear whether cation, can be found in supplementary Tables S10-11. these structural changes are related to medication use or The brain regions that were detected in the multivariate to symptom improvement. In contrast to these normal- analysis are partially consistent with the results from izing effects of SRIs, the previous univariate meta-analyses previous univariate ENIGMA-OCD meta-analyses and and mega-analyses of the ENIGMA-OCD study found mega-analyses6,7. Medicated adult OCD patients showed significantly thinner cortices in medicated adult OCD thinner frontal, temporal and parietal cortices, and patients and smaller cortical surface area in medicated smaller hippocampal and larger pallidum volumes com- pediatric OCD patients, but could not detect significant pared to HC, whereas no differences were found for differences in cortical and subcortical gray matter unmedicated adult patients. Although we were unable to between unmedicated OCD patients and HC, with the detect significant features for medicated patients and exception of larger thalamic volumes in unmedicated unmedicated patients vs. HC classifications, these earlier pediatric OCD patients6,7. Together, these cross-sectional results fit with the finding that classification performance studies suggest that medication use alters brain structure for medicated patients vs. HC was better than for rather than necessarily normalizing it to the level of unmedicated patients vs. HC, as differences in brain healthy individuals, an hypothesis which needs to be anatomy of unmedicated patients appear to be minimal. assessed in appropriately powered longitudinal studies. Interestingly, the classification between medicated vs. The identification of which brain regions contributed unmedicated patients was even better, which suggests that most to the classification resulted from a multivariate this results from the minimization of heterogeneity in analysis, and the localization of these regions should stratified patient groups as opposed to the larger hetero- therefore be interpreted with caution. MVPA techniques geneity seen in case-control comparisons. Finally, as the typically result in better discriminative ability between medication used for treating OCD is also used for treat- groups compared to standard univariate analyses by taking ment of many other psychiatric disorders, we anticipate the distributed nature of effects into account, but they do that these results are not specific to OCD. Bruin et al. Translational Psychiatry (2020)10:342 Page 8 of 12 Another point that deserves emphasis concerns the sites. Second, standardized FreeSurfer protocols were different ways of dealing with confounding non-imaging used for MR data processing to ensure reproducibility variables (e.g., age and sex) when using neuroimaging data across sites. It has been shown that FreeSurfer tends to for MVPA classification. Although recent studies were overestimate subcortical volumes in children48, and that unable to detect differences in predictive performance MR field strength can affect regional cortical estima- when comparing different approaches for dealing with tions49. However, these nonsystematic effects are expec- confounds in MVPA studies (nor differences in the ted to affect patients and HC equally and are therefore not weights assigned by the models46,47), results from our expected to influence our results. Third, limited infor- confound control experiments suggest otherwise. We mation on medication use was available. We were there- chose to add the covariates age, sex and data collection fore only able to distinguish patients on antidepressants site ID directly as features to our model as our initial with or without adjuvant antipsychotics vs. those who had approach. The underlying principle of this approach is not received any medication. Medication history, medi- that all relevant variables should be included in the model cation dosage, and duration of use were unknown. and that their relative contribution to the final predictive Nonetheless, these coarsely defined medication groups model will be recovered during model training, without enabled better case-control discrimination and good the need for manual confound adjustment procedures47. classification of medicated vs. unmedicated cases. Fourth, This approach resulted in high classification performance there is a lack of information on OCD subtypes in our for medication classifications (>0.8 AUC). However, sev- dataset. Particular OCD subtypes may have different eral sites included only medicated patients while in others neural correlates, and this might limit the ability of MVPA no patients had received medication, which could suggest models to find generalizable patterns in brain struc- that this high performance was achieved through classi- ture14,50. Fifth, it should be noted that the age cut-off used fiers detecting site-effects directly from covariates (e.g., to split the data in pediatric (age below 18) and adult (age site ID) rather than brain data. The latter is supported by 18 and older) samples may not be optimal with respect to the finding that classifications using covariates only (age, the development of the brain, but this was done in sex, and site) also resulted in high AUC (>0.8 AUC), accordance with the initial collection of pediatric and whereas regressing these covariates out from brain data adult samples and previous ENIGMA work51,52. Finally, it resulted in lower performance, with only classifications is possible that the brain features used for classification for medicated vs. control classification remaining sig- led to sub-optimal performance. OCD is thought to derive nificant (supplementary Tables S9). Our control experi- from abnormalities distributed at the network-level rather ments also show that the FreeSurfer data itself is likely to than focused on a single brain area, and FreeSurfer fea- be confounded by site-effects as well, as classifications tures might not be sufficiently sensitive to detect subtle using brain data only (without regressing out covariates) alterations associated with OCD. resulted in relatively high classification performance for Taken together, this study provides a realistic estimate medicated OCD vs. HC and medicated vs. unmedicated of the classification performance that can be achieved in a OCD classifications (with 0.70 and 0.75 AUC, respec- large, ecologically valid, multi-site sample of OCD parti- tively). This could be explained by classifiers being able to cipants using data on regional brain structure. Our find- identify sites through specific sample characteristics ings show that parcellated structural MRI data do not (demography and inclusions criteria used) resulting in enable a good overall distinction between patients with different brain anatomy, and methodical differences such OCD and HC. However, classifying subgroups of patients as types of scanner and imaging protocols used. Inter- based on medication status enables fair identification at estingly, control experiments for main diagnosis classifi- the individual subject level, which implies that medication cations showed that these were relatively unaffected by use is associated with substantial distributed differences in different ways of dealing with confounds (supplementary brain anatomy. This underlines the need for longitudinal Table S8). This is likely due to the fact that the classes for studies on the short-term and long-term effects of psy- diagnosis classifications (i.e., number of OCD patients and chiatric medication on brain structure. HC) are more balanced across samples. A number of limitations deserve emphasis. First, we Acknowledgements used a sample pooled from existing data across the world, The ENIGMA-Obsessive Compulsive Disorder Working-Group gratefully without harmonized protocols for scanning, inclusion acknowledges support from the NIH BD2K (Big Data to Knowledge) award U54 EB020403 (PI: Dr. Thompson) and Neuroscience Amsterdam, and an IPB grant criteria or demographic and clinical characteristics. These to Dr. Schmaal and Dr. van den Heuvel. Supported by the Japan Society for the sources of heterogeneity may limit classification perfor- Promotion of Science (JSPS; KAKENHI Grants No. 18K15523 to Dr. Abe, No. mance, but this also provides an opportunity for model 16K19778, No. 18K07608 to Dr. Nakamae, No. 16K04344, 19K03309 to Dr. Hirano, and No. 26461762 to Dr. Nakagawa); the Fundação de Amparo à development using independent data sets and the dis- Pesquisa do Estado de São Paulo (FAPESP, São Paulo Research Foundation; covery of biomarkers that are reproducible across study Grant No. 2011/21357-9; 2018/04654-9 and 2018/21934-5); the EU FP7 project Bruin et al. Translational Psychiatry (2020)10:342 Page 9 of 12 4 TACTICS (Grant No. 278948 to Dr. Buitelaar); the National Natural Science Department of Psychiatry, Graduate School of Medical Science, Kyoto Foundation of China (No. 81560233 to Dr. Cheng); the International Obsessive- Prefectural University of Medicine, Kyoto, Japan. 5Department of Psychiatry, Compulsive Disorder Foundation (IOCDF) Research Award to Dr. Gruner; the Bellvitge University Hospital, Bellvitge Biomedical Research Institute-IDIBELL, Dutch Organization for Scientific Research (NWO) (grants 912-02-050, 907-00- L’Hospitalet de Llobregat, Barcelona, Spain. 6Centro de Investigación 012, 940-37-018, and 916.86.038); the Netherlands Society for Scientific Biomèdica en Red de Salud Mental-CIBERSAM, Barcelona, Spain. 7Department Research (NWO-ZonMw VENI grant 916.86.036 to Dr. van den Heuvel; NWO- of Clinical Sciences, University of Barcelona, Barcelona, Spain. 8The Margaret ZonMw AGIKO stipend 920-03-542 to Dr. de Vries), a NARSAD Young and Wallace McCain Centre for Child, Youth and Family Mental Health, Investigator Award to Dr. van den Heuvel, and the Netherlands Brain Campbell Family Mental Health Research Institute, The Centre for Addiction Foundation (2010(1)-50 to Dr. van den Heuvel); the Federal Ministry of and Mental Health, Department of Psychiatry, Faculty of Medicine, University of Education and Research of Germany (No. BMBF-01GW0724 to Dr. Kathmann); Toronto, Toronto, Canada. 9Centre for Brain and Mental Health, The Hospital for the Alberta Innovates Translational Health Chair in Child and Youth Mental Sick Children, Toronto, Canada. 10Department of Psychiatry, Yale University Health and funding from the Ontario Brain Institute to Dr. Arnold; the School of Medicine, New Haven, CT 06510, USA. 11Mathison Centre for Mental Deutsche Forschungsgemeinschaft (DFG; Grant No. KO 3744/7-1 to Dr. Koch); Health Research and Education, Hotchkiss Brain Institute, Cumming School of the Helse Vest Health Authority (No. 911754, 911880 to Dr. Kvale) and the Medicine, University of Calgary, Calgary, Alberta, Canada. 12Department of Norwegian Research Council (No. HELSEFORSK 243675 to Dr. Kvale); the Psychiatry, Cumming School of Medicine, University of Calgary, Calgary, AB Wellcome Trust and a pump priming grant from the South London and T2N 1N4, Canada. 13Laboratory of Neuropsychiatry, Department of Clinical and Maudsley Trust, London, UK (Project Grant No. 064846 to Dr. Mataix-Cols); the Behavioral Neurology, IRCCS Santa Lucia Foundation, Rome, Italy. 14Psychiatry Generalitat de Catalunya (AGAUR 2017 SGR 1247 to Dr. Menchón); the PhD- and Clinical Psychobiology, Division of Neuroscience, Scientific Institute iHES program (FCT fellowship Grant No. PDE/BDE/113601/2015 to Dr. Moreira); Ospedale San Raffaele, Milano, Italy. 15Department of Psychology, Humboldt- the Japanese Ministry of Education, Culture, Sports, Science and Technology Universität zu Berlin, Berlin, Germany. 16Department of Clinical Neuroscience, (Grant-in-Aid for Scientific Research (C) 22591262, 25461732, 16K10253 to Dr. Centre for Psychiatric Research and Education, Karolinska Institutet, Stockholm, Nakao); the Italian Ministry of Health (No. RC13-14-15-16-17-18A to Dr. Sweden. 17Amsterdam UMC, Vrije Universteit Amsterdam, Department of Spalletta, Dr. Piras Fabrizio, Dr. Piras Federica); the National Center for Psychiatry, Amsterdam Neuroscience, Amsterdam, The Netherlands. 18 Advancing Translational Sciences (Grants No. UL1TR000067/KL2TR00069 to Dr. Amsterdam UMC, Vrije Universiteit Amsterdam, Department of Anatomy and Stern); the SA MRC funding and the National Research Foundation of South Neurosciences, Amsterdam Neuroscience, Amsterdam, The Netherlands. 19 Africa to Dr. Stein and Dr. Lochner; the Canadian Institutes of Health Research, Obsessive-Compulsive Disorder (OCD) Clinic Department of Psychiatry Michael Smith Foundation for Health Research, and British Columbia Provincial National Institute of Mental Health and Neurosciences, Bangalore, India. 20 Health Services Authority funding Dr. Stewart; the Netherlands Organization Department of Child and Adolescent Psychiatry and Psychotherapy, for Scientific Research (NWO/ZonMW Vidi 917.15.318 to Dr. van Wingen); the Psychiatric Hospital, University of Zurich, Zurich, Switzerland. 21University of Wellcome-DBT India Alliance (Grant No. 500236/Z/11/Z to Dr. Zurich and ETH Zurich, Neuroscience Center Zurich, Zurich, Switzerland. 22 Venkatasubramanian); the National Natural Science Foundation of China (No. McLean Hospital, Harvard Medical School, Belmont, MA 02115, USA. 23 81371340) and the Shanghai Key Laboratory of Psychotic Disorders (No. Donders Institute for Brain, Cognition and Behaviour, Radboud University, 13dz2260500) to Dr. Wang; the Government of India grants from the Nijmegen, Netherlands. 24Karakter Child and Adolescent Psychiatry University Department of Science and Technology (Grants No. SR/S0/HS/0016/2011 to Center, Nijmegen, The Netherlands. 25Department of Child and Adolescent Prof. Y.C. Janardhan Reddy, and DST INSPIRE faculty grant-IFA12-LSBM-26 to Dr. Psychiatry and Psychology, Institute of Neurosciences, Hospital Clínic Janardhanan C. Narayanaswamy) and from the Department of Biotechnology Universitari, Barcelona, Spain. 26Department of Medicine, University of (Grants No. BT/PR13334/Med/30/259/2009 to Prof. Y.C. Janardhan Reddy, and Barcelona, Barcelona, Spain. 27Centro de Investigación Biomédica en red de No. BT/06/IYBA/2012 to Dr. Janardhanan C. Narayanaswamy); the Dana Salud Mental (CIBERSAM), Barcelona, Spain. 28Department of Psychiatry, First Foundation and NARSAD to Dr. Fitzgerald; the NIMH (Grants No. R01 Affiliated Hospital of Kunming Medical University, Kunming, China. 29Institute MH107419 and No. K23MH082176 to Dr. Fitzgerald, Grant No. R21MH101441 to of Human Behavioral Medicine, SNU-MRC, Seoul, Republic of Korea. 30 Dr. Marsh, Grant No. R21MH093889 to Drs. Marsh and Simpson, Grant No. K23- Netherlands Institute for Neuroscience, Royal Netherlands Academy of Arts MH104515 to Dr. Baker; Grant No. R33MH107589 to Dr. Stern, Grant No. and Sciences, Amsterdam, The Netherlands. 31Department of Psychiatry, Icahn R01MH111794 to Dr. Stern; Grant No. R01 MH081864 to Drs. O’Neill and School of Medicine at Mount Sinai, New York, NY 10029, USA. 32Department of Piacentini; Grant No. R01 MH085900 to Drs. O’Neill and Feusner, Grant No. K23 Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, CA MH115206 to Dr. Gruner); the NIMH and the David Judah Fund at the 94612, USA. 33Department of Psychiatry, University of Michigan, Ann Arbor, MI Massachusetts General Hospital (Grant No. K23-MH092397 to Dr. Brennan); the 48109, USA. 34Department of Psychiatry and Mental Health, University of Cape Marató TV3 Foundation (Grants No. 01/2010 and 091710 to Dr. Lazaro); the Town, Cape Town, South Africa. 35Department of Neuroradiology, Klinikum Carlos III Health Institute (PI040829 to Dr Lazaro, Grant CPII16/00048 and rechts der Isar, Technische Universität München, München, Germany. 36TUM- Project Grants PI13/01958 and PI16/00889 to Dr. Soriano-Mas), co-funded by Neuroimaging Center (TUM-NIC) of Klinikum rechts der Isar, Technische FEDER funds/European Regional Development Fund (ERDF), a way to build Universität München, München, Germany. 37Max Planck UCL Centre for Europe; the AGAUR (2017 SGR 881 to Dr. Lazaro); the Michael Smith Computational Psychiatry and Ageing Research, London, UK. 38Wellcome Foundation for Health Research funding Dr. Jaspers-Fayer and Dr. Stewart; the Centre for Human Neuroimaging, University College London, London, UK. 39 Carlos III Health Institute (PI14/00419 to Dr. Alonso, Grant No. FI17/00294 to Research Center for Child Mental Development, Chiba University, Chiba, Ignacio Martínez-Zalacaín, Project Grant No. PI16/00950 to Dr. Menchón); the Japan. 40Departamento e Instituto de Psiquiatria do Hospital das Clinicas, IPQ Japan Agency for Medical Research and Development (AMED Brain/MINDS HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, SP, Beyond program Grant No. JP19dm0307002 to Dr. Shimizu); Oxford Health Brasil. 41Shanghai Mental Health Center Shanghai Jiao Tong University School Services Research Committee grant to Dr A James and National Institute on of Medicine, Shanghai, China. 42De Bascule, Academic Center for Child and Drug Abuse to Dr. Ivanov (NIH/NIDA Grant No. R21/DA046029 and No. R21/ Adolescent Psychiatry, Amsterdam, The Netherlands. 43Department of child DA045218); the Wellcome and Royal Society (211155/Z/18/Z) to Tobias U and adolescent psychiatry Amsterdam UMC, Amsterdam, The Netherlands. 44 Hauser; the Jacobs Foundation (2017-1261-04) to Tobias U Hauser, the Medical Division of Child and Adolescent Psychiatry, Department of Psychiatry, Icahn Research Foundation, and the Brain and Behavior Research Foundation (2018 School of Medicine at Mount Sinai, New York, NY 10029, USA. 45Department of NARSAD Young Investigator grant; 27023) to Tobias U. Hauser; the Swiss Psychiatry, , Oxford University, Oxford, UK. 46University of British Columbia, National Science Foundation (SNSF 320030_130237) to Susanne Walitza; the Vancouver, BC V6T 1Z4, Canada. 47Bergen Center for Brain Plasticity, Haukeland Harmann Müller foundation (1460) to Silvia Brem; NHMRC Career University Hospital, Bergen, Norway. 48Department of Clinical Psychology, Development Fellowship (1140764 to Dr. Schmaal). University of Bergen, Bergen, Norway. 49Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of Korea. 50 Department of Brain and Cognitive Sciences, Seoul National University Author details College of Natural Sciences, Seoul, Korea. 51SAMRC Unit on Risk and Resilience 1 Amsterdam UMC, University of Amsterdam, Department of Psychiatry, in Mental Disorders, Department of Psychiatry, Stellenbosch University, Amsterdam Neuroscience, Amsterdam, Netherlands. 2Department of Stellenbosch, South Africa. 52Institut d’Investigacions Biomèdiques August Pi i Physiology, Anatomy and Genetics, Oxford, UK. 3Department of mathematics Sunyer (IDIBAPS), Barcelona, Spain. 53Life and Health Sciences Research and applied mathematics, University of Cape Town, Cape Town, South Africa. Bruin et al. Translational Psychiatry (2020)10:342 Page 10 of 12 Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal. from the National Institute on Drug Abuse in the last 3 years. Dr. Pittenger has 54 ICVS/3B’s, PT Government Associate Laboratory, Braga/Guimarães, Portugal. received research support and/or honoraria for consultation from Biohaven 55 Clinical Academic Center-Braga, Braga, Portugal. 56Columbia University Irving Pharmaceuticals, Blackthorn Therapeutics, Abide Therapeutics, and Brainsway, Medical Center, Columbia University, New York, NY 10027, USA. 57The Division and royalties or honoraria from Oxford University Press and Elsevier in the past of Child and Adolescent Psychiatry, New York State Psychiatric Institute, 3 years. Dr. Feusner has received an honorarium from Pfizer and consultation Columbia University, New York, NY 10027, USA. 58Department of Clinical fees from NOCD, Inc. Dr. Piacentini has received research support from Pfizer Neuroscience, Centre for Psychiatry Research, Karolinska Institutet, Stockholm, Pharmaceuticals for research unrelated to the topic of this manuscript. Dr. Sweden. 59Department of Psychiatry and Behavioural Neurosciences, McMaster Soreni has received support for Investigator Initiated Clinical Trial from University, Hamilton, ON L8S 4L8, Canada. 60Department of Neuropsychiatry, Lundbeck LLC unrelated to the topic of this study. Dr. Buitelaar has been in the Graduate School of Medical Sciences, Kyushu University, Fukuoka, Japan. past 3 years a consultant to/member of advisory board of/and/or speaker for 61 Division of Child and Adolescent Psychiatry, Jane and Terry Semel Institute Shire, Roche, Medice, and Servier. He is not an employee of any of these For Neurosciences, University of California, Los Angeles, CA 94612, USA. companies, and not a stock shareholder of any of these companies. He has no 62 Magnetic Resonance Image Core Facility, IDIBAPS (Institut d’Investigacions other financial or material support, including expert testimony, patents, Biomèdiques August Pi i Sunyer), Barcelona, Spain. 63University of Illinois at royalties. Dr. Mataix-Cols receives royalties for contributing articles to Urbana-Champaign, Champaign, IL 61820, USA. 64University of Zürich, UpToDate (Wolters Kluwer Health), and for editorial work from Elsevier, all University Hospital Zürich, Dept. Neuroradiology, Zürich, Switzerland. unrelated to the current work. In the last three years, Dr. Simpson has received 65 University Department of Geriatric Medicine Felix Platter, Basel, Switzerland. research support for an industry-sponsored clinical trial from Biohaven 66 ATR Brain Information Communication Research Laboratory Group, Kyoto, Pharmaceuticals, royalties from UpToDate, Inc, and a stipend from JAMA Japan. 67Center of Mathematics, Computing and Cognition, Universidade Psychiatry for her role as Associate Editor. Dr. Tolin has received research Federal do ABC, Santo Andre, Brazil. 68Orygen, Parkville, VIC, Australia. 69Centre support from Biohaven Pharmaceuticals. All other individually-named authors for Youth Mental Health, The University of Melbourne, Melbourne, VIC 3052, in- and outside of the ENIGMA-OCD working group reported no biomedical Australia. 70Department of Cognitive Behavioral Physiology, Graduate School of financial interests or potential conflicts of interest. Medicine, Chiba University, Chiba, Japan. 71Center for OCD and Related Disorders, New York State Psychiatric Institute, New York, NY 10032, USA. 72 Pediatric OCD Consultation service, Anxiety Treatment and Research Center, Publisher’s note St. Joseph’s HealthCare, Hamilton, ON L9C 0E3, Canada. 73Offord Child Center, Springer Nature remains neutral with regard to jurisdictional claims in McMaster University, Hamilton, ON L8S 4L8, Canada. 74Department of published maps and institutional affiliations. Psychobiology and Methodology of Health Sciences, Universitat Autònoma de Barcelona, Barcelona, Spain. 75Beth K. and Stuart C. Yudofsky Division of Supplementary Information accompanies this paper at (https://doi.org/ Neuropsychiatry, Department of Psychiatry and Behavioral Sciences, Baylor 10.1038/s41398-020-01013-y). College of Medicine, Houston, TX 77030, USA. 76Department of Psychiatry, New York University Langone School of Medicine, New York, NY 10016, USA. 77 Nathan Kline Institute for Psychiatric Research, Orangeburg, NY 10962, USA. Received: 30 March 2020 Revised: 9 September 2020 Accepted: 14 78 Olin Neuropsychiatry Research Center, Hartford Hospital, Hartford, CT 06106, September 2020 USA. 79Department of Psychiatry, Yale University School of Medicine, New Haven 06510, USA. 80British Columbia Children’s Hospital, Vancouver, BC V6H 3N1, Canada. 81British Columbia Mental Health and Addictions Research Institute, Vancouver, BC V6H 3N1, Canada. 82Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA. 83James J. Peters VA Medical Center, References Bronx, New York, NY 10468, USA. 84Institute of Living/Hartford Hospital, 1. Ruscio, A. M. et al. The epidemiology of obsessive-compulsive disorder in Hartford, CT 06119, USA. 85Yale University School of Medicine, New Haven, CT the National Comorbidity Survey Replication. Psychiatry Res. 15, 513–517 06510, USA. 86Shanghai Key Laboratory of Psychotic Disorders, Shanghai, (2010). China. 87Seoul National University Hospital, Seoul, Republic of Korea. 2. American Psychiatric Association. Diagnostic and statistical manual of mental 88 Yeongeon Student Support Center, Seoul National University College of disorders, 5th edn. (DSM-5). Diagn. Stat. Man. Ment. Disord. 280, https://doi. Medicine, Seoul, Republic of Korea. 89Department of Cognitive Neuroscience, org/10.1176/appi.books.9780890425596.744053. (2013). Donders Institute for Brain, Cognition and Behaviour, Donders Centre for 3. Abramowitz, J. S., Taylor, S. & McKay, D. Obsessive-compulsive disorder. Lancet Cognitive Neuroimaging, Radboud University Medical Centre, Nijmegen, The 374, 491–499 (2009). Netherlands. 90Imaging Genetics Center, Stevens Institute for Neuroimaging 4. Menzies, L. et al. Integrating evidence from neuroimaging and neu- and Informatics, Keck School of Medicine, University of Southern California, Los ropsychological studies of obsessive-compulsive disorder: the orbitofronto- Angeles, CA 90007, USA. 91SAMRC Unit on Risk and Resilience in Mental striatal model revisited. Neurosci. Biobehav. Rev. 32, 525–549 (2008). Disorders, Department of Psychiatry and Neuroscience Institute, University of 5. van den Heuvel, O. A. et al. Brain circuitry of compulsivity. Eur. Neuropsycho- Cape Town, Cape Town, South Africa pharmacol. 26, 810–827 (2016). 6. Boedhoe, P. S. W. et al. Distinct subcortical volume alterations in pediatric and Code availability adult OCD: a worldwide meta- and mega-analysis. Am. J. Psychiatry 174, 60–69 The computer code for the above-described analyses is publicly available (2017). (https://github.com/WillemB2104/ENIGMA-OCD-2020). 7. Boedhoe, P. S. W. et al. Cortical abnormalities associated with pediatric and adult obsessive-compulsive disorder: findings from the ENIGMA obsessive- Conflict of interest compulsive disorder working group. Am. J. Psychiatry 175, 453–462 (2018). Dr. Baker has received research support from the National Institute of Mental 8. Kong, X. Z. et al. Mapping cortical and subcortical asymmetry in obsessive- Health (NIMH) and Valera Health. Dr. Brennan has received consulting fees compulsive disorder: findings from the ENIGMA Consortium. Biol. Psychiatry from Rugen Therapeutics and Nobilis Therapeutics and research grant support 1–13 https://doi.org/10.1016/j.biopsych.2019.04.022. (2019). from Eli Lilly, Transcept Pharmaceuticals, and Biohaven Pharmaceuticals. Dr. 9. de Wit, S. J. et al. Multicenter voxel-based morphometry multi-center mega- Walitza has received lecture honoraria Opopharma in the last 3 years. Her work analysis of structural brain scans in obsessive-compulsive disorder. Am. J. was supported in the last 3 years by the Swiss National Science Foundation Psychiatry 171, 340–349 (2013). (SNF), diverse EU FP7s, HSM Hochspezialisierte Medizin of the Kanton Zurich, 10. Rotge, J. Y. et al. Provocation of obsessive-compulsive symptoms: a quanti- Switzerland, Bfarm Germany, Zinep, Hartmann Müller Stiftung, Olga tative voxel-based meta-analsysis of functional neuroimaging studies. J. Psy- Mayenfisch. Dr. Dan J. Stein has received research grants and/or consultancy chiatry Neurosci. 33, 405 (2008). honoraria from Lundbeck and Sun in the past 3 years. Dr. Paul M. Thompson 11. Orrù, G., Pettersson-Yeo, W., Marquand, A. F., Sartori, G. & Mechelli, A. Using has received research grant support from Biogen, Inc., for research unrelated to Support Vector Machine to identify imaging biomarkers of neurological and the topic of this manuscript. Dr. Ivanov has received honoraria from Lundbeck psychiatric disease: a critical review. Neurosci. Biobehav. Rev. 36, 1140–1152 as a member of the Data Safety Monitoring Committee and research grants (2012). Bruin et al. Translational Psychiatry (2020)10:342 Page 11 of 12 12. Arbabshirani, M. R., Plis, S., Sui, J. & Calhoun, V. D. Single subject prediction of 33. Bradley, A. P. The use of the area under the ROC curve in the evaluation of brain disorders in neuroimaging: promises and pitfalls. Neuroimage https://doi. machine learning algorithms. Pattern Recognit. 30, 1145–1159 (1997). org/10.1016/j.neuroimage.2016.02.079. (2016). 34. Abraham, A. et al. Deriving reproducible biomarkers from multi-site resting- 13. Klöppel, S. et al. Automatic classification of MR scans in Alzheimer’s disease. state data: an Autism-based example. Neuroimage 147, 736–745 (2017). Brain 131, 681–689 (2008). 35. Hanley, J. A. & McNeil, B. J. The meaning and use of the area under a receiver 14. Bruin, W., Denys, D. & van Wingen, G. Diagnostic neuroimaging markers of operating characteristic (ROC) curve. Radiology 143, 29–36 (1982). obsessive-compulsive disorder: Initial evidence from structural and functional 36. Mason, S. J. & Graham, N. E. Areas beneath the relative operating character- MRI studies. Prog. Neuro-Psychopharmacology Biol. Psychiatry 0–1 https://doi. istics (ROC) and relative operating levels (ROL) curves. Q. J. R. Meteorol. Soc. org/10.1016/j.pnpbp.2018.08.005. (2018). 128, 2145–2166 (2002). 15. Varoquaux, G. Cross-validation failure: small sample sizes lead to large error 37. Storch, E. A. et al. Defining clinical severity in adults with obsessive-compulsive bars. Neuroimage 1–10 https://doi.org/10.1016/j.neuroimage.2017.06.061. disorder. Compr. Psychiatry 63, 30–35 (2015). (2017). 38. Strobl, C., Boulesteix, A. L., Zeileis, A. & Hothorn, T. Bias in random forest 16. Woo, C.-W., Chang, L. J., Lindquist, M. A. & Wager, T. D. Building better bio- variable importance measures: illustrations, sources and a solution. BMC markers: brain models in translational neuroimaging. Nat. Neurosci. Rev. 20, Bioinformatics 8, 25 (2007). 365–377 (2017). 39. Thomas, R. M., Bruin, W., Zhutovsky, P. & van Wingen, G. Dealing with missing 17. Wolfers, T., Buitelaar, J. K., Beckmann, C. F., Franke, B. & Marquand, A. F. From data, small sample sizes, and heterogeneity in machine learning studies of estimating activation locality to predicting disorder: A review of pattern brain disorders. In Machine Learning 249–266 (Elsevier, 2020). recognition for neuroimaging-based psychiatric diagnostics. Neurosci. Biobe- 40. Czéh, B. et al. Chronic social stress inhibits cell proliferation in the adult medial hav. Rev. 57, 328–349 (2015). prefrontal cortex: hemispheric asymmetry and reversal by fluoxetine treat- 18. Nunes, A. Using structural MRI to identify bipolar disorders–13 site machine ment. Neuropsychopharmacology https://doi.org/10.1038/sj.npp.1301275. learning study in 3020 individuals from the ENIGMA Bipolar Disorders Working (2007). Group. Mol. Psychiatry https://doi.org/10.1038/s41380-018-0228-9. (2018). 41. Kodama, M., Fujioka, T. & Duman, R. S. Chronic olanzapine or fluoxetine 19. Nieuwenhuis, M. et al. Classification of schizophrenia patients and healthy administration increases cell proliferation in hippocampus and prefrontal controls from structural MRI scans in two large independent samples. Neu- cortex of adult rat. Biol. Psychiatry https://doi.org/10.1016/j. roimage 61, 606–612 (2012). biopsych.2004.07.008 (2004). 20. Franke, K., Ziegler, G., Klöppel, S. & Gaser, C. Estimating the age of healthy 42. Soumier, A. et al. Mechanisms contributing to the phase-dependent regula- subjects from T1-weighted MRI scans using kernel methods: exploring the tion of neurogenesis by the novel antidepressant, agomelatine, in the adult rat influence of various parameters. Neuroimage 50, 883–892 (2010). hippocampus. Neuropsychopharmacology https://doi.org/10.1038/npp.2009.72. 21. Schnack, H. G. & Kahn, R. S. Detecting neuroimaging biomarkers for psychiatric (2009). disorders: Sample size matters. Front. Psychiatry 7, 50 (2016). 43. Homberg, J. R., Schubert, D. & Gaspar, P. New perspectives on the neurode- 22. Boedhoe, P. S. W. et al. An empirical comparison of meta- and mega-analysis velopmental effects of SSRIs. Trends Pharmacol. Sci. 31, 60–65 (2010). with data from the ENIGMA Obsessive-Compulsive Disorder Working Group. 44. Gilbert, A. R. et al. Decrease in thalamic volumes of pediatric patients with Front. Neuroinform. 12, 1–8 (2019). obsessive-compulsive disorder who are taking paroxetine. Arch. Gen. Psychiatry 23. Silverman, W. K., Saavedra, L. M. & Pina, A. A. Test-retest reliability of anxiety 57, 449 (2003). symptoms and diagnoses with the anxiety disorders interview schedule for 45. Hoexter, M. Q. et al. Gray matter volumes in obsessive-compulsive disorder DSM-IV: child and parent versions. J. Am. Acad. Child Adolesc. Psychiatry https:// before and after fluoxetine or cognitive-behavior therapy: a randomized doi.org/10.1097/00004583-200108000-00016. (2001). clinical trial. Neuropsychopharmacology 37, 734–745 (2012). 24. Kaufman, J. et al. Schedule for affective disorders and schizophrenia for school- 46. Linn, K. A., Gaonkar, B., Doshi, J., Davatzikos, C. & Shinohara, R. T. Multivariate age children-present and lifetime version (K-SADS-PL): initial reliability and pattern analysis and confounding in neuroimaging. Int. J. Biostat. 12, 31 (2016). validity data. J. Am. Acad. Child Adolesc. Psychiatry 36, 980–988 (1997). 47. Rao, A., Monteiro, J. M. & Mourao-Miranda, J. Predictive modelling using 25. Sheehan, D. V. et al. Reliability and validity of the mini international neu- neuroimaging data in the presence of confounds. Neuroimage 150, 23–49 ropsychiatric interview for children and adolescents (MINI-KID). J. Clin. Psy- (2017). chiatry 71, 313–326 (2010). 48. Schoemaker, D. et al. Hippocampus and amygdala volumes from magnetic 26. First, M. B. Structured Clinical Interview for the DSM (SCID). In The Encyclopedia resonance images in children: assessing accuracy of FreeSurfer and FSL of Clinical Psychology 1–6 (John Wiley & Sons, Inc., 2015). against manual segmentation. Neuroimage https://doi.org/10.1016/j. 27. Sheehan, D. V. et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): neuroimage.2016.01.038 (2016). the development and validation of a structured diagnostic psychiatric inter- 49. Govindarajan, K. A., Freeman, L., Cai, C., Rahbar, M. H. & Narayana, P. A. Effect of view for DSM-IV and ICD-10. in. J. Clin. Psychiatry 59, 22–33 (1998). intrinsic and extrinsic factors on global and regional cortical thickness. PLoS 28. Desikan, R. S. et al. An automated labeling system for subdividing the human ONE 9, 1–15 (2014). cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 50. Mataix-Cols, D. & van den Heuvel, O. A. Common and distinct neural correlates 31, 968–980 (2006). of obsessive-compulsive and related disorders. Psychiatr. Clin. North Am. 29, 29. Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning (2nd 391–410 (2006). edn.) (Elements, 2009). 51. Leckman, J. F. et al. Obsessive-compulsive disorder: a review of the diagnostic 30. Rasmussen, C. E. & Williams, C. K. I. Gaussian processes for machine learning. Int. criteria and possible subtypes and dimensional specifiers for DSM-V. Depres- J. Neural Syst. 14, 69–106 (2004). sion Anxiety https://doi.org/10.1002/da.20669 (2010). 31. Chang, C. & Lin, C. LIBSVM: a library for support vector machines. ACM Trans. 52. Anholt, G. E. et al. Age of onset in obsessive-compulsive disorder: admixture Intell. Syst. Technol. 2, 1–39 (2013). analysis with a large sample. Psychol. Med. https://doi.org/10.1017/ 32. Chen, T. & Guestrin, C. XGBoost: a scalable tree boosting system. In Proceedings S0033291713000470. (2014). of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining vols 13-17-785–794 (ACM Press, 2016). Author details ENIGMA-OCD Working Group Yoshinari Abe 4, Pino Alonso5,6,7, Stephanie H. Ameis8,9, Alan Anticevic10, Paul D. Arnold 11,12, Francesca Assogna13, Nerisa Banaj13, Nuria Bargalló62,92, Marcelo C. Batistuzzo40, Francesco Benedetti 14, Jan C. Beucke15,16, Premika S. W. Boedhoe17,18, Irene Bollettini14, Anushree Bose19, Daniel Brandeis93,94,95, Bruin et al. Translational Psychiatry (2020)10:342 Page 12 of 12 Silvia Brem20,21, Brian P. Brennan22, Willem B. Bruin 1, Jan K. Buitelaar 23,24, Geraldo F. Busatto96,97,98, Anna Calvo62, Rosa Calvo 25,26,27, Yuqi Cheng28, Kang Ik K. Cho29, Valentina Ciullo13,99, Sara Dallaspezia14, Damiaan Denys 1,30, Renate Drechsler100, Benjamin A. Ely 31, Madalena Esteves53,54,55, Andrea Falini101, Yu Fang102, Jamie D. Feusner32, Martijn Figee1,103, Kate D. Fitzgerald33, Martine Fontaine56, Jean-Paul Fouche34, Egill A. Fridgeirsson1, Patricia Gruner10, Margot Gueguen104, Deniz A. Gürsel35,36, Sayo Hamatani39, Gregory L. Hanna102, Bjarne Hansen47,48, Tobias U. Hauser 20,37,38, Yoshiyuki Hirano 39, Marcelo Q. Hoexter40, Hao Hu41, Chaim Huyser 42,43, Keisuke Ikari60, Iliyan Ivanov44, Anthony James 45, Fern Jaspers-Fayer46, Norbert Kathmann15, Christian Kaufmann15, Kathrin Koch35,36, Masaru Kuno39, Gerd Kvale47,48, Jun Soo Kwon 49,50, Yanni Liu33, Christine Lochner51, Luisa Lázaro25,52,26,27, Ricardo Magalhães53,54,55, Paulo Marques53,54,55, Rachel Marsh56,57, Ignacio Martínez-Zalacaín5,7, Yasutaka Masuda105, David Mataix-Cols58, Koji Matsumoto105, José M. Menchón 5,6,7, Euripedes C. Miguel40, Luciano Minuzzi59, Pedro S. Moreira53,54,55, Astrid Morer25,52,26,27, Pedro Morgado 53,54,55, Akiko Nakagawa39, Takashi Nakamae 4, Tomohiro Nakao60, Janardhanan C. Narayanaswamy19, Erika L. Nurmi32, Joseph O’Neill61, Jose C. Pariente62, Chris Perriello22,63, John Piacentini 32, Fabrizio Piras 13, Federica Piras13, Christopher Pittenger106, Sara Poletti14, Y. C. Janardhan Reddy19, Oana G. Rus-Oswald 64,65, Yuki Sakai66,4, João R. Sato67, Lianne Schmaal 68,69, Eiji Shimizu39,70, Jonathan P. Shock 3, H. Blair Simpson56,71, Noam Soreni72,73, Carles Soriano-Mas 5,6,74, Nuno Sousa53,54,55, Gianfranco Spalletta13,75, Dan J. Stein 91, Emily R. Stern76,77, Michael C. Stevens 78,79, S. Evelyn Stewart46,80,81, Philip R. Szeszko82,83, Jumpei Takahashi39, Luke Taylor2, Rajat M. Thomas1, Paul M. Thompson90, Anders L. Thorsen47,48,107, David F. Tolin84,85, Aki Tsuchiyagaito39,108, Daniela Vecchio13, Dick J. Veltman17, Ganesan Venkatasubramanian 19, Susanne Walitza20, Zhen Wang 41,86, Anri Watanabe4, Xiufeng Xu28, Jian Xu109, Kei Yamada110, Tokiko Yoshida39, Je-Yeon Yun 87,88, Mojtaba Zarei111, Qing Zhao41, Cong Zhou28, Froukje E. de Vries17, Stella J. de Wit17, Daan van Rooij89, Guido A. van Wingen1, Odile A. van den Heuvel17,18 and Ysbrand D. van der Werf18 92 Image Diagnostic Center, Hospital Clínic, Barcelona, Spain. 93Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric Hospital, University of Zurich, Zurich, Switzerland. 94Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Mannheim, Medical Faculty Mannheim/Heidelberg University, Mannheim, Germany. 95Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland. 96Laboratory of Psychiatric Neuroimaging (LIM21), Hospital das Clinicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, Brazil. 97Department and Institute of Psychiatry, Faculty of Medicine, University of Sao Paulo, Sao Paulo, Brazil. 98Center for Interdisciplinary Research on Applied Neurosciences (NAPNA), University of Sao Paulo, Sao Paulo, Brazil. 99Department of Neurosciences, Psychology, Drug Research and Child Health (NEUROFARBA), University of Florence, Firenze, Italy. 100 Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric Hospital, University of Zurich, Zurich, Switzerland. 101Neuroradiology, Division of Neuroscience, Scientific Institute Ospedale San Raffaele, Milano, Italy. 102Department of Psychiatry, University of Michigan, Ann Arbor, MI 48109, USA. 103 Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York 10029, USA. 104Department of Anatomy and Neurosciences, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Neuroscience, Amsterdam, The Netherlands. 105Department of Radiology, Chiba University Hospital, Chiba, Japan. 106 Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06510, USA. 107Amsterdam UMC, Vrije Universteit Amsterdam, Department of Anatomy and Neurosciences, Amsterdam, The Netherlands. 108Laureate Institute for Brain Research, Tulsa, OK 74136, USA. 109Department of Internal Medicine, First Affiliated Hospital of Kunming Medical University, Kunming, China. 110Department of Radiology, Graduate School of Medical Science Kyoto Prefectural University of Medicine, Kyoto, Japan. 111Institute of Medical Science and Technology, Shahid Beheshti University, Tehran, Iran