PLOS ONE RESEARCH ARTICLE Neuromodulatory effects and reproducibility of the most widely used repetitive transcranial magnetic stimulation protocols Justine Magnuson1,2,3, Mehmet A. Ozdemir ID1,2,4, Elon Mathieson1, Sofia Kirkman1, Brice Passera1,2, Sumientra Rampersad5,6, Alyssa B. Dufour7,8, Dana Brooks ID6, Alvaro Pascual-Leone ID2,9,10, Peter J. Fried1,2, Mouhsin M. Shafi1,2☯*, Recep A. Ozdemir ID1,2☯* 1 Berenson-Allen Center for Noninvasive Brain Stimulation, Department of Neurology, Beth Israel a1111111111 Deaconess Medical Center, Boston, MA, United States of America, 2 Department of Neurology, Harvard a1111111111 Medical School, Boston, MA, United States of America, 3 Health and Exercise Sciences, University of British a1111111111 Columbia Okanagan, Kelowna, BC, CA, 4 Department of Biomedical Engineering, Izmir Katip Celebi a1111111111 University, Izmir, Turkey, 5 Department of Physics, University of Massachusetts, Boston, MA, United States a1111111111 of America, 6 Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, United States of America, 7 Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States of America, 8 Hinda and Arthur Marcus Institute for Aging Research, Hebrew Senior Life, Boston, MA, United States of America, 9 Hinda and Arthur Marcus Institute for Aging Research and Deanne and Sidney Wolk Center for Memory Health, Hebrew SeniorLife, Boston, MA, United States of America, 10 Guttmann Brain Health Institute, Institut Guttmann de Neurorehabilitació, OPEN ACCESS Universitat Autonoma de Barcelona, Badalona, Spain Citation: Magnuson J, Ozdemir MA, Mathieson E, ☯ These authors contributed equally to this work. Kirkman S, Passera B, Rampersad S, et al. (2023) * rozdemir@bidmc.harvard.edu (RAO); mshafi@bidmc.harvard.edu (MMS) Neuromodulatory effects and reproducibility of the most widely used repetitive transcranial magnetic stimulation protocols. PLoS ONE 18(6): e0286465. https://doi.org/10.1371/journal.pone.0286465 Abstract Editor: François Tremblay, University of Ottawa, CANADA Received: February 3, 2023 Background Accepted: May 16, 2023 Repetitive transcranial magnetic stimulation (rTMS) is widely used in both research and clin- Published: June 23, 2023 ical settings to modulate human brain function and behavior through the engagement of the mechanisms of plasticity. Based upon experiments using single-pulse TMS as a probe, the Copyright: © 2023 Magnuson et al. This is an open access article distributed under the terms of the physiologic mechanism of these effects is often assumed to be via changes in cortical excit- Creative Commons Attribution License, which ability, with 10 Hz rTMS increasing and 1 Hz rTMS decreasing the excitability of the stimu- permits unrestricted use, distribution, and lated region. However, the reliability and reproducibility of these rTMS protocols on cortical reproduction in any medium, provided the original excitability across and within individual subjects, particularly in comparison to robust sham author and source are credited. stimulation, have not been systematically examined. Data Availability Statement: Raw de-identified data files are already uploaded to National Institute of Mental Health archive in accordance with the rules of the local ethics committee and can be Objectives accessed in the following link; https://nda.nih.gov/ In a cohort of 28 subjects (39 ± 16 years), we report the first comprehensive study to (1) edit_collection.html?id=2944. assess the neuromodulatory effects of traditional 1 Hz and 10 Hz rTMS on corticospinal Funding: Dr. Pascual-Leone is supported by grants excitability against both a robust sham control, and two other widely used patterned rTMS from the National Institutes of Health protocols (intermittent theta burst stimulation, iTBS; and continuous theta burst stimulation, (R24AG06142, and P01 AG031720), the National Science Foundation, and the Barcelona Brain cTBS), and (2) determine the reproducibility of all rTMS protocols across identical repeat Health Initiative funded primarily by La Caixa. Dr. sessions. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 1 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Shafi is supported by the Football Players Health Results Study (FPHS) at Harvard University, and the National Institute of Health (R01 MH115949, At the group level, neither 1 Hz nor 10 Hz rTMS significantly modulated corticospinal excit- R01AG060987, P01 AG031720-06A1). The content ability. 1 Hz and 10 Hz rTMS were also not significantly different from sham and both TBS of this paper is solely the responsibility of the protocols. Reproducibility was poor for all rTMS protocols except for sham. Importantly, authors and does not necessarily represent the official views of Harvard University and its affiliated none of the real rTMS and TBS protocols demonstrated greater neuromodulatory effects or academic health care centers, or the National reproducibility after controlling for potential experimental factors including baseline corti- Institutes of Health. cospinal excitability, TMS coil deviation and the number of individual MEP trials. Competing interests: A. Pascual-Leone is a co- founder of Linus Health and TI Solutions AG; Conclusions serves on the scientific advisory boards for Starlab Neuroscience, Magstim Inc., Hearts Radiant, These results call into question the effectiveness and reproducibility of widely used rTMS TetraNeuron and MedRhythms; and is listed as an techniques for modulating corticospinal excitability, and suggest the need for a fundamental inventor on several issued and pending patents on rethinking regarding the potential mechanisms by which rTMS affects brain function and methods and applications for noninvasive brain stimulation, and the real-time integration of behavior in humans. noninvasive brain stimulation with electroencephalography and magnetic resonance imaging (Patent Number: 6571123, Method and apparatus for recording an electroencephalogram during transcranial magnetic stimulation, Inventors: John R. Ives, Alvaro Pascual-Leone, Introduction Date of patent: May 27, 2003). None of these Over the past several decades, repetitive transcranial magnetic stimulation (rTMS) has become companies or interests influenced in any way the the most widely used non-invasive technique for modifying human brain function in both work reported and do not alter our adherence to PLOS ONE policies on sharing data and materials. health and disease. Conventional rTMS protocols consist of repeated trains of pulses delivered at low (1 Hz) or high frequencies (� 10 Hz) [1, 2], with the goal of suppressing (inhibitory) or facilitating (excitatory) neural excitability, respectively [3–5]. More recently, adopted from ani- mal studies, accelerated high-frequency rTMS protocols called theta burst stimulation (TBS) have been developed to induce similar inhibitory and excitatory neuromodulatory effects [6]. These TBS protocols have received increased attention in recent years owing to their shorter durations and low stimulation intensities. The widespread use of all rTMS protocols in both clinical and experimental research is motivated by the assumption that they can induce lasting and reliable changes in neuronal function (e.g., excitability) in the stimulated brain region (e.g., motor cortex) and its connected circuits, and thus modulate corresponding behaviors and/or cortical responses (e.g., motor-evoked potentials, MEPs). While initial studies reported expected neuromodulatory effects [6, 7] and partial reproduc- ibility of certain rTMS protocols [8], growing evidence consistently suggests that neuromodu- latory effects of rTMS are highly variable across subjects, protocols and sessions. Recent work has extensively focused on the effectiveness and reproducibility of TBS protocols and reported both high inter- and intra-individual response variability with poor reproducibility across repeat TBS sessions [9–16]. Surprisingly, however, the only study assessing intra-individual reliability of traditional 1 Hz and 10 Hz rTMS protocols was reported more than two decades ago [8] in a relatively smaller sample size (n = 20), with no delayed post-rTMS measurements, on consecutive days when meta-plasticity effects might be present, and, most importantly, without having a robust sham rTMS control. Our recent work using intermittent and continu- ous TBS protocols demonstrated that most of the significant TBS aftereffects on cortical and corticospinal responses were not statistically different from sham TBS [12]. More importantly, none of the significant neuromodulatory effects of TBS were reproducible across identical ses- sions [12]. These results endorse sham controls as an essential part of any rTMS design and suggest that the neuromodulatory effects and reproducibility of the two most widely used clini- cal rTMS protocols (1 Hz and 10 Hz) need to be validated against sham stimulation. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 2 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Given the lack of sham-controlled studies assessing reproducibility of 1 Hz and 10 Hz rTMS protocols, we performed a comprehensive study to (1) assess the neuromodulatory effects of traditional low- and high-frequency rTMS protocols against a robust sham control, (2) compare their neuromodulatory effects and reproducibility with two widely used intermit- tent (iTBS) and continuous TBS (cTBS) protocols in a single cohort, and (3) determine the reproducibility of their neuromodulatory effects across identical repeat sessions. We applied 1 Hz, 10 Hz, iTBS, cTBS and sham-rTMS to the hand region of the left motor cortex (M1), where the neuromodulatory effects of rTMS are typically studied. We measured peak-to-peak MEP amplitude in response to single-pulse TMS (spTMS), as it is the most commonly used metric in both experimental and clinical TMS research, to index immediate (5 min post rTMS) and delayed (25 min) changes in corticospinal excitability following each rTMS proto- col. Furthermore, as MEP responses are reported to be sensitive to a collection of methodolog- ical and individual factors [17], we performed a series of control analyses to examine whether (1) coil position deviations from the motor hotspot, (2) the number of MEP trials in a given block, and (3) the magnitude of baseline (pre-rTMS) corticospinal excitability affect neuromo- dulatory rTMS responses and their reproducibility. Methods Participants Data from 28 healthy participants (10 females, 39 ± 16 years old) across 10 sessions for each participant (280 sessions) were analyzed. All participants met the inclusion criteria, including no known history of psychiatric or neurological diseases, no systemic illness, no contraindica- tions to TMS or magnetic resonance imaging (MRI), and no drug abuse or dependence as measured by the mini-international neuropsychiatric interview (MINI). All experimental pro- tocols were approved by the Institutional Review Board of the Beth Israel Deaconess Medical Center (Boston, MA) and explained to participants while obtaining written informed consent. Data acquisition Real and sham TMS were applied with a 75-mm (outer wing diameter) MagPro Cool B-65 coil and A/P Cool-B65 coil, respectively, attached to a MagPro X100 stimulator (MagVenture A/ S). High-resolution anatomical MRIs were obtained for each subject and used for targeting TMS within and across sessions using the neuronavigation system Brainsight1 (Rogue Research Inc). Electromyography (EMG) was recorded from the right first dorsal interosseous (FDI) muscle and digitized using the BrainVision actiCHamp amplifier system and recorder software (Version 1.20.0601, Brain Products GmbH). Ag-AgCl surface electrode pairs were placed in a monopolar fashion over the FDI belly and tendon, with a ground electrode placed on the right ulnar styloid. EMG data were sampled at a rate of 5000 Hz and visualized over a 150 ms window length from 50 ms before to 100 ms after the TMS pulse. Experimental procedures This report is a part of a larger TMS-EEG-EMG-behavior study, with the current manuscript focusing on the MEP results to evaluate cortico-spinal excitability (Fig 1). All participants underwent ten separate experimental visits consisting of five rTMS conditions: 1 Hz, 10 Hz, iTBS, cTBS, and sham-rTMS (randomly assigned to one of the four real rTMS protocols) repeated 1 month apart. Sham-rTMS was randomly selected among the four real rTMS proto- cols for each participant (1 Hz, 10 Hz, iTBS, cTBS) to account for non-specific effects of the experimental procedure on cortical excitability measures. Sham coils generated a similar PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 3 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Fig 1. Protocol timeline pre- and post- repetitive transcranial magnetic stimulation (rTMS) application. Baseline motor evoked potential (MEP) amplitudes were obtained by applying 150 single pulses (sp) of TMS to the motor cortex (M1) at 120% resting motor threshold (RMT) and measuring peak-to-peak amplitudes of the elicited MEPs. 150 single pulses were again applied to M1 at 5 (T5) minutes post-rTMS and 60 single pulses delivered at 25 (T25) minutes post-rTMS to assess corticomotor reactivity. Resting state EEG (rsEEG) and behavioural outcomes from a motor task were also obtained before and after rTMS. https://doi.org/10.1371/journal.pone.0286465.g001 sound to that of the real coil, and surface-stimulating electrodes were placed above and beside the left eye to simulate somatosensory sensations arising from real TMS. The order of the first five visits was randomized across participants and scheduled at least four days apart to avoid carry-over effects. These visits were then repeated one month later, in the same order for the remaining five visits to evaluate test-retest reliability. Females were tested at approximately the same point in their menstrual cycle and visits were completed at approximately the same time of day for each participant to minimize variability due to circadian fluctuations. Participants were seated in a chair and wore in-ear headphones with integrated ear-protec- tion throughout all stimulation protocols. Auditory white noise was also used to mask the TMS coil click throughout the stimulation protocols, with the volume set to the maximum level the subject could comfortably tolerate. Following International Federation of Clinical Neurophysiology (IFCN) guidelines, resting motor threshold (RMT) was determined on the FDI hotspot as the minimum stimulation intensity eliciting at least five MEPs (>50 mV) out of ten pulses in the relaxed FDI using biphasic (posterior-anterior in the brain) current wave- forms. Active motor threshold (AMT) was then assessed as the minimum stimulus intensity that produced a MEP of at least 200 μV by a cortical silent period in at least five of the 10 trials while subjects were asked to abduct their index finger towards their thumb to slightly contract their FDI muscle. The intensity of spTMS for pre- and post-rTMS corticospinal excitability measurements was set to 120% of RMT. Baseline (pre-rTMS) corticospinal excitability was assessed by applying 150 spTMS trials with randomly jittered (3 to 5 s) inter-stimulus intervals to M1 during the relaxation of the FDI muscle and measuring peak-to-peak amplitudes of the elicited MEPs. Corticospinal excitability was reassessed at 5 (T5) and 25 (T25) minutes follow- ing real or sham rTMS with 150 and 60 spTMS trials, respectively. A sequential finger tapping task [18] was also performed at the beginning of each session and 15 min post-rTMS (Fig 1). Participants were asked to press a five-element sequence (such as 4-1-3-2-4) using their index, middle, ring and little finger. By applying novel sequences in each block, we aimed to mini- mize transfer and learning effects between blocks. The intensity of rTMS protocols was set based on a prior study evaluating the efficacy of rTMS effects at different TMS intensities [19]. 10 Hz rTMS was applied at 120% RMT in 4-sec- ond trains, with a 26-second inter-train interval for 37.5 minutes (3000 pulses total). 1 Hz rTMS was applied at 110% RMT as a continuous train for 15 minutes (900 pulses total). iTBS and cTBS were applied at 80% AMT in three-pulse bursts at 50 Hz with a 200-ms inter-burst interval. For iTBS, this pattern was delivered in two-second trains with an 8-second inter-train interval for 192 seconds (600 pulses total). For the cTBS protocol, the pattern was applied con- tinuously for 40 seconds (600 pulses total). Finally, sham rTMS/TBS was applied with the shielded side of a MagPro A/P Cool-B65 (MagVenture) coil over the at M1 hotspot with a 3-cm thick plastic spacer to further minimize any residual magnetic field reaching the cortex. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 4 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability EMG pre-processing All collected EMG data was processed offline using customized automated scripts running in Matlab R2021a (Math-Works Inc., USA). EMG data were bandpass filtered at 10–2000 Hz and baseline corrected by subtracting the mean value from 50 to 5 ms pre-TMS stimulation from the entire elicited signal. Next, the root mean square (RMS) of the EMG signal from -20 ms pre-TMS pulse to 13 ms post-TMS pulse, omitting -2 to +2 ms to avoid pulse artifact, was cal- culated to identify trials with artifacts, such as concurrent muscle activity. Trials with RMS val- ues greater than 2.5 standard deviations (SD) from the average RMS of the entire block of trials were removed. On average, 2.1 ± 0.9% of the trials across all participants were more than 2.5 SD from the mean. Peak-to-peak MEP amplitudes were then calculated for each remaining trial as the peak-to-peak voltage from 18 ms to 50 ms post-TMS. Statistical analyses The following statistical analyses were performed using a two-tailed 95% confidence interval. Data were tested for normality using the Shapiro-Wilk test and all non-normal data were log- transformed. P-values for individual tests were Bonferroni-corrected to minimize type-I error. For the neuromodulatory effects of rTMS, our goal was to test for a main effect of Time (Baseline, T5 and T25) for individual rTMS protocols in the initial (test) Visit 1 (V1) and, if so, whether we can reproduce these effects in the retest Visit 2 (V2) with an independent set of analyses. Thus, mean MEP amplitudes from each spTMS block for each condition (1 Hz, 10 Hz, iTBS, cTBS, Sham) and visit (V1 and V2) were entered into separate linear mixed-effects models (LMMs) to examine the main effect of Time. To compare MEP changes from each real rTMS protocol to sham-rTMS, we calculated the ratio of mean MEP amplitudes at post-rTMS time points (T5 and T25) to baseline for each proto- col. These ratio values for sham-rTMS at T5 and T25 were subtracted from real rTMS (1 Hz, 10 Hz, iTBS, cTBS) ratio values at T5 and T25 at the subject level. These sham-subtracted ratios were entered into separate mixed-effects linear regression models for each real rTMS condition and visit. A sham-subtracted ratio of greater than zero implies facilitation induced by the real rTMS condition relative to sham, whereas a sham-subtracted ratio of less than zero implies inhibition. To test whether neuromodulatory effects differ across rTMS protocols, post-rTMS ratio val- ues from each protocol (including sham-rTMS) were entered into a LMM with Protocol (1 Hz, 10 Hz, iTBS, cTBS, sham-rTMS) and Visit (V1 and V2) as main factors and “Protocol*Visit” as the interaction factor. Power analyses We performed post-hoc analyses for estimating statistical power to detect significant rTMS effects in our sample (n = 28) using previously reported effect sizes. Chung et al. [20] found that iTBS resulted in significant and moderately large MEP increase at T5 and T25 with a pooled effect size (SMD) of 0.69 and 0.71 (Cohen’s D), respectively. They also reported signifi- cant and large cTBS effects with -0.9 and -0.69 at T5 and T25 post-cTBS measurements, respectively. We used separate linear mixed-effect model (LMMs) to examine the main effect of Time (Baseline vs T5 vs T25) for each rTMS protocol. However, as power calculations for LMMs are complex and difficult to implement [21], we performed repeated-measures ANOVA as a proxy of LMMs. With a sample size of 28 and an alpha of 0.05, we will have 80% power to detect changes with an effect size (Cohen’s f) of 0.29. For post-hoc t-test comparisons, with three comparisons and a Bonferroni-corrected alpha of 0.0167, we will have 80% power to detect changes with an effect size (Cohen’s d) of 0.57 (a moderate effect). We noted that these effects sizes are considerably lower than what has been reported in the meta-analyses of PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 5 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Chung et al. [20], suggesting that our study is sufficiently powered to detect even smaller effects of rTMS on cortico-spinal excitability. Control analyses to examine potential sources of low reliability To ensure that coil deviations from the motor hotspot and cumulative trend effects due to the high number of MEP trials did not confound the neuromodulatory effects of rTMS, we first examined whether the high number of MEP trials at baseline and T5 blocks induces changes in M1 excitability (See Figure S1 in S1 File). We then ran a series of exploratory analyses look- ing at coil handling/position and trial hysteresis as potential sources of variance. Specifically, we reran the mixed-effects linear regression models for the neuromodulatory effects of rTMS (i.e., a model with a main effect of Time) using subsets of the data by selecting 25 MEP trials that are closest to the motor spot based on Euclidian distance measures, and the first 25 MEP trials regardless of the coil position for each spTMS block, respectively. At least 25 trials were included in each block for all mean MEP assessments [17]. The group-averaged number of MEP trials across all rTMS sessions were following; 140±18.07, 127 ±17.55 and 58±1.30 for visit-1; and 141±18.08, 133±14.86 and 59±1.06 for visit-2 within base- line (Pre) and post rTMS blocks (T5 and T25), respectively. In total, 11/840 (1.3%) of the blocks were discarded due to noisy and artifactual trials, 15/840 (1.8%) due to a scaling issue in the data, and 8/840 (1.0%) due to noisy data. Further, for the closest 25 analyses, 62/840 (7.3%) of the data were lost because of triggers not properly sending between brain vision and the EMG recording software. In total, 4% of the data were lost for analyses involving all trials and the first 25 trials, and 11% were lost for the closest 25 trial analyses. Additionally, to examine the impact of the inter-individual difference in baseline excitabil- ity, each visit for each participant was classified as either high or low excitability, depending on whether the average baseline MEP value of a given participant visit was above or below the median baseline MEP value (0.714 mV) across all visits and participants. Baseline excitability groups (High and Low) were then included in a mixed-effects linear regression model as a main factor with Time and Visit to assess for any main or interaction effects in a separate mixed-effect analysis using all MEP trials. Finally, reproducibility of rTMS effects, including the control analyses, were assessed using interclass correlation coefficients (ICCs). ICC values < 0.25 indicate little to no reliability, 0.25 to 0.5 reflect low reliability, 0.5 to 0.75 indicate moderate reliability, and > 0.75 indicate high- reliability [22, 23]. Results Neuromodulatory rTMS effects Group averaged and log transformed MEP amplitudes for each participant at each block (Base- line, T5, T25) across rTMS conditions and visits are shown in Fig 2A and 2B. For each rTMS protocol and visit, we tested for a main effect of Time (Baseline, T5 and T25). Across all trials, no statistically significant main effect of Time was found for any rTMS protocol (p � 0.0558), except for 10 Hz in Visit 2 (F(2,50) = 3.959, p = 0.0253, η2 = 0.137) (Fig 2A). However, this result was no longer significant after correction for multiple comparisons. Sham-corrected rTMS effects No main effect of Time was statistically significant for any of the rTMS protocol visits (p = 0.391, Fig 3A and 3B). These results indicate low rTMS modulatory effectiveness when accounting for sham. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 6 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Fig 2. Repetitive transcranial magnetic stimulation (rTMS) protocol effectiveness and repeatability for all motor evoked potential (MEP) trials in each block. (A) Group-averaged MEP at baseline, T5 and T25 during Visit 1 (left) and Visit 2 (right) for intermittent theta burst stimulation (iTBS), continuous theta burst stimulation (cTBS), 1 Hz, 10 Hz, and Sham protocols. Data are mean ± standard error. (B) Individual participant data for Visit 1 (top, V1) and Visit 2 (bottom, V2) for each protocol. White dots show group means. https://doi.org/10.1371/journal.pone.0286465.g002 Neuromodulatory effect differences among rTMS protocols across visits The mixed models found that MEP ratios (Post-iTBS/Baseline) at T5 showed a main effect of Visit (F(1,28) = 4.479, p = 0.0435, η2 = 0.138), however, this effect was no longer significant after correction for multiple comparisons. No other main effect or interactions were statisti- cally significant for T5 or T25 ratios (p � 0.291). rTMS effects when accounting for baseline excitability, the distance of TMS to the hotspot, and the number of trials Baseline excitability, the distance of the TMS coil to the hotspot, and the number of trials were all included in the mixed model assessing a main effect of time to determine whether these fac- tors would account for any MEP amplitude changes in response to the various rTMS proto- cols. rTMS effects were not improved when accounting for only the first 25 trials as reflected by the lack of statistical significance for the main effect of Time for these data (p = 0.0643) (Fig 4) except for 1 Hz (F(2,54) = 3.451, p = 0.0389, η2 = 0.113) and iTBS in visit 1 (F(2,46) = 3.425, p = 0.0411, η2 = 0.130). However, when correcting for multiple comparisons (p > 0.005), these results were no longer significant, and consistent effects were not present in Visit 2. When accounting for the closest 25 trials to the motor hot spot only, a significant main effect of Time PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 7 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Fig 3. Sham-corrected effects of intermittent theta burst stimulation (iTBS), continuous theta burst stimulation (cTBS), 1 Hz and 10 Hz. (A) Group-averaged motor evoked potential (MEP) ratio values for visit 1 and visit 2 with sham subtracted out separately for each participant. Data are mean ± standard error. (B) Individual sham-corrected data for visit 1 (V1) and visit 2 (V2) for each repetitive transcranial magnetic stimulation (rTMS) protocol. Data are presented as MEP ratio for each rTMS protocol with sham MEP percent change subtracted from these effects separately for each participant. https://doi.org/10.1371/journal.pone.0286465.g003 was found for 10 Hz in visit 2 (F(2,46) = 5.789, p = 0.0057, η2 = 0.201), however, again this finding did not survive multiple comparisons (Fig 5). No other main effect of Time was found for the closest 25 trials (p � 0.0843). Fig 4. Repetitive transcranial magnetic stimulation (rTMS) protocol effectiveness and reproducibility for the first 25 motor evoked potential (MEP) trials in each block. Group-averaged MEP at baseline (BL), T5 and T25 during Visit 1 (V1; left) and Visit 2 (V2; right) for intermittent theta burst stimulation (iTBS), continuous theta burst stimulation (cTBS), 1 Hz, 10 Hz, and Sham protocols. Data are mean ± standard error. https://doi.org/10.1371/journal.pone.0286465.g004 PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 8 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Fig 5. Repetitive transcranial magnetic stimulation (rTMS) protocol effectiveness and repeatability for the 25 trials corresponding to the single pulse stimulation applied closest to the hotspot. Group-averaged motor evoked potential (MEP) at baseline, T5 and T25 during visit 1 (left) and visit 2 (right) for intermittent theta burst stimulation (iTBS), continuous theta burst stimulation (cTBS), 1 Hz, 10 Hz, and Sham protocols. Data are mean ± standard error. https://doi.org/10.1371/journal.pone.0286465.g005 For the baseline excitability assessment, a significant main effect of Baseline Excitability was found for all protocols (p < 0.0001), confirming that the high and low baseline excitability groups are significantly different from each other (Fig 6). A significant interaction between Time and Baseline excitability was found for both the 1 Hz (F(2,85) = 3.255, p = 0.0434, η2 = 0.071) and cTBS (F(2,102) = 5.515, p = 0.0053, η2 = 0.098), however, only the cTBS protocol effect remained significant after correction for multiple comparisons. Post-hoc analyses showed that the low baseline MEP group had a significant increase in MEP amplitude after cTBS at T5 compared to baseline (t = -3.582, p = 0.0006). Notably, this effect was only signifi- cant for Visit 1 and was not reproduced independently in Visit 2. No significant main effects or interactions were found for any of the other protocols (p = 0.142). Reproducibility of neuromodulatory rTMS effects The reliability of treatment effects was measured using intraclass correlation coefficients. Scat- ter plots showing a correlation between visit 1 and visit 2 for the ratio data at T5 and T25 are presented in Fig 7A for all trials and 7B for sham-corrected trials. ICC results for MEP ratios at T5 and T25 for all protocols are shown in Table 1. Each protocol time point showed poor Fig 6. Scatter plots between baseline motor evoked potential (MEP) amplitude and post repetitive transcranial magnetic stimulation (rTMS) MEPs as a percent of baseline averaged across visits for T5 (top) and T25 (bottom) for each protocol; dots represent individual participants. https://doi.org/10.1371/journal.pone.0286465.g006 PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 9 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Fig 7. Scatter plots between T5 (blue) and T25 (red) ratio data during visit 1 (V1) and visit 2 (V2) for each protocol with a single dot representing each participant for (A) all trials and (B) sham-corrected data. https://doi.org/10.1371/journal.pone.0286465.g007 reliability across visits when using all trials to calculate T5 and T25 ratio values. Poor reliability was also found for the T5 and T25 ratio values based on the first 25 trials only. After correcting for multiple comparisons, only the sham T5 closest 25 trials showed moderate reliability across visits (c = 0.624, p = 0.002). Baseline excitability was also moderately reliable across all 10 visits (c = 0.5774, p < 0.001). Descriptive classification of corticospinal responses to rTMS protocols We classified rTMS response types using the percentage change in MEPs from baseline, as pre- viously described [24]. In particular, facilitation response is defined as an increase of at least 10% from baseline (μ�110%), suppression response is defined as a decrease of at least 10% from baseline (μ�90%) and others within these ranges (110%<μ>90%) is defined as no change/responder. At the group level, we did not see expected dominant pattern in any of the PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 10 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Table 1. Intra-participant variability results showing c and p-values for the intraclass correlation coefficient (ICC) between visit 1 and visit 2 using ratio data at T5 and T25 for all trials, first 25 trials, closest 25 trials and sham-corrected data. P-values less than 0.05 are highlighted in light gray, and p-values less than 0.005 (surviv- ing multiple comparisons) are highlighted in dark gray. All trials First 25 Closest 25 Sham-corrected r-value p-value r-value p-value r-value p-value r-value p-value iTBS T5 0.323 0.051 0.298 0.067 0.595 0.004 -0.255 0.892 iTBS T25 0.247 0.118 0.460 0.011 0.397 0.043 0.056 0.403 cTBS T5 -0.224 0.875 0.092 0.319 -0.219 0.838 -0.127 0.737 cTBS T25 0.266 0.087 0.152 0.221 0.087 0.350 0.378 0.032 1 Hz T5 0.169 0.192 -0.115 0.721 -0.005 0.508 0.216 0.137 1 Hz T25 0.192 0.165 0.079 0.345 0.183 0.203 0.151 0.238 10 Hz T5 -0.016 0.531 0.248 0.099 0.001 0.498 0.198 0.158 10 Hz T25 0.149 0.231 -0.080 0.651 0.407 0.031 -0.109 0.690 Sham T5 0.166 0.196 0.040 0.418 0.417 0.035 - - Sham T25 0.052 0.397 0.129 0.262 -0.190 0.761 - - https://doi.org/10.1371/journal.pone.0286465.t001 rTMS protocols including sham. For example, for iTBS we observed that roughly half of the participants (48% in Visit-1 and 52% in Visit-2) showed the expected facilitatory responses at T5 for both visits, but these responder ratios were very similar to the ratio of facilitatory responses following sham stimulation (47% in Visit-1 and 45% in Visit-2). Moreover, individ- ual response patterns were also not consistent across visits such that a given person classified as responder to iTBS with increased MEPs in Visit-1 may show opposite or no-response to the same stimulation in Visit-2 (Please see Fig 8A). Similar, high variability in intra-individual responses was observed for all rTMS blocks. Discussion In this study, we assessed whether 1 Hz and 10 Hz rTMS protocols (1) significantly decrease or increase corticospinal excitability, respectively, against sham-rTMS, (2) differ from widely used TBS protocols (iTBS and cTBS) on modulating corticospinal excitability, and (3) whether effects of both traditional rTMS and TBS protocols on corticospinal excitability are reproduc- ible across identical sessions. We found that neither 1 Hz nor 10 Hz rTMS had any significant effect on modulating corticospinal excitability at the group level, as indexed by the amplitude of MEP responses. MEP responses to these protocols were also not significantly different from both sham-rTMS and two widely used TBS protocols. Reproducibility of the neuromodulatory effects was poor for both real rTMS and TBS protocols except for iTBS at T5 for a subset of MEP trials (n = 25) when coil position was closest to the stimulation target. Importantly, none of the real rTMS protocols demonstrated consistent corticospinal neuromodulation or repro- ducibility after controlling for baseline corticospinal excitability, TMS coil deviation and num- ber of individual MEP trials within a given block. These results, along with the recent reports showing substantial variability and poor replicability in TBS applications [9, 10, 12, 14, 16, 25] call for a fundamental rethinking of the potential mechanisms by which rTMS affects brain function and behavior in humans. Recently, we have witnessed a reproducibility/replication crisis in almost every field of brain research including psychology [26], neuroimaging [27–30] and genomics [31], and research in non-invasive modulation of brain function in humans is no exception. Pascual- Leone and colleagues [3], performed the first experimental rTMS study on the human motor cortex and showed increased MEP amplitudes following 10 Hz rTMS of M1, and Wassermann and colleagues [32] reported diminished MEP amplitudes for the first time following rTMS of PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 11 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Fig 8. Individual response characteristics to rTMS. (A) Classification of individual responses and responder types across rTMS protocols for each visit. Pie charts show the percentage of facilitation (Red: μ�110%), suppression (Blue: μ� 90%) and no change (Gray: 110%<μ>90%) for each rTMS protocol in Visit-1 (upper pie charts) and Visit-2 (Lower pie charts). (B) Bar graphs showing group averaged MEP percentage changes across visits (Visit 1 and Visit) and blocks (T5 and T25). The colored dots (blue for Visit 1 and red for Visit 2) denotes the individual response and the black lines connected to the dots represent and track individual response changes across visits (MEP %Δ to Baseline; motor evoked potential percentage change to baseline). https://doi.org/10.1371/journal.pone.0286465.g008 M1 at 1 Hz. Similarly, the first study [6] assessing the neuromodulatory effects of TBS proto- cols reported increased and decreased MEP amplitudes following iTBS and cTBS of the human motor cortex, respectively. Following these initial results, rTMS and TBS protocols have been broadly employed as non-invasive therapeutic tools to induce persistent plasticity in targeted brain networks for the treatment of a wide range of neurological [5, 20, 33] and psy- chiatric [34–36] disorders. These protocols are also popular neuromodulation techniques to promote transient changes in neural excitability and thereby modulate human behavior and cognition in single-session experimental designs [5, 37]. Randomized clinical trials employing multi-session rTMS interventions have provided accumulating evidence for the therapeutic benefits of these rTMS and TBS protocols, resulting in Food and Drug Administration (FDA) approved therapies in major depressive disorder (MDD) [38–40], obsessive-compulsive disor- der (OCD) [41, 42] and migraine [43, 44]. Yet, an increasing number of experimental research applying single-session rTMS [24, 45] and TBS protocols [10, 25] in humans have failed to show effective and reliable modulation of neuronal excitability, which is presumed to be the primary neurophysiological mechanism underlying the clinical effects of rTMS. Previous studies assessing neuromodulatory effects of TBS in humans reported high inter- and intra-individual variability in corticospinal and cortical excitability following TBS [9], and with poor reproducibility of neuromodulatory effects in repeat sessions [12, 22]. Similarly, an early-dated review of 1 Hz and 10 Hz protocols on corticospinal excitability [5] also showed PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 12 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability that only less than half of the studies reported expected changes with increased excitability fol- lowing 10 Hz and decreased excitability following 1 Hz. Unfortunately, however, none of the studies reporting expected rTMS changes [4, 45–53] have included a robust sham protocol to control for placebo or non-specific effects of rTMS, or performed repeat sessions to test the consistency of reported rTMS effects. It is important to note that we could also reach similar conclusions in the current study if we would remove sham comparisons and repeat tests from the statistical analyses. For example, iTBS increased MEPs from baseline to T25 in Visit-1 but this effect was not significant compared to the sham and was not reproduced in Visit-2 (Fig 2). Overall, we have demonstrated that it is not possible to draw valid conclusions regarding the effects of TBS protocols on corticospinal and cortical excitability without having sham-con- trols and performing repeat tests. At this point, it is critical to question why a growing number of experimental reports have failed to identify the cortical excitability modulations that are presumed to underlie rTMS effects, while evidence for the clinical efficacy of repeated sessions of rTMS is only growing. The discrepancy between clinical and experimental rTMS studies could be explained by a number of hypothetical factors including, but not limited to, measurement tools and metrics for examining rTMS effects on brain function, methodological differences between clinical and experimental studies, and alternative mechanisms of action for clinical rTMS effects that are yet to be identified. To date, peak-to-peak MEP amplitude has been the most commonly used outcome measure for determining rTMS dose and for assessing the neuromodulatory effects of rTMS applications [54]. Despite such widespread use, MEP amplitudes are known to have substantial variability between successive trials only a few seconds apart, even with con- stant stimulation parameters and experimental conditions [10, 55–58]. A large body of prior research [17] has demonstrated that variability in MEP amplitudes may be attributed to a col- lection of experimental and methodological factors including stimulation parameters, devices, stimulated muscles, number of consecutive pulses, millimetric deviations from the hotspot, and the momentary fluctuations in brain state at the time of stimulation. In this regard, we performed a series of control analyses by re-grouping subjects based on their pre-rTMS MEP amplitudes, selecting a subset of MEPs when the coil was closest to the motor hotspot, as well as analyzing only the first 25 MEP trials, to account for potential experimental confounds such as baseline excitability of the stimulated muscle, variability in coil positioning, and cumulative intra-block effects of single-pulse TMS, respectively. Despite these efforts, however, we did not find the expected TMS-induced modulation of corticospinal excitability, nor improved repro- ducibility of neuromodulation across sessions, suggesting that these factors are unlikely to have significant contributions to high inter- and intra-individual response variability to rTMS reported in this study. Another possibility is that the MEP amplitude itself, as an outcome measure, is not specific enough to the neurophysiological alterations that are specifically occurring within cortical circuits. As MEP responses to rTMS reflect a combination of changes along the entire motor pathway (i.e., cortical, spinal and peripheral excitability), it is possible that endogenous fluctuations in the excitability of spinal motoneurons confound rTMS effects or contribute to their variability. In fact, pharmacological studies [59, 60] in humans showed that blocking N-methyl-D-aspartate receptors (NMDARs), a primary synaptic signaling mech- anism underlying long-term potentiation (LTP) or depression (LTD) like plasticity [61], sup- pressed excitatory and inhibitory aftereffects of iTBS and cTBS, respectively. Importantly, however, NMDAR blockage did not alter resting and active motor thresholds when compared to placebo agents, showing that NMDAR blockage mainly eliminated rTMS-induced neuro- modulation but did not affect baseline cortical excitability [59, 60]. These results support the idea that subthreshold rTMS mainly induces NMDA-dependent synaptic changes within intra-cortical circuits [62, 63] and suggest that magnitude of these cortical changes may not be PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 13 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability adequately and consistently captured by the MEP amplitudes due to their high inter- and intra-subject variability and susceptibility to confounding fluctuations from subcortical and spinal sources. Alternatively, the lack of change in corticospinal excitability following NMDAR blockage may also indicate that MEP responses to sTMS reflect neural activation mechanisms that are not directly involved in rTMS-induced neuroplasticity mediated by acti- vation of NMDAR. For example, previous research has shown that suprathreshold sTMS of the human motor cortex generates high-frequency descending volleys (~700 Hz) in corticosp- inal tracts with ISIs around 1.5ms [62, 64–66]. NMDA receptors, however, are known to have slow-evolving kinetics with discharge rates longer than 2ms [66, 67], thus they may act as a low-pass filter and are unlikely to involve in producing high-frequency descending volleys fol- lowing suprathreshold sTMS due to their slower depolarization rates. If sub-threshold rTMS mainly exerts its’ neuromodulatory effects on slower NMDAR and if high-frequency descend- ing volleys in pyramidal tracts following sTMS are preferentially mediated by faster receptors (~700 Hz), it is then conceivable to assume that the MEP response to sTMS may not even be the target outcome measure to detect NMDA-dependent LTP and LTD induced by rTMS in intra-cortical circuits. Future studies should perhaps focus on outcome measures that reflect spatial-temporal and oscillatory characteristics of neuronal function directly at the cortical level. Combining TMS with scalp electroencephalography (TMS-EEG) enables assessment of TMS-evoked potentials (TEPs) which, compared to MEPs, are typically considered as direct measures of cortical excit- ability [68–70]. However, a recent study by our group showed no reliable changes in TEPs fol- lowing theta burst stimulation [25], supporting the current findings that rTMS is likely not modulating local cortical excitability in the way we currently assume it does. Thus, more work is needed in this field to help identify the mechanisms by which rTMS is achieving its effects. For example, spectral dynamics of spontaneous neural oscillations captured during resting brain state (Rest-EEG) may be sensitive to changes in NMDA-dependent plasticity in humans [71], and could be considered another alternative outcome measure to assess rTMS effects. Another important potential factor for the current discrepancy between experimental investigations showing a lack of consistent changes in cortical excitability and the validated clinical rTMS effects is that in clinical applications, rTMS is typically applied in multiple ses- sions over several weeks. In contrast, the majority of experimental studies assessing the effects of rTMS on cortical excitability employ single session pre-post designs and measure neuromo- dulatory effects before and after rTMS with multiple follow-up blocks up to 60–90 mins [5]. The different patterns of application may result in different mechanisms of action. Research in animal models, for example, has shown that a single bout of high-frequency rTMS changes neuronal activity mainly through modulation of membrane potentials [72], with increased steady state neuronal currents, while daily sessions of rTMS protocols at the same frequency resulted in substantially increased BDNF levels [73]. It is possible that short-lived activity- dependent modulation of synaptic plasticity is highly variable both between and within indi- viduals following single bouts of rTMS, but that multiple sessions of rTMS induce persistent expression of neurotrophic factors [74] that result in more consistent network-level changes and therapeutic effects. Finally, it is possible that the cortical excitability hypothesis itself may be insufficient to explain the observed long-term changes in human brain function following multiple rTMS interventions. Emerging evidence in animal models suggested that, apart from cortical excitability, rTMS induces changes in gene expression, modulation of neurotransmit- ter release, reduced oxidative stress, inflammation, and activation of neurotrophic factors, which together may contribute to the lasting clinical effects of rTMS [63]. However, it is important to note that none of the molecular mechanisms of rTMS has been rigorously tested and directly validated in humans [63]. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 14 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability One limitation of the current study is the use of many different types of assessments (i.e. EEG, motor task, MEP, etc.) to assess rTMS effectiveness and reliability. Our main goal was to measure multiple electrophysiological and behavioral outcomes to comprehensively examine the neuromodulatory effects of rTMS at different levels. However, this makes it possible that one of the variables influenced another, independent of rTMS protocol effects. For example, it is possible that baseline corticospinal excitability reported in the current study was influenced by the motor task. Importantly, these rTMS protocols are typically used in clinical settings to improve outcomes in individuals with depression, schizophrenia, etc. and, therefore, assessing outcomes in these specific populations would be beneficial. Lastly, although we controlled for caffeine intake, sleep and menstrual cycle by instructing participants to follow their regular caf- feine intake/sleep patterns, we did not control for other factors such as physical activity levels, which could have affected inter-participant variability. However, factors affecting intra-partici- pant variability were largely controlled for including testing at the same time of day and the use of neuronavigation to ensure similar TMS coil positioning. Conclusions and future directions Repetitive TMS has become a very popular non-invasive therapeutic tool to attempt to treat a wide variety of neuropsychiatric disorders. Apart from FDA-approved therapies in MDD, OCD and migraine, these rTMS protocols have been investigated as a therapeutic tool for stroke recovery [75], addiction [76, 77], schizophrenia [78], autism [79], post-traumatic stress disorders [80], epilepsy [81], tinnitus [82], chronic pain [83], and memory deficits [33], among many others. Despite this widespread use, however, the mechanisms via which rTMS changes human brain function and achieves therapeutic benefits are not well understood. The predom- inant hypothesis for the mechanism of rTMS effects has been via modulation of cortical excit- ability. However, our results (and those of others) suggest that this hypothesis may not be valid, at least the way it is typically measured in humans; rTMS does not produce reliable changes in corticospinal excitability either within or across individuals. As long as the complex biological and neurophysiological mechanisms underlying rTMS effects on human brain func- tion and behavior remain unexplained, individual and experimental factors contributing to substantial inter- and intra-subject response variability to rTMS cannot be adequately con- trolled for, and thus will stand as a major obstacle for efforts to personalize rTMS to improve its consistency and clinical effectiveness. Future studies are immediately needed that use alter- native measurement tools and metrics to directly measure brain responses to rTMS, employ longitudinal research designs to monitor changes in brain function and structure and their relationship to behavioral changes over repeated sessions, and study mechanistic hypotheses to understand the role of individual and experimental parameters on rTMS effects. Supporting information S1 File. This file contains all the supporting text and figure. (DOCX) Author Contributions Conceptualization: Mouhsin M. Shafi, Recep A. Ozdemir. Data curation: Justine Magnuson, Mehmet A. Ozdemir, Brice Passera, Sumientra Rampersad. Formal analysis: Justine Magnuson, Mehmet A. Ozdemir, Alyssa B. Dufour, Peter J. Fried, Recep A. Ozdemir. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 15 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Funding acquisition: Mouhsin M. Shafi. Investigation: Elon Mathieson, Sofia Kirkman, Mouhsin M. Shafi, Recep A. Ozdemir. Methodology: Brice Passera, Dana Brooks, Alvaro Pascual-Leone, Mouhsin M. Shafi, Recep A. Ozdemir. Project administration: Elon Mathieson, Sofia Kirkman, Brice Passera, Mouhsin M. Shafi. Software: Recep A. Ozdemir. Supervision: Dana Brooks, Alvaro Pascual-Leone, Peter J. Fried, Mouhsin M. Shafi, Recep A. Ozdemir. Validation: Mehmet A. Ozdemir. Visualization: Justine Magnuson, Mehmet A. Ozdemir. Writing – original draft: Justine Magnuson. Writing – review & editing: Mehmet A. Ozdemir, Elon Mathieson, Sofia Kirkman, Sumientra Rampersad, Alyssa B. Dufour, Dana Brooks, Alvaro Pascual-Leone, Peter J. Fried, Mouhsin M. Shafi, Recep A. Ozdemir. References 1. Wassermann EM, Lisanby SH. Therapeutic application of repetitive transcranial magnetic stimulation: A review. Clinical Neurophysiology. 2001. https://doi.org/10.1016/s1388-2457(01)00585-5 PMID: 11459676 2. Wassermann EM. Risk and safety of repetitive transcranial magnetic stimulation: Report and suggested guidelines from the International Workshop on the Safety of Repetitive Transcranial Magnetic Stimula- tion, June 5–7, 1996. Electroencephalogr Clin Neurophysiol—Evoked Potentials. 1998; 108. https://doi. org/10.1016/S0168-5597(97)00096-8 PMID: 9474057 3. Pascual-leone A, Valls-solé J, Wassermann EM, Hallett M. Responses to rapid-rate transcranial mag- netic stimulation of the human motor cortex. Brain. 1994;117. https://doi.org/10.1093/brain/117.4.847 PMID: 7922470 4. Chen R, Classen J, Gerloff C, Celnik P, Wassermann EM, Hallett M, et al. Depression of motor cortex excitability by low-frequency transcranial magnetic stimulation. Neurology. 1997; 48: 1398–1403. https://doi.org/10.1212/wnl.48.5.1398 PMID: 9153480 5. Fitzgerald PB, Fountain S, Daskalakis ZJ. A comprehensive review of the effects of rTMS on motor cor- tical excitability and inhibition. Clinical Neurophysiology. 2006. pp. 2584–2596. https://doi.org/10.1016/j. clinph.2006.06.712 PMID: 16890483 6. Huang YZ, Edwards MJ, Rounis E, Bhatia KP, Rothwell JC. Theta burst stimulation of the human motor cortex. Neuron. 2005; 45. https://doi.org/10.1016/j.neuron.2004.12.033 PMID: 15664172 7. Maeda F, Keenan JP, Tormos JM, Topka H, Pascual-Leone A. Modulation of corticospinal excitability by repetitive transcranial magnetic stimulation. Clin Neurophysiol. 2000; 111: 800–805. https://doi.org/ 10.1016/s1388-2457(99)00323-5 PMID: 10802449 8. Maeda F, Keenan JP, Tormos JM, Topka H, Pascual-Leone A. Interindividual variability of the modula- tory effects of repetitive transcranial magnetic stimulation on cortical excitability. Exp Brain Res. 2000; 133: 425–430. https://doi.org/10.1007/s002210000432 PMID: 10985677 9. Hamada M, Murase N, Hasan A, Balaratnam M, Rothwell JC. The role of interneuron networks in driving human motor cortical plasticity. Cereb Cortex. 2013;23. https://doi.org/10.1093/cercor/bhs147 PMID: 22661405 10. Hordacre B, Goldsworthy MR, Vallence AM, Darvishi S, Moezzi B, Hamada M, et al. Variability in neural excitability and plasticity induction in the human cortex: A brain stimulation study. Brain Stimulat. 2017; 10: 588–595. https://doi.org/10.1016/j.brs.2016.12.001 PMID: 28024963 11. López-Alonso V, Cheeran B, Rı́o-Rodrı́guez D, Fernández-Del-Olmo M. Inter-individual variability in response to non-invasive brain stimulation paradigms. Brain Stimulat. 2014; 7. https://doi.org/10.1016/j. brs.2014.02.004 PMID: 24630849 PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 16 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability 12. Boucher PO, Ozdemir RA, Momi D, Burke MJ, Jannati A, Fried PJ, et al. Sham-derived effects and the minimal reliability of theta burst stimulation. Sci Rep. 2021; 11. https://doi.org/10.1038/s41598-021- 98751-w PMID: 34707206 13. Vallence AM, Goldsworthy MR, Hodyl NA, Semmler JG, Pitcher JB, Ridding MC. Inter- and intra-subject variability of motor cortex plasticity following continuous theta-burst stimulation. Neuroscience. 2015; 304. https://doi.org/10.1016/j.neuroscience.2015.07.043 PMID: 26208843 14. Schilberg L, Schuhmann T, Sack AT. Interindividual variability and intraindividual reliability of intermit- tent theta burst stimulation-induced neuroplasticity mechanisms in the healthy brain. J Cogn Neurosci. 2017; 29. https://doi.org/10.1162/jocn_a_01100 PMID: 28129054 15. Fried PJ, Jannati A, Davila-Pérez P, Pascual-Leone A. Reproducibility of single-pulse, paired-pulse, and intermittent theta-burst TMS measures in healthy aging, Type-2 diabetes, and Alzheimer’s disease. Front Aging Neurosci. 2017; 9. https://doi.org/10.3389/fnagi.2017.00263 PMID: 28871222 16. Perellón-Alfonso R, Kralik M, Pileckyte I, Princic M, Bon J, Matzhold C, et al. Similar effect of intermittent theta burst and sham stimulation on corticospinal excitability: A 5-day repeated sessions study. Eur J Neurosci. 2018; 48: 1990–2000. https://doi.org/10.1111/ejn.14077 PMID: 30022548 17. Corp DT, Bereznicki HGK, Clark GM, Youssef GJ, Fried PJ, Jannati A, et al. Large-scale analysis of interindividual variability in single and paired-pulse TMS data. Clin Neurophysiol. 2021; 132: 2639– 2653. https://doi.org/10.1016/j.clinph.2021.06.014 PMID: 34344609 18. Walker MP, Brakefield T, Morgan A, Hobson JA, Stickgold R. Practice with Sleep Makes Perfect: Sleep-Dependent Motor Skill Learning. Neuron. 2002; 35: 205–211. https://doi.org/10.1016/s0896- 6273(02)00746-8 PMID: 12123620 19. Turi Z, Lenz M, Paulus W, Mittner M, Vlachos A. Selecting stimulation intensity in repetitive transcranial magnetic stimulation studies: A systematic review between 1991 and 2020. European Journal of Neuro- science. 2021. https://doi.org/10.1111/ejn.15195 PMID: 33754397 20. Chung SW, Hill AT, Rogasch NC, Hoy KE, Fitzgerald PB. Use of theta-burst stimulation in changing excitability of motor cortex: A systematic review and meta-analysis. Neuroscience and Biobehavioral Reviews. 2016. https://doi.org/10.1016/j.neubiorev.2016.01.008 PMID: 26850210 21. Johnson PC, Barry SJ, Ferguson HM, Müller P. Power analysis for generalized linear mixed models in ecology and evolution. Methods Ecol Evol. 2015; 6: 133–142. https://doi.org/10.1111/2041-210X.12306 PMID: 25893088 22. McGraw KO, Wong SP. Forming Inferences about Some Intraclass Correlation Coefficients. Psychol Methods. 1996; 1. https://doi.org/10.1037/1082-989X.1.1.30 23. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016; 15. https://doi.org/10.1016/j.jcm.2016.02.012 PMID: 27330520 24. Nettekoven C, Volz LJ, Leimbach M, Pool E-M, Rehme AK, Eickhoff SB, et al. Inter-individual variability in cortical excitability and motor network connectivity following multiple blocks of rTMS. Neuroimage. 2015; 118: 209–218. https://doi.org/10.1016/j.neuroimage.2015.06.004 PMID: 26052083 25. Ozdemir RA, Boucher P, Fried PJ, Momi D, Jannati A, Pascual-Leone A, et al. Reproducibility of cortical response modulation induced by intermittent and continuous theta-burst stimulation of the human motor cortex. Brain Stimulat. 2021; 14: 949–964. https://doi.org/10.1016/j.brs.2021.05.013 PMID: 34126233 26. Nosek BA, Cohoon J, Kidwell MC, Spies JR. Estimating the Reproducibility of Psychological Science. OSF Preprints; 2016. https://doi.org/10.31219/osf.io/447b3 27. Marek S, Tervo-Clemmens B, Calabro FJ, Montez DF, Kay BP, Hatoum AS, et al. Reproducible brain- wide association studies require thousands of individuals. Nature. 2022; 603: 654–660. https://doi.org/ 10.1038/s41586-022-04492-9 PMID: 35296861 28. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013; 14: 365–376. https:// doi.org/10.1038/nrn3475 PMID: 23571845 29. Eklund A, Nichols TE, Knutsson H. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proc Natl Acad Sci. 2016; 113: 7900–7905. https://doi.org/10.1073/pnas. 1602413113 PMID: 27357684 30. Gratton C, Nelson SM, Gordon EM. Brain-behavior correlations: Two paths toward reliability. Neuron. 2022; 110: 1446–1449. https://doi.org/10.1016/j.neuron.2022.04.018 PMID: 35512638 31. Bridgeford EW, Wang S, Wang Z, Xu T, Craddock C, Dey J, et al. Eliminating accidental deviations to minimize generalization error and maximize replicability: Applications in connectomics and genomics. PLOS Comput Biol. 2021; 17: e1009279. https://doi.org/10.1371/journal.pcbi.1009279 PMID: 34529652 32. Wassermann EM, Wedegaertner FR, Ziemann U, George MS, Chen R. Crossed reduction of human motor cortex excitability by 1-Hz transcranial magnetic stimulation1These data were previously PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 17 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability published in abstract form in: Wassermann, E.M., Wedegaertner, F.R., George, M.S. and Chen, R., Electroenceph. clin. Neurophysiol., 103 (1997) 151.1. Neurosci Lett. 1998; 250: 141–144. https://doi. org/10.1016/S0304-3940(98)00437-6 PMID: 9708852 33. Chou Y, Ton That V, Sundman M. A systematic review and meta-analysis of rTMS effects on cognitive enhancement in mild cognitive impairment and Alzheimer’s disease. Neurobiol Aging. 2020; 86: 1–10. https://doi.org/10.1016/j.neurobiolaging.2019.08.020 PMID: 31783330 34. Fitzgerald P. Reduced plastic brain responses in schizophrenia: a transcranial magnetic stimulation study*1. Schizophr Res. 2004; 71: 17–26. https://doi.org/10.1016/j.schres.2004.01.018 PMID: 15374568 35. George MS, Lisanby SH, Sackeim HA. Transcranial magnetic stimulation: applications in neuropsychia- try. Arch Gen Psychiatry. 1999; 56: 300–311. https://doi.org/10.1001/archpsyc.56.4.300 PMID: 10197824 36. George MS, Taylor JJ, Short EB. The expanding evidence base for rTMS treatment of depression. Curr Opin Psychiatry. 2013; 26: 13. https://doi.org/10.1097/YCO.0b013e32835ab46d PMID: 23154644 37. Lefaucheur J-P, Aleman A, Baeken C, Benninger DH, Brunelin J, Di Lazzaro V, et al. Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS): An update (2014–2018). Clin Neurophysiol. 2020; 131: 474–528. https://doi.org/10.1016/j.clinph.2019.11.002 PMID: 31901449 38. George MS, Nahas Z, Molloy M, Speer AM, Oliver NC, Li X-B, et al. A controlled trial of daily left prefron- tal cortex TMS for treating depression. Biol Psychiatry. 2000; 48: 962–970. https://doi.org/10.1016/ s0006-3223(00)01048-9 PMID: 11082469 39. George MS, Wassermann EM, Williams WA, Callahan A, Ketter TA, Basser P, et al. Daily repetitive transcranial magnetic stimulation (rTMS) improves mood in depression. Neuroreport Int J Rapid Com- mun Res Neurosci. 1995. https://doi.org/10.1097/00001756-199510020-00008 PMID: 8547583 40. Pascual-Leone A, Rubio B, Pallardó F, Catalá MD. Rapid-rate transcranial magnetic stimulation of left dorsolateral prefrontal cortex in drug-resistant depression. The Lancet. 1996; 348: 233–237. https://doi. org/10.1016/s0140-6736(96)01219-6 PMID: 8684201 41. Trevizol AP, Shiozawa P, Cook IA, Sato IA, Kaku CB, Guimarães FB, et al. Transcranial magnetic stim- ulation for obsessive-compulsive disorder: an updated systematic review and meta-analysis. J ECT. 2016; 32: 262–266. https://doi.org/10.1097/YCT.0000000000000335 PMID: 27327557 42. Berlim MT, Neufeld NH, Van den Eynde F. Repetitive transcranial magnetic stimulation (rTMS) for obsessive–compulsive disorder (OCD): an exploratory meta-analysis of randomized and sham-con- trolled trials. J Psychiatr Res. 2013; 47: 999–1006. https://doi.org/10.1016/j.jpsychires.2013.03.022 PMID: 23615189 43. Dodick DW, Schembri CT, Helmuth M, Aurora SK. Transcranial magnetic stimulation for migraine: a safety review. Headache J Head Face Pain. 2010; 50: 1153–1163. 44. Conforto AB, Amaro E Jr, Gonçalves AL, Mercante JP, Guendler VZ, Ferreira JR, et al. Randomized, proof-of-principle clinical trial of active transcranial magnetic stimulation in chronic migraine. Cephalal- gia. 2014; 34: 464–472. https://doi.org/10.1177/0333102413515340 PMID: 24326236 45. Sommer M, Wu T, Tergau F, Paulus W. Intra-and interindividual variability of motor responses to repeti- tive transcranial magnetic stimulation. Clin Neurophysiol. 2002; 113: 265–269. https://doi.org/10.1016/ s1388-2457(01)00726-x PMID: 11856631 46. Fitzgerald P. Intensity-dependent effects of 1 Hz rTMS on human corticospinal excitability. Clin Neuro- physiol. 2002; 113: 1136–1141. https://doi.org/10.1016/s1388-2457(02)00145-1 PMID: 12088710 47. Stinear CM, Byblow WD. Impaired modulation of corticospinal excitability following subthreshold rTMS in focal hand dystonia. Hum Mov Sci. 2004; 23: 527–538. https://doi.org/10.1016/j.humov.2004.08.022 PMID: 15541533 48. Romero JR, Anschel D, Sparing R, Gangitano M, Pascual-Leone A. Subthreshold low frequency repeti- tive transcranial magnetic stimulation selectively decreases facilitation in the motor cortex. Clin Neuro- physiol. 2002; 113: 101–107. https://doi.org/10.1016/s1388-2457(01)00693-9 PMID: 11801430 49. Muellbacher W, Ziemann U, Boroojerdi B, Hallett M. Effects of low-frequency transcranial magnetic stimulation on motor excitability and basic motor behavior. Clin Neurophysiol. 2000; 111: 1002–1007. https://doi.org/10.1016/s1388-2457(00)00284-4 PMID: 10825706 50. Siebner HR, Tormos JM, Ceballos-Baumann AO, Auer C, Catala MD, Conrad B, et al. Low-frequency repetitive transcranial magnetic stimulation of the motor cortex in writer’s cramp. Neurology. 1999; 52: 529–529. https://doi.org/10.1212/wnl.52.3.529 PMID: 10025782 51. Jennum P, Winkel H, Fuglsang-Frederiksen A. Repetitive magnetic stimulation and motor evoked potentials. Electroencephalogr Clin Neurophysiol Mot Control. 1995; 97: 96–101. https://doi.org/10. 1016/0924-980x(94)00293-g PMID: 7537209 PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 18 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability 52. Lorenzano C, Gilio F, Inghilleri M, Conte A, Fofi L, Manfredi M, et al. Spread of electrical activity at corti- cal level after repetitive magnetic stimulation in normal subjects. Exp Brain Res. 2002; 147: 186–192. https://doi.org/10.1007/s00221-002-1237-z PMID: 12410333 53. Plewnia C, Lotze M, Gerloff C. Disinhibition of the contralateral motor cortex by low-frequency rTMS: NeuroReport. 2003; 14: 609–612. https://doi.org/10.1097/00001756-200303240-00017 PMID: 12657896 54. Rothwell JC, Hallett M, Berardelli A, Eisen A, Rossini P, Paulus W. Magnetic stimulation: motor evoked potentials. The International Federation of Clinical Neurophysiology. Electroencephalogr Clin Neuro- physiol Suppl. 1999; 52: 97–103. PMID: 10590980 55. Kiers L, Cros D, Chiappa KH, Fang J. Variability of motor potentials evoked by transcranial magnetic stimulation. Electroencephalogr Clin Neurophysiol Potentials Sect. 1993; 89: 415–423. https://doi.org/ 10.1016/0168-5597(93)90115-6 PMID: 7507428 56. Rösler KM, Roth DM, Magistris MR. Trial-to-trial size variability of motor-evoked potentials. A study using the triple stimulation technique. Exp Brain Res. 2008; 187: 51–59. https://doi.org/10.1007/ s00221-008-1278-z PMID: 18231784 57. Wassermann E. Variation in the response to transcranial magnetic brain stimulation in the general pop- ulation. Clin Neurophysiol. 2002; 113: 1165–1171. https://doi.org/10.1016/s1388-2457(02)00144-x PMID: 12088713 58. Goetz SM, Luber B, Lisanby SH, Peterchev AV. A novel model incorporating two variability sources for describing motor evoked potentials. Brain Stimulat. 2014; 7: 541–552. https://doi.org/10.1016/j.brs. 2014.03.002 PMID: 24794287 59. Schwenkreis P, Witscher K, Janssen F, Addo A, Dertwinkel R, Zenz M, et al. Influence of the N-methyl- D-aspartate antagonist memantine on human motor cortex excitability. Neurosci Lett. 1999; 270: 137– 140. https://doi.org/10.1016/s0304-3940(99)00492-9 PMID: 10462113 60. Huang Y-Z, Chen R-S, Rothwell JC, Wen H-Y. The after-effect of human theta burst stimulation is NMDA receptor dependent. Clin Neurophysiol. 2007; 118: 1028–1032. https://doi.org/10.1016/j.clinph. 2007.01.021 PMID: 17368094 61. Grant SG O’dell TJ, Karl KA, Stein PL, Soriano P, Kandel ER. Impaired long-term potentiation, spatial learning, and hippocampal development in fyn mutant mice. Science. 1992; 258: 1903–1910. https:// doi.org/10.1126/science.1361685 PMID: 1361685 62. Di Lazzaro V, Ziemann U. The contribution of transcranial magnetic stimulation in the functional evalua- tion of microcircuits in human motor cortex. Front Neural Circuits. 2013; 7: 18. https://doi.org/10.3389/ fncir.2013.00018 PMID: 23407686 63. Cirillo G, Di Pino G, Capone F, Ranieri F, Florio L, Todisco V, et al. Neurobiological after-effects of non- invasive brain stimulation. Brain Stimulat. 2017; 10: 1–18. https://doi.org/10.1016/j.brs.2016.11.009 PMID: 27931886 64. Lazzaro VD, Restuccia D, Oliviero A, Profice P, Ferrara L, Insola A, et al. Effects of voluntary contrac- tion on descending volleys evoked by transcranial stimulation in conscious humans. J Physiol. 1998; 508: 625–633. https://doi.org/10.1111/j.1469-7793.1998.625bq.x PMID: 9508823 65. Di Lazzaro V, Oliviero A, Profice P, Saturno E, Pilato F, Insola A, et al. Comparison of descending vol- leys evoked by transcranial magnetic and electric stimulation in conscious humans. Electroencephalogr Clin Neurophysiol Mot Control. 1998; 109: 397–401. https://doi.org/10.1016/s0924-980x(98)00038-1 PMID: 9851296 66. Lazzaro VD, Oliviero A, Profice P, Pennisi MA, Pilato F, Zito G, et al. Ketamine increases human motor cortex excitability to transcranial magnetic stimulation. J Physiol. 2003; 547: 485–496. https://doi.org/ 10.1113/jphysiol.2002.030486 PMID: 12562932 67. Collingridge GL, Herron CE, Lester RA. Frequency-dependent N-methyl-D-aspartate receptor-medi- ated synaptic transmission in rat hippocampus. J Physiol. 1988; 399: 301–312. https://doi.org/10.1113/ jphysiol.1988.sp017081 PMID: 2900333 68. Hoogendam JM, Ramakers GMJ, Di Lazzaro V. Physiology of repetitive transcranial magnetic stimula- tion of the human brain. Brain Stimulat. 2010; 3: 95–118. https://doi.org/10.1016/j.brs.2009.10.005 PMID: 20633438 69. Thut G, Miniussi C. New insights into rhythmic brain activity from TMS–EEG studies. Trends Cogn Sci. 2009; 13: 182–189. https://doi.org/10.1016/j.tics.2009.01.004 PMID: 19286414 70. Bortoletto M, Veniero D, Thut G, Miniussi C. The contribution of TMS–EEG coregistration in the explora- tion of the human cortical connectome. Neurosci Biobehav Rev. 2015; 49: 114–124. https://doi.org/10. 1016/j.neubiorev.2014.12.014 PMID: 25541459 71. Burgdorf JS, Christian EP, Sørensen L, Stanton PK, Leaderbrand K, Madsen TM, et al. A translational EEG-based approach to assess modulation of long-lasting NMDAR-dependent synaptic plasticity. PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 19 / 20 PLOS ONE Reproducibility of rTMS induced corticospinal excitability Psychopharmacology (Berl). 2019; 236: 3687–3693. https://doi.org/10.1007/s00213-019-05341-w PMID: 31392357 72. Banerjee J, Sorrell ME, Celnik PA, Pelled G. Immediate Effects of Repetitive Magnetic Stimulation on Single Cortical Pyramidal Neurons. Choi J, editor. PLOS ONE. 2017; 12: e0170528. https://doi.org/10. 1371/journal.pone.0170528 PMID: 28114421 73. Gersner R, Kravetz E, Feil J, Pell G, Zangen A. Long-Term Effects of Repetitive Transcranial Magnetic Stimulation on Markers for Neuroplasticity: Differential Outcomes in Anesthetized and Awake Animals. J Neurosci. 2011; 31: 7521–7526. https://doi.org/10.1523/JNEUROSCI.6751-10.2011 PMID: 21593336 74. Soundara Rajan T, Ghilardi MFM, Wang H-Y, Mazzon E, Bramanti P, Restivo D, et al. Mechanism of Action for rTMS: A Working Hypothesis Based on Animal Studies. Front Physiol. 2017; 8. Available: https://www.frontiersin.org/articles/10.3389/fphys.2017.00457 PMID: 28713286 75. Effects of Repetitive Transcranial Magnetic Stimulation on Motor Functions in Patients With Stroke | Stroke. [cited 2 Aug 2022]. Available: https://www.ahajournals.org/doi/full/10.1161/STROKEAHA.111. 649756 76. Song S, Zilverstand A, Gui W, Li H, Zhou X. Effects of single-session versus multi-session non-invasive brain stimulation on craving and consumption in individuals with drug addiction, eating disorders or obe- sity: A meta-analysis. Brain Stimulat. 2019; 12: 606–618. https://doi.org/10.1016/j.brs.2018.12.975 PMID: 30612944 77. Slotema CW, Blom JD, Hoek HW, Sommer IEC. Should We Expand the Toolbox of Psychiatric Treat- ment Methods to Include Repetitive Transcranial Magnetic Stimulation (rTMS)? A Meta-Analysis of the Efficacy of rTMS in Psychiatric Disorders. J Clin Psychiatry. 2010; 71: 19531. https://doi.org/10.4088/ JCP.08m04872gre PMID: 20361902 78. Freitas C, Fregni F, Pascual-Leone A. Meta-analysis of the effects of repetitive transcranial magnetic stimulation (rTMS) on negative and positive symptoms in schizophrenia. Schizophr Res. 2009; 108: 11–24. https://doi.org/10.1016/j.schres.2008.11.027 PMID: 19138833 79. Enticott PG, Fitzgibbon BM, Kennedy HA, Arnold SL, Elliot D, Peachey A, et al. A Double-blind, Ran- domized Trial of Deep Repetitive Transcranial Magnetic Stimulation (rTMS) for Autism Spectrum Disor- der. Brain Stimulat. 2014; 7: 206–211. https://doi.org/10.1016/j.brs.2013.10.004 PMID: 24280031 80. Osuch EA, Benson BE, Luckenbaugh DA, Geraci M, Post RM, McCann U. Repetitive TMS combined with exposure therapy for PTSD: A preliminary study. J Anxiety Disord. 2009; 23: 54–59. https://doi.org/ 10.1016/j.janxdis.2008.03.015 PMID: 18455908 81. Tergau F, Naumann U, Paulus W, Steinhoff BJ. Low-frequency repetitive transcranial magnetic stimula- tion improves intractable epilepsy. The Lancet. 1999; 353: 2209. https://doi.org/10.1016/S0140-6736 (99)01301-X PMID: 10392988 82. Kleinjung T, Eichhammer P, Langguth B, Jacob P, Marienhagen J, Hajak G, et al. Long-Term Effects of Repetitive Transcranial Magnetic Stimulation (rTMS) in Patients with Chronic Tinnitus. Otolaryngol Neck Surg. 2005; 132: 566–569. https://doi.org/10.1016/j.otohns.2004.09.134 PMID: 15806046 83. Lefaucheur J-P, Fenelon G, Ménard-Lefaucheur I, Wendling S, Nguyen J-P. Low-frequency repetitive TMS of premotor cortex can reduce painful axial spasms in generalized secondary dystonia: a pilot study of three patients. Neurophysiol Clin Neurophysiol. 2004; 34: 141–145. https://doi.org/10.1016/j. neucli.2004.07.003 PMID: 15501683 PLOS ONE | https://doi.org/10.1371/journal.pone.0286465 June 23, 2023 20 / 20