ORIGINAL RESEARCH published: 17 May 2019 doi: 10.3389/fnins.2019.00447 Test–Retest Reliability of the Effects of Continuous Theta-Burst Stimulation Ali Jannati 1* , Peter J. Fried 1 , Gabrielle Block 1† , Lindsay M. Oberman 2† , Alexander Rotenberg 1,3 and Alvaro Pascual-Leone 1,4* 1 Berenson-Allen Center for Noninvasive Brain Stimulation and Division of Cognitive Neurology, Department of Neurology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States, 2 Neuroplasticity and Autism Edited by: Spectrum Disorder Program, Department of Psychiatry and Human Behavior, E.P. Bradley Hospital, Warren Alpert Medical Olivier David, School, Brown University, East Providence, RI, United States, 3 Neuromodulation Program and Division of Epilepsy Institut National de la Santé et de la and Clinical Neurophysiology, Department of Neurology, Boston Children’s Hospital, Harvard Medical School, Boston, MA, Recherche Médicale (INSERM), United States, 4 Institut Guttman de Neurorehabilitació, Universitat Autónoma de Barcelona, Barcelona, Spain France Reviewed by: Objectives: The utility of continuous theta-burst stimulation (cTBS) as index of cortical Silvia Casarotto, University of Milan, Italy plasticity is limited by inadequate characterization of its test–retest reliability. We thus Giuseppe D’Avenio, evaluated the reliability of cTBS aftereffects, and explored the roles of age and common Istituto Superiore di Sanità (ISS), Italy single-nucleotide polymorphisms in the brain-derived neurotrophic factor (BDNF) and *Correspondence: apolipoprotein E (APOE) genes. Ali Jannati jannati@gmail.com; Methods: Twenty-eight healthy adults (age range 21–65) underwent two identical ajannati@bidmc.harvard.edu Alvaro Pascual-Leone cTBS sessions (median interval = 9.5 days) targeting the motor cortex. Intraclass apleone@bidmc.harvard.edu correlation coefficients (ICCs) of the log-transformed, baseline-corrected amplitude of † Present address: motor evoked potentials (1MEP) at 5–60 min post-cTBS (T5–T60) were calculated. Gabrielle Block, School of Medicine, New York Adjusted effect sizes for cTBS aftereffects were then calculated by taking into account Medical College, Valhalla, NY, the reliability of each cTBS measure. United States Lindsay M. Oberman, Results: 1MEP at T50 was the most-reliable cTBS measure in the whole sample Center for Neuroscience (ICC = 0.53). Area under-the-curve (AUC) of 1MEPs was most reliable when calculated and Regenerative Medicine, Department of Medical and Clinical over the full 60 min post-cTBS (ICC = 0.40). cTBS measures were substantially more Psychology, Uniformed Services reliable in younger participants (< 35 years) and in those with BDNF Val66Val and APOE University of the Health Sciences, ε4– genotypes. Bethesda, MD, United States Conclusion: cTBS aftereffects are most reliable when assessed 50 min post-cTBS, or Specialty section: This article was submitted to when cumulative 1MEP measures are calculated over 30–60 min post-cTBS. Reliability Neural Technology, of cTBS aftereffects is influenced by age, and BDNF and APOE polymorphisms. a section of the journal Reliability coefficients are used to adjust effect-size calculations for interpretation and Frontiers in Neuroscience planning of cTBS studies. Received: 01 January 2019 Accepted: 18 April 2019 Keywords: transcranial magnetic stimulation, continuous theta-burst stimulation, plasticity, reliability, Published: 17 May 2019 BDNF, APOE Citation: Jannati A, Fried PJ, Block G, Abbreviations: %1, percent change from the baseline; 1B−A , visit-B minus visit-A; 1MEP, natural log-transformed, Oberman LM, Rotenberg A and baseline-corrected amplitude of motor evoked potentials; APOE, apolipoprotein E; AUC, area under-the-curve; BDNF, Pascual-Leone A (2019) Test–Retest brain-derived neurotrophic factor; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; GABA, gamma- Reliability of the Effects of Continuous aminobutyric acid; ICC, intraclass correlation coefficient; iTBS, intermittent theta-burst stimulation; LME, linear mixed- Theta-Burst Stimulation. effects regression analysis; LTD, long-term depression; LTP, long-term potentiation; NMDA, N-Methyl-D-aspartic acid; PCR, Front. Neurosci. 13:447. polymerase chain reaction; Rm-ANOVA, repeated-measures analysis of variance; SNP, single-nucleotide polymorphism; doi: 10.3389/fnins.2019.00447 T0–Tn, over the first n minutes following cTBS; TMS, transcranial magnetic stimulation; Tn, at n minutes post-cTBS. Frontiers in Neuroscience | www.frontiersin.org 1 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults INTRODUCTION populations such as individuals with Alzheimer’s disease (Freitas et al., 2011), autism spectrum disorder (Oberman et al., 2012), Transcranial magnetic stimulation (TMS) is a method for focal diabetes (Fried et al., 2016), and schizophrenia (McClintock et al., non-invasive stimulation of the brain through electromagnetic 2011) from healthy individuals. For comparison, at least three induction (Barker et al., 1985). Application of TMS within the studies have assessed the reliability of intermittent theta-burst recommended guidelines (Rossi et al., 2009; Rossini et al., 2015) stimulation (iTBS) aftereffects (Hinder et al., 2014; Fried et al., is a safe means of triggering or modulating neural activity in a 2017; Schilberg et al., 2017). given brain region or network (Pascual-Leone et al., 2011; Fox Full characterization of the test–retest reliability of cTBS et al., 2012; Valero-Cabré et al., 2017). A form of repetitive TMS aftereffects is essential to properly interpret results and plan (rTMS) known as continuous theta-burst stimulation (cTBS) for future studies. We thus aimed to address this need consists of 50 Hz bursts of three TMS pulses repeated at 5 Hz by systematically assessing the test–retest reliability of cTBS for a total of 600 pulses over 40 s (Huang et al., 2005). The aftereffects in 5- or 10-min intervals for 60 min post-cTBS in average amplitude of motor evoked potentials (MEPs) induced a sizeable sample of healthy adults. We also calculated adjusted by single TMS pulses is reduced by approximately 25% for up effect sizes for cTBS aftereffects by taking into account the to 50 min following cTBS of the primary motor cortex (M1) reliability (or lack thereof) of each cTBS measure (Friedman, (Wischnewski and Schutter, 2015). This neuromodulatory effect 1968; Wright, 2014; Fried et al., 2017). In addition, we is thought to involve mechanisms similar to long-term depression explored the effects of age group on the reproducibility of (LTD) (Pascual-Leone et al., 1994; Huang et al., 2005; Hallett, cTBS aftereffects, as well as of single-nucleotide polymorphisms 2007). Therefore, the pattern of cTBS-induced changes in MEPs (SNPs) in brain-derived neurotrophic factor (BDNF) and provides a neurophysiologic index of the mechanism of cortical apolipoprotein E (APOE) genes, which have been found to plasticity (Pascual-Leone et al., 2005, 2011; Oberman et al., 2010, influence neuroplasticity (White et al., 2001; Cheeran et al., 2012, 2014, 2016; Suppa et al., 2016). 2008; Nichol et al., 2009; Antal et al., 2010; Peña-Gomez et al., The neuromodulatory effect of cTBS applied to M1 or 2012; Lee et al., 2013; Chang et al., 2014; Di Lazzaro et al., other brain regions has been investigated for its potential as a 2015; Jannati et al., 2017). Our results can improve the utility neurophysiological biomarker and a therapeutic intervention in of cTBS as a neurophysiologic index of cortical plasticity in several neurological and psychiatric disorders (Koch et al., 2009, neurological and psychiatric disorders, help elucidate the sources 2012; Eberle et al., 2010; McClintock et al., 2011; Cazzoli et al., of intra-individual variability in cTBS responses, and ensure 2012; Oberman et al., 2012, 2014; Di Lazzaro et al., 2013, 2016; adequate sample size and power in future cTBS studies in Mori et al., 2013; Cantone et al., 2014; Chuang et al., 2014; Forogh clinical populations. et al., 2014; Li et al., 2014; Suppa et al., 2014; Carrette et al., 2016). Despite its growing popularity, however, cTBS responses show large inter-individual (Hamada et al., 2013; Goldsworthy MATERIALS AND METHODS et al., 2014; López-Alonso et al., 2014; Vallence et al., 2015; Guerra et al., 2017; Heidegger et al., 2017; Hordacre et al., 2017; Jannati Participants et al., 2017) and intra-individual variability (Vernet et al., 2014; Twenty-eight healthy adults (25 males, age range: 21–65) Vallence et al., 2015) that can limit the utility of cTBS for assessing participated in the study, which was approved by the local brain plasticity in clinical populations. Institutional Review Board in accordance with the Declaration Only two published studies have assessed the reproducibility of Helsinki. All participants provided written informed consent of cTBS aftereffects (Vernet et al., 2014; Vallence et al., prior to enrollment and received monetary compensation 2015). The first (Vernet et al., 2014) used a relatively upon completion. None of the participants had any TMS small sample size (n = 10) and did not report reliability contraindication (Rossi et al., 2009), and all had normal coefficients of cTBS aftereffects, which can be compared with physical and neurological examinations. Individual and group- the reliability coefficients of other TMS measures (Carroll level demographics are presented in Tables 1 and 2, respectively. et al., 2001; Kimiskidis et al., 2004; Christie et al., 2007; Livingston and Ingersoll, 2008; Bastani and Jaberzadeh, 2012; Neuropsychological Testing Ngomo et al., 2012; Hinder et al., 2014; Liu and Au-Yeung, Mini-Mental State Examination (Folstein et al., 1975; Crum 2014; Sankarasubramanian et al., 2015; Schambra et al., 2015; et al., 1993) and the Abbreviated Battery of Stanford–Binet Hermsen et al., 2016; Fried et al., 2017). The second study IV intelligence scale (Thorndike et al., 1986), including (Vallence et al., 2015) used an input–output curve approach that Verbal Knowledge and Non-Verbal Fluid Reasoning allowed assessment of cTBS aftereffects elicited over a range subscores, were completed. of stimulation intensities, but at the cost of fewer time-points. Specifically, assessments were only performed at 0, 15, and 30 min Genetic Analyses post-cTBS. This excluded the earliest time points, i.e., 5 and Saliva samples from 22 participants were assessed for BDNF 10 min post-cTBS, which typically exhibit the maximal cTBS Val66Met polymorphism and the presence of APOE ε4 allele, effects (Wischnewski and Schutter, 2015) and later time-points, as reported previously (Jannati et al., 2017). Aliquot (700 µL) i.e., 40–60 min post-cTBS, which capture the longer-lasting TBS extraction of genomic DNA was performed on saliva samples effects and have been found to be useful in differentiating clinical collected using the Oragene Discover OGR-250 Kit (DNA Frontiers in Neuroscience | www.frontiersin.org 2 May 2019 | Volume 13 | Article 447 Jannati et al. TABLE 1 | Participants’ demographics, neuropsychological measures, single-nucleotide polymorphisms, and inter-visit measures. Inter-visit |1B−A | Age range Education Verbal Non-verbal interval Visit-A Visit-B start time ID (years) Race Ethnicity (year) Handedness BDNF APOE MMSE IQ KN FR (days) start time start time (min) 1 20–25 White Non-Hispanic 16 R Val/Met ε3/ε3 30 130 17 13 10 10:20 11:18 58 2 26–30 Other Non-Hispanic 19 R Val/Val ε3/ε4 30 121 17 10 8 13:50 13:50 0 3 20–25 Multiracial Hispanic 16 R Val/Val ε2/ε4 30 106 11 11 33 14:50 14:52 2 Frontiers in Neuroscience | www.frontiersin.org 4 20–25 Multiracial Non-Hispanic 16 R – – 30 112 11 13 7 9:55 9:50 5 5 56–60 White Non-Hispanic 16 R Val/Met ε3/ε3 30 109 13 10 7 9:36 9:25 11 6 56–60 Black Non-Hispanic 14 R – – 30 97 9 10 28 10:50 11:43 53 7 46–50 White Non-Hispanic 20+ R Val/Val ε3/ε3 30 133 17 14 9 14:38 13:57 41 8 20–25 Asian Non-Hispanic 16 R – – 30 94 8 10 7 9:27 9:39 12 9 50–55 White Non-Hispanic 16 R Val/Val ε3/ε4 30 106 13 9 9 14:00 14:01 1 10 50–55 White Non-Hispanic 16 R Val/Val ε3/ε3 30 115 14 11 19 12:56 14:39 103 11 60–65 White Non-Hispanic 16 R Val/Met ε3/ε4 30 118 12 14 7 11:25 11:22 3 12 20–25 White Hispanic 15 R Val/Val ε3/ε4 30 109 14 9 10 9:42 9:52 10 13 20–25 White Hispanic 17 R Val/Val ε2/ε3 30 112 10 14 7 9:47 9:39 8 3 14 30–35 White Non-Hispanic 19 R Val/Val ε3/ε3 29 100 9 11 8 14:33 10:19 254 15 20–25 White Non-Hispanic 17 R – – 30 118 13 13 8 9:46 9:28 18 16 45–50 White Non-Hispanic 16 R – – 30 94 9 9 28 14:04 13:36 28 17 26–30 White Hispanic – R Val/Val ε3/ε3 30 115 13 12 12 12:04 13:10 66 18 26–30 White Hispanic – R Val/Val ε3/ε3 29 100 10 10 7 10:52 10:50 2 19 50–55 Black Non-Hispanic – R – – 30 97 10 9 11 9:12 12:07 175 20 20–25 White Non-Hispanic 17 R Val/Val ε2/ε3 30 127 16 13 7 14:14 14:15 1 21 20–25 Multiracial Hispanic 17 R Val/Met ε2/ε3 28 115 12 13 7 9:50 10:14 24 22 45–50 Black Non-Hispanic 18 R Val/Val ε3/ε4 29 118 16 10 14 11:29 14:09 160 23 20–25 Asian Non-Hispanic 17 R Val/Met ε3/ε4 30 103 9 12 17 9:41 9:40 1 24 45–50 Black Non-Hispanic 18 R Val/Met ε3/ε3 30 88 9 7 19 10:43 10:45 2 25 30–35 Asian Non-Hispanic 20+ R Val/Met ε3/ε4 30 103 8 13 14 9:53 9:41 12 26 60–65 Asian Non-Hispanic 20+ R Val/Val ε3/ε4 30 124 15 13 7 11:02 10:04 58 27 45–50 White Non-Hispanic 13 R Val/Val ε3/ε4 30 88 9 7 19 11:47 13:02 75 28 30–35 White Non-Hispanic 20+ L Val/Met ε3/ε4 29 97 10 9 20 13:55 13:48 7 |1B−A |, absolute inter-visit difference; APOE, apolipoprotein E; BDNF, brain derived neurotrophic factor; FR, fluid reasoning; IQ, intelligence quotient; KN, knowledge; Met, methionine; MMSE, Mini-Mental State Examination; Val, valine. Racial and ethnic categories were defined according to the National Institutes of Health policy and guidelines on the inclusion of minorities as subjects in clinical research (NIH Office of Extramural Research, 2001). May 2019 | Volume 13 | Article 447 cTBS Reliability in Healthy Adults Jannati et al. cTBS Reliability in Healthy Adults TABLE 2 | Participants’ demographics, single-nucleotide polymorphisms, neuropsychological results, and neurophysiological measures for the total sample and for the age and genetic subgroups. All Age < 35 Age ≥ 45 BDNF Met− BDNF Met+ APOE ε4− APOE ε4+ (N = 28) (n = 16) (n = 12) p (n = 14)† (n = 8)† p (n = 12)† (n = 10)† p Age (year, mean ± SD) 36.8 ± 14.5 25.3 ± 4.3 52.1 ± 6.5 N/ A 36.6 ± 14.4 37.8 ± 15.4 0.86 33.8 ± 12.6 40.8 ± 16.1 0.27 Sex (M : F) 25 : 3 15 : 1 10 : 2 0.56 13 : 1 8:0 1.00 11 : 1 10 : 0 1 Race (White : non-White) 16 : 12 9:7 7:5 1.00 10 : 4 4:4 0.39 9:3 5:5 0.38 Ethnicity (Hispanic : non-Hispanic) 6 : 22 6 : 10 0 : 12 0.02 9:5 7:1 0.35 7:5 9:1 0.16 Education (year, mean ± SD)∗ 17.0 ± 2.3 17.4 ± 1.9 16.4 ± 2.8 – 17.3 ± 2.4 17.8 ± 2.1 0.69 17.3 ± 1.6 17.7 ± 2.8 – BDNF (Met− : Met+)† 14 : 8 8:5 6:3 – 14 : 0 0:8 N/A 8:4 6:4 1.00 APOE (ε4− : ε4+)† 12 : 10 8:5 4:5 – 8:6 4:4 1.00 12 : 0 0 : 10 N/ A Handedness (Right: Left ) 27 : 1 15 : 1 12 : 0 1.00 14 : 0 7:1 0.36 12 : 0 9:1 0.46 MMSE score (mean ± SD) 29.8 ± 0.5 29.7 ± 0.6 29.9 ± 0.3 0.24 29.8 ± 0.4 29.6 ± 0.7 0.52 29.7 ± 0.7 29.8 ± 0.4 0.58 IQ (mean ± SD) 108.9 ± 12.3 110.1 ± 10.6 107.2 ± 14.7 0.55 112.4 ± 12.0 107.9 ± 13.1 0.42 112.5 ± 13.2 108.7 ± 11.5 0.49 Verbal KN score 11.9 ± 2.9 11.8 ± 3.0 12.2 ± 2.9 0.71 13.1 ± 2.9 11.3 ± 2.9 0.16 12.6 ± 2.9 12.3 ± 3.2 0.83 Non-verbal FR score 11.0 ± 2.0 11.6 ± 1.6 10.3 ± 2.4 0.08 11.0 ± 2.0 11.4 ± 2.4 0.70 11.6 ± 2.0 10.6 ± 2.3 0.30 RMT (% MSO, mean ± SD) Visit A 35.3 ± 7.6 35.4 ± 8.7 35.3 ± 6.4 0.97 32.4 ± 5.5 38.1 ± 9.7 0.09 31.2 ± 6.0 37.9 ± 8.2 0.054 Visit B 35.9 ± 7.7 35.9 ± 8.2 35.9 ± 7.4 0.99 33.6 ± 5.8 37.4 ± 10.1 0.27 32.4 ± 6.4 38.0 ± 8.3 0.09 AMT (% MSO, mean ± SD) Visit A 25.9 ± 5.2 26.6 ± 5.9 25.0 ± 4.0 0.44 24.3 ± 3.7 27.3 ± 6.2 0.18 24.6 ± 4.8 26.3 ± 5.1 0.42 Visit B 25.7 ± 4.6 25.9 ± 5.0 25.4 ± 4.3 0.78 24.3 ± 3.2 26.8 ± 5.9 0.21 24.3 ± 4.2 26.3 ± 4.6 0.29 Baseline MEP amplitude (mV, mean ± SD) Visit A 1.3 ± 1.5 1.1 ± 1.1 1.5 ± 1.9 0.42 1.2 ± 1.2 0.8 ± 0.5 0.45 1.1 ± 1.3 1.0 ± 0.5 0.75 Visit B 1.1 ± 1.0 1.2 ± 1.0 1.1 ± 1.0 0.82 1.4 ± 1.0 0.7 ± 0.4 0.08 1.2 ± 1.1 0.9 ± 0.5 0.43 Intervisit interval (days) (mean ± SD) 12.8 ± 7.4 11.4 ± 7.0 14.8 ± 7.8 0.24 12.1 ± 7.3 12.6 ± 5.6 0.86 12.1 ± 7.9 12.5 ± 5.0 0.89 |1B−A | Start Time 42.5 ± 62.2 30.0 ± 62.9 59.2 ± 59.7 0.23 55.8 ± 74.6 14.8 ± 19.0 0.15 47.7 ± 72.6 32.7 ± 51.9 0.59 (min, mean ± SD) Comparisons of proportions were conducted with Fisher’s exact test. Education and single-nucleotide polymorphisms were not statistically compared between the subgroups because the data were not available for the total sample. The p-values were not adjusted for multiple comparisons. The p-value < 0.05 is highlighted in bold font. |1B-A |, absolute inter-visit difference; AMT, active motor threshold; APOE, apolipoprotein E; APOE ε4+, ε2/ε4 or ε3/ε4 genotype; APOE ε4–, ε2/ε3 or ε3/ε3; BDNF, brain-derived neurotrophic factor; BDNF Met–, Val66Val; BDNF Met+, Val66 Met; FR, fluid reasoning; IQ, intelligence quotient; KN, knowledge; MEP, motor evoked potential; Met, metionine; MMSE, Mini-Mental State Examination; MSO, maximum stimulator output; RMT, resting motor threshold; SD, standard deviation; Val, valine. ∗ Education data were available for 26 participants. † BDNF and APOE results were available for 22 participants. Genotek Inc., Ottawa, ON, Canada). DNA was extracted from Participants were seated in a comfortable chair with the right samples using standard methodology and the prepIT L2P reagent arm and hand in a natural pronated ∼90◦ angle on a pillow in (DNA Genotek Inc., 2015). The rs6265 SNP of the BDNF gene, front of them. They were instructed to keep their right hand as and the rs429358 and the rs7412 SNPs of the APOE gene were still and relaxed as possible throughout the experiment. They analyzed using a TaqMan single-tube genotyping assay, which were also monitored for drowsiness and were asked to keep uses polymerase chain reaction (PCR) amplification and a pair of their eyes open during the TMS application. Single TMS pulses fluorescent dye detectors that target the SNP. During PCR, the and cTBS were applied to the left primary motor cortex (M1) at polymerase released the fluorescent probe into solution where 120% of individual resting motor threshold (RMT) and 80% of it was detected using endpoint analysis in an 7900HT Real- active motor threshold (AMT), respectively, as biphasic pulses Time instrument from Applied Biosystems, Inc. (Foster City, with an antero-posterior–postero-anterior (AP-PA) induced CA, United States). current direction using a MagPro X100 stimulator and a MC-B70 Butterfly Coil (outer diameter: 97 mm; MagPro, MagVenture Transcranial Magnetic Stimulation A/S, Farum, Denmark). The coil was held tangentially to the Two identical TMS visits (7–33 days apart; median participant’s head surface, with the handle pointing occipitally interval = 9.5 days) were conducted. The starting times of the two and positioned at 45◦ relative to the mid-sagittal axis of the visits were 0–254 min apart (interquartile range = 1–103 min; participant’s head. The optimal spot for the maximal responses median = 12 min). The inter-visit intervals and starting-time of the right first dorsal interosseous (FDI) muscle (“motor differences for individual subjects are presented in Table 1. hotspot”) was localized. A Polaris infrared-optical tracking All TMS procedures followed the recommended guidelines system (Northern Digital Inc., Waterloo, ON, Canada) and endorsed by the International Federation of Clinical a Brainsight TMS neuronavigation system (Rogue Research Neurophysiology (Rossi et al., 2009; Rossini et al., 2015). Inc., Montreal, QC, Canada) with a brain MRI template Frontiers in Neuroscience | www.frontiersin.org 4 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults (for 21 participants) or the participant’s brain MRI (for the Grand-average values for all cTBS measures were calculated remaining 7 participants) was used to ensure consistent targeting separately for both visits and were compared against zero using throughout the experiment. one-sample t-tests. Visit-B minus visit-A difference (1B−A ) and Surface electromyogram (EMG) was recorded from the |1B−A | were calculated for each neurophysiological measure right FDI with a PowerLab 4/25 data-acquisition device and (Table 3). All analyses were two-tailed, and the α level was set to LabChart 8 software (AD Instruments, Colorado Springs, CO, 0.05. When explicitly noted, false discovery rate (FDR) was used United States). Electrodes were placed over the FDI belly to adjust p-values for multiple testing (Benjamini and Hochberg, (negative) and the first interphalangeal joint of the second finger 1995; Benjamini and Yekutieli, 2001). (positive). The ground electrode was placed over the ipsilateral 1MEPs at T10 and T40 were previously found to be the ulnar styloid process. The TMS system delivered triggered pulses best predictors of inter-individual variability in cTBS aftereffects that synchronized the TMS and EMG systems. EMG signal was in visit-A (Jannati et al., 2017). Thus, to assess the effect of digitized at 1 kHz for 500 ms following each stimulus trigger and potential covariates on the intra-individual variability of cTBS 100 ms pre-trigger, amplified with a range of ±10 mV (band-pass aftereffects at T10 and T40, we conducted linear mixed-effects filter 0.3–1000 Hz). (LME) regression analyses with 1MEPs at T10 or T40 as Each TMS session began by localizing the motor hotspot for dependent variable, Visit (visit-A vs. visit-B) as a within-subject FDI and assessment of the RMT, defined as the lowest intensity factor, and potential covariates including RMT, AMT, baseline of stimulation that elicited MEPs ≥ 50 µV in at least five of ten MEP amplitude, number of days between the two visits (Inter- pulses in the relaxed right FDI. To assess pre-cTBS cortico-motor visit Interval), and the absolute inter-visit difference in starting reactivity, three blocks of 30 single TMS pulses were applied time (in minutes) (Time Difference) as between-subject factors. to M1, with a 5–10 min inter-block interval and at a random Based on previous studies that found in many situations a 4–6 s inter-pulse interval. In each block, individual MEPs > 2.5 regression model is likely to be reliable when the number of SD from the mean were excluded. Baseline MEP amplitude was candidate predictors is smaller than one-tenth of the number calculated as the average of the peak-to-peak amplitude of MEPs of subjects (Harrell, 2015) (p. 72), up to three between-subjects in the three blocks. The AMT was then assessed as the lowest predictors were considered for simultaneous inclusion in any intensity that elicited MEPs ≥ 200 µV in at least five of ten pulses regression model. with the FDI slightly contracted. After a 5-min break, during To assess test–retest reliability, intraclass correlation which participants were instructed to maintain hand relaxation coefficients (ICCs) (Portney and Watkins, 2009) were calculated to control the effects of voluntary hand movements on cTBS in the form of absolute agreement between the two visits for responses (Iezzi et al., 2008), cTBS was applied as 200 bursts of all neurophysiological measures. ICCs were calculated using a three pulses at 50 Hz, repeated at 200-ms intervals for 40 s (for a two-way mixed-effects model, with fixed column (C) effects and total of 600 pulses). Cortico-motor reactivity was reassessed at 5, random row (R) effects (McGraw and Wong, 1996): 10, 15, 20, 30, 40, 50, and 60 min post-cTBS (T5–T60). MSR − MSE ICC(A, 1) = Statistical Analyses MSR + (k − 1)MSE + nk (MSC − MSE ) Stata software version 13.1 (StataCorp, College Station, TX, United States) and MATLAB and Statistics and Machine where ICC(A,1) represents the degree of absolute agreement Learning Toolbox R2016b (The MathWorks, Natick, MA, of measurements made under the two fixed levels of the United States) were used for data analysis. Data from each column factor. k = the number of raters/measurements per TMS visit included: (a) RMT and AMT, expressed as percentage subject; MSR = mean square for rows (representing the of maximum stimulator output; (b) baseline MEP amplitude, individual subjects); MSE = mean square error; MSC = mean calculated as the average of baseline MEP amplitude in 3 blocks square for columns (representing the two visits); n = the of 30 single TMS pulses; and (c) percent change in the average number of subjects. amplitude of 30 MEPs at T5–T60 relative to baseline (%1) for Using this formula, ICC = 1 indicates maximum reliability each participant. and ICC ≤ 0 indicates no reliability [in the case that the within- The Shapiro–Wilk test found significant deviations in MEP group variance is equal to or higher than the between-groups values from normal distribution; thus, natural log-transformed, variance (Kenny et al., 2002)]. ICC values were interpreted as baseline-corrected MEP amplitude at each post-cTBS time point follows (Portney and Watkins, 2009): (i) ICC < 0.25: very low (1MEP) was averaged over all participants separately for each to no reliability; (ii) 0.25 ≤ ICC < 0.50: low reliability; (iii) visit. The following measures were also calculated: absolute MEP 0.50 ≤ ICC < 0.75: moderate reliability; and (iv) ICC ≥ 0.75: modulation at T5–T60 (|1MEP|), maximum suppression and high reliability. ICC values were statistically compared using two- maximum modulation of MEPs during 60 min post-cTBS, area way mixed-effects F statistics (McGraw and Wong, 1996, Table 8). under-the-curve (AUC) and the absolute AUC value (|AUC|) The effects on the ICCs of covariates that had a significant of 1MEPs over T0–T10, . . . , and T0–T60 intervals. Cumulative effect on 1MEPs were assessed by including the covariate in the AUC and |AUC| measures up to each time-point were calculated corresponding mixed-effects regression model and re-calculating as the summed products of the average 1MEP and the average the residual intraclass correlation. |1MEP|, respectively, across each two consecutive time-points Lack of reliability of a measure of interest attenuates the and the time in minutes between them. observed effect size compared to the population parameter Frontiers in Neuroscience | www.frontiersin.org 5 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults TABLE 3 | Neurophysiological measures (mean ± SD) and their test-retest reliability for the whole sample (N = 28). Reproducibility-adjusted effect sizes Visit A Visit B 1B−A |1B−A | ICC p Cohen’s Cohen’s Cohen’s d = 0.2 d = 0.5 d = 0.8 Motor threshold (% MSO) RMT 35.3 ± 7.6 35.9 ± 7.7 0.6 ± 2.2 1.8 ± 1.4 0.96 <0.001 0.20 0.50 0.79 AMT 25.9 ± 5.2 25.7 ± 4.6 −0.2 ± 1.8 1.5 ± 1.1 0.93 <0.001 0.20 0.49 0.78 Baseline MEP amplitude (mV) 1.3 ± 1.5 1.1 ± 1.0 −0.2 ± 1.0 0.6 ± 0.8 0.70 <0.001 0.18 0.46 0.72 Post-cTBS 1MEP T5 −0.05 ± 0.3 −0.07 ± 0.4 −0.02 ± 0.5 0.41 ± 0.3 0.16 0.213 0.13 0.31 0.48 T10 −0.02 ± 0.4 0.08 ± 0.4 0.11 ± 0.5 0.42 ± 0.3 0.11 0.289 0.12 0.28 0.44 T15 0.09 ± 0.3 0.00 ± 0.3 −0.09 ± 0.5 0.40 ± 0.3 −0.16 0.791 − − − T20 0.10 ± 0.3 0.03 ± 0.4 −0.07 ± 0.5 0.39 ± 0.3 0.20 0.157 0.13 0.33 0.51 T30 0.07 ± 0.4 −0.02 ± 0.5 −0.09 ± 0.5 0.36 ± 0.4 0.37 0.024 0.16 0.39 0.61 T40 0.07 ± 0.4 −0.06 ± 0.5 −0.14 ± 0.5 0.40 ± 0.3 0.26 0.076 0.14 0.35 0.55 T50 0.08 ± 0.4 0.07 ± 0.5 −0.01 ± 0.5 0.37 ± 0.3 0.53 0.002 0.17 0.42 0.67 T60 −0.04 ± 0.5 0.03 ± 0.6 0.08 ± 0.8 0.61 ± 0.5 −0.08 0.653 − − − Maximum suppression −0.50 ± 0.5 −0.53 ± 0.5 −0.03 ± 0.6 0.45 ± 0.3 0.38 0.024 0.16 0.39 0.61 Post-cTBS |1MEP| T5 0.29 ± 0.2 0.33 ± 0.2 0.04 ± 0.3 0.29 ± 0.2 −0.27 0.912 − − − T10 0.31 ± 0.2 0.28 ± 0.2 −0.03 ± 0.4 0.27 ± 0.2 −0.18 0.815 − − − T15 0.26 ± 0.2 0.26 ± 0.2 0.00 ± 0.3 0.21 ± 0.1 0.11 0.297 0.11 0.28 0.44 T20 0.29 ± 0.2 0.33 ± 0.3 0.03 ± 0.3 0.22 ± 0.1 0.29 0.068 0.15 0.36 0.57 T30 0.33 ± 0.3 0.37 ± 0.3 0.03 ± 0.3 0.20 ± 0.2 0.50 0.003 0.17 0.42 0.66 T40 0.32 ± 0.2 0.32 ± 0.3 0.00 ± 0.4 0.24 ± 0.3 0.17 0.190 0.13 0.32 0.49 T50 0.33 ± 0.3 0.35 ± 0.3 0.02 ± 0.4 0.29 ± 0.2 0.34 0.037 0.15 0.38 0.59 T60 0.37 ± 0.3 0.44 ± 0.4 0.07 ± 0.5 0.35 ± 0.3 −0.03 0.567 − − − Maximum modulation (|1MEP|) 0.76 ± 0.4 0.86 ± 0.4 0.10 ± 0.4 0.35 ± 0.3 0.31 0.045 0.15 0.37 0.58 AUC of 1MEPs T0–T10 −0.33 ± 2.3 −0.15 ± 2.4 0.18 ± 3.1 2.55 ± 1.6 0.13 0.257 0.12 0.29 0.46 T0–T15 −0.16 ± 3.6 0.05 ± 3.4 0.21 ± 4.6 3.94 ± 2.2 0.15 0.228 0.12 0.31 0.48 T0–T20 0.31 ± 4.7 0.10 ± 4.6 −0.20 ± 6.0 5.14 ± 2.9 0.18 0.183 0.13 0.32 0.50 T0–T30 1.13 ± 7.2 0.15 ± 8.0 −0.99 ± 9.2 7.41 ± 5.4 0.27 0.078 0.14 0.36 0.56 T0–T40 1.85 ± 10.2 −0.26 ± 11.9 −2.12 ± 13.2 10.28 ± 8.3 0.29 0.061 0.15 0.36 0.57 T0–T50 2.60 ± 13.3 −0.25 ± 15.4 −2.85 ± 16.2 12.70 ± 10.2 0.36 0.026 0.16 0.38 0.60 T0–T60 2.77 ± 16.5 0.26 ± 19.1 −2.51 ± 19.7 15.94 ± 11.5 0.40 0.018 0.16 0.39 0.62 |AUC| of 1MEP T0–T10 2.22 ± 1.2 2.36 ± 1.4 0.14 ± 2.1 1.74 ± 1.2 −0.35 0.961 − − − T0–T15 3.63 ± 1.7 3.70 ± 2.1 0.07 ± 3.1 2.51 ± 1.8 −0.32 0.946 − − − T0–T20 5.02 ± 2.1 5.16 ± 2.7 0.14 ± 3.5 2.72 ± 2.2 −0.08 0.647 − − − T0–T30 8.15 ± 3.3 8.64 ± 4.1 0.48 ± 4.4 3.48 ± 2.7 0.31 0.053 0.15 0.37 0.58 T0–T40 11.42 ± 5.0 12.05 ± 6.2 0.64 ± 6.2 4.66 ± 4.0 0.39 0.018 0.16 0.39 0.61 T0–T50 14.66 ± 6.7 15.37 ± 8.3 0.71 ± 7.9 5.96 ± 5.2 0.45 0.008 0.16 0.41 0.64 T0–T60 18.17 ± 8.1 19.33 ± 10.6 1.16 ± 9.9 6.00 ± 0.1 0.46 0.007 0.16 0.41 0.64 AUC and |AUC| of 1MEPs were calculated as the summed products of the average 1MEP and the average |1MEP|, respectively, across two consecutive time-points and the time in minutes between them. The ICC values with p < 0.05 are highlighted in bold font. Abbreviations: 1B-A , Visit B minus Visit A; |1B-A |, absolute inter- visit difference; AMT, active motor threshold; AUC, area under-the-curve; 1MEP, natural log-transformed, baseline-corrected MEP amplitude; ICC, intraclass correlation coefficient; MEP, motor evoked potential; MSO, maximum stimulator output; RMT, resting motor threshold; T0–Tn, over the first n minutes following cTBS. (Hunter and Schmidt, 1994). Following previously applied TMS measures would attenuate small, medium, and large effect methodology (Friedman, 1968; Wright, 2014; Fried et al., 2017), sizes, i.e., Cohen’s d values of 0.2, 0.5, and 0.8 (Cohen, 1992), we assessed how test–retest reliability (or lack thereof) of respectively, which assume perfect reproducibility. First, each Frontiers in Neuroscience | www.frontiersin.org 6 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults idealized Cohen’s d is converted to an r (Cohen, 1988) (p. 23): starting time for individual participants are presented in Table 1. Statistical comparisons of these measures between the two age dIDEALIZED groups are presented in Table 2. rIDEALIZED = q 2 dIDEALIZED +4 Genetic Analyses This idealized r is then adjusted for unreliability using the ICC Available BDNF and APOE results and comparisons of (Wright, 2014): all available measures between BDNF/APOE subgroups are √ presented in Tables 1 and 2, respectively. 2 2 rADJUSTED = rIDEALIZED ∗ ICC Among 22 participants with available DNA results, the frequencies of BDNF Val/Val and Val/Met genotypes were 0.64 Finally, the adjusted r is converted back to an adjusted d and 0.36, respectively, while the frequencies of APOE ε2/ε3, (Friedman, 1968) (p. 246): ε3/ε3, and ε3/ε4 genotypes were 0.14, 0.41, and 0.46, respectively. 2 ∗ rADJUSTED BDNF and APOE subgroups were comparable in all available dADJUSTED = q measures (Table 2). 2 1 − rADJUSTED Baseline Neurophysiological Measures Exploratory Analyses The RMT, AMT, and baseline MEP amplitude in each visit Though unintended, the age of our participants conformed to a and their inter-visit differences are summarized in Tables 2 and bimodal distribution. Thus, to explore the impact of age on our 3, respectively. reliability measures, the total sample was subdivided into two There were no significant differences in any of the baseline distinct age groups with a 10-year gap and a ∼27-year difference neurophysiological measures in either visit between age or in mean age: a Younger group with age < 35 (n = 16; range: 21–34; genetic subgroups (Table 2). There was also no significant mean ± SD, 25.3 ± 4.3) and an Older group with age ≥ 45 (n = 12; difference between the two visits in any of the baseline range: 45–65; mean ± SD, 52.1 ± 6.5). To explore the effect of neurophysiological measures for the whole sample (p’s > 0.14), age on the test–retest reliability of cTBS aftereffects: (i) the ICC for each age group (p’s > 0.17), or for each BDNF (p’s > 0.08) or values of TMS measures were calculated separately for the two APOE (p’s > 0.28) subgroup. age groups; (ii) separate repeated-measures analyses of variance The LME regression analyses of 1MEP at T10 found a (Rm-ANOVAs) were conducted with 1MEP at T10 or T40 as significant, negative effect of Time Difference in all models dependent variable, Age Group as a between-subjects factor, Visit (p’s < 0.025), but no significant effect of any of the baseline as a within-subject factor, and Age Group × Visit interaction. neurophysiological measures or Visit (p’s > 0.21). LME regression Because the proportion of Hispanic participants was significantly analyses of 1MEP at no other time point found a significant effect higher in the Younger group than in the Older group, we assessed of Time Difference (p’s > 0.41). the effect of Ethnicity as a categorical covariate in these Rm- ANOVAs. We also re-calculated all the ICCs for the Younger cTBS-Induced Plasticity Results group while controlling for Ethnicity. Grand-average 1MEPs in visits A and B are shown in To explore the roles of BDNF and APOE SNPs in the Figure 1. 1MEP and |1MEP| values, maximum suppression reliability of TMS measures, we calculated the ICC values of and maximum modulation as well as their inter-visit differences neurophysiological measures over the two visits separately for participants with BDNF Val/Val (Met–; n = 14) and Val/Met (Met+; n = 8) genotypes as well as for those with APOE ε2/ε3 or ε3/ε3 (ε4–; n = 12) and APOE ε2/ε4 or ε3/ε4 (ε4+; n = 10) genotypes. Because BDNF was previously found to influence the cTBS aftereffect at T10 (Jannati et al., 2017), we assessed the cTBS aftereffects separately for BDNF Met– and Met+ participants in each visit and conducted a Rm-ANOVA with 1MEP at T10 as dependent variable, BDNF Status (Met– vs. Met+) as a between- subjects factor, Visit (visit-A vs. visit-B) as a within-subject factor, and BDNF Status × Visit interaction. Further, we assessed the effect of BDNF Status as a covariate in the LME regression analyses at T10. FIGURE 1 | Grand-average 1MEPs recorded from the right FDI muscle at 5 to 60 min following cTBS of the left primary motor cortex in two identical visits. The 1MEPs did not significantly differ from zero at any post-cTBS time point RESULTS in either visit (p’s > 0.11). Error bars represent standard error of the mean. cTBS, continuous theta-burst stimulation; 1MEP, natural log-transformed, baseline-corrected amplitudes of the motor evoked potential; FDI, first Demographics, neuropsychological measures, inter-visit interval, dorsal interosseous. starting times of the two visits, and inter-visit differences in Frontiers in Neuroscience | www.frontiersin.org 7 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults FIGURE 2 | Average 1MEPs recorded from the right FDI muscle at 5 to 60 min following cTBS of the left primary motor cortex in two identical visits in the Younger (A) and Older (B) groups. The 1MEPs did not significantly differ from zero at any time point in either visit in the Younger group (p’s > 0.06). ∗ The 1MEPs in the Older group were significantly greater than zero at T20 in visit-A (PFDR = 0.029), but not at any other time point in either visit (p’s > 0.18). Error bars represent standard error of the mean. cTBS, continuous theta-burst stimulation; 1MEP, natural log-transformed, baseline-corrected amplitudes of the motor evoked potential; FDI, first dorsal interosseous; FDR, false discovery rate. are summarized in Table 3. Grand-average 1MEPs did not as reliability-adjusted effect sizes for each TMS measure in the significantly differ from zero at any time point in either visit whole group are presented in Table 3. (p’s > 0.11). There was also no significant difference in grand- The ICCs of baseline neurophysiological measures was not average 1MEP or |1MEP| between the two visits at any time significantly different between the two age groups (p’s > 0.19; point (p’s > 0.16). Figure 3). In contrast, 1MEPs in the Younger group were The 1MEPs in the Older group were significantly greater than significantly more reliable than in the Older group at T10 and zero at T20 in visit-A (PFDR = 0.029), but not at any other time T60 (PFDR ’s < 0.015), but not at other individual time points point in either visit (p’s > 0.18). The 1MEPs in the Younger (PFDR ’s > 0.058). Similarly, |AUC| measures were significantly group did not significantly differ from zero at any time point in more reliable in the Younger group than in the Older group over either visit (p’s > 0.06). The 1MEPs in the BDNF Met– group T0–T20 and beyond (PFDR ’s < 0.001; Figure 3). The reliability of were significantly less than zero at T10 in visit-A (PFDR = 0.042), other cumulative 1MEP measures was not significantly different but not at any other time point in either visit (p’s > 0.14). The between the two age groups (p’s > 0.058). After adjusting for 1MEPs in the BDNF Met+ group were not significantly different Ethnicity, none of the ICCs in the Younger group crossed our from zero at any time point in either visit (PFDR ’s > 0.05). cTBS pre-defined boundaries for interpretation of ICC values (see aftereffects in both visits for the two age subgroups and the two “Materials and Methods” section). BDNF subgroups are presented in Figures 2 and 4 respectively. To control for potential effects of gender, race/ethnicity, and The Rm-ANOVA on the 1MEP at T10 found a significant handedness on the test–retest reliability of cTBS aftereffects, effect of BDNF status, F(1, 20) = 8.28, p = 0.009, η2p = 0.29, we calculated the ICC values of baseline neurophysiological but no significant effects of Visit or BDNF × Visit interaction measures and 1MEP’s at T5–T60 among White, non-Hispanic, (p’s > 0.10). BDNF Met-carrier status had a significant positive and right-handed males (n = 11, Table 1). The ICC values of RMT, effect in all LME regression analyses of 1MEP at T10 (B̂’s > 0.28, AMT, and baseline MEP amplitude in this subgroup were 0.86 p’s < 0.027). There was no other significant effect in any of the (p < 0.001), 0.96 (p < 0.001), and 0.75 (p = 0.003), respectively. LME models (p’s > 0.08). The ICC values of 1MEP at T5, T10, T15, T20, T30, T40, T50, To control for potential effects of gender, race/ethnicity, and and T60 in this subgroup were 0.54 (p = 0.031), –0.09 (p = 0.602), handedness on cTBS-induced plasticity measures, we calculated 0.18 (p = 0.297), 0.71 (p = 0.005), 0.86 (p < 0.001), 0.67 1MEP’s at T5–T60 in a subgroup of White, non-Hispanic, and (p = 0.008), 0.79 (p = 0.001), and 0.17 (p = 0.309), respectively. right-handed males (n = 11, Table 1). In this smaller, but more- The ICCs of baseline neurophysiological measures were homogenous subsample, 1MEPs did not significantly differ from not significantly different between either the BDNF or zero at any time point in either visit (p’s > 0.23). the APOE subgroups (p’s > 0.16; Figure 5). In contrast, 1MEPs were significantly more reliable in BDNF Met– Test–Retest Reliability of TMS Measures participants than in BDNF Met+ participants at T20–T40 Measures of inter-visit variability and test–retest reliability for (PFDR ’s < 0.023), but not at other time points (p’s > 0.24). RMT, AMT, baseline MEP amplitude and cTBS measures, as well Maximum suppression, maximum modulation, and AUC of Frontiers in Neuroscience | www.frontiersin.org 8 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults FIGURE 3 | Test–retest reliability of baseline neurophysiological measures and post-cTBS 1MEP measures separately in the Younger (age < 35, n = 16) and Older (age ≥ 45, n = 12) groups. |AUC| of 1MEPs were calculated as the summed products of the average |1MEP| across each two consecutive time-points and the time in minutes between them over T0–T10, T0–15, . . . , T0–T60 intervals (marked by their end time point on the abscissa). The ICCs of the RMT, AMT, and baseline MEP amplitude were not significantly different between the two age groups (p’s > 0.19). The |AUC| of 1MEPs were significantly more reliable in the Younger group than in the Older group over T0–T15 and beyond (PFDR ’s < 0.001). The reliability of other cumulative 1MEP measures was not significantly different between the two age groups (p’s > 0.058). AUC, area under-the-curve; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; 1MEP, natural log-transformed, baseline-corrected MEP amplitude; ICC, intraclass correlation coefficient; MEP, motor evoked potential; T0–Tn, over the first n minutes post-cTBS. 1MEPs over T0–T30 and beyond were significantly more reliable the test–retest reliability of cTBS aftereffects during 60 min post- in BDNF Met– participants than in BDNF Met+ participants cTBS. The present study was designed to fill this gap by assessing (PFDR ’s < 0.032; Figure 5). The reliability of other cumulative the test–retest reliability of cTBS aftereffects at 5- or 10-minute 1MEP measures was not significantly different between the two intervals (T5–T60) and of cumulative cTBS aftereffects during BDNF subgroups (PFDR > 0.21). 60 min post-cTBS in healthy adults. Furthermore, in order to 1MEPs were significantly more reliable in APOE ε4– provide guidance and reference for future studies, we calculated participants than in APOE ε4+ participants at T5 and T20–T40 adjusted effect sizes that take into account the test–retest (PFDR ’s < 0.024), but not at other time points (PFDR ’s > 0.07). reliability of cTBS measures. Finally, we explored the influences All AUC measures were significantly more reliable in APOE ε4– of age group and common SNPs in BDNF and APOE genes on participants than in APOE ε4+ participants (PFDR ’s < 0.021; the reliability of cTBS aftereffects. Figure 5). There were no significant differences in the reliability of maximum suppression or maximum modulation between the Overall Reliability of Baseline two APOE subgroups (PFDR ’s > 0.27). Neurophysiological Measures Resting motor threshold had high test–retest reliability (Table 3), DISCUSSION which was comparable with the ICC values reported in most previous studies (Carroll et al., 2001; Kimiskidis et al., 2004; Test–retest reliability of TMS measures influences their utility as Livingston and Ingersoll, 2008; Bastani and Jaberzadeh, 2012; potential neurophysiologic biomarkers or targets for therapeutic Ngomo et al., 2012; Hinder et al., 2014; Schambra et al., 2015; intervention. As the use of plasticity-inducing rTMS protocols Fried et al., 2017; Davila-Pérez et al., 2018) and somewhat becomes more common, it is necessary to investigate the higher than other studies that found RMT ICCs in the magnitude and sources of their inter- and intra-individual 0.75–0.80 range (Fleming et al., 2012; Liu and Au-Yeung, 2014; variability. While some of the factors that contribute to the inter- Sankarasubramanian et al., 2015; Hermsen et al., 2016). The individual variability of these types of plasticity metrics among AMT also had high test–retest reliability (Table 3), which was healthy individuals have been identified (Cheeran et al., 2008, comparable with the results of previous studies (Ngomo et al., 2009; Antal et al., 2010; Hamada et al., 2013; Goldsworthy et al., 2012; Hinder et al., 2014; Fried et al., 2017). 2014; López-Alonso et al., 2014; Nettekoven et al., 2014, 2015; Baseline MEP amplitude had moderate test–retest reliability Vallence et al., 2015; Suppa et al., 2016; Hordacre et al., 2017; (Table 3). The ICC of baseline MEP amplitude found in the Jannati et al., 2017), few studies have assessed the intra-individual present study (ICC = 0.70) was moderate compared to the wide reliability of cTBS responses (Vernet et al., 2014; Vallence et al., range of ICC values for baseline MEP amplitude (–0.16 to 0.87) 2015) and no study, to our knowledge, has systematically assessed reported in previous studies (Kamen, 2004; McDonnell et al., Frontiers in Neuroscience | www.frontiersin.org 9 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults 2004; Christie et al., 2007; Fleming et al., 2012; Ngomo et al., the initial cTBS in the present study may have set into motion 2012; Hinder et al., 2014; Hermsen et al., 2016; Fried et al., subtle changes that were still present when the second cTBS was 2017; Davila-Pérez et al., 2018). Variability of baseline MEP applied. Further, it is possible that demographic, genetic, and amplitude was previously found to be associated with variability state-dependent factors modulate the metaplastic(-like) effects of of TBS aftereffects (Hordacre et al., 2017; Fried et al., 2017). The successive TBS sessions (Opie et al., 2017). moderate reliability of baseline MEP amplitude in the present The low test–retest reliability of several cTBS aftereffects study suggests that such variability was not the main cause of the resulted in adjustment of large- and medium effect sizes to low reliability of some of the cTBS measures reported here. medium and small effect sizes, respectively (Table 3). While calculating the cumulative cTBS measures improved the overall Overall Reliability of cTBS Aftereffects ICCs over T0–T30 and beyond (Figures 3, 5), the ICCs of the The finding that differences between the start times of the cumulative measures for the whole sample remained below 0.5. two visits influenced the cTBS aftereffects at T10 could be Attenuation of idealized effect sizes by this level of reproducibility due to the effect of circadian rhythm on the neuromodulatory indicates that detecting significant differences in cTBS responses effects of rTMS arising from changes in cortical excitability and between healthy and clinical populations may require sample synaptic efficiency during the day (Cohen et al., 2005). While sizes that are substantially larger than those used in most previous the present results cannot definitively conclude that circadian cTBS studies (Wischnewski and Schutter, 2015; Chung et al., factors influenced the intra-individual variability in plasticity at 2016; Suppa et al., 2016), unless steps can be implemented to T10, future studies could attempt to control for the time of day improve the reliability of this technique. or, perhaps even better, to individualize visits to coincide with the The finding that within White, non-Hispanic, and right- same relative point in each subject’s circadian cycle. handed males (n = 11), none of the 1MEPs at any time The finding that T5 had one of the lowest between-visit point in either visit differed significantly from zero suggests variabilities among post-cTBS time points is consistent with the that controlling for demographic factors such as gender, findings of a previous study (Vernet et al., 2014). Importantly, race/ethnicity, and handedness is not enough to overcome the however, the low between-visit variability of cTBS aftereffects large inter-individual variability in cTBS responses in either at T5 and T50 at the group level in the present study did visit. We found, despite comparable reliability of baseline not translate to high test–retest reliability measures, which take neurophysiological measures, T5 and T20–T50 1MEPs were into account both within-individual and between-individuals substantially more reliable in this more-homogenous subgroup, variability; while T50 was the most reliable post-cTBS time point suggesting that heterogeneity of these demographic factors (ICC = 0.53), T5 had very low reliability (ICC = 0.16). This influence the reliability of cTBS aftereffects in the whole sample. pattern of results underlines the importance of calculating the While the small sample precludes definitive conclusions about ICCs of TMS measures rather than relying only on measures of the effects of gender, handedness, and race/ethnicity, these results inter-visit variability at the group level. Further, the low ICCs nonetheless appear to suggest that demographic variation plays a at T5 and T10 indicate that the time points expected to show role in the test–retest reliability of cTBS responses. maximal effects of cTBS (Wischnewski and Schutter, 2015) do not necessarily exhibit high test–retest reliability. This remained Age and Reliability of cTBS Aftereffects true for the cumulative measures of cTBS aftereffects over the first The significant Age Group × Visit interaction effect on T10 20 min post-cTBS. 1MEP indicates that cTBS aftereffects at T10 in the Younger The very low test–retest reliability of 1MEPs at T10 group, but not in the Older group, switched from inhibitory (ICC = 0.11) could be due to two factors: (1) The 1MEPs in visit-A to facilitatory in visit-B (Figure 2). Such reversal at T10 in both visits could be the most influenced by BDNF could be due to metaplasticity or some other state-dependent polymorphism (Jannati et al., 2017). Consistent facilitation of factor. The distinct patterns of cTBS aftereffects at T10 among MEPs, at least numerically, in BDNF Met+ participants in both Younger and Older groups could be due to two factors: (i) Based visits may have resulted in higher test–retest reliability of T10 on animal studies that have found an age-related reduction in 1MEP in that subgroup (Figure 5). (2) Despite the relatively long the efficiency of gamma-aminobutyric acid- (GABA-) mediated inter-visit interval in the present study, T10 seemed to exhibit inhibition (Milbrandt et al., 1994; Billard et al., 1995; McQuail a metaplastic-like effect similar to those reported with shorter et al., 2012), it is possible that older participants have less- intervals (Maeda et al., 2000; Gentner et al., 2008; Valero-Cabré efficient GABAergic synaptic transmission, which is presumed et al., 2008; Oberman et al., 2016) in the overall results (Figure 1), to be involved in cTBS-induced plasticity (Stagg et al., 2009; as well in the Younger group (Figure 2A) and the BDNF Met– Trippe et al., 2009). The resulting reduced inhibitory effects of subgroup (Figure 4A). In all three cases, the direction of the cTBS, potentially combined with cumulative facilitatory effects of neuromodulatory effect of cTBS at T10 was reversed, at least successive single TMS pulses (Pellicciari et al., 2016), could have numerically, from inhibitory in visit-A to facilitatory in visit- resulted in facilitation of MEPs in the Older group, at least in B. Such reversals, when predominant at the individual level, visit-A. (ii) The finding that the Younger group, but not the Older would substantially reduce the test–retest reliability of cTBS group, showed metaplastic-like changes at T10 could be due to aftereffects at T10. Although a previous iTBS study found only age-related differences in the priming effect of TBS (Opie et al., inter-visit intervals shorter than 7 days to be associated with 2017), i.e., a stronger priming effect of cTBS in visit-A among metaplastic changes after iTBS in aging adults (Fried et al., 2017), younger participants. Frontiers in Neuroscience | www.frontiersin.org 10 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults FIGURE 4 | Average 1MEPs recorded from the right FDI muscle at 5 to 60 min following cTBS of the left primary motor cortex in two identical visits in the BDNF Met– (A) and Met+ (B) groups. ∗ The 1MEPs in the BDNF Met– group were significantly less than zero at T10 in visit-A (PFDR = 0.042), but not at any other time point in either visit (p’s > 0.14). The 1MEPs in the BDNF Met+ group were not significantly different from zero at any time point in either visit (PFDR ’s > 0.05). Error bars represent standard error of the mean. BDNF, brain-derived neurotrophic factor; BDNF Met–, Val66Val; BDNF Met+, Val66Met; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; 1MEP, natural log-transformed, baseline-corrected; MEP amplitude; MEP, motor evoked potential; Met, metionine; Val, valine. FIGURE 5 | Test–retest reliability of baseline neurophysiological measures and post-cTBS 1MEP measures separately in the BDNF Met–/Met+ and APOE ε4–/ε4+ groups. AUC of 1MEPs were calculated as the summed products of the average 1MEP across each two consecutive time-points and the time in minutes between them over T0–T10, T0–15, . . . , T0–T60 intervals (marked by their end time point on the abscissa). The ICC values of RMT, AMT, and baseline MEP amplitude were not significantly different between the two BDNF groups (p’s > 0.19) or the two APOE groups (p’s > 0.16). Maximum suppression, maximum modulation, and the AUC of 1MEPs over T0–T30 and beyond were significantly more reliable in the BDNF Met– group than in the BDNF Met+ group (PFDR ’s < 0.032). The reliability of other cumulative 1MEP measures was not significantly different between the two BDNF groups (PFDR > 0.208). All the AUC measures were significantly more reliable in the APOE ε4– group than in the APOE ε4+ group (PFDR ’s < 0.021). The maximum suppression, maximum modulation were not significantly different between the two APOE groups (PFDR ’s > 0.27). AMT, active motor threshold; APOE, apolipoprotein E; APOE ε4+, ε2/ε4 or ε3/ε4 genotype; APOE ε4–, ε2/ε3 or ε3/ε3 genotype; AUC, area under-the-curve; BDNF, brain-derived neurotrophic factor; BDNF Met–, Val66Val; BDNF Met+, Val66Met; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; ICC, intraclass correlation coefficient; 1MEP, natural log-transformed, baseline-corrected MEP amplitude; MEP, motor evoked potential; Met, metionine; RMT, resting motor threshold; T0–Tn, over the first n minutes following cTBS; Val, valine. Similarly, the finding that despite comparable ICCs of baseline studies (Milbrandt et al., 1994; Billard et al., 1995; McQuail et al., neurophysiological measures in the two age groups, cTBS 2012). These results indicate that in order to retain adequate aftereffects were substantially less reliable in older participants power to detect differences in cTBS measures of plasticity in (Figure 3) could be due to the age-related decrease in the future studies, it may be necessary to adjust effect sizes separately efficiency of GABAergic synaptic transmission reported in animal for younger and older age groups. Frontiers in Neuroscience | www.frontiersin.org 11 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults BDNF and APOE Polymorphisms and measures indicates that differences in heterogeneity of Reliability of cTBS Aftereffects demographic factors and baseline cortical excitability did not play a major role in the observed differences in test–retest Despite comparable ICC values of RMT, AMT, and baseline MEP reliability of cTBS aftereffects. amplitude in BDNF Met– and Met+ groups, cTBS aftereffects In addition to considering age, genetic polymorphisms, inter- at several time points and most cumulative measures of cTBS visit interval (Fried et al., 2017), the time of day (Cohen et al., aftereffects were substantially more reliable in BDNF Met– 2005), and the use of neuronavigation (Julkunen et al., 2009), participants than in Met+ participants (Figure 5). This pattern other factors that could improve the test–retest reliability of of results could be due to the following: BDNF Met carrier TMS measures include: ensuring comparable blood glucose levels status is known to be associated with impaired N-Methyl-D- and caffeine intake before and during each visit (Specterman aspartic acid- (NMDA-)dependent LTD (Woo et al., 2005), et al., 2005; Cerqueira et al., 2006; Badawy et al., 2013), aberrant GABAergic synaptic transmission (Abidin et al., 2008), comparable amount and quality of sleep the night before each reduced cTBS-induced inhibition of MEPs (Chung et al., 2016), visit (Civardi et al., 2001; Kreuzer et al., 2011), comparable and “paradoxical” cTBS-induced facilitation of MEPs in visit- intensity and duration of exercise before each visit (Samii A reported in our previous study (Jannati et al., 2017) and et al., 1997; Lentz and Nielsen, 2002), comparable phase of the a few other studies (Gentner et al., 2008; Goldsworthy et al., menstrual cycle across visits (Smith et al., 1999; Hattemer et al., 2012; Hellriegel et al., 2012; Brownjohn et al., 2014). The finding 2007), the use of robotic arms such as the TMS-Robot (Axilum that BDNF Met+ participants showed MEP facilitation, at least Robotics, Schiltigheim, France), which can reduce trial-to-trial numerically, at T10 in both visits (Figure 4B) supports an variability of MEP amplitude (Foucher et al., 2012), comparable association between BDNF Met+ status and facilitatory response baseline MEP amplitude across plasticity visits (Fried et al., to cTBS at T10 in the present sample. The noticeably lower test– 2017), and implementing closed-loop systems that trigger TMS retest reliability of cTBS aftereffects in BDNF Met+ participants pulses timed to real-time, EEG-defined indices of brain states (Figure 5) could be due to the less-efficient cTBS-induced (Zrenner et al., 2016, 2018). plasticity caused by aberrant GABAergic inhibition (Abidin et al., 2008), assumed to be involved in the LTD-like effects of cTBS (Stagg et al., 2009; Trippe et al., 2009). CONCLUSION Despite comparable ICC values of RMT, AMT, and baseline MEP amplitude in APOE ε4– and ε4+ participants, cTBS The present study assessed the test–retest reliability of cTBS aftereffects at several time points and most cumulative measures aftereffects in healthy adults. cTBS aftereffects at most individual of cTBS aftereffects were substantially more reliable in APOE time points had low to moderate reliability. Cumulative cTBS ε4– participants than in APOE ε4+ participants (Figure 5). measures over the first 30 min and beyond were relatively more These results could be due to the influence of APOE ε4 on reliable. Effect sizes adjusted for reliability of cTBS aftereffects NMDA-mediated synaptic plasticity, which has been found to are provided to help future studies retain adequate power for be involved in TBS aftereffects (Huang et al., 2007; Chen et al., comparing M1 cTBS responses between healthy and clinical 2010). These results are also consistent with the less-efficient populations. Those calculations resulted in adjustment of several rTMS-induced activation of brain networks in APOE ε4 carriers large and medium effect sizes to medium and small effect sizes, (Peña-Gomez et al., 2012). respectively, thereby substantially increasing the estimates of the The small number of participants in the BDNF and APOE required sample size to detect a significant difference in cTBS subgroups study limits the generalizability of the present findings responses between healthy and clinical populations. Exploratory on genetic influences on the test–retest reliability of cTBS analyses found cTBS aftereffects were substantially more reliable measures. Assuming that the noticeable differences in reliability in younger participants (age < 35 years) and those with BDNF of cTBS aftereffects in BDNF and APOE subgroups observed Met– and APOE ε4– genotypes. here (Figure 5) are confirmed in future studies, it would be advantageous to consider the expected proportions of BDNF and APOE subgroups and adjust effect sizes for each SNP subgroup ETHICS STATEMENT accordingly. For example, the minor allele frequencies of rs6265 (BDNF), rs429358 (APOE), and rs7412 (APOE) SNPs in the This study was approved by the Institutional Review Board admixed American population in the 1000 Genomes Project at Beth Israel Deaconess Medical Center in accordance with (Auton et al., 2015) are 0.1527, 0.1037, and 0.0476, respectively. the Declaration of Helsinki. All participants provided written As long as that the SNP frequencies among participants do informed consent prior to enrollment and received monetary not significantly deviate from Hardy–Weinberg equilibrium compensation upon completion. proportions (Guo and Thompson, 1992; Wigginton et al., 2005), these frequencies provide good approximations to the frequencies of minor BDNF and APOE alleles in future cTBS studies. AUTHOR CONTRIBUTIONS In comparisons of test–retest reliability of cTBS measures between age or genetic subgroups, the finding that the subgroups AJ, AR, and AP-L conceived and designed the study. AJ in each case were comparable in gender, race/ethnicity, and GB collected the data. AJ analyzed the data and drafted handedness, and the reliability of baseline neurophysiological the manuscript. AJ, PF, AR, and AP-L interpreted the data. Frontiers in Neuroscience | www.frontiersin.org 12 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults All authors revised the manuscript, approved the final version, Jr. Foundation, the NIH (R01 HD069776, R01 NS073601, R21 and agreed to be accountable for the content of the work. MH099196, R21 NS085491, R21 HD07616), the Football Players Health Study at Harvard University, and Harvard Catalyst | The Harvard Clinical and Translational Science Center (NIH FUNDING NCRR and NCATS, UL1 RR025758). The content is solely the responsibility of the authors and does not necessarily represent This study was primarily funded by the National Institutes the official views of Harvard Catalyst, Harvard University and of Health (NIH R01 MH100186). AJ was further supported its affiliated academic health care centers, or any of the listed by postdoctoral fellowships from the Natural Sciences and granting agencies. Engineering Research Council of Canada (NSERC 454617) and the Canadian Institutes of Health Research (CIHR 41791). PF was supported in part by the NIH (R21 NS082970, R21 AG051846). LO was further supported by the Simons ACKNOWLEDGMENTS Foundation Autism Research Initiative (SFARI) and the Nancy Lurie Marks Family Foundation. AR was further supported We thank Stephanie Changeau, Aaron Boes, and Simon by the NIH (R01 NS088583), The Boston Children’s Hospital Laganiere (Beth Israel Deaconess Medical Center) for assistance Translational Research Program, Autism Speaks, Massachusetts with physical/neurological examinations, and Ann Connor and Life Sciences, The Assimon Family, Brainsway, CRE Medical, Joanna Macone (Beth Israel Deaconess Medical Center) for Eisai, Neuroelectrics, Roche, Sage Therapeutics and Takeda regulatory oversight and compliance, and for assistance with Medical. AP-L was further supported by the Sidney R. Baer evaluation of participants’ health and medical history. REFERENCES and in the management of dementia. Clin. Neurophysiol. 125, 1509–1532. doi: 10.1016/j.clinph.2014.04.010 Abidin, I., Eysel, U. T., Lessmann, V., and Mittmann, T. (2008). Impaired Carrette, S., Boon, P., Dekeyser, C., Klooster, D. C. W., Carrette, E., Meurs, A., GABAergic inhibition in the visual cortex of brain-derived neurotrophic factor et al. (2016). Repetitive transcranial magnetic stimulation for the treatment heterozygous knockout mice: GABA release is impaired in visual cortex of of refractory epilepsy. Expert Rev. Neurother. 16, 1093–1110. doi: 10.1080/ BDNF heterozygous KO mice. J. Physiol. 586, 1885–1901. doi: 10.1113/jphysiol. 14737175.2016.1197119 2007.148627 Carroll, T. J., Riek, S., and Carson, R. G. (2001). Reliability of the input–output Antal, A., Chaieb, L., Moliadze, V., Monte-Silva, K., Poreisz, C., properties of the cortico-spinal pathway obtained from transcranial magnetic Thirugnanasambandam, N., et al. (2010). Brain-derived neurotrophic factor and electrical stimulation. J. Neurosci. Methods 112, 193–202. doi: 10.1016/ (BDNF) gene polymorphisms shape cortical plasticity in humans. Brain Stimul. S0165-0270(01)00468-X 3, 230–237. doi: 10.1016/j.brs.2009.12.003 Cazzoli, D., Muri, R. M., Schumacher, R., von Arx, S., Chaves, S., Gutbrod, K., et al. Auton, A., Abecasis, G. R., Altshuler, D. M., Durbin, R. M., Abecasis, G. R., Bentley, (2012). Theta burst stimulation reduces disability during the activities of daily D. R., et al. (2015). A global reference for human genetic variation. Nature 526, living in spatial neglect. Brain 135, 3426–3439. doi: 10.1093/brain/aws182 68–74. doi: 10.1038/nature15393 Cerqueira, V., de Mendonça, A., Minez, A., Dias, A. R., and de Carvalho, M. Badawy, R. A. B., Vogrin, S. J., Lai, A., and Cook, M. J. (2013). Cortical excitability (2006). Does caffeine modify corticomotor excitability? Neurophysiol. Clin. 36, changes correlate with fluctuations in glucose levels in patients with epilepsy. 219–226. doi: 10.1016/j.neucli.2006.08.005 Epilepsy Behav. 27, 455–460. doi: 10.1016/j.yebeh.2013.03.015 Chang, W. H., Bang, O. Y., Shin, Y.-I., Lee, A., Pascual-Leone, A., and Kim, Y.-H. Barker, A. T., Jalinous, R., and Freeston, I. L. (1985). Non-invasive magnetic (2014). BDNF polymorphism and differential rTMS effects on motor recovery stimulation of human motor cortex. Lancet 325, 1106–1107. doi: 10.1016/ of stroke patients. Brain Stimul. 7, 553–558. doi: 10.1016/j.brs.2014.03.008 S0140-6736(85)92413-4 Cheeran, B. J., Ritter, C., Rothwell, J. C., and Siebner, H. R. (2009). Mapping genetic Bastani, A., and Jaberzadeh, S. (2012). A higher number of TMS-elicited mep from influences on the corticospinal motor system in humans. Neuroscience 164, a combined hotspot improves intra- and inter-session reliability of the upper 156–163. doi: 10.1016/j.neuroscience.2009.01.054 limb muscles in healthy individuals. PLoS One 7:e47582. doi: 10.1371/journal. Cheeran, B. J., Talelli, P., Mori, F., Koch, G., Suppa, A., Edwards, M., et al. pone.0047582 (2008). A common polymorphism in the brain-derived neurotrophic factor Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: gene (BDNF) modulates human cortical plasticity and the response to rTMS: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57, BNDF polymorphism modulates response to rTMS. J. Physiol. 586, 5717–5725. 289–300. doi: 10.1111%2Fj.2517-6161.1995.tb02031.x doi: 10.1113/jphysiol.2008.159905 Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate Chen, Y., Durakoglugil, M. S., Xian, X., and Herz, J. (2010). ApoE4 reduces in multiple testing under dependency. Ann. stat. 29, 1165–1188. doi: 10.1186/ glutamate receptor function and synaptic plasticity by selectively impairing 1471-2105-9-114 ApoE receptor recycling. Proc. Natl. Acad. Sci. U.S.A. 107, 12011–12016. doi: Billard, J., Lamour, Y., and Dutar, P. (1995). Decreased monosynaptic 10.1073/pnas.0914984107 GABAB-mediated inhibitory postsynaptic potentials in hippocampal CA1 Christie, A., Fling, B., Crews, R. T., Mulwitz, L. A., and Kamen, G. (2007). pyramidal cells in the aged rat: pharmacological characterization and Reliability of motor-evoked potentials in the ADM muscle of older adults. possible mechanisms. J. Neurophysiol. 74, 539–546. doi: 10.1152%2Fjn.1995.74. J. Neurosci. Methods 164, 320–324. doi: 10.1016/j.jneumeth.2007.05.011 2.539 Chuang, W.-L., Huang, Y.-Z., Lu, C.-S., and Chen, R.-S. (2014). Reduced cortical Brownjohn, P. W., Reynolds, J. N. J., Matheson, N., Fox, J., and Shemmell, J. B. H. plasticity and GABAergic modulation in essential tremor. Mov. Disord. 29, (2014). The Effects of individualized theta burst stimulation on the excitability 501–507. doi: 10.1002/mds.25809 of the human motor system. Brain Stimul. 7, 260–268. doi: 10.1016/j.brs.2013. Chung, S. W., Hill, A. T., Rogasch, N. C., Hoy, K. E., and Fitzgerald, P. B. 12.007 (2016). Use of theta-burst stimulation in changing excitability of motor cortex: Cantone, M., Di Pino, G., Capone, F., Piombo, M., Chiarello, D., Cheeran, B., et al. a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 63, 43–64. (2014). The contribution of transcranial magnetic stimulation in the diagnosis doi: 10.1016/j.neubiorev.2016.01.008 Frontiers in Neuroscience | www.frontiersin.org 13 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults Civardi, C., Boccagni, C., Vicentini, R., Bolamperti, L., Tarletti, R., Varrasi, C., Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid et al. (2001). Cortical excitability and sleep deprivation: a transcranial magnetic estimation. Psychol. Bull. 70, 245–251. doi: 10.1037%2Fh0026258 stimulation study. J. Neurol. Neurosurg. Psychiatry 71, 809–812. doi: 10.1136/ Gentner, R., Wankerl, K., Reinsberger, C., Zeller, D., and Classen, J. (2008). jnnp.71.6.809 Depression of human corticospinal excitability induced by magnetic theta-burst Cohen, D. A., Pascual-Leone, A., Press, D. Z., and Robertson, E. M. (2005). Off-line stimulation: evidence of rapid polarity-reversing metaplasticity. Cereb. Cortex learning of motor skill memory: a double dissociation of goal and movement. 18, 2046–2053. doi: 10.1093/cercor/bhm239 Proc. Natl Acad. Sci. U.S.A. 102, 18237–18241. doi: 10.1073/pnas.0506072102 Goldsworthy, M. R., Müller-Dahlhaus, F., Ridding, M. C., and Ziemann, U. (2014). Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hilsdale. NJ: Inter-subject variability of LTD-like plasticity in human motor cortex: a matter Lawrence Earlbaum Associates. of preceding motor activation. Brain Stimul. 7, 864–870. doi: 10.1016/j.brs.2014. Cohen, J. (1992). A power primer. Psychol. Bull. 112, 155–159. 08.004 Crum, R. M., Anthony, J. C., Bassett, S. S., and Folstein, M. F. (1993). Population- Goldsworthy, M. R., Pitcher, J. B., and Ridding, M. C. (2012). The application of based norms for the mini-mental state examination by age and educational spaced theta burst protocols induces long-lasting neuroplastic changes in the level. JAMA 269, 2386–2391. doi: 10.1001/jama.1993.03500180078038 human motor cortex. Eur. J. Neurosci. 35, 125–134. doi: 10.1111/j.1460-9568. Davila-Pérez, P., Jannati, A., Fried, P. J., Cudeiro Mazaira, J., and Pascual-Leone, A. 2011.07924.x (2018). The effects of waveform and current direction on the efficacy and test– Guerra, A., López-Alonso, V., Cheeran, B., and Suppa, A. (2017). Variability in retest reliability of transcranial magnetic stimulation. Neuroscience 393, 97–109. non-invasive brain stimulation studies: reasons and results. Neurosci. Lett. doi: doi: 10.1016/j.neuroscience.2018.09.044 10.1016/j.neulet.2017.12.058 [Epub ahead of print], Di Lazzaro, V., Capone, F., Di Pino, G., Pellegrino, G., Florio, L., Zollo, L., Guo, S. W., and Thompson, E. A. (1992). Performing the exact test of Hardy- et al. (2016). Combining robotic training and non-invasive brain stimulation Weinberg proportion for multiple alleles. Biometrics 48, 361–372. doi: 10.2307/ in severe upper limb-impaired chronic stroke patients. Front. Neurosci. 10:88. 2532296 doi: 10.3389/fnins.2016.00088 Hallett, M. (2007). Transcranial magnetic stimulation: a primer. Neuron 55, 187– Di Lazzaro, V., Pellegrino, G., Di Pino, G., Corbetto, M., Ranieri, F., Brunelli, N., 199. doi: 10.1016/j.neuron.2007.06.026 et al. (2015). Val66Met BDNF gene polymorphism influences human motor Hamada, M., Murase, N., Hasan, A., Balaratnam, M., and Rothwell, J. C. (2013). cortex plasticity in acute stroke. Brain Stimul. 8, 92–96. doi: 10.1016/j.brs.2014. The role of interneuron networks in driving human motor cortical plasticity. 08.006 Cereb. Cortex 23, 1593–1605. doi: 10.1093/cercor/bhs147 Di Lazzaro, V., Rothwell, J. C., Talelli, P., Capone, F., Ranieri, F., Wallace, A. C., Harrell, F. E. Jr. (2015). Regression Modeling Strategies: with Applications to Linear et al. (2013). Inhibitory theta burst stimulation of affected hemisphere in Models, Logistic and Ordinal Regression, and Survival Analysis. Berlin: Springer. chronic stroke: a proof of principle, sham-controlled study. Neurosci. Lett. 553, Hattemer, K., Knake, S., Reis, J., Rochon, J., Oertel, W. H., Rosenow, F., et al. (2007). 148–152. doi: 10.1016/j.neulet.2013.08.013 Excitability of the motor cortex during ovulatory and anovulatory cycles: a DNA Genotek Inc. (2015). Laboratory Protocol for Manual Purification of DNA transcranial magnetic stimulation study. Clin. Endocrinol. 66, 387–393. doi: from 0.5 mL of Sample. Available at: http://www.dnagenotek.com/US/pdf/PD- 10.1111/j.1365-2265.2007.02744.x PR-006.pdf (accessed January 12, 2016). Heidegger, T., Hansen-Goos, O., Batlaeva, O., Annak, O., Ziemann, U., and Lötsch, Eberle, M.-C., Wildgruber, D., Wasserka, B., Fallgatter, A. J., and Plewnia, C. (2010). J. (2017). A data-driven approach to responder subgroup identification after Relief from chronic intractable auditory hallucinations after long-term bilateral paired continuous theta burst stimulation. Front. Hum. Neurosci. 11:382. doi: theta burst stimulation. Am. J. Psychiatry 167, 1410–1410. doi: 10.1176/appi.ajp. 10.3389/fnhum.2017.00382 2010.10070988 Hellriegel, H., Schulz, E. M., Siebner, H. R., Deuschl, G., and Raethjen, J. H. (2012). Fleming, M. K., Sorinola, I. O., Newham, D. J., Roberts-Lewis, S. F., and Bergmann, Continuous theta-burst stimulation of the primary motor cortex in essential J. H. M. (2012). The effect of coil type and navigation on the reliability of tremor. Clin. Neurophysiol. 123, 1010–1015. doi: 10.1016/j.clinph.2011.08.033 transcranial magnetic stimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 20, Hermsen, A., Haag, A., Duddek, C., Balkenhol, K., Bugiel, H., Bauer, S., et al. 617–625. doi: 10.1109/TNSRE.2012.2202692 (2016). Test–retest reliability of single and paired pulse transcranial magnetic Folstein, M. F., Folstein, S. E., and McHugh, P. R. (1975). Mini-mental state”. stimulation parameters in healthy subjects. J. Neurol. Sci. 362, 209–216. doi: A practical method for grading the cognitive state of patients for the clinician. 10.1016%2Fj.jns.2016.01.039 J. Psychiatr. Res. 12, 189–198. Hinder, M. R., Goss, E. L., Fujiyama, H., Canty, A. J., and Garry, M. I. (2014). Inter- Forogh, B., Yazdi-Bahri, S.-M., Ahadi, T., Fereshtehnejad, S.-M., and Raissi, G. R. and intra-individual variability following intermittent theta burst stimulation: (2014). Comparison of two protocols of transcranial magnetic stimulation for implications for rehabilitation and recovery. Brain Stimul. 7, 365–371. doi: treatment of chronic tinnitus: a randomized controlled clinical trial of burst 10.1016/j.brs.2014.01.004 repetitive versus high-frequency repetitive transcranial magnetic stimulation. Hordacre, B., Goldsworthy, M. R., Vallence, A.-M., Darvishi, S., Moezzi, B., Neurol. Sci. 35, 227–232. doi: 10.1007/s10072-013-1487-5 Hamada, M., et al. (2017). Variability in neural excitability and plasticity Foucher, J., Lorgouilloux, K., Turek, J., Pham, B.-T., Elowe, J., Bayle, B., et al. induction in the human cortex: a brain stimulation study. Brain Stimul. 10, (2012). “Robotic assistance in coil positioning improves reliability and comfort,” 588–595. doi: 10.1016/j.brs.2016.12.001 in Proceedings of the 3rd Annual Conference of the German Society for Brain Huang, Y.-Z., Chen, R.-S., Rothwell, J. C., and Wen, H.-Y. (2007). The after- Stimulation - Modulating Emotions, (Berlin). effect of human theta burst stimulation is NMDA receptor dependent. Clin. Fox, M. D., Halko, M. A., Eldaief, M. C., and Pascual-Leone, A. (2012). Measuring Neurophysiol. 118, 1028–1032. doi: 10.1016/j.clinph.2007.01.021 and manipulating brain connectivity with resting state functional connectivity Huang, Y.-Z., Edwards, M. J., Rounis, E., Bhatia, K. P., and Rothwell, J. C. (2005). magnetic resonance imaging (fcMRI) and transcranial magnetic stimulation Theta burst stimulation of the human motor cortex. Neuron 45, 201–206. doi: (TMS). NeuroImage 62, 2232–2243. doi: 10.1016/j.neuroimage.2012.03.035 10.1016/j.neuron.2004.12.033 Freitas, C., Mondragón-Llorca, H., and Pascual-Leone, A. (2011). Noninvasive Hunter, J. E., and Schmidt, F. L. (1994). “Correcting for sources of artificial brain stimulation in Alzheimer’s disease: systematic review and perspectives for variation across studies,” in The Handbook of Research Synthesis, eds H. Cooper the future. Exp. Gerontol. 46, 611–627. doi: 10.1016/j.exger.2011.04.001 and L. V. Hedge (New York, NY: Russell Sage Foundation), 323–336. Fried, P. J., Jannati, A., Davila-Pérez, P., and Pascual-Leone, A. (2017). Iezzi, E., Conte, A., Suppa, A., Agostino, R., Dinapoli, L., Scontrini, A., et al. (2008). Reproducibility of single-pulse, paired-pulse, and intermittent theta-burst tms Phasic voluntary movements reverse the aftereffects of subsequent theta-burst measures in healthy aging, type-2 diabetes, and alzheimer’s disease. Front. Aging stimulation in humans. J. Neurophysiol. 100, 2070–2076. doi: 10.1152/jn.90521. Neurosci. 9:263. doi: 10.3389/fnagi.2017.00263 2008 Fried, P. J., Schilberg, L., Brem, A.-K., Saxena, S., Wong, B., Cypess, A. M., et al. Jannati, A., Block, G., Oberman, L. M., Rotenberg, A., and Pascual-Leone, (2016). Humans with type-2 diabetes show abnormal long-term potentiation- A. (2017). Interindividual variability in response to continuous theta-burst like cortical plasticity associated with verbal learning deficits. J. Alzheimer’s Dis. stimulation in healthy adults. Clin. Neurophysiol. 128, 2268–2278. doi: 10.1016/ 55, 89–100. doi: 10.3233/JAD-160505 j.clinph.2017.08.023 Frontiers in Neuroscience | www.frontiersin.org 14 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults Julkunen, P., Säisänen, L., Danner, N., Niskanen, E., Hukkanen, T., Mervaala, E., Mori, F., Rossi, S., Piccinin, S., Motta, C., Mango, D., Kusayanagi, H., et al. (2013). et al. (2009). Comparison of navigated and non-navigated transcranial magnetic Synaptic plasticity and PDGF signaling defects underlie clinical progression stimulation for motor cortex mapping, motor threshold and motor evoked in multiple sclerosis. J. Neurosci. 33, 19112–19119. doi: 10.1523/JNEUROSCI. potentials. NeuroImage 44, 790–795. doi: 10.1016/j.neuroimage.2008.09.040 2536-13.2013 Kamen, G. (2004). Reliability of motor-evoked potentials during resting and active Nettekoven, C., Volz, L. J., Kutscha, M., Pool, E.-M., Rehme, A. K., Eickhoff, S. B., contraction conditions. Med. Sci. Sports & Exercise 36, 1574–1579. doi: 10. et al. (2014). Dose-dependent effects of theta burst rTMS on cortical excitability 1249%2F01.mss.0000139804.02576.6a and resting-state connectivity of the human motor system. J. Neurosci. 34, Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., and Kashy, D. A. (2002). The 6849–6859. doi: 10.1523/JNEUROSCI.4993-13.2014 statistical analysis of data from small groups. J. Personal. Soc. Psychol. 83, Nettekoven, C., Volz, L. J., Leimbach, M., Pool, E.-M., Rehme, A. K., Eickhoff, 126–137. doi: 10.1037%2F%2F0022-3514.83.1.126 S. B., et al. (2015). Inter-individual variability in cortical excitability and motor Kimiskidis, V. K., Papagiannopoulos, S., Sotirakoglou, K., Kazis, D. A., network connectivity following multiple blocks of rTMS. NeuroImage 118, Dimopoulos, G., Kazis, A., et al. (2004). The repeatability of corticomotor 209–218. doi: 10.1016/j.neuroimage.2015.06.004 threshold measurements. Neurophysiol. Clin. 34, 259–266. doi: 10.1016/j.neucli. Ngomo, S., Leonard, G., Moffet, H., and Mercier, C. (2012). Comparison of 2004.10.002 transcranial magnetic stimulation measures obtained at rest and under active Koch, G., Bonni, S., Giacobbe, V., Bucchi, G., Basile, B., Lupo, F., et al. conditions and their reliability. J. Neurosci. Methods 205, 65–71. doi: 10.1016/j. (2012). Theta-burst stimulation of the left hemisphere accelerates jneumeth.2011.12.012 recovery of hemispatial neglect. Neurology 78, 24–30. doi: 10.1212/WNL. Nichol, K., Deeny, S. P., Seif, J., Camaclang, K., and Cotman, C. W. (2009). Exercise 0b013e31823ed08f improves cognition and hippocampal plasticity in APOE ε4 mice. Alzheimers Koch, G., Brusa, L., Carrillo, F., Lo Gerfo, E., Torriero, S., Oliveri, M., Dementia 5, 287–294. doi: 10.1016/j.jalz.2009.02.006 et al. (2009). Cerebellar magnetic stimulation decreases levodopa-induced NIH Office of Extramural Research (2001). NIH Policy and Guidelines on dyskinesias in Parkinson disease. Neurology 73, 113–119. doi: 10.1212/WNL. the Inclusion of Women and Minorities as Subjects in Clinical Research. 0b013e3181ad5387 Available at: https://grants.nih.gov/grants/funding/women_min/guidelines_ Kreuzer, P., Langguth, B., Popp, R., Raster, R., Busch, V., Frank, E., et al. amended_10_2001.htm (accessed February 1, 2016). (2011). Reduced intra-cortical inhibition after sleep deprivation: a transcranial Oberman, L., Ifert-Miller, F., Najib, U., Bashir, S., Woollacott, I., Gonzalez- magnetic stimulation study. Neurosci. Lett. 493, 63–66. doi: 10.1016/j.neulet. Heydrich, J., et al. (2010). Transcranial magnetic stimulation provides means to 2011.02.044 assess cortical plasticity and excitability in humans with fragile X syndrome and Lee, M., Kim, S. E., Kim, W. S., Lee, J., Yoo, H. K., Park, K.-D., et al. autism spectrum disorder. Front. Synaptic Neurosci. 2:26. doi: 10.3389/fnsyn. (2013). Interaction of motor training and intermittent theta burst stimulation 2010.00026 in modulating motor cortical plasticity: influence of BDNF Val66Met Oberman, L. M., Eldaief, M., Fecteau, S., Ifert-Miller, F., Tormos, J. M., and polymorphism. PLoS One 8:e57690. doi: 10.1371/journal.pone.0057690 Pascual-Leone, A. (2012). Abnormal modulation of corticospinal excitability in Lentz, M., and Nielsen, J. F. (2002). Post-exercise facilitation and depression of M adults with Asperger’s syndrome: modulation of excitability in Asperger’s. Eur. wave and motor evoked potentials in healthy subjects. Clin. Neurophysiol. 113, J. Neurosci. 36, 2782–2788. doi: 10.1111/j.1460-9568.2012.08172.x 1092–1098. doi: 10.1016/S1388-2457(02)00031-7 Oberman, L. M., Ifert-Miller, F., Najib, U., Bashir, S., Gonzalez-Heydrich, J., Picker, Li, C.-T., Chen, M.-H., Juan, C.-H., Huang, H.-H., Chen, L.-F., Hsieh, J.-C., et al. J., et al. (2016). Abnormal mechanisms of plasticity and metaplasticity in autism (2014). Efficacy of prefrontal theta-burst stimulation in refractory depression: a spectrum disorders and fragile X syndrome. J. Child Adoles. Psychopharmacol. randomized sham-controlled study. Brain 137, 2088–2098. doi: 10.1093/brain/ 26, 617–624. doi: 10.1089/cap.2015.0166 awu109 Oberman, L. M., Pascual-Leone, A., and Rotenberg, A. (2014). Modulation of Liu, H., and Au-Yeung, S. S. Y. (2014). Reliability of transcranial magnetic corticospinal excitability by transcranial magnetic stimulation in children and stimulation induced corticomotor excitability measurements for a hand muscle adolescents with autism spectrum disorder. Front. Hum. Neurosci. 8:627. doi: in healthy and chronic stroke subjects. J. Neurol. Sci. 341, 105–109. doi: 10.1016/ 10.3389/fnhum.2014.00627 j.jns.2014.04.012 Opie, G. M., Vosnakis, E., Ridding, M. C., Ziemann, U., and Semmler, J. G. Livingston, S. C., and Ingersoll, C. D. (2008). Intra-rater reliability of a transcranial (2017). Priming theta burst stimulation enhances motor cortex plasticity in magnetic stimulation technique to obtain motor evoked potentials. Int. J. young but not old adults. Brain Stimul. 10, 298–304. doi: 10.1016/j.brs.2017. Neurosci. 118, 239–256. doi: 10.1080/00207450701668020 01.003 López-Alonso, V., Cheeran, B., Río-Rodríguez, D., and Fernández-del-Olmo, Pascual-Leone, A., Amedi, A., Fregni, F., and Merabet, L. B. (2005). The plastic M. (2014). Inter-individual variability in response to non-invasive brain human brain cortex. Annu. Rev. Neurosci. 28, 377–401. doi: 10.1146/annurev. stimulation paradigms. Brain Stimul. 7, 372–380. doi: 10.1016/j.brs.2014.02.004 neuro.27.070203.144216 Maeda, F., Keenan, J. P., Tormos, J. M., Topka, H., and Pascual-Leone, A. (2000). Pascual-Leone, A., Freitas, C., Oberman, L., Horvath, J. C., Halko, M., Eldaief, Modulation of corticospinal excitability by repetitive transcranial magnetic M., et al. (2011). Characterizing brain cortical plasticity and network dynamics stimulation. Clin. Neurophysiol. 111, 800–805. doi: 10.1016%2Fs1388-2457% across the age-span in health and disease with TMS-EEG and TMS-fMRI. Brain 2899%2900323-5 Topogr. 24, 302–315. doi: 10.1007/s10548-011-0196-8 McClintock, S. M., Freitas, C., Oberman, L. M., Lisanby, S. H., and Pascual-Leone, Pascual-Leone, A., Valls-Solé, J., Wassermann, E. M., and Hallett, M. (1994). A. (2011). Transcranial magnetic stimulation: a neuroscientific probe of cortical Responses to rapid-rate transcranial magnetic stimulation of the human motor function in schizophrenia. Biol. Psychiatry 70, 19–27. doi: 10.1016/j.biopsych. cortex. Brain 117, 847–858. doi: 10.1093%2Fbrain%2F117.4.847 2011.02.031 Pellicciari, M. C., Miniussi, C., Ferrari, C., Koch, G., and Bortoletto, M. (2016). McDonnell, M. N., Ridding, M. C., and Miles, T. S. (2004). Do alternate methods Ongoing cumulative effects of single TMS pulses on corticospinal excitability: of analysing motor evoked potentials give comparable results? J. Neurosci. an intra- and inter-block investigation. Clin. Neurophysiol. 127, 621–628. doi: Methods 136, 63–67. doi: 10.1016/j.jneumeth.2003.12.020 10.1016/j.clinph.2015.03.002 McGraw, K. O., and Wong, S. P. (1996). Forming inferences about some intraclass Peña-Gomez, C., Solé-Padullés, C., Clemente, I. C., Junqué, C., Bargalló, N., Bosch, correlation coefficients. Psychol. Methods 1, 30–46. doi: 10.1037%2F1082-989x. B., et al. (2012). APOE status modulates the changes in network connectivity 1.1.30 induced by brain stimulation in non-demented elders. PLoS One 7:e51833. McQuail, J. A., Bañuelos, C., LaSarge, C. L., Nicolle, M. M., and Bizon, J. L. doi: 10.1371/journal.pone.0051833 (2012). GABAB receptor GTP-binding is decreased in the prefrontal cortex Portney, L. G., and Watkins, M. P. (2009). Foundations of Clinical Research: but not the hippocampus of aged rats. Neurobiol. Aging 33, 1124.e1–1124.e12. Applications to Practice, 3rd Edn. Upper Saddle River, NJ: Prentice Hall. doi: 10.1016/j.neurobiolaging.2011.11.011 Rossi, S., Hallett, M., Rossini, P. M., and Pascual-Leone, A. (2009). Safety, ethical Milbrandt, J. C., Albin, R. L., and Caspary, D. M. (1994). Age-related decrease in considerations, and application guidelines for the use of transcranial magnetic GABAB receptor binding in the Fischer 344 rat i inferior colliculus. Neurobiol. stimulation in clinical practice and research. Clin. Neurophysiol. 120, 2008– Aging 15, 699–703. 2039. doi: 10.1016/j.clinph.2009.08.016 Frontiers in Neuroscience | www.frontiersin.org 15 May 2019 | Volume 13 | Article 447 Jannati et al. cTBS Reliability in Healthy Adults Rossini, P. M., Burke, D., Chen, R., Cohen, L. G., Daskalakis, Z., Di Iorio, R., facilitation to subsequent TMS-mediated behavioural disruptions. Eur. J. et al. (2015). Non-invasive electrical and magnetic stimulation of the brain, Neurosci. 27, 765–774. doi: 10.1111/j.1460-9568.2008.06045.x spinal cord, roots and peripheral nerves: basic principles and procedures for Vallence, A.-M., Goldsworthy, M. R., Hodyl, N. A., Semmler, J. G., Pitcher, J. B., routine clinical and research application. An updated report from an I.F.C.N. and Ridding, M. C. (2015). Inter- and intra-subject variability of motor cortex Committee. Clin. Neurophysiol. 126, 1071–1107. doi: 10.1016/j.clinph.2015. plasticity following continuous theta-burst stimulation. Neuroscience 304, 266– 02.001 278. doi: 10.1016/j.neuroscience.2015.07.043 Samii, A., Wassermann, E. M., and Hallett, M. (1997). Post-exercise depression of Vernet, M., Bashir, S., Yoo, W.-K., Oberman, L. M., Mizrahi, I., Ifert-Miller, F., motor evoked potentials as a function of exercise duration. Electroencephalogr. et al. (2014). Reproducibility of the effects of theta burst stimulation on motor Clin. Neurophysiol. 105, 352–356. doi: 10.1016/S0924-980X(97)00033-7 cortical plasticity in healthy participants. Clin. Neurophysiol. 125, 320–326. Sankarasubramanian, V., Roelle, S. M., Bonnett, C. E., Janini, D., Varnerin, doi: 10.1016/j.clinph.2013.07.004 N. M., Cunningham, D. A., et al. (2015). Reproducibility of transcranial White, F., Nicoll, J. A. R., Roses, A. D., and Horsburgh, K. (2001). Impaired magnetic stimulation metrics in the study of proximal upper limb muscles. neuronal plasticity in transgenic mice expressing human apolipoprotein E4 J. Electromyogr. Kinesiol. 25, 754–764. doi: 10.1016/j.jelekin.2015.05.006 compared to E3 in a model of entorhinal cortex lesion. Neurobiol. Dis. 8, Schambra, H. M., Ogden, R. T., Martínez-Hernández, I. E., Lin, X., Chang, Y. B., 611–625. doi: 10.1006/nbdi.2001.0401 Rahman, A., et al. (2015). The reliability of repeated TMS measures in older Wigginton, J. E., Cutler, D. J., and Abecasis, G. R. (2005). A note on exact tests adults and in patients with subacute and chronic stroke. Front. Cell Neurosci. of hardy-weinberg equilibrium. Am. J. Hum. Genet. 76, 887–893. doi: 10.1086/ 9:335. doi: 10.3389/fncel.2015.00335 429864 Schilberg, L., Schuhmann, T., and Sack, A. T. (2017). Interindividual variability Wischnewski, M., and Schutter, D. J. L. G. (2015). Efficacy and time course of theta and intraindividual reliability of intermittent theta burst stimulation-induced burst stimulation in healthy humans. Brain Stimul. 8, 685–692. doi: 10.1016/j. neuroplasticity mechanisms in the healthy brain. J. Cogn. Neurosci. 29, 1022– brs.2015.03.004 1032. doi: 10.1162/jocn_a_01100 Woo, N. H., Teng, H. K., Siao, C.-J., Chiaruttini, C., Pang, P. T., Milner, T. A., et al. Smith, M. J., Keel, J. C., Greenberg, B. D., Adams, L. F., Schmidt, P. J., Rubinow, (2005). Activation of p75NTR by proBDNF facilitates hippocampal long-term D. A., et al. (1999). Menstrual cycle effects on cortical excitability. Neurology 53, depression. Nat. Neurosci. 8, 1069–1077. doi: 10.1038/nn1510 2069–2069. doi: 10.1212%2Fwnl.53.9.2069 Wright, K. (2014). “Adjusting effect sizes in light of reliability estimates,” in Specterman, M., Bhuiya, A., Kuppuswamy, A., Strutton, P., Catley, M., and Paper Presented at the Annual Meeting of the Southwest Educational Research Davey, N. (2005). The effect of an energy drink containing glucose and caffeine Association, (New Orleans, LA). on human corticospinal excitability. Physiol. Behav. 83, 723–728. doi: 10.1016/j. Zrenner, C., Belardinelli, P., Müller-Dahlhaus, F., and Ziemann, U. (2016). Closed- physbeh.2004.09.008 loop neuroscience and non-invasive brain stimulation: a tale of two loops. Stagg, C. J., Wylezinska, M., Matthews, P. M., Johansen-Berg, H., Jezzard, P., Front. Cell. Neurosci. 10:92. doi: 10.3389/fncel.2016.00092 Rothwell, J. C., et al. (2009). Neurochemical effects of theta burst stimulation as Zrenner, C., Desideri, D., Belardinelli, P., and Ziemann, U. (2018). Real-time assessed by magnetic resonance spectroscopy. J. Neurophysiol. 101, 2872–2877. EEG-defined excitability states determine efficacy of TMS-induced plasticity doi: 10.1152/jn.91060.2008 in human motor cortex. Brain Stimul. 11, 374–389. doi: 10.1016/j.brs.2017. Suppa, A., Huang, Y.-Z., Funke, K., Ridding, M. C., Cheeran, B., Di Lazzaro, V., 11.016 et al. (2016). Ten years of theta burst stimulation in humans: established knowledge, unknowns and prospects. Brain Stimul. 9, 323–335. doi: 10.1016/ Conflict of Interest Statement: AP-L serves on the scientific advisory boards j.brs.2016.01.006 for Neuronix, Starlab Neuroscience, Neuroelectrics, Constant Therapy, Cognito, Suppa, A., Marsili, L., Di Stasio, F., Berardelli, I., Roselli, V., Pasquini, M., et al. NovaVision, and Neosync, and is listed as an inventor on several issued and (2014). Cortical and brainstem plasticity in Tourette syndrome and obsessive- pending patents on real-time integration of TMS with EEG and MRI. AR is a compulsive disorder. Mov. Disord. 29, 1523–1531. doi: 10.1002/mds.25960 founder and advisor for Neuromotion, serves on the medical advisory board for Thorndike, R. L., Hagen, E. P., and Sattler, J. M. (1986). The Stanford-Binet NeuroRex, and is listed as an inventor on a patent related to integration of TMS Intelligence Scale: Guide for Administering and Scoring. Rolling Meadows, IL: and EEG. Riverside Publishing Company. Trippe, J., Mix, A., Aydin-Abidin, S., Funke, K., and Benali, A. (2009). Theta The remaining authors declare that the research was conducted in the absence of burst and conventional low-frequency rTMS differentially affect GABAergic any commercial or financial relationships that could be construed as a potential neurotransmission in the rat cortex. Exp. Brain Res. 199, 411–421. doi: 10.1007/ conflict of interest. s00221-009-1961-8 Valero-Cabré, A., Amengual, J. L., Stengel, C., Pascual-Leone, A., and Coubard, Copyright © 2019 Jannati, Fried, Block, Oberman, Rotenberg and Pascual-Leone. O. A. (2017). Transcranial magnetic stimulation in basic and clinical This is an open-access article distributed under the terms of the Creative Commons neuroscience: a comprehensive review of fundamental principles and novel Attribution License (CC BY). The use, distribution or reproduction in other forums insights. Neurosci. Biobehav. Rev. 83, 381–404. doi: 10.1016/j.neubiorev.2017. is permitted, provided the original author(s) and the copyright owner(s) are credited 10.006 and that the original publication in this journal is cited, in accordance with accepted Valero-Cabré, A., Pascual-Leone, A., and Rushmore, R. J. (2008). Cumulative academic practice. No use, distribution or reproduction is permitted which does not sessions of repetitive transcranial magnetic stimulation (rTMS) build up comply with these terms. Frontiers in Neuroscience | www.frontiersin.org 16 May 2019 | Volume 13 | Article 447