Clinical Epidemiology Dovepress open access to scientific and medical research Open Access Full Text Article Methodology Sample size and power considerations for ordinary least squares interrupted time series analysis: a simulation study This article was published in the following Dove Press journal: Clinical Epidemiology Samuel Hawley 1 Abstract: Interrupted time series (ITS) analysis is being increasingly used in epidemiology. M Sanni Ali 1,2 Despite its growing popularity, there is a scarcity of guidance on power and sample size con- Klara Berencsi 1 siderations within the ITS framework. Our aim of this study was to assess the statistical power Andrew Judge 1,3,4 to detect an intervention effect under various real-life ITS scenarios. ITS datasets were created using Monte Carlo simulations to generate cumulative incidence (outcome) values over time. Daniel Prieto-Alhambra 1,5 We generated 1,000 datasets per scenario, varying the number of time points, average sample 1 Centre for Statistics in Medicine, size per time point, average relative reduction post intervention, location of intervention in the Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal time series, and reduction mediated via a 1) slope change and 2) step change. Performance Sciences, University of Oxford, measures included power and percentage bias. We found that sample size per time point had Oxford, UK; 2Faculty of Epidemiology and Population Health, London School a large impact on power. Even in scenarios with 12 pre-intervention and 12 post-intervention of Hygiene and Tropical Medicine, time points with moderate intervention effect sizes, most analyses were underpowered if the London, UK; 3MRC Lifecourse sample size per time point was low. We conclude that various factors need to be collectively Epidemiology Unit, University of Southampton, Southampton, UK; considered to ensure adequate power for an ITS study. We demonstrate a means of providing 4 Department of Translational Health insight into underlying sample size requirements in ordinary least squares (OLS) ITS analysis Sciences, University of Bristol, Bristol, of cumulative incidence measures, based on prespecified parameters and have developed Stata UK; 5GREMPAL Research Group, Idiap Jordi Gol and CIBERFes, Universitat code to estimate this. Autònoma de Barcelona and Instituto Keywords: epidemiology, interrupted time series, sample size, power, bias de Salud Carlos III, Barcelona, Spain Introduction Interrupted time series (ITS) analysis is being increasingly used in epidemiology.1–3 It is an accessible and intuitive method that can be straightforward to implement and has considerable strengths.4 A common application is when population-level repeated measures of an outcome and/or exposure are available over time, both before and after some well-defined intervention such as a health policy change1,2,5 or a naturally occurring event of interest.6,7 Despite the substantial growth in the use of ITS methods, relatively little practical guidance has been developed in terms of methodological standards within the ITS framework,1,3 including a scarcity of guidance on required sample size. Sample size planning is often a key component of designing a study and should be conducted prior to analysis,8 although this is an aspect very often overlooked in ITS studies, with many being underpowered.9 Information on the power associated with various numbers of repeated measures Correspondence: Samuel Hawley of an outcome (ie, time points) has been previously reported,10 with rules of thumb Botnar Research Centre, Windmill Road, Oxford OX3 7LD, UK concerning the minimum number of pre- and post-intervention time points needed, Email samuel.hawley@ndorms.ox.ac.uk such as 3,3 6,11 8,12 and ≥10.9 However, researchers seeking to aggregate patient-level submit your manuscript | www.dovepress.com Clinical Epidemiology 2019:11 197–205 197 Dovepress © 2019 Hawley et al. This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms. http://dx.doi.org/10.2147/CLEP.S176723 php and incorporate the Creative Commons Attribution – Non Commercial (unported, v3.0) License (http://creativecommons.org/licenses/by-nc/3.0/). By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms (https://www.dovepress.com/terms.php). Hawley et al Dovepress data into a population-level time series to conduct an ITS are ITS scenarios confronted with the practical issue of considering a suitable There are many factors within an OLS ITS framework underlying sample size of subjects/patients per aggregate that could conceivably influence the power to detect the time point.13 Although longer time series have been shown impact of an intervention. Although the following is not to have more power than shorter time series, it seems rea- an exhaustive list, we here describe the main factors that sonable to propose that ITS analyses (even those with many we investigated: time points) with only a small number of subjects per time 1. Total number of time points in the time series, N point may contain so much noise as to render it improbable (­Figure 1A and B): as described in the “Introduction” of detecting a true impact of an intervention under study. section, the ITS approach relies on repeated observations Although the ITS method has many strengths, if a given of an outcome event over time, usually at equally spaced analysis is not adequately powered it may lead to publication intervals such as days, weeks, months, quarters, or years. of weak and spurious findings.14,15 We investigated nine values for the total number of time Given this paucity of guidance on sample size calcula- points (N), ranging from 6 to 50. tion, our aim in this study was to use a simulation approach 2. Number of subjects per time point, n (Figure 1C and D): to estimate power in an ITS analysis case study of repeated the sample size per time point will impact the accuracy measures of cumulative incidence generated from routinely of outcome estimates and hence the dispersion of a given collected health care data. We aimed to quantify the power time series. It is therefore an important factor influencing available in relation to the underlying sample size per time the power to detect an “interruption”. We investigated 11 point, while varying a number of other key parameters of values for n, ranging from approximately 150 to 5,700 interest. Furthermore, we set out to make available Stata patients per time point, which for our specific case study code to be readily usable by epidemiologists as a tool to corresponded to a mean number of outcome events per generate estimates of required sample size for similar ITS time point that ranged from 5 to 200 (Supplementary applications. materials). 3. Nature of intervention impact (Figure 1A–D): the impact Methods of an intervention can be modeled as a “step” change Study design in the level of outcome and/or a “slope” change in the We used Monte Carlo simulations, the strengths of which trend of outcome.4,21 More complex realities can be have been well described previously.16,17 Briefly, simulation incorporated such as multiple interventions, waning or studies involve generating data with known characteristics delayed effects, and nonlinear responses.2,21 However, defined by prespecified input parameter values. Consequently, for the purpose of the current work, we only considered because the truth regarding these characteristics is known, intervention effects mediated through either 1) a step it is possible to empirically evaluate the performance of a change or 2) a slope change. given statistical model when fitted to the simulated data.18,19 4. Effect size, ie, magnitude of intervention impact: one of the assumptions of ITS analysis is that the pre- Aims intervention level and trend of outcome can be used Our aim was to describe the power associated with the mean to predict post-intervention counterfactual estimates, sample size per time point to detect a change in 1) level and ie. expected values of the outcome in the time period 2) trend in an outcome (cumulative incidence) following a after the intervention had pre-intervention level/trend defined intervention in the ITS framework, using ordinary of outcome continued uninterrupted.2,21 The impact of least squares (OLS) regression. We considered a range of intervention can then be expressed as the difference values for various other factors such as total number of time between the estimated counterfactual outcome value points, effect size, and location of intervention in the time for a given post-intervention time point vs the estimated series. We set out to apply the methods within the context of modeled outcome value for the same time point using a specific case study using a recent ITS analysis, where we the observed data.22 In practice, this has often been done evaluated the impact of a UK National Institute for Health for the midpoint of the post-intervention period to yield and Care Excellence (NICE) technology appraisal on the an average post-intervention change.5,20,23 We therefore cumulative incidence of joint replacement within the Clinical used the magnitude of this average post-intervention Practice Research Datalink (CPRD).20 change expressed as a relative % to express effect size, 198 submit your manuscript | www.dovepress.com Clinical Epidemiology 2019:11 Dovepress Dovepress Hawley et al A B 10 10 9 9 8 8 7 7 5 6 5 6 % % 4 4 3 3 2 2 1 1 0 0 10 11 12 13 14 15 11 16 21 26 31 36 41 46 51 1 2 3 4 5 6 7 8 9 1 6 Time point Time point C D 10 10 9 9 8 8 7 7 5 6 5 6 % % 4 4 3 3 2 2 1 1 0 0 15 17 11 13 19 21 23 25 27 29 11 13 15 17 19 21 23 25 27 29 1 3 5 7 9 1 5 3 7 9 Time point Time point Figure 1 Example simulation scenarios for (A) less time points vs (B) more time points; (C) smaller sample size per time point vs (D) larger sample size per time point. defined for mid-time series interventions as the step or on power as this will affect the balance in the number slope change resulting in a –15%, –34%, –50%, and of pre-intervention and post-intervention time points to –75% reduction. be modeled. Locations investigated were at one-third, 5. Mean pre-intervention level and trend of outcome: the midway, and two-thirds from the beginning of the time absolute pre-intervention level of outcome is an impor- series. For trend change scenarios in our case study, we tant factor. For example, a relative 50% reduction in a used the same pre-intervention and post-intervention common outcome should be easier to detect than a rela- trends when investigating early/late interventions as per tive 50% reduction in a rare outcome. Furthermore, a the corresponding midway intervention setting within pre-intervention trend in outcome may exist, which may each N scenario (Supplementary materials). also have an effect on power. We therefore considered two parameters: the mean pre-intervention outcome Data-generating process value (defined using the pre-intervention midpoint) in Data were generated using Stata v15.2 (StataCorp LLC, conjunction with a pre-intervention trend parameter. In College Station, TX, USA), the general principles of which main analyses, we only explored scenarios (based on have been described elsewhere.24 Empty time series datasets our prior CPRD study20), where mean pre-intervention were created of length N (total number of time points). cumulative incidence was 3.5% and there was either 1) Three ITS variables were inserted: time point identifier no pre-intervention trend (for step change scenarios) or (integer), post-intervention indicator (binary), and post- 2) an upward trend (for slope change scenarios), as shown intervention time point identifier (integer).21 The time in Figure 1. We scaled trend parameters according to N point identifier was created first, then used in combination so that absolute pre-intervention values were constant with the “location of intervention” parameter to generate across all mid-time series intervention scenarios. Exact the other two ITS variables. The underlying sample size parameter values for these are provided in the “Supple- for each time point (nt) was then simulated from a normal mentary materials” section. distribution with mean n (a key parameter of interest; 11 6. Location of intervention in time series: location of values investigated) and SD of n/3. The number of outcome intervention in the time series may also have an impact events occurring at each time point was then drawn as a Clinical Epidemiology 2019:11 submit your manuscript | www.dovepress.com 199 Dovepress Hawley et al Dovepress binomial random variate (nt, pt), where nt represents the  Average estimate across simulations  sample size and pt is the probability of outcome. pt was a  - True parameter value  Percentage bias =   *100 linear function defined using the ITS variables in combina-  True parameter value  tion with other scenario-specific parameter values (equa-     tion included in the “Supplementary materials” section). The number of events per time point and nt were used to Sensitivity analysis derive the cumulative incidence time series. A total of To explore the impact of pre-intervention level of outcome, 1,000 Monte Carlo repetitions were carried out for each we repeated main analyses investigating power for slope and unique scenario. step changes while keeping N constant (N=28) but varying pre-intervention level from 3.5% to 8% and then to 20%. Methods of analysis A segmented linear regression model was fitted to each cre- Stata program ated dataset. This took the form of model (1) for step change Although we based the current analyses on a case study scenarios and model (2) for slope change scenarios: exploring a range of parameter values adapted from our (1) Yt=β0 + β1*time pointt + β2*intervention_indicatort + e prior CPRD study as specified earlier,20 we also developed a Stata program (Supplementary materials) with associated (2) Yt=β0 + β1*time pointt + β3*post_intervention_timepointt + e documentation (Supplementary materials) to provide a where Yt is the value of outcome at time point t. b0 estimates ready-to-use means for assessing power associated with any the level of the outcome just before the beginning of the time valid list of (nine) input parameter values as described in the series. b1 estimates the pre-intervention trend, b2 estimates “Supplementary materials” section. the change in level between the time point immediately before vs after the intervention, and b3 estimates the change Results in trend occurring immediately after the intervention. e is Results from our case study are presented in the following the error term. paragraphs describing the impact of N and n on power within several ITS scenarios (Figure 2A and 2B). Results from analy- Estimands ses exploring different effect sizes (whilst keeping N constant) The target of inference was the change in outcome following are presented in Figures 3A and 3B. Although the main results a defined intervention, specifically testing the null hypoth- pertained to a setting where the mean pre-intervention level esis of no change (ie, β2=0 [model 1] or β3=0 [model 2]). of outcome for mid-time series interventions was 3.5%, the The outcome at each time point was a proportion, which in Stata program developed can be used to explore alternative our case study was the 5-year cumulative incidence of joint input parameter values (Supplementary materials). replacement in rheumatoid arthritis patients.20 Slope change Performance As expected, power increased as N and/or n increased The coefficients, standard error, and P-values from these (­Figure 2A) and as effect sizes became larger (Figure 3A). models were stored, and the empirical power to reject the Results for different N and n combinations for each effect null hypothesis of no post-intervention change was calculated size investigated are provided in the “Supplementary materi- as the proportion of simulations, where the P-value for the als” section. These indicated that nearly all mid-time series intervention variable coefficient (step/slope change) was intervention scenarios with a large effect size (–75%) had <0.05.19,24,25 This was represented graphically as contour plots at least 80% power when there were >24 total time points, across scenarios according to N and n. For the convenience even when there was a very small sample size per time of comparison, additional presentation was made for power point (approximately 150 subjects, which in this case-study, according to different effect size and location scenarios while corresponded to only five outcome events per time point). keeping N constant (N=28). In addition, the percentage bias19 However, when the effect size was small (–15%) then to of the regression coefficients was calculated for midway step achieve 80% power an analysis had to either contain a large and slope change scenarios (while keeping N constant), which N or very large n (Supplementary materials). While keeping is defined as follows: other factors constant (effect size =-34% and N=28), power 200 submit your manuscript | www.dovepress.com Clinical Epidemiology 2019:11 Dovepress Dovepress Hawley et al A Slope change 100 0 0 0 0 0 0 0 0 0 0 0 0 0 15 60 05 50 95 40 85 30 75 20 65 10 55 1, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, Sample size per time point 80 Empiracle power 60 40 20 0 6 12 18 24 30 36 42 48 Total number of time points B Step change 100 0 0 0 0 0 0 0 0 0 0 0 0 0 15 60 ,05 ,50 ,95 ,40 ,85 ,30 ,75 ,20 ,65 ,10 ,55 1 1 1 2 2 3 3 4 4 5 5 Sample size per time point 80 Empiracle power 60 40 20 0 6 12 18 24 30 36 42 48 Total number of time points Figure 2 Empiracle power to detect a relative 34% reduction in outcome, where mean pre-intervention incidence is 3.5%: by the number of time points and mean sample size per time point: (A) slope change (B) step change. was greater in scenarios with mid-time series interventions, of time points was moderate (14 pre-intervention and 14 with comparably less power in scenarios with earlier/later post-intervention time points), analyses were underpowered interventions (Figure 4A). The percentage bias in model if sample size per time point was low (Figure 3B and Supple- coefficients was small, and this trended toward zero as sample mentary materials). Interestingly, little difference was found size increased (Figure 5). in power following an early or late intervention as compared to when the intervention occurred midway through (Figure 4). Step change The percentage bias in model coefficients was small, and Similar to slope change scenarios, power increased as N this trended toward zero as sample size increased (Figure 5). and n became larger (Figure 2B) or as the effect size was larger (Figure 3B). Generally, there was less power in step Discussion change scenarios than in corresponding slope change sce- Main findings narios (­Figure 2A and B), with nearly all mid-time series This study demonstrates that simple rules regarding the num- intervention scenarios being inadequately powered when the ber of time points are not adequate by themselves to denote effect size was only –15% (Figure 3B and Supplementary an ITS analysis as sufficiently powered. Other factors such materials). Even when effect sizes were large and the number as the sample size per time point, expected effect size, loca- Clinical Epidemiology 2019:11 submit your manuscript | www.dovepress.com 201 Dovepress Hawley et al Dovepress A B 100 100 80 80 60 60 Power Power 40 40 20 20 0 0 1,000 2,000 3,000 4,000 5,000 6,000 0 1,000 2,000 3,000 4,000 5,000 6,000 Average sample size (per time point) Average sample size (per time point) 15% relative reduction 15% relative reduction 34% relative reduction 34% relative reduction 50% relative reduction 50% relative reduction 75% relative reduction 75% relative reduction Figure 3 Empirical power in the case studya (stratified by effect size) to detect an intervention resulting in (A) a slope change or (B) step change. Note: aAssuming a mean pre-intervention outcome of 3.5%, mid-time series intervention, and 28 total time points. A B 100 100 80 80 Power Power 60 60 40 40 20 20 0 1,000 2,000 3,000 4,000 5,000 6,000 0 1,000 2,000 3,000 4,000 5,000 6,000 Average sample size (per time point) Average sample size (per time point) Early intervention Early intervention Midway intervention Midway intervention Late intervention Late intervention Figure 4 Empirical power in the case studya (stratified by intervention location) to detect an intervention resulting in (A) a slope change or (B) step change. Note: aAssuming a mean pre-intervention outcome of 3.5%, 28 total time points, and an average 34% relative reduction post intervention (early/late slope changes were identical to midway scenario, therefore, achieved a different effect size). tion of intervention in the time series, and pre-intervention step change reduction, with 14 pre- and 14 post-intervention trends need to be considered. For example, in our case study time points, one needed over 1,000 subjects per time point where mean pre-intervention level of outcome was 3.5%, to (ie, >28,000 total subjects), which may or may not be realistic achieve 80% power to detect a relative 34% post-intervention for a given study. However, three pre- and post-intervention 202 submit your manuscript | www.dovepress.com Clinical Epidemiology 2019:11 Dovepress Dovepress Hawley et al 6 4 Percentage bias 2 0 −2 −4 −6 0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000 5,500 6,000 Sample size Step Slope Figure 5 Percentage bias in estimates of intervention impact in the case studya: stratified by the nature of impact. Note: aAssuming a mean pre-intervention outcome of 3.5%, total of 28 time points, and an average 34% relative reduction post intervention. time points were equally sufficient to achieve 80% power difference between post-intervention values and counterfac- in relatively rare situations of large intervention effect sizes tuals, which in the case of slope change scenarios continued combined with very large sample sizes per time point (Sup- to increase as per the pre-intervention slope and therefore plementary materials). These results underline the importance made detection of a change more probable. of robust pre-study sample size planning. Estimates arising Within scenarios with a slope change, we found power from scenarios with a very small n were only slightly biased, to be greater in settings with a balanced number of pre- which disappeared as n increased (Figure 3). intervention and post-intervention time points (as opposed That power increases as N increases is an expected find- to earlier/later interventions), while the location of the inter- ing and has previously been shown for fixed ratios of effect vention had little impact on power to detect step changes and size to the SD of the time series.10,26 However, we in this was even marginally greater when the intervention occurred study addressed the previously undescribed trade-off between early. Although this was unexpected, it is not without some N and n. This is an important consideration and a helpful support from previous work.10 development. First, the SD of a given number of repeated population-level outcome measures may likely be difficult Limitations for applied researchers to estimate in advance of a proposed Our study is subject to various limitations. Each time point ITS study. Second, exploring this trade-off between N and n was a cumulative incidence, and given that individual sub- informs to what extent it may be beneficial (in terms of power) jects/patients could only be included in a single time point, when generating an aggregate ITS dataset to sacrifice sample we treated time points to be independent. As such, we did not size per time point to increase the number of time points (or explore what impact autocorrelation may have on estimates, vice versa). It allows a combination of N and n to be selected although this remains a subject for further investigation. to optimize power. Although the exact nuances of this unique Despite the availability of ITS approaches that explicitly trade-off were scenario specific, in most cases only very little model autocorrelation, such as autoregressive integrated gain in power was achieved when a time series was lengthened moving average (ARIMA) models,27 it would seem that at the expense of time point sample size, although gains were where the assumptions of OLS regression are met then this more noticeable where a very short time series was lengthened. is preferable for epidemiological studies where the goal is To the best of our knowledge, a differential power accord- likely to be causal inference rather than future prediction. ing to whether an intervention impact is mediated via a slope Indeed, while autocorrelation needs to be addressed where or step change has not previously been investigated. We present, it has been noted that in epidemiological studies it found that power was greater in slope change scenarios, a can often be accounted for by controlling for other variables,2 likely explanation being that our effect size was the average and interestingly of a recent review of over 200 drug utiliza- Clinical Epidemiology 2019:11 submit your manuscript | www.dovepress.com 203 Dovepress Hawley et al Dovepress tion studies implementing ITS analysis, 50% were found to Conclusion use segmented linear regression.1 Specification of ARIMA Multiple factors influence the power of OLS ITS analysis, models are frequently cited to require a minimum of 50 time and these should be collectively taken into account when points,28 with >100 being preferable,27 yet it is common to considering the feasibility of a proposed ITS study. We have have less than this minimum available in epidemiology con- demonstrated how a simulation approach can be used to texts using routinely collected data.10,21,23,29 For these reasons, estimate the power available within specific ITS scenarios our focus in this study was on “short” time series where we and provide Stata code to facilitate pre-analysis sample size considered 50 time points as a maximum and used Durbin– planning of future ITS studies within similar applications. Watson statistics to confirm that first-order autocorrelation was not present. Previous work investigated the relationship Acknowledgments between the number of time points and power in the presence This study was partially supported by Oxford NIHR Biomedi- of autocorrelation,10,30 where positive autocorrelation has cal Research Unit. Andrew Judge was partially supported been shown to reduce power and negative autocorrelation by the NIHR Biomedical Research Centre at the University to increase power.10 Similarly, we did not consider season- Hospitals Bristol NHS Foundation Trust and the University of ality nor situations where there may be a delay or waning Bristol. The views expressed in this publication are those of intervention effect. the authors and not necessarily those of the NHS, the National Another limitation is that our definition of effect size Institute for Health Research, or the Department of Health. as the difference between post-intervention time points and counterfactual time points (ie, what would have been Author contributions observed had pre-intervention level/slope continued unin- SH, DP-A, and AJ contributed to study conception and terrupted) involves extrapolation and therefore uncertainty. design. SH, MSA, and KB contributed to data simulation. SH, While this is often done in practice, with uncertainty of model KB, and MSA contributed to data analysis. SH contributed to estimates expressed using CIs,22 there is still the assumption drafting the manuscript. All authors contributed to revising that pre-intervention trends would have continued unchanged. the manuscript critically for important intellectual content, We only investigated scenarios where the repeated out- gave final approval of the version to be published, and agree come measure is a cumulative incidence (ie, a proportion). to be accountable for all aspects of the work. This is a common epidemiological measure, but incorporat- ing other common measures such as person-year rates, means Disclosure (eg, length of hospital stay or drug doses prescribed), and AJ has received consultancy fees from Freshfields Bruck- frequencies is a logical next step and remains the subject for haus Deringer and is a member of the Data Safety and imminent further investigation. Monitoring Board (which involved receipt of fees) from Anthera Pharmaceuticals, Inc., outside the submitted work. Strengths DP-A’s research group has received unrestricted research The disentangling of N and n is a key strength and novel grants from Servier Laboratoires, AMGEN, and UCB aspect of the current study, as is the separate consideration Pharma. SH, MSA, and KB report no conflicts of interest of post-intervention step and slope changes. Although we did in this work. not investigate the impact of varying all of the parameters defined, the development and inclusion of a Stata program References 1. Jandoc R, Burden AM, Mamdani M, Lévesque LE, Cadarette SM. are important features of the investigation, facilitating Interrupted time series analysis in drug utilization research is increas- researchers to estimate sample size requirements for future ing: systematic review and recommendations. J Clin Epidemiol. 2015;68(8):950–956. ITS studies in similar applications and thereby promoting 2. Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series the avoidance of carrying out underpowered analyses. We regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017 Feb 1;46(1):348–355. are currently working on using this tool as the basis for an 3. Ewusie JE, Blondal E, Soobiah C, et al. Methods, applications, inter- online calculator. It is also worth mentioning that we based pretations and challenges of interrupted time series (ITS) data: protocol the parameter values for our case study on a “real-world” for a scoping review. BMJ Open. 2017;7(6):e016018. 4. Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regres- clinical scenario20 to increase the applicability of the findings, sion based quasi-experimental approach when randomisation is not an rather than starting from arbitrary parameter values. option: interrupted time series analysis. BMJ. 2015;350:h2750. 204 submit your manuscript | www.dovepress.com Clinical Epidemiology 2019:11 Dovepress Dovepress Hawley et al 5. Hawton K, Bergen H, Simkin S, et al. Long term effect of reduced 17. Ripley BD. Stochastic Simulation. New York, USA: John Wiley and pack sizes of paracetamol on poisoning deaths and liver transplant Sons Ltd; 2006. activity in England and Wales: interrupted time series analyses. BMJ. 18. Morris T, White I, Crowther M. Using simulation studies to evaluate 2013;346:f403. statistical methods. Tutorial in Biostatistics. 2017. 6. Laliotis I, Ioannidis JPA, Stavropoulou C. Total and cause-specific 19. Burton A, Altman DG, Royston P, Holder RL. The design of simulation mortality before and after the onset of the Greek economic crisis: an studies in medical statistics. Stat Med. 2006;25(24):4279–4292. interrupted time-series analysis. Lancet Public Health. 2016;1(2): 20. Hawley S, Cordtz R, Dreyer L, Edwards CJ, Arden NK, Delmestri e56–e65. A, Cooper C, Judge A, Prieto-Alhambra D. The Impact of Biologic 7. Craig P, Cooper C, Gunnell D, et al. Using natural experiments to evalu- Therapy Introduction on Hip and Knee Replacement Among Rheuma- ate population health interventions: new Medical Research Council toid Arthritis Patients: An Interrupted Time Series Analysis Using the guidance. J Epidemiol Community Health. 2012;66(12):1182–1186. Clinical Practice Research Datalink [Abstract]. Arthritis Rheumatol. 8. Lenth RV. Some practical guidelines for effective sample size determina- 2016;68(suppl 10). tion. Am Stat. 2001;55(3):187–193. 21. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented 9. Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. Interrupted regression analysis of interrupted time series studies in medication use time series designs in health technology assessment: lessons from two research. J Clin Pharm Ther. 2002;27(4):299–309. systematic reviews of behavior change strategies. Int J Technol Assess 22. Zhang F, Wagner AK, Soumerai SB, Ross-Degnan D. Methods for Health Care. 2003;19(4):613–623. estimating confidence intervals in interrupted time series analyses of 10. Zhang F, Wagner AK, Ross-Degnan D. Simulation-based power calcu- health interventions. J Clin Epidemiol. 2009;62(2):143–148. lation for designing interrupted time series analyses of health policy 23. Hawton K, Bergen H, Simkin S, et al. Effect of withdrawal of co- interventions. J Clin Epidemiol. 2011;64(11):1252–1261. proxamol on prescribing and deaths from drug poisoning in England 11. Fretheim A, Zhang F, Ross-Degnan D, et al. A reanalysis of cluster and Wales: time series analysis. BMJ. 2009;338:b2270. randomized trials showed interrupted time-series studies were valuable 24. Feiveson AH. Power by simulation. Stata J. 2002;2(2):107–124. in health system evaluation. J Clin Epidemiol. 2015;68(3):324–333. 25. Sayers A, Crowther MJ, Judge A, Whitehouse MR, Blom AW. Determin- 12. Penfold RB, Zhang F. Use of interrupted time series analysis in ing the sample size required to establish whether a medical device is evaluating health care quality improvements. Acad Pediatr. 2013;13(6 non-inferior to an external benchmark. BMJ Open. 2017;7(8):e015397. Suppl):S38–S44. 26. McLeod AI, Vingilis ER. Power computations in time series analyses 13. Cordtz RL, Hawley S, Prieto-Alhambra D, et al. Incidence of hip for traffic safety interventions. Accid Anal Prev. 2008;40(3):1244–1248. and knee replacement in patients with rheumatoid arthritis following 27. Box GEP, Jenkins GM, Reinsel GC, Ljung GM. Time Series Analysis: the introduction of biological DMARDs: an interrupted time-series Forecasting and Control. 5th ed. Hoboken, NJ: Wiley; 2015. analysis using nationwide Danish healthcare registers. Ann Rheum Dis. 28. Chatfield C. The Analysis of Time Series: An Introduction. Chatfield CTM, 2018;77(5):684–689. Zidek J, editors. Taylor & Francis e-Library: CHAPMAN & HALL/CRC; 14. Van Calster B, Steyerberg EW, Collins GS, Smits T. Consequences of Boca Raton, Florida. 2004. relying on statistical significance: Some illustrations. Eur J Clin Invest. 29. Hawley S, Leal J, Delmestri A, et al; REFReSH Study Group. Anti- 2018;48(5):e12912. Osteoporosis Medication Prescriptions and Incidence of Subsequent 15. Button KS, Ioannidis JP, Mokrysz C, et al. Power failure: why small Fracture Among Primary Hip Fracture Patients in England and Wales: sample size undermines the reliability of neuroscience. Nat Rev Neu- An Interrupted Time-Series Analysis. J Bone Miner Res. 2016;31(11): rosci. 2013;14(5):365–376. 2008–2015. 16. Chang M. Monte Carlo Simulation for the Pharmaceutical Industry: 30. McLeod AI, Vingilis ER. Power Computations for Intervention Analysis. Concepts, Algorithms, and Case Studies. Boca Raton: CRC Press; 2010. Technometrics. 2005;47(2):174–181. Clinical Epidemiology Dovepress Publish your work in this journal Clinical Epidemiology is an international, peer-reviewed, open access, systematic reviews, risk and safety of medical interventions, epidemiol- online journal focusing on disease and drug epidemiology, identifica- ogy and biostatistical methods, and evaluation of guidelines, translational tion of risk factors and screening procedures to develop optimal pre- medicine, health policies and economic evaluations. The manuscript ventative initiatives and programs. Specific topics include: diagnosis, management system is completely online and includes a very quick prognosis, treatment, screening, prevention, risk factor modification, and fair peer-review system, which is all easy to use. Submit your manuscript here: https://www.dovepress.com/clinical-epidemiology-journal Clinical Epidemiology 2019:11 submit your manuscript | www.dovepress.com 205 Dovepress