Contemporary Clinical Trials Communications 20 (2020) 100678 Contents lists available at ScienceDirect Contemporary Clinical Trials Communications journal homepage: http://www.elsevier.com/locate/conctc Inclusion of non-inferiority analysis in superiority-based clinical trials with single-arm, two-stage Simon’s design Miguel Sampayo-Cordero a, l, *, Bernat Miguel-Huguet b, José Pérez-García a, c, David Páez a, d, Ángel L. Guerrero-Zotano e, Javier Garde-Noguera f, Elena Aguirre g, Esther Holgado a, h, Elena López-Miranda a, h, Xin Huang i, Andrea Malfettone a, l, Antonio Llombart-Cussac a, j, Javier Cortés c, k, a a Medica Scientia Innovation Research (MedSIR), Barcelona, Spain b Hospital Universitari de Bellvitge, L’Hospitalet, Barcelona, Spain c IOB, Institute of Oncology, QuironSalud Group, Barcelona and Madrid, Spain d Hospital de la Santa Creu i Sant Pau, Universitat Autònoma de Barcelona, Barcelona, Spain e Vanderbilt-Ingram Cancer Center; Vanderbilt University Medical Center, Nashville, TN, USA f Hospital Arnau de Vilanova, Valencia, Spain g Hospital QuironSalud Group, Zaragoza, Spain h Ramón y Cajal University Hospital, Madrid, Spain i Pfizer Global Research and Development, La Jolla, USA j FISABIO - Hospital Arnau de Vilanova, Valencia, Spain k Vall d’Hebron Institute of Oncology (VHIO), Barcelona, Spain l Medica Scientia Innovation Research (MedSIR), Ridgewood, NJ, USA A R T I C L E I N F O A B S T R A C T Keywords: Introduction: Non-inferiority (NI) analysis is not usually considered in the early phases of clinical development. In Non-inferiority some negative phase II trials, a post-hoc NI analysis justified additional phase III trials that were successful. Switching to non-inferiority However, the risk of false positive achievements was not controlled in these early phase analyses. We propose to Two-stage preplan NI analyses in superiority-based Simon’s two-stage designs to control type I and II error rates. Single-arm Methods: Simulations have been proposed to assess the control of type I and II errors rates with this method. A Phase II Group sequential designs total of 12,768 two-stage Simon’s design trials were constructed based on different assumptions of rejection response probability, desired response probability, type I and II errors, and NI margins. P-value and type II error were calculated with stochastic ordering using Uniformly Minimum Variance Unbiased Estimator. Type I and II errors were simulated using the Monte Carlo method. The agreement between calculated and simulated values was analyzed with Bland-Altman plots. Results: We observed the same level of agreement between calculated and simulated type I and II errors from both two-stage Simon’s superiority designs and designs in which NI analysis was allowed. Different examples has been proposed to explain the utility of this method. Conclusion: Inclusion of NI analysis in superiority-based single-arm clinical trials may be useful for weighing additional factors such as safety, pharmacokinetics, pharmacodynamic, and biomarker data while assessing early efficacy. Implementation of this strategy can be achieved through simple adaptations to existing designs for one- arm phase II clinical trials. hypothesis and strategies in a clinical setting [1,2]. Phase II studies in oncology are often multistage trials. Two-stage designs are becoming 1. Introduction increasingly more common, allowing for early trial termination in cases with low response rates (RRs) towards avoiding wasting time resources A single-arm phase II trial is the proof-of-concept stage in drug on ineffective treatments. These trials aim to determine whether the new development, and focuses on the evaluation of new therapeutic * Corresponding author. Medica Scientia Innovation Research (MedSIR), Torre Glòries - Planta 27, Avinguda Diagonal, 211, 08018, Barcelona, Spain. E-mail address: sampayo.mc@gmail.com (M. Sampayo-Cordero). https://doi.org/10.1016/j.conctc.2020.100678 Received 13 July 2020; Received in revised form 2 November 2020; Accepted 22 November 2020 Available online 28 November 2020 2451-8654/© 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 specified amount, called NIM. The null and alternative hypotheses Abbrevations should be defined as follows [6]: H0: p0 – p ≥ NIM (p is inferior to the control (p0) by NIM or more); (1) α The planned level of type I error 1‒β The planned level of power Ha: p0 – p < NIM (p is inferior to the control (p0) by less than NIM). (2) a The prespecified superiority efficacy boundary ani The prespecified no-inferiority efficacy boundary A challenging point in NI analysis is to distinguish an effective CI Confidence interval treatment from a less effective or ineffective treatment. The presence of FDA Food and drug administration assay sensitivity in a NI trial should be stated from: i) historical evidence M1 The assumed effect of the active control against placebo of sensitivity to drug effects based on well controlled trials and a robust M2 The largest loss of effect that would be clinically statistical and clinical judgment (e.g. A treatment cannot be used as a acceptable control arm if the superiority against placebo was inconclusive in his­ n The number of patients the end of the trial torical studies); and ii) appropriate conduct of the trial that adheres n1 The number of patients at the interim analysis closely to the design of the trials used to determine that historical evi­ NI Non-inferiority dence of sensitivity to drug effects exists [12]. The margin chosen for a NIM Non-inferiority margin NI trial should be defined prior to study initiation, taking into these p The effect of the test drug historical evidences. Although the NI margin used in a trial can be no p0 The effect of the drug under the null hypothesis (or the larger than the entire assumed effect of the active control against pla­ effect of the active control therapy) cebo (M1), it is generally desirable to choose a lower margin (M2) that p1 The effect of the drug under the alternative hypothesis reflects the largest loss of effect that would be clinically acceptable [6]. RR Response rates Showing NI to M1 provides assurance that the test drug had an effect UMVUE Uniformly Minimum Variance Unbiased Estimator greater than zero, but, in many cases, that is not sufficient to conclude that the test drug had a clinically acceptable effect [6]. In a fixed margin approach, the NIM could be considered as the risk ratio or risk differ­ ence, reflective of the average effect of the active control over placebo in regimen is superior to a pre-specified RR (often 5%) or experience with previous studies [(pcontrol/pplacebo) > 1 or (pcontrol - pplacebo) > 0], for the standard of care, whereas the alternative hypothesis is that RR is example: somewhat higher, say 20% [3,4]. Nevertheless, the non-inferiority (NI) question might also be relevant Relative risk = 2.64, 95% confidence interval (CI): (1.72 to 3.56). (3) in the phase II setting [5]. A typical scenario is one in which an exper­ Risk difference = 0.15, 95% CI: (0.07 to 0.22). (4) imental treatment is potentially less toxic, less costly, easier to admin­ ister or with novel biological pathways than a conventional treatment, We selected the 95% CI lower bound (1.72 or 0.07) and adjusted to but these do not represent a reduction of efficacy or in percentage of retain at least 50% of the historical effect of active control versus pla­ patient with clinical benefit [6]. cebo arm ([1.72^(1-0.5) = 1.31] or [0.07*(1-0.5) = 0.035]) [6]. In accordance with the European Medicines Agency guidelines [7], Accordingly, the calculated NIM describes a ratio or a difference in any superiority trial where NI may be an acceptable outcome, it is reflecting the largest loss of effect in control group RR (p0) that would be prudent to specify a NI margin in the protocol to avoid serious diffi­ clinically acceptable. Therefore, the null and alternative hypothesis of culties that can arise from later designation. Specification of a margin NI analysis can be defined as follows and depending on p0/NIM: after viewing the results can produce an increase in the alpha error rate H0: p ≤ (p0 / NIMas ratio) or H0: p ≤ (p0 - NIMas difference); (5) [8]. In 2016, the Food and Drug Administration published guidelines to establish effectiveness in NI trials. The statistical issues associated with H1: p > (p0 / NIMas ratio) or H1: p > (p0 - NIMas difference); (6) NI studies, and procedures used to determine the NIM, have been extensively described [6]. Risk ratio is preferred because it is less affected than risk differences Among all available multistage designs, the most popular is a two- by variability in the event rates in the placebo group [6]. stage design with a futility stopping point based on Simon’s minimax or optimal criterion [9]. The simplicity of Simon’s design may account 2.2. Include a NI analyses in a superiority based single-arm design for its popularity. However, the inference procedures used in two-stage designs are often not corrected to account for these designs’ adaptive In a superiority analysis design with tumor response as the primary nature [10]. For point estimation, previous authors have developed a endpoint, analyze firstly a NI hypothesis does not inflate the type I error method to calculate the Uniformly Minimum Variance Unbiased Esti­ rate when NI analysis and NIM are properly pre-specified [7]. Addi­ mator (UMVUE) for Simon’s designs and to achieve optimal results. tionally, the final number of responders needed to achieve the NI What is more, p-values and type II errors can be calculated with sto­ objective will always equal or lower than the prespecified superiority chastic ordering of the UMVUE [9,11]. These methods can be used when efficacy boundary (a) [7].Therefore, we assumed the same number of the realized sample size at the stopping stage is different from that patients as in superiority analysis (n); and ani (number of responding specified in the initial design, and this property makes them very useful patients in NI analysis) is chosen as the lowest integer satisfying the type for designing and analyzing two-stage phase II trials [9]. I error rate in NI analysis (αni) ≤ α: The aim of the present study is to assess the validity of the UMVUE- B(ani|n, p0 / NIM) ≥ 1- α. (7) based calculation method planning two-stage Simon’s design phase II trials with a superiority analysis, where NI analysis is also allowed. The power should be calculated: 2. Methods 1 - βni = B(ani − 1|n, p0/NIM); where ani ≤ a; 1 - β ≤ 1 - βni. (8) Accordingly, the study will achieve a positive finding when “p” is 2.1. NI analysis equal or higher than “p0/NIM” and significance level evaluated by binomial test in NI analysis is ≤ α. As the NI analysis has the same ex­ In a NI analysis, the goal of the study is to show that the effect of the pected accrual and lower or equal number of responders needed to test drug (p) is not inferior to the effect of the active control (p0) by a declare significance than superiority analysis (ani ≤ a), power always 2 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 will be equal or greater in NI than superiority criteria. Thus, this design crossed by more than 2.5% of the differences. This finding suggests that can assess superiority and NI criteria with the same sample size, type I calculated p-values tended to be slightly lower than the simulated alpha and type II error rates used in the superiority strategy (as outlined in the values (Fig. 1). Supplementary Methods) [7]. However, it is important to consider that these two biases are com­ Additionally, in a single-arm two-stage Simon’s design, UMVUE- mon in superiority designs with (NIM>1) and without (NIM = 1) NI based calculation of p-value is still valid when the realized sample size analyses. Additionally, the percentage of values crossing the 95% CI and number of responders to achieve a positive finding are different at boundary is equivalent in both designs (Fig. 1). Comparison of type II the stopping stage from that specified in the design (as outlined in the errors also reflected that calculated type II errors tended to be slightly Supplementary Methods) [9]. Thus, include a NI strategy in a superiority lower than the simulated type II errors. However, superiority design based single-arm trial may be implemented in Phase II Simon’s designs. with and without NI strategy displayed equivalent results with about A numerical example has been proposed in Results section. 95% limits of agreement (Fig. 2). In the superiority scenarios (NIM = 1), we observed that the cloud of 2.3. Simulations points was grouped in values of 0.01, 0.05, and 0.1 for the p-value, and 0.1 and 0.2 for the type II error. This is consistent with the pre-specified A total of 12,768 two-stage, single-arm designs were computed based design constrains. However, we did not observe this behavior in sce­ on different assumptions of p0, p1, α1, 1 - β. The package “Clinfun” narios where NIM >1, because the designs’ sample sizes were pre- (function “ph2simon”) from R software [13] was used for computing specified for superiority analysis. So, type I and II errors that fulfilled these designs [14]. design constraints for different values of NIM showed higher variability. The NIMs selected to formulate the rejection proportion ranged be­ Moreover, there were seven times more scenarios in NIM >1 analysis tween 1 and 1.45 in 0.05 increments [8]. P-values and type II errors in than in superiority analysis (NIM = 1). every design were calculated with stochastic ordering of UMVUE [9]. A The maximum differences between calculated p-values and type I user-defined function in the R software was used for calculating these errors were 0.0015, 0.002, and 0.004 for 0.01, 0.05 and 0.1 type I errors, P-values and type II errors. Alpha and beta errors were simulated with respectively (Fig. 1). Therefore, in a study with a 0.01 significance level, the Monte Carlo method. Number of random samples generated were the maximum simulated type I error ranged from 0.0085 to 0.0115. based on the need to attain 95% confidence, so that simulated values of Studies with a 0.05 significance level had maximum simulated type I alpha and beta errors were within 0.5% and 1% of the true values, errors ranging from 0.048 to 0.052. Finally, studies with 0.1 significance respectively [15]. Agreement between calculated and simulated values levels had maximum simulated type I errors that ranged from 0.096 to were analyzed with Bland-Altman plots (as outlined in the Supplemen­ 0.104 (Fig. 1). tary Methods) [16,17]. The R code detailing this process and illustrating Regarding beta errors, the maximum bias in designs with 0.1 and 0.2 the simulations can be found in the Supplemental Files 1 and 2. The type II errors was 0.005, respectively. Therefore, calculated type II er­ datasets generated in these analyses can be found in supplemental rors ranged from 0.195 to 0.205 and from 0.095 to 0.105 for designs datasets 0, 1, and 2. A summary detailing the information reported in with 80% and 90% of simulated power, respectively (Fig. 2). These re­ these supplementary files can be found in the SupplementaryDataSum­ sults suggest that differences between calculated and simulated scores mary document. are not relevant. Collectively, our findings implied that the minimum and maximum 3. Results differences for calculated and simulated values were equivalent in su­ periority (NIM = 1) and NI (NIM > 1) scenarios (Figs. 1 and 2). Addi­ 3.1. Agreement between simulated and calculated statistical errors tionally, results suggest that the agreement between calculated and simulated statistical errors decrease the greater the error tolerated in the The results showed a proportional bias between calculated p-values statistical design and the calculated statistical errors tend to be lower and simulated alpha levels. Higher levels of type I error related to than simulated ones. Therefore, it would not be recommended to plan greater absolute differences between the calculated p-values and simu­ single-arm designs with error constraints more relaxed than usually lated alpha errors. This is not surprising considering that high statistical accepted levels (alpha and beta errors equal or lower than 0.2) [18]. error is likely reflective of small sample size and high imprecision. Absolute differences have been plotted against average of calculated Moreover, we observed that the lower boundary of the 95% CI was and simulated scores. The type I errors values considered were 0.1, 0.05 Fig. 1. Agreement between calculated p-value and simulated alpha errors in Simon’s two-stage clinical designs. 3 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 Fig. 2. Agreement between calculated and simulated type II error in Simon’s two-stage clinical designs. and 0.01, and type II errors values were 0.2 and 0.1. The NIMs selected the baseline patients’ characteristics (e.g.: prior therapies received, to formulate the rejection proportion (p0/NIM) were (1, 1.15, 1.2, 1.25, or prior lines of therapy) [24]. 1.30, 1.35, 1.4, and 1.45). A maximum of 2.5% deviation defined the 95% limits of agreement. Thus, rejecting a 20% rate of responders would be a conservative Absolute differences have been plotted against average of calculated approach to assess this therapy (null hypothesis). Moreover, we planned and simulated scores. The type I values considered were 0.1, 0.05 and to detect an improvement greater than or equal to 45% (which repre­ 0.01, and type II errors were 0.2 and 0.1. The NIMs selected to formulate sented the alternative hypothesis). We proposed an optimal two-stage the rejection proportion (p0/NIM) were (1, 1.15, 1.2, 1.25, 1.30, 1.35, Simon’s design with the same error constraints accepted for the safety 1.4 and 1.45). A maximum of 2.5% deviation defined the 95% limits of objective (α = 0.1 and 1 ‒ β = 0.9). For the design parameters: agreement. (p0, p1, α*, 1 ‒ β*) = (0.20, 0.45, 0.1, 0.9) (9) 3.2. Numerical example The optimal design was given by: We suggest to use a single-arm trial designed to assess a new inves­ (a1 / n1, a / n) = (4 / 14, 8 / 25) (10) tigational therapy combined with an approved drug which represents where a1, n1, a, and n are the number of responders needed to move to the standard of care [19], or a dose modification of standard of care the second stage, the number of accrued patients at first stage, the aiming to reduce the probability of related adverse events and improve number of responders to achieve a positive result at final analysis, and quality of life [20,21]. The safety profile improvement of the standard of the total number of patients recruited, respectively. Calculations were care will only be acceptable if it is accompanied by a NI result evaluating implemented in the R “Clinfun” library (function “ph2simon (0.20, 0.45, the treatment activity. So, it makes sense to propose a NI contrast for the 0.1, 0.1)”) [14]. evaluation of early efficacy. In addition, the reduction of toxicity could increase adherence to treatment, so the analysis of efficacy in terms of 3) Thirdly, we calculated the number of responders who were needed to superiority should be also considered. This strategy allows us to better achieve a significant result at final analysis based on the NI hy­ weigh the risk-benefit ratio of this intervention compared with the pothesis. We assumed a NIM of 1.2. The number of patients recruited unique analysis of the safety profile. Our proposal explains how to in each stage and interim boundaries (a1, n1, and n) were the same pre-plan this study design to control type I error and avoid the post-hoc than previous superiority design. However, the number of re­ data review [6]. sponders to achieve a positive result in the final analysis were We designed a proof-of-concept study to evaluate if the new inves­ reduced: tigational therapy is promising, with less than 30 patients recruited. The sequence of the analysis at the end of study firstly assessed the safety (a1 / n1, a / n) = (4/ 14, 7 / 25) (12) profile, then the NI efficacy objectives, and finally, the superiority effi­ cacy objective. In accordance with fixed sequence method, the NI and The calculation of p-value and 95% confidence intervals were superiority analyses only could achieve a positive finding if previous implemented in the R “clinfun” or “OneArmPhaseTwoStudy” library assessment met the requirement for statistical significance [22]: [11,28]. In addition, three potential scenarios of analyses describe why our 1) Firstly, we assumed that the safety objective will be achieved at the proposal is more informative than the usual alternatives, as follows: end of the study. On the contrary, if the safety objective had not been reached, the efficacy analyzes would not make sense. • Scenario 1: 2) Secondly, we planned the statistical design for early efficacy objec­ tive based on two-stage Simon’s design for superiority analyses. We At the end of recruitment, a common issue is that clinical trials planned a 35% historical response rate in standard of care. This passed partially the pre-designed number of recruited patients. In response rate could range between 30% and 40% in accordance with accordance, our study achieved the interim objective to continue to the second stage (with at least 4 responders) and passed the target sample 4 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 size of two patients (N = 27) at final analysis. The total number of re­ the boundary will be achieved with a higher number of patients sponders observed at the end of study was 7 (x = 7). However, according recruited. However, if we based the test in the UMVUE method, the to the expected sample size, in the first 25 recruited patients only 6 analysis could be conducted with a different number of patients than responses were observed. Analyzing final data with the UMVUE based expected in the sample size [9]. Results based on the UMVUE method method allowed to analyze all the patients recruited in the study and estimated a 32.9% rate of responders. The p-values for the NI and su­ account the adaptive nature of the study due to the interim futility periority analysis were 0.085 (<0.1 significant criteria achieved) and analysis [9,27]. The results based on the UMVUE method estimated a 0.167 (>0.1 non-significant), respectively. As example: 32.2% rate of responders. The p-values for the NI and superiority analysis were 0.097 (<0.1 significant criteria achieved) and 0.194 (>0.1 NI analysis: twostage.inference(x, (a1-1), n1, n, p0/NIM) (23) non-significant), respectively. These calculations could be performed in twostage.inference(6, 3, 14, 20, 0.2/1.2) (24) open-access statistical libraries (packages “clinfun” or “OneArmPha­ seTwoStudy”, R statistical software) [11,28]: Superiority analysis: twostage.inference(x, (a1-1), n1, n, p0) (25) Package “clinfun”: twostage.inference(6, 3, 14, 20, 0.2) (26) NI analysis: twostage.inference(x, (a1-1), n1, n, p0/NIM) (13) Alternatively, the maximum likelihood method underestimated the twostage.inference(7, 3, 14, 27, 0.2/1.2) (14) rate of responders (30% < 32.9%) and did not declare neither significant differences in NI (p-value = 0.102) nor superiority analyses (p-value = Superiority analysis: twostage.inference(x, (a1-1), n1, n, p0) (15) 0.199). As example: twostage.inference(7, 3, 14, 27, 0.2) (16) NI analysis: binom.test(x = 6, n = 20, p = 0.2/1.2, alternative = ’greater’) (19) Package “OneArmPhaseTwoStudy”: Superiority analysis: binom.test(x = 6, n = 20, p = 0.2, alternative = ’greater’) (20) NI analysis: get_p_KC(x, (a1-1), n1, n, p0/NIM) (17) get_p_KC(7, 3, 14, 27, 0.2/1.2) (18) Moreover, regarding the scenario previously reported, a significant reduction in the rate of adverse events with a 32.9% response rate Superiority analysis: get_p_KC(x, (a1-1), n1, n, p0) (19) represented a promising result to design a future randomized trial. get_p_KC(7, 3, 14, 27, 0.2) (20) • Scenario 3: Alternatively, we could analyze results based on the pre-designed sample size boundaries. We considered the first 25 patients with only It could be argued that designing a clinical trial with a NI analysis is 6 responders (<7 responders, so the result was non-significant). More­ unnecessary because a more conservative null hypothesis (16.7%) could over we could conduct the analysis based on maximum-likelihood be assumed, and the same result should be obtained. However, consid­ method without including the interim decision in the analysis [9]. ering this alternative, the superiority objective could not be assessed, This method only considers the total number of responders (x = 7) and such as during a dose-finding phase I trial [33]. In this context, the study the total number of patients recruited (n = 27) (function “binom.test”, design of dose escalation trials usually includes patients who are allo­ package “stats”, R statistical software). We observed that the maximum cated in different cohorts and have received different dose levels in likelihood method underestimated the rate of responders (25.9% < order to identify the maximum tolerate dose (MTD). It is a common 32.3%) and did not declare significant differences neither in NI (p-value practice to expand the number of patients recruited in the MTD cohort to = 0.151) nor superiority analyses (p-value = 0.287). As example in R explore activity and extend the evaluation of safety results. The inclu­ code: sion of a NI hypothesis in the superiority-based analysis of this extended MTD cohort will help to rank the activity data with the other endpoints. NI analysis: binom.test(x = 7, n = 27, p = 0.2/1.2, alternative = ’greater’) (21) In accordance, we planned a single-stage, single-arm A’hern design [23]. We accepted the same constraints which were reported in the Superiority analysis: binom.test(x = 7, n = 27, p = 0.2, alternative = ’greater’) previous examples, except for type II error (20%): (22) (p0, p1, α, 1 ‒ β, NIM) = (0.20, 0.45, 0.1, 0.2, 1.2) (21) However, a significant reduction in the rate of clinically relevant adverse events with a 32.2% response rate was a promising result to Among 19 patients recruited, we will achieve a positive finding in the design a randomized trial. This result probably would have led to the NI and superiority analyses with 6 (31.5%) and 7 responders (36.8%), decision to develop the randomized clinical trial even if the efficacy NI respectively [23]. In addition to evaluate the clinical pharmacokinetic analysis was not formally considered. Whenever efficacy is ranked with profile of the drug, a phase I study offers the opportunity to test the safety, quality of life, cost or any other endpoints, efficacy is always prespecified pharmacodynamic and biomarker hypotheses that were evaluated in terms of NI among standard of care [25,29–31]. Never­ obtained in previous pre-clinical studies [34,35]. The aim of these phase theless, it was not formally defined and this post-hoc analyses usually I studies is to enrich the data obtained in early setting, that in turn will increase type I error [6]. allow the study design improvement of further randomized trials [35–37]. In accordance with our example, if data suggest both a good • Scenario 2: tolerability with a positive result in the NI analysis, a well-characterized pharmacokinetic profile, and a novel target in molecular pathway, the We would like to propose a scenario whose patients’ recruitment is randomized trial could be designed combining experimental therapy discontinued prior to reach the pre-designed sample size. The study with previous standard of care. Alternatively, the experimental therapy started the stage II, but it is finally discontinued with 6 responders and a will be evaluated as monotherapy in further randomized clinical trials if total of 20 patients recruited. Reasons for the study discontinuation were an equivalent toxic profile compared with previous example is obtained, not due to safety or efficacy results but to external issues (e.g. a logistic with a positive result in superiority analysis, and a similar molecular problem or a global pandemic) [32]. If we analyze these data based on target respect to standard of care. Finally, a vey toxic treatment with a the pre-designed sample size boundaries, we cannot conclude if the higher efficacy than standard of care, or a non-toxic treatment without a experimental therapy is promising or not. The number of responders was clinically meaningfully activity will not be tested in a new randomized lower than efficacy boundary (≥7 responders), but we cannot reject that 5 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 clinical trials if it had a negative risk-benefit ratio [38,39]. So, the iso­ authorities [53]. This suggests that the NI question is relevant in the lated assessment of activity, safety, pharmacokinetics, or pharmacody­ phase II setting and it is not so rare. However, the NI hypothesis was not namic data would have no sense. In order to combine these data in a planned, and it was only considered to deal with negative findings in single evaluation, it is important to grade qualitatively the intensity of proof-of-concept trials. This strategy leads to an increase in the proba­ clinical activity. Thus, analyzing both NI and superiority objectives is bility of type I error and the number of false positives [6]. In accordance, more informative than only consider one of them separately [40,41]. both systematic reviews reported and high rate of negative phase III trials after a negative phase II superiority trial (between 64% and 85%) 4. Discussion [36,53]. However, if the decision to include a NI analysis had been preplanned and included in the statistical design, investigators would Single-arm phase II trials are considered the first efficacy screenings have additional information about magnitude of response (non-inferior of new investigational agents in humans. Additionally, these trials are or superior) or other parameters without a type I error inflation. In important milestones towards testing safety and adapting the biomarker addition, the NI comparison with an historical control has probed its strategies validated in preclinical stages [34]. Although the NI question validity identifying subgroups of patients in adjuvant setting who can can be relevant in the phase II setting [5], it is not usually considered on avoid the toxic effects of chemotherapy [26]. designing single-arm clinical trials during early clinical development The most popular design for phase II cancer clinical trials is a single- [42,43]. Previous forays into precision medicine have shown that arm, two-stage Simon’s design. Numerous extensions have been pro­ identifying correctly both the target population and potential predictive posed for the Simon’s design, including randomized multi-arm trials biomarkers may be more critical for treatment success than simple designed to select the winner among the proposed therapeutics (Pick- demonstration of superior efficacy against an alternative [36,37,44]. the-winner study design) [54]. However, the inference procedures used The use of biomarker information in clinical trials has great potential for in two-stage designs are often not corrected to account for the adaptive efficiently identifying patients most helped by specific treatments [45, nature of these designs. A maximum likelihood estimator of the RR, the 46]. These biomarkers must have proven their clinical validity in pro­ number of positive responses/total number of patients, is biased. CI and spective, randomized trials with a superiority design in the enriched p-value should not be computed as if the data were obtained in a single population [47]. Consequently, designing a proof-of-concept phase II stage due to the possibility of early termination [10]. Different methods study that permits NI and superiority analyses will allow for more have been proposed to obtain a proper inference from the Simon’s informed decisions that rank the magnitude of clinical activity and other two-stage design [10]. The use of the UMVUE to estimate RR is rec­ parameters, such as safety, pharmacokinetic, and pharmacodynamic ommended as it addresses situations when the actual number of patients data [7]. As NI analysis is allowed, it could be assumed that this design recruited is equal to or different from preplanned values [27]. The facilitates that ineffective therapies were evaluated in a randomized calculation of RR, p-values and CIs has been incorporated in open-access study. However, this only could be true if we evaluated the treatment statistical libraries (packages “clinfun” and “OneArmPhaseTwoStudy”, exclusively based on the NI result, forgetting the other objectives. This R statistical software) based on previously published methods [11,28]. makes no sense, because any new drug development plan performs a We observed some differences between calculated (p-value) and risk-benefit assessment weighing all the endpoints [38]. In accordance, a simulated values (type I error). However, they were not relevant to the new therapy non-inferior to the standard of care with the same safety most common design constraints used in phase II single-arm trials profile, and biological target will not be much promising. On the con­ (0.01–0.1 type I errors and 0.1 to 0.2 type II errors). Using the UMVUE- trary, a new therapy with a good safety profile, that demonstrates to be based calculation method, we proved that the same level of agreement non-inferior to the standard of care would be a promising companion if between calculated and simulated values (type I and II errors) results it targets a different biological pathway compared with the standard from both two-stage Simon’s superiority designs and designs in which NI therapy [48]. analysis was allowed. Our findings suggest that the proposed method for Previous studies have proposed different strategies to evaluate analyzing NI in superiority-based designs did not introduce a bias under multiple endpoints (efficacy and toxicity) in phase I and II single arm the most usual constrains for alpha and beta errors. So, our results trials. EFFTox, Gumbel model, or continual reassessment method ad­ recommend that single-arm study designs can be planned to attain the aptations for Bayesian perspective, or single-stage binomial trials and usual levels of statistical error (alpha and beta errors equal or lower than Simon two-stage adaptations for frequentist paradigm. However, these 0.2). In study designs with error rates which are higher than usual levels, designs have been planned to evaluate the superiority of the new drug it would be necessary to simulate alpha and beta errors to confirm that activity among a theorical or historical reference. They not considered to statistical errors are attained at desired level. compare non-inferiority and superiority hypothesis to grade the The major limitations of this method are based on bringing together magnitude of clinical activity in the early clinical stages [49–52]. the inherent complexities of a study with historical controls and a NI A prior systematic review, which aimed to evaluate the character­ analysis. However, these limitations are quite common in comparative istics of phase II trials that best predict for phase III outcome, selected study designs of NI, because NIM must be established based on historical 270 single-arm phase II studies between 1981 and 2012. All these controls evidence [6]. Additionally, some bias (e.g. as selection of selected studies led to a phase III clinical trial. The meta-analysis showed inappropriate patients, poor compliance and insufficient follow-up, that that 168 single-arm trials were not positive, while 61 (36.3%) achieved a can lead to erroneously conclude that a treatment is not inferior to positive phase III results despite not having obtained a positive result in placebo in a comparative study) go against a positive achievement when the proof-of-concept study [36]. Additionally, Jardin DL and colleagues the comparison is done among a theoretical rate of efficacy deduced reported that 10% of FDA anti-cancer drugs approvals between January from historical controls. As comparative designs of NI, to declare a 2009 and June 2014 obtained a negative result in prior phase II clinical therapy as non-inferior in a single-arm trial, we need to demonstrate trials [53]. In a number of these studies, despite the low response rates, assay sensitivity based on an adequate trial design and conduct [12]. In clinical activity was considered not worse than conventional treatments accordance, lower sample sizes in phase II single-arm trials are not more and allowing to continue with Phase III trials according to other pa­ challenging for NI analyses respect to superiority ones if trial is properly rameters, such as the prolonged duration of clinical benefit or the pos­ preplanned and conducted. Our results suggest that we can conserve the itive safety profile [25,29–31]. Overall, these data demonstrate that same levels of alpha and beta errors after including the NI analysis combining the superiority and NI analyses is useful to rank the intensity without increase sample size. of clinical activity with other relevant parameters. This strategy of Furthermore, the NI analysis with the UMVUE-based calculation analysis of phase II trials has enriched the design of successful phase III method may be extended to two-stage designs with both futility and leading to the marketing authorization of the product from regulatory superiority boundaries (as outlined in the Supplementary Methods) 6 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 [55]. We limited our results to two-stage designs that are most popularly References for phase II cancer clinical trials, but the methods discussed in this article could be extended to phase II trials with any number of stages [9]. [1] M. Quintela-Fandino, A. Lluch, L. Manso, I. Calvo, J. Cortes, J.A. García-Saenz, M. Gil-Gil, N. Martinez-Jánez, A. Gonzalez-Martin, E. Adrover, R. de Andres, Likewise, we can design a single-arm time-to-event study with a NI G. Viñas, A. Llombart-Cussac, E. Alba, J. Guerra, B. Bermejo, E. Zamora, F. Moreno- hypothesis based on the exponential maximum likelihood estimator, Anton, S. Pernas Simon, A. Carrato, A. Lopez-Alonso, M.J. Escudero, R. Campo, one-sample log-rank tests or other approximations to the Kaplan-Meyer E. Carrasco, J. Palacios, F. Mulero, R. Colomer, 18F-fluoromisonidazole PET and activity of neoadjuvant Nintedanib in early HER2-negative breast cancer: a estimations [55–57]. If we assume the same number of patients as in window-of-opportunity randomized trial, Clin. Canc. Res. 23 (2017) 1432–1441, superiority analysis, we would formulate the NI rejection hazard rate https://doi.org/10.1158/1078-0432.CCR-16-0738. (λ0ni) from the hazard rate assumed under H0 in the superiority analysis [2] A. Llombart-Cussac, J. Cortés, L. Paré, P. Galván, B. Bermejo, N. Martínez, M. Vidal, S. Pernas, R. López, M. Muñoz, P. Nuciforo, S. Morales, M. Oliveira, L. de (λ0) and the NIM estimated from historical studies [6] (λ0NI = λ0/NIM). la Peña, A. Peláez, A. Prat, HER2-enriched subtype as a predictor of pathological Including a NI analysis in superiority-based clinical trials, may be complete response following trastuzumab and lapatinib without chemotherapy in useful for weighing additional factors such as clinical benefit duration, early-stage HER2-positive breast cancer (PAMELA): an open-label, single-group, multicentre, phase 2 trial, Lancet Oncol. 18 (2017) 545–554, https://doi.org/ safety, cost, or biomarker strategy while assessing activity and early 10.1016/S1470-2045(17)30021-9. efficacy rate. The results of previous single-arm designs leading to a [3] L. Rubinstein, M. Leblanc, M.A. Smith, More randomization in phase II trials: successful phase III trials or identifying subgroups of patients who can necessary but not sufficient, J. Natl. Cancer Inst. 103 (2011) 1075–1077, https:// avoid the toxic effects of chemotherapy suggest that NI question is doi.org/10.1093/jnci/djr238. [4] M. Sampayo-Cordero, B. Miguel-Huguet, A. Pardo-Mateos, A. Malfettone, J. Pérez- relevant in non-comparative studies. Implementation of this strategy can García, A. Llombart-Cussac, J. Cortés, M. Moltó-Abad, C. Muñoz-Delgado, be achieved through simple adaptations to the Simon’s two-stage design M. Pérez-Quintana, J. Pérez-López, Agreement between results of meta-analyses and other existing designs for one-arm phase II clinical trials. from case reports and clinical studies, regarding efficacy and safety of idursulfase therapy in patients with mucopolysaccharidosis type II (MPS-II). A new tool for evidence-based medicine in rare diseases, Orphanet J. Rare Dis. 14 (2019), https:// Declaration of competing interest doi.org/10.1186/s13023-019-1202-6. [5] B. Neuenschwander, N. Rouyrre, N. Hollaender, E. Zuber, M. Branson, A proof of concept phase II non-inferiority criterion, Stat. Med. 30 (2011) 1618–1627, We wish to draw the attention of the Editor to the following facts https://doi.org/10.1002/sim.3997. which may be considered as potential conflicts of interest and to sig­ [6] FDA, FDA GUIDANCE: Non-inferiority Clinical Trials to Establish Effectiveness, nificant financial contributions to this work: 2016. https://www.fda.gov/downloads/Drugs/Guidances/UCM202140.pdf. [7] Committee for Proprietary Medicinal Products (CPMP), the European Agency for Miguel Sampayo-Cordero reports personal fees from Hospital Vall the Evaluation of Medicinal Products, Points to Consider on Switching between d’Hebron, Roche, Nestle Health Science, Laboratorios Leti, Medica Sci­ Superiority and Non-inferiority. http://Www.Ema.Europa.Eu/Docs/En_GB/Doc entia Innovation Research (MedSIR), Syntax for Science, Ability Pharma ument_library/Scientific_guideline/2009/09/WC500003658.Pdf, 2000. [8] S. Tanaka, Y. Kinjo, Y. Kataoka, K. Yoshimura, S. Teramukai, Statistical issues and and Scienco Klinico, outside the submitted work. recommendations for noninferiority trials in oncology: a systematic review, Clin. Bernat Miguel-Huguet declares no conflict of interest. Canc. Res. 18 (2012) 1837–1847, https://doi.org/10.1158/1078-0432.CCR-11- José Pérez-García has received consulting and advisor fees from: 1653. Roche and Eli Lilly. [9] S.-H. Jung, Statistical issues for design and analysis of single-arm multi-stage phase II cancer clinical trials, Contemp. Clin. Trials 42 (2015) 9–17, https://doi.org/ David Páez declares a scientific advisory role for Amgen, Sanofi, 10.1016/j.cct.2015.02.007. Merck Serono, F. Hoffmann-La Roche Ltd, Lilly and Servier. [10] T. Koyama, H. Chen, Proper inference from Simon’s two-stage designs, Stat. Med. Ángel L. Guerrero-Zotano declares no conflict of interest. 27 (2008) 3145–3154, https://doi.org/10.1002/sim.3123. [11] S.-H. Jung, K. Owzar, S.L. George, T. Lee, P-value calculation for multistage phase Javier Garde Noguera declares no conflict of interest. II cancer clinical trials, J. Biopharm. Stat. 16 (2006) 765–775, https://doi.org/ Elena Aguirre has received consulting and advisor fees from Pfizer 10.1080/10543400600825645. ; discussion 777-783. and honorarias from: Roche, Novartis, Celgene, Eisai and Pfizer. [12] Choice of control group and related issues in clinical trials : ICH E10 (n.d), http://www.ich.org/products/guidelines/efficacy/efficacy-single/article/choice- Esther Holgado declares no conflict of interest. of-control-group-and-related-issues-in-clinical-trials.html. (Accessed 15 October Elena López-Miranda declares no conflict of interest. 2018). Xin Huang is a statistician at Pfizer Oncology. [13] R.C. Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2016. https://www. Antonio Llombart-Cussac has received consulting and advisor fees R-project.org/.2016. from Roche, GlaxoSmithKline, Novartis, Celgene, Eisai, and AstraZeneca [14] Venkatraman E. Seshan, Package “Clinfun” - Clinical Trial Design and Data and has stock options, patents and intellectual property from MedSIR. Analysis Functions. Version 1.0.14. https://Cran.r-Project.Org/Web/Packages/Cl infun/Clinfun.Pdf, 2017. Andrea Malfettone declares no conflict of interest. [15] William Oberle, Weapons and Materials Research Directorate, ARL. Monte Carlo Javier Cortés has received consulting and advisor fees from: Roche, Simulations: Number of Iterations and Accuracy, Available from: Celgene, Cellestia, AstraZeneca, Biothera Pharmaceutical, Merus, Seat­ file:///Users/becario1/Downloads/ADA621501%20(1).pdf, 2015. tle Genetics, Daiichi Sankyo and Erytech. In addition, Javier Cortés has [16] H. Potte, Critical review of method comparison studies for the evaluation of estimating Glomerular filtration rate Equations| Nephrology and kidney failure received honorarias from: Roche, Novartis, Celgene, Eisai, Pfizer and Journal|SciForschen (n.d.), https://www.sciforschenonline.org/journals/nephrolo Samsung. Add more, Javier Cortés has received research funding fees to gy-kidney/IJNKF-1-102.php. (Accessed 15 October 2018). the institution from Roche. Finally, Javier Cortés has stock options, [17] J. Ludbrook, Confidence in Altman-Bland plots: a critical review of the method of differences, Clin. Exp. Pharmacol. Physiol. 37 (2010) 143–149, https://doi.org/ patents and intellectual property from MedSIR. 10.1111/j.1440-1681.2009.05288.x. [18] Sin-Ho Jung, Randomized phase II cancer clinical trials, in: Chapman and Hall/ Aknowledgements CRC, 2013 (n.d.). [19] H.S. Rugo, L. Seneviratne, J.T. Beck, J.A. Glaspy, J.A. Peguero, T.J. Pluard, N. Dhillon, L.C. Hwang, C. Nangia, I.A. Mayer, T.F. Meiller, M.S. Chambers, R. We would like to acknowledge the reviewers for their valuable W. Sweetman, J.R. Sabo, J.K. Litton, Prevention of everolimus-related stomatitis in comments and contributions to this paper. women with hormone receptor-positive, HER2-negative metastatic breast cancer using dexamethasone mouthwash (SWISH): a single-arm, phase 2 trial, Lancet Oncol. 18 (2017) 654–662, https://doi.org/10.1016/S1470-2045(17)30109-2. Appendix A. Supplementary data [20] H.M. Earl, L. Hiller, A.-L. Vallier, S. Loi, K. McAdam, L. Hughes-Davies, A. N. Harnett, M.-L. Ah-See, R. Simcock, D. Rea, S. Raj, P. Woodings, M. Harries, D. Howe, K. Raynes, H.B. Higgins, M. Wilcox, C. Plummer, J. Mansi, I. Gounaris, Supplementary data to this article can be found online at https://doi. B. Mahler–Araujo, E. Provenzano, A. Chhabra, J.E. Abraham, C. Caldas, P.S. Hall, org/10.1016/j.conctc.2020.100678. C. McCabe, C. Hulme, D. Miles, A.M. Wardley, D.A. Cameron, J.A. Dunn, R. Agarwal, H. Algurafi, R. Allerton, C. Archer, A. Armstrong, C. Bale, L. Barraclough, U. Barthakur, C. Bedi, K. Benstead, D. Bloomfield, R. Bowen, C. Bradley, J. Brown, M. Butt, M. Churn, S. Cleator, J. Cliff, P. Crellin, M. Daly, S. De Silva-Minor, A. Dhadda, O. Din, S. Down, H. Earl, D. Eaton, A. Eichholz, D. Epurescu, C. Goh, A. Goodman, R. Grieve, M. Hadaki, C. Harper-Wynne, 7 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 M. Harries, L. Hayward, A. Humphreys, H. Innes, M. Jafri, A. Jegannathen, [26] F. Cardoso, L.J. van’t Veer, J. Bogaerts, L. Slaets, G. Viale, S. Delaloge, J.-Y. Pierga, M. Kelleher, H. Kristeleit, D. Lee, S. Lupton, C. MacGregor, Z. Malik, J. Mansi, E. Brain, S. Causeret, M. DeLorenzi, A.M. Glas, V. Golfinopoulos, T. Goulioti, J. Marshall, K. McAdam, T. McGolick, R. Mehra, D. Miles, N. Mithal, C. Moss, S. Knox, E. Matos, B. Meulemans, P.A. Neijenhuis, U. Nitz, R. Passalacqua, A. Moss, M. Mukesh, A. Neal, D. Nelmes, H. Neville-Webbe, J. Newby, S. O’Reilly, P. Ravdin, I.T. Rubio, M. Saghatchian, T.J. Smilde, C. Sotiriou, L. Stork, C. Straehle, P. Ostler, M. Persic, L. Pettit, S. Raj, F. Raja, D. Rea, C. Reed, A. Rigg, H. Roe, G. Thomas, A.M. Thompson, J.M. van der Hoeven, P. Vuylsteke, R. Bernards, N. Shah, P. Simmonds, E. Sims, S. Smith, N. Storey, W. Taylor, N. Thanvi, K. Tryfonidis, E. Rutgers, M. Piccart, 70-Gene signature as an aid to treatment K. Tipples, J. Vaidya, M. Varughese, A. Vinayan, N. Walji, S. Waters, P. Woodings, decisions in early-stage breast cancer, N. Engl. J. Med. 375 (2016) 717–729, K. Wright, S. Yahya, 6 versus 12 months of adjuvant trastuzumab for HER2-positive https://doi.org/10.1056/NEJMoa1602253. early breast cancer (PERSEPHONE): 4-year disease-free survival results of a [27] R. Porcher, K. Desseaux, What inference for two-stage phase II trials? BMC Med. randomised phase 3 non-inferiority trial, Lancet 393 (2019) 2599–2612, https:// Res. Methodol. 12 (2012) 117, https://doi.org/10.1186/1471-2288-12-117. doi.org/10.1016/S0140-6736(19)30650-6. [28] S.-H. Jung, Statistical issues for design and analysis of single-arm multi-stage phase [21] X. Pivot, G. Romieu, M. Debled, J.-Y. Pierga, P. Kerbrat, T. Bachelot, A. Lortholary, II cancer clinical trials, Contemp. Clin. Trials 42 (2015) 9–17, https://doi.org/ M. Espié, P. Fumoleau, D. Serin, J.-P. Jacquin, C. Jouannaud, M. Rios, S. Abadie- 10.1016/j.cct.2015.02.007. Lacourtoisie, L. Venat-Bouvet, L. Cany, S. Catala, D. Khayat, L. Gambotti, [29] L.T. Vahdat, B. Pruitt, C.J. Fabian, R.R. Rivera, D.A. Smith, E. Tan-Chiu, J. Wright, I. Pauporté, C. Faure-Mercier, S. Paget-Bailly, J. Henriques, J.M. Grouin, C. Piprot, A.R. Tan, N.A. Dacosta, E. Chuang, J. Smith, J. O’Shaughnessy, D.E. Shuster, N. L. Cals, L. Chaigneau, F. Demarchi, T. N’Guyen, U. Stein, C. Villanueva, J. Bréau, L. Meneses, K. Chandrawansa, F. Fang, P.E. Cole, S. Ashworth, J.L. Blum, Phase II A. Chouahnia, P. Saintigny, F. Boué, P. deSaint-Hilaire, I. Guimont, N. Grossat, study of eribulin mesylate, a halichondrin B analog, in patients with metastatic B. Valenza, E. Lévy, J. Médioni, C. Delbaldo, J. Grenier, D. Pouessel, S. Lavau- breast cancer previously treated with an anthracycline and a taxane, J. Clin. Oncol. Denès, C. Falandry, C. Fournel-Fédérico, G. Freyer, S. Tartas, V. Trillet-Lenoir, 27 (2009) 2954–2961, https://doi.org/10.1200/JCO.2008.17.7618. F. Bons, G. Auclerc, S. Chièze, N. Raban, C. Tournigand, S. Trager-Maury, [30] J. Cortes, L. Vahdat, J.L. Blum, C. Twelves, M. Campone, H. Roché, T. Bachelot, G. Bousquet, C. Cuvier, S. Giacchetti, A. Hocini, C. LeMaignan, J. Misset, A. Awada, R. Paridaens, A. Goncalves, D.E. Shuster, J. Wanders, F. Fang, D. Avenin, C. Beerblock, J. Gligorov, P. Rivera, H. Roché, P. Bougnoux, N. Hajjaji, R. Gurnani, E. Richmond, P.E. Cole, S. Ashworth, M.A. Allison, Phase II study of the O. Capitain, R. Delva, P. Maillart, P. Soulié, H. Bonnefoi, M. Durand, N. Madranges, halichondrin B analog eribulin mesylate in patients with locally advanced or L. Mauriac, P. Chollet, A. Dillies, X. Durando, J. Ferrière, C. Mouret-Reynier, metastatic breast cancer previously treated with an anthracycline, a taxane, and J. Nabholtz, I. Van Praagh, P. Cottu, V. Diéras, A. Durieux, M. Galotte, V. Girre, capecitabine, J. Clin. Oncol. 28 (2010) 3922–3928, https://doi.org/10.1200/ S. Henry, I. Iurisci, M. Jouve, V. Laurence, L. Mignot, S. Piperno-Neumann, JCO.2009.25.8467. P. Tresca, B. Coudert, E. Ferrant, F. Mayer, A. Vanneuville, J. Bonneterre, [31] J. Cortes, J. O’Shaughnessy, D. Loesch, J.L. Blum, L.T. Vahdat, K. Petrakova, V. Servent, L. Vanlemmens, P. Vennin, J. Guastalla, P. Biron, L. Dupuy-Brousseau, P. Chollet, A. Manikas, V. Diéras, T. Delozier, V. Vladimirov, F. Cardoso, H. Koh, L. Lancry, I. Ray-Coquard, P. Rebattu, O. Trédan, J. Extra, F. Rousseau, C. Tarpin, P. Bougnoux, C.E. Dutcus, S. Seegobin, D. Mir, N. Meneses, J. Wanders, C. Twelves, M. Fabbro, E. Luporsi, L. Uwer, B. Weber, D. Berton-Rigaud, E. Bourbouloux, EMBRACE (Eisai Metastatic Breast Cancer Study Assessing Physician’s Choice M. Campone, J. Ferrero, P. Follana, R. Largillier, V. Mari, B. Costa, H. Curé, versus E7389) investigators, Eribulin monotherapy versus treatment of physician’s J. Eymard, N. Jovenin, D. Lebrun, J. Meunier, G. Yazbek, D. Gedoin, B. Laguerre, choice in patients with metastatic breast cancer (EMBRACE): a phase 3 open-label C. Lefeuvre, E. Vauléon, A. Chevrier, C. Guillemet, M. Leheurteur, O. Rigal, randomised study, Lancet 377 (2011) 914–923, https://doi.org/10.1016/S0140- I. Tennevet, C. Veyret, E. Brain, M. Guiterrez, F. Mefti-Lacheraf, T. Petit, F. Dalenc, 6736(11)60070-6. L. Gladieff, H. Roché, F. André, S. Delaloge, J. Domont, J. Ezenfis, M. Spielmann, [32] WHO Rolling updates on coronavirus disease (COVID-19), WHO characterizes P. Guillet, V. Boulanger, J. Provençal, L. Stefani, C. Alliot, D. Ré, C. Bellaiche- COVID-19 as a pandemic (n.d.), https://www.who.int/emergencies/disease Miccio, G. Boutan-Laroze, R. Vanica, P. Dion, A. Hocini, G. Sadki-Benaoudia, s/novel-coronavirus-2019/events-as-they-happen. (Accessed 30 March 2020). A. Marti, A. Villing, B. Slama, J. Dutel, S. Nguyen, R. Saad, O. Arsène, Z. Merad- [33] H. Pan, R. Lin, Y. Zhou, Y. Yuan, Keyboard design for phase I drug-combination Boudia, H. Orfeuvre, J. Egreteau, M. Goudier, R. Lamy, B. Leduc, C. Sarda, trials, Contemp. Clin. Trials 92 (2020) 105972, https://doi.org/10.1016/j. B. Salles, C. Agostini, I. Cauvin, A. Dufresne, M. Mangold, S. Lebouvier-Sadot, cct.2020.105972. B. Audhuy, J. Barats, S. Cluet-Dennetière, D. Zylberait, G. Netter, L. Gautier- [34] I. Romero, M.J. Rubio, M. Medina, R. Serrano, E.M. Guerra, A.C. Salgado, B. Pérez, Felizot, I. Cojean-Zelek, A. Plantade, S. Vignot, E. Guardiola, P. Marti, L. Minig, A. Casado, P. Coronado, J.F. Cueva, A. Vilar, A. Torné, J. Ordi, S. Cros, I. deHartingh, R. Diab, A. Dietmann, S. Ruck, C. Portois, E. Guardiola, S. Oddou- G. Salinas, M. Santacana, J.A. Schoenenberger-Arnaiz, A. Llombart-Cussac, Lagranière, F. Campos-Gazeau, A. Bourcier, F. Priou, J. Geay, D. Mayeur, P. Gabez, A. Poveda, Preoperative olaparib in early-stage endometrial cancer (EC): a phase 0, R. ElAmarti, M. Combe, J. Ezenfis, P. Raichon-Patru, P. Amsalhem, J. Dauba, window of opportunity trial to evaluate the PARP inhibition effect, targeting cell D. Paraiso, F. Guinet, B. Duvert, M. Litor, F. Kara-Slimane, A. Bichoffe, N. Denizon, cycle-related proteins (POLEN study), J. Clin. Orthod. 36 (2018), https://doi.org/ J. Meunier, P. Soyer, F. Morvan, S. Van-Hulst, L. Vincent, C. Alleaume, P. Ibanez- 10.1200/JCO.2018.36.15_suppl.5598, 5598–5598. Martin, A. Youssef, Z. Tadrist, E. Carola, C. Pourny, J. Toccanier, N. Al-Aukla, [35] M. Arnedos, M. Roulleaux Dugage, J. Perez-Garcia, J. Cortes, Window of K. Mahour-Bacha, J. Salvat, L. Cals, P. Nouyrigat, S. Clippe, M. Gouttebel, Opportunity trials for biomarker discovery in breast cancer. Curr. Opin. Oncol 31 L. Vedrine, G. Clavreul, O. Collard, D. Mille, Y. Goubely, J. Grenier, R. Hervé, (2019) 486–492, https://doi.org/10.1097/CCO.0000000000000583. S. Kirscher, F. Plat, V. Delecroix, V. Ligeza-Poisson, D. Coeffic, L. Dupuy-Brousseau, [36] J.G. Monzon, A.E. Hay, G.T. McDonald, J.L. Pater, R.M. Meyer, E. Chen, B.E. Chen, D. Fric, C. Garnier, C. Leyronnas, T. Kreitman, R. Largillier, E. Teissier, P. Martin, J.E. Dancey, Correlation of single arm versus randomised phase 2 oncology trial S. Rohart deCordoue, C. ElKouri, J. Ramée, C. Laporte, O. Bernard, T. Altwegg, characteristics with phase 3 outcome, Eur. J. Canc. 51 (2015) 2501–2507, https:// A. Darut-Jouve, J. Dujols, F. Darloy, C. Giraud, V. Pottier-Kyndt, N. Achour, doi.org/10.1016/j.ejca.2015.08.004. S. Drony, M. Moriceau, C. Sarrazin, J. Legueul, J. Mandet, D. Besson, A. Hardy- [37] M. Schwaederle, M. Zhao, J.J. Lee, A.M. Eggermont, R.L. Schilsky, J. Mendelsohn, Bessard, J. Cretin, P. Houyau, E. Achille, D. Genêt, H. Thévenot, A. Moran-Ribon, V. Lazar, R. Kurzrock, Impact of precision medicine in diverse cancers: a meta- J. Pavlovitch, P. Ardisson, I. Moullet, B. Couderc, V. Fichet, F. Burki, A. Auliard, analysis of phase II clinical trials, J. Clin. Orthod. 33 (2015) 3817–3825, https:// C. Levaché, G. Auclerc, P. Cailleux, F. Schaeffer, N. Albin, D. Sévin-Robiche, doi.org/10.1200/JCO.2015.61.5997. J. Domas, S. Ellis, P. Montcuquet, G. Baumont, M. Bégue, S. Gréget, J. Ratoanina, [38] B. Ratitch, N. Goel, C. Mallinckrodt, J. Bell, J.W. Bartlett, G. Molenberghs, P. Singh, A. Vanoli, C. Bielsa, M. Bonichon-Lamichhane, D. Jaubert, H. Laharie-Mineur, I. Lipkovich, M. O’Kelly, Defining efficacy estimands in clinical trials: examples L. Alcaraz, J. Cretin, E. Legouffe, H. Bourgeois, G. Cartron, F. Denis, O. Dupuis, illustrating ICH E9(R1) guidelines, Ther Innov Regul Sci 54 (2020) 370–384, G. Ganem, S. Roche-Forestier, L. Delzenne, E. Chirat, J. Baticle, E. Béguier, https://doi.org/10.1007/s43441-019-00065-7. S. Jacquot, E. Janssen, H. Lauché, A. LeRol, J. Chantelard, G. L’Helgoualc’h, [39] International conference on harmonisation; guidance on statistical principles for E. Antoine, A. Kanoui, J. Llory, J. Vannetzel, J. Vignoud, C. Bruna, T. Facchini, clinical trials; availability–FDA. Notice, Fed. Regist. 63 (1998) 49583–49598. K. Moutel-Corviole, A. Voloch, A. Ghoul, D. Loiseau, K. Mahour-Bacha, N. Barbet, [40] EMEA, Points to Consider on Switching between Superiority and Non-inferiority, N. Dohollou, K. Yakendji, 6 months versus 12 months of adjuvant trastuzumab in 2000. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guid early breast cancer (PHARE): final analysis of a multicentre, open-label, phase 3 eline/2009/09/WC500003658.pdf. randomised trial, Lancet 393 (2019) 2591–2598, https://doi.org/10.1016/S0140- [41] G.D. Murray, Switching between superiority and non-inferiority, Br. J. Clin. 6736(19)30653-1. Pharmacol. 52 (2001), 219–219. [22] U.S. Department of Health and Human Services Food and Drug Administration, [42] L. Mauri, R.B. D’Agostino, Challenges in the design and interpretation of Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation noninferiority trials, N. Engl. J. Med. 377 (2017) 1357–1367, https://doi.org/ and Research (CBER), Multiple Endpoints in Clinical Trials - Guidance for Industry. 10.1056/NEJMra1510063. https://Www.Fda.Gov/Downloads/Drugs/Guidancecomplianceregulatoryinforma [43] T.A. Althunian, A. de Boer, O.H. Klungel, W.N. Insani, R.H.H. Groenwold, Methods tion/Guidances/Ucm536750.Pdf, 2017. of defining the non-inferiority margin in randomized, double-blind controlled [23] R.P. A’Hern, Sample size tables for exact single-stage phase II designs, Stat. Med. trials: a systematic review, Trials 18 (2017), https://doi.org/10.1186/s13063-017- 20 (2001) 859–866, https://doi.org/10.1002/sim.721. 1859-x. [24] A. Llombart-Cussac, X. Pivot, L. Biganzoli, H. Cortes-Funes, K.I. Pritchard, J.- [44] M. Sampayo-Cordero, B. Miguel-Huguet, A. Pardo-Mateos, M. Moltó-Abad, Y. Pierga, I. Smith, C. Thomssen, S. Srock, M. Sampayo, J. Cortes, A prognostic C. Muñoz-Delgado, J. Pérez-López, Agreement between the results of meta- factor index for overall survival in patients receiving first-line chemotherapy for analyses from case reports and from clinical studies regarding the efficacy of HER2-negative advanced breast cancer: an analysis of the ATHENA trial, Breast 23 laronidase therapy in patients with mucopolysaccharidosis type I who initiated (2014) 656–662, https://doi.org/10.1016/j.breast.2014.06.017. enzyme replacement therapy in adult age: an example of case reports meta- [25] P.A. Tang, S.J. Cohen, C. Kollmannsberger, G. Bjarnason, K. Virik, M.J. MacKenzie, analyses as an useful tool for evidence-based medicine in rare diseases, Mol. Genet. L. Lourenco, L. Wang, A. Chen, M.J. Moore, Phase II clinical and pharmacokinetic Metabol. 123 (2) (2018) 69–75, https://doi.org/10.1016/j.ymgme.2018.01.002. study of aflibercept in patients with previously treated metastatic colorectal [45] MONARCH 2, Abemaciclib in combination with fulvestrant in women with HR+/ cancer, Clin. Canc. Res. 18 (2012) 6023–6031, https://doi.org/10.1158/1078- HER2− advanced breast cancer who had progressed while receiving endocrine 0432.CCR-11-3252. 8 M. Sampayo-Cordero et al. Contemporary Clinical Trials Communications 20 (2020) 100678 therapy, J. Clin. Oncol.: Vol 35, No 25, (n.d.), http://ascopubs.org/doi/full/10.12 M. Sampayo, E. Aguirre, J.M. Pérez-García, J. Cortés, A. Llombart-Cussac, Phase II, 00/JCO.2017.73.7585. (Accessed 15 October 2018). multicenter, single-arm trial of eribulin as first-line therapy for patients with [46] J.M. Perez-Garcia, C. Saura, E. Muñoz, D. Moreno-Fernandez, P. Gomez, M. Vidal, aggressive taxane-pretreated HER2-negative metastatic breast cancer: the S. Di Cosimo, J. Cortes, J. Baselga, M. Bellet, Role of progesterone receptor status MERIBEL study, Clin. Breast Canc. 19 (2019) 105–112, https://doi.org/10.1016/j. (PR) as predictive factor of pathologic complete response (pCR) to neoadjuvant clbc.2018.12.012. chemotherapy (NACT) in breast cancer patients, J. Clin. Orthod. 28 (2010), [51] S.M. Berry (Ed.), Bayesian Adaptive Methods for Clinical Trials, CRC Press, Boca https://doi.org/10.1200/jco.2010.28.15_suppl.628, 628–628. Raton, 2011. [47] J. Baselga, J. Cortés, S.-A. Im, E. Clark, G. Ross, A. Kiermaier, S.M. Swain, [52] W. Zhong, J.S. Koopmeiners, B.P. Carlin, A trivariate continual reassessment Biomarker analyses in CLEOPATRA: a phase III, placebo-controlled study of method for phase I/II trials of toxicity, efficacy, and surrogate efficacy, Stat. Med. pertuzumab in human epidermal growth factor receptor 2–positive, first-line 31 (2012) 3885–3895, https://doi.org/10.1002/sim.5477. metastatic breast cancer, J. Clin. Orthod. 32 (2014) 3753–3761, https://doi.org/ [53] D.L. Jardim, E.S. Groves, P.P. Breitfeld, R. Kurzrock, Factors associated with failure 10.1200/JCO.2013.54.5384. of oncology drugs in late-stage clinical development: a systematic review, Canc. [48] H.S. Rugo, R.S. Finn, V. Diéras, J. Ettl, O. Lipatov, A.A. Joy, N. Harbeck, Treat Rev. 52 (2017) 12–21, https://doi.org/10.1016/j.ctrv.2016.10.009. A. Castrellon, S. Iyer, D.R. Lu, A. Mori, E.R. Gauthier, C.H. Bartlett, K.A. Gelmon, D. [54] S.-H. Jung, S.L. George, Between-arm comparisons in randomized phase II trials, J. Slamon, Palbociclib plus letrozole as first-line therapy in estrogen receptor- J. Biopharm. Stat. 19 (2009) 456–468, https://doi.org/10.1080/ positive/human epidermal growth factor receptor 2-negative advanced breast 10543400902802391. cancer with extended follow-up, Breast Canc. Res. Treat. 174 (2019) 719–729, [55] Sin-Ho Jung, Randomized phase II cancer clinical trials, in: Chapman and Hall/ https://doi.org/10.1007/s10549-018-05125-4. CRC, 2013. [49] P.F. Thall, S.-C. Cheng, Optimal two-stage designs for clinical trials based on safety [56] Sample Size Calculator, Kengo Nagashima - The Institute of Statistical Mathematics and efficacy, Stat. Med. 20 (2001) 1023–1032, https://doi.org/10.1002/sim.717. (n.d.), https://nshi.jp/en/js/. (Accessed 18 June 2019). [50] V. Ortega, A. Antón, I. Garau, N. Afonso, L. Calvo, Y. Fernández, M. Martínez- [57] J. Wu, L. Chen, J. Wei, H. Weiss, A. Chauhan, Two-stage phase II survival trial García, E. Blanco, P. Zamora, M. García, J.J. Illarramendi, C.A. Rodríguez Sánchez, design, Pharmaceut. Stat. 19 (2020) 214–229, https://doi.org/10.1002/pst.1983. 9