www.nature.com/bjc ARTICLE Clinical Study Evaluating radiological response in pancreatic neuroendocrine tumours treated with sunitinib: comparison of Choi versus RECIST criteria (CRIPNET_ GETNE1504 study) Mª Pilar Solis-Hernandez1, Ana Fernandez del Valle2, Alberto Carmona-Bayonas3, Rocio Garcia-Carbonero4, Ana Custodio5, Marta Benavent6, Teresa Alonso Gordoa7, Bárbara Nuñez-Valdovino8, Manuel Sanchez Canovas9, Ignacio Matos10, Vicente Alonso11, Carlos Lopez12, Antonio Viudez13, Marta Izquierdo1, David Calvo-Temprano2, Enrique Grande14, Jaume Capdevila10 and Paula Jimenez-Fonseca1 BACKGROUND: The purpose of our study was to analyse the usefulness of Choi criteria versus RECIST in patients with pancreatic neuroendocrine tumours (PanNETs) treated with sunitinib. METHOD: A multicentre, prospective study was conducted in 10 Spanish centres. Computed tomographies, at least every 6 months, were centrally evaluated until tumour progression. RESULTS: One hundred and seven patients were included. Median progression-free survival (PFS) by RECIST and Choi were 11.42 (95% confidence interval [CI], 9.7–15.9) and 15.8 months (95% CI, 13.9–25.7). PFS by Choi (Kendall’s τ = 0.72) exhibited greater correlation with overall survival (OS) than PFS by RECIST (Kendall’s τ = 0.43). RECIST incorrectly estimated prognosis in 49.6%. Partial response rate increased from 12.8% to 47.4% with Choi criteria. Twenty-four percent of patients with progressive disease according to Choi had stable disease as per RECIST, overestimating treatment effect. Choi criteria predicted PFS/OS. Changes in attenuation occurred early and accounted for 21% of the variations in tumour volume. Attenuation and tumour growth rate (TGR) were associated with improved survival. CONCLUSION: Choi criteria were able to capture sunitinib’s activity in a clinically significant manner better than RECIST; their implementation in standard clinical practice shall be strongly considered in PanNET patients treated with this drug. British Journal of Cancer (2019) 121:537–544; https://doi.org/10.1038/s41416-019-0558-7 BACKGROUND (MKI) sunitinib, which targets VEGFR 1, 2, and 3, as well as PDGFR, Pancreatic neuroendocrine tumours (PanNETs) are a heteroge- improved progression-free survival (PFS).6,7 Other antiangiogenic neous, typically slow-growing group of neoplasms.1 One of their drugs that inhibit the VEGF/VEGFR pathway, such as lenvatinib, biological hallmarks is their rich microvascular density that pazopanib, cabozantinib, surufatinib or famitinib, are being actively originates in the dense vascular network of the pancreatic islet. evaluated in PanNETs.8 These molecules foster the appearance of This structure is sustained by a host of proangiogenic molecules, altered, meandering tumour vessels that are susceptible to leaks such as vascular endothelial growth factor receptors (VEGFR), associated with areas of necrosis within the tumour.9 The platelet-derived growth factor receptor (PDGFR) and others.2,3 As a tomographic correlate includes cystic degeneration or intratumoural result, PanNETs appear as hypervascular masses on computed hypodensity, common radiological findings that are considered tomographies (CT), with avid contrast enhancement in the arterial evidence of in vivo activity of the antiangiogenic agent.10 In phase.4 The intensity of radiological enhancement correlates with contrast, clinical trials with these drugs in PanNETs usually reveal low the microvascular density.5 rates of tumour shrinkage.6,11 In the phase III randomised trial SUN1111 conducted in advanced, Response Evaluation Criteria In Solid Tumors (RECIST) were progressive, grade 1/2 PanNET, the use of the multikinase inhibitor developed to quantify efficacy of cytotoxic chemotherapy 1 Medical Oncology Department, Hospital Universitario Central de Asturias, Oviedo, Spain; 2Radiology Department, Hospital Universitario Central de Asturias, Oviedo, Spain; 3 Hematology & Medical Oncology Department, Hospital Universitario Morales Meseguer, UMU, IMIB, Murcia, Spain; 4Medical Oncology Department, Hospital Universitario 12 de Octubre, imas12, CNIO, UCM, CIBERONC, Madrid, Spain; 5Medical Oncology Department, Hospital Universitario La Paz, CIBERONC CB16/12/00398, Madrid, Spain; 6Medical Oncology Department, Hospital Universitario Virgen del Rocío, Sevilla, Spain; 7Medical Oncology Department, Hospital Universitario Ramón y Cajal, Madrid, Spain; 8Medical Oncology Department, Hospital Clínico de la Universidad de Chile, Santiago de Chile, Chile; 9Hematology & Medical Oncology Department, Hospital Universitario Morales Meseguer, Murcia, Spain; 10Medical Oncology Department, Hospital Universitario Vall d’Hebron, Vall d’Hebron Institute of Oncology (VHIO), Universitat Autònoma de Barcelona, CIBERONC, Barcelona, Spain; 11Medical Oncology Department, Hospital Universitario Miguel Servet, Zaragoza, Spain; 12Medical Oncology Department, Hospital Universitario Marqués de Valdecilla, Santander, Spain; 13Medical Oncology Department, Complejo Hospitalario de Navarra, Pamplona, Spain and 14Medical Oncology Department, MD Anderson Cancer Center, Madrid, Spain Correspondence: Paula Jimenez-Fonseca (palucaji@hotmail.com) Received: 16 May 2019 Accepted: 9 August 2019 Published online: 3 September 2019 © The Author(s) 2019 Published by Springer Nature on behalf of Cancer Research UK Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 538 according to the degree of tumour regression.12 Despite their variation rate (AVR), tumour growth rate (TGR) and diameter wide acceptance and reproducibility, their limitations are becom- variation rate (DVR). ing better known when evaluating the antiproliferative and antiangiogenic effects of targeted molecular agents,13,14 especially Measures (acquisition, imaging analysis and criteria of evaluation) in slow-growing tumours such as PanNETs.15–17 This translates as Patients underwent contrast-enhanced CT imaging of the chest, an underestimation of treatment effect in clinical trials and the abdomen and pelvis. The equipment used in these studies was CT inappropriate discontinuation of effective therapies in the case multidetector (64– or 128–detector row scanners). All studies had of tumours that maintain hypo-attenuation as a correlate to be performed with intravenous contrast according to each for effective biological activity, despite a discrete increase in centre’s standard protocol, including arterial and portal venous diameters.10,16,18 phase imaging deemed optimal for accurate evaluation, with a In response to these challenges, Choi et al. were the first to minimum of 5-mm reconstruction intervals. Images were inde- propose that the combined variation in tumour density and size, pendently read by two radiologists, specialised in NETs, who were with lower thresholds than the ones used by RECIST, was an blind to local results and to clinical data, and reached a consensus appropriate method by which to monitor response to targeted regarding their evaluation of response. PFS was defined as the therapies.19–22 Comparative data suggest improved performance time in months from initiation of treatment with sunitinib until of Choi versus RECIST criteria in a number of solid tumours progression or death, censoring event-free subjects. including neuroendocrine tumours;10,18,16 however, further eva- Choi and RECIST v1.1 criteria were analysed as per the usual luations are needed to validate this promising preliminary results descriptions.12,23 The exact definitions followed to interpret the prior to recommending their widespread use for PanNETs in images are displayed in Supplementary Table 1. In keeping with clinical practice. RECIST v1.1, up to five target lesions were selected (maximum of Against this backdrop, the CRIPNET_GETNE1504-study two per organ involved) calculating the sum of target lesions in (NCT02841865) seeks to compare the usefulness of Choi versus conformity with pre-established criteria.12 The Hounsfield unit RECIST criteria in well-differentiated, advanced PanNETs and to (HU) value was obtained using the mean attenuation of the pixels discern which criterion best captures sunitinib’s activity in this of the region of interest within the contour of the target lesions in population. arterial phase. The result was averaged to obtain a mean measure 1234567890();,: of CT attenuation. Changes in size or attenuation were calculated in absolute or relative terms against baseline values and the METHODS previous CT. The AVR and DVR were calculated by dividing the Study design and patients relative changes in density and size by the time since CRIPNET_GETNE1504 is a multicentre study classified by the commencement of sunitinib until evaluation of response. The Spanish Agency of Medicines and Medical Devices (AEMPS, for its TGR was calculated according to the definition by Ferté.24 Tumour acronym in Spanish) as a post-authorisation trial of prospective size was defined as the baseline sum of target lesion diameters follow-up. It was conducted at 10 sites of the Spanish Group of (D0), while Dt is the same measure after time (t) has elapsed. Thus, Neuroendocrine and Endocrine Tumors (GETNE, for its acronym in tumour growth (TG) was calculated using the formula: TG = 3 Log Spanish) and recruited consecutive patients with advanced, well- (Dt/D0)/t. To express this magnitude in clinically relevant terms, TG differentiated PanNETs treated with sunitinib between 2012–2017. was expressed as a percentage of variation over 1 month, by The study did not include any kind of intervention. The decision to means of the transformation: TGR = 100 (exp(TG) −1). modify treatment was therefore based on the investigator’s criterion, according to local standard radiological evaluation as per Statistics RECIST. The study was approved by the Ethics Review Boards of Cohen’s kappa was used as a measure of concordance between the participating centres and by the healthcare authorities of each Choi/RECIST criteria. Correlation between PFS & OS was quantified geographical region. All subjects gave their written informed by means of Kendall’s τ associated with Clayton’s copula models consent to participate in the study. for bivariate survival data.25 Wilcoxon signed rank test was used Eligibility criteria included: histological confirmation of PanNET, for paired data to compare variations in size/attenuation between Ki-67 index ≤ 20%, availability of serial image studies, with arterial consecutive radiological studies. A landmark estimation was made and portal phase, within the 4 weeks prior to initiating treatment for PFS at 24 months, after 6 months of follow-up. The landmark with sunitinib and at least every 6 months until progression or the method estimates the likelihood of survival after a set period of end of the study. Individuals were excluded if they received other time and is suitable for time-dependent variables, such as tumour concurrent antiproliferative treatments, with the exception of response.26,27 In this case, the landmark coincides approximately somatostatin analogues prescribed for their anti-secretory effect; with the date of the evaluations of response conducted in the those with short exposure to sunitinib (fewer than two cycles) due patients. The prognostic effect was appraised by Cox proportional to causes other than demise or clinical decline, or with previous hazards regression, factoring in the time-dependent nature of the use of other antiangiogenic agents. Cases with suboptimal or non- evaluation of response. The Mantel–Byar test was used to standard CT assessment (e.g. insufficient enhancement with compare survival endpoints as per Choi or RECIST. This method contrast, insufficient body coverage), or the absence of measur- is a modification of the log-rank test that avoids bias in survival able disease were likewise excluded. analysis by tumour response.27 The analyses were executed with Sunitinib was started at the standard dose, 37.5 mg/day, and the RStudio statistical software (RStudio, Inc., Boston, MA, USA), the criteria for dose reductions and delays were done as per the including the landest, survival and rgl packages.28–30 summary of product characteristics and the clinical judgment of the attending medical oncologist, expert in the management of PanNETs. RESULTS Patients Objectives One hundred and seven patients were recruited, 22 of whom were The main objective of the study was to assess the association of excluded as they did not meet the quality requirements per RECIST v1.1 and Choi criteria on survival endpoints. Other aims protocol for imaging studies, including the adequate predefined were to analyse whether PFS as per Choi is a better surrogate frequency of procedures. The baseline characteristics of the 85 endpoint for overall survival (OS) than PFS according to RECIST, individuals included are presented in Table 1. Participants had a and to appraise the correlation with prognosis of attenuation median age of 59 years (range, 21–84) and 58% were male. Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 539 tumour response per RECIST and PFS (Mantel–Byar test, p = 0.4). It Table 1. Patients’ sociodemographic variables and baseline must be remembered that, by definition, this analysis only characteristics contemplated two categories (PR and SD), as the RECIST-based N = 85, % PD endpoint coincided with the date of the CT in remaining cases that were found to be in progression (infinite, non-calculable Sex, male 49 (58%) hazard ratio [HR]). In contrast, the Choi-based PD (n = 29) Age, median (range) 59 (21–84) identified a group at greater risk for progression as per RECIST ECOG PS 0/1/2 35/49/1 (HR 2.55, 95% CI, 1.09–5.96) and for death (HR 2.44, 95% CI, 0.99–5.97). Tumour progression, by both RECIST and Choi criteria, Functioning tumour 21 (25%) increased the risk of death with an equivalent magnitude of effect Sites of metastases in both cases. However, had only morphological criteria been Liver 80 (94%) applied, 7/29 patients with PD by Choi (24%) would have been Peritoneum 4 (5%) classified as SD by RECIST. This would have overestimated these Lymph nodes 26 (31%) subjects’ prognosis, given that the 24-month OS rate (landmark) was 81.7% (95% CI, 69.6–91.1%) for SD as per RECIST, versus 50.1% Lung 3 (4%) (95% CI, 33.7–67.5%) for PD according to Choi criteria (see Other 10 (12%) Supplementary Table 2). Furthermore, the median PFS in these Number of involved organs, median (range) 1 (0–4) seven subjects was 13.4 months when the endpoint was defined Ki67, median (range) 7 (1–20) by RECIST versus 5.5 months as per Choi, which potentially entails Prior SSA 55 (65%) 7.9 months of missed opportunity to change treatment Likewise, 26/44 (59%) with RECIST SD were reclassified as PR by Previous lines excluding SSA Choi. Applying only RECIST led to a discreet underestimation of 0 46 (54%) prognosis, since the 24-month OS rate (landmark) was 91.8% (95% 1 20 (23%) CI, 83.3–100%) for Choi PR, somewhat higher than RECIST- 2 17 (20%) calculated SD. Nevertheless, the increase in the PR rate after 3 2 (2%) applying Choi criteria did not devalue the prognostic meaning of response, which continued to exert an equivalent favourable Sunitinib as first-line therapy 22 (26%) effect, as per both Choi and RECIST. Thus, the 24-month PFS Primary tumour surgery 33 (39%) landmark for patients with Choi and RECIST PR was 40.4% (95% CI, Locoregional therapies 4 (5%) 28.4–57.5%) and 48.0% (95% CI, 24.5–93.8%), respectively. Individuals with PR according to Choi had better PFS and OS, with HR 0.69 (95% CI, 0.29–1.61) and 0.86 (95% CI, 0.34–2.16), respectively, versus those with SD, but the results were not PanNETs had a median Ki67% of 7 (range, 1–20) and in 62%, statistically significant. metastases were limited to a single organ. Sixty-five percent received sunitinib after a somatostatin analogue. Sunitinib was Evaluation of tumour response in the first 3 months maintained for a median of 52 weeks (range, 7–301), with dose Subsequently, the ‘earlier’ evaluation performed in 63 patients reductions reported in 36% of treated patients. after a median of 2.8 months (range, 1.1–4.4) was predictive of PFS/OS (see Supplementary Table 3). In this scenario, PD by RECIST Evaluation of PFS by RECIST versus Choi was observed to have a more significant and greater effect on the During the follow-up period, 73 progression events were detected risk of death (HR for OS of 2.22, 95% CI, 1.13–4.25) compared to PD by RECIST and 64 by Choi criteria. The median PFS centrally defined by Choi criteria (HR for OS of 1.28, 95% CI, 0.58–2.83). evaluated were 11.42 months (95% confidence interval [CI], However, no statistical evidence was found that pointed toward 9.7–15.9) as defined by RECIST and 15.8 (95% CI, 13.9–25.7) as early evaluation by RECIST significantly predicted PFS per Choi criteria. PFS by Choi exhibited greater correlation with OS (Mantel–Byar test, p = 0.3), although we must bear in mind that, (Kendall’s τ = 0.722, standard error [SE] = 0.046) than PFS by by definition, this analysis only has two RECIST-defined categories RECIST (Kendall’s τ = 0.439, SE = 0.068) (see scatter plot of (PR and SD; the category PD by RECIST coincides with the bivariate survival copulas in Fig. 1a, b). Of note, investigator- endpoint). Choi criteria evaluated at 3 months displayed a evaluated PFS was 14.5 months (95% CI, 11.8–18.0). significant association with PFS (Mantel–Byar test, p = 0.04), but not OS (Supplementary Table 3). Evaluation of tumour response in the first six months Of the 85 subjects with a valid baseline CT, 78 had a CT at Evaluation of tumour growth rate and attenuation variation rate 6 months. At this time point, agreement between RECIST and Choi Finally, the relation between tumour size and AVR was examined, was weak (Cohen’s Kappa for 2 Raters, 0.392, p < 0.001). The as was the influence of these parameters on prognosis. First of all, transition diagram for both methods is shown in Fig. 2. According scatter plots were constructed that yield a graphic illustration of to RECIST, 12.8% attained a partial response (PR), 56.4% stable the correlation between relative variations in HU and DVR (Δ sum disease (SD) and 30.7% progressive disease (PD). When Choi of diameters, %), at 6 and 3 months (Fig. 1c, d, respectively). These criteria were applied, the main change was the increase in the scatter plots reveal that changes in attenuation are significantly percentage of PR, up to 47.4%, at the expense of decreasing the associated with variations in size, accounting for approximately rate of SD to 15.3%, and of a slight increment in the percentage of 18% of their variation. In both cases, the slopes are moderate, as PD (37.1%). these are slow-growing tumours. As expected, the DVR exhibited a Supplementary Table 2 summarises the association of the 6- very high correlation with the TGR (Kendall’s τ = 0.960). Likewise, month tumour response assessment with survival endpoints. the AVR explains 21% of the variability in tumour volume at Landmark survival curve estimates are reported in Fig. 3. Choi 6 months. criteria are seen to be significantly effective at predicting both PFS To demonstrate how these changes projected on evaluations of and OS (Mantel–Byar tests, p ≤ 0.001, in both cases). Evaluation by response, spider plots were drawn that illustrate the cross RECIST was associated significantly with OS. On the other hand, no relationship between: (1) DVR versus Choi criteria, and (2) AVR statistical evidence was found to favour the association between versus RECIST (Supplementary Fig. 1A, B, respectively). In this case, Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 540 a Bivariate copula (PFS by RECIST vs. OS) b Bivariate copula (PFS by Choi vs. OS) 1.0 1.0 Event Kendall’s  = 0.439, SE = 0.068 Event Kendall’s  = 0.722, SE = 0.046 Consored Consored 0.8 0.8 0.6 0.6 F (t1) F (t1) ^ ^ 0.4 0.4 0.2 0.2 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ^ ^ F (t2) F (t2) c 60 d 30 Δ Sum of diameters (%) 40 Δ Sum of diameters (%) 0 0 –30 y = 2.84 + 0.38x –0.002x2 –40 y = –2.68 + 0.31x P-value = 0.002 P-value = 0.0003 Adj. R 2 = 0.185 Adj. R 2 = 0.173 –60 –50 0 50 100 –50 0 50 100 Δ Average density (HU, %) Δ Average density (HU, %) Fig. 1 a Scatter plot of the bivariate survival copulas (PFS by RECIST vs OS); b scatter plot of the bivariate survival copulas (PFS by Choi vs OS). Interpretation of axes (a, b): progression-free survival (vertical) and overall survival (horizontal). c Δ Sum of diameters (%) and average Δ of tumour density (%) at 6 months; d Δ sum of diameters (%) and average Δ of tumour density (%) at 3 months. HU Hounsfield units, PFS progression-free survival, OS overall survival, SE standard error given the slow growth rate, no significant changes in size were RECIST Choi observed between time point 0 and 3 or 6 months (Wilcoxon test, SD, n = 12, p > 0.1). In contrast, attenuation did decrease significantly in both 11 15.3% spans (Wilcoxon test, p < 0.001). Figure 4 is a 3D-waterfall plot that SD, n = 44, illustrates the relation between RECIST, AVR, and PFS in each 56.4% PanNET. 7 26 PD, n = 29, Next, we assessed whether changes in DVR, TGR and AVR could 37.1% predict prognosis during treatment with sunitinib. Supplementary Table 4 shows the association between DVR, TGR and AVR with 1 survival endpoints at 3 and 6 months. While both parameters are associated with PFS and OS in univariate analyses, only TGR is an PD, n = 24, 22 independent prognostic factor in the multivariate model for both 30.7% 1 endpoints (see Supplementary Table 4). PR, n = 37, 47.4% 10 PR, n = 10, DISCUSSION 12.8% Proper monitoring of response to antineoplastic treatments is Fig. 2 Transition plot with the reclassification between RECISTv1.1 fundamental to reduce toxicity and costs. In the specific case of and Choi criteria. n number, PD progressive disease, PR partial sunitinib, this antiangiogenic drug induces biological changes in response, SD stable disease the tumours, that are visible on CTs, such as variations in Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 541 a b 6-month RECISTv1.1 response criteria 6-month Choi response criteria Conditional PFS rate (%) at 24 months from 6-month landmark time 1.00 Conditional PFS rate (%) at 24 months from 6-month landmark time 1.00 SD 33.7% (Cl 95% 23.3–50.2) PD 8.1% (Cl 95% 3.3–18.2) PR 48.0% (Cl 95% 24.5–93.8) Progression–free survival, % Progression–free survival, % SD 33.0% (Cl 95% 11.8–92.6) PR 40.4% (Cl 95% 28.4–57.5) 0.75 0.75 Mantel-Byar test, p < 0.4 Mantel-Byar test, p < 0.001 0.50 0.50 0.25 0.25 0.00 0.00 0 6 12 18 24 30 36 42 48 0 6 12 18 24 30 36 42 48 Time (months) Time (months) Number at risk Number at risk SD 0 8 4 1 1 0 0 0 0 SD 1 28 22 13 9 6 6 5 4 PD 0 6 2 2 1 1 1 1 1 PR 0 24 24 17 10 8 6 5 4 PR 0 8 8 7 3 3 1 1 1 0 6 12 18 24 30 36 42 48 0 6 12 18 24 30 36 42 48 Fig. 3 a Kaplan–Meier curves for conditional progression-free survival from 6-month landmark time stratified by Choi. b Kaplan–Meier curves for conditional progression-free survival from 6-month landmark time by RECIST. It does not include the PD category by RECIST as it coincides with the endpoint. CI confidence interval, PD progressive disease, PFS progression-free survival, PR partial response, SD stable disease 130 120 110 100 90 (HU, %) 80 70 60 density 50 40 Δ Average 30 20 10 0 –10 –20 –30 –40 90 –50 80 70 60 50 s) nth 40 ( mo 30 PFS 20 RECIST V1.1 Partial response 10 Stable disease Tumour progression New lesions Censored Fig. 4 3D-waterfall plot with the Δ in the average of HU, RECIST v1.1 and PFS. HU hounsfield units, PFS progression-free survival density,10,18 and produces a reduction in tumour size in most In other hypervascular tumours, such as hepatocellular carci- cases, although these regressions rarely reach the threshold of PR noma, kidney cancer or gastrointestinal stromal tumours, Choi by RECIST.6 Choi criteria are based on subtle changes in criteria have been used to assess response to tyrosine-kinase attenuation or size, theoretically making them a more adequate inhibitors having an antiangiogenic effect, such as sorafenib, method by which to track sunitinib’s effect in vivo (Smith et al., axitinib, sunitinib, pazopanib, etc.31 Other criteria have also been 2010; Faivre et al., 2012; Raymond et al., 2018). Nonetheless, there developed to respond to the complexities specific to each is scant experience with monitoring treatment effect using Choi disease32,33 and treatment modality, such as the modified RECIST criteria in PanNETs treated with antiangiogenics.10,16,18 in hepatocellular carcinoma treated with chemoembolisation or Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 542 MASS (morphology, attenuation, size and structure) criteria in attenuation translates into sustained antiangiogenic activity. None- renal cancer treated with antiangiogenics.34,35 theless, the reader must be aware that this study is not statistically In the CRIPNET_GETNE1504 study, we have observed that powered to detect this kind of subgroup effect. It must also be assessment of response via RECIST v1.1 at 6 months categorises remembered that in patients not treated with antiangiogenics, more than half of the series as SD, with relatively few PR. In our hypoattenuated PanNETs are generally associated with tumour de- cohort, tumours with SD as per RECIST comprised a hetero- differentiation and poor prognosis.5 geneous group from a prognostic perspective. This was pre- The CRIPNET_GETNE1504 study has certain limitations. Given the sumably due to the fact that tumour growth was typically slow, low incidence of PanNETs, the sample size is relatively small. Despite regardless of the efficacy of sunitinib. When tumours were this, it is one of the largest series reported to date.10,16,18 Secondly, appraised by Choi criteria, the most noteworthy consequence the decision as to whether discontinue sunitinib or not was based was the dramatic increase in PR rate (Δ + 34.6%). All told, more on each centre’s clinical practice, separate from the centralised than half of the cases of RECIST SD were reclassified as PR by Choi. radiological criterion, which may have influenced the estimation of The possibility of re-assessing sunitinib’s activity in PanNETs is PFS. The reader must likewise be aware that, while certain minimum consistent with earlier information in the literature.10,16,18 Our data quality criteria were required, and 22 cases were excluded on the point toward response as defined by Choi criteria, despite it being basis of insufficient imaging quality, the studies were conducted in more ‘permissive’, had a favourable effect on PFS/OS comparable accordance with the local protocols of each centre. Choi criteria to RECIST-determined response. Since no evidence was found to should be generalised by means of the rigorous application of endorse a substantial decline in prognostic value, the conclusion is standardised protocols. This calls for prior consensus across the that Choi was able to capture a clinically significant biological radiodiagnostic and other departments involved in the manage- effect that went unnoticed by RECIST. Therefore, most individuals ment of patients with neuroendocrine tumours. Finally, interobser- in PR as per Choi criteria in our series, in all likelihood actually ver agreement between the two radiologists who analysed response benefitted from the drug. was not examined. Nonetheless, earlier studies point toward the Although the percentage of subjects with RECIST SD reclassified level of concordance is generally high.38,39 as PD was relatively low, had Choi criteria not been applied, In conclusion, in advanced PanNETs treated with sunitinib, RECIST prognosis would have been overestimated in some 16%. However, as the sole method of monitoring response entails an imprecise or the impact of PD according to Choi criteria is complex and subtle. erroneous prognostic prediction in almost half of the patients. On the one hand, Choi criteria employ a more restrictive threshold RECIST PFS was a worse surrogate for OS than Choi-estimated PFS. In of PD (10% versus 20% with RECIST). However, the definition contrast, Choi criteria capture the activity of sunitinib in a clinically crucially excludes those tumours that, despite having grown, meet significant manner and suit the follow-up of slow-growing PanNETs. the criterion of PR based on decreased attenuation. The overall If the data from this study are confirmed through data from clinical result was that, paradoxically, despite the slight increase in PD at trials, Choi criteria should be considered as standard for monitoring 6 months with Choi criteria, PFS evaluated by these same criteria advanced PanNETs response to antiangiogenic therapy. was higher than RECIST PFS. The reason for this was that, while there were more initial progressions with Choi criteria, the effect was offset by tumours that maintained hypo-attenuation at ACKNOWLEDGEMENTS 12 months, despite their slow growth. This latter observation is The authors acknowledge GETNE for sponsoring this study, Priscilla Chase Duran for in line with the data from the phase IV sunitinib trial.16 The editing and translating the manuscript, The MFAR S.L. team for the support of the reduction of the cut-off (from 20 to 10%) improved the OS website registry. prediction overall, but did not affect the specificity of the definition of PD. Lamarca et al. recently published similar data.15 Therefore, the crux of the matter is what definition of PFS is the AUTHOR CONTRIBUTIONS M.P.S.H. and P.J.F. developed the project; A.F.V., D.C. and M.P.S.H. analysed the most valid for use as a surrogate for OS in this setting. Imaoka imaging studies, and A.C.B. analysed the data. P.J.F. and A.C.B. drafted the et al. previously considered that RECIST-defined PFS was an manuscript. The other authors recruited patients and provided clinical information, acceptable OS surrogate in PanNETs.36 In contrast, our data reveal comments and improvements to the manuscript. All authors participated in the that the correlation between PFS according to Choi and OS is interpretation and discussion of data. intermediate, and low, in the case of RECIST-based PFS. In line with pre-set criteria, this is compatible with a lack of validity of the latter surrogate.37 Nevertheless, correlation between surrogates ADDITIONAL INFORMATION and patient-relevant endpoints alone should not be the sole Supplementary information is available for this paper at https://doi.org/10.1038/ criterion to be contemplated in the case of indolent tumours, such s41416-019-0558-7. as neuroendocrine tumours, with several effective treatment options after first-line. Be that as it may, Choi PFS displayed a Competing interests: M.P.S.H. declares travel grants from and travel accommoda- slightly better correlation with OS, although further studies are tions from Amgen, Astellas, Astrazeneca, BMS, Ipsen, Novartis Sanofi, Pfizer, PharmaMar and Roche. Financial support for educational programs from BMS, needed to prove this surrogate’s validity. MSD, Ipsen, Novartis Sanofi, Pfizer, PharmaMar and Roche. Honoraria for meetings Quite a different issue is whether the dichotomisation of response and/or lectures from Astrazeneca, Pfizer and Roche. A.C.B. declares travel grants from entails loss of information, by ignoring gradualness.17 AVR and TGR Ipsen, Roche and Novartis and consulting/advisory role for Roche, Rovi, LeoPharma, correctly predicted RECIST-based PFS. Said effect was already Pfizer, Esteve all outside of the scope of this work. R.G.C. declares having provided present in studies at 3 months; hence, AVR and TGR can be early scientific advice and/or received honoraria from AAA, Amgen, Bayer, Ipsen, Lilly, markers of clinically significant activity of the drug. These Merck, MSD, Novartis, PharmaMar, Pfizer, Roche and Sanofi-Aventis, and has received measurements of variation by unit of time worked better than research support from Pfizer and BMS.T.A.G. declares advisory role, travel grants and density- or size-related variations. Furthermore, both were non- research funding from IPSEN, Novartis, Roche, BMS, Pfizer, Sanofi, Janssen, Astellas, linearly interrelated; consequently, in multivariate analyses, only TGR Astrazeneca. B.N.V. has received honoraria for advisory role from Novartis, declares travel grants and accommodations for educational programs from Tecnopharma, predicted survival endpoints. However, several nuances may limit Pfizer, Bayer, Sanofi, MSD. M.S.C. has received honoraria for advisory role from the practical use of TGR in some patients. First of all, the difficulty in KyowaKirin, research funding from Leo Pharma, travel accommodations from Sanofi, discerning the prognostic significance of slight differences in size MSD, Esteve, Amgen, Servier, Angelini, financial support for educational programs must be taken into account, especially since they are the ones that from Angelini, Sanofi, Rovi. V.A. declares advisory role from Amgen, Roche, Merck, occur in most patients. In addition, it cannot be ruled out that AVR Servier, Sanofi, Bayer, Ipsen, Novartis; travel accommodations from Roche, Merck, might modulate the effect of discrete growth, insofar as hypo- Novartis, Ipsen, Sanofi; personal fees from Amgen, Roche, Merck, Servier, Sanofi, Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 543 Bayer, Ipsen, Novartis, AAA and non-financial support from Roche, Merck, Novartis, 13. de Mestier, L., Dromain, C., d’Assignies, G., Scoazec, J.-Y., Lassau, N., Lebtahi, R. Ipsen, Sanofi. C.L. has received honoraria from Roche, Merck, Sanofi, Pfizer, Eisai, et al. Evaluating digestive neuroendocrine tumor progression and therapeutic Ipsen, Bayer; consulting or advisory role from Amgen, Roche, Sanofi, Merck, Servier, responses in the era of targeted therapies, state of the art. Endocr. Relat. Cancer Pfizer, Ipsen, Bayer, Eisai; research funding from Amgen, Roche, Merck, Sanofi, Bayer, 21, 105–120 (2014). Ipsen, Eisai, Celgene and travel accommodations from Roche, Pfizer, Merck. Servier, 14. Garcia-Carbonero, R., Garcia-Figueiras, R., Carmona-Bayonas, A., Sevilla, I., Teule, A., Amgen, and Ipsen. E.G. has received honoraria for ad boards, meetings and/or Quindos, M. et al. Imaging approaches to assess the therapeutic response of lectures from Pfizer, BMS, IPSEN, Roche, Eisai, Eusa Pharma, MSD, Sanofi, AAA, gastroenteropancreatic neuroendocrine tumors (GEP-NETs), current perspectives Novartis, Pierre Fabre, Lexicon, and Celgene and has received unrestricted research and future trends of an exciting field in development. Cancer Metastas. Rev. 34, grants from Pfizer, Astra Zeneca, MTEM/Threshold, Roche, IPSEN and Lexicon. J.C. has 823–842 (2015). received honoraria for scientific consultancy role from Novartis, Pfizer, Ipsen, Exelixis, 15. Lamarca, A., Barriuso, J., Kulke, M., Borbath, I., Lenz, H.-J., Raoul, J. L. et al. Bayer, Eisai, AAA, Amgen, Sanofi, and Merck and research support from Eisai, Novartis, Determination of an optimal response cut-off able to predict progression-free Ipsen, Astrazeneca, Pfizer, and AAA. P.J.F. declares travel grants from Ipsen and survival in patients with well-differentiated advanced pancreatic neuroendocrine consulting/advisory role for Roche, Celgene, Bristol, Mylan, Rovi, LeoPharma, all tumours treated with sunitinib, an alternative to the current RECIST-defined outside of the scope of this work. The remaining authors declare no competing response. Br. J. Cancer 118, 181–188 (2018). interests. 16. Raymond, E., Kulke, M. H., Qin, S., Yu, X., Schenker, M., Cubillo, A. et al. Efficacy and safety of sunitinib in patients with well-differentiated pancreatic neuroendocrine Ethics approval and consent to participate: All procedures followed were in tumours. Neuroendocrinology 107, 237–245 (2018). accordance with the ethical standards of the committee responsible for human 17. Lamarca A., Crona J., Ronot M., Opalinska M., Lopez C. L., Pezzutti D., et al. Value of experimentation (institutional and national) and with the Helsinki Declaration of 1964 Tumor Growth Rate (TGR) as an Early Biomarker Predictor of Patients’ Outcome in and later versions. All institutional ethics committees that have approved this study Neuroendocrine Tumors (NET). The GREPONET Study. Oncologist https://doi.org/ are listed in Supplementary Table 5. Informed consent was obtained from all patients 10.1634/theoncologist.2018-0672 (2019). before being included in the study. 18. Luo, Y., Chen, J., Huang, K., Lin, Y., Chen, M., Xu, L. et al. Early evaluation of sunitinib for the treatment of advanced gastroenteropancreatic neuroendocrine neoplasms via CT imaging, RECIST 1.1 or Choi Criteria? BMC Cancer 17, 154 (2017). Funding: This study has been funded by a restricted grant from Pfizer, S.A. This entity 19. Choi, H., Charnsangavej, C., Faria, S. C., Macapinlac, H. A., Burgess, M. A., Patel, S. R. has not contributed to the design of the study, analyses, interpretation of results, et al. Correlation of computed tomography and positron emission tomography in drafting of the document or to the decision regarding publication. patients with metastatic gastrointestinal stromal tumor treated at a single insti- tution with imatinib mesylate, proposal of new computed tomography response Data availability: All the data generated or analysed in this study are included in the criteria. J. Clin. Oncol. 25, 1753–1759 (2007). manuscript or the supplementary information. 20. Benjamin, R. S., Choi, H., Macapinlac, H. A., Burgess, M. A., Patel, S. R., Chen, L. L. et al. We should desist using RECIST, at least in GIST. J. Clin. Oncol. 25, 1760–1764 (2007). Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims 21. Ronot, M., Bouattour, M., Wassermann, J., Bruno, O., Dreyer, C., Larroque, B. et al. in published maps and institutional affiliations. Alternative Response Criteria (Choi, European association for the study of the liver, and modified Response Evaluation Criteria in Solid Tumors [RECIST]) Versus RECIST 1.1 in patients with advanced hepatocellular carcinoma treated with sorafenib. Oncologist 19, 394–402 (2014). REFERENCES 22. Smith, A. D., Lieber, M. L. & Shah, S. N. Assessing tumor response and detecting 1. Nuñez‐Valdovinos, B., Carmona‐Bayonas, A., Jimenez‐Fonseca, P., Capdevila, J., recurrence in metastatic renal cell carcinoma on targeted therapy, importance of Castaño‐Pascual, A., Benavent, M. et al. Neuroendocrine Tumor Heterogeneity size and attenuation on contrast-enhanced CT. AJR Am. J. Roentgenol. 194, Adds Uncertainty to the World Health Organization 2010 Classification, Real‐ 157–165 (2010). World Data from the Spanish Tumor Registry (R‐GETNE). Oncologist 23, 422–432 23. Choi, H. Response evaluation of gastrointestinal stromal tumors. Oncologist 13 (2018). (Suppl 2), 4–7 (2008). 2. Teulé A., Martín L., Casanovas O. in Management of neuroendocrine tumors of the 24. Ferté, C., Fernandez, M., Hollebecque, A., Koscielny, S., Levy, A., Massard, C. et al. pancreas and digestive tract (eds. Raymond, E., Faivre, S. & Ruszniewski, P.) 29–41 Tumor growth rate is an early indicator of antitumor drug activity in phase I (Springer-Verlag, Paris, 2014). clinical trials. Clin. Cancer Res. 20, 246–252 (2014). 3. Jiménez-Fonseca, P., Martín, M. N., Carmona-Bayonas, A., Calvo, A., Fernández- 25. Weber, E. M. & Titman, A. C. Quantifying the association between progression‐ Mateos, J., Redrado, M. et al. Biomarkers and polymorphisms in pancreatic neu- free survival and overall survival in oncology trials using Kendall’s τ. Stat. Med. 38, roendocrine tumors treated with sunitinib. Oncotarget 9, 36894–36905 (2018). 703–719 (2019). 4. Sheth, S., Hruban, R. K. & Fishman, E. K. Helical CT of islet cell tumors of the pancreas, 26. Parast, L., Tian, L. & Cai, T. Landmark estimation of survival and treatment effect in typical and atypical manifestations. Am. J. Roentgenol. 179, 725–730 (2002). a randomized clinical trial. J. Am. Stat. Assoc. 109, 384–394 (2014). 5. Rodallec, M., Vilgrain, V., Couvelard, A., Rufat, P., O’Toole, D., Barrau, V. et al. 27. Anderson, J. R., Cain, K. C. & Gelber, R. D. Analysis of survival by tumor response Endocrine pancreatic tumours and helical CT, contrast enhancement is correlated and other comparisons of time-to-event by outcome variables. J. Clin. Oncol. 26, with microvascular density, histoprognostic factors and survival. Pancreatology 6, 3913–3915 (2008). 77–85 (2006). 28. Therneau T. M., Lumley T. Package ‘survival’. https://cran.r-project.org/web/ 6. Raymond, E., Dahan, L., Raoul, J., Bang, Y., Borbath, I., Lombard-Bohas, C. et al. packages/survival/index.html. (2016) Sunitinib malate for the treatment of pancreatic neuroendocrine tumors. N. Engl. 29. Adler D., Murdoch M. D. Package ‘rgl.’ https://cran.r-project.org/web/packages/ J. Med 364, 501–513 (2011). rgl/index.html. (2018). 7. Faivre, S., Niccoli, P., Castellano, D., Valle, J. W., Hammel, P., Raoul, J.-L. et al. 30. Parast L. Package ‘landest’. https://cran.r-project.org/web/packages/landest/landest. Sunitinib in pancreatic neuroendocrine tumors, updated progression-free survival pdf. (2015). and final overall survival from a phase III randomized study. Ann. Oncol. 28, 31. van der Veldt, A. A. M., Meijerink, M. R., van den Eertwegh, A. J. M., Haanen, J. B. A. 339–343 (2016). G. & Boven, E. Choi response criteria for early prediction of clinical outcome in 8. Grillo, F., Florio, T., Ferraù, F., Kara, E., Fanciulli, G., Faggiano, A. et al. Emerging patients with metastatic renal cell cancer treated with sunitinib. Br. J. Cancer 102, multitarget tyrosine kinase inhibitors in the treatment of neuroendocrine neo- 803–809 (2010). plasms. Endocr. Relat. Cancer 25, 453–466 (2018). 32. Smith, A. D., Shah, S. N., Rini, B. I., Lieber, M. L. & Remer, E. M. Morphology, 9. Hao, Z. & Sadek, I. Sunitinib, the antiangiogenic effects and beyond. Onco Targets Attenuation, Size, and Structure (MASS) criteria, Assessing response and pre- Ther. 9, 5495–5050 (2016). dicting clinical outcome in metastatic renal cell carcinoma on antiangiogenic 10. Faivre, S., Ronot, M., Dreyer, C., Serrate, C., Hentic, O., Bouattour, M. et al. Imaging targeted therapy. Am. J. Roentgenol. 194, 1470–1478 (2010). response in neuroendocrine tumors treated with targeted therapies, the 33. Lencioni, R. & Llovet, J. M. Modified RECIST (mRECIST) assessment for hepato- experience of sunitinib. Target Oncol. 7, 127–133 (2012). cellular carcinoma. Semin Liver Dis. 30, 52–60 (2010). 11. Yao, J. C., Shah, M. H., Ito, T., Bohas, C. L., Wolin, E. M., Van Cutsem, E. et al. 34. Gaur, S. K., Friese, J. L., Sadow, C. A., Ayyagari, R., Binkert, C. A., Schenker, M. P. Everolimus for advanced pancreatic neuroendocrine tumors. N. Engl. J. Med 364, et al. Hepatic arterial chemoembolization using drug-eluting beads in gastro- 514–523 (2011). intestinal neuroendocrine tumor metastatic to the liver. Cardiovasc. Interv. Radio. 12. Eisenhauer, E. A., Therasse, P., Bogaerts, J., Schwartz, L. H., Sargent, D., Ford, R. 34, 566–572 (2011). et al. New response evaluation criteria in solid tumours, revised RECIST guideline 35. Bhagat, N., Reyes, D. K., Lin, M., Kamel, I., Pawlik, T. M., Frangakis, C. et al. Phase II (version 1.1). Eur. J. Cancer 45, 228–247 (2009). study of chemoembolization with drug-eluting beads in patients with hepatic Evaluating radiological response in pancreatic neuroendocrine tumours. . . Mª P Solis-Hernandez et al. 544 neuroendocrine metastases, high incidence of biliary injury. Cardiovasc. Interv. Open Access This article is licensed under a Creative Commons Radio. 36, 449–459 (2013). Attribution 4.0 International License, which permits use, sharing, 36. Imaoka, H., Sasaki, M., Takahashi, H., Hashimoto, Y., Ohno, I., Mitsunaga, S. et al. adaptation, distribution and reproduction in any medium or format, as long as you give Progression-free survival as a surrogate endpoint in advanced neuroendocrine appropriate credit to the original author(s) and the source, provide a link to the Creative neoplasms. Endocr. Relat. Cancer 24, 475–483 (2017). Commons license, and indicate if changes were made. The images or other third party 37. Institute for Quality and Efficiency in Health Care (IQWiG). Validity of surrogate material in this article are included in the article’s Creative Commons license, unless endpoints in oncology. https://www.ncbi.nlm.nih.gov/books/NBK198799/. (2011). indicated otherwise in a credit line to the material. If material is not included in the 38. Thian, Y., Gutzeit, A., Koh, D.-M., Fisher, R., Lote, H., Larkin, J. et al. Revised Choi article’s Creative Commons license and your intended use is not permitted by statutory imaging criteria correlate with clinical outcomes in patients with metastatic renal regulation or exceeds the permitted use, you will need to obtain permission directly cell carcinoma treated with sunitinib. Radiology 273, 452–461 (2014). from the copyright holder. To view a copy of this license, visit http://creativecommons. 39. Beuzit, L., Edeline, J., Brun, V., Ronot, M., Guillygomarc’h, A., Boudjema, K. et al. org/licenses/by/4.0/. Comparison of Choi criteria and Response Evaluation Criteria in Solid Tumors (RECIST) for intrahepatic cholangiocarcinoma treated with glass-microspheres Yttrium-90 selective internal radiation therapy (SIRT). Eur. J. Radio. 85, 1445–1452 © The Author(s) 2019 (2016).