Bladder Cancer 8 (2022) 339–357 339 DOI 10.3233/BLC-220055 IOS Press Systematic Review Predicting Recurrence and Progression in Patients with Non-Muscle-Invasive Bladder Cancer: Systematic Review on the Performance of Risk Stratification Models Félix Guerrero-Ramosa,b,c,1,∗ , José Daniel Subielad,1 , Óscar Rodrı́guez-Fabae , Julia Aumatelle , Celeste Manfredif , Giorgio Bozzinig , Javier Romero-Oteroa,b and Felipe Couñagoh,i,j a ROC Clinic, Madrid, Spain b Department of Urology, Hospital Universitario HM Sanchinarro, Madrid, Spain c Department of Urology, Hospital Universitario 12 de Octubre, Madrid, Spain d Department of Urology, Hospital Universitario Ramón y Cajal, IRYCIS, Universidad de Alcalá, Madrid, Spain e Department of Urology, Fundació Puigvert, Barcelona, Spain f Unit of Urology, Department of Woman, Child and General and Specialized Surgery, University of Campania “Luigi Vanvitelli”, Naples, Italy g Department of Urology, ASST Lariana Ospedale Sant’Anna, Como, Italy h Genesis Care Madrid, Madrid, Spain i Hospital San Francisco de Ası́s, Madrid, Spain j Hospital La Milagrosa, Madrid, Spain Received 1 June 2022 Accepted 3 October 2022 Pre-press 20 October 2022 Published 14 December 2022 Abstract. BACKGROUND: Several classifications have been reported to stratify non-muscle-invasive bladder cancer (NMIBC) in risk groups according to the probability of recurrence and progression. OBJECTIVE: To systematically review the current evidence regarding risk stratification of NMIBC. METHODS: The systematic review was performed in accordance with the PRISMA statement. Studies providing data on development and/or external validation cohorts of models and risk stratification tables for recurrence and/or progression for patients with NMIBC, reporting at least one discrimination measure (AUC or C-Index) were included. RESULTS: Twenty-five studies involving 22,737 patients were included. Six classifications were identified, three of them were predictive models (EORTC, CUETO, EAU 2021) and three were based on expert opinion (EAU 2020, AUA, NCCN). A high risk of bias was present in the majority of the studies. Certain heterogenicity was found among the studies regarding adjuvant therapy, postoperative instillation or second resection. The definition of oncological outcomes was not standardized in the included studies. CUETO and EORTC scoring systems are the most validated. In general, validations showed a poor discrimination capability to predict recurrence, slightly better for progression. The EAU 2021 model overestimates the risk of progression in patients treated with BCG. Carcinoma in situ is underrepresented in all the studies analyzed. 1 Equal contribution for first authorship. ∗ Correspondence to: Félix Guerrero-Ramos MD PhD, ROC Clinic, Paseo General Martı́nez Campos, 17, 28010 Madrid, Spain. E-mail: felixguerrero@gmail.com. ISSN 2352-3727 © 2022 – The authors. Published by IOS Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (CC BY-NC 4.0). 340 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review CONCLUSIONS: The existing classifications show poor discrimination capability for recurrence and possibly helpful discrimination capability for progression in NMIBC patients. These results highlight the unmet need to develop novel accurate risk models for patients with NMIBC, which could be improved with the combination of clinicopathological and molecular information. Keywords: Bladder cancer, recurrence, progression, risk stratification, systematic review, scoring, survival, calculator INTRODUCTION transurethral resection (TURBT) was not performed, the BCG maintenance schedule differed from the Non-muscle-invasive bladder cancer (NMIBC) current recommendation, and the World Health Orga- accounts for approximately 75% of all newly diag- nization (WHO) 2004/2016 grading classification nosed bladder cancers [1]. Despite the treatment system was not used. All of these preclude a potential based on transurethral resection of the bladder tumor lead-time bias and limit use of the aforementioned (TURBT) and intravesical therapy, patients with predictive models and risk stratifications in current NMIBC exhibit significant heterogeneity in out- clinical practice. comes, showing recurrence and progression rates as In this context, it is uncertain which of these tools is high as 50–70% and 10–40%, respectively [1]. These better for the purpose of risk stratification of NMIBC. data highlight the need for continuous and expen- Therefore, the present study aims to systematically sive surveillance, positioning bladder carcinoma as review the literature on the state of the art of risk the most expensive neoplasm per patient. stratification of patients with NMIBC. As is well known, intravesical therapy is impor- tant for the oncological control of NMIBC due to the improvement that is achieved in the recurrence and progression rates [1]. In this regard, there is evidence MATERIALS AND METHODS showing the superiority of bacillus Calmette-Guérin (BCG) immunotherapy over chemotherapy for this A systematic review was performed in accordance purpose [2]. Nevertheless, there are some concerns: with the Preferred Reporting Items for Systematic BCG immunotherapy has non-negligible toxicity; Reviews and Meta-Analyses statement (PRISMA) therefore, a proportion of patients withdraw from the [9]. The PICO (Population, Intervention, Compara- treatment early [3]. In addition, in the current era of tor, Outcome) approach was used to summarize our BCG shortage, demand for BCG exceeds its produc- research strategy (Table 1). The search was performed tion capacity, which makes its availability difficult; using the databases PubMed/Medline and Embase. thus, BCG should be reserved for patients who have A combination of the following keywords was a higher risk of recurrence and progression [4]. used: EORTC, CUETO, EAU, non-muscle-invasive For risk-adapted management of patients with bladder cancer, progression, recurrence, survival, risk NMIBC and to provide recommendations on adjuvant stratification, scoring system, scoring model, risk treatment and surveillance, different international tables, risk calculator. Boolean operators (AND, OR) urological organizations, such as the European were used to refine the search (Supplementary Mate- Organization for Research and Treatment of Can- rial). The search included articles published between cer (EORTC) [5], Club Urológico Español de January 2000 and January 2022 and was limited to Tratamiento Oncológico (CUETO) [6], the European English language studies. The automatic search was Association of Urology (EAU) [1], the American complemented by a manual search to identify rele- Urological Association (AUA) [7], and the National vant articles for the page-by-page examination of the Comprehensive Cancer Network (NCCN) [8], have last 10 years’ issues of the following journals: Euro- provided different evidence-based predictive mod- pean Urology, European Urology Focus, European els or expert opinion risk stratification. However, in Urology Oncology, The Journal of Urology, World studies performed for the construction of predictive Journal of Urology, British Journal of Urology Inter- models, the patients did not receive a single post- national, Urologic Oncology: Seminars and Original operative intravesical dose of chemotherapy, second Investigations, and Urology. F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 341 Table 1 PICO (Population, Intervention, Comparator, Outcome) approach Clinical question: How should NMIBC be stratified in 2022? Population: (1) Human, (2) adults, (3) patients with NMIBC Intervention: Risk stratification using risk models Comparison: EORTC risk tables vs CUETO risk tables vs AUA groups vs EAU groups (until 2020 and 2021 versions) vs NCCN groups Outcomes: Risk of recurrence and progression in patients with NMIBC based on existing scoring models Types of study: All available clinical, prospective, and retrospective comparative and non-comparative observational studies Databases searched: PubMed, Embase Keywords: CUETO and non-muscle-invasive bladder cancer or bladder neoplasm, EORTC and non-muscle-invasive bladder cancer or bladder neoplasm, CUETO risk tables and non-muscle-invasive bladder cancer or bladder neoplasm, EORTC risk tables and non-muscle-invasive bladder cancer or bladder neoplasm, CUETO score and non-muscle-invasive bladder cancer or bladder neoplasm, EORTC score and non-muscle-invasive bladder cancer or bladder neoplasm, EAU risk groups and non-muscle-invasive bladder cancer or bladder neoplasm, AUA risk groups and non-muscle-invasive bladder cancer or bladder neoplasm, EAU score and non-muscle-invasive bladder cancer or bladder neoplasm, Nomogram and non-muscle-invasive bladder cancer or bladder neoplasm, Scoring models and non-muscle-invasive bladder cancer or bladder neoplasm Manual search: Page-by-page examination in peer-reviewed journals Eligibility criteria: (1) development and/or external validation cohorts of models that assessed more than one clinicopathological factor for recurrence and/or progression for patients with NMIBC, (2) external validation for expert opinion risk stratification tables, and (3) reporting of at least one discrimination measure for the risk model [area under the curve (AUC) and/or c-index]. Exclusion criteria: (1) Studies including molecular signatures for NMIBC, (2) studies including biomarkers to improve the predictive performance of existing models Risk of bias assessment: PROBAST tool Inclusion criteria and study eligibility (4) probability of recurrence and/or progression of the cohort according to the risk group strata for Three reviewers (FGR, JDS, and ORF) screened the model assessed, and (5) discrimination measure all abstracts and full-text articles independently. Dis- (AUC and/or c-index) for each outcome assessed agreement was resolved by a discussion between (recurrence and/or progression) in the model. We them. Articles that fulfilled the following criteria used a predefined form for data extraction and any were included for the evidence synthesis: (1) devel- disagreements were resolved by discussion and con- opment and/or external validation cohorts of models sensus. that assessed more than one clinicopathological fac- tor for recurrence and/or progression for patients with Quality assessment NMIBC, (2) external validation for expert opinion risk stratification tables, and (3) reporting of at least The quality of the included studies was ana- one discrimination measure for the risk model [area lyzed independently by three reviewers (FGR, JDS, under the curve (AUC) and/or c-index]. We excluded and ORF) using the prediction model risk of bias studies with molecular features because they did assessment tool (PROBAST) [10]. We assessed the not include clinicopathological variables or did not reporting quality of predictive modeling studies using report any discrimination measure for the model. the Transparent Reporting of a multivariable predic- Case reports, editorials, letters, review articles, and tion model for Individual Prognosis Or Diagnosis meeting abstracts were not eligible and thus were also (TRIPOD) statement [11]. excluded during the systematic review process. Data analysis Data extraction A narrative synthesis of the included studies was The data were extracted by three reviewers inde- performed. Frequencies and percentages were used pendently (FGR, JDS, and ORF), who consensually to report categorical variables. Continuous data were established as extractable data: (1) authors, year of described using mean and standard deviation, or, publication, country, study design, sample size, num- alternatively, median and range/interquartile range. ber of institutions, name of the model assessed, (2) Crude rates and the probability of recurrence and clinicopathological and NMIBC management data progression of each cohort when patients were strat- of the cohort included in the risk model, (3) abso- ified using risk groups for each proposed model at lute recurrence and/or progression of the cohort, available time points were presented. 342 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review Fig. 1. In the case of overlapping data assessing two or article was judged to be at low RoB [6]. Overall high more risk models in different manuscripts using the RoB was mainly caused by high RoB in the analysis same cohort of patients, we selected the one with the domain (13/25 studies) [12–14, 17–19, 24, 26–28, 31, largest sample size for evidence synthesis. 32, 34], followed by a high (7/25 studies) [14, 15, 20, 22, 24, 29, 33] and unclear (11/25 studies) [5, 12, 16, RESULTS 17, 23, 26–28, 30–32] RoB in the outcomes domain. The other two domains contributed little to the overall Evidence synthesis high RoB (Fig. 2). Figure 1 shows the flow diagram of the studies. A Patient and study characteristics total of 3,079 references were found through the lit- erature search strategy (3,054 automatic search and Study characteristics are summarized in Table 2. 25 manual search), of which 838 were duplicates; the The 25 studies comprised a total of 22,737 patients. remaining 2,241 were screened by title and abstract. The geographical distribution was: 14 European stud- Of these, 2,206 were excluded and 70 were selected ies [5, 6, 15–17, 19–21, 23, 26, 30, 31, 33, 34], for full-text screening. Finally, 25 studies were con- 5 American studies [24, 25, 27–29], 4 Asian stud- sidered eligible for inclusion [5, 6, 12–34]. ies [14, 18, 22, 32], and 2 Brazilian studies [3, 12]. All analyses were retrospective. In the construction Quality assessment cohorts of the EORTC [5] and CUETO models [6], prospectively collected data from various clinical tri- Twenty of the 25 included studies were judged to als were used. Eleven studies were multi-institutional. be at high risk of bias (RoB) [5, 12–34], and only one Three studies consisted of prediction model develop- F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 343 Fig. 2. Assessment of the (a) risk of bias and (b) applicability according to PROBAST. ment without external validation [5, 6, 30] and 22 only patients with primary tumors were recruited) studies were external validations of existing models [20, 22, 25, 29–31]and 4,290 (23.9%) had recur- [12–29, 31–34]. rent tumors. The temporality of recurrent tumors was The gender of patients was specified in 23 of the documented in 13 of the studies (52%) [5, 12–15, 25 studies (92%) [5, 12–16, 18–34] involving 21,638 19, 21, 24, 26–28, 32, 33] involving 3,964 patients; patients, of whom 17,155 (79.3%) were male and 2,202 (55.5%) recurrences occurred before 1 year of 4,483 (20.7%), female. Age was described in 17 follow-up and 1,762 (44.5%) at more than 1 year of of the 25 studies (68%) [5, 6, 12–17, 21–30, 32, follow-up. The number of tumors could be extracted 34] involving 11,335 patients; 7,006 (61.8%) were in 23 of the studies (92%) [5, 6, 12–22, 24, 26–34] younger than 70 years and 4,329 (38.2%) were 70 involving 21,435 patients; 13,318 (62.1%) were soli- years or older. Regarding pathological features, the tary and 8,117 (37.9%), multiple. Tumor size could be primary or recurrent tumor status could be extracted extracted from 24 of the studies (96%) [5, 6, 12–22, from 22 of the studies (88%) [5, 12–15, 17–28, 30, 24–34] involving 21,109 patients; 15,073 (71.4%) 32, 34] involving 17,921 patients; 13,631 (76.1%) tumors were smaller than 3 cm and 6,036 (28.6%) of the patients had primary tumors (in six studies were 3 cm or larger. Tumor stage could be extracted 344 Table 2 Summary of baseline characteristics of the studies included in the systematic review Author/year Study design Enrollment No. of institutions Country No. of Follow-up Study type Type of prognostic Evaluated period patients (median; according to model prediction months) TRIPODS model F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review Sylvester, 2006 Retrospective 1979–1989 Multi-institutional Netherlands and 2,596 46.8 1ba Prediction model EORTC Belgium development Fernández-Gómez 2009 Retrospective 1990–1999 Multi-institutional Spain 69 1ba Prediction model CUETO development Pillai, 2011 Retrospective 1983–1985 Single institution UK 109 60 4 External validation EORTC Fernández-Gómez 2011 Retrospective 1990–1999 Multi-institutional Spain 1,062 69 4 External validation EORTC Hernández 2011 Retrospective 1998–2008 Single institution Spain 417 59 3 External validation EORTC Xylinas, 2013 Retrospective 2000–2007 Multi-institutional France, USA, 4,689 57 4 External validation EORTC, Germany, Italy, CUETO Sweden, Canada, Austria Xu, 2013 Retrospective 2003–2010 Single institution China 363 36 4 External validation EORTC, CUETO Vedder, 2014 Retrospective 1979–2012 Multi-institutional Denmark, 1,892 120 4 External validation EORTC, Netherlands, CUETO Spain Choi 2014 Retrospective 1985–2011 Single institution Korea 531 58 4 External validation EORTC, CUETO Almeida 2016 Retrospective 2003–2010 Single institution Brazil 63.6 4 External validation EORTC Busato 2016 Retrospective 2003–2010 Single institution Brazil 205 63.6 4 External validation EORTC Kilinç 2017 Retrospective 2002–2010 Single institution Turkey 348 55.25 3 External validation EORTC Ravvaz, 2017 Retrospective 2006–2016 Multi-institutional USA 1,333 37 4 External validation EORTC, CUETO, NCCN Dalkilic 2018 Retrospective 2007–2016 Single institution Turkey 400 60.6 4 External validation EORTC, CUETO Krajewski 2019 Retrospective 1986–2016 Multi-institutional Poland/Spain 414 68 3 External validation CUETO Leo 2019 Retrospective 1990–2014 Multi-institutional USA 1,404 25 3 External validation EORTC, (recurrence) NCCN 50 (progression) F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review Kim 2019 Retrospective 2000–2013 Single institution Korea 970 38 3 External validation EORTC Zhang, 2019 Retrospective 2003–2011 Single institution Germany 479 60 4 External validation EORTC, CUETO Jobczyk 2019 Retrospective 2005–2015 Single institution Poland 322 48 3 External validation EORTC, CUETO, EAU until 2020 Ravvaz, 2019 Retrospective 2006–2016 Multi-institutional USA 38 4 External validation AUA Fujita 2020 Retrospective 1993–2019 Single institution Japan 480 51 External validation EUA until 2020 Ritch, 2020 Retrospective 2001–2017 Single institution USA 398 37 4 External validation AUA Sylvester, 2021 Retrospective 1990–2018 Multi-institutional Netherlands, 3,401 46.8 1ba Prediction model EAU 2021 Spain, France, development Italy, Czech Republic, Germany, Austria, Canada Dovey 2021 Retrospective 1995–1996 Multi-institutional Sweden 395 123 4 External validation EAU 2021 Lobo 2021 Retrospective 2000–2018 Single institution USA 529 47.3 3 External validation EORTC, CUETO, EAU 2021 345 346 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review from all 25 studies [5, 6, 12–34] involving 22,737 Table 3 patients; 13,928 (61.25) were Ta and 8,809 (38.75) Clinicopathological features of the patients recruited through the systematic review were T1. According to the WHO 1973 grading sys- Variable No. (%) tem, tumor grade was extractable in 22 of the studies (88%) [5, 6, 12–24, 27, 30–34] involving 21,868 Gender Female 4,483 (20.7) patients; 15,901 (72.7%) tumors were G1/G2 and Male 17,155 (79.3) 5,967 (27.3%), G3. WHO 2004/2016 tumor grade Age was extractable in six studies (24%) [16, 25, 27–30] <70 years 7,006 (61.8) ≥70 years 4,329 (38.2) involving 6,052 patients; 3,501 (57.8%) tumors were Tumor status low grade (LG) and 2,551 (42.2%) were high grade Primary 13,631 (76.1) (HG). The presence of carcinoma in situ (CIS) was Recurrent 4,290 (23.9) communicated in 24 of the studies (96%) [5, 6, 12–17, Temporarity of recurrent tumors <1 year 2,202 (55.5) 19–34] involving 22,257 patients, of whom 1,540 ≥1 year 1,762 (44.5) (6.9%) showed the presence of associated CIS. The Number of tumors performance of re-TURBT could be extracted from Solitary 13,318 (62.1) ten studies (40%) [15, 16, 18, 23, 25, 27–30, 34] Multiple 8,117 (37.9) Tumor size involving 7,580 patients; re-TURBT was performed <3 cm 15,073 (71.4) in 1,862 (24.6%) patients and no studies described ≥3 cm 6,036 (28.6) the stage at re-TURBT (Table 3). Tumor stage Regarding intravesical therapy, single postopera- Ta 13,928 (61.2) T1 8,809 (38.8) tive chemotherapy instillation was reported in 11 of Tumor grade; WHO 1973 the studies (44%) [5, 15, 18, 19, 21, 23, 25–27, 32, G1/G2 15,901 (72.7) 33] involving 11,678 patients, among whom 4,820 G3 5,967 (27.3) (41.3%) received it. On the other hand, adjuvant BCG Tumor grade; WHO 2004 Low-grade 3,501 (57.8) immunotherapy instillation was reported in all stud- High-grade 2,551 (42.2) ies [5, 6, 12–34], with 5,333 (23.5%) patients having Concomitant CIS received BCG, of whom only 1,068 (4.7%) received Present 1,540 (6.9) Absent 20,717 (93.1) a maintenance schedule. Adjuvant chemotherapy Re-TURBT information could be extracted from 15 studies (60%) Yes 1,862 (24.6) [5, 12, 13, 15, 16, 19, 21, 22, 27–32, 34] involv- No 5,718 (75.4) ing 14,494 patients, of whom 5,423 (37.4%) received Intravesical therapy Single postoperative chemotherapy instillation 4,820 (41.3) adjuvant chemotherapy instillations. BCG immunotherapy 5,333 (23.5) The number of patients stratified according to Adjuvant chemotherapy 5,423 (37.4) the different models that could be extracted was as follows: EORTC model from 11 out of 25 studies (44%) [5, 12, 16, 17, 19, 21, 22, 24, 26, 27, 32], 12, 14–17, 19–21, 24, 26, 27, 32, 34] of the studies, n = 7,929; CUETO model from 5 of 25 studies (20%) and 1- and 5-year recurrence probabilities accord- [6, 16, 23, 27, 32], n = 3,567; EAU until 2020 from ing to CUETO risk groups were reported in nine 3 of 25 studies (12%) [18, 25, 30], n = 4,388; EAU (36.0%) studies [6, 14–16, 20, 23, 26, 27, 32, 34] from 2021 from 3 of 25 studies (12%) [16, 25, 30], (Tables 5 and 6). For EORTC risk groups, median n = 8,121; AUA from 3 of 25 studies (12%) [16, 28, 1-year recurrence probability was 6.7% in group I, 29], n = 2,090; and NCCN from 2 of 25 studies (8%) 19.0% in group II, 38.0% in group III, and 76.0% [24, 28], n = 2,894. Table 4 shows the stratification of in group IV. In EORTC risk groups, median 5-year patients through risk models. recurrence probability was 27.5% in group I, 49.0% in group II, 66.8% in group III, and 88.0% in group Oncological outcomes IV. For CUETO risk groups, median 1-year recur- rence probability was 8.6% in group I, 20.0% in Overall recurrence rate was reported in 24 (96%) of group II, 41.5% in group III, and 43.9% in group the 25 studies [5, 6, 12–29, 31–34]. Of 22,208 patients IV. In CUETO risk groups, median 5-year recurrence analyzed, 9,648 (43.4%) presented recurrence. One- probability was 38.5% in group I, 44.5% in group II, and 5-year recurrence probabilities according to 63.0% in group III, and 85.0% in group IV. One- and EORTC risk groups were reported in 14 (56.0%) [5, 5-year recurrence probabilities according to AUA risk F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 347 Table 4 Stratification of patients through risk models Scoring model Recurrence Progression EORTC (n = 7,929) 0 905 0 1,121 1–4 3,213 2–6 3,147 5–9 3,139 7–13 2,626 10–17 672 14–23 679 CUETO (n = 3,567) 0 –4 1,740 0–4 1,706 5–6 937 5–6 441 7–9 697 7–9 805 10–16 193 10–14 615 EAU until 2020 (n = 4,388) Low 663 Low 663 Intermediate 1,978 Intermediate 1,978 High 1,747 High 1,747 EAU 2021, WHO 1973 (n = 8,121) Low 1,036 Intermediate 1,913 High 758 Very high 89 EAU 2021, WHO 2004 (n = 8,121) Low 1,865 Intermediate 1,058 High 1,160 Very high 242 AUA (n = 2,090) Low 519 Low 519 Intermediate 778 Intermediate 778 High 793 High 793 NCCN (n = 2,894) TaLG 1,579 TaLG 1,579 TaHG 390 TaHG 390 T1 any 626 T1 any 626 CIS any 229 CIS any 229 groups were reported in three (12.0%) of the studies 14–16, 20, 23, 26, 27, 32, 34]. In EORTC risk groups, [16, 28, 29], and 1- and 5-year recurrence probabili- median 1-year progression probability was 0.0% in ties according to NCCN risk groups were reported in group I, 1.0% in group II, 4.0% in group III, and two (8.0%) of the studies [24, 28]. For AUA, 1-year 21.2% in group IV. For EORTC risk groups, median recurrence probability risk was 11.6% for low risk, 5-year progression probability was 0.0% in group I, 48.9% for intermediate risk, and 47.9% for high risk. 5.8% in group II, 18.5% in group III, and 36.5% in Five-year recurrence probability was 31.6% for low group IV. In CUETO risk groups, median 1-year pro- risk, 67.0% for intermediate risk, and 70.3% for high gression probability was 0.0% in group I, 3.0% in risk. In NCCN risk groups, 1-year recurrence proba- group II, 6.0% in group III, and 18.0% in group IV. bility was 20.0% for Ta low grade (TaLG), 33.0% for For CUETO risk groups, median 5-year progression Ta high grade (TaHG), 40.0% for T1, and 39.0% for probability was 3.5% in group I, 11.7% in group II, CIS. Five-year recurrence probability was 42.0% for 22.0% in group III, and 35.4% in group IV. There TaLG, 52.0% for TaHG, 51.0% for T1, and 56.0% for was information from two studies regarding EAU risk CIS. groups from 2020 and previously [25, 30]. Median 1- Overall progression rate was reported in 24 year progression probability was 0.0% for low risk, (96.0%) of the 25 studies [3, 5, 6, 12–17, 19, 21–34]. 1.0% for intermediate risk, 3.85% for high risk, and From 22,208 patients analyzed, 2,627 (11.8%) pre- 6.9% for very high risk. Moreover, median 5-year sented progression (Tables 5 and 6). One- and 5-year progression probability was 0.1% for low risk, 3.5% progression probabilities according to EORTC risk for intermediate risk, 9.7% for high risk, and 16.7% groups were reported in 14 (56.0%) of the studies [5, for very high risk. Information was available from 12, 14–17, 19–21, 24, 26, 27, 32, 34] of the studies, three studies regarding EAU 2021 WHO 1973 risk and 1- and 5-year recurrence probabilities according groups [16, 25, 30]. Median 1-year progression rate to CUETO risk groups were reported in nine (36.0%) was 0.4% for low risk, 2.0% for intermediate risk, studies [6, 14–16, 20, 23, 26, 27, 32, 34], and 1- and 5- 11.5% for high risk, and 28.4% for very high risk. year progression probabilities according to CUETO There was also information from three studies regard- risk groups were reported in nine (36.0%) studies [6, ing EAU 2021 WHO 2004/2016 risk groups [16, 25, 348 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review Table 5 Oncological outcome domains reported in the 25 included studies Studies Tumor-related outcomes Survival Recurrence Progression OMR OCSMR Almeida, 2016 x x x x Busato, 2016 x x x Choi, 2014 x x Dalkilic, 2018 x x Dovey, 2021 x x Fernández-Gómez, 2009 x x Fernández-Gómez, 2011 x x Fujita, 2020 x x x Hernández, 2011 x x Jobazyk, 2019 x x Kilinç, 2017 x x Kim, 2019 x x Krajewsky, 2019 x x Leo, 2019 x x Lobo, 2021 x Pillai, 2011 x x x Zheng, 2019 x x Xu, 2013 x x Vedder, 2014 x x x x Xylinas, 2013 x x Sylvester, 2006 x x x x Sylvester, 2021 x Ritch, 2020 x x Ravvaz, 2019 x x Ravvaz, 2017 x x OMR, Overall mortality rate; OCSMR, overall cancer-specific mortality rate. 30]. Median 1-year progression probability was 0.4% rate (OCSMR) was reported in five (20.0%) studies for low risk, 1.3% for intermediate risk, 3.5% for high [5, 12, 13, 30, 31]. Among 8,299 patients analyzed, risk, and 16.0% for very high risk. Median 5-year pro- OCSMR was 5.6% (467 patients). gression rate for EAU 2021 WHO 1973 was 1.3% for low risk, 5.6% for intermediate risk, 21.0% for high Validation and discrimination measures risk, and 51.0% for very high risk. Median 5-year progression probability was 1.5% for low risk, 4.1% We identified wide heterogeneity among the stud- for intermediate risk, 9.6% for high risk, and 40.0% ies with respect to reporting of validation and for very high risk. One- and 5-year progression prob- discrimination measures for NMIBC stratification, abilities according to AUA risk groups were reported the period of assessment, and the discrimination in three (12.0%) of the studies [16, 28, 29], and 1- and method used (c-index or AUC). 5-year progression probabilities according to NCCN For the EORTC scoring system [5], and with risk groups were reported in two (8.0%) studies [24, respect to recurrence prediction (Fig. 3), 6 of 25 28]. AUA 1-year progression risk was 0.1% for low studies (24%) [5, 12, 17, 20, 26, 27], with 5,627 risk, 3.0% for intermediate risk, and 20.0% for high patients, assessed c-index at 1 year, with a range of risk. Five-year recurrence probability was 1.1% for 0.62–0.72; eight studies (32%) [5, 12, 15–17, 20, 26, low risk, 11.6% for intermediate risk, and 44.3% for 27], with 6,422 patients, reported c-index at 5 years, high risk. In the case of NCCN risk groups, 1-year with a range of 0.59–0.777, and another eight stud- progression probability was 0.2% for TaLG, 0.9% ies (32%) [14, 21, 23, 24, 31–34] assessed c-index at for TaHG, 6.3% for T1, and 8.8% for CIS. Five-year final follow-up (median 25.2–120 months), including progression probability was 1.3% for TaLG, 4.7% for 10,120 patients, with a range of 0.597–0.817. Two of TaHG, 11.0% for T1, and 16.0% for CIS. 25 studies (8%) [19, 20], including 739 patients, ana- Overall mortality rate (OMR) was reported in six lyzed AUC at 1 year (Fig. 4), with a reported range of (24.0%) of the 25 studies [5, 12, 13, 18, 26, 31]. 0.557–0.61, while three studies (12%) [19, 20, 22], Among 5,491 patients analyzed, OMR was 28.3% including 1,709 patients, reported AUC at 5 years, (1,553 patients). Overall cancer-specific mortality with a range of 0.547–0.7, and another three studies F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 349 Table 6 Recurrence and progression probability (median) reported in the 25 included studies Recurrence Progression Probability (median) Probability (median) 1 yr (%) 5 yr (%) 1 yr (%) 5 yr (%) EORTC I 6.7 27.5 0 0 II 19 49 1 5.8 III 38 66.8 4 18.5 IV 76 88 21.2 36.5 CUETO I 8.6 38.5 0 3.5 II 20 44.5 3 11.7 III 41.5 63 6 22 IV 43.9 85 18 35.4 AUA Low 11.6 31.6 0.1 1.1 Interm. 48.9 67 3 11.6 High 47.9 70.3 20 44.3 NCCN TaLG 20 42 0.2 1.3 TaHG 33 52 0.9 4.7 T1 any 40 51 6.3 11 CIS any 39 56 8.8 16 EAU (previous) Low 0 0.1 Interm. 1 3.5 High 3.9 9.7 Very high 6.9 16.7 EAU 21 (WHO 1973) Low 0.4 1.3 Interm. 2 5.6 High 11.5 21.0 Very high 28.4 51 EAU 21 (WHO 2004/2016) Low 0.4 1.5 Interm. 1.3 4.1 High 3.5 9.6 Very high 16 40 EORTC, European Organization for Research and Treatment of Cancer; CUETO, Club Urológico Español de Tratamiento Oncológico; AUA, American Urological Association; NCCN, National Comprehensive Cancer Network; EAU, European Association of Urology; yr, year. Fig. 3. External validations with c-index for EORTC discrimination of recurrence risk (*refers to the model construction series). 350 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review Fig. 4. External validations with AUC for EORTC discrimination Fig. 6. External validations with AUC for EORTC discrimination of recurrence risk. of progression risk. (12%) [14, 19, 21], including 1,296 patients, analyzed reported AUC at overall follow-up (55.3–59 months), AUC at overall follow-up (median 55.3–59 months), with a range of 0.773–0.832 (Fig. 6). reporting a range of 0.773–0.832. For CUETO recurrence risk assessment tables [6], Regarding EORTC progression prediction (Fig. 5), we identified 3 of 25 studies (12%) [6, 23, 27], includ- 5 of 25 studies (20%) [5, 13, 17, 26, 27], with ing 2,809 patients, that reported c-index at 1 year, 5,305 patients, reported c-index at 1 year, with a with a range of 0.657–0.72; five studies (20%) [6, range of 0.65–0.86; 7 of 25 studies (28%) [5, 13, 15, 16, 23, 27], with 3,604 patients, analyzed c-index 15–17, 27], including 6,100 patients, reported the at 5 years, with a range of 0.56–0.705, and six stud- c-index at 5 years, with a range of 0.67–0.82, and ies (24%) [14, 23, 31–34], including 8,368 patients, another eight studies (32%) [14, 21, 23, 24, 31–34], reported c-index at overall follow-up (median 36–120 with 10,120 patients, analyzed the c-index at overall months), with a range of 0.516–0.836 (Fig. 7). Only follow-up (median 25.2–120 months), with a range Choi et al. [14] reported an AUC, with a value of 0.662–0.939. We identified only one study (4%) of 0.894, for CUETO recurrence risk tables, based [19], based on 417 patients, that reported an AUC for on 531 individuals with a median follow-up of 58 progression at 1 year, with a value of 0.58; two studies months. (8%) [19, 22], with a total of 765 patients, analyzed With respect to CUETO progression prediction, AUC at 5 years, reporting values of 0.55 and 0.70, and 3 of 25 studies (12%) [6, 23, 27], including 2,809 three studies (12%) [14, 19, 21], with 1,296 patients, patients, reported c-index for progression risk at 1 Fig. 5. External validations with c-index for EORTC discrimination of progression risk (*refers to the model construction series). F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 351 Fig. 7. External validations with c-index for CUETO discrimination of recurrence risk (*refers to the model construction series). Fig. 8. External validations with c-index for CUETO prediction of progression (*refers to the model construction series). year, with a range of 0.687–0.79; five studies (20%) overall series, the authors found a c-index of 0.639 [6, 15, 16, 23, 27], with 3,604 patients, reported c- and 0.631, respectively, for recurrence at 1 and 5 index at 5 years, with a range of 0.697–0.881, and years; when only patients treated with BCG were con- six studies (24%) [14, 23, 31–34], including 8,368 sidered, the c-index for recurrence at 1 and 5 years individuals, reported c-index at overall follow-up was 0.560 and 0.540, respectively. For progression, (36–120 months), with a range of 0.616–0.82 (Fig. 8). c-index at 1 and 5 years in the overall sample was As for recurrence, only Choi et al. [14] reported an 0.811 and 0.785, respectively, and for those individ- AUC, with a value of 0.724, for CUETO progression uals who received BCG it was 0.696 and 0.635 at 1 risk tables, based on 531 individuals with a median and 5 years, respectively. follow-up of 58 months. Regarding EAU 2021 risk groups, internal vali- EAU risk tables in their previous versions (until dation with 3,401 patients revealed a c-index of 0.80 2020) have been validated by only one group, who for 5-year progression prediction for both WHO 1973 performed a retrospective analysis of 322 patients and 2004/2016 grading systems in patients not treated with a median follow-up of 48 months [20]. In the with BCG [30]. Just Dovey et al. [16] externally val- 352 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review idated these data in their retrospective series of 295 2020, and NCCN) were risk groups based on expert Swedish patients (25.6% of patients under BCG, with opinion [1, 7, 8]. In summary, we found a poor dis- unspecified schedule), reporting a c-index for pro- crimination capability in most of the validations of gression at 5 years of 0.83, again for WHO 1973 and these risk models and the expert opinion risk groups. 2004/2016 grading systems. In addition, when BCG-treated patients are intro- For the AUA stratification system [7], only the duced into models such as EORTC or EAU 2021, group of Ravvaz et al. [28] reported an external recurrence and progression tend to be overestimated validation, based on retrospective data of 1,297 and their discrimination capability is impaired [17, patients; c-index for recurrence was 0.63, 0.61, 25, 33]. and 0.61 at 1-year, 5-year, and overall (median 38 The most validated predictive model is that from months) follow-up, respectively. In the same way, the EORTC [5]. The EORTC study included data of they reported a c-index for progression of 0.77 at 1 2,596 patients from seven EORTC phase III trials, but and 5 years and also at overall follow-up (median 38 less than 200 patients were treated with BCG (and no months). maintenance schedule was administered). That is why Finally, for NCCN risk tables [8], two studies (8%) validation of the EORTC tables by Fernández-Gómez [24, 28] based on retrospective data of 2,701 patients et al. [17] indicated that they were associated with an reported external validation; with a median follow-up overestimation of the risk of recurrence in all risk cat- of 38 and 47.3 months, these studies reported a c- egories and overestimation of the risk of progression index of 0.54 and 0.61 for recurrence and a c-index of in high-risk patients when they had been treated with 0.62 and 0.74 for progression. In their 1,297 patients, BCG. Similarly, the EORTC scoring system overes- Ravvaz et al. [24, 28] also found a c-index for recur- timated the risk of progression in the data analyzed rence of 0.61 and 0.56 at 1 and 5 years, respectively, in this systematic review [6, 17, 30]. According to and a c-index of 0.75 for progression at 1 year. the data reviewed, the actual progression rate among NMIBC patients receiving BCG therapy was lower than the probability of progression reported in the DISCUSSION EORTC risk tables, especially in the high-risk tumor category. NMIBC often progresses to MIBC after multiple In the case of AUA risk tables, progression was far recurrences, and NMIBCs differ in terms of their more likely in high-risk cases than in intermediate- tumor characteristics, including histological grade, T and low-risk cases, particularly in the first year [16]. stage, tumor number, tumor size, and tumor location Accordingly, the AUA risk groups’ discrimination in each recurrence [1]. Given this aspect of NMIBC, capability for progression, with a c-index of 0.77, multiple-event analytical techniques should be used. is comparable to that of the EORTC and CUETO Time-dependent covariate analysis is more accurate risk models, at 0.77 and 0.78 respectively. More- than time-fixed analysis [35], but the latter is used in over, the AUA risk groups also demonstrate a c-index most studies predicting the prognosis and outcome of similar to that of the NCCN risk groups, which NMIBC patients. is 0.79. The AUA low-risk group was comparable Based on the above considerations, risk tables for to the EORTC score in identifying patients with NMIBC have been designed to cover the clinical need a solitary, low-grade, <3 cm Ta lesion [16]. These for individualization and optimization of therapy for patients demonstrated a significant reduction in the NMIBC patients [1, 5–8, 30]. These risk tables are risk of recurrence (20% lower risk of recurrence at indeed the best currently available prognostic models, 5 years) compared with the intermediate- and high- and their use, as recommended by various guidelines, risk groups. Similarly, patients considered low risk has become the standard of care [1, 7, 8]. In this con- by the AUA risk tables showed only 1% probability text, external validation of these prognostic models is of NMIBC progression at 5 years and had a less than crucial to assess their discrimination capability, accu- 1% probability of progression prior to death. racy, and generalizability on more contemporaneous In the present systematic review, we found patient datasets. significant heterogeneity in the assessment of dis- In the present study, six classification systems crimination capability; therefore, a direct pooled were identified; three of them (EORTC, CUETO, and analysis of dataset predictive models and expert EAU 2021) were patient dataset-constructed predic- opinion-based risk groups was not possible. Both tive models [5, 6, 30], and three (AUA, EAU until EORTC [5] and CUETO [6] scores are the most F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 353 validated models and those that achieve the high- tive series of 295 patients in all stages of NMIBC, est discrimination capability for recurrence and 25.6% of whom were treated with BCG. By con- progression at overall follow-up; however, mod- trast, Lobo et al. [25] showed a c-index of only 0.64 est discrimination capability was reported when a for those who received adequate BCG therapy in time-dependent c-index/AUC (at 1 or 5 years) was a series of 529 BCG-treated patients. These results assessed. reflect the fact that the exclusion of patients treated When assessing the discrimination capability of with BCG from the EAU 2021 scoring model leads predictive models, an AUC or c-index of less than to the same phenomenon described for the EORTC 0.70 reflects poor performance [36]. In the present model [5], where only 7% of the patients were study, we found that Choi et al. [14] reported an AUC treated with BCG. Therefore, the EAU 2021 scor- of 0.894 for the CUETO scoring model and Killinç ing model overestimates the risk of progression, with et al. [21] reported a c-index of 0.817 for the EORTC consequent impairment of the discrimination capa- scoring model for recurrence outcome. However, in bility for progression in those patients treated with most of the external validations included for these BCG [25]. predictive models (13 studies) [5, 16, 17, 20, 21, 23, Bias was introduced in various ways in the included 25–27, 31–34], we found a poor discrimination capa- articles. Although assessment using the PROBAST bility (AUC or c-index under 0.7). Based on these tool [10] showed studies to be homogeneous with data, we can state that these two predictive models regard to the domains of participants and predictors, have a poor discrimination capability for recurrence several concerns regarding the risk of bias arise in the in patients with NMIBC. outcomes and analysis domains. Regarding progression, a better discrimination In most studies, the recurrence and progression capability was reported by Killinç et al. [21] for the outcomes were not defined, and in some studies, EORTC scoring model, with a c-index of 0.939, and patients with relapse of MIBC were recorded as both by Dalkilic et al. [15] for the CUETO scoring model, recurrence and progression, which could result in an with a c-index of 0.881. Furthermore, while in the overestimation of recurrences. In this regard, Kamat present review we identified five studies with an AUC et al. [37] recently published a consensus document or c-index under 0.7 [6, 19, 23, 26, 33], 12 studies in which these outcomes were specifically defined reported an AUC or c-index of at least 0.7 [13–16, to standardize the studies involving patients with 20–22, 27, 31, 32, 34], and in six cases the AUC or c- NMIBC. However, most of the studies included in this index exceeded 0.8 [14–16, 20, 28, 31]. These results systematic review were published before the afore- highlight that both models (EORTC and CUETO) mentioned study. seem to present a better discrimination capability Probably, the major concerns regarding the risk of for progression than for recurrence. This fact is rel- bias were observed in the analysis domain. Although evant for daily practice; moreover, it explains why the current recommendations for external validation progression has been the only outcome used for the studies of predictive models suggest that at least 100 construction of the updated EAU prognostic factor patients with the outcome need to be included in order risk groups [30]. to draw solid conclusions [10, 11], in most of the On the other hand, with the twin aim of updating included studies there was a low number of patients their previous risk groups based on expert opinion with the outcomes (recurrence or progression), which from the EORTC scoring model and incorporat- could have affected the conclusions. ing the WHO 2004/2016 tumor grading system, Other issues derived from this systematic review the EAU NMIBC guidelines panel has designed a are: new predictive model based on an individual patient Only EORTC, CUETO, and EAU 2021 are pre- data analysis involving 3,401 patients treated with dictive models constructed from a patient dataset TURBT with or without adjuvant chemotherapy [30]. [5, 6, 30]. NCCN, AUA, and EAU until 2020 clas- An important concern with respect to this model, sifications relied on a consensus of expert opinion however, is the exclusion of patients treated with [1, 7, 8]. In consequence, there are insufficient data BCG. and methodologies to perform calibration and statis- In a recent external validation of the updated EAU tical assessment for the discrimination performance 2021 model, Dovey et al. [16] showed a good dis- of these last ones. crimination capability (c-index 0.83 for both WHO The EAU21 is the only data-driven model that 73 and WHO 04/16 scoring models) in a retrospec- includes the WHO 1973 and WHO 2004/2016 grad- 354 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review ing classification systems, which could limit the fitted and be recognized from medical data, minimiz- use of the WHO 1973 classification. On the other ing the error. hand, the EAU21 included only patients with primary On the other hand, several researchers have tumors not treated with BCG, which could explain described a molecular signature that allows for pre- the drop of its accuracy in certain NMIBC patient diction of outcomes in patients with NMIBC. In this subpopulations. context, Hedegaard et al. [43] analyzed the tran- Patients with CIS of the bladder are underrepre- scription profile of 460 NMIBC (UROMOL project), sented in all these predictive models [5, 6, 30], in that managing to define three molecular groups: class 1, the proportion is below the described incidence [38, characterized by the expression of early cell cycle 39]. Furthermore, the described clinicopathological activation genes; class 2, characterized by the expres- features for patients with CIS of the bladder (such sion of late cell cycle activation genes; and class as focality, clinical subtype, and absence of papillary 3, which showed expression of CD44, KRT5, and tumor) [40] are not present in these models. Conse- KRT15. Classes 1 and 2 showed features of the quently, patients with CIS tend to be misclassified luminal subtype (luminal-like) while class 3 showed by these models. Despite the limitations in the risk features of the basal subtype (basal-like) described assessment of patients with CIS of the bladder, it for MIBC. is widely known that there is an important tendency More recently, Lindskrog et al. [44], in a multi- toward recurrence and progression in these patients; omic analysis that included 834 patients from the therefore, they must be considered as high/very high UROMOL project, validated the three previously risk until the development of a more adequate risk described classes, but in addition restratified class model. 2 into class 2a (characterized by significant chro- Some subpopulations of patients with the same mosomal instability, disruption of the cell signaling stage are classified into different categories by these pathway involving p53, and APOBEC mutations), classifications. In this regard, patients with a T1LG this being the class with the worst prognosis, with a would be intermediate risk according to the AUA recurrence rate of 1.11 recurrence/year, and class 2b guidelines [7] whereas any T1 would be considered (characterized by significant infiltration of immune high risk according to EAU until 2020 [1], and when cells and chromosomal instability). However, despite stratification is performed using updated EAU 2021 various efforts, these molecular signatures have not risk groups, the classification of T1 depends on sev- yet been validated in the clinical setting. eral other factors [30]. Finally, with regard to new therapies, a signif- icant number of clinical trials are using immune Future directions checkpoint inhibitors as adjuvant therapy for NMIBC, such as POTOMAC, CREST, ALBAN, and In the last decade, strategies to advance preci- KEYNOTE-676 (NCT03528694, NCT04165317, sion medicine through the development of novel NCT03799835, and NCT03711032). These trials are diagnostic and prognostic methods have attracted focusing on the setting of high-risk BCG-naïve dis- considerable investment. In this regard, recent data ease, and predicting the risk of these patients will be using a recalibration of the CUETO and EORTC a challenge in the near future. models through deep learning as an artificial intel- ligence (AI) strategy showed, in patients treated with BCG and mitomycin C, a c-index of 0.651 CONCLUSIONS and 0.660, respectively, for recurrence-free survival and a c-index of 0.881 and 0.885, respectively, for Both existing dataset-based prediction models progression-free survival [41]. The phenomenon of (EORTC, CUETO, and EAU 2021 scoring sys- improvement in discrimination capability for AI tem) and expert opinion risk groups (AUA, EAU models has been observed previously [42]. This is until 2020, and NCCN) show poor discrimina- attributable to the classical models being based on tion capability for disease recurrence and possibly linear regression statistics, as a consequence of which helpful discrimination capability for progression in non-linear relationships between predictive factors NMIBC patients. The EORTC overestimates the risk and the outcome may not be fitted. However, using of disease recurrence and progression, and EAU machine learning or deep learning, non-linear inter- 2021 model overestimates the risk of progression in actions between predictors and the outcome may be patients treated with BCG. F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 355 Our results highlight the unmet need to develop Biotech, Astellas (clinical trials); Pfizer, Lacer, Ipsen, novel accurate risk models for patients with NMIBC. Janssen, Combat Medical, Rovi, Recordati (travel Therefore, in the era of precision medicine, the expenses). combination of clinicopathological and molecular Felipe Couñago-Lorenzo declares the following information using AI could be the key to better predic- conflicts of interest: Janssen (sponsored research); tion of clinical outcomes that allows accurate clinical Janssen, AstraZeneca (advisory/consultancy); decision-making in NMIBC patients. Janssen, Roche, AstraZeneca, Astellas (speaker honoraria); AstraZeneca, Astellas, Janssen, Roche ACKNOWLEDGMENTS (travel expenses). José Daniel Subiela, Óscar Rodrı́guez-Faba, Julia The authors have no acknowledgments. Aumatell, Celeste Manfredi, Giorgio Bozzini and Javier Romero-Otero declare that they have no con- FUNDING flicts of interest. The authors report no funding. SUPPLEMENTARY MATERIAL AUTHOR CONTRIBUTIONS The supplementary material are available in the electronic version of this article: https://dx.doi.org/ Félix Guerrero-Ramos: conception, performance 10.3233/BLC-220055. of work, interpretation of data, writing the article. José Daniel Subiela: conception, performance of work, interpretation of data, writing the article. REFERENCES Óscar Rodrı́guez-Faba: conception, performance of work, interpretation of data, writing the article. [1] Babjuk M, Burger M, Capoun O, Cohen D, Comperat EM, Dominguez Escrig JL, et al. European Association of Urol- Julia Aumatell: performance of work, interpreta- ogy Guidelines on Non-muscle-invasive Bladder Cancer tion of data, writing the article. (Ta, T1, and Carcinoma in Situ). Eur Urol. 2022;81:75-94. Celeste Manfredi: interpretation of data, writing [2] Malmstrom PU, Sylvester RJ, Crawford DE, Friedrich M, Krege S, Rintala E, et al. An individual patient data the article. meta-analysis of the long-term outcome of randomised stud- Giorgio Bozzini: interpretation of data, writing the ies comparing intravesical mitomycin C versus bacillus article. Calmette-Guerin for non-muscle-invasive bladder cancer. Javier Romero-Otero: conception, writing the arti- Eur Urol. 2009;56:247-56. [3] Brausi M, Oddens J, Sylvester R, Bono A, van de Beek cle. C, van Andel G, et al. Side effects of Bacillus Calmette- Felipe Couñago-Lorenzo: conception, writing the Guerin (BCG) in the treatment of intermediate- and high- article. risk Ta, T1 papillary carcinoma of the bladder: results of the EORTC genito-urinary cancers group randomised phase 3 study comparing one-third dose with full dose and 1 year ETHICAL CONSIDERATIONS with 3 years of maintenance BCG. Eur Urol. 2014;65:69-76. [4] Perera M, Papa N, Christidis D, McGrath S, Manning T, As a systematic review of the literature, and as no Roberts M, et al. The impact of the global bacille Calmette- Guerin shortage on treatment patterns: population-based animal or human research was involved, our study data. BJU Int. 2018;121:169-72. is exempt from any requirement for Institutional [5] Sylvester RJ, van der Meijden AP, Oosterlinck W, Wit- Review Board approval. jes JA, Bouffioux C, Denis L, et al. Predicting recurrence and progression in individual patients with stage Ta T1 bladder cancer using EORTC risk tables: a combined anal- CONFLICT OF INTEREST ysis of 2596 patients from seven EORTC trials. Eur Urol. 2006;49:466-5; discussion 75-7. Félix Guerrero-Ramos declares the follow- [6] Fernandez-Gomez J, Madero R, Solsona E, Unda M, Martinez-Pineiro L, Gonzalez M, et al. Predicting nonmus- ing conflicts of interest: Bristol-Myers Squibb, cle invasive bladder cancer recurrence and progression in AstraZeneca, Janssen, Pfizer, Combat Medical patients treated with bacillus Calmette-Guerin: the CUETO (advisory/consultancy); Astellas, Janssen, Pfizer, scoring model. J Urol. 2009;182:2195-203. [7] Chang SS, Boorjian SA, Chou R, Clark PE, Daneshmand AstraZeneca, Combat Medical, Palex, Rovi (speaker S, Konety BR, et al. Diagnosis and Treatment of Non- honoraria); Pfizer, Combat Medical, Cepheid, Muscle Invasive Bladder Cancer: AUA/SUO Guideline. J AstraZeneca, Bristol-Myers Squibb, Janssen, IDL Urol. 2016;196:1021-9. 356 F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review [8] Flaig TW, Spiess PE, Agarwal N, Bangs R, Boorjian SA, [21] Kilinc MF, Bayar G, Dalkilic A, Sonmez NC, Arisan S, Buyyounouski MK, et al. Bladder Cancer, Version 3.2020, Guney S. Applicability of the EORTC risk tables to predict NCCN Clinical Practice Guidelines in Oncology. J Natl outcomes in non-muscle-invasive bladder cancer in Turkish Compr Canc Netw. 2020;18:329-54. patients. Turk J Urol. 2017;43:48-54. [9] Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, [22] Kim HS, Jeong CW, Kwak C, Kim HH, Ku JH. Novel nomo- Petticrew M, et al. Preferred reporting items for system- grams to predict recurrence and progression in primary atic review and meta-analysis protocols (PRISMA-P) 2015 non-muscle-invasive bladder cancer: validation of predic- statement. Syst Rev. 2015;4:1. tive efficacy in comparison with European Organization of [10] Moons KGM, Wolff RF, Riley RD, Whiting PF, West- Research and Treatment of Cancer scoring system. World J wood M, Collins GS, et al. PROBAST: A Tool to Urol. 2019;37:1867-77. Assess Risk of Bias and Applicability of Prediction Model [23] Krajewski W, Rodriguez-Faba O, Breda A, Pisano F, Poleta- Studies: Explanation and Elaboration. Ann Intern Med. jew S, Tukiendorf A, et al. Validation of the CUETO scoring 2019;170:W1-W33. model for predicting recurrence and progression in T1G3 [11] Moons KG, Altman DG, Reitsma JB, Ioannidis JP, urothelial carcinoma of the bladder. Actas Urol Esp (Engl Macaskill P, Steyerberg EW, et al. Transparent Reporting Ed). 2019;43:445-51. of a multivariable prediction model for Individual Progno- [24] Leo MC, McMullen CK, O’Keeffe-Rosetti M, Weinmann sis or Diagnosis (TRIPOD): explanation and elaboration. S, Garg T, Nielsen ME. External validation of the EORTC Ann Intern Med. 2015;162:W1-73. and NCCN bladder cancer recurrence and progression risk [12] Almeida GL, Busato WF, Jr., Ribas CM, Ribas JMF, De calculators in a U.S. community-based health system. Urol Cobelli O. External validation of EORTC risk scores to Oncol. 2020;38:39 e21-39 e27. predict recurrence after transurethral resection of brazilian [25] Lobo N, Hensley PJ, Bree KK, Nogueras-Gonzalez GM, patients with non -muscle invasive bladder cancer stages Ta Navai N, Dinney CP, et al. Updated European Associ- and T1. Int Braz J Urol. 2016;42:932-41. ation of Urology (EAU) Prognostic Factor Risk Groups [13] Busato Junior WF, Almeida GL, Ribas CA, Ribas Filho JM, Overestimate the Risk of Progression in Patients with De Cobelli O. EORTC Risk Model to Predict Progression Non-muscle-invasive Bladder Cancer Treated with Bacillus in Patients With Non-Muscle-Invasive Bladder Cancer: Is Calmette-Guerin. Eur Urol Oncol. 2022;5:84-91. It Safe to Use in Clinical Practice? Clin Genitourin Cancer. [26] Pillai R, Wang D, Mayer EK, Abel P. Do standardised 2016;14:176-82. prognostic algorithms reflect local practice? Application [14] Choi SY, Ryu JH, Chang IH, Kim TH, Myung SC, of EORTC risk tables for non-muscle invasive (pTa/pT1) Moon YT, et al. Predicting recurrence and progression of bladder cancer recurrence and progression in a local cohort. non-muscle-invasive bladder cancer in Korean patients: a ScientificWorldJournal. 2011;11:751-9. comparison of the EORTC and CUETO models. Korean J [27] Ravvaz K, Walz ME, Weissert JA, Downs TM. Pre- Urol. 2014;55:643-9. dicting Nonmuscle Invasive Bladder Cancer Recurrence [15] Dalkilic A, Bayar G, Kilinc MF. A Comparison of EORTC and Progression in a United States Population. J Urol. And CUETO Risk Tables in Terms of the Prediction of 2017;198:824-31. Recurrence and Progression in All Non-Muscle-Invasive [28] Ravvaz K, Weissert JA, Downs TM. American Urologi- Bladder Cancer Patients. Urol J. 2019;16:37-43. cal Association Nonmuscle Invasive Bladder Cancer Risk [16] Dovey Z, Pfail J, Martini A, Steineck G, Dey L, Renstrom Model Validation-Should Patient Age be Added to the Risk L, et al. Bladder Cancer (NMIBC) in a population-based Model? J Urol. 2019;202:682-8. cohort from Stockholm County with long-term follow-up; [29] Ritch CR, Velasquez MC, Kwon D, Becerra MF, Soodana- A comparative analysis of prediction models for recur- Prakash N, Atluri VS, et al. Use and Validation of the rence and progression, including external validation of AUA/SUO Risk Grouping for Nonmuscle Invasive Blad- the updated 2021 E.A.U. model. Urol Oncol. 2022;40:106 der Cancer in a Contemporary Cohort. J Urol. 2020;203: e1-e10. 505-11. [17] Fernandez-Gomez J, Madero R, Solsona E, Unda M, [30] Sylvester RJ, Rodriguez O, Hernandez V, Turturica D, Martinez-Pineiro L, Ojea A, et al. The EORTC tables over- Bauerova L, Bruins HM, et al. European Association of estimate the risk of recurrence and progression in patients Urology (EAU) Prognostic Factor Risk Groups for Non- with non-muscle-invasive bladder cancer treated with bacil- muscle-invasive Bladder Cancer (NMIBC) Incorporating lus Calmette-Guerin: external validation of the EORTC risk the WHO 2004/2016 and WHO 1973 Classification Systems tables. Eur Urol. 2011;60:423-30. for Grade: An Update from the EAU NMIBC Guidelines [18] Fujita N, Hatakeyama S, Momota M, Tobisawa Y, Panel. Eur Urol. 2021;79:480-8. Yoneyama T, Yamamoto H, et al. Impact of substratifica- [31] Vedder MM, Marquez M, de Bekker-Grob EW, Calle ML, tion on predicting oncological outcomes in patients with Dyrskjot L, Kogevinas M, et al. Risk prediction scores for primary high-risk non-muscle-invasive bladder cancer who recurrence and progression of non-muscle invasive blad- underwent transurethral resection of bladder tumor. Urol der cancer: an international validation in primary tumours. Oncol. 2020;38:795 e9-e17. PLoS One. 2014;9:e96849. [19] Hernandez V, De La Pena E, Martin MD, Blazquez C, Diaz [32] Xu T, Zhu Z, Zhang X, Wang X, Zhong S, Zhang M, et al. FJ, Llorente C. External validation and applicability of the Predicting recurrence and progression in Chinese patients EORTC risk tables for non-muscle-invasive bladder cancer. with nonmuscle-invasive bladder cancer using EORTC and World J Urol. 2011;29:409-14. CUETO scoring models. Urology. 2013;82:387-93. [20] Jobczyk M, Stawiski K, Fendler W, Rozanski W. Validation [33] Xylinas E, Kent M, Kluth L, Pycha A, Comploj E, Svatek of EORTC, CUETO, and EAU risk stratification in predic- RS, et al. Accuracy of the EORTC risk tables and of the tion of recurrence, progression, and death of patients with CUETO scoring model to predict outcomes in non-muscle- initially non-muscle-invasive bladder cancer (NMIBC): A invasive urothelial carcinoma of the bladder. Br J Cancer. cohort analysis. Cancer Med. 2020;9:4014-25. 2013;109:1460-6. F. Guerrero-Ramos et al. / NMIBC Risk Stratification: A Systematic Review 357 [34] Zhang G, Steinbach D, Grimm MO, Horstmann M. [40] Subiela JD, Faba OR, Aumatell J, Krajewski W, Calderon J, Utility of the EORTC risk tables and CUETO scoring Parada R, et al. Impact of clinical and pathological subtypes model for predicting recurrence and progression in non- of carcinoma in situ (CIS) of the bladder: Lessons learned muscle-invasive bladder cancer patients treated with routine from long-term follow-up of a series of CIS patients treated second transurethral resection. World J Urol. 2019;37: with BCG. Urol Oncol. 2022;40:9 e-e17. 2699-705. [41] Jobczyk M, Stawiski K, Kaszkowiak M, Rajwa P, Rozanski [35] Ngwa JS, Cabral HJ, Cheng DM, Pencina MJ, Gagnon DR, W, Soria F, et al. Deep Learning-based Recalibration of the LaValley MP, et al. A comparison of time dependent Cox CUETO and EORTC Prediction Tools for Recurrence and regression, pooled logistic regression and cross sectional Progression of Non-muscle-invasive Bladder Cancer. Eur pooling with simulations and an application to the Framing- Urol Oncol. 2022;5:109-12. ham Heart Study. BMC Med Res Methodol. 2016;16:148. [42] Cho BJ, Kim KM, Bilegsaikhan SE, Suh YJ. Machine learn- [36] Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Dev- ing improves the prediction of febrile neutropenia in Korean ereaux PJ, et al. Discrimination and Calibration of Clinical inpatients undergoing chemotherapy for breast cancer. Sci Prediction Models: Users’ Guides to the Medical Literature. Rep. 2020;10:14803. JAMA. 2017;318:1377-84. [43] Hedegaard J, Lamy P, Nordentoft I, Algaba F, Hoyer [37] Kamat AM, Sylvester RJ, Bohle A, Palou J, Lamm DL, S, Ulhoi BP, et al. Comprehensive Transcriptional Anal- Brausi M, et al. Definitions, End Points, and Clinical Trial ysis of Early-Stage Urothelial Carcinoma. Cancer Cell. Designs for Non-Muscle-Invasive Bladder Cancer: Recom- 2016;30:27-42. mendations From the International Bladder Cancer Group. [44] Lindskrog SV, Prip F, Lamy P, Taber A, Groeneveld J Clin Oncol. 2016;34:1935-44. CS, Birkenkamp-Demtroder K, et al. An integrated multi- [38] Subiela JD, Palou J, Esquinas C, Fernandez Gomez JM, omics analysis identifies prognostic molecular subtypes Rodriguez Faba O. Clinical usefulness of random biopsies of non-muscle-invasive bladder cancer. Nat Commun. in diagnosis and treatment of non-muscle invasive bladder 2021;12:2301. cancer: Systematic review and meta-analysis. Actas Urol Esp (Engl Ed). 2018;42:285-98. [39] Subiela JD, Rodriguez Faba O, Aumatell J, Calderon J, Mer- cade A, Balana J, et al. Contemporary outcomes of bladder carcinoma in situ treated with an adequate bacille Calmette- Guerin immunotherapy. BJU Int. 2022;129:542-50.