European Radiology (2021) 31:1471–1481 https://doi.org/10.1007/s00330-020-07175-z COMPUTED TOMOGRAPHY Clinical pre-test probability for obstructive coronary artery disease: insights from the European DISCHARGE pilot study Sarah Feger 1 & Paolo Ibes 1 & Adriane E. Napp 1 & Alexander Lembcke 1 & Michael Laule 1 & Henryk Dreger 1 & Björn Bokelmann 1 & Gershan K. Davis 2,3 & Giles Roditi 4 & Ignacio Diez 5 & Stephen Schröder 6 & Fabian Plank 7 & Pal Maurovich-Horvat 8 & Radosav Vidakovic 9 & Josef Veselka 10 & Malgorzata Ilnicka-Suckiel 11 & Andrejs Erglis 12 & Teodora Benedek 13 & José Rodriguez-Palomares 14,15 & Luca Saba 16 & Klaus F. Kofoed 17 & Matthias Gutberlet 18 & Filip Ađić 19 & Mikko Pietilä 20 & Rita Faria 21 & Audrone Vaitiekiene 22 & Jonathan D. Dodd 23 & Patrick Donnelly 24 & Marco Francone 25 & Cezary Kepka 26 & Balazs Ruzsics 27 & Jacqueline Müller-Nordhorn 28 & Peter Schlattmann 29 & Marc Dewey 1,30,31 Received: 26 November 2019 / Revised: 3 June 2020 / Accepted: 10 August 2020 / Published online: 9 September 2020 # The Author(s) 2020 Abstract Objectives To test the accuracy of clinical pre-test probability (PTP) for prediction of obstructive coronary artery disease (CAD) in a pan-European setting. Methods Patients with suspected CAD and stable chest pain who were clinically referred for invasive coronary angiography (ICA) or computed tomography (CT) were included by clinical sites participating in the pilot study of the European multi-centre DISCHARGE trial. PTP of CAD was determined using the Diamond-Forrester (D+F) prediction model initially introduced in 1979 and the updated D+F model from 2011. Obstructive coronary artery disease (CAD) was defined by one at least 50% diameter coronary stenosis by both CT and ICA. Results In total, 1440 patients (654 female, 786 male) were included at 25 clinical sites from May 2014 until July 2017. Of these patients, 725 underwent CT, while 715 underwent ICA. Both prediction models overestimated the prevalence of obstructive CAD (31.7%, 456 of 1440 patients, PTP: initial D+F 58.9% (28.1–90.6%), updated D+F 47.3% (34.2–59.9%), both p < 0.001), but overestimation of disease prevalence was higher for the initial D+F (p < 0.001). The discriminative ability was higher for the updated D+F 2011 (AUC of 0.73 95% confidence interval [CI] 0.70–0.76 versus AUC of 0.70 CI 0.67–0.73 for the initial D+F; p < 0.001; odds ratio (or) 1.55 CI 1.29–1.86, net reclassification index 0.11 CI 0.05–0.16, p < 0.001). Conclusions Clinical PTP calculation using the initial and updated D+F prediction models relevantly overestimates the actual prevalence of obstructive CAD in patients with stable chest pain clinically referred for ICA and CT suggesting that further refinements to improve clinical decision-making are needed. Trial registration https://www.clinicaltrials.gov/ct2/show/NCT02400229 Key Points • Clinical pre-test probability calculation using the initial and updated D+F model overestimates the prevalence of obstructive CAD identified by ICA and CT. • Overestimation of disease prevalence is higher for the initial D+F compared with the updated D+F. • Diagnostic accuracy of PTP assessment varies strongly between different clinical sites throughout Europe. Keywords Coronary artery disease . Computed tomography angiography . Probability of disease . Prevalence Sarah Feger and Paolo Ibes contributed equally to this work. Abbreviations Electronic supplementary material The online version of this article AUC The area under the receiver-operating-characteristic (https://doi.org/10.1007/s00330-020-07175-z) contains supplementary curve material, which is available to authorized users. CAD Coronary artery disease * Marc Dewey CT Computed tomography dewey@charite.de D+F Diamond-Forrester ICA Invasive coronary angiography Extended author information available on the last page of the article 1472 Eur Radiol (2021) 31:1471–1481 pCRF Paper-based case report forms With Stable Chest Pain and Intermediate Risk of Coronary PTP Pre-test probability Artery Disease’ (DISCHARGE) trial (www.dischargetrial. eu; FP7 2007-2013, EC-GA 603266; trial registration https://www.clinicaltrials.gov/ct2/show/NCT02400229) [18]. Introduction The DISCHARGE trial compares the effectiveness of invasive versus non-invasive coronary angiography. Patients In clinical routine, coronary artery disease (CAD) can be di- were prospectively included by 25 clinical sites across Europe agnosed by using invasive and non-invasive diagnostic tests. from 16 European countries. Choosing the appropriate and most beneficial diagnostic test The DISCHARGE pilot study was conducted in patients for each patient is highly important, [1] since invasive and clinically referred for either CT or ICA. Both examinations non-invasive tests have different advantages and disadvan- were performed at the Department of Radiology/Cardiology at tages. Invasive coronary angiography (ICA) enables invasive each of the 25 clinical sites. The decision for either diagnostic therapeutic measures in the same session, but is associated test was made in an outpatient setting, and the patients were with a small risk of procedural complications [2, 3]. These then clinically referred to elective CT or ICA. Before alloca- risks can be avoided in clinical practice by using non- tion to the subsequent clinical test, the clinical parameters for invasive tests instead [4]. Coronary computed tomography PTP calculation were assessed in all patients. The complete (CT) angiography is a safe non-invasive diagnostic test [5] recruitment period was about 3 years, extending from April with a very high negative predictive value, making it especial- 2014 to July 2017. The mean recruitment period per clinical ly valuable for ruling out obstructive CAD [6–8]. The calcu- site was 8 months (8.1 ± 4.8 months), ranging from 1 to 17 lation of pre-test probability (PTP) of disease can facilitate months. selecting patients with stable angina pectoris for the most ben- eficial diagnostic procedure [9]. The current European Inclusion criteria Guidelines recommend the selection of the appropriate diag- nostic test according to the individual patient’s CAD proba- Inclusion and exclusion criteria were relatively broad. Patients bility [10, 11]. Thus, the correct assessment of the likelihood were eligible for inclusion into the non-randomised of CAD is important to select patients for the diagnostic pro- DISCHARGE pilot study if they had a routinely scheduled cedure with the highest expected clinical benefit [12]. clinical examination for suspected CAD and stable chest Multiple PTP calculators exist and have been established in discomfort/chest pain and were at least 30 years of age. clinical practice. The Diamond-Forrester (D+F) PTP calcula- Patients with known CAD including a history of prior myo- tion model, which is recommended by the current European cardial infarction or revascularization were not included. guidelines for patients with stable chest pain, was introduced Patients were excluded from the study if they were not in sinus as early as 1979 (initial D+F) and is based on three clinical rhythm, were pregnant, or required haemodialysis. parameters that have to be assessed in patients (the patient’s age, the type of angina presentation, and the patient’s gender) Study outcomes [13]. Studies have shown that the original D+F model overestimated the prevalence of CAD [14, 15]. The original The purpose of the pilot study was to acquire information on D+F was statistically updated by Genders et al in 2011 to the routine clinical practice of CTA and ICA at different clin- improve the prediction of obstructive CAD especially in ical sites. In this paper, we investigate the accuracy of PTP women and patients with atypical angina presentation (up- calculation of obstructive CAD using the Diamond-Forrester dated D+F 2011) being clinically referred to ICA [16]. The (D+F) prediction models in patients clinically referred to CT purpose of this study was to analyse the accuracy of PTP or ICA. Each of the 25 clinical sites planned to include at least calculation of CAD using the original and updated D+F pre- 60 patients routinely scheduled for the examinations (30 CTA diction models in patients with stable chest pain with a clinical and 30 ICA). referral for either CT or ICA in a multi-centre pan-European setting [17]. Ethics The ethics approval for the main DISCHARGE trial (EA1/ Materials and methods 294/13) included the pilot study. Depending on local site re- quirements for data acquisition, written and/or oral informed Study design and patient recruitment consent was obtained from all patients participating in the DISCHARGE pilot study. The study participants did not un- This study was performed as part of the pilot study of the dergo any follow-up examinations or investigational proce- European multi-Diagnostic Imaging Strategies for Patients dures. The DISCHARGE project is funded by the EU-FP7 Eur Radiol (2021) 31:1471–1481 1473 Framework Programme (FP7 2007-2013, EC-GA 603266, Assessment of obstructive CAD EC-GA 603266) but the clinical sites did not receive any funding for the pilot study which was an own contribution Obstructive CAD was defined by ICA and CT as the pres- by all. Only research staff at the coordinator site received ence of at least one 50% coronary artery diameter stenosis funding for coordinating the pilot study. on a per-patient level. As CT has a high sensitivity/ specificity for the detection of significant CAD, we used CT results equally as ICA results in this study [6]. If ICA Study conduct was performed after clinically indicated CT, the ICA result was selected for the final analysis of the accuracy of PTP, The ICA and CT examinations had to be performed according since ICA is the reference standard for the diagnosis of to local standards. Vendor-specific CT protocols and a 10-step CAD. All examinations were evaluated locally at each guide for CT and ICA were finalised by the DISCHARGE clinical site by qualified and trained readers for CT and consortium and distributed to all involved staff members at all cardiologists for ICA according to their local standard of clinical sites. In addition, we performed two cardiac CT work- care, e.g. by using quantitative coronary analysis. shops in Berlin for the physicians involved in the examina- tions at all clinical sites to create a comparable basis of CT reading abilities for all participating centres [19]. Statistical analysis Assessment of pre-test probabilities Values are given as arithmetic mean (standard deviation), as median (interquartile range; IQR), or as number of patients Patients were included independently of their estimated or cal- (percentages). We performed the statistical analysis by culated disease probability. PTP of all included patients were using SPSS version 20 and R 3.4.4 [21]. A p value of assessed by comparing the initial D+F model introduced in ≤ 0.05 was defined to indicate statistical significance. The 1979 and the updated D+F model from 2011 [13, 16]. Both dependent t test for paired samples was used for the com- models of pre-test calculation of CAD probability include only parison of continuous variables. To compare pared dichot- three clinical parameters that had to be assessed in all patients: omous variables, we performed McNemar’s test. We used the patient’s age, the type of angina presentation, and the pa- the area under the receiver-operating-characteristic curve tient’s gender (Fig. 1) [20]. The typicality of angina presenta- (AUC) to compare the discriminative power of the two tion was based on the presence of the following three criteria: pre-test prediction models. To compare the two models in retrosternal localisation of pain, precipitation by exertion, and terms of their diagnostic accuracy, we performed a logistic prompt relief by rest or after nitroglycerin. [20] Dependent on regression analysis with the outcome (CAD or no CAD) as a how many criteria were fulfilled patients were allocated to one dependent variable and the respective PTP and the method of the following categories: typical angina (all 3 criteria pres- of computation as a predictor. In order to take care of the ent), atypical angina (2 of 3 criteria), non-anginal chest discom- variability between sites, we applied a random intercept for fort (1 of 3 criteria) or other chest discomfort (no criteria ful- site and correlation between observations. In addition, we filled). PTP for all patients at all sites was calculated by the performed a net reclassification analysis using 50% as a coordinating site based on the information provided in the CRF. cutoff value for the PTP. Fig. 1 Three clinical parameters for pre-test probability estimation 1474 Eur Radiol (2021) 31:1471–1481 Results 86 years. The male-to-female ratio was slightly higher in the ICA group with 59% male patients (419 male and 296 female Study population patients) than in the CT group with 51% male patients (367 male and 358 female patients). It was planned to include 1523 patients (see flowchart in Fig. 2 for further details). Sixty of these patients were retrospectively excluded. The most frequent reason was Examination results and accuracy of pre-test proba- a lack of adherence to the inclusion and exclusion criteria. bility prediction Thus, 1463 eligible patients were included in the study and underwent CT or ICA according to clinical referral. The overall prevalence of obstructive CAD was 31.7% Nevertheless, 23 patients had non-diagnostic examina- (456 of 1441 patients; Figure 3). In detail, the prevalence tions and were excluded. For the final analysis, 1440 pa- was higher in the ICA group (45.1%; 322 of 715 patients) tients were included (654 female and 786 male patients, compared with the CT group (18.5%; 134 of 725 patients; 45% and 55%, respectively). In this final population, 725 p < 0.001), and the overall prevalence ranged between patients underwent CT (358 female, 367 male) and 715 15.3% and 49.2% among the 25 clinical sites underwent ICA (296 female, 419 male). (Supplementary material, Online Figure 1). In the ICA group, the prevalence at the individual clinical sites ranged between 23.3% and 76.7%, and between 3.4% Patient characteristics and 34.5% in the CT group. The prevalence of obstructive CAD identified by ICA The number of patients with typical chest pain was higher in and CT (31.7%, 456 of 1440 patients) was overestimated the ICA group than in the CT group with 53% (378 of 715) by both prediction models (PTP: initial D+F 58.9% (28.1– versus 35% (253 of 725; Table 1). There were more patients 90.6%) and updated D+F 47.3% (34.2–59.9%), both with atypical angina/non-anginal chest pain or other chest p < 0.001; Fig. 3). Comparison of both prediction models discomfort in the CT group (65%) compared with those in showed overestimation of CAD to be slightly less for the the ICA group (47%). The mean age of patients was 64 years updated D+F compared with the initial version (p < 0.001). in the ICA group (63.8 ± 9.7 years; Table 1) and 59 years in The PTP calculated with the initial D+F version was 51% in the CT group (58.7 ± 11.3 years) with an overall range of 30– the CT group (median 54.4% (18.6–79.4%)) versus 65% in Fig. 2 Flowchart of the pilot study. A total of 1463 patients were eligible for study inclusion. For the final analysis, we included 1440 patients (with 654 female and 786 male patients) Eur Radiol (2021) 31:1471–1481 1475 Table 1 Distribution of clinical parameters in the ICA and CT ICA group (N = 715) CT group (N = 725) group Symptoms Typical angina pectoris 378 (52.9%) 253 (34.9%) * Atypical angina pectoris 166 (23.2%) 247 (34.1%) * Non-anginal chest discomfort 142 (19.9%) 194 (26.8%) * Other chest discomfort 29 (4.1%) 31 (4.3%) Gender Female 296 (41.4%) 358 (49.3%) Male 419 (58.6%) 367 (50.6%) Age Median (IQR), year 64 (57–72) 58 (51–67) The number of patients with typical chest pain was higher in the ICA group than in the CT group (53% versus 35%). More patients in the CT group presented with atypical angina or non-anginal chest pain than in the ICA group (61% versus 43%). The mean age of patients was higher in the ICA group than in the CT group (64 versus 59 years). The male-to-female ratio was slightly higher in the ICA group compared with that in the CT group. *Statistical significance the ICA group (median 69.7% (32.4–92%)). In both groups, Discriminative ability and NET reclassification analysis the PTP of CAD was significantly lower when calculated with the updated D+F version: 43% in the CT group (medi- The discriminative ability was stronger for the updated D+F an 41.4% (29.8–53.2%)) and 53% in the ICA group (median 2011 with an area under the receiver operating curve of 0.73 51.8% (40.6–65.4%)). Thus, both pre-test calculators (AUC; 95% confidence interval [CI] 0.70–0.76; Fig. 4) com- overestimated the actual prevalence of obstructive CAD of pared with an AUC of 0.70 (CI 0.67–0.73) for the initial D+F 45% in the ICA group and 19% in the CT group (p < 0.001). prediction model (p < 0.001). The discriminative ability dif- The overestimation was higher for the initial D+F calcula- fered between the 25 clinical sites (Supplementary material, tion compared with the updated D+F version in both the CT Online Figure 2). The logistic regression analysis with random and the ICA group (p < 0.001) and was higher in the CT intercept for the site showed a significant effect of the method group compared with that in the ICA group (p < 0.001) for with a coefficient equal to 0.44. This corresponds to an odds both prediction models. ratio of 1.55 (95% CI 1.29–1.86; Table 2). Thus, this result is Fig. 3 Pre-test probability and coronary artery disease (CAD) prevalence. by both prediction models with a higher overestimation of disease A total of 725 patients underwent CT, and 715 underwent ICA. The prevalence for the initial version compared with the updated D+F (p < average prevalence of obstructive CAD in the entire population was 0.001). This overestimation again was higher in the CT group compared 32% with a higher prevalence in the ICA group (45%) versus the CT with that in the ICA group (p < 0.001) group (19%). The actual disease probability was relevantly overestimated 1476 Eur Radiol (2021) 31:1471–1481 CAD but with a PTP of ≥ 50% to a PTP < 50% compared with the initial D+F model. Discussion The results of this multi-centre European study show the following: 1. The PTP calculated by the D+F model (initial and statis- tically updated version) relevantly overestimates the actu- al prevalence of CAD in patients clinically referred for ICA and CT with stable chest pain. 2. The updated D+F performs slightly better than the initial D+F. 3. Overestimation is higher in patients clinically referred to Fig. 4 Comparison of the discriminative power of pre-test probability CT than in those clinically referred to ICA. prediction models. The discriminative ability was higher for the updated 4. There is tremendous variability in the diagnostic accuracy D+F 2011 compared with the initial D+F prediction model. The AUC, which is a parameter of the discriminative ability, was 0.73 for the of PTP assessment between different clinical sites which updated D+F (AUC; 95% confidence [CI] interval 0.70–0.76). The were trained in the basic concept of PTP assessment and initial D+F had an AUC of 0.70 (95% CI 0.67–0.73; p < 0.001) evaluation of chest pain type. in favour of the updated D+F model. There is also substantial variability between sites measured by the variance of the ran- Interpretation of the results in the clinical context dom intercepts with a value of 0.14. As a result of the NET reclassification analysis, there is In this multi-centre clinical trial, we compared the CAD PTP an improvement due to the updated D+F model (NRI cat- estimation of the initial D+F with the updated D+F in a non- egorical 0.11 95% CI (0.05–0.16); p < 0.001; Table 3). selective pragmatic cohort of both patients referred on clinical Thus, using a cutoff value of 50% PTP for CAD, the up- grounds to CTA as well as ICA. Our trial shows that the statis- dated D+F model particularly reclassifies patients without tically updated D+F has a slightly higher discriminative ability Table 2 Logistic regression analysis with random effects Random effects Groups name Variance Standard deviation Clinical site (intercept) 0.1371 0.3703 Number of observations, 2880; groups: site, 25 Fixed effects Estimate Standard error Z value Pr (>|z|) (Intercept) − 2.734 0.16884 − 16.192 < 2e–16*** Pre-test probability 3.04813 0.19998 15.243 < 2e–16*** Method (original D+F or 0.43833* 0.09312 4.707 2.51e–06*** updated D+F 2011) Correlation of fixed effects (Intercept) Pre-test probability Pre-test probability − 0.818 Method (original D+F − 0.566 0.392 or updated D+F 2011) We performed a logistic regression analysis with the outcome (prevalence; CAD or no CAD) as dependent and the respective pre-test probability and the method of computation as predictors using the stacked data set (method original D+F versus updated D+F 2011). In order to take care of the variability between sites, we applied a random intercept for site. There is a significant effect of the method with a coefficient equal to 0.44, corresponding to an odds ratio of 1.55 (95% CI 1.29, 1.86). Significance codes: 0 = ‘***’, 0.001 = ‘**’, 0.01 = ‘*’, 0.05 = ‘.’ 0.1 = ‘ ’ 1; *Statistical significance Eur Radiol (2021) 31:1471–1481 1477 Table 3 NET reclassification index Updated D+F 2011 (n) % Reclassified Outcome: no CAD (n = 984) Pre-test probability < 50% ≥ 50% Initial D+F < 50% 381 41 10 ≥ 50% 266 296 47 Outcome: CAD (n = 456) Pre-test probability < 50% ≥ 50% Initial D+F < 50% 66 23 26 ≥ 50% 79 288 22 Combined data Pre-test probability < 50% ≥ 50% Initial D+F < 50% 477 64 13 ≥ 50% 345 584 37 NRI (categorical) [95% CI], 0.1059 [0.0532–0.1585]; p value, 0.00008 Net reclassification analysis using 50% as a cutoff value for the pre-test probability. As a result, it turns out that there is an improvement due to the Genders model (NRI categorical 0.11 95% CI (0.05–0.16); p < 0.001); n number of patients and therefore tends to estimate the CAD probability more ac- There are only few other papers that have assessed the curately than the initial D+F [13, 16]. For patients routinely accuracy of the D+F model to patients referred for CTA. scheduled for ICA, the clinical estimation of PTP was more They all have in common to include only patients with exact than for patients referred for CTA. Possibly, the reason low PTP being clinically assigned to CT but do not com- is not primarily the diagnostic test itself for which the patients pare with patients clinically referred to ICA. A study of are scheduled, but the clinical presentation which resulted in the Wasfy et al [24] was in an American cohort including decision to refer the patients for either diagnostic test [22]. In patients being referred only for CTA based on a low our patient population, the prevalence was higher for the pa- PTP. A more recent study evaluated 3 scores among pa- tients scheduled for ICA than for the CT patients. In this non- tients with suspected CAD in the CTA randomised arm of randomised study setting, this is logical from the clinical point the SCOT-HEART study for the outcome of obstructive of view, since ICA offers the possibility of subsequent treat- CAD by coronary CTA [25]: the modified D+F, CAD ment with angioplasty and coronary stenting. Consortium clinical score (CAD2), and CONFIRM risk Our results show variable prevalence of obstructive CAD score (CRS). They found that the best calibrator of ob- in patients routinely scheduled for ICA and CTA for the 25 structive CAD was the updated D+F, which goes in line clinical sites included in our analysis. Since the prevalence of with the results of our study. Another study has shown disease influences the accuracy of the applied clinical tests, it PTP calculation according to D+F to overestimate PTP of is highly important to know the local disease prevalence [10]. CAD in a multi-centre study setting with patients being As our study sites are spread across Europe, multiple factors also only referred to CT not ICA [14]. This was shown in affect the local disease prevalence and discriminative ability. our study as well. However, in our patient collective, pa- Overestimation of PTP of CAD will lead subsequently to in- tients were referred to both, CTA and ICA, based on clin- creased downstream diagnostic testing which increases the ical estimation. The study population of Cheng et al was possibility of adverse events and costs for the health care also characterised by a relatively low prevalence of dis- system. As recent studies have shown [23], the prevalence ease of 15%. The recently published study of Foldyna of obstructive CAD is relatively low in patients electively et al showed overestimation of the D+F model in a large referred to ICA/CT to evaluate stable chest pain. multi-centric cohort from the PROMISE trial [26]. This is in accordance with the results of our study. Again, in Comparison with other studies comparison with our study, the study of Foldyna et al only included patients being randomised to CTA. This study includes a large-scale prospective European cohort Thus, the analysis of our European DISCHARGE cohort with both CTA and ICA being the combined gold standard in provides a unique opportunity to make comparisons between a non-specified patient cohort. Recent scientific data on the pre-test models for patients referred for both CTA and ICA in clinical application of PTP calculation in patients with 25 European sites providing performance analysis for both suspected CAD are rare. models in a non-selective pragmatic cohort. 1478 Eur Radiol (2021) 31:1471–1481 Strengths and limitations More accurate clinical prediction tools are needed to optimize clinical decision-making for the diagnostic management of This multi-centre study included patients from clinical sites patients with suspected CAD. across Europe. Our patient population is very robust, includ- ing patients from 25 different clinical sites, with a high overall Acknowledgements We thank all patients for participating in this pilot study of the DISCHARGE trial. Without the help of the research staff at patient number of more than 1000 patients, and an almost all 25 clinical sites, this study would not have been possible. We are similar male-to-female ratio in both the CT and ICA group. thankful to Bettina Herwig for copy editing. This pilot study was planned to prepare the subsequent randomised controlled trial. Therefore, patients were routinely Funding Open Access funding provided by Projekt DEAL. The sent for CT and ICA examinations per clinical indication. DISCHARGE project is funded by the EU-FP7 Framework Programme (FP7 2007-2013, EC-GA 603266, EC-GA 603266) but the clinical sites However, this study was not designed as a controlled did not receive any funding for the pilot study which was an own contri- randomised trial. bution by all. Only the research staff at the coordinator site received In this study, we used both CT and ICA examination as the funding for coordinating the pilot study. diagnostic gold standard. This is due to the fact that various clinical studies have proven the high diagnostic accuracy of Compliance with ethical standards coronary CT angiography and its high negative predictive value [27]. In our study population, the clinical presentation Guarantor The scientific guarantor of this publication is Prof. Marc significantly differed between the CT and ICA group. Due to Dewey. the design of the D+F model, patient gender and age are part Conflict of interest Sarah Feger: Dr. Feger reports grants from European of the PTP calculation and these differences do not reduce the Commission, other from Vital Images, other from AG Mednet, during the validity of our patient collective, but reflect routine clinical conduct of the study; grants from Siemens Medical Solutions; grants from decision-making. In this study, we did not acquire detailed GE Healthcare; grants from Toshiba Medical Systems; and grants from information on the patients’ medical histories (e.g. diabetes, Philips Medical Systems, outside the submitted work. Paolo Ibes: Dr. Ibes reports grants and other from European arterial hypertension, smoking status, family history of CAD), Commission, from Vital Images, and from AG Mednet, during the con- CT/ICA indicators of CAD (e.g. coronary calcium score, left duct of the study; grants from Siemens Medical Solutions; grants from ventricular function, myocardial perfusion, wall motion), or GE Healthcare; grants from Toshiba Medical Systems; and grants from adverse events in the follow-up. Thus, we were not able to Philips Medical Systems, outside the submitted work. Adriane E. Napp: Dr. Napp reports grants from European perform a more detailed analysis to assess the differences in Commission, other from Vital Images, and other from AG Mednet, dur- the prevalence of obstructive CAD; we observed between the ing the conduct of the study; grants from Siemens Medical Solutions; different clinical sites or using other tests for estimation of pre- grants from GE Healthcare; grants from Toshiba Medical Systems; and test probability (e.g. SCORE or the extended D+F model). As grants from Philips Medical Systems, outside the submitted work. Alexander Lembcke: Dr. Lembcke reports grants from European the current European guidelines recommend the D+F model Commission, other from Vital Images, and other from AG Mednet, dur- statistically updated by Genders et al in order to select patients ing the conduct of the study; grants from Siemens Medical Solutions; for further diagnostic tests, we decided to request only clinical grants from GE Healthcare; grants from Toshiba Medical Systems; grants data for the fast and intuitive initial and updated D+F estima- from Philips Medical Systems, and personal fees from PAREXEL International GmbH, outside the submitted work. tion model. Michael Laule: Dr. Laule has nothing to disclose. Both the AUCs of the D+F models and the prevalence of Henryk Dreger: Dr. Dreger has nothing to disclose. significant CAD, which reflects post-test probability, varied Björn Bokelmann: Mr. Bokelmann reports grants from European strongly among the European sites participating in the study. Commission, other from Vital Images, and other from AG Mednet, dur- ing the conduct of the study; grants from Siemens Medical Solutions; grants from GE Healthcare; grants from Toshiba Medical Systems; and grants from Philips Medical Systems, outside the submitted work. Conclusions Gershan K. Davis: Dr. Davis reports grants from the European Commission, during the conduct of the study. Giles Roditi: Dr. Roditi reports grants from the European This study demonstrates that the initial and updated D+F Commission, during the conduct of the study. models relevantly overestimate the PTP of CAD compared Ignacio Diez: Dr. Diez Gonzalez reports grants from the European with its actual prevalence in patients routinely selected for Commission, during the conduct of the study. both CT and ICA. The updated D+F model was slightly more Stephen Schröder: Dr. Schröder reports grants from the European Commission, during the conduct of the study. accurate than the initial D+F version. The prevalence of ob- Fabian Plank: Dr. Plank reports grants from the European structive CAD differs between clinical sites in Europe. Thus, Commission, during the conduct of the study. in order to choose the most beneficial diagnostic test for pa- Pal Maurovich-Horvat: Dr. Maurovich-Horvat reports grants from the tients as recommended by the European guidelines, the over- European Commission, during the conduct of the study. Radosav Vidakovic: Dr. Vidakovic reports grants from the European estimation of the actual prevalence of CAD and differences in Commission, during the conduct of the study. prevalence among European countries need to be considered. Eur Radiol (2021) 31:1471–1481 1479 Josef Veselka: Dr. Veselka reports grants from the European Balazs Ruzsics: Dr. Ruzsics reports grants from the European Commission, during the conduct of the study. Commission, during the conduct of the study. Malgorzata Ilnicka-Suckiel: Dr. Ilnicka-Suckiel reports grants from Jacqueline Müller-Nordhorn: Dr. Müller-Nordhorn reports grants the European Commission, during the conduct of the study. from the European Commission, during the conduct of the study. Andrejs Erglis: Dr. Erglis reports grants from European Commission, Peter Schlattmann: Dr. Schlattmann reports grants from the European during the conduct of the study; grants from Abbott Vascular; grants from Commission, during the conduct of the study. Boston Scientific; personal fees from Abbott Vascular; personal fees from Marc Dewey: Dr. Dewey reports grants from the European Boston Scientific; personal fees from Biotronik; personal fees from Commission, other from Vital Images, other from AG Mednet, during Biosensors; personal fees from Cordis; and personal fees from the conduct of the study; grants from Siemens Medical Solutions; grants Medtronik, outside the submitted work. from GE Healthcare; grants from Toshiba Medical Systems; grants from Teodora Benedek: Dr. Benedek reports grants from European Philips Medical Systems; and grants from Heisenberg Programme of Commission, during the conduct of the study; grants from Romanian DFG, outside the submitted work. Ministry of European Funds; the Romanian Government and the European Union; and grants from Romanian Ministry of European Statistics and biometry One of the authors (PM) has significant statis- Funds, the Romanian Government, and the European Union, outside tical expertise. the submitted work. José Rodriguez-Palomares: Dr. Rodriguez-Palomares reports grants Informed consent Depending on local site requirements for data acqui- from the European Commission, during the conduct of the study. sition, written and/or oral informed consent was obtained from all patients Luca Saba: Dr. Saba reports grants from the European Commission, participating in the DISCHARGE pilot study. The study participants did during the conduct of the study; personal fees from Springer; and personal not undergo any follow-up examinations or investigational procedures. fees from Francis and Taylor, outside the submitted work. Klaus Kofoed: Dr. Fuglsang Kofoed reports grants from European Commission, during the conduct of the study; grants from AP Møller Ethical approval The ethics approval for the main DISCHARGE trial og hustru Chastine McKinney Møllers Fond; grants from The John and (EA1/294/13) included the pilot study. Birthe Meyer Foundation; grants from Research Council of Rigshopitalet; grants from The University of Copenhagen; grants from Methodology The Danish Heart Foundation; grants from The Lundbeck Foundation; • prospective grants from The Danish Agency for Science, Technology and Innovation • non-randomised clinical trial by The Danish Council for Strategic Research; grants from CATCH-2 • multicenter study trial; grants from CSub320 trial; and grants from Toshiba Medical Corporation, other from Toshiba Medical Corporation, outside the sub- Open Access This article is licensed under a Creative Commons mitted work. Attribution 4.0 International License, which permits use, sharing, Matthias Gutberlet: Dr. Gutberlet reports grants from European adaptation, distribution and reproduction in any medium or format, as Commission, during the conduct of the study; grants from Bayer long as you give appropriate credit to the original author(s) and the Healthcare; grants and other from Bayer Healthcare; grants from source, provide a link to the Creative Commons licence, and indicate if Bracco; and grants from Philips; other from Philips Medical Systems, changes were made. The images or other third party material in this article from Siemens Medical Solutions, other from Siemens Medical are included in the article's Creative Commons licence, unless indicated Solutions, outside the submitted work. otherwise in a credit line to the material. If material is not included in the Filip Ađić: Dr. Ađić reports grants from the European Commission, article's Creative Commons licence and your intended use is not during the conduct of the study. permitted by statutory regulation or exceeds the permitted use, you will Mikko Pietilä: Dr. Pietila reports grants from European Commission, need to obtain permission directly from the copyright holder. To view a during the conduct of the study; personal fees from AstraZeneca, personal copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. fees from Novartis, personal fees from MSD, non-financial support from Bayer, and non-financial support from B. Braun, outside the submitted work. Rita Faria: Dr. Faria reports grants from the European Commission, during the conduct of the study; other from the Portuguese Ministry of Health, outside the submitted work. References Audronė Vaitiekienė Dr. Vaitiekiene reports grants from the European Commission, during the conduct of the study. 1. Pryor DB, Shaw L, McCants CB et al (1993) Value of the history Jonathan Dodd: Dr. Dodd reports grants from European Commission, and physical in identifying patients at increased risk for coronary during the conduct of the study; personal fees from co-author of the artery disease. Ann Intern Med 118:81–90 StatDx Cardiovascular textbook, 2nd Ed; other from Associate editor 2. Almdahl SM, Veel T, Halvorsen P, Rynning SE (2013) Immediate for Radiology, European Radiology, QJM and Respirology, outside the rescue operations after failed diagnostic or therapeutic cardiac cath- submitted work. eterization procedures. Interact Cardiovasc Thorac Surg 17:314– Patrick Donnelly: Dr. Donnelly reports grants from the European 317 Commission, during the conduct of the study; grants from Southeastern 3. Shimony A, Joseph L, Mottillo S, Eisenberg MJ (2011) Coronary Health and Social Care Trust Innovation Research Development Group artery perforation during percutaneous coronary intervention: a sys- Fund, outside the submitted work. tematic review and meta-analysis. Can J Cardiol 27:843–850 Marco Francone: Dr. Francone reports grants from the European 4. Moscariello A, Vliegenthart R, Schoepf UJ et al (2012) Coronary Commission, during the conduct of the study; personal fees from Bayer CT angiography versus conventional cardiac angiography for ther- Medical Imaging and personal fees from Bracco, outside the submitted apeutic decision making in patients with high likelihood of coro- work. nary artery disease. Radiology 265:385–392 Cezary Kepka: Dr. Kepka reports grants from the European 5. Williams MC, Hunter A, Shah ASV et al (2016) Use of coronary Commission, during the conduct of the study. computed tomographic angiography to guide management of pa- tients with coronary disease. J Am Coll Cardiol 67:1759–1768 1480 Eur Radiol (2021) 31:1471–1481 6. Dewey M, Rief M, Martus P et al (2016) Evaluation of computed 17. (1994) Randomised trial of cholesterol lowering in 4444 patients tomography in patients with atypical angina or chest pain clinically with coronary heart disease: the Scandinavian Simvastatin Survival referred for invasive coronary angiography: randomised controlled Study (4S). Lancet 344:1383–1389 trial. BMJ 355:i5441 18. Napp AE, Haase R, Laule M et al (2017) Computed tomography 7. Pugliese F, Mollet NR, Runza G et al (2006) Diagnostic accuracy of versus invasive coronary angiography: design and methods of the non-invasive 64-slice CT coronary angiography in patients with pragmatic randomised multicentre DISCHARGE trial. Eur Radiol stable angina pectoris. Eur Radiol 16:575–582 27:2957–2968 8. Dharampal AS, Papadopoulou SL, Rossi A et al (2012) Computed 19. Zimmermann E, Germershausen C, Greupner J et al (2010) tomography coronary angiography accuracy in women and men at Improvement of skills and knowledge by a hands-on cardiac CT low to intermediate risk of coronary artery disease. Eur Radiol 22: course: before and after evaluation with a validated questionnaire 2415–2423 and self-assessment. Rofo 182:589–593 9. Pryor DB, Harrell FE Jr, Lee KL, Califf RM, Rosati RA (1983) 20. Diamond GA (1983) A clinically relevant classification of chest Estimating the likelihood of significant coronary artery disease. Am discomfort. J Am Coll Cardiol 1:574–575 J Med 75:771–780 21. Team RC (2017) R: A language and environment for statistical 10. Montalescot G, Sechtem U, Achenbach S et al (2013) 2013 ESC computing. https://www.R-project.org guidelines on the management of stable coronary artery disease: the 22. Eagle KA (1991) Medical decision making in patients with chest Task Force on the management of stable coronary artery disease of pain. N Engl J Med 324:1282–1283 the European Society of Cardiology. Eur Heart J 34:2949–3003 23. Adamson PD, Newby DE, Hill CL, Coles A, Douglas PS, Fordyce 11. Moschovitis A, Cook S, Meier B (2010) Percutaneous coronary CB (2018) Comparison of international guidelines for assessment interventions in Europe in 2006. EuroIntervention 6:189–194 of suspected stable angina: insights from the PROMISE and SCOT- 12. Knuuti J, Ballo H, Juarez-Orozco LE et al (2018) The performance HEART. JACC Cardiovasc Imaging 11:1301–1310 of non-invasive tests to rule-in and rule-out significant coronary 24. Wasfy MM, Brady TJ, Abbara S et al (2012) Comparison of the artery stenosis in patients with stable angina: a meta-analysis fo- Diamond-Forrester method and Duke Clinical Score to predict ob- cused on post-test disease probability. Eur Heart J 39:3322–3330 structive coronary artery disease by computed tomographic angiog- 13. Diamond GA, Forrester JS (1979) Analysis of probability as an aid raphy. Am J Cardiol 109:998–1004 in the clinical diagnosis of coronary-artery disease. N Engl J Med 25. Baskaran L, Danad I, Gransar H et al (2018) A comparison of the 300:1350–1358 updated Diamond-Forrester, CAD consortium, and CONFIRM 14. Cheng VY, Berman DS, Rozanski A et al (2011) Performance of history-based risk scores for predicting obstructive coronary artery the traditional age, sex, and angina typicality-based approach for disease in patients with stable chest pain: the SCOT-HEART estimating pretest probability of angiographically significant coro- Coronary CTA Cohort. JACC Cardiovasc Imaging. https://doi. nary artery disease in patients undergoing coronary computed to- org/10.1016/j.jcmg.2018.02.020 mographic angiography: results from the multinational coronary 26. Foldyna B, Udelson JE, Karady J et al (2018) Pretest probability for CT angiography evaluation for clinical outcomes: an international patients with suspected obstructive coronary artery disease: re- multicenter registry (CONFIRM). Circulation 124(2423-2432): evaluating Diamond-Forrester for the contemporary era and clinical 2421–2428 implications: insights from the PROMISE trial. Eur Heart J 15. Zhou J, Liu Y, Huang L et al (2017) Validation and comparison of Cardiovasc Imaging. https://doi.org/10.1093/ehjci/jey182 four models to calculate pretest probability of obstructive coronary 27. Schuetz GM, Zacharopoulou NM, Schlattmann P, Dewey M artery disease in a Chinese population: a coronary computed tomo- (2010) Meta-analysis: noninvasive coronary angiography using graphic angiography study. J Cardiovasc Comput Tomogr 11:317– computed tomography versus magnetic resonance imaging. Ann 323 Intern Med 152:167–177 16. Genders TS, Steyerberg EW, Alkadhi H et al (2011) A clinical prediction rule for the diagnosis of coronary artery disease: valida- Publisher’s note Springer Nature remains neutral with regard to jurisdic- tion, updating, and extension. Eur Heart J 32:1316–1330 tional claims in published maps and institutional affiliations. Eur Radiol (2021) 31:1471–1481 1481 Affiliations Sarah Feger 1 & Paolo Ibes 1 & Adriane E. Napp 1 & Alexander Lembcke 1 & Michael Laule 1 & Henryk Dreger 1 & Björn Bokelmann 1 & Gershan K. Davis 2,3 & Giles Roditi 4 & Ignacio Diez 5 & Stephen Schröder 6 & Fabian Plank 7 & Pal Maurovich-Horvat 8 & Radosav Vidakovic 9 & Josef Veselka 10 & Malgorzata Ilnicka-Suckiel 11 & Andrejs Erglis 12 & Teodora Benedek 13 & José Rodriguez-Palomares 14,15 & Luca Saba 16 & Klaus F. Kofoed 17 & Matthias Gutberlet 18 & Filip Ađić 19 & Mikko Pietilä 20 & Rita Faria 21 & Audrone Vaitiekiene 22 & Jonathan D. Dodd 23 & Patrick Donnelly 24 & Marco Francone 25 & Cezary Kepka 26 & Balazs Ruzsics 27 & Jacqueline Müller-Nordhorn 28 & Peter Schlattmann 29 & Marc Dewey 1,30,31 1 Charité – Universitätsmedizin Berlin, Humboldt-Universität and 16 Department of Radiology, Azienda Ospedaliero Universitaria di Freie Universität zu Berlin, Berlin, Germany Cagliari, Cagliari, Italy 2 17 Department of Cardiology, Aintree University Hospital, Department of Cardiology and Radiology, Rigshospitalet, Liverpool, UK University of Copenhagen, Copenhagen, Denmark 3 18 University of Central Lancashire, Liverpool, UK Department of Diagnostic and Interventional Radiology, 4 UNIVERSITY LEIPZIG –Heart Center Leipzig, Leipzig, Germany Institute of Cardiovascular and Medical Sciences, Glasgow 19 University, Glasgow, UK Department of Cardiology, Institute for Cardiovascular Diseases of 5 Vojvodina, Faculty of Medicine, University of Novi Sad, Novi Department of Cardiology, Basurto University Hospital Bilbao, Sad, Serbia Bilbao, Spain 20 6 Turku PET Centre and Heart Centre, Turku University Hospital, Department of Cardiology, ALB FILS KLINIKEN, Turku, Finland Goeppingen, Germany 21 7 Department of Cardiology, Centro Hospitalar de Vila Nova de Gaia, Department of Radiology and Department of Cardiology, Medical Vila Nova de Gaia, Portugal University Innsbruck, Innsbruck, Austria 22 8 Department of Cardiology, Lithuanian University of Health Department of Radiology, Medical Imaging Centre, Semmelweis Sciences, Kaunas, Lithuania University, Budapest, Hungary 23 Department of Radiology, St. Vincent’s University hospital, School 9 Department of Cardiology, Clinical Hospital Center “Zemun”, of Medicine, University College Dublin, Dublin, Ireland Faculty of Medicine, University of Belgrade, Zemun, Belgrade, 24 Serbia Department of Cardiology, Southeastern Health and Social Care 10 Trust, Belfast, Ireland Department of Cardiology, Motol University Hospital and 2nd 25 School of Medicine, Charles University, Prague, Czech Republic Department of Radiological, Pathological and Oncological 11 Sciences, Sapienza University of Rome, Rome, Italy Department of Cardiology, Wojewodzki Szpital Specjalistyczny 26 We Wroclawiu, Wrocław, Poland Dept. of Coronary and Structural Heart Diseases, Institute of 12 Cardiology, Warsaw, Poland Latvian Centre of Cardiology, Pauls Stradins Clinical University 27 Hospital, Riga, Latvia Royal Liverpool and Broadgreen University Hospital, 13 Liverpool, UK Department of Cardiology, Cardio Med Medical Center Targu- 28 Mures, Târgu Mureș, Romania Berlin School of Public Health Berlin, Berlin, Germany 14 29 Hospital Universitari Vall d´Hebron, Department of Cardiology. Institut für Statistik, Medizinische Informatik, Datenwissenschaften Vall d’Hebron Institut de Recerca (VHIR). Universitat Autònoma Universitätsklinikum Jena, Leipzig, Germany de Barcelona, Barcelona, Spain 30 Charité—Universitätsmedizin Berlin Department of Radiology, 15 Centro de Investigación Biomédica en Red-CV, CIBER CV, Berlin Institute of Health, Berlin, Germany Barcelona, Spain 31 DZHK (German Centre for Cardiovascular Research), partner site Berlin, Germany