ORIGINAL RESEARCH published: 14 July 2022 doi: 10.3389/fcvm.2022.895917 Development and Validation of a Personalized, Sex-Specific Prediction Algorithm of Severe Atheromatosis in Middle-Aged Asymptomatic Individuals: The ILERVAS Study Marcelino Bermúdez-López 1 , Manuel Martí-Antonio 1 , Eva Castro-Boqué 1 , María del Mar Bretones 1 , Cristina Farràs 2,3 , Gerard Torres 4,5 , Reinald Pamplona 6 , Albert Lecube 7,8 , Dídac Mauricio 8,9 , José Manuel Valdivielso 1* and Elvira Fernández1 on behalf of the ILERVAS Project Collaborators Edited by: 1 Grupo de Investigación Translacional Vascular y Renal, IRBLleida, Red de Investigación Renal (RedInRen-ISCIII), Lleida, Corey Kalbaugh, Spain, 2 Centre d’Atenció Primària Cappont, Gerència Territorial de Lleida, Institut Català de la Salut, Barcelona, Spain, Clemson University, United States 3 Research Support Unit Lleida, Fundació Institut Universitari per a la Recerca a l’Atenció Primària de Salut Jordi Gol i Gorina Reviewed by: (IDIAPJGol), Barcelona, Spain, 4 Departament de Medicina Respiratòria, Hospital Universitari Arnau de Vilanova, Grup Jennifer Mancio, Recerca Translational Medicina Respiratòria, IRBLleida, Universitat de Lleida, Lleida, Spain, 5 CIBER de Enfermedades University of Porto, Portugal Respiratorias (CIBERES), Madrid, Spain, 6 Departament de Medicina Experimental, IRBLleida, Universitat de Lleida, Lleida, Xavier Rossello, Spain, 7 Departament d’Endocrinologia i Nutrició, Hospital Universitari Arnau de Vilanova, Grup de Recerca Obesitat i Hospital Universitario Son Espases, Metabolisme (ODIM), IRBLleida, Universitat de Lleida, Lleida, Spain, 8 Centro de Investigación Biomédica en Red Spain de Diabetes y Enfermedades Metabólicas Asociadas (CIBERDEM), Instituto de Salud Carlos III (ISCIII), Madrid, Spain, 9 *Correspondence: Departament d’Endocrinologia i Nutrició, Hospital de la Santa Creu i Sant Pau, Institut de Recerca Biomèdica Sant Pau (IIB José Manuel Valdivielso Sant Pau), Barcelona, Spain josemanuel.valdivielso@udl.cat Background: Although European guidelines recommend vascular ultrasound for the Specialty section: This article was submitted to assessment of cardiovascular risk in low-to-moderate risk individuals, no algorithm General Cardiovascular Medicine, properly identifies patients who could benefit from it. The aim of this study is to develop a section of the journal Frontiers in Cardiovascular Medicine a sex-specific algorithm to identify those patients, especially women who are usually Received: 14 March 2022 underdiagnosed. Accepted: 20 June 2022 Methods: Clinical, anthropometrical, and biochemical data were combined with a Published: 14 July 2022 12-territory vascular ultrasound to predict severe atheromatosis (SA: ≥ 3 territories Citation: Bermúdez-López M, with plaque). A Personalized Algorithm for Severe Atheromatosis Prediction (PASAP- Martí-Antonio M, Castro-Boqué E, ILERVAS) was obtained by machine learning. Models were trained in the ILERVAS cohort Bretones MdM, Farràs C, Torres G, Pamplona R, Lecube A, Mauricio D, (n = 8,330; 51% women) and validated in the control subpopulation of the NEFRONA Valdivielso JM and Fernández E on cohort (n = 559; 47% women). Performance was compared to the Systematic COronary behalf of the ILERVAS Project Risk Evaluation (SCORE) model. Collaborators (2022) Development and Validation of a Personalized, Results: The PASAP-ILERVAS is a sex-specific, easy-to-interpret predictive model that Sex-Specific Prediction Algorithm of Severe Atheromatosis stratifies individuals according to their risk of SA in low, intermediate, or high risk. New in Middle-Aged Asymptomatic clinical predictors beyond traditional factors were uncovered. In low- and high-risk (L&H- Individuals: The ILERVAS Study. Front. Cardiovasc. Med. 9:895917. risk) men, the net reclassification index (NRI) was 0.044 (95% CI: 0.020–0.068), and the doi: 10.3389/fcvm.2022.895917 integrated discrimination index (IDI) was 0.038 (95% CI: 0.029–0.048) compared to the Frontiers in Cardiovascular Medicine | www.frontiersin.org 1 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis SCORE. In L&H-risk women, PASAP-ILERVAS showed a significant increase in the area under the curve (AUC, 0.074 (95% CI: 0.062–0.087), p-value: < 0.001), an NRI of 0.193 (95% CI: 0.162–0.224), and an IDI of 0.119 (95% CI: 0.109–0.129). Conclusion: The PASAP-ILERVAS improves SA prediction, especially in women. Thus, it could reduce the number of unnecessary complementary explorations selecting patients for a further imaging study within the intermediate risk group, increasing cost- effectiveness and optimizing health resources. Clinical Trial Registration: [www.ClinicalTrials.gov], identifier [NCT03228459]. Keywords: cardiovascular disease, atherosclerosis, cardiovascular risk assessment, machine learning, recursive partitioning classification trees, vascular ultrasound INTRODUCTION shown to increase its predictive value in cardiovascular mortality (11). However, and even though the 2019 European guideline Atherosclerotic cardiovascular disease (ASCVD) is still the main recommends vascular ultrasounds in individuals with a low-to- preventable cause of cardiovascular mortality and disability. moderate ASCVD risk (1), its implementation is still limited ASCVD is a latent and progressive condition in which atheroma possibly because it is not easy to select candidates who could plaques develop in the artery wall, and it is usually widely benefit from a comprehensive imaging study. Thus, a new extended by the time symptoms, typically a cardiovascular algorithm that helps clinicians to identify those patients is highly event, occur. Current European guidelines on cardiovascular recommended, as it will be cost-effective and will optimize health prevention recommend initial assessment and stratification of resources allocation. the risk of ASCVD based on a probabilistic tool that includes Machine learning (ML) technologies can improve the traditional risk factors, followed by therapeutic intervention diagnosis of diseases developing risk prediction algorithms to when necessary (1). However, these traditional risk scores guide clinical decisions (12). Although several ML algorithms underestimate individual risk in several circumstances, such as have been described to predict cardiovascular events (13–17), women, youths, and individuals with a low-to-moderate ASCVD few predict the presence and extent of subclinical atheromatosis risk (2, 3). Indeed, the number of cardiovascular events in (18), and sex-specific algorithms are not available. Therefore, individuals with a low-to-moderate risk is unacceptably high we developed and validated a personalized model to predict (4) and, although ASCVD is often thought to be a disease the probability of severe atheromatosis (SA; the PASAP- with a higher prevalence in men, the annual cardiovascular ILERVAS) based on classification trees (19), integrating clinical mortality rate has remained greater in women (5). Despite these data, anthropometrical parameters, and routine, easy-to-obtain, numbers, women have been underrepresented in CVD-related affordable biochemical parameters in an asymptomatic, middle- clinical trials, and data are not often stratified by sex, limiting aged population with a low-to-moderate cardiovascular risk. The their interpretation. Indeed, better individualized prediction new algorithm could help clinicians to select patients who could algorithms are urgently required, especially in individuals with benefit from further vascular ultrasound examination and reduce moderate risk and in women. the number of unnecessary complementary explorations. One of the available tools to improve prediction algorithms is the use of non-invasive imaging techniques. Those techniques can detect the presence, estimate the extent, and evaluate MATERIALS AND METHODS the clinical consequences of ASCVD. Detection of subclinical atheromatosis in the arteries by vascular ultrasound is a reliable Study Design and Selection of method to predict future coronary events (6), and it improves Participants cardiovascular risk assessment in asymptomatic individuals (7). Discovery Cohort: The ILERVAS Study The most common method of estimating ASCVD burden It is an ongoing randomized, interventional, longitudinal clinical has been carotid ultrasound. Carotid artery atheromatosis has trial in a low-to-moderate cardiovascular risk population of a well-demonstrated role in the incidence of cerebrovascular the North-East region of Spain (ClinicalTrials.gov identifier: and cardiovascular events (8). Atherosclerotic lower extremity NCT03228459). The study was originally designed with two peripheral artery disease is increasingly recognized as an arms: (i) intervention group, thereafter called Mobile Unit important cause of cardiovascular morbidity and mortality (9). follow-up group, and (ii) no intervention group, thereafter called Nevertheless, femoral artery atheromatosis assessment has long Electronic Medical Record follow-up group. The intervention been underexplored, although femoral atheroma plaques show was the generation of a report sent to the primary care the strongest association with cardiovascular risk factors, and physician, including vascular ultrasound examination in carotid a higher sensitivity to predict calcified coronary disease (10). and femoral arteries assessing 12 territories, combined with Furthermore, extensive vascular territory assessment has been clinical, anthropometric, lifestyle, and biochemical parameters. Frontiers in Cardiovascular Medicine | www.frontiersin.org 2 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis The study population was randomly allocated to the groups Source of Information and Data by stratified sampling from the electronic clinical history Collection database of primary care. The no intervention group will Sociodemographic variables and clinical history of cardiovascular be used to compare the impact of the intervention on risk factors were collected from clinical records. In contrast, cardiovascular morbidity and mortality. Thus, this group anthropometric data, smoking habit, and blood samples were was not used for the present work. The intervention group collected at the moment of vascular examination. was formed by 8,330 participants who were enrolled from January 2015 to December 2018 from 32 basic health areas of the province of Lleida (Catalonia, Spain). The study was Anthropometric Data The same protocols were used in both cohorts. Weight and designed to (i) assess the prevalence, vascular distribution, height were measured according to guidelines to obtain BMI. severity, and progression of subclinical atheromatosis in a Waist perimeter was measured with a non-stretchable tape with middle-aged population with a low-to-moderate cardiovascular a precision of 0.1 cm to assess abdominal adiposity, which was risk; (ii) uncover potential new factors predicting severe; and defined as a waist perimeter ≥ 88 cm in women and ≥ 102 cm (iii) assess the impact of subclinical atheromatosis detection in men. Blood pressure was determined in triplicate, after 5 min on the incidence of cardiovascular events during a 10-year rest using an automated device (Omron M6 Comfort, Omron follow-up period. The inclusion criteria were: 50–70-year- Healthcare, Japan) at 2-min intervals. The mean of the three old women and 45–65-year-old men with at least one of recordings was calculated. the following CVD risk factors: hypertension, dyslipidemia, obesity (defined as body mass index, BMI ≥ 30), smoking, and/or first-degree relative who developed premature CVD Biochemical Parameters (with a threshold at age 55 years for men or 65 years In the ILERVAS cohort, a fasting dried blood spot sample was for women). The exclusion criteria were clinical history of obtained by a fingertip puncture according to standard protocols. diabetes, chronic kidney disease, cardiovascular pathology Creatinine, uric acid, and total cholesterol levels were assessed (angina, myocardial infarction, cerebral vascular accident, with the REFLOTRON Plus system (Roche Diagnostics, R peripheral arterial disease, intestinal, or other ischemia), history Germany). It is a validated clinical chemistry system with highly of arterial surgery, active neoplasia, less than 18 months of correlated results to well-standardized laboratory methods (21– expected life, long-term home care, and/or institutionalized 23). The glycosylated hemoglobin test was performed using population. The Ethics Committee of the Hospital Arnau de a point-of-care instrument (Cobas B101 , Roche Diagnostics, R Vilanova (Lleida, Spain) approved the protocol (CEIC-1410). Germany) that meets the generally accepted performance criteria All patients signed informed consent. The study was conducted for its measurement (24). In the NEFRONA cohort, biochemical according to the principles of the Declaration of Helsinki. parameters were obtained from a routine fasting blood test taken A more-detailed explanation of the study has been previously no more than 3 months apart from vascular examination. GFR published (3). was estimated according to international guidelines using the CKD-EPI equation in both cohorts (25). Validation Cohort: The NEFRONA Study It is an observational, multicenter, prospective study. For Atheromatous Plaque Assessment by the present study, 559 healthy asymptomatic individuals with Vascular Ultrasound normal kidney function were selected from the NEFRONA Vascular ultrasound was performed by nurses specialized cohort. Subjects were enrolled in Primary Care centers from in vascular imaging. Standardized scanning and reading October 2010 to June 2012. It was designed to evaluate the protocols were followed to decrease interoperator variability prevalence and natural history of subclinical atheromatosis in and type 2 errors. Intra-observer reliability assessment showed chronic kidney disease patients, and the contribution of vascular a k-coefficient of one (two repeated measurements; 1,007 ultrasound for a more precise cardiovascular risk assessment. observations). Overall inter-rater reliability for all operators The NEFRONA cohort included a control group of individuals showed a k-coefficient of 0.915 (95% CI: 0.892–0.944; 959 with normal kidney function, which is the subgroup used for observations). Readers were unaware of the patients’ clinical the validation. Inclusion criteria were asymptomatic individuals histories. The VIVID i BT09 model ultrasound system between 18 and 74 years of age with a glomerular filtration (GE Healthcare) equipped with a 12L-RS linear probe (6– rate (GFR) over 60 ml/min/1.73 m2 . Exclusion criteria were 13 MHz), and a pulsed Doppler ultrasound was used to assess active infections, pregnancy, active neoplasia, life expectancy hemodynamic repercussions. In the ILERVAS cohort, arterial shorter than 12 months, previous cardiovascular event, carotid ultrasound was performed in 12 territories, both carotid surgery, or any organ transplantation. The Ethics Committee (common, bifurcation, internal, and external) and femoral of the Hospital Arnau de Vilanova (Lleida, Spain) approved (common and superficial) arteries. In the NEFRONA study, 10 the protocol. All patients signed informed consent. The study territories were explored, both carotid (common, bifurcation, was conducted according to the principles of the Declaration and internal) and femoral (common and superficial) arteries (3, of Helsinki. A more-detailed explanation of the study has been 20). Subclinical atheromatosis was defined as the presence of any previously published (20). plaque in the explored areas. According to Mannheim consensus, Frontiers in Cardiovascular Medicine | www.frontiersin.org 3 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis an atheroma plaque was defined as a focal encroachment into The obtained model was called Personalized Algorithm for Severe the lumen of the artery ≥ 1.5 mm (26). SA was defined as ≥ 3 Atheromatosis Prediction (PASAP-ILERVAS). territories with atheroma plaque. Variable Importance Calculation A variable importance score was calculated using the Statistical Analysis improvement measure attributable to each predictor in its Variable Selection role as splitter, plus the goodness for all splits in which it was a Parameters that require a high degree of specialization or surrogate. The values of importance were scaled up to sum 100%. technical resources were ruled out. Thus, a set of easy-to- obtain and affordable variables were selected. The outcome Probability Calibration was SA, defined as ≥ 3 territories with atheroma plaque out In order to agglutinate terminal nodes into groups with similar of 12. The explanatory variables were age, sex, clinical data risk, the probabilities of SA were calibrated using a histogram- (history of hypertension and dyslipidemia and smoking habit), transformed method using the CalibratR (32). First, the original anthropometrical data [systolic blood pressure (SBP), diastolic probabilities with a maximum of 10 bins were plotted. Then, blood pressure (DBP), body mass index (BMI), abdominal the optimal number of partitions to maximize the sensitivity adiposity, and waist-to-height ratio], and biochemical parameters was tested. The calibration process evidenced three risk groups, (creatinine, GFR, uric acid, and total cholesterol). classified as low-, intermediate-, and high-risk nodes. Descriptive Analysis of the Cohorts Prediction of Severe Atheromatosis and Non-severe The clinical characteristics of the ILERVAS cohort (discovery Atheromatosis With Personalized Algorithm for cohort) and NEFRONA cohort (external validation cohort) were Severe Atheromatosis Prediction described as frequencies for categorical variables, and means and The Youden index (33) was computed to identify the cut-off standard deviation (SD) for quantitative variables. Differences probability to distinguish between non-SA and SA in the terminal between SA and non-SA were assessed by several univariate nodes. Terminal nodes with a lower or equal probability to the logistic regressions for each clinical predictor adjusted by age cut-off were considered as non-SA, whereas terminal nodes with and stratified by sex. The p-value of the likelihood-ratio test a higher probability were classified as SA. and odds ratio (OR) values with 95% confidence intervals (95% CI) were represented with the forestplot package (27). For each Performance Metrics numerical predictor, adjusted OR per 1-SD higher parameter The area under the curve (AUC) and the recommended metrics measure was estimated. derived from the confusion matrix were used to evaluate model performance (34). A report from the European Society of Sample Balancing Cardiology Prevention of CVD Programme is recommended to familiarize the reader with the risk prediction tools in The descriptive analyzed revealed that the prevalence of SA was cardiovascular disease prevention and performance metrics (35). lower than non-SA in the discovery cohort. Thus, a Random Over-Sampling Example (ROSE) function (28) was used to External Validation increase the sample size of individuals with SA separately The PASAP-ILERVAS model was externally validated in a for men and women. Individuals who presented missing subpopulation of the NEFRONA study and performance metrics data (Supplementary Table 1) were excluded prior to sample were also calculated. balancing. This new balanced data set was used to train ML models in both sexes. Systematic COronary Risk Evaluation Model A logistic binary model with the Systematic COronary Risk Machine Learning Evaluation (SCORE) model was developed in the discovery A Recursive PARTicioning (rpart) classification tree (29) balanced sample. The Youden index was computed to identify approach was used and plotted with rpart.plot (30). Trees the cut-off probability to distinguish between non-SA and SA and were constructed in the balanced sample stratified by sex as performance metrics were calculated in L&H-risk individuals. follows: first, the algorithm recursively selected itself the clinical predictors that provide an optimal split in each node based Improvement of Risk Prediction Between the on the generalized Gini index to obtain the maximal impurity Personalized Algorithm for Severe Atheromatosis reduction. The growth of the tree was controlled by pre-pruning Prediction and the Systematic COronary Risk techniques. A minimum of 200 individuals in each node was Evaluation Model required to try a split and at least 100 in the terminal nodes. The increment on AUC, the Net Reclassification Index (NRI), The maximum depth of the final tree was fixed on 6. Second, the and the Integrated Discrimination Index (IDI) were evaluated resultant tree was trimmed back to minimize overfitting by post- to quantify the differences between both models in L&H-risk pruning techniques, which select the complexity parameter (CP) individuals (36). The SCORE model was considered as reference, that produced the minimum internal three-fold cross-validation and the PASAP-ILERVAS as the new model whose improvement error to prune the full tree. The probability of SA and the 95% was evaluated (37, 38). The CI for all metrics was performed CI were represented in the terminal nodes with ggplot2 (31). by bootstrapping (2,000 bootstraps). The comparisons between Frontiers in Cardiovascular Medicine | www.frontiersin.org 4 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis models were performed by the DeLong test at 95% CI (39). and there were fewer non-smokers and more current smokers. To test the differences between performance metrics [sensitivity, The mean age was higher in participants with SA, and the specificity, accuracy, and the positive predictive value (PPV)], levels of systolic blood pressure, glycosylated hemoglobin, uric a two-sample test for equality of proportions with continuity acid, and total cholesterol. However, the association of these correction was performed (40). clinical predictors with SA showed some differences between sexes (Supplementary Figure 1). Age, systolic and diastolic blood pressure, total cholesterol, and being underweight showed a RESULTS stronger positive association with SA in men than in women (OR: 1.76 vs. 1.51; OR: 1.32 vs. 1.23; OR: 1.17 vs. 1.07; OR: Clinical Characteristics of the Discovery 1.24 vs. 1.13; and OR: 7.02 vs. 3.39, respectively). On the and Validation Cohorts contrary, clinical history of hypertension showed a stronger The workflow of the whole analysis is shown in Figure 1. The positive association with SA in women than in men (OR: 1.58 vs. clinical characteristics of the ILERVAS cohort stratified by sex is 1.24, respectively). Finally, the levels of glycosylated hemoglobin, shown in Table 1. The prevalence of SA, defined as ≥ 3 territories uric acid, and clinical history of dyslipidemia showed a similar with atheroma plaque out of the 12 studied, was 42.6 and 24.4% positive association in both sexes (OR: 1.11 vs. 1.09; OR: 1.10 in men and women, respectively. These patients showed a higher vs. 1.10; and OR: 1.19 vs. 1.21 in men vs. women, respectively; prevalence of clinical history of hypertension and dyslipidemia, Supplementary Table 2). FIGURE 1 | Analysis pipeline. ASCVD, atherosclerotic cardiovascular disease; OR, odds ratio; SD, standard deviation. Frontiers in Cardiovascular Medicine | www.frontiersin.org 5 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis TABLE 1 | Baseline characteristics of the ILERVAS cohort by sex. Men Women n = 4,108 n = 4,222 Severe atheromatosis Severe atheromatosis No Yes P-value No Yes P-value n (%) 2,358 (57.4%) 1,750 (42.6%) 3,190 (75.6%) 1,032 (24.4%) Age, years 53.68 (5.77) 56.87 (5.56) <0.001 59.17 (5.83) 61.59 (5.99) <0.001 Clinical history, n (%) Hypertension 832 (35.3%) 797 (45.5%) 0.002 1,205 (37.8%) 549 (53.2%) <0.001 Dyslipidemia 1,171 (49.7%) 959 (54.8%) 0.007 1,729 (54.2%) 638 (61.8%) 0.012 Smoking habit <0.001 <0.001 Non-smoker 737 (31.3%) 252 (14.4%) 1,797 (56.3%) 502 (48.6%) Former 912 (38.7%) 703 (40.2%) 728 (22.8%) 203 (19.7%) Current 709 (30.1%) 795 (45.4%) 665 (20.8%) 327 (31.7%) Anthropometrical data SBP, mmHg 130.60 (15.25) 136.51 (16.51) <0.001 127.98 (17.15) 133.29 (16.94) <0.001 DBP, mmHg 83.79 (9.32) 85.34 (9.48) <0.001 79.04 (9.02) 79.59 (8.93) 0.060 Body mass index, n (%) 0.098 0.028 Underweight 2 (0.1%) 9 (0.5%) 11 (0.3%) 11 (1.1%) Normal weight 396 (16.8%) 283 (16.2%) 792 (24.8%) 251 (24.3%) Overweight/obesity 1,960 (83.1%) 1,457 (83.3%) 2,386 (74.8%) 770 (74.6%) Abdominal adiposity, n (%) 1,158 (49.1%) 914 (52.3%) 0.688 2,713 (85.1%) 892 (86.4%) 0.847 Waist-to-height ratio 0.60 (0.07) 0.60 (0.07) 0.907 0.63 (0.08) 0.64 (0.08) 0.837 Biochemical parameters Creatinine, mg/dL 0.90 (0.21) 0.88 (0.18) <0.001 0.71 (0.16) 0.71 (0.16) 0.181 GFR, mL/min/1.73 m2 93.76 (14.35) 93.01 (13.99) 0.094 90.17 (14.09) 88.79 (14.24) 0.007 Glycosylated hemoglobin, % 5.49 (0.44) 5.56 (0.46) 0.002 5.56 (0.38) 5.61 (0.39) 0.014 Uric acid, mg/dL 6.09 (1.36) 6.25 (1.45) 0.003 4.69 (1.25) 4.86 (1.38) 0.007 Total cholesterol, mg/dL 196.49 (36.02) 203.03 (39.88) <0.001 211.39 (36.24) 214.98 (37.86) <0.001 Values are shown as means and standard deviations for quantitative variables. Clinical history data were obtained from electronic medical records and refers to patients who had the prior clinical diagnostic of hypertension or dyslipidemia. Underweight was defined as a BMI < 18.5 kg/m2 , normal weight as 18.5–24.9, overweight 25–29.9, and obesity ≥ 30. Abdominal adiposity was defined as an abdominal perimeter ≥ 88 cm in women or ≥ 102 in men. DBP, diastolic blood pressure; GFR, glomerular filtration rate; SBP, systolic blood pressure. A Personalized Algorithm for Severe in women, intermediate-risk nodes with 54.8% in men and 54.9% Atheromatosis Prediction in women, and high-risk nodes with 71.8% in men and 71.2% in women. These profiles are shown in Figures 2, 3 as green, yellow, Pruning the trees to 13 splits in men and 17 splits in and red nodes, respectively. Thus, clinicians could easily identify women showed the lowest internal cross-validation error, beyond the risk of severe atheromatosis according to the characteristics which tree complexity entailed no additional improvement of the patients. (Supplementary Figure 2). The structure of the classification tree Variable importance assessment revealed that in both sexes, in men and women in the balanced ILERVAS cohort is shown age, smoking habit, SBP, and total cholesterol were the most in Figures 2, 3, respectively. In both genders, patients were first important clinical predictors of severe atheromatosis, which classified by age, followed by the smoking habit. The combination account for 98.4% of the predictor importance in men and 82.3% of these two parameters conditioned the clinical threshold of the in women. Additionally, a GFR ≥ 107 ml/min/m2 increased other predictors, such as systolic blood pressure, total cholesterol, the risk in men. In women, other predictors, such as history of uric acid, GFR, BMI, waist perimeter, and history of dyslipidemia dyslipidemia, BMI, history of hypertension, uric acid, and waist and hypertension. perimeter were identified (Figure 4). A total number of 14 final nodes were obtained in men and 18 in women. The probability of severe atheromatosis and its 95% CI are shown in each node to improve visualization. Internal Validation of the Personalized In order to agglutinate individual terminal nodes into groups Algorithm for Severe Atheromatosis with similar risk, the probability of severe atheromatosis in each Prediction Algorithm node was calibrated. Supplementary Figure 3 shows the absolute Table 2 shows the internal validation metrics. Specificity, frequency of patients with or without severe atheromatosis and PPV, accuracy, and AUC were higher in L&H-risk individuals its probability of having the disease before and after calibration. compared to the total sample (p-value: < 0.001), which indicate a Three risk profiles were identified: low-risk nodes with a mean higher feasibility of the PASAP-ILERVAS in those patients. On prevalence of severe atheromatosis of 28.8% in men and 28.6% the contrary, the negative predictive value (NPV) showed no Frontiers in Cardiovascular Medicine | www.frontiersin.org 6 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis FIGURE 2 | Personalized Algorithm for Severe Atheromatosis Prediction in men. The structure of the classification tree in men is represented. The probability of severe atheromatosis, which corresponds to the proportion of affected patients, is shown inside circles in the final nodes. The barplot offers a clearer visualization of this prevalence with its 95% confidence interval. The colors green, yellow, and red indicate the level of risk (low, intermediate, or high) identified after calibration. The units were as follows: age, year; GFR, ml/min/m2 ; SBP, mmHg; total cholesterol, mg/dl. GFR, glomerular filtration rate, SBP, systolic blood pressure. differences (i-value: > 0.1), and sensitivity was lower in L&H-risk Although the prevalence of SA was lower in the NEFRONA individuals than in the total sample (i-value: < 0.001). cohort compared to the ILERVAS cohort, patients showed similar In L&H-risk patients, all parameters were similar in both characteristics (Supplementary Table 3). sexes. The NPV, the PPV, and accuracy were 0.71. Thus, the Table 2 shows the external validation metrics. In L&H-risk PASAP-ILERVAS showed a 71% of correct predictions (including individuals, the specificity was similar in both sexes (men: 0.795; both true positive and true negative). Finally, AUC was 0.734 in women 0.774, i-value = 0.783). The NPV was extremely high men and 0.730 in women. (men: 0.898; women: 0.941). Thus, PASAP-ILERVAS showed a high ability in identifying truly healthy individuals. The PPV was higher in men than in women (0.561 vs. 0.178, External Validation of the Personalized p-value: < 0.001). It is noteworthy that performance metrics were Algorithm for Severe Atheromatosis limited by the low prevalence of SA in the external validation Prediction Algorithm cohort (men: 31.2%; women: 10.7%). Sensitivity was similar in The external validation was performed in an asymptomatic, both sexes (0.744 vs. 0.500, p-value = 0.141). Accuracy was high kidney disease-free, subpopulation of the NEFRONA cohort. in both, indicating a 78.2% and a 75.0% of correct predictions Frontiers in Cardiovascular Medicine | www.frontiersin.org 7 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis FIGURE 3 | Personalized Algorithm for Severe Atheromatosis Prediction in women. The probability of severe atheromatosis, which corresponds to the proportion of affected patients, was shown inside circles in the final nodes. The barplot offers a clearer visualization of this prevalence with its 95% confidence interval. The colors green, yellow, and red indicate the level of risk (low, intermediate, or high, respectively) identified after calibration. Hypertension and dyslipidemia refer to patients who had prior clinical diagnostic of hypertension or dyslipidemia in their clinical records. The units were as follows: Age: year; BMI: kg/m2 ; SBP, mmHg; total cholesterol: mg/dL; uric acid: mg/dl; waist perimeter: cm. BMI, body mass index; SBP, systolic blood pressure. by the PASAP-ILERVAS. Finally, the AUC was 0 .799 in men were very similar in both models (specificity: 0.763 vs. 0.761, and 0.691 in women. p-value: 0.903; PPV: 0.718 vs. 0.703, p-value: 0.398; and accuracy: 0.715 vs. 0.694, p-value: 0.070). However, although the increment Comparison of the Personalized of AUC was not significant (p-value: 0.962), the PASAP-ILERVAS showed an NRI of 0.044 (95% CI: 0.020–0.068) and an IDI of Algorithm for Severe Atheromatosis 0.038 (95% CI: 0.029–0.048) compared to the SCORE model. Prediction Algorithm With the Systematic In women, the PASAP-ILERVAS showed a higher specificity COronary Risk Evaluation than the SCORE (0.753 vs. 0.457, p-value: < 0.001), a lower The performance of the PASAP-ILERVAS to predict SA in L&H- sensitivity (0.669 vs. 0.772, p-value: < 0.001), and a higher risk individuals was compared to the traditional SCORE model accuracy (0.713 vs. 0.607, p-value: < 0.001). The PPV was also (Figure 5 and Table 3). higher in the PASAP-ILERVAS (0.712 vs. 0.565, p-value: < 0.001). In men, the PASAP-ILERVAS showed a higher sensitivity than The new algorithm showed a significant increase in the AUC of the traditional score (0.661 vs. 0.620, p-value: 0.019). Thus, the 0.074 (95% CI: 0.062–0.087, p-value: < 0.001). Thus, PASAP- new algorithm had a higher ability to detect L&H-risk men who ILERVAS showed a higher percentage of correct predictions in actually were affected by the disease. In contrast, other metrics L&H-risk women than the traditional SCORE, especially those Frontiers in Cardiovascular Medicine | www.frontiersin.org 8 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis FIGURE 4 | Clinical predictor importance in the PASAP-ILERVAS in both sexes. A variable importance score was calculated using the improvement measure attributable to each predictor in its role as splitter, plus the goodness for all splits in which it was a surrogate. The values of importance were scaled up to sum 100%. SBP, systolic blood pressure; BMI, body mass index; GFR, glomerular filtration rate. TABLE 2 | Internal and external validation of the PASAP-ILERVAS. PASAP-ILERVAS Internal validation External validation Men Women Men Women Absence of severe atheromatosis 0.500 0.499 0.688 0.893 Negative predictive value 0.688 0.695 0.876 0.940 Low and high risk 0.712 0.714 0.898 0.941 Specificity 0.628 0.614 0.624 0.710 Low and high risk 0.763 0.753 0.795 0.774 Prevalence of severe atheromatosis 0.500 0.501 0.312 0.107 Positive predictive value 0.658 0.656 0.493 0.205 Low and high risk 0.718 0.712 0.561 0.178 Sensitivity 0.715 0.733 0.805 0.625 Low and high risk 0.661 0.669 0.744 0.500 Accuracy 0.671 0.673 0.681 0.701 Low and high risk 0.715 0.713 0.782 0.750 AUC 0.710 0.708 0.755 0.693 Low and high risk 0.734 0.730 0.799 0.691 women who truly had severe atheromatosis, showing an NRI namely those classified as intermediate risk. Non-invasive of 0.193 (95% CI: 0.162–0.224) and an IDI of 0.119 (95% CI: imaging techniques can detect the presence of atheromatosis 0.109–0.129) compared to the SCORE model. and estimate the burden, which has been proven to improve cardiovascular risk assessment (7), and it is recommended when the coronary artery calcium score is unavailable or not DISCUSSION feasible (41). Furthermore, it is a cheap and accessible in most medical offices. However, the implementation of vascular In this study, we presented a novel, sex-specific, machine ultrasound, which could help increase accuracy in those patients learning-based algorithm for SA prediction in asymptomatic has not been accomplished in the routine clinical practice due middle-aged individuals. The PASAP-ILERVAS integrates clinical to several reasons. Among them, the high-time burden added history data, anthropometrical measurements, and affordable to each visit is likely one. Therefore, identifying patients in biochemical parameters to obtain a hierarchical, flexible, easy- whom a vascular ultrasound could add prognostic value is of to-interpret, predictive model. This model was validated in paramount importance. an external cohort, showing excellent performance in the The novel PASAP-ILERVAS stratifies individuals according to L&H-risk individuals. Thus, the algorithm could be useful to their risk of SA as low, intermediate, and high. First, patients were select individuals who can benefit from a vascular ultrasound, classified by age, followed by smoking habit. The combination Frontiers in Cardiovascular Medicine | www.frontiersin.org 9 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis FIGURE 5 | Comparison of the PASAP-ILERVAS with the SCORE risk score. Barplots contain the main performance metrics of PASAP-ILERVAS and SCORE risk model in L&H-risk individuals: (A) specificity; (B) sensitivity; (C) accuracy; (D) positive predictive model; (E) ROC curve in men; (F) ROC curve in women. The ROC curves reflect the area under the curve between sensitivity and complementary of specificity with both models. SCORE, Systematic COronary risk Evaluation; ROC, Receiver Operating Characteristic. of these two parameters conditioned the clinical threshold of the ILERVAS cohort with asymptomatic middle-aged individuals other predictors. Thus, the ML approach evidenced the additive without established CVD, revealed that in metabolically healthy effect of cardiovascular risk factors. In addition, the algorithm individuals the presence of subclinical atherosclerosis increased revealed that biological thresholds which conditioned ASCVD across BMI categories, whereas fewer differences were observed risk, must be considered individually. for metabolically unhealthy individuals. Thus, the presence Age, smoking habit, SBP, and total cholesterol accounts of subclinical atherosclerosis observed in patients with an 98.4% of the predictor importance in men and 82.3% in abnormal BMI was mainly attributed to the coexistence of other women. Additionally, a GFR ≥ 107 ml/min/m2 increased the cardiovascular risk factors, such as hypertension, diabetes, and risk in men. Although it is amply proven that kidney failure dyslipidemia (45). A recent study in a subpopulation of the PESA increases ASCVD risk as kidney function decreases, few studies study revealed that individuals with metabolic syndrome or its linked an increased glomerular filtration with ASCVD risk. individual components (central obesity, hypertension, low HDL- Glomerular hyperfiltration is an initial step of kidney damage in C, triglycerides, and altered glucose metabolism) showed bone diabetes and obesity, but its threshold remains elusive. However, marrow activation, even in the absence of systemic inflammation, there are data showing that glomerular hyperfiltration of 107– which is associated with early atherosclerosis (46). Similarly, uric 115 ml/min/m2 is independently associated with increased acid levels have been positively correlated with cardiovascular cardiovascular risk in middle-aged healthy individuals (42). diseases, including hypertension and atherosclerosis, through In women, other predictors related to obesity were identified. oxidative stress and an inflammatory response (47). Women Obesity is associated with an increased cardiovascular risk are often older than men when they suffer from their first since the pro-inflammatory cytokines produced by the adipose cardiovascular event (48). This difference is attributed to tissue itself induce atherosclerotic plaque formation (43), and the protective role of circulating estrogens on the vascular a larger waist perimeter was independently associated with endothelium, maintaining oxide nitric release leading to recurrent atherosclerotic cardiovascular disease (44). The PESA vasodilation (49), regulating prostaglandin production and study, which is another large Spanish cohort similar to the inhibiting smooth muscle cell proliferation (50). On the contrary, Frontiers in Cardiovascular Medicine | www.frontiersin.org 10 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis TABLE 3 | Improvement in risk prediction of PASAP-ILERVAS respect to SCORE in L&H-risk women than the traditional SCORE, especially in model. healthy women (specificity:0.753 vs. 0.457, p-value: < 0.001) and Men Women in those who were positive in screening test and truly had severe atheromatosis (PPV:0.712 vs. 0.565, p-value: < 0.001). Indeed, NRI 0.044 (0.020, 0.068) 0.193 (0.162, 0.224) our algorithm showed a significant increase in the AUC of 0.074 IDI 0.038 (0.029, 0.048) 0.119 (0.109, 0.129) (95% CI: 0.062–0.087, p-value: < 0.001) in women. 1AUC 0.001 (–0.012, 0.013) 0.074 (0.062, 0.087) Several limitations should be considered when interpreting The increment on AUC (1AUC), the Net Reclassification Index (NRI), and the this study. First, results are based on a cross-sectional analysis Integrated Discrimination Index (IDI) with 95% CI were shown. from the ILERVAS study cohort. However, the inclusion of history factors, such as hypertension and dyslipidemia, partially overcomes this limitation. Second, the ILERVAS at menopause, women show endothelial dysfunction and lipid cohort consists of middle-aged symptomatic participants with deposition in the artery walls, which can promote atherosclerosis relatively homogeneous socioeconomic, lifestyle, and ethnic development (51). In women under 55 years of age, smoking is characteristics. However, having a homogenous population the most important preventable cause of CVD, increasing their increases internal validity, can unveil hidden clinical associations, risk 7-fold (52). Large cohort studies have demonstrated that and is not uncommon (45, 58). Even though the PASAP- high SBP is an important risk factor for cardiovascular disease ILERVAS was externally validated in another cohort, further and a 5-mm Hg reduction of SBP reduced the risk of major analysis would be considered to reinforce data extrapolation. cardiovascular events by about 10% (53). Strikingly, hypertension Third, the derivation cohort is different not only in size (< 10% is more strongly associated with CVD in women compared of individuals), and time period, but also in terms of incidence to men (54). of subclinical atheromatosis, and even baseline characteristics of The PASAP-ILERVAS can be used as a prescreening model the patients. Fourth, the levels of glycosylated hemoglobin can to determine patients who will further benefit from a vascular be used to identify asymptomatic individuals at higher risk of ultrasound examination. It identified three risk profiles (low, subclinical atherosclerosis on top of traditional cardiovascular intermediate, and high). The internal validation showed that risk factors (58). However, unfortunately, we could not study the specificity, PPV, accuracy, and AUC were higher in L&H-risk contribution of HbA1c in the PASAP-ILERVAS due to a high individuals compared to the total sample (p-value: < 0.001), percentage of missing values in the external validation cohort. which indicate a higher feasibility of the PASAP-ILERVAS in Finally, recursive partitioning may be complex or overfit the data. those patients. In L&H-risk individuals, all parameters were This issue was addressed by a post-pruning technique based on similar in both sexes. The NPV and the PPV were higher than the CP, which shows the trade-off between the tree complexity 71%, which indicated that PASAP-ILERVAS algorithm correctly and how well the tree fits the data. identified patients with SA (true positives) or without SA (true In contrast, our study has several strengths. First, the study negatives). The external validation results were limited by the population was randomized, and a stratified sampling was low prevalence of SA in that cohort. However, the accuracy performed from primary care records to reduce selection bias, was high in both sexes (men: 0.782; women: 0.750), indicating and to obtain a representative cohort of the entire province. a high percentage of correct predictions. Thus, patients with Second, contrary to recent atheromatosis prediction algorithm, a an estimated low- or high-risk are very accurately classified by sex-stratified analysis was performed to unveil clinical differences the algorithm and are not candidates for vascular ultrasound in men and women. Third, rpart is a non-parametric ML examination. On the contrary, patients in whom the prediction method that can handle highly skewed data, does not require is not so accurate (intermediate-risk) are strong candidates to data categorization, and generates an easy-to-interpret graphical undergo a vascular ultrasound to truly determine whether they representation, which is very convenient in the daily clinical have SA in order to adjust treatments and/or follow-up visits. practice. Finally, the algorithm combines clinical history data, The NPV was extremely high (men: 0.898; women: 0.941). anthropometrical measurements, and affordable biochemical Thus, PASAP-ILERVAS showed a high ability in identifying truly parameters that can be easily obtained in non-hospital settings, healthy individuals. such as primary care centers and even pharmacies. Blood The SCORE is a cardiovascular risk assessment tool that biochemical parameters were obtained by dried blood spot tests, estimates the 10-year risk of fatal CVD (55). At the moment, which are highly validated methods. there is no internationally accepted tool to predict subclinical atheromatosis, so new methods to do so have been previously compared to the SCORE in similar middle-aged asymptomatic CONCLUSION cohorts (18, 56, 57). The comparison of the PASAP-ILERVAS with the SCORE model revealed a better performance in both We developed a personalized, sex-specific, easy-to-interpret sexes, showing a higher sensitivity in men, and a higher PPV, model for severe atheromatosis prediction in asymptomatic specificity, accuracy, and AUC in women. Importantly, PASAP- middle-aged individuals. The PASAP-ILERVAS predicted ILERVAS improved patients’ reclassification in both sexes, but accurately in L&H-risk patients. However, in intermediate- was much higher in women [NRI in men: 0.044 (95% CI: 0.020– risk individuals a vascular imaging exploration would be 0.068); women: 0.193 (95% CI: 0.162–0.224)]. Thus, the new recommended. Thus, the present algorithm could reduce algorithm showed a higher percentage of correct predictions the number of unnecessary complementary explorations Frontiers in Cardiovascular Medicine | www.frontiersin.org 11 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis selecting candidates for a further imaging study, increasing cost- Agència de Salut Pública de Catalunya, Departament de Salut, effectiveness and optimizing health resources. IRBLleida, Universitat de Lleida, Lleida, Spain; CIBER de Epidemiología y Salud Pública (CIBERESP), Madrid, Spain; Montse Martinez-Alonso: Unitat de Bioestadística, IRBLleida, DATA AVAILABILITY STATEMENT Departament de ciències Mèdiques Bàsiques, Universitat de Lleida, Lleida, Spain. The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation. AUTHOR CONTRIBUTIONS ETHICS STATEMENT MB-L, JV, and EF: study concept and design. EC-B and CF: data acquisition. MB-L, MM-A, JV, EF, and MB: data The studies involving human participants were reviewed and interpretation. MM-A and MB-L: statistical analysis and drafting approved by CEIC Hospital Universitario Arnau de Vilanova. of the manuscript. EF: study supervision. ILERVAS investigators: The patients/participants provided their written informed collected baseline and prospective follow-up data of the cohort. consent to participate in this study. All authors contributed to the critical revision of the manuscript for important intellectual content and article and approved the submitted version. ILERVAS PROJECT COLLABORATORS Eva Miquel and Marta Ortega: Centre d’Atenció Primària Cappont, Gerència Territorial de Lleida, Institut Català de la FUNDING Salut, Barcelona, Spain; Research Support Unit Lleida, Fundació This work was supported by grants from the Diputació de Institut Universitari per a la recerca a l’Atenció Primària de Lleida, Instituto de Salud Carlos III (RETIC RD16/0009/0011) Salut Jordi Gol i Gorina (IDIAPJGol), Barcelona, Spain; Ferran and Ministerio de Ciencia, Innovación y Universidades (IJC2018- Barbé, Jessica González, Silvia Barril, and Manuel Sánchez- 037792-I). de-la-Torre: Departament de Medicina Respiratòria, Hospital Universitari Arnau de Vilanova, Grup Recerca Translational Medicina Respiratòria, IRBLleida, Universitat de Lleida, Lleida, ACKNOWLEDGMENTS Spain; CIBER de enfermedades respiratorias (CIBERES), Madrid, Spain; Manuel Portero-Otín and Mariona Jové: Departament We would like to thank Virtudes María, Marta Elias, Teresa de Medicina Experimental, IRBLleida, Universitat de Lleida, Molí, Cristina Domínguez, Noemí Nova, Alba Prunera, Núria Lleida, Spain; Marta Hernández and Ferran Rius: Departament Sans, Meritxell soria, Francesc Pons, Rebeca Senar, Pau Guix, d’Endocrinologia i Nutrició, Hospital Universitari Arnau de Fundació Renal Jaume Arnó, and the Primary Care teams of the Vilanova, Grup de Recerca Obesitat i Metabolisme (ODIM), province of Lleida for recruiting participants and their efforts in IRBLleida, Universitat de Lleida, Lleida, Spain; Centro de the accurate development of the ILERVAS project. Samples were Investigación Biomédica en Red de Diabetes y Enfermedades obtained with support from IRBLleida Biobank (B.0000682) and Metabólicas Asociadas (CIBERDEM), Instituto de Salud PLATAFORMA BIOBANCOS PT17/0015/0027. Carlos III (ISCIII), Madrid, Spain; Josep Franch-Nadal and Esmeralda Castelblanc: Centro de Investigación Biomédica en Red de Diabetes y Enfermedades Metabólicas Asociadas SUPPLEMENTARY MATERIAL (CIBERDEM), Instituto de Salud Carlos III (ISCIII), Madrid, Spain; Departament d’Endocrinologia i Nutrició, Hospital The Supplementary Material for this article can be found de la Santa Creu i Sant Pau, Institut de Recerca Biomèdica online at: https://www.frontiersin.org/articles/10.3389/fcvm. Sant Pau (IIB Sant Pau), Barcelona, Spain; Pere Godoy: 2022.895917/full#supplementary-material REFERENCES low-to-moderate cardiovascular risk population: the ILERVAS study. Rev Esp Cardiol. (2020) 74:1042–53. doi: 10.1016/j.rec.2020.09.015 1. Mach F, Baigent C, Catapano AL, Koskinas KC, Casula M, Badimon L, 4. Marrugat J, Vila J, Baena-Diez JM, Grau M, Sala J, Ramos R, et al. [Relative et al. 2019 ESC/EAS guidelines for the management of dyslipidaemias: validity of the 10-year cardiovascular risk estimate in a population cohort of lipid modification to reduce cardiovascular risk. Eur Heart J. (2020) 41: the REGICOR study]. Rev Esp Cardiol. (2011) 64:385–94. doi: 10.1016/j.rec. 111–88. 2010.12.017 2. López-Melgar B, Fernández-Friera L, Oliva B, García-Ruiz JM, Peñalvo JL, 5. Gbd 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases Gómez-Talavera S, et al. Subclinical atherosclerosis burden by 3D ultrasound and injuries in 204 countries and territories, 1990-2019: a systematic analysis in mid-life: the PESA study. J Am Coll Cardiol. (2017) 70:301–13. doi: 10.1016/ for the global burden of disease study 2019. Lancet. (2020) 396:1204–22. j.jacc.2017.05.033 6. Simon A, Chironi G, Levenson J. Performance of subclinical arterial disease 3. Bermúdez-López M, Martínez-Alonso M, Castro-Boqué E, Betriu À, Cambray detection as a screening test for coronary heart disease. Hypertension. (2006) S, Farràs C, et al. Subclinical atheromatosis localization and burden in a 48:392–6. doi: 10.1161/01.HYP.0000236507.76042.72 Frontiers in Cardiovascular Medicine | www.frontiersin.org 12 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis 7. Inaba Y, Chen JA, Bergmann SR. Carotid plaque, compared with carotid (2004-2006-2011). An update on behalf of the advisory board of the 3rd, 4th, intima-media thickness, more accurately predicts coronary artery disease and 5th watching the risk symposia, at the 13th, 15th, and 20th European events: a meta-analysis. Atherosclerosis. (2012) 220:128–33. doi: 10.1016/j. stroke conferences, Mannheim, Germany, 2004, Brussels, Belgium, 2006, and atherosclerosis.2011.06.044 Hamburg, Germany, 2011. Cerebrovasc Dis. (2012) 34:290–6. doi: 10.1159/ 8. Li H, Xu X, Luo B, Zhang Y. The predictive value of carotid ultrasonography 000343145 with cardiovascular risk factors-A “SPIDER” promoting atherosclerosis. Front 27. Gordon M, Lumley T. Lumley MGaT Forestplot: Advanced Forest Plot Using Cardiovasc Med. (2021) 8:706490. doi: 10.3389/fcvm.2021.706490 ‘grid’ Graphics. Vienna: The R Foundation (2020). 9. Aday AW, Matsushita K. Epidemiology of peripheral artery disease 28. Lunardon N, Menardi G, Torelli N. ROSE: a package for binary imbalanced and polyvascular disease. Circ Res. (2021) 128:1818–32. doi: 10.1161/ learning. R J. (2014) 6:82–92. doi: 10.32614/RJ-2014-008 CIRCRESAHA.121.318535 29. Therneau T, Atkinson B, Ripley B. Rpart: Recursive Partitioning and Regression 10. Laclaustra M, Casasnovas JA, Fernández-Ortiz A, Fuster V, León-Latre M, Trees. Vienna: The R Foundation (2019). Jiménez-Borreguero LJ, et al. Femoral and carotid subclinical atherosclerosis 30. Milborrow S. rpart.plot: Plot ‘rpart’ Models: An Enhanced Version of association with risk factors and coronary calcium: the AWHS study. J Am Coll ‘plot.rpart’. Vienna: The R Foundation (2020). Cardiol. (2016) 67:1263–74. doi: 10.1016/j.jacc.2015.12.056 31. Wickham H. ggplot2: Elegant Graphics for Data Analysis. New York, NY: 11. Valdivielso JM, Betriu A, Martinez-Alonso M, Arroyo D, Bermudez-Lopez Springer-Verlag New York (2016). doi: 10.1007/978-3-319-24277-4_9 M, Fernandez E, et al. Factors predicting cardiovascular events in chronic 32. Schwarz J, Heider D. GUESS: projecting machine learning scores to well- kidney disease patients. Role of subclinical atheromatosis extent assessed by calibrated probability estimates for clinical decision-making. Bioinformatics. vascular ultrasound. PLoS One. (2017) 12:e0186665. doi: 10.1371/journal. (2019) 35:2458–65. doi: 10.1093/bioinformatics/bty984 pone.0186665 33. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Müller MJ. pROC: an 12. Shameer K, Johnson KW, Glicksberg BS, Dudley JT, Sengupta PP. Machine open-source package for R and S+ to analyze and compare ROC curves. BMC learning in cardiovascular medicine: are we there yet? Heart. (2018) 104:1156– Bioinformatics. (2011) 12:77. doi: 10.1186/1471-2105-12-77 64. doi: 10.1136/heartjnl-2017-311198 34. Trevethan R. Sensitivity, specificity, and predictive values: foundations, 13. Motwani M, Dey D, Berman DS, Germano G, Achenbach S, Al-Mallah MH, pliabilities, and pitfalls in research and practice. Front Public Health. (2017) et al. Machine learning for prediction of all-cause mortality in patients with 5:307. doi: 10.3389/fpubh.2017.00307 suspected coronary artery disease: a 5-year multicentre prospective registry 35. Rossello X, Dorresteijn JA, Janssen A, Lambrinou E, Scherrenberg M, analysis. Eur Heart J. (2017) 38:500–7. doi: 10.1093/eurheartj/ehw188 Bonnefoy-Cudraz E, et al. Risk prediction tools in cardiovascular disease 14. Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N. Can machine-learning prevention: a report from the ESC prevention of CVD programme led by improve cardiovascular risk prediction using routine clinical data? PLoS One. the European association of preventive cardiology (EAPC) in collaboration (2017) 12:e0174944. doi: 10.1371/journal.pone.0174944 with the acute cardiovascular care association (ACCA) and the association of 15. Ambale-Venkatesh B, Yang X, Wu CO, Liu K, Hundley WG, McClelland cardiovascular nursing and allied professions (ACNAP). Eur J Prev Cardiol. R, et al. Cardiovascular event prediction by machine learning: the multi- (2019) 26:1534–44. doi: 10.1177/2047487319846715 ethnic study of atherosclerosis. Circ Res. (2017) 121:1092–101. doi: 10.1161/ 36. Pickering JW, Endre ZH. New metrics for assessing diagnostic potential of CIRCRESAHA.117.311312 candidate biomarkers. Clin J Am Soc Nephrol. (2012) 7:10. doi: 10.2215/CJN. 16. Kakadiaris IA, Vrigkas M, Yen AA, Kuznetsova T, Budoff M, Naghavi M. 09590911 Machine learning outperforms ACC / AHA CVD Risk calculator in MESA. 37. Inoue E. nricens: NRI for Risk Prediction Models with Time to Event and Binary J Am Heart Assoc. (2018) 7:e009476. doi: 10.1161/JAHA.118.009476 Response Data. R Package Version 1.6. Vienna: The R Foundation (2013). 17. Alaa AM, Bolton T, Di Angelantonio E, Rudd JHF, van der Schaar M. 38. Kundu S, Aulchenko YS, Cecile A, Janssens JW. PredictABEL: assessment Cardiovascular disease risk prediction using automated machine learning: of risk prediction models. Eur J Epidemiol. (2011) 26:261–4. doi: 10.1007/ a prospective study of 423,604 UK biobank participants. PLoS One. (2019) s10654-011-9567-4 14:e0213653. doi: 10.1371/journal.pone.0213653 39. Pickering JW, Endre ZH. rap: Risk Assessment Plot and Reclassification Metrics. 18. Sánchez-Cabo F, Rossello X, Fuster V, Benito F, Manzano JP, Silla JC, R Package Version 1.03. Vienna: The R Foundation (2021) et al. Machine learning improves cardiovascular risk definition for young, 40. Pruim R, Kaplan DT, Horton NJ. The mosaic package: helping students to asymptomatic individuals. J Am Coll Cardiol. (2020) 76:1674–85. doi: 10.1016/ ‘think with data’ using R. The R J. (2017) 9:26. doi: 10.32614/RJ-2017-024 j.jacc.2020.08.017 41. Visseren FLJ, Mach F, Smulders YM, Carballo D, Koskinas KC, Bäck M, et al. 19. Fors MM, Viada CE, Gonzalez P. Use of recursive partitioning analysis in 2021 Esc guidelines on cardiovascular disease prevention in clinical practice. clinical trials and meta-analysis of randomized clinical trials, 1990-2016. Rev Eur J Prev Cardiol. (2021) 42:3227–337. Recent Clin Trials. (2017) 12:3–7. doi: 10.2174/1574887111666160916144658 42. Dupuis ME, Nadeau-Fredette AC, Madore F, Agharazii M, Goupil R. 20. Junyent M, Martínez M, Borràs M, Coll B, Valdivielso JM, Vidal T, et al. Association of glomerular hyperfiltration and cardiovascular risk in middle- Predicting cardiovascular disease morbidity and mortality in chronic kidney aged healthy individuals. JAMA Netw Open. (2020) 3:e202377. doi: 10.1001/ disease in Spain. the rationale and design of NEFRONA: a prospective, jamanetworkopen.2020.2377 multicenter, observational cohort study. BMC Nephrol. (2010) 11:14. doi: 43. Carbone S, Canada JM, Billingsley HE, Siddiqui MS, Elagizi A, Lavie 10.1186/1471-2369-11-14 CJ. Obesity paradox in cardiovascular disease: where do we stand? 21. Statland BE. A review of the analytic performance of the reflotron system for Vasc Health Risk Manag. (2019) 15:89–100. doi: 10.2147/VHRM.S16 cholesterol testing. Clin Ther. (1990) 12:281–6. 8946 22. Kridde G, Hirschberger J. [The determination of creatinine in blood and 44. Mohammadi H, Ohm J, Discacciati A, Sundstrom J, Hambraeus K, Jernberg plasma of small animals using the dry chemical system reflotron]. Tierarztl T, et al. Abdominal obesity and the risk of recurrent atherosclerotic Prax. (1991) 19:447–9. cardiovascular disease after myocardial infarction. Eur J Prev Cardiol. (2020) 23. Cattozzo G, Franzini C, Hubbuch A, Tritschler W. Evaluation of 27:1944–52. doi: 10.1177/2047487319898019 determination of uric acid in serum and whole blood with the Reflotron. Clin 45. Rossello X, Fuster V, Oliva B, Sanz J, Fernández Friera LA, López- Chem. (1988) 34:414–6. doi: 10.1093/clinchem/34.2.414 Melgar B, et al. Association between body size phenotypes and subclinical 24. Cosentino F, Grant PJ, Aboyans V, Bailey CJ, Ceriello A, Delgado V, et al. atherosclerosis. J Clin Endocrinol Metab. (2020) 105:3734–44. doi: 10.1210/ 2019 ESC guidelines on diabetes, pre-diabetes, and cardiovascular diseases clinem/dgaa620 developed in collaboration with the EASD. Eur Heart J. (2020) 41:255–323. 46. Devesa A, Lobo-González M, Martínez-Milla J, Oliva B, García-Lunar I, 25. Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF, Feldman HI, et al. Mastrangelo A, et al. Bone marrow activation in response to metabolic A new equation to estimate glomerular filtration rate. Ann Intern Med. (2009) syndrome and early atherosclerosis. Eur Heart J. (2022) 43:1809–28. doi: 10. 150:604–12. doi: 10.7326/0003-4819-150-9-200905050-00006 1093/eurheartj/ehac102 26. Touboul PJ, Hennerici MG, Meairs S, Adams H, Amarenco P, Bornstein 47. Yu W, Cheng JD. Uric acid and cardiovascular disease: an N, et al. Mannheim carotid intima-media thickness and plaque consensus update from molecular mechanism to clinical perspective. Frontiers in Cardiovascular Medicine | www.frontiersin.org 13 July 2022 | Volume 9 | Article 895917 Bermúdez-López et al. Personalized Prediction of Severe Atheromatosis Front Pharmacol. (2020) 11:582680. doi: 10.3389/fphar.2020.58 56. Fernández-Alvira JM, Fuster V, Pocock S, Sanz J, Fernández-Friera L, 2680 Laclaustra M, et al. Predicting subclinical atherosclerosis in low-risk 48. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, individuals: ideal cardiovascular health score and fuster-BEWAT Score. J Am et al. Executive summary: heart disease and stroke statistics–2016 update: a Coll Cardiol. (2017) 70:2463–73. doi: 10.1016/j.jacc.2017.09.032 report from the American heart association. Circulation. (2016) 133:447–54. 57. Raiko JR, Magnussen CG, Kivimäki M, Taittonen L, Laitinen T, Kähönen M, doi: 10.1161/CIR.0000000000000366 et al. Cardiovascular risk scores in the prediction of subclinical atherosclerosis 49. Mendelsohn ME, Karas RH. The protective effects of estrogen on the in young adults: evidence from the cardiovascular risk in a young Finns cardiovascular system. N Engl J Med. (1999) 340:1801–11. doi: 10.1056/ study. Eur J Cardiovasc Prev Rehabil. (2010) 17:549–55. doi: 10.1097/HJR. NEJM199906103402306 0b013e3283386419 50. Hermenegildo C, Oviedo PJ, Cano A. Cyclooxygenases regulation by 58. Rossello X, Raposeiras-Roubin S, Oliva B, Sánchez-Cabo F, García-Ruíz JM, estradiol on endothelium. Curr Pharm Des. (2006) 12:205–15. doi: 10.2174/ Caimari F, et al. Glycated hemoglobin and subclinical atherosclerosis in people 138161206775193136 without diabetes. J Am Coll Cardiol. (2021) 77:2777–91. doi: 10.1016/j.jacc. 51. Taddei S, Virdis A, Ghiadoni L, Mattei P, Sudano I, Bernini G, et al. Menopause 2021.03.335 is associated with endothelial dysfunction in women. Hypertension. (1996) 28:576–82. doi: 10.1161/01.HYP.28.4.576 Conflict of Interest: The authors declare that the research was conducted in the 52. Njølstad I, Arnesen E, Lund-Larsen PG. Smoking, serum lipids, blood absence of any commercial or financial relationships that could be construed as a pressure, and sex differences in myocardial infarction. A 12-year follow-up of potential conflict of interest. the Finnmark Study. Circulation. (1996) 93:450–6. doi: 10.1161/01.CIR.93.3. 450 Publisher’s Note: All claims expressed in this article are solely those of the authors 53. Collaboration BPLTT. Pharmacological blood pressure lowering for primary and do not necessarily represent those of their affiliated organizations, or those of and secondary prevention of cardiovascular disease across different levels the publisher, the editors and the reviewers. Any product that may be evaluated in of blood pressure: an individual participant-level data meta-analysis. Lancet. this article, or claim that may be made by its manufacturer, is not guaranteed or (2021) 397:1625–36. endorsed by the publisher. 54. Yusuf S, Hawken S, Ounpuu S, Dans T, Avezum A, Lanas F, et al. Effect of potentially modifiable risk factors associated with myocardial Copyright © 2022 Bermúdez-López, Martí-Antonio, Castro-Boqué, Bretones, Farràs, infarction in 52 countries (the INTERHEART study): case-control Torres, Pamplona, Lecube, Mauricio, Valdivielso and Fernández on behalf of the study. Lancet. (2004) 364:937–52. doi: 10.1016/S0140-6736(04)17 ILERVAS Project Collaborators. This is an open-access article distributed under the 018-9 terms of the Creative Commons Attribution License (CC BY). The use, distribution 55. Conroy RM, Pyörälä K, Fitzgerald AP, Sans S, Menotti A, De Backer G, or reproduction in other forums is permitted, provided the original author(s) and et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the copyright owner(s) are credited and that the original publication in this journal the SCORE project. Eur Heart J. (2003) 24:987–1003. doi: 10.1016/S0195- is cited, in accordance with accepted academic practice. No use, distribution or 668X(03)00114-3 reproduction is permitted which does not comply with these terms. Frontiers in Cardiovascular Medicine | www.frontiersin.org 14 July 2022 | Volume 9 | Article 895917