Received: 8 July 2022 | Accepted: 17 August 2022 DOI: 10.1002/hep4.2091 ORIGINAL ARTICLE Noninvasive predictors of clinically significant portal hypertension in NASH cirrhosis: Validation of ANTICIPATE models and development of a lab-­based model Anahita Rabiee1 | Yanhong Deng2 | Maria Ciarleglio2 | Jean L. Chan3 | Monica Pons4 | Joan Genesca4,5 | Guadalupe Garcia-­Tsao1,6 1 Section of Digestive Diseases, Yale School of Medicine, New Haven, Abstract Connecticut, USA 2 Clinically significant portal hypertension (CSPH), defined as hepatic venous Department of Biostatistics, School of Public Health, Yale University, New pressure gradient (HVPG) ≥ 10 mm Hg, identifies patients with compensated Haven, Connecticut, USA 3 cirrhosis at a high risk of decompensation. However, HVPG is an invasive Conatus Pharmaceuticals at the time of study conduct, San Diego, California, USA and nuanced method. The ANTICIPATE models, which include liver stiff- 4 Liver Unit, Department of Internal ness measurements by transient elastography (TE) and platelet count ± body Medicine, Hospital Universitari Vall d'Hebron, Vall d'Hebron Institut de mass index, are robust noninvasive surrogates of CSPH but required exter- Recerca (VHIR), Vall d'Hebron Barcelona Hospital Campus, Universitat Autònoma nal validation in patients with nonalcoholic steatohepatitis (NASH) cirrhosis. de Barcelona, Barcelona, Spain Additionally, TE is not widely available worldwide. The aims of the study 5 Centro de Investigacion Biomédica en Red de Enfermedades Hepáticas y were: (1) to externally validate the ANTICIPATE models using baseline data Digestivas, Instituto de Salud Carlos III, from patients with compensated NASH cirrhosis screened/enrolled in a mul- Madrid, Spain 6 VA-­CT Healthcare System, New Haven, ticenter international randomized controlled trial; and (2) to develop and ex- Connecticut, USA ternally validate a model using only laboratory values. Regarding aim 1, both Correspondence ANTICIPATE models showed good calibration and discrimination (area under Guadalupe Garcia-­Tsao, VA Connecticut Healthcare System, 950 Campbell the curve [AUC] > 0.8) in our cohort (n = 222). Regarding aim 2, a new lab-­ Avenue, West Haven Campus, West based model using the Fibrosis-­4 index (FIB-­4 [age, aspartate aminotrans- Haven, CT 06516, USA. Email: guadalupe.garcia-tsao@yale.edu ferase, alanine aminotransferase, platelet count]) plus serum albumin was Funding information developed. The discrimination in the training cohort (n = 309) was good (AUC Yale Liver Center, Grant/Award Number: of 0.78 [95% confidence interval [CI]:0.72–­0.83]). It was then externally vali- P30 DK34989 dated in a separate cohort of 245 patients with compensated NASH cirrhosis (AUC of 0.8 [95% CI: 0.75–­0.86]). Given the difference in the prevalence of CSPH between training (74%) and validation (39%) cohorts, the model required an update of the baseline risk to achieve a good calibration. The updated model was named FIB4+. In conclusion, both ANTICIPATE mod- els performed well in predicting the presence of CSPH in NASH cirrhosis. A model using FIB-­4 plus albumin (FIB4+) can be used to predict CSPH where TE is not available. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. © 2022 The Authors. Hepatology Communications published by Wiley Periodicals LLC on behalf of American Association for the Study of Liver Diseases. This article has been contributed to by U.S. Government employees and their work is in the public domain in the USA. 3324 | wileyonlinelibrary.com/journal/hep4  Hepatology Communications. 2022;6:3324–3334. HEPATOLOGY COMMUNICATIONS     | 3325 I NTRO DUCTI O N M ETHO DS Portal hypertension is the main driver of cirrhosis de- Participants compensation, which is defined as the development of ascites, variceal hemorrhage, or hepatic encepha- Baseline data of patients screened/enrolled in a RCT lopathy. Portal pressure is determined by the hepatic of emricasan versus placebo in patients with NASH ­venous pressure gradient (HVPG) with a normal HVPG cirrhosis were included in the current study.[8] The trial ranging between 3 and 5 mm Hg. A study derived from enrolled patients from 59 sites in the United States and the “timolol” randomized controlled trial (RCT), which Europe. The key inclusion criterion for the main trial included 213 patients with compensated cirrhosis, was NASH cirrhosis with an HVPG ≥ 12 mm Hg, and showed that an HVPG ≥ 10 mm Hg was the main pre- 263 patients were randomized in the trial, 188 of whom dictor of development of cirrhosis decompensation.[1] had compensated cirrhosis.[8] However, for the current Therefore, this threshold HVPG now defines the en- study, we also included data from 121 screened patients tity known as clinically significant portal hypertension with compensated cirrhosis who could not be included (CSPH). In addition to its prognostic value, the strat- in the RCT because screening HVPG was < 12 mm Hg. ification of patients with cirrhosis into those with and The sponsoring company, Conatus, kindly provided a without CSPH has important clinical implications, as deidentified database of baseline characteristics of pa- it has been shown that, in patients with cirrhosis and tients included/screened in the trial, including BMI, lab- CSPH, decompensation can be prevented by the use oratory values, LSM (using TE) and HVPG values that of nonselective beta blockers (NSBBs).[2] However, constituted the basis for the current study. For the first obtaining the HVPG requires an invasive procedure aim of the study, the following patients were excluded: and is not readily available in daily clinical practice. (1) those without LSMs, or with poor-­quality LSM de- Recently, liver stiffness measurements (LSMs) ob- fined as LSM with interquartile range > 30% or success tained by transient elastography (TE), a noninvasive rate < 60% on 10 measurements (similar to the criteria test, have been found to be useful in predicting the used by ANTICIPATE and ANTICIPATE-­NASH studies); presence of CSPH.[3–­6] (2) those with poor-­quality HVPG tracing (from an ordinal In a multicenter study by Abraldes et al. (ANTICIPATE scale of excellent/good/fair/poor per the central reader); study), among several noninvasive markers, a com- and (3) those without an available platelet count. For the bination of LSM and platelet count (ANTICIPATE second aim of the study, patients with an incomplete set model) was found to be predictive of CSPH, among of laboratory values at baseline were excluded. other outcomes. This study included patients with External validation cohort consisted of 245 patients LSM ≥ 10 kPa, which defines compensated advanced with compensated NASH cACLD and a LSM ≥ 10 kPa chronic liver disease (cACLD). Most patients had enrolled in the Pons et al. study.[7] This was an interna- hepatitis C (66%), and only a minority (5%) had non- tional cohort of patients from different centers across alcoholic steatohepatitis (NASH).[6] Subsequently a Europe and Canada. Two-­thirds of the cohort consisted multicenter study by Pons et al. aimed to externally of patients with biopsy-­proven NASH enrolled prospec- validate the ANTICIPATE model in patients with tively between 2016 and 2018. The remaining cohort cACLD of all etiologies.[7] In this study, 248 patients consisted of retrospective data from previously pub- (29.7%) had NASH cirrhosis, and the ANTICIPATE lished databases. model did not perform as well in patients with NASH who were obese. Therefore, a model was developed incorporating body mass index (BMI) in addition to Predicted outcome LSM and platelet count (ANTICIPATE-­N ASH model), which performed well in patients with NASH cirrho- The outcome of interest was CSPH at screening/ sis, both obese and nonobese.[7] Our study had two baseline defined as an HVPG ≥ 10 mm Hg. Per the objectives. The first was to externally validate the pre- Emricasan protocol, HVPG was performed by experi- dictive value of the ANTICIPATE and ANTICIPATE-­ enced teams following standard guidelines.[8] Tracings NASH models in determining the presence of CSPH were read independently by a senior experienced in a cohort of patients with compensated NASH cir- ­investigator (GGT). rhosis included in an international multicenter RCT. Because TE is not available in many centers world- wide, our second objective was to determine whether Models a model excluding LSM and including easily obtain- able laboratory indices/values could also predict We first externally validated the ANTICIPATE and CSPH in these patients. ANTICIPATE-­NASH models. 3326 |    NONINVASIVE PREDICTORS OF CLINICALLY SIGNIFICANT PORTAL HYPERTENSION The ANTICIPATE model includes LSM and platelet calibration intercept and slope, which was estimated by count to predict the presence of CSPH in patients with regressing the binary outcome on the log odds (fitted compensated cirrhosis[6] and uses the following for- logit) of the calibration model. The calibration slope has mula: Logit = −6.3320165 + 3.1001014 × ln(LSM) –­ 0.0 a target value of one, and the calibration intercept has 20481545 × Platelet count, with LSM expressed in kilo- a target value of zero.[15] pascals and with platelet count values > 150 introduced Discrimination refers to the model's ability to differ- as 150. entiate between patients with different outcomes.[14] The ANTICIPATE-­NASH model includes LSM, plate- The discrimination was assessed via receiver operat- let count, and BMI to predict the presence of CSPH in ing characteristic (ROC) curves and their correspond- patients with compensated NASH cirrhosis.[7] It uses ing C-­statistics. The two models were first validated in the following formula: Logit = −3.9529402 + 2.2835809 the entire cohort and then in the subset of obese pa- × log(LSM) –­  0.033777725 × BMI –­  0.014490895 × Pl tients with NASH. atelet count, with LSM expressed in kPa. We then developed a logistic regression model in We then developed a model using two markers that which the outcome was CSPH and the predictors were had been shown to predict either CSPH or decompen- predetermined and included FIB-­4 and albumin. The sation in patients with cirrhosis (mostly hepatitis C virus model was internally validated and corrected for opti- or alcohol),[1,9–­11] which consisted of serum albumin and mism by bootstrapping.[16] When bootstrapping, 500 Fibrosis- ­4 index (FIB- ­4).[12] Serum albumin has been data sets of the same size were generated by random found to be predictive of decompensation in two sep- selection with replacement from the main data set. The arate cohorts.[1,9] FIB-­4,[12] which includes age, aspar- discrimination of the model was evaluated by ROC tate aminotransferase (AST), alanine aminotransferase curve analysis, and the reported C-­statistic. Calibration (ALT), and platelet count (using the following formula: was evaluated by plotting the agreement of predicted (Age [year] × AST [U/L]) / (platelet count [109/L]) × (ALT and observed probabilities. We then assessed the diag- [U/L])1/2) has been shown to predict CSPH[11] and de- nostic performance of our model in the validation cohort compensation.[10] Our team also validated FIB-­ 4 for of 245 patients from the Pons et al. study. Diagnostic prediction of CSPH in two separate cohorts using indi- accuracy of the model was evaluated by using the Brier vidual patient data from two previously published stud- score, which is the mean difference between predicted ies[3,13] (validation data available upon request). and observed probabilities. The score ranges from 0 to 1, with smaller scores indicating higher accuracy. Finally, a nomogram was developed based on the final Variables corrected logistic regression model. p values ≤ 0.05 were considered statistically significant. Data were analyzed LSM was measured by TE (FibroScan) using either M using SAS version 9.4 (SAS Institute) and R Statistical or XL probes. Screening demographic and laboratory Software (R Core Team [2020], R: A language and en- data collected for the study were used. We defined vironment for statistical computing; R Foundation for obesity as BMI ≥ 30. Statistical Computing) using the rms (Harrell FE Jr. rms: Regression Modeling Strategies: R Package Version 5.1–­2. 2018; https://CRAN.R-­proje​ct.org/packa​ge5rms) Statistical analysis and ggplot2 (Wickham H. ggplot2: Elegant Graphics for Data Analysis, 2016; Springer-­Verlag) packages. For the first aim, we assessed the performance of the ANTICIPATE and the ANTICIPATE-­NASH models by evaluating the model's calibration and discrimination in R ESULTS our cohort using the methodology described by Moons et al.[14] Calibration is the agreement between observed AIM 1: External validation of ANTICIPATE and predicted risks.[14] Calibration was assessed by (1) and ANTICIPATE-­NASH models “calibration-­in-­the-­large,” which compares the average predicted risk with the overall event rate (i.e., when the A total of 222 patients were included in the analysis average predicted risk is higher than the overall event (Figure 1); 165 of 222 (74%) had CSPH and 171 of rate, the model overestimates risk in general; and when 222 (77%) were obese. CSPH was present in 129 of it is lower, the model underestimates the risk); (2) cali- 171 (75%) obese patients and in 36 of 51 (71%) non- bration plot, which plots the observed and predicted obese patients. Baseline demographics of patients are risks across deciles of predicted risk, and a measure summarized in Table 1. The prevalence of CSPH was of goodness-­ of-­ fit assessed using the Hosmer and somewhat higher in our cohort (74%) compared with the Lemeshow test (a p value > 0.05 or failure to reject the ANTICIPATE study (66%),[6] and much higher than that null hypothesis of adequate fit indicates insufficient evi- observed in patients with NASH in the ANTICIPATE-­ dence to conclude lack of fit or poor calibration); and (3) NASH study (39%)[7] (Table S1). HEPATOLOGY COMMUNICATIONS     | 3327 ANTICIPATE model of 222 [74%]); therefore the model showed good cali- bration in-­the-­large. The calibration intercept was not The predicted probability of CSPH using the model in significantly different from 0 (−0.0353, 95% confidence our population was 171 of 222 (77%), which was very interval [CI] −0.4760 to 0.4053), and the calibration close to the observed proportion of CSPH cases (164 slope was not significantly different from 1 (0.7713, 95% CI 0.5351 and 1.0075), all indicative of a good calibra- tion. The p value for the goodness-­of-­fit test (Hosmer Lemeshow test) was 0.26, indicative of absence of poor calibration. The model discrimination was evaluated by ROC curves. The area under the curve (AUC) was 0.8213 (95% CI 0.7575–­0.8850), indicating good dis- crimination (Figure 2). Given the performance of the model, an update did not seem necessary; however, with a small change to the baseline log odds (adding −0.2781 to the in- tercept), the calibration in-­ the-­ large was improved to predict CSPH in 164 of 222 (74%), which was the same as observed (164 of 222). It is important to note, however, that calibration in-­the-­large does not necessarily mean that the model was able to predict the exact same patients who had the outcome, and although the numbers are identical, the model could be identifying a patient who does not have CSPH as having CSPH or vice versa. We also checked the calibration of the model in the obese subpopulation (n = 171; Figure S1) and in three different BMI categories (nonobese [BMI < 30; n = 51], obesity class I [BMI between 30 and 35; n = 86], and F I G U R E 1 Study flow chart. Abbreviations: CSPH, clinically significant portal hypertension; HVPG, hepatic venous pressure obesity classes II and III [BMI ≥ 35; n = 85]). The results gradient; IQR, interquartile range; LSM, liver stiffness measurement. are summarized in Figure S2. TA B L E 1 Baseline characteristics of the cohort for Aim 1: Validation of ANTICIPATE models Median (IQR) or n (%)a No CSPH (n = 58) CSPH (n = 164) Total (n = 222) p value Age (years) 60 (54–­68) 62 (56–­68) 62 (56–­68) 0.41 Gender (female) 29 (50%) 94 (57%) 123 (55%) 0.34 Race (Caucasian)b 49 (85%) 146 (90%) 195 (88%) 0.09 Type 2 diabetes 36 (62%) 123 (75%) 159 (72%) 0.06 NSBB use 6 (10%) 49 (30%) 55 (25%) 0.003 Obese (BMI ≥ 30) 43 (74%) 128 (78%) 171 (77%) 0.54 BMI (kg/m2) 32 (30–­36) 34 (31–­38) 33 (30–­38) 0.15 Albumin (g/L) 4.4 (4.0– ­4.5) 4.1 (3.8– ­4.4) 4.1 (3.8– ­4.4) 0.001 ALT (U/L) 44 (26–­51) 33 (25–­45) 35 (25–­48) 0.013 Creatinine (mg/dl) 0.7 (0.6– ­0.9) 0.6 (0.5– ­0.8) 0.7 (0.5– ­0.8) 0.013 Platelets (K/mm3) 144 (110–­179) 95 (71–­125) 104 (75–­143) <0.001 INRb 1.1 (1.0–­1.2) 1.1 (1.1–­1.2) 1.1 (1.1–­1.2) <0.001 AST (U/L) 41 (30–­57) 44 (33–­57) 43 (32–­57) 0.30 MELD- ­Na score 7 (6–­8) 8 (7–­10) 8 (7–­9) <0.001 CTP scoreb 5 (5–­5) 5 (5–­5) 5 (5–­5) 0.1606 LSM (kPa) 21 (12–­27) 34 (23–­48) 28 (21–­42) <0.001 Abbreviations: ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; CSPH, clinically significant portal hypertension; CTP, Child-­Turcotte-­Pugh score; INR, international normalized ratio; MELD-­Na, Model for End-­Stage Liver Disease–­Sodium; NSBB, nonselective beta blocker. a Continuous variables are summarized using median (IQR), and categorical variables are summarized using the number of patients (percent of total). b Missing values: Race not available for 1 patient in the no-­CSPH group; INR was not available in 2 patients in the CSPH group; CTP was not available in 1 patient in the no-­CSPH group and 4 patients in the CSPH group. 3328 |    NONINVASIVE PREDICTORS OF CLINICALLY SIGNIFICANT PORTAL HYPERTENSION F I G U R E 2 ANTICIPATE model performance in the entire cohort (n = 222). Left panel: Receiver operating characteristic (ROC) curves showing the discriminative value of the ANTICIPATE model. The 45-­degree line is the line of no discrimination. Right panel: Calibration plots. The 45-­degree line denotes the perfect agreement between predicted and observed risk. The gray line approximates the agreement between predicted and observed risks across subgroups of participants ranked by increasing predicted risks. The rug histogram shows the distribution of patients' risks in the cohort. ANTICIPATE-­NASH model calibration. The p value for the goodness-­of-­ fit test (Hosmer Lemeshow test) was 0.21, which is indica- Using the ANTICIPATE-­NASH model in our population, tive of absence of poor calibration. The model's dis- the predicted probability of CSPH was 151 of 222 (68%), crimination was good with an AUC of 0.8174 (95% CI as opposed to the observed proportion of CSPH, which 0.7445–­0.8903) (Figure 3). was 164 of 222 (74%); therefore, the model slightly un- derestimated the presence of CSPH. The calibration intercept was 0.3735 (95% CI −0.00496 to 0.7520), and AIM 2: Development of a model using only the calibration slope was 1.0301 (95% CI 0.7127 and laboratory values 1.3475), which indicates good calibration. The p value from the goodness-­of-­fit test (Hosmer Lemeshow test) Using prespecified parameters including FIB-­ 4 (age was 0.51, which is indicative of absence of poor cali- [years] × AST [U/L])/(platelet count [109/L]) × (ALT bration. The model also showed good discrimination as [U/L]1/2) and serum albumin, which are easily available evaluated by ROC curves. The AUC was 0.8182 (95% using lab tests, we developed a model to predict CSPH CI 0.7563–­0.8802) (Figure S3). (called “FIB4+”). Because this lab-­based model does Given the performance of the model, an update did not use LSM, we were able to include all 309 patients not seem necessary; however, with a small change to screened/enrolled in the trial, even those without LSM the baseline log-­odds (adding +0.0131 to the intercept), or those with poor-­quality LSM (Figure 1). Baseline de- the calibration in-­the-­large was improved to predict 164 mographics of the study participants for Aim 2 are sum- of 222 (74%). marized in Table 2. The prevalence of CSPH was 74%. We then applied the ANTICIPATE-­NASH model to In the lab-­based model, albumin did not reach statis- only the obese subpopulation (n = 171). The predicted tical significance, however we chose to retain it, given probability of CSPH was 115 of 171 (67%) compared its clinical importance in the prediction of decompensa- with the observed proportion of CSPH, which was 128 tion.[1,9] The model showed good discrimination with an of 171 (75%); therefore, the calibration in-­ the-­ large AUC of 0.78 (95% CI 0.7193–­0.8343) in the training co- shows that the model slightly underestimated the prob- hort (Figure S4). We then internally validated the model ability of CSPH. The calibration intercept was 0.5039 by the bootstrapping method (500 replications). (The (95% CI 0.0824–­ 0.9253) and calibration slope was model formula and the bootstrapping validation are 0.9942 (95% CI 0.6462–­1.3423), which indicate good shown in Figure S4). We additionally externally validated HEPATOLOGY COMMUNICATIONS     | 3329 F I G U R E 3 ANTICIPATE–­NASH performance in the obese subpopulation (n = 171). Left panel: Receiver operating characteristic (ROC) curves showing the discriminative value of the ANTICIPATE-­NASH model. The 45-­degree line is the line of no discrimination. Right panel: Calibration plots. The 45-­degree line denotes the perfect agreement between predicted and observed risk. The gray line approximates the agreement between predicted and observed risks across subgroups of participants ranked by increasing predicted risks. The rug histogram shows the distribution of patients' risks in the cohort. TA B L E 2 Baseline characteristics of the cohort for Aim 2: Development of FIB4+ model Median (IQR) or n (%)a No CSPH (n = 81) CSPH (n = 228) Total (n = 309) p value Age (years) 60 (53–­67) 62 (56–­68) 62 (56–­68) 0.32 Gender (female) 43 (53%) 128 (56%) 171 (55%) 0.63 Race (Caucasian)b 72 (89%) 208 (92%) 280 (91%) 0.09 Type 2 diabetes 52 (64%) 176 (77%) 228 (74%) 0.022 NSBB use 11 (14%) 71 (31%) 82 (27%) 0.002 Obese (BMI ≥ 30) 61 (75%) 178 (78%) 239 (77%) 0.61 BMI (kg/m2) 32 (30–­37) 34 (31–­39) 34 (30–­39) 0.06 Albumin (g/L) 4.4 (4.1– ­4.5) 4.0 (3.7– ­4.4) 4.1 (3.8– ­4.4) <0.001 ALT (U/L) 43 (24–­51) 33 (24– ­46) 35 (24–­48) 0.048 Creatinine (mg/dl) 0.7 (0.6– ­0.8) 0.6 (0.5– ­0.8) 0.6 (0.5– ­0.8) 0.035 Platelets (K/mm3) 141 (104–­179) 94 (71–­124) 102 (75–­140) <0.001 INRb 1.1 (1–­1.1) 1.1 (1.1–­1.2) 1.1 (1–­1.2) <0.001 AST (U/L) 39 (27–­56) 43 (32–­56) 41 (31–­56) 0.040 MELD- ­Na score 7 (6–­8) 8 (7–­10) 8 (7–­9) <0.001 CTP scoreb 5 (5–­5) 5 (5–­5) 5 (5–­5) 0.0126 LSM (kPa)b 21 (12–­28) 32 (22–­48) 28 (19–­41) <0.001 Abbreviations: ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; CSPH, clinically significant portal hypertension; CTP, Child-­Turcotte-­Pugh score; INR, international normalized ratio; MELD-­Na, Model for End-­Stage Liver Disease–­Sodium; NSBB, nonselective beta blocker. a Continuous variables are summarized using median (IQR), and categorical variables are summarized using the number of patients (percent of total). b Missing values: Race not available for 1 patient in the no-­CSPH group; LSM was not available for 14 patients in the no-­CSPH and in 27 patients in the CSPH group; INR was not available in 4 patients in the CSPH group; CTP was not available in 1 patient in the no-­CSPH group and 4 patients in the CSPH group. our model in 245 patients with NASH cACLD from the validation cohort (Figure 4). The model's initial reliability Pons et al. group. The model showed good discrimina- plot in the validation cohort grossly overestimated the tion with AUC of 0.8039 (95% CI 0.7480–­0.8599) in the presence of CSPH (predicted 63% vs. observed: 39%); 3330 |    NONINVASIVE PREDICTORS OF CLINICALLY SIGNIFICANT PORTAL HYPERTENSION F I G U R E 4 FIB4+ model performance in the validation cohort (n = 245). Top panel: Receiver operating characteristic (ROC) curves showing the discriminative value of the FIB4+ model. The 45-­degree line is the line of no discrimination. Bottom-­left panel: Calibration plots before updating the model. The 45-­degree line denotes the perfect agreement between predicted and observed risk. The gray line approximates the agreement between predicted and observed risks across subgroups of participants ranked by increasing predicted risks. The rug histogram shows the distribution of patients' risks in the validation cohort. Bottom-­right panel: Calibration plots after updating the model. Abbreviation: AUC, area under the curve. however, the overestimation was consistent across all The calibration in-­the-­large in the updated lab-­based risk categories (Figure 4). Therefore, the lab-­ based model was 95 of 248 (38%), very close to the observed model was updated by adding −1.2621 to the baseline 97 of 248 (39%). The calibration intercept after the up- risk to account for the difference in the prevalence of date was −0.0212 (95% CI −0.3584 to 0.3161), and CSPH (74% vs. 39% in the derivation and validation calibration slope (before and after the update) was cohorts, respectively), resulting in a good calibration. 0.9610 (95% CI 0.6465–­1.2754), which indicates good HEPATOLOGY COMMUNICATIONS     | 3331 in performance of the ANTICIPATE models in our co- hort as compared with the Pons et al. cohort. First, this could be related to a higher prevalence of CSPH in our cohort, which was 74% compared with 66% in the ANTICIPATE study,[6] and 39% in the NASH population of the ANTICIPATE-­NASH study.[7] This higher preva- lence is likely due to the fact that patient enrollment in our cohort was skewed toward patients with a higher FIGURE 5 FIB4+ model nomogram likelihood of having CSPH (to meet the trial eligibility) before undergoing HVPG measurement. Nevertheless, in prior series of patients with compensated cirrho- calibration. The p value for the Hosmer Lemeshow sis from multiple etiologies, the prevalence of CSPH goodness-­of-­fit test was 0.29, which also indicates no has been reported to range from 63% to 73%.[3,21–­25] evidence of poor calibration. The Brier score for the up- Second, it could be that when the population with NASH dated FIB4+ model in the validation cohort was 0.169, presents with very advanced disease (compensated which indicates that the model provides accurate pre- cirrhosis with high HVPG), the correlation of LSM and dictions for the presence of CSPH. The final calibrated HVPG behaves more like other etiologies of cirrhosis. FIB4+ model uses the following formula: log odds Although the nomogram from the original studies (CSPH) = 0.7207 –­ (0.6729 × albumin) + (0.4408 × FIB-­4). provides a more accurate assessment of each individ- We then developed a nomogram using the updated ual patient's risk of having CSPH, Baveno consensus FIB4+ model, which estimates the risk of having CSPH has proposed simplified prediction cutoffs to rule in based on each patient's individual data (Figure 5). CSPH in patients with viral or alcohol-­related etiologies for a greater applicability in clinical practice. These cut- offs include LSM ≥ 25 kPa, LSM 20–­25 kPa with platelet D I SCUSS I O N count ≤ 150K, and LSM 15–­20 kPa with platelet count ≤ 110K.[18,26] However, Pons et al. cautioned against Noninvasive methods for detection of CSPH in patients the use of a single cutoff of LSM ≥ 25 kPa in patients with cirrhosis are critical, as the paradigm is shifting with NASH, particularly in patients with obesity, as it toward starting treatment with NSBBs once CSPH is significantly overestimated the presence of CSPH in detected, to prevent decompensation.[17,18] their population. In their study, only 63% of patients with obese NASH with LSM ≥ 25 actually had CSPH, whereas in patients with cirrhosis of viral or alcohol-­ Aim 1: Validation of ANTICIPATE models related etiologies, more than 90% of patients with this cutoff had CSPH. In our cohort, CSPH was present in In the validation study by Pons et al., although the 114 of 132 (86%) of patients with LSM ≥ 25 in the en- ANTICIPATE model had excellent performance in tire cohort and in 91 of 105 (87%) of the obese sub- patients with viral, alcohol, and nonobese NASH eti- population. Among patients with LSM 20–­25 kPa and ologies, when it was applied to the obese NASH popu- platelet count of ≤150K, CSPH was present in 24 of lation it overestimated the presence of CSPH. NASH 32 (75%), and among patients with LSM 15–­ 20 kPa population has been challenging for many reasons. It and platelet count ≤110K, CSPH was present in 12 of has been underrepresented in all previous studies, and 17 (71%) of patients. Nevertheless, and similar to the there are specific aspects of this population, such as Pons et al. study, the performance of a single cutoff technical difficulty with performing elastography[19] and of LSM ≥ 25 kPa was fairly poor in our cohort of pa- possible increase in liver stiffness with severe steatosis tients with NASH with an AUC of only 0.69, sensitiv- and steatohepatitis, which makes accurate assessment ity of 70%, specificity of 69%, positive predictive value of LSM challenging.[20] (PPV) of 86%, and negative predictive value (NPV) of Our study was able to externally validate both the 44% (Figure S5). Interestingly, despite the high preva- ANTICIPATE and ANTICIPATE-­ NASH models. The lence of CSPH in our cohort, which would significantly discrimination of both models was >0.8 in the entire co- increase the PPV of a test, the PPV was only 86%. hort and in the obese subpopulation. In terms of calibra- In summary, per the first aim of our study, we tion, the ANTICIPATE model slightly overestimated the were able to validate both the ANTICIPATE and presence of CSPH (calibration in the large: expected ANTICIPATE-­ NASH models in our cohort of NASH event rate 77% vs. 74% observed event rate), but the cirrhosis, with and without obesity. Although we per- ANTICIPATE-­NASH slightly underestimated the pres- formed a minor update to the intercept to fit our cohort, ence of CSPH in our cohort (calibration in the large: we still recommend using the original ANTICIPATE expected event rate 68% vs. 74% observed event rate). and ANTICIPATE-­NASH formula/nomogram, as these There are two potential explanations for the difference cohorts are more representative of patients seen in 3332 |    NONINVASIVE PREDICTORS OF CLINICALLY SIGNIFICANT PORTAL HYPERTENSION clinical practice compared with those selected for a of each variable or addition of a new variable) were randomized controlled trial.[6,7] needed to improve its calibration. Because, as men- tioned previously, the validation cohort (Pons et al. cohort) is more representative of patients with NASH Aim 2: Development of FIB4+ model seen in the outpatient clinics compared with those in our training cohort (preselected for a RCT), we recom- Because many centers in the United States and world- mend the use of the updated model, which we call the wide do not have the resources to acquire a TE device FIB4+ model in clinical practice. Based on our FIB4+ to perform LSMs, we considered it of great importance nomogram (Figure 5), patients with FIB-­4 ≥ 6 and albu- to develop a model that would forego LSMs and would min < 3.6 have a risk of CSPH > 75%. To facilitate this use readily available laboratory test results. Similar to calculation, we have created an online calculator that LSM, the FIB-­4 has been used to determine the pres- includes the five variables of the FIB4+ model (http:// ence of advanced liver fibrosis/cirrhosis in patients with www.fib4p​lus.com). chronic liver disease. FIB-­4 has also been shown to be a strong predictor of presence of varices (an indi- cator of CSPH) in patients with compensated cirrho- Strengths and limitations of our study sis.[27] This is not surprising, as, in its formula it includes AST (a marker of endothelial dysfunction) and platelet The strengths of our study are the inclusion of a large count, which according to the ANTICIPATE study was number of patients with NASH cirrhosis, which is cur- the second-­most important predictor of CSPH (after rently the most common etiology of chronic liver dis- LSM) and could solely predict CSPH.[28] FIB-­ 4 has ease,[29] with a significant proportion of patients with been used previously in patients with other etiologies of CSPH, which allowed for the validation of previous cirrhosis for prediction of CSPH.[11] Additionally, FIB-­4 models. The development of an inexpensive model has been used to predict clinical outcomes and is an that can be applied in any center and the external important predictor of decompensation in patients with validation of this model in a large sample of patients compensated cirrhosis.[10] from a very diverse setting is also a major strength.[30] In fact, in an as-­yet-­unpublished study in which our The fact that our FIB4+ model performed well in a group analyzed data obtained from the timolol study, separate cohort from different institutions/centers and in which the prevalence of CSPH was 63% (139 of with very different baseline characteristics suggests 219)[13] and validated it in a second cohort,[3] we found that our model can be generalizable to other popula- that FIB-­4 had a very good discriminative ability and tions of patients with compensated NASH cirrhosis. calibration for prediction of CSPH. We added serum Another major strength of our study is evaluation of ­albumin because, after HVPG, it was the most import- the performance characteristics of previous models ant predictor of the development of decompensation in and our own model, not only by discrimination (ROC patients with compensated cirrhosis.[1,9] curves and their corresponding C-­statistics) but also Although in the training cohort the addition of al- by detailed evaluation of different aspects of calibra- bumin to the model did not change its discriminatory tion (calibration in the large or the calibration intercept ability, in the validation cohort the model including al- and the calibration slope). bumin performed better (AUC = 0.794 [FIB-­4 alone] Our study has several limitations. First, the data vs. 0.8039 [FIB4+]). Another reason for keeping the used for our study were collected as part of screen- albumin in the final model is its potential for prediction ing for enrollment of patients in a RCT. The rea- of outcomes.[9] Because predicting CSPH is a surro- son for higher prevalence of CSPH in our cohort is gate in the prediction of cirrhosis decompensation, that enrollment was skewed toward patients with we think that maintaining albumin in the model will higher likelihood of having CSPH, before undergo- strengthen its ability to predict outcomes, although ing HVPG measurement. Given the slow recruitment this needs to be further investigated in future pro- (a large proportion of patients undergoing HVPG spective studies. measurement did not meet the criteria for study in- While the discrimination of our model was 0.8 in the clusion [HVPG ≥ 12]), the study added another inclu- validation cohort, calibration was not good, and the sion criterion, which was a platelet count ≤ 125K or model overestimated the presence of CSPH in all risk LSM ≥ 20 kPa during screening. This change in proto- categories (predicted risks were systematically too high col was implemented halfway through the study and but remained proportionally accurate). By adjusting the occurred at different times at each study center due to baseline risk of our original prediction model to that of the internal review board approval process; therefore, individuals included in the validation cohort, the calibra- we could not identify the patients enrolled before or tion was easily improved without changing its discrim- after this change in protocol. This skewed the popula- inatory ability. It is important to mention that no other tion toward patients with higher pretest probability of updates to the model (such as change in the coefficient CSPH. Although this could affect the PPV and NPV, HEPATOLOGY COMMUNICATIONS     | 3333 the baseline prevalence does not affect sensitivity REFERENCES and specificity, and thus the overall discrimination of 1. Ripoll C, Groszmann R, Garcia-­Tsao G, Grace N, Burroughs A, the model. Furthermore, our model still showed ac- Planas R, et al. Hepatic venous pressure gradient predicts clin- ical decompensation in patients with compensated cirrhosis. ceptable performance characteristics in a separate Gastroenterology. 2007;133:481– ­8. validation cohort with a significantly lower prevalence 2. Villanueva C, Albillos A, Genescà J, Garcia-­Pagan JC, Calleja of CSPH. Second, the patients could have been on JL, Aracil C, et al. β blockers to prevent decompensation of cir- NSBBs, which could have artificially decreased the rhosis in patients with clinically significant portal hypertension portal pressure gradient. Although NSBBs were held (PREDESCI): a randomised, double-­blind, placebo-­controlled, multicentre trial. Lancet. 2019;393:1597–­608. 24 h before HVPG measurement, this could have still 3. Berzigotti A, Seijo S, Arena U, Abraldes JG, Vizzutti F, García–­ decreased the portal pressures and misclassified Pagán JC, et al. Elastography, spleen size, and platelet count them as not having CSPH, while in fact their HVPG identify portal hypertension in patients with compensated cir- would have been ≥10 if measured while not being on rhosis. Gastroenterology. 2013;144:102–­11.e1. NSBBs. Finally, the updated FIB4+ model requires 4. Bureau C, Metivier S, Peron JM, Selves J, Robic MA, Gourraud PA, et al. Transient elastography accurately predicts presence further validation in a separate cohort of patients with of significant portal hypertension in patients with chronic liver NASH cirrhosis to demonstrate generalizability. disease: investigating the performance of transient elastogra- phy. Aliment Pharmacol Ther. 2008;27:1261–­8. 5. Lemoine M, Katsahian S, Ziol M, Nahon P, Ganne-­Carrie N, Conclusions Kazemi F, et al. Liver stiffness measurement as a predictive tool of clinically significant portal hypertension in patients with compensated hepatitis C virus or alcohol-­ related cirrhosis. Our study showed that both ANTICIPATE and Aliment Pharmacol Ther. 2008;28:1102–­10. ANTICIPATE-­NASH models perform very well in pa- 6. Abraldes JG, Bureau C, Stefanescu H, Augustin S, Ney M, tients with NASH cirrhosis, with good calibration and Blasco H, et al. Noninvasive tools and risk of clinically signifi- discrimination. The ANTICIPATE-­ NASH model also cant portal hypertension and varices in compensated cirrhosis: performed well in the subset of patients with obesity. the “Anticipate” study. Hepatology. 2016;64:2173–­8 4. 7. Pons M, Augustin S, Scheiner B, Guillaume M, Rosselli M, We were also able to develop a simple model using Rodrigues SG, et al. Noninvasive diagnosis of portal hyper- FIB-­4 and albumin to predict CSPH in patients with tension in patients with compensated advanced chronic liver compensated NASH cirrhosis and externally validate disease. Am J Gastroenterol. 2021;116:723–­32. and update the model in a separate validation cohort 8. Garcia-­Tsao G, Bosch J, Kayali Z, Harrison SA, Abdelmalek (Pons et al. cohort). The use of our final updated model, MF, Lawitz E, et al. Randomized placebo-­controlled trial of em- ricasan for non-­alcoholic steatohepatitis-­related cirrhosis with FIB4+, will be particularly useful for centers in the world severe portal hypertension. J Hepatol. 2020;72:885–­95. that lack TE to perform LSM. 9. Ripoll C, Bari K, Garcia-­Tsao G. Serum albumin can identify patients with compensated cirrhosis with a good prognosis. J ACKNOWLEDGMENT Clin Gastroenterol. 2015;49:613–­9. The authors thank the following investigators for 10. Guha IN, Harris R, Berhane S, Dillon A, Coffey L, James MW, et al. Validation of a model for identification of patients with kindly sharing their data from the ANTICIPATE-­N ASH compensated cirrhosis at high risk of decompensation. Clin study: Salvador Augustin, Bernhard Scheiner, Maeva Gastroenterol Hepatol. 2019;17:2330–­8.e1. Guillaume, Matteo Rosselli, Susana G. Rodrigues, 11. Procopet B, Cristea VM, Robic MA, Grigorescu M, Agachi PS, Horia Stefanescu, Mang M. Ma, Mattias Mandorfer, Metivier S, et al. Serum tests, liver stiffness and artificial neural Mayka Mergeay-­ Fabre, Bogdan Procopet, Philipp networks for diagnosing cirrhosis and portal hypertension. Dig Liver Dis. 2015;47:411–­6. Schwabl, Arnulf Ferlitsch, Georg Semmler, Annalisa 12. Sterling RK, Lissen E, Clumeck N, Sola R, Correa MC, Berzigotti, Emmanuel Tsochatzis, Christophe Montaner J, et al. Development of a simple noninvasive index Bureau, Thomas Reiberger, Jaime Bosch, and Juan to predict significant fibrosis in patients with HIV/HCV coinfec- G. Abraldes. tion. Hepatology. 2006;43:1317–­25. 13. Groszmann RJ, Garcia-­ Tsao G, Bosch J, Grace ND, Burroughs AK, Planas R, et al. Beta-­blockers to prevent gas- F U N D I N G I N F O R M AT I O N troesophageal varices in patients with cirrhosis. N Engl J Med. Supported by the Yale Liver Center, National Institutes 2005;353:2254– ­61. of Health (P30 DK34989). 14. Moons KGM, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External val- CONFLICT OF INTEREST idation, model updating, and impact assessment. Heart. 2012;98:691– ­8. Nothing to report. 15. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW, on behalf of Topic Group “Evaluating diag- ORCID nostic tests and prediction models” of the STRATOS initiative. Anahita Rabiee https://orcid. Calibration: the Achilles heel of predictive analytics. BMC Med. org/0000-0002-8833-3592 2019;17:230. 16. Steyerberg EW, Harrell FE Jr, Borsboom GJJM, Eijkemans Joan Genesca https://orcid. MJC, Vergouwe Y, Habbema JDF. Internal validation of predic- org/0000-0002-0831-8422 tive models. J Clin Epidemiol. 2001;54:774–­81. 3334 |    NONINVASIVE PREDICTORS OF CLINICALLY SIGNIFICANT PORTAL HYPERTENSION 17. Garcia-­ Tsao G, Abraldes JG. Nonselective beta-­ blockers in 26. de Franchis R. Expanding consensus in portal hypertension. J compensated cirrhosis: preventing variceal hemorrhage or pre- Hepatol. 2015;63:743– ­52. venting decompensation? Gastroenterology. 2021;161:770–­3. 27. Deng H, Qi X, Guo X. Diagnostic accuracy of APRI, AAR, FIB-­ 18. de Franchis R, Bosch J, Garcia-­Tsao G, Reiberger T, Ripoll C, 4, FI, King, Lok, Forns, and fibroindex scores in predicting the Abraldes JG, et al. Baveno VII—­renewing consensus in portal presence of esophageal varices in liver cirrhosis: a systematic hypertension. J Hepatol. 2022;76:959–­74. review and meta-­analysis. Medicine. 2015;94:e1795. 19. Castéra L, Foucher J, Bernard PH, Carvalho F, Allaix D, 28. Abraldes JG, Garcia-­Tsao G. Simple clinical tools to predict Merrouche W, et al. Pitfalls of liver stiffness measurement: a decompensation in patients with compensated cirrhosis: an 5-­year prospective study of 13,369 examinations. Hepatology. unmet need. Clin Gastroenterol Hepatol. 2019;17:2179–­81. 2010;51:828–­35. 29. Moon AM, Singal AG, Tapper EB. Contemporary epidemiol- 20. Petta S, Maida M, Macaluso FS, di Marco V, Cammà C, ogy of chronic liver disease and cirrhosis. Clin Gastroenterol Cabibi D, et al. The severity of steatosis influences liver stiff- Hepatol. 2020;18:2650– ­6 6. ness measurement in patients with nonalcoholic fatty liver 30. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. disease: steatohepatitis/metabolic liver disease. Hepatology. Substantial effective sample sizes were required for external 2015;62:1101–­10. validation studies of predictive logistic regression models. J 21. Villanueva C, Albillos A, Genescà J, Abraldes JG, Calleja JL, Clin Epidemiol. 2005;58:475–­83. Aracil C, et al. Development of hyperdynamic circulation and response to β-­ blockers in compensated cirrhosis with por- S U P P O R T I N G I N F O R M AT I O N tal hypertension: liver failure/cirrhosis/portal hypertension. Hepatology. 2016;63:197–­206. Additional supporting information can be found online 22. Reiberger T, Ferlitsch A, Payer BA, Pinter M, Schwabl P, Stift J, in the Supporting Information section at the end of this et al. Noninvasive screening for liver fibrosis and portal hyper- article. tension by transient elastography—­a large single center expe- rience. Wien Klin Wochenschr. 2012;124:395–­402. 23. Augustin S, Millán L, González A, Martell M, Gelabert A, Segarra A, et al. Detection of early portal hypertension with How to cite this article: Rabiee A, Deng Y, routine data and liver stiffness in patients with asymptomatic liver disease: a prospective study. J Hepatol. 2014;60:561–­9. Ciarleglio M, Chan JL, Pons M, Genesca J, et al. 24. Colecchia A, Montrone L, Scaioli E, Bacchi–­Reggiani ML, Colli Noninvasive predictors of clinically significant A, Casazza G, et al. Measurement of spleen stiffness to eval- portal hypertension in NASH cirrhosis: Validation uate portal hypertension and the presence of esophageal var- of ANTICIPATE models and development of a ices in patients with HCV-­related cirrhosis. Gastroenterology. lab-based model. Hepatol Commun. 2022;6:​ 2012;143:646–­5 4. 25. Kitson MT, Roberts SK, Colman JC, Paul E, Button P, Kemp W. 3324–­3334. https://doi.org/10.1002/hep4.2091 Liver stiffness and the prediction of clinically significant portal hypertension and portal hypertensive complications. Scand J Gastroenterol. 2015;50:462– ­9.