Article Open Access Machine Learning Improves Risk Stratification in Myelofibrosis: An Analysis of the Spanish Registry of Myelofibrosis Adrián Mosquera-Orgueira1, Manuel Pérez-Encinas1, Alberto Hernández-Sánchez2, Teresa González-Martínez2, Eduardo Arellano-Rodrigo3, Javier Martínez-Elicegui2, Ángela Villaverde-Ramiro2, José-María Raya4, Rosa Ayala5, Francisca Ferrer-Marín6, María-Laura Fox7, Patricia Velez8, Elvira Mora9, Blanca Xicoy10, María-Isabel Mata-Vázquez11, María García-Fortes12, Anna Angona13, Beatriz Cuevas14, María-Alicia Senín15, Angel Ramírez-Payer16, María-José Ramírez17, Raúl Pérez-López18, Sonia González de Villambrosía19, Clara Martínez-Valverde20, María-Teresa Gómez-Casares21, Carmen García-Hernández22, Mercedes Gasior23, Beatriz Bellosillo8, Juan-Luis Steegmann24, Alberto Álvarez-Larrán3, Jesús María Hernández-Rivas2, Juan Carlos Hernández-Boluda25, on behalf of the Spanish MPN Group (GEMFIN). Correspondence: Adrián Mosquera-Orgueira and Juan Carlos Hernández-Boluda (adrian.mosquera.orgeira@ sergas.es; hernandez_jca@gva.es). ABSTRACT Myelofibrosis (MF) is a myeloproliferative neoplasm (MPN) with heterogeneous clinical course. Allogeneic hematopoietic cell transplan- tation remains the only curative therapy, but its morbidity and mortality require careful candidate selection. Therefore, accurate disease risk prognostication is critical for treatment decision-making. We obtained registry data from patients diagnosed with MF in 60 Spanish institutions (N = 1386). These were randomly divided into a training set (80%) and a test set (20%). A machine learning (ML) technique (random forest) was used to model overall survival (OS) and leukemia-free survival (LFS) in the training set, and the results were validated in the test set. We derived the AIPSS-MF (Artificial Intelligence Prognostic Scoring System for Myelofibrosis) model, which was based on 8 clinical variables at diagnosis and achieved high accuracy in predicting OS (training set c-index, 0.750; test set c-index, 0.744) and LFS (training set c-index, 0.697; test set c-index, 0.703). No improvement was obtained with the inclusion of MPN driver mutations in the model. We were unable to adequately assess the potential benefit of including adverse cytogenetics or high-risk mutations due to the lack of these data in many patients. AIPSS-MF was superior to the IPSS regardless of MF subtype and age range and outperformed the MYSEC-PM in patients with secondary MF. In conclusion, we have developed a prediction model based exclusively on clinical variables that provides individualized prognostic estimates in patients with primary and secondary MF. The use of AIPSS-MF in combination with predictive models that incorporate genetic information may improve disease risk stratification. 1 Hospital Clínico Universitario, Santiago de Compostela, Spain 16 Hospital Universitario Central de Asturias, Oviedo, Spain 2 Hospital Clínico, Salamanca, Spain 17 Hospital General, Jerez de la Frontera, Spain 3 Hospital Clínic, Institut d’Investigacions Biomèdiques August Pi i Sunyer, 18 Hospital Virgen de la Arrixaca, Murcia, Spain Barcelona, Spain 19 Hospital Marqués de Valdecilla, Santander, Spain 4 Hospital Universitario de Canarias, Tenerife, Spain 20 Hospital de la Santa Creu i Sant Pau, Barcelona, Spain 5 Hospital Universitario 12 de Octubre, Madrid, Spain 21 Hospital Dr Negrín, Las Palmas de Gran Canaria, Spain 6 Hospital Morales Meseguer, Universidad Católica San Antonio de Murcia, Centro 22 Hospital General, Alicante, Spain de Investigación Biomédica en Red de Enfermedades Raras, Murcia, Spain 23 Hospital La Paz, Madrid, Spain 7 Vall d’Hebron Institute of Oncology (VHIO), Vall d’Hebron Hospital Universitari, 24 Hospital de La Princesa, Madrid, Spain Vall d’Hebron Barcelona Hospital Campus, Spain 25 Hospital Clínico Universitario-INCLIVA, Valencia, Spain 8 Hospital del Mar, Barcelona, Spain AM-O and JCH-B have senior joint authorship. 9 Hospital Universitario La Fe, Valencia, Spain Supplemental digital content is available for this article. 10 Hospital Germans Trias i Pujol, Institut Català d’Oncologia, Josep Carreras Copyright © 2022 the Author(s). Published by Wolters Kluwer Health, Inc. on Leukemia Research Institute, Universitat Autònoma de Barcelona, Badalona, behalf of the European Hematology Association. This is an open access article Spain distributed under the Creative Commons Attribution License 4.0 (CCBY), which 11 Hospital Costa del Sol, Marbella, Spain permits unrestricted use, distribution, and reproduction in any medium, provided 12 Hospital Virgen de la Victoria, Málaga, Spain the original work is properly cited. 13 Hospital Josep Trueta, Institut Catalá d’Oncologia, Girona, Spain HemaSphere (2023) 7:1(e818). 14 Hospital Universitario de Burgos, Burgos, Spain http://dx.doi.org/10.1097/HS9.0000000000000818. 15 Institut Catalá d’Oncologia, L’Hospitalet de Llobregat, Spain Received: September 3, 2022 / Accepted: November 17, 2022 1 Mosquera-Orgueira et al Improving Risk Stratification in Myelofibrosis INTRODUCTION January 2000 and October 2021 in 60 centers. This is a nation- Myelofibrosis (MF) is a rare chronic myeloproliferative wide registry (GEM-MIE-2014-01) contributed by centers affil- neoplasm (MPN) that appears de novo (primary myelofibro- iated to the Grupo Español de enfermedades Mieloproliferativas sis [PMF]), or after previous polycythemia vera (PV) or essen- Filadelfia Negativas (GEMFIN). Informed consent for inclu- tial thrombocythemia (ET) (secondary myelofibrosis [SMF]).1 sion in the registry was obtained from all patients. In every Constitutive signaling through the JAK-STAT pathway via patient, MF diagnosis was made according to the World Health activating mutations of JAK2, CALR, and MPL genes plays a Organization (WHO) criteria in use at the time of first obser- key role in its pathogenesis, while concomitant somatic muta- vation. The study was approved by the GEMFIN scientific tions, mostly affecting epigenetic modifiers or spliceosome com- board and was conducted in accordance with the Declaration ponents, may influence clinical phenotype or promote disease of Helsinki. progression.2,3 Variables with a low level of missing data (<15%) were The median survival of MF patients is about 6 years, but selected for the analyses. The selected variables were: age there is great individual variability, with the main causes of at MF diagnosis, sex, MF subtype, palpable splenomegaly death being leukemic transformation (20% of cases), disease or hepatomegaly, splenomegaly-related symptoms, constitu- progression and infections.4,5 Management of MF is challenging tional symptoms, hemoglobin levels, platelet count, leukocyte because there are no disease-modifying drugs,6,7 and the only count, monocyte count, circulating blast cells, and leukoer- curative treatment is allogeneic hematopoietic cell transplan- ytroblastosis (ie, presence of nucleated red cells, immature tation (allo-HCT),8–10 which is associated with significant mor- granulocytes and dacryocytes in peripheral blood). After bidity and mortality. Therefore, the risks of allo-HCT must be filtering out cases with incomplete annotations, a total of weighed up against expected survival without transplantation in each patient. According to consensus recommendations of the NCCN, ELN, and EBMT/ELN,8–10 allo-HCT should be consid- ered in eligible MF patients with an expected survival below 5 Table 1 years. Consequently, accurate prediction of survival is critical Baseline Characteristics of Patients in the Training and Test Sets for transplant decision-making.11 Variable Training Test Conventional prognostic models in use are the International Prognostic Scoring System (IPSS),12 which should be used at N 1109 277 diagnosis, and the Dynamic IPSS (DIPSS) and DIPSS plus,13,14 Agea, y 68.79 (18.65–94.25) 68.77 (23.07–91.67) applicable at any time during the clinical course. These mod- Age > 65 y, % 61.77 59.93 els were derived from patients with PMF, so a scoring system Male/female sex, % 57.71/42.29 59.93/40.07 named the Myelofibrosis Secondary to PV and ET-Prognostic Primary myelofibrosis, % 59.51 63.90 Model (MYSEC-PM) was subsequently developed to improve ECOG performance status >1, % 12.62 14.06 risk stratification in patients with SMF.15 Constitutional symptoms, % 42.20 38.27 Including cytogenetic and molecular data in contemporary Palpable splenomegaly, % 75.95 75.64 stratification models has led to more comprehensive prognos- Palpable hepatomegaly, % 18.13 20.53 tication. In this respect, the Molecular enhanced International Leukocyte countsa, ×109/L 9.88 (0.90–101.10) 10.50 (0.67–104.00) Prognostic Score System (MIPSS70) is based on clinical, his- Leukocytes >25 × 109/L, % 10.55 10.62 tological, and molecular factors,16 while the MIPSS70+ v2.0 Hemoglobin levela, g/dL 10.9 (3.2–19.4) 10.8 (4.4–18.7) includes clinical, cytogenetic, and molecular risk factors,17 and Hemoglobin < 10 g/dL, % 33.12 32.84 the Genetically Inspired Prognostic Scoring System (GIPSS) is Platelet counta, ×109/L 316 (5–2091) 309 (14–1640) restricted to genomic and cytogenetic data only.18 Finally, the Platelets < 100 × 109/L, % 12.59 16.11 prognostic calculator developed by Grinfeld et al19 differ in that Blood blasts countb, % 0.94 (0–15) 1.02 (0–11) rather than discriminating risk groups, it makes an individual Blood blasts ≥1%, % 39.5 42.11 prediction of survival and risk of acute myeloid leukemia (AML) Leukoerythroblastosis, % 60.8 58.2 based on clinical characteristics, cytogenetics, and comprehen- Serum LDH levelsa, IU/L 638 (68–4037) 673 (180–2374) sive molecular data. However, the main limitation to the general WHO bone marrow fibrosis grade 14.05 16.53 applicability of these models in clinical practice is the need for 0-1, % cytogenetic studies, which in many cases cannot be performed WHO bone marrow fibrosis grade 85.95 83.47 on bone marrow samples due to failure to obtain marrow (dry 2-3, % tap),12,20,21 and the need for next-generation sequencing (NGS). JAK2-mutated, % 59.69 57.40 Therefore, conventional risk models based primarily on clinical CALR-mutated, % 15.42 13.36 variables are still widely used in the clinical setting.10,11,22–24 MPL-mutated, % 3.33 2.89 The recent development of machine learning (ML) in medi- Triple negative, % 3.88 7.58 cine has become key to overcoming some of the limitations of Genotype not fully annotated, % 17.67 18.77 classical prognostic scores.25 ML is a field of artificial intelligence IPSS low risk, % 11.09 11.19 in which prediction is based on modeling of outcomes consider- IPSS intermediate-1, % 23.25 23.47 ing the complex interactions between multiple variables derived IPSS intermediate-2, % 26.42 27.80 from real examples, rather than the application of human-made IPSS high risk, % 21.28 18.77 rules. In MF, these advanced techniques can provide person- Median follow-up, y 6.56 6.51 alized survival predictions based on the clinical experience of JAK inhibitor treatment, % 38.95 37.54 thousands of patients. With this in mind, we aimed to develop a Allogeneic hematopoietic cell 9.64 10.47 new model for MF risk stratification using basic clinical infor- transplantation, % mation obtained at disease diagnosis. Median overall survival (95% CI), y 6.64 (6.06–7.55) 6.35 (5.35–7.93) Progression to AML, % 8.30 10.47 MATERIALS AND METHODS a Median (range). b Mean (range). Data source AML = acute myeloid leukemia; ECOG = Eastern Cooperative Oncology Group; LDH = lactate We retrieved original data included in the Spanish Registry of dehydrogenase; IPSS = International Prognostic Scoring System; LDH; WHO = World Health Myelofibrosis from 1617 patients diagnosed with MF between Organization. 2 (2023) 7:1 www.hemaspherejournal.com 1386 patients remained for analysis (tier 1 patients, miss- Variable selection and model development ing data rate 2.76%). These patients were randomly divided Univariate Cox regression (survival package)26 was used into a training set (80% of the cohort, N = 1109) and a test to evaluate the association of each variable with OS in the set (20% of the cohort, N = 277). Two hundred thirty-one training set. Variables with a P value <0.01 were selected for patients with a high rate of incomplete annotations in these a multivariate model using random forests (randomForestSRC variables (30.98%) were not included in the initial analysis package).27 Missing variables were imputed in each cohort sep- (tier 2 patients). arately using a missing data algorithm developed by Ishwaran et al.27 Random forests were created with 1000 trees. For cross-validation, sampling was performed without replace- Main study outcomes ment, which by default takes 0.632 times the sample size. Overall survival (OS) was defined as time from MF diag- Predictions were cross-validated in the training set and then nosis to death from any cause. In some analyses, for com- validated in the test cohort. This was done to rule out overfit- parison with other prognostic models, survival was censored ting of performance metrics in the training set related to either at the time of allo-HCT. Leukemia-free survival (LFS) was variable selection or the imputation process. The discrimina- defined as time from MF diagnosis to date of leukemic tive capacity of the random forest models in the training set transformation (uncensored) or last contact/date of death was evaluated with out-of-bag estimates of the concordance (censored). index (c-index). Figure 1. (A and B) Kaplan–Meier plots representing overall survival for the training and test sets, respectively. (C and D) Kaplan–Meier plots repre- senting leukemia-free survival for the training and test sets, respectively. 3 Mosquera-Orgueira et al Improving Risk Stratification in Myelofibrosis The precision of the different predictors was assessed using for the ML model (0.730 and 0.738 in the training and test time-dependent areas under the curve (AUCs) derived from Cox sets) and substantially superior to those of the IPSS (c-indexes survival models.28 For these calculations, cross-validation was of 0.664 and 0.674 in the training and test sets). We also eval- performed with the bootcv algorithm using 500 cycles. In each uated the performance of the IPSS-related variables as input to cycle, 75% of samples were used for training and 25% for test- the random forests model, revealing a better performance than ing. The c-indexes of these Cox models were computed with the IPSS (c-indexes 0.704 and 0.718), but still inferior to the bootstrapping in both the training and test set (500 cycles).29 new model. In the particular case of the training set, all random forest pre- Importantly, the model performed equally well in patients dictions used as input for downstream analysis were out-of-bag diagnosed before and after 2012, when ruxolitinib became to reduce the risk of overfitting during the training phase of the available in Spain for the treatment of MF patients. In the model. training set, the bootstrapped c-index of the ML model in the training set was 0.723 and 0.742 for patients diagnosed before and after 2012. These results were superior to those of the IPSS Mutational analysis (c-indexes; 0.674 and 0.680, respectively). In the test set, the DNA from bone marrow or peripheral blood was used for c-indexes of the ML model were 0.750 and 0.730 for patients mutation analysis. MPN driver mutations were detected using diagnosed before and after 2012, whereas the c-index of the standard methods.30,31 NGS was used to detect somatic muta- IPSS was 0.686 and 0.660. Additionally, we evaluated the per- tions involved in myeloid malignancies. Only pathogenic or formance of the model among patients who were subsequently probably pathogenic mutations with a variant allele frequency treated with any JAK inhibitor. Bootstrapped c-indexes for the (VAF) ≥ 5% were considered. High molecular risk was defined ML model and the IPSS score among patients treated with JAK by the presence of mutations in ASXL1, EZH2, IDH1, IDH2, inhibitors were 0.700 and 0.650 in the training set; and 0.736 SRSF2 genes or by the presence of the U2AF1Q157 mutation.17 and 0.689 in the test set. We then evaluated the capacity of the model to predict LFS. Initially, we used the same OS predictors created by the ML RESULTS algorithm to model the time to AML transformation using Cox Study population, clinical outcomes, and variable selection regression. C-indexes after bootstrapping were 0.697 and 0.703 Baseline characteristics and main clinical outcomes of in the training and test set, respectively, indicating that the same patients included in each cohort are shown in Table 1 and model retains substantial predictive ability for leukemic trans- Figure 1. Eight variables were associated with OS with a P formation. Time-dependent cross-validated AUCs also indicate value ≤0.01 in the training set (Table 2). These variables were that these predictions achieved high accuracy (Figure 3A and age at MF diagnosis, sex, hemoglobin level, percentage of blasts B; Suppl. Table S1). A random forest model based on the raw in peripheral blood, leukocyte count, platelet count, leukoery- values of the 8 variables achieved similar accuracy in both troblastosis in peripheral blood, and the presence of constitu- cohorts (c-indexes; 0.702 and 0.676 in the training and test sets, tional symptoms. respectively). Prediction of survival and AML transformation Comparison of the ML model with the IPSS for predicting OS A random forest model was created to predict OS with the The performance of the ML model was assessed using significant variables in the training set. This model achieved cross-validated time-dependent AUCs, and a comparison with an out-of-bag c-index of 0.750 in the training set and 0.744 the IPSS was performed (Suppl. Table S2). The ML model in the test set (Figure 2). For comparison, the c-indexes of the achieved a higher AUC at all evaluated time points (2.5, 5, 7.5, IPSS predictor using random forests were 0.599 and 0.685 in and 10 years) compared with the IPSS, in both the training and the training and test cohorts, respectively. We also evaluated the test cohorts (Figure 3C and D). The 5- and 10-year AUCs of c-indexes using traditional Cox regression to rule out underper- the ML model were 80.78% and 80.50% in the training set, formance of random forests with the IPSS scores, using the ML and 80.61% and 81.95% in the test set. These metrics were model predictions as input. Bootstrapped c-indexes were similar superior to those of the IPSS in the same patients: 78.87% and 73.14% in the training cohort, 75.22% and 76.83% in the test set. The difference in performance remained substantial Table 2 regardless of whether patients were censored or not at time of allo-HCT (Suppl. Figure S1). Next, the IPSS groups were com- Variable Association With Overall Survival in the Training Set pared with patient risk quartiles according to survival predic- (Cox Regression) tions made by the ML algorithm. The distribution of the IPSS Variable HR P value groups was unbalanced, with 50.77% and 48.37% of patients being assigned to a lower or higher risk group by the ML algo- Age at diagnosis 1.06 <0.0001 rithm, a fact that particularly affected IPSS intermediate-1 and Sex 1.78 <0.0001 -2 groups (Figure 4). Furthermore, we were able to confirm the PMF vs PET-MF 0.81 0.05 superiority of our model over the IPSS in patients both <60 and PMF vs PPV-MF 1.03 0.83 ≥60 years (Suppl. Figure S2). Importantly, the model retained a Palpable splenomegaly 1.25 0.05 similar prognostic performance in PMF and SMF, whereas the Palpable hepatomegaly 1.10 0.38 results of the IPSS were poorer in SMF than in PMF (Figure 5). Symptomatic splenomegaly 1.16 0.17 For the present study, PMF patients with grades 0-1 and 2-3 Constitutional symptoms 1.44 <0.0001 bone marrow fibrosis were considered to have prefibrotic and Hemoglobin level 0.98 <0.0001 overt PMF, respectively. The ML performed similarly in both Leukocyte count 1.03 <0.0001 groups, and its prognostic performance was superior to that Monocyte count 1 0.14 of the IPSS (Suppl. Figure S3). Finally, the performance of the Platelet count 0.99 <0.0001 ML predictor was evaluated in patients within the lower (low, Blasts in peripheral blood (%) 1.22 <0.0001 intermediate-1) and higher (intermediate-2, high-risk) IPSS Leukoerythroblastosis 1.50 <0.0001 categories. Notably, the performance metrics of the ML clas- PET-MF = postessential thrombocythemia myelofibrosis; PMF = primary myelofibrosis; PPV-MF = sifier were similar regardless of IPSS risk categories (Suppl. postpolycythemia vera myelofibrosis. Figure S4). 4 (2023) 7:1 www.hemaspherejournal.com Figure 2. Survival according to the ML model. Patients were split according to the predicted tertile of overall survival according to the model. Each branch rep- resents a tertile of patients with either low-risk (red), intermediate-risk (green), or high-risk (blue) disease. (A and B) The association with overall survival for the training and test cohorts, respectively, and (C and D) the association with time to leukemic transformation in the training and test sets, respectively. ML = machine learning. Comparison of the ML model with the MYSEC-PM in SMF training and test set. Among them, high-risk cytogenetic abnor- We compared the ML predictor with the MYSEC-PM in SMF malities as defined by the DIPSS plus model were present in 23% (Suppl. Table S3; Figure 6). Sufficient information to calculate and 19% of patients in each cohort, respectively. No substantial the MYSEC-PM was available in 449 and 100 SMF patients benefit was observed from the addition of high-risk cytogenetic in the training and test cohorts, respectively. Time-dependent data to the ML model for the prediction of OS or LFS (Suppl. cross-validated AUCs revealed a superior performance of the Tables S4 and S5; Suppl. Figure S5 A-B). ML predictor compared to MYSEC-PM in both cohorts. The Then, we analyzed the effect of high-risk mutations as defined 5-year AUCs in the training set were 82.55% and 75.93% by the MIPSS70-plus v2.0 (in ASXL1, SRSF2, EZH2, IDH1, for the ML model and the MYSEC-PM groups, respectively. IDH2, and U2AF1Q157 genes17) that were detectable with a VAF Likewise, the 5-year AUCs in the test set were 74.82% for the ≥ 5%. Overall, 247 patients (205 in the training set and 42 in the ML model and 68.26% for the MYSEC-PM. test set) were fully characterized at the molecular level for high- risk mutations, and patients harboring any of them were consid- Integration of other potential risk factors into the ML model ered high-risk for the present analysis. No benefit was obtained We evaluated whether other clinically relevant variables col- by including these molecular abnormalities in the prediction lected in a variable number of patients could potentially improve of OS (Suppl. Table S4; Suppl. Figure S6 A-B) or LFS (Suppl. the model. First, we studied the potential benefit of including Figure S7; Suppl. Table S5). Next, we assessed the prognostic high-risk cytogenetics in the prediction model. Overall, cyto- effect of classifying patients into 4 groups according to MPN genetic annotation was available for 277 and 74 cases in the driver mutation (JAK2-mutated, CALR-mutated, MPL-mutated, 5 Mosquera-Orgueira et al Improving Risk Stratification in Myelofibrosis Figure 3. Time-dependent cross-validated AUCs in the training (A) and test (B) cohorts for the prediction of leukemia-free survival according to the ML model and the IPSS score. Time-dependent cross-validated AUCs in the training (C) and test cohorts (D) for predicting overall survival according to the ML model and the IPSS score. AUCs = areas under the curve; IPSS = International Prognostic Scoring System, ML = machine learning. and triple negative). A total of 1138 patients were annotated Figure S8). Finally, we tested the potential benefit of including for this variable, 913 in the training set and 225 in the test set. Eastern Cooperative Oncology Group (ECOG) performance Nonetheless, no benefit was observed by adding this variable to status33 and serum lactate dehydrogenase (LDH) levels at MF the ML model (Suppl. Table S4; Suppl. Figure S6 C-D). diagnosis34 in the model. Overall, ECOG was available in We reasoned that the grade of bone marrow fibrosis could 615 patients (483 and 132 cases in the training and test set, play a prognostic role,16 but excluded this variable in the initial respectively) and LDH values were retrieved from 757 patients random forests model due to the different histological classi- (602 in the training set and 155 in the test set). None of these fications of bone marrow fibrosis during the study period and variables improved prediction in either cohort when added to the lack of centralized assessment, taking into account previous the Cox model along with the ML predictor (Suppl. Table S4; reports showing poor concordance in categorizing the degree Suppl. Figure S9). of fibrosis among pathologists in multicenter studies.32 Indeed, univariate analysis revealed that this variable was associated Evaluation of the predictions in tier 2 patients with prognosis in the training and test set (P value 2.49 × 10−7 A total of 231 patients initially excluded from the analysis and 0.01 in the training and test sets, respectively). However, due to high missing variable rate composed the tier 2 cohort. random forest survival models that included this additional Baseline characteristics for these patients can be seen in Suppl. variable did not show improved prognostication (Suppl. Table S6. The missingness rate among the final selected variables 6 (2023) 7:1 www.hemaspherejournal.com Figure 4. Transition plots representing the flow of patients between the different IPSS groups to the different survival quartiles predicted by the AIPSS-MF model in the training (A) and test (B) cohorts. However, it is important to note that the AIPSS-MF does not assign patients into risk groups but provides individual predictions of overall and leukemia-free survival. AIPSS-MF = Artificial Intelligence Prognostic Scoring System for Myelofibrosis; IPSS = International Prognostic Scoring System. was lower than among the original set of evaluated variables into arbitrary cutoff points. One exception is patient sex, not (25.16% versus 30.98%). Even so, up to 53.25% of patients included in these other models despite the better survival in had insufficient data for the calculation of the IPSS score. We MF women than men reported in several studies.20,35 Likewise, used parameter imputation in these patients to evaluate the male MPN patients are more prone to acquire high-risk results of the ML prognostic models in this cohort. Using the mutations, which have been linked to disease progression.36 random forest model for OS prediction, a c-index of 0.730 was Anemia has been associated with reduced survival in men, obtained. In the case of LFS, the Cox model based on the OS but not in women, and male sex has been associated with a predictors achieved a bootstrapped c-index of 0.606, whereas higher risk of leukemic transformation in PMF patients.37,38 the random forest model based on the original 8 variables Finally, in a personalized prognostic model for MPN, male obtained a c-index of 0.654. sex has been linked to increased risk of transformation, and An interactive web calculator of the AIPSS-MF (Artificial mutations in the spliceosome, epigenetic regulators and RAS Intelligence Prognostic Scoring System for Myelofibrosis) pathway genes.19 To our knowledge, the other factor, leuko- model can be accessed in the following link: https://genetic- erythroblastosis, has not been previously identified as a prog- soncohematology.com/MF/. The prediction of the model in a nostic risk factor in MF. According to the WHO and ICC clinical case using the interactive calculator is illustrated in classification, this feature is a minor criterion for overt PMF Figure 7. but not for early PMF, and an additional criteria for post-ET MF and post-PV MF, and probably reflects the grade of bone marrow fibrosis.1,39–41 In our series, about 60% of patients DISCUSSION had leukoerythroblastosis, and this factor correlated with a In the present study, we developed an ML model to predict higher degree of bone marrow fibrosis and a higher frequency survival in patients with MF based on data from the Spanish of anemia, marked leukocytosis, and circulating blast cells, Myelofibrosis Registry. The algorithm is a supervised random each of which has been associated with adverse prognosis in forest model composed of eight variables obtained at the time MF.12,14,16,20,41–43 of MF diagnosis: age, sex, constitutional symptoms, hemoglo- The main limitations of the present study derive from its bin level, leukocyte count, percentage of blasts in peripheral registry-based nature. Data quality depends on local physi- blood, platelet count, and leukoerytroblastosis. The AIPSS-MF cians entering data at many different centers over a long fol- model outperforms currently established prognostic systems, low-up period without centralized review. We have considered such as the IPSS and MYSEC-PM. In addition, the AIPSS-MF PMF patients with grades 0-1 and 2-3 bone marrow fibro- model produces balanced risk groups, reclassifying roughly sis to have prefibrotic and overt PMF, respectively, but the half of the patients into a different risk category from the IPSS. diagnosis of prefibrotic PMF was only formally established Other important advantages of the new model are (1) it behaves in a minority of patients. Therefore, the performance of the similarly in PMF and SMF; (2) it provides a personalized risk AIPSS-MF in patients with prefibrotic PMF requires further estimate for each individual patient; and (3) it is not based on validation. Informative cytogenetic data was only available genomic data, enabling its implementation in all types of health- for 25% of patients. The registry includes patients diagnosed care facilities. from 2000 onwards, and although most (82%) of the series Most of the variables in the new model are among the risk were annotated for MPN driver mutations, only a minority factors included in conventional risk-scoring systems, but here had NGS panel data evaluating additional somatic mutations. the prognostic impact of continuous variables is not stratified Therefore, we were unable to adequately assess the potential 7 Mosquera-Orgueira et al Improving Risk Stratification in Myelofibrosis Figure 5. Performance of the ML model and IPSS score in predicting overall survival in primary myelofibrosis ([A and B] for the training and test set, respectively) and secondary myelofibrosis ([C and D] for the training and test set, respectively). IPSS = International Prognostic Scoring System, ML = machine learning. benefit of including adverse cytogenetics or high-risk somatic their broad applicability, our results indicate that disease prog- mutations data in our prognostic model. For the same reason, nostication can be improved by reinterpreting a limited num- it was not possible to compare the predictive accuracy of our ber of classical variables.44 Now, more than ever, developing model with those that include genetic information, such as high-quality data banks on patients treated in the real world or DIPSS plus, MIPSS70 or MIPSS70+ v2.0. The performance in clinical trials will become a defining moment for precision of the model was evaluated on a test set comprising 20% of medicine in MF. the patients in our registry, but further external evaluation In conclusion, we present a new survival model in MF based using data from other sources is warranted. Nonetheless, the on data from the Spanish Myelofibrosis Registry. The present large size of the patient series, followed over a long observa- model can provide patient-specific predictions at disease diag- tion period, is a principal strength of our study, reflecting the nosis, and outperforms other well-established risk stratifying actual clinical course of the entire MF population without systems such as the IPSS and MYSEC-PM. Furthermore, this the set of inclusion and exclusion criteria used in controlled model is equally predictive of survival in both primary and clinical trials. secondary MF and does not require cytogenetic or molecular The present work contributes further evidence of the pro- data, which facilitates its applicability in all healthcare settings. found impact that ML may have in hematology-oncology, However, we think that the present model should be used in reshaping our previous conceptions of disease prognostication combination with other models that include genetic informa- and potentially changing our clinical practice. Although ML tion, such as MIPSS70 or MIPSS70+, especially when transplant models are often developed with large datasets that could limit decisions are involved. 8 (2023) 7:1 www.hemaspherejournal.com Figure 6. Comparison of the ML model and the MYSEC-PM model in predicting overall survival in secondary myelofibrosis in the training (A) and test (B) sets. MYSEC-PM = Myelofibrosis Secondary to PV and ET-Prognostic Model. Figure 7. Use of the interactive online calculator to predict overall and leukemia-free survival in a clinical case. This is a 60-year-old male patient who presented with fatigue and a palpable spleen 8 cm below the left costal margin. His blood test showed: Hb 11 g/dL, leukocytes 2.0 × 109/L (3% blast cells), platelets 75 × 109/L and leukoerythroblastosis. The patient was assigned to the intermediate-1 risk group by the IPSS model (median predicted survival of 7.9 years). However, the AIPSS-MF model predicted a median survival for this patient of 4.68 years and a leukemia-free survival of 87% at 5 years from diagnosis. As can be seen in the figure, this model is more sensitive than the IPSS to the adverse prognostic features present in patients with the myelodepletive phenotype of myelofibrosis. Unlike the IPSS, the model considers all prognostic information from continuous variables (eg, 3% blood blasts is worse than 1% blood blast, etc.) and does not rely on cutoff points. AIPSS-MF = Artificial Intelligence Prognostic Scoring System for Myelofibrosis; IPSS = International Prognostic Scoring System. 9 Mosquera-Orgueira et al Improving Risk Stratification in Myelofibrosis AUTHOR CONTRIBUTIONS 20. Passamonti F, Giorgino T, Mora B, et al. A clinical-molecular prog- nostic model to predict survival in patients with post polycythemia The GEMFIN group collected the data; AM-O performed the machine vera and post essential thrombocythemia myelofibrosis. Leukemia. learning analysis; JCH-B, MPE, and AM-O wrote the paper; all remaining 2017;31:2726–2731. co-authors critically evaluated the manuscript and made substantial recom- 21. Luque Paz D, Riou J, Verger E, et al. Genomic analysis of primary and mendations. All authors approved the submission of this manuscript. secondary myelofibrosis redefines the prognostic impact of ASXL1 mutations: a FIM study. Blood Adv. 2021;5:1442–1451. DISCLOSURES 22. Harrison CN, Mead AJ, Drummond MW, et al. A physician survey on the application of the British Society for Haematology guidelines for the The Spanish Registry of Myelofibrosis was initially sponsored by a grant diagnosis and management of myelofibrosis in the UK. Br J Haematol. from Novartis Pharmaceuticals, Inc. The data supporting the findings of 2020;188:e105–e109. this study are not publicly available due to privacy or ethical restrictions 23. McLornan DP, Sirait T, Hernández-Boluda JC, et al. European wide but are available on request from the corresponding authors. The study was survey on allogeneic haematopoietic cell transplantation practice for approved by the scientific board of GEMFIN. The authors have no conflicts myelofibrosis on behalf of the EBMT chronic malignancies working of interest to disclose. party. Curr Res Transl Med. 2021;69:103267. 24. Breccia M, Baratè C, Benevolo G, et al. Tracing the decision-making pro- cess for myelofibrosis: diagnosis, stratification, and management of rux- REFERENCES olitinib therapy in real-word practice. Ann Hematol. 2020;99:65–72. 1. Swerdlow SH, Campo E, Harris NL, et al, eds. WHO Classification of 25. Shouval R, Fein JA, Savani B, et al. Machine learning and artificial intel- Tumours of Haematopoietic and Lymphoid Tissues. Revised 4th edition. ligence in haematology. Br J Haematol. 2021;192:239–250. Lyon: International Agency for Research on Cancer; 2017:585. 26. Therneau TM, Grambsch PM. Modeling Survival Data. New York, NY: 2. Nangalia J, Green AR. Myeloproliferative neoplasms: from origins to out- Springer; 2013. comes. Hematol Am Soc Hematol Educ Program. 2017;2017:470–479. 27. Ishwaran H, Kogalur U, Blackstone E, et al. Random survival forests. 3. Spivak JL. Are polycythemia vera, essential thrombocytosis, and pri- Ann Appl Statist. 2008;2:841–860. mary myelofibrosis 1, 2, or 3 diseases? Leukemia. 2021;35:1890–1893. 28. Gerds TA, Kattan MW. Medical Risk Prediction Models: With Ties to 4. Tefferi A, Guglielmelli P, Larson DR, et al. Long-term survival and blast Machine Learning. 1st ed. Boca Raton: CRC Press; 2021. (Chapman & transformation in molecularly annotated essential thrombocythemia, poly- hall/crc biostatistics series). cythemia vera, and myelofibrosis. Blood. 2014;124:2507–13; quiz 2615. 29. rms: Regression Modeling Strategies version 6.3-0 from CRAN 5. Pastor-Galán I, Hernández-Boluda JC, Correa JG, et al. Clinico-biological [Internet]. Available at: https://rdrr.io/cran/rms/. Accessed May 19, characteristics of patients with myelofibrosis: an analysis of 1,000 cases from 2022. the Spanish Registry of Myelofibrosis. Med Clin (Barc). 2020;155:152–158. 30. Vannucchi AM, Lasho TL, Guglielmelli P, et al. Mutations and prognosis 6. Coltro G, Loscocco GG, Vannucchi AM. Classical Philadelphia-negative in primary myelofibrosis. Leukemia. 2013;27:1861–1869. myeloproliferative neoplasms (MPNs): a continuum of different disease 31. Tefferi A, Lasho TL, Finke C, et al. Type 1 vs type 2 calreticulin muta- entities. Int Rev Cell Mol Biol. 2021;365:1–69. tions in primary myelofibrosis: differences in phenotype and prognostic 7. Tefferi A. Primary myelofibrosis: 2021 update on diagnosis, risk-stratifi- impact. Leukemia. 2014;28:1568–1570. cation and management. Am J Hematol. 2021;96:145–162. 32. Pozdnyakova O, Rodig S, Bhandarkar S, et al. The importance of central 8. Barbui T, Tefferi A, Vannucchi AM, et al. Philadelphia chromo- pathology review in international trials: a comparison of local versus some-negative classical myeloproliferative neoplasms: revised man- central bone marrow reticulin grading. Leukemia. 2015;29:241–244. agement recommendations from European LeukemiaNet. Leukemia. 33. Tam CS, Kantarjian H, Cortes J, et al. Dynamic model for predict- 2018;32:1057–1069. ing death within 12 months in patients with primary or post-polycy- 9. Myeloproliferative neoplasms. NCCN Guidelines version 2.2022 themia vera/essential thrombocythemia myelofibrosis. J Clin Oncol. [Internet]. NCCN. Available at: https://www.nccn.org/guidelines/guide- 2009;27:5587–5593. lines-detail. Accessed May 14, 2022. 34. Shah S, Mudireddy M, Hanson CA, et al. Marked elevation of serum 10. Kröger NM, Deeg JH, Olavarria E, et al. Indication and management of lactate dehydrogenase in primary myelofibrosis: clinical and prognostic allogeneic stem cell transplantation in primary myelofibrosis: a consen- correlates. Blood Cancer J. 2017;7:657. sus process by an EBMT/ELN International Working Group. Leukemia. 35. Hernández-Boluda JC, Pereira A, Kröger N, et al. Allogeneic hemato- 2015;29:2126–2133. poietic cell transplantation in older myelofibrosis patients: a study of 11. England J, Gupta V. Novel therapies vs hematopoietic cell transplanta- the chronic malignancies working party of EBMT and the Spanish tion in myelofibrosis: who, when, how? Hematol Am Soc Hematol Educ Myelofibrosis Registry. Am J Hematol. 2021;96:1186–1194. Program. 2021;2021:453–462. 36. Karantanos T, Chaturvedi S, Braunstein EM, et al. Sex determines the 12. Cervantes F, Dupriez B, Pereira A, et al. New prognostic scoring sys- presentation and outcomes in MPN and is related to sex-specific differ- tem for primary myelofibrosis based on a study of the International ences in the mutational burden. Blood Adv. 2020;4:2567–2576. Working Group for Myelofibrosis Research and Treatment. Blood. 37. Nicolosi M, Mudireddy M, Lasho TL, et al. Sex and degree of sever- 2009;113:2895–2901. ity influence the prognostic impact of anemia in primary myelo- 13. Passamonti F, Cervantes F, Vannucchi AM, et al. A dynamic prognostic fibrosis: analysis based on 1109 consecutive patients. Leukemia. model to predict survival in primary myelofibrosis: a study by the IWG- 2018;32:1254–1258. MRT (International Working Group for Myeloproliferative Neoplasms 38. Vallapureddy RR, Mudireddy M, Penna D, et al. Leukemic transforma- Research and Treatment). Blood. 2010;115:1703–1708. tion among 1306 patients with primary myelofibrosis: risk factors and 14. Gangat N, Caramazza D, Vaidya R, et al. DIPSS plus: a refined dynamic development of a predictive model. Blood Cancer J. 2019;9:12. international prognostic scoring system for primary myelofibrosis that 39. Alaggio R, Amador C, Anagnostopoulos I, et al. The 5th edition of incorporates prognostic information from karyotype, platelet count, the World Health Organization Classification of haematolymphoid and transfusion status. J Clin Oncol. 2011;29:392–397. tumours: lymphoid neoplasms. Leukemia. 2022;36:1720–1748. 15. Passamonti F, Giorgino T, Mora B, et al. A clinical-molecular prognostic 40. Arber DA, Orazi A, Hasserjian RP, et al. International consensus classi- model to predict survival in patients with post polycythemia vera and post fication of myeloid neoplasms and acute leukemia: integrating morpho- essential thrombocythemia myelofibrosis. Leukemia. 2017;31:2726–2731. logical, clinical, and genomic data. Blood. 2022;140:1200–1228. 16. Guglielmelli P, Lasho TL, Rotunno G, et al. MIPSS70: mutation-en- 41. Thiele J, Kvasnicka HM. Hematopathologic findings in chronic idio- hanced international prognostic score system for transplantation-age pathic myelofibrosis. Semin Oncol. 2005;32:380–394. patients with primary myelofibrosis. J Clin Oncol. 2018;36:310–318. 42. Vener C, Fracchiolla NS, Gianelli U, et al. Prognostic implications of 17. Tefferi A, Guglielmelli P, Lasho TL, et al. MIPSS70+ version 2.0: muta- the European consensus for grading of bone marrow fibrosis in chronic tion and karyotype-enhanced international prognostic scoring system idiopathic myelofibrosis. Blood. 2008;111:1862–1865. for primary myelofibrosis. J Clin Oncol. 2018;36:1769–1770. 43. Carobbio A, Guglielmelli P, Rumi E, et al. A multistate model of survival 18. Tefferi A, Guglielmelli P, Nicolosi M, et al. GIPSS: genetically inspired prediction and event monitoring in prefibrotic myelofibrosis. Blood prognostic scoring system for primary myelofibrosis. Leukemia. Cancer J. 2020;10:100. 2018;32:1631–1642. 44. Mosquera Orgueira A, González Pérez MS, Diaz Arias J, et al. 19. Grinfeld J, Nangalia J, Baxter EJ, et al. Classification and person- Unsupervised machine learning improves risk stratification in newly alized prognosis in myeloproliferative neoplasms. N Engl J Med. diagnosed multiple myeloma: an analysis of the Spanish Myeloma 2018;379:1416–1430. Group. Blood Cancer J. 2022;12:76. 10