Since January 2020 Elsevier has created a COVID-19 resource centre with
free information in English and Mandarin on the novel coronavirus COVID-
  19. The COVID-19 resource centre is hosted on Elsevier Connect, the
             company's public news and information website.



   Elsevier hereby grants permission to make all its COVID-19-related
research that is available on the COVID-19 resource centre - including this
  research content - immediately available in PubMed Central and other
publicly funded repositories, such as the WHO COVID database with rights
for unrestricted research re-use and analyses in any form or by any means
   with acknowledgement of the original source. These permissions are
 granted for free by Elsevier for as long as the COVID-19 resource centre
                             remains active.
Journal Pre-proof


ABC2 -SPH risk score for in-hospital mortality in COVID-19 patients:
development, external validation and comparison with other available
scores

Milena S. Marcolino MD, PhD , Magda C. Pires PhD ,
Lucas Emanuel F. Ramos , Rafael T. Silva ,
Luana M. Oliveira MSc , Rafael L.R. Carvalho PhD ,
Rodolfo Lucas S. Mourato , Adrián Sánchez-Montalvá MD, PhD ,
Berta Raventós MSc , Fernando Anschau MD, PhD ,
José Miguel Chatkin MD, PhD , Matheus C.A. Nogueira MD ,
Milton H. Guimarães Júnior MD, MSc ,
Giovanna G. Vietta MD, PhD , Helena Duani MD, PhD ,
Daniela Ponce MD, PhD , Patricia K. Ziegelmann PhD ,
Luı́s C. Castro MD, PhD , Karen B. Ruschel PhD ,
Christiane C.R. Cimini MD, MSc , Saionara C. Francisco MSc ,
Maiara A. Floriani MSc , Guilherme F. Nascimento MSc ,
Bárbara L. Farace MD , Luanna S. Monteiro MD ,
Maira V.R. Souza-Silva MD , Thais L.S. Sales MSc ,
Karina Paula M.P. Martins MD, MSc ,
Israel J. Borges do Nascimento , Tatiani O. Fereguetti MD ,
Daniel T.M.O. Ferrara , Fernando A. Botoni MD ,
Ana Paula B.S. Etges PhD , Eric Boersma MD, PhD ,
Carisi A. Polanczyk MD, PhD ,
Alexandre V. Schwarzbold MD, PhD , Amanda O. Maurı́lio MD ,
Ana Luiza B.A. Scotton MD , André P. Weber ,
André S.M. Costa MD , Andressa B. Glaeser ,
Angélica Aparecida C. Madureira MD , Angelinda R. Bhering MD ,
Bruno Mateus de Castro , Carla Thais C.A. da Silva ,
Carolina M. Ramos MD , Caroline D. Gomes ,
Cı́ntia A. de Carvalho , Daniel V. Silveira MD, MSc , Edilson Cezar ,
Elayne C. Pereira PhD , Emanuele Marianne S. Kroger MD ,
Felipe B. Vallt , Fernanda B. Lucas MD , Fernando G. Aranha MD ,
Frederico Bartolazzi MD , Gabriela P. Crestani ,
Gisele A.N. Bastos MD, ScD , Glı́cia Cristina de C. Madeira MD ,
Helena Carolina Noal MSc , Heloisa R. Vianna MD, MSc ,
Henrique C. Guimarães MD, PhD , Isabela M. Gomes ,
Israel M. Romero MD, PhD , Joanna D.L. Batista MD, PhD ,
Joice C. de Alvarenga MD , Júlia D.S.S. Guimarães MD ,
Júlia D.P. de Morais MD , Juliana M. Rugolo ,
Karen Cristina J.R. Pontes , Kauane Aline M. dos Santos ,
Leonardo S. de Oliveira , Lı́lian S. Pinheiro ,
Liliane S. Pacheco MD, MSc , Lucas de D. Sousa MD ,
Luciana S.F. Couto , Luciane Kopittke PhD ,
Luis Cesar S. de Moura MD, PhD , Luisa Elem A. Santos ,
Máderson A.S. Cabral MD , Maı́ra D. Souza MD ,
Marcela G.T. Tofani MD , Marcelo Carneiro MD, PhD ,
Maria Angélica P. Ferreira PhD ,
Maria Aparecida C. Bicalho MD, PhD , Maria Clara P.B. Lima ,
Mariana F. Godoy MD , Marilia M.A. Cardoso PhD ,
Meire P. Figueiredo , Natália C.S. Sampaio ,
Natália L. Rangel MD , Natália T. Crespo MD ,
Neimy R. de Oliveira MD , Pedro L. Assaf MD ,
Petrônio José de L. Martelli MD, PhD , Rafaela S.C. Almeida ,
Raphael C. Martins , Raquel Lutkmeier ,
Reginaldo Aparecido Valacio , Renan G. Finger MD ,
Ricardo B. Cardoso MSc , Roberta Pozza MD, PhD ,
Roberta X. Campos MD , Rochele M. Menezes MD ,
Roger M. de Abreu MD , Rufino de F. Silva MD ,
Silvana M.M. Guimarães MD , Silvia F. Araújo MD ,
Susany Anastácia Pereira , Talita F. Oliveira MD ,
Tatiana Kurtz MD, PhD , Thainara C. de Oliveira ,
Thaı́za Simônia M.A. Araújo , Thulio Henrique O. Diniz MD ,
Veridiana B. dos Santos , Virginia Mara R. Gomes ,
Vitor Augusto L. do Vale , Yuri C. Ramires MD

PII:                    S1201-9712(21)00605-6
DOI:                    https://doi.org/10.1016/j.ijid.2021.07.049
Reference:              IJID 5585


To appear in:           International Journal of Infectious Diseases

Received date:          11 June 2021
Revised date:           18 July 2021
Accepted date:          20 July 2021

Please      cite    this    article    as:       Milena S. Marcolino MD, PhD ,         Magda C. Pires PhD ,
Lucas Emanuel F. Ramos , Rafael T. Silva , Luana M. Oliveira MSc , Rafael L.R. Carvalho PhD ,
Rodolfo Lucas S. Mourato ,            Adrián Sánchez-Montalvá MD, PhD ,             Berta Raventós MSc ,
Fernando Anschau MD, PhD ,             José Miguel Chatkin MD, PhD ,          Matheus C.A. Nogueira MD ,
Milton H. Guimarães Júnior MD, MSc ,         Giovanna G. Vietta MD, PhD ,         Helena Duani MD, PhD ,
Daniela Ponce MD, PhD ,                 Patricia K. Ziegelmann PhD ,               Luı́s C. Castro MD, PhD ,
Karen B. Ruschel PhD ,             Christiane C.R. Cimini MD, MSc ,             Saionara C. Francisco MSc ,
Maiara A. Floriani MSc ,              Guilherme F. Nascimento MSc ,                  Bárbara L. Farace MD ,
Luanna S. Monteiro MD ,                   Maira V.R. Souza-Silva MD ,                 Thais L.S. Sales MSc ,
Karina Paula M.P. Martins MD, MSc , Israel J. Borges do Nascimento , Tatiani O. Fereguetti MD ,
Daniel T.M.O. Ferrara ,                Fernando A. Botoni MD ,                  Ana Paula B.S. Etges PhD ,
Eric Boersma MD, PhD ,          Carisi A. Polanczyk MD, PhD ,         Alexandre V. Schwarzbold MD, PhD ,
Amanda O. Maurı́lio MD , Ana Luiza B.A. Scotton MD , André P. Weber , André S.M. Costa MD ,
Andressa B. Glaeser ,           Angélica Aparecida C. Madureira MD ,             Angelinda R. Bhering MD ,
Bruno Mateus de Castro ,                  Carla Thais C.A. da Silva ,               Carolina M. Ramos MD ,
Caroline D. Gomes ,        Cı́ntia A. de Carvalho ,        Daniel V. Silveira MD, MSc ,        Edilson Cezar ,
Elayne C. Pereira PhD ,                  Emanuele Marianne S. Kroger MD ,                      Felipe B. Vallt ,
Fernanda B. Lucas MD , Fernando G. Aranha MD , Frederico Bartolazzi MD , Gabriela P. Crestani ,
Gisele A.N. Bastos MD, ScD ,          Glı́cia Cristina de C. Madeira MD ,        Helena Carolina Noal MSc ,
Heloisa R. Vianna MD, MSc ,                Henrique C. Guimarães MD, PhD ,               Isabela M. Gomes ,
Israel M. Romero MD, PhD ,             Joanna D.L. Batista MD, PhD ,             Joice C. de Alvarenga MD ,
Júlia D.S.S. Guimarães MD ,                  Júlia D.P. de Morais MD ,                 Juliana M. Rugolo ,
Karen Cristina J.R. Pontes ,             Kauane Aline M. dos Santos ,              Leonardo S. de Oliveira ,
Lı́lian S. Pinheiro , Liliane S. Pacheco MD, MSc , Lucas de D. Sousa MD , Luciana S.F. Couto ,
Luciane Kopittke PhD ,              Luis Cesar S. de Moura MD, PhD ,                  Luisa Elem A. Santos ,
Máderson A.S. Cabral MD ,                    Maı́ra D. Souza MD ,                  Marcela G.T. Tofani MD ,
Marcelo Carneiro MD, PhD , Maria Angélica P. Ferreira PhD , Maria Aparecida C. Bicalho MD, PhD ,
Maria Clara P.B. Lima ,                 Mariana F. Godoy MD ,                    Marilia M.A. Cardoso PhD ,
Meire P. Figueiredo ,      Natália C.S. Sampaio ,       Natália L. Rangel MD ,      Natália T. Crespo MD ,
Neimy R. de Oliveira MD ,            Pedro L. Assaf MD ,             Petrônio José de L. Martelli MD, PhD ,
 Rafaela S.C. Almeida , Raphael C. Martins , Raquel Lutkmeier , Reginaldo Aparecido Valacio ,
 Renan G. Finger MD ,                 Ricardo B. Cardoso MSc ,             Roberta Pozza MD, PhD ,
 Roberta X. Campos MD ,                  Rochele M. Menezes MD ,             Roger M. de Abreu MD ,
 Rufino de F. Silva MD ,                Silvana M.M. Guimarães MD ,             Silvia F. Araújo MD ,
 Susany Anastácia Pereira , Talita F. Oliveira MD , Tatiana Kurtz MD, PhD , Thainara C. de Oliveira ,
 Thaı́za Simônia M.A. Araújo ,        Thulio Henrique O. Diniz MD ,       Veridiana B. dos Santos ,
 Virginia Mara R. Gomes , Vitor Augusto L. do Vale , Yuri C. Ramires MD , ABC2 -SPH risk score
for in-hospital mortality in COVID-19 patients: development, external validation and compar-
ison with other available scores, International Journal of Infectious Diseases (2021), doi:
https://doi.org/10.1016/j.ijid.2021.07.049




This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition
of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of
record. This version will undergo additional copyediting, typesetting and review before it is published
in its final form, but we are providing this version to give early visibility of the article. Please note that,
during the production process, errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.

© 2021 Published by Elsevier Ltd on behalf of International Society for Infectious Diseases.
This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
Highlights


      ABC2-SPH mortality risk score employs seven variables upon hospital presentation

      The model had high discriminatory value (AUROC 0.844, 95% CI 0.829 to 0.859)

      External validation confirmed these results in Brazilian and Spanish cohorts

      ABC2-SPH outperformed other available scores.

      It is implemented in a freely available online risk calculator (https://abc2sph.com/).




                                                                                                1
ABC2-SPH risk score for in-hospital mortality in COVID-19
patients: development, external validation and comparison
with other available scores


Milena S. Marcolino, MD, PhD1,* (Marcolino MS - milenamarc@ufmg.br, 0000-0003-4278-

3771)


Magda C. Pires, PhD2* (Pires MC – magda@est.ufmg.br, 0000-0003-3312-4002)


Lucas Emanuel F. Ramos3 (Ramos LEF - luckermos19@gmail.com, 0000-0001-7844-0581)


Rafael T. Silva3 (Silva RT - rafaelsilva@posteo.net, 0000-0002-9270-5328)


Luana M. Oliveira, MSc4 (Oliveira LM - luanalmo19.09@gmail.com, 0000-0003-4639-4546)


Rafael L.R. Carvalho, PhD5 (Carvalho RLR - rafaelsjdr@hotmail.com, 0000-0003-3576-3748)


Rodolfo Lucas S. Mourato6 (Mourato RLS - rodolfo_use@hotmail.com, 0000-0002-0251-0691)


Adrián Sánchez-Montalvá, MD, PhD7 (Montalvá AS - adrian.sanchez.montalva@gmail.com,

0000-0002-2194-5447)


Berta Raventós, MSc8 (Raventós B - berta.raventos@vhir.org, 0000-0002-4668-2970)


Fernando Anschau, MD, PhD9 (Anschau F - afernando@ghc.com.br, 0000-0002-2657-5406)


José Miguel Chatkin, MD, PhD10 (Chatkin JM - jmchatkin@pucrs.br, 0000-0002-4343-025X)


Matheus C. A. Nogueira, MD11 (Nogueira MCA - mathnogueira42@gmail.com, 0000-0002-

0241-9046)




                                                                                          2
Milton H. Guimarães Júnior, MD, MSc12 (Guimarães Júnior MH -

miltonhenriques@yahoo.com.br, 0000-0002-2127-8015)


Giovanna G. Vietta, MD, PhD13 (Vietta GG - ggvietta@gmail.com, 0000-0002-0756-3098)


Helena Duani, MD, PhD14 (Duani H - hduani@yahoo.com.br, 0000-0001-9345-018X)


Daniela Ponce, MD, PhD15 (Ponce D - daniela.ponce@unesp.br, 0000-0002-6178-6938)


Patricia K. Ziegelmann, PhD16 (Ziegelmann PK - patriciakz99@gmail.com, 0000-0002-2851-

2011)


Luís C. Castro, MD, PhD 17 (Castro LC - pharmlucamsc@gmail.com, 0000-0003-2379-0167)


Karen B. Ruschel, PhD18 (Ruschel KB - karenbruschel@gmail.com, 0000-0002-6362-1889)


Christiane C.R. Cimini, MD, MSc19 (Cimini CCR - christiane.cimini@gmail.com, 0000-0002-1973-

1343)


Saionara C. Francisco, MSc20 (Francisco SCF - saionaracf@gmail.com, 0000-0002-9655-6294)


Maiara A. Floriani, MSc21 (Floriani AM - maiara.floriani@hmv.org.br, 0000-0002-2981-9445)


Guilherme F. Nascimento, MSc 22 (Nascimento GF - guilhermefagundesn@hotmail.com, 0000-

0001-9064-7067)


Bárbara L. Farace, MD23 (Farace BL - barbarafarace@gmail.com, 0000-0002-6172-1093)


Luanna S. Monteiro, MD24 (Monteiro LS - luannasmonteiro@gmail.com, 0000-0002-6621-3338)


Maira V. R. Souza-Silva, MD25 (Souza-Silva MVR - mairavsouza@gmail.com, 0000-0003-2079-

7291)


Thais L. S. Sales, MSc 26 (Sales TLS - thaislorennass30@yahoo.com.br, 0000-0002-1571-3850)




                                                                                             3
Karina Paula M.P. Martins, MD, MSc27 (Martins KPMP - kkpmprado2@gmail.com, 0000-0002-

8313-7429)


Israel J. Borges do Nascimento28 (Borges do Nascimento IJ - israeljbn@ufmg.br, 0000-0001-

5240-0493)


Tatiani O. Fereguetti, MD29 (Fereguetti TO - tatianifereguetti@gmail.com, 0000-0001-5845-

0715)


Daniel T.M.O. Ferrara30 (Ferrara DTMO - daniel@taiar.com.br, 0000-0003-0886-9627)


Fernando A. Botoni, MD31 (Botoni FA - fbotoni@medicina.ufmg.br, 0000-0001-6268-8507)


Ana Paula B.S. Etges, PhD32 (Etges APBS - anabsetges@gmail.com, 0000-0002-6411-3480)


Eric Boersma, MD, PhD33 (Boersma E - h.boersma@erasmusmc.nl, 0000-0002-2559-7128)


Carisi A. Polanczyk, MD, PhD34 (Polanczyk CA - carisi.anne@gmail.com, 0000-0002-2447-2577)


Alexandre V. Schwarzbold, MD, PhD35 (Schwarbold AV - alexvspoa@gmail.com, 0000-0002-

5535-6288)


Amanda O. Maurílio, MD36 (Maurílio AO - amandaoliveira.maurilio@gmail.com, 0000-0002-

9355-9596)


Ana Luiza B. A. Scotton, MD37 (Scotton ALBA - analuiza.bahia@yahoo.com.br, 0000-0002-5857-

2031)


André P. Weber38 (Weber AP - andrepweber@gmail.com, 0000-0002-1386-2553)


André S. M. Costa, MD39 (Costa ASM - andresmc@gmail.com, 0000-0002-9153-1186)


Andressa B. Glaeser40 (Glaeser ABG - andressa.glaeser@hmv.org.br, 0000-0002-1534-0900)




                                                                                             4
Angélica Aparecida C. Madureira, MD41 (Madureira AAC - angelicaacmadureira@gmail.com,

0000-0002-8465-5674)


Angelinda R. Bhering, MD42 (Bhering AR - lindabhering@gmail.com, 0000-0002-7391-1951)


Bruno Mateus de Castro43 (Castro BM - brunocastro1199@gmail.com, 0000-0002-7013-2127)


Carla Thais C. A. da Silva44 (Silva CTCA - carlatcas@gmail.com, 0000-0003-1726-4494)


Carolina M. Ramos, MD45 (Ramos CMR - carol.marques@live.com, 0000-0002-8258-0891)


Caroline D. Gomes46 (Gomes CD - carolinegomes.pesquisaclinica@gmail.com, 0000-0002-5254-

4320)


Cíntia A. de Carvalho47 (Carvalho CA - carvalho.cintiaa@gmail.com, 0000-0001-8240-2481)


Daniel V. Silveira, MD, MSc42 (Silveira DV - danielvez@gmail.com, 0000-0002-7381-1651)


Edilson Cezar48 (CezarE-crfsae3_hu@centrodepesquisaclinica.com.br, 0000-0002-4006-4354)


Elayne C. Pereira, PhD49 (Pereira EC - elaynepp@yahoo.com.br, 0000-0002-1178-669X)


Emanuele Marianne S. Kroger, MD45 (Kroger EMS - manu.kroger@gmail.com, 0000-0001-5108-

4337)


Felipe B. Vallt50 (Vallt FB - vallt@medufmg.com, 0000-0001-5111-1921)


Fernanda B. Lucas, MD51 (Lucas FB - fbl_lucas@yahoo.com.br, 0000-0003-3926-900X)


Fernando G. Aranha, MD52 (Aranha FG - fgaranha@icloud.com, 0000-0001-9173-8892)


Frederico Bartolazzi, MD51 (Bartolazzi F - fredlazzi@hotmail.com, 0000-0002-9696-4685)


Gabriela P. Crestani46 (Crestani GP - gabrielapetryc@gmail.com, 0000-0002-4991-4941)


Gisele A. N. Bastos, MD, ScD53 (Bastos GAN - gisele.nader@hmv.org.br, 0000-0001-9668-1434)


                                                                                          5
Glícia Cristina de C. Madeira, MD41 (Madeira GCC - glicia.cristina.castro@gmail.com, 0000-

0002-9375-5319)


Helena Carolina Noal MSc54 (Noal HC - helenacnoal@gmail.com, 0000-0001-9912-2881)


Heloisa R. Vianna, MD, MSc55 (Vianna HR - hrvianna@hotmail.com, 0000-0003-1144-6262)


Henrique C. Guimarães, MD, PhD56 (Guimarães HC - hcerqueirag@gmail.com, 0000-0001-6511-

7400)


Isabela M. Gomes28 (Gomes IM - isabela_moraes1@yahoo.com.br, 0000-0002-4653-6447)


Israel M. Romero, MD, PhD57 (Romero IM - Israel.molina@fiocruz.br, 0000-0001-6642-7515)


Joanna D. L. Batista, MD, PhD58 (Batista JDL - joannalyra@gmail.com, 0000-0002-3703-2845)


Joice C. de Alvarenga, MD47 (Alvarenga JC - joice-alvarenga@hotmail.com, 0000-0003-3536-

7112)


Júlia D. S. S. Guimarães, MD59 (Guimarães JDSS - jusabatino@gmail.com, 0000-0002-4738-

7754)


Júlia D.P. de Morais, MD55 (Morais JDP - juliadrumond_@hotmail.com, 0000-0002-9538-6828)


Juliana M. Rugolo60 (Rugolo JM - jr.machado@unesp.br, 0000-0003-3984-4959)


Karen Cristina J.R. Pontes58 (Pontes KCJR - karen_lua3@hotmail.com, 0000-0002-8068-5331)


Kauane Aline M. dos Santos43 (Santos KAM - kauane.ams@gmail.com, 0000-0003-0041-7271)


Leonardo S. de Oliveira61 (Oliveira LS - seixasleo@yahoo.com.br, 0000-0003-1575-6559)


Lílian S. Pinheiro62 (Pinheiro LS - lilian.pinheiro98@hotmail.com, 0000-0003-2957-7674)




                                                                                             6
Liliane S. Pacheco, MD, MSc54 (Pacheco LS - lilianespacheco@gmail.com, 0000-0002-0229-

0692)


Lucas de D. Sousa, MD37 (Sousa LD - ld-sousa@hotmail.com, 0000-0002-0281-0897)


Luciana S. F. Couto63 (Couto LSF - lucianasiuves@gmail.com, 0000-0002-7948-222X)


Luciane Kopittke, PhD64 (Kopittke L - kluciane@ghc.com.br, 0000-0002-6606-7756)


Luis Cesar S. de Moura, MD, PhD65 (Moura LCS - luis.moura@tacchini.com.br, 0000-0002-8138-

6042)


Luisa Elem A. Santos66 (Santos LEA, luisa_elem@hotmail.com, 0000-0003-0264-5497)


Máderson A. S. Cabral, MD25 (Cabral MAS - madersonalvares@hotmail.com, 0000-0001-8556-

1998)


Maíra D. Souza, MD67 (Souza MD - mairadiassouza@gmail.com, 0000-0003-3546-4000)


Marcela G.T. Tofani, MD68 (Tofani MGT - mgttofani@gmail.com, 0000-0003-2738-0747)


Marcelo Carneiro, MD, PhD69 (Carneiro M - marceloc@unisc.br, 0000-0003-3603-1987)


Maria Angélica P. Ferreira, PhD70 (Ferreira MAP - mpiferreira@hcpa.edu.br, 0000-0003-0961-

524X)


Maria Aparecida C. Bicalho, MD, PhD71 (Bicalho MAC - macbicalho@gmail.com, 0000-0001-

6298-9377)


Maria Clara P.B. Lima63 (Lima MCPB - mariaclarapontellobl@gmail.com, 0000-0003-0461-2787)


Mariana F. Godoy, MD72 (Godoy MF - mfdegodoy@gmail.com, 0000-0002-6631-8826)


Marilia M. A. Cardoso, PhD60 (Cardoso MMA - marilia.cardoso@unesp.br, 0000-0002-6231-

5425)


                                                                                             7
Meire P. Figueiredo51 (Figueiredo MP - mpfenf@yahoo.com.br, 0000-0002-0724-6893)


Natália C.S. Sampaio73 (Sampaio NCS - natsamster@gmail.com, 0000-0002-5799-8006)


Natália L. Rangel, MD74 (Rangel NL - netx_11@hotmail.com, 0000-0001-6968-2271)


Natália T. Crespo, MD74 (Crespo NT - nat_trifiletti@hotmail.com, 0000-0002-1009-5299)


Neimy R. de Oliveira, MD73 (Oliveira NR - neimyramos@gmail.com, 0000-0001-5408-9459)


Pedro L. Assaf, MD75 (Assaf PL - pedro.ledic@hmdcc.com.br, 0000-0002-8419-5719)


Petrônio José de L. Martelli, MD, PhD76 (Martelli PJL - petroniocarla@uol.com.br, 0000-0001-

6920-6435)


Rafaela S.C. Almeida64 (Almeida RSC - rafacharao@gmail.com, 0000-0002-2091-5772)


Raphael C. Martins65 (Martins RC - rapahaelcm@gmail.com, 0000-0003-2167-3645)


Raquel Lutkmeier64 (Lutkmeier R - raquellutk83@gmail.com, 0000-0002-9519-261X)


Reginaldo Aparecido Valacio67 (Valacio RA - ravalacio@hotmail.com, 0000-0003-2666-7092)


Renan G. Finger, MD58 (Finger RG - renanfinger@yahoo.com.br, 0000-0002-5569-7787)


Ricardo B. Cardoso, MSc32 (Cardoso RB - ricardobcardoso@gmail.com, 0000-0002-4931-4081)


Roberta Pozza MD, PhD65 (Pozza R - robertapozza@tacchini.com.br, 0000-0003-1733-6398)


Roberta X. Campos, MD59 (Campos RX - xaviercampos.roberta@gmail.com, 0000-0002-2963-

8994)


Rochele M. Menezes, MD69 (Menezes RM - rochelemenezes@unisc.br, 0000-0002-1548-1607)


Roger M. de Abreu, MD75 (Abreu RM - roger.abreu@hmdcc.com.br, 0000-0002-2153-1340)


Rufino de F. Silva, MD36 (Silva RF - rufino@ufsj.edu.br, 0000-0003-1952-9193)

                                                                                               8
Silvana M. M. Guimarães, MD68 (Guimarães SMM - smangeon@gmail.com, 0000-0002-9655-

6294)


Silvia F. Araújo, MD68 (Araújo SF - silviaferreiragastro@gmail.com, 0000-0003-4782-5440)


Susany Anastácia Pereira28 (Pereira AS - susany2808@gmail.com, 0000-0001-5039-443X)


Talita F. Oliveira, MD67 (Oliveira TF - talitafischeroliveira@gmail.com, 0000-0002-4614-3109)


Tatiana Kurtz MD, PhD69 (Kurtz T - kurtz@unisc.br, 0000-0003-2124-711X)


Thainara C. de Oliveira48 (Oliveira TC - thainarastaehler@hotmail.com, 0000-0002-1248-2305)


Thaíza Simônia M.A. Araújo28 (Araújo TSMA - thaizaalbino16@gmail.com, 0000-0002-2082-

9786)


Thulio Henrique O. Diniz, MD36 (Diniz THO - thuliodinizmed@gmail.com, 0000-0003-1798-

4643)


Veridiana B. dos Santos64 (Santos VB - veridbaldon@hotmail.com, 0000-0002-9372-5785)


Virginia Mara R. Gomes77 (Gomes VMR - vgvirginiagomes@gmail.com, 0000-0001-7592-5604)


Vitor Augusto L. do Vale59 (Vale VAL - vitoravale@gmail.com, 0000-0002-9910-2861)


Yuri C. Ramires MD38 (Ramires YC - yuri.ramires@gmail.com, 0000-0002-0730-8716)




Affiliations, positions and addresses:

1
    Associate Professor and Internal Medicine Physician. Department of Internal Medicine,

Medical School; and Telehealth Center, University Hospital, Universidade Federal de Minas

Gerais. Avenida Professor Alfredo Balena 190, sala 246, Belo Horizonte, Brazil. Researcher.




                                                                                                9
Institute for Health Technology Assessment (IATS/ CNPq). Rua Ramiro Barcelos, 2359. Prédio

21|Sala 507, Porto Alegre, Brazil.

2
    Associate Professor and Statistician, Department of Statistics, Universidade Federal de Minas

Gerais. Av. Presidente Antônio Carlos, 6627, ICEx, sala 4071, Belo Horizonte, Brazil

3
    Undergraduate Statistics Students, Universidade Federal de Minas Gerais. Av. Presidente

Antônio Carlos, 6627, Belo Horizonte, Brazil

4
    Ph Student in Business and Administration. Center for Research and Graduate Studies in

Business Administration, Universidade Federal de Minas Gerais. Av. Presidente Antônio Carlos,

6627, Belo Horizonte, Brazil. Researcher. Institute for Health Technology Assessment (IATS/

CNPq). Rua Ramiro Barcelos, 2359. Prédio 21 | Sala 507, Porto Alegre, Brazil.

5
    Nurse and Researcher. Institute for Health Technology Assessment (IATS/ CNPq). Rua Ramiro

Barcelos, 2359. Prédio 21 | Sala 507, Porto Alegre, Brazil.

6
    Undergraduate Medical Student. Universidade Federal de São João del-Rei. R. Sebastião

Gonçalves Coelho, 400, Divinópolis, Brazil.

7
    Physician and Researcher. Infectious Diseases Department, Vall d’Hebron University Hospita,

Universitat Autònoma de Barcelona, PROSICS, Barcelona, Spain. Passeig de la Vall d'Hebron,

119-129, Edificio Mediterránea, despacho 119, 08035, Barcelona, Spain.

8
    Infectious Diseases Department, Vall d’Hebron University Hospital, Universitat Autònoma de

Barcelona, Barcelona, Spain

9
    Coordinator of the Research Sector of Grupo Hospitalar Conceição. Professor of the

Graduation Program on Evaluation and Production of Technologies for the Brazilian National

Health System, Hospital Nossa Senhora da Conceição and Hospital Cristo Redentor. Av.

Francisco Trein, 326, Porto Alegre, Brazil.


                                                                                              10
10
     Professor. School of Medicine, Pontifícia Universidade Católica do Rio Grande do Sul (RGS),

Porto Alegre, Brazil; Head in Pneumology Department, Hospital São Lucas PUCRS, Porto Alegre,

Brazil. Rua João Cateano, 79/503. Porto Alegre, Brazil.

11
     Physician and Researcher. Internal Medicine Department, Rede Mater Dei de Saúde. Av. do

Contorno, 9000, Belo Horizonte, Brazil.

12
     Teaching and Research Coordinator. Hospital Márcio Cunha. Av. Kiyoshi Tsunawaki, 41,

Ipatinga, Brasil.

13
     Professor and epidemiologist, School of Medicine. Universidade do Sul de Santa Catarina

(UNISUL). Avenida Pedra Branca, 25, Cidade Universitária Pedra Branca, Palhoça, Brazil.

Epidemiologist, Dissertare Scientific Advice, Rodovia João Paulo, 1030, Lá opera, 401b,

Florianópolis, Brazil. Collaborating researcher, SOS Cardio Hospital, Rodovia, SC-401, 121,

Florianópolis, Brazil.

14
     Professor and Infectious Diseases Physician. Internal Medicine Department. University

Hospital, Universidade Federal de Minas Gerais. Av. Prof Alfredo Balena, 110, Belo Horizonte,

Brazil.

15
     Professor and Nephrologist. Faculdade de Medicina de Botucatu - Universidade Estadual

Paulista "Júlio de Mesquita Filho". Av. Prof. Mário Rubens Guimarães Montenegro, s/n - UNESP

- Campus de Botucatu, Botucatu, Brazil.

16
     Professor and Statistician. Universidade Federal do Rio Grande do Sul and Institute for Health

Technology Assessment (IATS/ CNPq). Hospital Tacchini. Rua Ramiro Barcelos, 2359, Prédio 21

| Sala 507, Porto Alegre, Brazil.

17
     Clinical Pharmacist - UDM Coordinator of Specialized Assistance Service. Pharmaceutical

Assistance Research Center of Vale do Taquari. Av. Rio Branco, 1127, Estrela, Brazil.



                                                                                                11
18
     Researcher. Hospital Mãe de Deus, Hospital Universitário de Canoas, Universidade Federal do

Rio Grande do Sul and Institute for Health Technology Assessment (IATS/ CNPq). Rua Ramiro

Barcelos, 2359. Prédio 21 | Sala 507, Porto Alegre, Brazil.

19
     Adults’ Intensive Care Physician. Adjunct Professor. Mucuri Medical School – FAMMUC,

Universidade Federal dos Vales do Jequitinhonha e Mucuri – UFVJM. Hospital Santa Rosalia.

R.do Cruzeiro, 01, Teófilo Otoni, Brazil.

20
     Coordinator of the Teaching and Research Center, Hospital Metropolitano Doutor Célio de

Castro. Rua Dona Luiza, 311, Belo Horizonte, Brazil.

21
     Head of Value Management Office. Moinhos Research Institute. 910 Ramiro Barcelos Street,

5 floor, Porto Alegre, Brazil.

22
     Emergency and Internal Medicine Physician, Hospital Unimed BH. Av. do Contorno, 3097,

Belo Horizonte, Brazil.

23
     Internal Medicine Resident. Hospital Risoleta Tolentino Neves. Rua das Gabirobas, 01, Belo

Horizonte, Brazil.

24
     Internal Medicine Physician and Master’s Student. Hospital Metropolitano Odilon Behrens.

Medical School and University Hospital, Universidade Federal de Minas Gerais. Avenida

Professor Alfredo Balena 190, Belo Horizonte, Brazil.

25
     Physician and Master’s Student. Medical School and University Hospital, Universidade

Federal de Minas Gerais. Avenida Professor Alfredo Balena, 190, sala 246, Belo Horizonte,

Brazil.

26
      Pharmacist and PhD student. Universidade Federal de São João del-Rey. R. Sebastião

Gonçalves Coelho, 400, Divinópolis, Brazil.




                                                                                             12
27
     Internal Medicine Physican and PhD student. Medical School and University Hospital,

Universidade Federal de Minas Gerais. Avenida Professor Alfredo Balena 190, Belo Horizonte,

Brazil.

28
     Undergraduate Medical Student. Medical School, Universidade Federal de Minas Gerais.

Avenida Professor Alfredo Balena, 190, Belo Horizonte, Brazil.

29
     Infectious Diseases Physician. Hospital Eduardo de Menezes. R. Dr. Cristiano Rezende, 2213 -

Bonsucesso, Belo Horizonte. Brazil.

30
     UX Designer. Universidade FUMEC. R. Cobre, 200, Belo Horizonte, Brazil.

31
     Professor and Physician. Medical School, Universidade Federal de Minas Gerais. Hospital Julia

Kubitschek. Avenida Professor Alfredo Balena 190, Belo Horizonte, Brazil.

32
     Researcher. Universidade Federal do Rio Grande do Sul and Institute for Health Technology

Assessment (IATS/ CNPq). Rua Ramiro Barcelos, 2359. Prédio 21 | Sala 507, Porto Alegre,

Brazil.

33
     Professor. Erasmus MC, University Medical Center Rotterdam, Department of Cardiology,

Rotterdam, The Netherlands.

34
     Professor and Physician. Internal Medicine Department. Universidade Federal do Rio Grande

do Sul. Coordinator of the Institute for Health Technology Assessment (IATS/ CNPq). Rua

Ramiro Barcelos, 2359. Prédio 21 | Sala 507, Porto Alegre, Brazil.

35
     Professor. Hospital Universitário de Santa Maria. Av. Roraima, 1000, prédio 22, Santa Maria,

Brazil.

36
     Hospital São João de Deus. R. do Cobre, 800, São João de Deus, Brazil.

37
     Physician. Hospital Regional Antônio Dias. R. Maj. Gote, 1231, Patos de Minas, Brazil.



                                                                                               13
38
     Physician. Hospital Bruno Born. Av. Benjamin Constant, 881, Lajeado, Brazil.

39
     Hospitais da Rede Mater Dei. Belo Horizonte, Brazil.

40
     Biomedical and Master’s Student. Hospital Moinhos de Vento. R. Ramiro Barcelos, 910, Porto

Alegre, Brazil.

41
     Undergraduate Medical Student. Faculdade Ciências Médicas de Minas Gerais. Alameda

Ezequiel Dias, 275, Belo Horizonte, Brazil.

42
     Physician. Hospital UNIMED BH. Av. do Contorno, 3097, Belo Horizonte, Brazil.

43
     Universidade Federal do Rio Grande do Sul. Av. Paulo Gama, 110, Porto Alegre, Brazil.

44
     Undergraduate Nursing Student. Faculdade de Ciências Humanas de Curvelo. Av. JK, 1441,

Curvelo, Brazil.

45
     Medical Clinic Resident. Hospital Julia Kubitschek. R. Dr. Cristiano Rezende, 2745, Belo

Horizonte, Brazil.

46
     Hospital Mãe de Deus. R. José de Alencar, 286, Porto Alegre, Brazil.

47
     Hospital João XXIII. Av. Professor Alfredo Balena, 400, Belo Horizonte, Brazil.

48
     Hospital Universitário Canoas. Av. Farroupilha, 8001, Canoas, Brazil.

49
     Professor. Universidade do Sul de Santa Catarina (UNISUL). Avenida Pedra Branca, 25, Cidade

Universitária Pedra Branca, Palhoça, Brazil.

50
     Universidade Federal de Minas Gerais. Av. Presidente Antônio Carlos, 6627, Belo Horizonte,

Brazil

51
     Hospital Santo Antônio. Praça Dr. Márcio Carvalho Lopes Filho, 501, Curvelo, Brazil.

52
     Physician. Hospital SOS Cárdio. Rodovia, SC-401, 121, Florianópolis, Brazil.


                                                                                             14
53
     Medical Manager. Hospital Moinhos de Vento. R. Ramiro Barcelos, 910, Porto Alegre, Brazil.

54
     Hospital Universitário de Santa Maria. Av. Roraima, 1000, prédio 22, Santa Maria, Brazil.

55
     Hospital Universitário Ciências Médicas. Rua dos Aimorés – 2896, Belo Horizonte, Brazil.

56
     Hospital Risoleta Tolentino Neves. Rua das Gabirobas, 01, Belo Horizonte, Brazil.

57
     Vall d’Hebron University Hospital, PROSICS Barcelona. Passeig de la Vall d'Hebron, 119,

08035, Barcelona, Spain. Instituto René Rachou-FIOCRUZ Minas. Av. Augusto de Lima, 1715,

Belo Horizonte, Brazil.

58
     Hospital Regional do Oeste. R. Florianópolis, 1448 E, Chapecó, Brazil.

59
     Undergraduate Medical Student. Pontífica Universidade Católica de Minas Gerais. Av. Dom

José Gaspar, 500, Belo Horizonte, Brazil.

60
     Hospital das Clínicas da Faculdade de Medicina de Botucatu. Av. Professor Mário Rubens

Guimarães Montenegro, Botucatu, Brazil.

61
     Hospital Santa Rosália. R. Dr. Onófre, 575, Teófilo Otoni, Brazil.

62
     Undergraduate Medical Student. Universidade Federal dos Vales do Jequitinhonha e Mucuri.

R. Cruzeiro, 1, Teófilo Otoni, Brazil.

63
     Undergraduate Medical Student. Universidade Federal de Ouro Preto. R. Diogo de

Vasconcelos, 122, Ouro Preto, Brazil.

64
     Hospital Nossa Senhora da Conceição. Av. Francisco Trein, 326, Porto Alegre, Brazil.

65
     Hospital Tacchini. R. Dr. José Mário Mônaco, 358, Bento Gonçalves, Brazil.

66
     Undergraduate Medical Student. Centro Universitário de Patos de Minas. R. Maj. Gote, 808,

Patos de Minas, Brazil.


                                                                                                 15
67
     Hospital Metropolitano Odilon Behrens. R. Formiga, 50, Belo Horizonte, Brazil.

68
     Hospital Semper. Alameda Ezequiel Dias, 389, Belo Horizonte, Brazil.

69
     Hospital Santa Cruz. Rua Fernando Abott, 174, Santa Cruz do Sul, Brazil.

70
     Professor. Hospital de Clínicas de Porto Alegre. Av. Ramiro Barcellos, 2350, Porto Alegre,

Brazil.

71
     Fundação Hospitalar do Estado de Minas Gerais – FHEMIG. Cidade Administrativa de Minas

Gerais, Edifício Gerais – 13º andar, Rodovia Papa João Paulo II, 3777, Belo Horizonte, Brazil.

72
     Hospital São Lucas da PUCRS. Av. Ipiranga, 6690, Porto Alegre, Brazil.

73
     Hospital Eduardo de Menezes. R. Dr. Cristiano Rezende, 2213, Belo Horizonte, Brazil.

74
     Hospital Márcio Cunha. Av. Kiyoshi Tsunawaki, 41, Ipatinga, Brasil.

75
     Hospital Metropolitano Doutor Célio de Castro. Rua Dona Luiza, 311, Belo Horizonte, Brazil.

76
     Hospital das Clínicas da Universidade Federal de Pernambuco. Universidade Federal de

Pernambuco, Centro de Ciências da Saúde. Av. da Engenharia, s/n, Bloco D 1º Andar, Recife,

Brazil.

77
     Centro Universitário de Belo Horizonte (UniBH). Av Professor Werneck, 1685, Belo Horizonte,

Brazil.

#
Correspondence to: Milena Soriano Marcolino

University Hospital, Universidade Federal de Minas Gerais

Avenida Professor Alfredo Balena, 110 Room 107. Ala Sul.

Santa Efigênia – Belo Horizonte – MG. Brazil.

CEP 30130-100
*
These authors contributed equally to the work.


                                                                                                 16
Abstract

Objectives: The majority of available scores to assess mortality risk of coronavirus disease 19

(COVID-19) patients in the emergency department have high risk of bias. Therefore, our aim

was to develop and validate a score at hospital admission for predicting in-hospital mortality in

COVID-19 patients, and to compare this score with other existing ones.

Methods: Consecutive patients (≥18 years) with confirmed COVID-19 admitted to the

participating hospitals were included. Logistic regression analysis was performed to develop a

prediction model for in-hospital mortality, based on the 3978 patients admitted between

March-July, 2020. The model was validated in the 1054 patients admitted during August-

September, as well as in an external cohort of 474 Spanish patients.

Results: Median (25th-75th percentile) age of the model-derivation cohort was 60 (48-72)

years, and in-hospital mortality was 20.3%. The validation cohorts had similar age distribution

and in-hospital mortality. Seven significant variables were included in the risk score: age, blood

urea nitrogen, number of comorbidities, C-reactive protein, SpO2/FiO2 ratio, platelet count and

heart rate. The model had high discriminatory value (AUROC 0.844, 95% CI 0.829 to 0.859),

which was confirmed in the Brazilian (0.859 [95% CI 0.833 to 0.885]) and Spanish (0.894 [95%

CI 0.870 to 0.919]) validation cohorts, and displayed better discrimination ability than other

existing scores. It is implemented in a freely available online risk calculator

(https://abc2sph.com/).


Conclusions: We designed and validated an easy-to-use rapid scoring system based on

characteristics of COVID-19 patients commonly available at hospital presentation, for early

stratification for in-hospital mortality risk of patients with COVID-19.

Key words: COVID-19; SARS-CoV-2; mortality; prognosis; risk factors; hospitalizations; score




                                                                                               17
Introduction


           Coronavirus disease 2019 (COVID-19) is still the main global health, social and

economic challenge, overwhelming health care systems in many countries and heavily

burdening others.1,2 Case rates continue to rise, and some hospitals are nearly at their full

capacity of intensive care unit beds. The emergence of the new variants of SARS-CoV-2 in

England, South Africa, Brazil and India, with very high viral growth, more transmissible, less

detectable with the RT-PCR technique or deadlier than the wild-type SARS-CoV-2 and evidence

of lower vaccine efficacy, is currently a cause of huge concern3-6.

           Fast and efficient assessment of prognosis of the disease is needed to optimize the

allocation of health care and human resources, to empower early identification and

intervention of patients at higher risk of poor outcome. A proper assessment tool will guide

decision making to develop an appropriate plan of care for each patient7. In this context, rapid

scoring systems, which combine different variables to estimate the risk of a poor outcome,

may be extremely helpful for quick and effective assessment of those patients in the

emergency department8.

           Although different scores have been proposed to assess prognosis in COVID-19

patients, the majority of them lack benefit to clinical decision making, and there is a lack of

reliable prognostic prediction models9,10. Most scores were developed from small cohorts, at

high risk for bias, with selected study samples and relatively few outcome events, without

clear details of model derivation and validation, as well as unclear reporting on intended use11-
15
     . These issues lead to a high risk of model overfitting, thus their predictive performance when

used in clinical practice may be different than that reported10,11, and external validation was

rarely done 16 10.

           In this context, our aim was to develop and validate an easy applicable score that

employs routinely available clinical and laboratory data at hospital presentation, to predict in-


                                                                                                 18
hospital mortality in patients with COVID-19, able to discriminate high vs non-high risk

patients. Additionally, we aimed to compare this score with other existing ones.




Methods

        This study is part of the Brazilian COVID-19 Registry, an ongoing multicenter

observational study described elsewhere17, and a collaboration with Vall d’Hebron University

Hospital, in Barcelona, Spain, for independent external validation. The Brazilian COVID-19

Registry is being conducted according to a predefined protocol, in 36 Brazilian hospitals,

located in 17 cities, from five Brazilian states.

        Model development, validation and reporting followed guidance from the Transparent

Reporting of a Multivariable Prediction Model for Individual Prediction or Diagnosis (TRIPOD)

checklist and Prediction model Risk Of Bias ASsessment Tool (PROBAST) (Supplementary

Material)18,19.




Study subjects

        Consecutive adult patients (≥18 years-old) with laboratory-confirmed COVID-1920

admitted to the participating hospitals from March 1 to September 30, 2020 were enrolled.

Patients who were transferred between hospitals and admission data from the first hospital

(as we aimed to develop a score to be used in the first assessment) or the last hospital was not

available were excluded (Table S1), as well those who were admitted for other reasons and

developed COVID-19 symptoms during their stay (Figure 1).




Measurement and data quality assessment




                                                                                             19
        Demographic information, clinical characteristics, laboratory and outcome data were

collected from the medical records by using a prespecified case report form applying Research

Electronic Data Capture (REDCap) tools21,22. Data were collected by trained hospital staff or

interns. A detailed data management plan (DMP) was developed and provided to all

participating centers, and an online DMP training was mandatory before local research

personnel were allowed to start collecting study data23.

        We undertook comprehensive data quality checks to ensure high quality. A code was

developed in R software to identify values likely related to data entry errors for vital signs and

laboratory variables, based on expert-guided rules. Data were sent to each center for checking

and correction.




Potential predictors for in-hospital mortality

        All variables used to calculate the risk score were obtained at hospital admission. A set

of potential predictor variables for in-hospital mortality was selected a priori, as

recommended19, taking into account the evidence in literature of association with worse

prognosis in patients with COVID-19 or pneumonia, and availability of predictor measurement

at the time the model would be used, i.e., hospital admission11. All laboratory tests were

performed at the discretion of the treating physician. Imaging test results were not included,

as X-ray and CT scan are not always performed at patient admission and their interpretation

involve subjective judgement.


        Candidate predictor variables which were not available for at least two thirds of

patients within the derivation cohort (more than one third of missing data) were excluded.




Data analysis



                                                                                               20
        Continuous variables were summarized using medians and interquartile ranges (IQR),

whereas counts and percentages were used for categorical variables. We reported 95%

confidence intervals, and a p-value less than 0.05 was considered statistically significant.

Statistical analysis was performed with R software (version 4.0.2) with the mgcv, finalfit, mice,

glmnet, pROC, rms, rmda, and psfmi packages.




Missing data

        Considering missing at random after analyzing missing data patterns, multiple

imputation with chained equations (MICE) was used to handle missing values on candidate

variables (outcomes were not imputed). Mortality outcome was used as a predictor in MICE in

the derivation dataset, but not in the validation dataset. We used predictive mean matching

(PMM) method for continuous predictors and polytomous regression for categorical variables

(two or more unordered levels). The results of ten imputed datasets, each with ten iterations,

were combined following Rubin’s rules24.



Development of the risk score model

        Patients who were admitted before July 31 were included in the development cohort

(Figure S1). We conducted predictor selection based on clinical reasoning and literature review

before modeling. Generalized additive models (GAM) were used to examine the relationships

between in-hospital mortality and continuous (through penalized thin plate splines) and

categorical (as linear components) predictors. During this stage, variable selection was based

on D1- (multivariate Wald test) and D2-statistic (pools test statistics from the repeated

analyses).




                                                                                              21
        Subsequently, for an easier application of the risk score model at bedside, continuous

variables were categorized based on widely accepted cut points, current evidence and/or

categories defined in stablished scores for pneumonia and sepsis.

        Lastly, we used least absolute shrinkage and selection operator (LASSO) logistic

regression to derive the mortality score by scaling the (L1 penalized) shrunk coefficients. The

penalty parameter λ in LASSO was chosen using 10-fold cross-validation methods based on

mean squared error criterion.

        Risk groups were proposed based on predicted probabilities: low risk ( < 6.0%),

intermediate risk (6.0 – 14.9%), high risk (15.0 – 49.9%), and very high risk (≥ 50.0%), as

recommended by TRIPOD Guidelines18. Risk strata was based on other scores25.



External validation

        We performed an external (temporal) validation analysis using patients who were

admitted from August 1 to September 30, 2020.

        Independent external validation was also performed in a cohort of patients from Vall

d’Hebron University Hospital, in Barcelona, Spain, admitted from March 1 to May 31, 202026.

Inclusion and exclusion criteria were the same as the beforementioned ones. All patients

included were followed for at least 28 days. The aforementioned methods for data imputation

were used.



Performance measures

        We evaluated overall performance using Brier score27. Calibration was assessed

graphically by plotting the predicted mortality probabilities against the observed mortality,

testing intercept equals zero and slope equals one. The area under the receiver operating

characteristic curve (AUROC) described model’s discrimination. Confidence intervals (95% CI)




                                                                                            22
for AUROC were obtained through 2000 bootstrap samples. Positive and negative predictive

values of the derived risk groups were calculated.




Model comparisons


          The developed model was compared within the validation cohort with existing scores.

These scores were identified through a literature search of Medline, medRxiv and BioRxiv, with

no language or date restrictions, using the search terms “COVID-19,” “COVID”, “SARS-CoV-2,”

“coronavirus” combined with “score” and “mortality”. The last search was performed on

November 19, 2020. Two authors independently performed article selection and data

extraction. Additionally, we also included established scores for pneumonia and sepsis28-32.


          From the set of identified scores, we selected those which with predictors were

available within the database and had accessible methods for calculation. Model comparisons

were performed using AUROC and decision curve analysis.




ABC2-SPH risk score calculator


          Risk score calculator was developed in Javascript, using the Svelte framework while the

website was developed in R language (blogdown package).




Results

          The derivation cohort included 3978 patients, from 267 cities in Brazil (Figure 2).

Median age was 60 [IQR, 48-72] years, 53.8% were male, 70.1% had at least one comorbidity

and 20.3% died during hospitalization. Table 1 shows demographic, clinical characteristics and

laboratory findings for the derivation and validation datasets.

                                                                                               23
Development of the risk score model

        Thirty-six potential predictor variables were identified (Table S2). Number of

comorbidities was created as a composite of ten individual comorbidities shown to have

prognostic impact in COVID-9 (hypertension, diabetes mellitus, obesity, coronary artery

disease, heart failure, atrial fibrillation or flutter, cirrhosis, chronic obstructive pulmonary

disease, cancer and previous stroke)33,34, as in other scores25,35. Twelve variables were excluded

due to the excessive number of missing values, two for high collinearity, and one was not

recorded within database. Inotrope use was combined with blood pressure. Therefore, 20

variables were tested.


        Through generalized additive model (GAM), a combination of seven variables was

selected as the best predictor of in hospital mortality (Table S3). For an easier application to

the risk score model at bedside, continuous selected predictors were categorized for LASSO

logistic regression. All categories were defined a priori, based on widely accepted cut points,

current evidence and/or categories defined in stablished rapid scoring systems from

pneumonia and sepsis19.


        All variables were statistically significant predictors for in hospital mortality (Table S4

and Figure S2). Shrunk coefficients were scaled to provide a prognostic index and we denoted

it as the ABC2-SPH risk score (Table 2). The sum of the prediction scores ranges between 0 and

20, with a high score indicating higher risk of in-hospital mortality.

        Risk groups were proposed based on predicted probabilities (Table 3): low risk (0-1

score, observed in hospital mortality 2.0%), intermediate risk (2-4 score, 11.4%), high risk (5-8

score, 32.0%), and very high risk (≥ 9 score, 69.4%). Subject-specific risks can be assessed using

the developed ABC2-SPH risk score Web-based calculator (https://abc2sph.com/), freely

available to the public, and it can also be assessed through infographics (Figure S3).


                                                                                                24
        As well as GAM and LASSO, ABC2-SPH risk score showed good overall performance

(Brier score: 0.114) and good discrimination (AUROC equal 0.842; 95% CI 0.840–0.843) within

the derivation cohort (Table 4).



External validation – Brazilian cohort

        A total of 1054 patients were included in the validation cohort. Median age was 62

(IQR 48-73) years, 55.2% were male, 70.7% had at least one comorbidity and 19.7% died

during hospitalization. The distribution of patients across range ABC 2-SPH score in derivation

and validation cohorts are presented in Figure 3.

        We observed ABC2-SPH risk score had AUROC=0.859 (95% CI 0.833 to 0.885), good

overall performance (Brier = 0.108) and calibration (slope = 1.138, intercept = 0.114, p-value =

0.184; Figure S4a) under the validation cohort (Figure 4). The good performance is also

demonstrated in sensitivity analyses using complete case data (Table S5).

        Low, intermediate and high-risk groups showed good negative predictive values

(99.7%, 88.1% and 71.0%, respectively). A positive predictive value of 73.7% was observed in

patients classified as at very high mortality risk.



External validation – Spanish cohort

        A second external (geographic) validation was performed within a Spanish cohort with

856 patients and 172 (20.1%) in-hospital mortality. The demographic and clinical

characteristics at admission are listed in Table 1. The median follow-up time was 21 (IQR, 7-40)

days.

        ABC2-SPH score showed AUROC= 0.894 (95% CI 0.870 to 0.919; Figure 4), good overall

performance (Brier = 0.093). Calibration is shown in Figure S4b.




Literature review

                                                                                             25
       The literature search identified 39 scores to predict mortality in COVID-19 patients

(Table 5). In 36% the derivation cohort was from China, 21% from the United States and none

from South America.




Comparison with other scores

       Based on complete case validation cohort, the ABC2-SPH score achieved better

discrimination (Table 6, Figure 5a) than other prediction scoring systems for COVID-19,

pneumonia and sepsis (0.85; 95% CI: 0.82 – 0.88). Xie’s and Zhang’s score7,36,37 showed good

discrimination, but the number of complete cases and deaths were relatively small.

Considering clinical utility (Figure 5b), ABC2-SPH showed a better performance compared to

the three most discriminating scores for in-hospital mortality that were tested in more than

600 patients (4C Mortality Score, A-DROP and CURB-65) 25, 29.




Discussion

       ABC2-SPH score is simple, objective, easily available at hospital admission and easily

calculated, employing seven well defined and routinely recordable variables. It has shown to

be a reliable tool to estimate in-hospital mortality in COVID-19 patients. Model performance

compared surpassed other existing scores.

       The majority of developed scores are limited by methodological bias in development

cohorts. Robust models require large sample sizes, which produce more reliable and accurate

results18. Among the models analyzed for comparison, only 30.8% used a sample with more

than 1000 patients, 41.0% used a sample with less than 500 patients, and 41.0% were

developed and validated in a sample with less than 100 events.




                                                                                          26
        Most of the models we analyzed for comparison (69.2%) did not perform or did not

describe whether imputation methods were used for the missing data, therefore, there is a

high risk of bias related to the treatment of missing data. The approach of excluding the

missing data and performing the analysis with the complete cases can lead to biased results,

since the complete cases may not adequately represent the entire original study sample,

generating a selection bias18.

        In 43.6% of the analyzed studies, external validation of the developed model was not

performed. As the accuracy of a prediction model is always high whether the model is

validated on the development cohort used to derive the model only, the assessment of

accuracy in those studies may be overoptimistic18.

        Previous studies have observed the variables included in the ABC2-SPH score as risk

factors for severe COVID-19, what shows that our results are in line with the available

evidence. Age and number of comorbidities were reported as independent risk factors for

developing severe COVID and mortality in several publications 9,25,38. The strong age gradient

per decade after 60 years-old is in line with other series9,13. Aging is associated with a well-

known decline in the adaptive and innate immunity, which plays a major role in the increased

susceptibility of infections39. It is also related to an increased severity in pro-inflammatory

response and increased cytokine production, what is believed to increase patient vulnerability

to the unregulated inflammatory response in COVID-1940.

        The number of comorbidities indicates the importance of pre-existing conditions to the

severity of COVID-19. Even though comorbidities are age-dependent factors, the number of

comorbidities remained as an independent risk factor in the final model.

        As we aimed to use variables easily available at ED admission of any institution, we

opted to evaluate the SpO2/FiO2 ratio (SF ratio), instead of the ratio of arterial oxygen partial

pressure over the fraction of inspired oxygen (PaO2/FiO2 ratio), like the COVID-AID score41.


                                                                                              27
Arterial blood gas puncture and analysis is an invasive and complex procedure, which may be

time consuming for the team. SF ratio was already validated as a substitute for the PaO2/FiO2

ratio in assessing the oxygenation criterion of patients with acute lung injury and acute

respiratory distress syndrome42.

          COVID-19 associated hyperinflammation and coagulopathy are correlated to with a

wide deviation in various inflammatory markers and hemostasis parameters, and thus these

are potential prognostic markers of increased mortality in COVID-1943,44. Consistent with prior
          45,46
studies           , we also observed utility of CRP, an acute phase reactor with established

prognostic prediction role in intensive care septic and non-septic patients47,48, and

thrombocytopenia. The prognostic value of thrombocytopenia in COVID-19 patients has shown

in a recent meta-analysis49, and it was also included in other scores9,45. The exact explanation is

still unknown, and it is probably multifactorial, related to direct infection of bone marrow cells

by the virus, resulting in abnormal hematopoiesis; platelet destruction by the immune

system; endothelial damage triggering platelet activation, aggregation and microthrombi in

the lung; and abnormal platelet defragmentation in the lungs49.


          A recent meta-analysis (16 studies, n=3480) has shown significantly higher levels of D-

dimer on admission in patients who died compared to the ones who were discharged 43. This

exam was included as a predictor in different scores12,15,46,50,51. However, the value has to be

determined with the same methodology, preferably from the same manufacturer, and this

information was not available in any of the studies. A recent publication highlighted confusion

and potential for misinformation in reporting D-dimer data in COVID-1952. The authors

provided examples of serious errors in the reported values and/or units as reported in the

literature related to COVID-19, even in high impact journals.


          Most studies have not reported how they dealt with cases who were transferred

between hospitals. As the score is intended to be used at hospital admission, we opted to


                                                                                                28
exclude patients who were transferred between hospitals and admission data from the first

hospital was not available, to avoid the possibility the patients were already with critical

disease.


        Blood urea nitrogen elevation was a strong predictor for mortality, what is in line with

other scores25,53,54. Although autopsy studies did not find conclusive evidence of SARS-CoV2

infection in the kidney, some authors hypothesize that the damage may be mediated by direct

cytopathic effects of SARS-Cov2 on the kidney tissue, immune-mediated damage due to virus-

induced immune complexes, as well as the effects of the inflammatory response, hypoxia and

shock55-57.


        A major strength of the ABC2-SPH score is its simplicity, with the use of objective

parameters, what helps to reduce inter-user variability, easily available at the emergency

department presentation, even in under-resourced settings. The present study followed strict

methodological criteria, recommended by TRIPOD18 checklist and PROBAST19, and was based

on robust sample of patients with confirmed SARS-CoV-2 infection from 36 hospitals in four

Brazilian states, to ensure diversity of the population studied and representativeness of the

intended target population. The majority of published scores were developed in China or the

US (56.4%) and Europe (25.6%), this is the first study in the Latin American population. Data

were obtained by detailed medical chart reviews, and we were able to collect comprehensive

data from a large number of patients and follow 98.5% of the patients from admission to

discharge or death. Decisions about which predictors to retain in the final model did not rely

on potentially biased univariable selection of predictors. They were based on clinical

reasoning, previous evidence from other cohorts and systematic reviews on prognostic factors

for COVID-19 patients and availability of predictor measurement at hospital admission18.

        With regards to study limitations, due to the pragmatic study design, laboratory tests

were performed at the discretion of the treating physician, and we did not have a full dataset


                                                                                             29
on all laboratory parameters of interest available. Some laboratory parameters, which proved

to be of prognostic relevance in other studies, were not available for at least 2/3 of patients in

our sample. Therefore, we cannot rule out that variables with a higher proportion of missing

data would have had a significant impact on mortality prediction. Additionally, we were unable

to assess the predictive ability of some scores, as some required variables were not available.


        In the Spanish validation cohort, true mortality risk was somewhat underestimated in

very high-risk patients. The majority of those patients came from the beginning of the

pandemic, and at that time doctors used to admit patients earlier in the course of the disease.

Therefore, patients were admitted in a less severe condition than Brazilian ones, what can be

inferred from the difference in SF ratio. This is probably the explanation for the

underestimation in very high-risk patients among the Spanish cohort. Even with this

difference, the score was able to show very good discrimination to differentiate patients with

high vs. non-high risk, what is the main goal in using such scores.




Implications for clinicians and policymakers

        ABC2-SPH score may be very useful in a real-world setting, to provide healthcare

practitioners the decision support that is needed to help them better identify and prioritize the

care of patients who have the higher risk of death. Its development and validation followed

strict methodological criteria, and the score fulfils the characteristics of an ideal score 58. It can

be used in all emergency departments, regardless of the level of resource settings. The results

represent the experience of 36 hospitals in 17 cities in Brazil, and one hospital in Spain, and

they are highly relevant to the current pandemic. It can be easily calculated at bedside or could

be easily integrated to the electronic medical records for an automatic computation. It may

help clinicians to identify high-risk patients from the triage phase, as well as to identify those

most appropriate to be enrolled into therapeutic trials, may make possible to expand inclusion

                                                                                                   30
criteria through the early identification of patients who may benefit from therapy 59. It might

also be useful to help guiding recommendations for early palliative consultation60,61.


        Different from what has been mistakenly suggested25,62-64, the results from this study

do not suggest that patients from low-risk group may be discharged for home treatment. No

score so far has specifically tested this hypothesis. A recent editorial has highlighted the

importance of taking into account the “treatment paradox”: patients identified to be at the

low risk group were at low risk due to the interventions received in hospital 65. It must not be

interpreted as the risk to a patient if no actions are taken. Sperrin & McMillan counterfactual

prediction modelling as a potential solution to minimize bias from treatment paradox65. More

importantly, due to the treatment paradox, scoring systems developed and validated in in-

hospital settings cannot be used in outpatient settings without further validation, as it has

been mistakenly suggested66.




Unanswered questions and future research

        We believe that ABC2-SPH score may hold potential generalizability for other countries.

However, prediction models are population specific and may produce different results in

different populations67. Considering that thresholds for admission may vary, hospitalized

COVID-19 population may be different, the outcome events are different and patient

management may be different, further validation (and re-calibration) in different health care

settings is recommended. In particular, we learned that our model might underestimate

mortality in high-risk individuals.


        As we opted to develop the score focusing on information available at admission, as

this would make it more useful for clinicians, other important factors during hospitalization




                                                                                             31
that may impact prognosis were not included. Further analysis involving these factors are

required.


        ABC2-SPH score may help clinicians to make a prompt and reasonable decision to

optimize patient management and potentially reduce mortality. However, further prospective

studies are needed to investigate whether the use of the score in the emergency department

indeed trigger actions that result in reduced complications and hospital mortality. Additionally,

due to the rapidly changing nature of the COVID-19 and the disease management, model

performance should be monitored closely over time and space65.


        Future studies may also investigate risk factors for mortality among patients who

develop COVID-19 symptoms during hospital admission due to other conditions.




Conclusion


        In conclusion, we developed and validated the ABC2-SPH rapid scoring system and a

web-based risk calculator. This score, based on age, number of comorbidities, blood urea

nitrogen, C-reactive protein, platelet count, peripheral oxygen saturation and oxygen support

at admission is an inexpensive tool, showed to objectively and accurately predict in-hospital

mortality in COVID-19 patients. It may be used at bedside for earlier identification of in-

hospital mortality risk and, thus, inform clinical decisions and the assignment to the

appropriate level of care and treatment for COVID-19 patients.




Declarations


Ethics approval and consent to participate




                                                                                              32
        The study protocol was approved by the Brazilian National Commission for Research

Ethics (CAAE 30350820.5.1001.0008). Individual informed consent was waived due to the

severity of the situation and the use of deidentified data, based on medical chart review only.

For the independent external validation cohort, it was approved by the and Vall d’Hebron

University Hospital Research Ethics Committee (PR(AG)183/2020).



Conflicts of interest


        The author(s) declared no potential conflicts of interest with respect to the research,

authorship, and/or publication of this article.




Data availability statement


Data are available upon reasonable request.




Data Access, Responsibility, and Analysis


The lead authors (MSM and MCP) had full access to all the data in the study and take

responsibility for the integrity of the data and the accuracy of the data analysis.




Transparency declaration


The lead authors (MSM and MCP) affirm that the manuscript is an honest, accurate, and

transparent account of the study being reported; that no important aspects of the study have

been omitted; and that any discrepancies from the study as originally planned (and, if relevant,

registered) have been explained.



                                                                                             33
Funding


        This study was supported in part by Minas Gerais State Agency for Research and

Development (Fundação de Amparo à Pesquisa do Estado de Minas Gerais - FAPEMIG) [grant

number APQ-00208-20], National Institute of Science and Technology for Health Technology

Assessment (Instituto de Avaliação de Tecnologias em Saúde – IATS)/ National Council for

Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e

Tecnológico - CNPq) [grant number 465518/2014-1], and CAPES Foundation (Coordenação de

Aperfeiçoamento de Pessoal de Nível Superior) [grant number 88887.507149/2020-00]. AS was

supported by a Postdoctoral grant “Juan Rodés” (JE18/00022) from Instituto de Salud Carlos III

through the Ministry of Economy and Competitiveness, Spain.




Role of the funder/sponsor


        The sponsors had no role in study design; data collection, management, analysis, and

interpretation; writing the manuscript; and decision to submit it for publication. MSM and MP

had full access to all the data in the study and had responsibility for the decision to submit for

publication.




Author contribution


Substantial contributions to the conception or design of the work: MMS, LSM, KPMPM, IJBN,

CAP, LMO.


Substantial contributions to the acquisition, analysis, or interpretation of data for the work:

MMS, LSM, KPMPM, MVRSS, IJBN, TLSS, RLRC, MCP, RTS, LMO, AVS, AOM, ALBAS, APW, ABG,

                                                                                               34
AACM, ARB, BLF, BMC, CTCAS, CMR, CDG, CCRC, CAC, DVS, DP, DHV, EC, ECP, EMSK, FBL, FA,

FAB, FGA, FB, GPC, GGV, GANB, GCCM, GFN, HCN, HD, HRV, HCG, IMG, JDLB, JCA, JMC, JDSSG,

JDPM, JMR, KBR, KCJRP, KAMS, LSO, LSP, LSP, LDS, LSFC, LK, LCC, LCSM, LEAS, MASC, MAF,

MDS, MGTT, MC, MAPF, MCPBL, MFG, MMAC, MCAN, MPF, MHGJ, NCSS, NLR, NTC, NRO, PKZ,

PLA, PJLM, RSCA, RCM, RL, RAV, RGF, RBC, RP, RXC, RMM, RLSM, RMA, RFS, SCF, SMMG, SFA,

SAP, TFO, TK, TCO, TSMAA, THOD, VBS, VMRG, VALV, YCR, DTMOF, AS, BR.


Drafted the work: MMS, MCP, LMO.


Revised the manuscript critically for important intellectual content: all authors.


Final approval of the version to be published: all authors.


Agreement to be accountable for all aspects of the work in ensuring that questions related to

the accuracy or integrity of any part of the work are appropriately investigated and resolved:

MSM and MCP.




Acknowlegments


        We would like to thank the hospitals which are part of this collaboration, for

supporting this project: Hospital Bruno Born; Hospital Cristo Redentor; Hospital das Clínicas da

Faculdade de Medicina de Botucatu; Hospital das Clínicas da UFMG; Hospital das Clínicas da

Universidade Federal de Pernambuco; Hospital de Clínicas de Porto Alegre; Hospital Santo

Antônio; Hospital Eduardo de Menezes; Hospital João XXIII; Hospital Julia Kubitschek; Hospital

Mãe de Deus; Hospital Márcio Cunha; Hospital Mater Dei Betim-Contagem; Hospital Mater Dei

Contorno; Hospital Mater Dei Santo Agostinho; Hospital Metropolitano Dr. Célio de Castro;

Hospital Metropolitano Odilon Behrens; Hospital Moinhos de Vento; Hospital Nossa Senhora

da Conceição; Hospital Regional Antônio Dias; Hospital Regional de Barbacena Dr. José

Américo; Hospital Regional do Oeste; Hospital Risoleta Tolentino Neves; Hospital Santa Cruz;

                                                                                             35
Hospital Santa Rosália; Hospital São João de Deus; Hospital São Lucas da PUCRS; Hospital

Semper; Hospital SOS Cárdio; Hospital Tacchini; Hospital Unimed-BH; Hospital Universitário

Canoas; Hospital Universitário Ciências Médicas; Hospital Universitário de Santa Maria.


        We also thank all the clinical staff at those hospitals, who cared for the patients, and all

undergraduate students who helped with data collection.




Conflicting of interest statement

        The author(s) declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.




                                                                                                  36
References




1.      WHO. Coronavirus disease (COVID-19) Weekly Epidemiological Update and Weekly

Operational Update. 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-

2019/situation-reports/ (accessed 4 June 2021).


2.      Emanuel EJ, Persad G, Upshur R, et al. Fair Allocation of Scarce Medical Resources in

the Time of Covid-19. New England Journal of Medicine 2020; 382(21): 2049-55.


3.      Conti P, Caraffa A, Gallenga C, et al. The British variant of the new coronavirus-19

(Sars-Cov-2) should not create a vaccine problem. Journal of Biological Regulators Homeostatic

Agents 2121; 35(1).


4.      Zhang S. A Troubling New Pattern Among the Coronavirus Variants. 2021.

https://www.theatlantic.com/health/archive/2021/01/coronavirus-evolving-same-mutations-

around-world/617721/ (accessed 22 May 2021).


5.      Faria NR, Clar IM, Candido D, et al. Genomic characterisation of an emergent SARS-

CoV-2 lineage in Manaus: preliminary findings. 2021. https://virological.org/t/genomic-

characterisation-of-an-emergent-sars-cov-2-lineage-in-manaus-preliminary-findings/586

(accessed 4 June 2021).


6.      Rubin R. COVID-19 Vaccines vs Variants—Determining How Much Immunity Is Enough.

JAMA 2021; 325(13): 1241-3.


7.      Zhang S, Guo M, Duan L, et al. Development and validation of a risk factor-based

system to predict short-term survival in adult hospitalized patients with COVID-19: a

multicenter, retrospective, cohort study. Critical Care 2020; 24(1): 1-13.




                                                                                                37
8.      Leeuwenberg AM, Schuit E. Prediction models for COVID-19 clinical decision making.

The Lancet Digital Health 2020; 2(10): e496-e7.


9.      Fumagalli C, Rozzini R, Vannini M, et al. Clinical risk score to predict in-hospital

mortality in COVID-19 patients: a retrospective cohort study. BMJ Open 2020; 10(9): e040729.


10.     Gupta RK, Marks M, Samuels THA, et al. Systematic evaluation and external validation

of 22 prognostic models among hospitalised adults with COVID-19: An observational cohort

study. European Respitatory Journal 2020; 56(6).


11.     Wynants L, Calster BV, Collins GS, et al. Prediction models for diagnosis and prognosis

of covid-19: systematic review and critical appraisal. British Medical Journal Publishing Group

2020; 369: m1328.


12.     Wang J, Zhang H, Qiao R, et al. Thrombo-inflammatory features predicting mortality in

patients with COVID-19: The FAD-85 score. Journal of International Medical Research 2020;

48(9): 1-14.


13.     Allenbach Y, Saadoun D, Maalouf G, et al. Development of a multivariate prediction

model of intensive care unit transfer or death: A French prospective cohort study of

hospitalized COVID-19 patients. PloS One 2020; 15(10): e0240711.


14.     Kim I-C, Song JE, Lee HJ, et al. The Implication of Cardiac Injury Score on In-hospital

Mortality of Coronavirus Disease 2019. Journal of Korean Medical Science 2020; 35(39): e349.


15.     Zhou J, Huang L, Chen J, et al. Clinical features predicting mortality risk in older

patients with COVID-19. Current Medical Research and Opinion 2020; 36(11): 1753-9.


16.     Goel S, Jain T, Hooda A, et al. Clinical Characteristics and In-Hospital Mortality for

COVID-19 Across The Globe. Cardiology and Therapy 2020; 9(2): 553-9.




                                                                                                  38
17.     Marcolino MS, Ziegelmann PK, Souza-Silva MV, et al. Clinical characteristics and

outcomes of patients hospitalized with COVID-19 in Brazil: results from the Brazilian COVID-19

Registry. International Journal of Infectious Diseases 2021.


18.     Moons KG, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable

prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.

Ann Intern Med 2015; 162(1): W1-73.


19.     Wolff RF, Moons KGM, Riley RD, et al. PROBAST: A Tool to Assess the Risk of Bias and

Applicability of Prediction Model Studies. Annals of Internal Medicine 2019; 170(1): 51-8.


20.     Organization WH. Diagnostic testing for SARS-CoV-2: interim guidance, 11 September

2020: World Health Organization, 2020.


21.     Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data

capture (REDCap)—A metadata-driven methodology and workflow process for providing

translational research informatics support. Journal of Biomedical Informatics 2009; 42(2): 377-

81.


22.     Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: Building an international

community of software platform partners. Journal of Biomedical Informatics 2019; 95: 103208.


23.     Gregory KE, Radovinsky L. Research strategies that result in optimal data collection

from the patient medical record. Appl Nurs Res 2012; 25(2): 108-16.


24.     Rubin DB. Multiple imputation for nonresponse in surveys: John Wiley & Sons; 2004.


25.     Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with

covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation

of the 4C Mortality Score. BMJ 2020; 370: m3339.




                                                                                                    39
26.     Montalva AS, Nadal JS, Pereiro JE, et al. Early outcomes of tocilizumab in adults

hospitalized with severe COVID19. An initial report from the Vall dHebron COVID19

prospective cohort study. MedRxiv 2020.


27.     Rufibach K. Use of Brier score to assess binary predictions. Journal of Clinical

Epidemiology 2010; 63(8): 938-9.


28.     Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia

severity on presentation to hospital: an international derivation and validation study. Thorax

2003; 58(5): 377-82.


29.     Liu JL, Xu F, Zhou H, et al. Expanded CURB-65: a new score system predicts severity of

community-acquired pneumonia with superior efficiency. Scientific Reports 2016; 6: 22911.


30.     Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with

community-acquired pneumonia. N Engl J Med 1997; 336(4): 243-50.


31.     Cag Y, Karabay O, Sipahi OR, et al. Development and validation of a modified quick

SOFA scale for risk assessment in sepsis syndrome. PloS One 2018; 13(9): e0204608.


32.     Olsson T, Terént A, Lind L. Rapid Emergency Medicine Score: a new prognostic tool for

in‐hospital mortality in nonsurgical emergency department patients. Journal of Internal

Medicine 2004; 255(5): 579-87.


33.     Harrison SL, Fazio-Eynullayeva E, Lane DA, Underhill P, Lip GYH. Comorbidities

associated with mortality in 31,461 adults with COVID-19 in the United States: A federated

electronic medical record analysis. PLoS Med 2020; 17(9): e1003321.


34.     Harrison SL, Fazio‐Eynullayeva E, Lane DA, Underhill P, Lip GY. Atrial fibrillation and the

risk of 30‐day incident thromboembolic events, and mortality in adults≥ 50 years with COVID‐

19. Journal of Arrhythmia 2020; (00): 1–7.


                                                                                                   40
35.     Tuty Kuswardhani RA, Henrina J, Pranata R, Anthonius Lim M, Lawrensia S, Suastika K.

Charlson comorbidity index and a composite of poor outcomes in COVID-19 patients: A

systematic review and meta-analysis. Diabetes & Metabolic Syndrome: Clinical Research &

Reviews 2020; 14(6): 2103-9.


36.     Xie J, Hungerford D, Chen H, et al. Development and external validation of a prognostic

multivariable model on admission for hospitalized patients with COVID-19. MedRxiv 2020.


37.     Zhang H, Shi T, Wu X, et al. Risk prediction for poor outcome and death in hospital in-

patients with COVID-19: derivation in Wuhan, China and external validation in London, UK.

Preprints with The Lancet 2020.


38.     Liang W, Liang H, Ou L, et al. Development and validation of a clinical risk score to

predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Internal

Medicine 2020; 180(8): 1081-9.


39.     Fuentes E, Fuentes M, Alarcon M, Palomo I. Immune system dysfunction in the elderly.

Anais da Academia Brasileira de Ciências 2017; 89(1): 285-99.


40.     Sherwani S, Khan MWA. Cytokine Response in SARS-CoV-2 Infection in the Elderly.

Journal of Inflammation Research 2020; 13: 737-47.


41.     Hajifathalian K, Sharaiha RZ, Kumar S, et al. Development and external validation of a

prediction risk model for short-term mortality among hospitalized U.S. COVID-19 patients: A

proposal for the COVID-AID risk tool. PloS One 2020; 15(9): e0239536.


42.     Rice TW, Wheeler AP, Bernard GR, et al. Comparison of the SpO2/FIO2 ratio and the

PaO2/FIO2 ratio in patients with acute lung injury or ARDS. Chest 2007; 132(2): 410-7.




                                                                                                41
43.     Gungor B, Atici A, Baycan OF, et al. Elevated D-dimer levels on admission are

associated with severity and increased risk of mortality in COVID-19: A systematic review and

meta-analysis. The American Journal of Emergency Medicine 2021; 39: 173-9.


44.     Nascimento IJBd, Groote TCv, Mathúna DPO, et al. Clinical, laboratory and radiological

characteristics and outcomes of novel coronavirus (SARS-CoV-2) infection in humans: A

systematic review and series of meta-analyses. PloS One 2020; 15(9): e0239235.


45.     Nicholson CJ, Wooster L, Sigurslid HH, et al. Estimating Risk of Mechanical Ventilation

and Mortality Among Adult COVID-19 patients Admitted to Mass General Brigham: The VICE

and DICE Scores. Medrxiv 2020; 17: 1-33.


46.     Weng Z, Chen Q, Li S, et al. ANDC: an early warning score to predict mortality risk for

patients with Coronavirus Disease 2019. Journal of Translational Medicine 2020; 18(328): 1-10.


47.     Qu R, Hu L, Ling Y, et al. C-reactive protein concentration as a risk predictor of

mortality in intensive care unit: a multicenter, prospective, observational study. BMC

Anesthesiology 2020; 20(292): 1-9.


48.     Koozi H, Lengquist M, Frigyesi A. C-reactive protein as a prognostic factor in intensive

care admissions for sepsis: A Swedish multicenter study. Journal of Critical Care 2020; 56: 73-9.


49.     Bashash D, Hosseini-Baharanchi FS, Rezaie-Tavirani M, et al. The Prognostic Value of

Thrombocytopenia in COVID-19 Patients; a Systematic Review and Meta-Analysis. Archives of

Academic Emergency Medicine 2020; 8(1): e75.


50.     Shang Y, Liu T, Wei Y, et al. Scoring systems for predicting mortality for severe patients

with COVID-19. EClinicalMedicine 2020; 24: 100426.


51.     Hu C, Liu Z, Jiang Y, et al. Early prediction of mortality risk among severe COVID-19

patients using machine learning. MedRxiv 2020.


                                                                                                  42
52.     Favaloro EJ, Thachil J. Reporting of D-dimer data in COVID-19: some confusion and

potential for misinformation. Clinical Chemistry and Laboratory Medicine 2020; 58(8): 1191–9.


53.     Ko H, Chung H, Kang WS, et al. An Artificial Intelligence Model to Predict the Mortality

of COVID-19 Patients at Hospital Admission Time Using Routine Blood Samples: Development

and Validation of an Ensemble Model. Journal of Medical Internet Research 2020; 22(12):

e25442.


54.     Levy TJ, Richardson S, Coppa K, et al. Development and validation of a survival

calculator for hospitalized patients with COVID-19. MedRxiv 2020.


55.     Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel

coronavirus in Wuhan, China. The Lancet 2020; 395(10223): 497-506.


56.     Lemos JA, McGuire DK, Drazner MH. B-type natriuretic peptide in cardiovascular

disease. The Lancet 2003; 362(9380): 316-22.


57.     Yao XH, Li TY, He ZC, et al. [A pathological report of three COVID-19 cases by minimal

invasive autopsies]. Zhonghua Bing Li Xue Za Zhi 2020; 49(5): 411-7.


58.     Oprita B, Aignatoaie B, Gabor-Postole D. Scores and scales used in emergency

medicine. Practicability in toxicology. Journal of Medicine and Life 2014; 7(Spec Iss 3): 4-7.


59.     Haimovich AD, Ravindra NG, Stoytchev S, et al. Development and validation of the

quick COVID-19 severity index: a prognostic tool for early clinical decompensation. Annals of

Emergency Medicine 2020; 76(4): 442-53.


60.     Altschul DJ, Unda SR, Benton J, et al. A novel severity score to predict inpatient

mortality in COVID-19 patients. Scientific Reports 2020; 10(1): 16726, 1-8.


61.     Sheehan J, Ho KS, Poon J, Sarosky K, Fung JY. Palliative care in critically ill COVID-19

patients: the early New York City experience. BMJ Supportive & Palliative Care 2020; 0: 1-5.

                                                                                                   43
62.     Gavelli F, Castello LM, Bellan M, et al. Clinical stability and in-hospital mortality

prediction in COVID-19 patients presenting to the Emergency Department. Minerva Medica

2020: 1-18.


63.     Halalau A, Imam Z, Karabon P, et al. External validation of a clinical risk score to predict

hospital admission and in-hospital mortality in COVID-19 patients. Annals of Medicine 2021;

53(1): 78-86.


64.     Galloway JB, Norton S, Barker RD, et al. A clinical risk score to identify patients with

COVID-19 at high risk of critical care admission or death: an observational cohort study.

Journal of Infection 2020; 81(2): 282-8.


65.     Sperrin M, McMillan B. Prediction models for covid-19 outcomes. 2020; 371: m3777.


66.     Zhang C, Qin L, Li K, et al. A Novel Scoring System for Prediction of Disease Severity in

COVID-19. Front Cell Infect Microbiol 2020; 10: 318.


67.     KGM M, DG A, JB R, et al. Transparent Reporting of a multivariable prediction model

for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Annals of internal

medicine 2015; 162(1): W1-W73.


68.     Garibaldi BT, Fiksel J, Muschelli J, et al. Patient Trajectories Among Persons

Hospitalized for COVID-19: A Cohort Study. Annals of Internal Medicine 2021; 174(1): 33-41.


69.     Sourij H, Aziz F, Bräuer A, et al. COVID‐19 fatality prediction in people with diabetes

and prediabetes using a simple score upon hospital admission. Diabetes, Obesity and

Metabolism 2020; 23: 589–98.


70.     Kazemi MA, Ghanaati H, Moradi B, et al. Prognostic factors of chest CT findings for ICU

admission and mortality in patients with COVID-19 pneumonia. MedRxiv 2020.




                                                                                                   44
71.     Nunez-Gil IJ, Fernandez-Perez C, Estrada V, et al. Mortality risk assessment in Spain and

Italy, insights of the HOPE COVID-19 registry. Intern Emerg Med 2020.


72.     Gomez NFP, Lobo IM, Cremades IG, et al. [Potential biomarkers predictors of mortality

in COVID-19 patients in the Emergency Department]. Rev Esp Quimioter 2020; 33(4): 267-73.


73.     Bello-Chavolla OY, Bahena-López JP, Antonio-Villa NE, et al. Predicting mortality due to

SARS-CoV-2: A mechanistic score relating obesity and diabetes to COVID-19 outcomes in

Mexico. The Journal of Clinical Endocrinology & Metabolism 2020; 105(8): 2752-61.


74.     Yoo E, Percha B, Tomlinson M, et al. Development and calibration of a simple mortality

risk score for hospitalized COVID-19 adults. MedRxiv 2020.


75.     Yadaw AS, Li YC, Bose S, Iyengar R, Bunyavanich S, Pandey G. Clinical features of

COVID-19 mortality: development and validation of a clinical prediction model. The Lancet

Digit Health 2020; 2(10): e516-e25.


76.     Faisal M, Mohammed MA, Richardson D, Fiori M, Beatson K. Development and

validation of automated computer aided-risk score for predicting the risk of in-hospital

mortality using first electronically recorded blood test results and vital signs for COVID-19

hospital admissions: a retrospective development and validation study. MedRxiv 2020.


77.     Mei Q, Wang AY, Bryant A, et al. Development and validation of prognostic model for

predicting mortality of COVID-19 patients in Wuhan, China. Sci Rep 2020; 10(1): 22451.


78.     Lu J, Hu S, Fan R, et al. ACP risk grade: a simple mortality index for patients with

confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19)

during the early stage of outbreak in Wuhan, China. Preprints with The Lancet 2020.




                                                                                                45
79.      Soto-Mota A, Marfil-Garza BA, Rodriguez EM, et al. The low-harm score for predicting

mortality in patients diagnosed with COVID-19: A multicentric validation study. Am Coll Emerg

Physicians Open 2020; 1(6): 1436-43.


80.      Yan L, Zhang H-T, Goncalves J, et al. An interpretable mortality prediction model for

COVID-19 patients. Nat Mach Intell 2020; 2: 283-8.


81.      Williams RD, Markus AF, Yang C, et al. Seek COVER: Development and validation of a

personalized risk calculator for COVID-19 outcomes in an international network. MedRxiv

2020.


82.      Gue YX, Tennyson M, Gao J, Ren S, Kanji R, Gorog DA. Development of a novel risk

score to predict mortality in patients admitted to hospital with COVID-19. Sci Rep 2020; 10(1):

21379.


83.      Das AK, Mishra S, Gopalan SS. Predicting CoVID-19 community mortality risk using

machine learning and development of an online prognostic tool. PeerJ 2020; 8: e10083.


84.      Chen R, Liang W, Jiang M, et al. Risk Factors of Fatal Outcome in Hospitalized Subjects

With Coronavirus Disease 2019 From a Nationwide Analysis in China. Chest 2020; 158(1): 97-

105.


85.      Sarkar J, Chakrabarti P. A machine learning model reveals older age and delayed

hospitalization as predictors of mortality in patients with COVID-19. MedRxiv 2020.




                                                                                                 46
Table 1. Demographic and clinical characteristics for derivation and validation cohorts of patients admitted to hospital with COVID-19

                                             Derivation cohort (n=3,978)       Brazilian validation cohort (n=1,054)           Spanish validation cohort (n=856)


Characteristics                     Frequency (%) or                           Frequency (%) or                              Frequency (%) or       Non missing cases (%)
                                                            Non missing                                Non missing

                                                               cases (%)                                  cases (%)
                                        median (IQR)                               median (IQR)                                  median (IQR)


In hospital mortality                    806 (20.3%)       3,978 (100%)             208 (19.7%)       1,054 (100%)                172 (20.1%)                 856 (100%)


Age (years)                          60.0 (48.0, 72.0)     3,978 (100%)         62.0 (48.2, 73.0)     1,054 (100%)            62.0 (49.0, 74.0)               856 (100%)


Sex at birth                                              3,976 (99.9%)                               1,054 (100%)                                            856 (100%)


 Male                                  2,138 (53.8%)                                582 (55.2%)                                   498 (58.2%)


Comorbities


Hypertension                           2,147 (54.0%)       3,978 (100%)             563 (53.4%)       1,054 (100%)                377 (44.0%)                 856 (100%)


Coronary artery disease                   215 (5.4%)       3,978 (100%)                60 (5.7%)      1,054 (100%)                   65 (7.6%)                856 (100%)


Heart failure                             269 (6.8%)       3,978 (100%)                58 (5.5%)      1,054 (100%)                   46 (5.4%)                856 (100%)


Atrial fibrillation or flutter            139 (3.5%)       3,978 (100%)                27 (2.6%)      1,054 (100%)                  89 (10.4%)                856 (100%)


Stroke                                    146 (3.7%)       3,978 (100%)                43 (4.1%)      1,054 (100%)                   42 (4.9%)                856 (100%)




                                                                                                                                                                     47
COPD                        253 (6.4%)    3,978 (100%)     60 (5.7%)   1,054 (100%)     48 (5.6%)   856 (100%)


Diabetes mellitus         1,151 (28.9%)   3,978 (100%)   297 (28.2%)   1,054 (100%)   151 (17.6%)   856 (100%)


Obesity (BMI>30kg/m2)      696 (17.5%)    3,978 (100%)   181 (17.2%)   1,054 (100%)   202 (23.6%)   856 (100%)


Cirrhosis                     25 (0.6%)   3,978 (100%)      9 (0.9%)   1,054 (100%)      5 (0.6%)   856 (100%)


Cancer                      194 (4.9%)    3,978 (100%)     65 (6.2%)   1,054 (100%)     35 (4.1%)   856 (100%)


Number of comorbidities                   3,978 (100%)                 1,054 (100%)                 856 (100%)


 0                        1,189 (29.9%)                  309 (29.3%)                  325 (38.0%)


 1                        1,173 (29.5%)                  328 (31.1%)                  222 (25.9%)


 2                        1,013 (25.5%)                  269 (25.5%)                  167 (19.5%)


 3                         429 (10.8%)                   106 (10.1%)                   91 (10.6%)


 4                          131 (3.3%)                     33 (3.1%)                    34 (4.0%)


 5                           43 (1.1%)                     9 (0.9%)                    17 (2.0%)


Clinical assessment at

admission




                                                                                                          48
SF ratio                   428.6 (332.1, 452.4)   3,845 (96.7%)   433.3 (339.3, 452.4)   1,034 (98.1%)   457.1 (423.8, 466.5)   842 (98.4%)


Respiratory rate (irpm)             20 (18, 24)   3,236 (81.3%)            20 (18, 24)    870 (82.5%)             20 (18, 26)   452 (92.3%)


Heart rate (bpm)                   88 (78, 100)   3,787 (95.2%)           88 (77, 100)   1,020 (96.8%)           93 (80, 105)   842 (98.4%)


Glasgow coma score                  15 (15, 15)   3,695 (92.9%)            15 (15, 15)    982 (93.2%)             15 (15, 15)   838 (97.9%)


Systolic blood pressure                           3,762 (94.6%)                          1,014 (96.2%)                          843 (98.5%)


  90 (mm Hg)                    3,076 (81.8%)                            825 (81.4%)                            829 (98.3%)


 < 90 (mm Hg)                      510 (13.6%)                            146 (14.4%)                             14 (1.7%)


 Inotrope requirement               176 (4.7%)                              43 (4.2%)                             0


Diastolic blood pressure                          3,776 (94.9%)                          1,022 (97.0%)                          842 (98.4%)


 > 60 (mm Hg)                    3,541 (93.8%)                            962 (94.1%)                            712 (84.6%)


   60 (mm Hg)                       59 (1.6%)                              17 (1.7%)                            130 (15.4%)


 Inotrope requirement               176 (4.7%)                              43 (4.2%)                             0


Laboratory parameters


Hemoglobin (g/L)              13.3 (12.1, 14.4)   3,871 (97.3%)      13.3 (11.9, 14.5)   1,021 (96.9%)      13.4 (12.2, 14.6)   851 (99.4%)



                                                                                                                                       49
Platelet count (109/L)               196.0 (154.0, 257.0)        3,824 (96.1%)       203.0 (154.0, 260.2)       1,016 (96.4%)           188.0 (149.0, 243.0)                       851 (99.4%)


NLR                                          4.7 (2.8, 7.8)      3,759 (94.5%)               4.9 (3.0, 8.4)       989 (93.8%)                              NA                              NA


Lactate (mmol/L)                             1.4 (1.1, 1.9)      2,742 (68.9%)               1.5 (1.2, 2.1)       720 (68.3%)                              NA                              NA


C-reactive protein (mg/L)               77.0 (38.0, 143.0)       3,487 (87.7%)          74.1 (33.8, 143.0)        881 (83.6%)              98.4 (41.8, 186.3)                      581 (67.9%)


BUN (mg/dL)                              16.3 (11.5, 24.3)       3,636 (91.4%)           17.3 (12.9, 25.2)        942 (89.4%)               16.8 (12.1, 25.2)                      653 (76.3%)


Creatinine (mg/dL)                           0.9 (0.8, 1.2)      3,765 (94.6%)               1.0 (0.8, 1.3)       967 (91.7%)                    0.8 (0.7, 1.1)                    847 (98.9%)


Sodium (mmol/L)                      137.0 (135.0, 140.0)        3,550 (89.2%)       137.0 (134.3, 140.0)         930 (88.2%)           136.1 (134.0, 138.0)                       846 (98.8%)


Bicarbonate (mEq/L)                      23.0 (21.0, 25.0)       3,222 (81.0%)           23.0 (20.6, 25.0)        807 (76.6%)                              NA                              NA


pH                                           7.4 (7.4, 7.5)      3,232 (81.2%)               7.4 (7.4, 7.5)       808 (76.7%)                              NA                              NA


pO2 (mmHg)                               75.0 (63.0, 96.0)       3,183 (80.0%)           73.4 (63.0, 94.6)        800 (75.9%)                              NA                              NA


pCO2 (mmHg)                              35.0 (31.3, 39.0)       3,194 (80.3%)           34.0 (30.0, 38.0)        801 (76.0%)                              NA                              NA


BMI: body mass index; BUN: blood urea nitrogen; COPD: chronic obstructive pulmonary disease; NA: not available; NLR: neutrophils-to-lymphocytes ratio; SF ratio: SpO2/FiO2 ratio




                                                                                                                                                                                          50
Table 2. ABC2-SPH Score for in-hospital mortality in patients with COVID-19

                   Variable                        ABC2-SPH score

       A           Age (years)


                     < 60                                 0


                     60 - 69                              1


                     70 - 79                              3


                      80                                 5


       B           Blood urea nitrogen

                   (mg/dL)*


                     < 42                                 0


                      42                                 3


       C2          Comorbidities


                     0–1                                  0


                     2                                   1


                   C reactive protein (mg/L)


                     < 100                                0


                      100                                1


       S           SF ratio (%)


                     > 315                                0


                     >235 – 315                           1




                                                                              51
                                          >150 – 235                            3


                                           150                                 6


                      P               Platelet count (x109/L)


                                          > 150                                 0


                                          100 -150                              1


                                          < 100                                 2


                      H               Heart rate (bpm)


                                           90                                  0


                                          91 – 130                              1


                                           131                                 2


 * When converted to urea, the cut-off is 90 mg/dL.



             Table 3. Predicted mortality and mortality rates for ABC2-SPH Score risk groups

                                                       Derivation cohort                   Validation cohort
                              Predicted
     Risk Group
                                                  No of                              No of
                              mortality
                                                             No of deaths (%)                   No of deaths (%)
                                                 patients                           patients


Low (0-1)              < 6%                  1133           23 (2.0%)               290        1 (0.3%)


Intermediate (2-4)     6 - 14.9%             1470           168 (11.4%)             394        47 (11.9%)


High (5-8)             15 - 49.9%            907            290 (32.0%)             252        73 (29.0%)


Very high (9)                           468            325 (69.4%)             118        87 (73.7%)


Overall                   -                  3978           806 (20.3%)             1054       208 (19.7%)




                                                                                                               52
Table 4. Discrimination and model overall performance in derivation and validation cohorts

                             Derivation cohort                    Brazilian validation Cohort
   Model
                  AUROC (95%CI)            Brier Score     AUROC (95%CI)              Brier Score


   GAM            0.884 (0.879; 0.888)     0.101           0.871 (0.862; 0.879)       0.102


   LASSO          0.844 (0.842; 0.846)     0.115           0.859 (0.855; 0.862)       0.110


   ABC2-SPH       0.842 (0.840; 0.843)     0.114           0.857 (0.854; 0.860)       0.108


GAM: generalized additive models; LASSO: least absolute shrinkage and selection operator logistic

regression




                                                                                                    53
Table 5. Main characteristics of the studies




                                               54
                                                                                                                  Validation
                                                                                                 Develoment
                                                      Country of      Country of     Sample                       sample (n)
  Study          Study design    Patient time span                                              sample (n) (for                    Develoment population           Validation population
                                                      derivation      validation     size (n)                        (for
                                                                                                  mortality)
                                                                                                                  mortality)

                                                                                                                                                                   Confirmed SARS-CoV-2
                                                                                                                                                                   patients who required
                                                                                                                                                                   hospital admission at 8
                 Retrospective   March 1, 2020 to    United States   United States
 Halalau63                                                                            2025        Not clear         1290                  Not clear                hospitals in Beamount,
                    cohort        April 1, 2020       of America      of America
                                                                                                                                                                   excluding patients who
                                                                                                                                                                   remained hospitalized
                                                                                                                                                                    beyond May 12, 2020

                                                                                                                               Consecutive adult patients with
                 Retrospective February 22, 2020
Fumagalli9                                               Italy           Italy        516            516             NA        COVID-19 from 2 Italian tertiary
                    cohort      to April 10, 2020
                                                                                                                                         hospitals

                                                                                                                                                                      The same as the
                                                       England,        England,                                                Consecutive adult patients with
                  Prospective    May 21, 2020 to                                                                                                                  development population,
 Knight25                                            Scotland, and   Scotland, and   57824          35463           22361       COVID-19 from 260 hospitals,
                    cohort        June, 29 2020                                                                                                                    admitted after May 20,
                                                        Wales           Wales                                                   admitted up to May 20, 2020
                                                                                                                                                                           2020

                                  November 21,                                                                                  Patients with COVID-19 from       Data from hospitals not
                 Retrospective
  Liang38                        2019 to January        China           China         2300          1590             710        575 hospitals in 31 provincial        included in the
                    cohort
                                    31, 2020                                                                                       administrative regions          development cohort

                                                                                                                               Consecutive adult patients with
            45   Retrospective    First patient to   United States   United States                                             laboratory-confirmed COVID-19
Nicholson                                                                             1042           578             464
                    cohort         May 19, 2020       of America      of America                                                 patients from Mass General
                                                                                                                                      Brigham hospitals




                                                                                                                                                                              55
                                March 4, 2020 to
                                 April 24, 2020,                                                                   Consecutive confirmed COVID-
                Retrospective                       United States     United States
Garibaldi68                      with follow-up                                       832        832         NA     19 patients from 5 hospitals
                   cohort                            of America        of America
                                through June 27,                                                                     (John Hopkins Medicine)
                                      2020

                 Prospective
                                                                                                                   Adult patients with confirmed
                     and        April 15, 2020 to
 Sourij69                                              Austria             NA         238        238         NA    COVID-19 and diabetes or pre-                 NA
                retrospective    June 30, 2020
                                                                                                                              diabetes
                   cohort

                                                                                            Apparently, it                                                Adult patients with
                Retrospective
           62                 March 16, 2020 to                                             was developed                                                confirmed COVID-19
 Gavelli        single-center                           Italy             Italy       480                    480                 NA
                               April 22, 2020                                                 by expert                                                patients admitted to one
                   cohort
                                                                                             consensus                                                    university hospital

                                                                                                                    Adult patients with confirmed
                                                                                                                   COVID-19 who had undergone
                                                                                                                      CT scan <8 days from the
                                                                                                                       beginning of symptoms,
           70   Retrospective February 25, 2020                                                                    excluding the ones with RT-PCR
 Kazemi                                                  Iran              NA         91         91          NA                                                  NA
                   cohort      to April 25, 2020                                                                    more than 7 days from CT. CT
                                                                                                                    score developed not based on
                                                                                                                     the data. Authors tested CT
                                                                                                                   score and clinical variables in a
                                                                                                                                model

                                                                                                                   Patients with confirmed COVID-
                Retrospective   February 8, 2020
Núñez-Gil71                                         Spain and Italy        NA         908        908         NA     19 from centers in Italy (n=88)
                   cohort        to April 1, 2020
                                                                                                                          and Spain (n=820)




                                                                                                                                                                  56
                   Prospective                                                                              Adult patients with confirmed
                                  March 16, 2020 to
 Allenbach13      single-center                          France          France       152    152    131    COVID-19 from one tertiary care         Not described
                                    April 4, 2020
                     cohort                                                                                       university hospital

                  Retrospective                                                                             Adult patients with confirmed
                                February 19, 2020
    Kim14         single-center                          Korea             NA         38      38    NA     COVID-19 admitted to a tertiary              NA
                                to March 15, 2020
                     cohort                                                                                       university hospital

                                                                                                                                                 The same as the
                  Retrospective                                                                                                              development population
                                  March 1, 2020 to    United States   United States                        Patients with confirmed COVID-
  Altschul60      single-center                                                       4711   2355   2356                                        (spitted 50/50%,
                                   April 16, 2020      of America      of America                           19 from an academic hospital
                     cohort                                                                                                                  apparently by admission
                                                                                                                                                      date)

                                                                                                                                                 Adult patients with
                                                                                                                                                confirmed COVID-19
                                                                                                            Adult patients with confirmed
                                                                                                                                               patients presenting to
                                                                                                           COVID-19 patients presenting to
                                                                                                                                              emergency department
                  Retrospective   March 4, 2020 to    United States   United States                          emergency department of 2
Hajifathalian41                                                                       929    664    265                                           of 9 hospitals in
                     cohort        April 9, 2020       of America      of America                          hospitals in Manhattan (did not
                                                                                                                                              Massachusetts (did not
                                                                                                             exclude patients who were
                                                                                                                                               exclude patients who
                                                                                                             discharged within 24 hours)
                                                                                                                                              were discharged within
                                                                                                                                                      24 hours)

                                                                                                                                                  The same as the
                                                                                                                                             development population
                  Retrospective                                                                             Adult patients with confirmed
                                  January 28, 2020                                                                                              (the criteria used to
   Wang12         single-center                          China           China        243    199    44      COVID-19 from one university
                                  to March 4, 2020                                                                                           divide patients in training
                     cohort                                                                                            hospital
                                                                                                                                              and testing sets was not
                                                                                                                                                        clear)




                                                                                                                                                          57
                                                                                            Elderly patients (>60 years) with
             Retrospective   January 12, 2020
      15                                                                                    "clinically diagnosed" COVID-19
 Zhou        single-center    to February 26,     China     NA       118     118     NA                                                    NA
                                                                                             (RT-PCR or chest CT) from one
                cohort             2020
                                                                                                   university hospital

             Retrospective                                                                   Adult patients with suspected
                           February 24, 2020
 Goméz72     single-center                        Spain     NA       163     163     NA       COVID-19 admitted to one                     NA
                           to March 16, 2020
                cohort                                                                             university hospital

             Retrospective March 24, 2020 to                                                Patients with confirmed COVID-
Galloway64                                       England    NA      1157    1157     NA                                                    NA
                cohort      April 17, 2020                                                   19 from 2 academic hospitals

             Registry data
             from na open
                                                                                            Patients with confirmed COVID-       The same as the
                 source
                                                                                               19 from the open source      development population
  Bello-       database    First patient up to
                                                 Mexico    Mexico   51633   41307   10326     Mexican Ministry of Health   (split by random sampling
Chavolla73      from the      May 18, 2020
                                                                                               database (inpatients and      stratified by mortality
                Mexican
                                                                                                      outpatients)                    status)
               Ministry of
                 Health

                                                                                                                                     The same as the
                                                                                                                                development population
                                                                                            Adult patients with laboratory-
             Retrospective January 1, 2020 to                                                                                      (the criteria used to
 Weng46                                           China    China     301     176    125      confirmed COVID-19 from 2
                cohort     February 15, 2020                                                                                    divide patients in training
                                                                                                       hospitals
                                                                                                                                 and testing sets was not
                                                                                                                                           clear)

             Retrospective   Development                                                      Patients with COVID-19 (not       Patients with COVID-19
   Ko46                                           China    China     467     361    106
                cohort     cohort: January 10,                                               clear if laboratory-confirmed)     (not clear if (laboratory-
                            2020 to February                                                from one hospital, excluding 14        confirmed) from 3



                                                                                                                                             58
                                  24, 2020;                                                                 patients without a blood test             hospitals
                              Validation cohort:                                                            within 1 day after the hospital
                               February to July                                                                       admission
                                    2020

                                                                                                                                              Patients with confirmed
                                                                                                           Patients with confirmed COVID-
                                                                                                                                               COVID-19 from another
              Retrospective     January and                                                                19 from one hospital in Wuhan
 Xie36                                                China           China        444     299     145                                           hospital in Wuhan,
                 cohort        February 2020                                                                who had been discharged or
                                                                                                                                              excluding 6 patients who
                                                                                                                         died
                                                                                                                                                    died quickly

                                                                                                            Adult patients with confirmed
                                                                                                                                                 The same as the
                                                                                                           COVID-19 from 5 hospitals, up to
                                                                                                                                             development population:
              Retrospective   March 1, 2020 to     United States   United States                            99 years-old. The sample was
 Yoo74                                                                             4.840   1.613   1.614                                        andomly split in 3
                 cohort        April 28, 2020       of America      of America                             randomly split in 3 datasets, the
                                                                                                                                              datasets, the third one
                                                                                                              second one was used for
                                                                                                                                              was used for validation
                                                                                                                    development

                                                                                                                                                 Adult patients with
              Retrospective                                           United                                Adult patients with confirmed
Zhang37                         Not reported          China                        1001    775     226                                        confirmed COVID-19 from
                 cohort                                              Kingdom                                 COVID-19 from one hospital
                                                                                                                                                   another hospital

                                                                                                                                                   The same as the
                                                                                                              Inpatients and outpatients
                                                                                                                                              development population
                                                                                                            (including those attended by
              Retrospective                                                                                                                    (randomly split 80/20%)
                                                                                                             telehealth) with confirmed
         75        and         March 9, 2020       United States   United States                                                              and patients admitted to
Yadaw                                                                              5051    3841    961     COVID-19 from the Mont Sinai
               prospective     toApril 7, 2020      of America      of America                                                                Mont Sinai Hospitals who
                                                                                                           Health System (8 hospitals and
                 cohort                                                                                                                          were included in the
                                                                                                           over 400 ambulatory practices)
                                                                                                                                                  database (with the
                                                                                                                  until April 6, 2020
                                                                                                                                              outcome) on April 7, 2020




                                                                                                                                                          59
                                                                                                     Consecutive patients with        The same definition as
         50   Retrospective January 1, 2020 to                                                      confirmed COVID-19 from 2           the development
Shang                                              China     China    452    113        339
                 Cohort       March 27,2020                                                         hospitals in Wuhan, who had      population, but from a
                                                                                                      severe or critical illness     third hospital in Wuhan

                                                                                                                                     Consecutive adult non-
                                                                                                   Consecutive adult non-elective    elective or emergency
                                                                                                       or emergency medical           medical admissions
                                                                                                   admissions (COVID-19 and non-      (COVID-19 and non-
         76                   March 11, 2020 to    United    United                                 COVID-19 patients) from one     COVID-19 patients) from
Faisal        Registry data                                           6444   3924       2520
                               June 13, 2020      Kingdom   Kingdom                                hospital, who were discharged     another hospital, who
                                                                                                    over a course of three months   were discharged over a
                                                                                                     and had electronic NEWS2       course of three months
                                                                                                              recorded                 and had electronic
                                                                                                                                        NEWS2 recorded

                                                                                                    Adult patients with confirmed
                                                                                                      COVID-19, diagnosed with
                                                                                    Validation 1 = pneumonia by CT scan, from one
                              January 21, 2020                                                                                         The same as the
              Retrospective                                                            120 and     hospital in Wuhan. Patients who
Mei77                          to February 27,     China     China    492    237                                                   development population,
                 cohort                                                             validation 2 = died within the first 24 hours,
                                    2020                                                                                            from other 3 hospitals
                                                                                         135          with not clinical outcome
                                                                                                     available or who refused to
                                                                                                      participate were excluded

                                                                                                                                        Adult patients with
                              January 12, 2020
         7    Retrospective                                                                        Adult patients with confirmed    confirmed COVID-19 from
Zhang                           to February 9,     China     China    828    516        312
                 cohort                                                                             COVID-19 from one hospital         the same hospital in a
                                     2020
                                                                                                                                        different time span
                                                                                                                                     (February 8-9, 2020) and




                                                                                                                                                60
                                                                                                                                                              from another hospital



                Retrospective   January 21, 2020                                                                           Patients with confirmed or
   Lu78         single-center     to February 5,        China            NA          577           577           NA      suspected COVID-19 from one                   NA
                   cohort              2020                                                                                         hospital

                                                                                                                                                            Consecutive patients with
                                                                                                                                                            confirmed COVID-19 from
                Retrospective   April 30, 2020 to                                             Score developed
Soto-Mota79                                             Mexico           NA          400                         400                   NA                       12 hospitals, with
                   Cohort        May 20, 2020                                                   by consensus
                                                                                                                                                                 complete clinical
                                                                                                                                                            information and outcome

                                 Development                                                                              Adult patients with COVID-19
                              cohort: January 10,                                                                            (not clear if patients had
                                                                                                                                                                The same as the
                               2020 to February                                                                          laboratory-confirmed disease),
      80        Retrospective                                                                                                                               development population,
   Yan                             18, 2020;            China           China        485           375           110      from one hospital, excluding
                   cohort                                                                                                                                   admitted after February
                               Validation cohort:                                                                          patients with >20% missing
                                                                                                                                                                   18, 2020
                                February 19-24,                                                                             values and breast-feeding
                                     2020                                                                                             women

                                                                                                                            Healthcare database of 6             Adult patients with
                                                     United States                                                                                          confirmed with COVID-19,
                                  Development                                                                               countries, in which adult
                                                      of America,                                                                                              presenting at an initial
                                 cohort: any time                    South Korea,                                        patients with GP, EP or OP visit
                                                     South Korea,
           81   Retrospective     prior to 2020;                     Spain, United 2.126.78                                 with influenza or flu-like           healthcare provider
Williams                                                 Spain,                                  2,082,277      44.507
                   cohort       validation cohort:                     States of       4                                 symptoms, at least 365 days of      interaction in a GP, ER or
                                                       Australia,
                                January 1st 2020                       America                                             prior observation, and no         OP visit, and who had no
                                                         Japan,
                                 to April 20, 2020                                                                       symptoms in the preceding 60       diagnosis of influenzae or
                                                     Netherlands
                                                                                                                                      days                    pneumonia and no flu-
                                                                                                                                                                like symptoms in the




                                                                                                                                                                         61
                                                                                                                                             preceding 60 days


                                                                                                         Consecutive patients with
            Retrospective
      82                  March 10, 2020 to       United                                                confirmed COVID-19 from a
Gue         single-center                                           NA         316     316     NA                                                    NA
                           May 30, 2020          Kingdom                                                 general hospital, who had
               cohort
                                                                                                      clinical symptoms at admission


                                                                                                       Data shared by Korea Centers
                                                                                                          for Disease Control and
                                                                                                      Prevention, from 17 provinces.
            Retrospective   January 20, 2020
Das83                                          South Korea     South Korea     3.524   3.524   NA     Patients with confirmed COVID-                 NA
               cohort       to May 30, 2020.
                                                                                                           19, with availability of
                                                                                                        demographic, exposure and
                                                                                                      diagnosis confirmation features
                                                                                                          along with the outcome

                                                                                                        Adult patients with confirmed
                                                                                                        COVID-19 from 11 acute care
                                                                                                         hospitals in New York, from
                                                                                                         March 1, 2020 to April 23, 7          The same as the

            Retrospective                                                                             2020. Patients were excluded if development cohort from
       54        and        March 1, 2020 to   United States   United States                           they were still in the hospital at another hospital in New
Levy                                                                           8391    6162    2229                                        York from March 1, 2020
             prospective     May 12, 2020       of America      of America                               the study end point with a
               cohort                                                                                 length of stay less than 7 days; if to May, 7 2020, and all 12
                                                                                                                                            hospitals from April 24,
                                                                                                         they were transferred to a
                                                                                                        hospital outside of the health       2020 to May 6, 2020.
                                                                                                      system and their outcomes were
                                                                                                       unknown; or if they expired but
                                                                                                       were not marked as discharged



                                                                                                                                                      62
                                                                                                        in the EH

                                                                                             Patients with confirmed COVID-
                                                                                                  19 from 575 hospitals
              Retrospective The first patient to
Chen84                                               China        China   1590   1590   NA    throughout China, excluding                 NA
                 cohort      January 31, 2020
                                                                                             cases with incomplete medical
                                                                                                     records (20.8%)

                                               22 countries in
                            13th January, 2020                                                 Open source databased of
         85                                    Asia, Australia,
Sarkar        Registry data to 28th February,                      NA     115    115    NA    COVID-19 patients (inclusion                NA
                                                Europe and
                            2020                                                                 criteria is not clear)
                                               North America

                                                                                             Patients with severe confirmed
                                                                                             COVID-19 infection admitted to
                                                                                             one hospital in Wuhan. patients
                                                                                              who had >10% missing values,
                                                                                              stayed in the hospital <7 days,       The same as the
                              28 January 2020
              Retrospective                                                                      were afflicted by a severe     development population,
 Hu51                          and 11 March          China        China   247    183    64
                 cohort                                                                       disease before admission (e.g.      admitted at another
                                   2020
                                                                                                cancer, aplastic anaemia or             hospital
                                                                                             uraemia), were unconscious at
                                                                                                admission or were directly
                                                                                              admitted to the intensive care
                                                                                                 unit (ICU) were excluded




                                                                                                                                           63
Table 5. Continued

                                                                                                                                     Number of
                                                                                                                                                                    How many
                                                  Original                                                                         variables were
                                  Outcome                                                     Use of AI        Was a score                          Univariate    patients died in
   Study       Model outcome                     modelling             Imputation                                                   tested in the
                                    time                                                     techniques        produced?                             analysis    the development
                                                 approach                                                                           development
                                                                                                                                                                     dataset?
                                                                                                                                       cohort

                   Hospital                     Multivariate
  Halalau63    admission and in- In-hospital      logistic                 No                   No                 Yes               Not clear         No             Not clear
               hospital mortality               regression

                                               Cox regression
 Fumagalli9          Mortality   In-hospital                               No                   No                 Yes                  20             Yes              120
                                                  analysis

                                               LASSO logistic   Yes. Multiple imputation                    Yes (4C mortality
  Knight25           Mortality   In-hospital                                                  Yes. ML                                   21             No              11426
                                                 regression      with chained equations                           score)

                                                                Yes (if <20%). Predictive
               Composite of ICU                                 mean matching to impute
               admission, need                                  numeric features, logistic
                  of invasive               LASSO logistic        regression to impute
   Liang38                      In-hospital                                                     No         Yes (COVID-GRAM)             72             No             51 (3.2%)
                  mechanical                  regression          binary variables, and
                 ventilation or                                   Bayesian polytomous
                     death                                        regression to impute
                                                                     factor features

 Nicholson45       Need of       In-hospital    Multivariate               No                   No         Yes: one to predict          49             Yes         Not reported
                  mechanical                     logistic                                                 ventilation need (VICE




                                                                                                                                                                 64
               ventilation and in-                 regression                                         score) and another
               hospital mortality                                                                     one for death (DICE
                                                                                                            score)

                   In-hospital
                                                                                      Yes. NLP was
                mortality and a                                Yes. Imputed missing
                                                                                        used to         Yes: COVID-19
                 composite of                 Cox regression     values by chained
Garibaldi68                       In-hospital                                           identify        Inpatient Risk             24         No         131
                disease severity                 analysis      equations (MICE) with
                                                                                       presenting      Calculator (CIRC)
               (WHO scale) or in-                            predictive mean matching
                                                                                       symptoms
               hospital mortality

                                                   Multivariate
          69
 Sourij            Mortality         In-hospital     logistic         No                  No                  Yes              Not clear      Yes        58
                                                   regression

                                                  Multivariable
                                                     logistic
                 In-hospital
                                                 regression and
               mortality and in-
 Gavelli62                           In-hospital       Cox            No                  No          Yes (NOVARA score)           NA         No    NA (consensus)
               hospital clinical
                                                   Regression
                   stability
                                                     Hazard
                                                     models

                                                   Multivariate                                      Yes (authors created a
 Kazemi70          Mortality         In-hospital     logistic         No                  No         CT score not based on    Not available   No         11
                                                   regression                                               the data)

                                                   Multivariate
Núñez-Gil71        Mortality         In-hospital     logistic         No                  No                  Yes              Not clear      Yes        311
                                                   regression




                                                                                                                                                    65
                  Composite of ICU                 Multivariate
 Allenbach13        admission or      14 days        logistic                No               No        Yes             42       Yes         32
                       death                       regression

    Kim14            Mortality       In-hospital    Consensus                No               No        Yes             3        No          7

                                                   Multivariate
  Altschul60         Mortality       In-hospital     logistic                No               No        Yes          Not clear   Yes        621
                                                   regression

                                                   Multivariable
                                     7 days and                  Yes. Imputation by chained
Hajifathalian41      Mortality                        logistic                                No   Yes (COVID-AID)      38       Yes         93
                                      14 days                            equations
                                                    regression

                                                   Multivariable
   Wang12            Mortality        28 days         logistic               No               No    Yes (FAD-85)        41       No          24
                                                    regression

                                                   Multivariable
    Zhou15           Mortality       In-hospital      logistic               No               No    Yes (NLAUD)         37       No          51
                                                    regression

                                                   Multivariable
  Goméz72            Mortality        30 days         logistic               No               No    Yes (COVEB)         20       No          33
                                                    regression

                    Composite of
                                                 LASSO logistic
 Galloway64       transfer to ICU or In-hospital                             No               No        Yes             19       No         244
                                                   regression
                        death

                     Mortality        30 days          Cox                   No               No        Yes             12       No         4276
    Bello-
                                                   proportional



                                                                                                                                       66
Chavolla73                                    risk regression
                                                  analysis

                                                             Yes, for variables with
         46                                 LASSO logistic <10% missing values (>10%
 Weng             Mortality     In-hospital                                                No          Yes (ANDC)         24   No            21
                                              regression   were excluded from model
                                                               development). RF.

                                                Machine         Yes, imputed with mean   Yes, DLN
  Ko46            Mortality     In-hospital     learning        values for development    and RF       Yes (EDRnet)       73   Yes    212 (58.7%)
                                               techniques         and trainig datasets    model

                                               Multivariate
        36
  Xie             Mortality     In-hospital      logistic                No                No              Yes            28   No           155
                                               regression

                                                Gray`s K-
  Yoo74           Mortality     In-hospital   sample tests,              No                No              Yes            48   Yes   Not reported
                                              DeLong's test

                Death and poor
                   outcome
              (developing ARDS,
                   receiving                LASSO logistic
 Zhang37                        In-hospital                              No                No       Yes (DCS, DCSL, DL)   19   No         33 (4.3%)
                 intubation ou                regression
              ECMO treatment,
               ICU admission or
                     death)




                                                                                                                                     67
                                                                                       Yes.
                                                                                    Recursive
                                                                                     feature
                                                                                   elimination
                                                                                   method for
                                                                                     feature
                                         Artificial                                 selection,
                                                                                                    Yes (17F and 3F
Yadaw75     Mortality   In-hospital    inteligence         Yes, using means        and logistic                           17       No    313 (8.15%)
                                                                                                        models)
                                       techniques                                  regression,
                                                                                     SVM, RF
                                                                                   model, and
                                                                                     XGBoost
                                                                                   algorithms
                                                                                       for
                                                                                   prediction

                                                      Yes, multiple imputation
                                      LASSO logistic
Shang50     Mortality   In-hospital                  methods for variables with        No           Yes (CSS score)       52       No        49
                                        regression
                                                        <10% missing values

                                      Multivariable
                                                                                                  Yes (CARMc19_N and
Faisal76    Mortality   In-hospital      logistic                 No                   No                              Not clear   No        323
                                                                                                     CARMc19_NB)
                                       regression

       77                             LASSO logistic
 Mei        Mortality   In-hospital                               No                   No                Yes              43       No        105
                                        regression

                         14 days
                                      Cox regression   Yes. Multiple imputations
Zhang7      Mortality    and 28                                                        No                Yes              30       No        96
                                        analyses        (method not reported)
                          days




                                                                                                                                        68
                                              Cox regression
   Lu78          Mortality        12 days                      No       No                 Yes             Not clear   Yes           39
                                                 analysis

Soto-Mota79      Mortality      In-hospital    Consensus       No       No           Yes (LOW-HARM)           NA       No         200 (50%)

                                                                       Yes,
                                                Machine             XGBoost
   Yan80         Mortality      In-hospital     learning       No   machine                Yes                75       No           174
                                               techniques            learning
                                                                    algorithm

                Hospitalization
               with pneumonia,
                hospitalization
               with pneumonia In-hospital
                                                                      Yes, ML
                   requiring       and 30   LASSO logistic                         Yes, 3 scores (COVER-
Williams 81                                                    No   (train-test-                            31.917     No          11407
              intensive services days after   regression                                F for death)
                                                                       split)
                 or death and    index rate
               death in the 30
               days after index
                     date

                                              Multivariable
                                                                                     Yes (COVID-19
  Gue82          Mortality        30 days        logistic      No       No                                    15       No           145
                                                                                     Mortality Socre)
                                               regression

                                                                    Yes. SVM, K
                                                Logistic
      83
                                                                      nearest
   Das           Mortality      In-hospital regression and     No                    Yes (CoCoMoRP)           4        No            74
                                                                     neighbor,
                                               machine
                                                                     RFM and
                                               learning
                                                                     gradient



                                                                                                                             69
                                      techniques                                    boosting




                                                      Yes, imputation of means.
                                     LASSO logistic      Variables with >50%                        Yes (NOCOS
Levy54     Mortality     7 days                                                        No                           42   No        Not clear
                                       regression        missing values were                         Calculator)
                                                              excluded.

                                   Multivariate
                       14, 21 and
Chen84     Mortality              Cox regression                 No                    No          Yes (nomogram)   37   No           50
                        28 days
                                     analysis

                                       Machine                                       Yes, RF
Sarkar85   Mortality   In-hospital     learning                  No               classification        Yes         6    No           37
                                      techniques                                    algorithm

                                                                                  Yes. Logistic
                                                                                  regression,
                                                                                        PLS
                                                                                  regression,
                                                       Yes, using bagging tree.    EN model,
    51                             LASSO logistic       Variables with >30%         random
 Hu        Mortality   In-hospital                                                                      Yes         51   No           68
                                     regression         missing values were        forest and
                                                               excluded              bagged
                                                                                     flexible
                                                                                  discriminant
                                                                                    analysis
                                                                                      (FDA).




                                                                                                                              70
Table 5. Continued

                                                                              How are         AUC in            AUC in
                      Variables included in the final model      External
     Study                                                                   predictors      derivation        validation                             Limitations
                                  (for mortality)               validation
                                                                             combined?        cohort            cohort

                     Age, male sex, congestive heart failure,
                        end-stage renal disease, chronic
                                                                                                                              Selection bias: Excluded patients who were hospitalized
                     pulmonary disease, DM, hypertension,
                                                                                                                                 beyond May 12, 2020. Data on how the score was
                        obesity, nursing home residence,                     Points-based                     0.75 (0.71 –
   Halalau63                                                       Yes                      Not available                     developed not reported. Absence of an initial validation
                          immunocompromised status,                             score                            0.78)
                                                                                                                              cohort. Uniform scoring weights of different risk factors.
                       congenital heart disease, coronary
                                                                                                                                               Complete case analysis.
                     artery disease, end-stage liver disease
                                  and pregnancy

                        Age, number of comorbidities (CV
                     disease, hypertension, DM, depression,                                                                     Modest sample size. No external validation. Variables
                                                                             Points-based    0.90 (0.87 -
   Fumagalli9        dementia and cancer), respiratory rate,       No                                             NA            were selected by univariate analysis. Complete case
                                                                                score           0.93)
                        PaO2/FiO2, serum creatinine and                                                                                              analysis.
                      platelet count obtained on admission

                       Age, sex, number of comorbidities                                                                          Several potentially relevant comorbidities, such as
                        (chronic cardiac disease, chronic                                                                     hypertension, previous myocardial infarction, and stroke,
                     respiratory disease excluding asthma,                                                                         were not included in data collection. The authors
                        chronic renal disease defined as                                                                       considered that inclusion of these comorbidities might
                                                                             Points-based   0.786 (0.781 -   0.767 (0.760 -
    Knight25          estimated glomerular filtration rate         Yes                                                         have impacted upon or improved the performance and
                        ≤30, mild to severe liver disease,                      score           0.790)           0.773)         generalizability of the 4C Mortality Score. Secondly, a
                         dementia, chronic neurological                                                                        proportion of recruited patients (3.3%) had incomplete
                     conditions, connective tissue disease,                                                                      episodes, so there is a possibility of selection bias, if
                       DM, HIV or AIDS, and malignancy),                                                                        patients with incomplete episodes, such as those with
                         respiratory rate, SpO2, level of                                                                     prolonged hospital admission, had a differential mortality



                                                                                                                                                                         71
                    consciousness, urea and CPR obtained                                                                    risk to those with completed episodes.
                               on admission




                     Chest radiographic abnormality, age,
                    hemoptysis, dyspnea, unconsciousness,                                                               Modest sample size for score development and a
                        number of comorbidities (COPD,                                                              relatively small sample for validation. The data for score
                       hypertension, DM, coronary heart                                                               development and validation are entirely from China,
                    disease, chronic kidney disease, cancer,          Logistic     0.88 (0.85 -    0.88 (0.84 -      which could potentially limit the generalizability of the
  Liang38                                                      Yes
                     cerebrovascular disease, hepatitis B,           Regression       0.91)           0.93)        risk score in other areas of the world. Mortality was quite
                      immunodeficiency), cancer history,                                                            low (3.2%). Apparently, patients with cancer should gain
                    neutrophil-to-lymphocyte ratio, lactate                                                               points for both cancer history and number of
                      dehydrogenase and direct bilirubin                                                                             comorbidities, not clear.
                             obtained on admission

                      Age, sex, diabetes mellitus, chronic                                                         Modest sample sizes in both our derivation and validation
                    statin use, albumin, C-reactive protein,                                                        cohorts. The number of events on the derivation and
                                                                      Logistic     0.87 (0.83 –    0.80 (0.75 –
Nicholson45           neutrophil-lymphocyte ratio, mean        Yes                                                  validation cohort separately was not informed (211 in
                                                                     Regression       0.91)           0.85)
                    corpuscular volume, platelet count, and                                                         total). Variables were selected by univariate analysis.
                     procalcitonin obtained on admission                                                                            Complete case analysis.

                                                                                                                    Modest sample size. No external validation. Too many
                    Age, nursing home residence, sex, BMI,              Cox
               68                                                                                                   variables tested in the model for the number of events
Garibaldi           Charlson Comorbidity Index, SaO2/FiO2      No    regression   Not available    Not available
                                                                                                                      (24/131). To try to overcome that, authors tested
                          ratio obtained on admission                 analysis
                                                                                                                                      variables "in blocks"

          69                                                                      0.889 (0.837 -                     Small sample size and number of events. Number of
 Sourij               Age, arterial occlusive disease, CRP,    No    Nomogram                          NA
                                                                                      0.941)                        variables tested not clear. Complete case analysis, and
                    estimated GFR and aspartate AST levels
                                                                                                                    predictors with >20% missing values were excluded. No



                                                                                                                                                              72
                        obtained on admission                                                                                     external validation

                 Presence of comorbidity (any disease
                                                                                                                Score developed by consensus. Modest sample size.
           62        on active therapy), SpO2 and               Points-based
 Gavelli                                                   No                       NA         Not reported    Number of events is not clear. Single-center study. No
                   respiratory rate after a trial of 15            score
                                                                                                               external validation. AUC and accuracy not presented.
                  minutes with oxygen at a FiO2 0.5

                                                                                                                 Small sample size and number of events. Too many
                                                                                                                    variables tested for the low number of events.
                                                                                                                 Comorbidities were not well defined, percentage of
                 Age, sex, comorbidity (cardiovascular                                                            involvement included in CT score is subjective and
           70   and pulmonary), diffused distribution of         Logistic      0.73 (95% CI                     peripheral involvement is not well defined. Complete
 Kazemi                                                    No                                      NA
                  CT abnormality, total CT-score and            Regression     not reported)                   case analysis. High risk of selection bias: All 3 hospitals
                         dyspnea at admission                                                                    were referral centers for COVID-19 patients, so it is
                                                                                                               possible that the overall CT- score of the patients in this
                                                                                                                   study would not be representative of the general
                                                                                                                                      population

                                                                                                                  No external validation. Variables were selected by
                                                                                                                univariate analysis. Complete case analysis. Variables
                                                                                                                  included in the model not clearly defined. Authors
                                                                                                               reported that some incident events in the participating
                    Age, hypertension, obesity, renal
                                                                                                               centers may not have been diagnosed and/or not been
                 insufficiency, any immunosuppressive           Points-based   0.88 (0.85 –
Núñez-Gil71                                                No                                      NA         reported. The data analysis and modeling focused on only
                   condition, SpO2, CRP obtained on                score          0.91)
                                                                                                                   two countries (Italy and Spain) of the four initially
                                admission
                                                                                                              considered, since as previously mentioned heterogeneity
                                                                                                                 among countries with regard to clinical features and
                                                                                                                death-risk assessment could limit the representative
                                                                                                                                nature of the sampling.




                                                                                                                                                          73
                                                                                                                      Small sample size of both development and validation
                                                                                    0.786 for the   0.787 for the
                                                                      Points-based                                   samples. Too many predictors tested for a small number
                                                                                     composite       composite
                                                                       score (but                                     of events. Complete case analysis. External validation
                                                                                    outcome and     outcome and
                                                                           AUC                                       sample not described. The external sample consisted of
                     Age, WHO clinical scale, CRP and                              0.803 for death 0.827 for death
                                                                       presented                                      patients from a regional non-university hospital, which
 Allenbach13         lymphocytes count obtained on              No                      (after          (after
                                                                      based on the                                     could explain the differences on catchment area and
                              admission                                             correction for correction for
                                                                         logistic                                      patient recruitment. In the acute context of the first
                                                                                   over-optimism; over-optimism;
                                                                       regression                                      SARS-CoV-2 epidemic wave in France, we relied on a
                                                                                      IC95% not       IC95% not
                                                                         model)                                        sample prospectively defined by consecutive eligible
                                                                                      reported)       reported)
                                                                                                                                   patients in the study center.

                   Myocardial damage marker (creatine
                  kinase-MB [CK-MB] or troponin-I > the
                  99th percentile upper reference limit) +                                                             Score developed by consensus. Small sample size and
                  Heart failure marker (NT-proBNP ≥ 125                                                                  small number of events. Accuracy not assessed. The
                  pg/mL) + Electrical abnormality marker                                                                 protocol for the evaluation of cardiac injury was not
                     (first detected or newly developed                                                              controlled. The attending physician decided each category
                                                                      Points-based
    Kim14                supraventricular tachycardia,          No                   Not reported        NA          of the test according to the patient's condition at the time
                                                                         score
                    ventricular tachycardia, ventricular                                                             of the management. When the test was not performed, it
                    fibrillation, atrial fibrillation, bundle                                                           is assumed as a negative result because the physician
                           branch block, ST-segment                                                                  considered it as an unnecessary test or the result might be
                        elevation/depression, T-wave                                                                                           negative.
                   flattening/inversion, and QT interval
                             prolongation on ECG)

                   Age, sex, SpO2, MAP, INR, creatinine,
                                                                      Points-based 0.824 (0.814 to 0.798 (0.789 to    Complete case analyses, variables selected by univariate
  Altschul60        BUN, interleukin-6 (IL-6), CRP and          Yes
                                                                         score         0.851)          0.818)                               analyses
                   procalcitonin obtained on admission

Hajifathalian41     Age, mean arterial pressure, serum          Yes   Multivariate   7 days: 0.877   7 day (0.851    Modest sample size for development and validation, less



                                                                                                                                                                74
                    creatinine and severity of hypoxia at             logistic    (95%CI 0.831–        [0.781 to      than 100 events both in the development and validation
                           hospital presentation.                   regression      0.923); 14      0.921]); 14 day               cohorts, short follow-up time
                                                                                    days: 0.847      (0.825 [0.764
                                                                                  (95%CI 0.806–        to 0.887])
                                                                                      0.888)

                                                                                                    Not available      Single-center study, with small sample for development
                                                                    Logistic       0.871 (based
                                                                                                       (link for           and validation, less than 100 events both in the
           12      Age, ferritin and D-dimer obtained on           regression     on its optimal
    Wang                                                     Yes                                    supplemental         development and validation cohorts. Complete-case
                                  admission                           and         cut-off value =
                                                                                                    material does      analysis. D-dimer assay not described. AUC for external
                                                                   nomogram             85)
                                                                                                      not work)                 validation not available to the readers.

                                                                                                                        Single-center study, with small sample size, including
                   Lactate dehydrogenase, albumin, BUN,
         15                                                                       0.955 (95% CI                       cases not confirmed by RT-PCR, and less than 100 events.
    Zhou               NLR and D-dimer obtained on           No    Nomogram                              NA
                                                                                  not provided)                        Complete-case analysis and tests too many variables for
                                admission
                                                                                                                         the number of events. D-dimer assay not described.

                                                                                                                        Single-center study, with small sample size, including
                   Age, creatin, glucose and white blood                          0.874 (0.816-                       cases not confirmed by RT-PCR, and less than 100 events.
   Goméz72                                                   No     Not clear                            NA
                        cells obtained on admission                                  0.933)                            Complete-case analysis and tests too many variables for
                                                                                                                                        the number of events.

                   Age, sex, ethnicity, DM, hypertension,
                         chronic lung disease, SpO2,
                                                                   Points-based       0.697                            Modest sample size. No external validation. Complete
  Galloway64       radiographic severity score, neutrophil   No                                          NA
                                                                      score       (0.652,0.741)                                    case analysis. AUC < 0.70
                    count, respiratory rate, CRP, albumin,
                      creatinine obtained on admission

                                                                   Points-based   0.823 (95% CI     0.830 (95% CI       The use of data collected from a sentinel surveillance
Bello-Chavolla73     Age, diabetes, obesity, CKD, COPD,      Yes
                                                                      score       not reported)     not reported)      system model, what raises concern about data quality.
                   hypertension, immunosuppression and
                                                                                                                         The same score for inpatient and outpatients and



                                                                                                                                                               75
                  COVID-19 pneumonia                                                                      sensitivity analysis was not performed to assess accuracy
                                                                                                          for patients who were hospitalized. Apparently, complete
                                                                                                                                 case analysis.

         Age, neutrophil-to-lymphocyte ratio, D-           Nomogram                                        Small sample size for development and validation, with
                                                                          0.921 (0.835-   0.975 (0.947-
Weng46   dimer and C-reactive protein obtained       Yes   and logistic                                   <100 events in both cohorts. Variables with >10% missing
                                                                             0.968)           1.0)
                     on admission                          regression                                      values were excluded. D-dimer assay was not reported.

           Lymphocytes, neutrophils, albumin,
              LDH, neutrophil count (?), CRP,
           prothrombin activity, calcium, urea,
           estimated GFR, monocytes, globulin,                                                              Small sample size for development and validation, too
         eosinophils, glucose, RDW, bicarbonate,                                                           many variables tested for the limited number of events,
             RDW standard deviation, platelet                                                             high mortality rate, with possibility of selection bias. Not
 Ko46                                                Yes    AI model      Not reported    Not reported
          count, mean platelet volume, platelet                                                           clear if included laboratory-confirmed COVID-19 patients
         large-cell ratio, prothrombin time, total                                                          only. The number of predictors make it difficult to be
            protein, platelet distribution width,                                                                            applicable at bedside.
               aspartate aminotransferase,
            thrombocytocrit, eosinophil count,
                alkaline phosphatase, INR

                                                                                                          High risk of selection bias: the cohort was conducted early
                                                                                                          in the pandemic, there was a high mortality rate (51.8% in
                                                            Logistic
            Age, lymphocyte count, lactate                                                                 development cohort and 47.6% in the validation cohort),
                                                           regression     0.880 (95% CI   0.980 (0.958-
 Xie36   dehydrogenase and SpO2 obtained on          Yes                                                   and it may not accurately represent patients with mild or
                                                              and         not reported)       1.00)
                     admission                                                                             asymptomatic COVID-19 (as they were not being tested).
                                                           nomogram
                                                                                                            Small sample size for development and validation, less
                                                                                                                 than 100 events both. Complete case analysis.




                                                                                                                                                     76
                                                                                                             The authors reported that documentation of all kinds was
                                                                                                               inconsistent during the first wave of covid-19, and the
                                                                                              At admission    environments at different hospitals varied substantially.
                                                                              Not reported,
                                                                                             0.81; maximum While it is unlikely that a laboratory result or medication
              Glasgow coma scale, oxygen support                                as AUC was
                                                               Points-based                      through      administration was missed, inconsistencies in flowsheet
 Yoo74            level, BUN, age, lymphocyte            Yes                  used to define
                                                                  score                      admission 0.91; documentation during this period could mean that the
                     percentage, troponin                                      the variables
                                                                                              mean through timings of different modes of oxygen administration were
                                                                              for the score.
                                                                                             admission 0.92 not always accurately capture. The statistical test used to
                                                                                                                 produce the score is not adequate according to the
                                                                                                                      TRIPOD and may lead to overoptimism.

                                                                                                                   Authors reported that clinical datasets were collected
              DCS (demographic, comorbidities and                                                                when healthcare services were under severe strain. Data
                 symptoms): age, sex, chronic lung                                                              extraction sought to ensure consistency and accuracy, but
                     disease, DM, hypertension,                                                                    there is missing data in both datasets, and the analysis
             immunosuppression, cancer, CKD, heart                                                                was complete case based. Sample sizes for development
             disease, cough, dyspnea, diarrhea; DCSL                                                                and validation were small, with <100 events. Clinical
                    (demographic, comorbidities,                              DCS: 0.79; DCS:                   assessments at admission such as SpO2 were not available
                                                                                                DL: 0.74 (95%
        37     symptoms and laboratory tests): age,              Logistic      0.89; DL: 0.91                                         in either dataset.
Zhang                                                    Yes                                        CI not
              sex, chronic lung disease, DM, cancer,            regression      (95% CI not                       The external validation dataset has very different case-
                                                                                                 reported)
                  cough, dyspnea, CRP, creatinine,                               reported)                        mix, and only had follow-up to a fixed date (6-39 days).
              platelets, neutrophils and lymphocytes                                                              Although the Wuhan cohort includes many people with
                    counts; DL (demographic and                                                                       less severe disease, in the validation cohort most
                   laboratory tests): age, sex, CRP,                                                                 admitted patients are likely to have severe disease.
               creatinine, platelets, neutrophils and                                                           Although the authors reported all variables were included
             lymphocytes counts (around admission)                                                               in the model, for most of the included ones the 95% CI of
                                                                                                                                     the OR included 1.0

     75                                                                       0.91 (95% CI      0.91 (95% CI
Yadaw        17F: age, sex, ethnicity, encounter type,   Yes     Artificial                                         As it includes inpatients and outpatients, important
              temperature, diastolic blood pressure,           intelligence   not provided)     not provided)      laboratory parameters were not tested. The authors



                                                                                                                                                          77
               oxygen saturation at presentation,            (XGBoost)                                       reported that the clinical features available were limited
              minimum oxygen saturation, smoking,                                                             to those routinely collected during hospital encounters,
                asthma, COPD, obesity, DM, HIV,                                                                and they pointed out that development of even better
                cancer; 3F: age, minimum oxygen                                                              prediction models should be possible using a richer set of
                 saturation, and type of patient                                                                                      features.
                 encounter, obtained the day of
                            admission

                                                                                                             Small sample size in development (113 participants) and
                 Age, coronary heart disease, % of          Points-based   0.919 (95% CI   0.938 (95% CI
Shang50                                               Yes                                                    validation cohorts, with <100 events in the development
               lymphocytes, procalcitonin, D-dimer             score       0.870-0.970)    0.902-0.973)
                                                                                                             one. Too many variables tested for the number of events.

                                                                         CARMc19_NB =
                CARMc19_N: 10 [age, sex, COVID-19
                                                                          0.87 (95% CI CARMc19_NB =            Not exclusively for COVID-19 patients. COVID-19 was
                     (yes/no), NEWS2 score and
         76                                                 Points-based 0.85-0.89) vs    0.88 vs              identified by ICD-10 code which depends on clinical
Faisal        subcomponents] and CARMc19_NB: 18.      Yes
                                                               score      CARMc19_N    CARMc19_N =            judgment. Risk of selection bias, as only patients with
                 All variables from CARMc19_N + 7
                                                                          0.86 (95% CI     0.86              NEWS2 recorded were included. Complete case analysis.
                    blood test results + AKI score
                                                                           0.84-0.87)

                                                                                             VC1 = 0.928
                                                                                           (95% CI 0.884-     Risk of selection bias due to inclusion/exclusion criteria,
              Age, NLR, admission body temperature,         Points-based   0.912 (95% CI
Mei77                                                 Yes                                  0.971) and VC2    included only patients from Wuhan. Small sample size for
                         AST, total protein                    score       0.878-0.947)
                                                                                           = 0.883 (0.815-     development and validation. Complete case analysis.
                                                                                                0.952)

                                                                                            0.879 (95% CI,    Small sample size for evelopment and validation, <100
                                                                                            0.856–0.900)       events for both cohorst. The amount of missing data
                Age, LDH, NLR and direct bilirubin                         0.886 (95% CI
Zhang7                                                Yes   Nomogram                       and 0.839 (95%    differed between the survivor and non-survivor groups.
                     obtained on admission                                 0.873–0.899)
                                                                                              CI [0.798–       The study included a high population of patients who
                                                                                           0.880) for each   were severely ill, the authors pointed out there may be a
                                                                                              one of the         selection bias when identifying the risk factors of



                                                                                                                                                        78
                                                                                                hospitals                              mortality




                                                                  Cox                                          Included both patients with confirmed and not confirmed
     78                                                        regression                                       disease, small sample size with <100 events, number of
   Lu                        Age, CPR                   No                    Not reported         NA
                                                                analysis,                                        potential predictors tested was not clear. No external
                                                              decision tree                                                            validation.

                                                                                               Provided by
                                                                                              different cut-
                                                                                              offs, ranging
                                                                                              from 0.61 to
                Age, hypertension, white blood cell
                                                                                                0.90 (95%      Score developed by consensus. Not clear the moment it is
          79   count, lymphocyte count, myocardial              Logistic
Soto-Mota                                               No                         NA          ranges from     meant to be used. Risk of selection bias, high mortality in
               necrosis marker, creatinine, SpO2 (not          regression
                                                                                              0.59 to 0.93),                      the cohort (50%)
                      clear in which moment)
                                                                                              with best AUC
                                                                                              for 25 points
                                                                                              (0.90 [95% CI
                                                                                                0.87-0.93])

                                                               Multi-tree                                       Single-center study, with small sample for development
               LDH, lymphocytes and CRP obtained at                           0.978 (IC 95%   0.951 (CI 95%
  Yan80                                                 Yes    XGBoost                                            and validation, less than 100 events in the validation
                        hospital admission                                    not provided)   not provided)
                                                                model                                                 cohort. Apparently, complete-case analysis.




                                                                                                                                                         79
                                                                                                                  The auhtors reported they were unable to develop a
                                                                                                     CUIMC        model on COVID-19 patient data due the scarcity of
                                                                                                database 0.820   databases that contains this information in sufficient
                                                                                                (95% CI 0.796-   numbers. Based on seconday data, with possibility of
                                                                                                  0.840); HIRA   misclassifications of predictors (diseases is incorrectly
               Age, sex, history of cancer, COPD,                                               database 0.898   recorded in a patient’s history, incorrect recording of
                                                                 Points-based   0.896 (95% CI
Williams81   diabetes, heart disease, hypertension,        Yes                                  (95% CI 0.857-     influenza or COVID-19, and authors were unable to
                                                                    score        0.72 - 0.90)
              hyperlipidemia and kidney disease.                                                0.940); SIDIAP    include some suspected diseases predictors such as
                                                                                                 0.895 (95% CI BMI/obesity in the analysis due to the inconsistency with
                                                                                                 0.881-0.910); which these mesures are collected and reporte across the
                                                                                                    VA 0.717   databases included in the study. Patients may day after 30
                                                                                                 (0.642-0.791) days, and this will be recorded as a non-event. Apparently,
                                                                                                                                  complete case analysis.

                                                                                                                    Small sample size from a single center, no external
             Age, sex, hypertension, coronary artery
                                                                                                                 validation. Complete case analysis. Authors pointed out
             disease, heart failure, atrial fibrilation,
                                                                 Points-based   0.793 (95% CI                    that patients at the highest risk may be deemed too sick
  Gue82       oral anticoagulants, modified sepsis-        No                                        NA
                                                                    score       0.745–0.841)                         for maximal intervention and may be denied ICU
               induced coagulopathy (mSIC) score
                                                                                                                  treatment; predictors and their assigned weights in the
                (INR, platelet count, qSOFA score)
                                                                                                                                        final model.

             Age, sex, province (in South Korea) and
               exposure (nursing home, hospital,
                                                                                                                Risk of selecion bias (only patients wit complete data were
                religious gathering, call center,                   Logistic
     83                                                                         0.830 (95% CI                    included), unavailability of crucial clinical information on
  Das           community center, shelter and              No     regression                         NA
                                                                                not reported)                    symptoms, risk factors and clinical parameters. Less than
               apartment, gym facility, overseas                   (SMOTE)
                                                                                                                             100 events. No external validation
               inflow, contact with patients and
                             others)

 Levy54       Age, length of stay, SpO2, neutrophil,       Yes     Logistic     0.86 (95% CI     0.82 (95% CI     Data were imputed for variables with up to 50% missing



                                                                                                                                                            80
                   RDW, sodium urea (on admission and                       regression      not reported)   not reported)    values. Follow up was too short (7 days), what causes a
                             every 2 days)                                                                                   high risk of bias, as a significant proportion of patients
                                                                                                                               may die after 7 days. Authors did not show how to
                                                                                                                                                 calculate the score.

                       Age, coronary heart disease,
                                                                                                                            High risk of selectionbias (20.8% patients with incomplete
                    cerebrovascular disease, dyspnea,
          84                                                                                0.91 (95% CI,                      data were excluded), modest sample size, with <100
     Chen                procalcitonin, aspartate               No          Nomogram                             NA
                                                                                             0.85-0.97)                      events. No external validation. Complete case analysis.
                   aminotransferase, total bilirubin upon
                                                                                                                                Authors did not show how to calculate the score.
                                admission

                                                                                                                            Small sample size, with <100 events. High risk of selection
                   Age, sex, from Wuhan, visit to Wuhan,                         RF                                         bias: from 1085 patients, 652 (60.1%) were excluded due
                                                                                            0.97 (95% CI
    Sarkar85            days from symptom onset to              No         classification                        NA         to missing values, and the model was developed using one
                                                                                            not reported)
                                hospitalization                              algorithm                                      115 patients(10.6%). Data quality is questionable, as the
                                                                                                                            study is based in open source database.

                                                                                                                               Small sample size of both development and validation
                                                                                                                              samples, with <100 events. Too many predictors tested
                                                                                                                            for a small number of events. The authors did not exclude
                                                                                                                                 patients transferred from other hospitals (so the
                                                                                                                              assessment was not the first hospital assessment in all
                  Age, CRP, D-dimer, lymphocyte count at                   Points-based     0.895 (95% CI   0.881 (95% CI
      Hu51                                                      Yes                                                              patients). Single center study, patients from both
                                admission                                     score         not reported)   not reported)
                                                                                                                             derivation and validation sets were from Tongji Hospital,
                                                                                                                             which is one of the hospitals with a high level of medical
                                                                                                                             care in China (the authors reported that some critically ill
                                                                                                                            patients who recovered there might die in other hospitals
                                                                                                                                 with suboptimal or typical levels of medical care).

AUC: area under the curve; BMI: body mass index; CI: confidence interval; CPOD: chronic obstructive pulmonary disease; CPR: C-reactive protein; CT: computed
tomography; DLN: deep learning networks; DM: diabetes mellitus; GFR: glomerular filtration rate; ICU: intensive care unit; LASSO: least absolute shrinkage and selection



                                                                                                                                                                        81
operator logistic regression; NA: not applicable; RDW: red blood cell distribution width; PLS: partial least squares RF: Random Forest; SF ratio: SpO2/FiO2 ratio; SVM:
support-vector machine; XGBoost: eXtreme Gradient Boosting; WHO: World Health Organization.




                                                                                                                                                                    82
Table 6. Discrimination of risk scores within validation cohort (complete case)

Score                   Number of       Number of           AUROC (95%CI)

                        patients        deaths (%)


4C Mortality Score      625             110 (17.6%)         0.790 (0.750-0.830)


A-DROP                  704             148 (21%)           0.780 (0.740-0.820)


ABC2SPH                 779             148 (19%)           0.853 (0.822-0.885)


AID-14                  929             187 (20.1%)         0.752 (0.714-0.790)


AID-7                   929             187 (20.1%)         0.751 (0.713-0.789)


CURB65                  770             165 (21.4%)         0.748 (0.709-0.786)


E-CURB65                146             33 (22.6%)          0.768 (0.682-0.853)


NEWS-FAST               578             112 (19.4%)         0.739 (0.692-0.786)


NEWS2                   425             90 (21.2%)          0.746 (0.687-0.804)


NOVARA                  865             176 (20.3%)         0.656 (0.613-0.699)


qSOFA                   850             172 (20.2%)         0.653 (0.609-0.697)


REMS                    780             145 (18.6%)         0.753 (0.712-0.793)


SOFA                    288             59 (20.5%)          0.778 (0.712-0.843)


Xie                     475             93 (19.6%)          0.816 (0.768-0.863)


Yan                     431             81 (18.8%)          0.650 (0.603-0.697)


Zhang                   279             67 (24%)            0.810 (0.751-0.869)




                                                                                  83
Figure legends



Figure 1. Flowchart of COVID-19 patients included in the study



Figure 2. City of residence of patients within (a) development and (b) validation cohorts



Figure 3. ABC2-SPH Score in derivation and validation cohorts



Figure 4. Discrimination of ABC2-SPH Score in external validation cohorts



Figure 5. ROC curves (a) and decision curve (b) for best performing scores




                                                                                            84
                                           In-hospital patients with
                                             laboratoty confirmed
                                                  COVID-19

                                                   n=6845


                      Patients admitted between             Patients admitted between
                           March-July, 2020                  August-September, 2020

                               n=5287                                  n= 1558

          Excluded                                                                          Excluded

                                                                                    Age < 18 years, transferred
 Age < 18 years, transferred
                                                                                     from another hospital or
  from another hospital or
 admitted for other reasons                                                         admitted for other reasons

                                                                                              n= 488
      Lost n= 1248
           to follow-up                                                                 Lost to follow-up

       Transferred to                                                                     Transferred to
      another hospital                                                                   another hospital

            n=61          Development cohort                    Validation cohort             n=16

                               n=3978                                  N=1054




Figure 1. Flowchart of COVID-19 patients included in the study




                                                                                                              85
                         (a)                                         (b)




Figure 2. City of residence of patients within (a) development and (b) validation cohorts




                                                                                            86
Figure 3. ABC2-SPH Score in derivation and validation cohorts




                                                                87
Figure 4. Discrimination of ABC2-SPH Score in external validation cohorts




                                                                            88
                         (a)                                         (b)




Figure 5. ROC curves (a) and decision curve (b) for best performing scores




                                                                             89