Flaatten et al. Ann. Intensive Care (2021) 11:22 https://doi.org/10.1186/s13613-021-00815-7 RESEARCH Open Access Reliability of the Clinical Frailty Scale in very elderly ICU patients: a prospective European study Hans Flaatten1* , Bertrand Guidet2, Finn H. Andersen3,4, Antonio Artigas5, Maurizio Cecconi6, Ariane Boumendil7, Muhammed Elhadi8, Jesper Fjølner9, Michael Joannidis10, Christian Jung11, Susannah Leaver12, Brian Marsh13, Rui Moreno14, Sandra Oeyen15, Yuriy Nalapko16, Joerg C. Schefold17, Wojciech Szczeklik18, Sten Walther19, Ximena Watson20, Tilemachos Zafeiridis21 and Dylan W. de Lange22 on behalf of the VIP2 Study Group Abstract Purpose: Frailty is a valuable predictor for outcome in elderly ICU patients, and has been suggested to be used in various decision-making processes prior to and during an ICU admission. There are many instruments developed to assess frailty, but few of them can be used in emergency situations. In this setting the clinical frailty scale (CFS) is frequently used. The present study is a sub-study within a larger outcome study of elderly ICU patients in Europe (the VIP-2 study) in order to document the reliability of the CFS. Materials and methods: From the VIP-2 study, 129 ICUs in 20 countries participated in this sub-study. The patients were acute admissions ≥ 80 years of age and frailty was assessed at admission by two independent observers using the CFS. Information was obtained from the patient, if not feasible, from the family/caregivers or from hospital files. The profession of the rater and source of data were recorded along with the score. Interrater variability was calculated using linear weighted kappa analysis. Results: 1923 pairs of assessors were included and background data of patients were similar to the whole cohort (n = 3920). We found a very high inter-rater agreement (weighted kappa 0.86), also in subgroup analyses. The agree- ment when comparing information from family or hospital records was better than using only direct patient informa- tion, and pairs of raters from same profession performed better than from different professions. Conclusions: Overall, we documented a high reliability using CFS in this setting. This frailty score could be used more frequently in elderly ICU patients in order to create a more holistic and realistic impression of the patient´s condition prior to ICU admission. Keywords: Clinical frailty scale, Inter-rater variability, Intensive care, Octogenarians Background Frailty assessment is increasingly used in critically ill elderly patients and has in many studies been shown to correlate with outcomes [1–3]. Frailty assessment *Correspondence: Hans.flaatten@uib.no has recently been suggested as one of several ele- 1 Department of Anaesthesia and Intensive Care, Dep of Clinical Medicine, ments that could theoretically considered for decision Haukeland University Hospital Bergen Norway, University of Bergen, to admit patients to the ICU during the present pan- 5019 Bergen, Norway Full list of author information is available at the end of the article demic [4], although firm evidence for its use is lacking. © The Author(s) 2021. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat​iveco​ mmons​.org/licen​ses/by/4.0/. Flaatten et al. Ann. Intensive Care (2021) 11:22 Page 2 of 7 Traditionally, frailty assessments are performed within except for France and Switzerland using a validated ver- the context of a comprehensive geriatric assessment and sion in French [13]. require active participation from the patient [5]. Under- standably, this is not feasible in most acutely admitted or critical ill patients, and hence other methods have been Clinical Frailty Scale (CFS) developed to overcome this problem. One of the most The CFS was used to assess frailty in all recruited patients frequently used tool for frailty assessments in this set- as it presented prior to the acute event and admission ting is the Clinical Frailty Scale (CFS) [6] developed from to the ICU. The CFS is a pictographic scale from 1–9 the large Canadian studies of frailty that established the describing nine different grades of frailty with a short text cumulative deficit approach to frailty. The CFS has since attached [6]. Patients with scores from 1 to 3 are con- increasingly been used in intensive care as well as in sidered not frail, 4 is pre-frail or vulnerable, and 5 to 9 other emergency settings and was found in a recent sys- are considered to be frail. No specific training, except a tematic review to be the most frequent instrument used written explanation of the use of the CFS, was given to to assess frailty in ICU patients, but is only properly vali- the participating units where many, but not all units had dated in patients ≥ 65 years [7]. prior experience with using it. As in any assessment method, the psychometric prop- erties of the test are important. Regarding the CFS, the original publication [6] established construct validity by Assessment performed by different raters comparing it with the frailty index [8]. Inter-rater reli- In this study, two different study personnel from the ability of the CFS has been tested in a limited number ICU independently and blinded for each other results, of patients in three studies [9–11] and the feasibility was assessed the patient at admission (first 24 h in the ICU) clinically demonstrated in the VIP1 study where CFS was using the CFS with input from patients if possible, if not collected in 99.8% of the 5187 patients included [1]. As a from care-givers or the medical and nursing hospital pre-defined sub-study nested within the VIP2 study [12] notes. The second rater was free to use sources of input we additionally planned to perform a large international and was not constrained to use the same as rater 1. The assessment of CFS reliability. CFS score was noted for assessor 1 and 2 with informa- Interrater variability may vary for several reasons: indi- tion about the profession of the assessor: ICU nurse, ICU vidual differences in how to use the CFS, the rater’s pro- physician, dedicated study person, or other. Furthermore, fession and experience, and the source of information they documented the kind of information that was used necessary to perform a frailty score. Our hypothesis is to perform the score. These data were then recorded in that the CFS, being intuitive to perform, may vary little the electronic case record form (CRF) for the VIP-2 study with the rater’s profession and source of available infor- by the local study investigator. mation to perform the score. The assessors were named Rater 1 and Rater 2. In the The main aim of this study was to document inter-rater analysis of data, the CFS rating was considered as an reliability within a large prospective observation study ordinal variable, and the occupation of the assessors and assess the results of the score being derived from dif- were grouped as ICU nurse; ICU physician; research ferent raters and dissimilar information source, and in staff or other. The main source where the information addition study potential variances between countries. was obtained was classified into 4 groups: (a) from the patient; (b) from family/care-givers; (c) from hospital records; and (d); another source, and they could only Methods choose one option. Study design and setting The observational VIP-2 study was performed in acute ICU admissions of patients ≥ 80 years, and its primary Registration and ethics aim was to describe the influence and interaction of sev- This pre-defined sub-study was registered on Clinical eral geriatric syndromes: frailty, co-morbidity, the activity Trials.gov identifier NCT03370692 at the same time as of daily life, and cognition on many different outcomes. the main study. The main study was approved by ethical The study was performed over 12 months in 2018–2019 committees in all participating countries by institutional and included 3920 patients from 22 countries. More research boards, for details see the VIP-2 study main details and results can be found in the original publica- paper [12]. Since this study involved health profession- tion [12]. Units could voluntarily sign up additionally to als (raters) in some countries, this sub-study had to go participate in a pre-defined sub-study of the inter-rater through an independent review, and the rater then had to variability of CFS. The English version of CFS was used give informed consent to participate. Flaatten et al. Ann. Intensive Care (2021) 11:22 Page 3 of 7 Statistical analyses Table 1 Details of VIP-2 patients studied compared A statistical analysis plan was discussed in the VIP2 study to those not studied group and was decided to adopt to the guidelines for Included Not included reporting of reliability and agreement studies (GRAAS) [14], see Additional file 1. Number 1923 1997 Data were analysed using SPSS version 25.0 (IBM, Age (median IQR) year 84 (81–87) 84 (81–87) Armonk, NY USA) and with MedCalc 19.0 (http://www. Male sex (%) 53.8% 53.3% medca​lc.org Ostend, Belgium). The inter-rater reliabil- ICU LOS (median IQR) days 4 (1.96–8.89) 4 (1.71–7.12) ity was assessed using linear weighted kappa in order to SOFA score (median IQR) 6 (4–9) 6 (4–9) minimise outlier ratings and with intraclass correlation CFS fit (1–3) 789 (41%) 754 (38.1%) coefficient where raters for each subject were selected at CFS vulnerable (CFS 4) 407 (21.1%) 384 (19.4%) random and with a one-way random effects model. We CFS frail (CFS 5–9) 732 (37.8%) 840 (42.5%) first analysed the inter-rater variability using all pairs IQCODE (median, IQR) 3.25 (3–3.75) 3.19 (3–3.56) then compared raters from different professions, infor- Katz ADL score (mean) 4.8 mation sources and participating countries. In the manu- Co-morbidity/polypharmacy 10.6 (10.4–10.9) score (mean and 95% CI script, we further use the accepted grouping of weighted Admission groups (%) kappa: Poor: 0–0.2, Fair (0.21–0.4), Moderate (0.41–0.6), Acute respiratory failure 22.8 25.3 good (0.61–0.8) and very good (0.81–1.0) [14, 15]. Emergency surgery 15.4 12.3 Sepsis 14.5 13.0 Acute circulatory failure 13.8 13.8 Results Respiratory and circulatory 10.7 12.2 20 countries and 129 ICUs contributed to the inter-rater Trauma 6.1 4.6 study that included 1923 pairs of raters, and hence two Cerebral failure 5.8 4.0 independent CFS assessments. This represented 49.1% of Other 10.4 13.0 the whole VIP-2 study population, and patients’ details compared to patients not studied are given in Table 1. The CFS is the value from Rater 1, and the % relates to the column (within the subgroup) Overall the number of completed CFS in the VIP-2 SOFA sequential organ failure assessment, CFS clinical frailty scale, IQCODE study was 99.6%, higher than activity of daily life score informant questionnaire on cognitive decline in the elderly, ADL activity of daily (ADL): 88.6% or cognition (IQCODE): 76.0%, showing living, CPS Co-morbidity and Polypharmacy score very high compliance with this score. The profession of rater 1 and rater 2 were most often ICU physicians fol- lowed by ICU nurses, and the source of information for both pairs. The best results were obtained when both the rating was most often the family/care-givers (Fig. 1). raters were either nurses or physicians, and mixed pairs The mean CFS from rater 1 and 2 was 4.18 (± 1,764) and of assessors performed slightly worse. Likewise, there are 4.25 (± 1.76), respectively. Since the “other” group of better results when information does not come from the raters and information sources were few and not speci- patients. There is also a good performance of the CFS fied, we have excluded these from further analysis. across countries, but the three countries with the least The 9 different pairs of raters with regard to the profes- number of pairs included performed less well than the sion are given in Table 2. The weighted kappa for all pairs others, although most countries were overall classified as was 0.86 (95% CI 0.84–0.87). very good (weighted kappa ≥ 0.80). The intraclass correlation coefficient (absolute agree- We also performed a sensitivity analysis looking at two ment) was 0.93 for single measures and 0.96 for average subgroups according to rater 1: frail (CFS > 4) versus non- measures, and the weighted kappa for all measures was frail (CFS < 5). In the frail the kappa was 0.70 (95% CI 0.86 (95% CI 0.84–0.87) (Table 3). Worth noting is the 0.66–0.74) compared to 0.76 (95% CI 0.74–0.79). distribution of scores of 4 and 5 in Table 3. A notewor- thy number of rater one and two have scores above and below these values. Among rater one, 30 of 402 (7.4%) Discussion scored one or more CFS classes above 4 and by rater two In this large prospective study on frailty assessment in in 65 of 407 (16%) patients demonstrating some difficulty the ICU using the CFS, we found the overall agreement of judging vulnerable from frail patients. of inter-rater variability in patients > 80 years to be very The results in Table 4 demonstrate the variability good. We revealed, however, distinct variations between between pairs from different professions and less vari- groups of raters and between countries. The agreement ability when similar source for information was used for between obtaining CFS from hospital records or family Flaatten et al. Ann. Intensive Care (2021) 11:22 Page 4 of 7 Raters profession and source of information 70,0 % 60,0 % 50,0 % 40,0 % 30,0 % 20,0 % 10,0 % 0,0 % Profession ICU ICU nurse Research Source Patients Care-giver Hospital pysician staff records Rater 1 Rater 2 Fig. 1 Raters profession and source of information in the two groups Table 2 Distribution of pairs of Rater 1 versus Rater 2 was nearly identical but was lower when the patients were used as the primary source of information. Profession of raters Rater 1 Frailty is important in order to understand critical ill a Rater ­2 ICU nurse ICU physician Research staff patients, particularly in advanced age [16], and most ICU nurse 57 309 126 studies have demonstrated a close link between frailty ICU physician 162 785 120 and survival. Hence, knowledge of frailty status could be Research staff 46 96 138 important when issues such as ICU triage and limitation a of life-sustaining therapy are discussed. Recent guidelines Missing pairs in 4 patients Table 3 Intraclass variance Rater 1 (CFS1) and Rater 2(CFS2); weighted kappa (linear) 0.86 (0.84–0.87) CFS 2 CFS 1 1 2 3 4 5 6 7 8 9 Sum % 1 76 3 0 1 1 0 0 0 0 81 (3.5%) 2 0 7 1 0 0 0 0 0 0 196 (14.0%) 3 2 40 403 25 7 2 0 1 0 480 (25.0%) 4 1 4 54 313 22 7 1 0 0 402 (20.9%) 5 0 0 10 58 163 28 2 0 1 262 (13.6%) 6 0 0 5 5 43 177 17 2 0 249 (12.9%) 7 0 0 0 1 2 44 126 8 0 181 (9.4%) 8 0 0 0 1 0 3 13 46 2 65 (3.4%) 9 0 0 0 0 0 0 0 0 7 7 (0.4%) Sum % 94 (4.9%) 202 (10.5%) 493 (25.6%) 407 (21.2%) 240 (12.5%) 261 (13.6%) 159 (8.3%) 57 (3.0%) 10 (0.5%) 1923 Flaatten et al. Ann. Intensive Care (2021) 11:22 Page 5 of 7 Table 4 Weighted kappa in subgroups (physicians raters with a weighted kappa of 0.86. Since we had a large and nurses) and 8 countries (≥ 100 pairs) number of pairs to study, we could study results in sub- Group profession N Weighted 95% CI groups, both between raters from different professions, kappa the source of information as well as performance across countries. There seems to be better agreement when the ICU physician/ICU physician 785 0.87 0.85 to 0.89 raters are from the same profession; physicians or nurses. ICU nurse/ICU nurse 57 0.92 0.87 to 0.97 When the raters are from different professions the agree- Research staff/research staff 276 0.84 0.80 to 0.87 ment is slightly less. We also provide data showing that ICU nurse/ICU physician 162 0.77 0.71 to 0.83 obtaining information from family members and care- ICU physician/ICU nurse 309 0.80 0.77 to 0.84 givers or from written records in order to classify CFS Group: information source is in fact, better compared to the information obtained Patient/patient 394 0.85 0.82–0.88 from the patient. This may have a simple explanation that Family/family 818 0.89 0.87–0.90 many elderly patients, although seemingly awake and Hospital records/hospital records 187 0.89 0.85–0.92 co-operable, may not perform at their best at the time Group: countries of ICU admission. Hence important information may England 397 0.90 0.87 to 0.97 not be revealed for the rater. We have found that it can France 257 0.81 0.78 to 0.85 be a problem to differentiate between CFS 4 and 5. This Spain 186 0.89 0.85 to 0.93 could be important since 5 is the first stage on the frail Poland 163 0.89 0.85 to 0.92 part of the CFS and 4 is borderline. Recently the CFS was Greece 133 0.89 0.85 to 0.94 upgraded to version 2.0 and a more detailed guideline Germany 125 0.81 0.75 to 0.86 in how to understand and use the different levels in the Norway 110 0.71 0.63 to 0.78 scale have been published [21]. Portugal 106 0.82 0.76 to 0.89 Our study is in line with three recent studies of the inter-rater variability of the CFS. All studies are from single countries with a smaller number of pairs included, propose the use of frailty assessment as a part of the tri- and only the overall inter-rater variability was reported. age process to be used with COVID-19 [17]. However, In a study from Canada involving two ICUs [8], different use of frailty in triage setting has its limitations, and is assessors from a research coordinator, an occupational at present not confirmed in prospective studies. How- therapist, and a geriatric resident, performed CFS scores ever, use of CFS would be effective in analysing the clini- in 150 newly admitted ICU patients. They reported no cal decision-making process of an ICU team. There are significant differences between the three raters using several methods to assess frailty, and CFS is frequently Spearman’s rank correlation coefficient. In a more recent used in clinical studies with ICU patients [2] as well as study from six ICUs in Wales and Scotland, 101 patients in emergency admission [18] and is also used in routine were studied with two independent CFS assessments of clinical use in intensive care units outside study settings frailty by assessors from medical or nursing backgrounds [19]. [7]. They found a good agreement with a weighted kappa Using an instrument such as a frailty score requires of 0.74 between raters, and also that agreement differed knowledge about its performance and with special atten- slightly depending on the assessor’s background. tion to reliability and construct validity [20]. Of interest A more recent study comparing CFS scored in 158 is also its ability to predict risk for death, where CFS have adult ICU patients scored by geriatricians and intensiv- been found to perform well. This was recently confirmed ists reported however a poor agreement between raters in the VIP-2 study, where CFS alone had similar predic- [9]. The authors suggest that these two groups have a tive value for 30-day mortality as a model incorporating different conception of how frailty presents in critical ill cognition and functional disability [12], again providing patients as an explanation for this result. good criterion validity. Our study has its limitation: this was not a controlled The aim of the present study was to document several trial with regard to the choice of profession and source unanswered questions using CFS. What is the inter-rater of data used and may have been at the centres’ discre- variability when analysing more heterogeneous groups tion. We also have no information about the clinical of raters using a various source of data for the score? experience of the raters nor their age. The study has also Both aspects are important properties of a clinical test strengths. It has a very large sample size of nearly 2000 or score. The variation of a score in the same patient pairs of raters, with at least three important sources of between two raters is the inter-rater variability. Overall, variation: the profession of raters, source of information, our data proves a very high degree of agreement between and country. Flaatten et al. Ann. Intensive Care (2021) 11:22 Page 6 of 7 Conclusion were in fact considered as “test persons” and this was approved when neces- sary. Ethical approval from Norway (the country of the PI) had the number In very elderly ICU patients, the CFS has a high compli- 2018/87/REK sør-øst D. ance rate and exhibits high overall inter-rater agreement with a weighted kappa analysis of 0.86. Furthermore, Consent for publication Not applicable. there are minor variations in performance across dif- ferent health care professionals, countries and source of Competeing interests data. We found the best agreement using raters from the JCS declares that the Dept. of Intensive Care Medicine Bern has/had research and/or development/consulting contracts with (full disclosure): Orion Corpo- same health care professionals, but with no difference ration, Abbott Nutrition International, B. Braun Medical AG, CSEM SA, Edwards between pair of nurses or physicians. To determine CFS, Lifesciences Services GmbH/SA, Kenta Biotech Ltd, Maquet Critical Care AB, caution should be used to rely on the elderly ICU patients Omnicare Clinical Research AG, and Nestlé. Educational grants were received from Fresenius Kabi; GSK; MSD; Lilly; Baxter; Astellas; AstraZeneca; B. Braun as the sole source of information. Medical AG, CSL Behring, Maquet, Novartis, Covidien, Nycomed, Pierre Fabre Frailty assessment should be routine in the critical ill Pharma (Roba Pharma); Pfizer, Orion Pharma. The money went into depart- elderly patients, and the CFS is a good instrument in this mental funds. No personal financial gain applies. All other authors do not have any conflict of interest to declare related to this manuscript. respect, and will give a more holistic impression of the patient´s condition prior to admission. Author details 1 Department of Anaesthesia and Intensive Care, Dep of Clinical Medi- cine, Haukeland University Hospital Bergen Norway, University of Bergen, Supplementary Information 5019 Bergen, Norway. 2 Sorbonne Université, INSERM, Institut Pierre Louis The online version contains supplementary material available at https​://doi. D’Epidémiologie Et de Santé Publique, Saint Antoine Hospital, AP-HP, Hôpital org/10.1186/s1361​3-021-00815​-7. Saint-Antoine, Service de Réanimation, 75012 Paris, France. 3 Department of Anaesthesia and Intensive Care, Ålesund Hospital, Ålesund, Norway. 4 Dep of Circulation and Medical Imaging, NTNU, Trondheim, Norway. 5 Deparment Additional file 1. Guideline for reporting of reliability and agreement of Intensive Care Medicine, CIBER Enfermedades Respiratorias, Corpora- studies (GRAAS). cion Sanitaria Universitaria Parc Tauli, Autonomous University of Barcelona, Additional file 2. List of local investigators. Sabadell, Spain. 6 Department of Anesthesia and Intensive Care Medicine, Humanitas Clinical and Research Center–IRCCS, Via Alessandro Manzoni 56, 20089 Rozzano, MI, Italy. 7 AP-HP, Hôpital Saint-Antoine, Service de Réanima- Abbreviations tion, 75012 Paris, France. 8 Faculty of Medicine, University of Tripoli, Tripoli, CFS: Clinical frailty scale; VIP: Very old intensive care patients; ADL: Activity of Libya. 9 Department of Intensive Care, Aarhus University Hospital, Aarhus, daily life; IQCODE: Informant questionnaire on cognitive decline in the elderly; Denmark. 10 Division of Intensive Care and Emergency Medicine, Department GRAAS: Guidelines for reporting of reliability and agreement studies. of Internal Medicine, Medical University Innsbruck, Innsbruck, Austria. 11 Divi- sion of Cardiology, Pulmonology and Vascular Medicine, University Hospital Acknowledgements Düsseldorf, Heinrich-Heine- University, Düsseldorf, Germany. 12 Research Lead The study was endorsed by the European Society of Intensive Care Medicine Critical Care Directorate St George’s Hospital, London, UK. 13 Mater Misericor- (ESICM). diae University Hospital, Dublin, Ireland. 14 Faculdade de Ciências Médicas de Lisboa (Nova Médical School), Unidade de Cuidados Intensivos Neurocríticos Authors’ contributions E Trauma. Hospital de São José, Centro Hospitalar Universitário de Lisboa Cen- HF, BG, DDL designed the study and wrote the study protocol. AM provided tral, Lisbon, Portugal. 15 Department of Intensive Care 1K12IC, Ghent University geriatric expertise in designing the protocol. HF and AB performed the Hospital, Ghent, Belgium. 16 European Wellness International, ICU, Luhansk, statistical analysis. JF was responsible for data management and the electronic Ukraine. 17 Department of Intensive Care Medicine, Inselspital, Universitätsspi- CRF; HF drafted the first manuscript, and the writing group (BG, DDL, CJ and tal, University of Bern, Bern, Switzerland. 18 Intensive Care and Perioperative HF) discussed and finalised the manuscript. The rest of the authors were Medicine Division, Jagiellonian University Medical College, Kraków, Poland. 19 country coordinators and hence were responsible for data retrieval and have Heart Center, Linkoping University Hospital, Linkoping, Sweden. 20 ICU, participated reading and commenting the final draft of the manuscript. The St George’s University Hospital, London, UK. 21 Intensive Care Unit General VIP-2-study collaborators from all individual ICUs are listed in Additional file 2. Hospital of Larissa, Larissa, Greece. 22 Department of Intensive Care Medicine, Dutch Poisons Information Center (DPIC), University Medical Center, University Funding Utrecht, Utrecht, The Netherlands. Free support for running the electronic database and was granted from University of Aarhus, Denmark. Financial support for creation of the e-CRF Received: 10 November 2020 Accepted: 27 January 2021 and maintenance of the database was possible from a grant (open project support) by Western Health region in Norway 2018 who also funded the par- ticipating Norwegian ICUs. DRC Ile de France and URC Est helped conducting VIP-2 in France. References Availability of data and materials 1. Flaatten H, de Lange DW, Morandi A, et al. The impact of frailty on ICU Data are embedded in the main database, and as such cannot be opened for and 30-day mortality and the level of care in very elderly patients (≥ 80 general inspection. The main author can at request give an extract from the years). Intensive Care Med. 2017;43:1820–8. database for each pair of raters with necessary information about the inter- 2. Muscedere J, Waters B, Varambally A, et al. The impact of frailty on inten- rater agreement. sive care unit outcomes: a systematic review and meta-analysis. Intensive Care Med. 2017;43:1105–22. Ethics approval and consent to participate 3. Silva-Obregón JA, Quintana-Díaz M, Saboya-Sánchez S, et al. Frailty The VIP2 study that includes this cohort of patients received ethical approval as a predictor of short- and long-term mortality in critically ill older from all participating countries, as revealed in the original publication [12]. medical patients. J Crit Care. 2020;55:79–85. https​://doi.org/10.1016/j. Briefly this included informed consent at admission in some countries, and jcrc.2019.10.018. this was not necessary in others because of the observational nature of the study. Some countries did require informed consent from raters, since they Flaatten et al. Ann. Intensive Care (2021) 11:22 Page 7 of 7 4. Robert R, Kentish-Barnes N, Boyer A, et al. Ethical dilemmas due to 14. Kottner J, Audigé L, Brorson S, et al. Guidelines for Reporting Reliabil- the Covid-19 pandemic. Ann Intensive Care. 2020;10:84. https​://doi. ity and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. org/10.1186/s1361​3-020-00702​-7. 2011;64:96–106. 5. Fried LP, Ferrucci L, Darer J, Williamson JD, Anderson G. Untangling the 15. Altman DG. Practical statistics for medical research. London: UK, Chap- concepts of disability, frailty, and comorbidity: implications for improved man and Hall; 1991. targeting and care. J Gerontol A Biol Sci Med Sci. 2004;59:255–63. 16. Flaatten H, Jung C, Valle H, Guidet B. How Does Frailty Affect ICU Out- 6. Rockwood K, Song X, MacKnight C, et al. A global clinical measure of come? Curr Anesthesiol Rep. 2019;9:144–50. fitness and frailty in elderly people. Can Med Assoc J. 2005;173:489–95. 17. National Institute for health and Care Excellence. COVID-19 rapid 7. Pugh RJ, Ellison A, et al. Feasibility and reliability of frailty assessment in guideline: critical care in adults, 2020. https​://www.nice.org.uk/guida​nce/ the critically ill: a systematic review. Crit Care. 2018;2018(22):49. ng159​. Accessed 19 Jan 2021. 8. Rockwood K, Stadnyk K, MacKnight C, McDowell I, Hébert R, Hogan DB. A 18. O’Caoimh R, Costello M, Small C, et al. Comparison of Frailty Screening brief clinical instrument to classify frailty in elderly people [letter]. Lancet. Instruments in the Emergency Department. Int J Environ Res Public 1999;353:205–6. Health. 2019;16:3626. 9. Pugh RJ, Battle CE, Thorpe C, et al. Reliability of frailty assessment in the 19. Darvall JN, Bellomo R, Bailey M, et al. Frailty and outcomes from pneu- critically ill: a multicentre prospective observational study. Anaesthesia. monia in critical illness: a population-based cohort study. Br J Anaesth. 2019;74:758–64. 2020;2:21. 10. Shears M, Takaoka A, Rochwerg B, et al. Assessing frailty in the intensive 20. Flaatten H, Clegg A. Frailty: we need valid and reliable tools in critical care. care unit: a reliability and validity study. J Crit Care. 2018;45:197–203. Intensive Care Med. 2019;44:1973–5. 11. Surkan M, Rajabali N, Bagshaw SM, Wang X, Rolfson D. Interrater reliability 21. Rockwood K, Theou O. Using the clinical frailty scale in allocation scarce of the clinical frailty scale by geriatrician and intensivist in patients admit- health care resources. Can Geriatr J. 2020;23:210–5. ted to the intensive care unit. Can Geriatr J. 2020;23:235–41. 12. Guidet B, de Lange DW, Boumendil W, et al. The contribution of frailty, cognition, activity of daily life and comorbidities on outcome in acutely Publisher’s Note admitted patients over 80 years in European ICUs: the VIP2 study. Inten- Springer Nature remains neutral with regard to jurisdictional claims in pub- sive Care Med. 2020;46:57–69. lished maps and institutional affiliations. 13. Abraham PS, Courvoisier DS, Annweiler C, et al. Validation of the Clinical Frailty Score (CFS) in French language. BMC Geriatr. 2019;19:322.