Clinical Investigations Framework to Support the Process of Decision- Making on Life-Sustaining Treatments in the ICU: Results of a Delphi Study Monika C. Kerckhoffs, MD1; Jannien Senekal, MD1,2; Diederik van Dijk, MD, PhD1; Antonio Artigas, MD, PhD3,4; Jenie Butler, RN, MCN5,6; Andrej Michalsen, MD, MPH7; Margo M. C. van Mol, RN, PhD8; Rui Moreno, MD, PhD9; Filipa Pais da Silva, MD9; Edoardo Picetti, MD10; Pedro Póvoa, MD, PhD11; Annette Robertsen, MD12; Johannes J. M. van Delden, MD, PhD13 Objectives: To develop a consensus framework that can guide supported by caregivers, patients, and surrogate decision makers the process of decision-making on continuing or limiting life-sus- from multiple countries. taining treatments in ICU patients, using evidence-based items, Design: A three-round web-based international Delphi consensus study with a priori consensus definition was conducted with experts from 13 countries. Participants reviewed items of the 1 Department of Intensive Care Medicine, University Medical Center decision-making process on a seven-point Likert scale or with Utrecht, Utrecht University, Utrecht, The Netherlands. open-ended questions. Questions concerned terminology, con- 2 College of Intensive Care Medicine of Australia and New Zealand (ad- vanced trainee), Melbourne, VIC, Australia. tent, and timing of decision-making steps. The summarized results 3 Critical Care Department, CIBER Enfermedades Respiratorias Corpo- (including mean scores) and expert suggestions were presented racion Sanitaria Universitaria Parc Tauli, Autonomous University of Bar- in the subsequent round for review. celona, Sabadell, Spain. Setting: Web-based surveys of international participants repre- 4 Critical Care Department, University Hospitals Sagrado Corazon-Gen- eral de Cataluña, Quiron Salud, Barcelona, Spain. senting ICU physicians, nurses, former ICU patients, and surro- 5 Intensive Care Services, Royal Brisbane and Women’s Hospital, Her- gate decision makers. ston, QLD, Australia. Patients: Not applicable. 6 University of Queensland, Centre for Clinical Research, Herston, QLD, Interventions: Not applicable. Australia. Measurements and Main Results: In three rounds, respectively, 28, 7 Department of Anesthesiology and Critical Care, Medizin Campus Bo- 28, and 27 (of 33 invited) physicians together with 12, 10, and densee, Tettnang Hospital, Tettnang, Germany. seven (of 19 invited) nurses participated. Patients and surrogates 8 Department of Intensive Care Adults, Erasmus University Medical Center, Rotterdam, The Netherlands. were involved in round one and 12 of 27 responded. Caregivers 9 Unidade de Cuidados Intensivos Neurocríticos e Trauma, Hospital de were mostly working in university affiliated hospitals in Northern São José, Centro Hospitalar de Lisboa Central, Lisbon, Portugal. Europe. During the Delphi process, most items were modified in 10 Department of Anesthesia and Intensive Care, Parma University Hos- order to reach consensus. Seven items lacked consensus after pital, Parma, Italy. three rounds. The final consensus framework comprises the 11 Polyvalent Intensive Care Unit, Hospital de São Francisco Xavier, CHLO, content and timing of four elements; three elements focused on Lisbon, Portugal. 12 Department of Anesthesiology and Critical Care, Oslo University hos- ­caregiver-surrogate communication (admission meeting, follow-up pital, Oslo, Norway. meeting, goals-of-care meeting); and one element (weekly time-out 13 Department of Medical Humanities, Julius Center for Health Sciences meeting) focused on assessing preferences, prognosis, and pro- and Primary Care, University Medical Center Utrecht, Utrecht University, portionality of ICU treatment among professionals. Utrecht, The Netherlands. Conclusions: Physicians, nurses, patients, and surrogates gener- Copyright © 2019 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine. This is an open- ated a consensus-based framework to guide the process of deci- access article distributed under the terms of the Creative Commons sion-making on continuing or limiting life-sustaining treatments in Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), the ICU. Early, frequent, and scheduled family meetings combined where it is permissible to download and share the work provided it is prop- erly cited. The work cannot be changed in any way or used commercially with a repeated multidisciplinary time-out meeting may support without permission from the journal. decisions in relation to patient preferences, prognosis, and pro- DOI: 10.1097/CCM.0000000000004221 portionality. (Crit Care Med 2020; 48:645–653) Critical Care Medicine www.ccmjournal.org 645 Kerckhoffs et al Key Words: clinical decision-making; critical care; Delphi MATERIALS AND METHODS technique; intensive care units; patient care planning We conducted a three-round web-based Delphi consensus study, based on evidence on decision-making strategies, in accordance with guidelines and in analogy with recently per- formed ICU Delphi studies (16, 19–22). The Delphi technique O n almost 75,000 ICU beds across Europe, critically ill is widely used in health research to obtain consensus in serial patients are cared for every day (1). On most of these surveys, referred to as rounds (23). Key elements of the tech- days, a decision to continue treatment is taken, implic- nique are: 1) expert participants, 2) anonymity and individ- itly or explicitly. In at least 10% of all ICU patients, however, uality, and 3) providing a summary of results of the former the decision is made to limit life-sustaining ICU treatments (2, round at the start of the subsequent round. 3). Although some decisions concerning life-prolonging thera- Ethical approval was granted from the Institutional Review pies may seem straightforward, most decisions are the result of Board of the University Medical Centre Utrecht, The Netherlands a complex process of decision-making. (protocol number: 16/508), University Hospital Ghent, Belgium In 2003 and 2019, the prospective multicenter ETHICUS (BC/2368 LBG), and the University Hospital Oslo, Norway studies on end-of-life decisions in the ICU gave valuable insights (17/16124). Consent was waived, except for the Norwegian and into the variability and complexity of life-sustaining treatment Belgian participants who provided a written consent. decisions across European ICUs (3, 4). The variability between and within countries on those decisions as well as heteroge- Expert Panel Recruitment neity in prevalence of withdrawal of life-sustaining treatments Experts were defined as having theoretical knowledge or prac- within one ICU is only decreasing slowly (2, 4–6). The reason tical experience with the decision-making process on contin- for this variation is multifactorial. Patient-related factors (i.e., uing, withholding, or withdrawing life-sustaining treatments in age, comorbidities), geographic factors (e.g., southern Europe vs the ICU. To develop a framework that would comply with the central/northern Europe), and ICU characteristics all impact the needs of all stakeholders, the expert panel consisted of: 1) ICU tendency to make or avoid decisions on life-sustaining therapy physicians, 2) ICU nurses, and 3) former ICU patients and sur- (3, 4, 7–9). In addition, physician-related factors can cause dis- rogate decision makers. Physicians who were either member of cordances in prognostic estimates, and unknown preferences of the section on Ethics of the European Society of Intensive Care the patient can cause hesitation in decision-making (10–12). Medicine or known for their interest in ethics (clinically or scien- Although decisions are inevitably individual, they should tifically) were primarily invited to participate by email. Snowball always be the result of a careful process. The decision-making sampling, participants recommending acquaintances, was used process minimally involves a stepwise practice of gathering and to recruit additional physicians and nurses. Intended participants interpreting information, weighing different options, and ul- received an email containing a summary of the relevant literature timately taking a (shared-) evidence-based and personalized and the purpose, design, and time investment of the study. decision (13). Engaging in this process can both minimize sub- The former ICU patients and surrogate decision makers jectivity and biases and maximize ICU team, patient, and sur- were invited through five participating hospitals in four coun- rogate involvement (10, 14). tries. Patients were eligible if they had been admitted to the There is limited evidence on the required elements of the de- ICU for more than 7 days in the past 2 years. Surrogate deci- cision-making process on continuing or limiting ICU treatment sion makers were eligible when their family member met the and how each element contributes to a careful process and deci- former criteria and was discharged alive. For practical reasons, sion (15, 16). Strategies based on frequent caregiver-family meet- Dutch or English proficiency was required. Patients and surro- ings with predefined topics and integrated within standard ICU gate decision makers were treated as one expert group. care may improve process measures like “time to decision” and reduce nonbeneficial treatment days (16). In addition, it has been Delphi Design shown that adequate communication, including the opportunity The study was designed to consist of three rounds. Between to challenge the appropriateness of care within ICU teams, is a rounds, data were summarized and analyzed anonymously by prerequisite for the delivery of appropriate care (17). an independent researcher (J.S.) using Typeform and IBM SPSS Because several strategies contributing to careful deci- Statistics for Windows, version 25.0 (IBM, New York, NY) (Fig. sion-making have been described in the past, we hypothesized 1). Refinements made to the survey or framework were discussed that it would be possible to integrate this knowledge into a within the core research group (J.S., M.C.K., J.v.D., D.v.D.). Only framework that can support the process of decision-making participants who completed the previous round were invited on treatment intensity in the ICU. The use of a framework pro- to subsequent rounds. Participation could be withdrawn at any motes transparency of the process for all stakeholders, mini- time without reason. No financial compensation was offered. mizes the effect of coincidence, and encourages patients and Consensus was a priori defined as more than 70% agreement or surrogates to participate in decision-making (18). a mean Likert score more than or equal to 4 (on a scale of 1–7). The aim of this study was to develop a framework that can No consensus encompassed less than 70% agreement or a mean support the decision-making process to continue, withhold, or Likert score of less than 4. Items with 30–70% consensus were re- withdraw life-sustaining treatments in ICU patients. fined based on recommendations and feedback from participants 646 www.ccmjournal.org May 2020 • Volume 48 • Number 5 Clinical Investigations and presented in the subsequent round. Elements with less than An online survey was designed based on an outline of deci- 30% agreement were regarded as irrelevant and excluded. sion-making steps (elements) for which the participants were asked to specify the content through defining “items,” tim- Survey Design ing, and contributors (Fig. S1, Supplemental Digital Content A complete description of the survey design is described in 1, http://links.lww.com/CCM/F308). The survey consisted of the supplemental methods (Supplemental Digital Content 1, statements, questions using a seven-point Likert scale (seven is http://links.lww.com/CCM/F308). most positive), and open-ended questions. The online survey was pilot- tested in eight experts, profes- sionals, and surrogate decision makers, not participating in the Delphi study itself. The survey could be completed via a personal access-code ensur- ing single completion. Patients and surrogate de- cision makers received a paper copy of the first round of the survey in laymen’s terms with a prepaid return-envelop. They were given the opportunity to reflect on the basic draft and encouraged to share their view on the decision-making pro- cess. It was prespecified to in- volve them only in round one to minimize the burden. To maximize their input, their survey was enriched with more open-ended questions. “Round one” aimed at getting insight into expert’s views on the optimal ICU decision-making process, focused on communi- cation, gathering of background information, and the use of communication aids and tools. Importance and feasibility of each element were assessed. During “round two,” the ICU physicians and nurses were pro- vided the results and feedback from the first round, revised items (without consensus), and additional items based on rec- ommendations. Questions on the importance, feasibility, role, and triggers to involve palliative care teams and ethics consul- tants were added. During “round three,” the refined version of the deci- sion-making framework was presented for feedback re- garding usefulness and fea- Figure 1. Flowchart Delphi process. Graphical representation of the methods of the Delphi consensus process. sibility. Round three also Critical Care Medicine www.ccmjournal.org 647 Kerckhoffs et al finalized evaluation of items without consensus as well as addi- TABLE 1. Characteristics of Experts tional recommendations that had been made by participants. ICU Patients/ Characteristic Intensivists Nurses Surrogates Statistical Reporting Descriptive statistics (SPSS, version 25.0; IBM Corp) were used No. invited 33 19 27 to analyze and report data, including percentage agreement, No. of participants first 28 (85) 12 (63) 12 (44) mean Likert score, equimedian (defined as the median weighted round, n (%) by size of group), and interquartile ranges when appropriate. No. of participants 27 (96) 7 (58) NA completed all three RESULTS rounds, n (%) Seventy-nine experts consented to participate after the invita- Gender, n (%) tional email, of which 28 physicians, 12 nurses, and 12 patients Males 20 (74) 2 (17) Information and surrogate decision makers completed round one (Table 1; not available details in Table S1, Supplemental Digital Content 1, http://links. Years of experience NA lww.com/CCM/F308). The physicians and nurses were experi- in ICU (yr), n (%) enced caregivers, predominantly working in university affiliated 0–10 3 (11) 1 (8) hospitals. Ninety-two percent of all experts were from Europe. Former ICU patients and surrogate decision makers complet- 11–20 9 (33) 3 (25) ing the survey were from the Netherlands, United Kingdom, and > 20 15 (56) 8 (67) Belgium. Results are presented in Figure 2, A and B, Table 2, and Current practice, n (%) NA in the following paragraphs. Detailed results on each element are presented in Figure S2 and Table S2 (Supplemental Digital University affiliated 22 (81) 9 (75) hospital Content 1, http://links.lww.com/CCM/F308). “In round one,” 52 of the 79 invited experts responded District/general hospital 4 (15) 3 (25) (66%). Consensus was reached on importance and feasibility of Private practice 1 (4) 0 an admission meeting, gathering of information regarding pre- Characteristics of NA morbid health, and quality of life, a multidisciplinary meeting ICU(multiple options and a formal family meeting. No consensus was reached on their possible), n (%) timing. Consulting the general practitioner and other specialists Mixed 22 (81) 8 (67) to gather background information was considered important. In addition, using family meetings was chosen over questionnaires Medical 4 (15) 4 (33) as a source of information. There was consensus on the use of an Surgical 4 (15) 4 (33) informational brochure and communication preparation tool, Neuro 6 (22) 2 (17) as opposed to the use of video. Excluded items referred to man- Cardiothoracic 2 (7) 2 (17) datory participants of individual meetings. “After round one,” the framework was modified to a struc- Country of origin ture of four elements; three family meetings and a multidisci- (participants first round), n (%) plinary caregivers meeting. This meeting was retitled “time-out meeting” to elucidate the goal of the meeting, namely to address Australia 2 (7) 1 (8) 0 (0) whether to continue, withhold, or withdraw life-sustaining Belgium 4 (14) 3 (25) 2 (17) treatments. The item on religion and cultural background was Denmark 1 (4) 0 (0) 0 (0) rephrased to assess inclusion in the framework. New proposi- tions regarding the timing of the individual meetings were Germany 1 (4) 1 (8) 0 (0) made based on responses. Four additional items of the admis- Greece 0 (0) 1 (8) 0 (0) sion meeting and family meetings, additional options to sup- Israel 1 (4) 0 (0) 0 (0) port the gathering of information, as well as a recommendation Italy 5 (18) 1 (8) 0 (0) for additional services available to patients/surrogates were added based on recommendations from participants. Netherlands 4 (14) 2 (17) 9 (75) “In round two,” 38 of 40 experts responded (95%). Norway 2 (7) 2 (17) 0 (0) Consensus was reached on the four additional items included Portugal 4 (14) 0 (0) 0 (0) in the admission and formal family meeting and on three of four additional options to support the gathering of informa- Spain 1 (4) 0 (0) 0 (0) tion. Neither additional services to support the family nor the United Kingdom 2 (7) 1 (8) 1 (8) use of validated questionnaires to support gathering of infor- United States 1 (4) 0 (0) 0 (0) mation reached consensus. Six of the 13 previous noncon- NA = nonapplicable.Demographic characteristics of intensivists available for sensus items, which were adjusted to choose optional versus 27 participants, data not available for one participant. 648 www.ccmjournal.org May 2020 • Volume 48 • Number 5 Clinical Investigations mandatory inclusion in the framework, remained without regarding palliative care did not achieve consensus. The impor- consensus. tance and feasibility of the availability of an ethics consultant Importance, but not feasibility, of palliative care was estab- was established in an advisory role regarding ethics, policies, lished together with futility as a trigger. All other suggestions and legal implications, as well as resolution of conflict. Only the timing of the ad- mission- and time-out meet- ing reached consensus. “After round two,” the gathering of information was included as an item of the fol- low-up family meeting based on the suggestions of par- ticipants, where the timing was proposed to be day 2 or 3. The “formal family meet- ing” was retitled to “goals-of- care meeting” to emphasize the objective of this meeting. Three additional items related to prognosis and treatment limitations were added to the family and time-out meeting. The described role of palli- ative care was adjusted to eval- uate whether a protocolized role in the decision-making process would be appropriate. The pre- viously presented triggers were combined into “poor prognosis” as a single trigger. The trigger for ethics involvement was pro- posed to be conflict where other means had been ineffective. The use of video was rephrased to “include dig- ital information resources,” in order to include the use of in- ternet resources. “In round three,” 34 of 38 experts responded (89%). The framework was found useful and feasible to support the de- cision-making process in the ICU, and it was established that the framework should be made available to patients and surrogate decision makers. The three items on discuss- ing prognosis and limitations of treatment during time-out- and goals-of-care meeting achieved consensus. Consensus was reached on recommending the availability of religious sup- port, a social worker, and psy- Figure 2. Suggested format of the decision-making framework to use in practice. A, The three family meetings (admission, ICU follow-up and goals-of-care). chologist to support the family Critical Care Medicine www.ccmjournal.org 649 Kerckhoffs et al aimed to inform the family and clarify treatment goals and re- suscitation status; 2) a family follow-up meeting, inform and gather information on premorbid functioning; 3) a goals-of-care meeting, defining milestones and treatment goals in relation to pa- tient preferences and values; and 4) a (recurrent) time-out meet- ing where caregivers address prognosis and proportionality together with goals and/or lim- itations of treatment. At admis- sion, the framework would be offered to the patient/surrogate, informing them about the pro- cess of communication and de- cision-making steps including the agenda for each meeting. The local version of the framework can be enriched with available, recommended measures such as family support services. DISCUSSION We conducted a three-round International Delphi consensus study with physicians, nurses, former ICU patients, and sur- rogate decision makers from 13 countries, who cocreated a deci- sion-making framework describ- ing the content and timing of four evidence- and practice- based decision-making steps. Interpretation of the Figure 2. (Continued) B, The TIME-OUT meeting and additional resources. Framework The created framework should as well as the use of digital information resources. It was estab- be considered as an indicative aid to enhance timely, personal- lished that the palliative care team should not have a mandatory ized, and proportional decision-making, allowing the oppor- protocolled role in the ICU. tunity to modify the timing of the various elements according The minimum frequency of both the time-out and the to patient trajectory. Although our expert sample was relatively goals-of-care meetings was agreed upon to be weekly. small and skewed toward experienced European caregivers, the content and timing of the four decision-making steps largely Statistics align with international evidence and practice. For each option provided in the various rounds, the mean and equimedian on the seven-point Likert scale and percentage agreement on dichotomous questions were presented in the Framework in Relation to Scientific Evidence subsequent rounds. Data are presented in the Table S2 (Supple- The admission meeting, the follow-up meeting, and the mental Digital Content 1, http://links.lww.com/CCM/F308). weekly goals-of-care meeting all emphasize the importance of communication with patients or surrogate decision mak- Practical Implications Concerning the Use of the ers. This is supported by previous studies, as early patient Suggested Framework and family participation improves satisfaction with deci- The final result of the three rounds is presented in Figure 2, A and sion-making, decreases decisional conflict (24–27), and is B and consists of (in summary): 1) an ICU admission meeting, associated with a decreased length of stay for patients who 650 www.ccmjournal.org May 2020 • Volume 48 • Number 5 Clinical Investigations TABLE 2. Description of the Results of the Consensus Process Per Round Details per Meeting Round 1 2 3 ICU admission meeting Content items Update on current situation and disease/injury process Yes Determine presence of preexisting documented patient preferences Yes Yes Identify and confirm surrogate decision maker/legal representative Yes Provision of information regarding ICU a NA Yes Treatment plan and interventions a NA Yes Potentialv outcomes and prognosisa NA Yes Timing As soon as achievable within 24 hr X Yes Follow-up family meeting (including gathering of information) Content items Update similar to admission meeting NA NA Yes Independence with activities of daily living Yes Frailty Yes Cognition Yes Pain and discomfort on a daily basis Yes Religion and cultural background X Yes Timing Day 2 of ICU admission X X Yes Time-out meeting Content items Evaluate effectiveness of current management and control of X Yes symptoms (mandatory) Information regarding premorbid health and quality of life (mandatory) X Yes Discuss patient preferences Yes Set treatment goals and milestones for evaluation Yes Establish/review resuscitation status Yes Prognosis and expected outcomes b NA NA Yes Establish limitations of treatment b NA NA Yes Timing 5–7 d after admission X Yes Minimum frequency weekly NA X Yes Goals-of-care meeting Content items Update on current situation Yes Identify milestones to indicate progress or failure Yes Goals-of-care plan Yes Limitations of treatment (if any) Yes Identification of family needs and requirement for support a NA Yes Prognosis and expected outcomes b NA NA Yes Timing Predetermined and at physician discretion NA Yes Frequency: minimum weekly NA X Yes NA = nonapplicable, Yes = consensus achieved, X = no consensus achieved. Additional items added after round one. a Additional items added after round two. b Note that only selected items regarding content and timing which achieved consensus are presented. The complete consensus process is available in Table S2 (Supplemental Digital Content, http://links.lww.com/CCM/F308). Critical Care Medicine www.ccmjournal.org 651 Kerckhoffs et al ultimately die in the ICU (28). Furthermore, it may lead to with patients’ needs and expectations. Sixth, the use of re- a reduction in depressive symptoms in family members and fining comments, for example, the use of additional re- ICU caregivers (29). sources to provide information on ICU admission was Another consistent item in the framework for each family valued very important by patients and surrogate decision meeting is eliciting and reassessing patient treatment prefer- makers, therefore represented in the next round and subse- ences. Studies have shown that this is currently often omitted quently included in the framework. (15, 30, 31). This study also has limitations. First, by design, the results The time-out meeting, as the fourth element of the frame- are influenced by the participating experts. The snowball sam- work, can work as a cognitive aid. By discussing factors known pling method might have introduced a sampling bias resulting to influence prognosis, physicians are stimulated to use ana- in the majority of representatives being overall very experi- lytical thinking before they use their intuition to estimate out- enced, from North-European countries and working in uni- come for a specific patient (32, 33). In addition, weighing risks versity affiliated hospitals, which might impede generalization and benefits helps to structure the decision-making process, in other regions worldwide. The lack of consensus on some promotes more unbiased decisions, and improves prognostic items could reflect cultural differences regarding end-of-life accuracy (34). decisions. Second, we are not informed about the background The absence of palliative care team involvement in the of the patient and surrogates and how decision-making was framework is remarkable because previous studies have shown perceived in their situation. In addition, their response rate its effect on length of stay and quality of communication (35– was lower than expected and they were only involved in the 38). A possible explanation is that evidence on palliative care first round. Including patients and surrogate decision makers interventions mainly arises from Northern America, whereas in subsequent rounds could have influenced the framework, our experts were predominantly working in Europe. Delphi although most of their remarks are incorporated in the final participants mentioned that intensivists should have these version. Third, only 58% of nurses completed all three rounds. competences themselves. Future research should especially consider the applicability of the framework in various regions throughout the world and Framework in Practice the different perspectives of patients and surrogates (including The timing of the various meetings may appear late. After mul- surrogates of deceased patients). tiple Delphi rounds, only a small minority perceived the meet- ings as being too late. Therefore, the suggested timing could be regarded as the latest suitable timing, also granting the op- CONCLUSIONS International physicians, nurses, patients, and surrogates gen- portunity to advance meetings according to patient trajectory. erated a consensus-based framework to guide the process of In addition, the formal role of family members in deci- sion-making varies across countries and our expert sample decision-making on continuing or limiting life-sustaining may have not covered all strategies. Shared decision-making treatments in the ICU. Early, frequent, and scheduled family strategies, as opposed to paternalistic strategies, allows all meetings combined with a repeated multidisciplinary time- stakeholders including the patient’s substitutes to share re- out meeting may support decisions in relation to patient pref- sponsibility (38). This approach may promote earlier surrogate erences, prognosis, and proportionality. involvement than is suggested in the framework. Since the experts were encouraged to draft a framework ACKNOWLEDGMENTS applicable to most ICU patients, some items were rejected for We would like to acknowledge all participants: T. Aasmundstad, J. mandatory inclusion. However, addressing premorbid func- van den Akker, T. S. Andersen, A. Artigas, J. Benbenishty, D. Ben- tioning at admission or evaluating burden and rehabilitation oit, T. Boeter, V. Bosschem, E. Bull, J. Butler, A. Carneiro, G. Cite- planning during the time-out meeting can be very important rio, R. Curtis, C. Danbury, L. De Bus, P. Depuyt, B. Frommlett, A. and might be already valuable practice in some ICUs. Galazzi, L. Gale, R. Gerritsen, G Grasselli, C. Groeninx van Zoelen, E. Indseth, N. Latronico, A. Michalsen, J. Millo, L. Mavrommati, Strengths and Weaknesses M. Mol, R. Moreno, S. Oeyen, F. Pais Silva, E. Picetti, D. Poole, P. This study has several strengths. First, we followed the re- Povoa, M. Reade, A. Robertsen, D. Tjan, M. Trout, F. Van Lierde, C. porting standard for Conducting and Reporting of Delphi Verdonck, and all patients and surrogate decision makers. Studies ensuring clear consensus criteria and descriptions of designing subsequent rounds (22). Second, the survey Drs. Kerckhoffs and Senekal contributed equally to this work. was first piloted, to test understanding of the partici- Supplemental digital content is available for this article. Direct URL cita- pants. Third, the items included in the framework are ev- tions appear in the printed text and are provided in the HTML and PDF idence-based (16). Fourth, the Delphi experts originated versions of this article on the journal’s website (http://journals.lww.com/ from 13 countries reflecting different regions, various cul- ccmjournal). tural backgrounds, and several types of ICUs. Fifth, former Supported, in part, by the LIFE-PRIORITY foundation of the European Society of Intensive Care Medicine (King Baudouin Foundation: 2017- ICU patients and surrogate decision makers participated j5920780-208088). There was no involvement in design, conduction, or in the development of the framework ensuring compliance reporting of the study. 652 www.ccmjournal.org May 2020 • Volume 48 • Number 5 Clinical Investigations Drs. Kerckhoffs and van Dijk’s institutions received funding from the LIFE- 18. Stacey D, Légaré F, Col NF, et al: Decision aids for people facing PRIORITY foundation of the European Society of Intensive Care Medi- health treatment or screening decisions. Cochrane Database Syst cine (ESICM) (King Baudouin Foundation 2017-j5920780-208088). Dr. Rev 2014; 1:CD001431 Michalsen received funding from ESICM. The remaining authors have dis- 19. Turnbull AE, Sepulveda KA, Dinglas VD, et al: Core domains for closed that they do not have any potential conflicts of interest. clinical research in acute respiratory failure survivors: An interna- For information regarding this article, E-mail: m.c.kerckhoffs-2@umcu- tional modified Delphi consensus study. Crit Care Med 2017; 45: trecht.nl 1001–1010 20. Needham DM, Sepulveda KA, Dinglas VD, et al: Core outcome meas- ures for clinical research in acute respiratory failure survivors. An in- ternational modified Delphi consensus study. Am J Respir Crit Care REFERENCES Med 2017; 196:1122–1130 1. Rhodes A, Ferdinande P, Flaatten H, et al: The variability of crit- 21. Diamond IR, Grant RC, Feldman BM, et al: Defining consensus: a ical care bed numbers in Europe. Intensive Care Med 2012; 38: systematic review recommends methodologic criteria for reporting of 1647–1653 Delphi studies. J Clin Epidemiol 2014; 67:401–409 2. Mark NM, Rayner SG, Lee NJ, et al: Global variability in withhold- 22. Jünger S, Payne SA, Brine J, et al: Guidance on Conducting and ing and withdrawal of life-sustaining treatment in the intensive REporting DElphi Studies (CREDES) in palliative care: Recommen- care unit: A systematic review. Intensive Care Med 2015; 41: dations based on a methodological systematic review. Palliat Med 1572–1585 2017; 31:684–706 3. Sprung CL, Cohen SL, Sjokvist P, et al; Ethicus Study Group: End- 23. Linstone HA, Turoff M: The Delphi Method: Techniques and Applica- of-life practices in European intensive care units: The Ethicus study. tions. Digital copy of original version (1975). Newark, NJ, Addison- JAMA 2003; 290:790–797 Wesley, 2002 4. Sprung CL, Ricou B, Hartog CS, et al: Changes in end-of-life prac- 24. Chiarchiaro J, Buddadhumaruk P, Arnold RM, et al: Prior advance care tices in European intensive care units from 1999 to 2016. JAMA planning is associated with less decisional conflict among surrogates 2019; 322:1692–1704 for critically ill patients. Ann Am Thorac Soc 2015; 12:1528–1533 5. Barnato AE, Tate JA, Rodriguez KL, et al: Norms of decision making 25. Lautrette A, Darmon M, Megarbane B, et al: A communication strategy in the ICU: A case study of two academic medical centers at the and brochure for relatives of patients dying in the ICU. N Engl J Med extremes of end-of-life treatment intensity. Intensive Care Med 2012; 2007; 356:469–478 38:1886–1896 26. Kodali S, Stametz RA, Bengier AC, et al: Family experience with in- 6. Piers RD, Azoulay E, Ricou B, et al; APPROPRICUS Study Group of tensive care unit care: Association of self-reported family conferences the Ethics Section of the ESICM: Perceptions of appropriateness of and family satisfaction. J Crit Care 2014; 29:641–644 care among European and Israeli intensive care unit nurses and physi- cians. JAMA 2011; 306:2694–2703 27. Heyland DK, Rocker GM, Dodek PM, et al: Family satisfaction with care in the intensive care unit: Results of a multiple center study. Crit 7. Prendergast TJ, Claessens MT, Luce JM: A national survey of end-of- Care Med 2002; 30:1413–1418 life care for critically ill patients. Am J Respir Crit Care Med 1998; 158:1163–1167 28. Majesko A, Hong SY, Weissfeld L, et al: Identifying family members who may struggle in the role of surrogate decision maker. Crit Care 8. Azoulay E, Metnitz B, Sprung CL, et al; SAPS 3 investigators: End- Med 2012; 40:2281–2286 of-life practices in 282 intensive care units: Data from the SAPS 3 database. Intensive Care Med 2009; 35:623–630 29. Quenot JP, Rigaud JP, Prin S, et al: Suffering among carers working in critical care can be reduced by an intensive communication strategy 9. Benoit DD, Jensen HI, Malmgren J, et al; DISPROPRICUS study on end-of-life practices. Intensive Care Med 2012; 38:55–61 group of the Ethics Section of the European Society of Intensive Care Medicine: Outcome in patients perceived as receiving excessive care 30. Chiarchiaro J, Ernecoff NC, Scheunemann LP, et al: Physicians rarely across different ethical climates: A prospective study in 68 inten- elicit critically ill patients’ previously expressed treatment prefer- sive care units in Europe and the USA. Intensive Care Med 2018; ences in intensive care units. Am J Respir Crit Care Med 2017; 196: 44:1039–1049 242–245 10. Wilkinson DJ, Truog RD: The luck of the draw: physician-related vari- 31. Scheunemann LP, Ernecoff NC, Buddadhumaruk P, et al: Clinician- ability in end-of-life decision-making in intensive care. Intensive Care family communication about patients’ values and preferences in inten- Med 2013; 39:1128–1132 sive care units. JAMA Intern Med 2019; 179:676–684 11. Poulton B, Ridley S, Mackenzie-Ross R, et al: Variation in end-of- 32. Saposnik G, Redelmeier D, Ruff CC, et al: Cognitive biases associ- life decision making between critical care consultants. Anaesthesia ated with medical decisions: A systematic review. BMC Med Inform 2005; 60:1101–1105 Decis Mak 2016; 16:138 12. Wilson ME, Rhudy LM, Ballinger BA, et al: Factors that contribute to 33. Stiegler MP, Ruskin KJ: Decision-making and safety in anesthesi- physician variability in decisions to limit life support in the ICU: A qual- ology. Curr Opin Anaesthesiol 2012; 25:724–729 itative study. Intensive Care Med 2013; 39:1009–1018 34. Kahneman D: Thinking Fast and Slow. New York, NY, Farrar, Straus 13. Kon AA, Davidson JE, Morrison W, et al: Shared decision-making in and Giroux, 2011 intensive care units. Executive summary of the American College of 35. Braus N, Campbell TC, Kwekkeboom KL, et al: Prospective study of a Critical Care Medicine and American Thoracic Society Policy State- proactive palliative care rounding intervention in a medical ICU. Inten- ment. Am J Respir Crit Care Med 2016; 193:1334–1336 sive Care Med 2016; 42:54–62 14. Cook D, Rocker G, Marshall J, et al; Level of Care Study Investigators 36. Aslakson R, Cheng J, Vollenweider D, et al: Evidence-based palliative and the Canadian Critical Care Trials Group: Withdrawal of mechan- care in the intensive care unit: A systematic review of interventions. J ical ventilation in anticipation of death in the intensive care unit. N Palliat Med 2014; 17:219–235 Engl J Med 2003; 349:1123–1132 37. Mercadante S, Gregoretti C, Cortegiani A: Palliative care in intensive 15. Scheunemann LP, Cunningham TV, Arnold RM, et al: How clinicians care units: Why, where, what, who, when, how. BMC Anesthesiol discuss critically ill patients’ preferences and values with surrogates: 2018; 18:106 An empirical analysis. Crit Care Med 2015; 43:757–764 38. Curtis JR, Treece PD, Nielsen EL, et al: Randomized trial of communi- 16. Kerckhoffs MC, Kant M, van Delden JJM, et al: Selecting and evaluat- cation facilitators to reduce family distress and intensity of end-of-life ing decision-making strategies in the intensive care unit: A systematic care. Am J Respir Crit Care Med 2016; 193:154–162 review. J Crit Care 2019; 51:39–45 39. Sprung CL, Truog RD, Curtis JR, et al: Seeking worldwide profes- 17. Piers RD, Azoulay E, Ricou B, et al; Appropricus Study Group of the sional consensus on the principles of end-of-life care for the critically Ethics Section of the European Society of Intensive Care Medicine: ill. The Consensus for Worldwide End-of-Life Practice for Patients in Inappropriate care in European ICUs: Confronting views from nurses Intensive Care Units (WELPICUS) study. Am J Respir Crit Care Med and junior and senior physicians. Chest 2014; 146:267–275 2014; 190:855–866 Critical Care Medicine www.ccmjournal.org 653