healthcare Article Assessing Face Validity of the HexCom Model for Capturing Complexity in Clinical Practice: A Delphi Study Xavier Busquet-Duran 1,2,3, *, Eva Maria Jiménez-Zafra 1 , Magda Tura-Poma 1 , Olga Bosch-de la Rosa 4 , Anna Moragas-Roca 1 , Susana Martin-Moreno 1 , Emilio Martínez-Losada 1 , Silvia Crespo-Ramírez 4 , Lola Lestón-Lado 1 , Núria Salamero-Tura 1 , Joana Llobera-Estrany 1 , Núria Oriol-Peregrina 5,6 , Eduard Moreno-Gabriel 2,7 , Josep Maria Manresa-Domínguez 2,7,8 and Pere Torán-Monserrat 2,7 1 Home Care Program, Granollers Support Team (PADES), Vallès Oriental Primary Care Service, Catalan Health Institute, 08520 Granollers, Spain; evajiza@gmail.com (E.M.J.-Z.); turapoma@hotmail.com (M.T.-P.); amoragas_roca@hotmail.com (A.M.-R.); susmartin0@hotmail.com (S.M.-M.); emlosada1974@gmail.com (E.M.-L.); lestonlola@gmail.com (L.L.-L.); nsalamero.mn.ics@gencat.cat (N.S.-T.); jolloes@hotmail.com (J.L.-E.) 2 Multidisciplinary Research Group on Health and Society (GREMSAS), (2017 SGR 917), 08007 Barcelona, Spain; emoreno@idiapjgol.info (E.M.-G.); jmanresa@idiapjgol.info (J.M.M.-D.); ptoran.bnm.ics@gencat.cat (P.T.-M.) 3 Nursing Department, Fundació Universitària Bages (FUB), University of Vic, 08500 Vic, Spain 4 Red Cross Psychosocial Care Team (EAPS), 08402 Granollers, Spain; olga.bosch@creuroja.org (O.B.-d.l.R.); silvia.crespo@creuroja.org (S.C.-R.) 5 Degree in Speech and Language Therapy, University of Vic-Central University of Catalonia/UOC, 08242  Manresa, Spain; noriol@umanresa.cat  6 Sociosanitari Vallparadís, 08221 Terrassa, Spain 7 Research Support Unit Metropolitana Nord, Primary Care Research Institut Jordi Gol (IDIAPJGol), Citation: Busquet-Duran, X.; Jiménez-Zafra, E.M.; Tura-Poma, M.; 08303 Barcelona, Spain 8 Nursing Department, Faculty of Medicine, Universitat Autònoma de Barcelona, 08193 Barcelona, Spain Bosch-de la Rosa, O.; Moragas-Roca, * Correspondence: xbusquet@umanresa.cat A.; Martin-Moreno, S.; Martínez-Losada, E.; Crespo-Ramírez, Abstract: Capturing complexity is both a conceptual and a practical challenge in palliative care. S.; Lestón-Lado, L.; Salamero-Tura, The HexCom model has proved to be an instrument with strong reliability and to be valid for N.; et al. Assessing Face Validity of the HexCom Model for Capturing describing the needs and strengths of patients in home care. In order to explore whether it is also Complexity in Clinical Practice: A perceived to be helpful in enhancing coordinated and patient-centred care at a practical level, a Delphi Study. Healthcare 2021, 9, 165. methodological study was carried out to assess the face validity of the model. In particular, a Delphi https://doi.org/10.3390/ method involving a group of 14 experts representing the full spectrum of healthcare professionals healthcare9020165 involved in palliative care was carried out. The results show that there is a high level of agreement, with a content validity index-item greater than 0.92 both with regard to the complexity model and the Academic Editor: Magnolia Cardona HexCom-Red, HexCom-Basic, and the HexCom-Clin instruments, and higher than 0.85 regarding the Received: 28 December 2020 HexCom-Figure and the HexCom-Patient instruments. This consensus confirms that the HexCom Accepted: 29 January 2021 model and the different instruments that are derived from it are valued as useful tools for a broad Published: 4 February 2021 range of healthcare professional in coordinately capturing complexity in healthcare practice. Publisher’s Note: MDPI stays neutral Keywords: palliative care; Delphi technique; needs assessment; home care services; coordinated care; with regard to jurisdictional claims in complexity; methodological study published maps and institutional affil- iations. 1. Introduction Copyright: © 2021 by the authors. Due to the inclusion of all groups of pathologies within the scope of palliative care [1,2] Licensee MDPI, Basel, Switzerland. and the incessant aging of the population [3], substantial growth is expected in the demand This article is an open access article for these healthcare services in the upcoming years [4]. In the Spanish context, it is distributed under the terms and considered that 1.5% of the population and 11% of people over 65 need palliative care [5–7]. conditions of the Creative Commons This progressive and quantitative increase brings along a sophistication of this demand. Attribution (CC BY) license (https:// Accordingly, experts agree that the backbone of palliative care must be the notion of creativecommons.org/licenses/by/ complexity [8–11]. However, capturing complexity is both a conceptual and a practical 4.0/). challenge in palliative care [11]. Healthcare 2021, 9, 165. https://doi.org/10.3390/healthcare9020165 https://www.mdpi.com/journal/healthcare Healthcare 2021, 9, 165 2 of 18 The very nature of “complexity” is controversial [12]. This is partly due to the fact that it emerges from the concurrence of multiple interrelated factors, whether they are clinical, contextual, or related with the health system [13]. In an attempt to enhance the concept’s applicability and endorse a patient-centred clinical practice, a number of systemic theoretical frameworks have been developed and adopted over the last few years in the field of palliative care. The cumulative model puts the focus on the imbalance between the burdens and responsibilities of the patient on the one hand and their capacities on the other [14]. The vector model focuses on contextual factors, namely the interrelation and balance between socioeconomic, behavioural, genetic, environmental, cultural, and sociopolitical factors [5,15,16]. Finally, drawing on Bronfenbrenner’s ecological systems theory, Pask et al. [11] offer an innovative framework for understanding complexity by emphasizing the natural ecological environment of the individual, which is formed by the set of relationship structures that accommodate him/her. This systemic approach is particularly relevant for the practice of home care in Spain. In contrast with how palliative care is delivered at other, more institutionalised levels of care, foregrounding the patient’s specific needs and seeing them in relation to a broader and specific context should facilitate providing timely, adequate, individualised, and co- ordinated care [12,17]. Likewise, managing patients’ profiles drawing on their specific needs should assist in identifying those in need of specialised palliative care [11,18]. Ad- ditionally, this holistic perspective enhances home care teams in their responsibility to act as coordinators of different resources and levels of care involved in palliative care in Spain [19]. To the best of our knowledge, there are not available validated instruments based on this systemic approach, neither at a national level [20–24] nor at the international level [25–35]. To overcome this deficit, between 2006 and 2017 a model (HexCom) was designed to ultimately aid healthcare professionals caring for people with advanced illness and/or at the end of life. Originally, this model drew on the complexity levels and the inclusion and referral criteria for palliative care patients defined by the Catalan Government drawing on the conclusions of an interdisciplinary and multilevel work group endorsed by the Catalan Scientific Societies for Palliative Care and Family Medicine and Community [36]. Based on Ferris [37], the model proposes six areas of needs: clinical, psychoemotional, social/family, spiritual, ethical, and death-related. For each area, the level of complexity may be cate- gorised as low (L), medium (M), or high (H). Recently, based on Pask et al. [11], the model has been complemented adding five areas of systemic strength: microsystem, mesosystem, chronosystem, exosystem and macrosystem. For each system, the strength level can also be qualified as low, medium, or high. As a result of contrasting identified needs and strengths, an intuitive balance of complexity is proposed, emerging from considering whether a large amount of systemic strength may or may not reduce the resulting degree of complexity. The model has been published [38] and its content partially validated [39], with an interobserver Kappa of 0.92. The first results of its application in clinical practice have also been published recently [40]. Even though referrer professionals in hospitals and primary care are ultimately in charge of detecting cases with complex needs, palliative care is a basic responsibility of all healthcare professionals [12]. Accordingly, HexCom is deployed in five instrumental versions, which are complementary, suited to the degree of specialization of the service using it and the objective pursued (see Table 1 and Supplementary Materials Instruments S1 and Instruments S3). Healthcare 2021, 9, 165 3 of 18 Table 1. Instruments derived from the HexCom model. HC Professional HexCom Format Needs Strengths End-Users Red Table 6 areas - Generalists. Specialists with basic Basic Table 6 areas 5 systems equipment. 5 systems 6 areas subdivided Specialists Clin Table subdivided into into 18 subareas With full set-up. 9 subsystems 6 areas subdivided 5 systems Specialists into 18 subareas Figure Figure subdivided into with complete + evaluation at the 9 subsystems resources & training. beginning 6 areas 5 areas + 2 open Patient Table Specialists. + 1 open question questions In the Spanish context, specialist palliative care teams generally comprise physicians, nurses, social workers, and psychologists. However, in some areas, these can occasionally include professionals from other backgrounds, such as physiotherapy or even pain and pastoral care [19]. Given this diversity of tools and professionals involved in a complexity assessment, besides the need to achieve coordinated and patient-centred care in palliative care, it is critical to evaluate whether these instruments are consensually perceived, by a broad range of healthcare professionals, as valuable resources for their job of capturing the degree of complexity of all patients’ needs [41]. Hence, this study aims at assessing the face validity of both the Catalan and Spanish versions of the model and its derived instruments. 2. Materials and Methods This study has been conducted and reported following the recommendations for the conducting and reporting of Delphi studies (CREDES) [42]. 2.1. Design of the Study The present is a methodological study using a Delphi process to systematically evalu- ate the face validity of HexCom model and its five instruments. Delphi is a technique for determining the level of formal consensus and is used to obtain and evaluate the opinions of a group of experts with knowledge and experience in a specialised field [43–45]. In this case, the participants are considered experts in so far as they have knowledge and experience in delivering palliative care, therefore being capable of assessing whether the HexCom model is appropriate to the targeted construct (i.e., complexity) and assessment objectives (i.e., to identify complex patients/situations and allocate resources accordingly) [46]. The method structures a group communication process that is effective in allowing a group of individuals, as a whole, to address a complex issue [47]. The qualitative and quantitative approaches are combined into a single procedure, mitigating their respective limitations [48]. It entails the administration of questionnaires in different rounds until consensus is reached [43]. This method guarantees anonymity, controlled feedback, and a systematic assessment of face validity by way of an iterative process and the use of statistical techniques [49,50]. The procedure, including the previous stages of the HexCom model development, is shown in Figure 1. The two stages belonging to the Delphi study reported in this article are detailed below: (1) the preparation of the questionnaire by a coordinator-administrator group, and (2) the consensus methodology with a panel of experts. Healthcare 2021, 9, 165 4 of 18 Figure 1. Process of development and validation of HexCom. 2.2. Preparation of the Questionnaire The coordinator–administrator group was formed by the core members of PADES in Granollers, a home and palliative care support team whose composition is paradigmatic of this type of team in Catalunya (Spain). Namely, it consists of a majority of female professionals with different backgrounds and a longstanding experience in palliative care and home care (see Table 2 for a detailed description). Healthcare 2021, 9, 165 5 of 18 Table 2. Characteristics of the coordinating group. Coordinator Group (n = 11) Age From 29 to 45 years 8 (72.7%) From 45 to 63 years 3 (27.2%) Average (Standard deviation) 45.8 (8.8) Gender Women 9 (81%) Men 2 (19%) Profession Nursing 4 (36.3%) Medicine 3 (27.2%) Social work 2 (18.1%) Psychology 2 (18.1%) Years of Professional Experience From 7 to 10 2 (18.1%) From 11 to 20 4 (36.3%) From 21 to 39 5 (45.4%) Average (Standard deviation) 19.3 (8.0) Years of Palliative Experience From 7 to 10 2 (18.1%) From 11 to 20 6 (54.5%) From 21 to 39 3 (27.2%) Average (Standard deviation) 17.1 (8.9) Years of Home Care Experience From 1 to 10 6 (54.5%) From 11 to 20 2 (18.1%) From 21 to 39 3 (27.2%) Average (Standard deviation) 13.6 (10.8) Experience in Teaching and Research Graduate teaching 6 (54.5%) Postgraduate teaching 7 (63.6%) Publications 9 (81.8%) Drawing on their expertise and the existing literature [51,52], the coordinator– admin- istrator group defined the panel of experts, designed the assessment questionnaire, and provided feedback to the panel. Based on Oriol-Peregrina [53], the questionnaire contains a total of 60 statements or questions regarding the form, content, utility, and clarity of the HexCom model and its dif- ferent versions (see Table 1). More specifically, questions refer to the model (questions 2–15), Red (q. 16–23), Basic (q. 24–32), Clin (q. 33–38), Figure (q. 39–45), Patient (q. 46–59), the clarity of the translation into Spanish, its correspondence with the original Catalan version (60), and one last open question to collect generic qualitative feedback on the model or any of the instruments. Participants were asked to provide their degree of agreement with each statement using a Likert scale including the following grades: 1 = “Strongly disagree”, 2 = “Disagree”, 3 = “Indifferent”, 4 = “Agree”, and 5 = “Strongly agree”. All items included a blank space to allow explanatory comments and suggested modifications (e.g., alternative wording of a given item). 2.3. Delphi Process The Delphi technique has been widely used in palliative care [42]. The first areas where it was used were those of oncology pain [45], and it has been used specifically to assess complexity [54]. Healthcare 2021, 9, 165 6 of 18 2.3.1. Participants and Setting The number of participating experts recommended in the literature is between 7 and 30 [55,56]. Due to the small number of participants, the method is not intended to pro- duce statistically significant results but to collect the synthesis of thought from a special group [49]. The panel of experts was assembled using a purposive sampling process following the selection criteria of their knowledge and experience in palliative and home care at all levels of care (i.e., primary, secondary, etc.), as well as making sure it included those professions most often involved in end-of-life care. Hence, as in the coordinator–administrator groups and generally in the palliative and home care professions in Spain [40], the resulting panel comprised mostly female professionals and did not include professional profiles which are part of palliative care teams on very rare occasions (e.g., physiotherapists, speech therapists, and occupational therapists). Following the established recommendations [40,41] for this procedures, 15 people were selected, with the additional inclusion criteria of having over 7 years of experience in palliative care. Criteria for exclusion consisted of being currently unemployed, retired, or exempt of actual clinical practice for any given reason and having participated in the development of the HexCom model or any of its instruments. 2.3.2. Data Collection After the questionnaire was designed between January and March 2019, the experts were reached out to in March 2019 in order to present them with the project and ask for their participation. In case of acceptance, a personalised explanatory letter was sent by email, including a link to access the virtual anonymous platform containing complementary documentation, the different versions of the instrument (see Supplementary Materials Instruments S1 and Figure S2), and the questionnaire. 2.3.3. Ethical Considerations The participants were informed about the project’s objectives, indicating that their participation was voluntary and that the information was collected in an anonymous database so they could not be identified. The study was approved by the Ethics Committee of the IDIAP Fundació Jordi Gol (Ref. P15/171), the official institutional review board for primary care research in Catalunya. 2.3.4. Data Analysis To assess the degree of consensus after each round, the scores were analysed using the content validity by item index (CVI-I) and the number of experts that showed some disagreement. CVI-I allows assessing quantitatively the degree of agreement amongst experts on a given item, dividing the number of experts agreeing (scoring 4 or 5) with a given item by the total number of experts responding to that question [57]. Following the American Educational Research Association recommendations for studies with a panel of 6 or more experts, a CVI-I of over 0.78 was interpreted as significant consensus [58,59]. After obtaining and analysing the results of the previous round, those items with a CVI-I under 0.78 and/or where three or more experts expressed disagreement were resubmitted in a second and final round [38,60]. Based on Varela et al. [61], the qualitative data obtained in the first round were used in the second round to modify the item in question (see Table A2). 3. Results 3.1. Description of the Panel of Experts A description of the panel of experts whose participation was sought is displayed in Table 3. Fourteen of the 15 invited professionals agreed to take part of the study (93%). The average age of the panel was 52.3 years, with all professions at all levels represented, extensive work experience on average was 24.8 years, including long experience both in palliative care (18.1 years) and home care (13 years), as well as having teaching and research experience in the subject (see Table 3). Healthcare 2021, 9, 165 7 of 18 Table 3. Characteristics of the expert panel. Panel of Experts (n = 14) Age From 29 to 45 years 4 (28.6%) From 45 to 63 years 10 (71.4%) Sex Female 10 (71.4%) Male 4 (28.6%) Profession Nursing 3 (21.4%) Medicine 5 (35.8%) Social work 3 (21.4%) Psychology 3 (21.4%) Place of Current Work Social healthcare 4 (28.6%) Primary care 3 (21.4%) PADES 6 (42.8%) University 1 (7.1%) Years of Professional Experience From 7 to 10 0 (0%) From 11 to 20 5 (35.7%) From 21 to 39 9 (64.3%) Years of Palliative Experience From 7 to 10 4 (28.6%) From 11 to 20 3 (21.4%) From 21 to 39 7 (50%) Years of Home Care Experience From 1 to 10 8 (57.1%) From 11 to 20 1 (6.6%) From 21 to 39 5 (33.3%) Teaching and Research Experience Graduate 8 (57.1%) Postgraduate 12 (85.7%) Publications 13 (92.8%) 3.2. Delphi Survey The results of the first and the second Delphi rounds are shown in Table 4. Even though in 54 of the items (90%) the defined threshold for assuming consensus was achieved in the first round, all of those which received critical commentaries or appropriate suggested modifications (15 items) were revised, modified, and resubmitted for validation in the second round, alongside those six items scoring suboptimal CVI-I (<0.78) in the first round. All items scored above 0.78 CVI-I in the second and final round. Healthcare 2021, 9, 165 8 of 18 Table 4. Expert answers rounds 1 and 2, with average, standard deviation (SD), number of experts in disagreement, and the content validity by item index (CVI-I). Round 1 Round 2 Item Average Experts not in Average Experts not in N CVI-I N CVI-I (SD) Agreement (N) (SD) Agreement (N) Model 2 14 4.71 (0.47) 0 1 14 3 14 4.71 (0.47) 0 1 14 4.47 (1.06) 0 1 4 14 4.71 (0.47) 0 1 14 5 14 4.86 (0.36) 0 1 14 6 14 4.21 (0.58) 1 0.93 14 7 14 4.71 (0.47) 0 1 14 8 14 4.57 (0.51) 0 1 14 9 14 4.57 (0.51) 0 1 14 10 14 4.57 (0.51) 0 1 14 11 14 4.29 (0.61) 1 0.93 14 12 14 4.36 (0.50) 0 1 14 13 14 3.86 (0.66) 2 0.86 14 3.80 (0.88) 1 0.93 14 14 4.50 (0.52) 0 1 14 4.13 (0.99) 0 1 15 14 4.14 (0.66) 2 0.86 14 3.87 (0.83) 0 1 HexCom-Red 16 14 4.64 (0.50) 0 1 14 17 14 4.36 (0.63) 1 0.93 14 18 14 3.93 (0.73) 4 0.71 14 3.93 (1.11) 1 0.93 19 14 4.14 (0.53) 1 0.93 14 20 14 4.14 (0.53) 1 0.93 14 21 14 4.14 (0.77) 3 0.79 14 3.92 (1.06) 2 0.86 22 14 4.36 (0.93) 2 0.86 14 4.40 (1.07) 1 0.93 23 14 4.21 (0.43) 0 1 14 HexCom-Basic 24 14 4.57 (0.51) 0 1 14 25 14 4.29 (0.83) 1 0.93 14 26 14 4.14 (0.86) 2 0.86 14 3.86 (1.07) 1 0.93 27 14 4.21 (0.58) 1 0.93 14 28 14 3.93 (1.00) 3 0.79 14 3.86 (1.07) 1 0.93 29 14 4.00 (0.68) 3 0.78 14 3.85 (1.04) 3 0.78 30 14 4.14 (0.53) 1 0.93 14 31 14 4.14 (0.77) 1 0.93 14 32 14 4.00 (0.96) 4 0.71 14 3.87 (0.83) 0 1 HexCom-Clin 33 14 4.57 (0.51) 0 1 14 34 14 4.36 (0.84) 1 0.93 14 35 14 3.57 (1.16) 5 0.64 14 3.92 (1.13) 2 0.86 36 14 4.29 (0.47) 0 1 14 37 14 4.00 (0.78) 2 0.86 14 3.93 (0.88) 0 1 38 14 4.14 (0.86) 2 0.86 14 3.79 (0.90) 2 0.86 HexCom-Figure 39 14 4.57 (0.51) 0 1 14 40 14 4.29 (0.47) 0 1 14 41 14 3.93 (0.92) 4 0.71 14 3.66 (0.85) 2 0.86 42 14 3.79 (0.89) 5 0.64 14 3.87 (0.83) 0 1 43 14 4.07 (0.27) 0 1 14 3.87 (0.83) 0 1 44 14 3.64 (0.84) 4 0.71 14 3.72 (0.83) 2 0.86 45 14 4.36 (0.63) 1 0.93 14 HexCom-Patient 46 14 4.50 (0.52) 0 1 14 47 14 4.50 (0.52) 0 1 14 48 14 4.29 (0.61) 1 0.93 14 49 14 4.36 (0.63) 1 0.93 14 50 14 4.21 (0.80) 1 0.93 14 3.93 (0.88) 0 1 51 14 4.29 (0.47) 0 1 14 52 14 4.29 (0.47) 0 1 14 53 14 4.29 (0.61) 1 0.93 14 54 14 4.15 (0.80) 1 0.92 14 55 14 4.21 (0.70) 2 0.86 14 4.12 (1.15) 2 0.86 56 14 4.71 (0.47) 0 1 14 57 14 4.86 (0.36) 0 1 14 58 14 4.14 (0.53) 1 0.93 14 59 14 4.00 (0.55) 2 0.86 14 Translation 60 14 4.07 (0.83) 2 0.86 14 4.13 (0.99) 0 1 3.2.1. The HexCom Model As shown in Table 4, all items (2–15 and 60) assessing the model and its perceived usefulness reached the established degree of consensus in the first round (CVI-I > 0.86) Healthcare 2021, 9, 165 9 of 18 which increased in the second round (CVI-I > 0.92) after modifying four items drawing on the commentaries received from the panel. According to the qualitative feedback received, this very high consensus stems mainly from two key elements. On the one hand, the sound, conceptual underpinnings of the model: that complexity must be based on changing needs as the backbone of palliative care and the usefulness of a systemic approach to evaluate both needs and strengths in a given case. On the other hand, though slightly less significant, the model’s perceived transferability to all levels of care and, ultimately, the general public. 3.2.2. HexCom Instruments After adjusting those formal aspects (Item 18) suggested by the panel between rounds, HexCom-Red exceeded 0.92 in CVI-I for all items in the second round, except in the item (21) relating to the explanatory wording. Nevertheless, agreement upon this item improved from 0.79 in the first round to 0.86 in the second. Similarly, HexCom-Basic exceeded 0.92 in the CVI-I scores for almost all items in the second round (improving the scores in three of four items modified and resubmitted in the second round). Item 29, which refers to the explanatory wording for each area of systemic strengths is an exception to this trend. This scored a CVI-I of 0.78 in the first round and did not improve in the second round. This was partly due to the fact that three experts responded “indifferent” in the first round but did not provide alternatives to the wording. All items regarding HexCom-Clin achieved the established degree of consensus, scoring 0.86 or above in the CVI-I of all items considering both round 1 and 2. After registering insufficient degree of consensus in the first round, Item 35 (referring to the instrument’s format) was modified and validated in the second round, where it scored a CVI-I of 0.86. Item 38, asking about non-specialists’ understanding of the instrument was resubmitted with modifications in the second round, where it scored 0.86 in CVI-I, as in the first round. In the first round, the panel of experts assigned rather low scores to formal aspects (Item 41) and content issues (Item 42 and 44) of the HexCom-Figure. However, after the suggested modifications were introduced between rounds, all items exceeded 0.85 of the CVI-I score. All questions about the HexCom-Patient received were agreed upon in the first round, scoring CVI-I’s over 0.85. Even though two items were resubmitted in the second round after introducing minor modifications to the instruments, the overall degree of agreement increased slightly (one item reached a CVI-I of 1). 3.2.3. Open Comments The majority of the responses given to the final open question did not refer to the model or the instruments’ contents but to the difficulties of implementing them in non- specialised environments. These included comments on great care pressure, time constrains (both at primary care and hospital levels), fragmented work dynamics, and, as one expert put it, “the need for awareness, information, and specific training and practice”. In sum, the HexCom model is perceived as a good instrument for research in as much as “the model proposes an approach from ‘the whole’, taking into account all aspects that need to be considered, which facilitates monitoring and decision making”. Some experts pointed out the “perils” of forgetting that, before administering the instruments, it is necessary to create a bond and respond to emerging needs. 4. Discussion Establishing a consensus on the complexity of care in a field of clinical practice such as palliative care is not exempt from difficulties [62]. Aspects such as the elements that make it up, the criteria that define it, the best tools for detection and management or its adaptation to a certain healthcare field are still controversial [39]. Thus, studies using the Delphi method in palliative care [44,63] reach levels of agreement lower than in other fields, where up to 100% consensus is achieved in all items in the second round. Healthcare 2021, 9, 165 10 of 18 In our study, a group of highly qualified experts in palliative care reached an agreement that exceeds 92% with respect to the face validity of the HexCom model and more than 86% in relation to the utility of the different instruments that result from it. It can be argued that this high degree of consensus is associated with the systemic approach [11] underlying the HexCom model, which allows the detection of unmet needs and understands complexity not only as a multidimensional construction, but also a multireferential construction, taking into account the training of the diverse healthcare teams involved in complexity assessment [34]. The HexCom model is aligned with the majority of instruments validated both inter- nationally [25–29,32], as well as in the context of Spain [33], yet its high perceived usability may be explained by its capacity to broad the scope of application beyond oncological and hospitalised patients [37–39]. However, as the panel noted in their open comments, imple- menting the model in clinical environments characterised by considerable time constrains, amongst other barriers, is certainly challenging. Exploring efficient ways to integrate the model in these settings, taking into account not only “experts” but also potential users’ perspectives, emerges as a research and policy priority. Arguably, HexCom is appreciated by a wide range of professionals in so far as it improves the responsiveness of instruments such as the PCSPS (palliative care problem severity score), which only considers four areas and refers to features closer to assessing severity rather than complexity [26]. It can also be argued that HexCom is perceived to present certain advantages with respect to the needs assessment tool (NAT: PD-C) [29,31,32] since the former introduces a specific ethical area. Given that the HexCom model identifies complexity on every case where at least one of its dimensions is defined as highly complex, it can be argued that it is valued as more sensitive than similar tools such as PALCOM [33,34], which identifies a patient’s complexity only as a result of a summation of all of its areas. Similarly, it may be perceived as more patient-oriented than IDC-PAL® in as much as it is based on patients’ needs, rather than a set of risk factors [20–22]. Limitations of the Study A limitation in the Delphi technique is the bias risk in selecting the group of experts. The homogenization criterion (having expertise in home care) may have produced an approach with little representation, potentially limiting the breadth of experience included in the panel. To overcome this threat, the coordinating group consisted of a multidis- ciplinary group, with extensive and divergent experience, including psychologists and social workers, who are often not represented in other validation processes. However, the elevated interest in participating in the study (93% of the invitees participated) suggests that the participants are very representative of this professional community [56,64]. Even though heterogeneity between this sample was sought by including the different profes- sions involved in home care, it could have been greater if patients and relatives had been somehow involved in the study [63]. Another potentially problematic issue when carrying out Delphi studies is the way in which consensus is defined and operationalised, since it varies considerably amongst different studies [60], particularly in face-validity studies. To address this limitation, a quantitative criteria and established recommendations were followed (CVI-I and the number of experts in disagreement) combined with qualitative data to understand disagreement and enhance the consistency of the final results [47]. 5. Conclusions This study stems from the need to establish a shared understanding and widely accepted tools for capturing complexity as a critical step towards enacting coordinated and patient-oriented palliative care [33]. At a conceptual level, the HexCom model fulfils the prerequisites proposed by Pask, outlined drawing on patients’ and professionals’ experiences, in order to capture complexity [11]. These entail addressing it systematically; understanding the person as a whole; addressing it at the patient level (in a personal clinical visit and collecting clinical data); identifying needs and resources (resilience and Healthcare 2021, 9, 165 11 of 18 support networks included); defining levels of complexity; and using sensitive instruments that facilitate concrete interventions. This study complements the model’s robustness by validating HexCom face validity for the broad majority of its targeted end-users, namely healthcare professionals from different backgrounds. These positive results consolidate the broader project of validating this model, which will be followed by assessing the test–retest reliability, validity of criteria, concomitant and predictive reliability, as well as sensitivity and feasibility [65]. Accordingly, HexCom model is expected to be increasingly and widely adopted as a shared and valuable tool for needs assessment in palliative care, posing new challenges both for the model’s implementation and established practices and existing limitations for delivering optimal palliative care. Supplementary Materials: The following are available online at https://www.mdpi.com/2227-903 2/9/2/165/s1: Instruments S1: HexCom Instruments, English versions; Figure S2: HexCom Figure; Instruments S3: HexCom Instruments and figure, original version (Catalan). Author Contributions: Conceptualization: X.B.-D., E.M.J.-Z., and N.O.-P.; methodology, E.M.-G., N.O.-P.; software, J.M.M.-D.; validation, E.M.-G. and N.O.-P.; formal analysis, J.M.M.-D., X.B.-D.; investigation, A.M.-R., S.M.-M., J.L.-E. and X.B.-D.; resources, P.T.-M., N.S.-T.; data curation, E.M.-L., X.B.-D.; writing—original draft preparation, S.C.-R., X.B.-D.; writing—review and editing, E.M.-G., L.L.-L., and X.B.-D.; visualization, M.T.-P.; supervision, O.B.-d.l.R., P.T.-M.; project admin P.T.-M. All authors have read and agreed to the published version of the manuscript.” Funding: This research was funded by INSTITUT CATALÀ DE LA SALUT, grant number 7Z19/008. Institutional Review Board Statement: The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of IDIAP Fundació Jordi Gol (protocol code P15/171, approved 3rd December 2015). Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Data Availability Statement: The data presented in this study are available on request from the corresponding author. Acknowledgments: The authors would like to thank the following professionals for participating in the group of experts, from psychology: Marta López Elias, Jordi Royo, and Meritxell Naudeillo; from social work: Montse Metlikovtz, Agustin Pardo, and Mercè Riquelme; from nursing: Sònia Barrios, Jusa Adell, Lourdes Guanter, and Mercè Taló; from medicine: Cristina Charlez, Xavier Costa, Remei Tell, Miquel Domènech, and Merche Liras. They would also like to thank Xus Micó for administrative support, secretary of the PADES team in Granollers (Catalan Health Institute); and Cristina Pujol, for formal aspects and design. Conflicts of Interest: The authors declare no conflict of interest. Appendix A Table 1. Questionnaire to evaluate face validity of the HexCom model and the different associated instruments for complexity assessment. MODEL SD(1) D(2) I(3) A(4) SA(5) 2 The backbone of palliative care has to be the level of complexity. Complexity is determined by the needs and resources of the patient-family unit and 3 by the decisive capacity of the service that serves them. The HexCom model proposes a systematic approach to complexity, understanding 4 the person as a whole, and at the same time collecting needs and resources. The division in six areas that HexCom makes for the needs of the patient/family: 5 clinical, psycho-emotional, spiritual, sociofamily, ethics, and death-related. Dividing patient/family resources into five systemic areas: microsystem, 6 mesosystem, chronosystem, ecosystem and macrosystem. Healthcare 2021, 9, 165 12 of 18 Table 1. Cont. MODEL SD(1) D(2) I(3) A(4) SA(5) 7 The grading of levels of complexity proposed by the model (low, medium, and high). HexCom’s proposal for intervention (a benchmarking team for low complexity, 8 specialised team with intensity agreeing with the average complexity and specialised team with high intensity for high complexity). HexCom as a model of care can favourably affect the quality of care for people with 9 advanced illness and/or at the end of life. The use of HexCom favours teamwork by offering a common language between 10 different disciplines. The content of HexCom is understandable by the professionals of the different 11 disciplines that intercede in palliative care. The use of HexCom helps strengthen consensus among professionals involved in the 12 care of the case. The content of HexCom is understandable by non-specialised persons in 13 palliative care. The different instruments that are deployed from the model can respond to the 14 whole range of needs that can be found in the different services/teams. 15 The model seems applicable to all healthcare levels. HEXCOM-RED HexCom-Red as a tool for the management and derivation of complex cases can 16 favourably affect the management of care for people with advanced and/or end-of-life illness. 17 HexCom-Red offers information in a clear and visual way. The format of HexCom-Red is adequate in terms of letter type, measurement, and 18 distribution of information in space. The distribution of information in the grid of HexCom-Red facilitates it being filled 19 in easily. 20 The content of HexCom-Red is adequate in terms of its extent. The explanatory drafting for each area of HexCom-Red is clear enough and 21 understandable for people not specialised in palliative care. The levels of complexity and interpretation of grading at three levels seem adequate 22 and appropriate for the management of complex cases. The team intervention criteria for specialised teams according to the level of 23 complexity seem adequate and appropriate for the management of complex cases. HEXCOM-BASIC HexCom-Basic as a tool for comprehensive assessment of needs and resources can 24 favourably improve care for people with advanced illness and/or end-of-life illness. 25 HexCom-Basic offers information in a clear and visual way. The format of HexCom-Basic is adequate in terms of letter type, measurement, and 26 distribution of information in space. The distribution of information in the grid of HexCom-Basic facilitates it being filled 27 in easily. 28 The content of HexCom- Basic is adequate in terms of its extent. The explanatory drafting for each area of systemic resources in HexCom-Basic is 29 clear enough and understandable for people not specialised in palliative care. The levels of strength and interpretation of the three-level grading proposed in 30 HexCom-Basic seem adequate and appropriate for the assessment of the resources and strengths of the patient/family. Healthcare 2021, 9, 165 13 of 18 Table 1. Cont. MODEL SD(1) D(2) I(3) A(4) SA(5) The number of systems with a high level of strength seems a good indicator of the 31 strength of the patient/family. The criterion proposed in HexCom- Basic that three or more systems with high 32 strength can reduce the final complexity by one level seems adequate. HEXCOM-CLIN The HexCom-Clin as a tool for the comprehensive assessment of needs and 33 resources for specialised teams trained in its use can favourably affect the improvement of care for people with advanced illness and/or end-of-life illness. 34 HexCom-Clin offers information in a clear and visual way. The format of HexCom-Clin is adequate in terms of letter type, measurement, and 35 distribution of information in space. The distribution of information in the grid of HexCom-Clin facilitates it being filled 36 in easily. 37 The content of HexCom-Clin is adequate in terms of its extent. The explanatory drafting for each sub-area of needs and resources of HexCom-Clin 38 is clear enough and understandable for people not specialised in palliative care. HEXCOM-FIGURE HexCom-Figure as a tool for the comprehensive assessment of needs and resources 39 for specialised teams trained in its use can favourably affect the improvement of care for people with advanced illness and/or end-of-life illness. 40 HexCom-Figure offers information in a clear and visual way. The format of HexCom-Figure is adequate in terms of letter type, measurement, and 41 distribution of information in space. The distribution of information in the grid of HexCom-Figure facilitates it being 42 filled in easily. 43 The content of HexCom-Figure is adequate in terms of its extent. The explanatory drafting for each sub-area of needs and resources of 44 HexCom-Figure is clear enough and understandable for people not specialised in palliative care. HexCom-Figure as a tool for the comprehensive assessment of needs and resources for specialised teams trained in its use can favourably affect the improvement of care 45 for people with advanced illness and/or end-of-life illness when incorporating the initial and final evaluation, after release. HEXCOM-PATIENT HexCom-Patient as a interview guide for comprehensive assessment of needs and 46 resources of patients can favourably improve care for people with advanced illness and/or end-of-life illness. 47 HexCom-Patient offers information in a clear and visual way. The format of HexCom-Patient is adequate in terms of letter type, measurement, and 48 distribution of information in space. The distribution of information in the grid of HexCom-Patient facilitates it being 49 filled in easily. 50 The content of HexCom-Patient is adequate in terms of its extent. The explanatory drafting for each area of needs and resources of HexCom-Patient is 51 clear enough and understandable for people not specialised in palliative care. 52 The sections with which the information is distributed are appropriate. Healthcare 2021, 9, 165 14 of 18 Table 1. Cont. MODEL SD(1) D(2) I(3) A(4) SA(5) The questions formulated compile the main contents to understand the needs and 53 resources/strengths of the patient. The questions are formulated in a clear and understandable language so that they 54 can be understood by the patient. Collecting the perception of the passage of time is useful in order to have a global 55 approach on the experience of suffering by the patient. The indications of the NOTE of not doing the interview if there is denial, refractory 56 existential anguish, delirium or cognitive impairment seem appropriate. Introducing two open questions to understand the main problem and what is most 57 helpful to the patient favours empathy with them and can help give more appropriate indications. Grading the discomfort as low, medium, or high and its equivalence with VAS seem 58 adequate and appropriate, and it may be useful for complementing the interview. The strength gradation as a little, quite a lot, a lot, and its equivalence with inverse 59 VAS seems adequate and appropriate, and it may be useful for complementing the interview. HEXCOM TRANSLATION The Spanish version of the different instruments is the same as the original Catalan 60 sense. SD = Strongly disagree, D = Disagree, I = Indifferent, A = Agree, SA = Strongly agree. Appendix B Table A2. Modifications to the documents based on the responses of the group of experts. Model (Items 2–15) The word “changing” is introduced in relation to needs (Item 3). Comprehensibility for non-specialists is reviewed (Item 13). Not included are: A subsystem of cultural beliefs and values in macrosystem. This is justified above all because of the need for simplicity and it is considered that these are included as “values” in the micro-system and in the mesosystem community bonds (Item 6). Grade need in 4 levels. Justified so as not to break the initial model (Item 7). HexCom-Red (Items 16–23) “and/or end of life” added in the title (Item 16). The word “Discomfort” is added to improve comprehensibility (Item 21). The concept of “unadaptative emotionality” (intense, persistent emotion, which interferes with relationships and functionality), “rigid personality trait” (not adaptable to change) and “denial of the LDS” (unadaptative) (Item 21) is explained. Not included are: Put more space to specify the needs that cause discomfort. Justified by the need for simplicity (Item 19). Healthcare 2021, 9, 165 15 of 18 Table A2. Cont. HexCom-Basic (Items 24–32) Emphasis is placed on the intuitive and non-mathematical character of the final complexity balance “In principle, consider if more than half of the systems with high strength (H) could reduce the level of final complexity by one...) (Item 32). Not included are: Reduce the examples of systemic strengths. Justified by the comprehensibility of some little known concepts (Item 29). Modify the explanatory text for the strength systems, since there are no alternatives (Item 29). Grade the strengths the opposite way around. Justified to highlight precisely this opposite effect of the strengths (Item 30). Prioritise the micro-system. Justified by the need to consider all areas of strength alike (Item 30). HexCom-Clin (Items 33–38) The list of risk criteria for mourning is removed. Justified by the extension and because this instrument is for specialists (Item 37). HexCom-Figure (Items 39–45) The affiliation data in the central space is removed and the type is made larger (Item 41). Instructions on how to fill it in are included on the back (Item 44). HexCom-Patient (Items 46–59) Notes. New note about the need for a link and to respond to emerging needs (Item 46). Informal verb forms are maintained, for the primary need of creating a bond (Item 46). In the psycho-emotional area, “what degree of discomfort would you say” is substituted for “to what degree” (Item 51). In the “passing of time” area “with which you would relate” replaced by “with what” (Item 51). A note is placed on the limits of the question about the passage of time, which sometimes discriminates the ability to diversify attention and sources of gratification rather than suffering (Item 55). The VAS scales are included in the initial version (Items 58 and 59). Not included are: The consideration of changing the words sustaining and breaking in the spiritual sphere. Justified by the same concept of spiritual discomfort: biographical breakdown relates to lack of meaning and relational failure with lack of connection (Item 51). Translation into Spanish (Item 60) The questions are put right, “they produce discomfort” is added, and the psycho-emotional text is changed (Item 60). Notes: A review is made of the formal aspects, fonts, spaces, and distribution by an external person outside the coordinating group and a graphic designer reviews all the instruments (Items 18, 28, 35, 41, 48, 49, and 50). The formal and content changes are passed from one instrument to the other. References 1. McNamara, B.; Rosenwax, L.K.; Holman, C.D.A.J. A method for defining and estimating the palliative care population. J. Pain Symptom Manag. 2006, 32, 5–12. [CrossRef] [PubMed] 2. Kar, S.; Zengin, N. The relation between self-efficacy in patients with chronic obstructive pulmonary disease and caregiver burden. Scand. J. Caring Sci. 2019, 754–761. [CrossRef] [PubMed] 3. OECD. Preventing Ageing Unequally; OECD Publishing: Paris, France, 2017. 4. Etkind, S.N.; Bone, A.E.; Gomes, B.; Lovell, N.; Evans, C.J.; Higginson, I.J.; Murtagh, F.E.M. How many people will need palliative care in 2040? Past trends, future projections and implications for services. BMC Med. 2017, 15, 102. [CrossRef] [PubMed] 5. Gómez-Batiste, X.; Martínez-Muñoz, M.; Blay, C.; Amblàs, J.; Vila, L.; Costa, X.; Espaulella, J.; Espinosa, J.; Constante, C.; Mitchell, G.K. Prevalence and characteristics of patients with advanced chronic conditions in need of palliative care in the general population: A cross-sectional study. Palliat. Med. 2014, 28, 302–311. [CrossRef] [PubMed] 6. Blay, C.; Martori, J.C.; Limón, E.; Oller, R.; Vila, L.; Gómez-Batiste, X. Busca tu 1%: Prevalencia y mortalidad de una cohorte comunitaria de personas con enfermedad crónica avanzada y necesidades paliativas. Aten. Primaria 2017, 51, 71–79. [CrossRef] Healthcare 2021, 9, 165 16 of 18 7. Murtagh, F.E.; Bausewein, C.; Verne, J.; Groeneveld, E.I.; Kaloki, Y.E.; Higginson, I.J. How many people need palliative care? A study developing and comparing methods for population-based estimates. Palliat Med. 2014, 28, 49–58. [CrossRef] 8. Connor, S.; Gómez-Batiste, X. Assesing the need for palliative care in populations and contexts. In Building Integrated Palliative Care Programs and Service, 1st ed.; Gómez-Batiste, X., Connor, S., Eds.; World Hospice Palliative Care Alliance: Barcelona, Spain, 2017; pp. 79–92. 9. Groeneveld, E.I.; Cassel, J.B.; Bausewein, C.; Csikós, Á.; Krajnik, M.; Ryan, K.; Haugen, D.F.; Eychmueller, S.; Keller, H.G.; Allan, S.; et al. Funding models in palliative care: Lessons from international experience. Palliat. Med. 2017, 31, 296–305. [CrossRef] 10. Generalitat de Catalunya. Departament de Salut. Pla de Salut 2016–2010; Direcció General de Planificació en Salut: Barcelona, Spain, 2016. 11. Pask, S.; Pinto, C.; Bristowe, K.; van Vliet, L.; Nicholson, C.; Evans, C.J.; George, R.; Bailey, K.; Davies, J.M.; Guo, P.; et al. A framework for complexity in palliative care: A qualitative study with patients, family carers and professionals. Palliat. Med. 2018, 32, 1078–1090. [CrossRef] 12. Carduff, E.; Johnston, S.; Winstanley, C.; Morrish, J.; Murray, S.A.; Spiller, J.; Finucane, A. What does ‘complex’ mean in palliative care? Triangulating qualitative findings from 3 settings. BMC Palliat. Care 2018, 17, 12. [CrossRef] 13. Amblàs-Novellas, J.; Espaulella-Panicot, J.; Inzitari, M.; Rexach, L.; Fontecha, B.; Romero-Ortuno, R. En busca de respuestas al reto de la complejidad clínica en el siglo XXI: A propósito de los índices de fragilidad. Rev. Esp. Geriatr. Gerontol. 2017, 52, 159–166. [CrossRef] 14. Shippee, N.D.; Shah, N.D.; May, C.R.; Mair, F.S.; Montori, V.M. Cumulative complexity: A functional, patient-centered model of patient complexity can improve research and practice. J. Clin. Epidemiol. 2012, 65, 1041–1051. [CrossRef] [PubMed] 15. Safford, M.M.; Allison, J.J.; Kiefe, C.I. Patient complexity: More than comorbidity. The vector model of complexity. J. Gen. Intern. Med. 2007, 22, 382–390. [CrossRef] 16. Schaink, A.K.; Kuluski, K.; Lyons, R.F.; Fortin, M.; Jadad, A.R.; Upshur, R.; Wodchis, W.P. A scoping review and thematic classification of patient complexity: Offering a unifying framework. J. Comorbidity 2012, 2, 1–9. [CrossRef] [PubMed] 17. Hegelund, A.; Andersen, I.C.; Andersen, M.N.; Bodtger, U. The impact of a personalised action plan delivered at discharge to patients with COPD on readmissions: A pilot study. Scand. J. Caring Sci. 2019, 4, 909–918. [CrossRef] 18. Radbruch, L.; Payne, S. White paper on standards and norms for hospice and palliative care in Europe: Part 1. Eur. J. Palliat. Care 2009, 16, 278–289. 19. Instituto de Información Sanitaria. Atención a Los Cuidados Paliativos. Organización en Las Comunidades Autónomas. Madrid. 2010. Available online: https://www.mscbs.gob.es/estadEstudios/estadisticas/docs/siap/Atencion_a_los_Cuidados_ Paliativos_y_Dispositivos-2010.pdf (accessed on 21 September 2020). 20. Martin Rosello, M.L. Validación y Desarrollo del IDC-PAL, Instrumento Diagnóstico de la Complejidad en Cuidados Paliativos. PhD Thesis, Universidad de Málaga, Malaga, Spain, 2017. 21. Fernández-López, A.; Sanz-Amores, R.; Cía-Ramos, R.; Boceta-Osuna, J.; Martín-Roselló, M.L.; Duque-Granado, A.; Ollero-Baturone, J.M. Criterios y niveles de complejidad en Cuidados Paliativos. Med. Paliat. 2008, 15, 287–292. 22. Martín-Utrilla, S.; Villasana-Sarabia, A.; Gallart-Gomez, P.; Mancheno-Alvaro, A. Diagnóstico de la complejidad de la atención paliativa en hospitalización a domicilio versus hospitalización convencional. Therapeía 2016, 8, 31–45. 23. Comino, M.R.S.; Torres, N.G.; Cejudo, I.P.; Roselló, M.L.M.; García, V.R.; López, A.F. El valor del instrumento diagnostico de la complejidad en cuidados paliativos para identificar la complejidad en pacientes tributarios de cuidados paliativos. Med. Paliat. 2017, 24, 196–203. [CrossRef] 24. Comino, M.; Garcia, V.; López, M.; Feddersen, B.; Roselló, M.; Sanftenberg, L.; Schelling, J. Evaluation des spanischen Diagnostik- Tools für die Komplexität in der Palliativmedizin, IDC-Pal, durch Hausärzte in Deutschland: Eine Pilotstudie. Das Gesundh. 2018, 80, 871–877. [CrossRef] [PubMed] 25. Scandrett, K.G.; Reitschuler-Cross, E.B.; Nelson, L.; Sanger, J.A.; Feigon, M.; Boyd, E.; Chang, C.-H.; Paice, J.A.; Hauser, J.M.; Chamkin, A.; et al. Feasibility and effectiveness of the NEST13+ as a screening tool for advanced illness care needs. J. Palliat. Med. 2010, 13, 161–169. [CrossRef] [PubMed] 26. Masso, M.; Allingham, S.F.; Johnson, C.E.; Pidgeon, T.; Yates, P.; Currow, D.; Eagar, K. Palliative Care Problem Severity Score: Reliability and acceptability in a national study. Palliat. Med. 2016, 30, 479–485. [CrossRef] 27. Martin-Rosello, M.L.; Sanz-Amores, M.R.; Salvador-Comino, M.R. Instruments to evaluate complexity in end-of-life care. Curr. Opin. Support. Palliat. Care 2018, 12, 480. [CrossRef] 28. Rainbird, K.J.; Perkins, J.J. The Needs Assessment for Advanced Cancer Patients (Na-Acp): A Measure of the Perceived Needs of Patients with Advanced, Incurable Cancer. A Study of Validity, Reliability and Acceptability. Psychooncology 2005, 306, 297–306. [CrossRef] [PubMed] 29. Waller, A.; Girgis, A.; Currow, D.; Lecathelinais, C. Development of the palliative care needs assessment tool (PC-NAT) for use by multi-disciplinary health professionals. Palliat. Med. 2008, 22, 956–964. [CrossRef] 30. Waller, A.; Girgis, A.; Lecathelinais, C.; Scott, W.; Foot, L.; Sibbritt, D.; Kelly, B.; Johnson, C.; Gorton, E.; Kristjanson, L.; et al. Validity, reliability and clinical feasibility of a Needs Assessment Tool for people with progressive cancer. Psychooncology 2010, 733, 726–733. [CrossRef] Healthcare 2021, 9, 165 17 of 18 31. Waller, A.; Girgis, A.; Johnson, C.; Lecathelinais, C.; Sibbritt, D.; Forstner, D.; Liauw, W.; Currow, D.C. Improving outcomes for people with progressive cancer: Interrupted time series trial of a needs assessment intervention. J. Pain Symptom Manag. 2012, 43, 569–581. [CrossRef] [PubMed] 32. Girgis, A.; Waller, A. Palliative care needs assessment tools. In Oxford Textbook of Palliative Medicine; Cherry, N., Fallon, M., Kaasa, S., Portenoy, R.K., Currow, D.C., Eds.; Oxford University Press: Oxford, UK, 2015; pp. 363–375. 33. Waller, A.; Girgis, A.; Davidson, P.M.; Newton, P.J.; Lecathelinais, C.; Macdonald, P.S.; Hayward, C.S.; Currow, D.C. Facilitating needs-based support and palliative care for people with chronic heart failure: Preliminary evidence for the acceptability, inter-rater reliability, and validity of a needs assessment tool. J. Pain Symptom Manag. 2013, 45, 912–925. [CrossRef] [PubMed] 34. Tuca, A.; Gómez-Martínez, M.; Prat, A. Predictive model of complexity in early palliative care: A cohort of advanced cancer patients (PALCOM study). Support Care Cancer 2018, 26, 241–249. [CrossRef] [PubMed] 35. Codorniu, N.; Tuca, A. Complexity levels of interventions in a hospital support team: Descriptive study about prevalence, clinical characteristics and team organisation for each level. Palliat. Med. 2004, 18, 303–393. 36. Generalitat de Catalunya Departament de Salut. Descripció i Consens Dels Criteris de Complexitat Assistencial i Nivells D’intervenció en L’atenció al Final de la Vida; Direcció General de Planificació i Avaluació: Barcelona, Spain, 2010. 37. Ferris, F.D.; Balfour, H.M.; Bowen, K.; Farley, I.; Hardwick, M.; Lamontagne, C.; Lundy, M.; Syme, A.; West, P.J. A model to guide patient and family care: Based on nationally accepted principles and norms of practice. J. Pain Symptom Manag. 2002, 24, 106–123. [CrossRef] 38. Esteban-Pérez, M.; Grau, I.C.; Castells Trilla, G.; Bullich Marín, Í.; Busquet Duran, X.; Aranzana Martínez, A.; Besora Torradeflot, M.I.; Picaza Vila, J.M.; Tuca Rodríguez, A.; Valverde Vilabella, E. Complejidad asistencial en la atención al final de la vida: Criterios y nivelesde intervención en atención comunitaria de salud. Med. Paliativa 2015, 22, 69–80. [CrossRef] 39. Esteban-Pérez, M.; Fernández-Ballart, J.; Boira Senlí, R.M.; Martínez Serrano, T.; Nadal Ventura, S.; Castells Trilla, G. Concordancia entre la complejidad observada desde diferentes niveles asistenciales en pacientes crónicos complejos, con enfermedad avanzada o al final de la vida mediante un modelo de abordaje de la complejidad. Med. Paliativa 2018, 25, 236–244. [CrossRef] 40. Busquet-Duran, X.; Jiménez-Zafra, E.M.; Manresa-Domínguez, J.-M.; Tura-Poma, M.; Bosch-delaRosa, O.; Moragas-Roca, A.; Galera Padilla, M.C.; Martin Moreno, S.; Martínez-Losada, E.; Crespo-Ramírez, S.; et al. Describing Complexity in Palliative Home Care Through HexCom: A Cross-Sectional, Multicenter Study. J. Multidiscip. Healthc. 2020, 13, 297–308. [CrossRef] [PubMed] 41. Weissman, D.E.; Meier, D.E. Identifying patients in need of a palliative care assessment in the hospital setting: A consensus report from the Center to Advance Palliative Care. J. Palliat. Med. 2011, 14, 17–23. [CrossRef] 42. Jünger, S.; Payne, S.A.; Brine, J.; Radbruch, L.; Brearley, S.G. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat. Med. 2017, 31, 684–706. [CrossRef] [PubMed] 43. Red Española de Agencias de Evaluación de Tecnologías Sanitarias y Prestaciones del SNS. Elaboración de Guías de Práctica Clínica en el Sistema Nacional de Salud. Available online: https://redets.sanidad.gob.es/ (accessed on 18 October 2020). 44. Rayner, L.; Price, A.; Hotopf, M.; Higginson, I.J. Expert opinion on detecting and treating depression in palliative care: A Delphi study. BMC Palliat. Care 2011, 10, 10. [CrossRef] [PubMed] 45. Biondo, P.D.; Nekolaichuk, C.L.; Stiles, C.; Fainsinger, R.; Hagen, N.A. Applying the Delphi process to palliative care tool development: Lessons learned. Support. Care Cancer 2008, 16, 935–942. [CrossRef] [PubMed] 46. Hardesty, D.M.; Bearden, W.O. The use of expert judges in scale development. Implications for improving face validity of measures of unobservable constructs. J. Bus. Res. 2004, 57, 98–107. [CrossRef] 47. Oriol-Peregrina, N. Bioètica en la Presa de Decisions: Adaptacions Dietètiques en la Disfàgia. PhD Thesis, Universitat de Barcelona, Barcelona, Spain, 2018. 48. Moores, K.L.; Jones, G.L.; Radley, S.C. Development of an instrument to measure face validity, feasibility and utility of patient questionnaire use during health care: The QQ-10. Int. J. Qual. Heal. Care 2012, 24, 517–524. [CrossRef] 49. Torrado-Fonseca, M.; Reguant-Álvarez, M. El metodo Delphi. Reire. Rev. D’innovacio Recer. Educ. 2016, 9, 87–102. [CrossRef] 50. Kidholm, K.; Jensen, L.K.; Kjølhede, T.; Nielsen, E.; Horup, M.B. Validity of the Model for Assessment of Telemedicine: A Delphi study. J. Telemed. Telecare 2018, 24, 118–125. [CrossRef] [PubMed] 51. Pernambuco, L.D.A.; Espelt, A.; Magalhães Júnior, H.V.; Cavalcanti, R.V.A.; Lima, K.C. Screening for Voice Disorders in Older Adults (Rastreamento de Alterações Vocais em Idosos—RAVI)—Part I: Validity Evidence Based on Test Content and Response Processes. J. Voice 2016, 30, 246.e9–246.e17. [CrossRef] [PubMed] 52. Pernambuco, L.; Espelt, A.; Magalhães Junior, H.V.; Lima, K.C. de Recomendações para elaboração, tradução, adaptação transcultural e processo de validação de testes em Fonoaudiologia. CoDAS 2017, 29, 2–5. [CrossRef] [PubMed] 53. Oriol-Peregrina, N. Cuestionario de adherencia para las adaptaciones dietéticas en disfagia. Rev. Investig. Logop. 2018, 8, 91–114. [CrossRef] 54. Hui, D.; Mori, M.; Watanabe, S.M.; Caraceni, A.; Strasser, F.; Saarto, T.; Cherny, N.; Glare, P.; Kaasa, S.; Bruera, E. Referral criteria for outpatient specialty palliative cancer care: An international consensus. Lancet Oncol. 2016, 17, e552–e559. [CrossRef] 55. Landeta, J. El método Delphi. In Una Técnica de Previsión de la Incertidumbre; Ariel: Barcelona, Spain, 1999. 56. Dickman, A.; Bickerstaff, M.; Jackson, R.; Schneider, J.; Mason, S.; Ellershaw, J. Identification of drug combinations administered by continuous subcutaneous infusion that require analysis for compatibility and stability. BMC Palliat. Care 2017, 16, 22. [CrossRef] Healthcare 2021, 9, 165 18 of 18 57. Torabizadeh, C.; Yousefinya, A.; Zand, F.; Rakhshan, M.; Fararooei, M. A nurses’ alarm fatigue questionnaire: Development and psychometric properties. J. Clin. Monit. Comput. 2017, 31, 1305–1312. [CrossRef] 58. Polit, D.F.; Beck, C.T.; Owen, S.V. Is the CVI an acceptable indicator of content validity? Res. Nurs. Health 2007, 31, 341–354. [CrossRef] 59. Sireci, S.; Faulkner-Bond, M. Validity evidence based on test content. Psicothema 2014, 26, 100–107. [CrossRef] 60. Palmieri, P.A. La tecnica Delphi: Un método de consenso para la investigación en servicios de salud en Latino América. Ágora Rev. Científica 2018, 4, 1–9. [CrossRef] 61. Varela-Ruiz, M.; Díaz-Bravo, L.; García-Durán, R. Descripción y usos del método Delphi en investigaciones del área de la salud. Rev. Investig. Educ. Médica 2012, 1, 90–95. 62. Wallerstedt, B.; Benzein, E.; Schildmeijer, K.; Sandgren, A. What is palliative care? Perceptions of healthcare professionals. Scand. J. Caring Sci. 2019, 33, 77–84. [CrossRef] [PubMed] 63. Downing, J.; Knapp, C.; Muckaden, M.A.; Fowler-Kerry, S.; Marston, J. Priorities for global research into children’s palliative care: Results of an International Delphi Study. BMC Palliat. Care 2015, 14, 1–10. [CrossRef] [PubMed] 64. García Valdés, M.; Suárez, I.M.; Ii, M. Empleo del método Delphi en investigaciones sobre salud publicadas en revistas científicas cubanas. Rev. Cuba. Inf. Cienc. Salud. 2013, 24, 133–144. 65. Carvajal, A.; Centeno, C.; Watson, R.; Martínez, M.; Sanz, A. How is an instrument for measuring health to be validated? . Sist. Sanit. Navar. 2011, 34, 63–72. [CrossRef]