Received: 23 July 2019 DOI: 10.1002/mpr.1850 ORIGINAL ARTICLE - - Revised: 29 May 2020 Accepted: 28 July 2020 Development of a patient rated scale for mental health global state for use during humanitarian interventions Augusto E. Llosa1 | Carmen Martínez‐Viciana2 | Cristina Carreño2 | Stella Evangelidou3 | German Casas4 | Caroline Marquer1 | Marie Rose Moro5,6 | Bruno Falissard6 | Rebecca F. Grais1 1 Epicentre, Paris, France 2 Médecins Sans Frontières, Barcelona, Spain Abstract 3 Department of Psychiatry and Legal Objective: We present the results of a cross‐cultural validation of the Mental Medicine, Faculty of Medicine, Autonomous Health Global State (MHGS) scale for adults and adolescents (<14 years old). University of Barcelona, Barcelona, Spain Methods: We performed two independent studies using mixed methods among 103 4 Facultad de Medicina, Universidad de Los Andes, Bogotá, Colombia patients in Hebron, Occupied Palestinian Territories and 106 in Cauca, Colombia. 5 Médecins Sans Frontières, Université de The MHGS was analyzed psychometrically, sensitivity and specificity, ability to Paris, Paris, France detect clinically meaningful change, compared to the Clinical Global Impression‐ 6 Epidemiology and Population Health, CESP/ Severity scale (CGI‐S). Principal component analysis was used to reduce the number INSERM U1018, Paris, France of questions after data collection. Correspondence Results: The scale demonstrated good internal consistency, with a Cronbach alpha Caroline Marquer, Epicentre, Paris, France. Email: caroline.marquer@epicentre.msf.org score of 0.80 in both settings. Test retest reliability was high, ICC 0.70 (95% CI [0.41–0.85]) in Hebron and 0.87 (95% CI [0.76–0.93]) in Cauca; inter‐rater reliability Funding information was 0.70 (95% CI [0.42–0.85]) in Hebron and 0.76 (95% CI [0.57–0.88]) in Cauca. Médecins Sans Frontières Psychometric properties were also good, and the tool demonstrated a sensitivity of 85% in Hebron and 100% in Cauca, with corresponding specificity of 80% and 79%, when compared to CGI‐S. Conclusions: The MHGS showed promising results to assess global mental health thereby providing an additional easy to use tool in humanitarian interventions. Additional work should focus on validation in at least one more context, to adhere to best practices in transcultural validation. KEYWORDS conflict, global scale, humanitarian intervention, mental health, psychology Abbreviations: CEIC, Comité de Ética para la Investigación Científica; CGI‐S, Clinical Global Impression‐Severity scale; DAS, Disability Assessment Schedule 2.0; GAF, Global Assessment of Functioning; ICC, intra‐class correlation coefficients; KMO, Kaiser‐Meyer‐Olkin; MCID, minimal clinically important difference; MHGS, Mental Health Global State; MHPSS, Mental Health and Psychosocial Support; oPT, Occupied Palestinian Territories; PCA, Parallel Principal Component Analysis; ROC, receiver operating characteristics. This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial‐NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or adaptations are made. © 2020 The Authors. International Journal of Methods in Psychiatric Research published by John Wiley & Sons Ltd. Int J Methods Psychiatr Res. 2021;30:e1850. wileyonlinelibrary.com/journal/mpr 1 of 9 https://doi.org/10.1002/mpr.1850 2 of 9 - LLOSA ET AL. 1 | INTRODUCTION Hebron, MSF has been providing care and mental health support since 1991. The context is characterized as a chronic conflict where Populations exposed to natural and man‐made crises are at increased populations are exposed to conflict related violence as well as limi- risk of developing mental health problems (Fazel, Reed, Panter‐Brick, tations of movement, harassment, demolition of homes and restricted & Stein, 2012; Jones et al., 2009; Miller & Rasmussen, 2010; Reed, access to health care. In Cauca, MSF provided medical care beginning Fazel, Jones, Panter‐Brick, & Stein, 2012). Despite growing evidence in 1985 and mental health support between 2003 and 2016. At the of the benefit of a variety of mental health and psychosocial support time of the study, the population was suffering direct consequences (MHPSS) interventions in these settings, recommendations remain of a prolonged armed conflict, including repeated mass forced‐ largely based on expert opinion rather than robust measurement displacement, movement restrictions and poor access to healthcare (Interagency Standing Committee (IASC), 2006). Relying on expert (Gómez‐Restrepo et al., 2016; Sanchez‐Padilla, Casas, Grais, opinion may be the only possible recourse in situations without Hustache, & Moro, 2009). The most treated conditions were common empirical evidence for recommendations on patient care, but gen- mood and anxiety disorders including trauma and stress related eration of evidence remains imperative to improve MHPSS in- disorders. Patients with severe and chronic mental disorders (e.g., terventions. Additional data are needed to evaluate mental health in schizophrenia and bipolar disorders) were referred directly to part- conjunction with MHPSS interventions, and to promote evidence ners for specialized care, not provided within the MSF intervention driven practices (Charter, 2011; Tol et al., 2011). and none were included in this study. In both contexts, the mental This evidence generation aims to improve and adapt mental health interventions comprised a maximum of 16 psychological ses- health interventions to different crisis‐affected contexts and better sions per patient, delivered by a trained psychologist in as regular address the needs of these populations. intervals as possible, to adapt to patients that might be forced to Mental‐health measures, usually through scales, and particularly displace, or due to movement restrictions. Here, we present the those including the patient's perspective, provide important infor- development and validation studies of a long and short version for mation to improve and adapt mental care (Bolton & Tang, 2002; the MHGS scale for adults and adolescents (14 and above). The Rodin & van Ommeren, 2009; World Health Organisation, 2013, MHGS was psychometrically examined including inter‐rater, test‐ pp. 1–27). These scales, however, are often either disorder specific, retest reliability, and sensitivity to change analyses compared to the modulated around western concepts and populations, lack cross‐ Clinical Global Impression‐Severity scale (CGI‐S). cultural validation, require scoring by specialists, and are too lengthy and cumbersome to implement in many crisis‐affected contexts (Bolton & Tang, 2002; Chiumento, Khan, Rahman, & Frith, 2016; 2 | METHODS Dozio, Bizouerne, Feldman, & Moro, 2018; Van Ommeren, 2003; Viswanath & Chaturvedi, 2012; Wolpert et al., 2012; World Health The tool is interviewer‐administered and aims to capture the pa- Organisation, 2013, pp. 1–27). tient's perspective on their global mental state. The development of In humanitarian settings, especially conflict affected ones, there is the tool included two phases. The first phase included the identifi- sporadic and inconsistent access to patients due to insecurity along cation of the main domains of interest and a list questions to provide with a shortage and inequitable distribution of psychiatrists and psy- information on change in patients receiving the mental health chologists (Charlson et al., 2019; Patel, Chowdhary, Rahman, & Verdeli, intervention. Through focus groups in the community and pilot 2011). There is a clear need for tools and approaches which are easy to testing, understanding of the concepts described behind preselected use, can be easily administered by non‐specialized personnel and help questions were addressed before assessing psychometrics (Lohr assess overall changes in mental health. Despite a large number of et al., 1996). The second phase examined the psychometric proper- mental health status questionnaires, either addressing symptoms or ties of the tool and aimed to simplify the tool to a minimum set of function, few serve to measure global mental health state from the questions. These steps are described below. patient's perspective, and none have specifically addressed the speci- ficities of complex humanitarian conflict contexts (Kirmayer, Guzder, & Rousseau, 2013; Lohr, 1988; Rodin & van Ommeren, 2009). 2.1 | Creation of the list of questions To facilitate the evaluation of MHPSS interventions supported by Médecins Sans Frontières (MSF), we aimed to create a quick and Instrument domains and questions were selected through a versatile mental health global state (MHGS) scale for use in crisis‐ consensus group made up of experts in instrument development and affected populations. We aimed for an instrument that is adaptable validation and cross‐cultural psychology. These included mental to a variety of ages, mental conditions and contexts, which is easy to health professionals involved in the mental health interventions at administer, and possible to be done quickly when contact may be the two project sites (France, Spain, Colombia and oPT). The expert limited and intermittent. As such, we developed a short, easy to group met over the course of several weeks and derived a list of administer questionnaire, which captures the perspective of the pa- potential questions reflecting key domains from their professional tient. Two independent validations were performed, one in Hebron, expertise and previous work in these contexts. Once the list of Occupied Palestinian Territories (oPT) and one in Cauca, Colombia. In questions was developed through consensus, community focus LLOSA ET AL. - 3 of 9 groups in both project sites were held. Questions were then listed by cases where retest interviewers were not possible in person, these domain of interest and their phrasing was considered for simplicity took place by phone. After obtaining informed consent, participants and cross‐cultural interpretability (Kortmann, 1987, 1990; Kortmann were asked demographic questions and administered the MHGS, in & Ten Horn, 1988). A final list of questions was developed after an area offering privacy, by first introducing the questionnaire and follow‐up with additional expert consultation. explaining its purpose and how to rate each question of the MHGS with the aid of pictograms. The Disability Assessment Schedule 2.0 (DAS) was used to measure the patient's overall level of functioning 2.2 | Translation and adaptation of the tool (Üstün, 2010). All questions were read to participants, and their response recorded in study specific forms. Selected questions were translated into local languages (Arabic and The attending psychologist completed routine information for the Spanish) and back‐translated by professional translators in each patient dossier, including the Clinical Global Impression (for overall context. Discrepancies were resolved by consensus and questions severity assessment; CGI‐S) and the Global Assessment of Functioning revised accordingly. The instrument was also piloted in each context (GAF) during the patient's appointment (Hilsenroth et al., 2000; Guy, among professionals and laypersons from June to November 2013. The 1976). These and other study related data points were then trans- instrument was piloted by a dedicated study coordinator and provided ferred from the chart to the study forms by the study coordinator. to individuals working in various professions in the project as well as Patients in the test‐retest assessment subgroup were to be re‐ community members, none of whom participated in the study. Once interviewed by the study coordinator within 48 h. For the inter‐rater developed, professional local partners (members of the Ministries of reliability assessment subgroup, a cohabiting adult relative of the Health, psychiatrist, and psychologist) were asked to check the rele- patient was to be interviewed within 24 h, either by phone or in vance of questions. These steps aimed to assess interpretability of person depending on security constraints. The subgroup participating questions and to ensure that concepts represented are as intended and to the responsiveness testing were to be interviewed at different similarly understood in at least two cultural‐linguistic contexts. scheduled appointments, within five sessions from the study's base- line interview. For the study, the same routine clinic procedures were used. The patients who accepted to participated received one 2.3 | Training reminder call for their next appointment. Interviewers were the psychologists working in the MSF programs where the validation took place (oPT Cauca). They were specifically 2.6 | Data analyses trained over 2 days in interviewing, administration of the tool, the informed consent process, and were supervised by the study coor- Internal consistency of the instrument was assessed by Cronbach dinator (also a psychologist) at each study site. alpha (where alpha values 0.7 ¼ acceptable, 0.8 ¼ good, 0.9 ¼ excellent; DeVellis, 2016). Inter‐rater reliability compared assess- ments by the patient themselves with those of a cohabitating adult 2.4 | Study population and sampling family member. Test‐retest reliability was assessed between scores from the first study related assessment with those of the retest. Both Both contexts were selected due to the conflict affected the pop- reliability measures were estimated using intra‐class correlation co- ulations, and also long‐standing provision of mental health care in an efficients (ICC) and presented with corresponding 95% confidence insecure context. Study participants were recruited among patients intervals and p‐values (F test), following interpretation guidelines by directly affected by the conflict and receiving mental health care at Ciccetti (i.e., <0.40 bad; 0.40–0.59 fair; 0.60–0.74 good; 0.75–1.00 MSF supported clinics. After providing written informed consent, adult excellent; Cicchetti, 1994). Differences in distribution of initial and and adolescent participants, comprising any patient presenting for retest or second‐rater scores were compared with paired t‐tests or mental health and psychosocial care at one of the MSF clinics, were Wilcoxon signed‐rank tests, as appropriate, and p‐values presented. interviewed. Recruitment continued until the target sample size was Convergent validity was assessed comparing the results to CGI‐ reached (100). No differences were expected between those with ap- S, GAF and DAS, through Pearson & Spearman correlation tests, as pointments in different seasons. The baseline measure was considered appropriate depending on distribution of data, and interpreted as the session provided to the patient when included in the study. accordingly (0.1 small; 0.3 medium; 0.5 large; Cohen, 2013). Criterion validity was checked by calculation of sensitivity and specificity against the CGI‐S based on the clinician's assessment through 2.5 | Procedures structured interview. The Kaiser–Meyer–Olkin (KMO) test was used to determine sampling adequacy. A value 0.80 was set as the criteria Testing a minimum of 7–10 patients per question with no less than for exploring structure (34). Parallel principal component analysis 100 participants overall is standard practice (Kline, 2013; Terwee was performed to better understand the relationship between et al., 2007). Interviews were timed with the patient sessions. In questionnaire items (internal construct), compute factor loadings, to 4 of 9 - LLOSA ET AL. eliminate redundancies and specify the number of items for a on Hebron baseline data. The steep drop in factor loadings ei- shortened scale (35). The following goodness of fit statistics were genvalues over 100 replications in parallel principal component considered: chi‐squared test (acceptable model fit if p > 0.05); root analysis (PCA) suggested a single component model. Six questions mean square error of approximation (RMSEA, acceptable model fit if showed notably higher loading factors (superior to 0.30 in PCA; <0.06); comparative fit index (CFI, acceptable model fit if >0.96); Table 1). This model includes two questions (Q) on function (Q2) standardized root mean residual (SRMR, acceptable model fit if performance of daily activities and (Q3) getting along with others; <0.08; Hu & Bentler, 1999). two questions on specific symptoms of suffering (Q11) troubled The ability of the instrument to detect clinically important changes by thoughts or worries, and (Q12) feelings of sadness or over time was assessed by comparing the instrument's measurements nervousness; and two questions related to general well‐being, at least two different time points of the mental health care provided. addressing sleep problems (Q6) and (Q13) a general question on Detected differences were compared to those by another instrument suffering (Table 1). This analysis allowed for a simplified version (CGI‐S). Responsiveness was considered if the Area Under Curve including six questions. Confirmatory factor analysis was carried (AUC) was >0.70 ([CVO] American Psychiatric Association, 2005; out using the 6‐question version of MHGS on the Cauca dataset. Forkmann et al., 2011; Hall, 1995; Williams, 1976). The AUC of Goodness of fit statistics were consistent with the reduced model, receiver operating characteristics (ROC) curve is based on a 3‐point chi‐squared test ¼ 6.96, p ¼ 0.64 (acceptable model fit if p > 0.05); increase of the CGI‐S. The minimal clinically important difference RMSEA ¼ 0.00 (acceptable model fit if <0.6); CFI ¼ 1.00 (MCID) for the simplified version of the scale was calculated using (acceptable model fit if >0.90); excepting SRMR ¼ 0.28 (acceptable linear regression of MHGS against a 1‐point change in CGI‐S. model fit if <0.08). Results that follow are for the reduced version Data were analyzed using STATA SE 12.0 (STATA Corporation). of the instrument. The protocols received approval from ethics review boards of MSF, the University of Cauca, Colombia, and the Ministry of Health, oPT. All participants were already receiving free mental health care 3.2 | MHGS study participants in MSF programs, and were further referred to partners for psychi- atric and other medical care if needed. In Hebron, 103 participants were enrolled between January and May 2014. In Cauca, 106 participants were enrolled between December 2013 and August 2014. The majority of participants were female in 2.7 | Measures both Hebron and Cauca with a median age of 32 in Hebron and 33 in Cauca (Table 2). The domains selected were psychological functioning (cognitive In Hebron, most study participants were already in treatment at ability, interpersonal relation, and ability to perform daily activities), enrollment (median four sessions). Median of therapeutic session at symptoms of psychological suffering (somatizations, negative follow‐up was eight (n ¼ 30), with a median of five intra‐study thoughts or emotions, and difficulties in sleeping and eating) and therapeutic sessions (range 3–9). All participants responded directly general perception of psychological well‐being or suffering. Two to the interviewer (i.e., no proxy responders). Most (n ¼ 88, 85%) questions on coping mechanisms were also added as this was an area were in individual therapy. The most common principal diagnosis at perceived to be of therapeutic importance to MSF. One question on enrollment were other anxiety disorders (n ¼ 32, 31%), Post‐trau- perceived usefulness of the mental health and psychosocial sessions matic Stress Disorder (PTSD; n ¼ 20, 19%) and depression (n ¼ 16, was included but not intended to be part of the assessment scale. 16%). The complete version of MHGS was composed of 13 questions In Cauca, most study participants were on their first session at for adults and adolescents. The patient had to quote or point to the enrollment, with median of one session (n ¼ 106) and three sessions respective image for “not at all”, “a little,” “some,” “fair amount,” or “a at follow‐up (n ¼ 31). The median number of intra‐study therapeutic lot” using the following scale (Figure 1). The short version of MHGS sessions was two (range 2–3). Most participants (n ¼ 104, 98%) included a total of six questions with a minimum score of 1 (“not at responded themselves. Proxy responders were husband (n ¼ 1) or all”) and maximum of 5 (“a lot”) per question (5‐point Likert scale), mother (n ¼ 1), care takers of the patients who were not able to thus total scores for the scale range between 6 and 30 points, with answer directly. Of the 106 participants, 97 (92%) were enrolled in higher scores indicating greater difficulties (Figure 2). individual therapy. The most frequent diagnoses at enrollment were depression (n ¼ 29, 27%), PTSD (n ¼ 11, 10%) and prolonged bereavement (n ¼ 13, 12%). 3 | RESULTS 3.1 | Scale structure and item reduction 3.3 | Psychometrics properties Following the psychometrics properties result, the scale was Internal consistency was considered good, with a Cronbach alpha proved robust enough to undergo structure analysis (KMO ¼ 0.80) score of 0.80 for both Hebron and Cauca. LLOSA ET AL. - 5 of 9 F I G U R E 1 Adult/adolescent version of Mental Health Global State (MHGS) instrument, questions F I G U R E 2 Reduced, adult/adolescent version of Mental Health Global State (MHGS) instrument, questions 3.4 | Test‐retest reliability lower in both in Hebron (mean difference ¼ 2.5, SD ¼ 5.4, p < 0.05), and Cauca (mean difference ¼ 1.9, SD ¼ 3.8, p < 0.05). ICC In Hebron, 35 patients, and in Cauca 42 were re‐interviewed with corresponded to 0.70 (95% CI [0.41–0.85], p < 0.05) in Hebron and the MHGS scale to assess repeatability. Due to scheduling con- 0.87 (95% CI [0.76–0.93], p < 0.05) in Cauca. These results show straints and traceability of participants between appointments, the instrument tended to consistently indicate lower retest scores many patients were re‐interviewed after the planned 48 h; in for those with lower initial scores and higher retest scores for those Hebron, 98% of re‐tests occurred within 72 h of the baseline with higher initial scores, thus suggesting the instrument had good interview and in Cauca, 84%. On average scores at retest were test‐retest reliability. 6 of 9 - LLOSA ET AL. TABLE 1 Eigenvalues from PCA averaged over 10 replications (left), and Component loading values (right) Eigen values MHGS, Hebron Component loading values MHGS, Hebron PCA PA Dif MHGS question Category Component 1 Component 2 Component 3 Component 1 4.11 1.68 2.43 (1) How felt General 0.2139 0.3092 0.2589 Component 2 1.51 1.48 0.03 (2) Daily activities Function 0.3028 0.1891 0.0648 Component 3 1.2 1.34 0.14 (3) Getting along with others Function 0.2974 0.2252 0.4038 Component 4 0.93 1.23 0.29 (4) Pay attention and understand Function 0.2269 0.0703 0.2922 Component 5 0.91 1.13 0.22 (5) Appetite General 0.2157 0.0912 0.3589 Component 6 0.82 1.03 0.22 (6) Sleeping General 0.3048 0.0464 0.4196 Component 7 0.7 0.95 0.25 (7) Headaches and pain Symptom 0.2721 0.3107 0.0499 Component 8 0.65 0.87 0.22 (8) Aggression and loss of control Symptom 0.2927 0.2368 0.4421 Component 9 0.61 0.8 0.19 (9) Action to improve situation Coping 0.0085 0.5871 0.3125 Component 10 0.51 0.72 0.21 (10) Sought support comfort Coping 0.0025 0.5398 0.0457 Component 11 0.45 0.66 0.21 (11) Troubling thoughts and worries Symptom 0.3717 0.0023 0.1942 Component 12 0.35 0.58 0.23 (12) Feeling sad and nervous Symptom 0.4009 0.0713 0.1194 (13) Level of suffering Symptom 0.353 0.0933 0.1606 Abbreviations: MHGS, Mental Health Global State; PA, Parallel Analysis; PCA, Parallel Principal Component Analysis. TABLE 2 Age and gender of MHGS study participants, Hebron (left), Cauca (right), 2014 Hebron, oPT Female Male Total Cauca, CO Female Male Total Baseline n % n % n % Baseline n % n % n % 14–24 years 13 18.3 18 56.3 31 30 14–24 years 28 32.2 4 21.1 32 30.2 25–44 years 39 54.9 10 31.3 49 48 25–44 years 42 48.3 7 36.8 49 46.2 45–75 years 19 26.8 4 12.5 23 22 45–80 years 17 19.5 8 42.1 25 23.6 Total 71 100 32 100 103 100 Total 87 100 19 100 106 100 Abbreviation: oPT, Occupied Palestinian Territories. 3.5 | Inter‐rater reliability rho ¼ 0.48; DAS rho ¼ 0.59). For greater interpretability, correla- tions included participants who had measures for all tests at baseline, In Hebron, 36 patients were included in this subgroup, and in Cauca, n ¼ 71 in Hebron and n ¼ 84 in Cauca. Correlations were significant 42. In Hebron, 34 of 36 (94.4%) secondary raters were interviewed (p < 0.05) for all comparisons except with GAF in Hebron. the same day as the patient, while in Cauca all 42 were. On average Comparison of MHGS to CGI‐S was possible among the 32 family members tended to give lower scores than those directly by participants in Hebron and 31 in Cauca who underwent both baseline the patient; in Hebron, mean difference ¼ 1.7, SD ¼ 5.1, p < 0.05; in and follow‐up assessments. By this measure in Hebron, 7 (22%) Cauca, mean difference ¼ 0.2, SD ¼ 4.0, p ¼ 0.4. The ICC was 0.70 improved by 3 or more points in CGI‐S, 11 (34%) each by two and one (95% CI [0.42–0.84], p < 0.05) in Hebron and 0.77 (95% CI points, and 3 (9%) did not show improvement by CGI‐S. Similarly, in [0.57–0.88], p < 0.05) in Cauca. These results show that perceived Cauca, 3 (10%) improved by three points in CGI‐S, 12 (39%) by two severity by secondary raters was consistent with that given by pa- points, 11 (35%) by one point, and five (16%) showed no improvement. tients (i.e., lower scores with lower scores and higher with higher). The correlation of change in MHGS and CGI‐S was rho ¼ 0.38 (p < 0.05) in Hebron and rho ¼ 0.46 (p < 0.05) in Cauca for this version of the scale. 3.6 | Convergent validity When considering the psychologist's perspective expressed through CGI‐S as the gold standard measure, MHGS scale correctly Correlations with other instruments at baseline, were mostly me- classified improvements of 3 or more points among 81% of patients dium. In Hebron: these were CGI‐S rho ¼ 0.48; GAF rho ¼ 0.21; (85% sensitivity and 80% specificity) in Hebron using a cut‐off MHGS DAS rho ¼ 0.35. In Cauca these were: CGI‐S rho ¼ 0.49; GAF score change of eight or greater (AUC ¼ 0.85, 95% CI [0.67–1.00]). In LLOSA ET AL. - 7 of 9 1.00 1.00 0.75 0.75 Sensitivity Sensitivity 0.50 0.50 0.25 0.25 0.00 0.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 1 - Specificity 1 - Specificity Area under ROC curve = 0.8486 Area under ROC curve = 0.9048 F I G U R E 3 Receiver operating characteristic (ROC) curves for score change in six question version of Mental Health Global State (MHGS) for adolescents and adults compared to improvements in Clinical Global Impression (CGI), Hebron (left), and Cauca (right), 2014. Improvement was defined as a score change of three or more points in CGI; n ¼ 32 in Hebron, and n ¼ 31 in Cauca Cauca MHGS correctly classified improvements of three or more Despite this promising result, it should be noted that we faced points among 81% of patients (100% sensitivity and 79% specificity) security constraints leading to the minimum 100 participants in each using a cut‐off MHGS score change of 10 or greater (AUC ¼ 0.90, 95% setting. Also, correlations with baseline values from other in- CI [0.76–1.00]). ROC curves for both settings are shown in Figure 3. struments utilized in the study were mixed, albeit it correctly clas- There was strong agreement between the scales in more than sified over 80% of participants against therapist‐noted changes in half of participants assessed, and moderate to strong agreement in at function and symptomatology. It is important to note that patients' least three quarters of those assessed. There were no serious dis- assessment through MHGS asks about difficulties in the previous crepancies noted in either setting. No patient demonstrated strong week, while the psychologist might have taken a more global improvement by CGI‐S and deterioration by MHGS, nor strong perspective when rating patient functionality and symptomatology at improvement in MHGS and worsening by CGI‐S. the same point in time. The minimum clinically important difference (MCID) or clinically A few limitations also need to be considered. First, it should be relevant cut off for change in MHGS score was four points. This value noted that assessments took place at different points in the therapy. corresponds to the regression coefficient indicating the number of Given that continuity of care is not guaranteed in conflict settings MHGS point difference associated with a one‐point change in CGI‐S. (Sanchez‐Padilla et al., 2009), we aimed to have a tool that is practical For Hebron, this value was 2.3 (p < 0.05), and for Cauca 4.2 (p < enough to be applied at any session and would be able to discern 0.05). Considering the corresponding measurement errors for the change in global state within a few sessions. For this reason, we did scale were 2.9 and 3.3 for Hebron and Cauca, respectively, the most not require that the first and last interviews match the first and last appropriate cut off for improvement or worsening is a change in therapeutic session, nor a fixed number of sessions. Second, we relied score of 4 points in either direction. on a live‐in relative as proxy responder for two baseline interviews where the patient was unable to answer directly, and proxies were also used to assess inter‐rater reliability, compared to the patient's 4 | DISCUSSION own rating. Despite statistical agreement between the measures (i.e., by ICC), we recognize that proxy responders may not always be a valid To our knowledge, this is the first development and validation of an alternative to a patient's direct response. For instance, we noted on outcome scale for adults and adolescents (14 years old and above) average significantly lower MHGS scores by the secondary assessor created specifically from and for humanitarian interventions. It applies compared to the patients' own assessments. Nonetheless, when a a simple set of questions aimed at determining the global state of the patient is no longer available, an unfortunate reality of conflict set- patient at different time‐points in care, by applying a few simply wor- tings, a close, live‐in relative or caretaker may be the best suited ded questions on key domains of interest to mental health workers and alternative to inform the program about the patient's state. Third, populations in these contexts. The MHGS allows for monitoring change some additional challenges occurred by differences in test versus in global mental health state over time in several key outcomes of in- retest, as the original interview occurred in person and in most cases terest relating to function, symptoms and patient suffering. The results the re‐test occurred by phone, often days after the associated inter- suggest that Arabic and Spanish versions are dependable and offer a view. Scores where on average lower than initial ones, which could way to measure patient evolution throughout the care process. potentially be due to the phone interview. Additionally, the several 8 of 9 - LLOSA ET AL. days delay might have increased the chance of the effects of the A U TH O R C O N TR I B UT I O N S session or other external factors to play a role. It is to be noted as well, Conception and design of study: Augusto E. Llosa, Marie Rose Moro, that the MHGS scale requires two measures to detect change, and Bruno Falissard, Carmen Martínez‐Viciana, Stella Evangelidou, that a noted change is not, at any one point at individual level, German Casas, Rebecca F. Grais, and Caroline Marquer. Selection of necessarily a result of the intervention; it may reflect changes due to instrument's domain and questions: Marie Rose Moro, Bruno other life events. Fourth, despite formal training of psychologists on Falissard, Carmen Martínez‐Viciana, Stella Evangelidou, German the use of CGI‐S, it was a recent addition to routine assessments in Casas, Augusto E. Llosa, and Rebecca F. Grais. Conducted the these projects, and some changes in score were higher than expected. research: Augusto E. Llosa and Carmen Martínez‐Viciana. All authors While assessing concordance with MHGS was still possible, the MCID read and approved the final manuscript. will need to be confirmed in other settings. Strong training in the use of CGI‐S, which is recommended to be used alongside MHGS to E T HI CA L A PP R OV A L capture the therapists' perspective, is advisable. Last, the simplified The study received approval from the MSF Ethics Review Board, the scale was discerned statistically from results of the application of the Comité de Ética para la Investigación Científica (CEIC), Universidad longer version of the scale. The interspersion of questions from the de Cauca, Colombia, and clearance from the Palestinian Ministry of longer scale could have affected the responses of the questions Health and local authorities. All participants to the research signed included in the shorter version. Further testing of the simplified the informed consent. Participants younger than 18 years old signed version is therefore planned, though of note, the scale is already in use an assent form and their parent/caretaker signed the consent form. in several humanitarian contexts and performing well. At the programmatic level, the tool can help mental health staff O R CI D to assess impact of their interventions, adjust the objectives and Caroline Marquer https://orcid.org/0000-0002-9085-715X contents of the mental health interventions and care strategy, and provide an overall evaluation of programmatic impact in a brief and R EF ER E N CE S comprehensive manner. The use of the MHGS in combination with Bolton, P., & Tang, A. M. (2002). An alternative approach to cross‐cultural other evaluation support would hopefully improve the delivery of function assessment. Social Psychiatry and Psychiatric Epidemiology, mental health services during humanitarian intervention. Useful in 37(11), 537–543. Charlson, F., van Ommeren, M., Flaxman, A., Cornett, J., Whiteford, H., & acute and chronic crises, the tool may be most relevant for conflicts Saxena, S. (2019). New WHO prevalence estimates of mental dis- where humanitarian interventions may last for many years and in orders in conflict settings: A systematic review and meta‐analysis. some cases decades. In these contexts, there is ample opportunity to Lancet. 394(10194), 240–248. improve and refine programming. The instrument may be applicable Charter, H. (2011). The sphere project [Internet]. Response (Vol. 1. p. 402). Retrieved from www.practicalactionpublishing.org/sphere beyond humanitarian interventions, albeit additional validation work Chiumento, A., Khan, M. N., Rahman, A., & Frith, L. (2016). Managing would be needed to show it validity outside these contexts. ethical challenges to mental health research in post‐conflict settings. Developing World Bioethics. 16(1), 15–28. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for eval- uating normed and standardized assessment instruments in psy- 5 | CONCLUSION chology. Psychological Assessment. 6(4), 284 Cohen, J. (2013). Statistical power analysis for the behavioral sciences. The MHGS showed promising results to help assess MHGS thereby Biometrics. Academic press. providing an additional easy to use tool for both patient and mental (CVO) American Psychiatric Association. (2005). Global assessment of functioning (GAF) scale. In DSM‐IV‐TR manuel diagnostique et statis- health programmatic follow‐up in a humanitarian setting. Additional tique des troubles mentaux. work should focus on validation in at least one more context, to DeVellis, R. F. (2016). Scale development: Theory and applications. In Sage adhere to best practices in transcultural validation, and fine tune (Ed.), Scale development: Theory and applications (Vol. 26, pp. 109–110). MCID cut offs. Sage Publications. Dozio, E., Bizouerne, C., Feldman, M., & Moro, M. (2018). Operational and ethical challenges of applied psychosocial research in humanitarian A CK N O WL E D G M E NT emergency settings: A case study. Intervention. 16(1), 46 The authors wish to especially thanks Dr Ashour Hazem for his Fazel, M., Reed, R. V., Panter‐Brick, C., & Stein, A. (2012). Mental health of contribution and participation to the validation of the tool in Hebron. displaced and refugee children resettled in high‐income countries: The authors wish also to thank all of the participants in this study, Risk and protective factors. The Lancet, 379, 266–282. Forkmann, T., Scherer, A., Boecker, M., Pawelzik, M., Jostes, R., & and the MSF field teams and advisors. This study was funded and Gauggel, S. (2011). The clinical global impression scale and the in- supported by MSF, Operational Center Barcelona. Epicentre receives fluence of patient or staff perspective on outcome. BMC Psychiatry, core funding from MSF. The funder participated in the design of the 11(1), 83. study and participated in the revision of the report and manuscript. Gómez‐Restrepo, C., Aulí, J., Tamayo Martínez, N., Gil, F., Garzón, D., & Casas, G. (2016). Prevalencia y factores asociados a trastornos mentales en la población de niños colombianos, Encuesta Nacional C ON F L I C T O F IN T ER E S T S de Salud Mental (ENSM) 2015. Revista Colombiana de Psiquiatria. The authors declare that they have no competing interests. (45), 39–49. LLOSA ET AL. - 9 of 9 Guy, W. B. R. R. (1976). Clinical global impression (CGI). ECDEU Assessment Reed, R. V., Fazel, M., Jones, L., Panter‐Brick, C., & Stein, A. (2012). Mental Manual for Psychopharmacology. 217–222. health of displaced and refugee children resettled in low‐income and Hall, R. C. W. (1995). Global assessment of functioning: A modified scale. middle‐income countries: Risk and protective factors. The Lancet, Psychosomatics. 36(3), 267–275. 379, 250–265. Hilsenroth, M. J., Ackerman, S. J., Blagys, M. D., Baumann, B. D., Baity, Rodin, D., & van Ommeren, M. (2009). Commentary: Explaining enormous M. R., Smith, S. R, … Holdwick, D. J. (2000). Reliability and validity variations in rates of disorder in trauma‐focused psychiatric epide- of DSM‐IV axis V. American Journal of Psychiatry. 157(11), miology after major emergencies. International Journal of Epidemi- 1858–1863. ology, 38, 1045–1048. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in Sanchez‐Padilla, E., Casas, G., Grais, R. F., Hustache, S., & Moro, M.‐R. covariance structure analysis: Conventional criteria versus new (2009). The Colombian conflict: A description of a mental health alternatives. Structural Equation Modeling. program in the department of tolima. Conflict and Health. 3(1), 13 Interagency Standing Committee (IASC). (2006). IASC guidelines on mental Terwee, C. B., Bot, S. D. M., de Boer, M. R., van der Windt, D. A. W. M., health and psychosocial support in emergency settings [Internet]. Knol, D. L., Dekker, J., … de Vet, H. C. (2007). Quality criteria were Geneva, Switzerland: IASC 2006. proposed for measurement properties of health status question- Jones, L., Asare, J. B., El Masri, M., Mohanraj, A., Sherief, H., & van naires. Journal of Clinical Epidemiology, 60(1), 34–42. Ommeren, M. (2009). Severe mental disorders in complex emer- Tol, W. A., Barbui, C., Galappatti, A., Silove, D., Betancourt, T. S., Souza, R., … gencies. Lancet, 374(9690), 654–661. Van Ommeren, M. (2011). Mental health and psychosocial support in Kirmayer, L. J., Guzder, J., & Rousseau, C. (2013). Cultural consultation: humanitarian settings: Linking practice and research. The Lancet, 378, Encountering the other in mental health care, New York, USA: Springer 1581–1591. Science & Business Media. Üstün, T. B., Kostanjsek, N., Chatterji, S., & Rehm, J. (2010). Measuring Kline, P. (2000). The Handbook of Psychological Testing. Psychology Press. health and disability: Manual for WHO disability assessment schedule Kortmann, F. (1987). Problems in communication in transcultural WHODAS 2.0, Geneva, Switzerland: World Health Organization. psychiatry: The self reporting questionnaire in Ethiopia. Acta Van Ommeren, M. (2003). Validity issues in transcultural epidemiology. Psychiatrica Scandinavica, 75(6), 563–570. British Journal of Psychiatry, 182(5), 376–378. Kortmann, F. (1990). Psychiatric case finding in Ethiopia: Shortcomings of Viswanath, B., & Chaturvedi, S. K. (2012). Cultural aspects of major mental the self reporting questionnaire. Culture Medicine and Psychiatry, disorders: A critical review from an Indian perspective. Indian Journal 14(3), 381–391. of Psychological Medicine, 34(4), 306. Kortmann, F., & Ten Horn, S. (1988). Comprehension and motivation in Wolpert, M., Ford, T., Trustam, E., Law, D., Deighton, J., Flannery, H., & responses to a psychiatric screening instrument. Validity of the SRQ Fugard, R. J. (2012). Patient‐reported outcomes in child and in Ethiopia. British Journal of Psychiatry, 153(1), 95–101. adolescent mental health services (CAMHS): Use of idiographic and Lohr, K. N. (1988). Outcome measurement: Concepts and questions. standardized measures. Journal of Mental Health, 21(2), 165–173. Inquiry, 25, 37–50. World Health Organisation. (2013). Comprehensive mental health Lohr, K. N., Aaronson, N. K., Alonso, J., Burnam, M. A., Patrick, D. L., Perrin, action plan 2013–2020. Geneva 66th World Heal Assem [Internet] E. B., & Roberts, J. S. (1996). Evaluating quality‐of‐life and health (pp. 1–27). status instruments: Development of scientific review criteria. Clinical Therapeutics, 18(30), 979–984. Miller, K. E., & Rasmussen, A. (2010). War exposure, daily stressors, and How to cite this article: Llosa AE, Martínez‐Viciana C, mental health in conflict and post‐conflict settings: Bridging the Carreño C, et al. Development of a patient rated scale for divide between trauma‐focused and psychosocial frameworks. Social mental health global state for use during humanitarian Science & Medicine, 70(1), 7–16. interventions. Int J Methods Psychiatr Res. 2021;30:e1850. Patel, V., Chowdhary, N., Rahman, A., & Verdeli, H. (2011). Improving access to psychological treatments: Lessons from developing coun- https://doi.org/10.1002/mpr.1850 tries. Behaviour Research and Therapy. 49(9), 523–528.