TYPE Original Research PUBLISHED 09 January 2023 DOI 10.3389/fmed.2022.1014340 Measuring depression in Primary OPEN ACCESS Health Care in Spain: EDITED BY Simon Ching Lam, Tung Wah College, Hong Kong SAR, Psychometric properties and China REVIEWED BY diagnostic accuracy of HSCL-5 Emma Yun Zhi Huang, Macau University of Science and Technology, Macau SAR, China and HSCL-10 Fernanda Ávila, Fluminense Federal University, Brazil María Rodríguez-Barragán1,2,3 , *CORRESPONDENCE Ana Clavería María Isabel Fernández-San-Martín2,4 , Ana Clavería5,6,7*, anaclaveriaf@gmail.com Jean Yves Le Reste8 , Patrice Nabbe8 , Emma Motrico7,9 , SPECIALTY SECTION This article was submitted to Irene Gómez-Gómez7,9 and Eva Peguero-Rodríguez2,10,11 Family Medicine and Primary Care, 1 Primary Health Centre La Mina, Gerència Territorial d’Atenció Primària de Barcelona, Institut Català a section of the journal de la Salut, Sant Adrià de Besòs, Barcelona, Spain, 2 Institut Universitari d’Investigació en Atenció Frontiers in Medicine Primària Jordi Gol (IDIAP Jordi Gol), Barcelona, Spain, 3 Faculty of Medicine, Department of RECEIVED 08 August 2022 Pediatrics, Obstetrics, Gynecology and Preventive Medicine, Universitat Autònoma de Barcelona, ACCEPTED 21 December 2022 Barcelona, Spain, 4 Gerència Territorial d’Atenció Primària de Barcelona, Institut Català de la Salut, PUBLISHED 09 January 2023 Barcelona, Spain, 5 I-Saúde Group, South Galicia Health Research Institute [IISGS-Servicio Gallego de Salud (SERGAS)], Vigo, Spain, 6 Vigo Health Area, Servicio Gallego de Salud (SERGAS), Vigo, Spain, CITATION 7 Network for Research on Chronicity, Primary Care and Health Promotion (RICAPPS), Vigo, Spain, Rodríguez-Barragán M, 8 Department of General Practice, ER 7479 SPURBO Soins Primaires, Santé Publique, Registre des Fernández-San-Martín MI, Clavería A, Cancers de Bretagne Occidentale, Université de Bretagne Occidentale, Brest, France, 9 Department Le Reste JY, Nabbe P, Motrico E, of Psychology, Universidad Loyola, Andalucía, Spain, 10 Primary Health Centre El Castell, Gerència Gómez-Gómez I and Territorial d’Atenció Primària Metropolitana Sud, Institut Català de la Salut, Castelldefels, Barcelona, Peguero-Rodríguez E (2023) Spain, 11 Departamento de Ciencias Clinicas, Facultad de Medicina, Universidad de Barcelona, Measuring depression in Primary Barcelona, Spain Health Care in Spain: Psychometric properties and diagnostic accuracy of HSCL-5 and HSCL-10. Front. Med. 9:1014340. doi: 10.3389/fmed.2022.1014340 Background: Depression has a high prevalence among European countries. COPYRIGHT Several instruments have been designed to assess its symptoms in different © 2023 Rodríguez-Barragán, Fernández-San-Martín, Clavería, Le populations. The Hopkins Symptom Checklist 25 (HSCL-25) scale has been Reste, Nabbe, Motrico, Gómez-Gómez identified as valid, reproducible, effective, and easy to use. There are short and Peguero-Rodríguez. This is an versions of this scale that could be useful in Primary Care (PC) settings, but open-access article distributed under the terms of the Creative Commons their psychometric properties are unknown. Attribution License (CC BY). The use, distribution or reproduction in other Aim: To assess in PC patients the psychometric properties and diagnostic forums is permitted, provided the accuracy of the Spanish version of the HSCL-10 and the HSCL-5 consisting original author(s) and the copyright owner(s) are credited and that the of 10 and 5 items, respectively. original publication in this journal is cited, in accordance with accepted Methods: A multicenter, cross-sectional study was carried out at six PC academic practice. No use, distribution centers in Spain. The HSCL-25 was administered to outpatients aged 45–75 or reproduction is permitted which does not comply with these terms. who also participated in the structured Composite International Diagnostic Interview (CIDI). HSCL-10 and HSCL-5 were assessed and compared to HSCL-25 regarding total score correlation, internal consistency, and criterion validity against the gold-standard CIDI. This is a methodological study from a secondary data analysis and the primary data has been previously published. Frontiers in Medicine 01 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 Results: Out of 790 patients, 767 completed the HSCL-25 and 736 the CIDI interview (96.0%). Cronbach’s Alpha was 0.84 for HSCL-10 and 0.77 for HSCL-5. The known-group method and confirmatory factor analysis were acceptable for the establishment of construct validity. Sensitivity was 79.7% (CI95%, 67.7–88.0%) for HSCL-10, and 78.0% (CI95%, 65.9–86.6%) for HSCL- 5, whereas specificity was 83% (CI95%, 80.0–85.7%) for HSCL-10, and 72.8% (CI95%, 69.3–76.0%) for HSCL-5. Area under the curve against CIDI was 0.88 (CI95%, 0.84–0.92%) for HSCL-10, and 0.85 (CI95%, 0.81–0.89%) for HSCL-5. Optimum cutoff point calculated with Youden Index was 1.90 for the HSCL-10 and 1.80 for the HSCL-5. Conclusion: HSCL-10 and HSCL-5 are reliable and valid tools to detect depression symptoms and can be used in PC settings. KEYWORDS depression, Hopkins Symptom Checklist, Primary Health Care (MeSH), patient reported outcome measures (MeSH), diagnostic accuracy, psychometric properties (COSMIN) (8) for the language and population they Introduction are directed at. The Hopkins Symptom Checklist 25 (HSCL-25) (9) is a Mental health is an issue of increasing concern in Spain widely used, self-administered screening tool for depression and other European countries and represents a considerable and psychological distress; it is also employed for research percentage of Primary Care (PC) consultations. The clinical purposes. It has been shown to be valid and reliable for different diagnosis of mental disorders is based on the symptoms of the populations (10–14), useful in PC (15), and it is available in Diagnostic and Statistical Manual of Mental Disorders (DSM). several languages (16) including Spanish. The Spanish version In addition, there are many questionnaires available which of HSCL-25 was translated, culturally adapted, and validated assess mental health diseases in populations and are frequently following the COSMIN (17, 18). employed in PC (1). Such instruments are used for the purposes The HSCL-25 is a short version of the Symptom Checklist of screening and symptom detection to aid diagnosis. 90-Revised (SCL-90-R) (19). There are, however, other The United States Preventive Services Task Force (USPSTF) questionnaires with fewer items (20) which are of interest due to recommends universal screening for depression in the general their brevity and reduced completion time. Shorter versions are adult population (2) as it has been shown to reduce symptoms useful as they can save time in complex settings such us the PC and improve quality of life and functional status (3). Screening context. The HSCL-10 and HSCL-5 were developed by selecting should, however, be combined with adequate systems to ensure 10 and 5 items, respectively, due to their strong correlation with accurate diagnosis, effective treatment, and appropriate follow- up. the HSCL-25 mean score (19, 21). Items of the HSCL-10 and Up to 75% of patients with depression are treated exclusively HSCL-5 are included in the longer HSCL-25. Both the HSCL-10 in PC (4). Nevertheless, Family Doctors (FD) report difficulties and the HSCL-5 have shown a strong mean score correlation in the identification of mental health disorders (5) and with respect to the HSCL-25 and high reliability (Cronbach’s consider that evaluative tools permitting rapid diagnosis with Alpha Coefficient >0.80) (22). Both short versions, especially a limited number of items are required (6). The issue of the HSCL-10, have been used in different populations such as time is particularly relevant within the context of PC given adolescents (23–25), industry workers (26), population surveys its complexity and limited consultation schedules. Moreover, (22, 27), patients with alcohol use disorder (28), refugees (29), FD have been found to feel more confident in detecting and and also to measure symptoms of depression and anxiety due to managing depression when employing questionnaires (7). the COVID pandemic (30–32). Their psychometric properties The ideal questionnaire should be reliable, valid, ergonomic, recommend them for clinical use as screening and symptom and easy to use for both patients and healthcare professionals. assessment tools and for research purposes (25, 27, 33). It should be validated according to the Consensus-based The aim of this article is to report the psychometric Standards for the Selection of Health Measurement Instruments properties and diagnostic accuracy of the Spanish versions of Frontiers in Medicine 02 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 the HSCL-5 and HSCL-10 for their use as rapid and accessible Composite international diagnostic depression screening instruments in PC. interview (CIDI) The CIDI is a well-known, standardized interview designed Materials and methods by the World Health Organization (WHO) based on the fourth edition of the Diagnostic and Statistical Manual of Mental Data collection, study population, and Disorders (DSM-IV) and the International Classification of Diseases-10 (ICD-10) criteria (37). It is administered by trained variables interviewers and available in different languages (38). For this study, section E (questions referring to depression) of The present study was based on data from a cross-sectional the Spanish version was used. The CIDI was conducted by multicenter study designed to validate the HSCL-25 in a Spanish trained psychologists. PC population. This is a methodological study from a secondary As the diagnose of depression is a clinical interview data analysis and the primary data has been previously published performed by a trained professional and conducted using the (18). Participants were patients attending six Spanish Primary DSM criteria, the CIDI is considered the gold-standard in Health Centers (PHC) taking part in the EIRA study (34, 35). the present study. Inclusion criteria were to be aged 45–75 years and presenting two or more of the following: Smoking, low adherence to the Mediterranean dietary pattern, and insufficient physical activity. Exclusion criteria were: Advanced serious illness, cognitive Ethical considerations impairment, dependence in basic everyday activities, severe The study was developed according to national and mental illness, unable to attend the PHC, under treatment for international legislation (the Declaration of Helsinki and latest cancer or in end-of-life care, or planning to travel during the versions). The protocol was evaluated by the IDIAP Jordi Gol intervention period. Ethical Research Committee (approval number: P16/025) and Participants were recruited by consecutive sampling by the corresponding regional governments. Written consent of patients attending the PHC for any reason during a was obtained from the participants, and the questionnaires were 6-month-period in 2017. They were asked to complete codified with an identification number to protect anonymity and sociodemographic data (gender, age, nationality, marital confidentiality. status, current employment, and education level) and the self-administered HSCL-25 questionnaire (and other questionnaires/forms related to the EIRA study). Afterward, Statistical analysis trained professionals, blinded to the HSCL-25 score, conducted the gold-standard CIDI interview with all participants. Analysis was conducted with STATA version 15. Missing values for the HSCL-25 were replaced with the individual mean for the rest of the items. Subjects with ≥50% missing Hopkins symptom checklist-25 items were excluded. (HSCL-25) Total score was calculated for HSCL-25, HSCL-10, and HSCL-5 for the total population and in relation to gender and The HSCL-25 is a widely used, self-administered age categories, and by the following sociodemographic groups: questionnaire designed to measure anxiety and depression marital status, education level, and current employment. symptoms (9, 11) and takes 5–10 min to complete (13). It Reliability of the HSCL-10 and HSCL-5 was analyzed by consists of 10 and 15 items belonging to the anxiety and calculating the Cronbach’s Alpha Coefficient and for each of depression dimensions, respectively. The items are answered the two depression and anxiety subscales. A value of ≥0.7 was on a four-point Likert-like scale: 1 = “Not at all;” 2 = “A little;” considered adequate (39). Cronbach’s Alpha without the item 3 = “Quite a bit;” 4 = “Extremely.” The average score, ranging was also calculated to assess the contribution of each item to the from 1 to 4, is calculated by dividing the total score by the internal consistency of two versions. number of the items. A cutoff value of 1.75 is generally used for Construct validity was measured with the known-groups major depression diagnosis, as it is considered a valid predictor method by comparing the total score of the HSCL-10 and of mental disorder (10, 13, 36). the HSCL-5 by gender. The total score was expected to Items belonging to HSCL-25, HSCL-10, and HSCL-5 are be significantly higher among women (40, 41). Independent shown in Table 1. The corresponding cutoffs points are 1.85 for sample t-test was performed, a significant result (p < 0.05) the HSCL-10 and 2.00 for the HSCL-5 (22). The Spanish version was considered satisfactory (42). Confirmatory factor analysis of the HSCL-25 was used in this study (17). (CFA) was conducted to assess the structural validity. To Frontiers in Medicine 03 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 TABLE 1 Items belonging to HSCL-25, HSCL-10, and HSCL-5 scale. Item in HSCL-25 HSCL-10 HSCL-5 1. Being scared for no reason X Anxiety dimension 2. Feeling fearful X X 3. Faintness X 4. Nervousness X 5. Heart racing 6. Trembling 7. Feeling tense X 8. Headache 9. Feeling panic 10. Feeling restless 11. Feeling low in energy Depression dimension 12. Blaming oneself X 13. Crying easily 14. Losing sexual interest 15. Feeling lonely 16. Feeling hopeless X X 17. Feeling blue X X 18. Thinking of ending one’s life 19. Feeling trapped 20. Worrying too much X 21. Feeling no interest 22. Feeling that everything is an effort X 23. Worthless feeling X 24. Poor appetite 25. Sleep disturbance X Items belonging to depression and anxiety dimensions. evaluate the estimated model fit, the absolute fit index was of internal validity; positive and negative predictive values calculated with chi-squared distribution. Given that this value were also calculated. Both the sensitivity and specificity may be affected by the sample size, complementary indices of a screening test should be greater than 0.70 (42). For were employed: The root mean square error of approximation these calculations, cutoffs those proposed by Strand et al. (RMSEA), the standardized root mean square residual (SRMR), were followed: 1.85 for the HSCL-10 and 2.00 for the the comparative fit index (CFI), and the Tucker–Lewis fit HSCL-5 (22). Other authors have employed the same cutoff index (TLI) (43, 44). Cutoff values considered adequate were: points (47). SRMR < 0.05, RMSEA ≤ 0.08, CFI > 0.90, and TLI > 0.90 (45). Criterion validity and diagnostic accuracy were measured by calculating the ROC curve for the HSCL-10 and HSCL-5 Results scale in comparison with the gold-standard CIDI (8). The area under the curve (AUC) was estimated with 95% confidence Participants interval (CI95%). Best cutoff points for the study population and by gender were calculated with the Youden Index for From a total of 790 patients, 767 completed the HSCL-25 both HSCL-10 and HSCL-5. Youden Index is defined as (97.1% response rate). Participants’ mean age was 58.4 years “Sensitivity + Specificity −1,” it is a value that indicates (± 8.2), 54.4% were women, and there were no significant the validity of the instrument for a specific cutoff point gender differences among age categories. Table 2 shows the (46). Sensitivity and specificity were assessed as measures mean score and standard deviation (SD) of the three HSCL Frontiers in Medicine 04 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 TABLE 2 HSCL-25, HSCL-10, and HSCL-5 scale mean scores and standard deviation (SD) in relation to age, gender, marital status, education, and current employment. HSCL-25 HSCL-10 HSCL-5 n % Mean SD P-value Mean SD P-value Mean SD P-value Total 767 1.57 0.45 1.55 0.50 1.72 0.59 Gender Men 350 45.6 1.42 0.35 <0.001 1.41 0.39 <0.001 1.53 0.46 <0.001 Women 417 54.4 1.69 0.49 1.68 0.55 1.88 0.64 Age 45–54 298 38.7 1.63 0.47 0.006 1.62 0.52 0.005 1.71 0.61 0.060 55–64 265 34.7 1.55 0.49 1.54 0.51 1.71 0.61 65–75 204 26.6 1.50 0.40 1.47 0.44 1.64 0.54 Marital status Married/with a partner 550 71.9 1.54 0.43 0.028 1.52 0.47 0.029 1.69 0.56 0.084 Single 57 7.3 1.58 0.44 1.58 0.48 1.73 0.59 7-810-11 Separated or 103 13.3 1.68 0.48 1.69 0.56 1.82 0.67 divorced Widow (er) 57 7.5 1.64 0.51 1.60 0.61 1.86 0.65 Education Primary or lower 398 51.8 1.57 0.42 0.855 1.54 0.48 0.697 1.74 0.58 0.333 Secondary or higher 368 48.2 1.56 0.47 1.56 0.52 1.70 0.60 Current employment Employed 313 41.0 1.55 0.43 0.547 1.54 0.48 0.774 1.84 0.63 0.746 Housewife 111 14.5 1.64 0.45 1.61 0.51 1.82 0.60 Unemployed 83 10.9 1.65 0.46 1.65 0.55 1.82 0.60 Retired 209 27.3 1.45 0.40 1.43 0.44 1.57 0.54 Others (student, sick 48 6.3 1.88 0.53 1.88 0.58 1.98 0.58 leave, and disability) versions in relation to the sociodemographic characteristics of The least consistent was 25 “Sleep disturbance” followed by 1 the sample. “Being scared for no reason.” Item 25 “Sleep disturbance” had There were statistically significant differences in total scores a Cronbach’s Alpha without the item of 0.8424, as a result, this for the three versions regarding gender. Women scored higher item worsened the reliability of this version, as this value was with a minimum difference >0.25 points. There were also above 0.8417. In the HSCL-5, item 17 “Feeling blue” was also statistically significant differences in the total scores of the the most consistent whilst two “Feeling fearful” was the least HSCL-25 and HSCL-10, but not the HSCL-5, with respect to consistent although without affecting reliability. age and marital status. No differences were observed regarding education level or current occupation. Construct validity: Known-groups method and confirmatory factor Reliability: internal consistency analysis (CFA) Cronbach’s Alpha was 0.8417 and 0.7712 for the HSCL-10 The known-group method analysis showed that the total and HSCL-5, respectively. When analyzing the two dimensions score of the HSCL-10 indicated that women had significantly separately, this value was higher for the depression dimension higher scores (mean = 1.68; SD 0.03) than men (mean = 1.41; than the anxiety one. These values, and the value of the SD 0.02; t = 7.76; p < 0.001). Results were in the same direction coefficient without the item, are depicted in Table 3. with the HSCL-5, total score was significantly higher in women The most consistent item for the HSCL-10 was 17 “Feeling (mean = 1.88; SD 0.03) than in men (mean = 1.53; SD 0.02; blue” followed by 22 “Feeling that everything is an effort.” t = 8.51; p < 0.001). Frontiers in Medicine 05 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 TABLE 3 Cronbach’s alpha coefficient without the item and for total values. Item HSCL-25 HSCL-10 HSCL-5 1. Being scared for no reason 0.9148 0.8359 2. Feeling fearful 0.9137 0.8307 0.7580 3. Faintness 0.9127 0.8268 4. Nervousness 0.9109 0.7156 5. Heart racing 0.9136 6. Trembling 0.9151 7. Feeling tense 0.9109 0.8205 8. Headache 0.9163 9. Feeling panic 0.9156 10. Feeling restless 0.9117 11. Feeling low in energy 0.9110 12. Blaming oneself 0.9131 0.8279 13. Crying easily 0.9145 14. Losing sexual interest 0.9153 15. Feeling lonely 0.9121 16. Feeling hopeless 0.9127 0.8261 0.7427 17. Feeling blue 0.9085 0.8080 0.6820 18. Thinking of ending one’s life 0.9159 19. Feeling trapped 0.9129 20. Worrying too much 0.9129 0.7399 21. Feeling no interest 0.9129 22. Feeling that everything is an effort 0.9112 0.8188 23. Worthless feeling 0.9140 0.8315 24. Poor appetite 0.9167 25. Sleep disturbance 0.9152 0.8424 Total 0.9166 0.8417 0.7712 Anxiety subscale (items 1–10) 0.8306 0.6859 0.5404 Depression subscale (items 11–25) 0.8784 0.7804 0.7010 Table 4 shows the results of the CFA: the factor loading for that the HSCL-10 and the HSCL-5 do not have a stable each model and correlation in the two-factor models. All the factor structure. factor loadings were positive, statistically significant (p < 0.001), and above 0.30. In fact, all factor loadings in the different versions were above 0.45. The range of loadings was 0.45– Criterion validity and diagnostic 0.81 for the one factor HSCL-10, 0.50–0.83 for the one factor accuracy: Relation of HSCL-10 and HSCL-5, 0.47–0.82 for the two correlated factor HSCL-10, HSCL-5 with gold-standard CIDI and 0.51–0.85 for the two correlated factor HSCL-5. Item 17 “Feeling blue” was the item with the highest factor loadings in Of the 767 participants who completed the HSCL-25, 736 all the models analyzed. When analyzing the models with two also took part in the CIDI interview (96.0%). Depression correlated factors, a strong correlation between the two-factors prevalence varied depending on the questionnaire employed of depression and anxiety was observed for both the HSCL-10 (Table 5). Prevalence measured with the HSCL-10 was similar and the HSCL-5. to that obtained with the full HSCL-25 version and higher than The goodness-of-fit indices in the studied factor the value obtained with the HSCL-5. models can be consulted in Supplementary Table 1 of the Table 6 shows sensitivity, specificity, positive, and negative Supplementary materials. Globally, the indices showed predictive values for the total number of participants and by Frontiers in Medicine 06 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 TABLE 4 Confirmatory factorial analysis: Factor loading values and correlation between two anxiety and depression factors. One factor Two correlated factors HSCL-10 HSCL-5 HSCL-10 HSCL-5 Anxiety Depression Anxiety Depression Item 1 0.45 0.49 Item 2 0.52 0.50 0.56 0.54 Item 3 0.60 0.62 Item 4 0.64 0.71 Item 7 0.66 0.69 Item 12 0.57 0.58 Item 16 0.62 0.63 0.64 0.64 Item 17 0.81 0.83 0.82 0.85 Item 20 0.58 0.57 Item 22 0.69 0.68 Item 23 0.54 0.55 Item 25 0.47 0.47 Factor correlation 0.90 0.87 TABLE 5 Prevalence of depression according to CIDI, HSCL-25, HSCL-10, and HSCL-5, total and by gender. Male (n = 341) Female (n = 395) Total (n = 736) CIDI 4.7 (CI95% 2.7–7.5) 10.9 (CI95% 8.0–14.4) 8.0 (CI95% 6.2–10.2) HSCL-25 (cutoff = 1.75) 16.7 (CI95% 12.9-21-1) 38.7 (CI95% 33.9–43.7) 28.5 (CI95% 25.3–31.9) HSCL-10 (cutoff = 1.85) 12.6 (CI95% 9.3–16.6) 30.1 (CI95% 25.6–34.9) 22.0 (CI95% 19.1–25.2) HSCL-5 (cutoff = 2.00) 19.4 (CI95% 15.3–24.0) 41.5 (CI95% 36.6–46.6) 31.3 (CI95% 27.9–34.7) gender. Sensitivity was similar for both genders, especially in is depicted in Figure 2 and by gender in the Supplementary the HSCL-10, whilst specificity was better in men. Negative Figure 2. predictive values were >95% for both versions, these values were The optimum cutoff points for the study population were for the total population and by gender. All values were higher for calculated with the Youden Index. For the HSCL-10 it was 1.90 the HSCL-10 than for the HSCL-5. for the total population and for both genders. For the HSCL- 5 the optimum cutoff point was 1.80 for the total population and for women, and 2.00 for men. A table including sensitivity, HSCL10 vs. CIDI specificity, positive, and negative predictive values using optimal cutoff points is available in the Supplementary Table 2. The AUC between the HSCL-10 and the CIDI was 0.877 (CI95% 0.836–0.919). In the gender analysis it was greater in men with an AUC of 0.943 (CI95% 0.897–0.989) compared Discussion to women who had an AUC of 0.825 (CI95% 0.765–0.886). The ROC curve is depicted in Figure 1 and by gender in the This study is the first to analyze the psychometric properties Supplementary Figure 1. and diagnostic accuracy of the Spanish versions of the HSCL- 10 and HSCL-5, the results obtained now allow the use of both scales as screening tools for depression in the PC setting HSCL5 vs CIDI in Spain. The results demonstrated that in the Spanish PC population, the HSCL-10 presents high reliability and validity. The AUC between the HSCL-5 and the CIDI was 0.853 The HSCL-5 also showed acceptable psychometric properties (CI95% 0.812–0.894). In the gender analysis it was greater in although slightly worse than the HSCL-10. Both scales showed men with an AUC of 0.918 (CI95% 0.859–0.977) than in women adequate sensitivity and specificity when compared to the who had an AUC of 0.795 (CI95% 0.734–0.855). The ROC curve semistructured clinical interview CIDI conducted by trained Frontiers in Medicine 07 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 TABLE 6 Sensitivity, specificity, positive and negative predictive values, total and by gender in the HSCL-10, and the HSCL-5. Male (n = 341) Female (n = 395) Total (n = 736) Index CI 95% CI 95% Index CI95% CI 95% Index CI 95% CI 95% lower upper lower upper lower upper limit limit limit limit limit limit HSCL-10 cutoff Sensitivity 81.3 57.0 93.4 79.1 64.8 88.6 79.7 67.7 88.0 point = 1.85 Specificity 90.8 87.1 93.5 75.9 71.1 80.0 83.0 80.0 85.7 PPV 30.2 18.6 45.1 28.6 21.2 37.3 29.0 22.6 36.4 NPV 99.0 97.1 99.7 96.7 93.9 98.3 97.9 96.4 98.8 HSCL-5 cutoff Sensitivity 81.3 57.0 93.4 76.7 62.3 86.8 78.0 65.9 86.6 point = 2.00 Specificity 83.7 79.3 87.3 62.8 57.6 67.7 72.8 69.3 76.0 PPV 19.7 11.9 30.8 20.1 14.7 26.9 20.0 15.3 25.6 NPV 98.9 96.8 99.6 95.7 92.2 97.6 97.4 95.7 98.5 PPV, positive predictive value; NPV, negative predictive value. FIGURE 1 ROC curve and AUC HSCL-10 vs. CIDI. professionals. The optimal cutoff points obtained were very order to provide a similar diagnostic value that saves time close to those proposed by other authors. for both patient and healthcare professional (20, 53, 54). The PC setting is ideal for the detection, diagnosis, and Recently, the HSCL-10 and HSCL-5 have shown acceptable investigation of chronic, highly prevalent disorders as it allows reliability and validity (22, 27), particularly the HSCL-10 (25, an early study of such pathologies (48). With respect to 33, 47). depression, it is widespread in Spanish and European countries With respect to reliability, Cronbach’s Alpha results of (40) yet patients frequently consult their FD for other motives 0.84 and 0.77 were obtained for the HSCL-10 and HSCL- than their mood disorders (6). 5, respectively. As both were above 0.7, they are considered There are many questionnaires that can be employed as acceptable (55). Such findings are similar to those obtained screening tools to detect depression at all its stages (49–52), by other authors (33) and, as occurred in other studies (19, some of which have been validated within the PC setting 22, 27), the reliability of the two short versions was lower (1). Moreover, shortened versions have been proposed in than the HSCL-25. Frontiers in Medicine 08 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 FIGURE 2 ROC curve and AUC HSCL-5 vs. CIDI. In the full 25-item version, 17 “Feeling blue” was the most our study. Further validation studies should be done to assess consistent (18). This item which asks about sadness, a basic other alternative models of the factor structure of the HSCL-10 characteristic in patients with depression, is included in the and the HSCL-5. By examining the factor loading of 10 and 5 HSCL-10 and the HSCL-5 and was also the most consistent in items, respectively, all the items were significantly loaded to the the two short versions. Once removed, reliability diminished hypothesized construct, and all factor loadings were statistically to the minimum as can be seen in Table 3. The next most significant, positive, and above 0.45. Item 17 “Feeling blue” had consistent items in the HSCL-25 were four “Nervousness” and the highest factor loadings in all models tested and for both the seven “Feeling tense.” The former was included in the 5-item HSCL-10 and the HSCL-5. In the study of the two-factor models, version and the latter in the 10-item one. Item 24 was the least there was a factorial correlation of 0.90 between the dimensions consistent in the HSCL-25 and is not present in either of the in the HSCL-10 and of 0.87 in the HSCL-5, these findings short versions. The following least consistent items in the HSCL- indicated that the depression and anxiety dimensions strongly 25 were 8 “Headache” and 18 “Thinking of ending one’s life,” correlated in a positive manner. The correlation is expectable neither of which is included in the two short versions. as anxiety and depression are frequently found to be associated When analyzing Cronbach’s Alpha without the item in the comorbidities (56). short versions, it was observed that 25 “Sleep disturbance” The calculation of the total score is done in the same way in worsened HSCL-10 reliability, that is to say, by eliminating the different versions of the HSCL, by dividing the total score this item reliability improved. Such a finding concurs with that by the corresponding number of ítems answered. With respect reported by Kleppang et al. who employed the 10-item version to the total score, in comparison with the 25-item long version, with adolescents in Norway (25). The other items contributed to the mean score of the HSCL-25 was 1.57, very similar to the 1.55 good reliability in both the HSCL-10 and HSCL-5. obtained with the HSCL-10. The mean score increased to 1.72 Regarding analysis of the scale’s factorial structure, this with the five-item version. The means were significantly greater was performed with the CFA as the HSCL-25 has been widely in women for all three versions, this is unsurprising considering studied with one single factor or two correlated ones even that depression is more prevalent in the female gender (40, 41). though other models have been proposed (10). The HSCL- Significant differences reported in mean scores according to age 10 and the HSCL-5 maintain the same factor structure (27). and marital status for the 25 and 10-item versions were lost The goodness-of-fit indices for the studied models were not in the 5-item one. optimal, showing that the data did not fit the hypothesized factor With respect to prevalence, it was higher in the HSCL- structure of one factor and two correlated factors. The factor 5 followed by the HSCL-10 and then the HSCL-25. All three structure was unstable for both scales, this is a limitation of versions and the gold-standard CIDI showed greater prevalence Frontiers in Medicine 09 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 in women than men. Employing the cutoffs corresponding behavior. Adults often engage in two or more unhealthy to each version (22), sensitivity was similar for both genders behaviors simultaneously, the co-occurrence of unhealthy diet whilst specificity was greater for men in the two short versions. with insufficient physical activity ranges between 47 and Findings that concur with those reported for the HSCL-25 (18). 54%, unhealthy diet with smoking between 23 and 28%, and In a study comparing the HSCL-10 with the CIDI (33), slightly insufficient physical activity with smoking between 8 and 20% higher results were reported with respect to sensitivity, although (59). Unhealthy diet can be associated to depression (60). a different cutoff was employed. Another multicenter study Despite the limited age range and the selection criteria in conducted in General Practice in Norway and Denmark using the present study, the authors consider that the results are the HSCL-10 and the CIDI obtained similar results to ours in transferable given the large number of participants, moreover, terms of sensitivity and specificity (23). they had attended the PHC for any motive, consequently, the Cutoffs can be based on previous studies, cutoffs used sample is sufficiently representative. in clinical practice, cutoffs recommended by clinical practice When deciding which questionnaire to use, there are guidelines, or cutoffs recommended by the original authors (57). a few relevant factors to be taken into account, including There is very little literature on the appropriate cutoff point for the population involved and the setting. A balance must the HSCL-10 and the HSCL-5, Strand et al. recommend 1.85 for be achieved between psychometric properties and such the HSCL-10 and 2.00 for the HSCL-5 (22). By interpreting the pragmatic characteristics as self-administration, number of results from ROC curves, the accuracy for various cutoffs was items, simplicity/interpretability of scores, and accessibility (61). explored, and optimal cutoff values were obtained considering Scales with a reduced number of items may be the best option the maximum value of Youden Index. The optimum cutoff that as they are practical and feasible within the PC setting without we calculated for the study population is very close to that excessively losing reliability and validity. in the literature (22). Therefore, we considered that those of These findings indicate that the HSCL-10 and HSCL-5 1.85 and 2.00 for the HSCL-10 and HSCL-5, respectively, were questionnaires show adequate reliability and validity in order to appropriate to use showing adequate sensitivity, specificity, and be employed in PC to detect and evaluate depressive symptoms. AUC. The negative predictive values were >90% for the two With such a short number of items they are timesaving short versions and both genders whilst the positive predictive and facilitate the detection of cases of depression that could values were low. Such a finding reinforces the need to complete otherwise go unnoticed. the diagnosis of depression through a clinical interview. Other authors have analyzed validity between the HSCL-25 and the two short versions with ROC curves (22). As we had Conclusion the semi-structured CIDI interview for all our participants, we contrasted it against the two short versions with ROC curves. The Spanish versions of the HSCL-10 and HSCL-5, This is one of the strengths of the present study as sometimes especially the HSCL-10, are reliable and valid tools to detect there is a lack of gold-standard and the full version is used depressive symptoms and can be used in Primary Care settings. as a reference to assess validity. In other studies, only those participants who have a positive result on the scale and a small sample of those with a negative one, undergo the clinical Data availability statement interview of reference. The AUC was 0.88 for the HSCL-10 and 0.85 for the HSCL-5, both above the 0.75 cutoff considered to The raw data supporting the conclusions of this article will be of clinical utility, and greater than the 0.80 which confers a be made available by the authors, without undue reservation. “good” classification (≥0.90 is considered “excellent”) in terms of discriminative properties of the diagnostic accuracy (58). A study carried out by Haavet et al. (33) also obtained an AUC Ethics statement of 0.88 when comparing the HSCL-10 with the CIDI, other studies with similar methodology have obtained a lower AUC The studies involving human participants were reviewed (23). For both short versions the AUC was greater in men, and approved by the Clinical Research Ethics Committee of thus the probability of accuracy in diagnosis in males is greater. the University Institute Foundation for Research into Primary Such a gender difference has also been reported for the longer Health Care Jordi Gol from Barcelona (reference: P16/025). HSCL-25 (18). All the participants gave their written informed consent, The main limitation of our study is that the psychometric and the trial was conducted in accordance with the ethical properties of the two short versions were evaluated based on the standards set out in the Declaration of Helsinki. Data were responses to the 25-item scale as performed by other authors anonymously processed and were used only for the study (22, 27). The study population came from the EIRA study (34, objectives. Personal data confidentiality was ensured as set 35) and were patients aged 45–75 years presenting an unhealthy out in applicable regulations. Access to medical records was Frontiers in Medicine 10 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 gained as indicated in the legislation currently in force. The Network for Research on Chronicity, Primary Care, and Health patients/participants provided their written informed consent to Promotion (RICAPPS), with reference RD21/0022. This study participate in this study. received funding from Novartis. The funder was not involved in the study design, collection, analysis, interpretation of data, and the writing of this article or the decision to submit it Author contributions for publication. MR-B, MF-S-M, AC, JL, and PN: conceptualization. EM and IG-G: data collection. MR-B, MF-S-M, and AC: formal analysis. Acknowledgments MR-B, MF-S-M, and EP-R: writing—original draft preparation. All authors have reviewed and agreed to the published version We thank to the Family Practice Depression and of the manuscript. Multimorbidity group (FPDM) of the European General Practice Research Network (EGPRN). To the REDIAPP network, family physicians, other professionals, and the patients who participated in the study. Funding This study was supported by the Carlos III Health Institute, the Spanish Ministry of Economy and Competitiveness Conflict of interest via a health research grant (PI15/00114, PI15/00565, PI15/00762, PI15/01072, PI15/00896, PI15/01412, PI15/01151, The authors declare that the research was conducted in the PI15/00519, and PI15/01133) through the Research Network absence of any commercial or financial relationships that could in Preventive Activities and Health Promotion in Primary be construed as a potential conflict of interest. Care (redIAPP), (RD12/0005/0001, RD16/0007/0001, RD16/0007/0002, RD16/0007/0003, RD16/0007/0004, RD16/00 07/0005, RD16/0007/0006, RD16/0007/0008, RD16/0007/0009, Publisher’s note RD16/0007/0010, RD16/0007/0012, RD16/0007/0013, and RD16/0007/0015), the European Union ERDF funds, (European All claims expressed in this article are solely those of the Regional DevelopmentFund) and the Health Department authors and do not necessarily represent those of their affiliated (SLT002/16/00112) of the Generalitat de Catalunya. This study organizations, or those of the publisher, the editors and the received a predoctoral grant from the University Institute of reviewers. Any product that may be evaluated in this article, or Research in Primary Care IDIAP Jordi Gol (Spain) and another claim that may be made by its manufacturer, is not guaranteed grant from the Autonomous University of Barcelona (UAB)– or endorsed by the publisher. Novartis for research in Family and Community Medicine (reference: 7Z19/019), both in 2019. The study obtained funding from the Spanish Society of Family Medicine (semFYC) as it was Supplementary material awarded a predoctoral grant, “Isabel Fernández.” The project also received a research grant from the Carlos III Institute of The Supplementary Material for this article can be Health, Ministry of Science and Innovation (Spain) co-funded found online at: https://www.frontiersin.org/articles/10.3389/ with European Union—NextGenerationEU funds, through the fmed.2022.1014340/full#supplementary-material References 1. Nabbe P, Le Reste J, Guillou-Landreat M, Munoz Perez M, Argyriadou S, 4. Kroenke K. Depression screening and management in primary care. Fam Claveria A, et al. Which DSM validated tools for diagnosing depression are usable Pract. (2018) 35:1–3. in primary care research? A systematic literature review. Eur Psychiatry. (2017) 5. Garrido-Elustondo S, Reneses B, Navalón A, Martín O, Ramos I, Fuentes M. 39:99–105. doi: 10.1016/j.eurpsy.2016.08.004 Capacidad de detección de patología psiquiátrica por el médico de familia. Aten 2. Siu A, Bibbins-Domingo K, Grossman D, Baumann L, Davidson K, Ebell Primaria. (2016) 48:449–57. doi: 10.1016/j.aprim.2015.09.009 M, et al. Screening for depression in adults US preventive services task force 6. Latorre Postigo J, Navarro Bravo B, Parra Delgado M, Salguero J, Mae Wood C, recommendation statement. JAMA J Am Med Assoc. (2016) 315:380–7. doi: 10. Cano Vindel A. Evaluación e intervención de los problemas de ansiedad y depresión 1001/jama.2015.18392 en atención primaria: un problema sin resolver [Assessment and intervention of 3. Smithson S, Pignone M. Screening adults for depression in primary care. Med Anxiety and Depression disorders in Primary Care: an unsolved problem]. Rev Clín Clin North Am. (2017) 101:807–21. doi: 10.1016/j.mcna.2017.03.010 Med Fam. (2012) 5:37–45. doi: 10.4321/s1699-695x2012000100007 Frontiers in Medicine 11 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 7. Tavabie J, Tavabie O. Improving care in depression: qualitative study 27. Schmalbach B, Zenger M, Tibubos A, Kliem S, Petrowski K, Brähler investigating the effects of using a mental health questionnaire. Qual Prim Care. E. Psychometric properties of two brief versions of the Hopkins symptom (2009) 17:251–61. checklist: HSCL-5 and HSCL-10. Assessment. (2019) 28:617–31. doi: 10.1177/ 1073191119860910 8. Mokkink L, Prinsen C, Patrick D, Alonso J, Bouter LM, de Vet H, et al. COSMIN Study Design Checklist for Patient-Reported Outcome Measurement 28. Lien I, Bolstad I, Lien L, Bramness J. Screening for depression in patients in Instruments. La Maddalena: COSMIN (2019). p. 1–32. treatment for alcohol use disorder using the beck depression inventory-II and the Hopkins symptom checklist-10. Psychiatry Res. (2022) 308:114363. doi: 10.1016/J. 9. Derogatis L, Lipman R, Rickels K, Uhlenhuth E, Covi L. The Hopkins PSYCHRES.2021.114363 symptom checklist (HSCL): a self-report symptom inventory. Behav Sci. (1974) 19:1–15. 29. Nesterko Y, Jäckle D, Friedrich M, Holzapfel L, Glaesmer H. Factors predicting symptoms of somatization, depression, anxiety, post-traumatic stress 10. Glaesmer H, Braehler E, Grande G, Hinz A, Petermann F, Romppel disorder, self-rated mental and physical health among recently arrived refugees in M. The German version of the Hopkins symptoms checklist-25 (HSCL- Germany. Confl Health. (2020) 14:44. doi: 10.1186/s13031-020-00291-z 25) - factorial structure, psychometric properties, and population-based norms. Compr Psychiatry. (2014) 55:396–403. doi: 10.1016/j.comppsych.2013. 30. Harris S, Sandal G. COVID-19 and psychological distress in Norway: the 08.020 role of trust in the healthcare system. Scand J Public Health. (2021) 49:96–103. doi: 10.1177/1403494820971512 11. Nettelbladt P, Hansson L, Stefansson C, Borgquist L, Nordström G. Test characteristics of the Hopkins symptom check list-25 (HSCL-25) in Sweden, 31. Karlsson Stafseth S, Skogstad L, Raeder J, Strand Hovland I, Hovde using the present state examination (PSE-9) as a caseness criterion. Soc Psychiatry H, Ekeberg Ø, et al. Symptoms of anxiety, depression, and post-traumatic Psychiatr Epidemiol. (1993) 28:130–3. doi: 10.1007/BF00801743 stress disorder in health care personnel in Norwegian ICUs during the first wave of the COVID-19 pandemic, a prospective, observational cross-sectional 12. Lundin A, Hallgren M, Forsell Y. The validity of the symptom checklist study. Int J Environ Res Public Heal. (2022) 19:7010. doi: 10.3390/ijerph191 depression and anxiety subscales: a general population study in Sweden. J Affect 27010 Disord. (2015) 183:247–52. doi: 10.1016/j.jad.2015.05.024 32. Frounfelker R, Li Z, Santavicca T, Miconi D, Rousseau C. Latent class 13. Nabbe P, Le Reste J, Guillou-Landreat M, Gatineau F, Le Floch B, Montier T, analysis of COVID-19 experiences, social distancing, and mental health. Am J et al. The French version of the HSCL-25 has now been validated for use in primary Orthopsychiatry. (2022) 92:121–32. doi: 10.1037/ORT0000593 care. PLoS One. (2019) 14:e0214804. doi: 10.1371/journal.pone.0214804 33. Haavet O, Sirpal M, Haugen W, Christensen K. Diagnosis of depressed young 14. Kamacooko O, Bagiire D, Kasujja F, Mirembe M, Seeley J, King R. Prevalence people in primary health care-a validation of HSCL-10. Fam Pract. (2011) 28:233–7. of probable depression and factors associated with mean Hopkins symptom doi: 10.1093/fampra/cmq078 checklist (HSCL) depression score among young women at high risk aged 15–24 years in Kampala, Uganda. PLoS One. (2022) 17:e0270544. doi: 10.1371/JOURNAL. 34. Zabaleta-Del-Olmo E, Pombo H, Pons-Vigués M, Casajuana-Closas M, PONE.0270544 Pujol-Ribera E, López-Jiménez T, et al. Complex multiple risk intervention to promote healthy behaviours in people between 45 to 75 years attended in primary 15. Nabbe P, Le Reste J, Guillou-Landreat M, Beck-Robert E, Assenova R, Lazic health care (EIRA study): study protocol for a hybrid trial. BMC Public Health. D, et al. One consensual depression diagnosis tool to serve many countries: a (2018) 18:874. doi: 10.1186/s12889-018-5805-y challenge! A RAND/UCLA methodology. BMC Res Notes. (2018) 11:4. doi: 10. 1186/s13104-017-3111-x 35. Represas-Carrera F, Couso-Viana S, Méndez-López F, Masluk B, Magallón- Botaya R, Recio-Rodríguez J, et al. Effectiveness of a multicomponent intervention 16. Roncone R, Gone J, Barlati S, Nabbe P, Reste L, Lazic K, et al. . Nine forward- in primary care that addresses patients with diabetes mellitus with two or backward translations of the Hopkins symptom checklist-25 with cultural checks. more unhealthy habits, such as diet, physical activity or smoking: multicenter Front Psychiatry. (2021) 1:688154. doi: 10.3389/fpsyt.2021.688154 randomized cluster trial (EIRA Study). Int J Environ Res Public Heal Artic Public 17. Clavería A, Rodríguez-Barragán M, Fernández-San-Martín M, Nabbe P, Le Heal. (2021) 18:5788. doi: 10.3390/ijerph18115788 Reste J, Miguéns-Blanco I, et al. Translation and cross-cultural adaptation into 36. Veijola J, Jokelainen J, Läksy K, Kantojärvi L, Kokkonen P, Järvelin M, et al. Spanish, Catalan and Galician of the Hopkins symptom checklist-25 to identify The Hopkins symptom checklist-25 in screening DSM-III-R axis-I disorders. Nord depression in primary care. Aten Primaria. (2020) 52:539–47. doi: 10.1016/j.aprim. J Psychiatry. (2003) 57:119–23. doi: 10.1080/08039480310000941 2020.05.017 37. Kessler R, Bedirhan Üstün T. The World mental health (WMH) 18. Rodríguez-Barragán M, Fernández-San-martín M, Clavería-Fontán A, survey initiative version of the World Health Organization (WHO) composite Aldecoa-Landesa S, Casajuana-Closas M, Llobera J, et al. Validation and international diagnostic interview (CIDI). Int J Methods Psychiatr Res. (2004) psychometric properties of the Spanish version of the Hopkins symptom checklist- 13:93–121. 25 scale for depression detection in primary care. Int J Environ Res Public Health. (2021) 18:7843. doi: 10.3390/ijerph18157843 38. Harkness J, Pennell B, Villar A, Gebler N, Aguilar-Gaxiola S, Bilgen I. Translation procedures and translation assessment in the World Mental Health 19. Müller J, Postert C, Beyer T, Furniss T, Achtergarde S. Comparison of survey initiative. In: Kessler RC, Bedirhan T editors. The WHO World Mental eleven short versions of the symptom checklist 90-revised (SCL-90-R) for use in Health Surveys: Global Perspectives on the Epidemiology of Mental Disorders. the assessment of general psychopathology. J Psychopathol Behav Assess. (2009) New York, NY: Cambridge University Press, World Health Organization (2008). 32:246–54. doi: 10.1007/s10862-009-9141-5 p. 91–113. 20. Tambs K, Røysamb E. Selection of questions to short-form versions of 39. Terwee C, Bot S, de Boer M, van der Windt D, Knol D, Dekker J, et al. original psychometric instruments in MoBa ∗ . Nor Epidemiol. (2014) 24:195–201. Quality criteria were proposed for measurement properties of health status 21. Tambs K, Moum T. How well can a few questionnaire items indicate anxiety questionnaires. J Clin Epidemiol. (2007) 60:34–42. doi: 10.1016/j.jclinepi.2006. and depression? Acta Psychiatr Scand. (1993) 87:364–7. 03.012 22. Strand B, Dalgard O, Tambs K, Rognerud M. Measuring the mental health 40. Ferrari A. Global, regional, and national burden of 12 mental disorders in 204 status of the Norwegian population: a comparison of the instruments SCL-25, SCL- countries and territories, 1990-2019: a systematic analysis for the Global burden 10, SCL-5 and MHI-5 (SF-36). Nord J Psychiatry. (2003) 57:113–8. doi: 10.1080/ of disease study 2019. Lancet Psychiatry. (2022) 9:137–50. doi: 10.1016/S2215- 08039480310000932 0366(21)00395-3 23. Sirpal M, Haugen W, Sparle K, Haavet O. Validation study of HSCL-10, 41. Portellano-Ortiz C, Garre-Olmo J, Calvó-Perxas L, Conde-Sala J. Depresión HSCL-6, WHO-5 and 3-key questions in 14-16 year ethnic minority adolescents. y variables asociadas en personas mayores de 50 años en España. Rev Psiquiatr y BMC Fam Pract. (2016) 17:7. doi: 10.1186/s12875-016-0405-3 Salud Ment. (2018) 11:216–26. doi: 10.1016/j.rpsm.2016.10.003 24. Søberg Finbråten H, Kleppang A, Steigen A. Using the Rasch measurement 42. Huang E, Cheung J, Liu J, Kwan R, Lam S. Groningen frailty indicator– theory to assess the psychometric properties of the Hopkins symptom checklist-10 Chinese (GFI-C) for pre-frailty and frailty assessment among older people living in adolescents. Health Qual Life Outcomes. (2021) 19:248. doi: 10.1186/s12955-021- in communities: psychometric properties and diagnostic accuracy. BMC Geriatr. 01884-9 (2022) 22:788. doi: 10.1186/s12877-022-03437-1 25. Kleppang A, Hagquist C. The psychometric properties of the Hopkins 43. Jackson D, Gillaspy J, Purc-Stephenson R. Reporting practices in symptom checklist-10: a Rasch analysis based on adolescent data from Norway. confirmatory factor analysis: an overview and some recommendations. Psychol Fam Pract. (2016) 33:740–5. doi: 10.1093/FAMPRA/CMW091 Methods. (2009) 14:6–23. doi: 10.1037/a0014694 26. Ljoså C, Tyssen R, Lau B. Mental distress among shift workers in Norwegian 44. de Vet H, Terwee C, Mokkink L, Knol D. Measurement in Medicine. Practical offshore petroleum industry - relative influence of individual and psychosocial Guides to Biostatistics and Epidemiology. Cambridge: Cambridge University Press, work factors. Scand J Work Environ Heal. (2011) 37:551–5. doi: 10.5271/sjweh.3191 World Health Organization (2018). 338 p. Frontiers in Medicine 12 frontiersin.org Rodríguez-Barragán et al. 10.3389/fmed.2022.1014340 45. Viladrich C, Doval E, Penelo E. Fiabilidad y Validez. 12th ed. Bellaterra: 54. Petrowski K, Zenger M, Schmalbach B, Bastianon C, Strauss B. Psychometric Universitat Autònoma de Barcelona Laboratori d’Estadística Aplicada i de properties and validation of the English version Giessen subjective complaints list Modelització (2019). (GBB-8). BMC Psychol. (2022) 10:60. doi: 10.1186/s40359-022-00741-8 46. Brehaut, E, Neupane D, Levis B, Wu Y, Sun Y, Ioannidis J, et al. Optimal” 55. Nunnally J. Psychometric Theory. 2nd ed. New York, NY: McGraw-Hill cutoff selection in studies of depression screening tool accuracy using the PHQ- (1978). 9, EPDS, or HADS-D: a meta-research study. Int J Methods Psychiatr Res. 56. Martinussen L, Šaltytë Benth J, Almdahl I, Borza T, Selbæk G, Mcpherson B, (2022):e1956. doi: 10.1002/MPR.1956 et al. The effect of anxiety on cognition in older adult inpatients with depression: 47. Søgaard A, Bjelland I, Tell G, Røysamb E. A comparison of the CONOR results from a multicenter observational study. Heliyon. (2019) 5:e02235. doi: 10. mental health index to the HSCL-10 and HADS. Nor Epidemiol. (2003) 13:279–84. 1016/j.heliyon.2019.e02235 48. Violán Fors C, Odriozola G, Zabaleta-del-Olmo E, Moral E. La investigación 57. Cohen J, Korevaar D, Altman D, Bruns D, Gatsonis C, Hooft L, en atención primaria como área de conocimiento. Informe SESPAS 2012. Gac Sanit. et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: (2012) 26:76–81. doi: 10.1016/j.gaceta.2011.05.018 explanation and elaboration. BMJ Open. (2016) 6:e012799. doi: 10.1136/bmjopen- 2016 49. Reuland D, Cherrington A, Watkins G, Bradford D, Blanco R, Gaynes B. Diagnostic accuracy of Spanish language depression-screening instruments. Ann 58. Polo T, Miot H. Aplicações da curva ROC em estudos clínicos e Fam Med. (2009) 7:455–62. doi: 10.1370/afm.981 experimentais. J Vasc Bras. (2020) 19:e20200186. doi: 10.1590/1677-5449.20 50. Sanz J, Garcia-Vera M, Espinosa R, Fortún M. Adaptación española del 0186 inventario para la depresión de beck-II (BDI-II): 3. Propiedades psicométricas en 59. Meader N, King K, Moe-Byrne T, Wright K, Graham H, Petticrew M, pacientes con trastornos psicológicos. Clín Salud. (2005) 16:121–42. et al. A systematic review on the clustering and co-occurrence of multiple 51. Fernández-San Martín M, Andrade C, Molina J, Muñoz P, Carretero B, risk behaviours. BMC Public Health. (2016) 16:657. doi: 10.1186/s12889-016- Rodríguez M, et al. Validation of the Spanish version of the geriatric depression 3373-6 scale (GDS) in primary care. Int J Geriatr Psychiatry. (2002) 17:279–87. doi: 10. 60. Oliván-Blázquez B, Aguilar-Latorre A, Motrico E, Gómez-Gómez 1002/gps.588 I, Sarasa-Bosque C, Sanchez-Perez Á, et al. The Relationship between 52. Kakemam E, Navvabi E, Albelbeisi A, Saeedikia F, Rouhi A, Majidi S. adherence to the mediterranean diet, intake of specific foods and depression Psychometric properties of the Persian version of depression anxiety stress scale-21 in an adult population (45–75 Years) in primary health care. A cross- items (DASS-21) in a sample of health professionals: a cross-sectional study. BMC sectional descriptive study. Nutrients. (2021) 13:2724. doi: 10.3390/nu1308 Health Serv Res. (2021) 22:111. doi: 10.1186/s12913-022-07514-4 2724 53. Kroenke K, Spitzer R, Williams J, Löwe B. An ultra-brief screening scale 61. Kroenke K, Monahan P, Kean J. Pragmatic characteristics of patient-reported for anxiety and depression: the PHQ–4. Psychosomatics. (2009) 50:613–21. doi: outcome measures are important for use in clinical practice. J Clin Epidemiol. 10.1016/S0033-3182(09)70864-3 (2016) 68:1085–92. doi: 10.1016/j.jclinepi.2015.03.023 Frontiers in Medicine 13 frontiersin.org