Palmieri et al. BMC Nursing (2020) 19:23 https://doi.org/10.1186/s12912-020-00419-9 RESEARCH ARTICLE Open Access Hospital survey on patient safety culture (HSOPSC): a multi-method approach for target-language instrument translation, adaptation, and validation to improve the equivalence of meaning for cross-cultural research Patrick A. Palmieri1,2,3,4* , Juan M. Leyva-Moral5,6,7 , Doriam E. Camacho-Rodriguez7,8 , Nina Granel-Gimenez5 , Eric W. Ford9 , Kathleen M. Mathieson2 and Joan S. Leafman2 Abstract Background: The Hospital Survey on Patient Safety Culture (HSOPSC) is widely utilized in multiple languages across the world. Despite culture and language variations, research studies from Latin America use the Spanish language HSOPSC validated for Spain and the United States. Yet, these studies fail to report the translation method, cultural adaptation process, and the equivalence assessment strategy. As such, the psychometric properties of the HSOPSC are not well demonstrated for cross-cultural research in Latin America, including Peru. The purpose of this study was to develop a target-language HSOPSC for cross-cultural research in Peru that asks the same questions, in the same manner, with the same intended meaning, as the source instrument. Methods: This study used a mixed-methods approach adapted from the translation guideline recommended by Agency for Healthcare Research and Quality. The 3-phase, 7-step process incorporated translation techniques, pilot testing, cognitive interviews, clinical participant review, and subject matter expert evaluation. Results: The instrument was translated and evaluated in 3 rounds of cognitive interview (CI). There were 37 problem items identified in round 1 (14 clarity, 12 cultural, 11 mixed); and resolved to 4 problems by round 3. The pilot-testing language clarity inter-rater reliability was S-CVI/Avg = 0.97 and S-CVI/UA = 0.86; and S-CVI/Avg = 0.96 and S-CVI/UA = 0.83 for cultural relevance. Subject matter expert agreement in matching items to the correct dimensions was substantially equivalent (Kappa = 0.72). Only 1 of 12 dimensions had a low Kappa (0.39), borderline fair to moderate. The remaining dimensions performed well (7 = almost perfect, 2 = substantial, and 2 = moderate). (Continued on next page) * Correspondence: Patrick.Palmieri@jbisa.org; patrick.palmieri@uwiener.edu.pe; ppalmieri@atsu.edu; patrick.palmieri@mail.waldenu.edu 1 Office of the Vice Chancellor for Research, Universidad Norbert Wiener, Av. Arequipa 444, 15046 Lima, Peru 2 College of Graduate Health Studies, A. T. Still University, 800 West Jefferson Street, Kirksville, MO 63501, USA Full list of author information is available at the end of the article © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Palmieri et al. BMC Nursing (2020) 19:23 Page 2 of 13 (Continued from previous page) Conclusions: The HSOPSC instrument developed for Peru was markedly different from the other Spanish-language versions. The resulting items were equivalent in meaning to the source, despite the new language and different cultural context. The analysis identified negatively worded items were problematic for target-language translation. With the limited literature about negatively worded items in the context of cross-cultural research, further research is necessary to evaluate this finding and the recommendation to include negatively worded items in instruments. This study demonstrates cross-cultural research with translated instruments should adhere to established guidelines, with cognitive interviews, based on evidence-based strategies. Keywords: Hospital survey on patient safety culture, Patient safety, Organizational culture, Instrument translation, Instrument adaptation, Instrument validation, Cross cultural research, Agency for Healthcare Research and Quality, Spanish, Peru Background processes, and practices [24]. The HSOPSC is a validated As many as 16% of hospitalized patients experience a tool to measure the effectiveness of work environment medial error during their health encounter [1]. Hospital and organizational process associated with preventing culture and patient safety are interrelated as organizational the types of errors linked to consequential adverse failures and system driven errors contribute to the unin- events [25]. When administered yearly, the HSOPSC can tended events that produce poor quality outcomes [2]. provide leaders with a proxy measurement for the effect- Prominent international agencies, such as the World iveness of their quality improvement efforts focused on Health Organization (WHO) and the United States achieving patient safety [13]. Agency for Healthcare Research and Quality (AHRQ), Early instruments to measure safety culture were devel- recommend hospitals address this problem by improving oped in the late 1980’s and into early 1990’s [14] but did their organizational cultures [3]. As such, health services not gain popularity due to poorly performing psychomet- researchers are interested in the intersection of ric properties [15]. In a seminal review of contemporary organizational culture with patient safety [4, 5]. Safety cul- instruments [26], multiple safety culture instruments were ture can be defined as the overarching but emergent identified but varied considerably in the general character- healthcare property where professional attitudes and work istics, dimensions covered, and psychometric properties. climates result in the degree of system reliability and resili- Furthermore, the items, and associated dimensions, were ence to adverse outcomes [6]. not derived from a theoretically constructed framework Clinical quality and patient safety outcomes are linked [13]. In response to the need for a standard instrument to to the organizational culture dimensions that can be mea- measure safety culture, the AHRQ commissioned the de- sured with safety culture instruments for hospitals [7]. velopment of a psychometrically valid and reliable instru- Safety culture measures are correlated with employee per- ment [27–29], called the HSOPSC. formance (e.g. safety behavior), process and system errors, The HSOPSC, developed in English [28] for use in and accident rates across industries [8] and cultures [9] American hospitals, has demonstrated excellent psycho- with similar results from the health sector [10]. For ex- metric properties [27]. The instrument has been dissem- ample, researchers using the Hospital Survey on Patient inated to other countries, in multiple languages [8, 24, Safety Culture (HSOPSC) have reported strong correla- 30–36]; however, published studies generally neglect to tions between safety culture, adverse event frequency, and report the instrument translation method and validation patient outcomes [11, 12]. For these reasons, the measure- data. When provided, the information is either limited ment of safety culture has become the prerequisite for or poor quality such as reporting a simple post hoc psy- continuous quality improvement efforts to provide leaders chometric analysis. In this regard, a Spanish language with the essential feedback that stimulates organizational version of the instrument is available for use with Span- improvement [13–15]. ish speaking health workers in the United States [37]. In developing countries basic hospital safety indicators This version has also been used in Mexico [38, 39], are largely incomplete or unavailable [16–18]. The Colombia [40], and most recently Peru [24]. Also, a dif- sparse available data indicates adverse event prevalence ferent Spanish-language version of the instrument [41] in South American countries, including Peru [19–21], is was developed for use in Spain [42]. However, studies much higher than developed countries [22, 23]. Patient applying either of these Spanish language instruments safety is an emerging focus in the Peruvian health sector, for cross-culture research do not report the validation as demonstrated by the lack of basic safety programs, method and provide limited psychometric data about Palmieri et al. BMC Nursing (2020) 19:23 Page 3 of 13 reliability. According to the basic recommended trans- qualitative and quantitative methods and techniques lation strategies published by the AHRQ [43], there is [55, 56, 61, 63], including feedback questionnaires, no validated Spanish language version of the HSOPSC pilot testing, expert panels, and cognitive interviews reported in the literature for cross-cultural research [63–65]. As such, using an iterative mixed method in Peru, as well as in other countries in Latin Amer- approach to translate instruments [46], with cognitive ica. As the Spanish language differs between countries interviews [57, 66] ensures the resulting instrument and the meaning of words differs across cultures, a will have equivalence [51]; one that asks the same validated Spanish language version of the HSOPSC questions, in the same manner, with the same needs to be developed for cross-cultural research in intended meaning, as the source instrument. Peru, that confirms construct applicability and survey With globalization in the context of cross-cultural item integrity [44, 45]. research, evidence-based methods are required to pro- Researchers working with Spanish speaking popula- duce equivalent target-language instrument transla- tions [46, 47] have reported the traditional forward- tions from the sources [52, 67]. As such, this is a reverse translation technique does not result in a valid mixed-method approach adapted from the translation target-language instrument [48, 49]. However, the item guidelines recommended by AHRQ [43, 68, 69] with meanings, dimension integrity, and construct validity reference to other best practices and strategies [63, need to remain constant regardless of the language 70–72]. The approach adhered to the adapted version across cultures [50], with an attention to eliminating bias of the equivalence criteria for cross-cultural research and maximizing equivalence [51]. In order to study pa- with instruments [50, 55, 63, 73], see Table 1. The tient safety culture in a global perspective, the HSOPC AHRQ provided written permission by email (CRM: needs to be translated from the source language (original 00350304) to use the HSOPSC in English and Spanish language) into the target language (translated language) for this research project. In addition, permission was without losing the meaning [52] and context of the items granted to publish the resulting instrument from this and associated scales and/or dimensions during the study as well as the existing Spanish and English translation process [51]. As such, the purpose of this HSOPSC instruments. study was to test a mixed method approach for target- language instrument translation to produce a valid Data collection HSOPSC translation for cross-cultural research in Peru. Per the recommendations for cross-cultural instrument Developing a psychometrically sound target-language in- research where the source country, culture, and language strument for cross-cultural research from an English- are different than the target [72], the data collection language source, originally validated for use in the United process required three phases, including: Instrument States, is not straightforward [53]. In cross-cultural research, translation, Cultural-adaptation, content validation and the assumption that all instruments will automatically be equivalence [47, 72]. These phases included a forward equivalent across groups does not hold [54]. However, in- and reverse translation, cognitive interviews, targeted struments that are products of their local environments at participant review, structured pilot test, content valid- the time they were developed are likely to be more ation, and expert evaluation of equivalence. The work reliable [55]. Yet, researchers consistently fail to de- completed in each of the phases are described next. scribe how the HSOPSC was translated and validated [56] for cross-cultural comparability and compatibility Participants [57]. For cross-cultural research, the technical and se- Phase 1, the translation process, was completed by three mantic equivalence and cultural relevance of each bilingual (English/Spanish) professionals, with at least an item needs to be evaluated prior to data collection undergraduate degree in linguistics and extensive experi- [58–60]. Without a sufficient pre-data collection ence in translation and interpretation. Phase 2 included evaluation to ensure the correct representation of the nine participants (2 nurses and 7 physicians), called clin- instrument [60], the resulting factor analyses post- ical practice experts (CPEs), purposefully selected from data collection will be flawed and less rigorous [56]. licensed health professionals in Peru with current hos- pital work experience, fluency in Spanish, and advanced Methods English skills as self-reported and observed by the prin- Despite the increased adaptation of English language in- cipal researcher during a brief interview. Phase 3, struments for cross-cultural research, there is not con- included 7 participants (4 nurses, 2 physicians, 1 sensus about the gold standard method for instrument pharmacist), called subject matter experts (SMEs), pur- translation and validation, including cultural adaptation posefully selected as recommended by Grant and Davis [56, 61, 62]. However, proper cross-cultural research [74], intermediate to advanced English was required. All with instrument translation generally requires multiple participants verbally consented to participate in this Palmieri et al. BMC Nursing (2020) 19:23 Page 4 of 13 Table 1 Equivalence Criteria for Cross-Cultural Research with Instruments Criteria Definition Process Content The content of each item of the instrument is relevant to the phenomena of each — Research Team Experts. Equivalence culture being studied. — Clinical Practice Experts. — Subject Matter Experts. — Content validity index score. — Annotated survey dimension document. Semantic The meaning of each item is the same in each culture after translation into the — Translation guide from AHRQ. Equivalence language and idiom (written or oral) of each culture. — Qualified / experienced translators. — Forward- and reverse-translation. — Pilot test (cultural relevance & readability). — Confirmation of translation: Cognitive interviews and expert reviews. Technical The method of assessment is comparable in each culture with respect to the data — Translation guide from AHRQ Equivalence that it yields. — Experienced translators. — Subject Matter Experts. — Pilot test (evaluation scores). — Cognitive interviews. Criterion The interpretation of the measurement of the variable remains the same when — Research Team Experts. Equivalence compared with the norm for each culture studied. — Subject Matter Experts. — Pilot test (evaluation scores). — Cognitive interviews. Conceptual The instrument is measuring the same theoretical construct in each culture. — Translation guide from AHRQ. Equivalence — Qualified / experienced translators. — Forward- and reverse-translation. — Item to dimension selection/match. — Dual scoring process with content validity index and pilot test.. Adapted from Squires et al. [63], and Flaherty et al. [50] study approved by the A.T. Still University Institutional instrument content was generally adaptable for transla- Review Board (Protocol #01146). tion and implementation in Peru. As such, the instru- ment was sent for translation without changes [74]. Instrument HSOPSC The HSOPSC is a 42-item instrument grouped into 12 Step 2: forward translations with synthesis composite dimensions and nine non-dimensional items, The instrument was independently forward-translated including two safety assessment items and seven demo- from American English to Peruvian Spanish by two graphic items [69]. Each dimension is represented by translators [76, 77]. Then, the two versions were synthe- three to five items measured with a 5-point Likert scale sized and consolidated into a single instrument [75] by to ascertain agreement (strongly disagree to strongly the two healthcare experts and the primary investigator, agree) or frequency (never to always). The instrument all knowledgeable about the instrument properties and includes 59.52% positively worded items and 48% nega- scientific foundation. The translators were involved in tively worded items. The outcome measures are the two this process specifically for grammar and language single-item responses inquiring about the number of clarifications. events reported within the past 12 months and the over- all patient safety score (excellent to failing). For the re- Step 3: reverse translation with reconciliation ported error questions, errors are defined as any type, The goal of the reverse-translation process was to estab- without regard to harm. lish a conceptual rather than a literal “word-by-word” meaning [78]. An independent bilingual translator Instrument translation – phase 1 reverse-translated the synthesized instrument from Step 1: instrument review and translator selection Peruvian Spanish to American English. Then, the The source American English version of the HSOPC reverse-translated instrument was examined, in compari- was reviewed by two bilingual healthcare experts with son to the original instrument, by the two healthcare ex- the primary investigator to establish the content equiva- perts and the primary investigator. Although discrete lence, or the relevance and sensitivity [50, 75]. Although differences were identified in the reverse-translation, some challenging terms and phrases were noted in these were primarily related to nearly equivalent verb se- items, through discussion, the experts determined the lections. There were five instances where phrases Palmieri et al. BMC Nursing (2020) 19:23 Page 5 of 13 required clarification to achieve the same meaning in identify translation deficiencies by asking CPEs to de- Spanish as expressed in English. However, these clarifi- scribe their understanding of the item in Spanish and cations were noted for discussion during the cognitive then again after considering the original question in interview phase. Again, the translator provided clarifica- English. Furthermore, the other two Spanish HSOPSC tions specific to grammar and language. versions, United States [37] and Spain [41], were incor- porated into the process as resources to review, discuss, Cultural-adaptation – phase 2 analyze, and refine problematic items. The CPEs were Although a pilot test with a bilingual version of the instru- also asked to identify items with unfamiliar or inappro- ment has been recommended to determine the equiva- priate grammar and syntax. Cognitive interview data was lence between the newly translated and the original thematically coded to identify problematic items [81]. instruments [79], there were not enough bilingual partici- Notes were collected and compiled throughout the pants readily accessible for this study. Alternatively, this process. study used a pretest to review and refine the instrument through a series of three rounds. The cognitive interviews Step 6. Research team review with item revision followed each pretest to assesses the four stages of the The data from the pilot testing for each round was indi- participant engagement in completing the instrument, vidually and aggregately reviewed by the research team with including: Comprehension, retrieval, judgement, and the primary focus on improving items with ratings less than responding [80]. The procedure is described below. four. The notes from the cognitive interviews were also ref- erenced when reviewing each item. Revisions for problems Step 4. Pre-test with items was completed to improve the item performance Nine CPEs engaged in the pilot testing with three CPE per in the next round. The translators from the forward- and round across three sequential rounds. First, the CPEs com- reverse-translation were consulted with specific questions pleted the instrument without interruption. Then, the about word selections and phrase clarification. All revisions CPEs were asked for their general perspective about the in- were noted in the Excel spreadsheet to provide a record of strument. Finally, all CPEs were asked to complete the new the progressive evolution of each item across rounds. Each instrument and for a final assessment for item clarity and round was conducted sequentially and distinctly but all cultural equivalence. The assessment required each CPE to items were compared between and across rounds. respond to each item and then rank the item on a five- point Likert scale for language clarity (5 – completely read- able and understandable, 4 – mostly readable and under- Content validation and equivalence evaluation – phase 3 standable, 3 – readable and understandable, 2 – somewhat Step 7a. Subject matter expert evaluation of content readable and somewhat understandable; and 1 – not validity readable and understandable) and cultural equivalence The SMEs were provided the final instrument for review (5 – completely culturally relevant, 4 – mostly cultur- with a content validity questionnaire [82–87]. Each item ally relevant, 3 – culturally relevant, 2 – somewhat in the preliminary final instrument was rated on a 4 culturally relevant, 1 – not culturally relevant). This point-Likert scale (1-irrelevant, 2-little relevance; 3- coding schematic was necessary to identify the aggre- relevant; and 4-extremely relevant). In addition, the item gate level of item clarity and cultural relevance across evaluation used during the pilot testing for language participants. The assessment data was entered into an clarity (readability/understandability) and cultural rele- Excel database for the analysis. The item specific vance (context) was incorporated with dichotomous ac- scores guided the primary focus of the first round of ceptable or unacceptable scoring. Finally, there was a cognitive interviews; scores less than four were con- space for open ended comments for each item and at sidered opportunities for improvement. the end of the questionnaire. The content validity index (CVI) and the item evaluation for language and context Step 5. Cognitive interviews was calculated per item [83]. For analysis, the rankings Once the data entry was completed, the primary investi- were merged into two categories as items scored one gator conducted a cognitive interview with each CPE and two were considered irrelevant and items scored [69, 81], through a structured but open-ended discus- three and four were considered relevant [82]. Then, the sion, to understand how participants read, compre- CVI was calculated individually for each item (propor- hended, and responded to each item [57]. A cognitive tion of experts rating the item as relevant). The items interview script guided the content probes for additional were then calculated as part of the twelve dimensions, as evaluation of items scoring three or less in the first well as for the total instrument as the mean CVI [86]. A round and items scoring four or less in the subsequent significant CVI for this process was set, a priori, at 0.70 rounds. This probative process was constructed to or greater [82, 85, 86]. Palmieri et al. BMC Nursing (2020) 19:23 Page 6 of 13 Step 7b. Subject matter expert evaluation of equivalence Table 2 Participant Characteristics for Cognitive Interviews (N = The SMEs were provided with the final items from the 9) instrument and asked to indicate which items are associ- Mean Age (SD) 44.6 (6.7) ated with which dimensions. Each expert was required Male 6 (55.6) to select only one of the twelve dimensions for each Health Professions Specialty item, but they were also asked to indicate a secondary Internal medicine 3 (33.3) dimension, with a short rational, in the case they were Gynecology medicine 2 (22.2) unable to easily decide the dimension for the item. Through this process, the equivalence can be estab- Anesthesiology 1 (11.1) lished. The Kappa value was calculated for the SME Pediatric medicine 1 (11.1) agreement in identifying each item with the correct di- Intensive care nurse 1 (11.1) mension. The minimally acceptable Kappa value was set Medical-surgical nurse 1 (11.1) to 0.40 [88–90] as values below this point are not con- Mean Years of Experience (SD) 16.7 (6.5) sidered robust [91, 92]. According to the scale of Landis Hospital Setting a & Koch [93], the strength of the Kappa coefficients were defined in groupings (0.01–0.20 slight; 0.21–0.40 fair; Clinica 9 (100) 0.41–0.60 moderate; 0.61–0.80 substantial; 0.81–0.99 al- EsSalud 6 (66.7) most perfect, and 1.00 perfect). MINSA 5 (55.7) English Comprehension Results Advanced 6 (66.7) Sample characteristics Fluent 3 (33.3) The pre-tests with cognitive interviews were completed Note: Values are expressed as N (%) unless otherwise noted with nine Peruvian health care professionals (6 males a Participants could indicate experience in more than one hospital setting and 3 females) across three rounds with three different participants per round. The mean participant age was 44.6 years (± 6.7) and the mean experience level was relevance (Round 1 = 81.3%). The analysis evidenced al- 16.7 (± 6.5). In the previous five years, all participants though the initial translation was adequate from a lin- worked in private hospitals (known as ‘clínicas’ in Peru), guistics perspective, there were opportunities to improve six also worked in social security hospitals (Social Health the clarity and relevance from the targeted population Insurance of Peru), and five also worked in public hospi- perspective [94, 95]. With focused item revisions be- tals (Ministry of Health of Peru). All participants re- tween rounds, the cumulative item measurements for ported working between two to four different jobs, language clarity and cultural relevance improved by 10% either part-time, full-time, and/or contracted. All partici- in round 2 (79.4 to 89.2%) and improved 5% from round pants were advanced to fluent in English (6 advanced, 2 to round 3 (89.2 to 94.5%). From the interviews, par- and 3 fluent) and all participants were fluent in Peruvian ticipants identified problems understanding negatively Spanish (native language). Table 2 summarizes the worded items. However, the participant feedback re- demographic and professional characteristics. sulted in improvements to all negative items for the final version of the instrument. Table 3 summarizes the ag- Pilot testing and cognitive interviews gregate item analysis for all scores. The pilot testing of the instrument required on average Overall, the cognitive interviews identified translation 18 min (range 15 to 22 min), while the cognitive inter- issues across rounds. For example, Round 1 had 33 views required on average 50 min (range 40 min to 85 problems (language clarity 42.5%, cultural 33.3%, and min). From the item evaluations, the participants identi- general design 24.2%) but this number decreased to only fied multiple problem items: Round 1 there were 37 four problems in Round 3 (50.0, 25.0, and 25.0% problem items (14 unique to language clarity, 12 specific to cultural, and 11 mixed); Round 2 there were 14 prob- Table 3 Participant Item Rating (Aggregate Raw Score by lematic items (6 specific to language clarity, 5 specific to Measure Type by Round) cultural, and 3 mixed); and Round 3 there were four Testing Round Language Claritya Cultural Relevancea Totalb problematic items (2 specific to language clarity, 2 spe- Round 1 201.3 (77.4%) 211.3 (81.3%) 412.7 (79.4%) cific to cultural, and 0 mixed). The type, frequency, and Round 2 227.3 (87.4%) 236.7 (91.0%) 464.0 (89.2%) the mix of issues improved progressively across rounds. Round 3 243.0 (93.5%) 248.3 (95.5%) 491.3 (94.5%) In the aggregate analysis for each round, the language a Score calculation 51 items, 1–5 points per item, % is X points / 255 total clarity issues were the most common with the lowest b Score calculation 51 items, 1–5 points per item for each score, % is X points point percentage (Round 1 = 77.4%) followed by cultural / 510 total Palmieri et al. BMC Nursing (2020) 19:23 Page 7 of 13 Table 4 Problems Identified from Cognitive Interviews (Number Table 5 presents examples of the language clarity issues and Percent) identified by the participants. Regarding the cultural is- Round a Language Clarity Cultural Relevance Design Total sues, the predominant problems were related to defining Round 1 14 (42.5%) 11 (33.3%) 8 (24.2%) 33 (100%) definitions of concepts or seemingly general knowledge Round 2 7 (43.8%) 5 (31.2%) 4 (25.0%) 16 (100%) across cultures, with two specific causes: 1) Organizational constructs that differed between the Round 3 2 (50.0%) 1 (25.0%) 1 (25.0%) 4 (100%) a United States and Peru; and 2) References to health sys- Number reported per identified problem per item regardless of frequency per item tem constructs in the United States that are not present or not well known to professionals in Peru. Table 6 pre- respectively). Table 4 provides a summary of the item is- sents examples of the cultural relevance issues identified sues related to translation, context, and design. The by the participants. remaining four issues identified in Round 3 improved from Round 1 as multiple participants initially rated Content validity and equivalence these items as a one or two and, in the end, only a single From the seven SME invited to participate in the item participant rated one item as a three. The participant re- content review, only six completed the entire process. sponse for the sole issue in Round 3 was reportedly After reviewing the items (N = 42) for clarity, the S-CVI/ based on a negative impression about the item versus a Avg (scale-level content validity index, average) was 0.97 lack of understanding or concern about cultural rele- and the S-CVI/UA (scale-level content validity index, vance. At the conclusion of the three-phase process, the universal agreement) was 0.86 with a total item agree- expert panel believed each item was acceptable in terms ment of 36 of 42 items (4 items at 0.83, and 2 items at of readability, comprehension, and cultural relevance. 0.67) and for cultural relevance, the S-CVI/Avg was 0.96 Table 5 Examples of Language Clarity Issues Identified from Cognitive Interviews Translation issue Item Illustrative content example Spanish words do not convey intended Problem: Description of issue: “Crisis mode” translates to “modo de crisis” en construct Crisis mode or “modo de crisis” other Spanish speaking countries. The concept exists in Peru but translates differently. English question: We work in “crisis mode” trying to do too much, too quickly. Initial translation: Trabajamos en “modo de crisis”, tratando de hacer demasiado y muy rápidamente. Final translation: Trabajamos bajo presión intentando realizar demasiadas cosas muy rápidamente. Problem: Description: “Falls between the cracks” translates into Spanish as Perden or to lose for “falls through “perden” or to lose things which is slightly different in meaning the cracks” for participants. English question: Things “fall between the cracks” when transferring patients from one unit to another. Initial translation: Las cosas se pierden cuando los pacientes son transferidos de una unidad a otra. Final translation: La información y/o objetos de los pacientes se pierde cuando éstos se transfieren de una unidad a otra. Spanish words are unfamiliar or have Problem: Description of issue: Information is a generic word in Spanish and different meanings in some cultures or Importante información for specific the participants wanted further clarification about what different nationalities information important to patient information (e.g. documents or forms) is important. safety English question: Important patient care information is often lost during shift changes. Initial translation: Con frecuencia se pierde importante información sobre el cuidado del paciente durante los cambios de turno. Final translation: Se pierde a menudo información importante de cuidado de pacientes durante cambios de turno. Problem: Description of issue: The phrase for chance in Peru exists but Linking the meaning of mistakes to linking this to mistakes that occur in the hospital requires careful chance through the phrase “por word order arrangement. casualidad” English question: It is just by chance that more serious mistakes don’t happen around here. Initial translation: Aquí no suceden errores más serios sólo por casualidad. Final translation: Es sólo por casualidad que errores más serios no suceden aquí. Palmieri et al. BMC Nursing (2020) 19:23 Page 8 of 13 Table 6 Examples of Cultural Relevance Issues Identified from Cognitive Interviews Cultural Item Illustrative issue content example Definition of concepts or knowledge Problem: Different healthcare constructs in the Description of issue: Temporary staff and professionals are differ across cultures or nationalities provision of care between the United States and not frequently utilized in Peruvian healthcare facilities. Peru English Question: We use more agency/temporary staff than is best for patient care. Initial translation: Usamos más personal temporal o de agencia de lo conveniente para el cuidado del paciente. Final translation: Usamos más personal de reemplazo o temporal de lo que es mejor para el cuidado del paciente. Problem: Reference to errors and recording in Description of issue: The concept of blame and recording personnel files of errors in a personal file is not clear in Peru. English question: Staff worry that mistakes they make are kept in their personnel file. Initial translation: Al personal le preocupaba que los errores cometidos permanezcan en sus expedientes personales. Final translation: Cuando se comete un error, el personal teme que eso quede en su expediente. and the S-CVI/UA was 0.83 with a total item agreement represents a cultural difference in the construction of of 35 items (5 items = 0.83, 1 item = 0.67, and 1 item = the instrument that needs to be considered for interpret- 0.50). For the equivalence of the items to the dimension ation with the single measurement item. Table 7 for the entire instrument, the Kappa value was 0.72, or presents the additional questions identified by the partic- substantially equivalent. In terms of the individual di- ipants, but not included in the final instrument. mensions, only one of the twelve dimensions (dimension In addition to the translational, cultural, and general 2) had a Kappa value of 0.39, which is at the borderline design issues, some navigational issues were identified of moderate and fair. The remaining dimensions per- during the cognitive interviews. For example, two of the formed well (7 = almost perfect to perfect, 2 = substan- three participants in Round 3 indicated they were con- tial, and 2 = moderate). fused about when to turn the instrument pages to an- When aggregating the HSOPSC items, then averaging swer additional items. Similarly, participants found it the readability and cultural relevance scores for the three difficult to determine when they completed the entire participants for Round 1, there were 15 items (10 for instrument. Another participant indicated the choices readability and 5 for cultural relevance) which scored < for professions were not listed in a sensible “ranked” 3.00. For these items, 87% were negatively worded ques- order and yet another participant indicated the demarca- tions (readability 8 of the 10 items and cultural relevance tion for professional experience and years working in a all 5 items). In Round 2, of the 15 identified issues (those facility were not clear. Finally, one participant in Round < 3.00 in Round 1), only one negatively worded item 2 and two participants in Round 3 commented that remained problematic (average score = 2.67). In Round breaking item sections between pages was distracting 3, the only two items receiving an individual score less and seemed to disrupt the instrument flow. than four on either readability or cultural by participants were negatively worded. Finally, cognitive analysis re- Discussion vealed three additional items, recommended by partici- Similar to Levin et al. [47], the item analysis revealed pants, which should be added to the instrument. These two types of translational issues: 1) Words which when questions are specific to the reporting systems for errors translated from English to Spanish did not convey simi- and adverse events as the participants realized these sys- lar constructs; and 2) Phrases or specific words which tems are either not present or not widely utilized. This when translated from English to Spanish were not Table 7 Additional Questions Recommended by Participants Rational for recommendation Proposed question Question specific to the presence of a adverse event reporting system in the En su hospital funciona el Sistema de Notificación Hospitalario facility. There is another question specific to the reporting of adverse events but de eventos adversos. not one about the presence of the system. Question specific to the frequency the person has reported an adverse event. Cuando se reporta un evento ¿con qué frecuencia es analizado? Question specific to the rating of the patient safety in the hospital or clinic. There Por favor, dele a su HOSPITAL de trabajo una calificación is another question specific to rating the area or unit. promedio de acuerdo al grado de seguridad del paciente. Palmieri et al. BMC Nursing (2020) 19:23 Page 9 of 13 familiar to the participants as they had different mean- Across the cognitive interviews, participants consist- ings across cultures and/or national borders. Through ently identified similar issues with the identified prob- the cognitive interview process; however, the partici- lematic items. For example, in Round 2 about 82% of pants refined these issues into four distinct domains, in- the items rated by the participants as a three or less, at cluding: 1) translation (“reads wrong”); 2) culture least two of the three participants identified the same or relevance (“don’t understand”); 3) general instrument de- similar issue. Also, there were multiple items where the sign (“looks strange”); and 4) navigational issues with identification agreement highlighted items that the trad- completing instrument (“where do I go” or “what do I itional forward- and reverse-translation process would do” next). The fourth category has not been reported in have missed. For example, a cultural issue specific to the literature but describes some problems associated asking nurses and physicians about work hours in their with the format and flow of the instrument. Finally, the primary facility was captured by cognitive interviews. In participants realized one item asked about patient safety Peru, nurses and physicians routinely work enough outcomes. As such, they all recommended additional hours at two or three facilities to be equivalent to two items focused on the types and quantities of adverse er- full-time jobs in the United States. Often, professionals rors observed in practice or personally committed. are employed full-time by the public health system (or The general design issues identified during the cogni- semi-public) and then work full-time in the private tive probing process, can be classified into four categor- health system. As such, the four participants suggested ies: 1) Formal structures and process; 2) Physician work we needed to incorporate an additional question to cap- environment; 3) Professional domains; and 4) Termin- ture the total number of hours worked at all facilities in ology for the public and private systems. Although the addition to their primary work facility. first three categories were improved with relatively Most items translated were etic [52], or concepts that minor item adjustments, the final category necessitated were universally transferable. These items had very good an instrument design with words to satisfy the distinct clarity and substantial cultural relevance, without or with vocabulary of private hospitals (called clínicas) and pub- quite minor modification. Then, there were some emic lic hospitals (called hospitales). As such, the final instru- concepts that needed to be addressed by the research ment included specific words in combination such as team. These are the items reflecting culturally specific “hospital/clinica” and “area/unidad” to satisfy the differ- concepts in meaning, such as ideas and behaviors, in the ences in vocabulary additional items”. Table 8 presents source language [52], but prove to be inequivalent with examples of the four categories of general design issues. only translation [96]. For example, an English worded Table 8 Examples of General Design Issues Identified from Cognitive Interviews General design problem Illustrative example Health system differences between countries Description of issue: In Peru, there is not a formal adverse event and reporting system at every facility. As such, the concept is unfamiliar to many participants. Revision: Addition of a question specific to whether or not the participant is aware of the presence of a formal error reporting system at their respective facility. Work environment for physicians is different Description of issue: In Peru, most physicians work at more than one facility. Also, they are usually employed full-time by the public health system (or semi-public) and then work as contractors in the private health system. Revision: Incorporated a question to capture the number of hours worked at all health facilities in addition to their primary work facility in Peru. Professional domains are different or absent Description of issue: There are many professions in the United States and in Peru which are different. For example, registered nurses are present in both places but respiratory therapists do not exist in Peru. Revision: Reconstruct the questions specific to professional roles and positions in facilities in Peru Multiple concepts are different between the public Description of issue: Within Peru, there are differences between the reference terms for areas, and private health systems departments, and even facilities. Revision: For example, the public sector term for an acute care facility is “hospital” while the private sector term is “clinica.” Additional step: Incorporate changes by crafting two instruments: 1) Instrument specific for public facilities; and 2) Instrument for private facilities but only with changes to distinct descriptions and terminology but without changes to the item meaning. Description of issue: Within Peru, there are differences in job titles and descriptions between the public and the private health facilities. Revision: For public system, use the appropriate terms and for the private system use the specific terms. Additional step: Incorporate changes throughout the instrument for public facilities and then throughout the instrument for private facilities. Palmieri et al. BMC Nursing (2020) 19:23 Page 10 of 13 item, “We work in ‘crisis mode’ trying to do too much, the forward and reverse translation process were utilized too quickly” translated easily but the language did not as language consultants for the expert deliberations. In capture the meaning of the item. Although ‘crisis mode’ this regard, with justification, the “purity” of the forward translates easily to ‘modo de crisis’ in Spanish speaking and reverse translation was not maintained. A side-by- countries, the concept translates different in many coun- side instrument comparative each item, by dimensions, tries such as Peru. As such, the initial translation in- is provided in the Supplemental Table 1 for the AHRQ cluded the phrase ‘modo de crisis’ (Trabajamos en English and Spanish versions, the Spanish Version for “modo de crisis”, tratando de hacer demasiado y muy Spain, and the instrument produced from this study. rápidamente) required significant modification that did not include the phrase (Trabajamos bajo presión inten- Conclusion tando realizar demasiadas cosas muy rápidamente). This is the first study to report the application of the Another example of a seemingly discrete issue was dis- AHRQ recommended target-language translation covered with the word ‘chance’. Although ‘chance’ exists process, with the optional cognitive interviews, as an ef- in Spanish, when ‘chance’ is linked in a phrase with the fective method for cross-cultural research. Incorporating word ‘mistake’ the participants had difficulty under- cognitive interviews into a rigorous translation method standing the context of the item. In English, the item is resulted in an equivalent target-language HSOPSC for “It is just by chance that more serious mistakes don’t cross-cultural research. The feedback from the cognitive happen around here” and the initial translation was interviews contributed to substantially improved items “Aquí no suceden errores más serios sólo por casuali- as well as systematically validated the resulting Spanish dad”. By rearranging the word order and using an under- version of the HSOPSC for the Peruvian context. By line to emphasis the ‘sólo por casualidad’ the final pilot testing each item with a numerical rating provided translation was “Es sólo por casualidad que errores más a comparison within and between rounds, the resulting serios no suceden aquí”. In this case, the item score im- evaluation was robust in assessing the clarity and cul- proved significantly from an average of a 2.3 to a 4.7 in tural applicability, and provided a composite quality the next round and the cognitive interviews generated score. The sequential rounds revealed improved item no additional concerns. ratings for readability and cultural relevance and the With the final item analysis for grammar, syntax, and cognitive interviews resulted in successive reductions in other issues, we discovered negatively worded items per- item revisions. Furthermore, our analysis identified formed poorly, required multiple conversations during negatively worded questions are problematic for target- cognitive interviews, and generated disagreement be- language translations from American English to tween experts in panel discussions. With using only Peruvian Spanish. The inclusion of a content validity positively worded questions there is higher risk for ac- measurement provided additional evidence about se- quiescence bias [97–99]; however, the literature in this mantic equivalence. Finally, this is the first HSOPSC in- regard is primarily from English language countries and strument translation, adaptation, and validation study cultures. Similar to our finding about negatively worded reported in the literature for Latin America and appears questions, Solis-Salazar [100] reported combining posi- to be the only such study for all studies undertaking the tive and negative items can seriously damages the in- translation of the HSOPSC. ternal consistency of dimensions or subscales in Spanish language instruments. Supplementary information Finally, this study has three limitations. First, the cog- Supplementary information accompanies this paper at https://doi.org/10. nitive interviews required health professionals with ad- 1186/s12912-020-00419-9. vance to almost native levels of English comprehension. As such, these professionals might have more knowledge Additional file 1 Table S1. Item Comparison Across Instruments: Original English and Spanish Versions about safety culture than those without the same level of comprehension. In addition, the English language re- Abbreviations quirement narrowed the number of varied professionals AHRQ: Agency for Healthcare Research and Quality; CPE: Clinical practice to participate in the cognitive interviews. Second, this expert; CI: Cognitive interview; CVI: Content validity index; HSOPSC: Hospital study assumes the etic approach to cross-cultural re- Survey on Patient Safety Culture; I-CVI: Item-CVI; S-CVI: Scale-level content validity index; S-CVI/Avg: Scale-level content validity index, average; S-CVI/ search. In this regard, the theoretical assumptions and UA: Scale-level content validity index, universal agreement; SME: Subject operational constructs specific to safety culture were as- matter experts sumed to be translatable to Peru. However, the study method was a strength in providing evidence the lan- Acknowledgements We want to thank Dr. Adrian Anast, Director of the Online Writing Center at guage used to describe the assumptions and construct A.T. Still University for working with the research team to edit and revise this was transferable. Third, and last, the translators from manuscript in preparation for publication. Also, we want to thank Professor Palmieri et al. BMC Nursing (2020) 19:23 Page 11 of 13 Karen Dominguez-Cancino, Deputy Director of the EBHC South America: A Received: 21 July 2019 Accepted: 31 March 2020 Joanna Briggs Affiliated Group for her statistical advisement and technical as- sistance during the manuscript revisions. References Authors’ contributions 1. Jha AK, Prasopa-Plaizier N, Larizgoitia I, Bates DW. Research priority setting All authors participated in the final approval of the manuscript submitted for working group of the WHOWAfPS: patient safety research: an overview of journal review and are accountable for ensuring that questions related to the global evidence. Qual Saf Health Care. 2010;19:42–7. the accuracy and/or integrity of any part of the manuscript are appropriately 2. Clarke SG. The relationship between safety climate and safety performance: investigated and resolved. In the addition, the following authors were a meta analytic review. J Occup Health Psychol. 2006;11:315–27. involved in the: Study conception (PAP); study design (EWF, JLM, JSL, KMM,); 3. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a data collection (PAP); data analysis (PAP, JLM, JSL,); data interpretation (PAP, safer health system. Washington, D.C.: National Academy Press; 2000. EWF, JLM, JSL, KMM, NGG,); drafting the manuscript (EWF, JLM, JSL, PAP,); 4. Gershon RR, Karkashian CD, Grosch JW, Murphy LR, Escamilla-Cejudo A, developing the tables (DEC, JLM, NGG, PAP); substantial revision to the Flanagan PA, Bernacki E, Kasting C, Martin L. Hospital safety climate and its manuscript (DEC, JLM, NGG, PAP), critical revisions to the final manuscript relationship with safe work practices and workplace exposure incidents. Am (DEC, JLM, NGG, PAP). J Infect Control. 2000;28:211–21. 5. Scott T, Mannion R, Marshall M, Davies H. Does organisational culture influence health care performance? A review of the evidence. J Health Serv Author’s information Res Policy. 2003;8:105–17. PAP is a Fellow of the American Academy of Nursing and a nationally 6. Palmieri PA, Peterson LT. Attribution theory and healthcare culture: certified researcher (No. P0022630) at the Peruvian National Council of Translational management science contributes a framework to identify the Science, Technology, and Innovation (CONCYTEC). He is also the Director, etiology of punitive clinical environments. In: Savage GT, Fottler MD, editors. EBHC South America: A Joanna Briggs Institute Affiliated Group (Chile, Biennial Review of Health Care Management: Meso Perspective, vol. 8. Colombia, Peru); Vice Chancellor for Research, Universidad Norbert Wiener Bingley: Emerald Group Publishing; 2009. p. 81–111. Advances in Health (Peru); Adjunct Professor, College of Graduate Health Studies, A.T. Still Care Management. University (USA); Adjunct Professor, Doctoral Program, College of Health 7. Clark G. Organisational culture and safety: an interdependent relationship. Sciences, Walden University (USA); and Chairman of the Board of Directors, Aust Health Rev. 2002;25:181–9. Sigma Theta Tau International Foundation for Nursing (USA). 8. Hellings J, Schrooten W, Klazinga NS, Vleugels A. Improving patient safety culture. Int J Health Care Qual Assur. 2010;23:489–506. 9. Noort MC, Reader TW, Shorrock S, Kirwan B. The relationship between Funding national culture and safety culture: implications for international safety This research project was partially funded through a research dissemination culture assessments. J Occup Organ Psychol. 2016;89:515–38. grant from the Universidad Cooperativa de Colombia received by Dr. Doriam 10. Singer S, Falwell A, Gaba DM, Meterko M, Rosen A, Hartmann CW, Baker L. E. Camacho-Rodríguez. Identifying organizational cultures that promote patient safety. Health Care Manag Rev. 2009;34:300–11. Availability of data and materials 11. El-Jardali F, Dimassi H, Jamal D, Jaafar M, Hemadeh N. Predictors and The datasets used and/or analyzed during the current study available from outcomes of patient safety culture in hospitals. BMC Health Serv Res. 2011; the corresponding author on reasonable request. 11:1–12. 12. Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. Ethics approval and consent to participate 2010;6:226–32. The study protocol (#01146) was approved by the A.T. Still University 13. Palmieri PA, Peterson LT, Pesta BJ, Flit MA, Saettone DM. Safety culture as a Institutional Review Board (Mesa, Arizona). As approved by the Institutional contemporary healthcare construct: Theoretical review, research assessment, Review Board, each participant verbally consented to participate in this and translation to human resource management. In: Savage GT, Khatri N, study. Fottler MD, editors. Strategic Human Resource Management in Health Care, vol. 9. Bingley: Emerald Group Publishing Limited; 2010. p. 97–133. Advances in Health Care Management. Consent for publication 14. Shortell SM, Denise M, Rouseau DM, Gillies RR, Devers KJ, Simons TL. This consent is not applicable. Organizational assessment in intensive care units (ICUs): construct development, reliability, and validity of the ICU nurse-physician questionnaire. Med Care. 1991;29:709–23. Competing interests 15. Hofoss D, Deilkas E. Roadmap for patient safety research: approaches and The authors declare that they have no competing interests. roadforks. Scandanavian J Public Health. 2008;36:812–7. 16. World Alliance for Patient Safety. Forward Programme: 2008–2009. 1st ed. Author details Geneva: World Health Organization; 2008. 1 Office of the Vice Chancellor for Research, Universidad Norbert Wiener, Av. 17. Drösler SE, Klazinga NS, Romano PS, Tancredi DJ, Gogorcena Aoiz MA, Arequipa 444, 15046 Lima, Peru. 2College of Graduate Health Studies, A. T. Hewitt MC, Scobie S, Soop M, Wen E, Quan H, et al. Application of patient Still University, 800 West Jefferson Street, Kirksville, MO 63501, USA. 3School safety indicators internationally: a pilot study among seven countries. Int J of Nursing, Walden University, 100 S Washington Ave, Suite 900, Minneapolis, Qual Health Care. 2009;21:272–8. MN 55401, USA. 4EBHC South America: A Joanna Briggs Institute Affiliated 18. Kruk ME, Freedman LP. Assessing health system performance in developing Group, Universidad Norbert Wiener, Av. Arequipa 444, Lima, 15046 Lima, countries: a review of the literature. Health Policy. 2008;85:263–76. Peru. 5Departament d’Infermeria, Facultat de Medicina, Universitat Autonoma 19. Marcel JP, Alfa M, Baquero F, Etienne J, Goossens H, Harbarth S, de Barcelona, Avda. Can Domenech, Building M. Office M3/211, 08193 Hryniewicz W, Jarvis W, Kaku M, Leclercq R, et al. Healthcare-associated Bellaterra (Cerdanyola del Vallès), Barcelona, Spain. 6Center for Health infections: think globally, act locally. Clin Microbiol Infect. 2008;14:895– Sciences Research, Universidad María Auxiliadora, Av. Canto Bello 431, 15408 907. Lima, Peru. 7EBHC South America: A Joanna Briggs Institute Affiliated Group, 20. Hernandez K, Ramos E, Seas C, Henostroza G, Gotuzzo E. Incidence of and Universidad Norbert Wiener, Av. Arequipa 444, 15046 Lima, Peru. 8School of risk factors for surgical-site infections in a Peruvian hospital. Inf Contrl Hosp Nursing, Universidad Cooperativa de Colombia, Calle 30, Santa Marta, Epidemiol. 2005;26:473–7. Magdalena, Colombia. 9School of Public Health, University of Alabama at 21. Velasco E, Thuler LC, Martins CA, Dias LM, Goncalves VM. Nosocomial Birmingham, Ryals Public Health Building, 1665 University Blvd., Ryals 310E, infections in an oncology intensive care unit. Am J Infect Control. 1997;25: Birmingham, AL 35233, USA. 458–62. Palmieri et al. BMC Nursing (2020) 19:23 Page 12 of 13 22. Basu S, Andrews J, Kishore S, Panjabi R, Stuckler D. Comparative 46. Fujishiro K, Gong F, Baron S, Jacobson CJ, DeLaney S, Flynn M, Eggerth DE. performance of private and public healthcare systems in low- and middle- Translating questionnaire items for a multi-lingual worker population: the income countries: a systematic review. PLoS Med. 2012;9:e1001244. iterative process of translation and cognitive interviews with English-, 23. Ramirez-Wong FM, Atencio-Espinoza T, Rosenthal VD, Ramirez E, Torres- Spanish-, and Chinese-speaking workers. Am J Ind Med. 2010;53:194–203. Zegarra SL, Díaz Tavera ZR, Sarmiento López F, Silva Astete N, Campos https://doi.org/10.1002/ajim.20733. Guevara F, Bazan Mendoza C, et al. Surgical site infections rates in more 47. Levin K, Willis GB, Forsyth BH, Norberg A, Kudela MS, Stark D, Thompson FE. than 13,000 surgical procedures in three cities in Peru: findings of the Using cognitive interviews to evaluate the Spanish-language translation of a international nosocomial infection control consortium. Surg Infect. 2015;16: dietary questionnaire. Surv Res Methods. 2009;3:13–25. 572–6. 48. Lopez GI, Figueroa M, Connor SE, Maliski SL. Translation barriers in conducting 24. Arrieta A, Suárez G, Hakim G. Assessment of patient safety culture in private qualitative research with Spanish speakers. Qual Health Res. 2008;18:1729–37. and public hospitals in Peru. Int J Qual Health Care. 2018;30:186–91. 49. Usunier J-C. Language as a resource to assess cross-cultural equivalence in 25. Payne SC, Bergman ME, Beus JM, Rodríguez JM, Henning JB. Safety climate: quantitative management research. J World Bus. 2011;46:314–9. leading or lagging indicator of safety outcomes? J Loss Prev Process Ind. 50. Flaherty JA, Gaviria FM, Pathak D, Mitchell T, Wintrob R, Richman JA, Birz S. 2009;22:735–9. Developing instruments for cross-cultural psychiatric research. J Nerv Ment 26. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety Dis. 1988;176:257–63. climate: a review of surveys. Qual Saf Health Care. 2005;14:364–6. 51. van de Vijver FJR, Poortinga YH. Towards an integrated analysis of bias in 27. Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital cross-cultural assessment. Eur J Psychol Assess. 1997;13:29–37. survey on patient safety culture: psychometric analyses. J Patient Saf. 2009;5: 52. Waltz CF, Strickland OL, Lenz ER, editors. Measurement in nursing and 139–44. health research. 4th ed. New York: Springer Publishing Company; 2010. 28. Nieva VF, Sorra J. Safety culture assessment: A tool for improving patient 53. Knafl K, Deatrick J, Gallo A, Holcombe G, Bakitas M, Dixon J, Grey M. The safety in healthcare organizations. Qual Saf Health Care. 2003;12:ii17–23. analysis and interpretation of cognitive interviews for instrument 29. Sorra J, Dyer N. Multilevel psychometric properties of the AHRQ hospital development. Res Nurs Health. 2007;30:224–34. survey on patient safety culture. BMC Health Serv Res. 2010;10:199. 54. Thrasher JF, Quah ACK, Dominick G, Borland R, Driezen P, Awang R, Omar 30. Chen IC, Li H-H. Measuring patient safety culture in Taiwan using the M, Hosking W, Sirirassamee B, Boado M. Using cognitive interviewing and hospital survey on patient safety culture (HSOPSC). BMC Health Serv Res. behavioral coding to determine measurement equivalence across linguistic 2010;10:152. and cultural groups: an example from the international tobacco control 31. El-Jardali F, Jaafar M, Dimassi H, Jamal D, Hamdan R. The current state of policy evaluation project. Field Methods. 2011;23:439–60. patient safety culture in Lebanese hospitals: a study at baseline. Int J Qual 55. Squires A. Methodological challenges in cross-language qualitative research: Health Care. 2010;22:386–95. a research review. Int J Nurs Stud. 2009;46:277–87. 32. Pfeiffer Y, Manser T. Development of the German version of the hospital 56. Maneesriwongul W, Dixon JK. Instrument translation process: a methods survey on patient safety culture: dimensionality and psychometric review. J Adv Nurs. 2004;48:175–86. properties. Saf Sci. 2010;48:1452–62. 57. Willis GB. Cognitive interviewing: a tool for improving questionnaire design. 33. Sunol R, Vallejo P, Groene O, Escaramis G, Thompson A, Kutryba B, Garel P. Thousand Oaks: Sage; 2005. Implementation of patient safety strategies in European hospitals. Qual Saf 58. Erkut S. Developing multiple language versions of instruments for Health Care. 2009;18:i57–61. intercultural research. Child Dev Perspect. 2010;4:19–24. 34. Güneş ÜY, Gürlek Ö, Sönmez M. A survey of the patient safety culture of 59. Mason TC. Cross-cultural instrument translation: assessment, translation, and hospital nurses in Turkey. Collegian. 2016;23:225–32. statistical applications. Am Ann Deaf. 2005;150:67–72. 35. Raeissi P, Reisi N, Nasiripour AA. Assessment of patient safety culture in 60. Temple B. Nice and tidy: translation and representation. Sociol Res Online. Iranian academic hospitals: strengths and weaknesses. J Patient Saf. 2018;14: 2005;10:1–10. 213–26. 61. Acquadro C, Conway K, Hareendran A, Aaronson N. Literature review of 36. Granel N, M-DJ M, Barth A, Papp K, Bernabeu-Tamayo MD. Patient safety methods to translate health-related quality of life questionnaires for use in culture in Hungarian hospitals. Int J Health Care Qual Assur. 2019;32:412–24. multinational clinical trials. Value Health. 2008;11:509–21. 37. Agency for Healthcare Research and Quality. Cuestionario sobre la de 62. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P. seguridad de los pacientes en los hospitales. Rockville: Agency for Principles of good practice for the translation and cultural adaptation process Healthcare Research and Quality; 2009. for patient-reported outcomes (PRO) measures: report of the ISPOR task force 38. Fajardo-Dolci G, Rodriguez-Suarez J, Arboleya-Casanova H, Rojano- for translation and cultural adaptation. Value Health. 2005;8:94–104. Fernandez C, Hernandez-Torres F, Santacruz-Varela J. Cultura sobre 63. Squires A, Aiken LH, van den Heede K, Sermeus W, Bruyneel L, Lindqvist R, seguridad del paciente en profesionales de la salud. Cirugía y Cirujanos. Schoonhoven L, Stromseng I, Busse R, Brzostek T, et al. A systematic survey 2010;78:522–7. instrument translation process for multi-country, comparative health 39. Ramírez-Martínez ME, González Pedraza-Avilés A. Cultura de seguridad y workforce studies. Int J Nurs Stud. 2013;50:264–73. eventos adversos en una clínica de primer nivel. Enfermería Universitaria. 64. Beatty PC, Willis GB. Research synthesis: the practice of cognitive 2017;14:111–7. interviewing. Public Opin Q. 2007;71:287–311. 40. Gómez Ramírez O, Arenas Gutiérrez W, González Vega L, Garzón Salamanca 65. Weech-Maldonado R, Morales LS, Spritzer K, Elliott M, Hays RD. Racial and J, Mateus Galeano E, Soto Gámez A. Cultura de seguridad del paciente por ethnic differences in parents' assessments of pediatric care in Medicaid personal de enfermeria en Bogata, Colombia. Ciencia y enfermería. 2011;17: managed care. Health Serv Res. 2001;36:575–94. 97–111. 66. Garcia AA. Cognitive interviews to test and refine questionnaires. Public 41. Saturno Hernández PJ, Da Silva Gama ZA, de Oliveira Sousa SL, YA YAF, De Health Nurs. 2011;28:444–50. Souza Oliveira AC, editors. Análisis de la cultura sobre seguridad del 67. Harkness JA, Van de Vijver FJR, Mohler P. Cross-cultural survey methods. paciente en el ámbito hospitalario del Sistema Nacional de Salud español. Hoboken: Wiley; 2003. Madrid: Ministerio de Sanidad y Consumo; 2007. 68. Agency for Healthcare Research and Quality. Hospital Survey on Patient 42. Pinheiro MdP, Junior OCdS: Evaluación de la cultura de seguridad del Safety Culture: Background and information for translators. Rockville: Agency paciente en una organización hospitalaria de un hospital universitario. for Healthcare Research and Quality; 2009. Enfermería Global 2016, 16:309–324. 69. Agency for Healthcare Research and Quality. Pilot study: Validity and 43. Agency for Healthcare Research and Quality. Translation guidelines for the reliability of the hospital survey on patient safety. Rockville: Agency for surveys on patient safety culture. Rockville; 2010. Healthcare Research and Quality; 2004. 44. Flin R, Burns C, Mearns K, Yule S, Robertson EM. Measuring safety climate in 70. Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research health care. Qual Saf Health Care. 2006;15:109–15. instruments: language, setting, time and statistical considerations. BMC Med 45. Hutchinson A, Cooper KL, Dean JE, McIntosh A, Patterson M, Stride CB, Res Methodol. 2010;10:13. Laurence BE, Smith CM. Use of a safety climate questionnaire in UK health 71. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of care: factor structure, reliability and usability. Qual Saf Health Care. 2006;15: instruments or scales for use in cross-cultural health care research: a clear 347–53. and user-friendly guideline. J Eval Clin Pract. 2011;17:268–74. Palmieri et al. BMC Nursing (2020) 19:23 Page 13 of 13 72. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health- 100. Solís-Salazar M. The dilemma of combining positive and negative items in related quality of life measures: literature review and proposed guidelines. J scales. Psicothema. 2015;27:192–9. Clin Epidemiol. 1993;46:1417–32. 73. Squires A. Language barriers and qualitative nursing research: methodological considerations. Int Nurs Rev. 2008;55:265–73. Publisher’s Note 74. Grant JS, Davis LL. Selection and use of content experts for instrument Springer Nature remains neutral with regard to jurisdictional claims in development. Res Nurs Health. 1997;20:269–74. published maps and institutional affiliations. 75. Bracken BA, Barona A. State of the art procedures for translating, validating and using psychoeducational tests in cross-cultural assessment. Sch Psychol Int. 1991;12:119–32. 76. Brislin RW. Back-translation for cross-cultural research. J Cross-Cult Psychol. 1970;1:185–216. 77. Chapman DW, Carter JF. Translation procedures for the cross cultural use of measurement instruments. Educ Eval Policy Anal. 1979;1:71–6. 78. Jones PS, Lee JW, Phillips LR, Zhang XE, Jaceldo KB. An adaptation of Brislin’s translation model for cross-cultural research. Nurs Res. 2001;50:300–4. 79. Chang AM, Chau JPC, Holroyd E. Translation of questionnaires and issues of equivalence. J Adv Nurs. 1999;29:316–22. 80. Tourangeau R. Cognitive sciences and survey methods. In: Jabine TB, Straf ML, Tanur JM, Tourangeau R, editors. Cognitive aspects of survey methodology: Building a bridge between disciplines. Washington, DC: National Academy Press; 1984. p. 73–100. 81. Miller K. Conducting cognitive interviews to understand question-response limitations. Am J Health Behav. 2003;27:S264–72. 82. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health. 2006;29: 489–97. 83. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459–67. 84. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003;27:94–104. 85. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975; 28:563–75. 86. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35:382–5. 87. Zamanzadeh V, Rassouli M, Abbaszadeh A, Alavi Majd H, Nikanfar A, Ghahramanian A. Details of content validity and objectifying it in instrument development. Nurs Pract Today. 2015;1:163–71. 88. Okochi J, Takahashi T, Takamuku K, Matsuda S, Takagi Y. Reliability of a geriatric assessment instrument with illustrations. Geriatr Gerontol Int. 2005; 5:37–47. 89. Gorelick MH, Wagner D, McLellan SL. Development and validation of a self- administered questionnaire to measure water exposures in children. Ambul Pediatr. 2008;8:388–91. 90. Dufour S, Barkema HW, DesCôteaux L, DeVries TJ, Dohoo IR, Reyher K, Roy J-P, Scholl DT. Development and validation of a bilingual questionnaire for measuring udder health related management practices on dairy farms. Prev Vet Med. 2010;95:74–85. 91. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic. 1981;86:127–37. 92. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–82. 93. Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;33:363–74. 94. DeVillis RF. Scale development: theory and application. Thousand Oaks: Sage Publications; 2003. 95. Napoles-Springer AM, Santoyo-Olsson J, O'Brien H, Stewart AL. Using cognitive interviews to develop surveys in diverse populations. Med Care. 2006;44:S21–30. 96. Banville D, Desrosiers P, Genet-Volet Y. Translating questionnaires and inventories using a cross-cultural translation technique. J Teach Phys Educ. 2000;19:374–87. 97. Messick S. The psychology of acquiescence: an interpretation of research evidence. ETS Res Bull Ser. 1966;1966:1–44. 98. Ray JJ. Reviving the problem of acquiescent response bias. J Soc Psychol. 1983;121:81–96. 99. Lavrakas PJ, editor. Encyclopedia of survey research methods. Thousand Oaks: SAGE Publications; 2008.