Bonfill et al. BMC Health Services Research (2020) 20:505 https://doi.org/10.1186/s12913-020-05286-3 RESEARCH ARTICLE Open Access A survey to assess awareness and opinion of initiatives and recommendations on low- value diagnostic practices Xavier Bonfill1,2,3, Karla Salas-Gama1, Carolina Requeijo1* , Angela Merchán-Galvis1,4, Antonio Sánchez3,5, Elena Medarde5, M. Jesús Quintana1,2 and the DianaHealth Study Group Abstract Background: The need to reduce healthcare practices that provide no value has led to the development of initiatives that generate and publish recommendations to improve the appropriateness of clinical practice by identifying potentially inappropriate services, making recommendations, and proposing improvements. DianaHealth (www.dianahealth.com) identifies, classifies, and publishes recommendations from numerous scientific societies. The purpose of this study was to determine the awareness and perceived usefulness and applicability of published recommendations on low-value diagnostic measures, as judged by physicians who are recognised clinical leaders in their respective centres. Methods: We designed a questionnaire on the diagnostic recommendations considered relevant for each medical specialty and made it available, until September 2016, on DianaHealth. The survey was administered online to clinical leaders from 25 Spanish healthcare centres (hospitals and primary care centres). Results: A total of 413 (40.0%) physicians from 34 different specialties participated. The participation rate varied between centres (range 21.1%-100.0%) and specialties (range 12.5%-78.9%). Do Not Do (57.1%) was the most widely-known initiative. Most participants (82.6%; IQR 77.9%-94.9%) stated that they knew at least one of the 12 initiatives that identify non-recommended practices, and on average they were aware of four initiatives (range 1-12). The initiatives were perceived useful by 82.4% (IQR 73.3%-90.4%), and perceived applicable by 75.6% (IQR 67.4%-86.8%). A total of 531 recommendations were assessed. Sixty-three percent (IQR 53.6%-77.5%) of participants reported they were aware of the recommendations for their corresponding specialty. A total of 84.5% (IQR 75.0%-90.0%) stated they agreed with the recommendations and 84.5% (IQR 75.0%-90.0%) considered them useful. Among those who agreed with their respective recommendations, a median of 51.5% (IQR 41.4%-60.9%) perceived the guidelines as being fully implemented, 40.1% (IQR 31.9%-46.8%) considered them partially implemented, and 7.1% (IQR 3.7%-12.9%), not implemented. (Continued on next page) * Correspondence: crequeijo@santpau.cat 1 Centro Cochrane Iberoamericano. Servicio de Epidemiología Clínica y Salud Pública, Hospital de Sant Pau, IIBSant Pau, Barcelona, Spain Full list of author information is available at the end of the article © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Bonfill et al. BMC Health Services Research (2020) 20:505 Page 2 of 10 (Continued from previous page) Conclusions: Clinical leaders’ awareness of initiatives that generate and publish recommendations to improve clinical appropriateness remains low, although they did consider them useful. In general, participants were familiar with their speciality-specific diagnostic recommendations, agreed with them, and perceived them as useful and implemented in their centres. More needs to be done to raise awareness among professionals who do not know of or apply these recommendations. Keywords: Appropriateness, Low-value care, DianaHealth, Survey Background initiative and 97% of those who were acquainted with it Currently, there is a need to identify and reduce clinical considered it an appropriate source of information on practices of questionable value. Around 20% to 25% of low-value tests and interventions. A similar study was de- clinical practices have been reported to provide no bene- veloped in Spain at a tertiary hospital regarding the “Do fit to the patient and, therefore, present unnecessary Not Do” recommendations and it found that 90% of the directors of clinical units agreed with the recommenda- risks, with an economic impact that could even exceed tions [23]. 20% of total healthcare costs [1]. The objective of the current study was to determine Nowadays, optimising health care services is a world- the awareness of the above-mentioned initiatives and wide major interest, and several recently-published studies perceived usefulness and applicability in clinical practice have provided a reference framework for appropriate of its recommendations on diagnostic practices, in physi- health care [2–5]. Likewise, multiple national and inter- cians from different healthcare centres in Spain. national initiatives that generate and publish recommen- dations have been developed to improve the Methods appropriateness of care and clinical practice, by identifying We conducted a multi-centre, cross-sectional study co- potentially inappropriate services, making recommenda- ordinated by the Service of Clinical Epidemiology and tions, and proposing actions and clinical alternatives, Public Health at the Hospital de Sant Pau in Barcelona, when available, to reduce inappropriate use [6–20]. and the Comisión de Mejora de la Práctica Clínica Most initiatives are conducted by groups of scientific (Committee for Improving Clinical Practice) of the Con- societies who gather clinical recommendations [21]. sorci Sanitari de Terrassa (Barcelona). Choosing Wisely [7] is one such example that is active in From April to September 2017, we conducted a survey several countries [12–20]. Other initiatives include a) col- aimed at clinical leaders (directors, heads of depart- lections of articles identifying low-value health care prac- ments, and heads of clinical units or areas) from hospi- tices (e.g. the Less is More series in the Journal of the tals and primary care centres in Spain that had American Medical Association, JAMA [10]); b) ‘do not do’ previously communicated an explicit interest to partici- recommendations (e.g. the Do Not Do recommendations pate in the study. We invited only those professionals from the National Institute for Health and Care Excel- whose specialties included on the survey. We sent an lence, NICE [6]); c) appropriateness criteria such as those email to participate on the study which included a link by the American College of Radiology [15]; and d) local to the anonymous online questionnaire. projects to improve clinical practice (e.g. ESSENCIAL [9] by the Agency for Health Quality and Assessment of Cata- lonia (AQuAS), and the MAPAC initiative (in English, Im- Recommendation selection provement of Appropriateness in Clinical Practice and We selected recommendations based on the following Healthcare, from the Spanish Mejora de la Adecuación de criteria: la Práctica Asistencial y Clínica) [21]. In order to over- come the scattered fragmentation of recommendations  Inclusion criteria: recommendations on low-value generated by such initiatives, DianaHealth (www.diana- diagnostic tests, published on DianaHealth from in- health.com) identifies, classifies and publishes them to ception (2014) to September 10th, 2016. help clinicians identify the low-value clinical interventions  Exclusion criteria: recommendations on non- in their particular field [21]. diagnostic interventions, high-value recommenda- Few studies have been conducted to determine health tions, assessments with no explicit recommendation, professionals’ awareness and opinion of the various pub- recommendations with no specialty assigned, or lished recommendations on low-value clinical practices. A nursing recommendations. study among primary care physicians was conducted in the United States regarding the Choosing Wisely initiative DianaHealth allows users to select recommendations [22]. It found that 66% of participants were aware of the by specialty, so we generated a list for each one. When a Bonfill et al. BMC Health Services Research (2020) 20:505 Page 3 of 10 recommendation could be classified to more than one The survey was designed and carried out in accord- specialty, we included it in both lists. We excluded spe- ance with the Checklist for Reporting Results of Internet cialties with six or fewer low-value diagnostic recom- E-Surveys (CHERRIES) [24]. Before starting, we tested mendations. As a result, 34 specialties were selected. the survey for usability and technical functionality. Each Between 7 and 25 recommendations were selected for physician was invited to answer both sections of the each speciality based on their clinical relevance, general questionnaire through the electronic platform Clinapsis®. interest, potential clinical impact, and applicability. The survey was open for three months, from May to July 2017. The time needed to complete the survey was Questionnaire development around 10 to 20 minutes, depending on the number of The questionnaire consisted of two parts (Additional file recommendations for each speciality. 1): the first contained general questions on different ini- We conducted a descriptive data analysis for all the tiatives that generate and publish recommendations and collected variables, expressed as percentage of respon- the second contained specific questions related to each dents, using the software Stata version 14.0. Participation of the selected recommendations. The responses to the rate was calculated considering those clinical leaders that questions about awareness, agreement, usefulness, and answered the survey divided by the target population applicability were classified as disagreement (“totally dis- (clinical leaders who were invited to answer the survey). agree” and “disagree”), neither agree nor disagree, and The study was approved by the Clinical Research Eth- agreement (“agree" and "totally agree”). ics Committee of both co-ordinating centres. No Table 1 Clinician participation by centre Centre All invited Number of participants Participation % Hospitals (n=15) Catalonia H. de Sant Pau 118 81 68.6 H. U. Germans Trias i Pujol 54 35 64.8 C. Sanitari de Terrassa 43 27 62.8 H. General de Granollers 30 17 56.7 H. Sant Joan Despí M. Broggi 47 20 42.6 Corporació Sanitaria Parc Taulí 69 28 40.6 H. de Sant Rafael 35 14 40.0 Fundació Salut Empordà 25 10 40.0 Consorci Sanitari de l'Anoia 25 9 36.0 H. Sant Joan de Deu (Esplugues) 7 2 28.6 H. General de Catalunya 46 12 26.1 Madrid H. Universitario Ramón y Cajal 64 22 34.4 H. Puerta de Hierro 67 16 23.9 Basque Country H. Universitario Cruces 94 28 29.8 H. Universitario Donostiab 285 60 21.1 Primary Care Centres (n=3) Catalonia EAP Lledoners 2 2 100.0 EAP Sardenya 3 2 66.7 Consorci Castelldefels Agents de Salut 16 4 25.0 Othersa - 24 - Total 1,030 413 40.1 a Others: includes participants who did not specify their centre. b Donostia Hospital invited not only clinical leaders but also the entire medical team. Bonfill et al. BMC Health Services Research (2020) 20:505 Page 4 of 10 informed consent was required since the questionnaire Results was completed anonymously. Our online survey did not Initially, 25 centres agreed to participate in the study, al- require a signed consent, because it was completely an- though 7 were excluded from the analysis: 2 because onymous since it did not have any information that they did not reach the minimal participation rate (< 5%) could be used to glean the identity of the participants. and 5 because finally they decided not to take part of the Nevertheless, the protocol of the study was presented to survey. The majority of participating centres were hospi- two ethics committees and they considered not neces- tals (15), but three primary centres also participated (3). sary to be review by an ethical research committee since The target population consisted of 1,030 physicians, 413 it was not a biomedical research. of whom completed the survey (participation rate: Table 2 Participation in the survey by medical specialty. Medical speciality All invited Number of participants Participation % Rheumatology 19 15 78.9 Dermatology 22 15 68.2 Nuclear medicine 9 6 66.7 Medical oncology 20 14 66.7 Internal medicine 34 23 65.7 Occupational medicine 14 9 64.3 Geriatrics 16 10 58.8 Radiology 31 17 53.1 Haematology 29 15 51.7 Endocrinology 20 10 50.0 Pneumology 22 11 50.0 Neurology 41 21 50.0 Preventive medicine & public health 9 5 50.0 Infectious diseases 20 9 45.0 Physical medicine & rehabilitation 19 8 42.1 Psychiatry 30 13 41.9 Emergency medicine 41 17 41.5 Immunology & allergology 17 7 41.2 Obstetrics & gynaecology 39 17 40.5 Microbiology 21 8 38.1 Biochemistry 26 10 37.0 Otorhinolaryngology 39 14 35.9 Cardiology 34 12 35.3 Traumatology 29 10 34.5 General surgery 59 21 34.4 Gastroenterology 34 12 34.3 Vascular surgery 16 6 31.6 Intensive medicine 29 9 31.0 Paediatrics 40 12 29.3 Family medicine 64 21 28.8 Ophthalmology 32 8 25.0 Nephrology 22 5 22.7 Anaesthesiology 101 19 18.6 Urology 32 4 12.5 Total 1030 413 40.1 Bonfill et al. BMC Health Services Research (2020) 20:505 Page 5 of 10 Table 3 Awareness and opinion of the 12 initiatives to improve appropriateness, by medical specialty Medical Specialty N° of Awarenessa N. ° of initiatives Useful Applicable Partic. % Median Range % % Anaesthesiology 19 89.5 4 1 - 10 100.0 94.7 Gastroenterology 12 83.3 4 1 - 8 75.0 66.7 Biochemistry 10 100.0 4 1 - 10 100.0 80.0 Cardiology 12 100.0 5 2 - 11 16.7 25.0 General surgery 21 80.9 3 1 - 8 85.7 76.2 Vascular surgery 6 100.0 3 1 - 6 83.3 83.3 Dermatology 15 80.0 2 1 - 5 80.0 73.3 Endocrinology 10 80.0 3 2 - 9 80.0 70.0 Geriatrics 10 90.0 7 2 - 12 80.0 100.0 Haematology 15 66.7 2 1 - 6 73.3 46.7 Infectious diseases 9 77.8 4 2 - 8 100.0 88.9 Immunology & allergology 7 14.3 1 1 - 14.3 42.8 Occupational medicine 9 66.7 3 1 - 7 88.9 77.8 Family medicine 21 95.2 5 1 - 12 100.0 95.2 Physical medicine & rehabilitation 8 87.5 3 1 - 6 87.5 62.5 Intensive medicine 9 88.9 4 2 - 10 100.0 88.9 Internal medicine 23 78.3 6 2 - 11 82.6 69.6 Nuclear medicine 6 100.0 3 1 - 6 100.0 100.0 Preventive medicine & public health 5 100.0 7 2 - 12 60.0 60.0 Microbiology 8 75.0 3 1 - 10 87.5 87.5 Nephrology 5 100.0 3 1 - 4 80.0 80.0 Pneumology 11 81.8 3 1 - 5 90.9 90.9 Neurology 21 80.9 3 1 - 8 57.1 57.1 Obstetrics & gynaecology 17 94.1 4 2 - 8 88.2 82.3 Ophthalmology 8 66.7 2 1 - 6 37.5 62.5 Medical oncology 14 78.6 4 2 - 6 28.6 21.4 Otorhinolaryngology 14 50.0 3 1 - 7 64.3 71.4 Paediatrics 12 75.0 4 2 - 6 66.7 75.0 Psychiatry 13 84.6 5 2 - 8 84.6 84.6 Radiology 17 94.1 5 2 - 9 82.3 82.3 Rheumatology 15 80.0 3 2 - 6 73.3 73.3 Traumatology 10 100.0 3 1 - 5 80.0 70.0 Emergency medicine 17 70.6 4 1 - 7 94.1 88.2 Urology 4 100.0 3 1 - 7 100.0 75.0 Overall 413 82.6 4 1 - 12 82.4 75.6 a At least 1 initiative 40.1%). Six centres had a participation rate exceeding dermatology, nuclear medicine, medical oncology, in- 50% (Table 1). If we would have considered the data of ternal medicine, occupational medicine, geriatrics, radi- the two centres that were excluded the participation rate ology and haematology); in two specialties less than would have been similar (36.9%). 20.0% responded (anaesthesiology and urology). The median participation rate per specialty was 41.3% A total of 82.6% of participants (IQR 77.9%-94.9%) re- (range 12.5%-78.9%) (Table 2). Nine specialties had a ported that they knew of at least one of the 12 initiatives participation rate over 50.0% (rheumatology, identifying non-recommended practices included in Bonfill et al. BMC Health Services Research (2020) 20:505 Page 6 of 10 DianaHealth (Table 3 and Fig. 1). The mean number of compliant, and 40.1% (IQR 31.9%-46.8%) perceived par- known initiatives per specialty was 4 (range 1-12), the tial compliance (Fig. 2). Dermatology, paediatrics, and Do Not Do and Too Much Medicine initiatives being the immunology and allergy had the highest perception of most widely-known, with 57.1% and 54.5% or respon- full compliance. A median 7.1% perceived their depart- dents, respectively, being aware of them (Fig. 1). Most ment as non-compliant with the recommendations (IQR initiatives were familiar to fewer than 30% of partici- 3.7%-12.9%). pants. Regarding national initiatives, Essencial, No Hacer (by the Spanish Society of Family and Community Medi- Discussion cine), and MAPAC were the most widely-known initia- This study aimed to determine clinical leaders’ awareness, tives with awareness rates of 32.0%, 31.5%, and 24.5%, agreement and interest regarding existing initiatives to re- respectively. duce low-value clinical practices as well as the published Overall, perceived usefulness and applicability to clinical recommendations specific to each specialty. practice of the different initiatives was high: 82.4% (IQR In this first approach we decided to focus our survey 73.3%-90.4%) and 75.6% (IQR 67.4%-86.8%), respectively on recommendations on diagnostic practices and leave (Table 3). However, in four specialties (cardiology, haema- the development of a similar survey on therapeutic as- tology, immunology and allergy, and medical oncology) pects for future studies. less than half of the participants considered the initiatives The DianaHealth portal made obtaining the existing rec- useful or applicable in their clinical practice. ommendations very easy. This open-access portal, available Overall, a median of 63.1% of respondents were aware of in English and Spanish, periodically classifies the recom- the recommendations for their specialty (IQR 53.6%-77.5%) mendations generated from national and international ini- (Table 4). The three specialties with the highest awareness tiatives aimed at reducing low-value clinical practices and were endocrinology, anaesthesiology and radiology. improving clinical appropriateness [8]. The overall median agreement with recommendations Survey participation was not particularly high (40.1%, was 84.5% (IQR 75.0%-90.0%). The five specialties with or 47.4% with the exclusion of the centre that invited all the highest reported agreement were nuclear medicine medical staff instead of only clinical leaders), although otorhinolaryngology, radiology, family medicine and this is in line with results from other similar surveys. neurology (Table 4). The wide variability of participation across the different The recommendations were perceived as useful by a centres could reflect the presence or absence of active median of 84.5% overall (IQR 75.0%-90.0%). The special- institutional policies to reduce low-value practices. The ists that considered the recommendations more useful participation rate for the different specialties was prob- were those of otorhinolaryngology, radiology and micro- ably influenced to some extent by the specific character- biology (Table 4). istics of each centre. Among those specialists who agreed with the recom- Most participants claimed to know of at least one of mendations, for all specialties, a median of 51.5% (IQR the initiatives, although on average 4 of the 12 initiatives 41.4%-60.9%) perceived their department as being fully- were known. The only initiatives familiar to at least half Fig. 1 Awareness of initiatives identifying non-recommended practices included in DianaHealth® (N = 413). Bonfill et al. BMC Health Services Research (2020) 20:505 Page 7 of 10 Table 4 Awareness, agreement, and perceived usefulness regarding recommendations, by medical specialty N° of N° of Awareness Agreement Usefulness Partic. Recom. Median (%) IQR (%) Median (%) IQR (%) Median (%) IQR (%) Anaesthesiology 19 9 89.5 84.2 - 100.0 89.5 89.5 - 94.7 89.5 84.2 - 89.5 Biochemistry 10 25 60.0 40.0 - 90.0 90.0 80.0 - 100.0 90.0 70.0 - 100.0 Cardiology 12 19 83.3 66.7 - 91.7 91.7 79.2 - 100.0 91.7 83.3 - 100.0 Dermatology 15 12 40.0 18.3 - 53.3 76.7 56.7 - 86.7 66.7 60.0 - 80.0 Emergency medicine 17 9 52.9 47.1 - 64.7 88.2 76.5 - 94.1 82.4 76.5 - 88.2 Endocrinology 10 23 90.0 80.0 - 100.0 90.0 70.0 - 100.0 90.0 80.0 - 95.0 Family medicine 21 17 81.0 52.4 - 90.5 95.2 81.0 - 95.2 85.7 76.2 - 95.2 Gastroenterology 12 22 83.3 75.0 - 91.7 87.5 83.3 - 100.0 87.5 83.3 - 100.0 General surgery 21 13 52.4 42.9 - 71.4 71.4 57.1 - 81.0 71.4 57.1 - 81.0 Geriatrics 10 8 75.0 70.0 - 82.5 75.0 70.0 - 85.0 80.0 70.0 - 90.0 Haematology 15 8 76.7 51.7 - 81.7 76.7 65.0 - 80.0 76.7 71.7 - 81.7 Immunology & allergology 7 18 57.1 57.1 - 57.1 85.7 71.4 - 100.0 85.7 71.4 - 100.0 Infectious diseases 9 13 55.6 33.3 - 55.6 77.8 77.8 - 88.9 77.8 66.7 - 88.9 Intensive medicine 9 10 61.1 55.6 - 83.3 72.2 0.0 - 86.1 88.9 88.9 - 100.0 Internal medicine 23 22 60.9 53.3 - 68.5 78.3 66.3 - 91.3 78.3 69.6 - 90.2 Medical oncology 14 24 64.3 44.6 - 78.6 75.0 57.1 - 85.7 71.4 55.4 - 85.7 Microbiology 8 24 62.5 46.9 - 87.5 87.5 71.9 - 100.0 93.8 71.9 - 100.0 Nephrology 5 7 40.0 30.0 - 60.0 40.0 20.0 - 60.0 40.0 30.0 - 60.0 Neurology 21 23 47.6 38.1 - 69.1 95.2 81.0 - 100.0 90.5 76.2 - 95.2 Nuclear medicine 6 13 66.7 66.7 - 100.0 100.0 83.3 - 100.0 83.3 83.3 - 100.0 Obstetrics & gynaecology 17 25 70.6 52.9 - 88.2 88.2 64.7 - 94.1 88.2 70.6 - 94.1 Occupational medicine 9 11 77.8 61.1 - 88.9 88.9 77.8 - 88.9 88.9 83.3 - 88.9 Ophthalmology 8 6 43.8 37.5 - 59.4 81.3 75.0 - 87.5 75.0 75.0 - 84.4 Otorhinolaryngology 14 8 67.9 57.1 - 75.0 100.0 94.6 - 100.0 100.0 92.9 - 100.0 Paediatrics 12 25 83.3 75.0 - 100.0 91.7 91.7 - 100.0 91.7 91.7 - 100.0 Physical medicine & rehabilitation 8 7 62.5 18.8 - 93.8 0.0 0.0 - 87.5 75.0 75.0 - 93.8 Pneumology 11 24 63.6 31.8 - 86.4 90.9 43.2 - 90.9 90.9 43.2 - 90.9 Preventive medicine & public health 5 12 20.0 0.0 - 30.0 60.0 40.0 - 65.0 50.0 40.0 - 60.0 Psychiatry 13 14 61.5 34.6 - 67.3 73.1 69.2 - 82.7 76.9 63.5 - 84.6 Radiology 17 24 88.2 76.5 - 94.1 97.1 92.7 - 100.0 97.1 94.1 - 100.0 Rheumatology 15 19 46.7 16.7 - 60.0 80.0 73.3 - 86.7 73.3 66.7 - 86.7 Traumatology 10 8 50.0 37.5 - 65.0 70.0 57.5 - 72.5 60.0 52.5 - 72.5 Urology 4 17 75.0 50.0 - 75.0 50.0 25.0 - 75.0 50.0 25.0 - 100.0 Vascular surgery 6 12 83.3 66.7 - 87.5 83.3 79.2 - 100.0 91.7 83.3 - 100.0 Overall 413 531 63.1 53.6 - 77.5 84.5 75.0 - 90.0 84.5 75.0 - 90.0 of the participants were Do Not Do and Too Much Medi- Around 60% of the participants reported that they knew cine. The Spanish initiatives (Essencial, No Hacer, and the diagnostic recommendations for their specialty, with Compromiso por la Calidad) were known to less than substantial variability between specialties. Three-quarters one third of the participants. Three-quarters of the partici- of the participants agreed with the recommendations and pants judged the above-mentioned initiatives positively, considered them useful. It would be interesting for both in terms of usefulness and applicability, although this future studies to analyse in depth the differences be- perception was much lower in four specialties. Additional tween specialties regarding perceived usefulness. research about this difference should be conducted in fu- Around 50% of the participants estimated that profes- ture studies. sionals at their centre always complied with the Bonfill et al. BMC Health Services Research (2020) 20:505 Page 8 of 10 Fig. 2 Perceived total compliance by specialty with their corresponding recommendations. recommendations, 40% estimated partial compliance, The response rate was quite good (40%) in relation to and less than 10% estimated non-compliance, with what is described for web surveys [25, 26]. The research some variability between specialties. group followed the methodological steps of this type of Despite the many initiatives and projects generating survey to improve the response: make a pilot of the sur- recommendations to reduce low-value clinical practices, vey (to assess the design, order and duration), make the very few studies have been conducted to assess target invitation to participate, send a reminder and ensure physicians’ perception of these recommendations. Our easy access to the questionnaire [25]. However, no in- survey is the first to provide an overall assessment of centive was offered. existing local, national and global initiatives, by evaluat- ing awareness and perceived usefulness. DianaHealth has Conclusions proven to be a very valuable resource for identifying po- Overall, there is insufficient awareness among Spanish tential questionable practices. clinical leaders regarding worldwide initiatives to im- One of the potential limitations of our study is that par- prove health care appropriateness, despite perceived use- ticipation was lower than desired. Nevertheless, our re- fulness being high. Furthermore, according to the sponse rate was in relation to has been described for other participants, diagnostic recommendations specific to web surveys [25, 26]. Most of the participating centres each specialty are widely known, agreed with and ad- were in Catalonia, because the survey coordinators were hered to at each centre, which is very encouraging. It based there, and also because most of such initiatives have seems necessary for such initiatives, which periodically been developed there. These local initiatives are generally generate recommendations on reducing low-value prac- promoted in hospitals, which is why fewer primary care tices, to develop additional knowledge-translation ac- professionals were invited to participate. Another limita- tions addressed at the target physicians to improve tion of our study is a potential self-selection bias because awareness and compliance. of participants who answered the survey could be those Abbreviations more aware and familiar with the importance of prevent- IQR: Interquartile range; JAMA: Journal of the American medical association; ing low-value care. NICE: National institute for health and care excellence; MAPAC: In Spanish, Bonfill et al. BMC Health Services Research (2020) 20:505 Page 9 of 10 “Mejora de la Adecuación de la Práctica Asistencial y Clínica”; in English, was not carried out on patients, did not use biological samples, nor did it Improvement of Appropriateness in Clinical Practice and Healthcare contemplate the recording of clinical data, and therefore it was not necessary to review it by an ethical research committee. Acknowledgements Group authorship: collaborating author names of DianaHealth Study Group Consent for publication Members: Not Applicable. – Xavier Bonfill, M Jesús Quintana, Carolina Requeijo, Karla Salas, Ángela Competing interests M Merchán. Hospital de Sant Pau (Barcelona), CIBERESP. The authors declare that they have no competing interests. – Antonio Sánchez, Elena Medarde. Consorci Sanitari de Terrassa (Terrassa). Author details – Dimelza Osorio, Soledad Romea. Hospital Universitari Vall d’Hebron 1 Centro Cochrane Iberoamericano. Servicio de Epidemiología Clínica y Salud (Barcelona). Pública, Hospital de Sant Pau, IIBSant Pau, Barcelona, Spain. 2CIBERESP, – Francisco Baigorri. Corporació Sanitària Parc Taulí (Sabadell). Madrid, Spain. 3Universitat Autònoma de Barcelona, Barcelona, Spain. – Agustín Urrútia. Hospital Universitari Germans Trías i Pujol (Badalona). 4 Universidad del Cauca, Popayán, Colombia. 5Consorci Sanitari de Terrassa, – Josep Lluis Vega. Hospital Sant Joan de Déu (L’Hospitalet de Llobregat). Barcelona, Spain. – Pedro Armario. Hospital de Sant Joan Despí Moisès Broggi (Sant Joan Despí). Received: 31 January 2019 Accepted: 1 May 2020 – Matteo Fabbi. Hospital Universitari General de Catalunya (Sant Cugat del Vallès). – Anna Carol Pérez Segarra. Hospital Sant Rafael (Barcelona). – Xavier Martret. Parc Sanitari Sant Joan de Déu (Sant Boi de Llobregat). References – Miquel Vila; Hospital General de Granollers (Granollers). 1. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. – Marta Banqué. Yolima Cossio; Consorci Sanitari l’Anoia (Igualada). 2012;307(14):1513–6. – Javier Zamora, Jesús López Alcalde, Alfonso Muriel. Hospital 2. Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, Nagpal S, Universitario Ramón y Cajal (Madrid). Saini V, Srivastava D, Chalmers K, Korenstein D. Evidence for overuse of – José Ignacio Emparanza, Iratxe Urreta. Hospital Universitario Donostia medical services around the world. Lancet. 2017;390(10090):156–68. (Donostia). 3. Glasziou P, Straus S, Brownlee S, Trevena L, Dans L, Guyatt G, Elshaug AG, – José Ignacio Pijoan, Amaia Martínez Galarza. Hospital Universitario Janett R, Saini V. Evidence for underuse of effective medical services around Cruces (Barakaldo), CIBERESP. the world. Lancet. 2017;390(10090):169–77. – Agustín Gómez de la Cámara. Hospital Universitario 12 de Octubre 4. Saini V, Garcia-Armesto S, Klemperer D, Paris V, Elshaug AG, Brownlee S, (Madrid), CIBERESP. Ioannidis JPA, Fisher ES. Drivers of poor medical care. Lancet. 2017; – Ana Royuela. Hospital Puerta de Hierro (Majadahonda). 390(10090):178–90. – Blanca Lumbreras. Hospital Universitari Sant Joan d’Alacant (Alicante). 5. Elshaug AG, Rosenthal MB, Lavis JN, Brownlee S, Schmidt H, Nagpal S, – Pere Plaja. Hospital Fundació Salut Empordà (Figueres). Littlejohns P, Srivastava D, Tunis S, Saini V. Levers for addressing medical – Antoni Peris. Consorci Castelldefels Agents de Salut d'Atenció Primària underuse and overuse: achieving high-value health care. Lancet. 2017; (Castelldefels). 390(10090):191–202. – Carlos Brotons. Equip d’Atenció Primària Sardenya (Barcelona). 6. National Institute for Health and Care Excellence [Internet]. Savings and – Montserrat Ureña. Centres d’Atenció Primaria, Consorci Sanitari de productivity collection: Do not do recommendations. 2016. Available: Terrassa (Terrassa). https://www.nice.org.uk/about/what-we-do/our-programmes/cost-savings- – Joan Fernández Náger. Equip d’Atención Primaria Penitenciari de resource-planning. Lledoners (Sant Joan de Vilatorrada). 7. Choosing Wisely.org. [Internet]. ABIM Foundation. [cited 2016 Sept 1]. Available: http://www.choosingwisely.org/. 8. Ministerio de Sanidad, Servicios Sociales e Igualdad [Internet]. Compromiso We would also like to thank Mrs. Andrea Cervera Alepuz for translating and por la Calidad de las Sociedades Científicas. [cited 2016 Sept 1]. Available: Dr. M Jane Marshall for editing the manuscript. http://www.msc.es/organizacion/sns/planCalidadSNS/cal_sscc.htm. 9. Agència d’Avaluació i Qualitat Sanitàries de Catalunya (AquAS). [Internet]. Authors’ contributions Essencial, Afegint valor a la pràctica clínica [cited 2016 Sept 1]. Available: XB was involved in the conception and design of the study, analysis and http://essencialsalut.gencat.cat/ca/. interpretation of data, drafting the manuscript and revising it critically. CR, 10. The JAMA Network. [Internet]. Less is more collection [cited 2016 Sept 1]. AM and KS were involved with the design of the study, acquisition, analysis Available: https://jamanetwork.com/collections/44045/less-is-more. and interpretation of data, and drafting the manuscript. AS, MJQ and EM 11. U.S. Preventive Services Task Force. [Internet]. U.S. Preventive Services Task were involved with the design of the study and drafting the manuscript. The Force A and B recommendations. [cited 2016 Sept 1]. Available: http://www. author (s) read and approved the final manuscript. uspreventiveservicestaskforce.org/Page/Name/uspstf-a-and-b- recommendations/. Funding 12. Choosing Wisely Canada.org. [Internet]. Canadian Medical Association, The development of the online survey was partially funded by the MAPAC University of Toronto. [cited 2016 Sept 1]. Available: http://www. program of the Clinical Epidemiology Program of the Centre for Biomedical choosingwiselycanada.org/. Research in Epidemiology and Public Health Network (CIBERESP), Institute of 13. Elshaug AG, Watt AM, Mundy L, et al. Over 150 potentially low-value health Health Carlos III. Spain. care practices: an Australian study. Med J Aust. 2012;197(10):556–60. 14. Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, Chacko SJ, Availability of data and materials Borkar D, Gall V, Selvaraj S, Ho N, Cifu A. A decade of reversal: an analysis of The datasets generated and analysed during the current study are included 146 contradicted medical practices. Mayo Clin Proc. 2013;88(8):790–8. in the Clinapsis® electronic platform repository, but restrictions apply to the 15. American College of Radiology. [Internet]. The ACR Appropriateness Criteria. availability of these data, and they are not publicly available. Data are [cited 2016 Sept 1]. Available: http://www.acr.org/Quality-Safety/ however available from the authors upon reasonable request. Appropriateness-Criteria. 16. The Canadian Task Force for Preventive Health Care. [Internet]. The CTFPHC Ethics approval and consent to participate Guidelines. [cited 2016 Sept 1]. Available: https://canadiantaskforce.ca/. No informed consent was required since the questionnaire was completed 17. The British Medical Journal. [Internet]. Too Much Medicine. [cited 2016 Sept anonymously. Nevertheless, the protocol of the study was presented to two 1]. Available: http://www.bmj.com/specialties/too-much-medicine. Clinical Research Ethics Committees (Hospital de Sant Pau and Consorci 18. Sociedad Española de Medicina de Familia y Comunitaria semFYC. Sanitari de Terrassa) at it was not considered biomedical research since it [Internet]. Proyecto Recomendaciones «No hacer». [cited 2016 Sept 1]. Bonfill et al. BMC Health Services Research (2020) 20:505 Page 10 of 10 Available: http://e-documentossemfyc.es/recomendacion-para-no-hacer-de- la-sociedad-espanola-de-medicina-de-familia-y-comunitaria/. 19. Sociedad Española de Radiología Médica (SERAM). Recomendaciones de No Hacer. [cited 2016 Sept 1]. Available: http://seram.es/modules.php?name= documentos&op=getDocument&iddocument=397. 20. ChoosingWiselyAustralia.org.au. [Internet]. NPS MedicineWise: Choosing Wisely Australia. [cited 2016 Sept 1]. Available: http://www.choosingwisely. org.au/about-choosing-wisely-australia. 21. Bonfill X, Osorio D, Solà I, Pijoan JI, Balasso V, Quintana MJ, Puig T, Bolibar I, Urrútia G, Zamora J, Emparanza JI, de la Gómez C. Ferreira-González I. DianaHealth.com, an on-line database containing appraisals of the clinical value and appropriateness of healthcare interventions: database development and retrospective analysis. PLoS One. 2016;11(2):e0147943. 22. Buist DS, Chang E, Handley M, Pardee R, Gundersen G, Cheadle A, Reid RJ. Primary Care Clinicians' Perspectives on Reducing Low-Value Care in an Integrated Delivery System. Perm J. 2016;20(1):41–6. 23. Martin S, Miñarro R, Cano P, Aranda JM. Applicability of "do not do recommendations" from the National Institute for Health and Care Excellence in a quaternary care hospital. Rev Calid Asist. 2015;30(3):117–28. 24. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004; 6(3):e34. 25. Cantuaria ML, Blanes-Vidal V. Self-reported data in environmental health studies: mail vs. web-based surveys. BMC Med Res Methodol. 2019;19(1):238. 26. Fan W, Yan Z. Factors affecting response rates of the web survey: A systematic review. Comput Hum Behav. 2010;26(2):132–9. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.