F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 SYSTEMATIC REVIEW Effects of evidence-based clinical practice guidelines in cardiovascular health care quality improvements: A systematic review [version 3; peer review: 2 approved] Anggie Ramírez-Morera 1,2, Mario Tristan 1, Juan Carlos Vazquez2,3 1Cochrane Central America & Caribbean Spanish, IHCAI Foundation, San José, San José, 10904, Costa Rica 2Universitat Autònoma de Barcelona, Barcelona, Spain, 08041, Spain 3Iberoamerican Cochrane Centre, Barcelona, Spain First published: 10 Jul 2019, 8:1041 ( Open Peer Review v3 https://doi.org/10.12688/f1000research.18865.1) Second version: 20 Sep 2019, 8:1041 ( https://doi.org/10.12688/f1000research.18865.2) Reviewer Status Latest published: 14 Oct 2019, 8:1041 ( https://doi.org/10.12688/f1000research.18865.3) Invited Reviewers 1 2 Abstract Background: The development of evidence-based clinical practice guidelines (EB-CPGs) has increasing global growth; however, the certainty version 3 of impact on patients and health systems, as well as the magnitude of the published impact, is not apparent. The objective of this systematic review was to 14 Oct 2019 assess the effectiveness of the application of EB-CPGs for the improvement of the quality of health care in three dimensions: structure, process and results in the patient for the management of cardiovascular version 2 report disease. published Methods: We followed the methods described by the Cochrane Handbook 20 Sep 2019 and present a descriptive analysis because of the high heterogeneity found across the included studies. We searched the Cochrane Central Register of version 1 Controlled Trials, MEDLINE and EMBASE databases, as well as the grey published report report 10 Jul 2019 literature, between 1990 and June 2016. No language restrictions were applied. Only randomised clinical trials (RCTs) were selected. Three authors independently carried out the data extraction, using a modified 1 Yasser Sami Abdel Dayem Amer , King version of the Cochrane Effective Practice and Organization of Care form. Saud University Medical City, Riyadh, Saudi Results: Of the total of 84 interventions included in the nine RCTs evaluated, three (4%) were related to health care structure, 54 (64%) to the Arabia health care delivery process and 27 (32%) to patient outcomes. Regarding Alexandria University, Alexandria, Egypt the impact of using the EB-CPGs, in 55 interventions (65%), there were no significant differences between control and experimental groups. In four 2 Ignacio Marín León, Fundación Enebro, interventions (5%), the result favoured the control group, and the result Seville, Spain favoured the intervention group on 25 of the interventions (30%). Any reports and responses or comments on the Conclusions: This systematic review showed that EB-CPGs could be useful to improve the process and structure of health care and, to a lesser article can be found at the end of the article. extent, to improve the patients’ outcomes. After analysing many studies, we could have one more hypothesis for further research, which could shed more light upon those undiscovered variables that might interfere with the use of the EB-CPGs. Registration: PROSPERO CRD42013003589 Keywords Page 1 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Keywords Clinical Practice Guidelines. CPG, effect, health care quality Corresponding author: Anggie Ramírez-Morera (aramirez@ihcai.org) Author roles: Ramírez-Morera A: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing; Tristan M: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Methodology, Project Administration, Resources, Software, Writing – Original Draft Preparation, Writing – Review & Editing; Vazquez JC: Formal Analysis, Writing – Original Draft Preparation, Writing – Review & Editing Competing interests: No competing interests were disclosed. Grant information: IHCAI Foundation, the sponsor of the Cochrane Center for Central America and the Spanish-speaking Caribbean, provided the necessary to carry out this review such as access to databases, full-text articles and other resources. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2019 Ramírez-Morera A et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Ramírez-Morera A, Tristan M and Vazquez JC. Effects of evidence-based clinical practice guidelines in cardiovascular health care quality improvements: A systematic review [version 3; peer review: 2 approved] F1000Research 2019, 8 :1041 (https://doi.org/10.12688/f1000research.18865.3) First published: 10 Jul 2019, 8:1041 (https://doi.org/10.12688/f1000research.18865.1) Page 2 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Lugtemberg 2009 used the full Donabedian (1988) model only REVISED Amendments from Version 2 including studies from The Netherlands. This new version included 2 changes: This review sought to assess whether the quality of health Page # 14: We modify the Acknowledgments by: Anggie Ramirez thanks the professors of the School of Epidemiology … care improves in patients with the cardiovascular dis- Page # 14: We modify the name of the University in the affiliation ease when using EB-CPGs vs standard professional medi- of two authors and replace it Catalan language. cal practice. The primary aim was to assess the impact of the The director of the doctoral program suggested the changes. EB-CPGs for the management of cardiovascular diseases on Any further responses from the reviewers can be found at the healthcare quality, in terms of patient outcomes, management end of the article process, and healthcare structure. Methods We designed a methodology aimed to find and analyse studies Introduction measuring the impact of EB-CPGs on the improvement of This review refers to the changes in the quality of healthcare quality in health care services in the three areas proposed by services that are direct consequences of the systematic use of Donabedian (Donabedian, 1988; Moore et al., 2015): structure, evidence-based clinical practice guidelines (EB-CPGs). “Evidence- process, and patient outcomes. From the very beginning of the based” means that the recommendations are created using process when the authors wrote this review protocol, it was rigorous, unbiased and transparent methods of collation and clear that these items would not be easy to measure and, as the appraisal, alsousing scientific findings of the highest quality and search and data extraction moved forward, it became harder value to assist in providing optimal clinical care to the patient. to synthesise the information delivered by the different stud- (Guyatt, 1992; Sackett et al., 1996). EB-GPCs are evidence sum- ies included. The main obstacle was the inconsistency observed maries and include systematically developed recommendations in the different outcome measures used by the studies, which to assist physicians and patients in the process of making deci- included continuous as well as dichotomous values for different sions (Alonso-Coello et al., 2010; Glasziou et al., 2011). clinical conditions and interventions. The intervention EBCPGs are one of the most available and widely used tools definition read as any planned action taken to modify the clini- for translating the knowledge generated by scientific research cal practice and use of the clinical guidelines for influencing to practice. They should be based on the best scientific evi- in the clinical practice. The authors followed the methodo- dence available in order to improve the quality of patient care logical recommendations described in the Cochrane Hand- (Alonso-Coello et al., 2010; Grimshaw & Russell, 1993). Experts, book (Higgins & Green, 2012). This review is registered with institutions, and organizations worldwide have developed EBCPGs PROSPERO (ID: CRD42013003589). in all areas of medicine as a key tool to improve the quality of care Donabedian, 1988; Glasziou et al., 2011; Lugtenberg Study searches et al., 2009; Moore et al., 2015 states that before evaluating health We did a systematic search using the following electronic care, we must first decide how to define “quality” and whether databases for primary studies (randomised controlled tri- it depends only on the actions of physicians or if it also depends als (RCTs)): Cochrane Central Register of Controlled Trials on contributions from the patients and the healthcare system. (CENTRAL). The Cochrane Library, including the Cochrane EPOC (Cochrane Effective Practice and Organization of Care) Defining quality is challenging since it is not easy to character- specialised database; MEDLINE; EMBASE; CINAHL; ise coherently and objectively. Health must be analysed from PsycINFO; LILACS; Health Technology Assessment Data- a holistic point of view, and guideline developers must deter- bases and Web of Science, Science Citation Index, and Social mine the ideal amount of influence health should receive from Sciences Citation Index. individual preferences and social components. We must also understand the relationship between structural characteristics and The review authors combined search strategy for indexed terms healthcare processes, as well as their results in health services and developed free text terms. We included searches of grey (Donabedian, 1988; Moore et al., 2015). literature in different sources, such as reports of the world and regional conferences, academic theses and scientific reports Since the 1990s, increasing numbers of EB-CPGs are being not published in indexed journals. We searched for studies developed. However, it is unknown whether these high-quality published between January 1990 and June 2016, without any recommendations have a beneficial impact on patient health. language restriction. An advanced search strategy is available Despite the high number of recently published EB-CPGs, as Extended data, Appendix 1 (Ramirez et al., 2019). there are few studies on their effectiveness in improving clinical outcomes, the process and structure quality throughout Study selection the healthcare system. We analysed the studies found through the search strategy, and two authors independently selected the articles according to After an exhaustive search, only two systematic reviews the following inclusion criteria: RCT measuring the change in (SR) we found on this topic (Lugtenberg et al., 2009; health care when using EB-CPGs on cardiovascular disease. Worrall et al., 1997). Worrall et al., 1997 only analysed patient The study should measure the change in any of the three health outcomes missing two of the proposed Donadebian Model. care dimensions (structure, process, and patient outcomes). Page 3 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Data extraction A number of the 4,279 studies found in the initial search were Three authors independently performed data extraction using a not randomised clinical trials, as the authors described in their modified shorter version of the Cochrane Collaboration EPOC titles or abstracts. When assessing the full-text articles, we Data Collection Checklist translated into Spanish. Apart from found that many were cluster trials and observational studies preparing the Spanish version, we eliminated several items not with a “before and after” design. applicable to this review, given that we only included RCTs. We used a standardised digital form for data extraction and analy- Quality of evidence assessment sis. We used a standardised digital form for data extraction As for the quality of the evidence, we observed the presence of and analysis. We used Review Manager software (RevMan 5.3) a high or unclear risk of bias for allocation concealment (selec- for the data analysis. tion bias), blinding of participants and personnel (performance bias), and blinding of outcome assessment (detection bias), We assessed the risk of bias (quality) according to the Cochrane which can be explained by the nature of the interventions stud- Handbook (Higgins & Green, 2012). We found very high vari- ied. We found several types of systematic errors: random ability, so the study results were introduced as a narrative sequence generation (selection bias), incomplete outcome data in the Results. (attrition bias) and selective reporting (reporting bias). We found that the interventions measured yielded outcomes assessed Because of the variability between the measurements of the effect of the impact of EB-CPGs on the change of quality in the studies included in this review, it was not possible or appropri- ate to perform a meta-analysis; therefore, it was not possible to measure the statistical heterogeneity. Results Study identification and selection When the first version of the protocol of this Systematic Review was initiated, consider the inclusion of several clini- cal topics in a single systematic review. We decided to create a series of three systematic reviews on different clinical top- ics: Cardiovascular Health, Breast Cancer and Child and Mother Health. The initial searches included all those topics with the same inclusion criteria described. After removing dupli- cates, the search produced 4279 potential studies. After screen- ing by title and abstract, 4051 were excluded. After the full-text evaluation, 96 studies of all clinical subjects were selected, extended data, Appendix 2 (Ramirez et al., 2019). Study characteristics For the analysis, we organised the studies according to the topic or pathology being the core subject of the EB-CPG, and for this report, we only selected RCTs on cardiovascular diseases, because this theme accounts for the higher number of original articles. In total, we selected nine RCTs (Figure 1). Characteristics of the included studies are available as Extended data, Appendix 2 (Ramirez et al., 2019). We included nine RCTs analysing cardiovascular diseases (Beaulieu et al., 2004; Berner et al., 2003; Ellis et al., 2000; Guadagnoli et al., 2004; Hand et al., 2014; Jäntti et al., 2007; Kiessling & Henriksson, 2002; Tierney et al., 2003; Tsuyuki et al., 2015). All trials were carried out between 2000 and 2015, five in the United States of America, two in Canada, one in Finland and one in Sweden. Eight articles addressed outpatient care and three inpatient care. The EB-CPGs included in the selected trial looked at the following clinical problems: management of stable angina pectoris (over 65 years), unstable angina, dyslipidaemia, acute infarction, heart failure and blood pressure. Besides, the trials included perioperative cardiac evaluations of patients with non-cardiac surgery, cardiopulmonary resus- citation and secondary prevention in patients having coronary artery disease. Figure 1. Flow chart of the studies (PRISMA, Moher 2009). Page 4 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 with moderate to low evidence certainty according to the dimension, 54 interventions (64%) to the dimension health care GRADE classification. delivery and 27 interventions (32%) to the dimension of patient outcomes. Regarding the impact of EB-CPG use, we Risk of bias assessment found that in 55 interventions (65%) there was no significant Analysing the risk of bias in the nine included RCTs studies difference between the control and experimental groups. found the random sequence generation (selection bias) assess- ment had a low risk of bias (between 50–75%) for allocation In four interventions (5%) the outcome favoured the control concealment (selection bias) 33% low, 33% unknown and group (comparison of the measure of compliance of the recom- 33% high risk of bias. The blinding of participants and person- mendations (the average adjusted by the patient characteristics, nel (performance bias) obtained almost 50% high risk of bias. the setting, and the number of measures applied per patient)). The blinding of outcome assessments (detection bias) obtained The measure of effects (odds ratio (OR)) regarding the condi- a 50% low and 50% uncertain risk of bias. Incomplete outcome tions acute myocardial infarction and heart failure, separated data (attrition bias) obtained between a 50 and 75% low risk by care provided by a cardiologists or primary care physicians, of bias. For the selective reporting (reporting bias), there was were as follows: acute myocardial infarction, cared by a cardi- a 100% low risk of bias. Finally, other types of bias had a low ologist OR 0.81 (95% CI: 0.79 to 0.83), and cared by a primary risk of between 50 and 75% (Figure 2). care physician OR 0.73 (95% CI: 0.71 a 0.76); heart failure, cared by a cardiologist OR 0.88 (95% CI: 0.86 to 0.90), and care The studies with the lowest and highest risk of bias (Kiessling by a primary care physician OR 0.79 (95% CI: 0.76 to 0.81). & Henriksson, 2002 and Jäntti et al., 2007, respectively) were chosen to be evaluated with the GRADE methodology and to The result favoured the intervention group for 25 interventions build a summary of findings table. The aim of doing this (30%). Some of the recommendations were: use of antiplate- GRADE table was obtaining the rank of possible grades of the let medication during the 24-hour hospital stay, a study of evidence certainty found in the 84 interventions from the nine the left ventricle ejection fraction, total cholesterol and LDL studies included. The results of these studies produced mod- measurements, cardiopulmonary resuscitation, and degree of erate-to-low evidence certainty, according to the GRADE compliance with EB-CPGs recommendations. classification (see Table 1 and Table 2). Discussion Assessment of study outcomes Methodological efforts have been made to develop trustworthy The authors grouped the outcomes in simple relative and abso- EB-CPGs. However, it still seems that some of these EB-CPGs are lute numbers. There was not a global estimate of the meas- far from the reality of the clinical practice (Institute of Medicine urements of the effects included in the studies because of the (US) Committee on Standards for Developing Trustworthy considerable variability of measuring units, as well as the clinical Clinical Practice Guidelines, 2011). heterogeneity found among the studies included. The measure- ment of the outcomes reported in the studies was dichotomous, It is essential to emphasise the main findings from the continuous or nominal; the majority were dichotomous and outcome analysis. It is surprising that most of the studies did were used to measure the mastery of the process, e.g. the differentiate between the control and experimental groups regard- number of patients receiving an adequate treatment following a ing the improvements with the use of EB-CPGs. The effects of recommendation versus those who did not. recommendations of the interventions included in the nine RCTs on the areas of health care structure (4%) and patient Of the total of 84 interventions included in the nine RCTs eval- outcomes were the least studied (32%). This fact could lead uated, three (4%) corresponded to the health care structure us to assume that researchers have given more importance to Figure 2. Overall bias risk chart of all included studies. Page 5 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Table 1. Evaluation of the Certainty of the Evidence for the Jäntti 2007 study on cardiopulmonary resuscitation, according to the GRADE classification. Are the recommendations presented in the ERC 2005 CPG better than those presented in the ERC 2000 CPG more adequate to improve the outcomes for cardiopulmonary resuscitation? Patient or population: Manikins simulation of cardiopulmonary arrest Intervention: CPG ERC 2005 Control: CPG ERC 2000 Bibliography: Jäntti H1, Kuisma M, Uusaro A.. The effects of changes to the ERC resuscitation guidelines on no flow time and cardiopulmonary resuscitation quality: a randomised controlled study on manikins. Resuscitation November 2007;2(75):338–44. [PubMed: 17628319] Average duration (seconds ± SE) Relative Certainty of the Outcomes № of participants (studies) Relevance Average with Average with effect (p) evidence (GRADE) ERC 2000 ERC 2005 34 Control Group: 17 Total time without ⊕⊕⊕◯ 393 ± 19 190 ± 23 p < 0.001 Intervention Group: 17 Critical flow (s) MODERATE1,2 (1 RCT (Randomized Controlled Trial) 34 Control Group: 17 Delay to start ⊕⊕◯◯ 8±6 8±4 p = 0.949 Intervention Group: 17 Critical CPR (s) LOW1,3 (1 RCT (Randomized Controlled Trial) ERC: European Resuscitation Council SE: Standard deviation p: significance GRADE Working Group grades of evidence High certainty: We are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect Explanations 1. For the generation of the random sequence (selection bias) the risk is High: sealed, numbered and opaque envelopes were used for the randomisation of the cases to be treated. It is not clear why people with less work experience or no academic degree were assigned to the ERC 2000. For the allocation concealment (selection bias) the risk is High: It is likely that by the nature of the study the allocation concealment of the selection could not be made. For blinding of participants and staff (performance bias) the risk is High: It is likely that the nature of the study could not prevent participants from knowing which group they belonged to (which CPG they were using). For blinding of the outcome evaluation (detection bias) the risk is Low: A computer automatically collected it. For incomplete results data (attrition bias) the risk is Low: There was no loss of follow-up since it was a single session. For the particular report (notification bias) the risk is Low: A computer automatically collected it. In other risks of bias, the risk of bias is High: The description of the study design was not clear; therefore, we assumed that the study has a high risk of bias. 2. P < 0.001 3. P 0.949. Downgraded -1 for imprecision. the evaluation of using (or not using) the recommendations results (Tierney et al., 2003), with 13 interventions, but found no in the area of the process, instead of their direct impact on meaningful differences in the results between the groups. the patient health. Most of the studies (eight) reported results in the process area The changes observed in the patient progression were gener- and showed favourable results for the intervention in six studies ally modest. In the six studies evaluating patient outcomes with 16 measurements. This fact reflects the reluctance of (27 interventions), four of them reported a positive result for health care providers to use EB-CPGs and means that only seven measurements (26%) of the interventions. For example, 30% of these interventions followed the recommendations the study by Ellis et al. (2000) measured the difference in the given by the EB-CPGs. average change of total cholesterol (mg/dl) and reported a decrease of 7.4 mg/dl in favour of the intervention. It is Our findings in this review coincide with some of those essential to mention that these measures were mostly surro- captured by Gagliardi et al. (2012) and Kastner et al. (2015) most gate variables. Only one study reported variables for clinical of the studies included in this review used multiple strategies Page 6 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Table 2. Evaluation of the Certainty of the Evidence for the secondary prevention of coronary artery disease according to the GRADE classification: (Kiessling & Henriksson, 2002) study. Is the active implementation more effective compared to passive implementation as it was used in the control group to improve secondary prevention of coronary artery disease? Patient or population: person with coronary artery disease. Intervention: Active implementation (general practitioners participated in learning dialogues using the CPG on secondary prevention of coronary artery disease, method of recurrent cases in their primary care centers) Control: Passive implementation (the CPG was mailed to general practitioners and presented at a conference) Bibliography: Kiessling A1, Henriksson P. Efficacy of case method learning in general practice for secondary prevention in patients with coronary artery disease: randomised controlled study. BMJ October 2002;7369(325):877-80. [DOI: 12386042] Average Change of active versus passive implementation of the CPG (%) Relative effect № of participants Certainty of the evidence Outcomes (95% CI) Relevance (p) (studies) (GRADE) Control Intervention Group Group 176 Control Group: 88 Difference in 0.7% Intervention percentage of LDL -9.3 % ⊕⊕⊕◯ (CI -4.1 a p < 0.05 Group: 88 IMPORTANT change at 2 years (CI -15.8 a -2.9) MODERATE1,2 5.9) (1 RCT [Mean (mmol / L)] (Randomized Controlled Trial) 176 Difference in Control Group: 88 percent change 1.8% Intervention -6.0 % ⊕⊕◯◯ in total cholesterol (CI -2.2 a p > 0.05 Group: 88 IMPORTANT (CI -10.4 a -1.5) LOW1,3 at 2 years [Mean 5.9) (1 RCT (mmol / L)] (Randomized Controlled Trial) CI: Confidence interval p: significance GRADE Working Group grades of evidence High certainty:We are very confident that the true effect lies close to that of the estimate of the effect Moderate certainty:We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty:Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty:We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect Explanations 1. For the generation of the random sequence (selection bias) the risk of bias is Uncertain: Not described. For allocation concealment (selection bias) the risk is low: general practitioners and patients did not know in which research group they were assigned. For blinding of participants and staff (performance bias) the risk is low: General practitioners were not aware of being involved in the study and a blinded nurse on which group each patient belonged to, was the one who handled the paperwork, protocols of the investigation and had no contact with general practitioners. For blinding of the outcome assessment (detection bias), the risk is Low: The research codes and databases were not disclosed until the authors completed the statistical analysis. For incomplete results data (attrition bias), the risk is Low: The study used the intention-to-treat analysis and indicated that the follow-up for two years was 86%. For the particular report (notification bias) the risk is Low: The research codes and databases were not disclosed until the authors completed the statistical analysis. For Other biases the risk is Low: None known 2. p < 0.05 3. p > 0.05. Downgraded -1 for imprecision. to implement the EB-CPGs (electronic notifications in a digital very clearly that although clinicians have the new evidence file, cell phone applications, letters, phone call memos, printed at hand, using it means a change in the behaviour and habits material), and the authors do not tailor to every recommendation of the clinical management of patients that is complex to (the implementation strategy is usually the same for every achieve unless appropriate strategies are applied. A more spe- recommendation within a EB-CPG). Cosby (2006) formulates cific approach, based on the results of the analysis of the Page 7 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 obstacles hindering the adoption of every recommendation health care quality improvements. The second RD will cover separately, might improve the use and effect of the recommenda- Breast Cancer and the third child and maternal health. We cre- tions in practice. ated an interactive https://lnkd.in/dwTxaD8 LIVE GUIDELINES program following the CDC https://lnkd.in/d5r4z6m Conclusions There is an imbalance between the number of EB-CPGs Since research in this field is so new, and the results of this study developed and the number of high-quality studies evaluat- were not conclusive, more research is needed to evaluate the ing their effectiveness. After analysing many studies, we can change that EB-CPGs could make to the quality of health care, have one more hypothesis for further research for more light emphasising the less studied areas, such as the structure of upon those undiscovered variables that might interfere with the health care services and patient outcomes. use of the EB-CPGs. Therefore, more studies of good quality are still needed. Data availability Underlying data The variation in the effects of the recommendations of the All data underlying the results are available as part of the article EB-CPGs suggests that it would be useful to focus on the anal- and no additional source data are required. ysis of the adherence limitations, as well as on designing imple- mentation strategies by adapting every recommendation, instead Extended data of considering the EB-CPGs as a whole. Further research is Open Science Framework: Effects of Evidence-Based Clinical still needed to determine which factors related to the EB-CPGs Practice Guidelines in cardiovascular health care quality and their specific recommendations are essential to predict improvements- A Systematic Review. https://doi.org/10.17605/ the use of EB-CPGs, and thus achieve better patient outcomes. OSF.IO/9A5FM (Ramirez et al., 2019). Implications for practice This project contains the following extended data: The initial objective of this review was to strengthen the development programs for EB-CPG by evaluating their effects on • Appendix 1 Advanced Search Strategy the quality of health care and to give reliable evidence to sustain • A  ppendix 2 List of studies selected after screening and the decision-making process related to the construction of EB- assessing the full text CPGs in medium- and low-income countries. Even the research evidence is not strong enough to support the EB-CPGs as a tool • A  ppendix 3 Characteristics of included studies English to improve the practice for a better quality of care, the results version of this review need to be interpreted with caution. Definitely. • Some electronic data extraction forms It seems the standard application used so far must be reviewed and must incorporate new psychosocial strategies oriented Reporting guidelines toward driving change in clinical practice and the doctor-patient Open Science Framework: PRISMA checklist for “Effects relationship We need reaching the right hands at the right time. of evidence-based clinical practice guidelines in cardiovascu- It is necessary to emphasise that the standard CPG implementa- lar health care quality improvements: A systematic review”. tion used so far must be reviewed, and must incorporate new https://doi.org/10.17605/OSF.IO/9A5FM (Ramirez et al., 2019). psychosocial strategies oriented toward driving change in clinical practice and the doctor-patient relationships. Acknowledgments For an adequate implementation of a EB-CPG, it is necessary to Anggie Ramirez thanks the professors of the School of take into account the possible costs, risks, and benefits that will be Epidemiology of the University of Frontera Temuco, Chile, and necessary to know the expected results precisely. The health sys- the advisor of the Master’s thesis, Mario Delgado MD, PhD, tems that build EB-CPGs must maximise their efforts to get health for his contribution and suggestions of the proposed protocol care personnel to follow the recommendations of the EB-CPGs of the review. and make an effort to evaluate their impact. Author information Implications for research Anggie Ramírez-Morera and Juan Carlos Vazquez are PhD This is the first of series of three systematic reviews on the effects candidates Program in Biomedical Research Methodology and of Evidence-Based Clinical Practice Guidelines (EB-CPGs) on Public Health, Universitat Autònoma de Barcelona References Alonso-Coello P, Irfan A, Solà I, et al.: The quality of clinical practice guidelines pectoris and mass dissemination of therapeutic guidelines: a randomized over the last two decades: a systematic review of guideline appraisal studies. controlled trial. QJM. 2004; 97(1): 21–31. Qual Saf Health Care. 2010; 19(6): e58. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text Berner ES, Baker CS, Funkhouser E, et al.: Do local opinion leaders augment Beaulieu MD, Brophy J, Jacques A, et al.: Drug treatment of stable angina hospital quality improvement efforts? A randomized trial to promote Page 8 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 adherence to unstable angina guidelines. Med Care. 2003; 41(3): 420–31. Clinical Practice Guidelines: Clinical practice guidelines we can trust R. Graham PubMed Abstract | Publisher Full Text et al., eds., Washington (DC): National Academies Press (US). 2011. Cosby JL: Improving patient care: the implementation of change in clinical PubMed Abstract | Publisher Full Text practice. Qual Saf Health Care. 2006; 15(6): 447. Jäntti H, Kuisma M, Uusaro A: The effects of changes to the ERC resuscitation Publisher Full Text | Free Full Text guidelines on no flow time and cardiopulmonary resuscitation quality: a Donabedian A: The quality of care. How can it be assessed? JAMA. 1988; randomised controlled study on manikins. Resuscitation. 2007; 75(2): 338–344. 260(12): 1743–1748. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text Kastner M, Bhattacharyya O, Hayden L, et al.: Guideline uptake is influenced by Ellis SL, Carter BL, Malone DC, et al.: Clinical and economic impact of six implementability domains for creating and communicating guidelines: a ambulatory care clinical pharmacists in management of dyslipidemia in realist review. J Clin Epidemiol. 2015; 68(5): 498–509. older adults: the IMPROVE study. Impact of Managed Pharmaceutical Care PubMed Abstract | Publisher Full Text on Resource Utilization and Outcomes in Veterans Affairs Medical Centers. Kiessling A, Henriksson P: Efficacy of case method learning in general practice Pharmacotherapy. 2000; 20(12): 1508–1516. for secondary prevention in patients with coronary artery disease: randomised PubMed Abstract | Publisher Full Text controlled study. BMJ. 2002; 325(7369): 877–880. Gagliardi AR, Brouwers MC, Bhattacharyya OK: The guideline implementability PubMed Abstract | Publisher Full Text | Free Full Text research and application network (GIRAnet): an international collaborative Lugtenberg M, Burgers JS, Westert GP: Effects of evidence-based clinical to support knowledge exchange: study protocol. Implement Sci. 2012; 7: 26. practice guidelines on quality of care: a systematic review. Qual Saf Health PubMed Abstract | Publisher Full Text | Free Full Text Care. 2009; 18(5): 385–392. Glasziou P, Ogrinc G, Goodman S: Can evidence-based medicine and clinical PubMed Abstract | Publisher Full Text quality improvement learn from each other? BMJ Qual Saf. 2011; 20 Suppl 1: Moore L, Lavoie A, Bourgeois G, et al.: Donabedian’s structure-process-outcome i13–17. quality of care model: Validation in an integrated trauma system. J Trauma PubMed Abstract | Publisher Full Text | Free Full Text Acute Care Surg. 2015; 78(6): 1168–75. Grimshaw JM, Russell IT: Effect of clinical guidelines on medical practice: a PubMed Abstract | Publisher Full Text systematic review of rigorous evaluations. Lancet. 1993; 342(8883): 1317–1322. Ramirez A, Tristan M, Vazquez JC: Effects of Evidence-Based Clinical Practice PubMed Abstract | Publisher Full Text Guidelines in cardiovascular health care quality improvements- A Systematic Guadagnoli E, Normand SL, DiSalvo TG, et al.: Effects of treatment Review. OSF. 2019. recommendations and specialist intervention on care provided by primary http://www.doi.org/10.17605/OSF.IO/9A5FM care physicians to patients with myocardial infarction or heart failure. Am J Sackett DL, Rosenberg WM, Gray JA, et al.: Evidence based medicine: what it is Med. 2004; 117(6): 371–9. and what it isn’t. BMJ. 1996; 312(7023): 71–72. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text | Free Full Text Guyatt G, Evidence-Based Medicine Working Group: Evidence-based medicine. Tierney WM, Overhage JM, Murray MD, et al.: Effects of computerized guidelines A new approach to teaching the practice of medicine. JAMA. 1992; 268(17): for managing heart disease in primary care. J Gen Intern Med. 2003; 18(12): 2420–2425. 967–976. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text | Free Full Text Hand WR, Bridges KH, Stiegler MP, et al.: Effect of a cognitive aid on adherence to Tsuyuki RT, Houle SK, Charrois TL, et al.: Randomized Trial of the Effect of perioperative assessment and management guidelines for the cardiac evaluation Pharmacist Prescribing on Improving Blood Pressure in the Community: The of noncardiac surgical patients. Anesthesiology. 2014; 120(6): 1339–49, quiz 1349-53. Alberta Clinical Trial in Optimizing Hypertension (RxACTION). Circulation. 2015; PubMed Abstract | Publisher Full Text | Free Full Text 132(2): 93–100. Higgins JPT, Green S (editors): Cochrane Handbook for Systematic Reviews PubMed Abstract | Publisher Full Text of Interventions Version 5.1.0 [updated March 2011]. Barcelona: The Cochrane Worrall G, Chaulk P, Freake D: The effects of clinical practice guidelines on Collaboration. 2012; [Accessed February 21, 2019]. patient outcomes in primary care: a systematic review. CMAJ. 1997; 156(12): Reference Source 1705–1712. Institute of Medicine (US) Committee on Standards for Developing Trustworthy PubMed Abstract | Free Full Text Page 9 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Open Peer Review Current Peer Review Status: Version 2 Reviewer Report 10 October 2019 https://doi.org/10.5256/f1000research.22588.r54141 © 2019 Marín León I. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Ignacio Marín León Fundación Enebro, Seville, Spain It is OK, the new version of the article clearly improve the original one and include the reviewers suggestions. Competing Interests: No competing interests were disclosed. Reviewer Expertise: Internal Medicine. I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Version 1 Reviewer Report 22 August 2019 https://doi.org/10.5256/f1000research.20676.r51028 © 2019 Marín León I. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Ignacio Marín León Fundación Enebro, Seville, Spain Thank you for the invitation to review this well-conducted systematic review. The study under review is very interesting and necessary for the Clinical Practice Guidelines (CPGs) community and for the implementation science and healthcare quality communities. I would like to congratulate the authors for this difficult challenge with some minor comments to contribute to improve the content and message of this necessary study Page 10 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 to improve the content and message of this necessary study ABSTRACT: Conclusion "There are probably still undiscovered variables that interfere with the use of the CPGs". That could not be a conclusion from the study, given that the aim of the work was not to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, then no conclusion could be provided. A coherent conclusion could be the need to improve and systematize this kind of implementation studies. Introduction: pag 3, para 4, left column "After an exhaustive search, only two systematic reviews (SR) we found on this topic," It is needed to provide both references RESULTS: pag 3, 4 and figure 1. Study identification and selection. "Of these, we selected 96 studies after assessing the full texts". It is needed, in order to strengthen the transparency and reproducibility of the work, to display in an extended data appendix, the full reference list of rejected originals and reasons to be rejected (96 found minus 9 selected = 87 rejected). It should be clarified what is meant in the figure 1 tab "# 87 of studies to include in other analyzes". Pag 4, left column, para 3, rows 4-6: "When assessing the full-text articles, we found that many were cluster trials" There is no clear reasons to reject RCT clusters studies, when the intervention could be randomized to cluster offices, and the outcomes effect measured at individual patient level. It is a well-accepted methodology in implementation science, given that can control for bias and non-measured variables. (Please see 2 references provided: Eccles and Romero).1,2 , Pag 5, Table 1, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)" It is difficult to understand why the certainty of evidence for the outcomes "time without flow" and "delay to start CPR" are classified differently when, as displayed in the explanations, the bias and systematic errors are the same, and there is not a loss of patients to follow up. It is true that both effects measure score different statistical significance, but with data provided the readers feel with the same certainty (either moderate or low) that one effect is significantly different from the control group, and for the other outcomes (delay) there are not differences with the control group. Pag 6, Table 2, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)" The same reasoning for the difficulty to understand why are classified differently the certainty of evidence for the outcomes "LDL cholesterol" and "Total cholesterol". Discussion: Pag 7, para 2, rows 7-13 "This fact could lead us to assume that researchers have given more importance to the evaluation Page 11 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 "This fact could lead us to assume that researchers have given more importance to the evaluation of using (or not using) the recommendations in the area of the process, instead of their direct impact on the patient health". Another explanation could be to emphasize that it use to be easier to measure process of care than patient health outcomes. These health outcomes take a longer period of observations than to measure how professionals adopt CPG recommendations. That can explain the "preferences" of researchers. Conclusions The same considerations exposed in the abstract regarding the sentences "There are probably still undiscovered variables that interfere with the use of the CPGs". That could not be a conclusion from the study, given that was not the aim of the work to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, them no conclusion could be provided. A coherent conclusion could be the needs to improve and systematize these kinds of implementation studies. References 1. Eccles M, Grimshaw J, Campbell M, Ramsay C: Research designs for studies evaluating the effectiveness of change and improvement strategies.Qual Saf Health Care. 2003; 12 (1): 47-52 PubMed Abstract | Publisher Full Text 2. Romero A, Alonso C, Marín I, Grimshaw J, de Villar E, Rincón M, Reyes A, Briones E, Calderón E, Ramón Lacalle J, Jaffey J: Efectividad de la implantación de una guía clínica sobre la angina inestable mediante una estrategia multifactorial. Ensayo clínico aleatorizado en grupos. Revista Española de Cardiología. 2005; 58 (6): 640-648 Publisher Full Text Are the rationale for, and objectives of, the Systematic Review clearly stated? Yes Are sufficient details of the methods and analysis provided to allow replication by others? Partly Is the statistical analysis and its interpretation appropriate? Yes Are the conclusions drawn adequately supported by the results presented in the review? Partly Competing Interests: No competing interests were disclosed. Reviewer Expertise: Internal Medicine. I confirm that I have read this submission and believe that I have an appropriate level of Page 12 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Author Response 05 Sep 2019 Anggie Ramirez, Autonomus University of Barcelona, Barcelona, Spain Dear Ignacio Marín León Thank you for your Review Report This response was concomitant between Dr. Ramirez and Dr. Tristan, part of the authors of this systematic review. APPROVED WITH RESERVATIONS Thank you for the invitation to review this well-conducted systematic review. The study under review is very interesting and necessary for the Clinical Practice Guidelines (CPGs) community and for the implementation of science and healthcare quality communities. I would like to congratulate the authors for this difficult challenge with some minor comments to contribute to improving the content and message of this necessary study #1 ABSTRACT: Conclusion "There are probably still undiscovered variables that interfere with the use of the CPGs". That could not be a conclusion from the study, given that the aim of the work was not to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, then no conclusion could be provided. A coherent conclusion could be the need to improve and systematize this kind of implementation studies. Answer: We change the text for a better understanding of our meaning. After analyzing many studies, we can have one more hypothesis for further research for more light upon those undiscovered variables that might interfere with the use of the CPGs. #2 Introduction: pag 3, para 4, left column "After an exhaustive search, only two systematic reviews (SR) we found on this topic," It is needed to provide both references Answer: We agree and change for: After an exhaustive search, only two systematic reviews (SR) we found on this topic (Worrall, 1997; Lugtenberg, 2009). Worral (1997) only analysed patient outcomes missing two of the proposed Donadebian Model. Lugtemberg 2009 used the full Donadebian (1988) model only including studies from The Netherlands. #3 RESULTS: pag 3, 4 and figure 1. Study identification and selection. "Of these, we selected 96 studies after assessing the full texts". It is needed, in order to strengthen the transparency and reproducibility of the work, to display in an extended data appendix, the full reference list of rejected originals and reasons to be rejected (96 found minus 9 selected = 87 rejected). It should be clarified what is meant in the figure 1 tab "# 87 of studies to include in other analyses". Answer: We agree . The Appendix 2: List of studies selected after screening and assessing the full text is added at OSF Home, the link is: https://osf.io/9a5fm/?view_only=0ec7b68ed1004bf6b1aa74a19ed1912d We did not reject the 87 publications, not analyzed. This SR is the first of series that has three Page 13 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 https://osf.io/9a5fm/?view_only=0ec7b68ed1004bf6b1aa74a19ed1912d We did not reject the 87 publications, not analyzed. This SR is the first of series that has three systematic reviews. This two new ones will include the topics breast cancer, type 2 diabetes and others. #4 Pag 4, left column, para 3, rows 4-6: "When assessing the full-text articles, we found that many were cluster trials" There is no clear reasons to reject RCT clusters studies, when the intervention could be randomized to cluster offices, and the outcomes effect measured at individual patient level. It is a well-accepted methodology in implementation science, given that can control for bias and non-measured variables. (Please see 2 references provided: Eccles and Romero).1,2 , Answer: The authors decide when preparing the protocol to include only RCT. #5 Pag 5, Table 1, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)" It is difficult to understand why the certainty of evidence for the outcomes "time without flow" and "delay to start CPR" are classified differently when, as displayed in the explanations, the bias and systematic errors are the same, and there is not a loss of patients to follow up. It is true that both effects measure score different statistical significance, but with data provided the readers feel with the same certainty (either moderate or low) that one effect is significantly different from the control group, and for the other outcomes (delay) there are not differences with the control group. Answer: Please check the note explanations both have different statistical significance due to this fact the certainty of evidence is downgraded for “Delay to start CPR (s)”. #6 Pag 6, Table 2, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)" The same reasoning for the difficulty to understand why are classified differently the certainty of evidence for the outcomes "LDL cholesterol" and "Total cholesterol". Answer: Please check the note explanations both have different statistical significance due to this fact the certainty of evidence is downgraded for “Difference in percent change in total cholesterol at 2 years [Mean (mmol / L)]”. #7 Discussion: Pag 7, para 2, rows 7-13 "This fact could lead us to assume that researchers have given more importance to the evaluation of using (or not using) the recommendations in the area of the process, instead of their direct impact on the patient health". Another explanation could be to emphasize that it use to be easier to measure process of care than patient health outcomes. These health outcomes take a longer period of observations than to measure how professionals adopt CPG recommendations. That can explain the "preferences" of researchers. Answer: You are right. #10 Conclusions The same considerations exposed in the abstract regarding the sentences "There are probably still undiscovered variables that interfere with the use of the CPGs". That could not be a conclusion from the study, given that was not the aim of the work to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, them no conclusion could be provided. A coherent conclusion could be the needs to improve and systematize these kinds of implementation studies. Page 14 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 implementation studies. Answer: We change the text for a better understanding of our meaning. After analyzing many studies, we can have one more hypothesis for further research for more light upon those undiscovered variables that might interfere with the use of the CPGs. Competing Interests: We are the author of this SR. Reviewer Report 24 July 2019 https://doi.org/10.5256/f1000research.20676.r51034 © 2019 Amer Y. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Yasser Sami Abdel Dayem Amer 1 Clinical Practice Guidelines Unit, Quality Management Department, King Saud University Medical City, Riyadh, Saudi Arabia 2 Alexandria Center for Evidence-Based Clinical Practice Guidelines, Alexandria University, Alexandria, Egypt Thank you for the invitation to review this well-conducted systematic review. The study under review is very interesting for the evidence-based clinical practice guidelines (CPGs)- and for the healthcare quality and safety- research communities. Moreover, I would like to congratulate the authors for this work with some minor comments with the aim of improving the content and message of this distinguished study. Abstract: 1. I recommend replacing the phrase 'results in patients' with 'patient outcomes' as it is more commonly used in the field of quality and safety in healthcare. Additionally, the authors have used it for the rest of the MS. Introduction: 2. I would recommend adding recent definitions of both CPGs and healthcare quality (HCQ) as nowadays clinicians are getting increasingly interested in the fields of evidence-based medicine (EBM) and HCQ either from a research perspective or through engagement in their respective healthcare organizational accreditation. The HCQ definition and description of EB-CPGs are already in the PROSPERO protocol, please include them here as well. Additional relevant studies to cite here: (i) about the relation between EBM and HCQ: Glasziou P, Ogrinc G, Goodman S. Can evidence-based medicine and clinical quality improvement learn from each other?. BMJ Qual Saf. 2011;20 Suppl 1(Suppl_1):i13–i17. doi:10.1136/bmjqs.2010.0465241 Page 15 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 doi:10.1136/bmjqs.2010.0465241 (ii) an earlier similar key review: Grimshaw J, Russell I: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. The Lancet. 1993; 342 (8883): 1317-13222 page 3, left column, para 4: 3. '..After an exhaustive.....only two systematic reviews (SRs)....., but 'these SRs' did not....' Please add citations for these two references (SRs). 4. '...and international perspective together.' What do you mean by an 'international perspective'; of what; (i.e. of CPGs, of healthcare quality, or of cardiovascular healthcare?), please clarify or re-write. page 3, right column, para 4: 5. Regarding the data extraction modified and translated from the EPOC checklist, was it validated and if not, why the authors felt they did not need to? Additionally, please provide the readers who would be interested in the replication of this research with this tool as an online supplementary here. It was indeed mentioned with a link in the PROSPERO protocol ( https://fs20.formsite.com/mtristan/form41/index.html, attached on Appendix 3) but the link did not work, giving an error! Moreover, I checked the (Extended data) links in the OSF Home but could not retrieve it. 6. Please delete the duplicated sentence; 'We used a standardized digital ..........and analysis'. page 3, right column, para 7: 7. The #87 studies that were planned to be included in another analysis were not mentioned here since the final included were 9 rather than 96 studies. Please elaborate on this in this location. 8. Figure 1. PRISMA flow chart: #87 of studies to include in other 'analyses'. Tables 1 and 2.: the GRADE quality assessments and the explanations for the certainty of evidence were very well conducted and written. 9. Table 1: please, if feasible, replace the question in the GRADE SoF table; '.....the results for cardiopulmonary resuscitation?' with '.....the outcomes for cardiopulmonary resuscitation?' page 7, left column para 1: 10. Once more, please replace 'patient results' with 'patient outcomes'. page 7, left column, para4: Discussion: 11. '..However, it still seems that some of these CPGs are...' page 7, right column, para 2: 12. As part of discussing 'multiple strategies to implement the CPGs', I would recommend adding insights and citing these relevant keynote articles that discuss multi-faceted CPGI strategies and their importance: Page 16 of 22 12. F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 (i) Grol R, Wensing M. Improving patient care: the implementation of change in health care (in Dutch: Implementatie: effective verbetering van de patiëntenzorg). Amsterdam: Reed Business Education; 2013. (ii) Kastner M, Bhattacharyya O, Hayden L, et al. Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review. J Clin Epidemiol. 2015;68(5):498–509. doi: 10.1016/j.jclinepi.2014.12.013.3 (iii) Gagliardi A, Brouwers M, Bhattacharyya O, et al. A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools. Implement Sci. 2014;9:98. doi: 10.1186/s13012-014-0098-84 (iv) Gagliardi A, Brouwers M, Palda V, Lemieux-Charles L, Grimshaw J. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26. doi: 10.1186/1748-5908-6-26.5 page 7, right column, para 4: 13. '...... as well as on designing implementation strategies adapt every recommendation...' Do you mean "designing implementation strategies AND adapt every recommendation....if so add a comma ',' " or "designing implementation strategies (by) adapting every recommendation". Additionally, I suggest to add: '....initiate a pilot implementation of specific recommendation(s) instead of considering implementation of the CPGs as a whole'. page 7, right column, para 5: 14. Implications for practice The initial objective of this review was to strengthen the development programs for CPGs (plural) 15. The results of this review must need to be interpreted with caution. It reads better either (the results of this review must be interpreted with caution.) or (the results of this review need to be interpreted with caution.). 16. 'It is necessary to emphasise that the standard CPG implementation used so...' page 7, right column, para 6: 17. 'For an adequate implementation of a CPG, it is necessary to take into account the possible costs, risks, and benefits Finally, I would like to congratulate the authors once more on this work and invite all interested researchers to conduct similar research using this robust methodology to study the impact of EB-CPG implementation on all different healthcare specialties in addition to their effects on the interdisciplinary workflows and transitions of care between different specialties and services in healthcare. References 1. Glasziou P, Ogrinc G, Goodman S: Can evidence-based medicine and clinical quality improvement learn from each other?. BMJ Qual Saf. 2011; 20 Suppl 1: i13-17 PubMed Abstract | Publisher Full Text 2. Grimshaw JM, Russell IT: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations.Lancet. 1993; 342 (8883): 1317-22 PubMed Abstract | Publisher Full Text 3. Kastner M, Bhattacharyya O, Hayden L, Makarski J, Estey E, Durocher L, Chatterjee A, Perrier L, Graham ID, Straus SE, Zwarenstein M, Brouwers M: Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review.J Clin Epidemiol. 2015; 68 (5): 498-509 PubMed Abstract | Publisher Full Text Page 17 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 2015; 68 (5): 498-509 PubMed Abstract | Publisher Full Text 4. Gagliardi AR, Brouwers MC, Bhattacharyya OK, Guideline Implementation Research and Application Network: A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools.Implement Sci. 2014; 9: 98 PubMed Abstract | Publisher Full Text 5. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A conceptual framework of implementability.Implement Sci. 2011; 6: 26 PubMed Abstract | Publisher Full Text Are the rationale for, and objectives of, the Systematic Review clearly stated? Yes Are sufficient details of the methods and analysis provided to allow replication by others? Yes Is the statistical analysis and its interpretation appropriate? Yes Are the conclusions drawn adequately supported by the results presented in the review? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Pediatrics, child healthcare, evidence-based healthcare, knowledge translation, quality and safety in healthcare, healthcare informatics, and implementation and improvement sciences and research in general I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Author Response 05 Sep 2019 Anggie Ramirez, Autonomus University of Barcelona, Barcelona, Spain Dear Yasser Sami Abdel Dayem Amer Thank you for your Review Report This response was concomitant between Dr. Ramirez and Dr. Tristan, part of the authors of this systematic review. Thank you for the invitation to review this well-conducted systematic review. The study under review is very interesting for the evidence-based clinical practice guidelines (CPGs)- and for the healthcare quality and safety- research communities. Moreover, I would like to congratulate the authors for this work with some minor comments with the aim of improving the content and message of this distinguished study. Abstract: 1. I recommend replacing the phrase 'results in patients' with 'patient outcomes' as it is more commonly used in the field of quality and safety in healthcare. Additionally, the authors have used it for the rest of the MS. Answer: We agree and we will make the change. Page 18 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Answer: We agree and we will make the change. Introduction: 1. I would recommend adding recent definitions of both CPGs and healthcare quality (HCQ) as nowadays clinicians are getting increasingly interested in the fields of evidence-based medicine (EBM) and HCQ either from a research perspective or through engagement in their respective healthcare organizational accreditation. The HCQ definition and description of EB-CPGs are already in the PROSPERO protocol, please include them here as well.Additional relevant studies to cite here:(i) about the relation between EBM and HCQ:Glasziou P, Ogrinc G, Goodman S. Can evidence-based medicine and clinical quality improvement learn from each other?. BMJ Qual Saf. 2011;20 Suppl 1(Suppl_1):i13–i17. doi:10.1136/bmjqs.2010.0465241(ii) an earlier similar key review:Grimshaw J, Russell I: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. The Lancet. 1993; 342 (8883): 1317-13222Answer: We agree and we will make the change CPGs for EB-GPGs and added the recommended references the are very useful. page 3, left column, para 4: 1. '..After an exhaustive.....only two systematic reviews (SRs)....., but'these SRs' did not....'Please add citations for these two references (SRs). Answer: We agree, and we will make the change. 1. '...and international perspective together.' What do you mean by an 'international perspective'; of what; (i.e. of CPGs, of healthcare quality, or of cardiovascular healthcare?), please clarify or re-write. Answer: We agree, and we will make the change for: after an exhaustive search, only two systematic reviews (SR) we found on this topic (Worrall 1997, Lugtenberg 2009). Worral (1997) only analysed patient outcomes missing two of the proposed Donadebian Model. Lugtemberg 2009 used the full Donadebian (1988) model only including studies from The Netherlands. page 3, right column, para 4: 1. Regarding the data extraction modified and translated from the EPOC checklist, was it validated and if not, why the authors felt they did not need to? Additionally, please provide the readers who would be interested in the replication of this research with this tool as an online supplementary here.It was indeed mentioned with a link in the PROSPERO protocol ( https://fs20.formsite.com/mtristan/form41/index.html, attached on Appendix 3) but the link did not work, giving an error!Moreover, I checked the (Extended data) links in the OSF Home but could not retrieve it. Answer: We agree, and we will make the change. The Some electronic data extraction forms the link is not available now it is a temporary link during the active time of the survey period in the form site server. Now some results of the on line survey are available in this link https://osf.io/9a5fm/?view_only=0ec7b68ed1004bf6b1aa74a19ed1912d as Some electronic data extraction forms/Short version of EPOC Check list.docx. The link in the OSF Home is working. 1. Please delete the duplicated sentence; 'We used a standardized digital ..........and analysis'. Answer: We agree, and we will make the change page 3, right column, para 7: 1. The #87 studies that were planned to be included in another analysis were not mentioned Page 19 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 1. The #87 studies that were planned to be included in another analysis were not mentioned here since the final included were 9 rather than 96 studies. Please elaborate on this in this location. Answer: We agree, and we will make the change. And we add the following: When the first version of the protocol of this Systematic Review was initiated, consider the inclusion of several clinical topics in a single systematic review. We decided to create a series of three systematic reviews on different clinical topics: Cardiovascular Health, Breast Cancer and Child and Mother Health. The initial searches included all those topics with the same inclusion criteria described. After removing duplicates, the search produced 4279 potential studies. After screening by title and abstract, 4051 were excluded. After the full-text evaluation, 96 studies of all clinical subjects were selected. For this SR, only nine studies covering the cardiovascular issue were included. This is the first of series of three systematic reviews on the effects of Evidence-Based Clinical Practice Guidelines (EB-CPGs) on health care quality improvements. The second RD will cover Breast Cancer and the third child and maternal health. We created an interactive https://lnkd.in/dwTxaD8 LIVE GUIDELINES program following the CDC https://lnkd.in/d5r4z6m 1. Figure 1. PRISMA flow chart: #87 of studies to include in other 'analyses'. Answer: We agree, and we will make the change Tables 1 and 2.: the GRADE quality assessments and the explanations for the certainty of evidence were very well conducted and written. 1. Table 1: please, if feasible, replace the question in the GRADE SoF table; '.....the results for cardiopulmonary resuscitation?' with '.....the outcomes for cardiopulmonary resuscitation?' Answer: We agree, and we will make the change page 7, left column para 1: 1. Once more, please replace 'patient results' with 'patient outcomes'. Answer: We agree, and we will make the change page 7, left column, para4:Discussion: 1. '..However, it still seems that some of these CPGs are...' Answer: We agree, and we will make the change page 7, right column, para 2: 1. As part of discussing 'multiple strategies to implement the CPGs', I would recommend adding insights and citing these relevant keynote articles that discuss multi-faceted CPGI strategies and their importance:(i) Grol R, Wensing M. Improving patient care: the implementation of change in health care (in Dutch: Implementatie: effective verbetering van de patiëntenzorg). Amsterdam: Reed Business Education; 2013. (ii) Kastner M, Bhattacharyya O, Hayden L, et al. Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review. J Clin Epidemiol. 2015;68(5):498–509. doi: 10.1016/j.jclinepi.2014.12.013.3 (iii) Gagliardi A, Brouwers M, Bhattacharyya O, et al. A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools. Implement Sci. 2014;9:98. doi: 10.1186/s13012-014-0098-84 (iv) Gagliardi A, Brouwers M, Palda V, Lemieux-Charles L, Grimshaw J. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26. doi: 10.1186/1748-5908-6-26.5 Page 20 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Answer: Thank you very much for the recommended references. We agree that they will give strength to the implementation aspect, one of the most sensitive components for the use of CPG. In relation to the reference Gagliardi 2011, We will not take it in the article because the Research Implementation Guide and Application Network (GiRAnet), is not active and in its link http://www.cebhc.co.za/guideline- implementability-research -y-application-network-giranet-2 /, one link does not work and the other is restricted access. We had access to another article by the same author of 2012, which we share ideas that we have already implemented and this study is in protocol: Gagliardi, A. R., Brouwers, M. C., & Bhattacharyya, O. K. (2012). The guideline implementability research and application network (GIRAnet): an international collaborative to support knowledge exchange: study protocol. Implementation Science, 7, 26. https://doi.org/10.1186/1748-5908-7-26 The new paragraph reads as follows: Our findings in this review coincide with some of those captured by Gagliardi (2012) and Kastner (2015) most of the studies included in this review used multiple strategies to implement the EB-CPGs (electronic notifications in a digital file, cell phone applications, letters, phone call memos, printed material), and the authors do not tailor to every recommendation (the implementation strategy is usually the same for every recommendation within a EB-CPG) Cosby (2006) formulates very clearly that although clinicians have the new evidence at hand, using it means a change in the behaviour and habits of the clinical management of patients that is complex to achieve unless appropriate strategies are applied. A more specific approach, based on the results of the analysis of the obstacles hindering the adoption of every recommendation separately, might improve the use and effect of the recommendations in practice. page 7, right column, para 4: 1. '...... as well as on designing implementation strategies adapt every recommendation...'Do you mean "designing implementation strategies AND adapt every recommendation....if so add a comma ',' " or "designing implementation strategies (by) adapting every recommendation". Additionally, I suggest to add:'....initiate a pilot implementation of specific recommendation(s) instead of considering implementation of the CPGs as a whole'. Answer: We partially agree, and we will make the change. We will not mention or suggest of pilot program implementation because most of them majority yield very little or nothing. page 7, right column, para 5: 1. Implications for practice The initial objective of this review was to strengthen the development programs for CPGs (plural) Answer: We agree, and we will make the change 1. The results of this review must need to be interpreted with caution. It reads better either (the results of this review must be interpreted with caution.) or (the results of this review need to be interpreted with caution.). Answer: We agree, and we will make the change 1. 'It is necessary to emphasise that the standard CPG implementation used so...'Answer: We agree, and we will make the change page 7, right column, para 6: 1. 'For an adequate implementation of a CPG, it is necessary to take into account the possible costs, risks, and benefits Answer: We agree, and we will make the change Page 21 of 22 F1000Research 2019, 8:1041 Last updated: 16 OCT 2019 Answer: We agree, and we will make the change Competing Interests: We are the author of this SR. The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias You can publish traditional articles, null/negative results, case reports, data notes and more The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com Page 22 of 22