Coronado Daza et al. BMC Nephrology (2019) 20:192 https://doi.org/10.1186/s12882-019-1387-x RESEARCH ARTICLE Open Access Quality assessment of clinical practice guidelines for chronic kidney disease: a systematic review Jorge Coronado Daza1,2* , Robin W. M. Vernooij3,4†, Karla Salas5†, Dimelza Osorio6† and Gerard Urrútia Cuchí3,4,7† Abstract Background: Chronic kidney disease (CKD) is a worldwide public health problem. Clinical practice guidelines (CPGs) are being developed and implemented in order to improve clinical practice related to the detection and treatment of CKD. The objective of our study was to evaluate the quality of CPGs regarding the CKD and to examine whether there are factors which influence their quality. Methods: A systematic search was conducted to identify all CPGs regarding the early diagnosis and treatment of CKD. The CPGs quality were evaluated by three reviewers using the AGREE II instrument to decide if the guidelines are recommended for their use in clinical practice. Results: In total, 13 CPGs were identified: five from America, six from Europe, one from Asia, and one from Oceania. Five CPGs were recommended for their use in clinical practice; since all their domains achieved the medium or high category. Furthermore, six CPGs were recommended with modifications, as the stakeholders’ involvement, applicability, and editorial independence domains were evaluated as low category. These domains, as well as the rigor of the development domain, reached the very low category in those CPGs that were not recommended for its use in clinical practice. In all CPGs, the domains with the lowest average were the stakeholder involvement and the applicability. When comparing the domains of the CPGs according to the origin, type of developer group, the checklist used during the development and the publication period, a significantly higher average in the domain stakeholder involvement was found in the CPGs from Asia and Oceania compared to the ones in Latin America. Additionally, a significantly higher average in the applicability domain was found in the CPGs developed by CPGs developer organizations compared to those developed by medical societies. Conclusions: In total, 85% of the CPGs regarding CKD were recommended or recommended with modifications. The stakeholder involvement and applicability domains are assessed in the low category, which might affect the CPGs implementation. In order to save resources in low- and middle-income countries, an adaptation of the recommended CPGs should be considered. Keywords: Chronic kidney disease, Clinical practice guidelines, Quality assessment, Early diagnosis * Correspondence: jocodada@yahoo.es † Robin W. M. Vernooij, Karla Salas, Dimelza Osorio and Gerard Urrútia Cuchí contributed equally to this work. 1 Facultad de Medicina, Universidad de Cartagena, Cartagena 130014, Colombia 2 Nefrologia y Diálisis SAS, Cartagena 130001, Colombia Full list of author information is available at the end of the article © The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 2 of 12 Background database, National Guideline Clearinghouse (via https:// Chronic kidney disease (CKD) is considered since 2005 www.ahrq.gov/topics/national-guideline-clearinghouse- a worldwide public health problem. Furthermore, since ngc.html), National Institute for Health and Care Excel- 2007, all countries have been urged to adopt actions for lence (NICE), Scottish Intercollegiate Guidelines Net- the control of CKD [1, 2]. The interest in establishing work (SIGN), Guidelines International Network (G-I-N), CKD as a public health issue was crucial to establish and the National Health System library of Spain. An control measures and stopping the increase of CKD inci- additional search was conducted in the international dence. However, to date, there is no evidence stating that nephrology societies or associations webpages of each the use of control strategies has contributed to a de- country or region. Additionally, a systematic search was crease in the incidence of CKD. For instance, in the performed in MEDLINE and EMBASE, investigating the United States of America (USA), the incidence of CKD medical terms headers related to CKD, applying CPG increased from 353 per million population (pmp) in filters. 2005 to 370 pmp in 2014 [3]. One strategy to tackle the problem has been the devel- Clinical practice guidelines assessment opment of CPGs for use of the stakeholders (physicians, Three authors independently (JC, RV, KS) assessed the patients, educators, health care providers, and govern- quality of each CPG, under the guidance of an expert in ment regulatory agencies). The main objectives of CPGs Investigation Methodology (DO). The AGREE II instru- are the identification of the population at risk of CKD, ment [5], which consists of 23 items organized in six do- early detection, and strategies to avoid the progression mains, followed by 2 items of global score (overall of the disease. To achieve these objectives, appropriate assessment), was used. Each domain embraces a unique stakeholders must get involved in the development dimension in the CPG quality: scope and purpose, stake- process of CPGs, as recommended by the AGREE holder(s) involvement, clarity of presentation, rigour of (Appraisal of Guidelines for Research & Evaluation) development, applicability, and editorial independence. and its updated version, AGREE II [4, 5]. The AGREE The overall assessment includes a score for the general II instrument was also created to help achieving ad- quality of the CPG and an assessment whether it is rec- equate reporting of the development process and ommended for use in clinical practice. Each item was presentation of CPGs. assessed using a 7-point scale (from 1 “strongly disagree” In two recent systematic reviews the quality of CPGs to 7 “strongly agree”), even if it was not applicable. The addressing CKD has been analysed. The AGREE II do- domain scores was expressed as a percentage over the mains with the lowest scores are stakeholder’s involve- highest possible score using the following formula: ment and applicability [6, 7]. This is similar to CPGs evaluations in other diseases [8–10]. Obtained score − Minimum possible score In this study, we present a quality assessment of CPGs Maximum possible score − Minimum possible score regarding the early detection and management of CKD. Additionally, we evaluated whether there was a differ- The final score is the sum of the total scores assigned ence in the quality according to the region where the to each domain by each reviewer. The maximum pos- CPGs were developed, type of developer organization, sible score is 7 (strongly agree), multiplied by the num- self-assessment process in its development, and publica- ber of items in the domain and the number of reviewers. tion period. The minimum possible score is 1 (strongly disagree) multiplied by the number of items in the domain and Methods the number of reviewers. For the overall assessment, a We included national and regional CPGs for the early score from 1 to 7 was consigned, as well as a recommen- detection and management of CKD. We included CPGs dation regarding the use of the CPG in clinical practice which included: 1) recommendations based on syste- classified as: recommended, recommended with modifi- matic evidence synthesis; 2) employing a grading sys- cations, and not recommended. tem to rate the quality of evidence, 3) published in English, Spanish or languages that were feasible to Data analysis translate for the authors; and 4) published between Descriptive statistics were applied to analyse every do- 2008 and 2016. main (percentage; mean and standard deviation; median and interquartile range). The mean of the domains score CPGs identification was categorized as high (≥80%), medium (60–79%), low A systematic search was conducted in the main data- (40–59%), or very low (≤40%). The overall mean of each bases of organisations that develop or compile CPGs in- of the CPGs domains were compared using Student’s cluding: the Turning Research Into Practice (TRIP) t-tests for independent samples (the test was two-tailed, Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 3 of 12 and statistical significance was considered for P-values CPGs included were from Scotland [11], Spain [12], of less than 0.05), according to: CPG region, developer Netherlands [13], Mexico [14], Argentina [15], Chile [16], group, use of a self-assessment instrument, and publica- Malaysia [17], England [18], Italy [19], Sociedad Latinoa- tion period (2008–2011 versus 2012–2016). mericana de Nefrología e Hipertensión (SLANH) [20], The degree of agreement between reviewers was deter- Australasia [21], USA [22] and United Kingdom (NICE mined by the measurement of intraclass correlation co- guideline) [23]. efficient (ICC) and its 95% confidence interval. An ICC General characteristics of the 13 included CPGs are of > 0.9 was considered “very good”, between 0.71 and presented in Table 1. Six CPGs were developed in 0.9 “good”, between 0.51 and 0.7 “moderate”, between Europe (46.2%), four in Latin America (30.7%), one in 0.31 and 0.5 “fair”, and < 0.31 “poor” or “non-existent”. Asia (7.7%), one in Oceania (7.7%), and one in the USA (7.7%). Eight CPGs (61.5%) were published between Results 2008 and 2011 and five between 2012 and 2016 (38.5%). The systematic search retrieved 893 records. After delet- Regarding the language, five CPGs were written in Span- ing duplicates and reviewing titles and abstracts, we iden- ish (38.4%), six in English (46.2%), one in Italian (7.7%), tified 24 CPGs that were potentially eligible (Fig. 1). and one in Dutch (7.7%). Eight CPGs (61.5%) were is- Consequently, after the full-text evaluation, 11 CPGs were sued by organizations that develop CPGs, three (23.1%) excluded because they did not concern national CPGs, did by medical societies, and two (15.4%) by medical soci- not include recommendations regarding screening or early eties with an organization dedicated to developing CPGs diagnosis of CKD, or did not present recommendations (15.4%). Concerning the CPG developer panel, 12 (92%) based on the evidence (Additional file 1). Finally, 11 included nephrologists, four (30.7%) experts in method- national CPGs and two regional CPGs were included. The ology, and four (30.7%) the target population. During Fig. 1 Flowchart of bibliographical research Table 1 General characteristics of the included guidelines Guidelines name Country Year of Language of Type of Guideline The GDG The GDG The GDG Checklist Grading Adapted publication publication Developers include include include target system nephrologists methodologists population a Diagnosis and Management of Chronic Kidney Scotland 2008 English Organization YES NO YES NO SIGN NO Disease. A National Clinical Guideline b Guías SEN para el manejo de la Enfermedad España 2008 Spanish Medical society YES NO NO NO Adapted NO Renal Crónica (ERC) Avanzada y Pre-Diálisis of CTFPHC Coronado Daza et al. BMC Nephrology Richtlijn voor de behandeling van patiënten Netherlands 2009 Dutch Medical society YES NO NO NO Not specified NO met Chronische Nierschade (CNS) Prevención Diagnóstico y Tratamiento de la México 2009 Spanish Organization YES YES NO NO Several YES ERC Temprana Guía de Práctica Clínica sobre Prevención y Argentina 2010 Spanish Medical society YES NO NO NO SIGN YES (2019) 20:192 Detección Precoz de la ERC en Adultos en el and organization Primer Nivel de Atención Guía Clínica Prevención de ERC Chile 2010 Spanish Organization YES NO NO NO Ministry YES of Health Management of chronic kidney disease in adults Malaysia 2011 English Organization YES YES NO NO Modified NO SIGN; CTFPHC Renal Association Clinical Practice Guideline on England 2011 English Organization YES NO NO AGREE GRADE NO Detection Monitoring and Management of Patients with CKD Identificazione, prevenzione e gestione della Italy 2012 Italian Organization YES YES YES Checklist: Several YES malattia renale cronica nell’adulto SNLG-ISS Guías Latinoamericanas de Practica Clínica Sobre Latino América 2012 Spanish Medical society YES NO NO NO GRADE YES la Prevención, Diagnostico y Tratamiento de los and organization Estadios 1–5 de la ERC KHA-CARI Guidelines: Chronic Kidney Disease Australasia 2013 English Organization YES NO YES NO GRADE NO Screening, Monitoring, and Treatment of stage United States 2013 English Medical society NO YES NO NO ACP NO 1 to 3 Chronic Kidney Disease: A Clinical Practice Guideline From the American College of Physicians Chronic kidney disease United Kingdom 2014 English Organization YES NO YES NICE NICE NO a b Government agencies, disease-specific organizations, non-profit health delivery systems, guidelines development agencies; Professional associations; GDG Guideline Development Group, AGREE Appraisal of Guidelines for Research & Evaluation, SNLG-ISS Sistema nazionale linee guida (National Guideline System), SIGN Scottish Intercollegiate Guidelines Network, CTFPHC Canadian Task Force on Preventive Health Care, GRADE Grading of Recommendations Assessment, Development and Evaluation, KHA-CARI Kidney Health Australia – Caring for Australasians with Renal Impairment, ACP American College of Physicians, NICE National Institute for Health and Clinical, ERC Enfermedad renal crónica (Chronic kidney disease) Page 4 of 12 Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 5 of 12 the CPGs development process, only three CPGs (23.1%) Domain 3: rigour of development used a checklist to verify if they were fulfilling the re- The rigour of development domain considers the pro- quirements that a CPG should have. The system used to cess used to gather and synthesize the evidence, the classify the quality of evidence and the strength of the methods to formulate the recommendations, and the up- recommendations differed among the included CPGs; dating process (items 7–14). The overall average score three (23.1%) used the GRADE (Grading of Recom- was 64.9% (median = 68.4%; IQR, 22.9 to 93.1%). Fur- mendations Assessment, Development and Evaluation) thermore, 38.4% of the CPGs were considered within the system, three used the CPG developers’ own system high category, 23% within the medium category, and (23.1%), three used mixed systems (23.1%), two (15.4%) 38.6% within the low or very low category. The average the SIGN (Scottish Intercollegiate Guidelines Network) score of the CPGs that were recommended for use in tool, one (7.7%) used CTFPHC (Canadian Task Force on clinical practice was 85.3%, for CPGs recommended with Preventive Health Care) system, and one did not specify modifications was 61.7%, and for those not recom- the used system (7.7%). Five CPGs (38.5%) were adapted mended 31.9%. from international CPGs: all four Latin America and one European. Domain 4: clarity of presentation The clarity of presentation domain refers to the lan- CPGs quality general assessment guage, structure, and the CPGs format (items 15–17). The degree of agreement between the three reviewers was The overall average was 88.9% (median = 92.59%; IQR, good, with an ICC of 0.88 (95% CI: 0.67–0.96) for the 62.96 to 96.3%). In total, 92.3% of the CPGs were con- overall score. The CPG quality score varied from 3 to 7 sidered within the high category. The average score of and the reviewers recommended five CPGs (38.5%; CPG the CPGs recommended for use in clinical practice was scores between 5 and 7: Scotland, Malaysia, Australasia, 93.3%, for those recommended with modifications 91%, USA and United Kingdom). Six CPGs were recommended and for those not recommended 72.2%. with modification (46.1%; CPG scores between 4 to 5: Mexico, Argentina, Chile, England, Italy and SLANH), Domain 5: applicability and two CPGs were not recommended (15.4%; CPG This domain is related to the barriers and facilitators to scores between 3 to 4: Spain and Netherlands). The aver- its implementation, strategies to improve uptake, and age score of each domain of all included CPGs and their resource implications when applying the CPGs (items respective recommendation can be observed in Table 2, 18–21). The overall average score was 47.9% (median = and Fig. 2. 43.75; IQR, 2.78 to 87.5%). In total, 23% of the CPGs were assessed as the high category, 15.3% as the medium Domain 1: scope and purpose category, and 61.7% as the low or very low category. The This domain refers to the CPG general objective, the spe- average score of the recommended CPGs was 69.4%, for cific health aspects, and the population to whom this CPG those recommended with modifications 45.8%, and for is intended (items 1–3). The overall average score was those not recommended 6.9%. 80.1% (median = 88.9%; interquartile range (IQR), 50 to 98.2%). Furthermore, 61.5% of the included CPGs had Domain 6: editorial independence an average score of 80% or higher, which is considered Finally, the editorial independence domain refers to the high category. The average score of the recommended conflicts of interest of the panel members and the role CPGs was 93, 77.5% for the CPGs recommended with of the funding body (items 22–23). The overall average modifications, and 62% for the CPGs that were not was 62.7% (median = 59.72%; IQR, 19.44 to 94.44%). In recommended. total, 23% of the CPGs were in the high category, 30.7% in the medium category, and 46.3% in the low Domain 2: stakeholder involvement or very low category. The average score of the recommen- This domain refers to the degree to which stakeholders ded CPGs was 85%, those recommended with modi- have developed the CPG and the representation of the fications scored 55.6%, and those not recommended points of view of the intended users (items 4–6). The scored 36.1%. overall average was 58.2% (median = 59.3%; IQR, 27.8 to 94.4%). Furthermore, 15.3% of the CPGs were evaluated CPGs assessment according to recommendation as high category, 15.3% as medium category, and 69.4% When comparing the AGREE II domains between the as low or very low category. The CPGs that were recom- recommended CPGs and the CPGs recommended with mended had an average score of 74.4%, the CPGs rec- modifications, we found a significant difference in the ommended with modifications scored 53.4%, and the stakeholder involvement (74.4 ± 14.3 vs. 53.4 ± 8.2 re- CPGs that were not recommended scored 39.8%. spectively; p = 0.013), rigor of development (85.3 ± 5 Table 2 Domain-standardized scores of each clinical practice guidelines, as assessed using the Appraisal of Guidelines Research Instrument Guideline / Country or Region Scope and Stakeholder Rigor of Clarity and Applicability (%) Editorial Overall Recommendation Purpose (%) Involvement (%) Development (%) Presentation (%) Independence (%) Mean (%) Diagnosis and Management of Chronic 94.44 94.44 87.50 92.59 79.17 86.11 89.04 Recommended Kidney Disease. A National Clinical Guideline: Scotland Guías SEN para el manejo de la Enfermedad 64.81 27.78 22.92 81.48 11.11 19.44 37.92 Not recommended Renal Crónica Avanzada y Pre-Diálisis: España Richtlijn voor de behandeling van patiënten 59.26 51.85 40.97 62.96 2.78 52.78 45.10 Not recommended met Chronische Nierschade (CNS): Netherlands Coronado Daza et al. BMC Nephrology Prevención Diagnóstico y Tratamiento de la 98.15 59.26 76.39 92.59 40.28 58.33 70.83 Recommended Enfermedad Renal Crónica Temprana: México with modifications Guía de Práctica Clínica sobre Prevención y 90.74 59.26 73.61 88.89 52.78 66.67 71.99 Recommended Detección Precoz de la Enfermedad Renal with modifications Crónica en Adultos en el Primer Nivel de Atención: Argentina (2019) 20:192 Guía Clínica Prevención de Enfermedad Renal 88.89 57.41 49.31 92.59 84.72 55.56 71.41 Recommended Crónica: Chile with modifications Management of chronic kidney disease in 94.44 62.96 81.25 94.44 87.50 72.22 82.14 Recommended adults: Malaysia Renal Association Clinical Practice Guideline 59.26 38.89 50.69 90.74 47.22 61.11 57.99 Recommended on Detection, Monitoring and Management with modifications of Patients with CKD: England Identificazione, prevenzione e gestione della 50.00 48.15 63.19 88.89 30.56 55.56 59.06 Recommended malattia renale cronica nell’adulto: Italy with modifications Guías Latinoamericanas de Practica Clínica 77.78 57.41 56.94 92.59 19.44 36.11 56.71 Recommended Sobre la Prevención Diagnostico y Tratamiento with modifications de los Estados 1–5 de la Enfermedad Renal Crónica 2012: Sociedad Latinoamericana de Nefrología e Hipertensión Arterial (SLANH) KHA-CARI Guidelines: Chronic Kidney 92.59 74.07 81.94 90.74 61.11 77.68 79.69 Recommended Disease: Australasia Screening, Monitoring, and Treatment of stage 88.89 59.26 82.64 96.30 33.33 94.44 75.81 Recommended 1 to 3 Chronic Kidney Disease: A Clinical Practice Guideline From the American College of Physicians: United States of America (USA) Chronic kidney Disease: United Kingdom 94.44 81.48 93.06 92.59 86.11 94.44 90.35 Recommended Mean (SD) 80.09 (17.12) 58.18 64.87 88.89 (8.97) 47.92 (30.39) 62.73 (22.3) (17.38) (21.19) Median (range) 88.89 (50–98.15) 59.26 73.61 92.59 45.83 61.11 (27.78–94.44) (62.96–96.30) (27.78–94.44) (2.78–87.50) (19.44–94.44) Page 6 of 12 Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 7 of 12 Fig. 2 AGREE II Domain scores of Guidelines according to recommendation and category vs. 61.7 ± 11.5 respectively; p = 0.002), and the editorial 31.9 ± 12.3 respectively; p = 0.02) and clarity of presen- independence domains (85.0 ± 10 vs. 55.6 ± 10.4 respect- tation domains (91.1 ± 1.8 vs. 72.2 ± 13.1 respectively; ively; p = 0.001). Between the recommended CPGs and p = 0.006). the CPGs that were not recommended there was a statisti- cally significant difference in all domains. A statistically CPGs assessment according to subgroups significant difference was found between the CPGs recom- The overall averages and the domain score of the in- mended with modifications compared to those not recom- cluded CGPs according to each subgroup are described mended in the rigor of development (61.7 ± 11.5 vs. as follows and resumed in Table 3. Table 3 AGREE II domain scores of Guidelines according to different subgroups Subgroup Scope and Stakeholder Rigor of Clarity and Applicability Editorial Overall Purpose Involvement Development Presentation Mean ± SD Independence Mean ± SD Mean ± SD Mean ± SD Mean ± SD Mean ± SD Mean ± SD Region Latinoamérica 88.89 ± 8.42 58.34 ± 1.07 64.06 ± 13.06 91.67 ± 1.85 49.31 ± 27.32 54.17 ± 12.93 67.74 ± 7.36 Europe 70.37 ± 19.24 57.10 ± 25.64 59.72 ± 27.14 84.88 ± 11.5 42.83 ± 34.57 61.57 ± 26.73 62.74 ± 22.13 Asia+Oceania 93.52 ± 1.31 68.52 ± 7.85 81.60 ± 0.49 92.59 ± 2.62 74.31 ± 18.66 74.95 ± 3.86 80.91 ± 1.73 Developers Medical societies 70.99 ± 15.75 46.30 ± 16.46 48.84 ± 30.63 80.25 ± 16.7 15.74 ± 15.8 55.56 ± 37.58 52.94 ± 20.12 Organization 84.03 ± 18.49 64.58 ± 18.05 72.92 ± 16.59 91.90 ± 1.7 64.58 ± 22.89 70.13 ± 14.9 74.69 ± 13.00 Medical societies and organization 84.26 ± 9.17 58.33 ± 1.31 65.28 ± 11.79 90.74 ± 2.62 36.11 ± 23.57 51.39 ± 21.61 64.35 ± 10.80 Self-assessment Yes 67.90 ± 23.45 56.17 ± 22.4 68.98 ± 21.77 90.74 ± 1.85 54.63 ± 28.51 70.37 ± 21.03 68.13 ± 19.27 No 85.00 ± 13.31 60.37 ± 16.71 65.35 ± 21.68 88.52 ± 9.83 47.22 ± 30.85 61.93 ± 22.67 68.07 ± 16.44 Year of publication 2008–2011 81.25 ± 16.98 56.48 ± 19.47 60.33 ± 22.67 87.04 ± 10.52 50.70 ± 32.32 59.03 ± 19.23 65.80 ± 17.60 2012–2016 80.74 ± 18.36 64.07 ± 13.46 75.55 ± 14.97 92.22 ± 2.75 46.11 ± 27.12 71.65 ± 25.48 71.72 ± 14.98 Recommendation Recommended 92.96 ± 2.41 74.44 ± 14.25 85.28 ± 4.99 93.33 ± 2.11 69.44 ± 22.76 84.98 ± 9.96 83.41 ± 6.19 Recommended with modifications 77.47 ± 19.08 53.40 ± 8.23 61.69 ± 11.46 91.05 ± 1.82 45.83 ± 22.46 55.56 ± 10.39 64.17 ± 7.97 Not recommended 62,04 ± 3.93 39,81 ± 17.02 31,94 ± 12.77 72,22 ± 13.09 6,94 ± 5.89 36,11 ± 23.57 41.51 ± 5.07 Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 8 of 12 CPGs assessment according to the region CPGs assessment according to the time period of Out of the six CPGs developed in Europe, two were rec- publication ommended (Scotland and United Kingdom), two recom- For the period between 2008 and 2011, eight CPGs mended with modifications (England and Italy), and two (61.5%) were published, out of which two were recom- were not recommended (Spain and Netherlands). The mended, four were recommended with modifications, four Latin American CPGs (Mexico, Argentina, Chile and two were not recommended. In the time frame be- and SLANH) were all recommended with modifications. tween 2012 and 2016, five CPGs (38.5%) were published, The CPGs from Asia (Malaysia), Oceania (Australasia) out of which three were recommended and two were and USA were all recommended. When comparing the recommended with modifications. domain scores of the CPGs from Asia and Oceania with The overall average score of the CPGs published in the Latin American CPGs, we found a statistically the period between 2008 and 2011 was 65.8 ± 17.6 and significant difference in the stakeholder involvement do- for the CPGs published in the period between 2012 and main (68.5 ± 7.9 vs. 58.3 ± 1 respectively; p = 0.04), how- 2016 it was 71.7 ± 15 (p = 0.55). In both periods, the do- ever, when comparing with the European CPGs, no mains with the lowest scores were stakeholder involve- difference was found (68.5 ± 7.9 vs. 57.1 ± 25.6 respect- ment and applicability with no significant statistical ively; p = 0.57). difference between them. In Latin American CPGs, the domains with the lowest scores were stakeholder involvement, applicability, and Synthesis of recommendations for chronic kidney disease editorial independence. Similarly, in European CPGs, the We identified general recommendations for CKD in the domains with the lowest scores were stakeholder in- included CPGs: early diagnosis, investigations or diag- volvement and applicability. When comparing Latin nostic test, interventions for slowing the progression and American and European CPGs, we found no significant criteria for referral to the nephrologist (Table 4). For the difference in the AGREE II domains. early diagnosis of CKD, all CPGs recommended to study CKD in patients with diabetes mellitus and systemic ar- CPGs assessment according to the types of development terial hypertension. Furthermore, the majority of the groups CPGs stated that patients with cardiovascular diseases, In total, three CPGs were developed by medical societies systemic diseases that affect the kidney and family his- (Spain, Netherlands and USA), two were developed jointly tory of end-stage kidney disease must be investigated. by medical societies and organizations that develop CPGs To evaluate the presence of CKD, all CPGs recom- (Argentina and SLANH) and the remaining eight CPGs mended estimating the glomerular filtration with the for- by organizations responsible for developing CPGs. mulas based in the serum creatinine and to measure the For the organizations that develop CPGs and medical albuminuria. The formula most used to estimate the societies, the domains with the lowest scores were glomerular filtration is MDRD (Modification of Diet in stakeholder involvement and applicability. When com- Renal Disease). paring these domains between the organizations that To avoid the progression of CKD, all CPGs recom- develop CPGs and medical societies, no statistically mended stabilizing the arterial pressure and using angio- significant difference was found in the stakeholder in- tensin converting enzyme inhibitors or angiotensin II volvement (64.6 ± 18.1 vs. 46.3 ± 16.5 respectively; p = receptors blockers. The majority of the CPGs recom- 0.16) domain, however, we found a difference in the mended controlling proteinuria, stabilize weight, avoid applicability domain (64.58 ± 22.9 vs. 15.74 ± 15.8 re- smoking, and controlling lipids. spectively; p = 0.009). Eleven (85%) included CPGs reported criteria to obtain a referral to the nephrology department, this concerns CPGs assessment according to the self-assessment during especially patients with a low estimated glomerular fil- the process of development tration and proteinuria or albuminuria. The majority of Three CPGs (23%) used a checklist during the develop- the CPGs (73%) recommend a referral to a nephrologist ment process to verify if they were fulfilling reporting re- when the estimated glomerular filtration is below 30 quirements (England, Italy and United Kingdom). The mL/min/1.73m2. overall score of the CPGs that self-assessed the fulfill- ment of the requirements was 68.1 ± 19.3 compared to Discussion the overall score of 68.1 ± 16.4 for the CPGs that did not The 13 CPGs focused on the early detection of the CKD self-asses their reporting (p = 0.10). In both groups, the and included in this study represent the current scenario domains with the lowest scores were stakeholder in- worldwide, since we included CPGs published between volvement and applicability with no significant statistical 2008 and 2016 in different languages, with a geograph- difference between them. ical variation over all continents. Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 9 of 12 Table 4 General guidelines recommendations for chronic Table 4 General guidelines recommendations for chronic kidney disease kidney disease (Continued) Number Percent Number Percent 2 Screening for Chronic Kidney Disease (CKD) < 45 mL/min/1.73m 1 9 Diabetes mellitus 13 100 Proteinuria or albuminuria 11 100 Hypertension 13 100 Glomerular hematuria 8 73 a Cardiovascular disease 11 85 Modification of Diet in Renal Disease b Chronic Kidney Disease Epidemiology Collaboration Family history of end-stage kidney disease 10 77 Multisystem diseases with potential kidney 8 62 Our review shows that the majority of CPGs focused involvement on the early diagnosis of the CKD are of good quality Structural renal tract disease 8 62 and developed by organizations that develop CPGs. The Chronic use of nephrotoxic drugs 6 46 recommended CPGs have a medium to high score in all Proteinuria 6 46 the domains. The CPGs recommended with modifica- Aged over 60 years 4 31 tions only have a medium to high score on scope and purpose, rigor of development and clarity of presenta- Isolated Hematuria 3 23 tion domains. However, the domains on the stakeholder Low socioeconomic status 3 23 involvement, applicability and editorial independence Cigarrete smoking 3 23 did not reach a score of medium category. Obesity 1 8 Investigations for CKD Comparison with the existing literature Creatinine-based equations for estimation 13 100 In general, all CPGs have their lowest averages in the renal function stakeholder involvement, applicability and editorial inde- a MDRD 11 85 pendence domains, which could partly explain why the incidence of the CKD hasn’t decreased [3]. In the review Cockcroft Gault 6 46 b by López-Vargas et al., regarding the CPGs published be- CKD-EPI 4 31 tween 2002 and 2011, similar results have been found Albuminuria 13 100 [6]. The domains with the lowest average were stake- Proteinuria 10 77 holder involvement, applicability, and editorial inde- Hematuria 5 39 pendence. The review of Sekercioglu et al., regarding Renal ultrasound 2 15 CPGs published between 2003 and 2015 that focused in the alterations in the bone mineral metabolism in CKD, I nterventions for slowing the rate of progression of CKD reported as well that the domains with the lowest Optimal blood pressure range 13 100 averages (all being in the category very low) were stake- holder involvement, applicability, and editorial inde- Use of angiotensin converting enzyme 13 100 inhibitors or angiotensin receptor blockers pendence [7]. Our results are similar to those found by Gagliardi and Brouwers; they analyzed systematic re- Optimal proteinuria reduction 12 92 views regarding different pathologies, which included Optimal weight 12 92 CPGs published since 2008, finding that the three do- Smoking cessation 12 92 mains with the lowest scores were stakeholder involve- Lipid lowering with statin therapy 12 92 ment, applicability, and editorial independence. Among Sodium restriction 10 77 the factors associated with the applicability, there was a Exercise 10 77 significantly higher average in the CPGs elaborated by groups that develop CPGs. Unlike our study, they found Optimal glycemic control 9 69 a significantly higher average in the CPGs that were Patient education 5 39 published between 2010 and 2012. In their conclusions, Moderate intake of alcohol 4 31 they state that the applicability of the CPGs has not in- Referral to the nephrologist creased in relation to the those published before 2008 Statement of criteria for referral 11 85 and that the cost to elaborate CPGs is not being Reduced glomerular filtration rate or 11 100 rewarded by their applicability [24]. creatinine clearance < 30 mL/min/1.73m2 8 73 Strategies for implementation the CPGs < 60 mL/min/1.73m 2 2 18 The findings of our review and those of others point to the same weakness in the current CPGs. Due to the Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 10 of 12 weaknesses in the implementation, the recommendations is useful for the development, report and evaluation ac- with the best evidence are not being used fully in the daily tivities related to the credibility, optimization, reliability practice. This, as we mentioned before, can be one of the and implementability of the CPGs recommendations. It determining factors in the increasing incidence of CKD, is expected to be ready for publication in the following despite the existing recommendations in the CPGs regard- years. ing prevention. To enhance the applicability of the CPGs, it has been recommended, among other strategies, to Implications for clinical practice endow clinics with instruments that facilitate the im- The general recommendation especially in low- and plementation of these CPGs. The idea is to simplify the middle-incomes countries, is the adaptation of high recommendations to be executed and to achieve the facili- quality CPGs to their context by using recommendations tation of the application and understanding of the patients based in the best evidence available and focused in the and the healthcare providers [25, 26]. In a recent study, all early diagnosis of the CKD [32]. Ours results allow the ideal characteristics for a CPGs’ implementation instru- endorsement of the CPGs developed in Scotland, ment were explored, and identified 12 items. Among these Malaysia, Australasia and the United Kingdom (NICE items, the following were highlighted: identification of tar- guideline) to support other developer groups to create get users; involvement of target users were in the tool de- their own CPGs or adapt them to their context. For this velopment; and conduction of pilot-test of the tool in last purpose, the CPG from the USA is not endorsed target users [27]. Kastner et al. found that the factors asso- due to a low and very low score on the stakeholder in- ciated with the implementation of the CPGs are the ones volvement and applicability domains, respectively. related to the creation of the content of the CPG and the We considered that the adapted CPGs from Mexico, proper communication of its content [28]. In relation to Argentina, Chile, Italy and SLANH have recommenda- the content, they consider the stakeholder involvement tions of utility in their context but they require an im- and the feasibility important among others, similarly to provement in the domains stakeholder involvement and our study. Another study was executed to develop a model applicability in future updates so they can truly contrib- based in the evidence for the implementation of the CPGs ute to make an impact over the incidence of the CKD. of clinical practice. Based on their results, the GUIDE-M (Guideline Implementability Decision Excellence Model) Strengths and weaknesses was created with the purpose of helping organizations As far as we know, our study represents the first analysis that developed CPGs to create CPGs with recommen- published on CPGs quality over the last 8 years world- dations easier to implement, facilitate the users’ adop- wide, with no restriction regarding language, focused on tion and to motivate researchers for deeper investigations the early diagnosis of the CKD including recommenda- in the topic [29]. tions based on the latest evidence. Among the strengths, On the other hand, the AGREE II instrument has we included a systematic search of the published CPGs, been used more as an instrument for assessing CPGs’ a high degree of agreement among reviewers, a great ex- quality despite the fact that it was also created to help pertise of methodological experts in the evaluation of achieve the requirements of the development and pres- CPGs using the AGREE II instrument. Additionally, two entation of the CPGs. In our study, we reported that of the reviewers are specialists in the area of nephrology. only 23.1% (3/13) of the CPGs, performed a process of We highlight as a strength the fact that the nephrologist self-assessment or compliance verification of the requi- reviewers work in different continents and are native sites needed for the development of the CPG, hence from different countries. this could have influence the low scores of the stake- Among the weaknesses, we mentioned that our inclu- holder involvement and applicability domains. In 2016 sion criteria only admitted potentially high quality CPGs, the AGREE Reporting Checklist was published, which which can be a selection bias. Although no quality seeks, among its objectives, to help the CPGs’ devel- threshold has been established in the AGREE II instru- opers take into account each of the quality require- ment, we accepted as satisfactory an average score of ments proposed in AGREE II [30]; with the use of this 60% or more in the domains, which may be argued by instrument, it would be possible in future CPGs to other authors. overcome the weaknesses found in our study and in the other ones aforementioned. Within the AGREE plat- Conclusions form, a research project is registered to create a new in- The majority of the CPGs focused on the early diagnosis strument that complements AGREE II, which has been of the CKD are recommended for their use in clinical named AGREE-REX (Appraisal of CPGs REsearch and practice. However, in clinical practice we notice an in- Evaluation – Recommendations EXcellence) [31]. The crease in incidence of CKD, which suggests that the purpose of this project is to create an instrument that recommendations probably are not being properly Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 11 of 12 applied. We found that the AGREE II domains with Received: 30 August 2017 Accepted: 15 May 2019 the lowest average in all CPGs are the stakeholder in- volvement and the applicability, which may be factors in- fluencing implementation. The cost to elaborate CPGs is References not being rewarded by their implementation, for this 1. Schoolwerth A, Engelgau M, Hostetter T. A public health action plan is needed for chronic kidney disease. Adv Chronic Kidney Dis. 2005;12:418–23. reason, the general recommendation especially in low- 2. Levey AS, Andreoli SP, DuBose T, Provenzano R, Collins AJ. CKD: common, and middle-incomes countries, is the adaptation of high harmful, and treatable—world kidney day 2007. Am J Kidney Dis. quality CPGs to their context. 2007;49:175–9. 3. United States Renal Data System. 2016 USRDS annual data report: Epidemiology of kidney disease in the United States. Bethesda: National Additional file Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases; 2016. https://www.usrds.org/2016/view. Accessed 19 Sept 2016 4. AGREE Collaboration. Development and validation of an international Additional file 1: Summary of the excluded guidelines and exclusion appraisal instrument for assessing the quality of clinical practice guidelines: criteria. We reported the titles of the guidelines that were excluded and the AGREE project. Qual Saf Health Care. 2003;12:18–23. the inclusion criteria they failed to meet. (DOCX 19 kb) 5. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in Abbreviations health care. CMAJ. 2010;182:E839–42. AGREE: Appraisal of Guidelines for Research & Evaluation; CKD: Chronic 6. López-Vargas P, Tong A, Sureshkumar P, Johnson D, Craig J. Prevention, kidney disease; CPG: Clinical practice guideline; NICE: National Institute for detection and management of early chronic kidney disease: a systematic Health and Care Excellence review of clinical practice guidelines. Nephrology. 2013;18:592–604. 7. Sekercioglu N, Al-Khalifah R, Ewusie J, Elias R, Thabane L, Busse J, et al. A critical appraisal of chronic kidney disease mineral and bone disorders Acknowledgements clinical practice guidelines using the AGREE II instrument. Int Urol Nephrol. The authors thank Pablo Alonso-Coello, Iberoamerican Cochrane Centre, for 2017;49:273–84. his support in the literature search and his thoughtful comments. 8. Alonso-Coello P, Irfan A, Sola I, Gich I, Delgado-Noguera M, Rigau D, et al. The quality of clinical practice guidelines over the last two decades: a Funding systematic review of guidelines appraisal studies. Qual Saf Health Care. No sources of support supplied. 2010;19:e58. https://doi.org/10.1136/qshc.2010.042077. 9. Arévalo-Rodríguez I, Pedraza O, Rodríguez A, Sánchez E, Gich I, Solá I, et al. Availability of data and materials Alzheimer’s disease dementia Guidelines for diagnostic testing: a systematic The datasets used and/or analyzed during the current study are available review. Am J Alzheimers Dis Other Dement. 2013;28:111–9. from the corresponding author on reasonable request. 10. Chen Y, Hu S, Wu L, Fang X, Xu W, Shen G. Clinical practice guidelines for hypertension in China: a systematic review of the methodological quality. Authors’ contributions BMJ Open. 2015. https://doi.org/10.1136/bmjopen-2015-008099. JC, GU conceptualized the study. JC, RV, KS, DO analyzed the data. JC, RV, 11. Diagnosis and Management of Chronic Kidney Disease: a National Clinical DO, GU contributed to interpretation and manuscript preparation. All authors Guideline. https://www.sign.ac.uk/archived-guidelines.html. Accessed 19 read and approved the final manuscript. May 2015. 12. Alcázar R, Egocheaga MI, Orte L, et al. Documento de consenso SEN- semFYC sobre la enfermedad renal crónica [SEN-SEMFYC consensus Authors’ information document on chronic kidney disease]. Nefrología. 2008;28:273–82 PMID: JC is a physician specialized in Internal Medicine and Nephrology, with a 18590493. Master degree in Clinical Epidemiology. He is a teacher in the Faculty of Medicine of the Universidad de Cartagena, Colombia and leader of the Alta 13. Rensma PL, Hagen EC, van Bommel EFH et al. Richtlijn voor de behandeling Tension research group; also, is a PhD candidate in Methodology of Biomedical van patiënten met Chronische Nierschade [directive for the treatment of Research, Universidad Autónoma de Barcelona, Spain. patients with chronic kidney damage]. http://docplayer.nl/629519-Richtlijn- voor-de-behandeling-van-patienten-met-chronische-nierschade-cns.html. Accessed 18 May 2015. Ethics approval and consent to participate 14. Guía de práctica clínica. Prevención, diagnóstico y tratamiento de la Not applicable. enfermedad renal crónica temprana [clinical practice guideline. Prevention, diagnosis and treatment of early chronic kidney disease]. http://www. Consent for publication cenetec.salud.gob.mx/descargas/gpc/CatalogoMaestro/335_IMSS_09_ Not applicable. Enfermedad_Renal_Cronica_Temprana/EyR_IMSS_335_09.pdf. Accessed 18 May 2015. Competing interests 15. Guía de práctica clínica sobre prevención y detección precoz de la The authors declare they have no competing interests. enfermedad renal crónica en adultos en el primer nivel de atención. [Clinical practice guideline on prevention and early detection of chronic kidney disease in adults in the first level of care]. https://www.argentina. Publisher’s Note gob.ar/salud/calidadatencionmedica/guiaspracticaclinica. Accessed 25 May Springer Nature remains neutral with regard to jurisdictional claims in 2015. published maps and institutional affiliations. 16. Guía clínica. Prevención de enfermedad renal crónica [Clinical Guide. Prevention of chronic kidney disease]. http://diprece.minsal.cl/le- Author details informamos/auge/acceso-guias-clinicas/. Accessed 25 May 2015. 1 Facultad de Medicina, Universidad de Cartagena, Cartagena 130014, 17. Clinical Practice Guidelines. Management of chronic kidney disease in Colombia. 2Nefrologia y Diálisis SAS, Cartagena 130001, Colombia. adults. Malaysia; 2011. http://www.msn.org.my. Accessed 20 June 2015 3 Iberoamerican Cochrane Centre, 08041 Barcelona, Spain. 4Biomedical 18. MacGregor MS, Taal MW. Renal association clinical practice guideline on Research Institute Sant Pau, 08025 Barcelona, Spain. 5Hospital de la Santa detection, monitoring and management of patients with CKD. Nephron Clin Creu i Sant Pau, 08041 Barcelona, Spain. 6University Hospital Vall d’Hebron, Pract. 2011;118(S1):c71–100. 08035 Barcelona, Spain. 7CIBER Epidemiología y Salud Pública (CIBERESP), 19. Identificazione, prevenzione e gestione della malattia renale crónica Barcelona, Spain. nell’adulto [Identification, prevention and management of chronic kidney Coronado Daza et al. BMC Nephrology (2019) 20:192 Page 12 of 12 disease in adults]. http://documenti.sinitaly.org/malattia-renale-cronica. Accessed 25 May 2015. 20. Guías Latinoamericanas de Practica Clínica Sobre la Prevención Diagnostico y Tratamiento de los Estados 1–5 de la Enfermedad Renal Crónica [Latin American Guidelines on Clinical Practice on the Prevention and Diagnosis of Chronic Kidney Disease 1-5]. http://www.slanh.net/?p=1471. Accessed 21 June 2015. 21. KHA-CARI Guidelines: Chronic Kidney Disease. http://www.cari.org.au. Accessed 25 May 2015. 22. ScreeningMonitoring, and treatment of stage 1 to 3 chronic kidney disease: a clinical practice guideline from the American College of Physicians. Ann Inter Med. 2013;159:835–47. https://doi.org/10.7326/0003-4819-159-12- 201312170-00726. 23. Chronic kidney disease in adults: assessment and management. https://www.nice.org.uk/guidance/cg182. Accessed 20 June 2015. 24. Gagliardi A, Brouwers M. Do guidelines offer implementation advice to target users? A systematic review of guideline applicability. BMJ Open. 2015. https://doi.org/10.1136/bmjopen-2014-007047. 25. Patel V, Arocha J, Diermeier M, Greenes R, Shortliffe E. Methods of cognitive analysis to support the design and evaluation of biomedical systems. J Biomed Inform. 2001;34:52–66. 26. Gagliardi A, Brouwers M, Palda V, Lemieux-Charles L, Grimshaw J. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26. 27. Gagliardi A, Brouwers M, Bhattacharyya O, et al. A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools. Implement Sci. 2014;9:98. 28. Kastner M, Bhattacharyya O, Hayden L, et al. Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review. J Clin Epidemiol. 2015;68(5):498–509. 29. Brouwers M, Makarski J, Kastner M, et al. The guideline Implementability decision Excellence model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field. Implement Sci. 2015;10:36. 30. Brouwers MC, Kerkvliet K, Spithoff K. AGREE next steps consortium. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ. 2016. https://doi.org/10.1136/bmj.i4852. 31. AGREE: Innovations to enhance the capacity of practice guidelines to improve health and health care systems: Recommendation EXcellence (AGREE-REX). https://www.agreetrust.org/resource-centre/agree-rex- recommendation-excellence/. Accessed 6 June 2016. 32. Jha V, Arici M, Collins AJ, et al. Understanding kidney care needs and implementation strategies in low- and middle-income countries: conclusions from a “kidney disease: improving global outcomes” (KDIGO) controversies conference. Kidney Int. 2016;90(6):1164–74.