Received: 30 January 2022 DOI: 10.1002/ueg2.12231 ORIGINAL ARTICLE - Accepted: 31 March 2022 AGREE‐S: AGREE II extension for surgical interventions – United European Gastroenterology and European Association for Endoscopic Surgery methodological guide Patricia Logullo1 | Ivan D Florez2,3 | George A Antoniou4,5 | Sheraz Markar6,7 | Manuel López‐Cano8 | Gianfranco Silecchia9 | Sofia Tsokani10 | 10,11 2 12,13 Dimitrios Mavridis | Melissa Brouwers | Stavros A Antoniou | The GAP Consortium 1 Department Nuffield of Orthopaedics, Rheumatology & Musculoskeletal Sciences, UK EQUATOR Centre, Centre for Statistics in Medicine, University of Oxford, Oxford, UK 2 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada 3 Department of Pediatrics, University of Antioquia, Medellin, Colombia 4 Department of Vascular and Endovascular Surgery, Manchester University NHS Foundation Trust, Manchester, UK 5 Division of Cardiovascular Sciences, School of Medical Sciences, The University of Manchester, Manchester, UK 6 Nuffield Department of Surgery, University of Oxford, Oxford, Oxfordshire, UK 7 Department of Molecular Medicine and Surgery, Karolinska Institute, Stockholm, Sweden 8 Abdominal Wall Surgery Unit, Val d’ Hebrón University Hospital, Universidad Autónoma de Barcelona, Barcelona, Spain 9 Department of Medico‐Surgical Sciences and Translation Medicine, Faculty of Medicine and Psychology, Sapienza University of Rome, Rome, Italy 10 Department of Primary Education, School of Education University of Ioannina, Ioannina, Greece 11 Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, Paris, France 12 Department of Surgery, Mediterranean Hospital of Cyprus, Limassol, Cyprus 13 European University Cyprus, Nicosia, Cyprus Correspondence Stavros A Antoniou, Department of Surgery, Abstract Mediterranean Hospital of Cyprus, Limassol, Background: The Appraisal of Guidelines for Research and Evaluation (AGREE) II 3117, Cyprus. Email: stavros.antoniou@hotmail.com instrument has been developed to inform the methodology, reporting and appraisal of clinical practice guidelines. Evidence suggests that the quality of surgical guide- Funding information lines can be improved, and the structure and content of AGREE II can be modified to United European Gastroenterology; European Association for Endoscopic Surgery and other help enhance the quality of guidelines of surgical interventions. Interventional Techniques Objective: To develop an extension of AGREE II specifically designed for guidelines of surgical interventions. Methods: In the tripartite Guideline Assessment Project (GAP) funded by United European Gastroenterology and the European Association for Endoscopic Surgery, (i) we assessed the quality of surgical guidelines and we identified factors associated with higher quality (GAP I); (ii) we applied correlation analysis, factor analysis and - This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial‐NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or adaptations are made. © 2022 The Authors. United European Gastroenterology Journal published by Wiley Periodicals LLC on behalf of United European Gastroenterology. United European Gastroenterol J. 2022;10:425–434. wileyonlinelibrary.com/journal/ueg2 425 426 - UNITED EUROPEAN GASTROENTEROLOGY JOURNAL the item response theory to inform an adaption of AGREE II for the purposes of surgical guidelines (GAP II); and (iii) we developed an AGREE II extension for sur- gical interventions, informed by the results of GAP I, GAP II, and a Delphi process of stakeholders, including representation from interventional and surgical disciplines; the Guideline International Network (GIN); the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group; the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) initiative; and rep- resentation of surgical journal editors and patient/public. Results: We developed AGREE‐S, an AGREE II extension for surgical interventions, which comprises 24 items organized in 6 domains; Scope and purpose, Stakeholders, Evidence synthesis, Development of recommendations, Editorial independence, and Implementation and update. The panel of stakeholders proposed 3 additional items: development of a guideline protocol, consideration of practice variability and sur- gical/interventional expertise in different settings, and specification of in- frastructures required to implement the recommendations. Three of the existing items were amended, 7 items were rearranged among the domains, and one item was removed. The domain Rigour of Development was divided into domains on Evidence Synthesis and Development of Recommendations. The new domain Development of Recommendations incorporates items from the original AGREE II domain Clarity of Presentation. Conclusion: AGREE‐S is an evidence‐based and stakeholder‐informed extension of the AGREE II instrument, that can be used as a guide for the development and adaption of guidelines on surgical interventions. KEYWORDS AGREE II, AGREE‐S, evidence, guidelines, methodology, quality INTRODUCTION Key summary The landscape of clinical practice guidelines has changed over the 1. AGREE II was developed to inform the methodology of past decades. The guideline ecosystem has shifted from a consensus‐ clinical practice guidelines. based approach, to a structured, transparent and evidence‐informed 2. Research suggests that adaption of AGREE II to be used development of recommendations for clinical practice and health in guidelines of surgical interventions is needed. policy.1–6 Major organizations and initiatives, such as the Grading of 3. The AGREE‐S methodological guide is an AGREE II Recommendations Assessment, Development and Evaluation extension for surgical interventions, and has been (GRADE), the Guidelines International Network (GIN), the Scottish developed through a systematic, structured, evidence‐ Intercollegiate Guidelines Network (SIGN), the National Institute for based and stakeholder‐informed consensus. Health and Care Excellence (NICE), and the World Health Organi- zation (WHO), among other, have fostered a culture of evidence‐ informed methodology in guideline development.7–12 The AGREE (Appraisal of Guidelines for Research and Evaluation) initiative has created a comprehensive framework for optimal devel- important proportion of surgical guidelines is poor.16 Development of 13 opment, reporting and appraisal of clinical practice guidelines. The an instrument to enhance methodological rigour and improve the AGREE instrument was developed in 2003 and updated as AGREE II in quality of surgical guidelines appears a reasonable scientific objective. 2010 by a multidisciplinary team of researchers and experts in health Surgery is a discipline with distinct characteristics in health care policy and guideline development.14 AGREE II is a generic instrument delivery and with unique features of its evidence ecosystem. Un‐like that can be applied across different disciplines in healthcare.15 non‐surgical and pharmacological guidelines, surgical experience and Empirical and research evidence suggests that the quality of an expertise play an essential role in evaluation of the evidence and in LOGULLO ET AL. - 427 the external validity of practice recommendations. Furthermore, EQUATOR) and quality guideline development and adaption meth- organizational and health system infrastructures affect the imple- odology (GRADE, GIN), and for diversity in geographic and ethnic mentation of guidelines on surgical interventions. New interventional backgrounds. techniques are mastered and frequently advocated by experts in the We asked Delphi panelists to nominate new items to be included field, whose input is often indispensable in the development of in the AGREE II extension. We thematically summarized their re- relevant guidelines; however, their contribution may be considered sponses and presented them in the first Delphi round, together with to be biased due to competing interests. These parameters are not the findings of GAP I and GAP II, asking candidate items to be specifically addressed in the AGREE II instrument. In addition, some included in, or excluded from, the AGREE II extension. We asked items of the AGREE II instrument might not be fully applicable to panelists to rate their importance on a 5‐point Likert scale, 1 indi- surgical guidelines in their original form. For instance, resource im- cating disagreement and 5 indicating agreement, and suggest a plications can rarely be considered within an evidence‐based context, modification of the proposed items if they did not fully agree with the due to the scarcity of cost‐effectiveness studies in the field of sur- content or the formulation. gery. Applicability considerations need to take into account not only We modified items according to panelists' responses and pre- the healthcare setting, but also the surgeons' experience and sented them in a second iterative round. Consensus was defined as expertise, which is underreported in surgical guidelines. Such con- agreement (score of 4 or 5 on the Likert scale) among at least siderations prompted an international, multidisciplinary group of 80% of panelists. If consensus was reached in the first or second stakeholders to develop an extension of AGREE to be used in the round, the item was considered eligible for inclusion, and if no development of surgical guidelines. consensus was reached after 2 rounds, the item was discarded. This article reports on the development of the AGREE‐S meth- After two rounds of Delphi, 5 new items and 1 modified AGREE II odological guide, an AGREE II extension for guidelines on surgical item were shortlisted for inclusion, and 4 items were shortlisted interventions. The AGREE‐S reporting checklist and the AGREE‐S for exclusion. appraisal tool will be published separately. The steering group discussed the findings of GAP I, GAP II and GAP III in an in‐person consensus meeting. The findings of the three parts of the project informed the development of the AGREE‐S – METHODS AGREE II extension for surgical interventions. The development of this project was supported financially by The project was named ‘Guideline Assessment Project (GAP): Filling United European Gastroenterology and the European Association the GAP in surgical guidelines’ and consisted of three parts, GAP I, for Endoscopic Surgery (EAES). The intellectual property lies GAP II and GAP III. The steering group developed a project protocol with the authors of this manuscript, the GAP Consortium, and in advance.17 EAES. In GAP I, we performed a systematic search to identify guidelines published by surgical organizations with an international scope. We assessed the quality of 67 guidelines using the original AGREE II AGREE‐S METHODOLOGICAL GUIDE appraisal instrument. We performed exploratory and post hoc ana- lyses to identify variables associated with higher quality. These var- Scope and purpose iables were (i) development by an organization regularly involved in guideline production (≥1 guideline per year), (ii) using the GRADE 1. The guideline is developed according to a protocol. – New item methodology, and (iii) development by an organization with a guidelines committee.16 A priori development of a protocol serves as a reference for the In GAP II, we performed correlation and reliability analyses, and guideline development (or adaption) group and it reduces bias in the applied the item response theory using the AGREE II scores obtained development process. The protocol does not need to be excessively from GAP I. The statistical models suggested that removing and detailed, but it should provide fundamental methodological principles rearranging some items across domains and reducing the number of (e.g., key questions, evidence search and synthesis approaches, domains to 4 or 5 may increase the reliability of AGREE II in the funding source, and panel members, among others). Deviations from context of surgical guidelines.18 the protocol may be necessary, but these must be justified and re- In GAP III, which is reported herein, we selected a panel of ported in the executive summary. The topic of the guideline is usually stakeholders to participate in a Delphi process to inform the devel- prespecified, however the selection of specific question frameworks opment of an AGREE II extension for surgical interventions. We may be informed by the panel.19 The steering group may prepare the aimed for representation of various groups of stakeholders, including protocol in collaboration with the evidence synthesis group and the guideline developers, guideline users, patient advocates, represen- guideline panel. It may also be made available through international tatives of organizations advocating transparency in research guidelines registries,20 the organizational website, the social media, (Enhancing the QUAlity and Transparency Of health Research – or other means, for public input. 428 - UNITED EUROPEAN GASTROENTEROLOGY JOURNAL 2. The guideline has (a) specific overall objective(s). The guideline panel should include all stakeholders affected by the content of the guideline. These may be other than surgical and The guideline should have a specific and clear objective that interventional specialists, such as primary care physicians, nurses, and drives its development (or adaption). Guidelines on surgical in- physiotherapists. For example, a guideline on robotic colorectal sur- terventions may aim to improve the outcomes and experience of gery might involve general and colorectal surgeons, oncologists, ra- patients with a specific surgical disease or condition, to improve the diotherapists, pathologists, nurses, healthcare economists, surgical diagnostic process, to enhance the prevention of disease and/or to technology specialists, and patient representatives. Patient repre- inform policymakers and healthcare authorities. sentatives or patient advocates should participate as ordinary panel members, with equal contribution and voting rights from the start of 3. There is/are specific health question(s) to be covered by the the guideline development process. The involvement of patient rep- guideline [patient, interventions/procedures, outcomes]. – Modi- resentatives might not always be feasible, for example, in a guideline fied item on orthopaedic surgery in the octogenarians. On such occasions, pa- tient advocacy groups or representatives of caregiver support groups Defining the clinical question(s) early in the process of guide- might need to be involved. Guideline development (or adaption) line development is of paramount importance. The clinical question groups should aim for at least two representatives from each stake- (s) should be structured within a question framework (e.g., PICO – holders' group. Particular care should be given to avoid the over‐ or patient, interventions/procedures, outcomes, for questions on in- underrepresentation of stakeholders, depending on the topic of the terventions; patient, index test, reference test, outcomes for guideline. Furthermore, the guideline development process should be questions on diagnostic tests). Framing the question avoids making supported from a diverse group of people, such as experts in evidence spurious assumptions and judgements when developing the rec- outreach (e.g., health information specialists), evidence synthesis (e.g., ommendations. Generic questions such as “Which is the best statisticians, methodologists), and health economists. Individual roles approach to treat patients with malignant biliary obstruction?” do and tasks should be specific and defined at the outset. not define the competing interventions and will inevitably lead to confusion among the evidence research group, panel members, and 6. There are specific target users of the guideline. – Modified item other involved stakeholders. A structured question, such as “Should percutaneous biliary drainage be preferred over endo- The guideline should be developed (or adapted) to be used by scopic stent placement for the management of patients with ma- specific stakeholders. These may be surgeons, physicians, nurses, lignant obstruction of the extrahepatic biliary system?” will allied healthcare professionals, policymakers, or the public. Different facilitate individual processes of evidence outreach, summary, considerations apply depending on the perspective of the guideline. appraisal and development of recommendations. Defining the For example, a guideline designed to inform policymakers has to outcomes is best made through panel consensus (including patient consider cost and implementation issues. In contrast, a guideline representatives or advocates), for example, by grading their developed to inform exclusively patient decision making might not importance21; therefore, outcome measures may not be available need to focus on such issues. Other considerations refer to the at the outset. appraisal of the certainty (quality) of the evidence. Guidelines developed to inform population‐based practices are best informed by intention to treat analyses (which are usually provided by random- Stakeholders ized trials on surgical interventions). In contrast, from the patient perspective, per‐protocol analyses are more relevant.22 4. The guideline is supported by a guideline development committee, including a guideline methodologist. – New item Evidence synthesis Experts in the field facilitate the process of guideline develop- ment (or adaption). A guideline methodologist will define key meth- 7. Systematic methods are used to search for evidence. odological parameters, will instruct the panel how to appraise the evidence and will coordinate the process, will accommodate the Guidelines should be informed by all available evidence on a discussion on how the evidence and other considerations will inform specific topic and avoid biased consideration of a fragment of evi- the recommendations (e.g., evidence to decision framework), and will dence. This makes a systematic review of the literature imperative. safeguard the quality of the guideline development process. Evidence The review is ideally performed de novo by an evidence review team. suggests that surgical guidelines produced by organizations with an This is usually a labour‐intensive process and requires time and ad hoc guidelines committee are of higher quality.16 financial resources, and methodological expertise. Furthermore, such process requires development of an appropriate search strategy, with 5. The guideline development group includes individuals from all careful selection of thesaurus and search terms, subject headings, relevant professional groups and patients. – Modified item truncated terms, search limits, and additional topic‐specific search LOGULLO ET AL. - 429 term combinations to capture the desired study design(s). Collabo- Development of recommendations ration among the evidence search group, the evidence synthesis group, the guideline panel, and the steering group is of particular 10. The views and preferences of the target population (patients, importance. public, etc.) are considered. Alternatively, the guideline development group may use one or more existing systematic reviews to summarize the evidence that will A guideline development group is expected to involve patients inform the development of recommendations. However, these must and/or the public in the guideline development or adaption process. be recent and of high quality (e.g., evaluated using the AMSTAR – A Patient representatives or patient advocates should be involved from MeaSurement Tool to Assess systematic Reviews 2 tool23), and they the outset, be regular members of the guideline panel, and have equal should provide evidence on the outcomes prioritized by the guideline opportunities to express their opinion and the same voting rights as panel. Another option is to update an existing systematic review, for the other panel members. Alternatively, patient/public views and example, through collaboration with the authors of the original re- preferences can be informed by available research, ad hoc surveys, or view. This will allow updating of their work, tailoring it to the needs interviews with focus groups. Patients' and public input is particularly of the guideline project, and extracting information of interest that important for selecting patient‐centred critical and important out- was not captured in the original review. comes, and for defining clinically meaningful differences.19,21,25 Guideline developers may find the AGREE‐REX (AGREE Recom- 8. There are specific criteria for selecting the evidence. mendation EXcellence) useful when formulating their recommenda- tions under consideration of patients'/public's views.26. As in any systematic review, a guideline needs to specify criteria for selecting the evidence. These refer to the study design or type of 11. There are specific methods for formulating the recommenda- evidence (e.g., randomized trials, observational studies, diagnostic tion(s). test accuracy studies, surveys, focus groups) and the question framework (patient/population, intervention or index diagnostic test Guideline developers (or adaptors) should predefine the meth- of interest, comparator or reference test). These criteria are ideally odology that will be used to formulate the recommendation(s). A specified at the protocol stage; however, the guideline development specific, predefined, transparent methodology to develop (or adapt) group might need to deviate from prespecified criteria and consider the recommendation(s) will avoid the common pitfall of arbitrary alternative sources depending on the available evidence. For decisions on the strength and the direction of the recommendation instance, a guideline focussing on the management of abdominal (s). For instance, GRADE suggests using an evidence to decision aortic aneurysms in patients over the age of 80 might need to be framework, which consists of the certainty of the evidence, the bal- indirectly informed from evidence on patients over the age of 65, ance between benefits and harms, acceptability, feasibility, equity, when appropriate evidence for the former group is not available. economic/resources considerations, and patients' views and prefer- Such considerations should be transparently documented and be ences.27 Agreement among most of these parameters in favour of an made available to guideline users. intervention, in the presence of at least moderate certainty evidence on critical outcomes, suggests that a strong recommendation may be 9. The strengths and limitations of the body of evidence are provided. Methods to reach consensus among panel members are considered. ideally defined a priori and can involve iterative discussions, Delphi process, or voting. The guideline should consider and document the strengths and limitations of the evidence that informed the recommendation(s). 12. The health benefits, side effects, and risks are considered in Limitations may be related to the risk of bias of individual studies, formulating the recommendations. indirectness of the evidence (available evidence not addressing the guideline‐specific population, intervention or diagnostic test, In the context of evidence appraisal, harms of an intervention are comparator, reference test, and/or outcome of interest), the certainty outcomes in which the comparator is better than the intervention. It of the comparative effect estimate (imprecision), and other issues, is of specific importance to consider both benefits and harms when such as heterogeneity and publication bias. Such considerations are developing recommendations. Failure to consider harms may result in summarized in the GRADE assessment of the certainty (quality) of the false confidence that an intervention is superior or inferior to the the evidence24 and can be presented in evidence tables. Guideline comparator. The balance between benefits and harms may need to developers (or adaptors) should explain how these limitations be informed by the magnitude of the effect of the benefits and affected the recommendation(s). For instance, GRADE suggests that the respective magnitude of harms, along with the importance of the overall certainty of evidence on a question framework be defined these outcomes. by the lowest certainty of evidence on critical outcomes. A strong recommendation can rarely be provided when the overall certainty is 13. There is an explicit link between the recommendations and the below moderate. supporting evidence. 430 - UNITED EUROPEAN GASTROENTEROLOGY JOURNAL Guidelines are informed by research and empirical evidence, and Notably, the guideline manuscript should not contain in the sup- through the input of stakeholders and experts in the field. Guideline porting text statements that may be considered recommendations. users should be provided with sufficient information to review what type of evidence informed the recommendation(s) and how. Such 17. The potential resource implications of applying the recommen- information can be summarized in evidence summaries, systematic dations are considered. observation forms to retrieve expert‐based evidence, and summaries of evidence on cost, patients' values and preferences, acceptability Resource considerations, including the cost of interventions, in- and feasibility. For example, the link between the evidence and rec- struments, hospitalization, reinterventions, infrastructures (e.g., the ommendations is summarized in evidence tables and the evidence to availability of a hybrid angiosuite for combined surgical and endo- decision framework tables within the context of GRADE.27 vascular interventions), and specific features of organizations or health services should be considered when developing the recom- 14. The recommendations are specific and unambiguous. mendation(s). This is particularly important for guideline developers, who wish to adapt existing guidelines to be used in specific settings The recommendation(s) should be unambiguous, concise, with defined resources. Every effort should be taken to identify actionable and clearly formulated. The wording of the recommen- relevant published evidence (e.g., economic evaluations, such as cost‐ dation(s) should be precise and reflect its/their strength (e.g., “We effectiveness analyses), or to perform such analyses during the recommend…” for strong recommendations and “We suggest…” for guideline development or adaption process. Suppose no such infor- weak/conditional recommendations). It should also indicate the di- mation is available and there are no sufficient resources to perform rection of the recommendation (e.g., “laparoscopic hysterectomy de novo analyses. In that case, the guideline development group may over open hysterectomy”). Lengthy recommendations with complex seek input from an expert in economic analyses, or summarize the language may result in misconceptions about the recommended panel's collective experience regarding the use of resources. High course of action and the strength of the recommendation. Such demands on resources (e.g., high cost, limited availability of an wording as “Perioperative thromboprophylaxis may be recom- interventional technology, or lack of wide‐scale expertise) may mended…“, “…could be suggested”, or “should be considered” produce prompt guideline developers to provide a weak recommendation or confusion among target users as to whether an intervention is even recommend against an intervention, even when it is more strongly recommended or is suggested as the best alternative in most effective than the comparator. situations. Furthermore, recommendations may need to specify the required setting, resources, or surgical/interventional expertise for 18. The guideline considers potential variability in surgical expertise their implementation; for example, “Robotic prostatectomy is rec- of those performing the interventions/procedures. – New item ommended over laparoscopic or open prostatectomy when a robotic platform is available, and surgical expertise is in place”. Surgical and interventional experience and expertise may affect the outcome of an intervention. Studies comprising the background 15. The different options for management of the condition or health evidence of a recommendation may report on the experience and issue are considered. expertise of those performing the interventions. The guideline should specify the experience and expertise required to perform Different interventional and non‐interventional options and al- the recommended interventions (e.g., previous courses, hands‐on ternatives for treatment, management, diagnosis, prevention or training, previous experience with a specific number of proced- screening might be appropriate for different patient populations or ures/interventions, training of operation room staff). If expertise in people at risk, and in different settings, regarding the availability of an intervention is not widely available in a given context or financial resources and infrastructures, surgical or interventional geographical region, the guideline panel may abstain from providing experience and expertise. Such options should be taken into a strong recommendation. Alternatively, they may issue a strong consideration when formulating the recommendations. recommendation in a given context, such as in the presence of interventional or surgical expertise, or after appropriate training. 16. Key recommendations are easily identifiable. Such considerations will also inform the decision of guideline adaption groups to issue a strong or weak recommendation in a Recommendations should be highlighted in the guideline publi- specific setting. cation, executive report, organizational website, etc. They must be clearly distinguished from supporting information. The authors may provide a list or table with the recommendations, and they may also Editorial independence be highlighted in the text, to facilitate identification. Algorithms or flow charts may be helpful in transferring a recommendation or a set 19. The views of the funding body have not influenced the content of recommendations to clinical practice and decision making. of the guideline. LOGULLO ET AL. - 431 The guideline should be independent of any type of financial, The guideline should specify its validity period, which may inform intellectual, or professional influence. The funding body (which in- the timing of its updating. This may be related to expected advances cludes for‐profit or non‐profit organizations) should not influence the in the field, identification of ongoing trials, publication of new content of the guideline, and its role should be transparent, specific research, or accumulation of experience with new technologies. The and defined in advance. guideline development group may need to perform a scoping search of clinical trial registries to identify ongoing trials, or they may map 20. Competing interests of guideline development group members the trend of new publications over a specific time period, which may are recorded and addressed. predict the publication of new evidence after a specific period of time. All individuals who contribute to or influence the content of the guideline should disclose any direct (financial) or indirect (in- 23. The guideline provides advice and/or tools on how the recom- tellectual) conflict of interest. The Guidelines International mendations can be implemented. Network recommends that the guideline chair and the guideline panel be free from either direct or indirect conflicts, and that The guideline development group may consider providing tools professionals with expertise but with conflict of interest related to and resources to facilitate application, such as guideline summary the interventions, diagnostic tests etc. discussed in the guideline documents, patient or public versions or lay summaries, treatment/ may act as consultants, but do not participate in the decision management algorithms, how‐to manuals, solutions linked to barrier process of the direction and the strength of the recommendation analysis, tools to capitalize on guideline facilitators, the outcome of 28 (s). A strong opinion favouring an intervention, course of action, the pilot test and lessons learnt, decision aids, or smartphone diagnostic test etc., may also constitute an indirect conflict of in- applications. terest. The guideline steering committee, the guideline chair(s) and/or the methodologist may need to reallocate functions and 24. The guideline presents monitoring and/or auditing criteria. responsibilities, or exclude (from parts or the whole guideline) contributors with relevant conflicts of interest. Assessment of the applicability, uptake and impact of the Conflicts of interest should be documented at least at the outset guideline should be among the objectives of guideline developers. and upon completion of the guideline development process. Stan- This can be achieved through a survey of target users a period of time dardized declaration forms of peer‐reviewed journals usually do not after publication, in order to appraise applicability, and to develop require declaration of indirect conflicts; therefore, a detailed state- and measure indicators (e.g., reduction in the incidence of surgical ment may need to be made available by different means; for example, site infection), related to key recommendations.29 This can be pro- in a supplementary file or file repository. vided in a supplementary file or other resources. A summary of the AGREE‐S methodological guide is provided in Table 1. Implementation and update 21. The guideline considers facilitators and barriers to its WHO SHOULD USE THIS DOCUMENT application. This document is intended to be used by: The guideline development (or adaption) group should identify any potential barriers to the implementation of the recommendation ‐ Guideline developers and adaptors, as a summary of critical (s), such as lack of wide‐scale experience and expertise, limited re- methodological principles. sources (e.g., financial, technologies or instruments), resistance to ‐ Guideline committees of organizations on surgical interventions, to change, organizational culture, limited awareness of the evidence, inform the process and methodology of guideline development. etc. Furthermore, the guideline development (or adaption) group may ‐ Educators, to teach core competencies in guideline development. identify potential facilitators, such as cost‐effectiveness, reduced requirement of resources, or ease of implementation. Both facilita- tors and barriers should be considered when developing or adapting HOW YOU SHOULD USE THIS DOCUMENT the guideline. They may also inform the decision on the strength of the recommendation(s), for example, through an evidence to decision This methodological guide does not intend to replace methodolo- 27 framework. Such information can be collected directly from gies for development of guidelines in healthcare, such as GRADE stakeholders (e.g., the guideline panel) or through pilot testing of the and SIGN 50,30,31 or guides detailing the development process, guideline before widespread implementation. such as the GIN‐McMaster Guideline Development Checklist.32 It is intended to inform the de novo development of guidelines on 22. There is a procedure for updating the guideline. surgical interventions about best practices, summarizing key 432 - UNITED EUROPEAN GASTROENTEROLOGY JOURNAL TABLE 1 Summary of the AGREE‐S methodological guide Scope and purpose 1. The guideline is developed according to a protocol. 2. The guideline has (a) specific overall objective(s). 3. There are specific health question(s) to be covered by the guideline [patient, in- terventions/procedures, outcomes]. Stakeholders 4. The guideline is supported by a guideline development committee, including a guideline methodologist. 5. The guideline development group includes individuals from all relevant professional groups and patients. 6. There are specific target users of the guideline. Evidence synthesis 7. Systematic methods are used to search for evidence. 8. There are specific criteria for selecting the evidence. 9. The strengths and limitations of the body of evidence are considered. Development of recommendations 10. The views and preferences of the target population (patients, public, etc.) are considered. 11. There are specific methods for formulating the recommendations. 12. The health benefits, side effects, and risks are considered in formulating the recommendations. 13. There is an explicit link between the recommendations and the supporting evidence. 14. The recommendations are specific and unambiguous. 15. The different options for management of the condition or health issue are considered. 16. Key recommendations are easily identifiable. 17. The potential resource implications of applying the recommendations are considered. 18. The guideline considers potential variability in surgical expertise of those performing the interventions/procedures. Editorial independence 19. The views of the funding body have not influenced the content of the guideline. 20. Competing interests of guideline development group members are recorded and addressed. Implementation and update 21. The guideline considers facilitators and barriers to its application. 22. There is a procedure for updating the guideline. 23. The guideline provides advice and/or tools on how the recommendations can be implemented. 24. The guideline presents monitoring and/or auditing criteria. methodological features and complementing existing resources. CONCLUSION The present document can also support the process of adaption of guidelines on surgical interventions, after appraisal and selection of The AGREE‐S methodological guide is intended to inform the process candidate guidelines using the AGREE‐S and the AGREE II of development and adaption of guidelines on surgical interventions. 14,33 appraisal instruments, and AGREE‐REX. Guidelines on surgical interventions with pharmacological or non‐surgical components A C KN O W L E D G EM E NT S (e.g., on perioperative antibiotic prophylaxis) may additionally need The Guideline Assessment Project (GAP) III received financial support to be informed by the original AGREE II guide. The AGREE‐S from the United European Gastroenterology (UEG) and the European methodological guide with supporting resources can also be Association for Endoscopic Surgery and Other Interventional Tech- accessed on the AGREE‐S website at https://agree‐s.org/ and the niques (EAES), both non‐profit organizations. The funders had no role AGREE Trust website at https://www.agreetrust.org/. in the design or development of this project. LOGULLO ET AL. - 433 C O N F LI C T O F I N T E R ES T Subcommittee. Surg Endosc. 2021;35(3):1238–46. Epub 2020 Apr 2. The authors declare no conflicts of interest related to this work. Ivan https://doi.org/10.1007/s00464‐020‐07494‐8 7. Djulbegovic B, Guyatt GH. Progress in evidence‐based medicine: a D Florez and Melissa Brouwers are leads of the AGREE Consortium. quarter century on. Lancet. 2017;390(10092):415–23. Epub 2017 Detailed conflicts of interest statements of the authors are provided Feb 17. https://doi.org/10.1016/S0140‐6736(16)31592‐6 in http://osf.io/fau4d. 8. Terracciano L, Brozek J, Compalati E, Schünemann H. GRADE sys- tem: new paradigm. Curr Opin Allergy Clin Immunol. 2010;10(4): 377–83. PMID: 20610980. https://doi.org/10.1097/ACI.0b013e E T H IC S A P P R O V A L S T A T E MEN T 32833c148b The NHS/HSC Research Ethics Committee evaluated the project and 9. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der ethics approval was waived. Wees P. Board of trustees of the guidelines international Network. Guidelines international Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7): P AT I E N T C O N SE N T S T A T EM EN T 525–31. PMID: 22473437. https://doi.org/10.7326/0003‐4819‐ Not applicable. 156‐7‐201204030‐00009 10. Lowe G, Twaddle S. The scottish intercollegiate guidelines Network P E R M I SS IO N T O R EP R O D U C E M A T ER I AL F R O M O T H E R (SIGN): an update. Scot Med J. 2005;50(2):51–2. https://doi.org/10. SOURCES 1177/003693300505000202 11. National Institute for Health and Care Exce. llence (NICE). Devel- The authors obtained permission from the AGREE Consortium to oping NICE guidelines: the manual Guidance NICE. 2022. [online]. reproduce parts of the AGREE II User's Manual and 23‐item https://www.nice.org.uk/process/pmg20/chapter/introduction. Acce Instrument. ssed 27 January 2022. 12. Norris SL, Ford N. Improving the quality of WHO guidelines over the last decade: progress and challenges. Lancet Global Health. 2017;5(9): D A TA A V AIL AB IL I T Y S T A T EM E N T e855–6. PMID: 28807174. https://doi.org/10.1016/S2214‐109X(17) The data that support the findings of this study are available from the 30253‐X corresponding author upon reasonable request. 13. Makarski J, Brouwers MC, Agree Enterprise. The AGREE Enterprise: a decade of advancing clinical practice guidelines. Implement Sci. 2014;9:103. PMID: 25123781; PMCID: PMC4148001. https://doi. O R C ID org/10.1186/s13012‐014‐0103‐2 Ivan D Florez https://orcid.org/0000-0002-0751-8932 14. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Sofia Tsokani https://orcid.org/0000-0002-6258-8861 et al. AGREE Next Steps Consortium. AGREE II: advancing guideline Stavros A Antoniou https://orcid.org/0000-0002-4630-6748 development, reporting and evaluation in health care. J Clin Epidemiol. 2010;63(12):1308–11. Epub 2010 Jul 24. PMID: 20656455. https:// doi.org/10.1016/j.jclinepi.2010.07.001 R E FE R EN C E S 15. AGREE Next Steps Consortium. The AGREE II instrument [electronic 1. Zhang R, Yan SY, Wang YY, Huang Q, Ren XY, Tan R, et al. Analysis version]. 2009. Retrieved from http://www.agreetrust.org 20 of the status and trends of Chinese clinical practice guideline December 2021. development between 2010 and 2020: a systematic review. Front 16. Antoniou SA, Tsokani S, Mavridis D, López‐Cano M, Antoniou GA, Med. 2021;8:758617. https://doi.org/10.3389/fmed.2021.758617 Stefanidis D, et al. Guideline assessment project: filling the GAP in 2. Zhang S, Wu L, Wang Y, Zhou Q, Luo X, Mathew JL, et al. Meth- surgical guidelines: quality improvement initiative by an interna- odological and reporting quality of pediatric clinical practice guide- tional working group. Ann Surg. 2019;269(4):642–51. PMID: lines: a systematic review. Ann Transl Med. 2021;9(15):1258. 30188402. https://doi.org/10.1097/SLA.0000000000003036 https://doi.org/10.21037/atm‐21‐2686 17. Antoniou GA, Mavridis D, Tsokani S, López‐Cano M, Flórez ID, 3. Siedler MR, Lamadrid P, Humphries MN, Mustafa RA, Falck‐Ytter Brouwers M, et al. Protocol of an interdisciplinary consensus project Y, Dahm P, et al. The quality of physical activity guidelines, but aiming to develop an AGREE II extension for guidelines in surgery. not the specificity of their recommendations, has improved over BMJ Open. 2020;10(8):e037107. PMID: 32784259; PMCID: time: a systematic review and critical appraisal. Appl Physiol Nutr PMC7418673. https://doi.org/10.1136/bmjopen‐2020‐037107 Metabol. 2021;46(1):34–45. Epub 2020 Sep 29. https://doi.org/10. 18. Tsokani S, Antoniou SA, Moustaki I, López‐Cano M, Antoniou GA, 1139/apnm‐2020‐0378 Flórez ID, et al. Guideline Assessment Project II: statistical calibra- 4. Barker TH, Dias M, Stern C, Porritt K, Wiechula R, Aromataris E, tion informed the development of an AGREE II extension for surgical et al. Guidelines rarely used GRADE and applied methods inconsis- guidelines. Surg Endosc. 2021;35(8):4061–8. Epub 2021 Jun 22. tently: a methodological study of Australian guidelines. J Clin Epi- PMID: 34159464. https://doi.org/10.1007/s00464‐021‐08604‐w demiol. 2021;130:125–34. Epub 2020 Oct 29. https://doi.org/10. 19. Wiercioch W, Nieuwlaat R, Zhang Y, Alonso‐Coello P, Dahm P, Iorio 1016/j.jclinepi.2020.10.017 A, et al. New methods facilitated the process of prioritizing ques- 5. Oxman AD, Schünemann HJ, Fretheim A, Boyd EA, Bero LA. tions and health outcomes in guideline development. J Clin Epi- Tan‐torres edejer T. Improving the use of research evidence in demiol. 2021;143:91–104. Epub ahead of print. PMID: 34843861. guideline development. Knowledge centre for the health services at https://doi.org/10.1016/j.jclinepi.2021.11.031 the Norwegian Institute of public health (NIPH), report from Nor- 20. Chen Y, Guyatt GH, Munn Z, Florez ID, Marušić A, Norris SL, et al. wegian knowledge centre for the health services (NOKC); 11 2007. Clinical practice guidelines registry: toward reducing duplication, 6. Antoniou SA, Tsokani S, Mavridis D, Agresta F, López‐Cano M, improving collaboration, and increasing transparency. Ann Intern Muysoms FE, et al. EAES Consensus, Guideline Subcommittee. Med. 2021;174(5):705–7. https://doi.org/10.7326/m20‐7884 Insight into the methodology and uptake of EAES guidelines: a 21. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. qualitative analysis and survey by the EAES Consensus & Guideline GRADE guidelines: 2. Framing the question and deciding on 434 - UNITED EUROPEAN GASTROENTEROLOGY JOURNAL important outcomes. J Clin Epidemiol. 2011;64(4):395–400. Epub 28. Schünemann HJ, Al‐Ansary LA, Forland F, Kersten S, Komulainen J, 2010 Dec 30. PMID: 21194891. https://doi.org/10.1016/j.jclinepi. Kopp IB, et al. Qaseem A; board of trustees of the guidelines in- 2010.09.012 ternational Network. Guidelines international Network: principles 22. Higgins JPT, Savović J, Page MJ, Sterne JAC. On behalf of the RoB2 for disclosure of interests and management of conflicts in guidelines. Development Group. Revised Cochrane risk‐of‐bias tool for ran- Ann Intern Med. 2015;163(7):548–53. PMID: 26436619. https://doi. domized trials (RoB 2). https://www.riskofbias.info/welcome/rob‐2‐ org/10.7326/M14‐1885 0‐tool/current‐version‐of‐rob‐2. Accessed 23 January 2022. 29. Nothacker M, Stokes T, Shaw B, Lindsay P, Sipilä R, Follmann M, et al. 23. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. Guidelines international Network (G‐I‐N) performance measures AMSTAR 2: a critical appraisal tool for systematic reviews that working group. Reporting standards for guideline‐based performance include randomised or non‐randomised studies of healthcare in- measures. Implement Sci. 2016;11:6. PMID: 26772173; PMCID: terventions, or both. BMJ. 2017;358:j4008. PMID: 28935701; PMC4714427. https://doi.org/10.1186/s13012‐015‐0369‐z PMCID: PMC5833365. https://doi.org/10.1136/bmj.j4008 30. Schünemann H, Brożek J, Guyatt G, Oxman A. Handbook for grading 24. Guyatt G, Gutterman D, Baumann MH, Addrizzo‐Harris D, Hylek the quality of evidence and the strength of recommendations using EM, Phillips B, et al. Grading strength of recommendations and the GRADE approach; 2013. Available in: https://gdt.gradepro.org/ quality of evidence in clinical guidelines: report from an American app/handbook/handbook.html. Accessed 23 January 2022. College of Chest Physicians Task Force. Chest. 2006;129(1):174–81. 31. SIGN 50: a guideline developer's handbook (PDF). 2019. https://www. PMID: 16424429. https://doi.org/10.1378/chest.129.1.174 sign.ac.uk/media/1050/sign50_2019. Accessed 20 October 2021 25. Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, et al. 32. GIN‐McMaster guideline development checklist. 2022. https:// The GRADE Working Group clarifies the construct of certainty of cebgrade.mcmaster.ca/guidecheck.html. Accessed 23 January 2022. evidence. J Clin Epidemiol. 2017;87:4–13. Epub 2017 May 18. 33. The AGREE‐S Consortium. AGREE‐S appraisal instrument. 2022. PMID: 28529184; PMCID: PMC6542664. https://doi.org/10.1016/j. Accessed 31 January 2022. jclinepi.2017.05.006 26. Brouwers MC, Spithoff K, Kerkvliet K, Alonso‐Coello P, Burgers J, Cluzeau F, et al. Development and validation of a tool to assess the quality of clinical practice guideline recommendations. JAMA Netw How to cite this article: Logullo P, Florez ID, Antoniou GA, Open. 2020;3(5):e205535. 2020.5535. PMID: 32459354; PMCID: PMC7254179. https://doi.org/10.1001/jamanetworkopen Markar S, López‐Cano M, Silecchia G, et al. AGREE‐S: AGREE II 27. Alonso‐Coello P, Oxman AD, Moberg J, Brignardello‐Petersen R, Akl extension for surgical interventions – United European EA, Davoli M, et al. GRADE Working Group. GRADE Evidence to Gastroenterology and European Association for Endoscopic Decision (EtD) frameworks: a systematic and transparent approach Surgery methodological guide. United European Gastroenterol to making well informed healthcare choices. 2: clinical practice guidelines. BMJ. 2016;353:i2089. PMID: 27365494. https://doi.org/ J. 2022;10(4):425–34. https://doi.org/10.1002/ueg2.12231 10.1136/bmj.i2089