Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 https://doi.org/10.1186/s12874-020-01004-z RESEARCH ARTICLE Open Access Current methods for development of rapid reviews about diagnostic tests: an international survey Ingrid Arevalo-Rodriguez1* , Karen R. Steingart2, Andrea C. Tricco3,4,5, Barbara Nussbaumer-Streit6, David Kaunelis7, Pablo Alonso-Coello8,9, Susan Baxter10, Patrick M. Bossuyt11, José Ignacio Emparanza12 and Javier Zamora1,13 Abstract Background: Rapid reviews (RRs) have emerged as an efficient alternative to time-consuming systematic reviews— they can help meet the demand for accelerated evidence synthesis to inform decision-making in healthcare. The synthesis of diagnostic evidence has important methodological challenges. Here, we performed an international survey to identify the current practice of producing RRs for diagnostic tests. Methods: We developed and administered an online survey inviting institutions that perform RRs of diagnostic tests from all over the world. Results: All participants (N = 25) reported the implementation of one or more methods to define the scope of the RR; however, only one strategy (defining a structured question) was used by ≥90% of participants. All participants used at least one methodological shortcut including the use of a previous review as a starting point (92%) and the use of limits on the search (96%). Parallelization and automation of review tasks were not extensively used (48 and 20%, respectively). Conclusion: Our survey indicates a greater use of shortcuts and limits for conducting diagnostic test RRs versus the results of a recent scoping review analyzing published RRs. Several shortcuts are used without knowing how their implementation affects the results of the evidence synthesis in the setting of diagnostic test reviews. Thus, a structured evaluation of the challenges and implications of the adoption of these RR methods is warranted. Keywords: Rapid reviews, Tests, Diagnosis, Knowledge synthesis, Decision-making Background susceptible to be used to provide timely evidence include Rapid reviews (RRs) have emerged as an efficient alter- the identification of challenges to disease surveillance in native to resource-intensive systematic reviews (SRs). the context of the crisis in Syria, the impact of e-health RRs can speed up evidence synthesis by implementing for rural residents in Australia and the adequate man- methods and strategies to streamline the review process agement of new emergent diseases as the COVID-19 dis- [1–4]. RRs can inform best practices for a diverse variety ease [8–10]. of clinical and public health topics requiring a quick The methods for performing systematic reviews are turnaround [5–7]. Examples of topics where RRs are now well established for the field of medical test accur- acy as with other areas of healthcare [11]. For diagnostic accuracy reviews, key characteristics include clearly- * Correspondence: inarev7@yahoo.com; ingrid.arevalo@salud.madrid.org 1 Clinical Biostatistics Unit, Hospital Universitario Ramón y Cajal, IRYCIS, CIBER stated objectives and eligibility criteria; a systematic lit- of Epidemiology and Public Health, Madrid, Spain erature search; an assessment of methodological quality; Full list of author information is available at the end of the article © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 Page 2 of 7 and a systematic synthesis and presentation of the find- defined RR as “a knowledge synthesis strategy using lim- ings from the included studies [11–14]. In response to ited or accelerated methods to expedite the time required the demand for accelerated evidence syntheses to inform to obtain a conclusive answer” [6]. In addition, we defined clinical decisions and policy, efforts have been made to a diagnostic test as “any method for collecting additional standardize the methods and strategies for carrying out information about the current or future health status of a RRs while often extrapolating from effectiveness and patient” [23]. safety RRs [4, 6, 7, 15, 16]. The RRs of diagnostic evi- The questions in our survey focused on the method- dence however, present particular challenges given the ology that responders use to conduct RRs, and drew on fundamental differences between the methods used to those elements that we previously identified in our scop- summarize the evidence for interventions and those for ing review [5, 20] including methods to: diagnostic evidence. For instance, in contrast to SRs of interventions, SRs of diagnostic accuracy identify eligible a. limit the scope of the review question (narrow the studies from electronic search strategies that often in- scope) volve screening thousands of titles and abstracts. The b. focus on methodological tailoring of review steps use of methodological filters can limit the volume of ci- according to the needs of decision-makers (review tations retrieved and is strongly discouraged [17, 18]. shortcuts) Moreover, in diagnostic accuracy SRs, the synthesis of c. increase the intensity of the work on review evidence requires statistical knowledge to fit the com- processes (parallelization of tasks) plex statistical models needed for conducting meta- d. use new technologies to fast-track selected review analyses [11, 12, 19]. tasks (automation) In a previous scoping review, we examined the charac- teristics of RRs of diagnostic tests by scrutinizing reposi- To gauge the level of experience and skills of the team, tories of Health Technology Assessment (HTA) agencies we included questions about the number of RRs of diag- and papers published in indexed journals [20]. We found nostic tests previously conducted, the structure of the 191 RRs developed by international agencies since review team, and strategies for completing and publish- 2013—there was a clear increase better and more rapid ing the final report. synthesis of diagnostic evidence. We also observed that We developed several drafts of the questionnaire prior most RRs were broad in scope and assessed multiple to finalizing a pilot version. The pilot was tested by five index tests, outcomes, and test applications. We further researchers external to the research team who were found that well-known methodological tailoring strat- asked to assess the usability and technical functionality egies such as setting limits for literature searching by of the survey. After this revision, the final version of the date, language, or number of databases were rarely re- questionnaire consisted of 10 items: Six were multiple- ported. Due to an incomplete description of the methods choice questions, and four were open answers. A copy of used in the RRs, as well as inclusion of only published the survey is available as a supplementary file. reports, we could not provide a detailed account of the current practice for the development of RRs of diagnos- Recruitment process and description of the sample tic tests [20]. To better understand how RR methods are On April 2019, we invited representatives from institu- currently used to synthesize diagnostic evidence, we per- tions around the world who perform evidence syntheses formed an international survey to identify methodo- to participate in this closed survey. The invited represen- logical practices used in the development of RRs for tatives were from institutions belonging to the Inter- diagnostic tests. national Network of Agencies for Health Technology Assessment (INAHTA), the World Health Organization Methods (WHO) collaborating centers on Health Technology As- We developed and administered an online questionnaire sessment (HTA), the Health Technology Assessment seeking information about the methods and resources Network of the Americas (REDETSA), and the Health involved in the performance of RRs of diagnostic tests. Technology Assessment International Network (HTAi, We published a protocol summarizing the methods used non-profit members). to conduct this survey [21]. We followed the Checklist All initial contacts were made by the principal investi- for Reporting Results of Internet E-Surveys (CHERRIES) gator; participants were invited via an email message guidance to report the findings of our research [22]. that included a personal invitation letter and a copy of the research protocol [21]. In addition, we provided a Design of the survey participant information sheet with details regarding the We developed and administered our questionnaire using purpose of the study, reasons for their invitation, proce- SurveyMonkey software (https://surveymonkey.com/). We dures involved in participation, as well as privacy and Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 Page 3 of 7 confidentiality processes. Participating institutions were they performed RRs of diagnostic tests. They formed the asked to nominate a member of the team with experi- final sample for our following analyses. The characteris- ence in developing RRs to complete the survey. tics of participants (n = 25) are shown in Table 1. Four reminders were sent by email to non-responders The median number of RRs of diagnostic tests com- every 3 weeks until we closed the survey in July 2019. pleted per institution at the time of the survey was 10 Participation in the survey was voluntary, and partici- (Interquartile range from 5 to 18). Three institutions pants could leave the study at any time. No monetary in- performed more than 100 RRs. Nine institutions indi- centives were offered. cated the use of a handbook or guideline to develop RRs (36%). Twenty-two institutions reported that these RRs Survey administration are developed within a constrained time schedule (88%). After receiving confirmation of participation in the re- In addition, the involvement of highly trained staff was a search project, we sent an email containing the anon- frequent element of RR conduct at participating institu- ymized link to access the online questionnaire. tions (88%). However, less than half of the institutions The questions in our survey were not randomized or involved more than two reviewers or more than one alternated. Only the first question of the survey (Does team (i.e. to address different types of evidence, such as your agency conduct rapid reviews (RRs) of diagnostic health outcomes and cost-effectiveness) during the per- tests?) was mandatory; a negative answer to this question formance of RRs (10 agencies; 40%). Ten agencies re- terminated the survey because the responder was deemed ported active participation of stakeholders during the not to have the experience needed to answer the remaining development of RRs (40%; Table 1). questions. With regards to the general methods involved in the Participants could review and modify their answers be- performance of RRs, 16 institutions developed and fore submission using the back button. All responses followed a protocol (64%). Most institutions reported a re- were entered via the internet and automatically cap- vision of the RR findings by an external and/or internal tured. Multiple entries from the same IP address were peer-review process (24 institutions; 96%). Only eleven in- not allowed while the survey was open. stitutions considered publishing the final RR (Table 1). Data analysis and ethical issues Table 1 Characteristics of participating institutions This study was exempt from requiring approval by our Country N (%) Ethics Committee in accordance with Spanish National Africa 1 (4) Regulations. A positive response to the invitation email America 9 (36) was considered to be an agreement/informed consent to participate in the study. Asia 3 (12) Survey responses were de-identified and anonymized Europe 11 (44) for all analyses. Duplicate entries were eliminated before Oceania 1 (4) analysis; only the final entry was kept for further ana- Number of RRs of diagnostic tests developed lyses. Incomplete surveys were not included in the final Less than 10 RR 10 (40) analysis. We did not use the time spent to complete the 10 to 30 RR 9 (36) survey as a criterion to exclude answers (questionnaires submitted with an atypical timestamp). We did not rely More than 30 RR 6 (24) on statistical methods for assessing representativeness of Availability of RR methodological guidance (i.e. handbook) the final sample. We performed all descriptive analysis Yes 9 (36) using STATA 15.0. Structure of RRs team High level of training 22 (88) Results Involvement of more than two reviewers or more than 10 (40) Data were collected from April to July 2019. A total of one team 74 institutions were contacted by email, and we received Stakeholder involvement in several activities 10 (40) 39 replies (53% response rate). The responders were lo- cated in Europe (20 agencies; 52%), America (11 agen- General methods of RRs cies; 28%), Asia (4 agencies; 10%), Africa and Australia Development of a protocol 16 (64) (2 agencies each one; 5% respectively). All but one of the Use of a reporting template 12 (48) institutions that had initially agreed to participate com- Peer-review process 24 (96) pleted the survey (97% participation rate). The average Public consultation of the draft review 4 (33) time to complete the survey was around 11 min. Publication of the final review 11 (44) Twenty-five (64%) of the 39 participants indicated that Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 Page 4 of 7 Methods used to establish the scope of the review Table 2 Rapid review methods reported by the survey participants All participants reported the implementation of one or Method N (%) more methods to pre-define the scope of a RR during Narrow the scope the planning stage (range from 1 to 7 methods). A con- Defining a structured PICO question 23 (92) siderable number of participants reported implementing a Discussing the clinical pathway for the target condition 17 (68) actions related to planning such as defining a structured PICO question (92%) and discussing the clinical pathway Limiting the population b 17 (68) c and the target condition with stakeholders (i.e., the role Limiting the number of index tests 19 (76) of the test in current practice and the intended applica- Limiting the number of comparisons d 14 (56) tions) (68%). Only eleven participants reported limiting Limiting the number of outcomes e 11 (44) the number of outcomes (44%). All but one agency in- Limiting the number of applications of the tests f 20 (80) cluded accuracy as one of the main outcomes assessed Review shortcuts in the development of RR of diagnostic tests (96%). Only one strategy was used by more than 90% of participants: Using a previous review as a starting point g 23 (92) defining a structured PICO question. One participant re- Limiting search strategies to one database 2 (8) ported an additional method to narrow the scope of the Limiting search strategies by language 21 (84) RR: limiting the reference standards accepted for the tar- Limiting search strategies by date 17 (68) get condition (Table 3). Limiting the syntax of search strategies h 8 (32) Limiting search strategies results using methodological filters 14 (56) Methodological shortcuts All participants reported the use of one or more methodo- Excluding additional searches i 22 (88) logical shortcuts when developing a RR of a diagnostic test Limiting screening of titles & abstracts: one reviewer only 12 (48) (range from 4 to 13 methods) with six institutions report- Limiting the selection of full texts: one reviewer only 15 (60) ing the use of up to five different shortcuts. The most Limiting the data abstraction: one reviewer only 16 (64) common strategy was the use of a previous review as a Limiting the quality appraisal: one reviewer only 10 (40) starting point (e.g., to update the findings or to replicate Performing a narrative synthesis of findings j 19 (76) the electronic searches; 92%) followed by limiting search strategies by language (e.g., to English only; 84%), and ex- Excluding a GRADE assessment of findings 15 (60) cluding additional sources of evidence such as conference Parallelization of tasks proceedings (88%). Twenty-four participants reported Multiple reviewers completing the eligibility screening 8 (32) using two or more limits when conducting search strat- Multiple reviewers completing the data abstraction 7 (28) egies (96%). In addition, nine agencies reported perform- Multiple reviewers completing the quality appraisal 8 (32) ing the screening, final selection, data abstraction, and Automation quality appraisal with only one reviewer (36%) (Table 2). An important number of participants did not perform Used to assist in the screening/selection of references 3 (12) a meta-analysis in RRs (76%) nor assessed the certainty Used to assist in the data abstraction 2 (8) of evidence (60%). Participants suggested two additional Used to assist in the quality appraisal 1 (4) methodological shortcuts that were not included in our a Notes: including the role of the test in the current clinical practice, its survey: partial verification of the screening, final selec- intended application, and prior/alternative tests; b ideally to one single population; c ideally to one single test; d ideally to one single comparison; e tion, data extraction, and quality assessment by a second ideally to one single outcome; f ideally to one single application: i.e. reviewer; and the choice of pre-existing evidence synthe- monitoring, screening, diagnosis; g i.e. stepwise approach with an emphasis on higher levels of evidence, update of existing SR, re-run search strategies; h, e.g. sis as the only eligible source of evidence (i.e. no primary focused subject headings, terms in title only; i, e.g. conference abstracts; studies are included) (Table 3). search on the internet; j instead of a meta-analysis of data Parallelization of tasks participant reported parallelization of review activities by Twelve participants reported using one or more strat- performing selected activities simultaneously instead of egies to parallelize review tasks (48%) while three of consecutively (such as data extraction of known studies these participants used all parallelization methods de- while a search of new studies is conducted) (Table 3). scribed in the survey. The two most frequently used strategies were the involvement of several reviewers in Automation of review tasks the screening of citations and quality appraisal (eight Algorithms and machine learning techniques are agencies; 32%) (Table 2). One participant reported that methods rarely used by participating institutions. Only parallelization could also be used to assess the certainty five participants reported the use of one or more of of evidence using the GRADE approach. Another these techniques to perform RR tasks. Three agencies Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 Page 5 of 7 Table 3 Additional rapid review methods reported by the survey participants New methodologies Comments Establishing the scope Limiting the accepted reference standards Ideally to a single reference standard Limitations and shortcuts Selecting pre-existing synthesis of evidence only e.g., systematic reviews, HTA reports Screening of titles & abstracts: selected verification Two reviewers involved, checking a sample of/all references for accuracy Selection of full texts performed: selected verification Two reviewers involved, checking a sample of/all references for accuracy Data abstraction: selected verification Two reviewers involved, checking the sample/all references for accuracy Quality appraisal: selected verification Two reviewers involved, checking the sample/all references for accuracy Parallelization and Multiple reviewers assessing the certainty of the i.e. using the GRADE approach automation evidence Performing selected review activities simultaneously e.g., data extraction of known studies while a search of new studies is conducted reported the use of software to assist in the screening standards of preferred reporting items [26–28]. These and selection of references (11%; Table 2). may not exist for all topics. Regarding the resources available for RR development, Discussion we found that a considerable number of institutions in- To the best of our knowledge, this is the first inter- volved trained staff in the development of diagnostic national survey assessing current practice of methods for RRs although there were usually only two reviewers in- diagnostic test RRs. With 25 participants from across all volved. Selection and data abstraction by a single re- continents, we managed to generate a broad inter- viewer were common; however, it is possible that some national sample. We obtained additional information institutions may prefer to perform selective verification about current strategies in use for development of these by a second reviewer on a sample of the total citations. RRs in order to complement the findings from our pre- This strategy was suggested by the survey’s participants. vious scoping review [20]. We found that roughly one-third of institutions involved Briefly, the general methods involved for the develop- stakeholders in the development of RRs. While most ment of RR can be broadly classified into two groups: standard SRs do not involve stakeholders in their pro- those limiting the scope and/or affecting the rigor of RR duction, RRs might be more relevant for decision- development; and those increasing the resources avail- making in certain situations [29–31]. We further found able for RR development [5, 6, 24, 25]. In the first group, limited use of task parallelization perhaps due to the lack we found that most strategies to narrow the scope are of studies about the usability and impact of these strat- not used as a standard method; however, our survey in- egies both in general and for diagnostic test RRs in par- dicated a greater usage of limits in the scope than our ticular [32–34]. previous scoping review suggested [20]. In addition, we Previous surveys of producers of knowledge syntheses found a high number of participants imposing limits on reported slightly higher levels of adoption of RR the search, e.g., by limiting the language or date of elec- methods compared to our findings [35]. These levels of tronic searches. We noticed that more than half of re- adoption are also higher than those found in our previ- spondents use methodological filters during the ous scoping review. It is possible that RRs using methods literature search indicating that respondents are willing similar to those used in SRs have a greater chance of be- to potentially miss some studies in order to retrieve a ing published [24, 36]. While we found that few RR manageable number of search results given the project’s methods are used by more than 90% of participants, we shortened timeframe [17, 18]. We also confirmed that also observed that some SR tasks—such as developing a the participants often use a narrative synthesis to de- protocol and performing peer-review—are commonly scribe their findings rather than a formal data meta- implemented despite the time required for implementa- analysis [20]. Participants also reported that they con- tion. One possible explanation for this is that the extent sider the inclusion of previous evidence synthesis to be of methodological modifications relies on a request from useful for streamlining. While the use of pre-existing re- different stakeholders and therefore, in some cases, RRs views was one strategy proposed for development of can be produced following many of the same methods RRs, it is important to note that this strategy depends on used in standard SRs [20, 37]. In addition, it is known the availability of existing SRs that satisfy the updated that decision-makers are willing to accept only a small Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 Page 6 of 7 risk for an inaccurate answer in exchange for a rapid prod- Abbreviations uct; thus, current RR developers would be reluctant to CHERRIES: Checklist for Reporting Results of Internet E-Surveys; GRADE: Grading of Recommendations Assessment, Development and compromise the validity of results in exchange for imple- Evaluation; HTA: Health Technology Assessment; INAHTA: International mentation of methodological shortcuts and limits [37, 38]. Network of Agencies for Health Technology Assessment; REDETSA: Health We acknowledge several potential limitations in this Technology Assessment Network of the Americas; RRs: Rapid reviews; SRs: Systematic reviews; WHO: World Health Organization study. The descriptive goal of our research do not involve specialized statistical analysis. We obtained a 53% response Acknowledgements rate from invited institutions based around the world; non- Ingrid Arevalo-Rodriguez is funded by the Instituto de Salud Carlos III through the “Acción Estrategica en Salud 2013-2016 / Contratos Sara Borrell responders were located mainly in Europe and Latin Amer- convocatoria 2017/CD17/00219” (Co-funded by European Social Fund 2014- ica. Although we obtained replies from institutions based in 2020, “Investing in your future”). Andrea C. Tricco is funded by a Tier 2 similar locations, these missing data could have generated a Canada Research Chair in Knowledge Synthesis. Pablo Alonso-Coello is sup- ported by a Miguel Servet investigator contract from the Instituto de Salud risk of selection bias in our findings. We also found that 13 Carlos III (CPII15/0034). out of 39 institutions replying to our invitation do not con- duct RRs and/or RRs of diagnostic tests. Also, participants Authors’ contributions IAR, KS, ACT, BS, DK, PAC, SB, PB, JIE and JZ contributed to the conception in our survey were mainly representatives of local, national, and design of the study. IAR, KS, ACT, BS, DK, PAC, SB, PB, JIE and JZ were and regional HTA agencies. The reports performed by involved in the development of the protocol of this project. IAR coordinated these institutions might have characteristics that differ from the project. IAR, KS, ACT, JZ and BN collected information and analyzed the data. IAR, KS, ACT, BS, DK, PAC, SB, PB, JIE and JZ participated in writing the other reviews (i.e. classic systematic reviews) produced in original draft. IAR, KS, ACT, BS, DK, PAC, SB, PB, JIE and JZ read and approved academic and research settings. the final manuscript. Conclusions Funding This study is funded by the Fundación para la Investigación Biomédica To the best of our knowledge, this is the first inter- (FIBIO)-Hospital Universitario Ramon y Cajal (Internal grant IMP 18–05/2018). national survey assessing the current practice of The funder did not have a role in the design of the study and collection, methods for diagnostic test RRs. Our survey indicates a analysis, interpretation of data, or in writing the final manuscript. greater use of shortcuts and limits versus our previous Availability of data and materials scoping review findings that was based on published RRs The datasets used and/or analyzed during the current study are available of diagnostic tests. from the corresponding author on reasonable request. However, while our findings suggest that SR methods are Ethics approval and consent to participate sometimes preserved in the current practice of diagnostic In accordance with Spanish National Regulations, this study was exempt of test RRs, more general RR methods are usually applied to requiring approval by our Ethics Committee for Investigation (Hospital Ramon y Cajal, communication received on November 6 of 2018). We diagnostic evidence without a structured evaluation of the obtained written consent to participate from all enrolled institutions (positive impact of their implementation. Nevertheless, due to the reply to invitation letter and agreement to participate, see more information different characteristics of evidence synthesis used with in Methods). diagnostic tests, there is a need to evaluate the commonly Consent for publication used methodological shortcuts specific to data sets com- Not applicable. prising the RR on diagnostic tests as other researchers have claimed for their own fields [24]. In order to investigate this Competing interests The authors declare that they have no competing interests. further, the next stage of our work is to conduct a series of interviews with experts in the diagnostic field to explore the Author details 1 potential challenges and implications of the adoption of Clinical Biostatistics Unit, Hospital Universitario Ramón y Cajal, IRYCIS, CIBER of Epidemiology and Public Health, Madrid, Spain. 2Department of Clinical these RR methods [21]. The findings of our research pro- Sciences, Liverpool School of Tropical Medicine, Liverpool, UK. 3Li Ka Shing gram on the RRs of diagnostic tests will be useful for the Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, Toronto, development of clear and tailored guidance in this field as Canada. 4Epidemiology Division, Dalla Lana School of Public Health and Institute of Health Policy, Management and Evaluation at the University of well as to provide recommendations about adequate Toronto, Toronto, Canada. 5Queen’s Collaboration for Health Care Quality, methods for rapid synthesis of diagnostic evidence for Joanna Briggs Institute Centre of Excellence, Queen’s University, Kingston, decision-making and policy development. Canada. 6Cochrane Austria, Danube University Krems, Krems, Austria. 7 Canadian Agency for Drugs and Technologies in Health (CADTH), Ottawa, Canada. 8Iberoamerican Cochrane Center-Servicio de Epidemiología Clínica y Salud Pública, Biomedical Research Institute (IIB-Sant Pau), Barcelona, Spain. Supplementary information 9 CIBER of Epidemiology and Public Health, Barcelona, Spain. 10School of Supplementary information accompanies this paper at https://doi.org/10. Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK. 1186/s12874-020-01004-z. 11 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centres, University of Amsterdam, Additional file 1. Copy of online survey. Amsterdam, The Netherlands. 12Clinical Epidemiology Unit, Hospital Universitario Donostia, BioDonostia, CIBER of Epidemiology and Public Health, San Sebastian, Spain. 13Barts and the London School of Medicine and Dentistry, Queen Mary University, London, UK. Arevalo-Rodriguez et al. BMC Medical Research Methodology (2020) 20:115 Page 7 of 7 Received: 20 March 2020 Accepted: 30 April 2020 23. Knottnerus J, Frank B. The evidence base of clinical diagnosis. 2nd ed. London: BMJ Books; 2009. 24. Campbell F, Weeks L, Booth A, Kaunelis D, Smith A. A scoping review found increasing examples of rapid qualitative evidence syntheses and no References methodological guidance. J Clin Epidemiol. 2019;115:160–71. 1. Moher D, Stewart L, Shekelle P. All in the family: systematic reviews, rapid 25. Tsertsvadze A, Chen YF, Moher D, Sutcliffe P, McCarthy N. How to conduct reviews, scoping reviews, realist reviews, and more. Syst Rev. 2015;4:183. systematic reviews more expeditiously? Syst Rev. 2015;4:160. 2. Hartling L, Guise JM, Kato E, Anderson J, Aronson N, Belinson S, et al. EPC 26. World Health Organization. WHO handbook for guideline development methods: an exploration of methods and context for the production of Geneva. Switzerland: World Health Organization; 2014. rapid reviews. Research white paper. Rockville: Agency for Healthcare 27. Pieper D, Koensgen N, Breuing J, Ge L, Wegewitz U. How is AMSTAR Research and Quality; 2015. applied by authors - a call for better reporting. BMC Med Res Methodol. 3. Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence 2018;18(1):56. summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10. 28. McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, Clifford T, 4. Polisena J, Garritty C, Kamel C, Stevens A, Abou-Setta AM. Rapid review et al. Preferred reporting items for a systematic review and meta-analysis of programs to support health care and policy decision making: a descriptive diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA. 2018; analysis of processes and methods. Syst Rev. 2015;4:26. 319(4):338–96. 5. Langlois EV, Straus SE, Antony J, King VJ, Tricco AC. Using rapid reviews to 29. Wilson MG, Lavis JN, Gauvin FP. Developing a rapid-response program for strengthen health policy and systems and progress towards universal health health system decision-makers in Canada: findings from an issue brief and coverage. BMJ Glob Health. 2019;4(1):e001178. stakeholder dialogue. Syst Rev. 2015;4:25. 6. Tricco AC, Langlois EV, Straus SE, editors. Rapid reviews to strengthen health 30. Moore G, Redman S, Rudge S, Haynes A. Do policy-makers find policy and systems: a practical guide. Geneve: World Health Organization; 2017. commissioned rapid reviews useful? Health Res Policy Syst. 2018;16(1):17. 7. Pandor A, Kaltenthaler E, Martyn-St James M, Wong R, Cooper K, Dimairo M, 31. Hartling L, Guise JM, Hempel S, Featherstone R, Mitchell MD, Motu'apuaka et al. Delphi consensus reached to produce a decision tool for SelecTing ML, et al. Fit for purpose: perspectives on rapid reviews from end-user approaches for rapid reviews (STARR). J Clin Epidemiol. 2019;114:22–9. interviews. Syst Rev. 2017;6(1):32. 8. Banbury A, Roots A, Nancarrow S. Rapid review of applications of e-health 32. Marshall IJ, Wallace BC. Toward systematic review automation: a practical and remote monitoring for rural residents. Aust J Rural Health. 2014;22(5): guide to using machine learning tools in research synthesis. Syst Rev. 2019; 211–22. 8(1):163. 9. Ismail SA, Abbara A, Collin SM, Orcutt M, Coutts AP, Maziak W, et al. 33. O'Connor AM, Tsafnat G, Thomas J, Glasziou P, Gilbert SB, Hutton B. A Communicable disease surveillance and control in the context of conflict question of trust: can we build an evidence base to gain trust in systematic and mass displacement in Syria. Int J Infect Dis. 2016;47:15–22. review automation technologies? Syst Rev. 2019;8(1):143. 10. Paules CI, Marston HD, Fauci AS. Coronavirus infections—more than just the 34. Beller E, Clark J, Tsafnat G, Adams C, Diehl H, Lund H, et al. Making progress common cold. JAMA. 2020;323(8):707–8. with the automation of systematic reviews: principles of the international 11. Leeflang MM. Systematic reviews and meta-analyses of diagnostic test collaboration for the automation of systematic reviews (ICASR). Syst Rev. accuracy. Clin Microbiol Infect. 2014;20(2):105–13. 2018;7(1):77. 12. Deeks J, Bossuyt P, Gatsonis CE. Cochrane handbook for systematic reviews 35. Tricco AC, Zarin W, Antony J, Hutton B, Moher D, Sherifali D, et al. An of diagnostic test accuracy. London: The Cochrane Collaboration; 2010. international survey and modified Delphi approach revealed numerous 13. Mustafa RA, Wiercioch W, Cheung A, Prediger B, Brozek J, Bossuyt P, et al. rapid review methods. J Clin Epidemiol. 2016;70:61–7. Decision making about healthcare-related tests and diagnostic test 36. Kelly SE, Moher D, Clifford TJ. Quality of conduct and reporting in rapid strategies. Paper 2: a review of methodological and practical challenges. J reviews: an exploration of compliance with PRISMA and AMSTAR guidelines. Clin Epidemiol. 2017;92:18–28. Syst Rev. 2016;5:79. 14. Mustafa RA, Wiercioch W, Falavigna M, Zhang Y, Ivanova L, Arevalo- 37. Schunemann HJ, Moja L. Reviews: Rapid! Rapid! Rapid! ...and systematic. Syst Rodriguez I, et al. Decision making about healthcare-related tests and Rev. 2015;4:4. diagnostic test strategies. Paper 3: a systematic review shows limitations in 38. Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. most tools designed to assess quality and develop recommendations. J Clin Trading certainty for speed - how much uncertainty are decisionmakers and Epidemiol. 2017;92:29–37. guideline developers willing to accept when using rapid reviews: an 15. Haby MM, Chapman E, Clark R, Barreto J, Reveiz L, Lavis JN. What are the international survey. BMC Med Res Methodol. 2017;17(1):121. best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review. Health Res Policy Syst. 2016;14(1):83. Publisher’s Note 16. Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an Springer Nature remains neutral with regard to jurisdictional claims in emerging approach to evidence synthesis in health technology assessment. published maps and institutional affiliations. Int J Technol Assess Health Care. 2014;30(1):20–7. 17. Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, et al. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev. 2013;9:Mr000022. 18. van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta- epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol. 2014;67(11): 1192–9. 19. Leeflang MM, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Syst Rev. 2013;2:82. 20. Arevalo-Rodriguez I, Moreno-Nunez P, Nussbaumer-Streit B, Steingart K, Gonzalez Peña L, Buitrago-Garcia D, et al. Rapid reviews of medical tests used many similar methods to systematic reviews but key items were rarely reported: a scoping review. J Clin Epidemiol. 2019;116:98–105. 21. Arevalo-Rodriguez I, Tricco AC, Steingart KR, Nussbaumer-Streit B, Kaunelis D, Alonso-Coello P, et al. Challenges of rapid reviews for diagnostic test accuracy questions: a protocol for an international survey and expert consultation. Diagn Prognostic Res. 2019;3(1):7. 22. Eysenbach G. Improving the quality of web surveys: the checklist for reporting results of internet E-surveys (CHERRIES). J Med Internet Res. 2004; 6(3):e34.