F1000Research 2020, 9:226 Last updated: 31 MAR 2022 RESEARCH ARTICLE Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement [version 3; peer review: 2 approved] Previously titled: Quality of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement Carles Fernández-Jané 1,2, Mireia Solà-Madurell1, Mingkun Yu3, Changhao Liang3, Yutong Fei3, Mercè Sitjà-Rabert 1,2, Gerard Úrrutia 4 1School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain 2Global Research on Wellbeing (GRoW) Research Group, Ramon Llull University, Barcelona, Spain 3Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China 4Institut d'Investigació Biomèdica Sant Pau (IIB Sant Pau). CIBER de Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain v3 First published: 02 Apr 2020, 9:226 Open Peer Review https://doi.org/10.12688/f1000research.22843.1 Second version: 23 Oct 2020, 9:226 https://doi.org/10.12688/f1000research.22843.2 Approval Status Latest published: 20 Nov 2020, 9:226 https://doi.org/10.12688/f1000research.22843.3 1 2 version 3 Abstract (revision) view view Background: The completeness of reporting of acupuncture 20 Nov 2020 interventions is critical to ensure the applicability and reproducibility of acupuncture clinical trials. In the past, different publications have version 2 evaluated the completeness of reporting of acupuncture interventions (revision) view for different clinical situations, such as knee osteoarthritis, 23 Oct 2020 neurological diseases or cancer. However, this has not been done for acupuncture trials for chronic obstructive pulmonary disease (COPD). version 1 02 Apr 2020 view Objective: To assess the completeness of reporting of acupuncture interventions in trials for COPD. 1. Erik Cobo , Polytechnic University of Methods: A total of 11 English and Chinese databases were screened Catalonia, Barcelona, Spain up until May 2019 for randomised or quasi-randomised control trials of acupuncture for COPD. The STRICTA checklist was used to 2. David Blanco , Universitat Internacional determine the quality of the reporting of acupuncture interventions. de Catalunya, Barcelona, Spain Results: A total of 28 trials were included in our review. Out of the 16 Any reports and responses or comments on the STRICTA checklist subitems analysed, only 4 were considered article can be found at the end of the article. appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 30%. Page 1 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Conclusion: The adherence to STRICTA guidelines of acupuncture trials for COPD is suboptimal, and future efforts need to be addressed to improve the completeness of reporting. Keywords Quality of Reporting, Acupuncture, COPD Corresponding author: Carles Fernández-Jané (carlesfj@blanquerna.url.edu) Author roles: Fernández-Jané C: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing; Solà-Madurell M: Conceptualization, Investigation, Writing – Review & Editing; Yu M: Investigation, Writing – Review & Editing; Liang C: Investigation, Writing – Review & Editing; Fei Y: Conceptualization, Methodology, Writing – Review & Editing; Sitjà-Rabert M: Methodology, Writing – Review & Editing; Úrrutia G: Methodology, Writing – Review & Editing Competing interests: No competing interests were disclosed. Grant information: The author(s) declared that no grants were involved in supporting this work. Copyright: © 2020 Fernández-Jané C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Fernández-Jané C, Solà-Madurell M, Yu M et al. Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement [version 3; peer review: 2 approved] F1000Research 2020, 9:226 https://doi.org/10.12688/f1000research.22843.3 First published: 02 Apr 2020, 9:226 https://doi.org/10.12688/f1000research.22843.1 Page 2 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 currently no publications regarding the completeness of reporting REVISED Amendments from Version 2 of acupuncture interventions in COPD trials. The term “quality of reporting”, has been changed to “completeness of reporting” as “quality” is an ambiguous and Evaluating the adherence of acupuncture clinical trials for problematic word, and to use. COPD to SRICTA guidelines is crucial to detect underreported A mistake in the result section in the abstract has been subitems and therefore highlight current deficiencies. This will corrected. help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future Also, several clarifications have been made in the introduction and methods sections. research. In the results section, a new paragraph has been added to give The aim of this study is to comprehensibly evaluate the degree of an “overall summary” of the results. completeness of reporting for each STRICTA item in randomised Finally, a new paragraph has been added in the discussion trials using acupuncture. section to address possible interventions to improve the completeness of reporting. Methods Any further responses from the reviewers can be found at the end of the article Study selection In this study, we used the results of our previous systematic review, which included randomised or quasi-randomised tri- als using filiform needle acupuncture for COPD2. Published studies were comprehensively searched in the following data- Introduction bases from their inception to May 2019: Cochrane Central Recent systematic reviews have assessed acupuncture’s Register of Controlled Trials (CENTRAL), Medline, Embase, efficacy for chronic obstructive pulmonary disease (COPD)1–3. CINAHL, AMED (Ovid), PEDro, PsycINFO, CNKI, VIP, These reviews have concluded that, even though acupuncture Wanfang, and Sino-Med. Detailed descriptions of the inclu- could have some benefits, the risk of bias of the included trials sion criteria, information sources, search strategies and study is too high to draw any strong conclusion. Risk of bias is selections are published elsewhere2 and the protocol is available certainly a critical aspect in randomised control trials; however, at http://www.crd.york.ac.uk/PROSPERO/display_record.asp? to be able to adequately assess sources of bias, complete ID=CRD42014015074. information needs to be reported. Data collection process Complete and clear information regarding clinical trial method- Data from each trail was extracted independently by two ology is not only essential to adequately assess health research reviewers (CFJ, MSM, MY and CHL) using a standardised data but also for its applicability and reproducibility. This is extraction form, and disagreements were solved by consensus. especially important in complex interventions, such as acupunc- ture, that can be practiced in many different styles and varia- For data collection and STRICTA assessment, a specific tions. Aspects such as point selection, depth of the insertion, extraction table with instructions of how to assess each stimulation method and response sought, which may be very of the 16 STRICTA subitems was created by the authors different between practitioners, could have an impact on (Extended data12). This extraction table was tested with pilot the therapeutic effect4. Therefore, to be able to replicate an data of 3 papers to solve disagreements on its understanding acupuncture intervention in clinical practice or reproduce it in and ensure its usability. The data of the pilot test was included another trial, it is necessary to fully describe how it is applied. in the final analysis. The STandards for Reporting Interventions in Controlled Trials Our aim was to assess the acupuncture interventions, and of Acupuncture (STRICTA) guidelines were created to improve therefore, the 17th STRICTA subitem “precise description of the completeness of reporting in acupuncture trials and facili- the control or comparator” was not considered. However, we tate transparency in published reports5. These guidelines were did include the 16th STRICTA subitem “rationale of the chosen updated in 2010 as an extension of the CONsolidated Stand- comparator”, since it is critical to justify which component of the ards Of Reporting Trials (CONSORT) guidelines6 and consist acupuncture treatment is being assessed. of 6 key items and 17 subitems addressing aspects such as acu- puncture rationale, needling details, treatment regime, other Since some STRICTA subitems refer to multiple aspects components of the treatment, practitioner background and details (e.g., “names of points used” subitem refers not only to the name about the control or comparator. or location of points but also to if they are used unilaterally or bilaterally), besides considering items just as reported or not Although there is evidence that STRICTA guidelines have reported, we also considered partially reported items and helped to improve acupuncture’s reporting, there is still a lot recorded the reasons for there being. This was done to pro- of room for improvement7. This has also been seen in recent vide more detailed information regarding the aspects that should publications for some specific conditions such as neurological be improved in the reporting of future trials. Partial reporting diseases8–10 or cancer11, concluding that reporting is still subop- was also considered when the authors reported information in timal in these conditions. However, to our knowledge, there is other sections, such as the introduction or the discussion. Page 3 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Although some subitems were considered that could potentially Completeness of reporting be “not applicable” (NA) for some pragmatic designs, none of Out of the 16 STRICTA checklist subitems analysed, only 4 the trials required this. were considered appropriately reported in more than 70% of the trials; style of acupuncture, variation extent, retention time Data analysis and frequency and duration of treatment sessions. We also found A descriptive analysis was used to summarise the results using that 7 other subitems were correctly reported in less than 30% percentages and absolute numbers. of the trials; depth of insertion, needle type, number of treat- ment sessions, details of other interventions administered, set- Results ting and context of treatment, description of participating Number of publications and characteristics acupuncturists and rationale for the control. In our systematic review, we screened all 5030 unduplicated titles and abstracts retrieved, and obtained 166 full text articles. Ratings for STRICTA domains are summarized in Figure 2. Finally, we included 28 trials using filiform needles for COPD Details for each trial are shown in Table 1, including reasons for (Figure 1). Of those, only 6 were published in English-language considering partial reporting. journals, 1 in a German-language journal and 21 in Chinese- language journals. Details regarding the study characteristics Acupuncture rationale. Acupuncture rationale was considered and inclusion process have been published elsewhere2. adequately reported in 20 trials (71%) regarding acupuncture Figure 1. Flow diagram. Page 4 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Figure 2. Summary of completeness of reporting according to STRICTA guidelines. style, 18 trials (64%) regarding reasoning of the treatment and trials (100%), the “number of treatment sessions” was 24 trials (85%) regarding treatment variation. Trials classi- considered completely reported in none of them (0%). Although fied with partial reporting mentioned acupuncture style (3 tri- this might seem a contradiction, since the number of treat- als) and reasoning (4 trials) in the introduction section but ment sessions can be calculated from the treatment regime, not in the methods section. in STRICTA, the number of treatment sessions does not only include the planned number of sessions but also the actual Details of needling. Adequate reporting of needling details number of treatments received. This information was missing or was very heterogenous along all 4 subdomains. Best reported considered not clear in all trials. subdomains were “needle retention time” (21 trials, 75%) and “needle stimulation” (19 trials, 67%). Worse reported Other components of treatment. Other components of treat- items were “name of the points” (10 trials, 35%), “depth of ment were one of the poorer described items. “Details of other insertion” (8 trials, 28%) and “needle type” (6 trials, 21%). interventions administrated” was only reported in 8 trials (28%) Partial reporting was observed in great proportion in “name and “settings and context of treatments”, which refers to “instruc- of acupuncture points” (16 trials, 57%) being the main rea- tions to practitioners that might modify their normal practice, son not describing if points were used unilaterally or bilater- for example, prescribing or proscribing explanations to patients ally. Regarding the 46% of the trials classified with a partial about their diagnosis”, were only reported in 2 trials (7.1%). reporting in the “needle type” item, there was missing information about the needle manufacturer (32.1%, 9 trials), Practitioner background. This item was only addressed com- material (25%, 7 trials) and length and diameter (10.7%, pletely in 1 trial (3.5%), and only 2 more trials (7.1%) partially 3 trials). reported it without stating practitioner’s years of experience. Treatment regime. While “frequency and duration of the treat- Control or comparator. The “rational of the chosen comparator” ment sessions” was considered adequately reported for all was only correctly described in 2 trials (7.1%), while in 2 Page 5 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Table 1. Completeness of reporting according to STRICTA guidelines. STRICTA Items Reported, Not reported, Partially reported, % (n) % (n) % (n) Reason, % (n) 1) Acupuncture 1a) Style of acupuncture 71.4 (20) 14.2 (4) 14.2 (4) rationale    Not reported in the correct section, 14.2 (4) 1b) Reasoning for treatment provided, 64.2 (18) 21.4 (6) 14.2 (4) based on historical context, literature    Not reported in the correct section, sources, and/or consensus methods, 14.2 (4) with references where appropriate 1c) Extent to which treatment was varied 82.1 (23) 17.8 (5) 0 (0) 2) Details of needling 2a) Number of needle insertions per 46.4 (13) 28.5 (8) 17.8 (5) subject per session   Do not mention the number of needles, 17.8 (5) 2b) Names (or location if no standard 35.7 (10) 7.14 (2) 57.14 (16) name) of points used (uni/bilateral)   Do not mention uni or bilateral insertion, 53 (15)    Not all point locations are described, 3.5 (1) 2c) Depth of insertion, based on a 28.5 (8) 71.4 (20) 0 (0) specified unit of measurement or on a particular tissue level 2d) Responses sought 53.5 (15) 46.4 (13) 0 (0) 2e) Needle stimulation 67.8 (19) 17.8 (5) 14.2 (4)    Manual stimulation is mentioned but the specify the method is not, 10.7 (3)    Electrical stimulation is mentioned but not parameters used, 3.5 (1) 2f) Needle retention time 75 (21) 25 (7) 0 (0) 2g) Needle type 21.4 (6) 28.5 (8) 50 (14)   Material is not reported, 25 (7)    Manufacturer is not reported, 32.1 (9)    Diameter and length are not reported, 10.7 (3) 3) Treatment regime 3a) Number of treatment sessions 0 (0) 0 (0) 100 (28)    The actual number of treatments received is not reported or not clear, 100 (28) 3b) Frequency and duration of 100 (28) 0 (0) 0 (0) treatment sessions 4) Other components 4a) Details of other interventions 28.5 (8) 57.1 (16) 14.2 (4) of treatment administered to the acupuncture group 4b) Setting and context of treatment, 7.1 (2) 89.2 (25) 3.5 (1) including instructions to practitioners, and information and explanations to patients 5) Practitioner 5) Description of participating 3.5 (1) 89.2 (25) 7.1 (2) background acupuncturists   Years of experience not mentioned, 7.1 (2) 6) Control or 6a) Rationale for the control or 7.1 (2) 85.7 (24) 7.1 (2) comparator comparator   Not reported in the correct section: interventions 7.1 (2) STRICTA, STandards for Reporting Interventions in Clinical Trials of Acupuncture. Page 6 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 more trials (7.1%), this was mentioned in the introduction Strengths and limitations but not in the methods section. To our knowledge, this is the first study to assess the completeness of reporting of acupuncture interventions for Discussion COPD. We included all acupuncture trials for COPD published We found important limitations in the completeness of report- until May 2019 with no language restriction, which is important ing of acupuncture interventions in trials for COPD, espe- since we only found 6 acupuncture trials published in English. cially regarding “depth of insertion”, “needle type”, “number Also, we did not only assess if STRICTA subitems were of treatment sessions”, “details of other interventions admin- adequately reported or not but also analysed partial reporting and istered”, “setting and context of treatments”, “description of its reasons, which might be more helpful for authors to realise acupuncturist” and “rationale for the control or comparator” with what specific information is currently missing. less than 30% of the trials reporting them completely. Limitations of this study include that the STRICTA guidelines Recently, several similar studies have been published. Lu are not a rating scale; therefore, there are no clear indications et al.11 and Hughes et al.13 used STRICTA to evaluate trials of how to judge each subitem and when to consider it fully with cancer patients, and Wei et al. used STRICTA to evalu- reported. This issuer must be addressed in the future by ate trials with stroke patients8. Although they all concluded developing a proper completeness of reporting assessment tool that reporting should be improved, there were also some for acupuncture interventions. To minimise this problem, each differences. While Hughes et al. found better reporting item was assessed by two reviewers independently, and a regarding details of needling and treatment regimen, other standardised extraction form was developed to unify reviewers’ reviews found lower reporting on these subitems, especially criteria. Also, it would have been interesting to compare the regarding number of needles per session and depth of inser- completeness of reporting depending on the language of tion. Poor reporting on “details of other interventions admin- publication, so we could explore differences in journal stand- istered”, “description of acupuncturist” and “rationale for ards. However, due to the low number of non-Chinese-language the control or comparator” subitems was found in all studies. publications found (1 in German and 6 in English), we decided not to do so. Finally, STRICTA guidelines are specific for the The differences mentioned above could be due to several filiform needle acupuncture technique and are not suitable to reasons. First, Lu’s and Weis’s reviews, as well as our own, assess other interventions. Therefore, we did not include trials included Chinese-language trials, while Hugs’ study included only using only moxibustion, acupressure or transcutaneous electrical English-language trials. Trials published in English-language nerve stimulation. journals seem to have greater completeness of reporting than those in Chinese-language journals, according to Lu’s review. Conclusion However, Wei et al. found better reporting of the subitems “treat- The completeness of reporting of acupuncture interventions ment reasoning” and “response sought” in Chinese journals in COPD trials according to STRICTA guidelines is subopti- and better reporting on “practitioner’s background” in English mal. Strategies for improving the understanding of the guides journals. for authors, reviewers and journal editors are needed, as well as to improve its implementation. Second, since STRICTA does not clearly specify how items should be judged, authors might have used different criteria. For Data availability example, regarding the subitem “number of treatment sessions”, Underlying data the STRICTA statement says that “the actual number of Figshare: Raw data file, https://doi.org/10.6084/ treatments received by participants should be reported in the m9.figshare.11999970.v115. Results section” not only the planned ones. Whereas in our review, we did not consider that this subitem was fully reported Extended data unless this information was explicitly stated; others might Figshare: Extended data 1 Extraction form, https://doi.org/ have been more permissive. Also, the criteria to consider 10.6084/m9.figshare.11999994.v112. proper reporting on “other components of treatment” might vary widely between reviewers. Data is available under a Creative Commons Attribution 4.0 International license (CC-BY 4.0). Third, sometimes information might have been reported in sections such as the introduction and the discussion. While some authors might not have given much importance to this, Acknowledgements we decided to take it into consideration. We would like to thank the students from the Beijing University of Chinese Medicine Liu Kexin, Zhang Zixuan, Ming To try to improve the adherence to reporting guidelines Yang, Gao Jiaqi, Li Yilin, Zhang Zhijia, Luo Lingxiao, Shi Yixin, several strategies have been proposed including training on the Zhao Luming, Zhang Bingrui, Zhao Leying, Liu Jinjun, Ding use of the guidelines, improving understanding, encouraging Maoyu, Fan Shixuan and Ren Yiming for participating in the adherence, checking adherence and providing feedback, and project and helping to define the suitability of the data extrac- the involvement of experts. Unfortunately, the effectiveness of tion form. The corresponding author is a PhD candidate at the many of those interventions is still unknown14 Universitat Autònoma de Barcelona, Spain. Page 7 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 References 1. Coyle ME, Shergis JL, Huang ET, et al.: Acupuncture therapies for chronic controlled trials of electroacupuncture for stroke. BMC Complement Altern obstructive pulmonary disease: a systematic review of randomized, Med. 2016; 16(1): 512. controlled trials. Altern Ther Health Med. 2014; 20(6): 10–23. PubMed Abstract |� �� Publisher Full Text | Free Full Text PubMed Abstract 9. Luo G, Lu L, Zeng J: Quality of reporting of randomised controlled trials of 2. Fernández-Jané C, Vilaró J, Fei Y, et al.: Filiform needle acupuncture for copd: A acupuncture for neurological diseases conducted in China. Acupunct Med. systematic review and meta-analysis. Complement Ther Med. 2019; 47: 102182. 2014; 32(5): 411–7. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text 3. Hsieh PC, Yang MC, Wu YK, et al.: Acupuncture therapy improves health- 10. Zeng J, Lin G, Li L, et al.: Assessment of reporting quality in randomised related quality of life in patients with chronic obstructive pulmonary controlled trials of acupuncture for post-stroke rehabilitation using the disease: A systematic review and meta-analysis. Complement Ther Clin Pract. CONSORT statement and STRICTA guidelines. Acupunct Med. 2017; 35(2): 2019; 35: 208–218. 100–106. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text 4. Shi GX, Yang XM, Liu CZ, et al.: Factors contributing to therapeutic effects 11. Lu L, Liao M, Zeng J, et al.: Quality of reporting and its correlates among evaluated in acupuncture clinical trials. Trials. 2012; 13: 42. randomized controlled trials on acupuncture for cancer pain: application PubMed Abstract | Publisher Full Text | Free Full Text of the CONSORT 2010 Statement and STRICTA. Expert Rev Anticancer Ther. 5. MacPherson H, Altman DG, Hammerschlag R, et al.: Revised STandards 2013; 13(4): 489–98. for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): PubMed Abstract | Publisher Full Text Extending the CONSORT statement. J Evid Based Med. 2010; 3(3): 12. Fernández C: Extended data 1 Extraction form. figshare. Dataset. 2020. 140–55. http://www.doi.org/10.6084/m9.figshare.11999994.v1 PubMed Abstract | Publisher Full Text 13. Hughes JG, Lewith G, MacPherson H, et al.: Assessment of the quality of 6. Schulz KF, Altman DG, Moher D: CONSORT 2010 statement: updated reporting in studies of acupuncture for patients with cancer using the guidelines for reporting parallel group randomised trials. BMJ. 2010; 340: STRICTA guidelines. Acupunct Med. 2019; 37(4): 223–227. c332. PubMed Abstract | Publisher Full Text PubMed Abstract | Publisher Full Text | Free Full Text 14. Blanco D, Altman D, Moher D, et al.: Scoping review on interventions to 7. Svenkerud S, MacPherson H: The impact of STRICTA and CONSORT on improve adherence to reporting guidelines in health research. BMJ Open. reporting of randomised control trials of acupuncture: a systematic 2019; 9(5): e026589. methodological evaluation. Acupunct Med. 2018; 36(6): 349–357. PubMed Abstract | Publisher Full Text | Free Full Text PubMed Abstract | Publisher Full Text 15. Fernández C: Raw data file. figshare. Dataset. 2020. 8. Wei JJ, Yang WT, Yin SB, et al.: The quality of reporting of randomized http://www.doi.org/10.6084/m9.figshare.11999970.v1 Page 8 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Open Peer Review Current Peer Review Status: Version 3 Reviewer Report 01 December 2020 https://doi.org/10.5256/f1000research.30829.r75127 © 2020 Cobo E. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Erik Cobo Statistics and Operations Research Department, Polytechnic University of Catalonia, Barcelona, Spain Thanks for considering most of my suggestions. I think the paper is now much clearer and probably more reprodicible. Competing Interests: No competing interests were disclosed. Reviewer Expertise: Clinical trials; Reporting Guidelines; meta-research; research waste I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Reviewer Report 23 November 2020 https://doi.org/10.5256/f1000research.30829.r75126 © 2020 Blanco D. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. David Blanco Physiotherapy Department, Universitat Internacional de Catalunya, Barcelona, Spain I am happy with this new version of the manuscript. It is clear and well-reported. Congratulations on your work and thank you for the opportunity to review it! Competing Interests: No competing interests were disclosed. Page 9 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Reviewer Expertise: Journal policies, peer review, reporting guidelines I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Version 2 Reviewer Report 13 November 2020 https://doi.org/10.5256/f1000research.30190.r73827 © 2020 Blanco D. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. David Blanco Physiotherapy Department, Universitat Internacional de Catalunya, Barcelona, Spain Thank you for allowing me to review this manuscript. The subject is the evaluation of the degree of adherence to STRICTA guideline items in the context of acupuncture trials. Overall, I find the manuscript attractive and well-written. I hope the authors find the following suggestions helpful: General comments ○ Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines1 2 . Abstract ○ If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item. Introduction ○ In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper. ○ Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each Page 10 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 STRICTA item in randomised trials using acupuncture” or something similar. Methods ○ It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at. ○ Were the three papers you used for the pilot also included in the final analysis? This should be mentioned. Discussion ○ In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?) ○ The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.” ○ Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such3. ○ I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods. ○ For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies4 5 6 7 8 as well as different scoping and systematic reviews/surveys exploring this issue. References 1. Stevens A, Shamseer L, Weinstein E, Yazdi F, et al.: Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review.BMJ. 2014; 348: g3804 PubMed Abstract | Publisher Full Text 2. Blanco D, Altman D, Moher D, Boutron I, et al.: Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open. 2019; 9 (5). Publisher Full Text 3. Logullo P, MacCarthy A, Kirtley S, Collins GS: Reporting guideline checklists are not quality evaluation forms: they are guidance for writing.Health Sci Rep. 2020; 3 (2): e165 PubMed Abstract | Publisher Full Text 4. Hair K, Macleod MR, Sena ES, IICARus Collaboration: A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus).Res Integr Peer Rev. 2019; 4: 12 PubMed Abstract | Publisher Full Text 5. Barnes C, Boutron I, Giraudeau B, Porcher R, et al.: Impact of an online writing aid tool for Page 11 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial.BMC Med. 2015; 13: 221 PubMed Abstract | Publisher Full Text 6. Blanco D, Schroter S, Aldcroft A, Moher D, et al.: Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial. BMJ Open. 2020; 10 (5). Publisher Full Text 7. Cobo E, Cortés J, Ribera JM, Cardellach F, et al.: Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011; 343: d6783 PubMed Abstract | Publisher Full Text 8. Hopewell S, Boutron I, Altman DG, Barbour G, et al.: Impact of a web-based tool (WebCONSORT) to improve the reporting of randomised trials: results of a randomised controlled trial.BMC Med. 2016; 14 (1): 199 PubMed Abstract | Publisher Full Text Is the work clearly and accurately presented and does it cite the current literature? Partly Is the study design appropriate and is the work technically sound? Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes If applicable, is the statistical analysis and its interpretation appropriate? Yes Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Journal policies, peer review, reporting guidelines I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Author Response 15 Nov 2020 Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain We thank you for your time and comments on the manuscript. Please find below a point by point response to them: Page 12 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 General comments ○Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines1 2. Response: We agree with the suggestion and have changed “quality of reporting” for “completeness of reporting”. Abstract ○If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item. Response: We have now added the information at the beginning of the results section. Introduction ○ In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper. Response: We agree with the reviewers comments and therefore, we modified our last sentence to show that risk of bias and its quality of reporting are connected “Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.” ○ Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar. Response: We have reformulated the aim of the study to avoid confisions as the reviewer suggested. Methods ○ It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at. Response: The structure of the standardized data extraction table is fully available as extended data ○ Were the three papers you used for the pilot also included in the final analysis? This should be mentioned. Response: Yes, he pilot data extraction was included in the analysis. This has been clarified Page 13 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 in the data collection section Discussion ○ In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?) Response: This has been now described “We found important limitations in the quality of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.” ○ The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.” Response: This has been corrected ○ Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such3. Response: This is explained in the limitations section “Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper quality of reporting assessment tool for acupuncture interventions.” ○ I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods. Response: It was already explained in the methods section “Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion.”. However we have now clarified also in the discussion section “Third, sometimes information might have been reported in sections such as the introduction and the discussion.” ○ For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies4 5 6 7 8 as well as different scoping and systematic reviews/surveys exploring this issue. Page 14 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 Response: A new paragraph in the discussion section about the topic has been added “To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown 14” Competing Interests: No competing interests are disclosed Version 1 Reviewer Report 06 October 2020 https://doi.org/10.5256/f1000research.25221.r71710 © 2020 Cobo E. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Erik Cobo Statistics and Operations Research Department, Polytechnic University of Catalonia, Barcelona, Spain I read this paper and I think it could be accepted provided authors correct my three major suggestions and address my minor suggestions. Major suggestions: ○Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify. ○ Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on. ○ Please, allow access to your data to other scientists. Please, consider a public repository. Minor suggestions: ○As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a Page 15 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”. ○ In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper. ○ Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”1 ○ Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers? ○ Please, consider to explain the pilot process and how did you “ensure” its usability. References 1. Goodman S, Fanelli D, Ioannidis J: What does research reproducibility mean?. Science Translational Medicine. 2016; 8 (341). Publisher Full Text Is the work clearly and accurately presented and does it cite the current literature? Yes Is the study design appropriate and is the work technically sound? Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes If applicable, is the statistical analysis and its interpretation appropriate? Yes Are all the source data underlying the results available to ensure full reproducibility? No Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: RCTs, reporting guidelines, causal modeling, meta-researcch I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have Page 16 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 significant reservations, as outlined above. Author Response 11 Oct 2020 Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain We thank you for your time and comments on the manuscript. Please find below a point by point response to them: Major suggestions: Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify. As it is already explained in the figure, one trial included two interventions, that is why the total number of studies included is 62 but the summation of all the interventions is 63. Studies has been changed for articles in the diagram Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on. Conclusions have been re-written to include the mentioned recommendations “The quality of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.” Please, allow access to your data to other scientists. Please, consider a public repository. The data is already public and available in the “data availability” section. Minor suggestions: As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”. We agree on highlighting the need of developing a reliable tool for assessing the quality of reporting and this has been added in the discussion section. We would prefer not to change the nomenclature as the real the objective of the study is to Page 17 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 assess the quality of reporting, and using the adherence to current reporting guidelines is the method to do so. We believe that introducing the term “completeness of the reporting” may confuse reader. In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper. This has been clarified in the introduction section “Risk of bias of bias of the studies is certainly a critical aspect in randomised control trials; however, the quality of reporting of the published papers is also a key point.” Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”1 Here replication and reproducibility are used in a common language sense and the meaning is clarified at the end of the introduction section: “Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.” We believe that using the terms “methods reproducibility”, “Results reproducibility” or “inferential reproducibility” might not clarify but confuse the reader. Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers? Every paper was analysed by two reviewers, this has been now clarified in the methods section: “Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.” Please, consider to explain the pilot process and how did you “ensure” its usability. This has been now clarified in the “data collection process” section. “This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability.” Competing Interests: No competing interests were disclosed. Page 18 of 19 F1000Research 2020, 9:226 Last updated: 31 MAR 2022 The benefits of publishing with F1000Research: • Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com Page 19 of 19