PLOS ONE RESEARCH ARTICLE Nutrimedia: A novel web-based resource for the general public that evaluates the veracity of nutrition claims using the GRADE approach Montserrat Rabassa1, Pablo Alonso-Coello1,2*, Gonzalo Casino1,3* 1 Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain, 2 CIBER de Epidemiologı́a y Salud Pública (CIBERESP), Barcelona. Spain, 3 Departament of Communication, Pompeu Fabra University, Barcelona, Spain a1111111111 * palonso@santpau.cat (PAC); gonzalo.casino@upf.edu (GC) a1111111111 a1111111111 a1111111111 a1111111111 Abstract The objective of Nutrimedia is to evaluate, based on the scientific evidence, the veracity of nutrition claims disseminated to the public by the media. In this article, we describe the methodology, characteristics and contents of this web-based resource, as well as its web OPEN ACCESS traffic and media impact since it was launched. Nutrimedia uses a systematic process to Citation: Rabassa M, Alonso-Coello P, Casino G evaluate common beliefs, claims from newspapers and advertising identified and selected (2020) Nutrimedia: A novel web-based resource by its research team, as well as questions from the public. After formulating a structured for the general public that evaluates the veracity of question for each claim, we conduct a pragmatic search, prioritizing guidelines and/or sys- nutrition claims using the GRADE approach. PLoS ONE 15(4): e0232393. https://doi.org/10.1371/ tematic reviews. We evaluate the certainty of the evidence using the Grading of Recommen- journal.pone.0232393 dations, Assessment, Development and Evaluation (GRADE) approach, and classify the Editor: David Meyre, McMaster University, veracity of each claim into seven categories (true, probably true, possibly true, possibly CANADA false, probably false, false, and uncertain). For each evaluation, we develop a scientific Received: November 29, 2019 report, a plain language summary, a summary of findings table, and, in some cases, a video. From November 2017 to May 2019, we published 30 evaluations (21 were related to Accepted: April 9, 2020 foods, six to diets, and three to supplements), most of which were triggered by questions Published: April 30, 2020 from the public (40%; 12/30). Overall, nearly half of the claims were classified as uncertain Peer Review History: PLOS recognizes the (47%; 14/30). Nutrimedia received 47,265 visitors, with a total of 181,360 pages viewed. benefits of transparency in the peer review The project and its results were reported in 84 written media and 386 websites from Spain process; therefore, we enable the publication of all of the content of peer review and author and 14 other countries, mostly from Latin America. To our knowledge, Nutrimedia is the first responses alongside final, published articles. The web-based resource for the public that evaluates the certainty of evidence and the veracity editorial history of this article is available here: of nutrition claims using the GRADE approach. The scientific rigor combined with the use of https://doi.org/10.1371/journal.pone.0232393 friendly presentation formats are distinctive features of this resource, developed to help the Copyright: © 2020 Rabassa et al. This is an open public to make informed choices about nutrition. access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Introduction Data Availability Statement: All the results of the evaluations are available in the corresponding Most common chronic diseases such as cardiovascular diseases, cancers and diabetes are lead- technical reports freely accessible on the ing causes of death, accounting for 71% of all deaths and 43% of the global burden [1]. Modifi- Nutrimedia website (https://www.upf.edu/web/ able behavioural risk factors such as poor eating or food habits increase the risk of chronic PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 1 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims nutrimedia); all referenced papers are available in diseases [2]. Raising awareness of scientific knowledge about food and nutrition among the the usual databases (MedLine, Cochrane Database public can help improving health overall, and prevent chronic diseases [3]. of Systematic reviews, etc.). Data of the media The dissemination of nutrition information in the media is a potential mean for promoting impact from Factiva can be reproduced by a search with the term "Nutrimedia". Data on web traffic knowledge about appropriate food choices [4–5]. Media (including radio, television, newspa- from Google Analytics is not legally or ethically pers, internet and social media) today include a wealth of information about food and nutri- restricted and it is not necessary to replicate the tion. For example, a recent Google search showed over half a million results related to the results –it changes constantly with time. terms “nutrition advice”. Media has been shown to have a potential impact on knowledge and Funding: Nutrimedia has been partially supported awareness of the public of health issues related to the field of nutrition [6–7]. However, this by the Spanish Foundation for Science and information is often misleading and contradictory [8–10]. According to recent research, the Technology (FECYT) grants from the Spanish public is regularly exposed to mediocre or poor quality information about nutrition on web- Ministry of Science, Innovation and Universities sites [8] as well as in newspapers [9]. Approximately two-thirds of newspapers related to die- (FCT-16-11294 and FCT-17-12460). The funders had no role in the study design, data collection and tary advice are based on low-quality scientific evidence [10]. analysis, decision to publish, or preparation of the Online resources or websites (e.g., Google searches, YouTube) were the most popular manuscript. source of nutrition information, between 2003 and 2018, among adults [11–12]. However, to Competing interests: The authors have declared the best of our knowledge, there are no online resources that formally evaluate and communi- that no competing interests exist. cate the veracity of contemporary claims about nutrition based on scientific evidence. There- fore, we have developed a nutrition web-based information resource for the general public Abbreviations: CPG, Clinical Practice Guideline; GRADE, Grading of Recommendations, named Nutrimedia that provides rigorous evaluations of claims about nutrition. In this article, Assessment, Development and Evaluation; PICO, we describe how we have developed this resource and present an overview of its characteristics, Participant, Intervention, Comparison and Outcome contents and media impact. The research protocol is available in Spanish from the authors (s); RCT, Randomized controlled trial; SoF, upon request. Summary of Findings; SR, Systematic Review. Methods Identification and selection of nutrition claims To identify claims, we developed a preliminary list of claims based on news from the media, Google searches of current topics in nutrition and our knowledge and expertise. This prelimi- nary list has been updated over time. We classified the topics identified in the following four types of claims: • Common beliefs related to the health effects of certain diets, foods or nutrients. • Claims in pieces of news published in newspapers that report about nutrition or nutrition research (newspaper claims). • Advertising claims in traditional or online media (newspapers, magazines, radio, television, websites, blogs and social networks) about the effect of a particular food (or an ingredient) on consumer health or performance. • Claims based on questions from the public collected via an online survey posted on Nutri- media website (e.g., the question "Is meat carcinogenic?" becomes the claim "meat is carcinogenic"). Common beliefs and claims from newspapers or advertising were selected from the prelim- inary list based on two criteria: 1) achieving the highest interest score (we scored on a 5-point Likert scale the interest of each claim), and 2) ensuring a balanced selection of different types of claims and foods. To collect questions from the public, we published an online survey from November 20, 2017, until May 4, 2018. The survey included 10 closed-ended questions selected from the pre- liminary list of claims (to be rated by users on a 5-point Likert scale from 1–5; definitely not to definitely yes interested), one open-ended question and space for comments and suggestions. The questions from the public that were evaluated were selected from those most highly rated PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 2 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims among users in the 10 closed-ended questions and those posed in the open-ended question (the selection was made considering interest and feasibility). Scientific evaluation of nutrition claims For each claim selected, we developed a scientific report using a systematic and explicit process shown in Fig 1 that includes: 1. Formulation of structured clinical question. For each nutrition claim, we formulated a structured clinical question in a PICO (participant, intervention, comparison and outcome (s)) format. We included a maximum of four key public-important outcomes (e.g., death, can- cer or cardiovascular events). 2. Identification and selection of the evidence. To identify and select the best available scientific evidence, we conducted a search prioritizing clinical practice guidelines (CPG) and systematic reviews (SR). We searched on electronic databases (e.g., we searched in MEDLINE and Cochrane Database of Systematic Reviews for SR) (S1 Table), combining at least one MeSH term and free-text terms, with title and/or abstract restrictions, for each PICO compo- nent. We also searched grey literature (e.g. Google Scholar), governmental and institutional sources (e.g. World Health Organization, U.S. Department of Agriculture, Spanish Agency for Food Safety and Nutrition) and scientific societies websites (e.g., Sociedad Española de Nutri- ción Comunitaria). We included CPGs and/or SRs that used systematic methods to search and identify the evi- dence in two or more databases (e.g., MEDLINE, EMBASE) and that evaluated the risk of bias Fig 1. Nutrimedia evaluation process. https://doi.org/10.1371/journal.pone.0232393.g001 PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 3 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims of research evidence [13]. When we obtained more than one CPG or SR of similar quality we prioritized the most recent. If not available, we included primary studies prioritising random- ized clinical trials. 3. Synthesis of the evidence. We summarized the following information of each of the documents selected: 1) objectives, 2) methods, 3) main results, and 4) conclusion of the authors (if applicable) (S2 Table). 4. Evaluation of the certainty of the evidence. We assessed the certainty of the body of evidence (CPG or SR or primary studies) about the effects of nutrition interventions using the GRADE approach [14]. GRADE classifies the certainty of evidence as high, moderate, low, or very low for each outcome of interest within the same clinical question. GRADE categorizes randomized controlled trials (RCTs) as high certainty, whereas observational studies start as low certainty. An available body of RCTs can be downgraded on the basis of five factors: study limitations (risk of bias), indirectness, imprecision, inconsistency of results, and publication bias. In the case of observational studies, one can consider rating up on the basis of three fac- tors: large magnitude of an effect, dose-response gradient, and plausible residual confounding (Table 1). For each evaluation, we generated a Summary of Findings (SoF) table using GRADEPro [15]. SoF tables provide a structured outline of the number of studies and number of partici- pants included for each outcome of interest, certainty (or confidence) of the evidence, and the results both in relative and absolute terms (S3 Table). If the certainty of evidence differed across critical outcomes, the overall certainty of evidence is the lowest certainty of any of the public-important outcomes [16]. 5. Conclusion. We classified the veracity of each claim according to the certainty of the evidence into seven categories: true, probably true, possibly true, possibly false, probably false, false, and uncertain. For example, we assigned true or false, probably true or false, possibly true or false and uncertain conclusion when the overall certainty of the evidence related to a claim was high, moderate, low and very low (or no studies), respectively (Table 2). Development of web content for the evaluated nutrition claims For each claim, we developed the web content in plain language and we structured it in layers and sections, combining text with other formats (images, static and interactive tables, symbols, videos in some cases, and others), with the aim of making it more user-friendly and under- standable. We included the following sections in each evaluation [17]: Table 1. Factors of the certainty of the evidence. Study design Certainty of the Factors that can reduce the Factors that can increase the certainty of the evidence Certainty of the evidence grade certainty of the evidence evidence grade (final) (initial) Randomized High Risk of bias (study limitations) Large magnitude of effect High trial Inconsistency of results (or Dose-response gradient Moderate heterogeneity) Indirectness of evidence (PICO All plausible confounding and bias, which would reduce a and applicability) demonstrated effect or would suggest a spurious effect if no effect was observed Observational Low Imprecision (number of events Low study and confidence intervals) Publication bias Very low https://doi.org/10.1371/journal.pone.0232393.t001 PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 4 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims Table 2. The certainty of the evidence and the veracity of the claims evaluated. Claim Certainty� Statement# Veracity Desirable claim (claim with a beneficial effect) High Reduces/improves the risk True False Undesirable claim (claim with a harmful effect) Moderate Probably reduces/improves the risk Probably true Probably false Low May reduce/improve the risk Possibly true Possibly false Very low Uncertain whether improves/reduces the risk Uncertain � High certainty means that the authors have a lot of confidence that the true effect is similar to the estimated effect; moderate certainty means that the authors believe that the true effect is probably close to the estimated effect; low certainty means that the true effect might be markedly different from the estimated effect, and very low certainty means that the true effect is probably markedly different from the estimated effect. GRADE certainty ratings taken from BMJ Best Practice. What is GRADE? September 2019. Available: https://bestpractice.bmj.com/info/us/toolkit/learn-ebm/what-is-grade/ # The phrase of the statement should be elaborated taking into consideration the magnitude of the desirable and undesirable effects (see Table 3 and S3 Table). https://doi.org/10.1371/journal.pone.0232393.t002 • Headline: it summarizes the result of the evaluation in a sentence or, in some cases, it poses the question of the evaluation in plain language. • Introduction and contextualization: we provided a brief introduction and contextualization about the claim being analysed. • Conclusion: we stated the result of the evaluation with its corresponding symbol. • Summary: we summarized the evaluation in plain language making the full scientific report available in PDF format. For some claims, we also produced a short video explaining the evaluation [18]. • What does the evidence say? (Qué dice la ciencia): this section provides a reasoned explana- tion of the certainty of the research results. When the result of the evaluation was not uncer- tain, an interactive SoF table was also provided. SoF tables are user-friendly formats to communicate research findings to the public and other stakeholders [19]. • To know more (Para saber más): this section includes relevant resources and links related to the evaluated claim. The Nutrimedia website (https://www.upf.edu/web/nutrimedia) is responsive and multi- layered. It is hosted by Pompeu Fabra University, which provided the technical support for its development and maintenance. Media impact and website traffic analysis We used two main methods to assess the media impact of Nutrimedia: the Factiva database, which includes newspapers, magazines and news agencies from all over the world; and the media monitoring service Acceso 360. The estimated economic value of each piece of news, provided by Acceso 360, was calculated based on the cost of the advertising space it occupies in a particular newspaper. We used Google Analytics to analyse website traffic. Results Nutrimedia was launched in Spanish in November 20, 2017. During the first year and a half, we published 30 evaluations, of which 12 (40%) were questions from the public, 9 (30%) were PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 5 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims common beliefs, 7 (23%) were newspaper claims, and 2 (7%) were advertising claims. Most evaluations were related to raw or processed foods (70%; 21/30), followed by diets (20%; 6/30) and supplements (10%; 3/30); and the majority of them were based on SR (80%; 24/30). Nearly half of the claims were classified as “uncertain” (47%; 14/30), followed by “possibly true” (13%; 4/30), probably true (13%; 4/30), “probably false” (10%; 3/30), “possibly false” (7%; 2/30), false (7%; 2/30) and “true” (3%; 1/30) (Table 3). An example of the evaluation of the veracity of a claim is presented in the S4 Table. Nutrimedia has other contents such as “Eating with science” (Comer con ciencia), “To know more” (Para saber más), and “About Nutrimedia” (Sobre Nutrimedia). The “To know more” section provides short videos to promote critical thinking and to facilitate the under- standing of some methodological concepts (e.g., what is the GRADE approach and how cer- tainty of the evidence was evaluated; guidelines for the public to interpret information on nutrition; guidelines for journalists to report on nutrition; relevant links in the field of nutri- tion, and a glossary). In the section “About Nutrimedia”, users can find out who we are, the scientific methodology that we applied to evaluate claims on nutrition, the press releases and the media coverage. In the section “Ask Nutrimedia”, available from November 20, 2017, to May 5, 2019, the public assessed the interest of 10 questions in an online survey. Of the 12 questions from the public that were evaluated, 6 were the most voted of the 10 closed-ended ques- tions and 6 were posed by users in the survey space for comments and suggestions. Some other questions about nutrition of general interest, most of them posed by the public, were answered by experts in articles and interviews (provided as podcasts) in the “Eating with science” section. The number of online survey respondents was 333 (55.2%, 182/333, gen- eral public; and 45.8%, 151/333, health professionals). Fifty-eight respondents (17.4%, 58/ 333) provided positive (51.7%, 30/58) (e.g., “An excellent idea to combat all the misinfor- mation”; “Thank you for helping me get back to believing in science”) and neutral (5.2%, 3/58%) comments or suggestions (43.1%, 25/58) (e.g., “It would be interesting to receive web updates by email”; “It would be tremendously helpful to give healthy and tasty recipes”). The number of website users was 47,265 (46,052 new users) since the website was launched (November 20, 2017) and for the first 18 months (until May 20, 2019). The number of page views in this period was 181,360. During this period, Nutrimedia was cited by 94 newspapers, news agencies and newswires included in Factiva Dow Jones database; of these 94 citations, 55 were in Spanish, 33 in Catalan, 4 in English and 2 in Portuguese. According to the data pro- vided by Acceso 360, Nutrimedia was cited 78 times by newspapers and 386 times by websites from Spain and 14 other countries, mostly in Latin America. In addition, the Nutrimedia proj- ect and its evaluations were broadcasted on several Spanish radio and television channels. In that period, the estimated economic value of the 78 pieces of news published in printed edi- tions of newspapers was €152,507.55. Nutrimedia has also been cited by prominent Spanish dieticians and nutritionists. For example, Nutrimedia is referenced in 5 of its 12 recommendations for the general public in the 2018 food-based dietary guideline "Small changes to eating better" by the Public Health Agency of Catalonia [20]. Finally, Nutrimedia was well placed in nutrition searches in Spanish with Google, whose algorithm rates the pages according to the criteria of expertise, authority and trustworthiness [21]; e.g., in a Google search with the terms “meat cancer” (search performed on 2019 Septem- ber 30, using a logged-out Chrome browser cleared of cookies and previous search history), Nutrimedia was ranked as the third listed information source in a list headed by the World Health Organization. PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 6 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims Table 3. Type and veracity of the evaluated claims in https://www.upf.edu/web/nutrimedia until May 20, 2019. Claim� Type Intervention Critical outcome(s)# Certainty of Statement(s) Veracity of evidence the claim Moderate alcohol Common Alcohol Breast cancer High Alcohol consumption (in any amount) False consumption is beneficial beliefs increases breast cancer risk to health To stay healthy is better Newspaper Five servings of All-cause mortality Moderate Habitual consumption of vegetables Probably to eat more than five daily claims vegetables and fruits and fruits probably reduces the risk of true servings of fruits and daily all-cause mortality vegetables Palm oil is more harmful Questions Palm oil vs similar fats Cardiovascular disease Very low We are uncertain whether palm oil Uncertain to health than other from the (vegetable oils/ partially (lipids levels) increases the risk of cardiovascular similar fats public hydrogenated oils/ disease in comparison of other similar animal oils) fats Danacol lowers high Advertising Danacol LDL cholesterol High Danacol reduces the level of LDL True cholesterol up to 10% claims cholesterol Antioxidant supplements Common Antioxidant Mortality/ cardiovascular Low Antioxidant supplements Probably prevent diseases beliefs supplements diseases / cancer consumption may reduce the risk of false all-cause mortality, cardiovascular disease and cancer Added sugar to food is Newspaper Added sugar Coronary diseases Moderate Added sugar to food probably Probably harmful to health claims increases the risk of coronary disease true Gluten-free diet is Questions Gluten-free diet Coronary diseases Low Gluten-free diet probably slightly Probably beneficial for the health from the reduces the risk of coronary disease false of healthy adults public Chocolate consumption Newspaper Chocolate Mortality from Very low We are uncertain whether chocolate Uncertain prevents cardiovascular claims cardiovascular diseases/ consumption reduces the risk of disease cardiovascular disease cardiovascular disease and cardiovascular mortality Soy products are effective Common Soy products Menopause symptoms Very low We are uncertain whether soy Uncertain in treating the symptoms beliefs (vasomotor and vaginal products and soy supplements associated with symptoms)/ cognitive consumption reduces the risk of menopause function menopause symptoms Coffee consumption is Newspaper Coffee Mortality/ Mortality from Very low We are uncertain whether coffee Uncertain harmful to health claims cardiovascular disease/ consumption increases the risk of all- cardiovascular disease/ cause mortality, cardiovascular cancer mortality, cardiovascular disease and cancer White bread favors Questions White bread vs whole Body weight / abdominal Low White bread may make little or no Possibly overweight more than from the bread circumference difference to weight and abdominal true whole wheat bread public circumference/ Whole wheat bread may make no difference to weight and abdominal circumference Energy drinks Common Energy drinks and Injuries/ cognitive Moderate Alcohol mixed with energy drinks Probably consumption counteracts beliefs alcohol/ Energy drinks/ function/ and behavioral consumption probably no reduces the false the cognitive effects of Alcohol disorders negative cognitive effects of alcohol alcohol consumption consumption Light food products Common Low-fat foods/ low- Body weight Very low We are uncertain whether low-fat Uncertain consumption reduces beliefs calorie foods (light food foods reduces body weight weight products) Omega-3 fatty acid Questions Omega-3 supplements Dementia Low Omega-3 supplements may make little Probably supplements help prevent from the or no difference to develop dementia false dementias public It’s just as healthy to Common Fruit juices (100% fruit)/ Body weight, diabetes and Very low We are uncertain whether fruit juices Uncertain drink a fruit juice than a beliefs Fruit juices with added cardiovascular risk increases or reduces body weight and whole fruit intake sugar the risk of diabetes and cardiovascular (Continued ) PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 7 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims Table 3. (Continued) Claim� Type Intervention Critical outcome(s)# Certainty of Statement(s) Veracity of evidence the claim Artificial sweeteners are Common Artificial sweeteners Diabetes/ obesity/ satiety Moderate Artificial sweeteners consumption Probably harmful to health beliefs and appetite slightly improves the metabolic control false in diabetics patients, supervised by healthcare professionals/ Artificial sweeteners consumption (instead sugar) slightly reduces (or maintain) body weight in structured weight-loss programs supervised by healthcare professionals/ Artifical sweeteners consumption probably make little or no difference to hormonal regulators of satiety and appetite Vegan diet is beneficial to Questions Vegan diet/ Vegetarian All-cause mortality/ Very low We are uncertain whether vegan diet Uncertain health from the diet cancer reduces the risk of all-cause mortality public and cancer Lactose free milk is easier Advertising Lactose free milk Gastrointestinal No studies No studies were found between lactose Uncertain to digest claims symptoms and diseases free milk consumption and gastrointestinal symptoms and diseases Alkaline diet prevents Questions Alkaline diet Cancer Very low We are uncertain whether alkaline diet Uncertain cancer from the reduces the risk of cancer public Meat is carcinogenic Questions Meat/ red meat/ Colorectal cancer Low Habitual red meat consumption may Possibly from the processed meat increase the risk of colorectal cancer/ true public Habitual processed meat probably increases the risk of colorectal cancer Intermittent fasting is Questions Intermittent fasting Coronary artery disease/ Very low We are uncertain whether intermitent Uncertain beneficial to health from the diabetes fasting reduces the risk of coronary public artery disease and diabetes Breastfeeding prevents Questions Breastfeeding Obesity and overweight Low Breastfeeding may reduce the risk to Possibly obesity from the develop obesity and overweight true public Habitual nuts intake Newspaper Nuts All-cause mortality/ Moderate Habitual nuts intake probably reduces Probably reduces cardiovascular claims cardiovascular disease the risk of all-cause mortality and true risk cardiovascular disease Habitual garlic intake Questions Garlic Cancer Very low We are uncertain whether habitual Uncertain helps prevent cancer from the garlic intake reduces the risk to public develop cancer Mediterranean diet Newspaper Mediterranean diet/ Depression Very low We are uncertain whether Uncertain reduces the risk of claims other healthy diets mediterranean diet and other healthy depression diets reduce the risk of depression Vitamin D supplements Common Vitamin D suplements Fracture High Vitamin D no reduces the risk of False reduce the risk of fracture beliefs fracture Dairy intake helps Newspaper Dairy Cardiovascular disease Low Dairy intake (>two servings/day vs Possibly prevent cardiovascular claims zero) may reduce the risk of true disease cardiovascular disease To lose weight, it is better Questions Olive oil Body weight Moderate Olive oil consumption (instead other Probably to consume olive oil than from the oils or fats) probably reduces body true other oils public weight Daily egg consumption Common Egg Cardiovascular disease Low We are uncertain whether habitual egg Uncertain increases the risk of beliefs consumption (one serving/day) cardiovascular diseases increases the risk of cardiovascular disease (Continued ) PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 8 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims Table 3. (Continued) Claim� Type Intervention Critical outcome(s)# Certainty of Statement(s) Veracity of evidence the claim Ecological foods intake is Questions Ecological foods Cancer Very low We are uncertain whether ecological Uncertain beneficial for health from the foods consumption (instead public conventional foods) reduces the risk of cancer � Chronologically ordered from the oldest to the newest published on the web. # Only those outcomes that we deemed as critical. https://doi.org/10.1371/journal.pone.0232393.t003 Discussion Nutrimedia is the first web-based resource for the general public that evaluates the veracity of media claims about nutrition based on the certainty of evidence, and to communicate the results in plain language and friendly presentation formats. We have evaluated 30 claims and classified nearly half of them as uncertain based on GRADE. During the first year and a half, Nutrimedia had a considerable impact on the media (about one impact per day). There are some other online resources with different purposes and target populations. “Practice-based evidence in nutrition” provides the latest evidence in practice-based nutrition questions applying the GRADE approach, but this evidence-based decision support system is intended for dieticians/nutritionists and students [21]. “Behind the Headlines”, from the UK National Health Service, analyses critically media claims related to nutrition, but does not assess the certainty of the evidence [22]. Cochrane Nutrition provides nutrition-related Cochrane systematic reviews, but there is often a communication gap between these reviews and the public [23]. To avoid this, Cochrane centers implement alternative ways to make evi- dence accessible in plain language summaries [24] and blogshots (infographics that summarize the evidence) in English [25], Spanish [26] and other languages. By overcoming these limita- tions, Nutrimedia is an innovative resource for disseminating quality nutrition information, and making it accessible to the public. Therefore, we believe that Nutrimedia is a pioneering initiative that promotes critical thinking and can have an impact on the food and nutrition lit- eracy of the general public [27]. Nutrimedia has several strengths. The website is user-friendly because no registration is required, the evaluations are accessible within a few clicks and its content is multi-layered and multiformat. For each evaluation, we have applied a rigorous and explicit methodology using GRADE. GRADE approach represents a systematic, explicit and transparent methodological framework for grading the certainty of evidence and it has already been endorsed or used by over 100 organizations, the World Health Organization and the Cochrane Collaboration [28– 29] among others. Finally, our project team has extensive knowledge and experience in the fields of nutrition, evidence-based medicine, methods, communication and journalism. Nutrimedia also has some limitations. Firstly, it is only available in Spanish. Secondly, we used pragmatic search strategies favoring precision over sensitivity [30]. Thirdly, this project has no system yet for monitoring and continuous updating of the evaluations. Finally, analysis of its impact, usefulness, accessibility and understandability is still limited. Implications for practice and research General public, journalists and communicators can use Nutrimedia to stay informed and mak- ing informed decisions about nutrition. Researchers interested in evaluating topics about PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 9 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims nutrition can use our approach. More research is needed about the impact, usefulness, accessi- bility and understandability of Nutrimedia. Supporting information S1 Table. Resources and search strategies. (DOCX) S2 Table. Key reporting aspects of the evidence. (DOCX) S3 Table. Standardised statements about effect according to the GRADE approach. (DOCX) S4 Table. Text box. An example of a scientific evaluation of a nutrition claim. (DOCX) Acknowledgments We would like to thank Darı́o Lopez, Andrea Juliana Sanabria, Mónica Ballesteros, Carolina Requeijo Lorenzo, Karla Salas Gama, Paulina Fuentes, Laura Martı́nez Garcı́a, and Alba Irigo- yen for their contribution in this project. We would also like to thank to Victoria Leo for her assistance with the English text. Author Contributions Conceptualization: Montserrat Rabassa, Pablo Alonso-Coello, Gonzalo Casino. Data curation: Montserrat Rabassa, Pablo Alonso-Coello, Gonzalo Casino. Formal analysis: Montserrat Rabassa, Pablo Alonso-Coello, Gonzalo Casino. Funding acquisition: Gonzalo Casino. Investigation: Montserrat Rabassa, Pablo Alonso-Coello, Gonzalo Casino. Methodology: Montserrat Rabassa, Pablo Alonso-Coello, Gonzalo Casino. Project administration: Gonzalo Casino. Resources: Pablo Alonso-Coello, Gonzalo Casino. Supervision: Pablo Alonso-Coello, Gonzalo Casino. Validation: Gonzalo Casino. Writing – original draft: Montserrat Rabassa. Writing – review & editing: Pablo Alonso-Coello, Gonzalo Casino. References 1. World Health Organization (WHO). Noncommunicable diseases. Key facts. 2018 June 1 [cited 2019 Oct 25]. In: WHO website [Internet]. https://www.who.int/news-room/fact-sheets/detail/ noncommunicable-diseases 2. GBD 2017 Diet Collaborators. Health effects of dietary risks in 195 countries, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2019; 393: 1958–1972. https://doi.org/ 10.1016/S0140-6736(19)30041-8 PMID: 30954305 3. Mozaffarian D. Dietary and policy priorities for cardiovascular disease, diabetes, and obesity: A compre- hensive review. Circulation. 2016; 133: 187–225. https://doi.org/10.1161/CIRCULATIONAHA.115. 018585 PMID: 26746178 PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 10 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims 4. Helm J, Jones RM. Practice Paper of the Academy of Nutrition and Dietetics: Social Media and the Die- tetics Practitioner: Opportunities, Challenges, and Best Practices. J Acad Nutr Diet. 2016; 116: 1825– 1835. https://doi.org/10.1016/j.jand.2016.09.003 PMID: 27788767 5. Viswanath K, Bond K. Social determinants and nutrition: reflections on the role of communication. J Nutr Educ Behav. 2007; 39: S20–4. https://doi.org/10.1016/j.jneb.2006.07.008 PMID: 17336801 6. Nagler RH. Adverse outcomes associated with media exposure to contradictory nutrition messages. J Health Commun. 2014; 19: 24–40. https://doi.org/10.1080/10810730.2013.798384 PMID: 24117281 7. Williams G, Hamm MP, Shulhan J, Vandermeer B, Hartling L. Social media interventions for diet and exercise behaviours: a systematic review and meta-analysis of randomised controlled trials. BMJ Open. 2014; 4: e003926. https://doi.org/10.1136/bmjopen-2013-003926 PMID: 24525388 8. Gholizadeh Z, Papi A, Ashrafi-Rizi H, Shahrzadi L, Hasanzadeh A. Quality evaluation of Persian nutri- tion and diet therapy websites. J Educ Health Promot. 2017; 6: 48. PMID: 28616415 9. Kininmonth AR, Jamil N, Almatrouk N, Evans CEL. Quality assessment of nutrition coverage in the media: a 6-week survey of five popular UK newspapers. BMJ Open. 2017; 7: e014633. https://doi.org/ 10.1136/bmjopen-2016-014633 PMID: 29284712 10. Cooper BE, Lee WE, Goldacre BM, Sanders TA. The quality of the evidence for dietary advice given in UK national newspapers. Public Underst Sci. 2012; 21: 664–73. https://doi.org/10.1177/ 0963662511401782 PMID: 23832153 11. Quaidoo EY, Ohemeng A, Amankwah-Poku M. Sources of nutrition information and level of nutrition knowledge among young adults in the Accra metropolis. BMC Public Health. 2018; 18: 1323. https:// doi.org/10.1186/s12889-018-6159-1 PMID: 30497442 12. Ostry A, Young ML, Hughes M. The quality of nutritional information available on popular websites: A content analysis. Health Educ Res. 2008; 23: 648–55. https://doi.org/10.1093/her/cym050 PMID: 17897928 13. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Hand- book for Systematic Reviews of Interventions version 6.0. 2019 August [cited 2019 Oct 25]. In: Cochrane website [Internet]. www.training.cochrane.org/handbook 14. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336: 924–6. https://doi.org/10.1136/bmj.39489.470347.AD PMID: 18436948 15. GRADE working group. Windows-based GRADEpro 3.0 software. 2015 [cited 2019 Oct 25]. In: GRA- DEpro GDT website [Internet]. http://gdt.guidelinedevelopment.org/app/ 16. Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, et al. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. J Clin Epidemiol. 2013; 66: 158–72. https://doi.org/10. 1016/j.jclinepi.2012.01.012 PMID: 22609141 17. Nutrimedia. Mensajes Evaluados. ¿Es realmente cancerı́gena la carne? 2019 [cited 2 March 2020]. In: Nutrimedia website [Internet]. https://www.upf.edu/web/nutrimedia/-/comer-carne-puede-elevar- ligeramente-el-riesgo-de-cancer-pero-este-riesgo-es-muy-inferior-al-de-fumar-o-consumir-alcohol#. Xlzh5df0nbi 18. Nutrimedia. Para saber más. Vı́deos para el pensamiento crı́tico. 2019 [cited 2 March 2020]. In: Nutri- media website [Internet]. https://www.upf.edu/web/nutrimedia/videos-para-el-pensamiento-critico 19. Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol 2010; 63: 620e6 20. Public Health Agency of Catalonia. Health Department. “Small changes to eat better” [Internet]. Public Health Agency of Catalonia. 2018 Nov. [cited 2019 Oct 25]. http://salutpublica.gencat.cat/web/.content/ minisite/aspcat/promocio_salut/alimentacio_saludable/02Publicacions/pub_alim_salu_tothom/Petits- canvis/La-guia-peq-cambios-castella.pdf 21. Google. Search quality general guidelines. 2019 Dec 5 [cited 2020 Feb 24]. In: Google [Internet]. https://static.googleusercontent.com/media/guidelines.raterhub.com/en// searchqualityevaluatorguidelines.pdf 22. Practice-Based Evidence in Nutrition (PEN). The global resource for nutrition practice. 2019 [cited 2019 Oct 25]. In: PEN website [Internet]. http://www.pennutrition.com/index.aspx 23. National Health Service (NHS). Behind the headlines: food and diet. 2019 [cited 2019 Oct 25]. In: NHS website [Internet]. https://www.nhs.uk/news/food-and-diet/?order=newest 24. Cochrane Nutrition. Evidence. Cochrane nutrition reviews. 2019 [cited 2019 Oct 25]. In: Cochrane Nutri- tion website [Internet]. https://nutrition.cochrane.org/evidence 25. Cochrane UK. Blogshot & infographic archive. 2019 [cited 2019 Oct 25]. In: Cochrane UK website [Inter- net]. http://uk.cochrane.org/blogshot-infographic-archive PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 11 / 12 PLOS ONE Nutrimedia: A web-based resource on the veracity of nutrition claims 26. Cochrane Iberoamérica. Blogshots. 2019 [cited 2019 Oct 25]. In: Cochrane Iberoamérica website [Inter- net]. http://es.cochrane.org/blogshots_temas#n 27. Yuen EYN, Thomson M, Gardiner H. Measuring Nutrition and Food Literacy in Adults: A Systematic Review and Appraisal of Existing Measurement Tools. Health Lit Res Pract. 2018; 2: e134–e160. https://doi.org/10.3928/24748307-20180625-01 PMID: 31294289 28. Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. [GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction]. Gac Sanit. 2018; 32: 166.e1–e10. 29. Alonso-Coello P, Rigau D, Sanabria AJ, Plaza V, Miravitlles M, Martinez L. Quality and strength: the GRADE system for formulating recommendations in clinical practice guidelines. Arch Bronconeumol. 2013; 49: 261–7. https://doi.org/10.1016/j.arbres.2012.12.001 PMID: 23434203 30. Martinez Garcia L, Sanabria AJ, Araya I, Lawson J, Sola I, Vernooij RW, et al. Efficiency of pragmatic search strategies to update clinical guidelines recommendations. BMC Med Res Methodol. 2015; 15: 57. https://doi.org/10.1186/s12874-015-0058-2 PMID: 26227021 PLOS ONE | https://doi.org/10.1371/journal.pone.0232393 April 30, 2020 12 / 12