Science & Society Involving society in science Reflections on meaningful and impactful stakeholder engagement in fundamental research Helen Garrison1, Marta Agostinho2, Laura Alvarez3, Sofie Bekaert4 , Luiza Bengtsson5, Elisabetta Broglio6 , Digna Couso7 , Raquel Arau jo Gomes8 , Zoe Ingram5, Emma Martinez9, Ana Lucia Mena8 , Dörthe Nickel10, Michael Norman9, In^es Pinheiro11 , Marta Solıs-Mateos6 & Michela G Bertero6,12,* T he concepts of Open Science and (Burns et al, 2021). This can include How can stakeholder engagement be Responsible Research and Innovation increased mutual trust and mutual learning, achieved? call for a more transparent and improved social relevance of research, and collaborative science, and more participation improved adoption of results and knowl- Importantly, the recent COVID-19 crisis has of citizens. The way to achieve this is through edge (Cottrell et al, 2014). Finally, stake- emphasized the need for improving public cooperation with different actors or “stake- holder engagement is often regarded as an understanding of the scientific process. The holders”: individuals or organizations who important factor to sustain public invest- time needed between fundamental discov- can contribute to, or benefit from research, ment in the life sciences (Burns et al, 2021). eries and application, and the evolution of regardless of whether they are researchers ...................................................... scientific knowledge through questioning, “Stakeholder engagement in themselves or not. Examples include funding revisiting and self-correcting acquired agencies, citizens associations, patients, and knowledge, are concepts that need to be policy makers (https://aquas.gencat.cat/ research can inform and broadly communicated. Greater public web/.content/minisite/aquas/publicacions/ understanding of the scientific process will empower citizens, help render 2018/how_measure_engagement_research_ not only contribute to openness and saris1_aquas2018.pdf). Such cooperation is research more socially accept- increased trust in science, but also help to even more relevant in the current, challenging able and enable policies develop critical, analytical, and transparent times—even apart from a global pandemic— grounded on evidence-based attitudes. when pseudo-science, fake news, nihilist atti- knowledge” Advocates for public engagement have tudes, and ideologies too often threaten social ...................................................... been arguing for decades about the need to and technological progress enabled by overcome a deficit view, according to science. Stakeholder engagement in Here, we discuss different levels of stake- which the public lacks sufficient informa- research can inform and empower citizens, holder engagement by way of example, tion about science and technology. Instead, help render research more socially accept- presenting various activities organized by to increase scientific literacy, encourage able, and enable policies grounded on European research institutions. Based on participation, and foster public acceptance evidence-based knowledge. Beyond, stake- these experiences, we propose ten reflection of science, engagement should be a holder engagement is also beneficial to points that we believe should be considered dialogue in which different stakeholders researchers and to research itself. In a by the institutions, the scientists, and the actively participate (Stilgoe et al, 2014). recent survey, the majority of scientists funding agencies to achieve meaningful and Another development is the move away reported benefits from public engagement impactful stakeholder engagement. from trying to achieve consensus to 1 Vetenskap & Allm€ anhet, Stockholm, Sweden 2 EU-LIFE Alliance of Research Institutes Advocating for Excellent Research in Europe, Barcelona, Spain 3 CeMM Research Center for Molecular Medicine of the Austrian Academy of Sciences, Vienna, Austria 4 VIB, Ghent University, Ghent, Belgium 5 Max Delbru €ck Center for Molecular Medicine in the Helmholtz Association (MDC), Berlin, Germany 6 Centre for Genomic Regulation (CRG), Barcelona Institute of Science and Technology, Barcelona, Spain 7 Universitat Auto noma de Barcelona, Barcelona, Spain 8 Instituto Gulbenkian de Ci^encia (IGC), Oeiras, Portugal 9 Babraham Institute, Cambridge, UK 10 Institut Curie, PSL Research University, Paris, France 11 Institut Curie, PSL Research University, Nuclear Dynamics Unit, Paris, France 12 University Pompeu Fabra (UPF), Barcelona, Spain *Corresponding author. E-mail: michela.bertero@crg.eu DOI 10.15252/embr.202154000 | EMBO Reports (2021) 22: e54000 ª 2021 The Authors Published under the terms of the CC BY 4.0 license EMBO reports 22: e54000 | 2021 1 of 7 EMBO reports Helen Garrison et al eliciting and accepting diverse views Examples of stakeholder engagement communication activities provided by the (Mohr, 2008). Stakeholder engagement can across different levels institutions involved in writing this paper thus help to acquire scientific knowledge, was much larger. For example, we specifi- create attitudes that value science as part of Here, we present examples of stakeholder cally omitted important, but commonly cultural development, and enable an active engagement in the life sciences at different undertaken school outreach activities. role of citizens and scientists in social levels and on different scales, from informing debates (Godin & Gingras, 2000). to collaborating and empowering. Stakehold- Inform ...................................................... ers can represent diverse groups of people, “Advocates for public engage- from patients, artists, pupils at school, policy Even if informing—whereby researchers makers, or other researchers to citizens in disseminate knowledge about discoveries ment have been arguing for general. The selected examples demonstrate and technologies in an unbiased way—repre- how this layering of engagement approaches sents the first step of engagement, it is still a decades about the need to over- can be used to engage citizens of any back- necessary activity. Informing often presents come a deficit view, according ground. Through identification and discussion fewer barriers to non-expert audiences and to which the public lacks suffi- of the benefits and challenges, we propose acts as an entry point to more engagement. cient information about science ten reflection points to achieve meaningful At the Research Center for Molecular and technology.” and impactful stakeholder engagement. Medicine (CeMM) of the Austrian Academy ...................................................... The examples—activities carried out by of Sciences, researchers and science commu- several European research institutes that nicators developed a Virtual Reality App There are multiple frameworks to define participate in the EU-LIFE alliance and dif- specifically to promote the research at the stakeholder engagement in research. To better ferent Europe-wide projects, such as LifeTime institute. At its launch, CeMM created a characterize the examples, we adopted Gabriele Initiative and ORION Open Science—high- printed quiz in the form of a card game that Bammer’s model, which defines different levels light the main outcomes, the challenges, the was distributed widely during the “BE OPEN of engagement with different stakeholders as main contributions by researchers, and stake- —Science & Society Festival” in September well as the responsibilities of researchers at holders as well as any potential tensions. 2018 (https://www.fwf.ac.at/en/about-the- each level (https://i2insights.org/2020/01/07/ They were chosen based on their originality, fwf/be-open-science-society-festival). More research-modified-iap2-spectrum/). It encom- their success, and how well they represent than 30,000 citizens in total attended the passes five stages from informing—one- the different levels of engagement. Although festival (Fig 1). There CeMM team members way communication—to collaborating and the selected framework helps to categorize invited visitors to download the app on their empowering via two-way or even “multi- different activities, we recognize that engage- phone and play the quiz, using the cards as way” dialogue (Table 1). It should be ment represents a continuum, from unidirec- triggers, which generated good feedback. stressed that different levels often overlap tional communication to two-way dialogue, Since, the app has been further developed and complement each other and that all are and many of the activities fall within multiple into an Augmented Reality version to teach of value depending on the specific objec- categories and represent multiple levels of younger audiences about the building blocks tives and audiences. engagement. In fact, the portfolio of science of life (DNA, proteins, cells) and provide Table 1. The table illustrates the different levels of stakeholder engagement, from informing to empowering, as well as researchers’ responsibilities at each level. Inform Consult Involve Collaborate Empower Stakeholder Researchers provide Researchers obtain Researchers work directly Researchers partner Researchers assist participation stakeholders with stakeholder with stakeholders to with stakeholders for stakeholders in goal balanced and feedback on ensure that stakeholders salient aspects of conducting their own objective the research concerns and aspirations the research research information are consistently understood to assist them in and considered in the understanding research the research Promise made to We will keep We will keep you informed, We will work with you We will look to you We will provide stakeholders from you informed listen to and acknowledge to ensure your concerns for advice and advice and assistance researchers your concerns and aspirations are directly innovation in as requested in line and aspirations reflected in the research, designing and with your decisions for and provide feedback on and we will provide conducting the research designing and how your input influenced feedback on how your and incorporate your conducting your the research input influenced the advice and research, as well as research recommendations for implementing the to the maximum findings extent possible Source: Gabriele Bammer 2 of 7 EMBO reports 22: e54000 | 2021 ª 2021 The Authors Helen Garrison et al EMBO reports Lisbon, which is one of the top European festivals. IGC scientists and science commu- nicators at the festival offer a broad range of activities, from speed-dating with scientists to hands-on activities, games, exhibitions, and demonstrations. A different theme is selected each year and specific activities designed accordingly. For example, in 2018, a darkroom was set up to create an environ- ment where visitors could view biological samples under fluorescence microscopes and bioluminescent microorganisms. Over the three days of the festival, an average of 1,500 visitors interacted with IGC scientists, mostly teenagers, and young adults who can be otherwise difficult to reach through tradi- tional school outreach activities. Consult Consulting, the next level of stakeholder Figure 1. “FWF BE OPEN” Science & Society Festival in Vienna, 2018. © Hans Leitner/CeMM. engagement, helps to inform research deci- sions, whether at a project or institutional level, and to strengthen relationships and users with insight into the human body as intensive. Yet, comments and questions trust. they zoom into the cell all the way down to received on YouTube indicated that it In 2019, the Babraham Institute in the genome. Unfortunately, the number of successfully conveyed the concentrated and Cambridge, UK, conducted a two-stage public people reached has not been as high as yet fun atmosphere of a life science lab. dialogue on genome editing in fundamental hoped for, due to a lack of funding and Research work was demystified (“not much research. The first stage included develop- human resources to support the continued action”), biological concepts, and research ment of the specification and methodology, development. Nevertheless, the tool success- explained, and career pathways to science followed by review and consultation with fully served two purposes: informing others were showcased. It also provided uncensored stakeholder representatives to set expecta- about the research being carried out at the access to restricted areas and presented the tions, establish a common approach that institute and educating students about scientists as they really are: a diverse, inter- would suit all stakeholders, and develop the molecular biology. national group of people with ideas, dedica- materials for the second stage. The second Inspired by the popularity of reality TV, tion, and hobbies (https://www.youtube. stage involved a deliberative workshop in the Max-Delbr€ uck-Center for Molecular com/watch?v=iNCZUEk3JGA&t=13s). Cambridge over one and a half days. An Medicine (MDC) in Berlin, Germany, orga- ...................................................... agency specialized in market research and “Each day from 9 am to 6 pm, nized a live stream on YouTube from their public consultation recruited the participants laboratories as part of the 2020 Berlin to ensure they were nationally representative. Science Week. Each day from 9 am to 6 pm, scientists were filmed during The recruitment of experts and scientists to scientists were filmed during their work with participate in the workshops was challenging, their work with cameras rolling cameras rolling non-stop, even though, at as they had to donate a total of 12 h of their times, the lonely centrifuge whirring away non-stop, even though, at scarce free time without receiving any was the only thing on display. The live TV times, the lonely centrifuge compensation. format was interjected with special features, whirring away was the only The dialogue was organized as part of the such as interviews with scientists and other thing on display” European ORION Open Science project with staff, Q&A sessions with the public, plus ...................................................... the overall aim to explore public attitudes on “scientainment” events, such as Lab Olym- genome editing, specifically within the context pics (“How fast can you pipet?”) and a Often it is challenging to reach audiences of fundamental research (https://www.orion- music concert in the lab, for which lab that are not interested in science. To address openscience.eu/publications/reports-papers/ equipment was used as music instruments. this, the Instituto Gulbenkian de Ci^encia 202103/public-attitudes-genome-editing-life- While around 150 people were watching at (IGC) in Lisbon, Portugal, has been taking sciences-research). Another important objec- any given moment, the total reach was science to music festivals (Le~ ao, & Castro, tive was to understand how to better engage about 3,500 people, which is comparable to 2012). This initiative has been made possi- people from different backgrounds with a regular open day at the MDC. ble due to a partnership between the IGC potentially controversial scientific topics. The Reality TV experiment was techni- and “Everything is New”, the promoter of All scientists who participated valued the cally challenging and time and resource the NOS Alive Music Festival in Alges, near opportunity to have in-depth conversations ª 2021 The Authors EMBO reports 22: e54000 | 2021 3 of 7 EMBO reports Helen Garrison et al with members of the public. One of them participating in the scientific process. The of the latest technologies and two further stated that “We need to acknowledge the institutions promoting the forum found that public dialogues on future priority research knowledge of the public participants and the initiative gave them a greater insight into topics. recognise how it complements the exper- citizens’ views, opinions, and wishes, and Another example is the Embodying tise of the researchers to come to conclu- enabled them to gather unique innovative Memories project by the IGC (Matias et al, sions on how new technology can be ideas to reach different audiences. However, 2021). Using science and art, a multidisci- introduced in a way that benefits society promoting such forums can be very costly plinary team of science communicators, as a whole”. and time-consuming for all parties and it is researchers, and art education professionals If communication about science only essential that the goals are clearly defined designed a co-creation project together with pulls in those who are already interested, it from the start to make sure that the citizen a community at risk of social exclusion. The fails to reach out to people who are most deliberative forum is the best tool. Finally, main objectives were to increase awareness vulnerable to scientific misinformation. In follow-up of the proposals’ implementation and engagement in science, and to improve order to address this problem, the IGC, the needs to be periodically communicated to the community’s willingness to participate Instituto de Tecnologia Quimica e Biol ogica participants and the general public to avoid in new experiences, rather than aiming to in Lisbon (ITQB NOVA), and the Oeiras creating disillusionment. increase the target audience’s knowledge of municipality in Portugal organize a series of a certain topic. The project was tailored to annual citizen deliberative forums. The Involve the needs and interests of the audience, a focus of the first forum, held in February community of 14 women, aged between 64 2020, was how to make science accessible to The next level of engagement is involve- and 84 with low literacy levels and socioeco- citizens and involve citizens in science. Prior ment, whereby researchers collaborate with nomic status. to the forum, an open “Idea Contest” was stakeholders, to listen to their concerns and During the design phase, the communica- promoted through social media, posters, and aspirations and ensure these are addressed. tors met with the participants and from fliers distributed through schools, senior This was the rationale for an online these conversations the topic of universities, and local associations in Oeiras. public dialogue on research strategy orga- neuroscience emerged. The implementation Ten of the 30 submitted ideas were then nized by the Centre for Genomic Regulation consisted of six indoor and two outdoor selected by a jury. These ideas were later (CRG) in Barcelona, Spain. The dialogue sessions. Each indoor session explored a dif- presented to the participants in the forum, sought to gain opinions on fundamental ferent aspect of neuroscience, with different which lasted two days and involved 30 citi- research and discuss ethical and societal activities developed for each. The two zens selected to reflect the demographics of aspects with representatives from civil soci- outdoor sessions consisted of guided tours Oeiras. During an initial learning phase, ety and other stakeholders. The objective to an art museum (the Gulbenkian Museum participants heard from three experts about was to explore how to address societal in Lisbon) and a research center (the IGC). a variety of approaches to science communi- views and concerns in the next CRG strate- For the participants, this was their first expe- cation and public engagement and were also gic plan (2021–2024). Thirty-one citizens, rience of a museum and a research center, presented with the 10 winning ideas from who were selected by a market research and the visits were greatly appreciated. The the contest. With this knowledge and infor- company to reflect a representative sample project results suggest that, in the short mation, and based on their own personal of the Spanish population and who were term, tailored science engagement programs experiences, participants discussed in small financially compensated for participating, can be effective in reaching and involving groups and designed their own proposals. In and 22 other stakeholders participated in a socially excluded publics. Yet, for medium- the final deliberation phase, the participants 13-day online dialogue, owing to the term impact, such communities need to be narrowed down the proposals to three that COVID-19 pandemic. further supported to facilitate access to were then presented to a panel of decision- The exercise was highly valued by both cultural and scientific experiences. makers, who gave feedback and discussed the participants, who showed great interest ...................................................... possibilities for implementation. in CRG research and operations and raised a ...................................................... number of relevant issues and expectations, “The project results suggest and the scientists, who found the experience that, in the short term, tailored “If communication about very enriching and altered their perception science engagement science only pulls in those who of how the public views them. Interestingly, some members of CRG’s senior management programmes can be effective in are already interested, it fails initially expressed skepticism about involv- reaching and involving socially to reach out to people who are ing citizens in the CRG’s strategic plan. excluded publics” most vulnerable to scientific However, the organizers remained focused ...................................................... misinformation” on the initial objectives of the dialogue and ...................................................... sought to respond to participants’ opinions Chronic liver disease is a major cause of and requests, which thereby resulted in a morbidity and mortality worldwide; non- Subsequent evaluation highlighted that high level of engagement. Importantly, the alcoholic steatohepatitis (NASH) is an the participants valued having their voices dialogue led to two key new actions being advanced type of non-alcoholic fatty liver heard and were particularly interested in incorporated in the CRG’s strategic plan: a disease (NAFLD) which can lead to cirrhosis learning more about science and series of regular talks on ethical implications associated with liver fibrosis and progressive 4 of 7 EMBO reports 22: e54000 | 2021 ª 2021 The Authors Helen Garrison et al EMBO reports loss of function and increased risk of hepato- cellular carcinoma. Early detection of NASH is key for prevention through lifestyle change and/or therapeutic intervention. The “Improving liver disease diagnosis” project, funded by the VIB Grand Challenges program in Ghent, Belgium, collected patient samples for analysis and storage to search for prognostic biomarkers for liver diseases. This project was inspired by the outcome of a previous initiative by the King Baudouin Foundation, in which several project part- ners were involved in a multi-stakeholder dialogue initiative with physicians, citizens, patients, and biobank experts to set priori- ties within research on liver diseases and to discuss how biobanks could effectively help leverage this research (Raeymaekers, 2019). The results informed the VIB-liver project in a number of ways to identify relevant research questions. Collaborate Figure 2. Co-creation event for the GENIGMA game with researchers, representative from The GENIGMA citizen science project, devel- patients’ associations, storytellers, science communicators, and art creators. © Elisabetta Broglio/ oped as part of ORION Open Science, is an CRG. example of productive and intense collabo- ration between researchers and citizens from different interest and expert groups. “everything is everywhere”, the Belgian of longevity and CRISPR”: to give citizens GENIGMA is a game for smartphones to “Soy-in-1000-gardens” project aimed to the impetus and the information to reflect investigate 3D genomic structures and alter- engage 1,000 citizens to grow soy in their on applications of the disruptive genome- ations in cancer cells. From the very start, gardens to “trap” nitrogen-fixating bacteria in editing technology CRISPR/Cas9. Even the game itself was conceived with the help their root nodules for further analysis. Partici- though this was a fictional situation, it was of teachers, representatives of patient associ- pants were recruited via “Mijn Tuinlab”, a an interesting exercise for critical thinking to ations, clinicians, designers, communicators, platform for citizen science projects and via an empower the public to reflect on this specific and gamers—more than 120 people partici- official launch and its press coverage. More question. pated in three different events (Fig 2). As than 5,000 citizens applied and 1,154 were ...................................................... the game developed, more citizens started to actively collaborate in the research project, selected and provided with seeds. A digi- tal interface collects data related to the growth “The holy grail of stakeholder initially by helping to test the validity of the of the soy plants. Additionally, the project also engagement is to empower data analysis pipeline and the game’s func- tested the effect of specific engagement actions citizens to make fact-based tionality and later by analyzing the data, (communication, information, etc.) on partici- decisions” while they played. Despite initial skepticism pants’ overall awareness and attitude toward and concerns about this highly collaborative sustainable gardening, food consumption, and ...................................................... approach, scientists subsequently recog- agriculture. Importantly, the project also ÆON describes a futuristic scenario in nized the value of including different included 100 farmers, who are introducing soy which a CRISPR-based rejuvenator exists. It perspectives. Participants’ feedback indi- as a crop in Flanders. Communication with gives the viewer a glimpse into the conse- cated they were very happy to contribute participants is key to the success of this kind quences of using vs. not using it by portray- their time and “non-scientific” skills from of citizen science projects, and keeping the ing a couple that made opposite decisions. the very beginning while their main motiva- participants’ community engaged is one of the The art piece shows what the couple looks tion for participating was being able to help main challenges. like, how they interact with each other and cancer research. with death, and forces the viewer to reflect Soy is a subtropical crop ill adapted to Empower on their own position. Through the addi- Northern European climatic conditions. One tional display of the scientific proof-of- major issue is the lack of (commercially The holy grail of stakeholder engagement is concept work, which shows that such a available) nitrogen-fixing bacteria in the soy to empower citizens to make fact-based deci- device, or at least a method for rejuvenation plants’ root nodules, which are necessary to sions. This was the ambition of the ORION at the molecular level, could actually achieve acceptable yields. Under the call Science and Art project ÆON “Trajectories become reality, the viewer is also forced to ª 2021 The Authors EMBO reports 22: e54000 | 2021 5 of 7 EMBO reports Helen Garrison et al think about the personal, social, and ethical implications. ÆON was developed as part of an arts residency hosted by the MDC. Following an international Open Call process, the artist Emilia Tikka was selected to spend three months in the MDC labs, where, assisted by scientists, she carried out her experi- ments to produce a proof of concept for her futuristic scenario. The collaboration for ÆON was intense and a proof in itself that facilitation by a public engagement profes- sional is key to success. In fact, this was the main challenge of the project: finding a common language and developing an under- standing of each other’s thinking as artists and scientists view the world differently and have different communication styles. Facilita- tion needs to be factored in for such projects to be successful: in the budget, in the project planning and the external communication activities. ÆON has been on display in art galleries and exhibitions throughout Europe and received considerable press coverage. The art was also used as an impulse for public Figure 3. Launch of the ÆON exhibition at the National Museum of Science and Technology in dialogues on genome editing in Sweden, Stockholm, 2020. © Ben Libberton/Vetenskap & Allma€nhet. Germany, Czechia, and the UK within the ORION Open Science project. The art piece was well received by the public and sparked lively discussions on what basic science can and many projects may actually achieve dif- and events such as a music festival, or and cannot achieve in terms of societal deci- ferent levels. Citizens are far better empow- engaging market research companies to sions (Fig 3). ered to collaborate in the scientific process recruit a representative group of partici- when they have been well informed, pants. Reflections consulted, and involved. It is interesting to note that sometimes the level of engagement Reflection 4. We cannot continue to Based on our experience with public and may increase during a project and even pretend that “talk is cheap” and rely on stakeholder engagement in the life sciences, surpass the expected level. This was the case minimum efforts and investments. Some we have identified ten reflection points for at the CRG, where the dialogues were initi- activities can be costly and/or might researchers and institutions who are inter- ally planned as a consultation activity but require external expertise. Ideally, institutes ested in engaging the public and other stake- developed toward the involvement level. will have funding to support stakeholder holders in their research. engagement; in other cases, funding can be Reflection 1. A fundamental aspect is that Reflection 3. It is important to reflect on the sought through national, European, or stakeholder engagement activities must be motivations of the target audience and the international calls. Time and flexibility in framed within the strategy of the research researchers who participate, and offer appro- research grants are also important factors, institute or the leading organization. Senior priate incentives to ensure their engagement. particularly when stakeholder insights can management, public engagement experts, A stakeholder mapping exercise should iden- influence the direction of research projects and scientists need to first reflect on what tify the key stakeholder groups according to (Lavery, 2018). they want to achieve to define the objec- their goals, motivations, expertise, and inter- tives, the stakeholders with whom to inter- ests; it is not sufficient to decide to involve Reflection 5. It is important to be clear act, different engagement activities and “the public” or “citizens” as they are a very about the expected level of stakeholder necessary resources, such as personnel and heterogeneous group. In addition, it is engagement in order to manage expectations funding. It helps to obtain commitment and important to recognize that some stakehold- and ensure credibility. Moreover, it is likely buy-in at all levels of the institution. ers are harder to engage than others and to that the number of stakeholders who can be think creatively about how to reach them, engaged will decrease as the engagement Reflection 2. The different levels of engage- for example, partnering with civil society deepens. This is the case, for example, with ment should be considered as a continuum, organizations, running activities at venues public dialogues that involve a smaller 6 of 7 EMBO reports 22: e54000 | 2021 ª 2021 The Authors Helen Garrison et al EMBO reports number of citizens to ensure meaningful beforehand and mitigated by involving Catalunya. We acknowledge funding from conversations and exchange of ideas. experts. European Commission (EC): 741527. Reflection 6. Another critical aspect is trans- Reflection 9. Stakeholder engagement activ- References parent communication of the results back to ity requires certain skills, and participating Burns JA, Holden S, Korzec K, Dorris ER (2021) the participants. Specifically in the case of researchers should be provided with suitable From intent to implementation: factors public dialogues, citizen science projects, or training along with explanations of what is affecting involvement in life science research. activities where stakeholders’ involvement expected, what can happen and how it can PLoS One 16: e0250023 and opinions can impact the course of the be dealt with. If researchers fear being Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S, research project, the participants should be misinterpreted or that their message will be Chang C, Hoomans T, Meltzer D, Noorani H, kept informed. Recognizing the value of decontextualized for political/ideological Robinson K et al (2014) Defining the benefits of participants’ contributions and providing reasons, specific support and training should stakeholder engagement in systematic reviews feedback on how their contributions will be be broadly provided by the institutions. This [Internet]. Rockville, MD: Agency for Healthcare used also increases the likelihood of a support can even evolve into a broader Research and Quality (US) deeper level of engagement in the future. collaboration between researchers and Godin B, Gingras Y (2000) What is scientific and professionals in science communication and technological culture and how is it measured? A Reflection 7. As unexpected behaviors or engagement. multidimensional model. Public Underst Sci 9: 43 – 58 outcomes can occur during stakeholder Lavery JA (2018) Building and evidence base for engagement activities, it is therefore impor- Reflection 10. Finally, as the principles of stakeholder engagement. Science 361: 554 – 556 tant to remain flexible and able to rapidly Open Science and Responsible Research and Le~ ao MJ, Castro S (2012) Science and rock. EMBO adapt to changing circumstances. A key part Innovation become embedded in the research Rep 13: 954 – 958 of engagement is “listening”, so attempts to process, stakeholder engagement should be Matias A, Dias A, Gonçalves C, Vicente PN, Mena control the direction that a discussion might included as part of academic productivity AL (2021) Science communication for social take should be avoided at all costs. Instead, criteria to encourage researchers to dedicate inclusion: exploring science & art approaches. J giving visibility to the existence of contro- the necessary time and resources to it. Sci Commun 20: A05 versy but also highlighting emerging consen- Mohr A (2008) An independent evaluation of the sus in a safe, respectful, and friendly Acknowledgements BBSRC and the MRC stem cell dialogue project atmosphere will allow for more realistic and cia Mena was funded by the Portuguese Ana Lu 2008. Nottingham: University of Nottingham, challenging views to emerge. Foundation for Science and Technology—FCT. UK: Institute for Science and Society Everything is New sponsored IGC’s participation in Raeymaekers P (2019) Multi-stakeholder dialogue Reflection 8. Right from the start, it is NOS Alive Music Festival. The Calouste Gulbenkian for priority setting in health research. Sharing important to discuss how engagement activi- Foundation funded IGC outreach activities at NOS experiences from 3 pilot projects. Brussels: King ties should be evaluated. Evaluation should Alive Music Festival and the “Embodying Baudouin Foundation: Mind the Gap ISBN:D/ focus on short- and long-term impacts as Memories” project. The Soy-in-1000-gardens 2893/2018/34 well as what we can learn to inform and project and the Liver project were funded within Stilgoe J, Lock SJ, Wilsdon J (2014) Why should we improve future work. This might require the Grand Challenges Program of VIB. This VIB promote public engagement with science? experts in social science, but if resources are Program received support from the Flemish Public Underst Sci 23: 4 – 15 limited, a simple evaluation will do too. Government under the Management Agreement Evaluation helps to collect evidence and to 2017–2021 (VR 2016 2312 Doc.1521/4). In^es define best practices that others can learn Pinheiro funded by ANR LabEx DEEP (11-LBX- from. There is always a risk that the desired 0044, ANR-10-IDEX-0001-02) and by Institut License: This is an open access article under the impact is not achieved or that an activity Curie. We acknowledge support of the Spanish terms of the Creative Commons Attribution License, may have negative consequences. Instead of Ministry of Science and Innovation to the EMBL which permits use, distribution and reproduction being ignored or feared, such risks should partnership, the Centro de Excelencia Severo in any medium, provided the original work is prop- be addressed through a risk assessment Ochoa and the CERCA Programme/Generalitat de erly cited. ª 2021 The Authors EMBO reports 22: e54000 | 2021 7 of 7