Special Issue: The Challenge of Social Impact for Research Methodologies International Journal of Qualitative Methods Volume 21: 1–9 Innovative Methodological Approach to © The Author(s) 2022 DOI: 10.1177/16094069221083373 Analyze Innovation and Social Impact journals.sagepub.com/home/ijq Joan Bellavista1, Carmen Elboj-Saso2 , Carme Garcı́a Yeste3 , and Beatriz Villarejo-Carballido4 Abstract The scientific literature has presented evidence of the links between innovation and change and has published excellent methodologies to analyze them. Nowadays, international scientific programs like Horizon Europe prioritize social impact and co-creation; researchers need to develop methodologies to analyze the link of innovation with change and new knowledge and specially with social impact. This paper presents an innovative methodological approach to this endeavor using Social Media Analytics to investigate citizens’ participation in paying attention to and differentiating between innovations with social impact and innovations without social impact. The method used to address this aim is Social Media Analytics, specifically through a Twitter sample on innovation and social impact composed of 16,794 tweets obtained during January–June 2021. The result obtained indicates that the definition of methodologies to capture citizens’ participation in paying attention to and differentiating between innovation and social impact is crucial for advancing this innovative methodological approach to analyze innovation with social impact. Keywords methodological approach, innovation, social media analytics, social impact Introduction innovation is a transversal aspect in most research areas; busi- ness, education, health, experimental sciences, engineering, and Methodologies for research innovation and social impact are a agriculture are good examples of this transversality. challenge nowadays. In this sense, one first step is to reflect on Understanding innovation as an outcome and process the meaning of innovation and how this concept could be linked (Kahn, 2018), a challenge is to know which methodologies are to social impact and possible methodologies to address it. The available for analysing this topic. In fact, diverse disciplines term “innovation” has different meanings depending on its ap- have applied various methodologies to analyze innovation plication, and usually there are some misunderstandings ac- linked with change or the creation of new knowledge. In this cording to Kahn (2018). There are two definitions for innovation: sense, the European Commission has prioritized that research the first one referred to an outcome and the second one to a process. Innovation can be applied in different domains in re- lation to outcomes, processes, mindset, strategies (Kahn, 2018). 1 Department of Sociology, University of Barcelona, Spain Moreover, innovation is also a current term linked with research 2 Department of Psychology and Sociology, University of Zaragoza, Spain and knowledge creation. For instance, one of the priorities for 3 Department of Pedagogy, University of Rovira-Virgili Tarragona, Spain scientific programs like Horizon Europe is to promote research 4 Department of Journalism and Communication Studies, Autonomous that includes co-creation, social impact, and innovation University of Barcelona, Spain (European Commission, 2020). Although the EU has considered Corresponding Author: impact and social impact in previous research and innovation Carmen Elboj-Saso, Departament of Psychology and Sociology, University of policies and framework programs, the current Horizon Europe Zaragoza, C/ Pedro Cerbuna 12, Zaragoza 50009, Spain. incorporates a special emphasis on these strategies. The field of Email: celboj@unizar.es Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 International Journal of Qualitative Methods outcomes, changes and new knowledge should be linked to innovation in social methods; for instance, linear models, the societal impact during last years. The introduction of mixed methods, systems frameworks, machine learning, and societal impact as a key indicator to evaluate the Framework new approaches to fieldwork are available. These studies often Program for Research (Flecha, 2018) is evidence of this ap- benefit from interdisciplinary research teams. According to the proach, and innovation should include this indicator as well. authors, “On the qualitative side, we see the value of new Social impact and societal impact both are terms used to technologies for more efficient data collections” (Druckman & explain the social improvements obtained through the trans- Donohue, 2020; p. 16). Other methodologies used are based ference of research results that represents effective steps to- on the conception of social labs (Timmermans et al., 2020). wards the fulfilment of officially defined social goals (Pulido Other methodologies used are based on the conception of et al., 2018), such as UN Sustainable Development Goals or social labs (Timmermans et al., 2020). The social lab is an Missions of Horizon 2020, among the social goals defined by inclusive methodology according to Timmermans et al. (2020) other institutions. Including a focus on social impact in the and consists in a space for “doing social experiments in a research work helps guarantee that the investment in it will practical context where experts and stakeholders join together contribute to improving people live conditions. This paper to initiate actions focused on tackling challenges without aims to review the different methodologies used to analyze being constrained by predetermined project plans, (…) social innovation, with particular attention to those contributions that labs provide precisely the ability to proactively experiment promote the integration of social impact. The innovative with circularity” (Timmermans, 2020:412). In relation to methodological approach is exemplified in this case with a methodologies addressed to specific fields, there is an example specific method, Social Media Analytics, addressed to in- of a methodology proficient in guiding successful investment vestigate citizens’ participation in paying attention to and in the IoT (Internet of Things) (Ammirato et al., 2019). Ac- differentiating between innovations with and without social cording to the authors, the methodology combines two key impact. sources: a) literature review and b) impressions collected from The article presents an overview in which a review of informal surveys and in-depth interviews. Other studies fo- diverse methodologies related to innovation is made and cused on methodologies related to innovation that help the classified based on different main topics. The first topic ex- replicability in a different context, promoting the anchoring plores the diversity of methodologies that have analyzed in- for scaling (Seifu et al., 2020). Specifically in education, the novation from different fields. The second topic is based on field usually links innovation with change. For instance, in a methodologies focused on assessing national innovation study from Higher education, the methodology based on in- systems. Finally, the third focuses on methodologies that have quiry about “innovative action” embraces three compelling analyzed research proposals’ innovation based mainly on the dimensions: a) the best alternative to achieve innovation, b) peer review process. The introduction ends with an expla- the best means (looking for efficiency and effectiveness) and nation of the trend of the Framework Program to link inno- c) the best results (evaluation of aims, means and results) vation with societal impact aimed to present an innovative (Penalva, 2021). In other fields, for instance, in business and methodological approach with the method used in this study, health, innovation is linked with solutions or solving prob- Social Media Analytics. lems. Sudbury-Riley et al. (2020) reveal the Trajectory Touchpoint Technique (TTT), a service design methodology aimed at increasing innovation at the service of customer Diverse Methodologies to Analyze Innovation experiences. The design and development of the TTT uses The literature review presents some findings extracted from design science research, a goal-oriented methodology that the papers obtained through the combination of the searchable produces practical solutions to organizational problems keywords “innovation” and “methodologies” of the last (Sudbury-Riley et al., 2020). From the healthcare innovation, 5 years in the Web of Science and some relevant documents on there is a study focused on methodologies based on problem- this topic. The selection of the papers has centered on con- based approach. In this case, and according to the authors, the tributions focused on methods that include innovation as a method needs to define the problem by applying techniques crucial topic. There are quantitative and qualitative methods such as ethnographic research, market analysis and stake- and mixed methods. Some of them focused on helping the holder exploration (Soliman et al., 2020). Still, there is a need guidance of innovation in different fields; others focused on to investigate how the different techniques interact with one analysing the innovation at macro and micro level, for in- another and how to develop the methodology to focus on how stance, the innovation capacity of a country, an organization, innovation should be oriented for the advancement of or a research program. The following contributions selected problem-driven or need-led innovation approaches (Soliman exemplify some of the findings extracted from the literature et al., 2020). review. Another topic is focused on methodologies focused on the One of the topics is the diversity of methodologies carried assessment of national innovation systems, as well as, for out for analysing innovation from different fields. According example, specific methodologies for studying innovative to Druckman and Donohue (2020), we live in an era of practices in policy. Regarding the assessment of the national Bellavista et al. 3 innovation systems (Freeman, 1995; Lundvall, 1992; Nelson, topical emphases (Zhang et al., 2016). In all, science and 1993; OECD, 1997), there are contributions focused on technology studies contribute with knowledge how to approach methodologies based on science and technology statistics in Responsible research and innovation (RRI) (Smith et al., 2021). order to evaluate the international position of those systems. As stated by Chaves et al. (2020), the methodology proposed enables intertemporal tracking of the trajectories of selected Innovation, Co-Creation and Social Impact in countries trajectories of selected countries “placed in cluster Framework Program for Research according to quantitative characteristics of their innovation systems” (Chaves et al., 2020; p. 44). Specifically, the International scientific programs like Horizon Europe prior- methodology proposed helps form a link between the analysis itizes social impact as a key indicator and co-creation as a of science and technology statistics and more structural as- pathway to reach real improvements where innovation is sessments of the global capitalist dynamic. Further, it helps included and addressed to this aim. According to Robinson clustering countries depending on the results obtained through et al. (2020) Horizon Europe integrated RRI as an overarching the dataset built (Chaves et al., 2020). Other studies, instead of principle following the three O’s: “Open Innovation, Open quantitative methods, use qualitative ones. This can be seen, Science, Open to the world ”(European Commission, 2016). for instance, in the use of the Benchmarking method to rank The co-creation with citizens plays a key role in this program, the countries in relation to innovation (Gerlitz et al., 2020), or and they have a triple role: being an end-user, being a relevant the use of the interviews in the example of the analysis of the voice in terms of values and expectations and being an active innovative practices on equal participation of women in player that can contribute with innovative ideas. Now, co- politics in Kerala linked to the Sustainable Development Goal creation approaches are a consolidated trend aimed to col- (SDG) of the United Nations Organization (Babu & Jose, laborate in the design and create a solution based on multi- 2020). Another study found the sustainability of urban sur- stakeholder collaboration (Robinson et al., 2020). This faces could be assessed with a method of SDG based on a top- strategy also has the concept “Open” as a key word following down approach for identifying innovations’ potential contri- the broad policies of the EU. Moreover, the co-creation in- bution to the achievement of SDG before its introduction tervention is also a trend in different fields, for instance, in (Henzler et al., 2020). Likewise, it is important to refer to the Education (Ruiz-Eugenio et al., 2021) Manuals the OECD (2015, 2018) has published in different There are different indicators and methodologies to evaluate the versions for decades. The Frascati Manual is being published impact (scientific, societal and policy) of the research developed in since 1963 and is devoted to the methodologies to collect and the reference report on Monitoring the impact of EU framework report data on research and experimental development. The programs (van des Besselaar et al., 2018). Institutional changes and Oslo Manual, on the other hand, deals, since 1992, with the MoRRI indicators represent a strong mechanism to asses en- methodologies related to data on innovation. gagement in Open Science (Robinson et al., 2020) and the societal The last topic is focused on how research funding programs impact (Flecha, 2018), contributing also to the evaluation of in- usually employ the peer review process to select which research novation. On the other hand, strategies identified to achieve social proposals should be funded to guarantee innovation. Assessing impact include a meaningful involvement of stakeholders and end- this peer review process through a methodology for analysing the users through the project lifespan and public deliberation with a innovation of proposals is one of the concerns pointed out in diverse public (Aiello et al., 2021). In fact, the evaluation of national these studies. Parreira et al. (2019) proposed a method that in- innovation systems is simplified with a specific proposal consid- volves the use a multicriteria decision model under a multi-step ering that innovation should be addressed to sustainable growth, decision-making process. However, other studies focus on the based on principles that take into account economic, social, and policy attitude within scientific innovation through qualitative environmental (ecological) aspects, according to Bielinksa-Dusza inquiry to better understand how the policy attitude operates, and Hamerska (2021). In this context, it is also useful to consider considering that this attitude includes a collaborative ideal and the Triple Helix model developed by Etzkowitz & Leydersdorf influence of values and organizations (Friberg & Englander, (1995). The authors analyze the relationship between universities, 2019). This point is crucial for the authors because science is an business and government, and further development with the fourth integrated part of the policy and industry field, and policy attitude helix incorporates society and impact. Moreover, the fifth helix influences directly how the knowledge is perceived; qualitative adds the natural environment, that is, the socio-ecological transition inquiry is needed to “to explicate the human scientific meaning of (Carayannis & Campbell, 2010). These models allow analysing in a specific attitude driven by an interest in a sociocultural context” more detail the national innovation systems. (Friberg & Englander, 2019; p. 1). Other studies are focused on There is a strong trajectory developing methodologies ad- how to identify innovation topics to assess the policymakers dressed to assess innovation, as seen in previous contributions related to funding research programs inside this field. The study reviewed in this article. The literature review identifies a trend to developed by Zhang et al. (2016) proposed an analytical method link innovation with social impact and results, particularity in the to cluster associated terms and phrases to constitute meaningful Framework Program for Research. However, there is still a need technological topics and their interactions and identify changing to develop methodologies to analyze this approach. To that end, 4 International Journal of Qualitative Methods the analysis of citizens’ perception of social impact should be Table 1 presents the total sample and the final dataset across included due to its relevance for open innovation. Hence, it is key the combination of keywords: to assess social impact through including citizens’ voices in Therefore, the final dataset included a total of 14,460 valid research and innovation, as without them co-creation, necessary tweets that were analyzed as detailed in the results’ section. for research and innovation as indicated by European Com- mission, is not possible. Dialogic Codebook The second step was the elaboration of the dialogic codebook for Method the Communicative Content Analysis, which was designed fol- lowing a dialogic approach. The categories to be included in the This study has used Social Media Analytics with the specific codification process were discussed based on an egalitarian dialogue method of Social Impact in Social Media (Pulido et al., 2020) in among researchers, and decisions were made based on validity order to analyze citizens’ participation in paying attention to claims instead of power claims. A first codebook was elaborated to and differentiating between innovation with or without social include the categories needed to achieve the aims of the research. impact. The social impact in social media (SISM) methodology Then, a pilot was conducted to test the codebook. The final version aims to identify quantitative and qualitative evidence of the of the codebook, which is presented below (Table 2), included both potential or real social impact of research shared on social predefined categories and categories emerged from the pilot. media. In this case, the focus is innovation instead of research. The steps of this methodology include (a) criteria selection of the social media datasets, (b) search strategies for extracting Communicative Content Analysis social media data, (c) application of ethical criteria on internet This study has followed the methodology in the field of Social and social media research, (c) elaboration of a codebook, and d) Media Analytics, which is the Communicative Content qualitative and quantitative analysis of the social media dataset. Analysis. Drawing upon the principles of the Communicative Methodology, this novel approach places co-creation of Data Collection knowledge in the core of the process (Gomez et al., 2019). Therefore, an egalitarian dialogue among plural and diverse In order to collect and analyze citizens’ voices around in- voices is ensured throughout the process. In line with previous novation and social impact, researchers established combi- studies related to Communicate Content Analysis on social nations of keywords that included (1) “Innovation” and (2) media and applying dialogic reliability (Pulido et al., 2020), in four keywords related to social impact: “results,” “improve- this study, researchers worked collaboratively and established a ment,” “impact,” or “social impact.” The data collection took constant egalitarian dialogue among them during the whole place in Twitter and included a maximum of 10,000 tweets process. All researchers discussed and had the codebook, which under each combination of keywords published from January oriented the Communicative Content Analysis, and used it to 1st, 2021 to June 30th, 2021. If more than 10,000 tweets were decide whether to assign one code or another to each message. found under a combination of keywords, the criterion was to If researchers had any doubts during this process and the select the 10,000 with more interactions among users, which classification of the tweet was not clear, the case was discussed was measured in terms of retweets. with other researchers using validity claims and reaching an In this study, researchers defined as a unit of analysis the agreement based on a consensus. This process ensures dialogic whole tweet, including videos, images, link, and websites reliability, as it is based on the plurality of voices and a constant attached in the message. In addition, the number of retweets and egalitarian dialogue among researchers. We have done the obtained by each message was also considered. In those cases qualitative analysis of each tweet of the final sample (1440 where the extracted tweet was part of a Twitter thread or tweets). The results are shared quantitatively because it is more conversation, the whole thread was included in the analysis. suitable with the aim of this paper to show whether there is or not a trend to link innovation with social impact. Sample The sample for this study included a total of 16,794 tweets Ethical Considerations which were extracted using python connected with the API of Regarding ethical considerations, the present research adheres to Twitter. During the codification process, some messages were international ethical criteria related to social media data collection excluded from the analysis for the following reasons: (1) the and corresponding analyses; in particular, we have followed the tweets contained broken links, deleted messages or pages with ethical guidelines for social media research recommended by restricted access, (2) the tweets did not include enough in- European Commission (2018). We have perceived the risk of formation to be codified and analyzed as evidence of inno- harm to and conserved the anonymity of users. Additionally, we vation with or without social impact, and (3) the tweets were have read the terms, conditions, and legalities of each of the spam or contained disrespectful language such as insults. social media channels, and we have used only public information Bellavista et al. 5 Table 1. Sample. Excluded due to Extracted Excluded due to lack of information or Combination of keywords messages lack of access spam messages Final sample Innovation + results 10,000 410 1001 8589 Innovation + improvement 5029 62 483 4484 Innovation + impact 701 14 115 572 Innovation + social impact 1064 31 218 815 Table 2. Codebook. Innovation Social Impact Code Name Definition 0 Not valid There is a broken link or there is not enough information to include the message in the analysis dataset. In addition, those messages that are spam or include disrespectful language (i.e., insult) 1 No There is no social impact as a result of an innovation 2 Potential Although the innovation has not promoted social impact yet, it will potentially achieve social impact. It includes prototypes, designs and guidelines to promote social impact or future transferences of innovations, among others 3 Real The innovation has achieved social impact and improved people’s lives Evidence of social impact Code Name Definition 0 Not applicable The message does not reflect social impact as a result of an innovation 1 No Users do not provide any evidence of how an innovation has achieved social impact 2 Supposed evidence of social impact Although users claim that the innovation has somehow improved people’s lives, there is no evidence source of this improvement 3 Evidence of social impact Users provide evidence of how innovations have improved people’s lives; links or source cited Area of innovation Code Definition - Not applicable o not valid E Education H Health B Business M Marketing C Climate change U Urbanism Other Other not mentioned previously Several Two or more areas involved without identifying any user. Likewise, the data were appro- innovation linked to social impact is crucial for defining the priately coded and anonymized to avoid the possibility of variables to apply in the sample related to this method. Ac- traceability. Sets of data have been secured, saved, and stored. cording to Table 3, there is a trend to link innovation with The dataset analyzed and the calculations performed are available social impact in the sample analyzed. in the Supplementary Materials (dataset). We cannot share all raw The global variables (YES or NO) help identify the per- data due to the current terms of the social media channels and the centage of the number of tweets that link innovation with General Data Protection Regulation (GDPR). social impact, being useful for a quick view. In addition, the division of potential and real social impact helps understand the gradient of social impact, whether it is potential or real Results (Pulido et al., 2018). The highest percentage of tweets with The first step is to identify whether citizens in social media are social impact (potential and real) is the combination keyword linking innovation with social impact or not. The category of of innovation and social impact (47.73%). Although this result 6 International Journal of Qualitative Methods Table 3. Innovation and Social Impact. NO YES Potential SI Real SI Keywords n % n % n % n % Innovation + results 7145 83.20 1443 16.80 943 10.98 500 5.82 Innovation + improvement 3775 84.19 702 15.65 432 9.63 270 6.02 Innovation + impact 312 54.55 257 44.93 150 26.22 107 18.71 Innovation + social impact 424 52.02 389 47.73 279 34.23 110 13.50 Table 4. Evidence of Social Impact. Regarding innovation and social impact, the first one is several NO Supposed YES areas (31.36%), followed by other areas (20.82%), and very close appears business (20.57%). In a lower level, although Keywords n % n % n % playing a role, health and education are areas where inno- Innovation + results 556 38.53 599 41.51 277 19.20 vation and social impact are a consolidated trend. Innovation + improvement 359 51.14 242 34.47 101 14.39 Last, Table 6 helps to view which type of tweets of in- Innovation + impact 109 42.41 90 35.02 58 22.57 novation with or without social impact capture more attention Innovation + social impact 234 60.15 106 27.25 49 12.60 from citizens. In this case, it is important to incorporate the average of the RT because it may clarify the results shown in Table 5. In all areas that are taken into consideration (business, education, health, climate change, urbanism, other areas, and was expected due to the election of that specific combination several areas) except for marketing, the average RT is higher in of keywords, it is important to note that not all the tweets under innovation linked to social impact than the one without this this combination have social impact, and the combination of link. Thus, considering the average RT, the tweets that link innovation and impact also has a high percentage of inno- innovation with social impact obtain more attention from vation and social impact (44.93%). Regarding the percentage citizens than the ones that do not link innovation with social of real social impact, the highest result is related to the impact. combination of innovation and impact (18.71%), more than The quantitative results shared above help draw a general five points above the combination of innovation and social overview of the innovation and social impact. Analysing impact (13.50). results from a qualitative side allows for a more in-depth The method should help identify those tweets that include analysis of the messages, delivering more qualitative infor- evidence of social impact or supposed evidence. The first one mation about the object of study. Some of the examples shared (social impact with evidence) helps citizens, policy makers quantitative evidence. For instance, on the area of climate and stakeholders go directly to the source that shares evidence change, there is a tweet that reflects on how the inclusion of of the improvements obtained, and the supposed evidence solar panels has achieved a 35% reduction of energy con- implies a contrasting task for trust in the statement checked. sumption due to the investment in leading technologies and The following Table 4 classifies tweets without social impact being at the forefront of innovation. On the area of urbanism, evidence from those tweets that have evidence or supposed there is a tweet on how innovation is linked to improve a better evidence of social impact. The results indicate that supposed sanitation, the innovation achieved this aim, and the evidence evidence of social impact has a higher percentage of tweets is disseminated in a newspaper. Before introducing the in- than tweets with evidence of social impact in all four com- novation, the municipality used to generate 180 tons of binations of keywords. garbage and now it has been reduced to 130 tons, and it is There is another interesting result to be analyzed and de- working to reduce it further. On the area of health, there is a fined as category (Area) with different possible variables tweet reflecting how the investment on innovation linked to defined in the codebook (see Table 5). This result is helpful in improve the life conditions through better care given has order to analyze which area has the highest percentage of achieved the improvement of 2500 patients. innovation with social impact. In the case of innovation and improvement, health is the area with a highest percentage (31.95%), as well as in the case of innovation and results Limitations (57.20%). In innovation and impact the variable several areas The method used to apply this innovative methodological (23.14%) is the one with the highest percentage together with approach to analyze the link between innovation and social other areas not included in the selected list (22.35%). Among impact poses some limitations. In particular, there are five the specific areas included, education has the highest per- limitations to highlight. The first is related to the keyword centage which represents the third position (16.08%). selection. There is a specific list proposed by researchers Bellavista et al. 7 Table 5. Areas with Innovation Linked to Social Impact. Business Education Health Marketing Climate Change Urbanism Other Several Keywords n % n % n % n % n % n % n % n % Innovation + results 127 18.12 85 12.13 401 57.20 29 4.14 199 28.39 70 9.99 298 42.51 231 32.95 Innovation + improvement 71 10.13 52 7.42 224 31.95 2 0.29 80 1141 18 2.57 181 25.82 73 10.41 Innovation + impact 37 14.51 41 16.08 30 11.76 4 1.57 21 8.24 6 2.35 57 22.35 59 23.14 Innovation + social impact 80 20.57 34 8.74 37 9.51 0 0.00 24 6.17 11 2.83 81 20.82 122 31.36 Table 6. Tweets and Retweets Innovation Linked to Social Impact. Innovation linked to social impact Innovation without link social impact Areas Tweets Retweets Average RT Tweets Retweets Average RT Business 315 439 1.39 1880 1709 0.91 Education 212 267 1.26 194 161 0.83 Health 692 1084 1.57 352 268 0.76 Marketing 39 29 0.74 770 581 0.75 Climate change 324 702 2.17 73 52 0.71 Urbanism 105 245 2.33 36 6 0.17 Other 613 1117 1.82 3556 4313 1.21 Several 486 1253 2.58 87 63 0.72 involved, but this list could be modified with other keywords 2020; Seifu et al., 2020; Soliman et al., 2020; Sudbury-Riley to extract other datasets. The results are limited to this par- et al., 2020; Timmermans et al., 2020). The literature reviewed ticular dataset. For getting a consolidated overview of it, it has identified a need to analyze research innovation linked to would be necessary to amply the list of these keywords. The societal impact (Aiello et al., 2021; Bielińska-Dusza & Ha- second limitation is the period. We have extracted a dataset merska, 2021; Robinson et al., 2020). Societal impact is within a specific time; hence, the results are limited to this crucial for the Framework Program for Research, and for this period of time. Therefore, it would be necessary to schedule a reason, there is a need to develop methodologies addressed to monitoring process during an extensive period, such as a year, assess innovation with social impact. The European Frame- for getting an in-depth overview. The third limitation is re- work Program published documentation on how to evaluate garding privacy settings; according to the ethical consider- the societal impact with a list of indicators and diverse ations, we cannot reveal in detail any information of the users methodologies (Flecha, 2018). Now is the time to delve deeper who tweet different messages. This limitation does not allow into the development of methodologies focused on innovation deepening on who publishes what regarding innovation and with social impact. This paper has presented an innovative social impact. In addition, the fourth limitation is that re- methodological approach to this endeavor with an example of searchers involved in his study have developed the dialogic the application of Social Media Analytics under this approach codebook regarding the decisions of different codes defined. to investigate citizens’ participation in Twitter related to The codes that were more challenging to identify have been paying attention to and differentiating between innovation those regarding the difference of real or potential innovation with or without social impact. The selection of categories and with social impact; the analysis of these tweets needed a variables and the presentation of the results quantitatively and dialogic contrast of researchers involved to classify in the qualitatively helps us to overview how citizens are paying corresponding code. Last, the fifth limitation is that of spam or attention and differentiating innovation with social impact. disrespectful messages, which is solved deleting them from Furthermore, this methodological approach addressed the the final dataset analyzed. social impact from the design, data collection, and analysis to help identify and analyze this topic. Including this innovative methodological approach could Discussion complement the assessment methods addressed to inno- Usually, innovation has been linked to change and the creation vation and enrich them due to the inclusion of societal of new knowledge with diverse methodologies in order to impact as a crucial element that the European Commission analyze them (Chaves et al., 2020; Druckman & Donohue, prioritizes. Finally, citizens are also important players 8 International Journal of Qualitative Methods paying attention to the core elements innovation and social for modification. Sustainability, 13(4), 2199. https://doi.org/10. impact. 3390/su13042199 Carayannis, E. G., & Campbell, D. F. J. (2010). Triple helix, qua- druple helix and quintuple helix and how do knowledge, in- Further Methodological Advances novation and the environment relate to each other? International Future applications of this innovative methodological ap- Journal of Social Ecology and Sustainable Development (IJ- proach will allow for relevant comparisons to be made to SESD), 1(1), 41–69. https://doi.org/DOI:10.4018/jsesd. understand different applications and the evolution of data 2010010105 related to innovation and social impact. Moreover, this Chaves, C. V., Ribeiro, L. C., dos Santos, U. P., & Albuquerque, methodology could be adapted for different social media and E. D. E. (2020). Innovation systems and changes in the centre- to find out the differences that may exist in the use and rel- periphery division: Notes on a methodology for determining evance of the same keywords. In addition, the elements that country trajectories from science and technology statistics. may explain these differences could be investigated in relation CEPAL REVIEW, 2020(130), 45–64. https://doi.org/10.18356/ to the specific characteristics and uses of the different social 256b68b2-es media. A further methodological aspect to deepen on in the European Comission (2020). Strategic plan 2020-2024 (pp. 1–59). analysis is how to delve into the reasons and justification for https://ec.europa.eu/info/sites/default/files/rtd_sp_2020_2024_ the opinions expressed through which the data is obtained. en.pdf Druckman, D., & Donohue, W. (2020). Innovations in social science Declaration of Conflicting Interests methodologies: An overview. American Behavioral Scientist, The author(s) declared no potential conflicts of interest with respect to 64(1), 3–18. https://doi.org/10.1177/0002764219859623 the research, authorship, and/or publication of this article. Etzkowitz, H., & Leydesdorff, L. (1995). A laboratory for knowledge based economic development. EASST Review, 14(1), 14–19. Funding European Commission (2016). Open innovation open science open to The author(s) disclosed receipt of the following financial support for the world. Directorate-General for Research and Innovation. https:// the research, authorship, and/or publication of this article: This work ec.europa.eu/digital-single-market/en/news/open-innovation-open- was supported by the INTURPYR: Tourism innovation in the Pyr- science-open-world enees for a unique tourist destination in the heart of the Pyrenees. EFA 026/ European Commission (2018). Ethics in social science and humanities. 15. Co-financed 65% by the European Regional Development Fund https://ec.europa.eu/research/participants/data/ref/h2020/other/hi/ through the Interreg V A Spain/France/Andorra Programme - POCTEFA h2020_ethics-soc-science-humanities_en.pdf Issue October 2014–2020. Flecha, R., European Commission (2018). Societal impact Moni- toring the iumpact of EU framework programmes. ORCID iDs Freeman, C. (1995). The ‘national system of innovation’ in historical perspective. Cambridge Journal of Economics, 19(1), 5–24. Carmen Elboj-Saso  https://orcid.org/0000-0003-0937-4861 https://doi.org/10.1093/oxfordjournals.cje.a035309 Carme Garcı́a Yeste  https://orcid.org/0000-0001-9717-8021 Friberg, T., & Englander, M. (2019). Taking policy for granted in the context of scientific innovation. International Journal References of Qualitative Methods, 18, 1–7. https://doi.org/10.1177/ 1609406919831588. Aiello, E., Donovan, C., Duque, E., Fabrizio, S., Flecha, R., Holm, P., Molina, S., Oliver, E., & Reale, E. (2021). Effective strategies that Gerlitz, L., Meyer, C., & Prause, G. (2020). Methodology approach enhance the social impact of social sciences and humanities re- on benchmarking regional innovation on smart specialisation search. Evidence and Policy, 17(1), 131–146. https://doi.org/10. (RIS3): A joint macro-regional tool to regional performance 1332/174426420X15834126054137 evaluation and monitoring in central Europe. Entrepreneurship Ammirato, S., Sofo, F., Felicetti, A. M., & Raso, C. (2019). A meth- and Sustainability Issues, 8(2), 1359–1385. https://doi.org/10. odology to support the adoption of IoT innovation and its appli- 9770/jesi.2020.8.2(80 cation to the Italian bank branch security context. European Gómez, A., Padrós, M., Rı́os, O., Mara, L.C., & Pukepuke, T. (2019). Journal of Innovation Management, 22(1), 146–174. https://doi. Reaching social impact through communicative methodology org/10.1108/EJIM-03-2018-0058 researching with rather than on vulnerable populations: The Babu, A., & Jose, S. (2020). From the homestead to centerstage: A Roma case. Frontiers in Education, 4(9). https://doi.org/10. journey of empowerment. International and Multidisciplinary 3389/feduc.2019.00009. Journal of Social Sciences, 9(3), 230–259. https://doi.org/10. Henzler, K., Maier, S. D., Jäger, M., & Horn, R. (2020). SDG-based 17583/rimcis.2020.4401 sustainability assessment methodology for innovations in the field Bielińska-Dusza, E., & Hamerska, M. (2021). Methodology for of urban surfaces. Sustainability, 12(11), 4466. https://doi.org/10. calculating the European innovation scoreboard—proposition 3390/su12114466 Bellavista et al. 9 Kahn, K. B. (2018). Understanding innovation. Business Hori- Robinson, D. K. R., Simone, A., & Mazzonetto, M. (2020). RRI zons, 61(3), 453–460. https://doi.org/10.1016/j.bushor.2018. legacies: Co-creation for responsible, equitable and fair inno- 01.011 vation in horizon Europe. Journal of Responsible Innovation, Lundvall, B. Å. (1992). In B. Å. Lundvall (Ed.), National systems of 8(2), 1–8. https://doi.org/10.1080/23299460.2020.1842633 innovation: Towards a theory of innovation and interactive Ruiz-Eugenio, L., Toledo del Cerro, A., Gómez-Cuevas, S., & learning. Pinter Publishers. Villarejo-Carballido, B. (2021). Qualitative study on dialogic Nelson, R. (1993). National innovation systems: A comparative literary gatherings as co-creation intervention and its impact on analysise. Oxford University Press. psychological and social well-being in women during the OECD (1997). National innovation systems. OECD Publications. COVID-19 lockdown. Frontiers in Public Health, 9(March), OECD (2015). Frascati manual 2015: Guidelines for collecting and 602964. https://doi.org/10.3389/fpubh.2021.602964 reporting data on research and experimental development, the Seifu, M., van Paassen, A., Klerkx, L., & Leeuwis, C. (2020). An- measurement of scientific, technological and innovation ac- choring innovation methodologies to ‘go-to-scale’; a framework to tivities. OECD Publishing. https://doi.org/https://doi.org/10. guide agricultural research for development. Agricultural Systems, 1787/9789264239012-en 182(2020), 102810. https://doi.org/10.1016/j.agsy.2020.102810 OECD (2018). Oslo manual 2018: Guidelines for collecting, Smith, R. D. J., Kamwendo, Z. T., Berndt, A., & Parkin, J. (2021). reporting and using data on innovation the measurement of Taking knowledge production seriously in responsible research scientific, technological and innovation activities (4th ed.). and innovation. Journal of Responsible Innovation, 8(2), 1–10. OECD Publishing. https://doi.org/https://doi.org/10.1787/ https://doi.org/10.1080/23299460.2021.1935584 9789264304604-en Soliman, E., Mogefors, D., & Bergmann, J. H. M. (2020). Problem- Parreiras, R. O., Kokshenev, I., Carvalho, M. O. M., Willer, A. C. M., driven innovation models for emerging technologies. Health Dellezzopolles, C. F., Nacif, D. B., & Santana, J. A. (2019). A and Technology, 10(5), 1195–1206. https://doi.org/10.1007/ flexible multicriteria decision-making methodology to support s12553-020-00450-5 the strategic management of science, technology and innovation Sudbury-Riley, L., Hunter-Jones, P., Al-Abdin, A., Lewin, D., & research funding programs. European Journal of Operational Naraine, M. V. (2020). The trajectory touchpoint technique: A Research, 272(2), 725–739. https://doi.org/10.1016/j.ejor.2018. deep dive methodology for service innovation. Journal of 06.050 Service Research, 23(2), 229–251. https://doi.org/10.1177/ Penalva, J. (2021). Innovation and leadership as design: A meth- 1094670519894642 odology to lead and exceed an ecological approach in higher Timmermans, J., Blok, V., Braun, R., Wesselink, R., & Nielsen, R. Ø. education. Journal of the Knowledge Economy, 13, 430–446. (2020). Social labs as an inclusive methodology to implement https://doi.org/10.1007/s13132-021-00764-3. and study social change: The case of responsible research and Pulido, C. M., Redondo-Sama, G., Sordé-Martı́, T., & Flecha, R. (2018). innovation. Journal of Responsible Innovation, 7(3), 410–426. Social impact in social media: A new method to evaluate the social https://doi.org/10.1080/23299460.2020.1787751 impact of research. PLOS ONE, 13(8), Article e0203117. https:// van des Besselaar, P., Flecha, R., & Radauer, A. (2018). Monitoring doi.org/10.1371/journal.pone.0203117 the impact of EU framework programmes. Pulido, C. M., Villarejo-Carballido, B., Redondo-Sama, G., & Zhang, Y., Zhang, G., Chen, H., Porter, A. L., Zhu, D., & Lu, J. (2016). Gómez, A. (2020). COVID-19 infodemic: More retweets for Topic analysis and forecasting for science, technology and in- science-based information on coronavirus than for false infor- novation: Methodology with a case study focusing on big data mation. International Sociology, 35(4), 377–392. https://doi. research. Technological Forecasting and Social Change, 105(1), org/10.1177/0268580920914755 179–191. https://doi.org/10.1016/j.techfore.2016.01.015