Original research DIGITAL HEALTH Volume 9: 1–11 Evaluation of response to incentive recruitment © The Author(s) 2023 Article reuse guidelines: strategies in a social media-based survey sagepub.com/journals-permissions DOI: 10.1177/20552076231178430 journals.sagepub.com/home/dhj Megumi Ichimiya1 , Hope Muller-Tabanera2, Jennifer Cantrell3 , Jeffrey B Bingenheimer1, Raquel Gerard1, Elizabeth C Hair2,3,4, Dante Donati5, Nandan Rao6,7 and W Douglas Evans1 Abstract Objective: This study aimed to examine varying incentives on acceptance to participate in an online survey on social media and to identify related demographic factors. Methods: The study used Facebook and targeted its users aged 18 to 24 years in the United States. During recruitment, par- ticipants were randomized to one of the three types of incentives for survey completion, (1) a $5 gift card, (2) a lottery for a $200 gift card, and (3) a $5 gift card plus a lottery for a $200 gift card. Acceptance rates for survey participation were com- pared across three incentives using percentages, 95% logit-transformed confidence intervals, and Pearson’s chi-squared tests. The survey asked about cognition and behaviors around smoking and vaping. Results: The ads had 1,782,931 impressions, 1,104,139 reaches, and 11,878 clicks. The average ad frequency was 1.615, and the click-through rate was 0.67%. Males clicked less than females when seeing the ads. The acceptance rates for the three incentives were 63.7%, 37.2%, and 64.6%, respectively. A Chi-square test confirmed that the lottery-only group had a lower acceptance rate compared to those guaranteed an incentive, including the gift card group and the gift card and lottery group. Further analyses indicated that males did not opt into the survey as often as females when given the lottery-only incentive option, and those who did not meet their financial expenses opted into the survey more often than those who had more money than their expenses when given the lottery-only incentive option. Conclusions: This study suggests that incentives guaranteed to all participants, even if the incentive’s value is small, may lead to higher acceptance rates compared to a lottery for a greater incentive in social media-based surveys. Keywords Digital health, eHealth, digital media, social media, incentive, recruitment, behavior change, smoking, vaping Submission date: 26 October 2022; Acceptance date: 10 May 2023 Introduction 1 Department of Prevention and Community Health, Milken Institute School The use of social media to recruit research participants for of Public Health, The George Washington University, Washington, DC, USA 2 health research has significantly increased in the past Schroeder Institute, Truth Initiative, Washington, DC, USA 3 Global Institute of Public Health, New York University, New York City, NY, decade.1–7 Researchers have successfully recruited USA general and targeted populations on various social media 4 Department of Health, Behavior and Society, Johns Hopkins Bloomberg platforms, including Facebook, Instagram, Twitter, and School of Public Health, Baltimore, MD, USA 5 others.2 Social media recruitment has also effectively 6 Marketing Division, Columbia Business School, New York, NY, USA accessed youth and young adults and hard-to-reach Universitat Autonoma de Barcelona, Catalunya, ES, Spain 7 Barcelona School of Economics, Catalunya, ES, Spain groups for research.2,4–7 Corresponding author: Incentives play a positive role in increasing response to Megumi Ichimiya, The George Washington University, 950 New Hampshire requests for research participation.8–15 The role of incen- Ave, NW Washington, DC 20052, USA. tives is based on the social exchange theory, which posits Email: imegumi7@gwu.edu Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/ open-access-at-sage). 2 DIGITAL HEALTH that the actions of individuals are motivated by the antici- A separate line of research has examined strategies for pated relationship between the associated rewards and recruitment on different social media platforms or in com- costs.16–18 Individuals may respond to a request to partici- parison to traditional recruitment methods, and many pate in research when they expect and trust that the studies have used incentives to recruit participants rewards of responding (i.e. monetary compensation, a online.9,26 A systematic review of 110 unique studies con- social reward, etc.) outweigh the costs of participating ducted on Facebook found that incentives offered included (i.e. time and effort).19,20 a small guaranteed gift or reimbursement for participation, a Although there has been debate about the ethics of chance to win a prize via lottery, or a combination of a small incentives, and some researchers point out that incentives guaranteed reimbursement plus a chance to win a prize via may be coercive, past literature suggests that incentivized lottery.5 However, none of these studies compared the surveys are not coercive when participation is voluntary effect of varying incentive strategies on recruitment and informed consent is offered after a full explanation of success. the purpose of the research, procedures, risks and benefits, Researchers have noted a critical need for effective and alternatives, and confidentiality.21–23 Explanation of the novel methods for research and experimentation in social cost and compensation is also an essential part of informed media.27 Given the substantial increase in social media consent, and the forms and amount of incentives should research and the lack of robust evidence based on methodo- clearly be communicated to potential participants before logical best practices, such research should include system- the start of the survey.22,23 Careful examination of the atic approaches to experimentally test varying recruitment data quality is required for any surveys, including online strategies, including incentives, on social media platforms. surveys, particularly when incentivized, as incentives can Incentives may increase participation and the likelihood of attract participants who are not genuinely interested in the retention in longitudinal studies,11–15,21,26–28 and may also survey. However, previous research suggests that incenti- increase engagement with interventions, including those vized surveys tend to have improved data quality compared delivered through social media. One important objective to nonincentivized surveys in several aspects, such as of future research using social media platforms is to deter- decreased drop-out rates and increased complete responses, mine how to optimize study designs and responses to in addition to survey response rates.8–15 Regarding the recruitment and intervention methods.29 content of survey responses, recent studies show that the This study aimed to (1) examine varying incentive strat- risks of bots that fill surveys with random responses have egies on acceptance to participate in an online research been increasing for incentivized online surveys and study survey on social media, (2) evaluate whether incen- methods to detect bots (e.g. collection of timestamps, tive effectiveness varied by participant demographic char- examine survey response patterns, and collect participant acteristics, and (3) examine Facebook advertising email address) should be employed in the survey proto- statistics (e.g. click-through rate or CTR) and their relation- col.24,25 The nature of incentives can vary depending on ship to the participants’ demographics. The goal of the the target population (e.g. vulnerable populations), the study was to provide evidence for optimal incentive strat- amount, and the research settings, and decisions should egies in future social media-based surveys, particularly be made with careful consideration by each institutional around smoking and vaping. review board. A large body of experimental research demonstrates that higher incentives increase response, but with diminishing Methods returns, cash or cash equivalents are preferred to gifts, and evidence for lotteries is mixed and inconclusive.8–10 Study design There is also limited empirical evidence of incentive This study was conducted as part of a larger, long-term effects on different subpopulations, particularly for online study examining the effects of social media-based antito- surveys.10 However, the majority of these studies have bacco advertising on young adult attitudes, beliefs, and been conducted in educational or university settings, pro- behavior. The larger study conducted a baseline survey fessional societies, or in ongoing online panels that partici- and two follow-up surveys to assess changes in smoking pants joined for survey opportunities and presumably have and vaping behaviors and cognitions. The surveys were some prior contact with the surveying entity.9,26 The extent pilot tested and updated by research staff before starting to which findings from these studies can be generalized to the study. social media platforms is unclear. The perceived legitimacy The eligibility for the study was (1) Facebook users, (2) of the entity contacting an individual for research is a sig- aged 18 to 24 years, and (3) in the United States. Facebook nificant factor in respondent trust, and this issue is espe- was selected because it is simple to facilitate surveys on the cially acute on social media platforms.10 Incentives may platform, and they offer detailed ad statistics on their dash- play an important role in establishing the validity of a board. The sample was a convenience sample and was not survey opportunity. representative of national demographics. Ichimiya et al. 3 Of the larger longitudinal study explained above, this from the target population, starts a baseline survey right study only used the acceptance of survey participation after study participation consent, and sends reminders and and demographics from the baseline survey. The baseline follow-ups until study completion based on retargeting survey had 39 questions in total, including 4 questions technology. The eligibility criteria can be specific and based on skip patterns that were shown to participants detailed for behavioral interventions and stratified by depending on their previous responses. Data was collected individual-level factors including age, gender, race, and between December 29, 2021, and February 15, 2022. ethnicity, geographic locations where the intervention is The study was reviewed and approved by the institu- conducted, and other relevant factors of interest. The plat- tional review board at The George Washington University form currently uses Facebook messenger and WhatsApp (NCR202837) and was determined to be exempt. This for surveys, and uses Facebook, Instagram, and Google article is informed by the CHERRIES framework for web- search results for retargeting. based surveys.30 The process of recruitment is summarized in Figure 1. Virtual Lab posted paid ads with images on users’ feeds to recruit study participants. Figure 2 shows the specific Incentives ad images used for this step. Ad images were broad on During recruitment, participants were randomized to purpose, and no information regarding the topic and the receive one of the three types of incentive offers, a $5 incentive group was provided in this phase to avoid bias. Amazon gift card (incentive 1), a lottery for a $200 The potential participants clicked on the ads, which redir- Amazon gift card (incentive 2), and a $5 Amazon gift ected them to Facebook Messenger, where the survey card plus a lottery for a $200 Amazon gift card (incentive took place. The survey was linked to the study’s 3), at their initial recruitment (see Figure 1). All participants Facebook page using the Virtual Lab platform. This in the first and the third group were told in advance that they allowed the study’s Facebook pages to send automated would receive a $5 Amazon gift card after completing the messages on Facebook messenger. Participants who baseline and an additional gift card after completing the clicked on the ads saw a welcome message and a question two follow-up surveys, for a total of $15 if they completed that asked permission to chat. If participants confirmed per- all three surveys. In these groups, a $5 incentive was guar- mission to chat, the experiment randomly assigned them to anteed because the gift card was offered after each survey one of the three incentive groups. They then saw an intro- for all respondents. Participants in the second group, on duction message that explained their incentive offer, an the other hand, were offered a lottery ticket to win a $200 explanation of the study, the IRB information, a consent Amazon gift card, only if they completed all three form, and an invitation to continue to the survey. If partici- surveys. The conditions clearly explained the incentive pants consented and accepted to participate, they began the options before they began the survey. Specific sentences baseline survey. Eligibility for the study was determined by used for each group were: “You will receive a $5 screening questions at the beginning of the survey. Amazon voucher after the completion of each survey, for a total of $15 if you complete all the 3 surveys”; “If you answer all the 3 surveys you will get a lottery ticket to Measures enter a contest to win a $200 Amazon voucher”; and Acceptance of survey participation was asked using a ques- “You will receive a $5 Amazon voucher after the comple- tion, “Do you accept to participate in the survey?”, after an tion of each survey, for a total of $15 if you complete all introduction message that explained their incentive offer, an the 3 surveys. Moreover, if you answer all the 3 surveys explanation of the study, the IRB information, and a you will also get a lottery ticket to enter a contest to win consent form. a $200 Amazon voucher”. Group allocation was automatic- Demographic questions included gender, age, race and ally generated by the online platform using simple ethnicity, education, and financial situation. The financial randomization. situation was asked using a question, “Considering your own income and the income from any other people who help you, how would you describe your overall personal Recruitment financial situation?”. Options included live comfortably, This study used a novel online platform for recruitment and meet needs with a little left, just meet basic expenses, and data collection developed by Virtual Lab LLC (https://vlab. doesn’t meet basic expenses. digital/).31 The platform enables researchers to reach, Ad statistics included impressions, reach, clicks, cost, fre- recruit, and interact with a target population and to com- quency, and CTR. Ad impressions refer to the number of plete all steps in a research study through social media plat- times the ads were shown on the screen. Reach is the forms online. It is run independently of social media number of people who saw the ads at least once, estimated companies. It screens potential participants based on the by the online platform. Ad impressions may include multiple proposed eligibility criteria, recruits study participants views of the ads by the same individual, whereas the reach 4 DIGITAL HEALTH Figure 1. Flowchart of recruitment process. does not include those. Click corresponds to the number of and 95% logit-transformed confidence intervals were calculated. clicks on the ads. The cost was recorded as the total cost A Pearson’s chi-squared test evaluated statistically significant spent for displaying each ad. The frequency refers to the independence of participation between the incentive options. average number of times each person saw the ad, which (Aim 2) Responses for demographic questions for the entire was calculated as the number of impressions divided by the sample were not available since those who denied participa- reach. The CTR corresponds to the percentage of times tion in the survey did not answer any questions. However, people saw the ad and performed a link click, calculated as frequencies and percentages of subcategories for the survey the number of clicks divided by impressions times 100. participant group were calculated. Pearson’s chi-squared tests evaluated statistically significant independence of par- ticipation between the incentive options by each demo- Analysis graphic group. (Aim 3) Ad impressions, reach, click, cost, frequency, and (Aim 1) The analyses compared acceptance rates for survey par- CTR were compared across gender using the data recorded ticipation by the three incentive options. Subgroup percentages on the ad management website. Ichimiya et al. 5 Figure 2. Ad images. Figure 3. Probability of survey participation by incentive option offered. Analyses were performed with Rstudio (Version 4.1) and the difference in proportions of acceptance of the survey the packages: tidyverse, ggplot2, gtsummary, rstatix, and between the types of incentives offered, X2 (2, N = 3143) freqtables. = 199.82, p = 4.07e−44. Figure 3 shows the probability of participation by incentive offered. Results Aim 2: Incentives and acceptance rates Aim 1: Incentives and acceptance rates by demographics A total of 3143 people clicked on the Facebook survey Table 1 specifies each incentive group’s random assign- advertisement. The acceptance rate for the $5 gift card ment and who accepted to participate in the survey. option (incentive group 1) and the $5 gift card plus a The survey collected demographics for those who lottery option (incentive group 3) were 63.7% and 64.6%. confirmed they would like to take the survey and were The acceptance rate for the lottery-only option (incentive within the age range (N = 1609). Table 2 shows the group 2) was lower, at 37.2%. A Chi-square test confirmed sample characteristics. The convenience sample of this 6 DIGITAL HEALTH Table 1. Acceptance of survey participation by incentive option offered. Offered an incentive Yes, took the survey No, opted out Chi-squared test Incentive 1: $5 gift card N = 1072 683 (63.7%) 389 (36.3%) X2 (2, N = 3143) = 199.8, p = 4.07e-44 Incentive 2: $200 gift card lottery N = 999 372 (37.2%) 627 (62.8%) Incentive 3: $5 gift card & $200 gift card lottery N = 1072 692 (64.6%) 380 (35.4%) Overall N = 3143 1747 (56%) 1396 (44%) study consisted of more female (68%) and Asian American Aim 3: Ad statistics or Native Hawaiian/Pacific Islander (AANHPI, 21%) com- The Facebook recruitment ads were displayed 1,782,931 pared to the US National survey, at 48.7% and 5.5%, times on the screen (ad impressions). A total of 1,104,139 respectively.23 Other demographics were similar across people saw the ads at least once (reach). The ad images our sample and the US national sample. are shown in Figure 2. These ads were clicked 11,878 Pearson’s chi-squared test confirmed the difference in times. The average frequency was 1.615. The cost of dis- proportions of males and females between the types of playing the ads was $8439. incentives offered, X2 (2, N = 1524) = 11.02, p = .004. The CTR was 0.67% on average. The CTR was different While 30% and 29% of the survey participants were across gender for both ads. The CTR among males for ad 1 males in the incentive groups that offered $5 gift cards, and ad 2 were 0.50% and 0.53%, respectively, and those only 21% were males in the lottery-only incentive among females were 1.07% and 0.95%. For both ads, group. On the other hand, 66% and 67% of the survey par- males clicked less than females when they saw the ads. ticipants were females in the incentive groups that offered Table 3 shows the ad statistics. guaranteed gift cards, and 75% of the survey participants were females in the lottery-only incentive group. Since the incentive groups were assigned randomly, we assume that males did not opt into the survey as often Discussion as females when given the lottery-only incentive option. Figure 4 compares incentive responses by participants’ Differences in recruitment rates by incentive type gender. The present study provides preliminary evidence for which A Pearson’s chi-squared test confirmed the difference in types of incentives were associated with survey acceptance. proportions of those who did not meet their regular house- This study found that acceptance rates were higher when hold expenses and those who did between the types of incentives were small but guaranteed, compared to a incentives offered, X2 (2, N = 1569) = 7.04, p = .030. lottery for a greater incentive. Acceptance of the small While 6.0% of the survey participants did not meet their but guaranteed incentive was 1.7 times that of the expenses in each incentive group that offered guaranteed lottery-only option. Adding a lottery to a small incentive gift cards, 10% of the survey participants did not meet did not increase survey participation. Although the survey their expenses in the lottery-only incentive group. On the asked about cognition and behaviors around smoking and other hand, 94% of the survey participants met or exceeded vaping, our findings on incentive would not be limited to their expenses in the incentive groups that offered guaran- these topics as these were not shared with participants at teed gift cards, and 90% of the survey participants met or the time of survey acceptance. exceeded their expenses in the lottery-only incentive The findings suggest that potential participants decided group. Since the incentive groups were assigned randomly, to take the survey based on the expected probability of we assume that when given the lottery-only incentive getting the incentive. While the odds of winning the option, those who did not meet their financial expenses lottery were not directly provided to participants since opted into the survey more often than those who had they were dependent on the number of participants, accept- more money than their expenses. According to chi-squared ance rates suggest that participants discounted the value of tests, all other demographics tested did not have a statistic- the lottery since the expected probability of winning it was ally significant different proportions. Figure 5 compares unknown. The small, guaranteed incentive was valued more incentive responses by participants’ self-reported financial highly. This study suggests that a small, guaranteed incen- situations. tive is more effective than a lottery in encouraging social Ichimiya et al. 7 Table 2. Survey participant sample description. Incentive 2, Chi-squared test Characteristic Overall, N = 1609 Incentive 1, N = 633 N = 343 Incentive 3, N = 633 p-value Gender 0.023 Female 1084 (68%) 412 (66%) 255 (75%) 417 (67%) Male 440 (28%) 189 (30%) 70 (21%) 181 (29%) Neither 66 (4.2%) 25 (4.0%) 13 (3.8%) 28 (4.5%) Missing 19 7 5 7 Age (mean, SD) 21.06(2.00) 21.08(1.97) 21.11(1.96) 21.01(2.06) Race and ethnicity 0.53 White, non-Hispanic 753 (49%) 294 (49%) 166 (51%) 293 (48%) Hispanic 276 (18%) 110 (18%) 60 (19%) 106 (17%) Black, non-Hispanic 125 (8.1%) 42 (6.9%) 29 (9.0%) 54 (8.8%) AANHPI, non-Hispanic 321 (21%) 137 (23%) 57 (18%) 127 (21%) Other race 64 (4.2%) 22 (3.6%) 11 (3.4%) 31 (5.1%) Missing 70 28 20 22 Financial situation 0.030 Doesn’t meet expenses 105 (6.7%) 35 (6%) 33 (10%) 37 (6%) Meets or exceeds budget 1464 (93%) 580 (94%) 300 (90%) 584 (94%) Missing 40 18 10 12 Level of education 0.50 Less than high school graduate 206 (15%) 79 (14%) 36 (12%) 91 (16%) High school graduate 711 (50%) 280 (50%) 159 (53%) 272 (49%) College graduate 501 (35%) 200 (36%) 106 (35%) 195 (35%) Missing 191 74 42 75 AANHPI: Asian American or Native Hawaiian/Pacific Islander; SD: standard deviation. media survey participation. Also, our findings suggest that all key factors in promoting effective social exchange.34 adding a lottery to a guaranteed incentive does not improve Ensuring these three factors are present may be particularly the chances of participating in a survey. important in studies with low response propensity, such as This finding is consistent with past studies, especially in online surveys. One hypothesis is that trust in the lottery psychology and behavioral economics.32,33 Whether expli- option (i.e. in the likelihood of a reward) was low, reducing citly or implicitly, people are affected by the expected prob- the propensity to accept the survey. ability of outcomes and prefer promised incentives rather In practice, the cost for a guaranteed incentive can be a than taking risks for an unpromised return, even if the fiscal challenge in research. This study compared a $5 gift incentive amount is larger. Rewards, costs, and trust are card and a $200 lottery, and the amount of reward was 40 8 DIGITAL HEALTH Figure 4. Survey participants’ gender. Figure 5. Survey participants’ financial status. times different from the participants’ view. Yet, those in the lottery-only incentive option.35 This finding has implica- guaranteed incentive group accepted the survey by 1.7 tions for studies that stratify by gender, as acceptance times that of the lottery group. However, for a guaranteed rates should be expected to be lower for male respondents. incentive, the total cost will increase as the number of par- ticipants increases, and careful consideration of the cost- effectiveness would be required for researchers. Further Online recruitment advertising research is required to examine the cost-effectiveness of This study used a novel, social media-based technology for guaranteed incentives and lottery to identify the most effect- survey recruitment. This methodology enables researchers ive strategy. In examining these factors, it is also important to reach many potential participants with lower costs and to investigate the application of bots in survey responses more efficient targeting by participant characteristics com- across guaranteed incentives and lottery.24,25 pared to conventional recruitment methods.31 We found This study also suggests gender differences in response that the recruitment using this methodology was less to survey incentives. Since the incentive groups were costly than other recruitment methods used by the assigned randomly, we concluded that males did not opt authors, such as using online ad platforms and traditional into the survey as often as females when given the data collection vendors such as large internet-based Ichimiya et al. 9 Table 3. Ad statistics. Impression Reach Click Total cost (USD) Male Ad 1: Amazon girl 639,894 362,019 3194 2892 Ad2: Banner brown 609,744 365,898 3232 2429 Female Ad 1: Amazon girl 311,154 198,780 3334 2063 Ad2: Banner brown 222,139 177,442 2118 1056 Total 1,782,931 1,104,139 11,878 8439 Frequency Click-through rate Male Ad 1: Amazon girl 1.768 0.50% Ad2: Banner brown 1.666 0.53% Female Ad 1: Amazon girl 1.565 1.07% Ad2: Banner brown 1.252 0.95% Total 1.615 0.67% research firms,31 at 8439 USD to reach 1,104,139 popula- two groups, as shown in Table 2. Since participants were tion and to collect 1609 study participants. randomly assigned and had an equal probability of receiv- We also found variations in recruitment ad CTR by ing each incentive offer, and thus each person clicked on gender. Since a lower percentage of males clicked the ads the ad to one of the three incentive groups, each group compared to females, we conclude that researchers may should have the same total sample size. We hypothesize need to pay more to attract male participants. Further that some people assigned to the lottery-only incentive research is needed to understand this tendency and what may have blocked the Facebook survey account after other factors (e.g. survey content, format, etc.) may seeing that we offered a lottery-only incentive rather than explain our findings. There is a large body of research something guaranteed. Additionally, this study did not finding that females are more likely to participate in investigate different amounts of incentives, such as the pos- research than males.36–38 sible difference between varying guaranteed incentive amounts either alone or combined with different lottery options. Also, the odds of winning the lottery were not pre- Future research sented as they were based on how many people opted into This research is part of a larger, long-term study examining the survey, which was not capped. The responses to further the effects of social media-based antitobacco advertising on variations in the types of incentives may differ and yield young adult attitudes, beliefs, and behavior. The incentive additional evidence on the response to the offered social evidence gathered here will be used in this long-term exchange. In addition, the present study only used research. We will conduct additional incentive experiments Facebook and may not be generalizable to other social to expand the evidence based on social media-based media platforms (e.g. Snapchat, Tik Tok). Since the use surveys and response probabilities, including cost and cost- of Facebook among young adults is not as common as effectiveness. Finally, we will compare retention and attri- for the older generations, it may not be the best platform tion by study and incentive conditions across groups to reach the masses of this population. However, using the follow-up surveys. Facebook was a good option for this study due to the con- venience of facilitating the survey through its message feature, which is not currently available on other social Limitations media platforms. Finally, the study used a convenience First, the number of participants was not consistent across sample collected via social media and is not representa- groups. We observed a difference in the sample size of tive of the overall US population of young adults aged those offered the lottery-only option 73 less than the other 18 to 24 years. 10 DIGITAL HEALTH Conclusion 5. Thornton L, Batterham PJ, Fassnacht DB, et al. Recruiting for health, medical or psychosocial research using Facebook: sys- The survey acceptance rate of the small but guaranteed tematic review. Internet Interv 2016; 4: 72–81. incentive (incentive 1) was 1.7 times that of the lottery-only 6. Whitaker C, Stevelink S and Fear N. The use of Facebook in option (incentive 2). Adding a lottery to a small incentive recruiting participants for health research purposes: a system- (incentive 3) did not increase survey participation. It is atic review. J Med Internet Res 2017; 19: e290. hypothesized that potential participants have decided on 7. Topolovec-Vranic J and Natarajan K. The use of social media their acceptance based on the probability of getting the in recruitment for medical research studies: a scoping review. incentive rather than the size of the incentive. This study J Med Internet Res 2016; 18: e286. suggests that incentives guaranteed to all participants, 8. Nicolaas G, Corteen E and Davies B. The use of incentives to recruit and retain hard-to-get populations in longitudinal even if the incentive’s value is small, may lead to higher studies. NatCen Soc Res 2019: 7–15. acceptance rates compared to a lottery for a greater incen- 9. Singer E. The use and effects of incentives in surveys. In: tive in social media-based surveys. Findings from this Vannette DL and Krosnick JA (eds) The Palgrave handbook study offer insights into methods for social media-based of survey research. Switzerland: Springer, 2018, pp.112–135. survey recruitment and suggest avenues for future research. 10. Toepoel V. Effects of incentives in surveys. In: Gideon L (ed.) Handbook of survey methodology for the social sciences. Contributorship: The manuscript was prepared by MI, HMT, JC, New York: Springer, 2012, pp.209–223. and WDE. MI, HMT, and JBB worked on data analysis. The 11. Yu S, Alper HE, Nguyen AM, et al. The effectiveness of a implementation of the study was done by DD, NR, and RG. JC, monetary incentive offer on survey response rates and ECH, and WDE reviewed and supervised the study and the response completeness in a longitudinal study. BMC Med manuscript. All authors reviewed and approved the final version Res Methodol 2017; 17: 77. of the manuscript. 12. Abdelazeem B, Hamdallah A, Rizk MA, et al. Does usage of monetary incentive impact the involvement in surveys? A sys- tematic review and meta-analysis of 46 randomized controlled Declaration of conflicting interests: The author(s) declared no trials. Gao Z, editor. PLoS One 2023; 18: e0279128. potential conflicts of interest with respect to the research, 13. Göritz AS. Incentives in Web Studies: Methodological Issues authorship, and/or publication of this article. and a Review. 2006. 14. Shamon H and Berning CC. Attention check items and Ethical approval: This study was reviewed and approved by the instructions in online surveys: boon or bane for data institutional review board at The George Washington University quality? Surv Res Methods 2020; 14: 55–77. (NCR202837). 15. Shaw MJ, Beebe TJ, Jensen HL, et al. The use of monetary incentives in a community survey: impact on response rates, data quality, and cost. Health Serv Res 2001; 35: 1339–1346. Funding: The author(s) disclosed receipt of the following 16. Blau PM. Exchange power in social life. New York: Wiley, financial support for the research, authorship, and/or publication 1964. of this article: This work was supported by the National Cancer 17. Homans GC. Social behavior: its elementary forms. Institute (grant number R01CA253013). New York: Harcourt, Brace, & World, 1961. 18. Thibaut JW and Kelly HH. The social psychology of groups. New York: Wiley, 1959. Guarantor: WDE. 19. Dillman DA. Mail and telephone surveys: the total design method. New York: Wiley, 1978. ORCID iDs: Megumi Ichimiya https://orcid.org/0000-0002- 20. Dillman DA, Smyth JD and Christian LH. Internet, mail, and 2665-6513 mixed-mode surveys: the tailored design method. 3rd ed. Jennifer Cantrell https://orcid.org/0000-0003-4730-2535 New York: Wiley, 2009. 21. Singer E and Bossarte RM. Incentives for survey participation when are they “coercive”? Am J Prev Med 2006; 31: 411–418. References 22. Joffe S, Cook EF, Cleary PD, et al. Quality of informed 1. Alshaikh F, Ramzan F, Rawaf S, et al. Social network sites as consent: a new measure of understanding among research a mode to collect health data: a systematic review. J Med subjects. JNCI J Natl Cancer Inst 2001; 93: 139–147. Internet Res 2014; 16: e171. Epub ahead of print 2014. 23. FERCAP Multi-Country Research Team, Karbwang J, 2. Darko EM, Kleib M and Olson J. Social media use for Koonrungsesomboon N, et al. What information and the research participant recruitment: integrative literature extent of information research participants need in informed review. J Med Internet Res 2022; 24: e38015. consent forms: a multi-country survey. BMC Med Ethics. 3. Frampton GK, Shepherd J, Pickett K, et al. Digital tools for 2018; 19: 79. the recruitment and retention of participants in randomised 24. Xu Y, Pace S, Kim J, et al. Threats to online surveys: recog- controlled trials: a systematic map. Trials 2020; 21: 478. nizing, detecting, and preventing survey bots. Soc Work Res 4. Reagan L, Nowlin SY, Birdsall SB, et al. Integrative review of 2022; 46: 343–350. recruitment of research participants through Facebook. Nurs 25. Pozzar R, Hammer MJ, Underhill-Blazey M, et al. Threats of Res 2019; 68: 423–432. bots and other bad actors to data quality following research Ichimiya et al. 11 participant recruitment through social media: cross-sectional 32. Kahneman D and Tversky A. Prospect theory: an analysis of questionnaire. J Med Internet Res 2020; 22: e23021. decision under risk. Econometrica 1979; 47: 263–291. 26. Wu MJ, Zhao K and Fils-Aime F. Response rates of online 33. Linnemayr S, MacCarthy S, Kim A, et al. Behavioral surveys in published research: a meta-analysis. Comput economics-based incentives supported by mobile technology Hum Behav 2022; 7: 1–2, 7–10. on HIV knowledge and testing frequency among Latino/a 27. Evans WD, Abroms LC, Broniatowski D, et al. Digital media men who have sex with men and transgender women: proto- for behavior change: review of an emerging field of study. Int col for a randomized pilot study to test intervention feasibility J Environ Res Public Health 2022; 19: 2–5, 11–12. and acceptability. Trials 2018; 19: 540. 28. Reveilhac M, Steinmetz S and Morselli D. A systematic litera- 34. Nord WR. Social exchange theory: an integrative approach to ture review of how and whether social media data can comple- social conformity. Psychol Bull 1969; 71: 174–208. ment traditional survey data to study public opinion. Multimed 35. Laguilles JS, Williams EA and Saunders DB. Can lottery Tools Appl 2022; 81: 10107–10142. incentives boost web survey response rates? Findings from 29. Tulsiani S, Ichimiya M, Gerard R, et al. Assessing the feasi- four experiments. Res High Educ 2011; 52: 537–553. bility of studying awareness of a digital health campaign on 36. Burg JAR, Allred SL and Sapp JH. The potential for bias due Facebook: pilot study comparing young adult subsamples. to attrition in the national exposure registry: an examination of JMIR Formative Res 2022; 6: e37856. reasons for nonresponse, nonrespondent characteristics, and 30. Eysenbach G. Improving the quality of web surveys: the the response rate. Toxicol Ind Health 1997; 13: 1–13. checklist for reporting results of internet E-surveys 37. Dunn KM, Jordan K, Lacey RJ, et al. Patterns of consent in (CHERRIES). J Med Internet Res 2004; 6: e34. epidemiologic research: evidence from over 25,000 respon- 31. Rao MN, Donati D, Olvera O, et al. Conducting surveys and ders. Am J Epidemiol 2004; 159: 1087–1094. interventions entirely online: a virtual lab practitioner’s 38. Galea S and Tracy M. Participation rates in epidemiologic manual. Washington, DC: World Bank Group, 2020. studies. Ann Epidemiol 2007; 17: 643–653.