Battling the Bots and Defending Against Fraudulent Responses in an International Community-Engaged Web-Based Survey With People Living With Long COVID: Methodological Study.

BACKGROUND: Web-based surveys involving self-reported questionnaires are vulnerable to fraudulent responses. Advancements in artificial intelligence and bots have introduced additional challenges to preventing and identifying fraudulent responses to online questionnaires. OBJECTIVE: This study aimed to describe our experiences with fraudulent responses, strategies for preventing and identifying fraudulent responses, lessons learned when conducting a web-based survey with adults living with Long COVID, and recommendations for web-based survey research. METHODS: The Long COVID and Episodic Disability Study is an international community-engaged study among adults living with Long COVID in Canada, Ireland, the United Kingdom, and the United States. We conducted a longitudinal web-based survey, with online administration of a self-reported questionnaire at 2 timepoints (Time 1 and Time 2), 1 week apart. We recruited through Long COVID community groups using social media, emails, and word of mouth. The survey was disrupted by fraudulent responses, including bots. To defend data integrity, we implemented the following strategies: (1) pausing our initial launch (Wave 1), (2) developing and implementing screening criteria to identify fraudulent responses, and (3) relaunching the web-based survey (Wave 2) with revised recruitment strategies and questionnaire design to prevent and identify fraudulent responses. RESULTS: We received 4663 responses for Time 1 and 1281 responses for Time 2, of which we retained 798 of 4663 (17%) responses and 629 of 1281 (49%) responses. Strategies for preventing fraudulent responses included enabling survey protection features in survey software, shutting down compromised survey links, avoiding recruitment via public social media groups, and removing mention of a financial incentive from recruitment materials. Strategies for identifying fraudulent responses included monitoring response completion times, start and end time stamps, geolocation, and screening for suspicious email address characteristics and duplicates. CONCLUSIONS: Our lessons learned fell into the following three areas: (1) survey-design and implementation to prevent and identify fraudulent and bot-generated responses, (2) recruitment strategies to mitigate the risk of disruption by bots, and (3) responding to disruptions caused by fraudulent and bot responses. We recommend the following tactics to prevent and mitigate the risks of fraudulent and bot responses when administering online web-based questionnaires: (1) review current literature and connect with researchers and Research Ethics Boards about strategies before launching, (2) invest in survey software with rigorous information security technology, (3) use bot-detection features available in survey software before launching, (4) design questionnaire items to identify bots and fraudulent actors, (5) tailor criteria for identifying fraudulent and bot responses to the characteristics of the target population, (6) avoid recruitment in public social media groups, (7) engage community leaders in tailored and targeted recruitment, (8) avoid advertising incentives, (9) shut down compromised links rapidly, (10) communicate with the Research Ethics Board about disruptions, and (11) combine automated and manual methods to identify potentially fraudulent responses on time.