Addressing fraudulent responses in online surveys: insights from a web-based participatory mapping study.
Abstract
Web-based studies of human dimensions are increasing across environmental and socio-ecological disciplines. However, the prevalence of fraud threatens research quality. Increased fraud rates should be expected as surveys move progressively more online, motivated by expanding reach, cost savings and/or in response to COVID-19. Web-based research must better account for fraud to maintain confidence in findings. Practical diagnostic tools and data quality protocols are required to detect fraud and ensure results quality. Drawing on our experience using an online participatory mapping case study, we discuss methods to detect potentially fraudulent responses-and identify some limitations. We begin by reviewing the current state of knowledge on fraudulent responses or 'fraudsters' and its relative absence in environmental and socio-ecological disciplines. We then describe our research approach, the indicators and variables we used to detect and assess fraud and our decision-making process to eliminate suspicious responses without jeopardizing research integrity. We found that despite several preventative measures, many fraudulent respondents could provide survey responses and effectively mimicked legitimate respondents at first glance. By assuming each response to be 'potentially fraudulent', we determined that the complete screening of each respondent, while time-consuming, can limit the prevalence of fraud. We also determined that the most common data consistency checks (e.g. duration, trap questions and straight-liner checks) are unlikely to guarantee valid respondents. If not acknowledged and addressed, fraud has the potential to undermine data integrity, discredit research findings and limit the utility of results for policy. This study contributes to environmental and socio-ecological research by reviewing existing fraudster literature and using our experience with fraud to provide recommendations for researchers to address this problem. We encourage researchers implementing online qualitative research methods to thoroughly assess and report fraud, when possible, to ensure widespread knowledge of this growing threat.