Online Survey Respondents’ Reactions to Required Questions

The availability of easy to use online survey software has led to a deluge of do-it-yourself survey researchers, many of whom have little experience in questionnaire or research design. One of the most common errors committed in such surveys is the improper use of “required” question, where the survey respondent is not allowed to continue in the survey unless the question is answered. The problem with this design feature is that if a valid choice is not offered, the survey respondent must either give an incorrect answer to proceed, or terminate the survey. In order to get an empirical sense of how survey-takers respond to such situations, two studies were conducted, both employing 600+ respondents of a Harris Interactive/Nielsen metropolitan area online panel. Respondents were asked their opinion of a fictitious feature of the Dodd-Frank Wall Street Reform and Consumer Protection Act under varying required and unrequired response options including whether “Don’t know” was offered as a choice. As expected, we found that Don’t Know was overwhelmingly chosen when offered. Also as expected, when a response was required and Don’t Know was not offered, respondents chose the midpoint of the 5-point scale. However, contrary to our expectations, even those who were not required to answer a question (could skip to the next one) chose the midpoint of the scales rather than choosing to skip the question. This was true of all scale formats tested. We conclude that there are unexpected threats to data quality when using required questions and such questions should be used with great caution.