bq. This experiment was designed to test the impact of factors hypothesized to increase the response rate but had the unanticipated effect of increasing survey error.
Daniel Merkle and Murray Edelman report on an exit poll experiment they conducted. They had interviewers approach a random subset of potential respondents with a colored folder that partially covered the traditional questionnaire. The folder was printed with the logos of the sponsoring media organization and had the words “Survey of Voters,” “Short,” and “Confidential” on it. The response rate in this condition was 5 points higher than in the “control” condition that didn’t have the folder. However, the error in the folder condition was higher: the Democrat’s share of the vote was overestimated.
Unresolved is the question of why Democratic voters like colored folders. Merkle and Edelman write:
bq. After conducting this study we initially hypothesized that the color logos of the national news organizations on the folder may have been perceived more positively by Democrats, leading to a greater propensity to respond among these voters. However, an experiment to test the effect of the logos by the second author using the 2000 exit polls did not find an impact of the logos on response rates or error.
bq. While this doesn’t rule out the logos as having an effect in this study, there are other possible explanations for the Democratic overstatement. The Folder Condition emphasized that the survey was short and confidential and also included more interviewer training on refusal conversion. These are standard methods for improving response rates but perhaps the message of “short and confidential” appealed more to Democrats than Republicans, and it may have been the case that Democratic refusals were more easily converted than Republican refusals.
More is here.