Surveys are widely used across the social sciences, business, and even the natural sciences to draw conclusions about human behavior. When important, high-stakes decisions are being made on the basis of such tools as surveys, we hope that the inferences that lead to those decisions are sound.
Unfortunately, bugs creep into surveys, much in the same way that bugs creep into computer programs: authors of surveys neglect edge cases, they introduce bias, and they fail to consider how the survey will be received in the wild. Many of these bugs are well-known to those who use surveys regularly in their research. Practitioners handle these bugs by running pilot studies and performing post-hoc statistical analyses on the results obtained, ensuring the quality of their responses.
Unfortunately, there are some bugs that are not as widely known, and there are some bugs that cannot be diagnosed through post-hoc analyses. SurveyMan addresses these problems by controlling the experimental setup of a survey. By automating large portions of the deployment and debugging of surveys, SurveyMan allows the experimenter to conduct reliable, repeatable surveys on the web at scale.
SurveyMan is a collaborative research project between the
departments of Computer Science and Linguistics at the University of
Massachusetts Amherst. The code is primarily the work
Other student collaborators on code and research
are @mmcmahon13 and
is the principle advisor, with @jpater providing input from