Back to the Basics – Survey Design
Decision Sciences & Innovation empowers our clients with end-to-end quantitative research solutions and custom advanced analytics. We deliver insights that are both easy to understand and actionable, enabling our clients to make better decisions for their business. Our comprehensive approach equips our team with the tools and confidence to solve complex business problems with our extensive and continuously evolving analytical toolbox and to translate data-driven insights into business solutions.
Survey design is key to obtaining quality data and results. Too many times, we as researchers are focused more on methods for keeping data clean or gaining as much information as possible that we often forget the fundamentals of survey design. As a result, we generate surveys that fall short of our intentions. With careful consideration and forethought, remembering survey fundamentals, in addition to utilizing data cleanliness traps, we generate quality and engaging surveys.
There is no perfect survey, but sticking to the five fundamentals below, will result in efficient, attractive, and insightful surveys.
Keep It Simple, Silly. One of the most common mistakes we can make in survey design is over-engineering. This is especially true with today’s technology, tools, and data-driven mind set. While it may seem engaging and useful to include multiple check-boxes, rankings, slider bars, drop-down menus, etc., it is crucial that we maintain simplicity and function when designing the survey. Without this forethought, the final survey will be excessively long, have high respondent fatigue, and potentially contain messy data.
Jargon
It is understandable to assume respondents know more than they do. Even though we include definitions and activities to expose respondents to material, whether conducting research among the general population or targeting respondents who work in specific fields, our experience suggests there is not a consistent understanding of terms or instructions. Thus, it is imperative that we ensure the survey is succinct and comprehensible, free of jargon and industry-specific acronyms.
Respondent Focus
Building on the previous point, we want to make the survey easy to follow, allowing respondents to give their full attention to the survey tasks. Respondents may be unable to recall the succinct and jargon-less information we have laid out in the survey or instructions for completing a task. Keeping this in mind, it is wise to continuously remind respondents of the information they need to recall or instructions they need to follow. This can easily be done via mouse overs or links to a separate tab without lengthening the overall survey with redundant information.
Plan Ahead
The previously mentioned points are easy to address individually, but planning ahead will allow you to address them all simultaneously. The biggest consideration when planning the survey is the overall objective of the project. What decisions are going to be made? What is the most important information to get insights on? What is the analytic plan? Focusing the survey on the main information and keeping it concise allows for a more effective and engaging survey for respondents.
Not So Basic: Develop the Analysis Plan First
This may seem counter-intuitive, but developing the analysis plan first is one of the best practices for ensuring a high quality survey is designed. Moreover, it further facilitates all of the previous points. Every analysis has its advantages and disadvantages. The analysis plan allows us to develop questions supporting the advantages and mitigating the disadvantages. Furthermore, each analysis requires specific types of data and assumptions, which lead to simple and correctly formatted questions. By knowing what the analyses are before jumping into designing and programming the survey, you prevent yourself from over- (or under-) engineering the survey. For example, the anticipated analytic techniques may not work with a ranking exercise, or you need to use a clearly defined algorithm for defining segments, which could rule out multiple check-box questions. Similarly, you know exactly what information is relevant, required, or expendable in the context of the end goal. This analytics-driven approach results in a survey that directly addresses and complements the research objective(s).
Sometimes, statisticians/data analysts/data scientists are brought in for one thing, running the analyses. However, bringing these individuals in early in questionnaire design leads to rich, high quality outcomes.
While there is much more to questionnaire design than what has been discussed here, being mindful of the basics and planning ahead with analytics will place researchers well on the path to having high quality design and data.
