Surveys are valuable for understanding your target audience better, and can be a great way to create original research that earns press mentions and backlinks. But creating a survey that accomplishes its intended goals is harder than you may think.
Survey errors have significant consequences. They can cause high rates of survey abandonment, cause your audience to feel annoyance with your brand, make you look sloppy or biased, or leave portions of your audience feeling left out. All of which is likely the opposite of what you intend to get out of the survey to begin with.
Learning to recognize and avoid common survey errors is a skill—and not one your average marketer necessarily has. Before you create your survey, make sure you know the most common problems so you can steer clear of them.
1. Overly long surveys
You get an email asking you to take a survey and think, “Sure, I’ve got a few minutes.” Ten minutes later, you’re running late for a meeting and not even sure how much more of this survey is left. This is a familiar experience, and one that’s sure to cause high rates of survey abandonment.
SurveyMonkey’s analysis of survey length found that completion rates dropped anywhere from 5% to 20% for any survey that took over 7 to 8 minutes to complete. People are often happy to complete a survey that will help create valuable research or provide customer feedback as long as the ask isn’t too high. Turns out, a time commitment of over 8 minutes is the tipping point for most.
Many survey tools will provide an estimate of how long your survey will take to complete. If possible, try to keep it around 5 to 6 minutes or less, and consider 8 minutes a hard cutoff. Figure out which of your questions are the most important, and be willing to ditch the rest.
2. Leading questions
Leading questions are worded in a way that makes the respondent more likely to choose one answer over another. Often the difference between a useful question and a leading one is as simple as the inclusion of a persuasive adjective. And while sometimes leading questions are intentional, survey creators can easily end up writing leading questions accidentally because they bring their own bias to the process. If you write a question hoping for a particular answer, your preference can seep into the wording itself.
A question like, “Will you endanger employees by forcing them to come to the office during a pandemic?” clearly expects a specific answer. Even if you have strong feelings on the subject, to get accurate answers you need to rework the question to remove your own opinion. Something like, “How likely are you to have employees come back into the office before a vaccine is available” will receive more accurate and measured responses.
3. Confusing questions
A good survey question will be simple and straightforward. But often, survey creators assume terms or phrasing that’s obvious to them will be easily understood by survey takers as well. Review your questions to look survey errors like jargon, acronyms, or wording that’s confusing.
If a respondent has to turn to Google to understand what a question is trying to say, they’re more likely to abandon the survey than bother with that extra step. Consider running your surveys by a friend or colleague that’s not in your industry to check for anything that may be confusing.
4. Question types that don’t match the question
When creating a survey, you’re able to choose between a number of question types (multiple choice, short answer, ranking answers, etc.). Selecting answer types or options that don’t adequately match the question is an easy error to make.
For example, a question that requires nuance to answer, but only provides yes and no responses with no room for explanation, will leave people struggling to answer accurately. And a multiple choice question that doesn’t include all possible answers will leave some of your respondents feeling left out. If you ask respondents how many kids they have, but don’t provide an option to say “none,” what are childfree respondents supposed to do?
When choosing your question type, consider carefully whether your choice makes sense for what you’re asking. And make sure your answer options cover all possible responses.
Pro tip: Including an Other choice with a write-in option for multiple choice questions can cover a lot of bases here!
5. Sending surveys too often, or too early
If a person gets hit with a survey the first time they visit your website, or a customer feels like they’re getting survey requests constantly, you won’t get responses. You’ll get annoyed visitors and customers.
Every time you send a survey, you’re essentially asking people for a small favor. You want to be cognizant of how often you’re asking people to do something for you. If your audience feels like they’re facing a constant barrage of survey requests, they won’t feel generous with their time. And it will negatively influence their feelings about your brand.
6. Asking two questions in one
This is sometimes referred to as a double-barreled question, and it will both confuse respondents and muddle results. Any time your question packs two different issues into one question, people are forced to answer one way even if their response would be different for the two parts of the question.
For example, asking “how concerned are you about this year’s budget and strategy?” is forcing anyone who feels one way about the budget and another about the strategy into providing one answer for both. And your survey results are less valuable when you can’t separate out which issue respondents are referring to.
7. Sending a donation solicitation or sales pitch disguised as a survey
This tactic seems to be especially common in an election year (looking at you, political parties). But it’s disingenuous, and highly likely to cause annoyance. Surveys should genuinely be for learning from and about your audience. If they’re instead a ploy used to seem like you’re listening just long enough to get their attention—only to then ask for donations or a purchase, you’re going to drive people away.
Before you create and send a survey, clarify what your goals are. If you’re only using it to get donations or increase product sales, you’re better off with other tactics. If you want to learn what your audience is thinking and feeling, then you’re on the right track.
8. Asking for unnecessary personal information.
For some types of surveys, collecting demographic information is an important part of getting the information you need. But often, it leads to survey errors. Asking someone to provide their gender, race, or income when it won’t make a difference to understanding your results could make some respondents feel uncomfortable, and may make them choose to forego completing the survey entirely.
If you do determine that you need demographic information, make sure you set up your questions to be inclusive. A question about gender that only offers male and female as options risks leaving some of your audience out. And consider including an explanation for why you need the information and how you’ll be using it, so respondents will know that you respect their privacy.
Create Surveys That Work For You And Your Audience
A well designed survey can provide you with valuable information that fuels more successful marketing campaigns, helps improve your company’s products, or becomes valuable data you can package into content. But you’ll lose out on that potential value if your survey is riddled with errors. Take care in designing your survey to ensure people will be willing to take it, and able to answer all your questions honestly and accurately.