Part
01
of one
Part
01
When performing survey questionnaires for customer satisfaction, what are the accepted standards for best answer formats? For example, "How satisfied were you with your hotel room today?"... Should the answer have a 1-10 scale? Or should the answer be 5 options of "very unhappy....unhappy...happy... etc..." ?Liechert scale
Hello! Thanks for allowing us to assist you with your customer satisfaction questionnaire. Proper survey question development is a specialty of mine, and I'm happy to share my experience and research with you. In short, you want to ensure that your answers are labeled clearly and unambiguously, with the lower value answers on the left and the higher value answers on the right. For most questions, five to seven options is the recommended standard, but this can vary depending on the needs of each question and the comfort level of your respondents. Please keep reading below where I go over all of this in more detail with examples.
METHODOLOGY
I have previously studied this topic extensively, and supplemented my knowledge with additional research here in order to provide you with the best quality recommendations. With the Likert and Likert-type scale questions, there can be different best practices depending on the situation and audience. Where relevant, I present explanations of how to determine the best approach for your specific situation.
OVERALL GUIDANCE
- Ask absolute questions and label all options in a way that minimizes personal interpretation. For example: "How well did you sleep? (1=poor, 2=okay, 3=good, 4=excellent)"
This question has a clear scale, but sleep quality is influenced by many different factors, so any answer will need a follow-up to be of any real value. The question also doesn't specify the type of sleep, so some respondents may count only night-time sleeping and some may also include naps during the day.
In cases like this, give all respondents the same measurable point of reference for comparison, such as: "Compared to your usual ___, rate your experience. (much better, slightly better, the same, slightly worse, much worse, N/A)"
- Ask your most important questions early to make sure they aren't influenced by respondent fatigue at the end. Also, ask overall sentiment questions before digging into specifics to avoid contaminating their original "gut feeling."
- Make sure the questions you ask are about things you can reasonably expect respondents to remember. Clearly state whenever appropriate whether or not you need an exact answer or if an estimate is acceptable, or if any other special instructions apply.
ENSURE RESPONDENT COMFORT
It's important that your respondents feel comfortable answering the questions of your survey. Confusing or controversial questions can turn off a respondent and cause them to give inaccurate answers or quit the survey altogether.
- Ask about one thing at a time and don't make assumptions about what respondents will answer. For example, consider the question, "was the dinner tasty and filling?" If the respondent didn't have the dinner, they need a way to indicate this. If they did have the dinner, what if it was tasty but not filling, or vice versa? This complex question addresses two variables and can be problematic and off-putting. Instead, ask simple questions that focus only on what you need to know and always give people an option that allows them to say "not applicable" or "not sure." If possible, allow optional free form explanations in case they have important information to express that can't be captured by the main survey question.
- Avoid excessive controversial questions which could cause respondents to withdraw. If you must ask questions about sensitive topics (including demographics such as race or income), put them at the very end of the survey. This way, if respondents choose not to answer them, it won't disrupt the positive flow of information in the earlier questions.
RANKING VS. RATING
The human brain's short term memory can only hold five to nine items for comparison, and the more items a respondent is asked to rank, the fuzzier the comparison becomes. For this reason, rating questions are often preferable to ranking questions, especially for verbal surveys with no visual reminders of previous decisions.
- Whenever possible, have the respondent provide ratings rather than rankings. That is, rather than ask, "rank these 10 options from best to worst," have them give each option an individual rating. Ranking also (usually) doesn't allow for indication of distance between rankings, which can be very important for both the respondents' expression of sentiment and your understanding of the data. On the other hand, ratings allow the respondent to focus only on one variable at a time (unless they choose to consider others to help them decide), making the task much less mentally challenging and the responses more objective. On your end, the ratings can usually be sorted to provide a ranking, anyway, but with built-in distance measures.
As an example, a rating of two items as "1" and "4" shows a distance/strength of 3, where as ranking the same two options as "first" and "second" wouldn't tell you how strongly the respondent felt about the comparison.
RATING SCALES
Now, touching on the question you mentioned in your request, there really isn't one definite rule for the number of options to use on a Likert scale. Providing five to seven options gives you reliable (in terms of accuracy) and valid data. Providing seven to 11 options is often more reliable, but is much more mentally burdensome for the respondent. As mentioned previously, the format of the survey also influences the number of options a respondent can comfortably manage. If the survey provides a visual reminder of the options for each question, it will be less taxing on your respondents to answer. Phone surveys should aim to keep the options as few as possible, with as little variation as possible from question to question.
Unless you have a group of highly motivated respondents, I recommend going with five to seven logical options. The key word here is logical, and will help you make the decision of whether to use an even number or an odd number of options.
Providing an even number of options (excluding N/A) leaves no middle ground, which can be important when your respondents have a definite opinion but perhaps they are apprehensive about making their feelings known. You should still only use as many options as make sense. For example, if your scale is "agree, somewhat agree, somewhat disagree, disagree, N/A" you're including two options that are logically redundant -- somewhat agree is arguably the same as somewhat disagree. Respondents are likely to be confused by this wording.
On the other hand, consider the options, "best, good, neither good or bad, bad, worst, N/A." The "neither" option gives the respondent a safe choice if they don't feel comfortable with the other answers. You will lose a little bit of accuracy by not forcing them to stand on one side of the line or the other, but (especially if you have a lot of questions like this), keeping the respondent comfortable might be the difference between a completed survey and an incomplete one. So, you'll have to weigh your needs and goals for the whole survey when deciding how much to press your respondents on any individual questions.
Things to keep in mind:
- One of the options should always be N/A, unless you ask a screener question before so respondents are only asked a question if it's applicable.
- However many options you use, the most important thing is that each option is clearly labeled and unambiguous. Avoid labeling only the ends of the scales, as each respondent can interpret the middle point differently (a little of both, neither, unsure, etc.)
- When people aren't sure of their answer, the scores will float toward the middle. That is, if you give people a middle (odd) option, most will choose that rather than a "don't know" option. When compounded with the previous point, you can see why it's very important to clearly explain what each value of your scale represents.
- There are differing opinions on the order in which scaled responses should be presented -- High/Best to Low/Worst, or vice versa. A traditional Likert scale is typically "Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree." However, it has been shown that there is a bias toward responses that are displayed on the left. Therefore, a survey using a traditionally-ordered Likert scale may show lower overall scores than a survey using a reverse Likert scale, with the higher/more positive values on the left. Since you are interested in the professional standard, I would recommend using the traditional Likert ordering of lowest values on the left and highest values on the right. It may make for a slightly more negative tone for your respondents, but will avoid the appearance of bias in your results. This document contains many examples of how to best word the options of your Likert scale questions.
CONSISTENT SCALES
Your goal will be to obtain mostly quantifiable responses for easier analysis of your data. In order to convert options such as, "happy, very happy, etc." into quantifiable values, it's absolutely critical that the scales be consistent.
- For example, take the question "how often did you use ___ while on vacation? (never, one time, two times, every day)" Spreading those options across a scale evenly would be difficult to do because every day could mean one time or 30 times, depending on the length of the vacation.
In these cases, consider the most logical scale or groupings based on what the answers mean. Is there really a big difference between one time and two times? If the question is, "how often did the brakes fail," yes, two times has a significantly different impact than one time. And there is little point in asking about "5-7 times," because realistically, you know there's a major problem if it happens more than once. Do this thinking beforehand so your respondents know that you understand what you're asking and you get the most meaningful information back from them.
- If you need to use multiple scales to accommodate your required questions, group same-scaled questions together so the respondent only has to remember one scale at a time. This is especially critical in phone surveys that lack the visual reminders of a paper or online survey.
- You must also ensure the scales align across multiple questions. For example, consider the following questions:
"How much did you like the movie? (did not like it, liked it a little, liked it a lot, N/A)"
"Do you think you will visit us again? (definitely not, probably not, not sure, probably yes, definitely yes)"
These two questions use different scales, so the data will have to be normalized if you intend to compare data from the two questions.
CONCLUSION
To sum up, the best customer questionnaire will have five to seven logical, unambiguous options for each response. The scales and order should be comparable across the survey and accommodate respondents' comfort levels to help ensure accurate feedback.
Thank you again for choosing Wonder for your research needs. I hope this information is helpful for you as you prepare your customer survey. If there is anything else we can do to assist, please let us know.