Plenty of business owners have learned not to rely on ‘gut feelings’ to make important business decisions. Getting input from your audience will help you broaden the scope of your business ideas. But are all the answers that you’re getting really useful? Have you ever wondered why not? Before you answer that question, think about what kind of data you need for your survey. Is it qualitative data or quantitative data? Do you want more interpretations, or are you willing to settle for just numbers?

I know, it’s a lot to think about. Don’t worry — we’ll help you! Surveys are what we’re good at.

Open or closed?

Survey questions come in two main varieties: open-ended and closed-ended. Each variety has its own strengths. With closed-ended questions, respondents are given a limited set of answer options. This makes it easier to code the results, because you can assign a value to each response and analyse them mathematically. With open-ended questions, respondents can create their own answers. This allows for a wider range of insight and interpretation, and helps you dive deeper to explore the nuances of an issue.

While there are some obvious benefits in including closed-ended questions to your survey, you should consider the pitfalls as well. You might be thinking, “But closed-ended questions will help me get right to the point!” But you should think again. It’s common to assume that closed-ended questions give you better results, but in reality, you are only limiting your respondents’ options.

Closed-ended questions include a list of pre-selected options for the respondents to choose from. For instance, do you want pizza or a burger for dinner? The problem is that you might not be interested in either — maybe what you actually want is fruit, but if that’s not one of the answer options, you can’t make that opinion count. So before you can use a closed-ended question to capture quantitative data, you need to do your qualitative research and figure out what options you need to offer your respondents to collect their opinions accurately.

Why does it matter?

According to Survey Questions: Handcrafting the Standardized Questionnaire, by Converse and Presser, 1986, “The open and closed-ended formats are different tasks for respondents. In the open-ended task, respondents write down what is readily available in their minds, whereas in the closed-ended task we have respondents focus their attention on specific responses chosen by the investigator.”

The above quote clearly explains how the type of question you select affects the answers even more than you might realize.

What’s the use of giving a survey when you’re actually forcing people to select from the answers you chose? The image below is a result for a study conducted by Pew Research in their November 2008 post-election survey. The respondents were asked, “What one issue mattered most to you in deciding how you voted for president?” The group was divided into those who received a closed-ended question and those who received an open-ended one.

This demonstrated that asking the same question in two different ways brings in different results. It is evident that 35% of the respondents were forced to choose an answer that wasn’t actually what they wanted, which is a HUGE problem for the accuracy of the survey. Hence you get a clearer analysis when you choose open-ended questions.

Strike a good balance.

So if open-ended questions are more effective at getting accurate answers, but closed-ended questions make it easier to tally your results, how do you decide which one to use? You don’t have to. A semi-structured questionnaire with a good blend of both types is the most effective way to tap into the power of both open-ended and closed-ended questions. This type of questionnaire can bring out answers you wouldn’t have even considered before.

Let’s end with an example.

What are the features you like about Zoho Survey?

• Real-time reporting

• Offline surveys

• Collaboration options

• Customizable themes

• Easy-to-use survey builder

Tell us below if there’s something else you like about us?

And that, folks, is how it’s done! You end up getting useful and accurate results.