Any research requires primary data collection for analysis, which is where the skill of designing a questionnaire for market research becomes handy. The Questionnaire Design Process which I will elaborately discuss with you today will be meaningful in whichever primary research and data analysis task you take up.
You might be in your graduation studies, designing your very first survey questionnaire on Google Forms, or might be a research analyst at a leading organization. Our discussion will help you cover all bases, at whichever level of detail you wish to achieve, in this comprehensive pack on tips and tricks for Questionnaire Design Process.
Why do you need a questionnaire?
Every market research study, be it exploratory, descriptive, or causal, needs you to collect significant amount of data for analysis. How will you collect this primary data? And how will you ensure the collected data is in a shape where it can be easily used to generate interesting insights in line with your research objective?
You will realize that, if not in touch with this crucial skill of designing questionnaires, you will struggle to get things right. To ease your anxiety and worries, I am here with a compendium of the most detailed and yet, most concise ‘set of protocols’ to ensure a decent questionnaire design. ‘Decent’, because questionnaire design process does not involve direct calculations or logic. It is an art that needs to be appreciated in its full form to gain the magical results.
I will share with you the basics of how to design a good questionnaire, along with various steps involved in questionnaire construction, in depth. But first, let us understand in detail, the aspects that we will cover in this discussion.
So, without much ado, let us dive deep into the rich concepts of a successful questionnaire design process.
Why must you design the questionnaire just right?
While qualitative analysis involves data collection through Focus Group Discussions, Depth Interviews, and Ethnography studies, quantitative analysis needs to be collected based on the following specifications, to achieve meaningful results.
Data Collection Technique should be -
Data should be -
Consider this sample questionnaire for market research PDF, to understand how questionnaires typically look like.
We will understand the steps needed to create a questionnaire that is complete in itself. But, first, let us briefly understand the overall flow of the process of designing and delivering an effective questionnaire, through this interesting video -
Now we will discuss the questionnaire design process which will guide you at each stage of the process to finally launch a well-reasoned questionnaire.
Also Read: How to become a Market Research Analyst
Steps to make a wholesome Questionnaire
There’s no magic number or protocol to create the most suitable questionnaire for your audience. The number of questions, survey structure, type of responses expected, everything is subjective, based on research purpose, target audience, and analysis objective.
Still, the process can be categorized into various steps involved in questionnaire construction. Remember, fine-tuning the questionnaire is the creativity of a skilled researcher, but comes easily with practice.
1 Specify Information Needed
This gathers pre-requisite understanding of the research design process. As your project progresses, your understanding of information required gets clearer. But, these few points should be affirmed before moving to the next step –
- 1Identify components of problem to analyze, mainly research question or hypotheses
- 2Identify type and format of information needed to analyze
- 3To ensure the information identified in point #2 is sufficient to answer the research question, create a dummy table. A dummy table will depict the questionnaire outcome which will be used for data analysis.
- 4Identify target population of respondents (this will influence many following steps such as questions wording, and type of opinion to collect). Ensure you choose a very specific target group, to avoid making a broad questionnaire that doesn’t give meaningful depth in responses (such as more no-opinion answers).
2 Specify Type of Interviewing Method
Questionnaire design process has delivery method based on the complexity, duration and variation of questions. For example, a face-to-face interview can have lengthy, scenario-based, complex questions to be answered by respondents (due to respondents’ physical presence) But, for mail or internet surveys, simple questions and detailed instructions are needed. Computerized or Computer-assisted interviews can have complex skip patterns and randomization of questions to eliminate order bias (bias in answering a question based on the previous sequence of questions and the resulting thought-process). Further, based on mode of questionnaire delivery, questions language can be decided as conversational or direct.
3 Individual Question Content
- 1Is the question necessary? – Ensure that the response from the question contributes directly and significantly to your data collection (the Dummy Table), and ensure that a particular piece of information is captured only once in a questionnaire (no repetition).
- 2Are multiple questions needed instead of one? – Avoid double-barrel questions. What are double-barrel questions? A single question that tries to cover two issues. For instance, “Do you think Amazon Pantry is a cheap and convenient option to purchase groceries?” is a double-barrel question, whose answer can have multiple parts. Hence, the response’s reliability and validity are at stake. Instead, break the question into multiple parts, and collect responses for each component in an individual question. So, "Do you think Amazon Pantry is a cheap option for groceries?" and "Do you think Amazon Pantry offers convenient purchase options for groceries?" are now two distinct questions.
4 Overcoming Inability to Answer
My mother was asked to fill a customer feedback form at a retail outlet while exiting post a purchase on a busy Sunday evening. The form was 4 pages long and had many sections of open-ended questions. She ended up partially filling the survey, and submitted it for consideration. Imagine a sizeable ratio of such submissions for your survey, and the amount of data cleaning each inaccurate response would require.
You need to identify common problems faced by respondents while filling up your questionnaire –
To ensure quality of responses (or that respondents are aware of what you are asking), add filter questions in the beginning. Filter questions are added to screen respondents so that only the target audience actually send the responses for consideration.
Unable to remember on cue
How many times did you wash your hands this week?
When was the last time you called any customer care?
Were you able to answer these questions? Didn’t they send you off at a tangent, and ended up frustrating and distracting you? Instead of collecting factual information, which more often than not, is incorrect by over 100% of the actual answer, information can be collected in a range of time duration, or qualitatively. For example, now consider answering the questions again based on these options –
E. Numerous times
Inability to remember leads to any one of these -
Unable to articulate
You might find it difficult to describe the atmosphere in an airline while flying, but if given options to choose from, you will be in a better position to choose the best fitting answer. It is good to facilitate options with visual cues such as pictures, maps, or even detailed descriptions.
Unwilling or Disinterested
This is my personal observation that people who are busy or will be occupied in work soon tend to avoid filling questionnaires or rush through to get done. Often times, while delivering the questionnaire to respondents, timing plays a key role in getting the survey filled with the desirable attention you expect. Few pointers to hold respondents’ interest –
5 Choosing Questionnaire Structure
Open-ended questions are ones where respondents are free to answer in any way, with any type of information. While they capture general attitudes and behaviour of respondents and are much less biased in responses, unstructured questions pose major disadvantages. Coding the responses is one of the major issues, where contextual and subjective answers are difficult to accommodate within the analysis framework. Also, they may involve interviewer bias where the interviewer may selectively include or exclude certain pieces of information.
These have a specific set of response alternatives to choose from while responding. These are of three types –
- 1Multiple Choice - Respondent is given choice of responses to choose one or multiple from. For instance, a question asking “Do you intend to buy a smartphone within the coming 6 months?” can have response options as - 'Definitely will not buy', 'Probably will not buy', 'Undecided', 'Probably will buy', and 'Definitely will buy'.
MCQ-type questions face two issues -
1 Number of Alternatives - Response choices must include all possible variations and must be mutually exclusive. If the question becomes too complex (involving high information-processing), consider splitting the question into two parts.
2 Order or position bias - Respondents tend to check the first and last options to observe the range of variations, and often, among numeric choices, tend to answer around the mean. To eliminate this bias, randomize the order of options.
- 2Dichotomous - Response choices come in pairs, with ‘either’ ‘or’ options (Yes/No). These must be applied very selectively, and should be avoided in case there is any grey-area in answering. One major issue still persists about the influence of the questionnaire structure in indicating response. To overcome this, the split ballot technique can be used with questions having biased connotations in language are given to 50% respondents in its double-negation form and the other 50% as is.Consider the following example -“Individuals are more to blame than social conditions for crime and lawlessness in this country” produced 59.6% agreement. However, the opposite statement “Social conditions are more to blame than individuals for crime and lawlessness in this country” produced a 43.2% agreement, as opposed to the expected 40.4%. How can we overcome this? Both the formats of the question must be sent to half the randomized audience.
- 3Scale - Consider the sample question of purchasing a smartphone in the coming 6 months, and consider the following response choices -Definitely will not buy - 1Probably will not buy - 2Undecided - 3Probably will buy - 4Definitely will buy - 5This is another format of offering question responses, similar to the multiple-choice technique, and directly offers coded responses for data analysis.
6 Choosing Question Wording
Language in which questions are framed are known to make or break the deal when it comes to respondents investing time and effort in responding to your questionnaire. It is a crucial step because –
What can we as researchers do to tackle such problems? A 7-step guide will help you –
- 1Define the issue - Complete the context and meaning of the question clearly by the six W's concept – who, what, when, where, why, and the way.
- 2Use ordinary words - You might be a Shakespeare in Literature, but a survey isn’t your canvas of creativity and flamboyance. Match the vocabulary level of the respondents, especially in cases where your target respondents are responding to a very domain-specific questionnaire.
- 3Use unambiguous words - Words in the question and choices must have only one meaning in the context. For instance, consider the two sets of response options for the question “In a typical month, how often do you hop at departmental stores?” -
'Never', 'Occasionally', 'Sometimes', 'Often', and 'Regularly'
'Less than once', '1 or 2 times', '3 or 4 times', '4 to 6 times', and 'More than 6 times a month'
Which options seem clearer and more intuitive to select from? The second one, right? It is because there is no perception bias from the respondent leading to inconsistent data. Remember that certain words which are all-inclusive (“all”, “always”, “every”) and all-exclusive (“any”, “none”) might be perceived at different degrees by people.
- 4Avoid leading questions - Leading questions are ones which clue the respondent about the desirable answer, which results in yea-saying or acquiescence bias. A question saying “Will you consume fast-food knowing the associated health problems?” will attract more ‘No’s since the associated ill-effect mentioned right next to it tends people to say/believe that they won’t indulge. Try asking the question as “Will you consume fast-food?” and It will collect a more realistic response.
- 5Avoid implicit alternatives and assumptions - Implicit alternatives make it unclear, and increase the probability of choosing the alternative mentioned in the question. For instance, “Would you like to fly when travelling short distances?” will answer more inaccurate ‘Yes’ than “Would you like to fly when travelling short distances, or would you rather drive?”. Close preferences can be missed while framing the question, but not in most cases.
- 6Avoid generalizations and estimates - Ask specific questions, which seek direct answers, and frame questions such that respondents create no generalizations or own assumptions while answering. To find out domain-specific numeric information from respondents, do not ask “What is your family’s per capita expenditure?”. Instead, ask direct and easy questions like average expenditure per week, and number of family members, and calculate per capita expenditure by self based on responses.
- 7Use both positive and negative statements - For attitude and personality-recording questionnaires, often questions are worded as statements, with responses to be collected as degrees of agreements or disagreements. In such cases, it is better to have dual statements randomly (both positive and negative) such that each questionnaire has half positive and half negative interspersed statements.
7 Determining Order of Questions
Designing the flow of the questionnaire accurately is important. Try to create a flowchart clearly sequencing the questions that lead to the respondent being navigated to different sections of the flowchart (called branching questions).
8 Form and Layout
General guidelines of maintaining uniform formatting, spacing and positioning of questions affect the accuracy of end outcome of the questionnaire design process. Precoding of questionnaire responses must be done prior to launch. Precoding means assigning a code to every conceivable response for data collection. Questions and sections must be numbered serially, to easily control the directional flow of information from the questionnaire.
9 Reproducing the Questionnaire
Carefully reproducing the questionnaire for administration is another critical step in the questionnaire design process to ensure it actually gets filled with the right intentions. Visually, it should be de-cluttered, should appear professional and the questions and section breaks must be clearly visible and distinctly demarcated. Improve readability through visual cues such as instructions to questions in different font style, and sections of the questionnaire color-coded differently.
Online questionnaires must ensure access permissions to respondents and must include appropriate settings about including or excluding the respondents’ email IDs.
Even after such as detailed scrutiny of the questionnaire design process, there might be elements whose representation might create some unknown issues in the mind of the respondents while responding. Keep it a habit of pretesting the questionnaire on a smaller group of respondents with an aim to improve the questionnaire by identifying and eliminating potential problems that they face. You will also get a view of the data being populated and will be able to make changes even in the responses. It is good to analyze data collected here to ensure your research objective or hypotheses will be answered through the data collected.
Two ways of proper pretesting are protocol analysis and debriefing.
Finally, edit the questionnaire based on inputs obtained from pretesting. Your questionnaire is now finally ready to hit the road!
While an all-encompassing summary of the questionnaire design process would offer a good summary to the vast yet wholesome topic of how to design a good questionnaire, few steps to keep in mind are –
So, the next time you are designing a questionnaire for market research, follow this checklist of items to achieve in a smooth roll-out of your survey questionnaire.