Q: How should I design and score a Yes/No survey?

A Yes/No survey is easy to administer and score (from the researcher’s perspective) and easy to complete (from the participant’s perspective), but it is not always easy to design. Here are some points you need to keep in mind when creating such a survey.

• It is meant to be an ‘entry point.’ As the survey is easy to complete, involving only a Yes or No to each question, participants are less ‘resistant’ to such a survey. Thus, it is easier to find participants for such a survey. However, a Yes/No survey is usually intended as a starting point for a deeper exploration into the problem being studied. The survey allows you to get an initial ‘feel’ and can lead to a more detailed questionnaire or even an interview seeking more insightful responses.
• It is used to segregate participants. A Yes/No survey can be used to narrow down your target audience from the pool of participants. Thus, for instance, if you are studying a link between obesity and exercise, some of your questions could be Do you exercise, Do you consider yourself obese, Do you know your weight, and so on. Based on that, from the set that considers themselves obese, you could identify two groups: one that exercises and one that doesn’t.
• It can be used to assess consistency in awareness and response. A Yes/No survey with many questions (perhaps as the one you are planning) can sometimes be used to determine if participants are being consistent in their responses. This is done by asking the same question (or a question around the same concept) in multiple ways. In the earlier example, for instance, you could frame multiple questions to check their awareness of obesity from multiple angles: Do you know the definition of obesity, Do you know the meaning of body mass index (BMI), Does ‘fat’ mean the same as ‘obese,’ and so on. Note of course that all the questions won’t be around the same concept; they will need to be interspersed with other questions around other concepts. Thus, in the same example, there will need to be other questions around concepts such as exercise, calories, metabolism, and so on.
• The meaning of each question should be clear. As the survey involves answering a simple Yes or No, there should be absolute clarity in the meaning and framing of each question. Continuing with our example, a question like Do you exercise regularly can be open to interpretation because ‘regular’ can mean once in five days to some people and once in two days to others. Thus, if you wish to understand their regularity, a better question could be Do you exercise every day.
• It needs to be devised carefully and with consideration. Given what we just discussed, you need to devise a Yes/No survey with care, keeping in mind the objective of the survey (and of your study as a whole) and therefore the exact nature of information you are seeking from the participants. The survey could thus be a part of a mixed-methods study, where the survey forms an initial quantitative component that leads into a more focused discussion with a few selected participants, which forms the qualitative component. If needed, you may also provide a text box or space (depending on whether you are doing it online or offline) for participants to elaborate on their responses. Going back to the example, a participant who answers that they exercise every day may wish to qualify it by saying that they exercise every week day but not on weekends.

Scoring a Yes/No survey is the easy part. You simply need to tally the Yes and No responses for each question for all the participants and divide it by the total number of participants to get the percentages of Yes and No for each question. Of course, if it is a mixed-methods study, you will have to score (evaluate) more qualitatively.

All the best with the survey creation!

Related questions:

• What are questionnaire surveys?
• Should I write the definition and working of an instrument or scale that I used in my study?
• Do I need to take permission to use a scale that was published in a paper?