The Effects of the Neutral Response Option on the Extremeness of Participant Responses

Abstract

Previous research suggests that when presented with a neutral response option, people will be more likely to select that option than report their actual opinion. The current study examined the extremeness of participant responses to sensitive statements as affected by the availability of a neutral response option on a Likert Scale. Contrary to past research, participants who reported their attitudes to sensitive topics on a five-point scale (neutral response option present) had more extreme responses than those who reported on a six-point scale. These findings suggest that the removal of a neutral response option may not always result in participants taking extreme stances on sensitive issues.

The Effects of the Neutral Response Option on the Extremeness of Participant Responses

Psychologists commonly use Likert scale questionnaires to collect opinion data. Since Renis Likert first introduced the Likert scale in 1932, the formatting of the Likert scale has been a source of dispute in regards to reliability and validity. Several topics within this dispute include the number of response options, the labeling of points, and the necessity of a neutral response option (Johns, 2005; Lozano, Garcia-Cueto, & Muniz, 2008).

Although Likert advocated for the use of the 5-point scale, researchers have since argued for more points to increase the reliability and validity of the scale. Weijters, Cabooter, and Schillewaert (2010) found that more options decreased the occurrence of extreme response styles, and Lozano, Garcia-Cueto, & Muniz (2008) suggested that reliability increases when there are more points. These findings provide evidence for including more points on a Likert scale but the issue is  ongoing.

Another question that has been raised in regards to formatting of the Likert scale is the labeling of points. Research has found three significant consequences of labeling points. First, respondents tend to be attracted to labeled points; thus labeling only end points may result in a bias towards extreme answers (Weijters et al., 2010). Second, labeling all points can lower extremeness of response due to the attractiveness of the intermediate options. Finally, labeling all points increases levels of positivity bias, the tendency to respond with a positive answer (i.e. agree or strongly agree) to a question regardless of what the question is asking, among participants (Krosnick, 1991). Therefore, there are negative sides to both labeling only end points and labeling all points on a scale.

The neutral response option is the biggest source of dispute surrounding the Likert scale. Originally offered in an effort to avoid false responses (Bishop, 1987), the neutral response option enabled people who were ignorant about or indifferent to a subject to select no opinion or neutral instead of being forced to choose a response that did not reflect their true beliefs (Johns, 2005; Krosnick et al., 2002). Although designed with the intention of reducing instances of false responses, studies show that the inclusion of a neutral or “no opinion” option significantly increases the number of people stating they have no opinion when they actually do (Bishop, 1987; Johns, 2005; Kalton, Roberts, & Holt, 1980; Krosnick et al., 2002; Nowlis, Kahn, & Dhar, 2002). Three factors likely influence a participant’s decision to falsely report via the neutral option: cognitive effort, ambivalence, and social desirability.

Reasons People Choose the Neutral Response Option

People have a tendency to satisfice, or avoid the cognitive effort required to pick a satisfactory answer when providing attitude reports (Krosnick et al., 2002). For each item on a questionnaire participants must interpret the question, recall related facts and memories, interpret the information to form an opinion, and then apply this opinion to the relevant Likert point (Johns, 2005). Particularly when unmotivated, participants may satisfice and choose a neutral option (Garland, 1991; Johns, 2005).

People also pick neutral options because of ambivalence. Bishop (1987) showed that people’s responses in public opinion polls tend to gravitate towards neutral because they want to avoid the negative feelings associated with their conflicting feelings on an issue. Additionally, picking a neutral option allows people to avoid the cognitive effort needed to choose between their positive and negative feelings on an issue (Nowlis et al., 2008).

Social desirability is a third factor that influences the choice of a neutral option. Krosnick et al. (2002) suggested that participants choose a neutral option when they are reluctant to voice a socially undesirable opinion; however, Stocke (2007) has shown that people tend to be more honest on a self-report survey than with an interviewer when the topic pertains to issues such as racial attitudes.

As a result of of the research on satisficing, ambivalence, and social desirability, some researchers suggest doing away with the neutral option on Likert scales for several reasons (Garland, 1991; Krosnick et al., 2002; Kalton et al., 1980). First, the removal of the neutral option forces people to use cognitive effort thus countering participant tendencies to satisfice (Krosnick et al., 2002). Secondly, removing a neutral option forces ambivalent persons to exert cognitive effort and use what they perceive to be the most important point of an argument to make a decision (Weijters et al., 2010; Nowlis et al., 2002). People with higher ambivalence will lean towards more extreme options (Nowlis et al., 2002). Finally, Garland (1991) argued that with the removal of the neutral option, people are forced to use cognitive effort to think about their true feelings on the subject. This negates the effect of social desirability bias without changing the participants’ opinions, decreasing instances of social desirability.

The Current Experiment

The current experiment examines whether the removal of a neutral response option increases the likelihood of a person taking a more extreme stance on a sensitive topic. Based on the Nowlis, et al. (2002) findings that people with higher ambivalence are more likely to pick an extreme option when no neutral response option is given, we expect to find that participants will take more extreme stances on sensitive topics when the neutral option is removed. The assumption is that participants will have higher ambivalence for sensitive topics. This assumption is based on previous research conducted by Tourangeau and Yan (2007), which concluded that a question is sensitive when it is intrusive (asks about a taboo or controversial subject), when a person perceives their anonymity to be threatened, or when the question has a socially undesirable response.  Boysen, Vogel, & Madon (2006) found that people are more likely to give their true opinion when a measure is given privately rather than publicly because their anonymity is protected, which supports the findings of Tourangeau and Yan (2007).

Krosnick (1991) suggested that the more experienced a person is at thinking about a topic, the easier it is to retrieve information for questions. Sensitive topics are discussed frequently in popular media because they pertain to so many people. Participants in our study will therefore likely have already formed opinions on the subjects and will be able to easily recall information on the subjects. The ease of recall (which decreases task difficulty for our participants), the increase in motivation (due to people wanting to express their opinions), and the increase in ability from participants with higher education all decrease the likelihood of satisficing occurring.  Of the two main components one should look at in a Likert scale, we are focusing on extremeness rather than direction of response (Peabody, 1962) because it is not pertinent to our study to know which direction people lean on sensitive topics. We are solely interested in the effects removing the neutral option from a Likert scale has on extremeness of response. In addition, this study is expected to provide further evidence for Garland’s (1991) suggestion that the removal of a neutral stimulus reduces social desirability.

Method

Participants

A sample of 99 college students (88 females, 11 males, age range: 18-23 years) volunteered to participate in the study in exchange for extra credit in undergraduate level psychology courses. The majority of participants were underclassmen (n = 69).

Materials and Procedure

All participants read a set of 12 items about four controversial topics: homosexuality, abortion, obesity, and gun control (see Appendices A and B for questionnaires). Examples of items included “homosexuality is a choice” and “abortion is murder.” Items were grouped based on their topics and tested for reliability (see Table 1).  All composites had sufficiently high reliability with the exception of the obesity composites; therefore, obesity was not included in the analyses.

The control group responded to each item on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree) that contained a neutral response option (3, unlabeled). The experimental group answered items on a 6-point Likert scale (1 = strongly disagree to 6 = strongly agree) that did not contain a neutral response option. Participants completed the experiment in a classroom setting. After providing consent participants completed a randomly assigned version of the questionnaire. The experimenters debriefed participants at the conclusion of the study.

Results

One participant’s data was excluded from the analysis due to failure to understand questionnaire. Prior to conducting this analysis, several steps had to be taken to ensure the viability of the data. First, any reversed items on the questionnaire had to be recoded. Following this step, a reliability analysis was run on the topics.

Because the items were rated on different scales (5 vs. 6 points), z-scores were created so that the scores would be comparable. Due to the controversial nature of the questionnaire items, participants could easily score at either end of the scale. For example, one participant could have an extreme z-score of -3 (indicating disagreement with an item) whereas another participant could have an extreme z-score of +3 (indicating agreement). Because these scores would cancel each other out, all z-scores were squared, resulting in data reflecting only extremeness, not direction.  Finally, topical composites were created by taking the mean of the squared z-scores for all items relevant to a given topic.

A Mixed Model ANOVA was conducted to test the effects of scale type (between-subjects) and topic (within-subjects) on response extremeness. There was a significant main effect of scale type, F (1, 97) = 5.12, p = .026. Participants answering items on the five-point scale (M = 1.28, SD = .72) scored significantly more extreme than those answering items on the six-point scale (M = .98, SD = 1.11), which did not support the hypothesis. Results of within-subjects’ effects were also found to be significant, F (2, 194) = 8.92, p < .001. Further exploration of the topics showed that the abortion topic (M = 1.45, SD = 1.10) elicited significantly more extreme responses than gun control (M = .98, SD = .72), p < .001, and homosexuality (M = .98, SD = 1.13), p = .001. The analysis of differences between homosexuality and gun control yielded no significant results, p = .997. The results suggest that certain topics may be more sensitive than others.

Discussion

Previous research has suggested that the presence of a neutral response option or a nonresponse option increases the likelihood of participants selecting this option (Kalton, Roberts, & Holt, 1980; Krosnick et al., 2002; Nowlis, Kahn, & Dhar, 2002; Bishop, 1987; Johns, 2005). Numerous explanations for this effect have been posed such as satisficing, ambivalence, and social desirability bias. However, the findings of the current study suggest the opposite. Participants who answered the sensitive items on a five-point scale had significantly more extreme responses than participants who responded to the same items on a six-point scale. These findings suggest that further research is necessary to analyze the relationship between the number of options on a scale and participants’ responses to sensitive topics.

The number of options given to participants may explain the results. Weijters, Cabooter, & Schillewaert (2010) found that the more options one has, the less extreme their responses. This theory was supported in our study with people who answered on the five-point Likert scale having more extreme responses than those who answered on the six-point Likert scale. Future research could examine the relationship between a four-point and a five-point scale.

Although we did not hypothesize an effect of topic on response extremeness, we found    that the abortion topic had significantly more extreme responses than gun control and homosexuality, while homosexuality and gun control did not differ. The significant difference between topics suggests that the topic itself affects how participants respond to items. The lack of an interaction between topic and presence or absence of a neutral response point suggests that responses to topics are consistent regardless of whether the participants answered on a five-point or six-point Likert scale.  The data suggests that people have stronger views on abortion than they do on gun control and homosexuality. These findings may be due to the emphasis social media places on these topics.

Mean response extremity served as the dependent variable in this experiment. Future studies on this subject could look at response variability, how widely the scores are distributed from the mean. Future studies should also make use of a preexisting attitude questionnaire with demonstrated reliability.

The topic of Likert scale types and participant responses is a field that needs further research. This research shows that there is a difference between scale types when used to test the extremeness of participant responses. This is contrary to previous findings by Krosnick et al. (2002), suggesting that further research is necessary to determine the relationship between scale types and sensitive topics. These results are important because it shows that Likert scale type has an effect on participant responses when sensitive topics are being surveyed. Further research is necessary to determine the nature of this effect.

References

Bishop, G. F. (1987). Experiments with the middle response alternative in survey questions. Public Opinion Quarterly, 51, 220-232. Retrieved from http://www.jstor.org/stable/2748994

Boysen, G. A., Vogel, D. L., & Madon, S. (2006). A public versus private administration of the implicit association test. European Journal Of Social Psychology, 36, 845-856. doi:10.1002/ejsp.318

Garland, R. (1991). The mid-point on a rating scale: Is it desirable? Marketing Bulletin, 2, 66-70.

Guy, R. F., & Norvell, M. (1977). The neutral point on a Likert scale. Journal of Psychology: Interdisciplinary and Applied, 95, 199-204. doi: 10.1080/00223980.1977.9915880

Jamieson, S. (2004). Likert scale: How to (ab)use them. Medical Education, 38, 1217-1218. doi: 10.1111/j.1365-2929.2004.02012.x

Johns, R. (2005). One size doesn’t fit all: Selecting response scales for attitude items. Journal of Elections, Public Opinion & Parties, 15, 237-264. doi: 10.1080/13689880500178849

Kalton, G. G., Roberts, J., & Holt, D. D. (1980). The effects of offering a middle response option with opinion questions. Journal Of The Royal Statistical Society. Series D (The Statistician), 29, 65-78. doi:10.2307/2987495

Krosnick, J.A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236. doi: 10.1002/acp.2350050305

Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, W., Kopp, R. J., & … Conaway, M. (2002). The impact of ‘no opinion’ response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66, 371-403. Retrieved from http://www.jstor.org/stable/3078768

Lozano, L.M., Garcia-Cueto, E., & Muniz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 4, 73-79. doi: 10.1027/1614-2241.4.2.73

Marquis, K. H., Marquis, M. S., &  Polich, J. M. (1986). Response bias and reliability in sensitive topics surveys. Journal of the American Statistical Association, 81, 381-389. Retrieved from http://www.jstor.org/stable/2289227

Nowlis, S. M., Kahn, B. E., & Dhar, R. (2002). Coping with ambivalence: The effect of removing a neutral option on consumer attitude and preference judgments. Journal of Consumer Research, 29, 319-334. doi: 10.1086/344431

Peabody, D. (1962). Two components in bipolar scales: Direction and extremeness. Psychological Review, 69, 65-73. doi: 10.1037/h0039737

Stocke, V. (2007). Determinants and consequences of survey respondents’ social desirability beliefs about racial attitudes. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 3, 125-138. doi: 10.1027/1614-2241.3.3.125

Tourangeau, R., Groves, R. M., & Redline, C. D. (2010). Sensitive topics and reluctant respondents demonstrating a link between nonresponse bias and measurement error. Public Opinion Quarterly, 74, 413-432. doi: 10.1093/poq/nfq004

Tourangeau, R. & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859-883. doi: 10.1037/0033-2909.133.5.859

Wakita, T., Ueshima, N., & Noguchi, H. (2012). Psychological distance between categories in the Likert scale: Comparing different numbers of options. Educational and Psychological Measurement, 72, 533-546. doi: 10.1177/0013164411431162

Weijters, B., Cabooter, E., & Schillewaert, N. (2010). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27, 236-247. doi: 10.1016/j.ijresmar.2010.02.004

Appendix A

Questionnaire used in Experimental Group of Study

For the following questions please write the number from the scale above that best reflects your opinion.

1—————-2——————3——————4——————-5——————6

Strongly Disagree                                                                                               Strongly Agree

  1. ________ Homosexuality is a choice.
  2. ________ Obesity can be helped.
  3. ________ A woman should have the right to choose if she wants an abortion.
  4. ________ Obese people should be required to exercise.
  5. ________ People have the right to bear arms.
  6. ________ Gun control is infringing on the 2nd amendment.
  7. ________ Abortion should be illegal.
  8. ________ Homosexuality is morally wrong.
  9. ________ Banning guns would save lives.
  10. ________ Homosexuals should have the right to be married.
  11. ________ Abortion is murder.
  12. ________ Obesity is a disease.

Please answer the following questions:

Age:_______

Gender: M    F

Class: Freshman     Sophomore     Junior     Senior

Appendix B

Questionnaire Used in Control Group of Study

For the following questions please write the number from the scale above that best reflects your opinion.

1————————2————————3————————-4————————5

Strongly Disagree                                                                                                       Strongly Agree

  1. ________ Homosexuality is a choice.
  2. ________ Obesity can be helped.
  3. ________ A woman should have the right to choose if she wants an abortion.
  4. ________ Obese people should be required to exercise.
  5. ________ People have the right to bear arms.
  6. ________ Gun control is infringing on the 2nd amendment.
  7. ________ Abortion should be illegal.
  8. ________ Homosexuality is morally wrong.
  9. ________ Banning guns would save lives.
  10. ________ Homosexuals should have the right to be married.
  11. ________ Abortion is murder.
  12. ________ Obesity is a disease.

Please answer the following questions:

Age:

Gender: M    F

Class:  Freshman      Sophomore      Junior      Senior

One Response

Leave a Comment


*