Most people don’t recognize the term “Likert scale,” but almost everyone knows the thing itself: the familiar questionnaire in which a respondent reads a statement and then picks an answer that best describes his or her reaction to it, ranging from, say, “strongly agree” to “strongly disagree.”

In the social sciences, the Likert scale is such a dexterous instrument that researchers can—and do—use it to gauge attitudes about everything from political issues to how romantic we consider ourselves to be. Not surprisingly, it has become the most widely used attitude scale in survey research.

 

Given its ubiquity, it would be easy to assume that the Likert scale is beyond reproach—an assumption Michael Lau, Assistant Professor of Counseling Psychology at Teachers College, believes would be a mistake.

 

“We take a lot of scientific methods for granted,” says Lau, who joined the TC faculty in 2007. “We’ve elevated science to a level where we trust it. Where we say: ‘There are no problems with these methods.’ Actually, if you look into methodology and philosophical assumptions, you’ll find a lot of gray areas. In other words, if you tinker with certain things methodologically, you can wind up with different results—or come up with spurious results.”

 

It is that possibility for distortion that drove Lau, an expert in quantitative methods, to examine some of the biases that can creep into Likert scales, specifically a phenomenon known as extreme response style (ERS), or the tendency of some respondents to choose the more extreme ends of a scale regardless of what a given statement might say.

 

There is nothing wrong with choosing a more extreme option, Lau says, if it is an accurate reflection of the degree to which a person endorses a given statement. If, however, the way that the scale is presented affects these responses, then the validity of the responses becomes suspect. So, as the focus of his dissertation at the University of Notre Dame, Lau tested three aspects of a Likert scale that could lead people to respond using the more extreme options.

 

One possible bias he examined was the use of absolute terms. Would, for example, a questionnaire that used the term “completely disagree” versus something less definitive, such as “strongly disagree,” lead more people to opt for an extreme response? He also analyzed whether labeling only the end points of a scale might lead to ERS. And he also considered the length of questionnaires and the role fatigue might play in leading people to choose an extreme answer.

 

Lau found that the length of a test and the labeling of only the extremes in a scale weren’t important. He detected little evidence of ERS in either case. The use of absolute terms, however, did result in evidence of ERS. The more absolute the terms, the more likely respondents were to choose one of the extremes.

 

Lau hastens to add, however, that he is not calling into question the overall accuracy of Likert scales. The effect he found was relatively small and would not have been significant enough to raise concern for most researchers. Nevertheless, he hopes his study will help to scholars who use Likert scales to understand potential biases.

 

“It’s really an issue of accuracy,” Lau says. “You want to get a sense that the results are not a function of how you put together the scale, but rather that it reflects solely a person’s attitude to whatever it is that you are measuring. My dissertation shows that how you label things in terms of absolute terms can have an effect.”

 

For now, Lau is at a loss to explain why an absolute term would make it more likely for people to select it than a less definite term. His original hypothesis was the opposite:  A less definitive term would more likely yield more extreme responses. One possible explanation, he says, is that a less definitive term simply doesn’t map fully a respondent’s attitude or feeling.

 

At TC, Lau has continued his general interest in research methodology by collaborating with a number of his colleagues in exploring methodological issues within a number of areas ranging from reviewing and critiquing research methods that could be effective in the study of racial microaggressions—a term that has come to mean various subtle insults directed toward minorities, often unconsciously—to using qualitative methodologies in building theories on gender roles in various ethnic groups. 

 

In the end, Lau says he has come away from his research with an appreciation of the power of the Likert scale—and its potential limitations.

 

“I think the goal of the dissertation was to inform,” he says. “If you are trying to run a study and putting together a questionnaire, what can you do to minimize issues of extreme responding? Can you use certain ways of phrasing the anchor points—the extremes—that might minimize extreme responding? The answer is that you can, and if you minimize potential methodological biases, you are going to get more valid and accurate results.”