Throughout academia, including in political science, women haven’t achieved parity with men. As this series explores, implicit bias holds women back at every stage, from the readings professors assign to the student evaluations that influence promotions and pay, from journal publications to book awards. These political and sociological problems deserve study as much as any of the other issues the academy investigates. Here’s the second piece in our two-week series on the gender gap in political science — and what we can do about it. — Kim Yi Dionne
How can colleges and universities — and, for that matter, the working world at large — overcome implicit bias against women, biases that aren’t as obvious as someone arguing openly that women can’t think as well as men?
One of the elements that holds women back in political science are the ways professors get evaluated — which influences decisions about pay, promotion, assignments, tenure and so on. A substantial body of research shows that students systematically give lower teaching evaluations to women and people of color than to white men, even when there are no differences in the teaching’s quality and effectiveness. Nevertheless, most universities and colleges continue to use student evaluations to inform decisions about who gets promoted.
Unintentional and automatic biases are hard to overcome. Students may have no idea they are evaluating faculty differently based on the professor’s gender, race or other characteristics. But previous research suggests that acknowledging and recognizing biases can help undo its effects, at least somewhat. We decided to measure whether telling students about such biases would change their evaluations.
How we did our research
In our experiment, we divided 52 classes into two groups. One group received an email informing them course evaluations were about to begin. The other group received the same email — but with an added statement that encouraged them to keep the research about both explicit and implicit biases in mind. We specifically mentioned that such biases can often result in professors being evaluated according to gender and racial stereotypes, and linked to a repository of studies about student evaluation bias, so that students who wanted to could learn more. Students in that second group also found a similar statement at the top of their evaluation forms. The evaluations were completed by 383 students who received the statement and 316 who did not.
We conducted this experiment in fall 2018 and again in spring 2019. Of the 52 classes in our study, female faculty taught 31 and male faculty taught 21. (All faculty were white, so we could not test whether our “bias statement” mitigated racial bias.) The evaluations asked students to rate faculty on a 1 to 5 scale on 10 different aspects of teaching, and included space for open-ended responses about faculty strengths and ways to improve teaching.
Informing students about bias changed the quality of their comments on female faculty
Building on the work of political scientists Kristina Mitchell and Jonathan Martin, we looked at how, in their comments section, the students referred to faculty. Did they call the faculty member a “professor” — or a “teacher” or “instructor?” Such a reference can be quite telling. In a male-dominated profession like academia, women are often assumed to be less qualified or of lower academic rank than men. When that happens, women are called “teachers” while equivalent men are called “professors.”
Among the students who received the statement about implicit bias, the teaching evaluations had a 25 percent increase in references to female faculty as professors — and a 40 percent reduction in referring to female faculty as an instructor or teacher.
Another telling point is whether students comment on the faculty member’s teaching and expertise, or on such personal qualities as physical appearance or fashion choices. Among the students who’d received the bias statement, comments on female faculty were substantially more likely to be about the teaching.
But being told about bias didn’t consistently change students’ numerical ratings
However, the bias statement didn’t have the same clear results when students ranked faculty on the 1 to 5 scale. Political scientist David Peterson and his colleagues found that a bias statement did raise students’ scores for women faculty. Our efforts in fall 2018 found something similar: The bias statement did slightly increase students’ evaluation scores for women faculty, at least on some questions.
But oddly, in the spring semester, the bias statement didn’t affect students’ evaluations of female faculty.
We also see the statement increasing men’s evaluations on some questions one semester and lowering them the next. What might that mean?
Students generally evaluate male faculty more highly than they do women, research shows. For instance, researcher Lillian MacNell and her colleagues examined online classes to see whether students evaluate a course instructor differently if they think they are being taught by a man or a woman. In identical online courses, an instructor who used a man’s name received higher evaluations than when the same professor taught the course using a woman’s name.
If alerting students to implicit bias in evaluations cannot raise women’s ratings, perhaps it can instead counter the male advantage. We want to be cautious in drawing conclusions here, however, since our results differed across semesters.
What does this mean for universities’ student evaluations going forward?
University administrators sometimes use student evaluations’ numerical ratings in decisions about faculty hiring, pay and promotion. So while including the bias statement helped reduce bias in the students’ comments, the ratings are what matter. Apparently, informing students about biases can sometimes help reduce bias in their faculty ratings — but sometimes may not be enough.
Since student evaluations have so many problems, why are they still so widely used? Probably for a few simple reasons. First, senior faculty, who are typically white men, are familiar with student evaluations — and generally benefited from them. Second, such evaluations take little class or faculty time. Third, student ratings have an air of objectivity, simply because they’re numerical. And so universities are in no rush to abandon student evaluations of teaching.
Informing students about racial and gender biases may protect against comments devaluing women. But an anti-bias statement isn’t enough. Students’ ratings won’t change unless they are motivated to and have some guidance about overcoming those biases.
Don’t miss anything! Sign up to get TMC’s smart analysis in your inbox, three days a week.
Ellen M. Key (@ellenmkey) is associate professor of political science at Appalachian State University.
Phillip J. Ardoin (@ardoinpj) is professor and chair of political science at Appalachian State University.