The study is described in a 2024 draft paper posted on the website of SSRN, formerly known as the Social Science Research Network. It is currently undergoing revisions with the academic journal Management Science.
The researchers detected grading bias against the end of the alphabet in a wide range of subjects. However, the grading penalty was more pronounced in the social sciences and the humanities compared to engineering, science and medicine.
In addition to lower grades, the researchers also found that students at the bottom of the alphabet received more negative and impolite comments. For example, “why no answers to Q 2 and 3? You are setting yourself up for a failing grade,” and “NEVER DO THAT AGAIN.” Top-of-the-alphabet students were more likely to receive, “Much better work on this draft, [Student First Name]! Thank you!”
The researchers cannot prove precisely why extra points are deducted for the Wilsons of the world, but they suspect it’s because instructors – mostly graduate students at the unnamed university in this study – have heavy grading loads and they get tired and cranky, especially after grading the 50th student in a row. Even before the era of electronic grading, it’s quite likely the instructors were not as fair to students at the bottom of the paper pile. But in the paper world, a student’s position in the stack was always changing, depending on when the papers were turned in and how the instructors picked them up. No student was likely to be in the bottom of the pile every time. In the LMS world, the U’s, V’s, W’s, X’s, Y’s and Z’s almost always are.
Another theory mentioned by the authors in the paper is that instructors may feel the need to be stricter if they’ve already given out a string of A’s, so as not to be too generous with high marks. Students at the bottom of the alphabet may be the victims of a well-intentioned effort to restrain grade inflation. It’s also possible that instructors are too generous with students at the top of the alphabet, but grade more accurately as they proceed. Either way, students at the bottom are being graded differently.
Some college instructors seem to be aware of their human frailty. In 2018, one posted on a message board at Canvas, asking the company to randomize the grade book. “For me, bias starts to creep in with fatigue,” the instructor wrote. “I grade a few, go away from it, grade a few more, take a break. Or that’s the goal when I’m not up against a deadline.”
If you’ve read this far, perhaps you are wondering how the researchers know that the grades for the U-to-Z students were unfair. Maybe they’re comparatively worse students? But the researchers matched the grades in Canvas with the student records in the registrar’s office and they were able to control for a host of student characteristics, from high school grades and college GPA to race, ethnicity, gender, family background and income. End-of-the alphabet surnames consistently received lower marks even among similar students who were graded by the same instructor.
The researchers also found that a tiny fraction of instructors tinkered with the default settings and graded in reverse alphabetical order, from Z to A. That led to the exact opposite results; students with end-of-the alphabet names earned higher grades, while the grades for A, B and C surnames were lower.
The bias against end-of-alphabet surnames is probably not unique to students who use the Canvas LMS. All four major LMS companies, which collectively control 90 percent of the U.S. and Canadian market with more than 48 million students, order submissions alphabetically for grading, according to the researchers. Even Coursera, a separate online learning platform, does it this way.
Wang’s solution is to shake things up and have the LMS present student work for grading in random order. Indeed, Canvas added a randomize option for instructors in May 2024, after the company saw a draft of this University of Michigan study. “It was something that we had on our radar and that we’d heard from some users, but had not completed it yet,” a company spokesman said. “The report from the University of Michigan definitely pushed that work to top priority.”
However, the default remains alphabetical order and instructors need to navigate to the settings to change it. (Changing this default, according to the study authors, has “low visibility” within system settings on the site.) I hope this story helps to get the word out.
This story about learning management systems was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters.