Gig Economy | Peer-Reviewed Research

Thumbs Up: Fixing the Bias in Gig-Worker Ratings

When a gig-work platform switched from a five-star rating system to a simple thumbs-up/down, it eliminated a 9% racial earnings gap for their workers.

Based on research by Sora Jun, Tristan L. Botelho (Yale), Demetrius Humes (Toronto) and Katherine A. DeCelles (Toronto)

“We saw that subtle racial bias was creeping into five-star ratings, quietly chipping away at non-white workers’ earnings. Then the rating system was changed,” Jun says, “and just like that, the earning gap was gone.”

Key findings:

Data from a home services platform revealed a 9% pay gap between white and non-white workers.
White workers were 3.5 percentage points more likely to receive a perfect rating under the five-star system.
Switching from a five-star rating system to a thumbs-up/thumbs-down scale eliminated this racial disparity entirely for new workers.

For millions of gig workers, a customer rating isn’t just feedback — it can determine access to more work, better pay and financial stability.

A new study published in Nature uncovers a hidden problem — racial bias in customer evaluations — which led to a 9% income gap between white and non-white gig workers in a home services platform. The study also proposes an unexpected but simple solution: changing how customers rate workers from a five-star system to a thumbs-up/thumbs-down.

Analyzing 100,759 completed jobs and 69,971 customer ratings from an online home services platform in the United States and Canada, the researchers found that racial minorities received lower ratings than white workers. (The study does not disclose the real name of the platform for confidentiality reasons.)

This small but consistent bias had big financial consequences: minority workers earned just 91 cents for every dollar their white counterparts made.

But when the platform made one simple change — switching from a traditional five-star rating system to a binary thumbs-up/thumbs-down scale — the racial gap disappeared.

“The numbers don’t lie,” says Sora Jun, a co-author and assistant professor of organizational behavior at Rice Business. “We saw evidence that subtle racial bias was creeping into five-star ratings, quietly chipping away at non-white workers’ earnings. When the rating system was changed, the earning gap was gone.”

The Power of “Dichotomization”

Customer ratings influence earnings on many gig platforms. A single decimal point can push workers into lower income brackets, reducing their pay for identical work.

This study reveals that five-star rating systems create space for subtle discrimination, where customers score minority workers slightly lower, with a 0.07-star difference (4.72 vs. 4.79).

While this gap may seem small, it had significant financial consequences. For example, if a gig worker earns $40,000 a year, a 9% pay gap means they are taking home $3,600 less annually — purely due to biased ratings.

Multiply this across an entire workforce, and the financial impact is staggering. According to the World Bank, an estimated 154 million to 435 million people worldwide are working gig jobs online, up to 12% of the global workforce.

The research team tested their hypothesis in multiple ways. First, they analyzed real-world data from a platform that changed its rating system abruptly, without informing users of the switch.

Before the change, white workers were 3.9 percentage points more likely to receive a perfect five-star rating compared to non-white workers. But after the introduction of the thumbs-up/thumbs-down system, this gap vanished.

The researchers also ran controlled experiments, where participants were asked to rate identical work by either white or non-white workers using different rating scales.

The results were clear: racial bias was more pronounced in the five-star system and significantly reduced in the binary system. White and non-White workers received positive ratings at nearly identical rates (95.9% vs 95.4%).

“Five-star ratings can feel objective, but they’re anything but,” Jun explains. “They may give customers too much room to let their biases slip in. When we force people to make a yes-or-no decision, they focus on performance, not personal perceptions.”

The Bigger Picture

Bias in ratings is not just a problem for one company — ratings are used across the gig economy, which spans transportation (Uber, Lyft), delivery services (DoorDash, Instacart) and freelance work (Upwork, Fiverr).

Many platforms rely on customer ratings to allocate opportunities, promotions and even account suspensions. If ratings are biased, so are the economic outcomes.

The findings suggest that platforms can meaningfully reduce racial inequities with a relatively simple intervention. While diversity training and anti-bias policies are common in corporate hiring, they’re impractical for gig economy customers, who aren’t employees and can’t be required to follow training or bias guidelines.

Changing the way ratings are structured, however, is a simple and scalable fix.

One potential concern is whether switching to a thumbs-up/thumbs-down system removes valuable detail from customer ratings. The researchers acknowledge that binary ratings provide less nuance than a five-star system, but they argue this loss of detail is largely irrelevant.

In practice, most customers inflate ratings, with 85% already giving five stars, meaning the extra precision of a five-star scale is more illusion than reality. Platforms should ask themselves: do we care more about fairness or fake precision? Because right now, the status quo is costing real people real money.

“If we’re serious about tackling bias, we can’t just rely on people to be more self-aware,” Jun says. “We need structural fixes. And sometimes, the smallest tweaks — like changing how we rate workers — can make a big impact.”

See Botelho, Jun, Humes, and DeCelles, “Scale Dichotomization Reduces Customer Racial Discrimination and Income Inequality.” Nature (2025). https://doi.org/10.1038/s41586-025-08599-7