The Hidden Bias Behind Ratings feat. Professor Sora Jun

[00:00]Brian Jackson: Welcome to Owl Have You Know, a podcast from Rice Business. This episode is part of our Up Next series, where faculty, researchers and alumni weigh in on the trends currently shaping the business.

Today, we're joined by Dr. Sora Jun, assistant professor of management and organizational behavior at Rice Business. Sora teaches across the MBA, undergraduate, and Ph.D. programs, and her work has been published in leading journals. Dr. Jun studies the hidden forces that shape how we see and treat one another — from the ways people navigate power and privilege across social hierarchies to how subtle biases show up in something as simple as a five-star rating. Her work uncovers how our minds process inequality and how even small design choices, like switching from stars to thumbs-up icons, can make systems fairer.

Dr. Jun, welcome to Owl Have You Know.

[00:55]Sora Jun: Thank you so much for having me.

[00:58]Brian Jackson: Well, you've done so much. And really, what I want to know is your journey, starting from studying finance in Toronto to becoming a researcher in organizational behavior.

[01:09]Sora Jun: Yeah. So, I decided to enroll in a business program — a commerce program is what the Canadian schools call it — because I was really interested in understanding, like, how the world works. And I think, back then, I thought the biggest mover of people's behavior was money. And so, that's why I enrolled in a business program. But I actually found that a lot of the classes I was taking were not really answering the questions that I was hoping to get answers for.

So, I ended up doing a minor in sociology, which I think was a much better fit for my interests back then. I then learned about what research is through my organizational behavior professor, who basically, kind of, plucked me out of the class and suggested that I work for her as a research assistant. And I still work with her. Her name is Dr. Katy Decelles.

And so, that's how I got into the world of research, was really just taking classes that I found interesting and then just digging deeper into that without really thinking about, like, jobs, which I probably should have thought about. But thankfully, I found my way into what I think fits my kind of interests much better. So, that's how I got into research.

[02:20]Brian Jackson: So, you ended up in Toronto, but before, you were raised moving from different places, right?

[02:25]Sora Jun: Yeah. My earliest memories are my family in Buffalo, New York. So, my family was there until I was seven, even though I was born in Korea, like I moved as soon as I was born. And then my family moved back to Korea, so I did elementary school and middle school in Korea. And then my sister had been born in the U.S. And so, she moved back to Buffalo by herself to study for high school, because public schooling was free. And my parents thought that I might want to go through a similar path, but they knew they wouldn't be able to, like, afford for me to study in school in the U.S. without a citizenship. So, they applied for Canadian immigration where my aunt lived. So, I ended up living with my aunt's family and did high school. And then I went to college in Canada. I know that's very complicated, but yeah, circumstances led me to move in different places.

[03:21]Brian Jackson: Hey, it sounds complicated, but then I think of, like, my friend group that's everyone's stories are like that. So, your background moving across, has it led to influence the questions that you're asking or the kinds of research that you're drawn to?

[03:35]Sora Jun: Yeah, I think so. So, I think, at the core of all of my research lies, like, this idea of social identity. And I think that was a big part of my upbringing. So, for example, when my family moved back to Korea when I was, like, seven, eight-ish, I looked Korean, but I didn't speak Korean. But I understood most of the Korean that was being spoken. I think that was really weird for everybody around me because they would just assume that I was like them, but I wasn't really.

And then when I moved back to Canada to study, I didn't have as much of a Korean accent in my English, but my vocabulary was stuck at, like, a seven-year-old's vocabulary. And so, that was, I think, also really weird for people to, kind of, understand what was going on with me.

And so, I really experienced that daily and got used to the different environments. But I think that just, kind of, gave me, like, a different way of seeing, like, how groups form and what that identity means. Like, what my Korean identity means for me versus other people or my Asian identity and so on. So, I think it did have an impact, for sure.

[04:41]Brian Jackson: So, you're a student. You are working with research. When did it click, that, “Okay, now I want to teach?”

[04:48]Sora Jun: So, I didn't even have any teaching experience until my first job at UT Dallas. I think I had always had this idea that I am responsible for, like, mentoring younger Ph.D. students. So, there was some training and mentoring there, but I don't think I really seriously thought about the implications of teaching undergrads or MBAs until I first started teaching. And then I found a lot of joy and fulfillment in doing that, but I just fell into it in a lucky way — especially what I teach at Rice, is really close to what I do in my research. And so, I think it's very difficult to often reach the public audience with academic research. But I feel like I'm able to do that, often in a deep way with students, curating what I think is important research, like cutting-edge research in the areas that I study, to different levels of students. And I feel like I'm making an impact in that way, which I really, really enjoy.

And I think, also, a lot of the students who end up taking my class, kind of, find comfort in understanding and making more sense of their lived experiences in a more, you know, like, research-based way. 

[05:58]Brian Jackson: And that in itself is a full-time job, on top of you're trying to do research, write papers, which I think is a segue into some of the research that you've done on the subtle systemic forms of bias. What drew you specifically into studying these small but consequential ways inequality shows up?

[06:17]Sora Jun: I think a lot of it is just, like, by my training. So, for example, like, the very first research project that I worked on, I think, was much more specific to how you manage your own social identity. So, I had a paper that ended up being published, but it started out as, like, an undergrad thesis, on how racial minorities basically erase cues that signal their race when they're applying for jobs to have a better chance at, you know, not being discriminated against and getting those interviews and jobs.

And so, I don't know if I was specifically initially focused on the, sort of, discrimination part, but I think it became a, sort of, automatic implication of, like, thinking about how your own social identity impacts how people react to you and respond to you. And a big part of that is discrimination, not just in the, sort of, negative way, but also in the positive way.

So, like, some people, just based on their identities, might be given the benefit of the doubt or, like, be presumed to be competent and all these things. So, yeah, that's probably how I got into it. Yeah.

[07:24]Brian Jackson: So, resumes. Like, what were the items that gave folks away? Was it affinity groups or geographic location, basic things like that?

[07:34]Sora Jun: So, it would often be in the names, like their first names or last names. Even if… so, let's say, like, my name is Sora Jun, like, some people might want to erase a name like that in the, kind of, Asian-ness by, you know, saying, like, SJ Jun, you know, like, even if the Jun, kind of, stays, like, pretty Asian.

But also, we looked at affiliation to ethnic groups. So, in an experiment we did, we, kind of, manipulated the degree of, like, what we call whitening, erasing racial cues in resumes and names and affiliation with student groups.

And so, the, kind of, takeaway from that paper was that, first, ethnic minorities, they are sensitive to discrimination, so they do tend to erase racial cues in their job packet when they apply for jobs to, kind of, avoid discrimination. But we also ran a field experiment where we varied the degree of whitening of resumes and applied for real jobs to look at the actual callback rates for those resumes. And we found that the more the resumes were whitened, meaning, like, they had fewer and fewer racial cues, the more likely the companies were to call the applicant back for an interview.

And we also actually had varied what kind of organizations these fake but real-looking resumes were sent to. And so, half of the organizations had the language in their, kind of, job postings that they value diversity and they welcome diversity. They appreciate it. And in the other half of the organizations that we sent our resumes to, those organizations didn't have any of that language. And we find that it doesn't matter what the organization says. The more a resume is whitened and has fewer racial cues, the better applicant's chances are to getting an interview, is what we found in that paper.

[09:19]Brian Jackson: How interesting! Even with how most companies have, what, an AI system that does a cursory run through that was happening?

[09:29]Sora Jun: So, this was, like, way before the age of AI, because it was a project that I had started when I was an undergrad student. But, yeah, I'm not sure about the implications of AI with that. Like, I think what you might be referring to is, like, the, kind of, blinding of identities, is that what you're talking about?

[09:48]Brian Jackson: Mm-hmm. That would just be interesting to know. I was curious. Well, I guess the timing here is everything, so this is way before that.

[09:54]Sora Jun: Yeah. But it is an interesting question. I do get that question a lot in my classes as well, like, can we just take away all the identifiers? Well, the research, at least the most recent research, like, in the past couple of years, not by myself, but other scholars, shows that it's not always beneficial to race cues either. It's, like, a much more sensitive and, kind of, delicate process, where it might be helpful if an organization has maybe had a lot of problems with bias and discrimination before. But if an organization already values diversity, truly not in a performative sense, then making these identifiers blind is not actually going to help.

[10:34]Brian Jackson: So, you've done a study on the gig worker ratings — which for those who don't know what gig worker ratings are, you might have to define — and the racial bias you uncovered in five-star systems. If you could just walk me through what that study looked like.

[10:47]Sora Jun: Yeah. So, this study did look at gig workers, but I don't think it's really limited to just gig workers. So, the starting point of this project was that a lot of online platforms that do hire gig workers, like Uber or Taskrabbit, use customer ratings to decide which workers will get more work and how much they'll get paid through their algorithms. But a lot of research before us has shown that there is a racial bias or racial discrimination in how customers rate these workers.

And so, generally, the pattern is that white workers tend to get better ratings and more pay than non-white workers. And so, we were trying to see if we could reduce that bias and discrimination by making a structural change specifically to how the workers are rated. Because a lot of these, sort of, platforms use a five-star rating scale, we wanted to look at whether switching to dichotomous, like thumbs-up or thumbs-down scale, might help reduce that gap.

[11:48]Brian Jackson: And so, did it ultimately reduce the gap? I’m curious.

[11:51]Sora Jun: Yes, it did. Yeah. So we were able to get data from, like, a home services online platform. And that company had made a pretty abrupt switch from a five-star rating scale to a thumbs-up/-down scale. So, none of the workers or the customers really knew that that change was coming. And what we're finding from their data is that, before the change, so, when the company was using a five-star rating scale, there is a racial gap in ratings, which also leads to a racial gap in the workers’ earning. So, for example, non-white workers were making, like, 91 cents to a dollar that the white workers were making. And after they made that switch to thumbs-up/-down scale, that gap is eliminated. So, there are no differences in the mean values of ratings for non-white versus white workers. And also, the income gap also was eliminated.

[12:46]Brian Jackson: And then, ultimately, the customer experience — unchanged, right? Did they show anything ultimately with the product?

[12:53]Sora Jun: No. So, we didn't find any evidence of that. But that was one of the concerns with the study design from the real data from the company. So, like, one thing that gave us comfort or, I guess, confidence in our results was that all of these gig workers were already vetted for, before they could be on the platform. And so, there was already, like, a test of quality beforehand. But we also ran a follow-up experiment to hold that quality constant. So, even… and we found again, like, support for predictions and replicating what the field data showed, with an experiment that held the quality of the workers constant.

[13:30]Brian Jackson: So, how do you balance the fairness of the gains of a binary system with the loss of nuance?

[13:36]Sora Jun: Yeah, I think that's a really tricky part. I do think it is a challenge. If we were to, like, imagine changing all these numerical rating scales to dichotomy scales, like, we would lose a lot of the fine-grained information. So, it probably depends a lot on the context. Like, I think, from our study, what we were finding was that the ratings using a five-point scale were already quite inflated, so there wasn't actually a ton of fine-grained information to be had from even the five-point scale information. So, like, everybody's ratings were between, like, a 4.5, for example, to a 5. This is just an example.

So, that's, kind of, one thing to keep in mind, is maybe assessing whether the fine-grained information is actually being produced from what you think is a good rating scale you have. Other things we've talked about as an authorship team were, like, it might be possible to use dichotomous skills when it's appropriate and ask for different dimensions of quality rather than just, like, lumping it all into, kind of, one question. Even though the cost of that would be, like, the strain on the customers, but that could be one way to at least increase more nuance about the quality of the work and workers.

[14:48]Brian Jackson: In talking about advantaged versus disadvantaged, from your side, why does framing matter so much when talking about pay gaps or wealth disparities?

[14:58]Sora Jun: So, we know from prior research that framing matters. Like, past research has found that any sort of gap or inequality could be described in at least one of two ways. So, it could be described as the powerful group having more power or the less powerful group having less power.

And a number of researchers have found that the way we frame the same inequality matters in how people understand what the causes of inequality are, who they really focus on when they're thinking about the inequality, and what kind of policies they support when trying to close those gaps. So, for instance, when you describe the gender pay gap as women earning less than men, compared to what is the logically equivalent framing of men earn more than women, people are more likely to focus on women as the cause of that pay gap. So, they're thinking, like, what are women doing to earn less than men? When, really, in a specific context, the issue could be more about the structure or the system, or it could be, like, men are doing something else or men are being treated in a different way to create that gap.

And so, framing of inequality matters, because even though what is being talked about is logically equivalent for an advantage or disadvantage frame, people understand it to be very different. And then they focus on different, I guess, objects.

[16:23]Brian Jackson: Yeah, it's so interesting. So, why do you think wealth inequality is often described in neutral terms while gender and racial gaps are framed as disadvantages?

[16:34]Sora Jun: Yeah. So, this is a question that my collaborators and I looked at in this paper called “Chronic Frames of Inequality.” And what we hypothesized about the differences between the different domains of inequality was that people have different conceptions of how fair or legitimate these inequalities are. And that perception of fairness also has implications for whether an advantage frame is more comfortable for people to use versus the disadvantage frame. And it gets quite technical after this.

[17:04]Brian Jackson: Yeah, I know.

[17:05]Sora Jun: What we find is that people generally think that race and gender inequalities are unfair. They often think that these identities are essential to people or more essential than, like, how much wealth or education you have.

So, that's the first part of it. And then we had all these predictions about how the perceptions of the legitimacy of the inequality might impact the framing. And so, when an inequality is considered unfair or illegitimate, it's in the interest of people on both sides of that inequality to frame it as a disadvantage, right? So, if you are, kind of, benefiting from these illegitimate or unfair hierarchies that you, kind of, have an understanding that it's unfair, it's much better not to draw attention to what you're getting more of, right? It doesn't feel great.

And this is, I guess, again, especially a case if you think that is coming out of some sort of unfair system, right? Also, on the other side of this is, if you are disadvantaged from this system of what you see as an unfair system of inequality, it's also, sort of, in your best interest to frame it as a disadvantage because any sort of remedies to fix that inequality will be more in the form of giving you or your group more things to correct for it than to bring the top down, which you might care about, but not as much as, like, your group being pulled up, right? That's, like, one side of it is that… 

And the opposite is true when you generally think that the inequality is fair. So, if you think that you, like, won all these awards for really legitimate reasons, you know, like, you did well, like, you had a lot of effort and talent involved, it's actually pretty good to highlight that part of it, rather than, like, other people not doing as well because they might not have had those things, right?

Whereas, if you think that you are, kind of, losing out because of the fair reason, you might not want to call it out as much, right? Like, you don't really want to highlight your…

[19:06]Brian Jackson: [inaudible 19:05].

[19:07]Sora Jun: Exactly. And so, I know I talked about more of an individual-level example there, but connecting it back again to this idea of, like, what people consider a fair or legitimate system of hierarchy, people generally think that class and equality is your wealth, your income is a fair system, even though, again, there's also a lot of evidence that it’s not actually fair. But people think it's more fair than at least race and gender inequalities. We find evidence that that idea of perceived fairness is what drives people to either not prefer to use any type of frame or if they are preferring a frame, it's the advantage frame.

[19:42]Brian Jackson: So, as a leader, or let's say, like, as an educator, what do we do? How do we adjust our language, our vernacular, when we address and approach inequality in the workplace?

[19:53]Sora Jun: Right. So, this is something that my other collaborator, Taylor Phillips, and I had thought about a lot when we were looking at academic journal articles doing the exact same chronic framing as, like, newspaper articles or lay people. And I find it very difficult to give definitive prescriptive advice.

So, I think my advice would be to do a self-check. Like, do I have this tendency to chronically frame in one way rather than the other? And then just try to frame it the other way. See if it's even possible to do that. And then, kind of, reflect on what the implications of that other frame might be.

[20:31]Brian Jackson: Going to shift a bit to anti-Asian discrimination. You know, you're working on a paper currently. I think it's going to be potentially out when this episode is released.

[20:41]Sora Jun: Hopefully,

[20:42]Brian Jackson: What inspired you to dig into this?

[20:44]Sora Jun: Yeah. So, one of my biggest interests is really understanding complex hierarchical systems, how our conceptions of intergroup dynamics or relationships between different groups can really change once we stop dichotomizing our more complex social hierarchies. So, one, kind of, obvious example of this is the racial hierarchy in the U.S. So, a lot of research on race in the U.S., specifically, has really focused on comparing the experiences of white people versus, usually, it's Black people, but often, like, racial minorities as a, sort of, monolith. There's more and more research coming out that separates the different racial minority groups. But I had always been interested in understanding, like, where Asians, sort of, fit into this dynamic of race in the U.S.

Obviously, because of my own Asian identity, I think this is one of the first questions I had as a grad student when I started to read more about race and, like, not really seeing that information about, like, what is happening to race. Like, how do these theories apply to Asian people and so on?

And so, that's, kind of, how it got really started. And then I really started to think about the implications of Asian Americans being in a, sort of, intermediate status group, having fewer privileges and advantages than white Americans, but more so than Black and Hispanic individuals.

And so, my collaborators and I, we had theorized that one, sort of, implication of that might be that Asians simultaneously doing well and having these advantages might obscure the disadvantages and discrimination that they experience, which led to that project.

What we were theorizing about was this idea that, when people try to recognize some sort of event as discriminatory, there's some research showing that people, kind of, have a schema or an idea of what discrimination looks like. They have a prototype of, like, “This situation fits into my understanding of discrimination. This is happening.”

We were theorizing that, because Asian Americans also have these simultaneous racial advantages, along with their disadvantages that people wouldn't really think about Asian individuals as, kind of, prototypical victims of discrimination. 

[23:03]Brian Jackson: You know, working through this paper, talking through the research, having other authors that you're working with, has this made you reflect on your own experience and even the experience of those close to you?

[23:16]Sora Jun: I think so. When you ask that question, I think the first thing that comes to mind is my own relationship with my Asian identity, which was, kind of, complicated, just because I did move around a lot. And I think it actually took me a while to really feel like I had enough legitimacy in standing and studying this in some ways because I felt like I didn't really the real, like, Asian American experience, given that I have been in so many different circles and I'm hearing so many different stories about what it means to be Asian for different people.

But I think really embracing that insecurity, almost, was useful because I think it just made me dig in deeper and realize, like, that might be part of that Asian experience is, like, feeling like there's so many different kinds of Asian experiences. I'm sure this is similar for other groups as well, but I think I've just become more appreciative of just asking people, like, “What's this been like for you?”

[24:15]Brian Jackson: Yeah. And you're pulling qualitative data. You're asking people their experiences. You're building that identity and piecing it together, right?

[24:24]Sora Jun: Yeah.

[24:24]Brian Jackson: Any big questions you want to explore, looking ahead five to 10 years?

[24:29]Sora Jun: I think I'm really interested in shedding more light on the advantage side of Asian Americans. It's, kind of, interesting.

I have a sense that it's almost easier to publish research that showcases the disadvantages of Asians. And I think people… the audience finds that a little more comfortable to take in. But my idea of, and what the research has said about Asian Americans’ position in the racial hierarchy in the U.S., is that they're in this in-between space. And so, there's not a lot of research on how Asians experience their advantages, which I think has a lot of other implications as well. Like, I don't think people think of Asians as, like, prototypical beneficiaries of the systems of racism. And they might not be the most benefiting group, but they still benefit more than some other groups, right?

So, like, it's that kind of nuanced “advantaging” side that I am more interested in and think it's important to explore further. Another question that I always have is, like, thinking beyond the context of race, like, what other hierarchies have this more fine grain, like, more than two-group situation? 

[25:40]Brian Jackson: Interesting. Okay, so you're going to be busy the next five to 10 years.

[25:45]Sora Jun: Yeah, but it's fun work.

[25:50]Brian Jackson: So, you teach across several programs here at Rice — the MBA, Ph.D., and now the new Virani Undergraduate business program. What has that experience been like for you? And how does teaching undergraduates compare to working with MBA students or even doctoral candidates?

[26:07]Sora Jun: Yeah, I think it's really exciting. I just like the fact that I could teach all these different levels. And I find it challenging, a lot of times, to talk with undergrads who might have less experience and they might be more, like, interested in the theory part of things. And then on the other hand, like, talking to Executive MBAs who have, like, so much experience, it's just a very different group of people that I end up having access to. Like, I often see my role as a teacher more as a facilitator than, like, a teacher, like, sharing wisdom. I'm like, I'm connecting all the science about these topics to you all who have these, like, life experiences.

And so, just the fact that I get to teach this, like, wide range of students and, kind of, also hear about their perspectives, I find that really, really exciting.

[26:56]Brian Jackson: When your students leave your classes, be it undergrad or EMBA, what's the one thing you hope they carry with them into their careers?

[27:04]Sora Jun: I do think, like, one thing that's just important in general as, like, a functioning human being. But I also find, you know, like, it's hard to practice this all the time. It's just recognizing that your reality is not going to be the same as the other person's reality, even in the same situation.

And so, just giving yourself that room and, kind of, grace to be compassionate about yourself, but also for another person's understanding of what's happening. I think that's really important. And I have found that, a lot of our students, I hear their experiences and how they've tackled different situations and I'm like, “You sound like a great manager. Like, I would love to work with you, you know, if you ever become a professor.”

And I think that's, kind of, what I'm noticing, is that what I find the commonality of those, kind of, students who I like to think really, really shine and have, like, already demonstrated in practice. Like, I think the connecting thread is that they're really, really good at perspective taking and just, like, not assuming that other people are experiencing the world in the same way that they are.

[28:06]Brian Jackson: Yeah, it's leading with empathy.

[28:09]Sora Jun: Yeah. That's a great word. I could have just said that.

[28:12]Brian Jackson: No, no, I'm a good summarizer. That's what I'm here for. Thank you, Sora, for joining us today on Owl Have You Know.

[28:22]Sora Jun: Thank you so much for hearing me out.

[28:27]Brian Jackson: Thanks for listening. This has been Owl Have You Know, a production of Rice Business. You can find more information about our guests, hosts, and announcements on our website, business.rice.edu.

Please subscribe and leave a rating wherever you find your favorite podcasts. We'd love to hear what you think.

The hosts of Owl Have You Know are myself, Brian Jackson, and Maya Pomroy.