AI Bias Detection in Image Generators

Knowledge at Wharton

Ask about Knowledge at Wharton

Episode Description

Wharton marketing professor Kartik Hosanagar and doctoral candidate Pushkar Shukla talk about software they developed with other experts to identify and correct biases in AI text-to-image generators.

The Knowledge at Wharton podcast features interviews with Wharton faculty and industry leaders about the business landscape and the impact of their work. Conversations range from the latest research and books by faculty authors, to Leading Diversity at Work with professor Stephanie Creary.

#ArtificialIntelligence #AIethics #GenerativeAI #ResponsibleAI ----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.

Launched in 1999, Knowledge at Wharton offers free access to:

- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox

Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU

Brief Summary

Kartik Hosanagar and Pushkar Shukla discuss their TIBET tool for detecting biases in AI-generated images and its societal implications.

Key points

  • Introducing TIBET: A New Tool. The TIBET tool aims to automatically detect biases in AI-generated images.
  • The Importance of Bias Detection. Biases in AI can perpetuate stereotypes at an unprecedented scale, affecting societal views.
  • Future Applications of TIBET. The TIBET tool's approach could be extended to other AI models, including language generation.

Transcript

This podcast is brought to you by Knowledge at Wharton. Angie Basiouny: Welcome to Knowledge at Wharton. I'm Angie Basiouny. If I ask you If I ask you to think of a doctor, your brain might conjure up the image of an older white male. If I ask you to think of a fitness enthusiast, it might be a younger, slender person. These are biases that are baked into our brains, and unfortunately, they also get baked into AI. I'm here today with Wharton Professor Kartik Hosanagar and his colleague, Pushkar Shukla, who's a PhD candidate at the Toyota Technological Institute at Chicago. They're part of a team that has created this new tool that can correct, detect, biases in text-to-image generators.

Those are the machine learning models that can produce a visual image for you with just a few prompts. Gentlemen, I'm so glad to have you here today. Kartik, it's our first time talking on this podcast. Welcome. Thanks for having me, Angie. A great pleasure to meet you in person. We've interacted many times before, but lovely to see you. Same here. And Pushkar, welcome to the Wharton family. Thank you for making some time and joining us today. Thank you so much, Angie. It's a pleasure meeting you and interviewing for this podcast. - Absolutely. So when I first read this paper, I just thought, what a difficult problem to solve for in real life. I can't even imagine how

difficult it is to solve for it on a technology platform. And that's why I'm really excited to share these details about this tool today. Let's just start with an overview, if you will, Professor. Give us an overview of this research, what you were trying to solve for, what the problems are. Yes. Our research was motivated by the widespread use of generative AI systems today to generate all kinds of content. Certainly text through systems like ChatGPT, but also a lot of image generation through tools like Stable Diffusion, Mid- journey, Meta's AI, and even ChatGPT does image generation. And so on. And while they're being used at an unprecedented volume or scale today, these systems do have biases. It's kind of what

you were mentioning earlier. They're trained on data. Those data are generated by humans who have their own biases, and so they pick up their biases, and in many ways institutionalize those biases, and kind of do that going forward. So if you look at image generation as an example, in these systems, if you type in, say, "computer programmer", you're probably likely to see most of the images being of males. And so these systems are prone to biases. But generally the biases get detected after the fact, and usually by a human user using them and kind of saying, "Hey, I'm noticing a pattern." Or a journalist like you might notice a pattern and write about it. So we've seen, for example, Google's Gemini having bias

issues, and people wrote about it. And similar things have been written about pretty much every image generation model. So we set out to come up with a system that can automatically detect biases in these systems. And that automatic detection is extremely important because of the scale at which they're being used, but it's also a very hard problem. And so we wanted to be able to detect them, and not just detect simple or the common biases, like, say, gender and age, but all kinds of potential biases, and to be able to provide explanations to human decision makers. So that's what we set out to do and solve with this project. I think that it would be helpful if we could show some of the biased images that we're talking about, and that's where I'm

hoping, Pushkar, that you can help us with that. Share your screen and show us some of these images that you're studying in this project. The first set of images talks about a computer programmer. So if you ask an AI generative model— and in this case, we used Stable Diffusion— to generate images of a computer programmer, this is how these images look like. Similarly, in the second set of images, I ask the generative AI model to generate images of old— of an old man at a church. Now, before I go on and talk about the possible directions in which these images can be biased, I want to take a second and ask you two, what do you think are the possible directions in which these images can be biased?

Well, most definitely gender, right? For the first one, a computer programmer, the images are always going to be male. I noticed in your paper that you had a collection of images when you prompt for, for example, childcare worker, and all the images were female. Another— another stereotype. I would also say maybe the— the image about an old man in a church. The church always has a similar look. So the image of the church in this case is the same. It appears to be maybe a Catholic Church. The stained glass windows, the arched— you know, the arched windows. So again, it's sort of an image that we have for maybe even from our school days of studying the Renaissance. Yeah. And I was going to add that— well, at least the

computer programmer images, in terms of race, seems to be diverse, but in terms of the old man at the church, from what I can see, they're all white male. I can— perhaps also the computer programmer seems to be based on these four images, all young. And so there's another stereotype I'm seeing there, but that's what I'm seeing visually, at least. Cool. So I think all the axes of biases that you mentioned are correct, but there is more to it. So when we talk about— you talked about gender, ethnicity, age being important axes of biases, but let's look at a few other axes of biases. All computer programmers are lean. So there is clearly a body type issue there. And that's— similarly, the physical

appearance of old men at a church are of a certain type. They look grim, slightly depressed or sad, and also things such as, there might be ableism bias. Or in our research, we found that oftentimes old men were portrayed to have a certain kind of disability. So the point that I'm trying to make is when we think about biases in AI, we generally think about the common dimensions of biases such as gender, ethnicity, culture, maybe age. But there is more to it. There are other equally important dimensions of biases, such as socioeconomic bias, disability, body type and health bias of some sort. So— and that is something that needs to be looked upon, and that is what makes it

a tough problem to solve. I'm going to ask both of you to answer this question. Why? Why do we need to look at it? Why is it important to correct this kind of bias? Well, quite simply, these biases will continue to be there in society, but also at a scale that we have probably not seen before. If you have a human being generating images in some marketing agency, and that's going to be used in ad collateral, or visuals that we see around us, then you know that bias with an individual human being affects a few 100, few 1000 images. But if you have an AI system that's doing this at scale for millions of images— and in the end, we're probably going to have like three, four image generation AI systems that

are powering a lot of the images that get created and used by all of us. So systematically, that will mean that all of us are seeing, you know, biased images everywhere, which ultimately affects our social constructs. You know, how do we imagine a CEO? Is that always a male? How do we imagine, you know, you brought up a childcare worker. Is it always a female? And so on. And so, I think those stereotypes will then propagate in society. And the challenge over here is that we can't always rely on manual human detection of these biases, because millions of images are being generated on a daily basis, and that scale and speed, manual approaches just will not work. And if I generate two images from the system, and I

use it, and those two images happen to show two males, now that doesn't mean there is bias in there. But if tens of 1000s of people are generating images, and each of their one or two images are biased, and collectively, we're skewing how we represent teachers or how we represent computer programmers and so on. And many times there may not be a human in the loop. And so if we are automatically using AI systems to create content and use them in some downstream activity or task, then we need some automated way to correct— detect and correct those biases. Otherwise, you know, they will go undetected and be used in society at large. Yeah. So it's not— it's not the individual user. It's not, you

know, the student who's creating an image to go with his term paper. It really is— we have to think bigger. We have to think about scale and really consider that. Pushkar, I know that bias detection in computer models is something that you study specifically. Can you— from your point of view, why is this important? I think because AI models in general sort of are a reflection of the society. So— and one thing we want to ensure is— is, like, make sure that the harmful aspects that are already prevalent in the society, first, are not, like, transferred into the AI models. And more so, they are not at least exacerbated by these AI models. So often times we see that a lot of these biases in, say in certain professions, are exacerbated

or worsened by how AI models depict them. So I think that's a very important problem that needs to be solved. If we were to— like, I'm all in for developing and deploying AI models at large scales. But we should sort of understand that they should be done in a efficient manner. Yeah. Well, so this model is called TIBET, which stands for— it's an acronym that stands for Text to Image Bias Evaluation Tool. Can you just briefly explain to us, how does it work? Yeah. So generally, the philosophy behind what we're doing is that if humans cannot handle the scale and speed of image generation, and if AI's strength is scale and speed, can we leverage AI itself to detect the problems in AI models? And

if we do it carefully, then it doesn't become a loop of, AI creates models that are biased, and AI detection tools cannot detect those biases, because they have the same problem. And so you have to do this carefully. And the way we do this just intuitively at a high level— the first thing is, if we think about a term like, say, "computer programmer" or "old man at a church," the two examples Pushkar showed, we first want to think about, what are the ways in which that image can be biased? And he asked us those questions, and we did a joint brainstorming of the ways in which it could be biased. So the first step in our tool is to ask an AI system, a large language model, a question about, what are the ways in which images

that are generated in response to this prompt by a user can be biased? And it would help identify, it's certainly a gender, but it could identify some of the other biases that Pushkar raised, like body type or even the visual environment for the image and so on. And then for every potential— we call it an axis. But every potential dimension or axis along which an image can be biased, we generate what we call as counterfactual prompts. Meaning, what are the other kinds of prompts that could have been entered by the users along the dimension of bias. So if we are worried about gender, a counterfactual prompt for "computer programmer" would be "male computer programmer" and "female computer programmer", or

"transgender computer programmer", things like that. And then we generate images for these counterfactual prompts. We compare those images with the image for our main prompt, again using AI to look at the concepts that are in the images and compare the concepts. And we might find in the example that Pushkar showed that the images for computer programmer have very similar concepts as the images for male computer programmer, but the concepts are very different than the images for, say, female computer programmer. In terms of concepts, meaning the dresses people are wearing, the jewelry they have on, whether they're wearing glasses or not. Things like that. We might also find that the concepts for computer programmer look

very similar to young computer programmer, but look very different from older computer programmer. Once we start to see that, we start to see, look. The results for computer programmer are nearly the same as male computer programmer, nearly the same as young computer programmer, but very dissimilar to female computer programmer or older computer programmer. And based on that, our system is able to generate scores for how biased they are and if it starts to look like it's very similar to some counterfactual prompts, but very dissimilar to others, our system knows that there's a bias here and is able to flag that and give a high score for that bias. That's really incredible. Now, you had mentioned just briefly Google

Gemini. And we're going to talk about that for just a little bit. This was a case where Google had put out this text to image generator in the earlier part of the year, and then they sort of found themselves embroiled in the culture wars, because the generator was actually over-correcting for bias. And one of the famous examples was, if you prompt it for an image of the Founding Fathers for the United States, it would include it would depict a Black man. If you asked for soldiers who fought in the Vietnam War, it would depict an Asian woman. So this is the case where it was actually over correcting. That brings me to asking you about the limitations of your model. How do you prevent the same problem from

happening in your model? Pushkar, you should take that. - Yeah, I think that problem happens because they come up with this one-size-fits-all approach. So any prompt, or most of the prompts, you just correct for culture, as well as, say, gender, and that's about it. Whereas what our approach does is it's a very dynamic approach which is sort of dependent on what the input is. So a typical example I give is capybaras getting married in Italy, versus images of a philosopher on Mars, might have completely different sets of biases. And capybaras getting married in Italy might not have any biases related to gender, but the way Italy is portrayed might be biased in that direction. So I think the way our approach differs is

that it considers the context in which the prompt is said. And based on that context, it generates a set of biases, and then does this investigation on those sets and says that, okay, I'm biased along these directions, or not biased along these directions, but I know what directions to check my biases to start, it's not like I'm going to check my biases on age, race and gender for every every prompt or every input. Yeah. Right. Well, I find it fascinating that what you're actually doing is using human intelligence to make the artificial intelligence better, right? So I wanted to ask you, though— but we are— this tool was specifically designed for text-to-image generation. Can it be used for other things?

Do you have plans to expand its abilities? Yeah, we do. Because if you think about the underlying approach, it really is an approach that can be generalized to, for example, large language models that are generating text. Because we are thinking about first asking an AI system to reason through, what are the ways in which it can be biased? And then, given the ways in which it can be biased, constructing counterfactual prompts. Other ways in which to construct those prompts to look at, for example, what output would be generated if we worded the prompt slightly differently along gender, around age, or along body type, and so on. And then looking at the output and comparing it and saying, "Hey,

the output changed a lot when I changed the prompt in terms of gender, let's say." So that approach is easily applicable to other settings. And in fact, we are currently working on applying the same kinds of ideas in the context of the use of large language models and enterprises. And so that is ongoing. We don't have a paper yet on it, but we are certainly extending our study in that direction. And I know that we're going to get this question, so I'm going to ask you now. Is this tool, the TIBET tool, is it available? Can people find it, use it, apply it? How do they do that? Pushkar, you've been putting a lot of effort in that direction, so please. I think it's currently in progress. The paper is out. We

are sort of finishing our tool. But it's going to be soon available, probably in a month or two. Yeah. So we're just trying to wrap up the nitty gritty details of it and sort of make it a tool that people can use. Yeah. Sounds like it's going to be really important when it hits— hits the public. Gentlemen, is there anything else that we need to know about this tool, or about bias and image generation that you'd like to add before we close out? Well, to what Pushkar just said, I wanted to add that while the— the tool is not fully available for anyone to use, we do provide a lot of the code that anyone can use to also apply it in their settings. We provide a lot of the data explanations and so on.

And so we do have that on a— on a website. The URL for that is Tibet-ai.github.io. So if people go to Tibet-ai.github.io, you'll find our paper there. You'll find explanations there, but you'll also find raw code that anyone can use in in their settings. In terms of a user- friendly tool with the UI, that's in the works, and hopefully down the road. These are— these are things that we all have to start thinking about as we're speeding rapidly in this evolution of AI, that, how we fix the problems, the speed bumps along the way. So I'm glad the two of you are working on this. Thank you so much for joining me today

and for sharing some of these details. I really appreciate it. Thanks for having us. If you enjoyed this conversation, we invite you to check out all our content at knowledge.Wharton.upenn.edu. I'm Angie Basiouny. Thank you for joining us. For more insight from Knowledge at Wharton, please visit knowledge.Wharton.upenn.edu.

Tension

45/ 100Some tension

Related episodes