Everyday AI Podcast – An AI and ChatGPT Podcast - EP 377: Confronting AI Bias and AI Discrimination in the Workplace

Episode Date: October 10, 2024

Think AI is neutral? Think again. This is the workplace impact you never saw coming. What happens when the tech we rely on to be impartial actually reinforces bias? Join us for a deep dive into AI bias and discrimination with Samta Kapoor, EY’s Americas Energy AI and Responsible AI Leader.Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan and Samta questions on AIUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTopics Covered in This Episode:1. Business Leaders Confronting AI Bias and Discrimination2. AI Guardrails3. Bias and Discrimination in AI Models4. AI and the Future of Work4. Responsible AI and the FutureTimestamps:02:10 About Samta Kapoor and her role at EY05:33 AI has risks, biases; guardrails recommended.06:42 Governance ensures technology is scaled responsibly.13:33 Models reflect biases; they mirror societal discrimination.16:10 Embracing AI enhances adaptability, not job replacement.19:04 Leveraging AI for business transformation and innovation.23:05 Technology rapidly changing requires agile adaptation.25:12 Address AI bias to reduce employee anxiety.Keywords:generative AI, AI bias, AI discrimination, business leaders, model bias, model discrimination, AI models, AI guardrails, AI governance, AI policy, Ernst and Young, AI risk, AI implementation, AI investment, AI hype, AI fear, AI training, workplace AI, AI understanding, AI usage, AI responsibilities, generative AI implementation, practical AI use cases, AI audit, AI technology advancement, multimodal models, AI tech enablement, AI innovation, company AI policies, AI anxiety.Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info)

This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live and Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. There's no getting around it as powerful as generative AI is.

There's an other side to it, right? It can be ugly sometimes. There's biases and there's discrimination built into these models. So I think it's important that we have those conversations and talk about what business leaders need to know when it comes to biases. and discrimination that makes its way into these models and how we can responsibly and ethically still use them to grow our companies and to grow our careers.

All right, I am excited for today's episode where we have a leader from EY joining the show. So if you're new here, thanks for tuning in. My name is Jordan Wilson, and this is Everyday AI. Before we get our show started off, I have to get a quick shout out to our partners at Microsoft. So if you haven't heard, the Work Lab podcast from Microsoft is made for leaders who want to understand the future of work. It offers expert insights on everything from how to approach digital transformation to what it takes to thrive in the AI area.

That's W-O-R-K-L-A-B, no spaces available wherever you get your podcasts. All right. So they just dropped a new episode last week. It was fantastic. Make sure you go check it out. And while you're checking things out, please go to your everyday AI.com. So we are a daily live stream podcast and free daily newsletter. And the newsletter, sometimes I get it.

You're listening to the podcast. You're on the treadmill. You're walking the dog. And our guest today, I guarantee you, you're going to miss some of the gems that she brings. So we're going to be recapping it all in today's newsletter. So make sure you go sign up for that. And if you are looking for the news, technically pre-recorded show here. We're debuting it live.

So we'll have that as always in today's. newsletter. All right, enough chit-chat, y'all. I'm very excited for our guest for today. So please help me welcome to the show, Samta Kapoor, the Energy AI and Responsible AI Leader at EY Americas. Samtha, thank you so much for joining the Everyday AI show. Thank you, Jordan. Thanks for having me. Oh, absolutely. So could you tell us a little bit, right? Like that sounds like a huge responsibility, right, the energy, AI and responsible AI leader for EY. Can you tell us a little bit about what you do in your role at EY? Yeah, clearly other than making sure I have enough AI in my title,

there's a lot that I work on, Jordan. It's basically, like, in a nutshell, it's thinking about how to help our clients across utilities, oil and gas, and mining, to transform using data and AI responsibly. So how do we make sure that the organizations can deliver value to all of their stakeholders, which could be board members, customers, you and I, everyone, their employees, but do so with having the core responsibility in it? And I'm sure most people know Ernst & Young, but maybe for those that aren't familiar,

can you tell us a little bit about EY and the work that you guys do globally? Absolutely. Absolutely. So, Unstyn Young is an approximately 400,000 plus employees company. And what we do is tax, assurance, audit, as well as consulting. So those are sort of our main lines of business. And we do that by keeping our people at the core of it and also by making sure that we're delivering client value every day. So let's just maybe skip to the end here sometime. And let's talk about how can business leaders actually confront AI bias and AI discrimination in these models? Because, yeah, we know they, you know, all this bad information also makes its way to these models that we all use.

So what should business leaders be looking out for? And let's start talking about some of those best practices. Yeah, the good and bad about AI is that these biases can creep in any step of the way, right? Right from the beginning when you are thinking about the use case. or what is it that you're going to do? How are you going to transform your business? Believe it or not, you can actually bring in bias. So from the design to thinking about the data that you're going to feed your algorithms

to using those algorithms and making sure that you are continuously monitoring those algorithms for data and model drifts without getting too technical is key to ensuring that you are not affecting anyone negatively and or not being in the news for wrong reasons. Yeah, we've seen plenty of that, especially, I would say, early on, right, when the world was still acclimating to what the heck a large language model is. Hopefully it's a little less, you know, rampant than it once was. But, you know, maybe let's start even talking a little bit about these, you know, guardrails, right? That's something we hear all the time. And I know, you know, some of you out there that just want to use AI and go faster and break things, right?

Guard rails can be boring, but Sompta, they're extremely important. What should people know specifically as it comes to, like when it comes to bias and discrimination about AI guardrails? Yeah, and Jordan, you know the cool part about AI is that there was always bias that could come in and creep in and exist. But Gen. AI actually came in with so many more risks than we hadn't seen with traditional, classical, nat, or whatever we want to call the other AI.

And so now it's all come to where, right? There are so many different things that have come around where everyone is nervous about what it's bringing and what it's not. So in terms of guardrails, there are a few things that I strongly recommend. When you're an organization, just again, these guardrails are not to stop innovation or these guardrails are not to say that, oh my God, like, AI is this big thing that you should not think about, like let it pass, right? That's not the motive. The motive is to make sure that you have enough governance, where you're still innovating, you can still be cutting edge, you can still understand what's happening, but you have enough governance to make sure that you are staying out of the news.

The other big piece, so again, like any basic, you know, things that we discuss, it's all about the people process and tech. It's not that the tech itself is harmful. It's, you know, it's like a knife. Like you use it to cut fruits and vegetables and get healthy. or you, you know, like kill someone and go to prison, right? But what do you do with the guardrails that you set to make sure that you're using it for fruits and vegetables and not landing in prison with that is very crucial?

So having good governance, having a good idea of where your models are, how they're being used, who is using it, how are they impacting the end customer? So for example, if you have built a model for customer segmentation, if you are using a model in a medical field to do diagnosis, Have you thought about minority? Have you thought about women of color, right? A little plug. But honestly, it's about making sure that across these different stages that we discussed up front, that you have enough oversight, which is going to ensure that your models and data is used responsibly,

is key to making sure that you're scaling and you are scaling at a pace that you can truly innovate. And again, like this is, I know popular view is the minute you talk about, guard drills and governance, everyone is nervous about, you know, oh my God, I can't use this technology. But this is honestly all about scaling. And believe it or not, having guardrails and having a strong governance, enable scaling faster. Because you have a good view of what you've built, where you can use it, how you can use it, how can you reuse things that you've built. So those are sort of the key things that I would encourage your listeners to think about. You know, and maybe this is oversimplifying things, but that's something I always tried to do right here on the everyday AI show.

You know, I like when companies hire us to consult or something like that, I tell them like, okay, do you have a hardware policy? Do you have a computer policy? Do you have a software policy? An internet policy and email, right? And all these companies are like, yes, yes, yes, we do. But then they still maybe don't understand why they need an AI policy, right? You know, in general, Why is something like guardrails, governance, something as simple as AI policy, why is that so important when it seems so fundamental or so easy to skip? Such a great question, Jordan. I'm going to state some news articles that we all would have read. And I'm also going to talk a little bit about regulation.

That might help companies appreciate the value of having AI policy and the governance in place. So there was very recently, there was a financial institution in the news. where they had actually eliminated a certain segment with the society when they were thinking about mortgages and like giving the discount-based mortgages, etc. There were also healthcare companies that have been in the news because of certain treatments of certain segments very differently. If that wasn't enough, let's go back and think through what is happening in the regulatory landscape. The EU has already put out the EU AI Act. So you have to make sure that you're in the forefront.

You're able to answer the questions they're asking. the models that are considered high risk are treated very differently than the others. Now, here back in the states, we have, while we have the guardrails, we have everything that's being set up by the White House. There are also states that are coming up and saying, by the time all of this gets into being, we are still going to go ahead and do something for our own good. So California as an example, New York as an example. You have to declare in New York why a certain person was selected and the other was not if you're using AI for recruiting in each other. Closer home in California where I am based, there is this other piece of, I'm not going to

college regulation, but let's just say there's another piece of piece that has been discussed and that's going to come out that's going to monitor the way AI is being used. So if these, you know, just the reputational loss wasn't enough, I strongly encourage companies to think about the regulation that is going to come. Again, no one has a crystal ball and no one knows when it's coming, but it's for your own good to get ahead of it and get a handle of how you're going to handle it, whether it's through policies, whether it's through having an AI inventory, whether it's through having a user knowledge, all those good things together. So I'd say that generally, at least I see, usually enterprise companies, you know, are a little

further ahead because, you know, they have bigger data teams. They maybe have had people, you know, on AAL machine learning teams for decades. But maybe for those mediums, businesses that this is kind of new territory or businesses that have really grown very quickly in the last couple of years since this generative AI boom. Can you explain a little bit why there's, like, how does this, you know, bias and discrimination actually end up in these AI models, right? Because one thing people say is, well, if there's humans training the models and there's, you know, reinforcement learning from human feedback, right, it should be bias-free. So why is this still a concern when in theory there are, you know, humans, you know, at these big tech companies, open AI, Google,

Anthropic, et cetera, right? Why does this bias in discrimination still make its way into the models that hundreds of millions of people use? Look, as human beings, you and I both, Jordan, whether we acknowledge or no, have unconscious bias. And I'm not saying we bring it to work. I'm not saying that that's how discrimination creeps in, but we also have to understand that historically the data was very tilted for certain areas in the society. So for example, there were roles in the past that only men were hired for or only men could do. So the data

is inclined towards that. Now think about using that data set in this today's world, in this new age that we're all in, where women can do that job as well, but historically there was never that. So when you're trying to shortlist resume is using AI, you would not shortlist Samta. Because Samtha has never done the job before. And yes, there is human in the loop. And yes, there is a lot of different things that are there. But unless your AI is truly giving the human the ability to pick and choose and giving the rationale, which it doesn't happen all the time because of the techniques that are used to train the model,

it's hard to say. So it's not that, again, right, like it's not that reinforcement learning or these data biases cannot be corrected. It's a matter of catching them at the right time before it affects the society or employees or the board or the company in a certain way. And there's always a way to mitigate bias, minimize it. In my personal opinion, this is not Eva's opinion, but in my personal opinion, it is really hard to completely eliminate bias. I think it would take a lot for all of us to get together and make sure that happens, mitigate it to an extent that it doesn't harm anyone and be very conscious of how you are using AI.

I think that's such a good point, right? Something I, you know, I say that, you know, models are both a reflection of the internet and the internet is a reflection of us, right? So if there's biases, yeah, that means there's probably bias in the mirror, whether you know it's inherent, whether you can recognize it or not. You know, I want to take a look at this from maybe a slightly different angle, you know, Because when you think of discrimination, you know, you just mentioned, you know, it can be, you know, discriminating against women, discriminating against certain minority groups. But there's also maybe discrimination against the type of work.

And I know even when it comes to AI, you know, people are looking at big companies and they're like, oh, they're investing billions of dollars. And maybe they're just trying to get rid of my type of position. And these big companies, right, they're investing all this money to maybe replace my type of work. How can, you know, I know that's stretch. the meaning of the word a little bit. But I think, you know, even we see plenty of articles and studies where, you know, people who are later in their careers, they feel, okay, well, you know, there's ageism here and they're, you know, doing these models to go after things that, you know, my, you know,

my demographic is good at or whatever it may be. Where do you stand on that? And how can you make sense of that? Because it's a tricky situation. Oh, it definitely is. The cool part I think about AI, Jordan, and I love saying it all the time. because it gives me a spot on the cool kids table. But the good part about AI is that as humans,

it's hard to point out that bias and say, oh, you know, what you're doing is not right, but at least you can fix algorithms, right? That's how I see it. So to your point around, yes, there is definitely anxiety, and we have, we as EBA have done a lot of surveys, we do a lot of our own studies and research. What we found is very interesting,

which is the investments in AI are going up, but so is the anxiety. in people about AI going up. One of the prime reasons for that is because employees don't feel like they are getting enough training, enough education around AI. The more we can give that to our employees,

whatever size of your company is, whatever you're trying to do, the more you can let people play with this technology, invest in their upskilling, they are learning. Take the fear out of their minds on like AI is going to do, replace my job. I'm not going to have it because that's not productive for anyone. It's not productive for

your company. It's not productive for your employees. So taking that fear out, mitigating it by providing them really good tools that they can play with. They can understand that this is not something that's going to take away my job, but it's probably going to be someone who is using AI to enhance their day to day versus someone who's not using AI to enhance their day to day. So it's not about I'm going to replace this complete person. It's about how quickly are you adapting? How quickly are you willing to learn. How much are you willing to learn? What are the different things you're willing to bring to the table? So I think there's like two, three dimensions to this, right? One, companies making sure that they're giving the employees what they need. Employees making sure that they're being

receptive to it and getting on board with that and playing with the tech themselves, understanding the pros and cons and how it works and what happens. And then making sure that that's been passed along. Those things, I think, would help the investments go up. I mean, we'll get the investments where they're at, but also get the employees where they need to be instead of the fear. All right, have to take a quick break and shout out our partners at Microsoft. So if you didn't know, the Work Lab podcast from Microsoft is made for leaders who want to understand how work is changing because effective leaders adapt. They stay on top of trends.

They embrace any edge that they can get. And effective leaders also know that the key to understanding artificial intelligence is to get better at understanding human intelligence. So for real world lessons and actionable insights to help you stay ahead, check out the WorkLab podcast. That's WorkLab, no spaces available wherever you get your podcast. All right, let's get back into our conversation. So, you know, speaking of investments, I think, you know, EY has invested somewhere north of, you know, $1.4 billion into, you know, AI. And I know that covers a lot of different areas.

So, you know, I'm wondering, you know, you don't obviously have to spill company secrets. But in general, right, you all have been putting a lot of effort, money, research, in time into AI. And all companies are trying to do it the right way. And all companies are trying to avoid bias and discrimination. So maybe speaking generally, what are some takeaways that you have seen so far, at least when it comes to, you know, bias and discrimination. So we, I'm so proud of my firm for doing what we're doing right now, Jordan, because we're

eating our own dog food, right? I told you initially we have around 400,000 employees. And what we're doing is we're making sure that they are getting all the training they want. They are able to do whatever they need with a very enclosed YQ environment where they can upload documents. Everything, of course, goes through a lot of confidentiality checks, as you can imagine. like we have our clients data that we also have access to. So we are very, very, very, very careful of that. So that's sort of one bucket that we think about when we make these investments so that we're bringing our people along, mitigating the anxiety, mitigating the risk.

And also honestly, being able to serve our clients better, being able to deliver value every day. Because our people have been playing with it long before this became a thing, right? So we wanted to make sure we're front and center, leading, cutting edge. The other thing that we're doing is we're thinking about our own business. and we're thinking we're a big people business. So yes, there are ways we can augment and make our employees, you know, happy and life easier. And honestly, like, make my life easier.

I've been using AI a lot personally to do a lot of different things. But it's about also making sure that we're thinking through how we transform our business and doing. So are there lines of business that need more, you know, that can be transformed or disrupted in a very different way using AI? So we're thinking about it in like two buckets and all the investments that are, that we're making are going in these two buckets very broadly. And everything in bucket too, which is disrupting our business, making sure everything is like coming to bear is what we're

also bringing to our clients because we're making a lot of investments with our clients, joint investments and we're delivering a lot of value by all the lessons learned not only internally, but across the board. So those are different things that we're doing with the investments and working towards mitigating the fee here. Right. Yeah. And, you know, we'll make sure in the newsletter to share some of those things that EY has done, some great surveys, reports, information that you guys have put out there, which I think can be a great resource for our listeners. One thing that I want to, you know, you mentioned you have so many AIs in your title, right? Like what is the thing in their, you know, responsible AI.

So when we talk about AI bias and AI discrimination, right? As a responsible AI leader, where does the onus ultimately fall? Because I think, you know, in 2023 and in the early part of 2024, when there were still medium-sized and small enterprise companies sitting on this generative AI fence, right? They were almost like passing that bucket around and they're like, no, you take this. No, you take this. You're right. Ultimately, for those companies that haven't already kind of quote-unquote gone all in on generative AI, who's, like, who does it fall on to do this in a responsible way? I'm going to say something different and controversial, Jordan.

Oh, love it. But I think the onus of ResponsiblyEye is on each and every individual. I can have it in my title. You can take the title and have it on your title, right? And yes, we need to have someone responsible who's going to take care of it. But it should be a part of each and every one of our day-to-day lives and day-to-day responsibilities. We can, as an enterprise, whether you're small and medium, whether you're a lot, large enterprise, there's only this much that you can do with the guardrails and with everything

that you're setting up, but making sure that this is becoming a second nature to all of your employees who are even touching AI in a certain way, right? Even if they're not data scientists, even if they're not building it, but they're using it. Making sure that everyone understands the implications of this technology is key. Now, I'm a firm believer of having one person who wakes up thinking about it 100%, but then it is also on all of us. to make sure that we have that lens on every time we play with this type. So, you know, you talk there about being in this and in the day-to-day, right? And I think that, you know, maybe bias in AI models has maybe kept companies from implementing it

or not knowing, right, who is going to be the one leading this thing forward. And, you know, people say, oh, maybe AI is, you know, generative AI is just hype. And I think we finally moved past a lot of those things. And companies have finally gotten this realization where it's like, okay, we have no choice, right? So when you sit here, you know, you're in this day to day, you've been in and around AI for longer than all of us, right? Or many of us. Where do you see this going next, right? I'm not asking you to predict the future here, but you're in a day to day.

You're working with some of the largest companies in the world on AI implementation and ethics and responsible AI. What should we be looking at next? So the technology is going to rapidly change, as you're probably seeing as well, every single day, Jordan. When you share the news, probably you wake up and you're like, oh, wow, this was very different yesterday. So I think where companies are moving and where things are coming are, like I'm going to talk about a little bit of the technology and then get into the guardrails and the responsible AI angle. I think the big piece here is to be agile and not to think about this as like a one-time framework set up. Like you could do historically with a lot of different tech. I'm not saying you could not.

But now with AI, it's very different. Where I see this going is that where I see companies being successful with where the technology is going in terms of responsible AI is being extremely agile, being very open to feedback, giving very strong feedback to the system, making sure that that feedback is incorporated, and making sure that you're able to be so agile that you can change the frameworks, tweak it a little bit. Like we are now seeing with the tech, there's multimodal models coming in. Agentic architecture. All these things existed for a little bit. one could argue, but now everyone's starting to think about the practical use of it, right? First comes like, yes, we build this cool tech, here's what it can do, then comes like, okay, I'm a big business or I'm a small and medium business. How am I going to use it? Is it going to disrupt me?

And now we're at a point where we're trying to, we're all doing like practical use cases, implementations, business redefinitions. And with that, I think comes in the angle of being agile with the framework that you're going to set up so that again, you can be very, very careful. The other thing I think that is very important for companies to think about is how do you tech enable your responsible AI? So how do you truly have an audit trail? How do you make your AI audit ready? Do you have your inventory in a certain, you know, you're like kind of making sure that you've logged it in a certain tech-based thing so that you don't have Excel sheets flying around, which can be easily changed, right? So you want to also start thinking about along with these frameworks,

how do I have a solid tech backing and a solid like nothing less than being audit ready, to be honest. That would help company scale a lot. So, Sando, we've covered so much in a very short period of time, right? We've talked about the importance of guardrails, you know, how companies are investing so much money. But sometimes the more money they invest, the more anxiety it causes and employees. And we've tackled AI discrimination bias, so many things. But what is, as we wrap up here, maybe what is your one most important takeaway? And I know that's hard, right, to wrap all this in a bow, so to speak.

But for those business leaders who have AI bias and AI discrimination on the top of their mind, what's your takeaway message for that? Please definitely think about transforming your organization, your business, the society using data and AI. Do not be nervous about that, but please do it responsibly. That would be right. straightforward to the point. I love it. I love it. This was so good. Sauta, thank you so much for joining the Everyday AI show. We really appreciate your time and your insights. Thank you, Jordan. Thanks for having me. It was a pleasure. All right. And hey, as a reminder, y'all, that was a lot. We got like an education and a half. If you miss anything, don't worry about it. It's going to be in our newsletter. So if you haven't already, please go to Your EverydayAI.com. Sign up for the free daily newsletter.

helpful. If so, tell a friend, right? Where else can you come and learn from the leaders in AI in the world right here? So thank you for tuning in. We hope to see you back tomorrow and every day for more everyday AI. Thanks y'all. Meet Firefly AI assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere express and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time. See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us.

If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily news. newsletter so you don't get left behind. Go break some barriers and we'll see you next time.