Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 2x08: Algorithmic Bias and Subjective Decision Making with Alf Rehn

Episode Date: February 23, 2021

Biases can creep into any data set, and these can cause trouble when this data is used to train an AI model. Alf Rehn, Professor of Innovation, Design, and Management at the University of Southern Denmark, joins Andy Thurai and Stephen Foskett to discuss the lessons he has learned about algorithmic bias based on his work with the Velux Foundations Algorithms, Data and Democracy project. Society is directing artificial intelligence to solving some problems and ignoring others, and this can create biases as surely as data selection in model training. Can we ever truly eliminate bias? If not how do we work against it? Can we keep the genie in the bottle even if we want to? And can machines ever make sound, ethical subjective decisions? Guests and Hosts Alf Rehn, Professor of Innovation, Design and Management at the University of Southern Denmark. Find Alf on Twitter as @AlfRehn or at AlfRehn.com Andy Thurai, technology influencer and thought leader. Find Andy’s content at theFieldCTO.com and on Twitter at @AndyThurai Stephen Foskett, Publisher of Gestalt IT and Organizer of Tech Field Day. Find Stephen’s writing at GestaltIT.com and on Twitter at @SFoskett Date: 2/23/2021 Tags: @SFoskett, @AndyThurai, @AlfRehn

Welcome to Utilizing AI, the podcast about enterprise applications for machine learning, deep learning, and other artificial intelligence topics. Each episode brings in enterprise infrastructure experts to discuss applications of today's data center of AI. I'm your host, Stephen Foskett, organizer of Tech Field and publisher of Gestalt IT. You can find me on Twitter at S Foskett. Today we're discussing algorithmic bias and how AI is basically only as good as the data that gets into it. So first let me introduce our guest, Alf Reijen from the,

well, you can tell us who you are. Pleasure to be here. Yes, my name is Alfred. I'm a professor of innovation, design, and management at the University of Southern Denmark. So coming to you live from Copenhagen today. And I study, among other things, how AI is utilized in innovation and leadership. I am Andy Thorey, founder and principal at thefieldcto.com, where we provide unbiased emerging tech advisory and consulting services. You can follow me on Twitter at Andy Thurai or on thefieldcto.com. Again, that's thefieldcto.com.

So Alf, you were brought to our attention because previously in episodes of Utilizing AI, we've talked about bias and how algorithmic bias can creep in almost anywhere whenever we're talking about applications of AI. We've had people before who were data scientists and interested in basically making sure that the data is clean, that you're building your model on. And I know that this is something that you've studied as well, but coming to it more from that side of things than from the AI side of things. So essentially, what I'd love for you to do is to share your overall message with our audience of enterprise and AI people. Where does the bias come from and what should we be looking out for that's thanks that's a that's a very good question i'm not sure i'm i can answer it fully you know actually i know i can't answer it fully so i've been long interested in knowledge knowledge is a

obviously all professors are interested in knowledge but i've been passionate about knowledge and particularly the boundaries of knowledge my entire life. That is, how do we define a field? How do we define what is okay to study and not to study? How do we define what kind of innovation is a good innovation or a bad innovation? So I've been looking at these things both from a kind of, let's call it, widely an ethics perspective, but also from a technological perspective for quite a while. And I've recently become more enmeshed in this because in Denmark, there is a tremendous amount of interest on artificial intelligences and algorithms. And one of the largest research foundations in the country decided that as their main project for basically years to come, they would fund something known as the ADD project, Algorithms, Data and Democracy,

in which they stated that because algorithms, their potential biases and artificial intelligences are becoming so incredibly central to society that they wanted top-level research on how we can keep up trust in this coming age of, well, everything AI. A 10-year research project, 100 million Danish crowns, which is a nice chunk of cash. And I was one of the consortium of six fine fine researchers who managed to get this funding. In this project, I have several different interests, but my key interest has been in my personal field, which is innovation. That is, if you followed what innovation evangelists are saying these days, they are looking to artificial intelligence and saying, listen, this is going to be the greatest things in sliced bread.

We'll have artificial intelligences going through ideas faster than any human could possibly ever do. It will generate iterations that we cannot even fathom. It will manage to kind of trawl the internet and connect things that no one's connected before. And it sounds like a beautiful idea. Humans and artificial intelligence is working together to generate innovations that will solve all our problems. But I've been an innovation researcher for quite a while, and I know a little bit about innovation.

And I know, for instance, that one of the things that has defined innovation for the last years has been actually how biased it's all been. People like you and I, both kind of white, middle-aged-ish, Western men. I mean, obviously I'm not as handsome as you, but we're both at least passable. We have been kind of awarded with this treasure trove of toys. I have so much innovation directed at me. Everyone wants to give me more apps with which to order food or designs or whatever else I like to think of. And at the same time a lot of the really big wicked problems, I mean god we haven't been able to create face masks that don't fog up glasses yet. We are not always solving the right problems.

And what I'm kind of now working on is trying to understand, if we create artificial intelligences to help us in innovation, how do we make sure that they actually gear towards the right kind of innovations? But it's not just the ALF show in which a helpful artificial intelligence tries to amuse me to death with more and more of the toys I already have too many of. How do we make sure that these artificial intelligences actually direct their interest also to the problems of, let's say, poor single mothers, the elderly, children in less developed countries. I am all for artificial intelligence and I'm all for innovation, but we must be clear that these must be governed so as not the current biases in innovation creep into the machine learning models and into the neural networks. That's a lot of good stuff in there. I got about 20 questions on that statement you made. But let's start over the first one.

The interesting acronym you have, ADD, which stands for something else here in the US, by the way. So interestingly enough that you lump algorithms, data and democracy together. And of course, a good AI project or a good solid results would depend as much on algorithm as well as on data, right? So data has become to a level of democratization now with a lot of collection there, a lot of democratization, so it's become easier, but algorithms is still very proprietary-ish.

Do you see in your experience, companies are moving more towards algorithmic democracy or more towards algorithmic proprietary? No, I think you're very right, but I'm not, I think you're partially right. Yes, data has become more kind of open and so on, but I think there's also still tremendous troves of dark data. And particularly as we see at how data collection structures are being set up, I'm seeing more and more of this dark data. I'm seeing less and less transparency in exactly what data is actually being put into data sets and so on. And as for companies,

I don't think it's quite gotten up to a kind of cold algorithmic war, in that there is still kind of an understanding that we need to share the wealth, we need to kind of try to understand best practices. And I see a lot of good communication between CIOs about what works and what doesn't in artificial intelligence. But at the same time, I am kind of starting to hear when I talk to CEOs and the likes, I'm starting to hear kind of, if not quite yet a weaponization of artificial intelligence, but still kind of, they are starting to talk in a way and go, oh, yeah, we must make sure that we don't get left behind. We must kind of strike first. A lot of the discourse is getting to an almost warlike level, because I think a lot of CEOs have realized that, oh my god, it's no longer just training a computer to play Go or recognizing kittens in videos. This actually has real world consequences. And they are quite keen on, whilst understanding they might not have found the killer app yet, but they're quite keen on

there being a killer app out there. And I think that what we've now seen in social media, for instance, has been really impactful on CEOs. And I will not name names here, but after the unpleasantness at the capital, I did talk to a European CEO who said, yeah, that seems like very, very good marketing. And you might now be shocked at somebody kind of looking at insurrection and seeing this as a marketing angle. But what he meant was, my god, yeah, Christ, people, if you could manipulate those people to not kind of cause havoc, but rather buy more of our brands, that'd be a fantastic thing. So I think that we're going to move more and more towards kind of a closed islands approach to artificial intelligences and where companies will become more and more protective. And I think this is dangerous.

And I think that this is why the algorithmic governance will be so very, very important. Because state and democracy, that democracy is a mighty big word to throw out there. But we've already seen that filter bubbles create very different images of what is democracy and how democracy works. It has created lies about democracies. And that's just in social media.

That's just on a still fairly small bit. If this starts to work also on how pricing works, how innovation works, how research decisions are taken, that can have huge impacts. And the reason I'm kind of obsessed with this now is because I started a simple thought experiment. My field, innovation, might seem far away from democracy in a sense, but at the same time, every innovation decision, every research funding decision that was made is, of course, to support one technological trajectory, but, and this is what people don't think about, it also cuts off a lot of other potential technological trajectories. So the decisions about innovation we make in the here and now will affect our children and their children 20, 30, 50 years into the future.

And this is an incredibly important notion to have a stable functioning democratic society to ensure the technological development, which has defined our societies over the last decades, still keeps on a good track. This is not just about who has the kind of cutest algorithm. This is about the worlds and futures we built. This is really interesting because, as I said in the past, we've talked quite a lot about how essentially a poisoned well of data can change the outcome of the artificial intelligence algorithm. But, you know, like you, I have a little bit of background in this, having studied science, technology and society as my undergraduate degree and focusing on things like technological determinism and Bertrand Russell and so on. I appreciate you bringing it up, which is essentially the choices we make about what algorithms to create and what problems to solve has as much bias problem in this way or that way is just as challenging from a data quality as the quality of the data that is used to build the model. Absolutely, absolutely. And I mean, we can look right now, it comes certainly as no shock to anyone that a lot of kind of money was put into studying pandemic

responses during the last year. So a lot of research funds were taken away from wherever else they would have gone and put into who can create PPEs faster, can we get a vaccine quicker, all these things, important things. But what we sort of often then forget is that by now walking around with, I've talked about pandemic blinders on, we simply do not see the many other issues we should be focusing on. And yes, exactly as you say, choosing what artificial intelligences, what algorithms to develop, we are making actually very, very big decisions without necessarily realizing what all this will lead to. And now, I'm not a Luddite by any kind of measure, and I'm not kind of saying that, oh, if we go down this path, we'll have Skynet in three years. But I do think that we don't pay enough attention to just how subtle bias can be. I mean, obviously, we've seen a few chatbots become

racist, which had its own shock effect and its own dark hilarity to it as well. But I think that the true algorithmic bias will be far subtler. It will function a little bit like sexism still works in society. I mean, as we all know, sexism doesn't work like we men get together in a man club and say, let's oppress women. I mean, I've never been invited to one of those meetings where, so if there are any, I'm seemingly out of the club. Instead, it's that little thing. It's when you interview and there's a good female candidate and a good male candidate, and you somehow think, nah, yeah, but he could be a really nice golf buddy. And then it's that subtle little thing that twists. A thing you might not even realize is sexism or bias, but just creeps in. And I think that is why I'm also kind of worried about using artificial intelligences in developing innovation.

I don't believe that they will just create toys for ALF. Of course not. But I think that they might generate ideas where those ideas that speak to a certain segment will get just a little bit of a bump up, just a few more extra ideas. And over time, those small, small shifts, of course, compound. kind of ideas that do not get pushed up, could have been the ones that created entirely new technology trajectories that could solve really what we call wicked problems. So not trying to be alarmist, but more saying it is the choices we make about where to focus and what to develop, but also how even a small, small bias, a small kind of shift, so subtle that it might not even be seen if you don't really kind of go look for it, actually can

have tremendous effects. That's interesting. So going back to your first answer, you talked about, you know, when it comes to AI, this is what people miss when it comes to, you know, before AI days. Well, there's no before AI days because AI has been around since 70s. But I'm talking about mainframe adoption of AI in the enterprise world. We all lived and died by the structured data domain. And that's how the decisions were made. But with AI, there's structured data and there's unstructured data. And then part of that could be a dark data,

but then there's truly dark data, which nobody knows the meaning of, right? So there can be bias in each one of those areas. Structured data bias is easy to eliminate. Unstructured data bias is going to be extremely difficult to eliminate simply because we are talking about videos, audios, soundbites, and text that's free form, articles. It's going to be extremely difficult to make

that subjective bias elimination decision, the unstructured and dark data. The data is not even available to do that. Plus on top of it, the algorithms, it's hard to decide whether it's biased or not. So how do you, I mean, there are so many variables in here. How do you think it's possible to eliminate the bias from all the facets of that? If somebody were to eliminate, particularly when they're doing innovation portion?

Well, to begin with, I don't believe bias can ever be truly eradicated. We can work with bias and we can try to minimize bias and we can call bias out when we see it, but I don't think there will ever be completely 100% unbiased systems. I mean, human society has a lot of biases in it. And even though we've maybe managed to work against some of the more horrific versions, we still, in our everyday life, we work with biases. To a degree, an element of bias is also even necessary for some forms of decision making. That is, we trust our gut, which is a bias. We kind of have a lot of kind of shortcuts in our head that help us make quick decisions. And these could be also understood as biases. But so I'm not saying we could eliminate,

but what I'm trying to say is that we need to have a broader conversation, a broader dialogue about what do we actually know about these biases and also what biases might we be missing because we don't even understand to look for them. And you're perfectly right. So with Rumsfeld's fantastic old statement, the unknown unknown,

that which we don't even know that we don't know, is of course impossible to, how can you take out bias from something you have absolutely no knowledge of, even having knowledge about? But that doesn't mean that we can't have a conversation about it. Now, innovation is always a journey into the unknown unknown. It is, yes, I was talking to a R&B manager earlier today who said that, yeah, you can have a decent picture about how his specific field, which has to do with fire extinguishing and intelligent systems for putting out fires, can evolve in two years. Because he knows where the money is gone. he knows what other people are working on, he knows approximately the speed at which his field works. He says, five years, I've no idea, 10 years, your guess is as good as mine. And he had, of course, the humility to kind of admit this,

because we human beings sometimes at least have the capacity of knowing that we don't know. But when I then look at how people are saying that, okay, we should use artificial intelligence systems to help us with innovation, how on earth could even the most magnificent algorithm fed the finest of historical data predict something such as, for instance, the emergence of the internet? I don't know if you kind of looked into instance the emergence of the internet i don't know if you kind of looked into the the early history of the internet but my god there were some weird decisions made and some some weird technologies pushed and lots of and i'm not just talking about californian

kids in garages now i'm talking way beforeANET, when the first ideas of the protocols that later emerged into the internet. If those funding decisions, allowing a bunch of, I think they could proudly call themselves geeks, working with highly untested ideas, super kind of weird and challenging notions, and the early days of the artificial intelligence community would be something similar, if they hadn't gotten that shot, if they hadn't gotten that chance, today we wouldn't have this conversation because Zoom wouldn't exist. Podcasts would be, what the hell is a podcast? I don't have a pod and I don't wear a cast. And that is why I am worried about allowing too much algorithmic logic into R&D, into artificial intelligence, into research funding decisions. Not that we shouldn't, for instance, have systems that help us out, that challenge us, that test things and so on. But I think that there will need to be a place for a screw it, let's do it attitude.

A I don't know where this will lead us, but let's test it anyway. Because innovation history is basically that. It is people trying an idea that is so wild, so out there, so seemingly pointless, ludicrous, even childish, that no sensible system, no intelligence would have funded it. Yet somebody did try it out. Somebody started building something. And today we have great things for it.

Sorry, I'm getting a little preachy here. I do apologize. No, so a couple of things you mentioned that requires or deserves follow-up. There was a thing you mentioned about, you know, we talked about the algorithmic proprietoriness and openness. Most companies are willing to have the data openness because it's somewhat available. You could either buy or available in a public domain. But most companies still feel and think that the proprietary algorithm they created is somewhat their domain. And some companies have moved away from that, saying that, you know what, if I were to create

something and only if I use it and my customers use it, then it's not exactly validated to find problems in it. For example, the original security protocols when we created, right? I'm talking about the encryption algorithms and whatnot. When it was not truly tested, there were holes. We are finding out holes even after 20, 30 years after the most experienced ones or the most evolved ones. How do you avoid such things?

The only way you could avoid that would be open that up to the public to test it out, but you move into a more of a business value domain. Data is democratized, algorithms are democratized, but you move into the business value domain saying that I know this business really well, and I can use this data, use this algorithm. I can give you a business decision only I can give. There are some companies moving towards that. But in your experience, do you think that movement is happening now in general, or you're still seeing very restricted domain? I think that the smartest companies realize that openness is good. And I think that we are, I'm happy to see that more and more companies

understand that transparency is actually good for all. The problem, of course, being that there are also companies who've been very good at utilizing secrecy and, well, proprietary technologies, and not least in the social media sphere we've seen a lot of this. And it is truly a wicked problem, because in the best possible world, I guess, we would have sort of the nation-state, and I'm not talking the government, but some form of independent agency representing the nation state, would kind of have lost, say, of algorithmic governance. You'd have super smart people who only try to benefit society by having governance structures for algorithms. And even as

I'm saying this, I realize how hopelessly naive that is, to imagine that that kind of super organization, A, could be created, and B, would not at some point be politically tainted, or even infiltrated. So it is really a wicked problem. We all know that the degree of trust we have on our governments isn't the highest it's ever been. It's in some places quite a lot lower than it's ever been. And so the notion for most companies that yes, you should allow government to take over which algorithms you are using and to test your artificial intelligences, I mean, then companies would laugh you out of the room. So I don't know. But I know that the debate about algorithmic governance, AI governance, is becoming critically important. And it's not just important from some

kind of, oh, I want to check everyone's algorithm. It's important for society as a whole. And it occurs to me as well that there's a connection here sort of to the development of other sort of pervasive technologies in that there's almost a genie in the bottle problem that if even if we have some kind of governance to control this, even if we are pushing back against bias, you know, water may flow to the lowest point, and we may end up having, you know, perhaps, you know, governments like yours, who are funding this research and looking and concerned about this might try to keep that genie in the bottle, but other governments around the world, or people, you know, extra governmental people might just say, forget it, we're going to do what we want to do. And, you know, you mentioned the history of the internet. I mean, I think that

one of the interesting aspects there is that for positive and for negative, it was a pretty wild west for a while in terms of what we're going to do. And this sort of, I don't know, these technicians and nerds who just sort of went forward with this belief that, a libertarian belief that no matter what happens, it will be okay because, you know, the best solution always wins. We don't know that to be the case. In fact, history has shown us that that may not be the case. One of the things that I, even now, I find it difficult in adopting the AI technology is the subjective decision-making. Going back to that, having enough quality data and then proper decision, you could make, machines can make a decision fairly quickly. I mean, missions with the capacity and computers, supercomputers and cloud now make decisions.

But there are times humans make the subjective decisions by looking at it, saying that, you know, the data tells me in one way, we had to go, I wrote a blog about this recently, but then I'm going to make a decision based on my gut feeling or a subjective decision.

Do you think missions will ever get to that stage? And even if it does decision do you think machines will ever get to that stage and even if it does do you think we'd be able to believe the decision subjective decisions machines make i'm i'm still a technological optimist i'm actually i'm actually a huge optimist on on most in most ways i i do believe that maybe not during my lifetime but at some point we will have machines that will be able to make sound, subjective, ethical decisions. However, I'm not entirely sure that they will always manage to make them. I'm not entirely sure that this can ever be fully trusted. So I do believe that oversight from humans who are actually, we are actually very good at a couple of things.

We're very good at realizing how things could go to hell. That is how a decision that might seem smart in the here and now can actually lead up to catastrophe somewhere down the line. And we're actually quite good at realizing, yeah, that sounds like a good solution, but you know what? That goes against my values. And those are things that I think we will always need to have a human for. Maybe more of an oversight character, because I think that we already, my Netflix queue is pretty smart as is, and I've only been developed for a little while. Over the decades, I could see that

machines could become really good at making fairly hefty decisions. But at the end of the run, we need to have some form of human oversight. I don't want to count out humanity quite yet. Well, thank you so much, Alf. Honestly, I think we could have talked with you about this for hours. But before we go, we do have a new tradition here on season two of Utilizing AI, where we surprise our guests with a couple of AI-related questions for just a quick, easy answer. And unfortunately, you being a researcher have probably already developed answers to most of the questions we would ask. So I'm going to ask them anyway and we'll see. I don't think I can stump you.

So let's call them low-end administrative jobs, I think will, I think it'll be a slow erosion, because there'll be still people needed to check and make sure that everything works. But definitely, over the next five years, I think, if not a huge percentage, but a couple of percentage points or maybe one of existing jobs I totally see being taken over by AIs, yes. All right. And just one more here. And this is a new one that we haven't asked anybody yet. Can you think of any fields of study or endeavor that have not been touched by AI to date? Yes, a couple. As far as I know, prostitution has been untouched by artificial intelligence. If it has, then... Although actually one of my... A friend and colleague over in the US actually has written on the ethics of sex robots.

So that clearly is coming as well. But it's getting fewer and fewer. But I can still think of a couple. But these days, I even know farmers who utilize AI in their work. Thank you so much for joining us today. We really enjoyed this conversation, and I would have loved to have spent a lot more time. Before we go, Alf, where can people connect with you and follow your thoughts on these subjects? I'm on Twitter with my name as a handle, at Alfren.

I'm easily found on most social media, such as LinkedIn, Medium, and the likes. And my webpage, alfren.com, is still up and actually being updated very soon. You can follow me on Twitter at Andy Thurai or LinkedIn or at thefieldcto.com. That's thefieldcto.com. And I'm Stephen Foskett. You can find me at S Foskett on most social media sites, including LinkedIn and Twitter. And I would love to connect with you there. Well, thank you everyone for listening to this episode of the Utilizing AI podcast.

If you enjoy these episodes, please do subscribe and rate the show on your favorite podcast platform since that does help our distribution. And if you enjoyed it, please do share it with your friends and continue the conversation online. This podcast is brought to you by gestaltit.com and thefieldcto.com.

For show notes and more episodes, go to utilizing-ai.com or find us on Twitter at utilizing underscore AI. Thanks, and we'll see you next time.