In Good Company with Nicolai Tangen - Ethan Mollick: AI Urgency, Leadership Responsibility, and Why Companies Must Act Now

Episode Date: June 11, 2025

Which companies will lead and which will be left behind as AI transforms the way we work? Nicolai Tangen connects with Ethan Mollick, Wharton professor and author of 'Co-Intelligence: Living and Working with AI,' to explore how organizations can harness AI's revolutionary potential. They discuss the growing adoption of AI tools across workforces, proven tactics for driving company-wide implementation, the rise of autonomous AI agents, and why traditional training approaches may be missing the mark. Ethan reveals insights from his research showing that AI works best as a collaborative teammate rather than a replacement. With AI capabilities advancing faster than expected, organizations face increasing urgency to act. Tune in for an insightful conversation! In Good Company is hosted by Nicolai Tangen, CEO of Norges Bank Investment Management. New full episodes every Wednesday, and don't miss our Highlight episodes every Friday.  The production team for this episode includes Isabelle Karlsson and PLAN-B's Niklas Figenschau Johansen, Sebastian Langvik-Hansen and Pål Huuse. Background research was conducted by David Høysæter and Yohanna Akladious. Watch the episode on YouTube: Norges Bank Investment Management - YouTubeWant to learn more about the fund? The fund | Norges Bank Investment Management (nbim.no)Follow Nicolai Tangen on LinkedIn: Nicolai Tangen | LinkedInFollow NBIM on LinkedIn: Norges Bank Investment Management: Administrator for bedriftsside | LinkedInFollow NBIM on Instagram: Explore Norges Bank Investment Management on Instagram Hosted on Acast. See acast.com/privacy for more information.

Hi everybody, Nicola Tangen from the Norwegian Sovereign Wealth Fund. And today I am here with Ethan Molyk, one of my favorite professors, a professor at Wharton and who was out not long ago with a book called Co-Intelligence, Living and Working with AI. And actually you can see it behind Ethan there, down to the right. If you haven't got it, run and buy it. Ethan, if you were a chief AI officer in a company for the next three months, what kind of top actions would you take straight away?

So I think that the most important thing is to get people actually aware of where the state of the art in AI is. I talk to companies all the time, and I think that a lot of executive level people may have tried AI a while ago or didn't use it personally and don't realize how potentially transformative it is. And I think that I have a sort of general idea that you need to involve your team leadership,

you need to involve a set up a lab that's doing research and you need to think about how to roll this out to the crowd, to everybody in the organization. So you've got to kind of bring the whole company with you, which is not always an easy thing to do. And how do you maximize the uptake in the organization? So I think that's a really good question. And I think we're still figuring out the answers. But it's kind of like any other thing you want to do. You have to think about incentives.

And you have to think about leadership, right? So why are people set device to use this? Now, the thing that makes AI interesting is everybody's already using it. So there's a new study that just came out that showed and represented a sample of American workers that usage went from 30% in February to 40% of people using AI at work,

a little over 40% as of April. So it's used everywhere. The thing is people aren't just showing you they're using it because they're hiding it, because they're not incentivized to show you. They're worried they'll be fired if they use it either because they'll be told that they're using it improperly or because people realize they're not necessary or there's less workers needed. And so there's less workers needed. And so there's incentive problems.

And then there's also, for everyone else who isn't using it, there's role modeling problems. What do I do to get started? How do I use this? Why is it important? So you've got to both deal with the skeptics and the people already adopting it who may not be showing you they're using it yet. Why would people not want to show that they use it? I mean, here, if people don't use it, I'm like,

you know, that's bad, right? You know, so if you think about the incentives of the average worker who's using AI, first of all, they look like geniuses when they're using AI. And one problem that we see is people don't want to show you they're using it because it makes them look like they're not geniuses, like people get the AI credit. The second reason is IT in many companies is viewed as a thing that is a cost cutting measure. So if I could show you that the AI does some of my work, do I get a reward for that or do we fire employees? Do I feel safe to reveal it? And a lot of people are just using it to work less. So you're working 90% less. Am I going to show you I'm using AI so I have

to do more work as a result? Am I going to get a credit for doing this? So there's a lot of incentives to not share, probably a lot more than to share. Wow. How do you measure AI usage in a company? So the metric thing is really interesting, right? So what I see a lot of people drive towards, we're in a world of sort of new innovation, right? So it can be a little challenging to have a direct, like easy answer to that question of like, what does AI do?

How should we be using it immediately? So chief AI officer, you're doing experimentation. That said, I think raw usage actually matters. I mean, the typical KPIs you see are how many people are using our AI app internally. The downside of that is in most companies I talk to, that maxes out at 20% or 30% of the population. And that's because an additional third

is probably secretly using AI or doesn't want to use your apps, your apps aren't good enough. And another third is kind of waiting for instructions. Like they're happy to use it, but they're the opposite of the people who were early adopters. They want clearer rules about how I should use it and why.

So I think you can measure adoption by use of your app. I think you can measure adoption by other internal measures of how much you're using AI at work. But I think you have to be aware that there's still this secret cyborg problem of people not revealing their AI use. So how are the best companies going about this? Seeing some really interesting examples of how this works, I can tell a few of the stories I can tell all of them.

One example is radically changing incentives. I've seen companies that offer $10,000 bonuses at the end of every week to every employee that uses, the employee who best uses AI to automate their job. They think they're saving money versus other approaches. I've seen people build this into their hiring process. So before you hire somebody, you have to, your team has to try and use AI to do their job. And then you adjust your request for hiring

based on that experience. Or before you request money, you need to show how you're using AI to do it. Moderna has this really great example that they put together. They use AI for everything. What they did was build around the process of annual reviews. Basically, they built a whole series of GPTs that help people uncover their own performances,

improvement needs, and what they've done over the year, and talk to the right people about their jobs and things so they can write a really good yearly update about themselves. And they said, well, if you don't use these GPTs, you're probably not gonna do as well on your performance reviews and that will hurt your annual salary. And everybody ended up using this series of things which introduced them to AI.

So putting these bottlenecks in place where people have to use it, thinking about building into internal processes in a way that encourages positive use rather than negative use. Those tend to be really effective methods. So you need a combination of the stick and the character. I think you do and I think you also need role modeling, right? A leader who uses AI will make sure AI seems critical. Someone who doesn't use AI and says use it is kind of a problem. I mean,

you mentioned the chief AI officer at the beginning. One of the things that worries me a little bit about that is there is no ability to have a chief AI officer really at this point. I mean, generative AI has been around three years. Everybody who's on their very first project. So if you hire chief AI officer, that is often somebody who is actually doing machine learning beforehand, which is great.

Like AI writ large, but not generative AI. But they're not gonna have any idea of how to make an organization transform with AI. It has to be at the leadership level, I think. It has to be at the C level of the company. But does it make sense to have a central unit which kind of disperses the best use cases and make sure that everybody is at it?

So that's the concept of the lab that I was talking about earlier. So you do want a centralized unit for experimentation, but that lab has to be staffed partially by the crowd, by the people who are the best users inside your company. And they're going to be sort of ambidextrous. They're going to be shipping out like, here's a great use case, and also building a philagetic system that doesn't quite work yet, that automates the entire job, right? But I think the traditional view of having IT produce things, I think

that actually is on its way out. Some of the most interesting experiments I'm seeing in companies are dispersing engineers from the IT department out into work with subject matter experts. Because Vibe coding has strong limitations, but it's absolute best when you have a senior coder working with a subject matter expert. You can do incredible things in a couple of days that used to be whole processes. So I think there's gonna be

centralization of some AI functions, but it has to disperse other parts of the organization in a more decentralized way. Yeah, yeah. We do that. But what about ambassadors and people across the firm being trained regularly? Yeah, and I think one of the things we've been struggling with a lot as we think about AI use and observant companies has been what training means. So I think one of the things I know is the approach that you take that I think is smartest about training is using the opportunity for contact hours with AI. I think one of the smartest things I said in the book, I didn't know it was the smartest

thing at the time, but it turned out to be quite useful, has been that you just need to use AI to get it, right? You need to bring it to every work task and you'll figure out what it's good or bad at 10 plus hours. And people are very resistant, even very smart, very well-meaning, very self-motivated people are often very resistant to using this for reasons they can't fully explain, right? It's a weird technology. It's not that easy to use.

It's uncomfortable in some ways to sort of confront, I talk about the need for an existential crisis. So I think that it's hard to see, adoption is strange that way, right? So training can kind of get you into adoption, but we don't actually have that much we can teach people about AI, like a lot of the property techniques you learn don't really work that well or aren't that important

as the model gets better. So it's not that experience. And I think you will develop this idea of these champions. Some people will just get it and it's really important, like you said, to get those people out there. They're the cutting edge. They're the representatives of the lab in the world. Will the people who are top performers without AI be the top performers with AI? This is one of the biggest questions we're facing. If you think about it, there's four possibilities for what happens in an AI world on skills.

The first effect that we saw, and we saw this in our study, the Boston Consulting Group study I did with my friends at Harvard and MIT in the University of Warwick, where we found big performance gains, and this came out like a year and a half ago, kind of made a big stir, 40% improvement in quality for people who use GPT-4 versus not, big speed improvements. And a lot of other studies like that have shown a leveling effect. So bottom performers get the biggest boost. When you really look at what's happening,

it's actually the AI doing the work of the bottom performers, right? So the AI is pretty good. So it moves everybody up to the eighth percentile. So one option is it boosts the bottom performers. A second option that could exist simultaneously is the idea that top performers get some sort of massive returns.

We have a couple of studies show that, but not that many. It's hard to study. There's actually a, one of the best pieces of evidence for that was actually turned out to be a fraudulent paper from MIT that didn't exist, right? But I think there's a lot of suspicion that top performers using AI can get a huge boost, just harder to measure.

So there's a possibility that maybe there's a hundred times return if you're already a good coder. And we'll know more about that in the near future. There's also a possibility that, you a possibility that AI lifts everybody up. So everybody's performance goes up by a similar amount. And then there is this sort of other possibility that there's AI whisperers who are just good at AI, and they're the ones who get all the returns.

So we don't know whether it's contrary in the lower end, on the top end, whether it lifts everybody up, or whether there's just sort of magical AI whisperers who are just built to do this. And then agents are coming to replace everybody if you listen to the AI labs. And what do you think? Do you think that will happen? So I, you know, I think... I mean, first of all, just tell us, an agent, what exactly does an agent do? Yeah, so nobody has a good definition of anything in AI.

So people will, if you are a leader in a company, you will have vendors selling you things with every possible label and they'll say it's agentic and everything else because there's no clear definition. But the simplest version, and this sort of overlaps with what the labs think, is imagine any AI system that can be given a goal and can autonomously go and try and accomplish that goal without further human intervention using its own judgment and tools. That's what an agent does. So an agent, I would say, prepare me for this podcast and it would do all the research necessary. We can even demo a little agent right now if we wanted to do that, a little bit of agentic

work. But it's a tool that goes out and kind of does things in the world. And the idea with an agent is if an agent can go out and do work for us, then I get to skip the whole problem of trying to figure out how to integrate AI in with my workers because the AI will basically be a worker. I'll say, write this code for me and then deploy it and then test it and come back. I think we're further from that than the labs think, but I do think narrow agents are already very possible.

Deepak Chakravartty Have you got a good one available? But I do think narrow agents are already very possible. So deep. Have you got a good one available? I mean, I think, yeah, let's pick an agent for fun here. I will actually share my screen and I'm going to log into an agent while we talk. Okay. So let's look at an early version of an AI agent called Manus. Manus is a small Chinese company. Actually, I don't know how big they are. But they use

Claude as their underlying data set, so Anthropics data. And I just want to show you an agent would do, right? So in this case, I asked it to come up with 20 ideas for marketing slogans for new mail or cheese shop, select the best one, build a financial marketing plan, build a website, carrying cheeses. And so what you'll see is what the model actually does here is it comes up with a do-do list and then it starts coming up with a plan.

So here, it comes with a bunch of slogans and ranks those slogans and then decides on the best one. Then it goes out and actually does market research on the cheese industry. Then it goes ahead and puts together a whole financial plan for us that's many pages long. Then it goes ahead and figures out, and here's a to list that goes through to do this.

It does market research. It comes up with a color scheme for the site. And then ultimately what it does is without any further intervention from me, launches a website. And this is 100% created by the AI. Right? I can go shop cheeses, there's a little built a shopping cart functionality. I've got a subscription model I can do that has forms built in. And this is without any further

intervention from me at all. Right? So the idea is that this is- So how big was your prompt? How big was your prompt here on this? You saw it. The prompt was literally this one, right? That's not that complicated. It figured out all my intent and everything else from this. So this is an example of an agent at work, right? And when will these be widely available? I mean, this is available right now.

And I would also argue that 03, which is the model that everyone can use right now, is basically an agentic model already. So if we just go to ChatUBTs03, and I can just ask a question saying, let's ask it to do something. Let's say something like, give me 10 ideas for a new on trend shoe design based on market research and develop pricing, Proforma financials for the best one and show me a photo shoot of it. And I'm not so interested in the prompt being amazing as to show you what I mean by this.

So this is, 03 is an example of both a reasoning model, a model that goes through this process of thinking. We can talk more about why that's important and interesting if you want to. But what you'll see, what I want to show you about this is it's come up with a plan. Right. I'm going to look up latest trends in footwear, and then I'll come up with 10 designs, then I'll pick the best one. Then I'll create a pricing strategy.

So it's come with a plan. And what you'll see is that it's going to go through this step wise. So just on that command, right, it's going to, you can see it's doing web searches already about to do market research. Now it's choosing a design based on the research that it did. So it's a stepwise, agentic approach, right, that the AI takes even for just a simple command.

This is kind of how an agent works. So you can see it's doing multiple web searches. It's thinking about what this all means for the financials. It's figuring out, that it made a mistake in doing a search, so it's looking for different things that it did before. That's an agent at work. So this agent could have done these whole poll calls, right? And pretended to be you and me. So almost, right? There's a weird disconnect. Video and audio is a little bit different than other approaches to AI, like large language models themselves, but increasingly so. Yes,

you can now bring a live agent onto a call and it has a pretty good conversation with you and live video is getting much better. So if not now within the next months, that's a pretty plausible thing to be able to do. Do you think you would do better than you and me? No. I mean, not just because I want to keep our jobs, right? And I'm not just trying to flatter you, because we still have interview left

and I want you to make me look good. But aside from those issues, right? I think the thing about AI is that mostly it's not superhuman. So you're a very good podcaster, and I would say that anyway. But at the top of your field, whatever you're top at, you're definitely probably better than AI. And by the way, we got our shoe here. And so I think that it's not as good as us yet.

That's the real question is, does it get to be? And does it get to be good across, as good as across every field, which we'd call artificial general intelligence, AGI, a machine smarter than a human expert across every field. And that's the biggest question in AI right now. Absolutely. When will we be there? So that is a thing I don't know the answer to. If you ask the AI labs, they think two years. You ask AI skeptics, they think 10 years, which is a weird place to be for skeptics. To be like, yeah, this is definitely happening. It's just not happening right now.

Yeah. So what aspects of human work do you think AI will complement and what will it replace? So I think that is a really interesting question. The whole premise of the book is about co-intelligence, the idea that the machine works best with a human. That's the same way, right? And so right now, because the AI has these,

has what we call a jagged frontier. It's really good at some stuff you'd expect and not good at other stuff. And it's also missing functions, right? Like it's hard to have it kind of connect the world together. If I ask you to prepare for the podcast, I'm gonna get only so far with it. It will probably give me great preparatory advice, but then I still need to go on the

podcast and have this conversation with you. And it doesn't use my email well yet, so I can't let it interact completely with your team. So there's all these jagged edges that make it hard to use as a universal person replacement. Again, the goal of the AI Labs is agents will solve this problem by doing the work for us. Again, I'm a little reluctant to believe that we're going to be there as fast as they think. But I think compliments is pretty wide. I think compliments across a wide range of tasks.

I think where you're weakest is where it will substitute, but almost at the individual level. So if you're not good at idea generation, the AI is probably better at idea generation than you. If you're good at idea generation, you will definitely get value out of using AI, but you should probably be using your own idea generation as well. If you are terrible at email communication,

the AI is probably better at that than you, but that doesn't mean that you don't have a role to play in making sure the email is shaped properly. Yeah, I use it for a lot of my emails just to help because my English isn't so good. So it just really improves my emails a lot. Now, do you think to which extent is it now being used for CFOs trying to cut costs and to which extent is just amplifying power and helping us to do things better. So this is where companies get to make choices and one of the things I worry about with AI is if the leadership isn't well informed in companies about how they work, they view this as another

normal technology in the sense of like this is a cost-cutting measure. So I can increase productivity by 20% so I can fire 20% of my staff. I think there's two things that worry me about that approach outside of any sort of moral or other kinds of concerns you might have, which is that first of all, no one knows how to use this, right? There is no off the shelf product

that just does things for you with AI yet, they'll come, but you have to figure out how to use it inside your own company. And doing that requires you to actually have experts figure out how it's used, and the experts are your own organization. Your HR department is your R&D. So if you start firing people for using it, because AI makes it more efficient,

everyone just stops showing they're using AI, and you're going to be in trouble. So I think there's some danger in making a cost-cutting move right away. That doesn't mean people aren't doing it. The second big danger for making cost-cutting is, if you believe we're on the edge of a real revolution in how work gets done, which I do, then the idea that you're going to slim yourself down. So if I get 20% performance improvement, I'll cut 20% of people, feels like a really bad solution

in a world where everybody else is going to have 20% performance gain overnight. And so I think that organizations that are in growth mode will tend to outperform those who are using this as a cost cutting technology. But we don't have all the models yet. People are still figuring this out. When we had a breakfast recently, we talked about the role of compliance or general counsel. How are you seeing that?

So the opinion that will probably get in the most trouble is the two most, not universally, but the two most risky places to assign all of your AI responsibility to is often IT and legal. That's not true in every case, right? But legal compliance, the issue is, this is a weird technology. A lot of people know about it is based on rumor. The number of companies I talked to that will refer, where the legal office will refer to an incident where Samsung's data was stolen by ChatGPT,

which never happened, right? What actually happened was that people in Samsung were worried about ChatGPT using their data, so they banned use in the very early days. So it's all rumor based. Right now, these AI models are, I can tell you they're being used at Moderna and they're being used at JP Morgan, like companies that are very worried about data use with legal restrictions are using it, right? You guys are using AI. There

are ways to get around at this point the legal issue. So if the legal team is holding you back, it's because they don't fully understand the problem. And on the IT side, there are some incredibly brilliant and innovative IT people out there who will run with this. But the traditional way that IT handles a project, right, is they'll build a product for you around this and check out vendors and do this approach. And they'll make AI and IT technology as opposed to everybody technology.

And that's another danger is if you just, like we need to build an application for AI, well, we actually need to figure out use cases. Everybody needs to be using this to get there. So those are two different danger spots. They're not universal. I've seen some incredible compliance officers who lead directions with AI, but I've also seen resistance happen from there.

Yeah. No, I think we got a tremendous compliance here. Really, really good. But it's interesting because there are very few cases where a compliance officer can kill a company. I mean, here, if you hold back the usage, you kill your company. Because competition is just pulling apart by 20% a year. And within two years, you're dead.

I think that that urgency you feel is really interesting. I talked to lots of executives and you see this light switch go off for them. A lot of them are treating this as like they put this down seven levels of their organization or they've hired a consultancy who's going to produce a report on their AI readiness. And then you see the executives who kind of get it and there's just night and day because once you get what's happening here, it's very hard to not feel urgency and to not be anxious about resistance everywhere. And that's in our previous conversation,

that's one of the things that struck me was that feeling like, oh, this is the big one and we need to figure this out. And organizations that haven't put that in the list aren't gonna be in trouble. What proportion of companies have got it now, you think? I am surprised by how quickly the religion is spreading, but not as many as you think.

I mean, I talked to a lot of top executives. I would say, it's gone from like two or 3% of people, you know, getting it to where 20% of executives in a lot of the firms that should feel urgency feeling it. But that's a pretty big increase in a short time. So, I mean, this technology is remarkably rapidly adopted. Slight change of topic. How do you stay updated through these extraordinarily fast changing times? So, I think I'm probably one of the most kind of, you know, current people on using this. I've got this virtuous cycle going, right, which is as somebody who does a lot of work

in AI and is influential in the space, all the AI labs come to me. I don't take money from them, but they all give me early access to stuff, so I know what's coming. I'm on all these weird private discords and conversations. I'm on X and blue sky. So, but you know, I'm a professor who is on sabbatical and spends a lot of time thinking about AI stuff, and I'm having trouble keeping up. I think keeping up is challenging.

At the other hand, I don't think you need to that much, right? I actually think if you go with like a chat GPT or Gemini and just use it a bunch, those tend to have the really up-to-date models, but it's hard to kind of keep up otherwise. I mean, I've got a newsletter, so people can subscribe to that. But there isn't one sort of great source on this. And AI is still, it's sort of like how it's treating organizations. For a lot of publications, it's like one,

it's spread across multiple parts of their organization. So it's reported a little bit in politics, a little bit in technology, a little bit in business. So nobody really has a full picture, I think, including people in the AI labs. People move from one company to the other, right? And it seems like these models are not ahead for a long period of time. They are being overtaken all the time by other things, right? Is this something that will continue? So a lot of questions of the future are unclear. I think, you know, so the frontier models, the best models are in one point. There's only a few companies that can afford to make them at this point. And so generally you want to stay close to one of the model makers, right?

So the people who make frontier closed source models are, you know, open AI and anthropic and Google by and large, right? There are some other options out there, but those are sort of the three big closed source ones. Generally, if you go with one of those, they're going to stay in the frontier for the foreseeable future. There's not a reason to suspect that they might fall four months behind for a little while.

If that matters to you, that's what your lab is supposed to be doing in your company is like, how good is the new model should we switch over? Somebody else has to be doing that testing 24 seven. Another thing always surprised me in companies is how few of them have people assigned 24 seven to just working with AI. Like it just, there's lots of other departments

that work on things, but there's very few people whose job is to stay on top of these things. So you're in the lead, you get all this stuff, you're invited to pre-releases. When you now look into the future here, what's been the biggest surprise for you lately? So I think the biggest surprise for a lot of us has been this idea of reasoner models that you kind of see here, right? So I showed you a little bit of this as an example earlier, but it turns out that models that sort of think out loud outperform those that don't. And this very kind of simple trick has

increased the ability of AI by a tremendous amount. So I think the capability curve is coming faster than I expected it to. And that's been a big surprise. And then the other side of it that's been a big surprise is how fast adoption has occurred. So this is a very fast adoptive technology, according to any historical precedent. We're probably up to a billion people using, you know, chat GPT at this point. The last numbers they released were somewhere between 500 million and a billion people. There's another few hundred million using other models. This is an insanely high adoption rate for technology that sometimes doesn't work or is weird to use and where we don't quite know what it's good or bad for yet. And so I think that the speed of adoption and the speed of capability gain are both faster than I thought.

By the end of the year, what is it that it can do that it can't do now? So I think the arbitrary end of year deadline comes right in the middle of where agentic systems may actually be useful. We're starting to see these narrow purpose agents work quite well. So if you use AI for coding work, for example, you're starting to see that be very

useful, but the agents where you just say, fix the code for me, still kind of flawed. A narrow agent like Deep Research is a very good research agent already and replaces a lot of what you'd have an analyst do and increasingly will go into territory that was lawyer territory and other places of doing research and analysis and pulling stuff together. We're not quite there yet, but it's getting there. So I think the question is how quickly you could just assign an agent, do this job for me and it does a reasonably good pass that you still need a human interaction with. And I think we might be there for many jobs before the end of the year.

When can I say, hey, I'm going to be in New York first week of November, please look up the 10 best Mexican restaurants and book a table 730. I mean, I, I'm going to be in New York first week of November. Please look up the 10 best Mexican restaurants and book a table, 730. I mean, I think we're there already. That's not a hard problem. The only question is the book a restaurant. And if you use Manus, it would probably be able to email that person or do that.

That's an easy one. And I think your current AI will do everything to book the table for you. And that's an example of a connector that we just need to connect. It does voice. In fact, Google has been, even before generative AI took off, has a model where you try and book a table at a restaurant

that doesn't take reservations. It will call and have a voice call them, and an AI try and book the reservation for you. So that's already there. But one of the interesting things about AI is it's already there almost. You can't just chat to BT to do it. There's a little bit of hoops to jump through. So it's not a capability problem. It's a user experience problem. It's a UI problem. It's a communication problem. And that's, I think, AI ability, Fireout strips our ability to use it.

Now you wrote about Cybernetic teammate. When have you seen teammates most strikingly or before my human colleague? So this is again, a case of co-intelligence, right? Of working with the AI. So the cybernetic teammate paper, we again, with my colleagues at Harvard and MIT, University of Warwick and at Penn, we went out and we did an experiment Procter and Gamble, where they gave us 776 of their employees and we did real work that they actually do. So, you know, whether that's product analysis work or

product development or marketing, and we had them either work alone or in cross-functional teams of two with one marketing person, one business person, and one technical person. What we found was that individuals working alone with AI had the same performance statistically as teams of two. They also produced more diverse ideas than if they were working alone, and they were happier.

That's the cybernetic teammate idea of working with the AI to do things. I think a lot of people are already doing cybernetic teamwork, right? I know you are. You consult AI about all sorts of things. Now you have a small team working for you, you know, a panel of experts. So I think that people who use this, this is the natural way to use AI actually, is as a teammate where it's helping you and you're filling on goals on it. So I think we're seeing pretty big gains across the board from that approach. How can we best develop the people in the usage of this? What's the best way to really drive? I mean, we talked a bit about the institution and how you drive the institution forward.

What's the best way to drive the individuals? So I think that we are still learning about how to do that. This is not like traditional training that we give you four or five rules for using AI. I mean, I do think there's some things to learn, but a lot of training focuses on like prompting techniques, and prompting techniques turn out to be less and less important as time goes on. AI models are fairly resistant. Why is that becoming this important?

Well, two things I think are happening. One is the models are getting better, and larger, better models get your intent better. And the second is it turns out we just don't know a lot about prompting. It's very contingent. We have actually a little study that we did at the General AI Lives at Wharton,

where we measured the accuracy of AI on answering questions if you were polite to it versus if you were mean to it. And it turns out that politeness matters a huge amount on certain answers. If you do 100 tests on a particular math problem, it turns out in that math problem, if you say please, you get more accurate answers than if you don't say please, right? But it turns out on a separate math problem, saying please makes it do worse and yelling at it does better.

And we don't know why it works in one case, but not the other. So the average effect is zero, basically, but the individual question effect is quite large. So if we told people, always be polite or always use this approach, it's sometimes going to work, sometimes it's going to backfire. And the best- So I normally say, please, I should just not do it. It doesn't seem to help.

But then again, I think there's value in the mental model of treating AI like a person, and if please helps you do that, even though it's not a person, then I think it's no problem. And it's just kind of hard otherwise, right? It feels weird to just order your computer around. And then if you talk to people who really believe super intelligence is coming soon, you better be polite now because they'll know. How do you test for AI literacy when you interview people in a job interview? I want to avoid over-indexing on today's literacy because we don't have a definition of AI literacy.

I mean, I teach AI and we don't have a definition of being AI literate. It tends to be you're an AI user and then the question is, are you a sophisticated user or not? Does that matter to you? Because it might turn out that they're not a sophisticated user, but a small amount of experience with a sophisticated user will make them good. If we're judging AI literacy today, you're really judging people's independent ability

to go out and figure out ways to use AI in their job. And that can be valuable, but that doesn't always tie in with subject matter expertise and other issues. So I would just ask people at this point, show me, you know, when we were hiring people for the generative AI lab, we both asked them how they used AI, we would give them a task and say, use AI to accomplish the impossible task and get as far as you can in an hour, right? And we would also be asking people about, you know, actually interviewing them about what's working, what isn't for their AI use and having them show us some of that that. So I mean, I think you would judge AI,

but that's kind of an example of usage and creativity with AI, and we're an AI lab, so we kind of need those people. I worry at this stage about companies picking AI literacy as the major issue. One of the things we found from the BCG study was that more junior people were actually worse at using AI in the organization than more senior people.

They may have gotten AI, but they didn't understand the organization. So they would produce a document and say, this is a great memo. And then someone with 20 years of experience would look at the memo and say, no, it needs to be better in these following six ways. They would have been better using AI than the junior person. So I worry, it kind of goes back to that chief AI officer example you gave earlier. Companies are trying to hire their way out of this problem. And I don't think there's a way of hiring your way out of the problem without the executives

also engaging deeply and making these decisions about what AI means in their organization. What personality type do you think will adopt the best? So the model, we talk about agency a lot. We don't fully know what it means. I teach entrepreneurship, so there's an entrepreneurial action or a tendency to entrepreneurship that I think is the same thing as agency, which is a feeling that you have control over your environment, a locus of control, that you are in control of your own destiny and you should seek out opportunities to do things that are different or better improved.

That is not something everybody has. That seems to play a role in AI, but we don't have measurements of it. So I'm a social scientist. I'm making stuff up here, which always makes me feel bad because we don't have a measure. I don't have an AI measurement

that makes people, that can tell you, but there is this sort of agentic sense. On the other hand, some of the most entrepreneurial interesting people I know are late adopters of AI. For whatever reason, they hit a barrier when they were using it. And I'm not sure what that is.

So again, we don't have easy answers to almost any question about AI at this point. Two slightly different ones. How do you see the regulation in Europe versus the US? So I'm not a regulation expert, but I will tell you that one of the interesting things that happened with the release of DeepSeq, which is a Chinese open source model, R1, DeepSeq R1 specifically, was it was the first time a Chinese model was on the frontier of capabilities. It did not beat American models, they're better again at this point, but it's a typical closed source open source.

There are a few months behind, but a very good model. It caused a bit of a panic in the US that I don't think was warranted necessarily of even from a great powers competition standpoint. But that sort of, that plus the new administration, I think has created this opportunity where there's very little regulation. I mean, in fact, one of the bills in front of Congress right now in the US is that this

would ban AI regulation at the state level for the next 10 years. I don't know whether that passes or not, but I think there is a desire to put in the accelerator and less regulation and more sort of letting AI rip. I think that my impression from talking to people in Europe has been there has been a move to less regulation from a very regulated viewpoint. I think part of this came down to the idea that a lot of the initial regulation was based around existential risk, which I think is important, but we don't know how to measure. And now I think we have to start moving towards a place where we're regulating harm. So even though there was this there's a bill saying you can't regulate AI, another bill went through Congress banning deepfakes. So I think we're going to have to regulate the outcomes of AI, but I think that Europe is still a much tighter regulatory

environment, but is probably looser than it was a couple of years ago on the AI front. You mentioned existential risk. Is there an existential risk? the AI front. You mentioned existential risk. Is there an existential risk? I have no idea, but a lot of very smart people think there is. For me, the idea that AI gets smart enough that it takes over the world doesn't feel real to me, but that doesn't mean that it isn't. As a good social scientist, I have to realize my own views are kind of secondary

to what does smart people say if we have a forecasting contest. And there's a lot of the founders of AI think that it's an existential danger to humanity. A lot of very smart people think that way. So we have to view it as an existential risk as well. The problem is we don't know what form that takes and we don't know what the level is. Originally a GPT-4 or GPT-03 class model was considered an existential risk level before we had these things. Now we have them, we're like, these aren't going to let anyone take over the world or spontaneously go out and destroy the economy. Is there a magical point to where that happens?

I just don't know. Well, Ethan, this has been 40 incredible minutes. You are at the very forefront of the very forefront. It's been so interesting talking to you. So keep up the good work. Look forward to staying in touch and see what the future brings. Thank you. It's a pleasure being here. And I will say it's exciting to talk to people who get the AI thing, because I think it is important and spreading that news is important. We can't go with eyes closed. We have to make decisions about the future

and you can only do that if you're using these things. Absolutely, fantastic. Yeah.