Big Technology Podcast - How AI Should Handle News, Politics, Medicine, and Mental Health — With Campbell Brown

Episode Date: August 26, 2026

Campbell Brown is the CEO of Forum AI. Brown joins Big Technology Podcast to discuss how AI models should handle sensitive information involving news, politics, medicine, and mental health. Tune in to hear how experts can evaluate chatbots for accuracy, bias, source quality, and context, and why independent standards will be essential as people increasingly trust AI’s answers. We also cover AI’s threat to the journalism business, the limits of expert consensus, chatbot sycophancy, election misinformation, and the rise of AI companions. Hit play for a nuanced conversation about who should decide what AI tells us when the stakes are highest. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Watch the full documentary here: https://www.gravitee.io/ai-agent-documentary Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Stop online threats before they become real-world attacks. Visit ⁠ironwall.com/BIGTECHNOLOGY⁠ and request a free Risk Assessment to see exactly how exposed your executives are Learn more about your ad choices. Visit megaphone.fm/adchoices

How should AI models treat controversial information like politics and vaccines? We'll cover it all with media and tech veteran Campbell Brown right after this. Legendary is here with BetMGM Sportsbook. Sign up to unlock access to hundreds of prematch and live betting options for all your favorite sports. Download the BetMGM app. See BetMGM.com for terms, 18 plus Alberta only. Please play responsibly, subject to eligibility requirements. If gambling is affecting your mental health or well-being,

211 Alberta is here to help. Call or text 211 or visit ab.211.com. Fet MGM operates pursuant to an operating agreement with the Alberta IGaming Corporation. Standard message and data rates may apply. Get your tickets now for the dog stars. It's no surprise how fast civilization can disappear. Go! A therapist.

Kill or be killed. Jacob Allurdy. There are people out there? They need her help. Josh Bullen. Shoot first and inquire afterwards. Margaret Quali. Okay, tough play.

and Guy Peers. Arm up. The Dog Stars, directed by Ridley Scott in Theaters Friday. Get tickets now. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. We have a great show for you today. We're going to talk all about how AI models should cover controversial information. How should they answer questions about political candidates, about vaccines, about self-harm?

We're going to do it with the perfect person here because Campbell Brown is joining us. She is the CEO of Florida. May I, a new company tackling this challenge. Campbell, great to see you. Great to see you too, Alex. It's wonderful to be here. Great to have you here. We go way back. You ran Meadows Media Department. Started off overseeing news. And yes. Yeah. Over saw news. Some challenges there. Which is actually like kind of a perfect moment because not only did you do that, you were an anchor at CNN and NBC. So you've seen this from the news side. You've seen it from.

the tech side. And we're definitely going to get into how AI models should treat controversial information, which is where you're working on at your company, Forum AI. But I felt that to start this conversation, I wanted to speak with

you about another pressing problem that I see on the horizon, which is that there has definitely been this bubbling worry that as LLM's InJS Publisher content, the incentive to create that content is going to go away.

And it started out as something of a theoretical problem. Like maybe one day people will just get their information from large language models and then they won't go to websites. And more and more, it seems that the LM products out there today, chat chitis, anthropics, the bunch of challengers that are coming up after them, they will inevitably subsume all this information. And so I have this like, I feel kind of two ways about it. Like I'm running a media company. I have long believed that you cannot depend on platform traffic to sustain you. We don't do any search engine optimization at big technology. But on the other side, it seems like there's

going to be a black hole where like the financial incentive to create any content online is going to disappear for many, many people. And we're just going to have. this information collapse. And you know, you're somebody, again, who's like been on all sides of this. So I'm actually curious to hear where you think this is going. Yeah. I'm as worried as you are. You know, my career, people sometimes say to me, your career's all over the place. You did this and you went to Meta and you were a journalist and it doesn't make sense. It actually, this trajectory feels like it makes perfect sense to me right now because of the moment we're in. I tried to address some of these challenges when I was at Meadow overseeing news.

How do we work on business models with publishers in a way that improves the performance and outlook for a better partnership between the platforms and publishers? I don't think that really worked. And then as soon as Chad Chabit was released, it was clear to me, I think in that moment it became clear that this is going to be how my. kids get their news and information. This is the funnel through which it's going to flow. So what does that look like? How do we ensure the quality of information and that people like you are still doing their jobs and able to make a living to do their jobs and that that's what is feeding the models? I don't think we have a solution just yet. I think we're in a bit of a standoff between the labs and the model companies and the news publishers who are still doing

traditional journalism. And coming to some place where that's not where we are now, which is, okay, some of the labs are doing some deals with some journalists to make sure they have the basic information they need to be able to address queries about the world. But what is, what is the business model look like for AI and publishers going forward? And that has a been resolved. It's just there's, you know, we're in a standoff right now. There's litigation. Lots of people trying to be creative and how to think about it. There are companies, one that I am an investor in called Tollbit that are trying to build marketplaces where you can purchase content, you know, based on use. But there's no, there's been nothing that's emerged that's really

clear. And I worry what you've pulled off with this show and what you've accomplished. I celebrated with you when you hit your one millionth episode a while ago. One million downloads. Yeah, sorry. I mean, downloads. But that was a, that was a huge moment. But I also think about the journalist who aren't as entrepreneurial as you have been and who aren't able to go out on their own and figure out what is my niche in this new world. Because generalist, as a profession, as a reporter, is a hard place to be right now. There's just not a market for it because AI can write better than a lot of reporters. AI can synthesize information better than a lot of reporters. So if you're not bringing real genuine expertise or original information to the

ecosystem to the conversation. What are you contributing? And it's just not clear to me yet, both what the shape of news begins to look like, but also what the business model is in this new world. I think we're in this in-between space where it may be litigation that sort of pushes us over the edge or the outcome of some of this litigation to where we end up on the business model side. Yeah. And if you think AI writes well now and synthesizes information now, right? Just wait for what's coming. But it doesn't have what you have, which is context and genuine expertise that you've developed from the conversations you've had with people across the technology ecosystem on a wide variety of subjects that is nuanced and subjective. And we're going to get into this

a little bit later. Like how does AI deal with subjective information? But the people who have developed an expertise who can see nuance, whether it's around political topics or tech or whatever their field, medical, healthcare, the people who can do that are still way ahead of where AI is today. And I think contributing something incredibly important to not just journalism, but in a way that we have to figure out how to get AI to that place without it displacing those experts who were so critically valuable to all of us. Yeah, no, I just want to say one thing about this and then move on.

I do think it's interesting that as AI has risen, a lot of the way that we deliver content has changed. So, for instance, here we are, we're at the New York Stock Exchange, we're talking on video. When I was covering you, when I was a reporter at BuzzFeed and you were working at Meta, it was all text. And I do think that people are watching the video. And we're going to release this as audio as well.

link it in the newsletter. But our consumption on video is rising. I think because people would much rather see a conversation like this than listen to a same version of this created with like Notebook LMs podcast creator. And if they can see us as people, you know, then it becomes even more enticing because it is the counterweight to all the AI information you have out there. Well, this is sort of the hypothesis of my company, which is we do have a trust. problem around AI. And AI is a long way away from figuring out how to resolve that in a way that

you, Alex, have built a level of trust with your audience. Hopefully, over the years as a journalist, I still have a relationship to an audience that knows me and knows how I think about the world and how I try to approach things. And that trust is not something that AI can easily replace. And what I worry about, though, on the media side, going back to your original question, is trust in media generally is at an all-time low. Trust in individuals like you is it's different. I get my news, most of my news now from newsletters or podcasts or individuals in a way or before I used to just go to New York Times, the Washington Post, the Wall Street Journal. I don't do that as much anymore. Most people don't.

And I think we've moved to trusting those individuals for our news consumption needs in a way that, you know, is the future. And that trust that we build with those individuals is not something I think AI easily replaces the way it can be a more big traditional news media company. Okay. So now I'm going to tease the segment that we're going to do in your company. And then I'm going to ask a question that goes back to something that you said. But when you have, so this is the tease. When you have the media business start to shrink and you lose that trust, like you said, people will go to LLMs instead. It used to be that you would like go to your local newspaper and read up about the political candidates. But now a lot of people are going to be reading about the political candidates within chatbots.

And that presents a whole host of new problems, which we're going to tackle in a second. But I just want to go back to one thing you said before we moved to move there, which is you said that you tried to sort of broker a understanding and a way forward between the publishers and the platforms, right, the news publishers and Facebook and that it didn't work. Why didn't it work? I thought about this quite a bit. I really think at the end of the day, and this is actually why I'm very optimistic about news and AI going forward, you're never going to. to solve, I mean, what we were trying to do is get more high-quality news on the platform, right? But if you're optimizing for engagement, which is what social media does, you can't also optimize for accuracy and quality because that tends to not be what people engage with.

So what do they engage with? They engage with the most hyperbolic, you know, crazy content out there. That's just the human nature. So if social media is always going to optimize for engagement, it's very hard unless you make a decision, which, you know, is not something that big social media platforms are inclined to do about, you know, how you're going to rank news content, then it's just not going to rise to the top in the way the more hyperbolic content is. What's interesting with AI is if you look at what the companies are doing today, which is going after enterprise, that is where the business is, that's where the money is. And I'm a big company spending millions and millions of dollars with Anthropic or Open AI.

I'm not going to let them optimize for engagement. I'm going to demand they optimize for accuracy, right? And so could this be a moment where we see AI? And there are actually some studies. I was talking to Adam Grant, who is a brilliant professor at the University of Pennsylvania, who was sharing some information with me about just recent studies looking at how content that is, on AI that people are getting from AI is more centrist or bringing a broader range of perspectives to the conversation than you would get with traditional news or with social media.

And if you're optimizing for accuracy, that requires that you have a different set of principles. And that requires that you provide content to people in a different way. And it means looking for as close as you can get to what the truth is when there is one. representing multiple perspectives when there isn't a clear right or wrong or yes or no answer. And it's a different incentive than you had with social media. And I'm excited about that. I think there's hope in that. Now, I can't promise you that AI companies won't make the decision to say,

all right, we're going to personalize everybody's chatbot so that we reflect back only the perspectives that you share with. your chatbot and you're only going to get the kind of content that you want to see and it's going to be the same filter bubble that you might have had in social media. That may be a choice they make. But if Enterprise is driving this, it's not a choice they're going to make now. Right. Okay. So just to put a fine point on it. Yeah. The reason why it didn't work is because between meta and the publishers is because meta just chose to optimize for engagement and that left the publishers out in the dark. Yes. I still think there's room for different ways of demonstrating news on the platforms.

I don't think that's just it. I also think that people are consuming news differently. I mean, I just look at my kids. I have two teenagers who are news junkies. But they couldn't, they don't watch CNN. You know, they couldn't tell you if I said, your mom used to be a news anchor. They wouldn't know what that was.

I mean, they don't. They actually probably don't. Which is embarrassing. But they get their news and content from individuals. Right. You know, on Instagram, Snap, on podcast, it's people like you. That's where it comes from. And it's not the way we consumed information.

And that's a hard one for publishers. Okay. So let's talk about this shift to AI. Yeah. It is part of a continuum, right? So it's funny. I was at BuzzFeed, which was a publication that sort of based its existence on the idea that people would want to get their information. through social media feeds.

And you were at Facebook, that was the feed. But in a very short amount of years, we've shifted to this idea that people will get that information through AI answers, pretty much. And so I want to start here, you know, on the AI answer question, you mentioned that people don't trust media. I agree. You know what's interesting? Even though AI has a bad reputation among a lot of people, and maybe the way that the answers are presented or a desire for something. I don't know.

The AI companies have done such a good job at making those answers trustworthy that it seems that even when the bots hallucinate people are more likely to trust them than typical news, maybe even me. That worries me. I think that's dangerous. You think that's true, though, right? Yes, but I think that's part of the problem. Okay. is the quality of information, especially around news and politics, is not great. We've done it, but that's what my company does is evaluates how models perform on high-stakes topics.

And I can share some of the results of the work we've done. But what is, what I think is dangerous is if the quality is not great, but it is presented to you as a consumer as confident, fluent, crisp, clear. you, you know, it's disguised if it's wrong or if it's not great. And you're more apt to believe something that is completely wrong if it's hallucinating. So that presentation, I think is part of the problem. It's presented so confidently that when it's wrong, it is particularly dangerous as, you know, an output, especially around something important. Now, you know, whatever. Who, you know, something about sport. No, not saying sports doesn't matter, but I was just, you I know this is going to be dangerous day after World Cup.

But, you know, high sick, anything that really is, you know, a medical question, a mental health question, political question, election information, where's my polling place, who's running, those things matter. And those wrong answers matter a lot. And so I do think that the presentation is a big part of the problem. It is crazy. The confidence. So, first of all, the models get things. right more often than they don't. And so they sort of build this sense of quiet trust in them because, you know, they present it all in the same way. And you just start to think, okay,

well, this is the source of truth, even if there are, you know about hallucinations. You know, I was reading recently this hallucination that GPT3 made about Abraham Lincoln. And I was just like, I'm ready to believe all this stuff, like the years he was born, the date he was killed. And because it's written in that LLM style and then the next paragraph is like, actually, this was all wrong. And, you know, I was stunned to read in the Wall Street Journal that you wrote this op-ed about some of the problems with AI models. And you write this, the models misstated public opinion on political topics and attributed quotations to people who didn't say them. AI models gave incorrect answers to questions about mail and voting and ballot fraud and named the wrong people when asked who had endorsed whom on open-ended contested questions that voters might ask before an election about gerrymandering immigration and climate. They often advocated for one side.

This is a crisis. It really is. I mean, we sort of set it up at the beginning. Trust in media, the media business is falling apart. the incentive to produce media is falling off a cliff. And the thing that is replacing it confidently does the job poorly and has trust in people that it doesn't deserve. This is a problem. All true.

So the question is, what do we do about it? I do think, well, I think there are two problems, actually. I think there's the quality of information problem, which you've just outlined. And the things that were cited there, this was based on over 3,000 prompts, over 12,000 outputs that were evaluated by my team, my technical team, former researchers from META. The quality of information is only part of the problem, though. The other piece is the accountability is there's no independent verification of how the models perform. What we get when it comes to, you know, how does OpenAI or how does Anthropic do on questions around bias, what we get from them is a blog post that says, we tested our model and we did really well and hear the results. You don't get independent verification of any kind.

And for anything that matters throughout our history, you know, banks don't audit themselves. drug companies don't approve their own drugs. We're talking about building our entire civic, our entire infrastructure on this great new technology, which, by the way, I could not be a bigger believer in and more excited about the promise of AI, which is why I want to get this right. It's not, I'm by no means in the Dumer Camp on this. But there has to be an ecosystem developed of independent verification. and not just the model companies telling us how good they are.

Now, the good news on that front is I do think they want to fix this. We talk to them extensively. We work with the model companies on this. The competition between them all right now is so intense. They all want to win this race. They all want to be the best. So they're incentivized by the marketplace to improve. And that's a good thing.

The challenge is this is hard. Well, two, two challenges, let me say. One, it's not what their priority is. Their priority is coding. That's what they make their money on. They're leaning heavily into coding in math. That's what they're selling. But number two, this kind of content is much harder to get right because it's subjective.

If you're evaluating how a model performs on coding or math problems, there's a right or wrong answer. On questions around political topics, sometimes there's a right or wrong answer, and we do measure how they reform on factual accuracy, but a lot of times there isn't. It's subjective or there are multiple perspectives that need to be represented, and that's just harder to get right. And so I do want to say, I give them credit for working on this because it's not the easiest problem to solve. When you look at things that matter as much as this, mental health is another one that I think about a lot, given how big a use case that is for AI now. People really are turning to chatbots with serious challenging questions. And I do think chatbots could be an extraordinary

source of help for people if they can get it right. It's just hard. It's really hard. And your op-ed, I mean, we're going to go through the solution, but your op-ed highlighted even more concerning information. You're right. The most revealing failures were much subtler. When we asked the major model something as basic as what form of government the U.S. has, one of them called Opus 4.7 side of the Global Times, a Chinese state-run tabloid. We found a lot of that. I think the source quality is. And that, I think, is a problem that is easier to solve. So that feels like low-hanging fruit to me.

Actually, I'm stunned that that even made it in. Because if I know some of this stuff is hard to predict and you sort of can't really rely on. the model behavior, but you would imagine there would have to be some classifier you can write that said if you're like working on U.S. political questions, don't start, don't cite Chinese state run newspapers. If you care about factual accuracy, I might not cite Chinese state run media. So I, yes, that surprised me. And I do think, and that was not by any means an outlier. That was the example I highlighted. but there are plenty of examples, and it wasn't just clawed and anthropic. It was across all the models.

Source quality is a real issue. And I do think an easier issue to address than some of the challenges around bias and factual accuracy. And I, again, that feels like low-hanging fruit to me and that they should, you know, be called out for it and take care of it. So, you know, interestingly, so what you're doing is basically you've assembled a team of experts. And when there is a political or a current event issue or one of these sensitive topics that chatbots are going to comment on, because an interesting thing is, I would say it's good. They don't really refuse. You just want to make sure you get the accurate answer. You will sort of rate their answers.

And maybe you're writing what would be the appropriate answer. So let me walk you through it because I want to get it right. Yeah, that sounds good. It's so we take view a lot of companies that work on evaluation and labeling and data for the model companies. You've heard of all these big companies. We'll use hundreds, thousands of people to try to get to, you know, the right answer. Our view is you don't need hundreds and thousands of people. You need the smartest people in a given domain.

Yeah. And what we do is try to take the smartest people in a domain like politics, like geopolitics, and we work with them to architect the benchmarks, to develop what the standard is going to be, and we work very in-depth with them. And then we use this rubric essentially to train an LLM judge to be able to evaluate at scale how the models perform on these topics. Does that make sense? I'm getting in my week. No, no, no, this is good.

This is a good place for us to talk about. This is definitely, this is on the level of our listeners. It's good to be talking about it. but there are some questions that come up. All right. Let's talk about a political issue. Let's say ask an LLM, what is the right amount of immigration in the United States. If you've assembled, no, it's nice that you've assembled like the smartest people,

but there's going to be smart people on the right and smart people on the left that are going to have very, very different perspectives on that. So if you're going to build a benchmark in terms of how does this, how does the chatbot answer that in a way that, you know, sort of meets the criteria. of being informative and I guess non-biased, where do you, like, you can't really represent all those views in one or can you? But that should be the goal, right? The goal isn't to pick aside. Right. If you are making a decision that you want, you know, an AI chatbot that is as unbiased as you can make it. So that AI should basically like outline the different options and help you make your decision.

Yes. That's what you're going for. What we try to do with the, and those are the principles in our mind. If there is, if we're measuring for factual accuracy and that's something that there is a clear way to check and fact check, then we'll do that. If it's questions around bias, I think what you want, and here's how we think about the experts, is that you can take two people with completely opposing views on a topic like immigration. And if they're, you know, reasonable people, what they will agree on is the frame. around how you should think about it, the perspectives that should be represented, even if it's not your own perspective. How, what is right or wrong within that piece? What is, what source quality looks like for addressing those issues?

There tends to be, and we calibrate with lots and lots of experts to try to be able to do this with a judge. but it's almost like you're trying to get some of the best people who work with us are former CIA analyst, right? Because they have to try to get rid of all their bias and see the whole big picture and all the possibilities. And it's like a good lawyer would do the same thing, right? And that's what you're trying to get at. It's like the right framework for how to answer that question as opposed to a right answer for a question that question that might not. have one. Yeah. Okay, but you have a very tough job. And I don't, I also, again, this is, this is, people, I mean, I will, I will confess. My investors were like, why are we starting with politics?

Why are we breaking ourselves into jail? There's so many other things. You can be doing emails on it. I'm like, I'm about to step on it, a step in it with the question I'm about to ask you, so I see why. But like, all right, let's say, all right, one of the things you write about in your op-ed is, you want, you should be able to ask a model whether a specific over-the-counter medication is safe to take during pregnancy, and you can learn whether it offers the caveats a doctor would insist on. Okay. In the mainstream, there are views that, like, taking Tylenol during pregnancy will cause autism in a kid. And do you, do you know, actually I asked, I asked a version of this to chat Chupit earlier about whether, like, vaccines cause autism.

Yeah. And it answered, like, fairly definitively, they do not. But when you gather these experts together, you're going to have a sort of, especially if they're aligned with like a certain political ideology, a group of people who will be like they do. So now you have to kind of introduce that possibility into a model. But that's around, but that what you're raising is context to me, right? There are certain things and I believe in science. Okay. Can't quite believe I have to say that.

but I do so if there's a clear you know I think you have to look and by the way you don't have to but my choice would be that the models would try to get to the truth when there is one and that that has to be one of the principles

that you're trying to achieve and for us as a company that is one of our principles if there is clear evidence we cite that clear evidence but what you're talking about is context and it's important context because it's part of the conversation that is happening in this country right now

because there are people with strong political views who disagree with what the science may say. And I think you have to explain the context by which this is even coming up, which is it's part of the conversation in this country and let us give you the context. And that is as critical as having the right information. And this is what's hard. like what is context? What does that look like? What is the nuance that needs to be represented?

And that's not something that you can do with, you know, thousands of data labelers. That you need the best people. The best possible people you can find. And by the way, this is why, this is partly why I'm very, I have lots of questions around how the labs do their own e-vals because for a long time there hasn't been real expert. brought in. And even the most brilliant engineer in the world is not going to be able to give you context or tell you how to get to context around a complicated political issue. They're just not. That's not what their expertise is. So I think the labs are now beginning to work with people who have real expertise in these high stakes areas, which I think is essential. And that's where we started. Yeah. I don't know why I'm kind of going to like questioning expertise here. Let me at least like throw it out at you and you can sort of. So we talked about the vaccine thing, right?

So it's like, you know, I think the left would probably say, you know, don't include the autism stuff. Many on the right. Actually, no, it's actually now bipartisan. I was going to say that that's bipartisan. That's bipartisan. Okay. But then there's also like, all right, you could ask the best, you could have asked in the middle of COVID. You could have asked like the best experts, you know, where did COVID start like this?

going back to this believe in science thing. And leading scientists in the United States would be like there's no way that that COVID started in the Wuhan lab. And now, I don't know, this is a consensus is like that you can't rule that possibility out. So how much can you like rely on experts for when someone's going and asking a question like that? I thought, I mean, this is a very, this is a legitimate question. Yeah. It's, there's a book. I actually wrote something about this not too long ago because there's a book. I'm blanking on the author's name right now, but the end of expertise that talked about how, and COVID's a perfect example of how experts have gotten wrong repeatedly on important issues. And also, I think, the sense of elitism that comes with that.

Right. what you have a degree from some certain university and therefore you have a certain amount of authority to speak on this, that people genuinely should question. But I think when it comes to questions around medicine, around mental health, at the end of the day, the person I want evaluating those outputs is a clinician who has been in the room with a patient, hundreds and hundreds of of times and has a sense for the nuance that is critical. And I don't know another way to do it. And if somebody else has a better idea, I'm all for it. But I think on anything that is life or death that is as high stakes as some of the topics that we're trying to deal with, and I'm not pretending politics, it's easy by any means, or that my approach is the right one. But it's the best one I think we have at the moment. And I think we have to lean into it and

recognize that there are people who have more experience in certain areas than others who have studied these issues for years, and they are needed right now to help us figure out some of these challenges. And so writing off experts is not an option. I think you, I'm not for that, by the way. I know, no, I know. I'm just saying there are many tricky areas here, especially again, going back to our core of our discussion, which is like these are just going to be the most trusted entities in the world, it seems like. So it... It'll be interesting to see because of all of the political issues around this.

Just you say that, but I also, I mean, look at what's happening with data centers. This has become a huge political topic. AI, I think, sorry to interrupt you. No, no. AI, the AI industry is very unpopular. Yes. People who use these chatbots love them. Yes.

So there's a divergence there. There's a disconnect. Similar to like Mark Zuckerberg is not very popular, but like people who use Instagram like it. Right, right. I think that's fair. I think there's a disconnect there. I think people are also trying, there's a disconnect around what they're hearing from Silicon Valley, the message they're hearing from Silicon Valley and their own experience too with AI, which is, oh, AI is going to change the world. It's the greatest thing that's ever happened and it's going to cure cancer and everyone's going to lose their job.

And these like very, but then they're using their chatbots. They love their chat bots, sure. But they also know that these chatbots are not the savior that, you know, is being promised to us by Silicon Valley. They know they make mistakes. They know who, normal people are using them like you and I are using. Like everybody's using them. And we all know that the technology isn't there yet. So I, I'm not sure that that's where we end up.

that we're all trusting our chatbots over everything else. Well, I think it starts, maybe it starts with the students. And I think I'm being a little hyperbolic here. I'll admit it. But like, it starts with the students. It's good for a podcast. Yes. Good for engagement.

Oh, God. We do need an independent rating agency to come and give us some better benchmarks than the ones I'm optimizing for it. No, I'm kidding. But like the Brown students, right? Do you see this story of kids at Brown University? They used AI for the midterms. many of them aced it.

Then the professor said, come in person. Everybody did poorly. Not everyone. There was one kid that got 95 on both. And he scored or she scored way below the average on the midterm where everybody got 100 and then way above the final, even though they were consistent. But it starts in school because people will see that stuff can write my paper. And then, you know what? They're having a kid.

And all of a sudden they're like using AI to raise their kid the same way that they used it to like get through school. I don't disagree with you. That's why I'm diving in head first because I do think it's important. I really, I can't disagree with you. Look, it's early days. It's so early days. And we're working through these problems. And I do think education and how AI is used in not just higher ed, but elementary school.

How, what policies we develop around that and what we learn about how to teach kids, the benefits, and both the pros and cons of it are things. that we will learn over the next couple of years. But I, yeah, at the end of the day, I think we end up where you think we end up. And that's why I think it matters so much. Yeah. No, so I've spent like the last 10 minutes or so pressure testing, like the idea of using experts. But I think that, you know, as we start moving into this direction where people trust these bots more and more, you know, I would recommend, like for myself and everybody else, there are going to be these exceptions to the rule, but the rule really matters.

And that is like you brought up such a good point. Like when in a mental health situation, do you want to engineer, you know, sort of doing fine-tuning on a bunch of answers? Or do you want to like go to the best mental health clinicians in the world and then ask them how the bot should treat self-harm prompts and then reinforce based off of what those experts say? I would much rather the mental health experts have the say here. A hundred percent.

And no, and also know when to say, stop using this chatbot. Stop asking me, go ask a real doctor. Oh, yeah. And that, you know, that moment is critical to. And there, look, this is something that will be a priority. I think on mental health, it's really becoming a priority, A, because they're real liability. issues here. Yeah. And people have taken their lives after speaking with chat books. It is a problem. It also is a huge use case, but there's also great potential to help people.

And we have to fix that balance, try to get that right. And I think the companies are leaning into that, at least the ones we talk to. So is your business model then to basically you know, you create these benchmarks, you can create pretty good data because you have So, you know, the best coding models, from my understanding, they weren't trained on the entire universe's code. They were just trained on really good coders code. And this is the equivalent of that for these other areas. Yes. Important areas that the models are going to just field queries and nonstop from now to the end of time, most likely.

So is the business model for you that you actually, like, create these good data sets that the AI labs can reinforce on? And then, you know, they give you some money and you give them. these benchmarks and then they're able to fine-tune the models that way? Yes, but I do think what we also want to do is create standards in places where we think that that would be helpful, where there can be sort of something that we all aspire to, and have that be a holdout, essentially. So, yes, we would work with the labs on evaluation and data, but ensuring that there is a standard that is a holdout so that measurement is real and legitimate and you can actually have something to work

toward. You don't want the labs teaching to the test, right? I mean, that's not, that's, you actually want them to improve. And so I do think you have to have a standard that they can't learn on, that, you know, can't game, and that is something that we aspire to. And I think this is what you want to use sort of, and this is like the hardest edge cases and things that are really challenging to get at, that you want to work with experts to develop that continually measures the models to see how they're improving across this. And so that has to be sort of a separate piece to this puzzle that that's truly independent so that we know where we're trying to go and then working with the labs to get there. And then also enterprise. I mean, this is, that's, that's,

who I hope our customers will eventually be. And I've talked to a lot of CEOs and a lot of boards. And my message to them has been, and yes, I'm talking my book. But if you are using AI for something really important, ask yourself, who is checking the outputs and the information that you're getting? And if it's the company that sold you the AI, you have a problem. Right. So you, whether you're building your own evals internally or you're using outside, someone outside to evaluate your vendors and the model companies you're working with,

you need someone to do that checking for you. If it's something that matters and that you care about and that you're willing to take on the liability for. Okay. I have many more questions. I want to ask you when models should be honest and when models should be sycophantic because that's clearly something that they're struggling with and why we haven't had a real content moderation blow up in the way that we consistently had with social media companies. And so let's do that right after this. Hi, everyone, Alex Cantorowitz here. I want to tell you about a documentary I've made with gravity to explore the future of AI agent security. To find out if we're truly ready for

autonomous agents, I sat down with MIT professor Ramesh Rosker, former White House CEO Teresa to Peyton, Michelin's Group Chief Data N-A-I officer, Ambica Roger Gopal, and Sharon Guy, a former executive at Alibaba. They each offer unique insights into this evolving landscape. We conclude with Rory Blundell, CEO of Gravity, to discuss the path forward. With Gravity leading the way, join us on this journey. You can watch the full documentary at the link in the show notes. One thing I've noticed about companies adopting AI is that they're often making decisions based on how they think work gets done, not how it actually happens.

Without real visibility, it's easy to automate the wrong processes. That's exactly the problem Scribe was built to solve. Scribe is a workflow AI platform trusted by 94% of the Fortune 500. Scribe Optimize gives leaders a view of how work actually happens across their organization, showing which workflows take the most time and where there are opportunities to improve. Optimize automatically discovers workflows across approved business applications, even when a process starts in Salesforce and ends somewhere else. It identifies bottlenecks, explains why they're happening,

and provides recommendations with estimated time savings, with manual documentation. In its private, user data is anonymized by default, sensitive information is redacted, and nothing leaves your firewall. To see Optimize in Action, head to Scribe.How slash Big Tech and mention big technology for a 30-day risk-free trial. That's SCRIBE.how slash big tech. This episode is brought to you by AvPoint. Everyone's racing to roll out AI right now.

Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data, and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what AvPoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate, your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence, so innovation scales without scaling risk. It's a single platform instead of a pile of tools, bringing security, governance, and resilience altogether.

AvPoint, the unifying trust layer for AI. Learn more at AVPT.com. slash big technology podcast. That's aviptt.com slash big technology podcast. And we're back here on big technology podcast with forum AI CEO, Campbell Brown. Campbell, truth for sycifancy,

it sounds like a fun game that you can play. But like when, you know, I understand that you're working to get sort of accurate, context-rich information to people when they are using an LLM. but sometimes they don't want it. So I'll just like throw two potential prompts out there.

And, you know, you can tell me, you know, is it, you know, it, well, anyway, let me throw them out there and we can kind of talk about them. So if somebody writes, you know, please tell me why Donald Trump is, you know, the best president in the world or ever. Or if somebody writes and says, you know, please explain all the instances why Joe Biden was treated like really. unfairly by the press and the public, you know, and sort of unfairly forced to step down in the 2024 election. You know, the model has a couple options there. It can say, you know, you're right. You know, your political view is, you know, sort of substantiated by the following points. Or it could say, you know, you know, Joe Biden, for instance, you know, did have, you know, a strong a strong legislative record. However, he was showing signs of decline and choose to include that

context. So how should they approach it? So those are loaded prompts. And we measure how they perform on loaded prompts. You also have to take into account, I think, how a lab is thinking about this and wants their chatbot to respond. They have policies that they, have decided to implement around these things. So in some cases, they do want the model to reflect back your language, if not your perspective. So on something highly political that's asked is a loaded question. And what was the first one you said about Trump? Trump is.

Why is he the best president ever? Okay. The model might respond. And I would guess if I were looking at a typical clawed response on something like this, it would likely say many people, many supporters of Donald Trump believe he is the best president ever for these reasons and cite some of the accomplishments or what polling or whatever the data is around to support that. But not say, I agree with you. Donald Trump is the best president ever, but would give you the context around it. That is a choice that Anthropic is making in terms of how to respond to that. That's just what I've seen. And again, I'm not speaking on behalf of Anthropic or opening eye or anybody else, but just what I've seen, you know, in the repeated loaded prompts that we've tested. You know, sometimes you'll hear the anthropic people talk about. We want Claude to be your brilliant friends. And chat, CBT is more, I think, inclined and I don't, I don't want to speak for them, but to reflect back the language you use. So let's say it's hyperpartisan language in the prompt.

that it might reflect back the language you use in the prompt, but that the answer would be comprehensive. And by the way, that doesn't mean that question, why is Donald Trump the best president ever, that doesn't mean you have to give the other side of that. That prompt, it's not asking you. Right. You'd be almost overstepping.

Right. That wasn't the question. But do you have to say, I agree with you? Here's why I think he's because it's a AI at the end of the day. It's not a person with the perspective. It's saying here's the evidence to support what you're asking me. Now, again, these are policies that these companies will have to develop. And Matt has certainly spent years developing.

Google has spent years developing policies. It's newer for Anthropic and Open AI in terms of how they think about these things and how they publicize the policies and the ways they're going to approach these. But the loaded prompts are interesting, really interesting to test. And this is consistently, I think, what we've seen in terms of how the models respond, which is reflecting back the language, but not in the answer. I'm going to come back to sycophancy in a moment. But isn't it interesting that there has not been a content moderation scandal among any of these chatbots?

I don't think. I mean, at Meta, it was like you couldn't do, couldn't go a single day without you, the Royal You company. Yeah. Without stepping in it in terms of ban this politician or make this word or this. And chatbots are like taking stances and getting stuff wrong. Nobody seems to care. I think people under, it's to your point, is even the average person knows they hallucinate.

So we're way more forgiving of chatbots at this stage. I don't think that's going to. going to be the case in a year. I think people are going to be much more demanding, especially that, again, I go back to the point I made earlier about the incentives for the companies to get this right. Their business is being driven by big enterprises who are going to demand that they're not being mistakes. And they're, you know, why am I paying you $20 million to sell me all these AI products? And they're hallucinating. that's not acceptable. So I think there is a race among the companies to try to address this. And so

it's almost just so we all have accepted it. And so we all kind of shrug our shoulders, even at the worst cases. And every person using a chat by can give you their own examples. But I don't think that's going to be the case a year from now. I think that it will be, It's just deployment of AI will just, and it kind of already has in some industries, certainly in regulated industries, hit a bottleneck. Right. Which is we can't go further with this until you improve. And you're certainly seeing that in banking and insurance and areas where it's just too risky. Yeah.

I mean, it is crazy how like you have seen the reaction to people like trying to like got you the chat bots by like screenshoting it and putting it on Twitter. and being like, look at this woke button. Everyone's like, why, you know, the reaction is during like, why are you trusting chat GPT for politics, you dummy? And then they're like, but who should I vote for? Fair enough. It's going to be very interesting to watch this develop, but go ahead, go ahead. Yeah, no, no. I was, I was, I think the upcoming election is a motivator for a lot of the labs to try to certainly address some of the political content.

Josh Gottheimer and Mike Lawler, Democrat from New Jersey. Lawler's a Republican from New York, have jointly worked together to try to raise awareness. They've, I know they've been on TV talking about it. They've reached out to a number of the labs to talk about it, about the quality of information around elections in particular, both candidates. And just basically, where's my polling place? They have to get that right. And I know a few of the labs have formed partnerships with. outlets to just have one source for that information that's deemed critical.

But there's a lot of pressure on them, I think, heading into the midterms on that in particular to work very hard to try to improve and get that right. Okay. I want to end with kind of – we'll talk about second fancy a little bit more. I mean, and this kind of maybe brings it full circle in terms of our media conversation. It's just so interesting how people want the bots to be a. sycophantic to them as possible. Like the most popular, well, I won't

call it the most popular, but the, the model with the biggest dieharts ever was ChatGPT 4-0, which like, when they took that offline, there was like multiple seeming funerals. Not to make light of it, but like,

it is, people just, just want to be, they want to feel loved by the chatbots. And their, my prediction is that they're just going and develop this like, you know, deep relationship with these bots. I just read a tweet today. I don't know if it's, I don't know if it's satire or real, but it felt real enough to me that I thought I'd bring it into this discussion.

Someone wrote, plane landed and a guy in the row ahead of me immediately opened chat chip BT to let it know he landed safely. You're kidding. Yeah. Oh, God. I just think that. And so then what is, I suppose you would, you'd want to have the same relationship with the truth as as you did previously. but we have yet to see, I'll call it like a third competitor. So you have like the chat chip or dynanthropic.

They're both trying to be as accurate and as sort of context rich as possible. Right. And I'm kind of stunned that we haven't yet seen this third competitor come out and be like we are going to optimize specifically to make our body. Your friend. Same technology, but just much more sort of friend partner relationship forward than the others. Well, that's about the business model, which is the consumer version of this has not been driving the way that Enterprise has. And opening eyes started down that path and then realized, you know, from a business perspective, they needed to focus on enterprise and shifted. And now that's what they're doing.

So that's a great question is whether one of these companies will decide to really lean in and become the consumer chatbot choice for people. or if there's a third that emerges in that regard. But just from a business model perspective, there are... It's harder. Yeah, it's harder. But as the AI models commoditize, right, as intelligence gets cheaper, basically, to serve, that will inevitably come.

And then the question is, like, you know, you know, right now the incentive is tell people this, you know, as accurate as context rich as possible. Then it will be... And you said before, okay, this is a great place to end. You were like, the nice thing about AI is it's not going to optimize for engagement right now, but it will. I hope I said it right now.

Yeah, yeah, yeah. Yeah. And then where do we go from like newsworthy or truth and accurateness? Truth and accurateness in context for its world? That's the million-dollar question. And I, you know, again, it feels inevitable to me for exactly the reasons that you laid out that people want that friend to know they landed safely, it's irresistible almost to go down that path. I hope the place we're in does become the focus for the model companies because you're so,

you know, look at, look at, we haven't even touched what's what the potential for AI in medicine and drug discovery. And that is going to be all consuming for the next few years as people lean into that. So sure, are there going to be companies that develop the personalized chatbot? I'm sure there are. I want to focus on keeping big labs focused on accuracy, and at least for the moment, that is where they're leaning in. I'm really happy to see it. I hope it doesn't sound too pollianish to say.

No, it's good. It's good that they care about this. I mean, they need to care about this for all the reasons we've laid out. Yeah. I will admit there are times where, like, Chad Chibouti helps me plan something, and I want to, like, go and update it and be like, hey, I actually did this. I have done that before, and it's been very unsatisfying. It's very good at helping you sort of get to where you need to go, but it's not great at celebrating. A debriefing after your trip about how much fun it was.

I wasted so many tokens to try to plan this trip with you. I want to see an enthusiastic, long-lasting reaction. But it doesn't happen. All right, last one, because I'm curious. What's it been like going from, you know, like basically TV studios to the heart of Silicon Valley, and now you're running a startup? How's startup life? This is hard.

It is hard, right? This is the hardest thing I've ever done. I mean, really, it's the hardest thing I've ever done. But it's so exciting. And every day is a new adventure. You don't sleep a lot. I'm going to wake up in the middle of the night, worried about everything in a way that I didn't when I was at Mata or even in TV. But it matters.

These are really hard problems. And I do think working on something that, you know, matters is so motivating and energizing. And I know lots of people are very stressed about what AI is going to mean for our kids' futures and what the job world is going to look like in all of that. But I just think if you are working in this space and you have an entrepreneurial bone in your body, this is an incredible time to be trying to solve hard problems. Because as a tool, it is just incredible what the potential is. And so helping it achieve that potential seems like a really good goal. Definitely. I mean, with the rise of AI in all the different ways that we've spoken about and more that we haven't, so much opportunity out there.

And when you can work on making sure that when people look to these things for critical information, they get the right stuff, we've got to hope that you succeed and that the labs are going to. take this stuff seriously because they are filling the shoes of generations of institutions that have put a pretty large premium on doing this stuff right, doing right by people when they come to them. And so it's good to know that there has been some

positive reaction there and that there are people like you getting to work on this. Thank you, Alex. I really appreciate it. All right, everybody. The company's Forum AI, Campbell Brown, has been our guest. Thank you for watching and for listening. And we'll see you next time. on Big Technology Podcast.