Big Technology Podcast - AI Predictions for 2025: Geopolitics, Agents, and Data Scaling — With Alexandr Wang

Episode Date: December 11, 2024

Alexandr Wang is the CEO and co-founder of Scale AI. He joins Big Technology Podcast to share his predictions for AI in 2025, including insights about emerging geopolitical drama in the AI field, AI agents for consumers, why data may matter more than computing power, and how militaries worldwide are preparing to deploy AI in warfare. We also cover quantum computing and why Wang believes we're approaching the current limits of what massive GPU clusters can achieve. Hit play for a mind-expanding conversation about where artificial intelligence is headed and how it will transform our world in the coming year. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology Questions? Feedback? Write to: [e-mail]

Scale AI founder and CEO, Alexander Wang, joins us to predict where AI is heading in 2025, looking at everything from geopolitics to AI agents. That's coming up right after this. Welcome to Big Technology Podcast, a show for cool, at a nuanced conversation of the tech world and beyond. So thrilled about the show that we're bringing to you today because we have Alexander Wang here. He's the founder and CEO of Scale AI, that company. It's worth $14 billion that raised a billion dollars this year. It creates data that powers LLMs from Open AI meta and other big companies.

And it also provides technical solutions to businesses and the U.S. government, which helps them build and deploy AI. So Alex is working with all the big companies really there in the heart of what they're doing and including, you know, not just companies, but the U.S. government. And we're definitely going to touch on that. So Alex, great to have you here. Thanks so much for coming on the show. Thanks for having me. Super excited to be chatting today. Yes.

And we're going to get into plenty of. your predictions. And I just want to kick off the one that I find the most interesting, which is that you see some geopolitical shifts coming up in the next year in the world of AI. Why don't you lead with that one? So I think one of the big questions of AI has, for the past decade, has always been the U.S. versus China arms race. And I think the question that's often asked is which of the U.S. or China is going to come out ahead on AI technology. And certainly it's been a pretty tight race at various points over the past decade, as we look at technology to technology, with autonomous vehicles, it was very close, and then now with military use cases of AI, it was very close,

and then now with generative AI and large language models, it's once again quite close. I do expect that the new admin will come in and help accelerate things to enable the US to compete more aggressively with China and ultimately come out ahead on the technology. But my prediction really is that we're going to be talking a lot more about not only which of the two super power wins, but which one has AI systems that are going to be adaptable, sorry, adoptable and exportable worldwide. So which country is going to have the AI technology that becomes sort of the infrastructure and the foundation of the world's AI systems? And, you know, there's a lot of countries that are kind of caught in the middle. Most of the globe is sort of caught in the middle between U.S. and China, and there's always these questions where I think both the U.S. and China ask them, hey, you have to pick aside when it comes to which technology you're going to rely on. And so, you know, we like to call these geopolitical swing states or, you know, many, many countries which are sort of, you know, they could go either way.

They could go to Western and U.S. technologies or they could go to Chinese technologies. I think one of the best examples of this was in the past year, the Biden admin posed to the UAE, hey, which way are you going to go in terms of AI technology? You could either go into the sort of Huawei China stack or you could go into the Microsoft United States technology stack for AI, and they ultimately pick the U.S. stack. But I think this is going to be one of the under-the-line battles that really defines the course of the next few decades of geopolitics, I don't think we can really afford another Chinese expansion expansionary expedition like the Belt and Road Initiative or Huawei's technology being exported

very broadly. We need to ensure that Western AI technology is dominant globally. So basically what you're positing is that there's a series of AI models that U.S. companies like OpenAI, Google, Amazon, meta are building. And then there's a series of models that Chinese companies like Huawei are building and they're going to be in competition with each other in the globe and it's important that the US wins or the Western version wins because we also have Mistral in France.

Why is that important? There's two sides of this. I think first there's the tactical question of, okay, which one is more powerful U.S. AI versus Chinese AI. And this is very relevant for national security. I mean, I think that like if you believe that there's some potential of some kind of conflict over Taiwan or some kind of other, like, hot conflict between the U.S. and China, then we really, the United States needs to ensure that we have the best possible

AI technology to ensure that we would prevail in any kind of hot conflict, that that democracy would prevail, and that ultimately that we're able to sort of continue ensuring our way of life. But having the better, having the better chat GPT isn't going to make you victorious in a conflict over Taiwan. Certainly, it will not be the only factor, but the history of war is a history of military technology. And time and time again, you know, you see when there's new technologies and new technological paradigms that come to warfare, it has the ability to fundamentally shift the tides. You know, we saw that most recently in Ukraine with drone warfare becoming

all of a sudden the major paradigm. By the way, I think that the drone warfare in Ukraine is becoming more and more enhanced by gender of AI and more advanced autonomy. So that's definitely one thread that is continuing. Before you move on, where would you say the U.S. and China are in terms of competitiveness on AI technology and especially not even broader, but like especially about the way that they apply it in war? So if you look at just the raw technology, the U.S. is ahead, but China is is moving, is fast following. You know, and we like to break it down across three dimensions.

So, AI really boils down to three pillars, it boils down to algorithms, computational power, and data. So algorithms are the kinds that, you know, folks at OpenAI or Google or other companies build. Computational power comes down to chips and GPUs, you know, the kind that Nvidia produces out of TSMC's factories or TSMC's fabs in Taiwan. And then lastly, is data, which is maybe the least focused on of the three pillars, but certainly just as important for the performance of these AI systems.

If we're at a rack and stack versus China, we're ahead on algorithms, we're head-on computational power, thankfully due to a lot of the export controls that the Commerce Department has put in place. And then on data, it's a little bit of a jump ball. You know, the conventional wisdom is that China is actually probably going to be ahead on data in the long run, because they don't care as much about sort of personal liberties and, you know, protecting personal data in the same way that we do in the West. And so right now, the U.S. is ahead. That being said, the sort of deployment of AI-2 military, you know, it's hard to track exactly. The PLA doesn't tell us exactly what they're doing. People at Liberation's Army out of China. They don't tell us exactly

what they're up to, but I certainly am worried that they're moving faster than we are in the U.S. And this has been the sort of pre-existing precedent when it comes to China's use of AI technology for national security or military use cases. So the best example of this is in the past decade, they rolled out facial recognition technology widespread across the whole country for things like Uighur expression or global surveillance of their citizen base. And they did that incredibly quickly, much faster than any comparable technology scale up in the United States.

So my expectation is that they will actually deploy AI to their military faster than the U.S., even though the U.S. is ahead on the core technology. Okay, so that's the military point. So basically you're going to want the Western countries to be stronger than China, and AI makes a big difference there. So it's important for the AI industries to be stronger because if you're not stronger than you're, there's a liability, especially as this stuff gets put into production on the battlefield with things like drones and computer vision. guess, applied on top of satellite imagery to figure out where people are stationed in the middle of hot conflicts. And there's a more subtle point, which is, which is that it actually, not only does it matter for hot conflict, for war, et cetera, it also matters just in terms of, okay, which

technology becomes the commercially or economically speaking, the global standard. Right. And this is your second point here. Yeah, exactly. And because in the U.S., you know, we benefit. as a country from being the global standard in a number of areas. You know, we are the global standard for currency. That is something that's incredibly beneficial to our economy and to everything that we do.

You know, certainly our search, so Google and a lot of our technology companies are the global standards. So for search and for social media, many of these are the sort of like global standards. We benefit a lot from these being the global standards. standards. And I think when it comes to AI, you know, it's a very interesting technology because not only is it a sort of technological utility, but it's also a cultural technology. Ultimately, if a lot of people within on the globe are talking to AIs to, you know, understand what to think or how to feel about certain things, then ensuring that the AI substrate that gets exported around the world is one that is democratic in nature, that is sort of believes in the

ideas of sort of free speech and sort of, you know, open conversation about whatever topic is necessary, you know, that's a really powerful cultural export that we can have from the United States that will, over time, I think, fulfill a lot of America's vision of ensuring that we have, you know, freedom and liberty for all. So I think it's one of these things that is unbelievably important, even beyond the sort of hot military implications, it's one that's important just for culturally ensuring that the United States is able to export our ideals. So you're saying there's a soft power issue here as well? Yes, exactly.

I want to ask you about China's development of AI because I always hear two contradictory things about how China's progressing with AI. The first is that they have the government that's willing to put all the resources that they can into building the compute power to train and run models. and they don't care about data privacy, so they have all the data that they need, right? And then the algorithms are, you know, they're basically all published in that Google paper.

You know, you can tweak them a little bit, but basically they have the algorithms. So they should be the lead. And then you look at what's actually going on on the ground, which is that, and you correct me if I'm wrong, right now China is using a lot of American models, open source models. In fact, meta's model, the Lama model,

which is an open source model that they have developed and released. We know for a fact has been used in applications by the Chinese military. So explain this one to me. How has China been able to effectively, you know, put all these resources toward the problem, but still has to rely on American open source technology to build the things that they want to build? Well, there's probably two major things. I mean, one undeniable trend over the past, let's call it five years, has been the sort of the collapse of the Chinese startup sector.

And this is really driven by policies from the CCP to significantly, you know, they killed certain startup industries. They really hampered the entire innovation ecosystem. And you see it in the numbers, the sort of amount of capital flowing into the Chinese innovation and ecosystem has fallen off cliff pretty precipitously. So why did they do that before you move on? Why did they do that? I know they also somewhat disappeared Jack Ma.

Like they had Chinese tech icons that have sort of gone away. Was it that the tech industry was growing so large it threatened the government or what it could be the possible logic there? Yeah, I do think that was that's the sort of fundamental risk. I mean, I think that if, if the government, if the CCP has a desire to ensure that they consolidate all the power, either they have to nationalize the tech firms or they have to ensure that they stay weak. And so, and there were some other, yeah, there was some other.

So yourself in the foot stuff. Totally. And I think it's a lot of this hinges on I think they do really see the world differently from the way that we do. I think we, you know, in the West it seems totally insane. But I think in certain doctrines or in certain, with certain ideals, I think can make total sense, right? But there is a death of the Chinese innovation ecosystem. So a lot of what they have to do in AI is just catch up and copy what we've been up to, which they have been pretty successful at. So, for example, the Open AI released 01 and released the 01 preview a number of months ago.

This is such reasoning model. Yeah, this is Open AI's advanced reasoning model, which is great at sort of scientific reasoning. and mathematical reasoning and reasoning and code, et cetera. And the very first replication of that model and of that paradigm of model actually came out of China from a lab called Deepseek, the Deepseek R1 model. So they certainly are extremely good at catching up. Now there is a very real hamperner in a lot of their progress too, which is the chip export controls.

And this has been an incredible effort, I think, from the U.S. to market. of commerce and the Biden administration in general to sort of hamper the ability of the Chinese AI ecosystem to build foundation models of the similar size, scale, and magnitude as the ones we have in the U.S., because, you know, they have not been able to get access to the cutting edge invidia GPUs that we have in the States. And so, you know, whether or not you think that's good or bad policy, it has hampered the progress of Chinese AI development, which enables us to stay ahead. So let's circle back to your prediction that you talked about how U.S. and China will be

head to head, trying to get their vision for AI adopted across the globe. So that's your prediction of like what's going to happen. Who do you think is going to win there? I think that the trend right now is currently very positively in the direction of the United States or of the West, broadly speaking. We have the most powerful models. We also have, I think, the most compelling value propositions. in terms of our models are going to keep getting better.

And yes, maybe the Chinese ones catch up over time, but we are the innovation ecosystem. We are going to be the ones who innovate far ahead of the adversaries. That being said, I think that there's, you know, on the flip side, you have to look at what's the total package that the CCP or China might be able to offer. You know, in the Belt and Road initiative, it was through this sort of like total package of technology plus infrastructure buildouts, plus. debt that sort of managed to move a lot of folks over to their side. And so I think we need to watch it closely to make sure that we always have a compelling

total value proposition. I do think, you know, one sort of sub-prediction that I have, too, which is important to mention here, is that, you know, the technology is moving so quickly that I do think that 2025 will be the year where we start to see several middle. militaries around the world start utilizing AI agents in active war-fighting environments to great effect. I think you're going to start seeing this in some of the hot wars that we have going, as well as some sort of military, advanced militaries who aren't at war, start utilizing AI agents. And so I think that the temperature, so to speak, on AI deployment to military

is going to go out pretty dramatically over the course the next year. Yeah, I just wrote a post on big technology about how AI is going to be an enterprise thing for a while, right? Like companies, B2B software companies, not exactly the most exciting stuff in the tech world. Is it going to be where this stuff is adopted because it solves a problem for them where they have loads of information. They can't organize it. They can't share it. They can't act on it. And generative AI in particular is quite good at handling that. And then you think about, well, where else could this be of use if it's not going to be for regular people, right? Like, we're not, we don't have an AI phone right now, but we have like plenty of companies working in

AI software and the military is just like the perfect example of where it could apply because of all of the information and the logistics issues. Yeah, exactly. And I think that this is, you're hitting on the core point, which I think is some is, is often glossed over. I think when people think about the military and think about a war, they often think about the, the literal battlefield and the sort of actions on top of the battlefield. But, you know, 80% of the effort that goes into any warfighting effort or any military is all of the logistical coordination that goes into, you know, the manufacturing of weapons or the manufacturing of various supplies, the logistics and sort of delivery of all the supplies to a battlefield, the

decision-making process, the sort of data processing of all the information that's coming in. And so most of what happens actually looks, to your point, a lot like an enterprise, the stakes are just dramatically higher. Yes. Yeah, military today is all about logistics. It's like the firing of the guns is like the last thing that happens, but it's a logistics game. And so just to you know, drill down a little bit on one of those sub-preditions that you made, so how do AI agents help in that case? So, you know, there's probably two core areas where I think AI agents are going to have immediate value. One is in, is in, you know, kind of to reference your point on

enterprises, it's in processing huge amounts of data. Right now, most militaries already have more information coming in the door than they have the ability to process. There's terabytes and terabytes of data that come in, whether it's data from the battlefield, data from their partners and allies, data from satellite networks, data from other data collection formats, and they need to process that into insight that actually can help them make real decisions about what they should be doing differently. So the first is just sort of, this like huge problem of massive data ingest into real decision making. That's and that that sort of general problem set fits a lot of sub areas, whether it's in

logistics or intelligence or military operation planning or whatever it might be. The second area where I see it having very, very real impact is just in fundamentally coordination and optimization of complex systems. And this is really where the, I think, the logistics or the manufacturing cases are very clear, where these are incredibly complex processes with lots and lots of moving parts. And it's hard for humans to get your hands around those processes and really optimize them effectively, whereas AI systems can ingest far more information about the processes than otherwise can run simulations on their own around what are various configurations that might operate better, and they can sort of self-optimized those processes. to perform better. And then there's, I think, the sort of third area, which are more sort of speculative or sci-fi, which is the use of AI agents more actively in drone autonomy or a lot of

the autonomous missions that are being run right now. And, you know, I think this is an area of active experimentation for a lot of militaries. But I think if you start to see that happen, then you're going to, you will have more autonomous drones that are able to be more and more lethal, more and more effective. And that's going to be a cat and mouse game in and of itself, a real race. That scares the shit out of me. Are you comfortable with that? I think it's no. I think I think ultimately we're going to need to have global conversations and global coordination around to what degree we actually want a lot of this, a lot of AI

agents to be used actually on the battlefield. That being said, there are hot wars going on right now where militaries and countries are desperate and I think they'll do whatever they need to in the near term to get the, to get the leg up. Yeah, it's one of those things that I feel like once it leaves the station, it ain't coming back. And when we talk about agents, it's basically like AI applications that make decisions on their own. If we end up having that, you know, a deployed in war, it's just going to, once somebody does

it's just everyone is going to do it. It's like the opposite of mutually restored destruction with nukes, I think, where that's like, oh, like, you know, if we do this, then the world is over. Whereas with like agents deciding what to bomb, where to bomb, how to attack, as long as they don't have access to nukes, it's really tough for that to go back in the barn because if you don't use it, you're going to be destroyed. Yeah. Yeah, I think the good news is that if you take nukes as an example,

What has happened with nukes is like we've built incredibly advanced technology, technology that has the ability to frankly be world ending, but that has actually led to more peace than without it because, you know, you have this deterrent threat of the utilization of nukes. And so my hope certainly is that while AI's application into military is something that is very concerning and potentially extremely powerful, it is the sort of same overall effect, which is to ultimately deter more conflict than create it. I hope you're right, and I'm wrong. And we did have Palmer Lucky on the show a couple months ago, and he talked about countries

don't start wars that they believe they're going to lose. And so maybe that adds to that. I mean, that's certainly been the case with nuclear. All right, I want to get into your second prediction. We already have brought up AI agents, but I think we should go a little bit deeper because, you know, I think people hear about AI agents and they say, is that supposed to be something on my computer that's going to like book me travel, book me tables at restaurants, look things up for me, do my expense reports if I need them to do that, or, you know, basically agents that

act on behalf of the individual. We haven't really seen those yet. We've seen some examples of companies and militaries using these things. And the average person doesn't get a chance to touch that. But you think it's going to change. Yeah, I do think that 20, yeah, I think that 2020, 2025 is really going to be the year where we start to see some kind of very basic primordial AI agents really start working in the consumer realm and creating sort of real consumer adoption. You know, another way that I think about this is, you know, we'll see something like a chat chippy moment in 2025 for AI agents, which is, you know, you'll see a product that starts

resonating, even though to technologists it may not seem like all that or may not seem like that big of a leap relative to what we had before. And I think a lot of that is going to come from probably two main threads. First, obviously, the models continuing to improve and getting more reliable and sort of, you know, getting down that curve. And the second is really evolving in the UI and experience of what an agent does. I mean, right now we're so stuck as a, I think, tech industry. still on the sort of like chat paradigm and, you know, having everything be a chat with one of these models.

And I think that's a constrictive paradigm to enable agents to actually really start working. And to me, what it really means for an agent to start working is, you know, me as a user or consumers in general, start actually outsourcing some real workflows to the agent that they would, would have had to do otherwise. And so we'll start to just sort of like fully trust the agent to do full end-to-end workflows. Maybe it'll be something around travel. Maybe it'll be something around calendaring. Maybe it'll be something even around just like, you know, producing presentations or managing your workflow. But we'll start to really offload some of the meaningful chunks of our work to the agents. And there will be something that really starts to take off. You know,

I don't know if it's going to be one of the big labs or it'll be a new startup that comes up with it because I think so much of it will come from kind of like experimenting and the natural innovation ecosystem working out. But, you know, what we see is that the models and their capabilities are certainly strong enough to enable a pretty incredible experience. You know, there's no, there's all this talk about whether or not we're, you know, we're hitting a wall or whatnot. But the models are really, really powerful, and we should see something big here. Okay. So just walk me through, like, what that experience might look like. You don't, you know, we don't have to stick with this. Like, it doesn't have to necessarily be the use case.

But since you've imagined the idea that AI agents could end up helping us in 2025, like what are some experiences that are in the realm of feasible for someone? So let's, first let's walk through what, what's an ideal AI agent? An ideal AI agent is one that I think is observing and naturally in all the sort of like core flows of information and core flows of context that you are in digitally. So, you know, it's in all your Slack threads. It's only your email threads. It like, you know, it reads your Jira or all of your tools to understand everything that's going on in your work life. And then it helps to sort of organize all that information to start taking certain actions. And so, like, one agent that I think would be super beneficial and one that I think is in the realm of feasible is, you know, something that starts to take a hand at responding to a lot of your emails, you know, flagging when it needs you for, like, additional context or information to be able to address your emails, can sort of,

summarize a lot of your emails for you naturally. And so something that just turns the experience of doing email from, hey, I'm like having to respond piece by piece to every single email to leveling you up, to being, hey, this is like all of the overall work streams and workflows. And how do you want to engage at a high level on top of those workflows? But this is a business use case. And I'm curious if you think that like how everyday people might end up using. AI agents, or is that just still a ways off? Like, maybe not in 2025.

Everyone works, you know, so. Give me an example outside of the work context. Yeah, I think one that's more personal. I mean, I think similarly, I think in everyone's personal lives, you're also juggling and navigating a whole set of various priorities. You know, I'm planning a trip with my friends over here and I'm I need to you know get gifts for my for my family and figure out what they what they want for Christmas and then I need to I have all of these sort of personal projects which are still sort of like sitting there and so I think in the same way helping you sort of like level up on top of all of the projects that you are navigating and sort of like help you sort of coordinate

between all of them more naturally I think that's something that that we're going to start seeing Now, I don't know the perfect way that that happens, right? I think that the product experience is so so important as a part of this and having a product experience where you don't expect it to be perfect, but you expect it to be pretty good. I think that's like 99% of the challenge. And that's why we haven't seen it yet, despite the fact that the models already can do a lot of this stuff pretty well. My 2025 prediction is that guys use AI agents to use dating apps for them and some get found out and some don't. And we're going to see some stories about how like

some guy like set it on autopilot and ended up, you know, lining up more dates than he could ever hope for. Yeah. Yeah, yeah, yeah. That, well, hopefully. Maybe that's already happening. Hopefully there'll be good dates. Yeah, I don't know. What are you seeing? What are what, you know, I know you had Benioff on the podcast a few, a little bit ago. What are you seeing as the things that seem to make sense from an AI agent's perspective? Well, I think that Mark Beniof, the Salesforce CEO, when he came on, talked pretty convincingly that we'll have AI agents at work. And again, this is like the work or the enterprise use case because work has all this data. And there are all these tasks that we do all throughout the day at work that are just arduous and really quite, you know, quite annoying, preparing.

making dashboards, going to meetings we don't need to be in, pulling out highlights from those meetings, sending them to our bosses, telling our bosses, you know, in the Salesforce instance, for instance, like how each conversation went and what our expected pipeline is to close that corridor and all this stuff can be used for AI. I think it can be used with AI. I think it's really interesting in the medical use case. I was just speaking with GE Healthcare about how they've now put in dashboards for doctors sort of summaries of cancer patients, medical histories, which run thousands of pages.

And the doctors never had a chance to read the whole history. And now the generative AI is summarizing it and going out and finding available treatments for them and notifying them when they missed tests. And I think this is also an example that Beniof gave about the healthcare example where that can actually be proactive in scaling medical advice and medical treatment in a way that you'd never hear from like your doctor after you showed up to an appointment and now can they create an agent that just kind of keeps you on your plan you know in terms of like follow up stuff that you need to do on the consumer side like for everybody else that's kind of where I wonder

because all of our internet has been designed to effectively combat bots but if we have agents that work on our behalf on the internet like travel sites dating sites social media sites I'm very curious, like, whether they're going to come up against these bot protection systems. Like, are they going to do CAPTCHAs on our behalf? Are they going to get the text messages and fill in those numbers? So they're able to log into different systems because, again, the whole Internet has been built to defend against these things. So I'm curious what you think. I mean, is this vision of, you know, personal agents that act on our behalf to do things like book travel, keep up with our health, take action on Internet services for us?

Is it even a feasible thing to do, given all of the particular? protections to sort of guard against them up until this moment? We will have to sort of fundamentally reformat how the internet works to be able to support it. And I think that like the, you know, we're going to need in some, in some senses like there will be like two webs. There will be the web that, that, that humans use when they need to navigate stuff on their own. And then there will be the web that agents use, which is sort of under the surface and something that humans will never see, but allows them to sort of, you know, conduct actions on our behalf more, more efficiently and easily. And that, I think, will be in the long run what ends up happening. And my honest take is,

I think that to the degree that most of us, you know, there's sort of like two kinds of usages of the Internet today. There's sort of consumption, which is where we're seeking out content and, you know, we're curious about things. And then there's utility-based usage. And I think the sort of addressable market, so to speak, for the agents is all the utility work. Like, everything where I'm using the internet just to, like, get something done, I want that to happen faster, easier, better. I would rather have to not have to do that actively at all. Let's say it's like booking and important and looking up a particular piece of information or, you know, figuring out how to like,

you know, you fill out my tax return or whatever it might be, like, that stuff should all be handled by the agents. And we're still going to have to, you know, do a lot of consumption of content just to sort of like, you know, as part of our, as part of what we like to do. And so, yeah, I think, I think it's a really good point. I mean, ultimately, I think agents are going to start in an area that they'll feel pretty. it'll feel like a toy, just like with any technology. So maybe, you know, we'll all start with like a language learning agent, or we'll start

with a cooking aid agent, or it'll just be something that feels pretty innocuous, but then we'll start to realize we can really rely on it, and then we'll start relying on it for a lot more. And that's kind of what happened, I think, with ChatGBT. Initially, it was sort of, we realized, you know, it was kind of a toy, and then people started doing a lot of homework with it. people start to code with it and then now people do all sorts of stuff with chat chbt and other chatbots that will be the the thread let me ask you this question before we move off of agents do you think

it's ethical for me to like have my AI agent which can type and talk uh go out and email and call a bunch of humans on our behalf people working you know let's say in customer service or uh i don't know if i'm applying to schools and they're trying to find out like information about like whether I qualify and what I need to submit. I mean, these processes, maybe they've been designed as arduous to sort of filter out the people who aren't willing to do the work to sort of get in or pass that application threshold. So it's in some way it's combating these guardrails that companies and institutions have

set up for us. On the other hand, it could end up wasting a lot of people's time. Like I really am anticipating like no agent policies from like certain schools or institutions being like, if you're going to reach out to us, it has to be a person versus an agent. What do you think? You know, I saw this thing on Reddit. There was this post of how a, an admissions officer, she sort of created all these, all these ways in which they could, they could track whether or not an essay was AI generated or not. And there were very detailed things. It was very specific. They're a list of maybe 20 or so criteria that they looked for. And I think that,

you know, to your point, it's, it was kind of heartbreaking to see because that means that, you know, let, yeah, for if a student used an AI to generate an essay, you know, they have to spend way more time just figuring out whether or not it was AI generated to like sift through all the noise. And so, yeah, I think I think you're totally right. I think we're going to need, there will almost in the same way that there will be like an internet for humans and internet for agents, there will be processes for humans, processes for agents. And a lot of things that are high intent or very expensive or otherwise special in some way are going to be reserved for humans only.

And it'll sort of be the sort of like more transactional stuff that can be handed off to agents in mass. That's right. I mean, in some ways, I'm looking forward to this future. On the other hand, I do sort of think, like, the more we talk about it, how much AI will take care of for us, I do sort of feel like we're cannonballing our way towards that Wally future where we're all fat and drinking big, big sodas and having Rumbas take us around the world. It's, yeah, I think, I think ease and convenience, which definitely are the direction. that technology has taken us, you know, clearly there should be limits at some point, but if they exist, we don't know where they are.

Exactly. And this idea of like removing friction, in some ways it's made the world great. In other ways, it sort of changes the brain chemistry of people where, like, we don't expect to go through hard things. And when we do, we lose our minds. And that's why you end up seeing the YouTube videos and the videos on X of people in the airport because we've removed so much friction and companies have competed on the base of customer experience to the point where now if something goes wrong, we're fragile. And we think that,

you know, we deserve better. And there is something to be said for friction. It toughens people up a little bit. Totally. All right. We're here with Alexander Wing, CEO and co-founder of Scale AI, $14 billion company that works with others to help. generate AI data, AI data for them, and also help them scale their AI solutions. We're going to talk a little bit more about Alex's third prediction when we come back right after this. Hey, everyone, let me tell you about the Hustle Daily Show, a podcast filled with business, tech news, and original stories to keep you in the loop on what's trending.

More than 2 million professionals read the Hustle's daily email for its irreverent and informative takes on business and tech news. Now, they have a daily podcast called The Hustle Daily Show, where their team of writers break down the biggest business headlines in 15 minutes or less and explain why you should care about them. So, search for The Hustle Daily Show and your favorite podcast app, like the one you're using right now. And we're back here on Big Technology Podcast with Alexander Wing, the CEO and co-founder of Scale AI. So, Alex, I want to ask you about this interesting shift that we're seeing, right? So up until this point, we've talked entirely about AI models on the basis of how many GPUs or chips they're trained on, right? It used to be that you could train a model on like 16 chips, right?

By the way, they're not cheap, like $20,000 to $40,000 each. Then I went to $1,000. And now towards the end of the year, we started hearing crazy numbers like $100,000, $200,000. I was just at Amazon's Reinvent Conference in Vegas. Matt Garman, the CEO of AWS, told me that they're going to train the next anthropic model on hundreds of thousands of GPUs, GPUs or GPU equivalents. And then I was like, oh, that's a lot. And as he's saying that, Elon Musk came out and was like, well, we are going to train the next

XAI model in Memphis on a million GPUs. So I think we're really hitting, like maybe we're hitting the limit. I don't know, of what you can do with chips. And so you believe that we're going to shift. this conversation beyond chips in terms of what makes the most powerful model. So I will tee you up for prediction number three. Yeah. And so so much of the dialogue to your point over the past few years has really been around GPUs and computational power. And I think what's going to happen in 2025 is we're going to, we aren't going to only be focused on who can create newer, better chips or

bigger data centers with more chips, but also who can create newer and better data. And one of the things that I think we're going to see is a focus of the focus shift from just computational power to computational power plus data being sort of considered nearly equally. You know, data really is at its core the raw material for intelligence. So the conversations around data are going to be really interesting. And one of the big topics that's been that's been bounced around for the past few months has been, And are we hitting a wall, have we hit the data wall, and are we hitting a wall on progress overall? And I think the interesting thing that's been happening is, you know, this has come from an approach

of scale up computational power at all costs. If we just scale up the number of GPUs and create huge, bigger and bigger, you know, data centers of GPUs without creating more and more data to train these models on, then we're going to hit issues and we're going to hit walls and barriers where we stop seeing the level of progress that we expect out of the models. So one of the big things that we see, especially in our work with a lot of the frontier labs, is it is true, they're scaling up the GPU clusters, they're scaling up the number of chips, you know, that's still a very aggressive path for them.

But the in parallel conversation is how do we scale up data? And that, you know, there's two sides of that. One is obviously scaling up the volumes, but also scaling up the complexity. So they're seeing the need to go towards more of what we call frontier data. So go towards advanced reasoning capabilities, agentic data to support the agents that we were just talking about, advanced multimodal data. We just saw today, for example, that Open AI release SORA. And so the needs for video data and more complex combinations of

video, text, audio, imagery, et cetera, altogether is going to be really, really interesting going into the next year. And so I think one of the lessons that's really played out more recently with the models is that you can't just scale GPUs and expect to get the same levels of progress. You need to have a strategy by which you're going to scale up all three of the pillars. You need a strategy to scale up the computer. You need a strategy to scale up data. you need a strategy to continue improving the models.

And it's only through the sort of concert of all three of those things that you're going to be able to get, keep pushing the boundaries and barriers on AI progress. But I'm curious what you think. I mean, you've talked to all these CEOs. What are they talking about? I mean, this is exactly the thing that they're talking about. We had Aidan Gomez from Cohere in a couple weeks ago. And he basically said that this has sort of been the path of training the models. whereas in the early days, you could effectively bring anybody off the street, take down anything

they had to say, and there would be new information for the models. And then you started to have to bring in grad students to talk about their, because that general knowledge base was built. So then you bring in grad students to talk about their area of discipline. Then you go to the PhDs. And then he goes, where do we go next? Because we have all this general knowledge. And now we have all the specialized knowledge that we've used to train these models on.

And by the way, it's just amazing the way. that they've improved and been able to sort of handle some complexity. It's really crazy. And so the question is like where to go next. And I think that's what you guys are working on now. And I'd be curious to hear what the process is like on your end for, you know, generating more data for these models to train on. Yeah.

So it's exactly what you just mentioned. Like a lot of what we're focused on is how do we bring in expertise and, and really this sort of expertise. from every field you might imagine, from, you know, medicine to law, to math, to physics, to computer science, to, you know, even knowing about really advanced systems of various kinds or being a great accountant or, you know, whatever field you might imagine, getting the sort of all of, what is all the arcane knowledge, what is all of the sort of really specific, deep knowledge that exists in each of these areas and pull that into, you know, large-scale

data sets that we can use to help train these models to keep improving in a lot of these areas. And a lot of the effort for us has been something that we call hybrid data. So how do we – so one of the things that we've seen over the past few – past year in particular is that synthetic data has not worked as well as, I think, everybody at hope, you know, pure synthetic data, just using data generated from the models, try to train future models. That can sometimes cause real issues for the models. And so one of the things that we've been really pushing forward is this idea of hybrid data.

So you have synthetic data, but you use human experts to mix in with the synthetic data to ensure that you're producing data that's really, really accurate and high quality, and it won't cause issues, but also you're able to do it very efficiently and at large scale. So you also have those PhDs that will sit down and kind of write what they know or dictate what they know and then you feed that into the models. Yeah, exactly. And a lot of times it's even more targeting that. You know, you run the model until you realize the model's making mistakes over and over again. And then you know you've hit sort of a limit of its knowledge or a limit as its capability and you have a PhD sort of come in and help, you know, set the model up on the right

track, so to speak. What's the limit then in terms of where are we going to get to? Because if we, let's say we have all these specialized fields input, their knowledge, does that eventually make, like, AI complete if it just kind of knows everything about every subject? Or does it have to hit, like, a new benchmark to really show that it has this, like, next level intelligence? Like, does it have to start making discoveries of its own?

What do you think the benchmark should be? Yeah, I think, well, to me, I think there's like clearly many more levels of improvement. So now it's it's sort of testing, okay, can it do each of these things right once or, or how, there's sort of, the first track was just reliability. So getting these models from doing something right once and five times to write 99.99% of the time. And that requires a lot of development just to get to that, you know, increase the level of reliability of the systems. And then to your point, it's really about how can the model start taking more and more actions in a row.

You know, one of the things that really is true in all the models today is that they're not that good at, you know, at taking multi-step actions. Whenever it has to take a few hops, whenever it has to chain a few things together, it'll invariably make mistakes along the way. And so the next level of improving reliability is really enabling the models to do more and more multi-turn, more and more multi-step reasoning to be able to enable them to sort of do more and more complex tasks. And then the last piece as we go is, and this is the key to where you're going, is like,

eventually it'll be able to start making its own hypotheses, running those tests on its own and sort of ultimately making its own sort of discoveries or realizations or sort of conduct its own research. And even then, it's still going to get stuck sometimes and still going to need a human PhD to come in and sort of help it, just in the same way that, like, you know, a PhD student these days, you know, still needs an advisor to sort of still give it the right nudge. And so, and so I don't think the sort of like the symbiosis, so to speak, between the humans and the AI will ever go away. Like I think we'll always be able to sort of, will always be very important in helping the models, you know, get on the right track.

and ensure that they always are continuing to improve. But we're going to see the model sort of level up in terms of what is the degree to which they're able to be autonomous and the degree to which they're able to operate on their own. And on the multi-step thing, right, taking a bunch of different steps, I heard something interesting from Moody's last week and I want to run it by you where they said basically they've created 35 individual agents. So let's say they want to evaluate something for their portfolio, like a company for their portfolio. you, they'll have one that will look at, one agent will look at the financial data. Another agent will look at the, let's say, weather risks. Another agent will look at the location that they're based in. Another one will look at the industry.

They have 35 different variables or whatever it is. And then they have, all of them come back and they deliver their results to this compiler agent, which evaluates all of it, and then runs the results by voting agents, which ask, okay, is this reliable or not? I walked away from that impressed by the idea, but also like kind of my reporter brain went off and was like, I don't know if this is real or not. So I'm curious what you think. Is that a possible solution and how feasible is that in terms of a way to get into these multi-step processes? So that's a very, in my work, it's a very sort of regimented way to try to enable the systems to do multi-step reasoning.

Because ideally, what you want the model to do is to, just like how a human does, be able to sort of go through and figure out what are the bits of pieces it needs to know as it goes along and be able to do so on its own dynamically without having to sort of like predetermined and preset this entire regimen for the models to need to go through. So you're saying that might be something that a model can do entirely on its own. That's pretty cool. I think in the future, like, we're going to, the models will improve to be able to get there. And I think the real, on the multi-step side and the multi-step reasoning point, I do think that the, there's a lot of blockers because this is the kind of thing that humans learn how to do kind of from a lot of trial and error and experimentation. like we'll try to do a complex task and then we'll realize we'll learn that oh we actually missed you know let's say you try to bake a cake for the first time for you know a reasonably complex endeavor and then you realize you missed a B C and D and then the next time around

you'll be like okay I'm definitely going to remember I just had a pan of flour that came out of the oven where did I go wrong um and uh but yeah exactly I mean like we learn a lot through trial and error. And right now the models are, the models are early in the process of doing the same thing, of going through and, and sort of, and being able to do these sort of dynamic processes where they learn through trial and error and they are able to continually learn from their mistakes. That's where we need to get to. Okay, great. I know we have just a couple of minutes left. So let me throw a couple quick hits at you and then we can head out. First of all, I'm just curious.

We talked a lot about how data is going to matter a lot, but I can't get my mind off the fact that Elon's going to try to build this million GPU super cluster. What's your prediction for what that spits out? I honestly think right now at where we are in AI development today, we are more bottlenecked by data than we are compute. It will just have an incremental improvement then with something like that. Yeah, I think the real step changes come from data. Okay. So just a quick follow-up to that if we, if we end up to, like I just saw there was news from Google today about this breakthrough they had in quantum computing, which we'll probably cover more on the Friday show. If we have working quantum computers, which can process data much faster, what do you think that does for AI?

I really think, so I had the opportunity to tour Google's quantum facility earlier this year. It's very impressive. I think quantum computing is on kind of like the way AI was back in 2018. It's on a few scaling laws where you can definitely sort of squint and see that in five to 10 years this is going to be a really, really impactful technology. And ultimately, I think what it's going to enable is it's going to speed up AI's ability to do scientific discovery. And so whether it's, you know, I think a lot of the use cases that excite people are in biology or chemistry or fusion or a lot of these very chaotic and difficult understand, you know, natural sciences.

I think that's where quantum computing has the ability to be pretty transformational fundamentally. And I think AI will be able to use it as a tool to be able to enable it to do incredible research in those fields. That's crazy. Okay. So, all right. Last one for you. We're in the middle of like this race where it seems like every week the foundational model companies put out a new development, whether that's Open AI, whether that's Anthropic or even XAI and Google. Amazon just released a set of new models last week.

So who do you think is in the lead at the end of 2025? Oh, that's hard to say. I mean, I think that one thing that we see today with the models is that. that because all the benchmarks that we're used today are what's called saturated, i.e., in other words, all the models do really well at the benchmarks, it's really hard to discern actually which models are fully on top versus not on top. There's a lot of argument, for example, on the internet, at least in the Twitter feeds that I see in terms of whether Claude is better or a one is better, and there's a lot

all these comparisons and between the two of them. So one of the things that I think we're going to need in 2025 are much, much harder benchmarks and much, much harder evaluations that are going to be able to help us figure out, you know, separate the wheat from the chaff a little bit. I don't know who's going to be in the lead, but I do think that I think that we need much better measurement to actually be able to discern between all of these incredible models that labs are pushing out right now. Okay, all right, we'll take it.

No prediction on who's going to be the best, but a definite, interesting perspective on evaluations. Alex, great to meet you. Thank you for coming on the show. I think these predictions have been fascinating, definitely stretched my mind in areas that I wasn't thinking about. So thank you, and we hope to have you back sometime soon.

Yeah, this was a lot of fun. Thanks for having me. Thanks for being here. All right, everybody, thank you so much for listening. We'll be back on Friday with Ranjan breaking down the news. We will see you then on Big Technology. podcast.