Your Undivided Attention - The AI ‘Race’: China vs. the US with Jeffrey Ding and Karen Hao

Episode Date: August 31, 2023

In the debate over slowing down AI, we often hear the same argument against regulation.   “What about China? We can’t let China get ahead.” To dig into the nuances of this argument, Tristan and Aza speak with academic researcher Jeffrey Ding and journalist Karen Hao, who take us through what’s really happening in Chinese AI development. They address China’s advantages and limitations, what risks are overblown, and what, in this multi-national competition, is at stake as we imagine the best possible future for everyone.CORRECTION: Jeffrey Ding says the export controls on advanced chips that were established in October 2022 only apply to military end-users. The controls also impose a license requirement on the export of those advanced chips to any China-based end-user.RECOMMENDED MEDIA Recent Trends in China’s Large Language Model Landscape by Jeffrey Ding and Jenny W. XiaoThis study covers a sample of 26 large-scale pre-trained AI models developed in ChinaThe diffusion deficit in scientific and technological power: re-assessing China’s rise by Jeffrey DingThis paper argues for placing a greater weight on a state’s capacity to diffuse, or widely adopt, innovationsThe U.S. Is Turning Away From Its Biggest Scientific Partner at a Precarious Time by Karen Hao and Sha HuaU.S. moves to cut research ties with China over security concerns threaten American progress in critical areasWhy China Has Not Caught Up Yet: Military-Technological Superiority and the Limits of Imitation, Reverse Engineering, and Cyber Espionage by Andrea Gilli and Mauro GilliMilitary technology has grown so complex that it’s hard to imitateRECOMMENDED YUA EPISODES The Three Rules of Humane TechA Fresh Take on Tech in China with Rui Ma and Duncan ClarkDigital Democracy is Within Reach with Audrey TangYour Undivided Attention is produced by the Center for Humane Technology. Follow us on Twitter: @HumaneTech_

Hey everyone, this is Tristan. And this is Aza. As we do our work, we often meet other people who are just as concerned about the fast pace of AI development as we are. And yet they're terrified for calling for a pause or a slowdown. Why is that? The main reason, we have to beat China. And this is like a trump card for the argument, right? It ends the debate. Because who's going to argue, no, I think we should let China win. But how much do we really know about China's development of AI?

What are the stakes in the AI race for the Chinese government? And is it even possible for China to overtake the U.S. in AI? Or is this just some kind of inflated threat? Yeah. So in this episode, we're going to be bringing you a conversation with two very well-respected experts on tech in China. And that's really good, because these takes go against at least my intuition. So Jeff Ding is an assistant professor of political science at George Washington University, and he was previously a postdoctoral fellow at the Stanford Center for International Security and Cooperation.

And Karen Howell is an award-winning journalist covering the impacts of AI on society. She was formerly a foreign correspondent covering China for the Wall Street Journal and a senior editor for AI at the MIT Tech Review. Welcome so much the show. Thank you so much for having us. Got to be here. So let's do a situational assessment. Like where is China with AI?

In what areas is China ahead of the U.S.? Where is China behind? What's overrated in terms of their capacity and underrated in terms of their capacity? And I know this is a complex topic because there's, are they ahead in hardware and chips and production pipeline? Are they ahead in like the software, the creation of LLMs? Is that like public or is that private? Is that research? Then there's the question of data.

Like who has access to what data streams and at what scales? And so I know it's a broad question, but love if you guys could break down, like where is China on AI and hopefully along some of these spectrums? Overall, we can pretty clearly say that the U.S. is ahead and AI and China is behind. And people will put various numbers here, but I think it's hard to put like a specific figure because it actually really does depend on the discipline, like the subfield within AI. We can have consumer applications of the technology. We can also have military applications of the technology and there are different stakes depending on which one you're talking about. When it comes to computer vision, China is potentially like head to head with the US or

maybe arguably like a little bit ahead because of the emphasis on surveillance. And so there was a lot of effort by the government to invest in this technology. There was just a lot of talent that was cultivated within computer vision. The government is willing to put all these cameras up and cameras are the basis of the data that you need to train computer vision systems. Okay, so just to jump in here, computer vision is AI that's applied to figuring out what's going on in an image. So you think of like a closed circuit camera, like a CCTV.

It can understand what's going on in the image. in a microsecond. This could include facial recognition. It can be used to spot a criminal in a crowded stadium, and also gate detection, where AI can figure out with 94% accuracy who a person is just by looking at how they walk. Jeff, I want to turn it to you. I see you nodding here. Why does AI matter to the U.S.-China competition from your perspective? And specifically, I know you focus on economic growth and productivity. I think one way that I think about AI and U.S.-China competition is through the lens of historical competitions over general purpose technologies like electricity. So AI is often referred to as

the new electricity. And there's this idea that in the past, all these different industries and application sectors were electrified. And now in the future, all these different industries and application sectors will be cognitized or intelligentized. So that gives sort of a specific pathway for why China cares about promoting artificial intelligence to sustain productivity growth and achieve productivity leadership. I've done translations of President Xi's speeches and different analyses of those speeches where Chinese commentators are specifically referring back to these past productivity leadership transitions, where one country seizes new opportunities in these general purpose technologies, is able to adopt them faster throughout their entire economy.

and become the leading economy. And then eventually that translates to more military and geopolitical influence. So I think that is definitely one way to frame the stakes and why both countries think of it as so important. So then how is China performing with large language models? China is definitely behind in that the calculus actually is quite different because when you look at just English language data versus Chinese language data globally, there is far more English language data because it is the lingua franca of the world and of science, of business, of like really high-quality subjects,

whereas you cannot find tons of like high-quality nature papers in Chinese language, for example. And so a lot of Chinese researchers say that now in the generative AI boom, focused on language, they do feel very much at a disadvantage. this regard and that they don't feel like they could ever really build like a chat GPT purely based on the data. It's very difficult for them to achieve that level of performance. And I think we'll start as transformers and as generative AI kind of consumes all other forms of AI, I think we'll continue to see the snowballing of the U.S. English language advantage and China just struggling with that data piece.

Jeff, I'm curious, do you agree? Do you have anything to add here? Yeah, I think I want to emphasize two other dimensions. One is who is doing frontier research, like paving new paradigms in AI development. So I think these U.S. Frontier Labs that you're talking about, they're setting and paving the way for new research directions in this area. And then second is computing limitations. So the computing hardware and the chip.

required to both train and run these large AI models. And I think both of these points, as we unpack them further, I think they cut against this argument about the U.S. not needing to regulate its own AI development or large language models because China is just going to race ahead. I don't think that's true when you look at what's happening specifically in large language models. So in terms of the paradigm shifts in large language models, models. We often forget this because it's the forerunner to chat GPT, which has set the whole

internet ablaze. But GPT3 was a huge innovation in terms of just massive amounts of data to train these large language models on entire swaths of the internet. Then Chinese researchers followed in the wake of GPT3 to produce their own large language models. And in a recent Center for the Governments of AI report, myself and Jenny Shao, we showed that it took about one and a half to two years for Chinese researchers to catch up to GPT3. And the best Chinese competitor in terms of chat GPT is still not very close to OpenAI's version. And you mentioned the other limitation to China's development of AI is compute.

And for listeners, compute we're referring to GPUs or these big advanced chips from Nvidia that are used to train large AI systems. It's kind of like how big is the computing power that you have? Just to give you a sense of the differences in terms of computing power, and how they play out, that Chinese version of GPT3, one of those versions is from Beijing Academy of Artificial Intelligence.

To train it, they had to go to a supercomputer in Qingdao, so to use national resources to train the model. Then once the model is trained, they copied it to a hard drive, and basically we're forced to just give it away for free to any Chinese company that wanted it because they didn't have the compute to run the model. But then all these other companies, okay, So now they have the model, it's trained, but they can't run it either because it costs so much in terms of compute.

And I'm not going to go into the details about Nvidia A100 chips being restricted now because of export controls. But that is definitely a bottleneck to China's AI development as well. That's super fascinating. Like having is different than being able to use. And actually opens up the general area of misconceptions and to hazard a guess. I think here in the U.S., when we think of, China, we tend to imagine all of their tech companies and their government acting as some kind of Borg that's all cooperating with like perfect cooperation. But I think we should

interrogate that a bit. There's like Huawei and Tencent and Alibaba and Baidu. They're all developing AI. What is the relationship between them and what is their relationship to the national universities and to the Chinese Communist Party? Yeah, the Chinese government and Chinese companies definitely do not see eye to eye on many things. And there's immense friction in that system. And it really depends company by company what the relationship is with the government. Like there's some companies that are really cozy and also some companies that the government is very intent on keeping an eye on. But there is lots of budding heads constantly behind the scenes that we don't necessarily hear about because companies will never ever publicize if they're

disagreeing with the government or, like, trying to negotiate with the government on certain things. But, like, one example that I really love that I heard, and I'm not going to mention the company, but there was once an instance where the local government was asking for a company to give up data to try and basically do some investigation into how a crime was committed, which is also a request that often American companies will receive from the U.S. government. and this company refused because of data privacy reasons and then the local government kept pushing and pushing and the company ended up printing out all the data on physical paper

and delivering it in boxes to the local government office and I love it so much because there are a lot of really innovative things that Chinese companies do to essentially retain their own agency and control yeah, we'll find these ways to troll the line a little bit. But I do think that in the same way that in the U.S., there are moments when Silicon Valley and Washington suddenly become aligned on a particular thing that also happens from China as well. Like right now, when Chinese companies are trying to restimulate their business,

they are looking to the government to be like, hey, you were the ones that regulated us and slowed down our business? So are you going to give us some kind of incentives now to help us build ourselves back up? I think what you're saying is really important because there is this notion that by law, Chinese companies do have to comply to the state intelligence service. And so it's important that you're noting places where companies in China will push back against the government. Jeff, what do you make of this aspect of the conversation?

Yeah, those examples make me want to plug this subreddit called malicious compliance, if you all are familiar with that. But yeah, I think the point that I want to add is there are similarities. in terms of pushback from companies and they're not all working in sort of one monolithic entity. But I think there are also differences in that the government does have more levers of control over companies.

So we've seen increasing influence of the party over large tech companies. And Karen has reported on some of these developments as well. We've seen more party committees and party representation in key leader. mechanisms or bodies at some of these large technology companies. And I think the recent generative AI regulations hammer home this influence of the government, especially the government's desire to control information, most obviously through censorship.

So with these large language models that we've been discussing, the recent Chinese AI regulations apply very stringent planks and requirements for companies to disclose their training data or for companies to ensure that the information that's producing is objective. So they're very concerned about these technology platforms that have the capacity to shape public opinion. And that is so key to the Chinese government's legitimacy, the ability to control information. So I do think that there are some pretty stark differences on the information control aspect. For someone who's not familiar with China's political system, why would releasing Open AI open source systems to the broad public be so threatening to the Chinese Communist Party?

Yeah, I think throughout the last few decades, as China has been really building up its cybersecurity regime and its censorship regime, you see that any time there is a technology that could change the dynamics in the information ecosystem, there will be a lot of stress and a lot of fear and quick reactions from the Chinese government to try to contain that. So every time you see a step change, you will get that kind of language to remind people, by the way, large language models are here, but you still can't have them spewing things that are misaligned with what the party wants. And so, yeah, you do definitely see, like right now there's an effort in China where the government is trying to go through compliance checks with each of the major providers of foundation models to make sure that the foundation models have been implemented in a way that. that do you have the right censorship controls, and then they're being allowed to be released. And so in terms of going back to this question of, like,

can a U.S. afford to slow down and regulate its companies or won't China just race to catch up? One of the counter arguments is that large language models are not being deployed massively in China without these stringent controls because they're so unpredictable. How do we think about the balance between the concerns about stability, so the rollout of LLMs and large language models undermining stability and how much the Chinese Communist Party values that versus the competitiveness, economic growth, military might, and every

incentive that they have to throw resources to catch up. I think even if the Chinese government weren't concerned about stability, it would be very difficult for China to not, I guess what we're saying is not catch up, but specifically to get ahead of the U.S. And that is partly because of the data limitations, partly because of the talent limitations and the compute limitations that Jeff mentioned and the sheer money and resources that go into these things. I talk with a lot of Chinese AI researchers all the time about this idea that there's no way an open AI could have happened in China because there's no version in which a Chinese organization would just be given $10 billion US dollars with no plans other than, yes,

we're going to build AI, and then let them play for eight years. years. That just doesn't really happen. And I think because of all of these different things, it would be difficult to create sort of the same conditions, especially now because of like the economic slowdown in China as well, you're not going to see these kind of conditions where the government is just going to throw all this money without a clear understanding of what's happening and just wait around for years and years for researchers to do kind of frontier level research. So what we're going to continue to see, I think, for a very long time is the U.S. and the U.K. still dominating in what Jeff was calling these frontier innovations because the amount of resources

that people are willing to pour into getting there is just so much higher. And the talent concentration is still so much higher. The U.S. is still benefiting from so many researchers around the world wanting to come to the U.S. to be part of the AI revolution there. And so then you layer on the fact that there are also reservations from the Chinese government around the way that these technologies could undermine their credibility and social stability. And you get into just a very layered set of interlocking challenges that I think the American discourse around U.S. China competition often misses. And going back to the should we be regulating these companies in the U.S. because China could catch up. Actually, I think if the U.S. regulated these companies, you would see a slowdown in China as well because a lot of these frontier innovations that might not be happening as fast in the U.S. are also then not going to be translated into China as fast. This is such a crucial point that you're making, which is that the fastest accelerant to China's progress is the U.S. releasing stuff faster.

But I think these kind of counterintuitive notions are super important to tease out because I don't think most policymakers hear this, right? they hear, if we stop, we're just going to lose to China. Jeff, you had your hand up. Let me add one more counterintuitive notion on top of that. I think oftentimes we assume that regulation is going to slow down progress. And I think historically smart, prudent regulation to make technology development safer, more sustainable, more trustworthy, which is extremely relevant in terms of AI and people's distrust of AI systems, unfamiliarity with more powerful AI systems,

smart regulations might actually lead to more sustainable and fast development of AI over the course of years or decades, depending on what time scale you're thinking about. So I think from the Chinese perspective, we've mentioned a few of the regulations that are targeted towards information control or making sure that these models don't say anything politically sensitive. But some of these other regulations are actually meant to ensure personal privacy protection, improve transparency about algorithms.

So some of these things, they might slow down development in the short run, but over the course of a few years, maybe even decades, it might lead to more sustainable adoption of AI across a bunch of different sectors of society. Yeah, what I'm hearing you say is that the race to deploy AI as quickly as possible be like the race to deploy nuclear power plants. before we figure out how to make them safe everywhere and you just end up with a volatile country and it should in fact be a race to deploy in such a way that it strengthens your society. But I want to make the case for why the U.S. should be alarmed about China's acceleration of AI progress.

One example that we often hear is just look at the number of papers that China is publishing versus U.S. They have a huge lead in the number of papers published so shouldn't we in fact be really scared that, yes, right now we're behind, but if you look at the dotted line of where things going,

like they have a population advantage, a data advantage. They're graduating more engineers, more computer scientists, more STEM talent. And I want to add, there are people like Eric Schmidt and Reid Hoffman and very influential American tech voices

saying, if we stop for just 10 seconds, we're going to lose this race. And so let's steal man, do they see something we don't see or haven't mentioned yet in this conversation? And feel free to either of you to react to what Aza was sharing. I think the talent,

The talent factor is a big one. China is currently behind in the talent, but they are pumping out a ton of engineers and researchers. The key is whether or not they'll be able to tap into that talent and whether they can retain that talent. This is a challenge China is facing. Also that the U.S. is facing because U.S. immigration laws are so difficult that a lot of actually Chinese researchers that want to come to the U.S. and stay and live and contribute to American AI innovation are being sent back. But in the long run, I do think that if you have more of the best and brightest minds, you will potentially start to overcome some of the challenges that we've mentioned. So I do think that in that sense, that something could come that would actually completely shift the dynamics. But I think that possibility is still quite unlikely based on the current scenario. I think the one thing that I would want to add is I do think that it's important to talk about AI through the competition lens. But so much of what people are developing in China has nothing to do with nationalism or geopolitics or anything

like that. Like, when you talk to research about why they're excited to get into AI, they're talking on things like, I really want to improve education, improve health care, like good for humanity things. And I have never met a Chinese researcher who's like, I'm doing this for my country, you know?

They just don't think in those terms. So I do want to just remind people that it's not like the moment that you go to China that everyone is just talking about like, how do we with the U.S., like, how are we going to build these technologies to make our country strong and be able to, like, overtake the U.S. as a superpower. Like, I do not ever hear those conversations.

Karen, that's fascinating because we do hear the, but we're competing with China all the time in the U.S. And it's funny then to realize in China, the vast majority developers are not saying, but we're competing with the U.S., and that is another way of reinforcing a point you made earlier, that it's actually the U.S., which is leading the competition, and if China's taking the stance of fast follow, then the race is defined entirely by our drumbeat. Yeah, I think to take the other side, really, to some extent, when it comes to frontier AI research,

it is a two-player game. In terms of countries that have concentrations of maybe 100 or so leading AI researchers that live in different clusters, like Beijing, Shanghai, Chen Zheng. There's not that many outside of China in the U.S., right? Maybe London.

So I sit in a modest office in Foggy Bottom where my incentive is just to find the truthful interpretation of something. Where does China stand on AI? Reid and Eric have some personal stakes, financial stakes, and a lot of different things. Their incentive is not necessarily to do the most systematic or rigorous analysis. No offense. I'm a policy debater, and I'm an academic and I have academic freedom to say these things.

So I have no instrumental reason to underinflate or inflate China's threat. Other people do. And also another thing is, who are you reading and who are you talking to? At least 50% of my consumption about China's AI scene is from Chinese language text. I would say Karen's probably at that ratio as well, or talking to people on the ground. Why would they have an incentive to underinflate China's progress? So the more you talk to people on the ground, the more you talk to people who have read Chinese language,

long-form investigative reports about these topics, the less likely you are to take this view that China is on the verge of surpassing the U.S. as an AI superpower. So, Jeff, that's really fair. Both Eric Schmidt and Reid Hoffman have a stake in policies that say that China's catching up and we have to let the U.S. companies rip because they are invested in them. But if I still try to be charitable to their perspective,

and assume and Steelman their perspective, I might assume that maybe they've seen classified information about military advances in China, or there's certain areas where China seems to be ahead in, say, hypersonics or drones or other kind of military developments, where maybe they're not ahead in AI, but there's this sort of blur your eyes,

smear across what you're looking at, say they're so advanced in certain areas that are dangerous, we don't want them adding AI to those things, and so we do need to slow them down or we can't let them catch up. But that's a separate conversation than large language model training. or is it? I just wanted to have one last chance here about for those who are really saying

we cannot afford to slow down even a year. Like the best possible case, there's some national security military argument. What is that? I hear this a lot and it's like the Trump card for people who want to say that there is still the risk that China is going to be ahead in these spaces because it's impossible to argue against. You have access to some classified intelligence that I don't have that says your side wins this argument. I think it's possible, right? It's possible that these people have access to privileged information about maybe Chinese actors exfiltrating key models or these mysterious military AI applications that no one's actually heard of. I think it's still unlikely for two main reasons.

One is AI is a unique technology in that so much of the cut. adding edge advances are released almost immediately because there's so much incentive to just publish your model on archive. Say we're doing the best work, come work for us. That's how you attract the best and brightest talent. So when people tell me, oh, China has all this shadow research that they're not publishing, there's no logical reason for that. Bidu, Alibaba, Tencent, they want to be publishing all their work in all the top forums because they're competing with the US labs for talent. I think the second argument goes back to some of the historical lessons we've talked about. In all these cases where the U.S. is overhyping its technological rival, Japan, Soviet Union,

I bet people were making this argument too. Now we actually have the declassified records that show, oh, the missile gap was illusory. Oh, this gap that you talked about in terms of ICBMs or all these other strategic military technologies was illusory. And that classified information actually went the other way. You know, I want to be respectful. Sometimes I'm not that respectful, so I don't get invited back to things, but I just don't find those arguments that convincing. Jeff, you wrote what I'd argue was a seminal paper that rejects some of the big assumptions that people have historically made about tech innovation. If I try to boil it down, I would say you argue that we shouldn't be measuring a country's lead by looking at the number of

of scientific papers that they're pumping out. Instead, we need to measure the country's capacity to diffuse the technology, that is to roll it out in factories, in businesses, and in universities around the country. Can you explain that a bit more? Yeah, I think oftentimes when we talk about scientific and technological leadership, that gets boiled down to this magical belief that innovation is all that matters. But much of the hard work comes after the innovation is pioneered. After the company trains the first large language model, how does that get transferred to the small, medium-sized business that's going to implement it in a specific sector? And I think if we take a more diffusion-oriented lens, China actually struggles in a lot of those metrics. China ranks as a

pretty middling scientific and technological power. So that was the argument that I was making in my paper about the diffusion deficit, that we should do a better job about measuring scientific and technological prowess from the stage that occurs after that initial Eureka moment. And then could you give us some historical examples because I think those anecdotes really drive this point home? Yeah, I think we saw a similar overestimation of the Soviet Union's scientific and technological prowess. It's a little bit eerie how similar the claims are from U.S. circles back during the Cold War when there's concerns about a scientific manpower gap in terms of the Soviet Union, more STEM Ph.D. students. And we saw that the Soviet Union was able to produce some

leading-edge innovations, but was not able to have that sort of fast-acting market-based diffusion process to spread these innovations throughout their entire economy. And actually, if you look back to the U.S. becoming a technological leader in the early 20th century, the U.S. was not dominating noble prizes or publications in leading scientific fields like chemistry. It was excelling and being able to translate these leading edge advances throughout the entire economy. Could you give some examples of that? I know in your paper you mentioned examples of

it was like ironworking and metalworking and electricity, and it's not about the number of people who are inventing discovering electricity, it's about the electricians or the electrical engineers, the people who are tinkering with how to implement it. And the more tinker's you have, the more you win, as opposed to the more Einstein's that come up with the first insight. Actually, just to make this comparison to how open AI has been releasing its technology. So on the one hand, we could say, Sam Altman's quote as I think,

you know, the best way to make this technology safe is to roll it out to everybody and have them iterate on it and test it. And this is not a safe way to deploy that technology. And so people are massively diffusing based on specifically OpenAI's lead in the fact that it checked out openaI.com became so popular. I'm curious any reactions you have to that. I do think this is an example of the U.S.'s diffusion capacity when it comes to AI. And the The fact that we have this strong open source ecosystem, the fact that we have an environment that allows companies to experiment without the government crushing platforms that have public opinion properties, a Chinese lab has not been able to release something like chat GPT in

the format of chat GPT being completely open to everyone to tinker and try it out. So I do think it is a testament to the US's diffusion capacity. here, which is the investment capital that's needed to develop large language models. And these models cost billions of dollars to research and develop. So the question I have for you is, does the fact that the Chinese government might censor the products or delay their release with red tape and regulation reduce the incentives for investors to invest? There is definitely a concern among investors around how willing they are to funnel money into

these technologies. And this could be part of the reason why also Chinese researchers say opening I could have never happened in China because you just won't get that kind of capital. But I do want to make the point. I do think that in the U.S. conversation, there is an over-indexing of the censorship being the limiting factor like the bottleneck for capital. One of the biggest things that investors are dealing with right now around generative AI is the lack of chips. They're just not willing to invest in these companies if the company hasn't figured out the chip problem. That is far more of the concern that's top of mind for them than the censorship thing.

If I'm an investor considering investing in a Chinese AI company and I see that Chinese AI company needs chips, but because of the Biden administration's Chips Act and the export controls on chips, I know that they're not going to get them. That makes it hard for me to invest in that company. Exactly. Because the compute, the chips are one of the fundamental ingredients to this thing even existing. So you can't start talking about censorship unless the thing exists. And so every investor right now, like when startups are pushing investors, they have a. slide in their slide deck just on their plan for chips. So a lot of people listening to this, they've heard how important and how central compute or chips are to being able to compete in AI.

And if China's currently behind in all that, what's to stop them from just choosing tomorrow to build chips domestically? What's getting in the way of that? How effective the chip's export control has been and how could they not just catch up? On the effectiveness of the export controls, I think one thing I want to stress is the October 2022 controls were targeted at military end uses. So the commercial labs can still have access to a lot of these chips. And also there are some loopholes in terms of implementation. Financial Times reporters have done some reporting where Chinese labs have even said on the record that they're just able to rent out Nvidia chip clusters and use cloud computing to access clusters that are located in other countries. The Chinese military would never want to do that because of security issues. But Chinese companies, if they're using AI for commercial purposes, to my knowledge, those loopholes and those enforcement gaps are still present in that Chinese companies still can access some of these high-end chips.

If they are completely cut off, it's hard to compete in this industry and to develop indigenous innovation in this industry because the leader in the fifth generation of chips is going to have a strong, first mover advantage to build the sixth generation and then the seventh generation because of accumulated experience, technical expertise, and also capital investments to build some of the plants that are required to design and make these chips. And just really quickly, why can't China just invest, I don't know, $100 billion, $300 billion? It seems like this is the most important strategic advantage moving to the future. So wouldn't China put all of their might, all of their resources against solving this bottleneck? They are definitely trying to solve the bottleneck, and they have put an enormous amount

of money into the semiconductor industry to try and create a domestic chip. I mean, you're seeing the U.S. now trying to also, with the Chips and Science Act, invest an enormous amount of money and re-shore all chips manufacturing. But the issue that both are facing, just as a baseline, is that the chips industry is so enormously complex, and it is the poster child for globalization. like every little piece of equipment or component, like everything that you need, there's one country that specializes in that one thing.

And then it's not like Chinese components are used at every stage, so they don't have that same kind of like sway. Their sway is like their market power. So they're trying to incentivize certain countries with these really critical equipments by saying, hey, we have this big market, we'll pay you lots of money for this. But to develop every single thing from scratch,

is very challenging. The lesson I take away is like both the U.S. and China see it as existential. They're both trying to do it domestically, and they're both struggling to get it right and mostly still relying on this one company in Taiwan, 90 miles off the coast of China to still make the chips, which sets up another constraint in the situation, which is how central Taiwan is and the control potentially of Taiwan. We have Dario, the CEO of Anthropics, saying in the last few weeks

that he believes that even with the best in class security practices that they are using and trying to use, he said a really truly determined state actor could steal the anthropic model that they're building. And just to give listeners a couple examples of this, in 2007, a Chinese specialist breached Lockheed Martin and stole confidential information and design in the electronic systems of the still under development F-35 jet fighter.

Another example, in 2013, the Washington Post reported a secret government report that listing more than two dozen major weapon systems whose designs were stolen over an unknown period by Chinese cyberspies. I just want our listeners to know that when the Chinese people's liberation army

has been determined to get certain information out of the U.S. or tech places, they're very good at this. So when we talk about the biggest risk to China catching up, we don't often talk about it's the U.S. building it

and then China stealing it. I'm curious how you both react to that kind of attack vector of concern. I think when it comes to AI, I'm not thinking, thoroughly convinced that there's actually a huge incentive to steal the technology versus to, like Jeff said, like so much of it is open source. So you would need to really feel like you couldn't access the thing in order to invest all those resources to steal the thing.

So I think that this idea that GPT7 is suddenly going to unlock all of these things around the military, I just don't really think that's the way that the military works. Like I don't think either the U.S. or the Chinese military just like swallows this hugely new emerging technology that hasn't yet been explored in many different ways and then suddenly integrates it into all of the military. It is much more methodical than that. And there is no like killer app right now for generative AI in the military. So I just don't see the incentive for stealing of the technology. And the other thing is China has been like really, Xi Jinping has really been beating the drum around like self-sufficiency. I think the key question for me is how much

of the frontier AI developments can be codified into something that can be stolen versus how much of those new capabilities are unable to be codified and they're more captured in tacit knowledge or in having the experience of actually playing around, training the model, doing all the tough engineering work. You can give the Chinese military a blueprint of a stealth fighter and that's where the news article stops. It doesn't go further and track. Is China, actually building stealth fighters on the level similar to the U.S.? No, because they don't have that tacit and managerial knowledge that the U.S. defense base has in terms of building effective stealth fighters.

So I would point listeners to the Gilly Twins. And international relations have an article about why China can't catch up and they talk about why even with the benefits of globalization, espionage, the Chinese military hasn't been able to build stealth fighters. So you start with the story of, oh, they stole all these blueprints. Let's check back in 10, 20 years later. Did they actually build an effective stealth fighter? No.

I want to close by really just imagining positive end games here. We have this world where we have a handful of companies racing to scale these massive new systems. GPT4 was trained with $100 million. GPT5 will be trained with a billion dollars. After that, it'll be trained with $10 billion. We're going to spend that much on compute to train something that will have read the entire Internet, reason across domains and fields, and be able to maybe in the future do automated science. We're building this incredibly powerful technology. It carries with incredible risks.

And we're going super, super fast because these kinds of things are coming in the next two years. The way to slow down would be to internationally coordinate. If we're going to slow down in a way that we don't just all lose, it's like climate change. Are we willing to cut our economic growth with a big carbon tax? We only want to do it if China wants to do it too. And so much of this seems like we've got to be able to come to the agreement with a shared view of the amount of risk here. and say, could we afford to collectively go at a pace that we can get this right? So what are your reactions to the promise or lack of promise with international agreement?

Do you think China could be a good faith actor and could the U.S. be a good faith actor in any discussions to rein in AI? I think China is hugely interested in international coordination, in part because they want to be part of the conversation to set international norms, so they want to be at the table. But also, like we saw just a few months ago there was this really big, conference that was held by the Beijing Academy of AI. And they convened the leading experts in China and the leading experts in the U.S. Sam Altman was there. There was an executive from Anthropic there.

Those dialed in virtually. But like there are organizations within China that are trying to create these forums for this kind of international dialogue, which is quite extraordinary at a time that we're in right now for these organizations to be doing that and have the convening power to do that. And part of it is because there is a lot of relationship building on the ground between Chinese and U.S. researchers and folks within the AI industry where they really do want to be talking to each other and they're trying to, from a bottom's up approach, get those, that coordination at the ground happening at higher levels. So I am optimistic long term that there are ways that the U.S. and China can set aside differences and find a way to coordinate with competitive elements. You really cannot develop this technology in a safe way that is beneficial for everyone if you don't have such a significant part of the world in the conversation. Yeah, so there is some coordination on the controllability of automated systems going on right now at the highest levels.

There's a joint technical committee that brings together the two most influential technical standards setting organizations in the world. I'm going to throw a lot of acronyms at you, but it's the ISO slash IEC Joint Technical Committee, and they have a specific subcommittee on AI. Chinese representatives lead a working group on building technical standards for the controllability of AI systems. So obviously those applied to systems that are more near-term or in operation now, and we're talking about more long-term, potentially more transformative AI applications, but that could be a building block of being able to talk about the controllability of these systems. today might help us have similar channels and get good ideas about how to control more powerful

systems tomorrow. I think the second thing I'll say is there are some historical loadstones and guiding points for U.S.-China cooperation on AI, especially in terms of safety and security issues. Even during the fiercest periods of the Cold War, the U.S. and the Soviet Union cooperated on this technology called permissive action links, which were electronic. locks that prevented unauthorized use of nuclear weapons. And the U.S. seriously considered sharing permissive action-link technologies with Pakistan, China. So there have been researchers, leading thinkers, policymakers who have proposed we need to

find the permissive action links for AI, guardrails, confidence-building measures, safety and security techniques that the U.S. and China, it would be in both of their interests to cooperate on. Even Eric Schmidt, he was a commissioner of national security commission. on AI, they mentioned permissive action links as a reference point for the U.S. and China to cooperate on restricting certain military applications of AI. So I do think that there are some templates for us to draw on. Just to quickly push back and just hear your reaction, as much as this does look optimistic between this care and this trip, you're talking about the Beijing trip or the

CEOs of the U.S. companies are meeting with some of the labs and leaders of the Chinese companies and also the academic institutions. But at the higher level, when asked between the Biden administration and Xi Jinping and his administration to get climate commitments shared, the Chinese Communist Party rejected those attempts for those discussions. As many times as John Kerry will fly over there, he's not getting the meeting to really actually come up with those commitments because it seems like trust is at an all-time low. So how do we reckon with the kind of difference between the high-level difficulty, it seems, and the Track 2-style more academic institution to academic institution, CEO-level conversations? I think it's exactly that. Like sometimes the coordination has to happen at the track two level. And that's, I think, sometimes the more effective coordination

because those are the people that are actually building the thing. So you want the coordination to be happening as close to the technology as possible. You want the engineers to be coordinating, the research to be coordinating, the CEOs to be coordinating. And yes, at the highest levels, there is not a lot of coordination anymore because there's just a lot of baggage that both governments are now dealing with where they do not want to be the one that moves first to try and meet the other country. But close to the technology, if we are seeing that coordination among the technology builders,

I think that in the long run is where my optimism stems from. I think that's a really good note to end on. We wanted to have this conversation because of the way the drumbeat of, we have to beat China. We have to beat China has become the trump card that ends every other conversation. Yes, and I would say that Karen and Jeff have argued for a much more nuanced understanding of the problem. While China might have academic publications, it doesn't have the same diffusion capacity as the U.S. And there are different types of AI.

China might lead in computer vision, but it faces massive structural challenges to the development of large language models. The final thing I think it's worth taking away from this episode is Jeffrey and Karen both said that the reason why China is going so fast is because the U.S. is going so fast. they are a fast second mover. So if we go slower, they argue, then so too does China. Your undivided attention is produced by the Center for Humane Technology, a non-profit working to catalyze a humane future. Our senior producer is Julia Scott.

Kirsten McMurray and Sarah McRae are our associate producers. Sasha Fegan is our managing editor. Mixing on this episode by Jeff Sudakin. Original music and sound design by Ryan and Hayes Holiday. and a special thanks to the whole Center for Humane Technology team for making this podcast possible. Do you have questions for us? You can always drop us a voice note at HumaneTech.com slash ask us,

and we just might answer them in an upcoming episode. A very special thanks to our generous supporters who make this entire podcast possible, and if you would like to join them, you can visit HumaneTech.com slash donate. You can find show notes, transcripts, and much more at HumaneTech.com. And if you made it all the way here,

Let me give one more thank you to you for giving us your undivided attention.