The AI Daily Brief: Artificial Intelligence News and Analysis - What the Head of the New UK AI Foundation Model Taskforce Thinks of AI

Episode Date: June 25, 2023

Entrepreneur and investor Ian Hogarth was recently named Chair of the UK's AI Foundation Model Taskforce. In today's Long Reads Sunday, NLW reads excerpts from Ian's writing on AI going back to 2018 to help us understand the perspective he's bringing to the taskforce and its £100M mission to make AI safer and more aligned.   Excerpts from: AI Nationalism (2018) We must slow down the race to God-like AI (April 2023) Taskforce announcement thread and post (June 2023)    The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI Breakdown, we long read a set of different excerpts from pieces from Ian Hogarth, the recently appointed chairman of the UK Foundation Model Task Force. The AI Breakdown is a daily podcast and video about the most important news and stories in AI. Like, subscribe and share, and go to Breakdown.network for more information. Hello, friends. Welcome back to the AI breakdown. Today we're doing something a little bit different with our long reads. Earlier this week, entrepreneur and investor Ian Hogarth was named the chairman of the UK Foundation Model. task force. This is Rishi Sunak's 100 million pound task force that's meant to help the UK become a hub

and a leader when it comes to AI regulation and policy. Now, Ian is a really dynamic and interesting choice, someone who comes from the tech and entrepreneurial world, but who has spent the last few years getting deeper and deeper into AI and thinking about it also from a policy perspective. So what we're going to do today is read some excerpts from a number of pieces that show his evolving thinking on AI. The first is from a PC called AI. nationalism in 2018. The second is from a more recent piece, we must slow down the race to godlike AI that appeared in the Financial Times. And the third is a thread and an op-ed around the announcement that he had become the chairman of the UK Foundation Model Task Force.

So let's first go back to 2018. On June 13th, Ian wrote, for the past nine months, I've been presenting versions of this talk to AI researchers, investors, politicians, and policymakers. I felt it was time to share these ideas with a wider audience. The central prediction I want to make and defend in this post is that continued rapid progress in machine learning will drive the emergence of a new kind of geopolitics. I have been calling it AI nationalism. Machine learning is an omnibus technology that will come to touch all sectors and parts of society. The transformation of both the economy and the military by machine learning will create instability at the national and international level, forcing governments to act. AI policy will become the single most important

area of government policy. An accelerated arms race will emerge between key countries and we will see increased protectionist state action to support national champions, blocked takeovers by foreign firms and attract talent. This arms race will potentially speed up the pace of AI development and shorten the timescale for getting to AGI. Nationalism is a dangerous path, particularly when the international order and international norms will be in flux as a result, and in the concluding section I discuss how a period of AI nationalism

might transition to one of global cooperation, where AI is treated as a global public good. Now, section two, we're going to skip over. It's called Progress in Machine Learning, and in it Ian points to a number of different recent updates that include Microsoft achieving human parity on Mandarin and English translation, deep mind building AlphaGo and then Alpha Zero, which beat World Go champions and later top-performing chess computers.

Section 3 is called three forms of instability. Ian writes, So why does this matter to nation states? There are three main ways in which accelerating progress in machine learning could create instability in the international order. Commercial applications of machine learning will create vast new businesses and destroy millions of jobs. In the extreme case, the country that invests the most efficiently may end up the strongest economically. Machine learning will enable new modes of warfare,

both sophisticated cyber offense and defense capabilities, but also various forms of autonomous and semi-autonomous weaponry. In the most extreme case, the country that invests the earliest and most aggressively may end up in a position of military supremacy. Eventually, more general-purpose AI will enable a fundamental speed-up in science and technology research. In my opinion, this might actually be the most profound source of instability. Consider, for example, the state whose leadership in AI enables them to be the first to develop a viable fusion reactor for power generation. Again, in the extreme case, this might enable a country to achieve Wakandan technological supremacy. Machine learning to use Jack Clark's term is a uniquely

omnibus technology that could impact almost every area of national policy. Human intelligence has shaped everything we see around us, so our ability to build machines with greater and greater intelligence could eventually have the same impact. Ambitious governments have already started to see machine learning as the core differentiating technology of the 21st century, and a race has already commenced. This race will come to bear some similarity to the nuclear arms race of the last century and geopolitical tensions and alliances between nation-states and multinational companies over oil. In Section 4, Ian discusses industry mix, labor cost, demographics, basically talking about how, while AI impact will have some common threads throughout the world,

the specific impacts will also vary country-to-country based on things like what types of industries they have and their approaches to welfare and redistribution. In Section 5, Ian identifies that there is a blurring line between public and private sectors. Ian writes, there are incredibly powerful non-state actors who are also competing furiously to develop this technology. All of the seven most important technology companies in the world, Google, Apple, Amazon, Facebook, Alibaba, Tencent, and Baidu, are making huge investments in AI, from low-level frameworks and silicon to consumer products. It goes without saying that their expertise in machine learning leads any state actor at the moment. As the applications of machine learning grow, the interactions between these companies in different

nation states will grow in complexity. States have historically played a crucial role in underwriting long-term high-risk research in science and technology by funding either academic research or the military. These technologies are often then commercialized by private companies. With the rise of visionary and wealthy technology companies like Google, we are seeing more high-risk long-term research being funded by the private sector. This creates tensions when the interests of a private company like Google and a state are not aligned. In Section 6, Ian identified that China in 2018 was way out ahead.

Indeed, he wrote in developing a national strategy for AI, China is way out ahead of everyone else. For China, over the past couple decades, protectionism has been a winning strategy in developing enduring domestic technology companies, and it has ultimately enabled China to be the only other country in the world with AI companies to rival Americas. Now, the piece, which is awesomely long and thorough, then goes through key events in the arms race so far, AI nationalism policies, what can countries that aren't China or America Adieu, the strange case of the UK, rogue actors and how they complicate this whole mix,

and then ultimately comes to his concluding section called Engineers Without Borders. In it, Ian gives his widest view. He says, personally, I believe that AI should become a global public good, like GPS, HTTP, TCPIP, or the English language. And the best long-term structure for bringing this to fruition is a non-profit, global organization with governance mechanisms that reflect the interests of all countries and people. The best shorthand I have for this is some kind of cross between Wikipedia and the UN. While the idea of AI as a public good provides me personally with a true north, I think it is naive to hope we can make a giant leap there today, given the vested interest and misaligned incentives

of nation-states, for-profit technology companies, and the weakness of international institutions. I believe that we are likely to go through a period of AI nationalism before we get to a place where AI is treated like a public good. And that, to use Orwell's distinction, a kind of AI patriotism, is likely to be a good thing for smaller countries in the short term. Taking the example of the UK, I am in favor of a more expansive national AI strategy to protect the UK's economic, military, and technological interests, and to give the UK a credible seat at the table when global issues around AI are being worked out. That will help ensure that the UK's economic interest and values are considered. I believe it is necessary for the UK government to take steps towards investing in

and protecting its homegrown AI companies and institutions to allow them to play a larger role on the world stage independent of America and China. I have lived. in both America and China, and during that time developed enormous respect and affection for both of those countries. That does not prevent me from believing the UK should protect the economic interests of its citizens, and I would like to see the UK play a material role in shaping the future of AI. So that is Ian speaking in 2018. Fast forward to April 13th of this year, the day after this podcast first came out. On that day, Ian wrote an extremely extensive piece for the Financial Times called We Must Slow Down the Race to Godlike AI.

starts that piece. On a cold evening in February, I attended a dinner party at the home of an artificial intelligence researcher in London, along with a small group of experts in the field. He lives in a penthouse apartment at the top of a modern tower block, with floor-to-ceiling windows overlooking the city's skyscrapers and a railway terminus from the 19th century. Despite the prime location, the host lives simply, and the flat is somewhat austere. During dinner, the group discussed significant new breakthroughs such as OpenAIs chat GPT and DeepMinds Gato and the rate at which billions of dollars have recently poured into AI. I ask one of the guests who has made important contributions to the industry, the question that

often comes up at this type of gathering. How far away are we from artificial general intelligence? AGI can be defined in many ways, but usually refers to a computer system capable of generating new scientific knowledge and performing any task that humans can. Most experts view the arrival of AGI as a historical and technological turning point, akin to the splitting of the atom or the invention of the printing press. The important question has always been, how far away in the future this development might be. the AI researcher did not have to consider it for long. He replied,

It's possible from now onwards. This is not a universal view. Estimates range from a decade to a half century or more. What is certain is that creating AGI is the explicit aim of the leading AI companies, and they are moving towards it more swiftly than anyone expected. As everyone at the dinner understood, this development would bring significant risks for the future of the human race. If you think we could be close to something potentially so dangerous, I said to the researcher, shouldn't you warn people about what's happening? He was clearly grappling. He was clearly grappling with the responsibility he faced, but, like many in the field, seemed pulled along by the rapidity of progress. When I got home, I thought about my four-year-old, who would wake up in a few hours.

As I considered the world he might grow up in, I gradually shifted from shock to anger. He felt deeply wrong that consequential decisions, potentially affecting every life on Earth, could be made by a small group of private citizens without democratic oversight. Did the people racing to build the first real AGI have a plan to slow down and let the rest of the world have a say in what they were doing? Now, next in this piece, Ian talks about his background, and the important thing to note is that this is not someone who has been in academia or policy all his life. He's a tech guy. He's always worked in tech. He sold a company. He's back more than 50 AI startups. So he's not coming at this from a

ludite perspective. And still, as Ian writes at that dinner in February, significant concerns that my work has raised in the past few years solidified into something unexpected, deep fear. Now, continuing to excerpt from the piece, Ian writes, acronym doesn't capture the enormity of what AGI would represent, so I will refer to it as what it is, Godlike AI, a superintelligent computer that learns and develops autonomously, that understands its environment without the need for supervision, and that can transform the world around it. To be clear, we are not here yet, but the nature of the technology means it is exceptionally difficult to predict exactly when we will get there. Godlike AI could be a force beyond our

control or understanding, and one that could usher in the obsolescence or destruction of the human race. Recently, the contest between a few companies to create godlike AI has rapidly accelerated. They do not yet know how to pursue their aims safely and have no oversight. They are running towards a finish line without an understanding of what lies on the other side. The current era has been defined by competition between two companies, DeepMind and OpenAI. There's something like the jobs and gates of our time. Now, he then goes on to talk about the forming and foundation of both DeepMind and its acquisition by Google as well as OpenAI and its conversion from a non-profit to a for-profit, and talks about how much of the emphasis of both of these companies has been on the application

of AI or the exploration, at least, of AI, in areas like gaming and chatbots. Ian writes, The focus on games and chatbots may have shielded the public from the more serious implications of this work. But the risks of Godlike AI were clear to the founders from the outset. In 2011, DeepMind's chief scientist Shane Legg described the existential threat posed by AI as the, quote, number one risk for this century, with an engineered biological pathogen coming a close second.

Any AI caused human extinction would be quick, he added. If a super intelligent machine or any kind of super intelligent agent decided to get rid of us, I think it would do so pretty efficiently. Earlier this year, Altman said, quote, The bad case, and I think this is important to say, is like lights out for all of us. Why are these organizations racing to create godlike AI? Ian continues, if there are potentially catastrophic risks. Based on conversations I've had with many industry leaders and their public statements,

there seem to be three key motives. They genuinely believe success would be. hugely positive for humanity. They have persuaded themselves that if their organization is the one in control of God-like AI, the result will be better for all. And finally, posterity. The individuals who are at the frontier of AI today are gifted. I know many of them personally, but part of the problem is that such talented people are competing rather than collaborating. Privately, many admit that they have not yet established a way to slow down and coordinate. I believe that many would sincerely welcome government stepping in. For now, the AI race is being driven by money. Since last November, when ChatGPT

became widely available, a huge wave of capital and talent has shifted towards AGI research. We've gone from one AGI startup, DeepMind, receiving $23 million in funding in 2012, to at least eight organizations raising $20 billion of investment cumulatively in 2023. Now, Ian then returns to some of the themes from his AI nationalism piece talking about the geopolitical dimension of this, and finally concludes with a quick discussion of alignment. Alignment, Ian writes, is essentially an unsolved research problem. We don't yet understand how human brains work, so the challenge of understanding how emerging AI brains work will be monumental. When writing traditional software, we have an explicit

understanding of how and why the inputs relate to outputs. These large AI systems are quite different. We don't really program them. We grow them. What is more concerning is that the number of people working on AI alignment research is vanishingly small. For the 2021 state of AI report, our research found that fewer than 100 researchers were employed in this area across the core AGI labs. As a percentage of headcount, the allocation of resources was low. DeepMind had just 2% of its total headcount allocated to AI alignment. OpenAI had about 7%. The majority of resources were going towards making AI more capable, not safer. We've made very little progress on AI alignment, in other words, and what we have done is mostly cosmetic. Now, Ian concluded that piece

with a set of very loose recommendations that basically amounted to governments need to get more involved, and people who want to have a say in this need to start getting louder. Well, as of last Monday, he will now have a chance to help lead that conversation. On June 18th, he tweeted, to be appointed as the chair of the UK's AI Foundation Model Task Force. I wrote in 2018 about how accelerating AI progress would create new geopolitical challenges. In April 23, I wrote an essay for the FT, we must slow down the race for Godlike AI that highlighted the risks of a small number of AI companies racing to create ever more capabilities without enough progress on AI safety or regulation. The world has significantly shifted since then. Pioneers in the field like Jeffrey Hinton

have spoken out. An open letter signed by a broad coalition of AI experts compared the risks to nuclear weapons or pandemics. And at a pivotal moment, Rishi Sunak has stepped up and is playing a global leadership role. He has pledged 100 million pounds on AI safety, the largest amount ever committed to this field by a nation state. The field of AI safety has been significantly under-resourced, even as funding for AGI companies has now crossed a cumulative 20 billion. We have 100 million pounds to spend on AI safety and the first global conference to prepare for. I want to hear from you and how you think you can help. The time is now, and we need more people to step up and help. I am fundamentally optimistic about the potential for science and technology to transform our lives for the better.

The opportunity for AI to be a force for good are truly remarkable, but we need to do it safely. Ian then echoed those themes in an op-ed for the times. Now, I'm enthusiastic about this appointment. I think Ian's a great choice, and I really like people who have this perspective of a base level of techno-optimism also coming into this murkier, newer space, and being able to rewrite some of their priors and look at it from first principles. I think that to achieve what we want to achieve here, it's going to take people who can translate society, policy, and industry. And I think, frankly, that for industry to respect the policy side, to some extent it's going to have to speak to them in their own language.

I also think that Ian is right to recognize that even in just the two months since he wrote that piece, the landscape for the AI safety conversation looks very different. To me, it's pretty obvious why. The hundreds of millions of people, if not billions of people, who have learned about this space because they tried a generative AI tool, such as ChatGyPT or Mid Journey or something like that, feel to me, frankly, much more comfortable

with the idea that these technologies could get out of our hands. When you've just experienced something that feels like magic for the first time, it's not hard to imagine that that magic might metastasize in ways that you can't understand or see now. I think this is borne out in people's response to Jeffrey Hinton and Yahshua Benjillo and all of this new media attention on AI risk. But I also think that people are going to want to move the conversation to what we actually do. The six-month pause idea not only didn't hit people as the right approach. The most common response, even for people who thought it might be a good idea in the short term was,

and then what? But the ground is fertile for the conversation, and I'm glad that we're actually having it. That is it for today's Longreed Sunday. I hope this was helpful. I hope this introduced you to Ian and how he's thinking about governments in the UK, and AI safety in general, check out his Twitter at Soundboy and he has a bunch of links pinned to his profile page

for where you can fill out a form to see if you can be involved with the Foundation Model Task Force. If you're enjoying the AI breakdown, please like, subscribe, and share it. Go check out the YouTube if you're listening. Go check out the podcast if you're watching. And until next time, peace.