Better Offline - Are We At Peak AI?

Episode Date: April 10, 2024

It’s been just under a year and a half since ChatGPT - an AI-powered chatbot launched by so-called non-profit OpenAI - ushered in a new era of investor and media hype around how artificial intellige...nce would change the world. But what if this we're actually at the peak of what generative AI can do? In this episode, Ed Zitron walks you through the four intractable problems that are stopping Large Language Models like ChatGPT in their tracks - and why they're all-but-impossible to overcome.See omnystudio.com/listener for privacy information.

Transcript
Discussion (0)
Starting point is 00:00:00 This is an IHeart podcast. Guaranteed Human. Run a business and not thinking about podcasting. Think again. More Americans listen to podcasts than adds supported streaming music from Spotify and Pandora. And as the number one podcaster, IHearts twice as large as the next two combined. Learn how podcasting can help your business. Call 844-844-I-Hart.
Starting point is 00:00:19 Another podcast from some SNL late-night comedy guy, not quite. Unhumor me with Robert Smygel and friends. Me and hilarious guests from Bob Odenkirk to David Letterman help make you. you funnier. This week, my guest, SNL's Mikey Day and head writer, Streeter Seidel, help an a cappella band with their between songs banter. Where does your group perform? We do some retirement homes. Those people are starving for banter. Listen to humor me with Robert Smigel and friends on the I-Heart Radio app, Apple Podcasts, or wherever you get your podcasts. Your 20s can be so exciting, but they can also be really overwhelming, confusing, and honestly, just kind of lonely.
Starting point is 00:00:58 May is Mental Health Awareness Month and the psychology of your 20s is breaking down the science behind the biggest roadblocks we face. I was six years into my career, the 80-hour weeks and just the first one in, the last one out, and I ended up burning out. There was a large chunk of my 20s that I was just so
Starting point is 00:01:15 wanting to be out of that phase out of my skin and I just really regret not living in the present more. You don't need to have everything figured out right now. You just need to understand yourself a little bit better. Listen to the psychology of your 20s on the IHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Also Media. Hello and welcome to Better Offline.
Starting point is 00:01:40 I'm your host Ed Zittron. It's been just under a year and a half since ChatGPT, an AI-powered chatbot, launched by so-called non-profit OpenAI, ushered in a new investor in media hype cycle around how AI would change the world. Chad GPT's instant success made both OpenAI and its CEO, Sam Altman, overnight celebrity. as a result of chat GPT's alleged intelligence, which seemingly allowed you to do everything from generate an entire essay from a simple prompt
Starting point is 00:02:19 to writing entire reams of software code. You can theoretically ask it anything, and it will spit out an intelligence-sounding response thanks to being trained on terabytes of text data, like a search engine that's able to think for itself. Eh, big problem is chat GPT doesn't think at all. It doesn't know anything. Chat GPT is actually probabilistic.
Starting point is 00:02:40 It uses a statistical model to generate the next piece of information in a sequence. If you ask it what an elephant is, it'll guess that the most likely answer to that prompt is that an elephant is a large mammal and then perhaps describe its features, such as a long trunk. Chat Chbd doesn't know what an elephant is, or what a trunk is, or what the elephant-hantai family is, it simply ingested enough information to reliably guess what an elephant is meant to be, or indeed know that that's what you're asking it. This is the technology underpinning the latest artificial intelligence boom. It's called generative artificial intelligence, and it's powered by large language models.
Starting point is 00:03:19 And they underpin tools like OpenAI's chat GPT, Anthropics Claude, X.com's horrifying chatbot Grok, and of course Google's Gemini. Essentially, they're AI systems that ingest vast quantities of written text or other data, and then through mathematics, try and identify patterns of relationships between words, or symbols, or basically any meaning from the text or thing they're being fed. And it almost seems like magic, because it's able to generate this plausible, seeming, almost human content at this remarkable speed. These models are now capable of generating text, images, and even video in response to
Starting point is 00:03:56 simple chat prompts, all by learning the patterns and structures of their data. Yet underneath the hood, there's always something a bit wrong. Generative AI at times authoritatively spits out incorrect information. which can range from funny, like telling you that you can melt an egg, to outright dangerous, like when the recently launched AI-powered New York City chatbot for small business owners started telling them that it was legal to fire somebody for refusing to cut their dreadlocks. This is why you'll see strange glitches in images generated by AI, hands with too many fingers, horrifying-looking people in the back of realistic-looking photos,
Starting point is 00:04:32 and so on and so forth. Because these models don't actually know what anything is. they don't have meaning. They don't have consciousness or intelligence. They're guessing. And when they guess, they sometimes hallucinate, which I'll get too soon. And while they might be really, really good at guessing,
Starting point is 00:04:53 there are effectively a very, very powerful version of autocomplete. I don't know anything. I really mean that. These things aren't even intelligent. But because these models seem like they know stuff and they seem to be able to do stuff and the things that they create almost seem right, the media and the vocal investor class on Twitter
Starting point is 00:05:11 have declared that large language models would change everything. To them, LLMs like ChatGPT, would upend entire business models, render once unassailable tech giants honorable, and rewrite our entire economic playbook by turning entire industries into something you tell a chatbot to do in a sentence. You know, it doesn't really matter that generative AI is mostly good at pumping out reams of generic slop,
Starting point is 00:05:34 and that it's also clogging services like Amazon's Kindle EBookstore, and I guess the rest of the internet with generative content, it doesn't matter at all, because it's kind of good. Obviously, I'm being sarcastic. This is all very, very bad. In the last year, there have been hundreds of mulling articles about how AI will replace everything from drive-through workers to medical professionals. Theoretically, you could just feed whatever information a potential customer could ask for
Starting point is 00:06:01 into a vast database and have an AI chew it up and then they could just generate exactly the answer you'd need. AI would just naturally slip into areas of disorganization and inefficiency and spit out remarkable new ideas, all with minimum human input. In every one of these stories carries with them a shared belief, one might even call it a shared hallucination. And they all believe that generative AI will actually be able to do these things. It'll actually be able to replace people.
Starting point is 00:06:32 And what we're seeing today is just the beginning of our glorious automated future. But what if it's not? What if Generative AI can't actually do much more than it can today? What if we're actually at peak AI? In the next two episodes, I'm going to tell you why I think that is. I'm going to tell you how I think this whole thing falls apart.
Starting point is 00:07:02 Another podcast from some SNL late-night comedy guy, Not quite. Unhumor me with Robert Smygel and friends. Me and hilarious guests from Jim Gaffigan to Bob Odenkirk to David Letterman, help make you funnier. This week, my guest, SNL's Mikey Day and head writer Streeter Seidel, help an Acapella band with their between songs banter. The worst singer in the group? The worst?
Starting point is 00:07:25 Yeah. Me. Is there anything to the idea that because you're from Harvard, you only got in because your parents made a huge donation. The yard birds, right? That's the name. The Harvard Yardt Yardt. They're open.
Starting point is 00:07:40 Do you have a name suggestion? We're open. Since you guys are middle aged, one erection. Listen to humor me with Robert Smigel and Friends on the I-Heart Radio app, Apple Podcasts, or wherever you get your podcast. Humor me. I need some jokes to make me seem funny. Run a business and not thinking about podcast. Podcasting, think again.
Starting point is 00:08:05 More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora. And as the number one podcaster, IHeart's twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only IHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business. Think IHeart. Streaming, radio, and podcasting. Let us show you at IHeartadvertising.com.
Starting point is 00:08:29 That's iHeartadvertising.com. agency, the ability to know that we're the experts in our own body. On the podcast cultivating her space, Dr. Dom and Terry Lomax create a space where black women can show up fully and be heard. I wholeheartedly think, you know, you hit 30. You shouldn't have to share one with anybody. Mm-hmm. From navigating friendships and healing to setting boundaries and prioritizing your mental health.
Starting point is 00:08:56 These are real honest conversations. We don't always get to have out loud. totally unreasonable with different parts of life, right? Like, oh, have all three meals and make sure you're mindful during all of them? Absolutely not. During one meal, I'm standing. I'm standing and handing my children food. Because healing, empowerment, and resilience aren't just ideas, their practices.
Starting point is 00:09:20 And this Mental Health Awareness Month, there's no better time to pour back into yourself. Listen to cultivating her space on the IHeart Radio app, Apple Podcast, or wherever you get your podcast. AI's media hype train has been fairly relentless since November 22 when ChatGBTGPT launched. And AI champions like OpenAI CEO Sam Altman have proven all too willing to grease its wheels, making bold promises about what AI could do and how today's problems are so easily overcome. It's also helped that the tech media has largely accepted these promises without asking basic questions like how and when will it do this stuff and can it do this stuff.
Starting point is 00:10:03 don't believe me. Go and look at any interview with Sam Altman from the last few years. Watch any of them. In fact, just look at any AI figurehead getting interviewed and count the amount of times they've actually received any pushback or been asked to elaborate on any specific issue. It's actually very rare. Let me play you one of the few times that anyone's actually interrogated an AI person,
Starting point is 00:10:26 specifically Joanna Stern of the Wall Street Journal, who you might remember from the Vision Pro episode of Better Offline. She interviewed OpenAI's chief technology officer, Mira Murati, about SORA, which is OpenAI's video-based version of ChatGPT, where you can theoretically ask it to generate videos. Just to be clear, it's unreleased and unclear whether it'll ever actually get released, and the videos look good at first, then they look really weird. But just listen to this particular question. It's Joanna asking Mira, the CTO of OpenAI, an $80 billion AI company, hey, did you train on YouTube? What data was used to train SORA? We used publicly available data and licensed data.
Starting point is 00:11:08 So videos on YouTube? Now, I encourage you to go and look up this clip, because at this point, Marathi makes the strangest face I've ever seen in a tech interview. I'm actually not sure about that. Okay, videos from Facebook, Instagram? You know, if they were publicly available, available yet publicly available to use,
Starting point is 00:11:36 there might be the data, but I'm not sure. I'm not confident about it. What about Shutterstock? I know you guys have a deal with them. I'm just not going to go into the details of the data that was used, but it was publicly available or licensed data. The remarkable part about this interview is that it's a relatively, simply simple question, you as the CTO of an $80 billion AI company.
Starting point is 00:12:03 What training data did you use to train your model? Did you use YouTube? It's a yes or no question, Mira. Mirror, answer the bloody question, Mira. All right, right. The answer is, of course, the OpenAI likely trained its video generating model SORA on YouTube videos, which might be why they're yet to launch it. And the videos generated by SORA also feature
Starting point is 00:12:27 some remarkably similar images to say, SpongeBob Squarepants, and I wouldn't be surprised if they carry with them multiple weird biases about race and gender that we'll see in the future. But also, when you watch these videos, much like most generative AI content, there's something a bit off about them.
Starting point is 00:12:46 In the Wall Street Journal's interview, you get to see some of the prompts that were used and some of the videos that came out, and you see crazy things happening, like a robot completely changing shape as it turns, cars disappearing and appearing behind the robot. It's not very good. It seems cool at first.
Starting point is 00:13:04 If you squint really hard, it looks real, but there's always something off. And that's because, as I've said before, these models don't know anything. They don't know what a robot looks. They can make a really good guess, though. Anyway, Stern's interview with Murati of Open AI
Starting point is 00:13:19 is a great example of how the entire AI falls apart at the slightest touch, because it's fundamentally flawed and not actually able to deliver the society-defining promises that Sam Mortman and the venture capital sect would have you believe. In a year and a half, despite billions of dollars of investment, despite every major media outlet claiming otherwise, generative artificial intelligence has proven itself incapable
Starting point is 00:13:43 of replacing or even meaningfully enhancing human work. And the thing is, all of these problems I'm talking about with generative AI, all of these hallucinations, all of these weird artifacts that are popping, up throughout these videos, the weird mistakes that the texts that are popped out by Chat GPT have. All of these problems are problems that aren't necessarily just technological. They're physics, they're mathematics. These aren't things you can just outrun.
Starting point is 00:14:13 And I believe that there are four intractable problems that will stop generative AI from progressing much further than it is today. The first is, of course, its energy demands, the massive amounts of power it requires. The second are its computational demands. The amount of compute power it requires to even crunch the simplest things out of chat GPT. Its hallucinations, the authoritative failures it makes when it spits out nonsense or creates a human hand with 18 fingers, and of course the fact that these large language models have an insatiable hunger for more training data. Now let me break that down.
Starting point is 00:14:52 Large language models are extremely technologically and environmentally demanding. The New Yorker reported in March 2024 that ChatGPT uses more than half a million kilowatt hours of electricity to respond to the 200 million requests it receives in a day, or 17,000 times the amount that the average American household uses in a day, and others have suggested it might be as high as 33,000 households worth. Generative AI models demand specialist chips called graphics processing. units, typically a souped-up version of the technology used to drive the graphics in a gaming console, albeit at a much higher cost, with each one costing tens of thousands of dollars each.
Starting point is 00:15:33 They do this because large language models like ChatGPT are highly computationally intensive. I'm going to break that down, don't worry. When you ask ChatGPT a question, it tokenizes it, breaking it down into smaller parts for the model to understand. It then feeds these tokens into various mechanisms, that help it understand the meaning of the thing you asked it to do. Based on the parameters that it learned in training, chat GPT generates a response by predicting the most likely sequence of things that you might want it to do, an answer to a question, an image, so on and so forth.
Starting point is 00:16:07 Each one of these steps is extremely demanding. Processing hundreds of billions of these parameters learned patterns from ingesting training data, such as how the English language works or what a dog looks like, to produce even the simplest thing. Training these models is equally intensive, requiring chat GPT to process massive amounts of data, another problem I'll get to in a bit, adjusting those hundreds of billions of parameters and developing new ones based on what the data says, as it quote-unquote learns more, though as we're clear,
Starting point is 00:16:37 chap GPT doesn't learn anything, it just makes new parameters to read things. A model like chat GPT grows, making it more complex, which in turn requires more data to train on and more compute power to both ingest the data, create more parameters, and turn it into something resembling an answer. And because it doesn't know anything, it's suggesting the most likely to be correct answer, which leads it to hallucinating incorrect things that, based on probability, kind of seem like the right thing to say.
Starting point is 00:17:06 These hallucinations are the dirty little secret of generative AI and are impossible to avoid thanks to the fact that every single thing these models say is a mathematical equation rather than any kind of intellectual exercise. If you ask chat GPT how many days they're on a week, it doesn't know that there are seven days, but it's been trained on patterns of language and generates a result based on those patterns,
Starting point is 00:17:28 which at times can be correct and can also be wrong. There's no way of fixing this problem. You can mitigate it, you can make it less likely it will mess up, but hallucinations will happen, because there is no consciousness. It is not learning anything. this thing has no knowledge.
Starting point is 00:17:48 More computing power would allow it more parameters to give it more rules so that a generative AI will be more likely to give a correct answer, but there's no eliminating them, and doing so may require more computing power than actually exists or is possible. Without an AI of consciousness, an impossible dream known as average generalized intelligence
Starting point is 00:18:08 that Sam Altman would have you believe is imminent, there's really no solving hallucinations. When you answer questions, using probability, you're always going to have mistakes because you're not actually answering them using knowledge, intellect or experience. You're using dice rolls. It's a bloody game of Dungeons and Dragons. We turned in Carter into Dungeons and Dragons. Anyway, a newly published paper by Tepo Fellin and Matthias Holwegg of the University of Oxford degrees, found in that large language models like ChatGPT are incapable of generating new knowledge.
Starting point is 00:18:41 It's a remarkably in-depth rundown of the fundamental differences between a large language model and a human brain, and it combines both psychological and mathematical research, going back to child psychology as well, the basic building blocks of how we consume and learn things and how we make decisions as a result. The paper, titled Theory is All You Need, AI, Human Cognition, and Decision Making, argues that AI's data and prediction-based orientation is an incomplete view of human cognition. and that the forward thinking theorizing of the human mind in layman's terms, the mess of the information we've learned over our lives, our experiences, and our ability to look forward and consider the future,
Starting point is 00:19:23 is just fundamentally different to a model that predicts things only based off of past data. Think of it like this. If you've read a book and you might think about writing a new book based on those ideas, you're not remembering every part of the book. You don't have a perfect memory. and you're also constantly thinking about things as your day goes on. The human brain is a goddamn mess. Generative AI is in some level stuck in amber.
Starting point is 00:19:50 Though the billions of parameters might change, the data never does. The way it consumes the data may be, but the data doesn't change. In essence, generative AI is held back by the fact that it can't consider the future and is actually permanently mired in the data of the past. Their largest problem might be a far simpler one, a far sillier and a kind of an ironic one. There might not be enough data for these bloody things to actually train on. Another podcast from some SNL, late-night comedy guide, not quite. Unhumor me with Robert Smygel and friends.
Starting point is 00:20:35 Me and hilarious guests from Jim Gaffigan to Bob Odenkirk to David Letterman, help make you funnier. This week, my guest, SNL's Mikey Day and head writer Streeter Seidel, help an a cappella band with their between songs banter. There's the worst singer in the group. The worst? Yeah. Me.
Starting point is 00:20:53 Is there anything to the idea that because you're from Harvard, you only got in because your parents made a huge donation. The group. The yard birds, right? That's the name. The Harvard yard, but they're open to change. Do you have a name suggestion? We're open.
Starting point is 00:21:08 Since you guys are middle aged. One erection. Listen to here. Humor me with Robert Smigel and Friends on the IHeart Radio app, Apple Podcasts, or wherever you get your podcast. Humor me. I need some jokes to make me seem funny. Run a business and not thinking about podcasting, think again. More Americans listen to podcasts than ads supported streaming music from Spotify and Pandora. And as the number one podcaster, IHeart's twice as large as the next two combined.
Starting point is 00:21:40 So whatever your customers listen to, they'll hear your message. Plus, only IHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business. Think IHeart. Streaming, radio, and podcasting. Let us show you at iHeartadvertising.com. That's iHeartadvertising.com. Agency, the ability to know that we're the experts in our own body.
Starting point is 00:22:02 On the podcast cultivating her space, Dr. Dom and Terry Lomax create a space where black women can show up fully and be heard. I wholeheartedly think, you know, you hit 30, you shouldn't have to share one with anybody. Mm-hmm. From navigating friendships and healing to setting boundaries and prioritizing your mental health. These are real honest conversations. We don't always get to have out loud. Totally unreasonable with different parts of life, right? Like, oh, have all three meals and make sure you're mindful during all of them?
Starting point is 00:22:34 Absolutely not. During one meal, I'm standing. I'm standing and handing my children food. Because healing, empowerment, and resists. Resilience aren't just ideas. They're practices. And this Mental Health Awareness Month, there's no better time to pour back into yourself.
Starting point is 00:22:50 Listen to cultivating her space on the IHeart Radio app, Apple Podcasts, or wherever you get your podcast. While the internet may at times feel limitless, a researcher recently told the Wall Street Journal, the only a tenth of the most commonly used web dataset, the Common Croll, are freely available, or 250 billion page dump of the web's information, is actually of high enough quality data
Starting point is 00:23:18 for large language models like chat GPT to actually train on. Putting aside the fact that I can't find a single definition of what high quality actually means, the researcher, Pat Blow-Villobos suggested that the next version of chat GPT would require more than five times the amount of data it took to train its previous version, GPT4. The new one is called GPT5, by the way.
Starting point is 00:23:41 and other researchers have suggested that AI companies are going to run out of training data in the next two years. Now, that sounds dire, but don't worry they've come up with a very funny and extremely stupid idea to fix it. One specifically posed by the Wall Street Journal is that the AI companies are going to create their own synthetic data to train their models, a computer science version of inbreeding the researcher Jason Sadowski calls Habsburg AI. This is, of course, an absolutely terrible idea. A research paper from last year found that feeding model-generated data into models to train them creates something called model collapse, a degenerative learning process, where models start forgetting improbable events over time as the model becomes poison with its own projection of reality.
Starting point is 00:24:29 The paper, called the Curse of Recursion, Training on Generated Data makes models forget, highlights an example where feeding a generative of AI its own data eventually destroys its ability, to answer questions, and within nine generations, one answered a simple prompt about architecture with an insane screed about jackrabbits full of at symbols and weird characters. So, not to worry again, the tech overlords have come up with a great idea to fix this problem. Their common retort to the problem of synthetic data is that you could use another generative AI to monitor the synthetic data being fed into a model to make sure it's right. At this point, I'd like to get slightly angry.
Starting point is 00:25:10 Are you kidding me? Are you fucking kidding me? You're saying that the way to make sure the data generated by an unreliable generative AI is to use another generative AI, one with the same goddamn problems, which also hallucinates information that knows nothing. You're going to use that AI to monitor whether the data that is created by an AI is any good. Are you completely insane? Are you insane?
Starting point is 00:25:33 You're going to feed the crap from the crap machine into another crap machine. to make it not make crap? Why am I reading journalists credulously printing this ridiculous solution in the New York God damn times? Every time, every time these bubbles are inflated because tech executives are able to get their half-assed, half-baked solutions parroted by reporters who should know better. You don't have to give them the better affair of the goddamn fucking doubt. This is how we got the bloody matter of us.
Starting point is 00:26:00 Pardon me. I've calmed down now. Anyway, anyway, if you're worried about model collapse, you're already too late, as these models are likely already being fed their own data. You see, these models are trained on the web, as I previously told you, and they're desperate. They need data. They need more stuff. They need more stuff to ingest so they can spit out more stuff. The problem is that these machines are purpose-built to make a lot of content. And so the web's already being filled with Generative AI.
Starting point is 00:26:33 Generative AI is already spamming the internet. A report from 404 Media from last week said that Google Books has already started to index several different works that were potentially written by AI, featuring the hallmark generic writing tropes of these models. 404 Media also reports that the same thing is happening over at Google Scholar, their index of scientific papers, with 115 different articles featuring the phrase, as of my last knowledge update, a specific phrase spat out by generative models. This is really bad, by the way, and this is only going to get worse.
Starting point is 00:27:08 When you have an internet economy that is built so that the people that can put the most out there will probably get the most traffic, they're going to use these tools. These tools are great for that. If you don't give a rat fuck about the quality, this is the best thing in the world for you. And that's the thing. This is a problem both created and caused by these models. You see, the other dirty little secret of generative AI is that these models unashamedly, plagiarized the entire web, needing outlets like the New York Times and authors like John Grisham
Starting point is 00:27:39 to sue Open AI for plagiarism. While OpenAI won't reveal exactly what their training data is, the New York Times was able to successfully make ChatGPT, reproduce content from the newspaper, and the company has repeatedly said that it trains on publicly available data from the internet, which will naturally include things like Google Scholar and Google Books. The Times also reports that OpenAI has become so desperate for data that they've used their whisper tool to transcribe YouTube videos into text to feed into chat GPT's training data.
Starting point is 00:28:11 Pretty sure that's plagiarism, but who am I to tell you? And as the web gets increasingly pumped full of this generative content, these models are going to just start eating their own swill, slowly corrupting themselves in a kind of ironic death. According to Zakar Shumulov,
Starting point is 00:28:30 one of the authors of the model collapsed paper at the University of Cambridge, the unique problem that synthetic data creates is that it lacks human errors. Human-made training data, by the nature of it being written by a human, includes errors and imperfections, and models need to be robust to such errors. So what do we do if models are trained off of content created without them? Do we introduce the errors ourselves? How many errors are there? How do we introduce them all? And indeed, what are the errors? What do they look like? Do we even know, are we conscious of the errors in the human language that make us human? The models aren't. Well, maybe they are. It's kind of unclear. It's kind of tough to express how deeply dangerous
Starting point is 00:29:12 the synthetic data idea is for AI. Models like ChatGBTGBT and Claude are deeply dependent on training data to improve their outputs, and their very existence is actively impeding the creation of the very thing they need to survive. While publishers like Axel Springer have cut deals to license their company's data to chat GPT for training purposes, this money isn't flowing to the writers that create the content, that Open AI and Anthropic need to grow their models much further. In fact, I don't think you're going to see more journalists get hired as a result of these deals, which kind of makes them a little bit stupid.
Starting point is 00:29:48 This puts AI companies in a kind of Kafkaesque bind, where they can't really improve a tool for automating the creation of content without human beings creating more content than they've ever created. it before. Just as said tool actively crowds out human-made data, it's all a little silly. The solution to these problems, if you ask OpenAI's Sam Altman, is always more money and power, which is why the information reports he is trying to convince OpenAI investor Microsoft to build him, and I'm not kidding, an $100 billion supercomputer called Stargate. This massive series of interconnected machines will require entirely new ways to
Starting point is 00:30:28 mountain cool processing units and is entirely contingent on OpenAI's ability to meaningfully improve CHETGPT, something Sam Mortman claims isn't possible without more computing power. To be clear, OpenAI already failed to build a more efficient model, dubbed Arachis, which ended up getting mothballed because it wasn't more efficient. It's also important to note that every major cloud company now has inextricably tied themselves to the generative AI movement. Google and Amazon have invested billions into ChatGPT competitor Anthropic, and both claim to be Anthropics' primary cloud provider, though isn't really obvious which one is. In doing so, they've guaranteed, according to a source of mine,
Starting point is 00:31:17 about $750 million a year of revenue for Google's cloud and $800 million a year of revenue. for Amazon Web Services, the cloud service from Amazon, by mandating that Anthropic uses their services to power their clawed model. This is similar to the $13 billion investment that Microsoft gave OpenAI last year, most of which was made up of credits for Microsoft's your cloud, and I somehow doubt that Microsoft is going to be the noble party that goes on their earnings and says, well, we don't want to count the credits that we gave Open AI. We want to be fair.
Starting point is 00:31:51 They're going to match that shit right back into their revenue. kind of a con. Kind of makes me angry when I think about it too. Anyway, let me just put that aside. I'm not going to get pissed off again. Look, I'm surprised more people on really upset about this very incestuous relationship between big tech and this supposedly independent generative AI movement. Microsoft, Google, and Amazon have effectively handed cash to one or two companies
Starting point is 00:32:17 that will eventually hand the cash back to them in exchange for cloud services that are necessary to make their companies work. and all three big tech firms are spending billions to expand their data center operations to capture this theoretical demand from generative AI. Every penny that OpenAI or Anthropic makes will now flow back to one of three big tech firms, even more so in the case of OpenAI because Microsoft's investment entitles Microsoft to a share of any future profits from OpenAI and ChatGPT. Yeah, it doesn't even really matter if they make one,
Starting point is 00:32:52 because Big Tech wins either way. Anthropic has to use Google Cloud and Amazon Web Services. OpenAI has to use Microsoft's Azure Cloud, and Microsoft is actively selling OpenAI's models to their Azure Cloud customers, and every time somebody uses OpenAI's models, that model is being run on Azure Cloud, generating revenue for Microsoft.
Starting point is 00:33:14 This is the rot economy in action, by the way. Big Tech has funded its biggest customers for their next growth revenue stream, justifying this massive expansion of their data center operations because AI is the future. And they're telegraphing growth to these brainless drones in the market who will buy anything, who never think too hard about what they're actually investing in. AI is this big, sexy, exciting and theoretically powerful way to centralize labor. And it's innovative sounding enough that it allows people to dream big about how it might change their lives
Starting point is 00:33:48 and how it might help them not pay real people to do shit. Yeah, here's the biggest worry I have. Here's the real pickle. Here's the thing that keeps me up at night. None of these companies seem to have appeared to consider something. What if Generative AI can't actually do any of the things they're excited about? What if Generative AI's content, as you probably see from anything chat GPT spits out, isn't really good enough?
Starting point is 00:34:19 Hey, is anyone checked if anyone's actually using these tools, if they're helpful to anyone? Is this actually replacing anyone's work? Huh. That's a bit worrying, mate. I didn't think about that before. Just kidding, I've been thinking about it for months. Look, here's the thing. I think that the big problem here is that Sam Altman and his cronies
Starting point is 00:34:44 have allowed the media, the markets, and big tech to fill in the gaps of their specious messaging. They've allowed everybody to think that open AI can do whatever anyone dreams. Yeah, I don't think that Generative AI can do much more than it is today. And also, from what I've seen, none of these Generative AI companies actually make a profit. And with each new model, they become less profitable. And I don't see that changing in the future.
Starting point is 00:35:17 And so I've dug in a little more, looking under the hood, all the demand that's spurring Microsoft Google and Amazon's Data Center operations might not have to have. actually be there. My friends, I think we're in the next tech bubble. And in the next episode, I'm going to walk you through how I think it might pop. Thank you for listening to Better Offline. The editor and composer of the Better Offline theme song is Mattosowski. You can check out more of his music and audio projects at Mattosowski.com.
Starting point is 00:36:00 You can email me at easy at betteroffline.com or check out Better Offline.com to find my newsletter. this podcast. Thank you so much for listening. Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website, coolzonemedia.com, or check us out on the IHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Another podcast from some SNL late-night comedy guy, not quite. Unhumor me with Robert Smigel and Friends,
Starting point is 00:36:46 me and hilarious guests from Bob Odenkirk to David Letterman help make you funnier. This week, my guest, SNL's Mikey Day and head writer Streeter Seidel help an a cappella band with their between songs banter. Where does your group perform? We do some retirement homes. Those people are starving for banter. Listen to humor me with Robert Smigel and friends on the IHeart Radio app, Apple Podcasts,
Starting point is 00:37:08 or wherever you get your podcasts. Your 20s can be so exciting, but they can also be really overwhelming, confusing, and honestly, just kind of lonely. May is Mental Health Awareness Month, and the psychology of your 20s is breaking down the science behind the biggest roadblocks we face. I was six years into my career, the 80-hour weeks, and just the first one in, the last one out, and I ended up burning out. There was a large chunk of my 20s that I, like, was just so wanting to, like, be out of that phase out of my skin.
Starting point is 00:37:38 And I just, like, really regret not living in the present more. You don't need to have everything figured out right now. You just need to understand yourself a little bit better. Listen to the psychology of your 20s on the IHeartRadio app, Apple Podcasts, or wherever you get your podcasts. This is Saigon, the story of my family and of the country that shaped us. From IHeart Podcasts, Saigon. You don't think I'm serious about a free Vietnam? One city, a divided country, and the war that tore America apart.
Starting point is 00:38:08 This is for Vietnam. They're pouring patriots all over here. Freedom for Vietnam! There's a fire coming to this country and it's going to burn out everything. Listen to Saigon on the IHeart Radio app, Apple Podcasts, or wherever you get your podcast. This is an I-Heart podcast Guaranteed Human

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.