The Chaser Report - The T In ChatGPT | Charles Firth

Episode Date: September 4, 2024

Charles Firth is back! Which can only mean one thing — he's talking about ChatGPT. In particular, Charles shares with Andrew what the T in ChatGPT stands for, and why it is so existentially horrifying. Good to have you back Charles. Hosted on Acast. See acast.com/privacy for more information.

The Chaser Report is recorded on Gatigal Land. Striving for mediocrity in a world of excellence, this is The Chaser Report. Hello and welcome to The Chaser Report with Dom and Charles, with Andrew. And not Dom. No, and not Dom, yes. And Charles is back. Hang on, I'm very confused. Who's back?

Why am I sitting here with you, Charles, and not Dominic Knight as I have been over the last however long? Oh, I don't know, probably because Dom's so lazy and just never turns up. Yeah, well, he didn't turn up to a lot of the episodes I did with him as well. Oh, really? But, yeah, I don't think anyone really noticed, though. No, I mean, I'm, because I can do a pretty good impression of Dom. So, in fact, I think the nine out of those tenish eps, it was just me talking to myself. Oh, really?

Oh, that's interesting. Yeah. Yeah, and then sort of, you know, pretending to be dumb in reply like this. Yeah, so, yeah, here, Germanic now. Oh, my God. I can see how everyone was fooled by the amazing impression. Yeah. It's pretty close, isn't it?

So how do the listeners know that you're not impersonating me? Oh, well, my impression of you is very different, Charles. You know, when I do the episodes with Charles Firth, they go, they go, Hello, everybody, it's here, you're here with Charles Firth. And my special companion, Andrew Hanson. So that's how they go, he's a grave infringement of copyright. What are you saying? What are you saying?

That's your voice, isn't it? No, my voice is, ah, my voice is this. Ah, fuck, oh, shit. Oh, that's right. It is. It actually is to, I'm keeping it very, I was actually writing a little thing about you the other day, which I'm not going to tell you anything more about it. It may never see the light of day. But that was one of the points I made when I was describing what it was like to have a conversation with you.

It's not a law suit, is it? You're not writing something for a law set. I can't tell you. It might be. It might be. It might be. But there's a lot of, ah, fuck. Oh, fuck.

Oh, I'm got to get it. Yeah. Okay. Well, so we're back. Today we're going to talk about chat GPT. And I'm going to explain what the T means in chat GPT. But before we do, I just want to tell you a little bit about how much people have been clamoring for me to return to this podcast. Well, you won't have too much to say on that topic, will you, Charles?

I see, you've got a blank page. I was flooded. I was flooded with nice, nice, nice. from people all around the world wanting clamoring for me to come back. Oh, well, you flooded, were you? Yeah, flooded. Right.

Why don't you tell us about the flood after this advertisement then? Because aren't we supposed to throw to an ad before you have flood? I think we are. So stay tuned and we'll hear about this flood of clamoring people just after this.

I'm a tad suspicious. Thank you for your patience. Your call is important. Can't take being on hold anymore. FIS is 100% online, so you can make the switch in minutes. Mobile plans start at $15 a month. Certain conditions apply. Details at fizz.ca.

I don't have any specifics if you're any specifics. I just, like no specific people, no specific comments you'd like to single out in this flood of people who've missed you. The flood of people who've missed you. It's the thing about a flood is you sort of don't know specifics. Like I don't identify each. rain drop in a flood, do I? Oh, so you were just swept away on a river of emotion. Yeah, it's just a tonne, just a torrent of just people clamouring for me to come back.

So I have, just twist my arm, I'm back. It's all right. Thank God, tells. Okay. So, well, don't feel pressured then to read any of the correspondence that came flooding in. If you don't feel equipped to do that. No, exactly. With correspondence.

I mean, it would take up the rest of the year. Yeah, and you would require some correspondence, too. That's an obstacle. So, what we're going to talk about is chat-chip-thet. Now, do you know what T stands for in Chat-C-C-T, Andrew? You're talking about the first T or the second? It does have two. I think the first T is the end of the word chat.

I know that much. G-G-P-T. So before 2017, it was just. just GP, chat GP, right? Oh, was it, was it real? I didn't know it was even existed before 2007. I thought the whole thing was quite new. No, no, 2017. Oh, 2017.

Well, no, I mean, I don't think it actually existed in 2017 either, but if it did, it was no such thing, like it was, the tea part of chat GPT was invented in 2017, right? And then it was chat GP. That sounds more like a sort of thing, like a medical app or something, isn't it?

I don't, look, I don't know what the G or the P stand for, let alone the T. So you have to fill me in. But the T stands for Transformer, right? Okay. Oh, okay. And it is the most amazing. So we've talked about it on this podcast before, not with you, Andrew, but with Dom.

And the whole, the whole interesting thing about CETT, which regular listeners will understand is that essentially the whole way AI works is, it's just statistics, right? It's just a massive, massive, fuck-off huge spreadsheet, right, of statistical probabilities of what is the next. it's most likely word to come, right, in any sentence, right? So what it does is it ingests all of known creation and text that humanity has ever made, which is mainly on Reddit, right? And then it absorbs that. And then the main engine for AI is to just statistically work out

what is the most next, most likely word in any given sentence, given all the sort of stuff that it knows and it's in context, and then spits that out. So if you say to it, write me an essay about, I don't know, Australian literature, it will just statistically work out what that's supposed to be. And the more defined you become, it creates a statistic. You know, if it's about what did Elizabeth Jolly think of Australian literature, it will work out statistically what that article is likely to be, right?

And it just works it out word by word, right? Okay. So it must be, I mean, if it's the word that's most likely to come next, why isn't it more full of the word um i feel like statistically surely the next word is is usually um um yeah especially on that particular topic but no because no because it's reading wikipedia and read it and things like that's the yeah so that's where it gets absorbs all of its Australian literature knowledge yeah and and and I don't know yeah and also presumably you know conversations between primary schoolers who want to talk about Australian literature and you know

Oh, of course, yeah, with their mates. Yeah, local conversations in bars late at night when you're just discussing... Yeah, that's right. Elizabeth Jolly, as we do. I mean, we're always talking about Elizabeth Jolly. I was wondering what... Yes. What would she think?

And David Mouf. I mean, you know, like, let's just not... No, I'm always wondering. When I have a problem in life, if I'm at a crossroads, I always think to myself, what would David Maloof do? Yes, exactly. And then you asked Chat GPD. But previously,

what I've said is a sort of very vague understanding of what the next step in the process and the breakthrough that came that really made chat GPT work right so they had known this since about 2012 they'd known that that was a very powerful way of creating AI but they're actually getting far more sort of results on things like videos and image generation stuff like that because you know create me an image that looks like a photo of a nuclear power plant you know, for the front page of the Liberal Party. Liberal Party's policy. Genius plan to save the environment.

It's their environmental policy. And arts and their arts policy. Nuclear power. Nuclear reactors. So, and that was sort of easier statistically. What the tea did was, and the way I've described it before on this podcast, is create a sort of semiotic understanding. of what, you know, how grammar works.

And so therefore it didn't just, you know, like statistically, it still made sense along, you know, it held in, in its brain a sort of sense of what words relate to, you know, what's more important, what's less important inside a sentence, right? And the right order to put in me in. Chat, GPT, can you re-explain in clearer language what Charles is telling you? No, that was my previous explanation, which was a bit shit, right? And I'm now going to tell you, because I've just read this fascinating article in a New Yorker,

which, understandably, you might want to just go and read yourself rather than have me explain it to you. Yeah, I'm reading it right now, actually. I've taken my headphones off. I'm just reading the article so that I can get to the end quickly. No, no. So the point is what actually, the breakthrough, the transformer breakthrough, was to go, actually, attention is all you need. You don't need a semiotic basis for.

for understanding how English grammar works to construct an English sentence. All you need to know from a computer AI perspective is which word is statistically more important in each sentence than the rest of them, right? And so say the word knife appears in a sentence, right? Statistically, that's right. That is likely to be a fairly important word to the meaning of that sentence, right? Well, yeah, especially if it's a serial killer saying the sentence or a chef. I suppose.

Yeah, exactly, or anything, because it's such a rare word. You would only ever use it in the sentence if it was really the point of the sentence. That's true. That's true. Unless you were describing, say, the entire contents of your kitchen in a list in that centre, then maybe knife wouldn't be so important. Is that right? Yes, but statistically, that's very, that's going to be on the bottom of the curve.

There's going to be a distribution of things. I'm always describing the entire contents of my kitchen, huh? No, yeah. I was just having that conversation this morning. Right. But if knife, okay, so knife, you've got knives, that's an important word. The is very unlikely to be particularly important in a sentence. Like, it'll probably be in a sentence.

But the amount of attention that it requires to construct the meaning of that sentence is not a huge. It's probably more important than an or, you know, R or something. But, you know, like, it's not going to be the main point of the sentence, right? Anyway, so what they worked out was you don't need to teach the AI some sort of understanding of how English grammar work. to get it to know how to write English. What you need to do is just attention is all you need. All you need is a statistical analysis that you can then put in a big spreadsheet

and work out in every sentence what the relative attention that you should be paying to each word is. And then the computer will teach itself how to construct English sentences that make sense. And not only that, it will teach itself how to make Japanese sentences that make sense and German and Latin and things. suddenly, like, and you don't have to, you don't have to sort of modify it based on each

language or type of gram that's going on. It does that itself simply through, it's called matrix multiplication, but it's essentially like looking at that attention thing and then multiplying it by itself and multiply it by itself. And so in every sentence that it's doing, it's looking up this graph of, well, what is the, you know, what's the word that I've just said? What's the most likely? And what's, what's the relative important? of that word within the thing, and it creates its own sense of what grammar should be, which is why, as soon as they invented this T, this transformer, which is what it's called, suddenly the chatbots were able to work, not just in English, but in any language, right?

And so in 2017, they were writing this paper, right, and they were going, oh, this is a pretty cool invention. Why don't we just upload a bit of Wikipedia and see whether this will work, right? And so they had like a couple of days to train their little AI model that they were doing. And so it only got through half of Wikipedia in that time. Me too. Analyzing the sort of statistical thing. But then they went, at the end of that, they went, okay, we're going to call this device a transformer.

Can you create seven articles that are plausible Wikipedia articles, having learned Wikipedia, about transformer, right, under the entry Transformer. And it created, to their incredible surprise, it created seven different definitions of the word article, right, and entries under the word transformer, right, about various things. Like one was a 1980s punk rock Japanese band called Transformer, which then broke up in the mid-1990s and went into sort of mystic blues or something like that for a while. And it was absolutely plausible, absolute bullshit, but completely plausible. And there were seven different articles like that. And they had cracked how to teach an AI how to learn grammar ride. But it wasn't through any understanding.

It was through it learned itself how to teach itself grammar. Not on the basis of here's an object, you know, here's a noun, here's an adjective. They have to go together. It said, go away and work it out for yourself. And that's how it all works. But they don't really know why it works so well. But it does. And because it's just literally based on statistics, right?

And the relative attention that you should be paying to each word in every sentence that's ever existed, it's very scalable. It means that if you throw more computing power at it and train it on everything, it gets better and better at being intelligent, right? But they don't know why. They have no understanding of what's going on there. And it's not based on any human understanding of semiotics or. or how we know how to construct sentences. It's just based on its own fucking thing. And it's based on a very thin, tiny little idea

that now just completely has transformed the entire world of AI and is going to transform our entire universe. And this happened like only fucking seven years ago. It's fucking incredible. It is incredible and mind-blowing and also immensely worrying and depressing for me personally. Thank you for your patience. Your call is important.

Can't take being on hold anymore. FIS is 100% online, so you can make the switch in minutes. Mobile plans start at $15 a month. Certain conditions apply. Details at FIS.ca. The Chaser Report. More news. Less often.

No, that is extraordinary. But is it fair to say maybe we are a bit the same as that in a way? Yes. Yes. When we grow up, you know, we don't really have a theoretical understanding of grammar. You know, when human children, they kind of, it's from what they've heard, it's from what they've heard other people say, and they echo little phrases and they echo, I notice it in my kids, they talk in certain shapes and in certain tones and that remind me of myself or Jess, because, you know, maybe they're not too far away from having a little spreadsheet in their brains in a way.

Well, I think that that's what people are understanding. And a perfect example of that is, you know how every language has a different order for ordering your adjectives? So if you say the big black brown, the big black brown bear, right? You know to sort of go size, then, you know, color, then the actual proper noun for the bear or whatever.

And then bloody French, there were French people put the adjectives after the noun. You have to wait. You have to wait to hear the description. And the Germans put their verbs right at the end, so you don't know what's going on. No, no idea. You're on a constant state of suspense being a German. Just hanging on there every word, wondering what's going to happen.

Yeah, yeah. Oh, you invade. Oh, okay. Yeah, oh, I wish you'd said that earlier in the sentence. We would have been prepared. She means it's an unfair advantage. I wonder how Polish constructs this in it. Do you think that's how they took them by surprise?

Other way around, yeah, other way around. The verb comes first. Yeah, big mistake. Never put the verb first. But, no, so the point is that you don't really clock that. You don't actually think about that at all as an English speaker until you start learning other languages and you realize, oh, this is not some sort of natural thing.

This is a sort of pre-coded order. And so what you're working off before you come to a sort of deeper understanding of it, which you've got to be taught, is you're right, a sort of statistical database of inputs that you've heard your parents and your colleagues and your friends at school say that actually orders your adjectives perfectly every time, which is exactly what ChatGPT is doing. So, yeah, you're right. Maybe.

So does this mean that ChatGPT, once we throw enough computer at it, is going to be sentient? Well, it doesn't sound like it. It's a bracelet. If it's just based on spreadsheets, I mean, it's close to sentience, isn't it? You just said your kids are basically spreadsheets. Oh, I see what you mean. Oh, I see what you mean. Well, it's kind of, yeah, yeah, but it's not exactly the same, I feel.

I feel that, you know, it means they don't, you can't look at the spreadsheet in the kids' head, or at least not yet, can you? No, but they're working on that too. No, no, but they don't know how AI works either. It's not like, yeah, right. Like, this is the whole thing is, and this is why there's this real feeling amongst people who are, you know, involved in AI, that it's incredibly dangerous. Oh, yeah, because they don't know how it works and what it's going to do next with this spreadsheet.

Because what they've done is, they've said, go away and work it out for yourself. And the fucking computer has. Look, can't we just change the permissions on the spreadsheet? Just, you know, just make sure the computer's not shared into that spreadsheet anymore. And then we'll be safe. You know, put a password on the bloody thing. Yeah, but when, like, don't you think that the world is too lazy for that to be the solution? Because, like, by the time that becomes a problem, like, oh, yeah, it's about to launch nuclear strike against us, right?

Yeah. It'll be like, oh, well, but if we disable AI, then Facebook won't work, and I won't get all the Instagram. Oh, that's true. And, yeah, and the Woolworth's refund chatbot won't work anymore, which would be a huge problem. We won't be able to ring up and talk to a computer about our... Yeah, no, and we're not going to want that, are we, as a human, No, we, well, I know from looking at what we've done so far, we're definitely going to let it do whatever it wants, even if it means nuking ourselves, aren't we? And I think, you know, in some ways, maybe Dutton's nuclear policy is based on AI.

Maybe he, as I've heard, he's a bit lazy. Maybe he just typed in, give me a good climate policy. And they went, and it just came back with that. Nuclear bombs. Solve climate change, nuclear bombs. Well, it would in a way, wouldn't it? because we wouldn't have to worry about the problem. Yeah.

I mean, only the cockroaches would be having to worry about it. And now they're probably Liberal Party members. So they're advantage. Clearly. Yeah. Okay, well, anyway, that's my explanation. Yeah, well, look, you can choose if you're listening to whether to believe Charles's explanation or do your... Well, you can read the New Yorker article, which I must admit is longer than this explanation.

But I think there's a pretty good fucking summary. Yeah. Oh, bloody hell. Yeah. Okay. Well, look, why don't we, you know, why do we come back with our own intelligences next time? And we'll solve all the problems that have resulted from this fucking mess that ChatGPT has created for us all. What I'll do next time is I'll get AI to process all the swamping, like, clamouring of the listeners for my return.

Oh, if you're a return. That's a good idea. And let it give a summary of that. clamoring. Yeah, that'd be lovely. That'd be really moving and soulful, like all those AI texts usually are. Yeah. I'm a bit disappointed, Andrew, that you didn't get me something for my return.

Well, look, I was, honestly, I was just that I was so heartbroken for me being away. The grief was so immeasurable while you weren't on the podcast. I got a lot of that actually in the... Oh, I couldn't get out of bed. I actually could not get out of bed. I just lay there, you know, recording the podcast with Dom under the covers. Yes.

So thank God, thank God you're back. Well, look, if you do want to send, what's the email address, Charles? If you want to send, you know, some sort of, you know, wonderful encomiums, Charles's way for having come back, where can people write to you? Podcast at chaser.com.com. Or just contact David Maloof and he'll pass it on. Yes. Or Elizabeth Jolly.

Or Elizabeth Jolly. You could send it to her. She's no longer with us, but that doesn't make much difference, I think, because you're not going to get any messages anyway. Fuck you. Fuck you. Well, I'm going on evidence. I'm like the AI spreadsheet.

I'm looking. There's nothing on the table. So statistically, the most likely thing to happen next is that there will be no messages arriving for you. Our gear is from Road, part of the Iconicless network. Fuck off, Andrew. Thank you for your patience. Your call is important.

Can't take being on hold anymore? FIS is 100% online, so you can make the switch in minutes. Mobile plans start at $15 a month. Certain conditions apply. Details at fizz.ca.