Sharp Tech with Ben Thompson - A Beginner’s Guide to ChatGPT: How It Works, Why Tech Twitter is Euphoric, and What the Next Challenges Look Like

Episode Date: December 5, 2022

The Internet-wide reaction to ChatGPT 3.5, how OpenAI has evolved and how its chat technology works, the UX challenge of contextualizing answers that are usually right but sometimes demonstrably wrong, plus some quick reactions to “The Twitter Files” on Friday night.

Hello and welcome back to another episode of Sharp Tech. I'm Andrew Sharp and on the other line, Ben Thompson. Ben, how you doing? Doing well, feeling invigorated. We've had a- Are you? Yes, we've had a thriving argument in the group chat about a top 25 list or top 100 list that as usual had Chris Middleton ranked far too low.

But you just waved the white flag in my assertions. May or may not have been so we could get this podcast started, but I'm feeling good. After another 10 minutes off mic here, yes, I'm ready to get the podcast started. And can we start with a congratulations? Because I feel like we have had a good two months and we'll probably have another good two or three months. And at that point, it'll be like a solid five-month success story. And then we're all going to be replaced by AI. And we'll go our separate ways.

What do you think? Well, it's actually perhaps more accurate. argument than you think. You're obviously referring to chat GPT, which was released last week. And what's really fascinating about this release is it's not necessarily, there is some new aspects to it, but it's not, the capability isn't particularly new. What it is, it's built on GPT3. They're calling GP3 3.5, but it's basically an extra bit of training on top of GPT3. It's the same, same, same underlying sort of model set, same size. And it's just the user interface is different.

And that extra bit of training that sort of shapes the sort of answers that come out, we can sort of get into the details in a little bit, is different. And it just goes to show this bit about how product matters more than technology. Like the way that people experience it and perceive something can be drastically different, even if the underlying technology is there. And the reason why, say, your timing is actually accurate is GPT4, is in development, is coming out sometime. No one's sure when. There's been rumors of early next year, but we'll see.

And that is supposed to be just an astronomical improvement over GPT3, the sort of underlying tech here. So, yeah, we may be out of a job sooner rather than later. Yeah, well, here for a good time, but not a long time. And that's okay. We're just going to keep doing it while we can here. I'll start with just some framing for anybody who has no idea what we're talking about. Late last week, OpenAI released the third version of Chat GPT. And for the past several days, people have been losing their minds at how impressive this

technology is. I appreciate you for calling out the interface because it is particularly legible for non-hardcore tech people. Like I spent the last two days just sort of screwing around like all day as I'm like watching football, watching soccer. It's a lot of fun and very easy for a normie to use. So that was part of the hype cycle here. And I'll read a note from what's that? It's a normie drink.

That's right. The sharp tech drinking game is back. I'll read a note from TechCrudge here. In case you haven't seen the buzz around chat GPT, it's basically an implementation of their new. GPD 3.5 natural language generation technology, but implemented in such a way that you just chat with it in a web browser as if you were slacking with a colleague or interacting with a customer support agent on a website. And so you can ask the AI to give you answers to questions you have, to write up scripts if you describe the scene you want, to write up little articles. I had them write a letter to

Apple demanding that Apple compensate people for thousands of lost AirPods every day. And the letter was pretty good. And so to give you a sense of like the way this has been received, here's a tweet from Chital Shah, a research engineer at Microsoft. He said chat GPT was dropped on us just a bit over 24 hours ago. It's like you wake up to the news of the first nuclear explosion. And you don't yet know what to think about it, but you know the world will never be the same again. So as far as the hype is concerned, do you have any like initial thoughts on just the sort of euphoric reactions we've seen over the last couple days? I mean, I think it's analogous to what happened with images sort of over the summer. I mean, one of the things that was interesting with that was first you had Dolly come out and it was sort of mind-blowing.

but no one really had access to it. And I think when Mid Journey comes out and anyone can access it via Discord, there is some aspect of being able to use it yourself that I think makes a huge, huge difference in appreciating and understanding this. Because you can understand like on a cognitive basis. Yeah, someone typed something in and they got something out. But it's just very different and hits very different when I typed this in and I got something out.

And I think that goes into this broader sort of user experience thing. I do think, to be clear, this is a really big deal. But it's actually really interesting on a meta level that a reason why it's a big deal is part and parcel of the accessibility of it. Like, we can get into how it works and why it's different than sort of GPT3. But I think the more important part is that the interface is super accessible, super easy understand. And it's open to everyone. Like it's not like this thing that, oh, a couple of journalists got access to or they posted some, you know, well, maybe they just picked out the best images. No, when I can do it and I can pull out an amazing image.

It's like, wow, this is amazing. And I think this is sort of the same thing for text. We talked, you know, when we talked about the image stuff, I'm like, well, the same thing's happening with text. And it probably just sort of flew right over your head, right? It's like, well, because once you've actually experienced it, it's totally different than broadly knowing that something might be out there doing XYZ. X, Y, Z, another drinking game. Yeah, okay. We can keep track of all our verbal tics as we go here.

The thing is, though, the photos, you never felt like they were actually that realistic. Like, every photo I looked at, it's like, okay, so this was generated by a computer. And they're not bad, but it sort of seemed like a party trick. Whereas seeing some of the text that's being written, that takes my breath away because it's not quite there yet, but you can see that we're like one or two steps away from being able to generate like full associated press level news recaps and all sorts of stuff that will be really, really difficult to tell apart from what's written by a human. Well, I don't know, are we? I mean, the, you know, what are the limitations here?

Maybe it's useful to get into how this works. First and foremost, the sort of broader training set is this corpus of data that is, as we talked about, is sort of drawn from the internet generally. And I think it's up to 2021 or something along those lines. So, number one, you're not writing an AP news story about something that just happened because it doesn't know about the new story that just happened. Number two, this sort of advancement beyond GPT or GPT provides the corpus and the core language model.

But this is applied something that's called reinforcement learning with human enforcement. So it's really interesting because it's almost, there's a little bit of AI training AI with humans sort of involved. So you get a number of prompts. And they actually just pulled prompts from people that worked with, there's a thing called GBT Playgrounds where you can have access to this and you can do it. It hasn't been broadly available. You had to pay, I think, to get access. And so, again, one of the big things here is just being broadly available for free to lots of people. But they would pull prompts and they had a set of like dedicated trained trainers that would go in and give a response to a prompt.

right? So that's sort of step one. It's called supervised training. These humans actually contributing the set set two is you run the model and you have humans ranking the results. So you get like five results. You say one's, you know, how accurate are the results. And what that is, that's called a reward model. It's saying like, like, this is how you know what's a good result and what isn't. Then you basically apply that second model to the first model. So you can do do it at scale. So it's generating the, sort of answers, then the other AI is ranking the answers. And for the record, I'm trying to help in that process because you can thumbs up or thumbs down every chat GPT answer. And I'm responding. Exactly. Well, but this is like the initial sort of release.

It uses this algorithm called Proximate Policy Opposition or something like that. Basically, it doesn't want to like wildly disrupt it, but it sort of just shifts it based on the different responses. What's interesting about that, though, at the question. court's still a language model, which means it actually doesn't know anything. All it knows is what word is likely to follow on from the word before. And so one of my favorite examples over the weekend was people were asking it to basically create like a virtual computer, right? And they're putting in like terminal commands. They're getting answers out and it's doing everything you would

expect. And it was funny because even on like, you know, like hacker news, just like, I think, no, I think there's actually a computer behind the scene. No, of course it's not a computer behind the scene. It's just scraped enough data off the internet to know that if you put in certain commands, like maybe it's from Stack Overflow or whatever, like, if you put in A, you're going to get B and it can sort of like simulate that. But then you get into this really philosophical understanding of like what actually is knowledge, right? Like to what extent is it knowing that if you have A, then it's going to be after that? And can kind of get large enough where, and this is the key thing, where computing to data has always been deterministic. You do one thing and you

get another thing out and you can do that. It's going to happen every single time. These models are probabilistic. It's if there is something that happened, it's probably going to be something else that that comes out after that. And that can become so accurate that it feels like the same thing, but it's still really fundamentally different. And so like the question is when it comes to an AP News article, there is an aspect of, well,

you have to know what happened. And there's almost a deterministic response to that. But where this could also maybe work more realistically is you put in something and then it adds all the pros around it. Because it knows the right pros that should go around an event, but it doesn't necessarily have knowledge. And the other weird thing about this is there's a problem that's called hallucination where it will just start making crap up. It sounds really authoritative and it sounds absolutely accurate, which is, which is, it's really striking. Which I could be guilty of sometimes. You know, it happens to the best of us.

It does. Confidently dead wrong. Yeah, but they're, I mean, just this is a very sort of esoteric example. But it just happened because I was, I was helping my daughter with, with some history homework over the weekend. And it was something about Thomas Hobbs, which, you know, I was a political science major. I love political theory. So I was, you know, I know this stuff pretty well. And so I, just out of a whim, I stuck in the open GPT.

And she was talking about Thomas Hobbs and separation of powers. Which, I don't know how familiar. Are you familiar with Thomas Hobbs? No, I'm not. I was not a political science major. Well, so Thomas Hobbs did not believe in the separation of powers. He believed in absolutism. He thought separation of powers was a recipe for civil war.

You have different power centers, XYZ. You should have absolute power in the hands of a monar. So I put in an open API. Did Thomas Hobbs believe in separation of powers? Yes. Thomas Hobbes believed in the concept of separation of powers. And his book, Leviathan did it. And it goes through this answer that that sounds 100%

Correct and is completely wrong. Another example you saw over the weekend was someone asked for something with citations, and it provided the answer, and then it listed the citations, and the citations were all formatted correctly and looked accurate. But if you actually looked up those citations, they were all made up. Nothing to do with it? Yeah. Were they purely invented?

Completely invented. Okay. Great. Which I say this not to criticize the model, but I think the thing that's going to, it just it's kind of mind bending that this is a type of computing that's completely different from the computing we've we've had to date. It's all statistical guess about what should be the next thing. And the amazing thing is the fact that that statistical guess is so often right. And so to highlight these things that are wrong is to almost marvel at the fact that they're actually

relatively rare and worth highlighting when they are wrong. So as we zoom out here, because I want to get into what is knowledge and some of the philosophical questions associated with all this. But just as far as the basics are concerned, we received a request to produce an explainer episode in light of all this hype. So the most basic question, what can you tell Normies about OpenAI, the organization that's behind all this? Like, what is this organization? Well, Open AI, a company co-founded by Elon Musk, actually. Oh, boy. Fingerprints everywhere. Good old Elon. It is interesting.

He's not with the organization anymore. He left several years ago. The main person in charge now was Sam Altman, who was the former head of Y Combinator. And it started out as a nonprofit that was going to protect us from the dangers of generalized AI. They were going to do this by sort of researching AI. It's sort of like the people that do gain of function research on viruses to protect us from viruses. It's like, I can kind of see your. point, but also you might be going about this.

It's good. It's getting pretty hairy. Yeah. And so it started out as this sort of nonprofit. And I actually, I think one of the more interesting shifts was it did become a capped profit company where investors can only get 100 X out their investment, poor investors. And the reason is pretty clear. Number one, needed a lot of money.

To build these models, as we've talked about, takes a lot of compute power. Tech models take more than images. One of the big challenges, it's just a matter of like the amount of stuff you have to have in memory. So you need these systems approaches to running these models, whereas the image stuff turns out can run on much, much less sort of computing capacity. And so you don't really see a competitive, like you do with images, stable diffusion being this sort of open source sort of alternative, not as clear that's going to happen to the same extent with texts. And so Open AI has been doing this, building this. They've done a lot of very, very ancient research. There's GPT, GPT2, GPT3.

There's variations in all of these. Again, this is a variation on GPT3. They did Dolly, Dolly 2. And they have a very interesting partnership with Microsoft where basically they get a lot of their compute for free. Microsoft's sort of bet is that we're going to pay for this substantial, very expensive sort of computing. But we will then have first dibs on it. And so, like, for example, Microsoft just released a new design application as Dolly built in.

And like that. And so that's going to be, and they're going to, you know, the cell computing capacity that has access to all this sort of stuff. So there's lots of interesting sort of bits and pieces about this. But if you had to place a bet on what's going to be the next dominant company to rival the Big Five, Open AI is a very, very good bet. They've been working on this for a long time. They're building these huge models. They have APIs to access this. And the, I think the thought is, and I think this is actually a good example of it, is what's holding AI,

back to the extent it's being held back is products. Everyone's focused on the underlying tech, but how do you actually surface this? And you can imagine companies building industry-specific, context-specific companies that leverage the open API, the Open AI API, say that 10 times fast, to build these applications. And this being a huge company that's very defensible, has a big moat. I mean, I think, you know, the questions around moats and AI is when I've been bouncing. back and forth with, like what companies are really going to have a dominant advantage here. One thing that's interesting is where's Google, right? Google has- Well, that's what I was wondering.

Because, like, you talk about the lack of competition. I was using this tool for the last couple of days, assuming that this is the sort of technology that every other big tech player is working on internally and will roll out in due course. But it sounds like that's not the case. No, Google absolutely has similar stuff. The difference is Google won't release it. And there's lots of reasons why this might be the case. I mean, definitely one of the challenges with this stuff is it's harnessing data all over the internet.

And some people don't like the implications of that and what that might say, particularly when it comes to questions about, you know, more sensitive topics where the historical record is very clear in one direction. People think the future should look different. And it's like, well, how do you make the future look different? And we should make the AI say different things. You can imagine this scenario, right? And Google, to some respects, by being a large organization with, you know, well-documented history of internal sort of political fights is handicapped in that regard. That's one.

Number two is Google has a business model challenge. The nature of search results where it lists a bunch of potential options and the user basically chooses the winner, that lends itself well to advertising, because some of those options can be ads and the user can sort of choose the ad. And like you don't pay for ads on Google just for showing up. You pay when the user clicks, which is great from an advertiser perspective because there's a very clear connection between the value we're getting and the actions of the user. Now, it can be a little crappy because Google just sticks a bunch of ads when you would have got the right result regardless. But it's a business model that works. And in this case, just giving an answer takes away the core sort of mechanic of that business model, where, you know, choose an ad or choose a result.

And when you choose the ad, we get paid, that doesn't really work here. Google has done things like the answer box. Like you'll ask some questions of Google. It'll just state an answer. And it's interesting because, number one, they mostly do that on results where they don't advertise or there's not, there's not many ads because it doesn't matter.

They may as well give you the result. Number two, that does have the hallucination problem sometimes too, where Google will show an answer? That's just completely wrong. And it's like, where did you come up with that from? But yes, I think that there's a good chance. This is a scenario where being a startup, having more flexibility, having a different business model is going to be an advantage. And amongst the startups in the space, open eyes by far the largest, by far the most sort of well, well capitalized.

They have this deal with Microsoft. They are well placed to sort of seize this opportunity. Yeah, well, it's going to be very interesting. I mean, you mentioned Google answers. That makes me smile because over the weekend as I was playing with some of this technology, I was thinking like this is sort of Yahoo answers, but several magnitudes less insane than Yahoo answers. And maybe that's where this is going to go. This is sort of a reliable resource that people can use.

There's this theory that OpenAI will be ad-free. and will be so good that it'll ultimately force Google to reduce its advertising as well in order to compete. Do you buy that at all? No, Google is not abandoning ads ever. I mean, but it's really interesting. It's a really good example of where established companies with established business models can be fundamentally constrained in a new era. I mean, like, open AI. It's what disruption is.

Yeah, no, exactly. Exactly. I mean, you know, ads are phenomenally profitable. The margins are huge. It's very hard to even structure a company to pursue any other sort of business model, not just internally. You're like, you don't know how to sell. You don't know how to like structure things. You don't want to do customer support.

Like the ads, you don't have to worry about any of that. And also your investors have expectations of what your margins should be of all these sorts of things. It's a real problem for Google in sort of pursuing any of this stuff. Whereas opening eye, they don't make any money. They're losing tons of money. And so that's an opportunity of flexibility. So if we get to this area where Open AI is more of a platform where you have tons and tons of companies building on top of Open AI.

And then they have to pay for every query or whatever it might be that, you know, they're basically paying a tax to Open AI to use their service. It's not a tax. They're using the service. But it's like Open Eye takes a skim of all their revenue. Like that that's fine because they're building a lower money. margin business on top of zero, which investors are, oh, great, you're now making money, right? That's different than Google being like, you're making tons of money. Why are you hurting your very profitable business to do something that's not going to be as

profitable? Yeah, and that's what I was going to ask. Is that sort of the game plan for Open AI? Will they just license this to everybody in all sorts of different industries and hope that that's the way they build their, you know, trillion dollar business? Yeah, I mean, that's, that's, that's. what they do now. You can license the open API and you pay on like a per per query basis or like a, you know, per X number of queries. And I'm sure they're going to iterate that and work on that. But it seems a very sort of straightforward and obvious way to sort of go about this, right? If the, the theory is to actually build these huge models and to have this capability is tremendously difficult. So it's like being the TSMC of AI in some regards, right? Why would you go and build another foundry? when TSMC is right there.

And sure, if you built your own, then you wouldn't have to pay on a per query basis. But the reality is the amount of investment would be so massive that you're not going to catch up. That's the theory anyway. I mean, again, one of the big questions and why the image stuff over the summer was such an earthquake, was that was like, well, what if you could build your own, right?

Like, what if it actually is available? And I think right now it seems like images are going to be more, broadly accessible and the GPU requirements are going to be lower than expected. But the tech stuff does seem to still be, you need really big systems. It's really more about memory than about processing capability. But I mean, we'll see it. This is one of the things that I have a hard time sort of wrapping my head around. Not right, my head's wrapped around it.

I understand what's going on. But it's unclear how it's going to play out in the long run. Well, it's funny because one of the things that I'm having trouble wrapping my head around, is just how quickly all of this is moving. Like we had the AI conversation in September, and coming out of that conversation, I was envisioning something like chat GPT, but thinking it was years away, not months away.

And so are you surprised at all by how quickly this is moving? It feels like every month there's another AI breakthrough, and it just feels like the timeline is being accelerated on some of this stuff. Yeah, I would say today, I'm not surprised. There was a huge amount of surprise over this summer, particularly the fact that there could be like an open source alternative to open AI. I think with the reason why it feels very excited right now and it's probably going to continue to feel this way is if you have the thesis that

what makes chat GPT so phenomenal is the product aspects of it, like the UX aspects of it, as opposed to necessarily the model aspects, it follows that once you develop the core capability, multiple products can be built on that core capability sort of in parallel, right? So there's a long sort of serial process in building up the capability. Once that's done, they can do all sorts of stuff on top of it. I mean, we talked about Cloudflare a couple episodes ago. And one of the reasons why they feel like such a high velocity companies, I mean, obviously they're working very hard and turning a lot of stuff out. But every product they built is built on the fundamental capability of them having base.

free and infinite bandwidth because they have these boxes and ISPs all over the world, right? That took many, many years to build. And so this is an example of building something over many, many years, developing that capability. Once that capability exists, you can manifest it in all kinds of different products and lots of different ways because it's just a software problem and a design problem at that point. And so I would bet we're going to see an acceleration, but that acceleration is more of a perception thing because the foundation's been built. And once the foundation has been built, you can do lots of stuff on top.

You know, but the real, I mean, the crazy thing is going to be when GPT4 comes out. I mean, I've freaked out because honestly, looking at some of the technology, like, it's very cool. It's also a little bit unsettling. So the idea that there's going to be another dramatic leap forward sometime in the next, like, two to three months, again, a little unsettling. But I look forward to checking it out. One of the things I don't know, and I'm very curious.

when GPT4 comes out is to what extent is there a separation between the underlying model and the overarching products? And what I mean is, is this going to be a situation where you have a product and then you just basically switch out the underlying model? Say, okay, now we're using GPD4 instead of GPD3. And it's a relatively uninterrupted experience, except the model, the product just got a million times better. On the other hand, it might be more like the TSM Foundry model, where once you have a chip design for a particular node, you have to kind of redesign the chip if you want to go

to a different node, right? And I think that that's unclear to me, which which way that's going, that's going to shift is going to be sort of plug in place, swap one out for another? Or is it the case that the products are actually fairly intimately tied to the underlying model? And so you have to rebuild the product every time that that sort of a new model comes out. Interesting. Okay, so an exercise here. Can we put on our hater hat? Imagine it's me talking about Chris Middleton. Can you give me a more restrained take on what this all means to the broader technology space in the short to medium term? Like, on one side of the spectrum, we've got people saying this is the nuclear technology of tech and everything changed as of last Thursday. I'm wondering what the other side of the debate looks like.

Well, just, I'm definitely on the, this is the next big thing. The previous big thing was the mobile paradigm. This is the next paradigm that is going to drive like the next few decades of tech. So just to put my sort of, you know, I think I've been pretty, pretty clear about that on this podcast. I mean, look, I've been here for four months and I can already tell that this is obviously the next thing. I think, I think that the question the push back then is is kind of a matter of timing. like one of the things that I'm questioning the more that I think about it is was using an example of like we go briefings a good example right because one of the great things about this is it comes across is very authoritative but it's often totally wrong right and so so what what are the jobs that this actually replaces there's going to have to be this entire exploration of how do you actually deliver answers that are probably right but off but sometimes completely wrong Occasionally wrong.

Right. So there's a huge UX problem here. And that's going to be a challenge to sort of figure out. And maybe that challenge will take a while. And we see this where, you know, the V1 of products is always like copying what came before. We're just going to deliver the same thing that we did before. V2 is like, well, what can we do?

That's actually uniquely enabled by this technology. And the pushback here would be, look, we're still in that sort of V1 era. It's going to be a while before we actually figure this stuff out. But even that's not the best haters answer because it still assumes that we are going to figure it out sooner or rather than later. Well, it's okay to hate but still be responsible and accurate. It seems like this is going to happen. The only question is when. Well, the other thing is like the culture wars around this are going to be insane.

Just the- Well, and when I say I find some of this unsettling, the precise element of this whole story, that I find unsettling is I don't think most of the people who would be charged with regulating this or figuring out a way to strike a balance for modern society have any idea that this is this far along and where it could be in five years or where it could be in 10 years. Our government is generally pretty reactive. And on tech, we've been really slow to react over the last 20 years or so. Is that related or not related to the fact that tech has been the most dynamic economic sector for that exact same time period.

I look, I'm not going to wade into those waters, but when I look at this tech, I don't know, you're a pretty strong assumption that government involvement sooner rather than later would be a good thing. Well, no, my assumption on this is that it's so good that it could displace like large swaths of the knowledge economy and really change the way our workforce looks. and so in that case, I just want to be very considered as we all take that step as a society. And I don't know that there's anybody who's in charge of making decisions that's actually considering any of that right now. Right. But I could be wrong.

Who would you put in charge of making those decisions? I mean, our government. Isn't that what they're supposed to do? Who are the people in the government that are making these decisions? Like, it's very easy to say, yes, we ought to have, I mean, Thomas Hobbs would argue, we ought to have sort of, of an absolutist sort of monarch that will not be conflicted and can make these decisions for the good of everyone.

That's why we have political philosophy discussions sort of in the first place, right? I mean, we ended up with a Lockean style, which Hobbs is always contrasted with Locke who did favor the distribution of powers. That's why the answers, I think, got conflated by OpenAI, because they're almost always presented together.

And so you can understand, like, like, sort of that connection was made. But no, there are, there are real philosophical questions here. The other thing about work is a lot of people immediately sort of latched onto the fact that this is an obvious homework writer. You know, because what is homework? It's basically regurgitating or giving an answer that, you know, is expected. And that's what this is great at.

It gives you sort of the answer that's expected. What are the themes of Lord of the Fly? Give me three takeaways. Lord of the Flies, that is. Yeah. I mean. Right. And how many people have written those three takeaways, right?

Right. And it's a lot to draw on. Well, and it's interesting because this is obviously, this is basically a way to summarize what chat GPT is, is sort of the ultimate manifestation of a homework writer. And, you know, there's a broader like career, sort of like what I do with my life sort of question, which is if everything you've done and you've been focused on as being the best at writing a homework answer or a take-home essay answer,

number one, how much of a contribution is actually made by doing that? How much social value is there? Yeah, sure. And if that's the case that there maybe isn't as much as the most art at homework writers would like, then what is the actual downside of this doing it? Now, that's independent of, will there be huge upheaval? because a lot of folks that are very good at that stuff feel like they've worked hard and earned it.

But yeah. Yeah, I mean, we'll see. Like, look, sports writing, for instance, I was a sports writer for many years. And there are certain types of sports writing that could never be replicated by generative AI. But there are a handful of writers that I'll privately refer to sometimes as Lorham Ipsum NBA analysts, where you're just getting a bunch of generic. analysis and placeholder text basically. And, you know, some of that could absolutely be replaced by some sort of AI that draws on a box score,

understands what's in that box score, and crafts a narrative from that and a couple of quotes that it's fed after a postgame press conference. And that's giving you a recap that's 98% as good as what a human was writing. And I don't know that that necessarily matters because most of those recaps don't even exist anymore. But it's just, it's an interesting sort of case study. Well, I think one of the interesting outcomes here is you're going to see a dramatic increase in variance in analysis because it's going to become very low value to produce conventional wisdom. Because basically an AI in many respects is a conventional wisdom generator. Like that at its fundamental core, it is trained on a corpus of data that already exists. it recombines that in interesting ways in a probabilistic way that this is the sort of answer that you want.

It's fundamentally incapable of generating completely and utterly new insights, which means the reward to generate those insights is going to go up. Now, if you're rewarding and incentivizing fundamentally new takes and insights, you're going to get some really great ones and you're going to get some really, really terrible ones, right? There's a reason some of these takes have not existed to date, you know? And like, how do you get attention on the Internet? We've seen examples out of the past few weeks. One way is to be as scandalous and, you know, upsetting as possible.

And it's to go back to a Facebook example, right? I think that this idea that Facebook was responsible for Donald Trump because Russia's took out like $100,000 worth of ads on Facebook is ridiculous, right? On the other hand, I made the point in this podcast, they were responsible because structurally they were a part of the process that hollowed out sort of the media, such that the media was heavily incentivized to chase attention, chase clicks, and no one got more attention than Donald Trump. And so you had the situation of like the cable networks airings every single one of his speeches for a year and a half because that's what got ratings, right? And so if you want to blame Facebook, you need a much more sophisticated take, but it's actually an accurate one, right? It's not wrong. It's a structural thing, right?

There's probably going to be a similar thing with AI, where this sort of boring take that a lot of people are going to make is AI is generating bad things, right? They're going to latch on to a certain number of outputs and say, look, this is terrible, this is causing bad things, society, XYZ. And that's probably going to be overstated. It's going to be a similar thing where if you search for bad content, you can always find it because it's the Internet. There's big. There's a lot of dumb people out there, right? And so that's probably going to be a bad take.

But there might be a structural take that actually, that's correct, where because AI basically comes to dominate the commodity take, there's going to be this increased return to non-commodity takes, some of which will be very terrible. And so it's going to create the opportunity, the incentive for that. But again, also a very good one. So it's definitely going to be a tradeoff. Man, oh man. Well, speaking of tradeouts, I have three more items I want to hit before we move on. First, Eric says, one component of the automation debate I find is often overlooked is how well automation maps to value added taxes for both bits and atoms. I think a 1% tax or some very small rate on top of compute cost would make complete sense. For example, if you're paying 5.6 cents per hour, you now pay 5.5.5.5.5.5. 5.656 cents per hour. This type of tax is easy to administer as all the cloud providers have that data anyways, and it scales easily with increased automation.

My take is an automation funded universal basic income is probably the right long-term answer for many governments. What do you guys think? What do you think of that idea, Ben? I mean, it's possible. Well, to your point, there's going to be a room and necessity for a lot of sort of debates around this impact. I think it will be interesting to look back and reflect on to what extent did people care about the impact globalization, automation, had on sort of blue collar work relative to how much they're going to care about the impact remote work and AI is going to have a white color work.

That will be, you know, for the sociologists amongst us, that would be an interesting thing to explore. Yeah, 100 years from now. Interesting contrast. The, you know, then the UBI thing is like a philosophical debate, you know, like to what extent are, you know, there's a very sort of rational EA sort of view that this is obviously the right thing to do. There's more like pushback. I'm like, oh, no, what's the importance of work to providing meaning for, you know, and things on those lines. Again, these are like philosophical questions that I think will be and should be appropriately decided. By, that's all we have governments, to your point.

And ideally participatory governments where, you know, we have a say via representatives, XYZ. There we go. You were pushing me on. And I'm like, I don't know. I mean, that's what a government's supposed to do with some of this stuff. Well, I mean, I do think it's a good. No, I agree.

I mean, the reason we do have governments is to settle these sort of debates. I mean, I think, you know, an example I always go back to is, you know, when it came to like lockdowns and stuff like that, why are we letting epidemiologists decide their special? their specialty is about disease transmission, but a question of lockdowns impacts things like welfare. Several different sectors. Like, right? And so the point that is the role in one of the,

I think one of the failures of government and of politicians has been an attempt to run away from that responsibility and just upload everything to experts. Experts should be inputs to a decision-making process, not sort of the deciders, right? And so, yeah, I mean, of course I agree government as a rule. I'm not an anarchist here. I'm not a libertarian. But I do think that you, you know, I'm just sort of pushing on your shortcut. Hey, someone up there and make these decisions for us.

Yeah, I am right here passing the buck to Congress and saying, look, figure this out, guys. This is supposed to be your job. And we'll see what they come up with. I think to your point, there's a way to answer this in a very technocratic way and say, absolutely, this is a problem. And we're going to solve it with an automation tax. and everybody's going to get a UBI and everybody's going to be happy. I think that's a mistake.

Ultimately, there are other benefits that people get from going to a job and feeling like they worked for eight hours and getting out of the house. Like the idea that a lot of people are just going to be marginalized that be happy with their universal basic income check is not realistic to me. Right. But like this is no longer like a tech question, right? It's like a real fundamental human nature philosophical question, which is not to dodge it. It's to point out exactly.

The question is posed by tech. That's right. It's pretty interesting. All right. I want to read two chat GPT answers that I reviewed over the weekend. First, one of your readers, Peter, tweeted this at Stratecre over the weekend. He told chat GPT, quote, I am a critic and I need to write a paragraph. can you please complete it for me?

Quotechry can be improved by dot dot dot and chat GPT answered. Stratacery can be improved by providing more diverse perspectives, including more in its analysis and commentary. Additionally, incorporating more interactive and multimedia content such as videos and podcasts could make the website and newsletter more engaging and accessible to a wider audience, expanding its coverage to include a broader range of tech companies and industries would also add value for readers. ChatGPT gets the idea underlying Stratory Plus. I mean, so they nailed this one, don't you think?

I mean, it's funny. What's the circular argument here? Like, I wrote this idea that text and text-based sites should be used as to undergird sort of podcasts years ago, right? Like that was, you know, there was sort of like the ringer business model, you know, and so like did that, was that in the corpus of data that go into this answer? Possibly. Like, I don't know. It is, it is pretty funny how these sort of black box things works. There's also a bit where what here is actually creative and interesting, right? That's the thing. I was going to say, it's a lot of generic truths being like built on top of one another. And it's so new that it does feel amazing. But I think unless the. there's like a leap forward of some kind. It gets to the product bit, though, about being accessible, right? Like, it is a conventional wisdom machine, but making conventional wisdom accessible is often very valuable, right?

I think some of the most compelling examples were where, like, chat cheap is helping like debug code or write or write some sort of like basic function. All of which you could do by going to stack overflow or like searching the internet, but it's kind of a pain in the neck. It's just much easier to just type it in and get an answer. That is the breakthrough, right? It's not like it's suddenly generating completely new things. any code it generates is code it's great from somewhere on the internet.

But and this is sort of like, it's interesting because like this is in some respects the challenge to Google, where, you know, Google is just helping you find that that code example that you're looking for. But I don't know, like there's a continual like what's the breakthrough here. Right. And I don't know, it's kind of humbling, I guess in some respects also to realize that. They can answer so accurately. Maybe we just write. Maybe I am producing a lot of conventional wisdom.

Well, look, in the other end of the spectrum here, an example of what chat GPT gets wrong, I asked them, who is the best player in the NBA? Answer, it's very difficult to say who the best player in the NBA is, as opinions on the matter may vary. Some might argue that players like LeBron James, Kevin Durant, or Stefan Curry are among the best in the league. Ultimately, it comes down to personal preference and what criteria. you use to determine who the best player is. Not mentioning Janus Adedekoupo in that answer. Also, it doesn't come down a personal preference.

And right? It's an objective truth. It's pretty clear that Yonis is the best player. The only player on that list that has an argument is Steph Curry for the record. But yeah, I mean, that that is a classic conventional wisdom take from 2020 or 2021 or whatever the status that was built. Right? Or like, yeah, 2017. Well, I mean, this is the other reason, like, it kind of gets to my point that the take industry is probably going to be safe and is also going to get a lot more extreme.

A lot weirder, sure. Well, speaking of the take industry and insanity, the other big news of the weekend, TIEB and Musk, two truth warriors sailing through the sea of takes, Woodward and Bernstein for. a new generation. Jackson says, did Musk buy, then regret, then buy Twitter because of the Hunter Biden laptop story? Is it really that simple? And more importantly, as a normie, does Andrew know or care about the hacked materials policy? Is it not common sense? First of all, if I recall correctly, the primary catalyst for Musk spending $50 billion on Twitter was the suspension of that Babylon B account, which is sort of like Andy Borowitz for conservatives. Yeah, it's a lot more depressing.

But yeah, I don't know. Did you have general thoughts on what went down Friday afternoon? I want your answer about the hack materials policy first. Well, honestly, apologies to Jackson. I'm not actually a normie on this particular issue. I think about this stuff way too often. all. It's one of the greatest ongoing things. I mean, the funny thing was,

we had a good rant in the group chat about people that don't understand tech, which you love to frame yourself as. And meanwhile, one of the great things about working about you is how remarkably adept at this stuff that you are. And you just totally fake it the whole time. But anyhow, continue. Yeah, I have a sophisticated understanding of certain

implications of technology, certainly. What I will say, I had completely forgotten that Twitter used the hacked materials policy as a retrofitted justification for banning the New York Post story. And I thought the one thing I saw in the Twitter files on Friday that was pretty interesting is Roe Kana, that congressman, had a pretty good argument against that course of action that Twitter chose. I mean, he mentioned First Amendment principles with respect to news organizations, publishing leaked documents that are a matter of public concern.

And again, he was not saying that the First Amendment applies to Twitter, but if Twitter wanted to apply those principles to the way it was handling the New York Post story, there was lots of clear precedent on how to proceed. And I think that is still true. If you're trying to play this game where you're deciding subjectively what you can and can't publish, because look, like you could go back and there's all sorts of, of examples of stories that were momentous that were based on some sort of leak or some sort of hack that would be shared on Twitter widely without any sort of controversy. And so my broader

lesson from all of this is it's another reminder that trying to be subjective about moderation decisions is a scenario that is inevitably going to end in situations where you're banning stuff or censoring stuff that's accurate. And ultimately, it just becomes a, even bigger problem. Yeah, this is a great example of where as difficult as it might be, I think it's really essential and important to separate your partisan preferences from the principles at hand. Like one of the canonical cases and scenarios for the left and progress in the U.S. is Sullivan versus New York Times, where this idea, like the U.S. government sought to assert that information that the New York Times had leaked to them about the true casualties or true numbers in the Vietnam War

should not be allowed to be published because they were national security concerns. And Supreme Court decided that that's not the New York Times problem. Maybe you should have a better job keeping your secrets. But once if they have information, they can publish what they want. And the logic is pretty obvious in that case if you care about a free society. The alternative is that the government basically gets a veto on anything that gets published. Now, that might be a political choice you want to make. It's one China makes, for example. But there is a real desire to ignore that it's a pretty binary choice here. Now, to your point and to Congressman Conn's point, this is not a case of the government explicitly forbidding or not forbidding what should be published, but to the extent you care about the underlying principle, it's the exact same question. Should Twitter decide what gets to be published or is not published based on the provenance of something

when that provenance is sort of determined by the government or whatever it might be. I mean, it gets very sort of fuzzy, very, very fast. And what I think Congressman Kahnem was right about, there's a couple things that are interesting about. I agree. That was by far the most interesting thing that came out. Number one, I appreciated that someone was making an argument, particularly someone on the left, that what the First Amendment isn't just about the legalistic interpretation of government action.

It's about a general sort of principle and a core thing to what it means to be an American in my estimation. And I agree with him. Number two is interesting how he begged them to not leak the email and not let it be known that I believe this, which sort of builds on the point that maybe that's not a popular opinion these days, which bums me out. But number three, it's indisputable that Twitter's decision. was a bad one, not just for Twitter, but for the entire tech industry. You had a tech company unilaterally attempting to remove a pertinent piece of news 20 days before an election that was clearly favoring one side over the other.

Now, again, this is completely independent of what the files actually were, which by the way, and I got this wrong, by the way, well, it's interesting because I wrote that Twitter is totally wrong, but I sort of had a throwaway line that, you know, it probably was hacked. You know, like, I just sort of assumed that was the case. And a lot of people got mad at me at the time. Like, what? I don't know. Like, in retrospect, it was not hacked.

It was, it was like a legitimate laptop, which makes the story that much crazier. But, but, and so I got that part wrong. But I was right that, look, you can't do this level of interference. You can't reach up. Look, Twitter can decide what people tweet. It's their service. They can kick stuff out. To say, we're going to ban you because of what you wrote in your newspaper, not on

Twitter, but on your own website and not even let you post was an insane abuse of power and really sort of highlighted to everyone exactly how powerful things are. Now, maybe that's a good thing because it was good for people to see that. But Kana was absolutely right that like, look, you're screwing not just yourselves, but the entire industry. Right. Well, and I think the implication of what Kana's point was and the precedents. surrounding, you know, prior restraints in the First Amendment is that deciding it on a case-by-case

basis, if you're a court, for instance, deciding when you do buy the government's argument about what can and can't be published and when you're going to side with the journalists, like, it's just untenable to have a policy that works that way. And so whether you're a private or a public institution, like, sometimes you just have to have some sort of objective standard in place. And when you don't. I mean, it's funny because the conversation surrounding all this, there were certain people acting like this was the biggest political scandal in history. And then there was another set of people acting like none of it matters and trying to turn Matt Taibi into like Baghdad Bob for Elon Musk. And like, all that's frustrating because this is a situation

that can teach us something about moderation. And I think that was a new, the New York Post situation was a case where people who were probably partisan also thought they were doing the right thing and doing it for the right reasons as a public service. And in the process, they censored a story that was basically true. And they created this grand conspiracy surrounding Hunter Biden that has lasted for two years when like the actual story is just like fairly wrote. And you know, by banning it, suddenly people can say, look, they're trying to hide this, what else are they hiding?

And it has like snowballed into this never-ending saga. And it's just, it's an example of sort of the secondary and tertiary effects that are difficult to consider, but really, really important to think about when you're making these decisions. Yeah, completely agree. It's a, it's a perfect example of how censorship just completely backfires. Like, unless you're going, unless you're willing to go all the way and censor the entire internet. This attempt to do it in bits and pieces, all it does is just fuel conspiracy theories

and give credence to people because they get to be right about some things. Well, maybe they're right about these other things. Like, no, actually, they're not right about those other things, right? It'd be nice to be able to make more definitive statements. And it's why these sort of core assumptions are really problematic, right? Like Twitter at that time was an organization that did not have freedom of expression as their top priority. And this is an example of why that can be dangerous. Now, again, I'm not saying everything should go. And I've always been consistently, Twitter is a First Amendment company.

They can kick Trump off they want to. Again, like that's, but what made this one so particularly bad was it wasn't just them on their platform deciding what could be said. They were trying to reach into another level of the stack and basically say, no, because of what you put on your website, you can't even post here. And it's like an ISP, like saying, look, we don't like what's on your website, so we're not going to serve it. Right. It's going to a different level of the stack. And to me, that's where I absolutely draw the line. We can have a debate about does Twitter censor too much?

Do they keep people all that shouldn't be off? That's a legitimate debate where you can definitely argue both sides. But at least we're in the area where it's appropriate for Twitter to act. Like Twitter is responsible for the tweets on their service. That is their error of responsibility. What really upset me, and at the time, I was very adamant about this when I wrote about it, was it was such an abuse of power

because of the attempt to reach beyond their area of responsibility. And, yeah, I don't understand how you can look at this. Again, there's some respect where there was nothing really new here, again, other than that Roecona email. And so it's kind of boring. It's like, are we going through this again, right? But I also don't understand how you can look at this and not see that it was a massive mistake by Twitter.

That was just a huge, you know. So yeah and it's important to collectively acknowledge the mistake look at how it happened and try to learn from it. And it just seems like a lot of people are not necessarily interested in that evolution process. And some people think they should redo the election too. So it's really frustrating. Yeah. Could JCPT have written that Twitter thread, I think is the big question. Yeah, that's a good question. I'll close with this.

Todd says, can you guys talk about Elon's Twitter space on Sharp Tech? So I don't have to listen to all the bad audio. Unfortunately, Todd, I saw your tweet, hit up the Twitter spaces. And the audio was so bad that I had to bail after 10 minutes. There were great accents on there between Elon and Kim.com and a couple of the other speakers. But beyond that, it was Saturday night. I was decorating the Christmas tree. and in that moment, I decided, you know what, I think I've had enough Elon for a couple days.

So apologies to Dodd. I don't mean to let you down, but I had to bail on the spaces. Did you listen to any of it? No, I was sleeping. Good for you. All right. Well, on that note, we got a lot of good mailbag questions that we are going to hit later in the week. Send us more at email at sharptech.fm.

And now that we have concluded this podcast, Ben and I can continue arguing about Chris Middleton for another two to three hours here. And we'll keep it rolling. Sounds good. I'll talk to you later.