The AI Daily Brief: Artificial Intelligence News and Analysis - OpenAI Is Officially Training GPT-5

Episode Date: November 13, 2023

Today on the AI Breakdown NLW looks at what GPTs are (and what they're not) and why many people are thinking about them the wrong way. Plus, OpenAI is officially training GPT-5, according to a new interview with CEO Sam Altman. Today's Sponsors: Listen to the chart-topping podcast 'web3 with a16z crypto' wherever you get your podcasts or here: https://link.chtbl.com/xz5kFVEK?sid=AIBreakdown  ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're talking about the real value and importance of OpenAI's custom GPTs. Before that on the brief, OpenAI is officially training GPT5. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our Discord, our YouTube, and our newsletter. Welcome back to the AI breakdown brief, all the AI headline news you need in around five minutes. Today we kick off with a couple of stories around OpenAI. Big surprise, that's basically been the kickoff of every episode for the last week or so. But where we start is with a really interesting interview between Sam Altman, who is, of course,

the CEO of OpenAI and the Financial Times. There are a few things in here which, although seemingly innocuous at first, actually I think have pretty big implications. The first, and what the FT chose to focus on, was the intimation that OpenAI would be raising even more money from their partners at Microsoft. From the interview, Altman said his company's partnership with Microsoft's chief executive Satya Nadella was working really well, and that he expected to, quote, raise a lot more over time from the tech giant, among other investors, to keep up with the punishing costs of building

more sophisticated AI models. Specifically when asked if Microsoft would keep investing further in OpenAI, Altman said, I'd hope so. There's a long way to go and a lot of compute to build out between here and AGI. Training expenses are just huge. Indeed, Altman also pointed out that although the company is over a billion dollar revenue run rate at this point, they're still unprofitable because of how high those training costs are. Alman said that the Microsoft partnership would ensure that, quote, we both make money on each other's success and everybody is happy. Now, another interesting part of this conversation, although not particularly unexpected

if you listen to Sam Altman talk with any sort of frequency, is the way that he describes what OpenAI's real products are. He said, right now people say you have this research lab, you have this API software, you have the partnership with Microsoft, you have this chat GPT thing, now there is a GPT store. But those aren't really our products. Those are channels into our one single product, which is intelligence, magic intelligence in the sky. I think that's what we're about. Indeed, Altman said that his specific time is split into, one, research into how to build superintelligence, and two, how to get the computing resources to do so.

Still, one of the most significant revelations from the interview is something which perhaps was widely expected, but represents a significant shift in the tone from OpenAI, which is that the company has confirmed that it's working on GPT5. Now, the reason that this is notable is that in all previous interviews when asked about this, Altman went out of his way to say that we are not training GPT5 yet. And while he still didn't commit to any sort of timeline for release, he did talk about it in more than passing detail. Indeed, it sounds like the interviewer, although we can't be sure because we only have these excerpts, not the actual interview, delved into it with Altman, who even got into

where the data would be coming from. On the one hand, it is going to be publicly available datasets, but on the other hand, it's going to be proprietary data from companies that they partner with. This, of course, gets to what we talked about last week with OpenAI's new data partnership initiative, which is clearly an attempt to go out and proactively recruit data that they didn't have access to previously. Alman also talked about how hard it was to predict exactly how GPD5 will be better than its predecessors, even though they can have pretty strong confidence that it will be better than those predecessors. Altman said,

Until we go train that model, it's like a fun guessing game for us. We're trying to get better at it because I think it's important from a safety perspective to predict the capabilities, but I can't tell you here's exactly what it's going to do that GPT4 didn't. Alman also discussed that there would be more to getting AGI than just training better and better LLMs. He called them, quote, one of the core pieces for how to build AGI, but there will be a lot of other pieces on top of it. However, what that other pieces are, he's not really sure. He used an analogy of Isaac Newton, saying basically that he couldn't just read more textbooks and talk to more professors. He had to get out there and actually do things. However, what those things are

when it comes to the context of AGI, Altman just isn't sure. He said, said, so the question is, what is the missing idea to go generate new knowledge for humanity? I think that's the biggest thing to go work on. Now, in the meantime, for those who are a little bit more focused on the here and now and the new launch of GPTs, this interview reaffirms that they are very much focused on these as the first step towards more complex and more capable agents. Said Altman, we will make these agents more and more powerful, and the actions will get more and more complex from here. The amount of business value that will come from being able to do that in every category, I think, is pretty good. Now, as I mentioned, a lot of people are

noting the fact that this $10 billion wasn't enough to compete in this incredibly expensive industry. Well, part of that is not just the computing costs of training ever more advanced models, but also the salary costs of the team members that they are trying to recruit. The information dropped a piece over the weekend. OpenAI's new weapon in talent war with Google? $10 million pay packages for researchers. Now, basically what's happening here is that there is this interesting little window where Open AI is on the verge of effectively tripling the valuation of the company

with this employee tender offer that we've talked about in the past. Effectively, OpenAI is giving some of its employees the ability to get liquidity, and they're doing so at a price tag that's between $80 and $90 billion. Apparently then, OpenAI's recruiters are taking that fact and bringing it to top AI employees at other companies like Google and basically saying, lock in a stock package now at the current valuation of $27 billion from earlier this year, and almost immediately it'll be worth $3x what it is on paper.

Now, in addition to those big, juicy pay packages, the recruiters are also making claims that researchers would have greater access to compute. However, exactly what that's based on isn't clear, given that there is still a perception that Google has an advantage in that area, even if Sam Altman has told some colleagues that he expects that advantage to shrink next year, as Microsoft makes more chips available to OpenAI. Now, while it doesn't sound like Google is trying to match these salary offers currently, they have clearly increased their willingness to compete on that vector, given that the flow hasn't just been from Google to OpenAI, but that Google has been successfully recruiting some

OpenAI employees, such as the person who led the development of code interpreter, who joined Google from OpenAI in October. Now, speaking of compute, while Nvidia's H-100 has set the tone and tenor for this latest generation of generative AI, today they unveiled the H-200. The H-200, as you might expect, is meant to be a GPU upgrade from the H-100. CNBC writes, The key improvement with the H-200 is that it includes 141 gigabytes of next-generation HBM3 memory that will help the chip perform inference.

using a large model after it's trained to generate text, image, or predictions. NVIDIA has said that the H200 will generate output nearly twice as fast at the H100. Now, interestingly, this is the same focus that AMD's new MI300 X-CH chip is supposed to have, a real focus on inference, and both are expected to come sometime in the first half of 2024. Now, back in the land of Google, Reuters is reporting that the company is in talks to invest in very hot AI startup Character AI. Character AI, which was founded by ex-Gougalers, gives people, particularly young people, the ability to chat with virtual celebrities, anime characters, or create their own chatbots.

According to similar web, 60% of Character AI's traffic is between 18 and 24. As Reuters puts it, the demographics helping the company position itself as the purveyor of more fun personal AI companions compared to other AI chatbots from OpenAIs chat GPT in Google's Bard. The company is apparently looking for a big valuation increase from back in March when it raised $150 million at a billion dollar valuation. Character AI is now apparently looking to raise funding at around a $5 billion valuation. Last up today, from the science in AI file, space.com is reporting about a new AI that can track the melting of icebergs 10,000 times faster than can humans. Right space, scientists are turning to artificial intelligence to quickly spot giant

icebergs and satellite images with the goal of monitoring their shrinkage over time. And unlike the conventional iceberg tracking approach, which takes a human a few minutes, to outline just one of these structures in an image, AI accomplished the same task in less than 0.01 seconds. Basically, the team in question trained AI to spot large icebergs, using images from the European Space Agency's Sentinel 1 satellite, and it did so with not only accuracy, but also with fewer mistakes. Again, from space, the AI tool didn't make the same mistakes as other more conventional automated approaches, such as the error of misconstruing individual bits of ice as one collective iceberg. And of course, basically with the observation taken out of the hands of

individual scientists, there's more time to spend focusing on actual solutions. Anyways, guys, that is going to do it for today's AI breakdown brief. Next up, the main AI breakdown. And now a word from today's sponsor. Are you interested in how two top-of-mind trends AI and crypto can work together? If so, I have the perfect podcast recommendation for you. Web3 with A16Crypto, the chart-topping show brought to you by venture firm and Dries in Horowitz.

Web 3 with A16Z Crypto is your definitive resource for the future of the internet. Whether you're already building in these spaces or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford Cryptography Professor Dan Bonay and former Google Xer Aliya in conversation with host Sonal Choxi about the intersection of AI and crypto. From fighting deep fakes and proving humanity to large language models like ChatGBT, they cover it all. I highly recommend checking it out, especially if you'd like to learn more about how

AI and crypto will impact our everyday lives. Beyond crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. So go ahead, listen to Web3 with A16Z crypto wherever you get your podcasts. Welcome back to the AI breakdown. Well, a week ago, at OpenAI's Dev Day, one of, if not the biggest announcement was around what they were calling GPTs or custom GPTs. GPs, they said, were customized versions of chat GPT that could combine instructions, extra knowledge, and take actions on behalf of the person interacting with it. They wrote, GPs are a new way for anyone to create a tailored version of chat GPT to be more

helpful in their daily life, at specific tasks, at work, or at home, and then share that creation with other. GPs can help you learn the rules to any board game, help teach your kids math, or design stickers. Now, a couple more things to note. One is that GPTs required no code. There was a builder, the chat GPT released at the same time, and two, there was the promise of a GPT store rolling out later this month in November. They wrote, starting today, you can create GPTs and share them publicly. Later this month, we're launching the GPT store featuring creations by verified builders. Once in the store, GPs become searchable and may climb the leaderboards. We will also spotlight the most useful and delightful GPDs we come across in categories like

productivity, education, and just for fun. In the coming months, you'll also be able to earn money based on how many people are using your GPT. So all of this was initially greeted with tons of excitement, but then, of course, the AI hypefluencers on Twitter slash X started doing their thing, and every other post that anyone who touches AI saw was some version of here's 10 mind-blowing examples of GPTs, which necessarily creates its own sort of fatigue. Staff engineer writes,

I'm already sick of seeing the word GPTs, when will it end? And on the one hand, I totally understand this point of view. It's why I've sort of railed against that type of content in the past. But at the same time, I also started to see many people posting critiques like this one from Daniel Lossie. I'm going to say it, custom GBT's just a fancy system prompt, lull. AI entrepreneur Bindu Reddy said the same thing. Most Gptys are glorified instruction prompts.

While these are fun to play with, I suspect the novelty will soon wear off. Most users will go back to using core chat GPT features versus messing around with these GPs. Data Chaz posted the Scooby-Doo reveal meme, where so-called mind-blowing custom GPT is actually revealed to be a fancy one-liner GPT4 prompt. Now, it is my contention that dismissing these as fancy system prompts completely misses the point

of what GPs are trying to do and where their real value lies. TLDR, they are just a fancy system prompt, but that's exactly the point. So let's talk about what GPs are not. One, they are not agents. OpenAI made it very clear that this was the case. Open AI does view them as their very first step towards the world of agents in which AIs have more context about their user and can actually go accomplish tasks, but they are intentionally and explicitly just a very small step in that direction.

At OpenAI Dev Day, Sam Altman used the introduction of custom GBT's to reinforce the company's belief that when it comes to a technology that has as many upsides but also downsides as autonomous AI agents, OpenAI believes that the only way to approach it is to actually see how things work in the real world rather than just trying to speculate on them in the lab. The reason it matters that GBT's are not agents is that the label of agent comes with it the baggage of inflated expectations. People expect more from these GPs than they're actually used. useful for. And because they don't achieve that idea that people had in their minds, they are dismissed as not

very useful, even though they were never intended to be that thing that people were imagining them to be. I think this is much the same with the idea of them as apps. Now, the comparison to apps comes not from OpenAI in any way, but simply because when we think of a store for stuff that users create that a platform cuts people in on, what we think of is the app store. And since these GPTs are going to be in some cases useful for similar types of things as are apps currently, it's a fairly natural jump to make. Apps, however, of course, are a totally different phenomenon. They're coded and built from the ground up with a specific purpose. They are not meant to be an excess point to an AI model, at least not in most cases. And more than anything, they're just more sophisticated and complex

than what custom GPs are trying to be. So again, the reason that it matters to identify what they're not is that I think part of the latent problem with these critiques is that they start from a standpoint of inflated expectations that it doesn't make any sense to actually apply to custom GPTs. But if that's what GPTs are not, they are not apps and they are not agents, then what are they? Well, one, as we said, in OpenAI's view, they are a first step towards agents. Logan, who does developer relations at OpenAI, actually responded to Daniel in that post that I just read and said, it's our first step towards agents. GPTs will get much better over time. Many people have been asking for the ability to customize chat GPT with files, tools,

and instructions. Excited we deliver that, but this is only the first step. So much more to come. So one part of what GPTs are is a first step towards agents. But that's not the thing that I actually think is important right now in terms of how users consider it. I said a moment ago that I think that the critique that GPTs are just glorified prompts is wrong because they are glorified prompts, but that's exactly the point. GPTs are workflows. As often happens, Professor Ethan Malik from Wharton sums it up really crisply and acutely. He writes, I think GPTs may have have actually unlocked the potential of structured prompts for many people. I see tons of people sharing GPTs where before, almost no one shared links to prompts. It's not really a capability

change, but a user experience change. What he means is that the most useful GPs, right now, are the ones that have come to embody the specific prompts that power users have figured out how to unlock value from, just created in a shareable way. Instead of a Twitter thread or a blog post or a video or some downloadable PDF in exchange for your email that tells you all about prompt engineering and how to copy someone's prompting, instead you can now just hand them one of these custom GPs. Have you perfectly figured out a way to get great writing critique for technical writing? Well, again, instead of teaching someone how to prompt chat GPT in the same way that you did to get those results, you can just give them your custom GPT that has all of that structure

built into it. GPTs are just saved, shareable, structured prompts. But the fact that they are saved and shareable is what makes them powerful, because that gets us to the next thing that Gptis are, which is use case simplifiers. For the vast majority of people, when they look at the blinking button of the main page of chat CPT that asks, how can I help you today? Their answer is, well, I'm not sure. That's why GPT has starter prompts. Mine currently read, design a database schema. Tell me a fun fact about the Roman Empire. Recommend a dish to impress a date. Explain options trading. Maybe randomly one of those things will be useful, or maybe I came to chat GPT because I had an idea of exactly what I wanted. But I think that the vast majority of people instead

come because they've heard this technology is incredibly powerful, and they want to see if it's powerful for them. There is nothing more intimidating in many ways than a blank page. And the vast majority of people who learn about artificial intelligence and generative AI tools, learn by one, seeing what other people are doing, and then two, trying that out themselves and taking it in slightly different directions that are more relevant for them personally. When I say that GPTs are use case simplifiers, what I mean is that instead of having to browse around X to get an idea of what other people are using chat GPD for, you can just look at the names of these GPs, all of which have some specific focus, and hone in directly on what's most useful for you.

their use-case simplifiers again, like Ethan said, not because they've added some fundamental new capacity, but because they've created a new user experience that makes it much easier to see what chat GPT can actually be useful for. And that gets us to one more thing that GPTs are, a way to transfer knowledge from the most to the least sophisticated chat GPT users. In the world that we live in, when it comes to tools of any type, a very, very tiny fraction of the population that uses those tools are going to be the innovators

that actually figure out how to use them. The vast majority of people instead will just copy what they see and really only learn a few workflows that ultimately capture a big chunk of what they need. On the other hand, there are a small handful of users who sit at the top of that power distribution and come up with the vast majority of use cases that ultimately people will copy, imitate, and adapt. These GBTs are the way that that very small cohort, that always an inevitably smaller portion of the larger chat chbtu using population is going to get what works and what is useful to everybody else. Again, as Ethan wrote, it's not really a capability change

but a user experience change, but that doesn't mean it's insignificant. In fact, in many ways, it seems likely to me that user experience changes more than capability changes are what is going to be most important to unlocking the value of generative AI to the next 100 million or billion users. Now, this is not to say that there aren't reasonable

critiques of custom GPTs. One thing that many have noted is that it appears that there are ways to reverse engineer the custom instructions underlying GPTs. That includes potentially getting access to the files that were uploaded as part of building the GPT. Those are real meaningful concerns. They are security holes, they are privacy considerations, and those are absolutely things to consider when using these tools, or in particular, building these tools. They also suffer for many of the same challenges that just generic use of Chad GPT does. For example, hallucinations, and just how compliant with these custom instructions GPTs really are. Some like Jerry Liu, the CEO of Lama Index, also argues that they exist in a weird space where they are perhaps

too complicated for the average user to create, but also not sophisticated enough for developers who use who might opt to use the API instead. There's also the question of what the store will actually look like. I think Boris did a really great summary here. He wrote, because of the word store, many people think that it'll be like the App Store and get excited about the money-making opportunity. I have a feeling that the better analogy is a Spotify-type subscription platform. Millions of interchangeable easy-to-make GPTs. The vast majority of them will be free with the $20 per month-loss subscription. There will be paid GPs, but not many people will buy them. Of course, a few hits will sell big numbers. Overall, the store will be one, great for the users,

two, wonderful for OpenAI, three, better than nothing for the most new, quote-unquote, GPT developers. Just like the vast majority of the musicians in Spotify, make pennies from streaming, the same will be true with the vast majority of the GPT developers. Most developers will publish quality GPs and use them as a free lead magnet to monetize in other ways. Some will make good money but nothing big from the revenue share when their GPs go viral. Overall, I think it'll be a net positive for everyone, or at least for most, and a good next step in the AI adoption wave. Now, I think that there are some reasons to think that Boris is right when it comes to the idea that GPs will be monetized not via the creators of GPT setting their

own price, but through something more like a subscription. If you go back to how OpenAI describe the GPT store, they write, in the coming months you'll be able to earn money based on how many people are using your GPT. That's a weird way of saying you'll be able to set the price and get paid each time someone uses it, meaning it seems much more like a revenue sharing type of agreement based on some black box usage formula that we probably won't have access to. Now all of that said, I don't want to give the impression that this critique is the main threat of conversation surrounding GPTs. Instead, it is definitely excitement and

entrepreneurial creativity. Replit Builder and Residence Martin Bowling writes, excited to unveil the custom GPT starter kit. It's designed to empower creators with the tools to build advanced GPTs with ease. And basically, this is a toolkit for an advanced custom GPT that can do things like integrate your own API, create shortcuts, and embed custom knowledge directly into that custom GPT. GPT's Dex is a new product that is racing up the product hunt charts right now that is, as you might expect, a third-party database of interesting GPs. Now, going back to my claim that GPs are use case simplifiers. One of the things that will or won't make that so is what OpenAI does or third parties do to make it easy for users to actually go out and find

the right ones for them. Yes, this is great that there's a visual interface for searching through thousands of GPs, but it's still an overwhelming amount of information. And there isn't really any sort of sophisticated search that allows me to go find exactly what I might need for a use case right now, which still kind of leaves people in the position of having to just scroll and see if anything triggers an idea of what might be useful for them. One of the major questions to me for what comes next with these GPTs is what sort of search and browse interface OpenAI puts around them. I think that'll be as significant as the monetization approach

when it comes to whatever they're planning for the GPT store. Plus, even though I'm more focused on the here and now, there is certainly an understanding of OpenAI's perspective that these are the beginning of personalized AI agents. Kevin Ruth's from the New York Times wrote a piece this weekend called Personalized AI Agents Are Here? Is the world ready for them? The piece concludes, if Open AI is right, we may be transitioning to a world in which AIs are less our creative partners than silicon-based extensions of us, artificial satellite

brains that can move throughout the world, gathering information and taking actions on our behalf. I'm not fully prepared for that world yet, but by the look of things, I better start getting ready. So, friends, that is my take about what GPs are and are not. I think when viewed through the lens, of being better ways of capturing structured prompts and making them shareable. It is an extremely use case expanding and community expanding feature update, and one that I think we're just barely scratching the surface on.

I'm sure later this week I will do some rundown of my favorite GPTs that I found so far. Now, if you are interested in talking more about GPTs specifically, you can either join us in the AI breakdown Discord, which is at bit.ly. Or I've also created another experimental discord just for focus on GPT building, called, as you might guess, GPT builders. You can find a link to that one at bit.ly slash gpt builders. In either case, I'm excited to learn alongside you. And of course, as always, I appreciate you listening or watching. Until next time, peace.