The AI Daily Brief: Artificial Intelligence News and Analysis - Ilya Sutskever Raises $1B for Safe Superintelligence

Episode Date: September 5, 2024

Former OpenAI founder Ilya Sutskever recently announced his new company Safe Superintelligence. Now he's announced a $1B pre-product raise. Concerned about being spied on? Tired of censored responses? AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://venice.ai/nlw ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠and enter the discount code NLWDAILYBRIEF. Learn how to use AI with the world's biggest library of fun and useful tutorials: https://besuper.ai/ Use code 'podcast' for 50% off your first month. The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, OpenAI co-founder Iliosuke's Safe Superintelligence, Inc, has raised $1 billion. Before then on the brief, the president of OpenAI Japan seems to indicate that GPT Next is coming, and it's 100 times more powerful than GPT4. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief Headlines edition, all the daily AI news you need in around five minutes. We kick off today with a conversation that ripped around Twitter slash X yesterday. At the KDD summit, a representative from OpenAI Japan shared a slide that seemed to indicate that GPT Next,

which was in quote, so isn't necessarily an official name, is set to be 100x more powerful than the GPT4 era models we have now. The speaker was to Dow Nagasaki, who was announced back in April as the president of OpenAI Japan. Now, the way that Twitter slash X is taking this is that this 100x improvement on GPT Next is a specific real estimate rather than a demonstration of the exponentiality of AI and that GPT Next is the name of the model. I'm not totally sure that something isn't being lost in translation. It seems fairly possible to me that this slide was meant to just represent the exponential nature of AI growth and that the slide which said GPT Next in 2024 and had it at 100x more powerful than the GPT4 era was not in fact saying that the next model would be called GPT

Next, that it would come out in 2024 or that it would be 100x more powerful, but instead was more meant as a demonstration and then just got picked up and ripped around the internet as the internet does. Still, I think the excitement around it shows just how hungry for whatever comes beyond GPT4 people are. And frankly, it was probably pretty dangerous to put this in a slide if it wasn't actually claiming that GBT Next was coming out this year and was going to be 100 times more powerful. One of the things we've been discussing a lot recently is Wall Street's relationship with AI. Nvidia had a terrible day yesterday, wiping out $279 billion.

That is the single biggest day of market cap loss in history. Over the last three sessions, since it reported earnings, it's now down 14%. Now, part of the specific reason for Nvidia's losses was that the U.S. Department of Justice has apparently sent a subpoena to Nvidia as part of a larger antitrust probe. Bloomberg characterizes the antitrust probe as escalating. As they put it, the DOJ, which has previously delivered questionnaires to companies, is now sending legally binding requests that oblige recipients to provide information. This takes the government a step closer to launching a formal complaint. The concern is that Nvidia is making it harder to switch to other

suppliers and penalize buyers that don't exclusively use its chips. Now, while some of the issues may be Nvidia specific, Nvidia's placed as a bellwether for the rest of the market means that these big losses have people looking at downtream implications as well. Again, Bloomberg writes, Invideo route has traders watching $100 share level amid vacuum. After earnings, there are a few potential catalysts on the horizon. Said Michael Kerbryd, portfolio manager at Evercore Wealth, we're in a bit of a void right now. We're through with earnings and there's lots of economic data coming up this month. There's a lot of caution ahead of that.

When you're in a trading vacuum, it becomes a shoot-first market that is very short-term in nature. And indeed, it does seem like broader uncertainty is coming home to Roos is the main market force. The information writes, just what we needed a stock market chill to remind us that summer is over. Investors on Wall Street were dumping stocks left right and center on Tuesday, after anemic manufacturing data reminded people of economic uncertainties they had put out of their minds. Tech stocks were a sea of red from the smallest to the biggest. Later they write,

enthusiasm for both Microsoft and Nvidia has been dampened by questions about the staying power of corporate spending on AI chips. Apple stock, meanwhile, has outperformed other big tech stocks in the past three months. Its rally may be premature, however, as Apple can't escape its own version of the AI question. The company next week unveils its newest iPhone, which means we're closer to seeing whether the addition of AI features can jumpstart stalled iPhone sales. There's no guarantee that consumers will be any more willing to spend for AI-powered services than businesses are. This should be an interesting fall. That was reflected in a conversation on CNBC with a portfolio manager who said,

we like Apple for its sustained business model, not its AI story. Anyways, lots of interesting things coming up in the weeks ahead when it comes to the markets. Next story today, we have yet another aqua-hire non-acquisition thing, Amazon has hired the founders of the industrial robotics company Co-variant. The company was last valued at $625 million back in April 2023, and once again, this same model of hiring the founders and licensing the startup's foundation models is the alternative to an actual acquisition.

Only around a quarter of the company's employees are joining Amazon, all who will be joining the fulfillment technologies and robotics team. I don't know how this trend ends, but I do know that if startups keep selling themselves and ditching out on the payday for half or more of the company, it's going to be a lot harder to hire people into startups. Lastly today, the Oprah AI special hasn't even come out, and already it's controversial.

Ars Technica writes, Oprah's upcoming AI television special sparks outrage among tech critics. AI opponents say Gates Altman and others will guide Oprah through an AI sales pitch. The show is, of course, airing on September 12th, and quote, aims to explore AI's impact on daily life. You can read the piece for yourself, but basically it shows once again, just how fraught the discourse around AI is. That's going to do it for today's AI Daily Brief

Headlines edition. Next up, the main episode. Today's episode is brought to you by Venice. Venice is a private, uncensored generative AI app. It accesses open source models to enable text, image, and code generation without the fear of being spied on or having your data exploited. Discuss anything with Venice without concern about it being monitored, sold, or given to advertisers and governments. Venice is different because your conversations and creations are kept securely within the browser, never stored or accessible by Venice. Unlike other AI apps, Venice won't tell you what's okay to say or not. Venice won't patronize you. It simply provides direct access to machine intelligence, no topics are off limits, no ideas, or taboo. With Venice, you're in control

of the AI as you should be. Pro subscriptions are available for $49 a year or $8 per month. AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit venice.aI. slash NLW and enter the discount code NLW Daily Brief. That's NLW Daily Brief, all one word. Today's episode is brought to you by Superintelligent, which is, of course, our platform that helps you learn how to use AI tools and perhaps even more importantly,

gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever. when you sign up for Super Intelligent, between now and the end of August, using code so back, your first month will be 100% free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast,

and with an eye to actually transforming how you get things done. We've just launched Super for Teams, so if you have a group of people at your company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using Super Intelligent as an individual, once again, if you sign up super intelligent between now and the end of the month using code so back you will get your first month 100% free. Go to B super.com. Welcome back to the AI Daily Brief. One of the big questions surrounding the firing and then ultimately rehiring of Sam Altman, the CEO of OpenAI last year, was whether there was some advance, some technical innovation, some breakthrough that had happened,

that had spooked some folks inside OpenAI, creating a rift between the them and Sam Altman in terms of how they would think about releasing that breakthrough onto the world. The way that this was popularized was a phrase, what did Ilya see? Elia was, of course, referring to Ilius Sartzgever, one of the original co-founders of OpenAI and the company's chief scientist. On November 24th, as the whole thing was happening, A16Z's Mark Andreessen wrote, seriously though, what did Ilya see? Elon Musk responded, yeah, something scared Ilya enough to want to fire Sam. What was it? For months this conversation persisted.

Despite the fact that all the people involved with OpenAI said that it wasn't an issue of safety that caused their firing of Sam Altman and a board review after that found the same thing, the question never really went away. And whatever the answer to the question was, it seemed pretty clear to many that Ilya was probably not long for OpenAI. Even though he had dramatically switched sides and come back to working with the company and supporting Sam Altman, he was largely absent for the following six months. Indeed, it was almost exactly six months until May, when Ilya announced on Twitter,

after almost a decade I've made the decision to leave OpenAI. He continued, the company's trajectory has been nothing short or miraculous, and I'm confident that OpenAI will build AGI that is both safe and beneficial. It was an honor and a privilege to have worked together and I will miss everyone dearly. So long and thanks for everything. I'm excited for what comes next, a project that is very personally meaningfully to me, about which I will share details in due time. A little over a month later, we started to get some details.

Ilya announced that he was starting a new company called Safe Super Intelligence or SSI. The company announced itself with a post on X, Super Intelligence is within reach. Building Safe Super Intelligence, SSI, is the most important technical problem of our time. We've started the world's first straight shot SSI lab with one goal in one product, a safe super intelligence. It's called Safe Super Intelligence, Inc. SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve

SSI. We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible, while making sure our safety always remains ahead. This way, we can scale in peace. Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures. The founders of the company included not only Ilya, but prominent AI investor Daniel Gross, as well as AI scientist Daniel Levy. Now, upon the announcement, there was some skepticism. A.I. Safetyist Eliasur Yudkowski wrote, If you have an alignment plan, I can't shoot down in 120 seconds, let's hear it.

So far, you have not said anything different from the previous packs of disaster monkeys, who all said exactly this almost verbatim, but I'm open to hearing better. Still, by and large, the sentiment was excitement, with more than a fair bit of speculation around just how much this company was going to raise. The sense around the AI space was that Ilya likely had a blank check. And now, a couple months later, that certainly appears to be the case. Today, once again, they tweeted, SSI is building a straight shot to safe superintelligence. We've raised $1 billion from NFDG, A16Z, Sequoia, DSD Global and SV Angel. The announcement also came with an exclusive in Reuters about what the company wanted to build. Additional detail shared in Reuters,

the $1 billion came at apparently around a $5 billion valuation, adding in perhaps the most duh sentence I've ever read. The funding underlines how some investors are still wanting to make outsize bets on exceptional talent focused on foundational AI research. The rest of the Reuters article doesn't have that much in the way of details. SSI is saying that it plans to partner with cloud providers and chip companies to fund computing power needs, but hasn't decided who it will work with. Reuters also writes, Sitzkever was an early advocate of scaling, a hypothesis that AI models would improve in performance given vast amounts of computing power.

The idea and its execution kicked off a wave of AI investment in chips, data centers, and energy laying the groundwork for generative AI advances like ChachyPT. Ilya says he will approach scaling in a different way than his former employer without sharing details. He said, everyone just says scaling hypothesis. Everyone neglects to ask, what are we scaling? Some people can work really long hours and they'll just go down the same path faster. It's not so much our style. But if you do something different, then it becomes possible for you to do something special.

So basically, there is a sense of some different approach that they're going to take, although what that different approach is remains obscure or hidden at least to the public. In terms of immediate reactions, some of the discussion was about technical debates. Karam Kaya writes, Elias Zitzgever and Daniel Levy disagree with Jan Lacoon, the chief AI scientist at Meta, on the idea that superintelligence can't be achieved with auto-regressive models.

A significant new wave is approaching, set to push the boundaries of the current architecture. Others pointed out that this probably had something to say about the current state of AI funding. Accelerate Harder, writes, if you were wondering if AI investment is drying up, here's a $1 billion raise for a company that probably doesn't even have a demo yet.

Sure, it's about the team, but that's still quite a data point. Mark Andreessen said we at A16Z are delighted to be on the SSI team and enthusiastic about the mission and strategy. Now, I want to talk about this idea, as reiterated here by Andrew Curran, that SSI's only goal and product is superintelligence. For many, this idea that you could raise a billion dollars on a $5 billion valuation just on the strength of a team will seem ludicrous. They will take it as a sign that the AI bubble is reaching some sort of peak. However, I think that there is a fundamental bifurcation in the way that people are looking at the AI space, particularly when it comes to AGI or superintelligence. On the one hand of those,

who view AI through a traditional business lens of needing to have a return on investment. This is where we get the type of analysis expressed in Sequoia's AI $600 billion question, although I will note that Sequoia was listed among the SSI investors. In any case, the idea here is that the big tech hyperscalers have spent a huge amount building out AI infrastructure and are significantly in the whole on that investment, at least a half trillion dollars. On the flip side, though, are those who think that the opportunity is so large that the possibility of winning that dwarfs any cost right now. As summed up by VC Sarah Tavill in a recent

blog post, if you're a big stack player like meta, Microsoft, Google, or any of the other foundation model pure plays, you have no choice but to keep raising your bet. The prize and power of winning is too great. If you blink, you are left empty-handed watching someone else count your chips. It's likely hundreds of billions will be destroyed and trillions earned. Later, she writes, the prize is theoretically so large, and if a clear winner emerges, their market opportunity so uncapped, you have to keep increasing your bet. This, I think, is the simple logic behind all of this investment. It's as simple as trillions of dollars of upside, handicapped to the possibility that any particular team wins. One of the things that I think is most interesting about SSI's approach is that while many are

wondering how they're ever going to return on investment, when they say they won't have any sort of business model or commercial distractions, I actually wonder if some of the investors are sitting there thinking themselves that that lack of distraction is actually the optimal way to pursue AGI. While XAI, Google, Meta, and Microsoft are all messing around with quarterly reports and the fickle vagaries of Wall Street expectations, SSI blissfully ignores any sort of commercial pressure. There's a certain logic there that I think people are missing. Then again, the question is where will the next billion dollars or really the next

$10 billion come from? As we've discussed recently, even having raised 13 or 14 billion, at this point, OpenAI is likely heading back to the well for more. Will SSI be able to raise similar-sized funds? Or, because they're not planning on releasing a chat GPT competitor, will their costs be dramatically lower? There are all these really interesting questions that to me keep coming back to the idea that a team that's totally unencumbered by any sort of commercial pressure or consumer product pressure might actually be in a better position to win this space. But of course, we're just going to have to wait and see. In any case, it's clear that they have some resources to go after

this new mountain as Ilya called it, and it will be very interesting to see what they come up with. For now, though, that's going to do it for today's AI Daily Brief. Until next time, peace.