Tech Brew Ride Home - Thu. 06/20 - Ilya Sutskever’s New AI Startup

Episode Date: June 20, 2024

Ilya Sutskever wants to go straight to Safe Superintelligence, do not pass go, but do probably collect hundreds of millions of dollars. Is Perplexity ignoring robots.txt files? Xreal’s hybrid AR glasses play. And how many apps did Apple sherlock at WWDC last week? Links: Ilya Sutskever Has a New Plan for Safe Superintelligence (Bloomberg) Perplexity Is a Bullshit Machine (Wired) For Apple’s AI Push, China Is a Missing Piece (WSJ) Xreal’s new gadget is a phone-sized Android tablet just for your AR glasses (The Verge) iOS 18 could ‘sherlock’ $400M in app revenue (TechCrunch) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Thursday, June 20th, 20th, 20th, 204. I'm Brian McCullough today. Ilyssus Soskever wants to go straight to safe superintelligence.

Do not pass go, but do probably collect hundreds of millions of dollars. It is perplexity ignoring robots.comptych files, X-Reels hybrid AR glasses play, and how many apps did Apple Sherlock at WWDC last week? Here's what you miss today in the world of tech. Well, we now know what Ilya Saskiever's next act will be, now that he's left Open AI, along with Daniel Gross and Daniel Levy, he is founding safe superintelligence, a new AI startup, quote, with one goal and one product, a safe superintelligence. Gross is the former AI lead at Apple and is Nat Friedman's frequent investing partner,

if you remember from that bonus episode with Nat Levy, worked alongside Suskever at OpenAI, quoting from Bloomberg. This company is special in that its first product, will be the safe superintelligence, and it will not do anything else up until then. Susgever says in an exclusive interview about his plans, it will be fully insulated from the outside pressures of having to deal with a large and complicated product and having to be stuck in a competitive rat race, end quote. Suskever declines to name safe superintelligences, financial backers, or disclose how much he's raised. As the company's name emphasizes, Susgever has made AI safety the top priority.

The trick, of course, is identifying what exactly makes one AI system safer than another, or really safe at all. Susgever is vague about this at the moment, though he does suggest that the new venture will try to achieve safety with engineering breakthroughs baked into the AI system, as opposed to relying on guardrails applied to the technology on the fly. By safe, we mean safe-like nuclear safety, as opposed to safe as in trust in safety, he says. I think the time is right to have such a project, says Levy. My vision is exactly the same as Ilias, a small, lean, cracked team with everyone focused on the single objective of a safe superintelligence. Safe superintelligence will have offices in Palo Alto, California, and Tel Aviv. Both Suskever and Gross grew up in Israel. Given Sutskev's near-mythical

place within the AI industry, the uncertainty around his status has been a Silicon Valley fixation for months. As a university researcher and later a scientist at Google, he played a major role in developing a number of key AI advances. His involvement in the early days of OpenAI helped it attract the top talent that's been essential to its success. Suskever became known as its major advocate for building ever larger models, a strategy that helped the startup surge past Google and proved crucial to the rise of ChatGBT. Safe superintelligence is in some ways a throwback to the original OpenAI concept, a research organization trying to build an artificial general intelligence that could equal and perhaps

surpass humans on many tasks. But OpenAI's structure evolved, as the need to raise vast sums of money for computing power became evident. This led to the company's tight partnership with Microsoft and contributed to its push to make revenue-generating products. All of the major AI players face a version of the same conundrum, needing to pay for ever-expanding computational demands as AI models continue to grow in size at exponential rates. These economic realities make safe superintelligence a gamble for investors who would be betting that Susquever and his team will hit on breakthroughs to give it an edge over rivals with larger teams and significant headstarts. Those investors will be putting down money without the hope

of creating profitable hit products along the way, and it's not at all clear that what safe superintelligence hopes to make is even possible. With superintelligence, the company is using AI industry lingo to refer to a system that would be in another league than the human-level AI's most of the larger tech players are pursuing. There's no consensus in the industry about whether such an intelligence is achievable or how a company would go about building one. That said, safe superintelligence will likely have little trouble raising money, given the pedigree of its founding team and the intense interest in the field. Out of all the problems we face, raising capital is not going to be one of them, says Gross. Researchers and intellectuals have contemplated making AI systems safer for decades,

but deep engineering around these problems has been in short supply. The current state of the art is to use both humans and AI to steer the software in a direction aligned with humanity's best interests. Exactly how one would stop an AI system from running amuck remains a largely philosophical exercise. Suskeever says that he's spent years contemplating the safety problems and that he already has a few approaches in mind. But safe superintelligence isn't yet discussing specifics. At the most basic level, safe superintelligence should have the property that it will not harm humanity at a large scale, Suskeever says. After this, we can say we would like it to be a force for good. We would like to be operating on top of some key values. Some of the values we were thinking about are maybe the values

that have been so successful in the past few hundred years that underpin liberal democracies, like liberty, democracy, freedom, end quote. So Skever says that the large language models that have dominated AI will play an important role within safe superintelligence, but that it's aiming for something far more powerful. With current systems, he says, you talk to it, you have a conversation, and you're done. The system he wants to pursue would be more general purpose and, expansive in its abilities. You're talking about a giant super data center that's autonomously developing technology. That's crazy, right? It's the safety of that that we want to contribute to, end quote. Meanwhile, while Forbes continues to threaten AI search company perplexity with legal action for allegedly

plagiarizing its content, a new analysis alleges that perplexity also seems to be scraping sites using seraptitious methods, ignoring robots. text files, with a perplexity tied machine doing so on Wired and other sites, quoting from Wired itself. A Wired analysis, and one carried out by developer Rob Knight suggests that Perplexity is able to achieve this, partly through apparently ignoring a widely accepted web standard known as the Robots' Exclusion Protocol to surreptitiously scrape areas of websites that operators do not want access by bots, despite claiming that it won't. Wired observed a machine tied to perplexity, more specifically one on an Amazon server and almost certainly operated by Perplexity doing this on Wired.com, and across

other Kandai Nass publications. Wired provided the perplexity chatbot with the headlines of dozens of articles published on our website this year, as well as prompts about the subjects of Wired reporting. The results showed the chatbot at times closely paraphrasing Wired stories and at times summarizing stories inaccurately and with minimal attribution. In one case, the text it generated falsely claimed that Wired had reported that a specific police officer in California had committed a crime. The AP similarly identified an instance of the chatbot attributing fake quotes to real people. Despite its apparent access to original Wired reporting and its site hosting original Wired art, though none of the IP addresses publicly listed by the company left any identifiable trace

in our server logs, raising the question of how exactly Perplexity's system works. Until earlier this week, Perplexity published in its documentation a link to a list of the IP addresses its crawlers use an apparent effort to be transparent, however, in some cases as both Wired and Knight were able to demonstrate, it appears to be accessing and scraping websites from which coders have attempted to block its crawler, called Perplexity Bot, using at least one unpublicized IP address. The company has since removed references to its public IP pool from its documentation. That secret IP address, 44.21.181.252, has hit properties at Kande Nas, the media company that owns Wired, at least 822 times in the past three months. One senior engineer at Kondi

Andes Nass, who asked not to be named because he wants to, quote, stay out of it, calls this a, quote, massive undercount because the company only retains a fraction of its network logs. Wired verified that the IP address in question is almost certainly linked to perplexity by creating a new website and monitoring its server logs. Immediately after a Wired reporter prompted the perplexity chatbot to summarize the website's content, the server logged that the IP address visited the site. This same IP address was first observed by night during a similar test. It also appears probable that in some cases, and despite a graphical representation in its user interface, that shows the chatbot reading specific source material before giving a reply to a prompt,

perplexity is summarizing not actual news articles, but reconstructions of what they say based on URLs and traces of them left in search engines like extracts and metadata, offering summaries purporting to be based on direct access to the relevant text. The magic trick that's made perplexity worth ten figures, in other words, appears to be that it's both doing. what it says it isn't and not doing what it says it is, end quote. An angle I hadn't thought of with regards to Apple intelligence is think about what's a big important market for Apple? China. What doesn't operate in China? Open AI. So, sources are telling the journal that Apple has held talks with Baidu, Alibaba, Baichuan AI, and others to help offer Apple intelligence in China. Quote, in China, Apple is

falling behind local rivals that have already incorporated AI functions into their phones. The iPhone dropped to third place by handset market share among smartphone brands in China in the first quarter of this year behind two local brands, according to Counterpoint Research. Apple has held talks with several Chinese companies that make AI models, including search engine company Baidu, e-commerce leader Alibaba Group, and a Beijing-based startup called Bakchuan AI, people familiar with the matter said. In China, companies must seek Beijing's approval before introducing AI chatbots built on large language models trained with huge databases of text, images, and video, vacuumed up from the

internet and other sources. Regulators vet the models to ensure they don't influence public opinion in a way the government doesn't approve of. As of March, Beijing's internet watchdog, the cyberspace administration of China had approved 117 generative AI products, none of which is foreign developed. Early this year, Apple explored the possibility of obtaining approval for a foreign large language model to be used in its devices in China, but it found that Chinese regulators were unlikely to approve it, people familiar with the matter said. That realization prompted Apple to step up talks with potential local partners, they said. Apple said the region that includes mainland China, Taiwan, Hong Kong, and Macau accounted for 18% of its global revenue in the quarter

ended March 30th. Its position is threatened by local companies. This year, Huawei is expected to account for 17% of China's smartphone market, up from 13% last year, while Apple's share is projected to drop to 16% from 18% according to counterpoint, end quote. X-Reel has unveiled the $199 Beam Pro, a 6.5-inch device that powers X-Reel's glasses, runs Android 14-based Nebula OS, uses a Snapdragon chip, and has two USB-C ports, the better to charge while you're also connected to the AR glasses. Quoting the Verge. It's a handheld device with the rough dimensions of a smartphone, but X-Rel thinks of it more as a companion to your glasses. On the back, there's a dual-loor. lens camera. You can use to take spatial and 3D videos for viewing in your glasses or your Vision Pro. Xreal says the Beam Pro's footage will work in your Apple headset too. When you have your glasses plugged in, you can use the Beam's screen as a touchpad and the device is also designed

to have two apps open side by side in your field of view. When you first plug in the glasses, it'll pop up a home screen of your apps, which you can open and control using the Beam Pro as a remote. The Beam Pro looks like a big upgrade on the Beam, which was essentially just a remote control for your X-Rail glasses. The Beam definitely solved a problem for X-Real owners, but it had some issues. A bunch of reviewers and users found it was fiddly and unreliable, and X-Reel had a hard time explaining to users why it even existed in the first place. The screen should make the pro much easier to use, and the camera makes it more than just a lesser smartphone replacement. You can, of course, still plug in your Steam Deck or a smartphone and use X-Reel's glasses that way,

but this feels like a more integrated approach. X-Reel's approach is much less integrated than what we're seeing from Apple and meta, both of which are determined to put a whole computer on your face. But there's something clever about X-Real's way. It's using a totally mature device category to do all the hard work, and doing as little on your face as possible. At least for now, it feels like a smart strategy, end quote. Finally today, stories like this one have been floating around since last week, so let me catch this one as representative of them all. According to app figures, iOS 18, could Sherlock apps that have an estimated $393 million in annual revenue, quoting TechCrunch. Apple's practice of leveraging ideas from its third-party developer community to become new iOS and Mac features and apps has a hefty price tag.

A new report indicates, with the release of iOS 18 later this fall, Apple changes may affect apps that today have an estimated 393 million in revenue and have been downloaded roughly 58 million times over the past year, according to an analysis by app intelligence firm app figures. Every June at Apple's Worldwide Developers Conference, the iPhone maker teases the upcoming releases of its software and operating systems, which often include features previously only available through third-party apps. The practice is so common now. It's even been given a name. Sherlocking, a reference to a 1990s search app for Mac

that borrowed features from a third-party app known as Watson. Now, when Apple launches a new feature that was before the domain of a third-party app, it is said to have Sherlocked the app. In earlier years, Sherlocking apps made some sense. After all, did the iPhone's flashlight really need to be a third-party offering, or would it be better as a built-in function? Plus, Apple has been able to launch features that have made its software better adapted to consumers once and needs by looking at what's popular among the third-party developer community. Of course, this practice also raises the question as to whether Apple is leveraging proprietary data to make its determinations about what to build next and whether

the apps it competes with are being offered an even playing field. For example, before Apple launched its own parental control system, it shut down many third-party apps that had built businesses in this space by saying their solutions were now non-compliant with its rules and policies. The apps weren't offered access to a developer API for managing Apple's built-in parental controls for years, prompting an antitrust investigation. In more recent years, Apple has Sherlock third parties with launches of features like continuity camera, medication tracking, sleep tracking and mood tracking, as well as apps like Freeform and Journal. This year, it turned its attention to password managers, call recording, and transcription apps,

those for making custom emoji, AI-powered writing tools, and math helpers, trail apps, and more. In an analysis of third-party apps that generated more than 1,000 downloads per year, app figures discovered several genres that had found themselves in Apple's crosshairs in 2024. In terms of worldwide gross revenue, those categories have generated significant income over the past 12 months, with the trail app category making the most at $307 million per year, led by Market Leader and 2023 Apple App of the Year, all trails. Grammer helper apps, like Grammally and others, also generated $35.7 million, while math helpers and password managers earn $23.4 million and $20.3 million, respectively.

Apps for making custom emoji generated $7 million. Of these, trail apps accounted for the vast majority of potentially Sherlock's revenue, or 78%, noted app figures as well as 40% of downloads of Sherlock apps. In May 2024, they accounted for an estimated 28.8 million in gross consumer spending and 2.5 million downloads to give you an idea of scale. Many of these app categories were growing quickly with math solvers having seen revenue growth of 43% year over year, followed by grammar helpers, up 40%, password managers, up 38% and trail apps up 28%. Emboji making apps, however, we're seeing declines of negative 17% year over year. Although these apps certainly have dedicated user bases that may not immediately choose to switch to a first-party offering,

Apple's ability to offer similar functionality built-in could be detrimental to their potential growth. Casual users may be satisfied by Apple's good enough solutions and won't seek out alternatives, end quote. There is nothing like international tournaments in the summer. Football on all day long while I work. Come on, England. Talk to you tomorrow.