Tech Brew Ride Home - Mon. 09/11 – ChatGPT Is Thirsty

Episode Date: September 11, 2023

The Instacart IPO is coming with a helluva haircut. Meta is planning a big new LLM trained on its own stuff. Some eye-watering details on how much water ChatGPT uses. If you don’t do that AI tech, that doesn’t mean someone else won’t. And what tomorrow’s iPhone event says as about Apple’s high-end strategy. Sponsors: Collective.com/ride DraftKings Sportsbook with code TECHMEME for $200 in bonus bets Links: Instacart targets up to $9.3 bln valuation for much-awaited US IPO (Reuters) Meta Is Developing a New, More Powerful AI System as Technology Race Escalates (WSJ) Roblox’s new AI chatbot will help you build virtual worlds (The Verge) Artificial intelligence technology behind ChatGPT was built in Iowa — with a lot of water (AP) The Technology Facebook and Google Didn’t Dare Release (NYTimes) Apple Renews Qualcomm Deal in Sign Its Own Modem Chip Isn’t Ready (Bloomberg) Apple Bets on Titanium and Cameras to Nudge Buyers Toward Pricier iPhones (Bloomberg) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Monday, September 11th, 2023. I'm Brian McCullough today. The Instacart IPO is coming with a hell of a haircut. Meta is planning a big new LLM trained on its own stuff. Some eye-watering details on how much water chat cheapy T uses. If you don't do that AI tech, that doesn't mean someone else won't.

And what tomorrow's iPhone event says about Apple's high-end strategy. Here's what you miss today in the world of tech. They're coming. I believe both. Clavio and Instacart are beginning the road shows for their respective IPOs today. But worth noting this, at the implied valuation, Clavio is looking to price that it's a bit under $9 billion. So that would be slightly, slightly under the valuation of their last venture capital round. Meanwhile, according to filings, Instacart is looking to raise up to $660 million in its IPO at an up to $7.73 billion valuation, which is a significant haircut from their last private market

valuation of $39 billion. So the last time they raised money, it was at a $39 billion valuation, and it is now down to $7.73 billion, who. Which makes this interesting, quoting from Reuters. In an unusual move, Cornerstone investors have agreed to buy up to $400 million worth of shares sold in the offering, which would account for nearly two-thirds of the total proceeds at the top end of the price range. Such investors include Norges Bank Investment Management, a division of Norges Bank, and entities affiliated with Venture Capital Firms TCV, Sequoia Capital, D1 Capital Partners, and Valiant Capital Management. Sequoia and D1 Capital are current backers of Insacart, end quote. So correct me if I'm wrong here, but this sounds like they're trying to gobble up a huge

percentage of the offering to maybe get a first day bump. This implies those existing investors are rolling the dice and trying to recoup some of that haircut by hoping shares slowly appreciate in the capital markets or maybe immediately appreciate. I mean, it wouldn't be insane to think that Instacart could have a few good quarters and raised to an, I don't know, $20 billion evaluation. Anyway, point is, Instacart, another COVID-Times high flyer, would be a good marker to determine where public markets believe this sort of consumer-facing service would be worth in normal times. But more importantly, as Dan Premack points out, the pipeline of other companies hoping to go public in coming months and years, a lot of them look

more like Clavio than Instacart, i.e. their SaaS. So I'll say it again, for the health of the tech IPO market and the tech market generally, the performance of the Arm IPO and the Instacart IPO are interesting, but the one to really keep your eye on is Clavio, as that potentially says more to and about the market. I said a while ago that it occurred to me that next to Nvidia and maybe Microsoft, the company really in the catbird seat in this new AI era is meta. Because think about it. You need content to train the LLMs on and who has more user-generated content than meta across their family of apps. Well, the journal says that meta will soon begin training a new LLM on its own stuff and hopes the model will be roughly as capable as OpenAI's

GPT4. Quote, the planned system, details of which could still change, would help other companies to build services that produce sophisticated text, analysis, and other output. It is the work of a group formed early this year by Meta Chief Executive Mark Zuckerberg to accelerate development of so-called generative AI tools that can produce human-like expressions. Meta expects to start training the new AI system known as a large language model in early 2024, some of the people said. The company is currently building up the data centers necessary for the job and acquiring more H-100s, the most advanced of the Nvidia chips used for such AI training.

While Meta joined with Microsoft to make Lama 2 available on Microsoft's cloud computing platform Azure, it plans to train the new model on its own infrastructure, some of the people said. Zuckerberg is pushing for the new model, like Meta's earlier AI offerings, to be open-sourced and therefore available free for companies to build AI-powered tools. Large language models generally get more powerful when trained on more data. The most powerful version of the Lama 2 model that Meta announced in July was trained on 70 billion parameters, a term for the variables in an AI system that is used to measure size. OpenAI hasn't disclosed the size of GPT4, but it is estimated to be roughly 20 times that size at 1.5 trillion parameters.

Some AI experts say there could be other methods to achieve GPT4's power without necessarily approaching its size, end quote. Another big AI story, Roblox, has unveiled Roblox Assistant, a conversational AI assistant that lets creators enter prompts to generate virtual environments, also to get help with coding and more. Coding the Verge. With the Roblox assistant, creators will be able to type in prompts to do things like generate virtual environments. In one demo, somebody types in, I want to make a game set in ancient ruins, and Roblox drops in some stones, moss-covered columns, and broken architecture. Make the player spawn by a campfire in the ruins, adds a campfire and a stool. Add some trees for the player to chop down, adds nearby trees. Roblox will grab assets from either its marketplace or your own.

own visual asset library, according to Roblox spokesperson Roman Schertifsky. I'm generally skeptical of generative AI, but I think this is a pretty interesting use for the technology. In an interview with the Verge, Roblox CTO Daniel Sterman described how the tool might be able to create basic gameplay behaviors like teleporting you to a place if you touch a door. The Roblox assistant can also help with coding and answer questions about developing on Roblox, which could be useful tools for creators on the platform. Unfortunately, creators won't be able to use Roblox Assistant right away, as Sturman said that the tool is going to launch at the end of this year or early next year. Down the line, Roblox has bigger visions for Roblox assistant,

and Sturman tees that it could generate sophisticated gameplay and even make 3D models from scratch. If all that works, it could bring Roblox in line with CEO David Buzuki's vision of a Westworld-like ease of design, end quote. Westworld, that's an interesting point. If you're not familiar with Roblox, it's sort of like meta-gaming as a platform. it's not just one game, it's thousands of games built on top of Roblox. Roblox itself is just the sandbox in which other people create games. Making it easier to create those games means more games available, but also imagine if

the gaming experience becomes, each gamer can create the game they want to play at any given time on the fly. One of the narratives around crypto has long been how expensive it is in terms of the resources used, the energy required to do the compute, but also the resources required to to keep the machines running doing the compute. Well, new research suggests that chat GPT consumes up to an estimated 500 milliliters of water for every 5 to 50 prompts. This might explain why Microsoft recently reported its water usage spiked 34% year-over-year in 2022. Google reported a 20% year-over-year increase, and that was, again, 2022 before this AI moment had really arrived. Quoting the Associated

press. Building a large language model requires analyzing patterns across a huge trove of human-written text. All of that computing takes a lot of electricity and generates a lot of heat. To keep it cool on hot days, data centers need to pump in water, often to a cooling tower outside its warehouse-sized buildings. It's fair to say the majority of the growth is due to AI, including its heavy investment in generative AI in partnerships with Open AI, said Shalai Ren, a researcher at the University of California Riverside, about Microsoft and who has been trying to calculate the environmental impact of generative AI products such as chat GPT. In a paper due to be published later this year, Wren's team estimates chat GPT gulps up to 500 milliliters of water close to what's in a 16-ounce

water bottle every time you ask it a series of between 5 to 50 prompts or questions. The range varies depending on where its servers are located and the season. The estimate includes indirect water usage that companies don't measure, such as to cool power plants that supply the data centers with electricity. Most people are not aware of the resource usage underlying chat GPT Wren said, if you're not aware of the resource usage, then there's no way that we can help conserve the resources, end quote. In response to questions from the Associated Press, Microsoft said in a statement this week that it is investing in research to measure AI's energy and carbon footprint, quote, while working on ways to make large systems more efficient in both training and application.

We will continue to monitor our emissions, accelerate progress while increasing our use of clean energy, to power data centers, purchasing renewable energy, and other efforts to meet our sustainability goals of being carbon-negative, water positive, and zero waste by 2030, the company's statement said. OpenAI echoed those comments in its own statement Friday, saying it's giving considerable thought to the best use of computing power, end quote. The narrative has been that Google sat on its AI advances, thereby allowing OpenAI to burst onto the scene, that meta was a bit behind the AI moment because it was maybe focused on other

things. Well, the Times has an interesting piece up outlining how meta and Google held back other tech, other AI tech that was used to recognize unknown people's faces due to privacy worries. But that had the unintended consequence of just opening the door for competitors. Startups like Clearview AI and Primeyes startups that privacy activists hate. Quote, in recent years, the startups Clearview AI and PIMIs have pushed the boundaries of what the public thought was possible by releasing face search engines paired with millions of photos from the public web, PIMEyes, or even billions. Clearview. With these tools available to the police in the case of Clearview AI and the public at

large in the case of Pimeyes, a snapshot of someone can be used to find other online photos where that face appears, potentially revealing a name, social media profiles, or information a person would never want to be linked to publicly, such as risque photos. What these startups had done wasn't a technological breakthrough, it was an ethical one. Tech giants had developed the ability to recognize unknown people's faces years earlier, but had chosen to hold the technology back, deciding that the most extreme version, putting a name to a stranger's face, was too dangerous to to make widely available. As early as 2011, a Google engineer revealed he had been working on a tool to Google someone's face and bring up other online photos of them. Months later, Google's chairman,

Eric Schmidt said in an onstage interview that Google, quote, built that technology and we withheld it. As far as I know, it's the only technology that Google built, and after looking at it, we decided to stop, Mr. Schmidt said. Advertently or not, the tech giants also helped hold the technology back from general circulation by snapping up the most advanced startups that offered it. In 2010, Apple bought a promising Swedish facial recognition company called Polar Rose. In 2011, Google acquired a U.S. face recognition company popular with federal agencies called Pit Pat, and in 2012, Facebook purchased the Israeli company Face.com. In each case, the new owners shut down the acquired company services to outsiders.

The Silicon Valley heavyweights were the de facto gatekeepers for how and whether the tech would be used. Facebook, Google, and Apple deployed facial recognition technology in what they considered to be relatively benign ways, as a security tool to unlock a smartphone, a more efficient way to tag known friends and photos, and an organizational tool to categorize smartphone photos by the faces of the people in them. In the last few years, though, the gates have been trampled by smaller, more aggressive companies such as Clearview AI and PIMEyes. What allowed the shift was the open source nature of neural network technology, which now underpins most artificial intelligence software, end quote. I guess that whole project inside of Apple to create their own modems, and at that point basically make the majority of the major components of the iPhone in-house, has hit a snag. Qualcomm says Apple has extended its modem deal for three more years, covering smartphone launches in 2024,

2025, and 26. The prior deal was supposed to end this year, quoting Bloomberg. The company's agreement had been set to end in 2023, and the latest iPhone due on Tuesday was expected to be one of the last to rely on the Qualcomm modem chip. Instead, Qualcomm will maintain its lucrative position within Apple's supply chain. The iPhone maker is Qualcomm's largest customer accounting for nearly a quarter of revenue, according to data compiled by Bloomberg. And their relationship helps validate Qualcomm's claims to having the best smartphone modem, a critical component that allows devices to connect to the internet and make calls. Starting with the iPhone 12 generation, the chip has supported speedier 5G networks. For Apple, the move suggests that building a modem

component has been more challenging than expected. The effort has been years in the works. The Cooper Tino-California-based company kicked off the project in 2018 and then bolstered it with the acquisition of Intel's smartphone chip business in 2019. By 2020, Apple heralded the development of its own modem as a key strategy transition. Johnny Shrugi, its chip chief, said at the time that the work was full speed ahead. Some analysts expected the component to be ready for the 2023 iPhone, but Qualcomm quashed that speculation last year. Apple was still looking. looking to ship the modem at either the end of 2024 or in early 2025, Bloomberg News previously reported. Now the project has a longer runway before it needs to be ready. It's been a difficult

undertaking. Apple needs a chip that can connect to various cellular networks globally without fail while offering as good or better performance as Qualcomm. It's run into trouble with battery life, and there are bureaucratic challenges such as certifying the modem with authorities, end quote. And finally, iPhone event tomorrow. There are various roundups of what we can expect in terms of the devices and the features. But I wanted to highlight this particular strategic framing from Mark German. Tomorrow might be all about the high-end iPhone, the expected price increase for that high-end iPhone, and features like iPhone 15 Pro Max's Periscope Lens, which is exclusive to that device. This, according to German, shows Apple is getting more aggressive with its premium strategy.

Quote, the company will further differentiate its iPhone 15 Pro models from the standard models by giving them better battery life, faster USBC data transfer speeds, thinner borders, and nicer screens. The Pro versions will also get a customizable action button and a faster chip. Apple also is adding an extra enticement to the top of the line Pro Max. The phone will offer a wider range of optical zoom via a so-called periscope lens. This system doubles the iPhone's ability to zoom in on an object via the hardware lens from 3x to about 6x.

In past years, the two Pro models typically only differed, in terms of screen size and battery capacity. They shared the same features, internal components, and specifications. The last time Apple differentiated its high-end phones was 2020, when the iPhone 12 Pro Max had slightly superior zoom and stabilization. But the Periscope Edition is more significant, so the new approach shows Apple is getting more aggressive with its premium strategy. The company will also push customers toward its more expensive accessories,

continuing a trend from the September 2020 launch. The new Apple Watches will all get a major performance up, upgrade, but users will need to spring for the pricier Ultra to get a titanium case, larger screen, and longer battery life. And if customers want AirPods that support USBC charging, they'll need to buy the costlier model, at least for now, end quote. So yeah, right there, you heard it, iPhone event tomorrow. So usual caveat that tomorrow's show will be slightly later than normal, so I can actually watch the event itself. Talk to you sometime tomorrow afternoon.