Tech Brew Ride Home - Tue. 04/09 – Google’s New AI Chips

Episode Date: April 9, 2024

Google has a ton of AI announces, including new Arm-based AI chips, utilizing Google search in Gemini, and more. They also released their own Find My network. Microsoft is confident they can release chips that can best Apple Silicon. And maybe OpenAI DID train on YouTube videos after all. Everybody is desperate for data right now. Sponsors: 1Password.com/ride Links: Google Expands In-House Chip Efforts in Costly AI Battle (WSJ) Google Shows AI Model Is Enterprise-Ready After Gemini Mishaps (Bloomberg) Gmail adding voice input, Gemini for Google Chat, Meet ‘Translate for me,’ & more (9to5Google) Google rolling out Find My Device network for Android (9to5Google) Microsoft is confident Windows on Arm could finally beat Apple (The Verge) Spotify launches personalized AI playlists that you can build using prompts (TechCrunch) How Tech Giants Cut Corners to Harvest Data for A.I. (NYTimes) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the TechMeme right home for Tuesday, April 9th, 2024. I'm Brian McCullough today. Google has a ton of AI announces, including new arm-based AI chips,

utilizing Google Search and Gemini and more. They also release their own FindMine network. Microsoft is confident they can release chips that can best Apple Silicon, and maybe OpenAI did train on YouTube videos after all. Everybody is desperate for data right now. Here's what you miss today in the world of tech. Google had a whole bunch of AI announcements. morning, including its new arm-based Axion chips for AI, with up to 30% better performance

over arm-based rivals set to launch to cloud customers later this year. Quoting the journal, The chip efforts promised to reduce Google's reliance on outside vendors and bring it into competition with longtime partners such as Intel and Nvidia analysts said. Google officials said they didn't view this as a competition. I see this as a basis for growing the size of the pie, said, I mean, Vadat, the Google Vice president overseeing the company's in-house chip operations. Axion is based on circuitry from the British chip design firm Arm, making Google the third big tech company after Amazon and Microsoft use that framework for a data center CPU. The shift has supplanted an old status quo where big

operators of server farms bought their CPUs almost exclusively from Intel and advanced micro devices. Google has resisted selling chips directly to customers to install in their own data centers. The move would push the company more directly into competition with Intel and Nvidia, the biggest winner of the AI boom so far with more than 80% of the market for chips used to develop and serve the technology. Becoming a great hardware company is very different from becoming a great cloud company or a great organizer of the world's information. Google's VAT said, Google has chosen instead to rent custom chips to cloud customers.

It said the Axion chips will become accessible to external customers later this year, and the latest generation of its TPUs was now widely available, end quote. But wait, there's more, more announces, including that Google thinks Gemini has finally cleaned up its act after initial missteps, quoting Bloomberg. At the company's annual cloud computing conference in Las Vegas on Tuesday, Cloud Chief Executive Officer Thomas Currian, showed off how Google's most powerful AI model Gemini can be used to create advertisements, ward off cybersecurity threats, and spin-up short videos and podcasts. corporate customers will be able to peg Gemini's query responses to reliable sources of information known as grounding. The company is rolling out the use of Google search results

as a source for the AI model's answers, thereby providing greater accuracy and freshness, Curion said, end quote. That had to happen sometime. Google has Google search, which nobody else has, so you'd think they'd have a natural leg up in terms of timeliness of data and thus responses. Also, maybe accuracy, TBD, I guess. Google also announced, Vids, an app to help companies and consumers make collaborative shareable video powered by Gemini, launching in public beta this summer. They also made the Imagine 2 image generator generally available in vertex AI and added in-painting and outpainting to remove or add elements and expand borders. They released Gemini 1.5 Pro in public preview on vertex AI with support for up to 1 million tokens,

and said the model will power new features in Code Assist. And some stuff that even non-AI developers could use soon, quoting 9 to 5 Google. Besides Vids, Google Workspace is getting a slew of new features and Gemini capabilities at Cloud Next 2024. Google is updating, help me write in mobile Gmail with voice prompting and input that lets you, quote, send emails easily when you're on the go. Meanwhile, an instant polish feature will, quote, convert rough notes to a complete email with one click. Google shared today that, quote, 70% of enterprise users who use help me write in docs or Gmail end up using Gemini's suggestions, end quote. Google Docs is adding a tabs feature that can, quote,

organize information in a single document instead of linking to multiple documents or searching through Drive to find what you're looking for. Additionally, Docs is getting support for full bleed cover images at the top with both those features entering general availability in the coming weeks. Google Sheets is getting a tables feature that, quote, does the heavy lifting to format and organize data with a sleek design. Gemini is coming to Google chat to summarize conversation and answer questions. On the web, it also appears as a side panel. Chat is also getting automatic

message translation later this year, while Google is increasing spaces capacity to 500,000 members. Google Meet is adding Translate for Me to automatically detect and translate captions in 52 new languages for a total of 69 or 4,600 language pairs. Meanwhile, take notes for me is in preview. At Google Cloud Next 2024, Google announced that Gemini in chat and Meet will be available as part of a new AI meetings and messaging add-on for enterprise that costs $10 per user per month, Google also announced a similarly priced AI security add-on that, quote, allows IT teams to automatically classify and protect sensitive files company-wide in Google Drive, end quote. Not AI, but earlier, Google also debuted its Find My Device Network in the U.S. and Canada,

initially for lost Android phones and tablets, though third-party tracker tag support is coming in May. Quoting 9 to 5 Google again. This crowdsourced FMD network will use over a billion Android 9 plus devices to find your lost items. It was first announced at Google I.O. 2023 and missed the original summer target. Previously, the Find My Device app required your lost phone to have an internet connection, Wi-Fi or cellular. This new offering works even if they're both offline by leveraging Bluetooth. The Pixel 8 and 8 Pro can be found even, quote, if they're powered off or the battery is dead.

Google credits specialized pixel hardware while it's working with other Android OEMs and chipmakers on expanding this capability to premium devices. Meanwhile, you'll be able to find compatible Bluetooth tracker tags in May, which is when iOS 17.5 is expected with unknown tracker alerts. Chipolo and Pebblebee will release FMD devices next month with Ufi, Geo, Motorola, and more expected to release accessories later this year. You can also expect JBL, Sony, and others to update their headphones to support the Find My device network. Google has nothing to share today about where OS smartwatch is getting this capability,

though. All this is accessed from the Find My Device app, which will show a list of your devices. Select one and tap Find Nearby to play sounds and get hints like devices close, very close, and it's here. It will take a moment to connect with Google using a material use shape that fills as you get closer to the device, end quote. With the whole chip explosion, we've been seeing thanks to AI, we can forget that the chip explosion. We can forget that the chip explosion, really started to happen when Apple Silicon came out and seemed to be a generational leap for traditional computers. Well, Tom Warren's sources are telling him that Microsoft is confident upcoming Windows laptops with Qualcomm's Snapdragon X Elite chips will beat an M3-powered

MacBook Air in CPU and AI performance. Quoting the verge, after years of failed promises from Qualcomm, Microsoft believes the upcoming Snapdragon X elite processors will finally offer the performance it has been looking for to push Windows on Arm much more aggressively. Microsoft is now betting big on Qualcomm's upcoming Snapdragon X elite processors, which will ship in a variety of Windows laptops this year and Microsoft's latest consumer-focused surface hardware. Microsoft is so confident in these new Qualcomm chips that it's planning a number of demos that will show how these processors will be faster than an M3 MacBook Air for CPU tasks, also AI acceleration, and even app emulation. Microsoft claims in internal documents seem to

by the verge that these new Windows AIPCs will have, quote, faster app emulation than Rosetta 2, the application compatibility later that Apple uses on its Apple Silicon Macs to translate apps compiled for 64-bit Intel processors to Apple's own processors. App emulation has been a big problem for Windows on Arm over the past decade, but Microsoft did deliver X-64 app emulation for Windows 11 more than two years ago. This helps ensure apps can run on Windows on arm devices when there isn't a native Arm 64 version. Native Arm apps are key for improved performance on upcoming Windows on Arm laptops, and Google just recently released its own Arm 64 version of Chrome, ready for these upcoming devices. I'm also told that Microsoft is planning to ship

consumer models of its upcoming Surface Pro 10 and Surface Laptop 6 with Qualcomm's Snapdragon X Elite processors inside instead of Intel's Core Ultra Chips. Microsoft has already announced business-focused versions of the Surface Pro 10 and Surface Laptop 6 that ship with Intel Core Ultra processors, but the ARM models will be marketed towards consumers instead. Microsoft describes devices running Qualcomm, Snapdragon X-Eleet processors as next-gen AI copilot PCs internally, meaning to differentiate them from existing PCs that run on AMD's latest chips or even Intel's core ultra processors. This new class of PCs will get access to new AI-powered Windows features first, including

an AI Explorer app that lets you, quote, retrieve anything you've ever seen or done on your device. Some things that we missed while I was away, like Spotify debuting AI playlist in beta for its premium users in the UK and Australia, letting Android and iOS users generate and revise playlist using text prompts, quoting TechCrunch. In addition to more standard playlist creation requests like those based on genre or time frame, Spotify's use of AI means people can ask for a wider variety of custom playlist like songs to serenade my cat or beats to battle a zombie apocalypse. Spotify suggests. Promps can reference all sorts of things like places, animals, activities,

movie characters, colors, or emojis. The company notes that the best playlists are generated using prompts that contain a combination of genres, moods, artists, and decades, however. Spotify also leverages its understanding of users' taste to customize the playlist it makes with the feature. After the playlist is generated, users can then use the AI to revise and refine the end result by issuing commands like less upbeat or more pop, for example. Users can also swipe left on any songs to remove them from the playlist. In terms of the technology, Spotify says it's using large language models to understand the user's intent. Then Spotify uses its personalization technology, the information it has about the listener's history and preferences,

to fulfill the prompt and create a personalized AI-generated playlist for the user. The company uses a range of third-party tools for its AI and machine learning experiences. The feature is found in the Your Library tab in Spotify's app by tapping on the plus button at the top right of the screen, a pop-up menu appears showing the AI playlist as a new option alongside the existing playlist and blend options. If a listener can't think of any prompts to try, Spotify offers prompt suggestions to help people get started, like Get Focused at Work with Instrumental Electronica, fill in the silence with background cafe music, get pumped up with fun, upbeat, and positive songs, or explore a niche genre like Witch House, and many others. To save an AI

playlist, tap the Create button to add it to the library, the company notes the AI has guardrails around it, so it will not respond to offensive prompts or those focused on current events or specific brands, end quote. Finally today, sources told the times that maybe OpenAI did indeed transcribe more than one million hours of YouTube videos through a program called Whisper and use that text to train GPT4. Google also might have transcribed YouTube videos itself to harvest text, though, so these details, if true, would contradict statements from both companies. Quoting the New York Times. Open AI researchers created a speech recognition tool called Whisper. It could transcribe the audio

from YouTube videos yielding new conversational text that would make an AI system smarter. Some OpenAI employees discussed how such a move might go against YouTube's rules, three people with knowledge of the conversation said. YouTube, which is owned by Google, prohibits the use of its videos for applications that are, quote, independent of the video platform. Ultimately, an open AI team transcribed more than one million hours of YouTube videos, the people said. The team included Greg Brockman, OpenAI's president, who personally helped collect the videos two of the people said. The texts were then fed into a system called GPT4, which was widely considered one of the world's most powerful AI models and was the basis of the latest version of the chat GPT chatbot.

But like OpenAI, Google transcribed YouTube videos to harvest text for its AI models. Five people with knowledge of the company's practices said that potentially violated the copyrights to the videos, which belonged to their creators. Last year, Google also broadened its terms of service. One motivation for the change, according to members of the company's privacy team and an internal message viewed by the Times, was to allow Google to be able to tap publicly available Google Docs, restaurant reviews on Google Maps, and other online material for more of its AI products, end quote. You get the sense that this is a free-for-all right now, all of these companies trying to get good training data. Here's more fuel to that fire. Get this wild anecdote.

quoting from the piece again. The volume of data is crucial. Leading chatbot systems have learned from pools of digital texts spanning as many as 3 trillion words or roughly twice the number of words stored in Oxford University's Bodleyian Library, which has collected manuscripts since 1602. The most prized data, AI researchers said, is high-quality information such as published books and articles, which have been carefully written and edited by professionals. Their situation is urgent. Tech companies could run through the high-quality data on the internet as soon as 2026, according to Epoch, a research institute. The companies are using the data faster than it is being produced. The only practical way for these tools to exist is if they can be trained on massive amounts

of data without having to license that data. Sydomley, a lawyer who represents Andresen Horowitz, a Silicon Valley venture capital firm, said of AI models last year in a public discussion about copyright law, the data needed is so massive that even collective licensing really can't work." End quote. Tech companies are so hungry for new data that some are developing synthetic information. This is not organic data created by humans, but text, images, and code that AI models produce themselves. In other words, the systems learn from what they themselves generate. At Meta, which owns Facebook and Instagram, managers, lawyers, and engineers last year discussed buying the publishing house Simon and Schuster to procure long works, according to recordings

of internal meetings obtained by the Times. They also conferred on gathering copyrighted data from across the internet, even if that meant facing lawsuits. Negotiating licenses with publishers, artists, musicians, and the news industry would take too long, they said. In January 2020, Jared Kaplan, a theoretical physicist at Johns Hopkins University, published a groundbreaking paper on AI that stoked the appetite for online data. His conclusion was unequivocal. The more data there was to train a large language model, the technology that drives online chatbots, the better it would perform. Just as a student learns more by reading more books, large language models can better pinpoint patterns in text and be more accurate with more information.

Everyone was very surprised that these trends, these scaling laws, as we call them, were basically as precise as what you see in astronomy or physics, said Dr. Kaplan, who published the paper with nine OpenAI researchers. He now works at the AI Startup Anthropic. Scale is all you need soon became a rallying cry for AI, end quote. Thank you to all of you who tweet. at me over the weekend and emailed me and we're like, don't be an ass, Brian, go do this family adventure, be with your kids. As you can tell, I did that. It was wild, truly a cool experience. Not so cool was the 10 hours we spent in the car yesterday going up to Northern Vermont and back, didn't get home till 11.30 last night. Excuse me while I go take a nap. Talk to you

tomorrow.