Tech Brew Ride Home - Wed. 12/06 – Google’s OpenAI Competitor Announced

Episode Date: December 7, 2023

Say hello to Gemini, Google’s new AI model designed to go head to head with OpenAI. Governments have been spying on your phone’s notifications, but Apple couldn’t tell you about that until now. Why is Twitch shutting down service in what is one of the biggest markets in the world for esports? And another way to hide your green bubbles in iMessage. Sponsors: CrashPlan.com/ridehome The Traceroute Podcast Links: Meet Gemini, the AI That Google Says Is Way, Way Better Than ChatGPT (Gizmodo) Google’s Gemini AI model is coming to the Pixel 8 Pro — and eventually to Android (The Verge) Google announces the Cloud TPU v5p, its most powerful AI accelerator yet (TechCrunch) Governments spying on Apple, Google users through push notifications - US senator (Reuters) Twitch plans to shut down in South Korea on February 27, 2024 (VentureBeat) There’s a new iMessage for Android app — and it actually works (The Verge) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Wednesday, December 6, 2023. I'm Brian McCullough today. Say hello to Gemini, Google's new AI model designed to go head-to-head with OpenAI. Governments have been spying on your phone's notifications, but Apple couldn't tell you about that until now. Why is Twitch shutting down service in what is one of the biggest markets in the world for esports? And another attempt to bridge the blue and green bubble divide in eye message. Here's what you missed today in the world of tech. Well, I guess they had faked us, so Mayaculpa? Google this. Google this. morning announced Gemini, a new AI model with three tiers, Ultra, Pro, and Nano, and plans a paid version in 2024. Google says the new Bard beats free chat GPT, but generally they want you to know

that Gemini is going to go head-to-head with everything OpenAI does, quoting Gizmodo. Gemini, the company's most powerful AI to date, comes to Bard and Pixel 8 Pro smartphones starting today and will soon integrate with other products across Google services, including Chrome, search, ads, and more. Google has a top-line message it wants you to hear. This thing is way better than anything you'll get from OpenAI. Gemini comes in three tiers. Gemini Ultra is Google's most powerful model, pitched as a competitor to OpenAI's GPT4. Gemini Pro is a mid-range model, powered to beat out GPT3.5, the baseline version of ChatGPT. Last is Gemini Nano, a more efficient model built to run on mobile devices. As of Wednesday, Bard is running on a

finely tuned version of Gemini Pro, said Sophie Sharon, Vice President of Google Assistant and Bard at a press conference. This will have more advanced reasoning, planning, understanding, and other capabilities, she said. Sheron said, Google will roll out a paid version of the chatbot running on Gemini Ultra early next year that the company calls Bard Advanced. She declined to share details on pricing. Google shared a long list of benchmarks showing that on almost every measure, the new Bard outperforms the free version of ChatGBTGPT. The company shared several demonstrations of Bard's new supercharged abilities, including a collaboration with YouTuber Mark Rober, in which the AI helps build a hyper-accurate paper airplane.

Along with Bard, Gemini is also coming to Pixel 8 Pro Android phones in a Wednesday update, albeit in a limited capacity. Gemini Nano now powers the summarized feature on Android's recorder app on Pixel 8 Pros. Google says the AI will also power Android's smart reply feature on the Pixel 8 Pro, but only if you're using the Google keyboard and only in WhatsApp. The company says Gemini is coming to more messaging apps and other parts of the operating system next year. For now, GPT4 is the most powerful model available to the public. Google says it has GPT4 beat, and Gemini Ultra will be the best AI on the market when it rolls out. With a score of over 90 percent, Gemini is the first AI model to outperform human experts on the industry standard

benchmark MMLU, said Eli Collins, vice president of product at Google Deep Mind. It's our largest and most capable AI model. MMLU, short from massive multitask language understanding, measures AI capabilities using standard tests in a combination of 57 subjects such as math, physics, history, law, and medicine. It's unclear when the public will get to see the proof, however, over the last week, the information reported that Google pushback the Gemini launch because the AI, quote, didn't reliably handle some non-English queer. series. Google's in-person Gemini demos, which were slated for this week, were postponed indefinitely. In response to questions about the alleged foreign language problems, Collins said,

quote, Gemini is actually quite performant with regards to multilingual capabilities. Google wouldn't get more specific than to say Gemini Ultra will be available early next year. Google stressed that Gemini is built for multimodal performance, meaning it can comprehend different kinds of information such as text, images, video, audio, and more. Google shared a video where a Gemini-powered Bard helps with, a student's physics homework starting with a photo of the assignment with handwritten questions. The AI then seamlessly transitions to written advice, complete with equations, and a step-by-step answer set, end quote. More on Gemini Nano coming to the Pixel 8 Pro, quoting the verge.

Gemini may be the biggest most powerful large language model or LLM Google has ever developed, but it's better suited to running in data centers than on your phone. With Gemini Nano, though, the company is trying to split the difference. It built a reduced version of its flagship LLM that can run locally and offline on your device. Well, a device anyway, the Pixel 8 Pro is the only nano-compatible phone so far, but Google sees the new model as a core part of Android going forward. If you have a Pixel 8 Pro starting today, two things on your phone will be powered by Gemini Nano, the auto-sumorization feature in the recorder app, and the smart reply of the G-board keyboard keyboard. Both are coming as part of the Pixel's December feature drop. Both work offline since the model is running on the device itself, so they should feel fast and native.

Google is starting out quite small with Gemini Nano. Even the smart reply feature is only Gemini powered in WhatsApp, though Google says it's coming to more apps next year. And Gemini as a whole is only rolling out in English right now, which means many users won't be able to use it at all. Your Pixel 8 Pro won't suddenly feel like a massively upgraded device, though it might over time if Gemini is as good as Google thinks it can be. And next year, when Google brings a Gemini-powered bard to assistant on pixel phones,

you'll get even more of the Gemini experience. Nano is the smallest, duh, of the Gemini models, but Demis Hesibis, the CEO of Google DeepMind, says it still packs a punch. It has to fit on a footprint, right? He says, the very small footprint of a pixel phone. So there's memory constraints, speed constraints, all sorts of things. It's actually an incredible model for its size, and obviously it can benefit from the bigger models by distilling from them,

and that sort of thing. The goal for Nano was to create a version of Gemini that is as capable as possible without eating your phone storage or heating the processor to the temperature of the sun. Right now, Google's TensorFlow 3 processor seems to be the only one capable of running the model. But Google is also working on a way to build Nano into Android as a whole. It launched a new system service called AI Core that developers can use to bring Gemini-powered features into their apps. Your phone will still need a pretty high-end chip to make it work, but Google's blog post announcing the feature mentions Qualcomm, Samsung, and Media Tech as companies making compatible processors. Developers can get into Google's early access program now, end quote.

Google, by the way, has also updated Bard to use its new Gemini Pro models, as I said, the middle tier of the Gemini series, and says it's the biggest and best upgrade yet that can match chat GPT in terms of performance. Speaking of developers and chips, Google has also announced the cloud TPU V5P, its most powerful AI accelerator yet. Quoting TechCrunch, a V5P pod consists of a total of 8,960 chips and is backed by Google's fastest interconnect yet with up to 4,800 GbPS per chip. Google trained Gemini on these new custom chips. It's no surprise that Google promises that these chips are significantly faster than the V4 TPUs, The team claims that the V5P features a 2x improvement in flops and 3x improvement in high bandwidth memory. That's a bit like comparing the new Gemini model to the older open-AigPT 3.5 model, though.

Google itself, after all, already moved the state of the art beyond the TPUV4. In many ways, though, the V5E pods were a bit of a downgrade from the V4 pod with only 256 V5E chips per pod versus 4,96 in the V4 pods, and a total of 197 T-flops, 16-bit floating point performance versus V5E chips versus 275 for the V4 chips. For the new V5P, Google promises up to 459 T-flops of 16-bit floating point performance, backed by the fastest interconnect. Google says all of this means the TPU V5P can train a large language model like GP-175B 2.8 times faster than the TPUV4, and do so more cost-effectively, too, though the TPUV-5E, while slower, actually offers more relative performance per dollar than the V5P, end quote. Senator Ron Wyden has said that unidentified governments have been surveilling smartphone users

via various apps push notifications, which is interesting enough, but further interesting, is the news that Apple at least knew about this, but had to keep quiet about it until now. Quoting Reuters. Apps of all kinds rely on push notifications to alert smartphone users to incoming messages, breaking news, and other updates. These are the audible dings or visual indicators users get when they receive an email or their sports team wins a game. What users often do not realize is that almost all such notifications travel over Google and Apple's servers. That gives the two companies unique insight into the traffic flowing from those apps to their users, and in turn puts them in, quote, a unique position to facilitate government surveillance of

how users are using particular apps, Wyden said. He asked the Justice Department to, quote, repeal or modify any policies that hindered public disclosures of push notification spying. In a statement, Apple said that Wyden's letter gave them the opening they needed to share more details with the public about how governments monitor push notifications. Quote, in this case, the federal government prohibited us from sharing any information, the company said in a statement, Now that this method has become public, we are updating our transparency reporting to detail these kinds of requests, end quote. The Department of Justice did not return messages seeking comment on the push notification surveillance, or whether it had prevented Apple or Google from talking about it.

Google did not return messages seeking comment. Wyden's letter cited a tip as the source of the information about the surveillance. His staff did not elaborate on the tip, but a source familiar with the matter confirmed that both foreign and U.S. government agencies have been asking Apple and Google for metadata, related to push notifications to, for example, help tie anonymous users of messaging apps to specific Apple or Google accounts. The source declined to identify the foreign governments involved in making the request, but describe them as democracies allied to the United States. The source said they did not know how long such information had been gathered in that way. Most users give push notifications a little thought, but they have occasionally attracted attention from technologists

because of the difficulty of deploying them without sending data to Google or Apple. end quote. This is kind of a big deal. Gaming live streamer Twitch is planning to shut down operations in South Korea on February 27th, saying the country was prohibitively expensive to operate in despite working to reduce costs. Quoting Venture Beat, Twitch stressed that the situation in South Korea is unique, as operating costs are significantly higher than other markets. According to stats from Sully Nome, Korean language channels brought in 98.8 million hours watched over the last last 30 days. This accounts for just shy of 5.5% of all hours watched on the platform. Twitch has offered help to Korean streamers that built their communities on the platform to find a new home

on other streaming services. To do this, Twitch plans to reach out to competitors in South Korea to support streamers, transitions to other services. According to Sali Nome, there are about 18,500 active affiliates and partners streaming in Korean. This closure will likely have a major impact on the e-sports market. South Korea is a key region. for League of Legends, among other titles. Major publishers such as Riot will have to shift operations quickly and rebuild their platforms elsewhere during the offseason, end quote. But why is South Korea so expensive? Quoting PC gamer. CEO Dan Clancy offered an explanation to the shutdown, quote, ultimately the cost to operate Twitch in Korea is prohibitively expensive, and we have spent

significant effort working to reduce these costs so that we could find a way for the Twitch business to remain in Korea, end quote. He goes on to say that despite reducing maximum source quality to 720P and experimenting with a peer-to-peer model, quote, our network fees in Korea are still 10 times more expensive than in most other countries, which has been operating in Korea at a significant loss, and unfortunately, there is no pathway forward for our business to run more sustainably in that country, end quote. The high fees come from South Korea's law that dictates content providers have to pay volume-dependent fees in order to deliver their digital goodies through Korean Internet service providers. It's proven a hot topic in the country over the last few years, especially as it targets

high bandwidth sites like Google and Netflix, with the latter previously embroiled in a lawsuit with one of South Korea's biggest providers over who should foot the bill, end quote. So these are basically anti-net neutrality fees to the equivalent of South Koreans' equivalent of Verizon and AT&T, I guess. One more bit of speculation, on my part, at least, wouldn't the fact that Twitch is streaming live as opposed to, you know, cashed downloaded video make things worse for Twitch since there are downloads and uploads at the same time. Sounds like that would be bandwidth intensive. And finally today, another brick taken out of the wall that separates blue bubbles from green

bubbles. Beeper has debuted Beeper Mini for Android, which lets users send iMessages using end-to-end encryption without having to use a new number or login with an Apple ID. They're only charging $2 a month for this privilege. Quoting the verge. Earlier this year, a developer slid into Eric Mijikovsky's DMs with a spectacular claim that he had reverse-engineered Apple's iMessage allowing any device, Android, Windows, whatever, to send messages as a blue bubble. Mijikovsky didn't believe what he was reading.

I said, BS, no one has done that. No one on Earth has done that, said Mijikovsky, CEO of the messaging startup beeper. He'd tried to do it himself, and he'd messaged everyone he could find who'd ever gotten close. No one had put all the pieces together, he said. But now there was this developer in his DMs, a 16-year-old high school student of all people linking him to a prototype. And it worked. That prototype became the basis for a new Android app called Beeper Mini that Mijikovsky's startup is launching today. Open the app and it'll look at all of your text message conversations, figure out which ones are from IMessage users, and switch them over to blue bubble conversations on Apple's

platform. From then on, whenever you message an iPhone user through Beeper Mini, you'll be using iMessage and there'll be none the wiser. I've been using the app for the past few weeks, and I've been surprised at how smoothly it works. Messages sent from Beeper Mini on my pixel 8 appear as blue bubbles on the iPhones of my friends and family members. Group chats I'm on automatically switched over to iMessage as soon as someone fired off a meme. Reactions, threads, photos, and videos without the messy text message compression all came through. The best thing I can say about Beeper Mini is that almost no one noticed I was using it. Blue Bubbles just started appearing no lost messages to speak of. Beeper Mini joins a growing list of apps trying to hack the iMessage experience onto Android,

but Mijikovsky is adamant that Beeper Mini is not like the other services out there. It is directly sending iMessages. Other services, including Beeper's previous iMessage implementation, would relay messages through a Mac hosted in the cloud. That poses real security problems, as recently exemplified by Sunbird, and it's nothing branded spin-off, nothing chats. Nothing's app was launched and pulled in just four days after serious security issues were discovered. Sunbird pulled its app shortly thereafter. Beeper Mini avoids some of those problems because it's operating in a fundamentally different way. Its developers figured out how to register a phone number with iMessage, send messages directly to Apple's servers,

and have messages sent back to your phone natively inside the app. It was a tricky process that involved deconstructing Apple's messaging pipeline from start to finish. Beeper's team had to figure out where to send the messages, what the messages needed to look like, and how to pull them back down from the cloud. The hardest part, Midgeikovsky said, was cracking what is essentially Apple's padlock on the whole system, a check to see whether the connected device is a genuine Apple product. The bigger question may be how long Beeper Mini can survive. Mityikovsky believes he's on the right side of the law.

He points to a copyright carve-out for reverse engineering and says there's no Apple code in Beeper Mini, and he believes it'd be too difficult for Apple to cut Beeper off without also breaking eye message for legions of genuine Apple devices. I'm not so sure on either count. Apple is deeply protective of iMessage as a lock-in mechanism for the iPhone, and it's hard to imagine the company ignoring the spate of third-party iMessage solutions forever, end quote. Guess I'd better break out that Pixel 8 Pro I got last month and see how this on-device AI stuff works. I'll let you know.

Talk to you tomorrow.