Tech Brew Ride Home - Wed. 08/14 – When It Comes To AI On Phones, Google Does It Itself

Episode Date: August 14, 2024

All the details from yesterday’s pixel event, but especially the AI features that show how far ahead Google is. At least when it comes to putting AI on phones. We have official post quantum computing cryptography standards. And why they’re using iPhones to make offsides calls in soccer this season. Sponsors: DataTribe.com/challenge Hims.com/ride Links: The Google Pixel 9’s AI Camera Features Let You Reshape Reality (Wired) Google Gemini’s voice chat mode is here (The Verge) Apple relents and approves Spotify app with EU pricing (The Verge) The first post-quantum cryptography standards are here (TechCrunch) The English Premier League Will Ditch Its Hated VAR Offside Tech for a Fleet of iPhones (Wired) Fantasy League Links: https://fantasy.premierleague.com/leagues/auto-join/s5r9c8  Code: s5r9c8 https://fplchallenge.premierleague.com/leagues/auto-join/8znkcc  Code: 8znkcc Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Wednesday, August 14th, 2024. I'm Brian McCullough today. All the details from yesterday's pixel event, but especially the AI features that show maybe how far ahead Google is,

at least when it comes to putting AI on phones. We have official post-quantum computing cryptography standards and why they're using iPhones to make off-sides calls in soccer this season. Here's what you missed today in the world of tech. So I am glad I waited to do the pixel stuff. Let's do a quick rundown of the hardware announces from yesterday so I can get to the more interesting bits of news. You had the Google Pixel 9 with a tensor G4 chip, a design with flat sides, a bigger 6.3-inch screen, and AI features. More on those in a second, all on sale starting August 22nd, starting at 799 bucks. A 6.3-inch Pixel 9 Pro and a 6.8-inch 9 Pro XL with 120 Hertz superactua display. Tenser G4, 16 gigabytes of RAM, 45-watt fast charging in four colors, and priced at 999 and 1099,

respectively, and up, of course, that's just the starting prices. There was the Pixel Watch 3 in 41-millimeter and 45-millimeter sizes with a 2,000-knit display, 16% smaller bezels starting at 349. There were new Pixel Buds Pro 2 for 229 with a TensorFlow AI. chip, smaller and lighter than the original pros, with improved noise cancellation and sound quality. There was also the $1,190 plus Pixel 9 Pro Fold, Google's second take on that foldable category, with a larger 8-inch and 6.3-inch inner and outer displays, 4,60 mill-amp-hour battery, shipping September 4th. Now, here's the interesting bits.

Of course, Google also announced a bunch of AI stuff, and quoting no less than Mark German on Twitter. After watching Google's latest AI announcements, it's hard to believe Apple is anything other than two to three years behind in this area, at least, end quote. Let's run down some of the interesting AI stuff from Wired, quote. First up, AdMe. You've probably been in a situation where you want to take a selfie with your partner or family in front of a subject like the Eiffel Tower, but someone has to take the picture, right? Instead of handing your $1,000 phone to a stranger, ADME accomplishes the same task. This is a special mode in the Pixel 9 phones that first asks you to scan the surrounding area briefly, then you'll snap a picture of your loved one in front of the subject,

and then swap places. When they take over photo capture duties, they'll see a faded out image of themselves in the camera preview, and the camera app will suggest a place for the second person to stand. Once they press the shutter button, it'll superimpose the images, so it appears as if both people were standing right next to each other even when they weren't. It worked well in my brief testing, and naturally I tried to see if I could duplicate myself. This worked once, but every other attempt failed. That's because Google says it was not designed for the same person to show up twice. Maybe if you change your shirt or try to look different enough, it might do the trick. I'll need to do more testing to see how well it works when you want to put your hand around another person's shoulder.

Next, re-imagine in Magic Editor. Re-imagine is the latest addition to Google's Magic editor, which currently lets you move objects around a photo or erase objects. This new tool lets you select an area of a photo and then a text prompt pops up, where you can type in what you want to see as your end result. This can be anything ranging from turning the photo from daytime to nighttime, adding stormy clouds, or, like I tried, adding a UFO over the Empire State Building. The more descriptive you are, the better the results. However, Google says it works best with backgrounds and objects instead of people. There are guardrails in place so that you don't alter how someone looks. It's similar to Samsung's sketched image feature in its latest folding

phones, except Samsung asks you to sketch what you want to see rather than using text. Reimagine isn't perfect. Sometimes it didn't produce results with what I typed in, and sometimes the results were just plain bad. But you do get four results to choose from, and you can always try again and be more descriptive. Finally, there's auto frame. Composition is important in photography, and if adding gridlines to your camera app doesn't help you line things up. Yes, most smartphone cameras offer this feature in the camera settings. Google thinks this is another task generative AI can help with. Autoframe lives in Magic Editor much like Reimagined. Once you're editing a photo, you'll see the option to select Auto Frame. Tap this, and it will generate four images with different framing.

For example, I intentionally took a photo where I was standing very close to the edge of the frame. Not great compositionally. I used Auto Frame, and it generated pixels above and to the right of me, pushing me closer to the center following the classic rule of thirds. It even gave me a vertical crop of an original horizontal photo. These generated pixels essentially understand the context of the photos and expand the edges of the frame so that it looks natural, even if it's all artificial. In the images I tested it with, it did not know how much of the tree was really to my left or how far the fence went, so it made assumptions. If you look closely, you can probably find some mistakes, but most people will never notice the difference. One more. Finally, there is zoom enhancement.

enhance. Google announced zoom enhance with the Pixel 8, but it never shipped because it wasn't ready. Now, it's finally launching in the Pixel 9 series and will arrive to Pixel 8 phones at a later date. Currently, if you zoom into a photo pre-capture, Google uses its super res zoom algorithm to ensure the image is sharper than what you'd get with a typically digitally zoomed in photo. Zoom enhanced, however, is post-capture as a feature. In the Google Photos app, select a photo that you want to zoom in on, tap the edit button, and then go to tools to find zoom enhance. You'll have to zoom to the area you want and then tap zoom enhance, and just like in the early 2000s, CSI shows it'll enhance the photo by generating pixels to make it appear sharper. I tried it on some faraway buildings and the results delivered sharper lines that looked much cleaner than the previously pixelated image, end quote. Google also announced

Gemini will replace Google Assistant full stop as the default on the Pixel 9 lineup. However, users can go back to the classic Google Assistant if they want. This is all in aid of the debut of Gemini Live, quoting the verge. Available for Gemini Advanced Subscribers, it works a lot like chat GPT's voice chat feature with multiple voices to choose from and the ability to speak conversationally, even to the point of interrupting it without tapping a button. Google says that conversations with Gemini Live can be free-flowing, so you can do things like interrupt and answer mid-sentence or pause the conversation and come back to it later. Gemini Live will also work in the background, or when your phone is locked. Google first announced that Gemini Live was coming during its I.O. developer

conference earlier this year, where it also said Gemini Live would be able to interpret video in real time. Google also has 10 new Gemini voices for users to pick from with names like Ursa and Dipper. The feature has started rolling out today in English only for Android devices. The company says it will come to iOS and get more languages, quote, in the coming weeks, end quote. Yeah, why partner with Open AI? when you can just insert your own AI as your Siri killer yourself. Let me do a quick sort of omnibus segment here to shoehorn in two different sort of follow-up stories. First, Apple has approved Spotify updating its app to show in-app pricing information for iPhone users in the EU, starting today,

after Spotify's, you know, years-long legal battle about all this, quoting the verge. One thing that's missing is the ability to click a link to, make those purchases from outside the Apple App Store. Spotify says it's opting into the music streaming services entitlement that Apple introduced after being served with a $2 billion EU antitrust fine in March for abusing its dominant position in music streaming, rather than accepting the complicated new developer terms Apple outlined last week. Unlike the entitlement, the latter would allow EU developers to link to external payment options with Apple taking a cut of off-platform sales. Spotify clearly doesn't want to

to do that, saying that Apple is demanding, quote, illegal and predatory taxes, end quote. And then, I believe the phrase I used was burning everything that was flammable just to stay alive. According to public filings, Intel sold 1.18 million shares that it owned in Arm during Q2, which would have raised around $147 million for the company based on the stock's average price in the quarter, end quote. U.S. NIST has published its first three post-quantum cryptography standards. IBM's director of research thinks quantum will hit an inflection point sometime around 2030. Quoting TechCrunch, it'll still be a while before quantum computers become powerful enough to do anything useful, but it's increasingly likely that we will see full-scale error-corrected quantum computers become operational within the next five to ten years. That'll be great for a scientist trying to solve hard computational problems in chemistry. and material science, but also for those trying to break the most common encryption schemes used today. That's because the mathematics of the RSA algorithm that, for example, keep the internet connection to your bank safe are almost impossible to break with even the most powerful

traditional computer. It would take decades to find the right key. But these same encryption algorithms are almost trivially easy for a quantum computer to break. This has given rise to post-quantum cryptography algorithms. And on Tuesday, the U.S. National Institute of Standards and Technology, NIST, published the first set of standards for post-quantum cryptography. MLKEM, originally known as Crystals Khyber, MLDSA, previously known as Crystal's DeLithium, and SLODSA initially submitted as Sphinx Plus. And for many companies, this also means that now is the time to start implementing these algorithms.

The ML-K-E-M algorithm is somewhat similar to the kind of public-private encryption methods used today to establish a secure channel between two servers, for example. At its core, it uses a lattice system and purposely generated errors that researchers say will be very hard to solve, even for a quantum computer. MLDSA, on the other hand, uses a somewhat similar scheme to generate its keys, but is all about creating and verifying digital signatures. SLHDSA is also all about creating digital signatures, but is based on a different mathematical foundation to do so. Two of these algorithms, MLKEM and MLDSA, originated at IBM, which has long been a leader in building quantum computers. To learn a bit more about why we need these standards now, I spoke to Dario Gill, the director of research at IBM.

He thinks that we will hit a major inflection point around the end of the decade, which is when IBM expects to build a fully error-corrected system, that is one that can run for extended periods without the system breaking down and becoming unusable. then the question is, from that point on how many years until you have systems capable of breaking RSA. That's open for debate, but suffice to say, we're now in the window where you're starting to say, all right, so somewhere between the end of the decade and 2035 at the latest, in that window, that it is going to be possible. You're not violating laws of physics and so on, he explained. Gill argues that now is the time for businesses to start considering the implications of what cryptography will look like once RSA is broken. A patient adversary could, after all, start

gathering encrypted data now and then in 10 years use a powerful quantum computer to break that encryption. But he also noted that few businesses and maybe even government institutions are aware of this, end quote. Finally today, the English Premier League is back starting this weekend, and thank God, would you believe me if I told you there's a tech angle to that? Wired takes a look at how this season, the Premier League is taking the off-sides decision out of the hands of VAR. That hated video refereeing technology, and it's doing so thanks to a system that literally just uses iPhone cameras. Quote, Dragon, according to Genius, will initially use at least 28 iPhone cameras at every stadium in the Premier League. More cameras may be used in certain

stadiums throughout the year, the company says. The system uses the built-in cameras of iPhone 14 models and newer. The iPhones are housed in a custom waterproof case, adorned with cooling fans that are connected to a power source. The team designed mounts that hold up to four iPhones clumped together. Once the iPhones are positioned around the pitch, together, they capture a constant stream of video for multiple angles. Camera mounts can be moved to change coverage zones in certain facilities, per genius, but will typically be stationary during actual play to ensure proper coverage and avoid recalibration needs on the fly. This wealth of visuals apparently gives Dragon the ability, to track between 7,000 and 10,000 points on each player at all times.

Dragon leverages the ability of iPhones to capture video and ultra-high frame rates, mitigating tricky instances of occlusion that can obscure the precise kickpoint of the ball. De Aurea offers a simple example. Watch some broadcast video of soccer balls being kicked, but slow the clips down enough so you watch the action progress frame by frame. You will in many instances miss the kickpoint, D'Arya says. The kickpoint will be in between two frames. of video. You go from one frame where the ball is not on the foot yet to the next frame, and the ball

has already left the foot and gone in the other direction, end quote. Most broadcast video today is captured at 50 or 60 frames per second. Dragon can capture up to 200 frames per second, potentially reducing those gaps between frames by 75%. The initial EPL system will be capped at 100 frames per second to balance latency, accuracy, and costs. The system can auto-detect important impending events, such as a possible off-side call, and scale up the frame rate of certain cameras temporarily, then scale back down when appropriate to save computing power. Facilitating this automation is Dragon's other key feature, a machine intelligence system running on the back end, known internally as object semantic mesh.

Utilizing genius's years of converting optical basketball data, this machine learning program has been trained on common soccer events or situations over several seasons. It's not just capturing movements, it's contextualizing them in real time, and in some cases even learning from them. In the AI community, that's not a very novel approach to have this kind of semantic understanding, says D'Aria. It's not just an image or a representation, but it's actually something you can reason about and you can interrogate. Don't worry about a full takeover from our robot overlords, though. While both EPL and Genius declined to provide specifics, some of which, including timing, are still being determined ahead of Dragons in

season launch, sources familiar with the setup confirmed that humans will make the final decision on all offside calls with the assistance of these AI tools, end quote. Speaking of soccer, as mentioned before, it's that time of year, the time where I get into fantasy soccer for a few weeks until I inevitably forget to update my team one week, at which point I kind of give up. But as I do every year, I have put together a Mutant Podcast Army League you can join, if that's your thing, also some sort of new thing called the FPL Challenge,

Codes to both are at the bottom of the show notes. Usually we get about 30 or 40 folks in the league, but I forget to announce who wins every year. If you won last year, make yourself known to me. Best of luck to everybody that joins the league. Talk to you tomorrow.