Tech Brew Ride Home - Fri. 06/14 – Now Microsoft Delays Recall

Episode Date: June 14, 2024

The recall of some Microsoft AI products continues as literally Recall gets delayed. Another deep dive into how Apple’s AI actually works. Dream Machine is an open source AI video generator you can use this weekend. And in the Weekend Longreads Suggestions, the people who have found everyday use cases for the Apple Vision Pro. Links: Microsoft’s all-knowing Recall AI feature is being delayed (The Verge) Here’s how Apple’s AI model tries to keep your data private (The Verge) Luma AI debuts ‘Dream Machine’ for realistic video generation, heating up AI media race (VentureBeat) As streaming becomes more expensive, Tubi cashes in on the value of free (Los Angeles Times) Weekend Longreads Suggestions: I Know What the Apple Vision Pro Is For (Intelligencer) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Friday, June 14th, 2024. I'm Brian McCalla today. The recall of some Microsoft AI products continues, as literally recall gets delayed. Another deep dive into how Apple's AI actually works. Dream Machine is an open source AI video generator you can use this weekend. And in the weekend, long-read suggestions the people who have found everyday use cases for the Apple Vision Pro. Here's what you miss today in the world of tech. Well, it continues. Microsoft is officially, delaying that new recall feature to test it within the Windows Insider program. Also, this means that Recall won't ship with those co-pilot plus PCs that are supposed to start shipping next week.

This is after Microsoft says it would make the feature opt-in. Quoting Tom Warren and the Verge. We are adjusting the release model for recall to leverage the expertise of the Windows Insider community to ensure the experience meets our high standards for quality and security, says Microsoft in an updated blog post. When Recall Preview becomes available in the Windows Insider program, we will publish a blog post with details on how to get the preview, end quote. This means that recall won't even be available initially to Windows Insiders or anyone who buys a Copilot Plus PC. I wrote in Notepad earlier today that Windows engineers were scrambling to get the security improvements tested and implemented in time for the June 18th launch date of Copilot Plus PCs. Now Microsoft

is essentially admitting here that it needs more time to test recall's security improvements. Microsoft first unveiled the recall feature as part of its upcoming Copilot Plus PCs last month, but since then, privacy advocates and security experts have been warning that recall could be a, quote, disaster for cybersecurity without changes. Microsoft committed to three major updates to recall last week, including making the AI-powered feature an opt-in experience instead of on by default, encrypting the database, and authenticating through Windows Hello. Recall uses local AI models built into Windows 11 to screenshot mostly everything you see or do on your computer and then give you the ability to search and retrieve items you've seen. An explorable timeline lets you scroll through those snapshots and look back on what you did on a particular day on your PC.

Everything in Recall is designed to remain local and private on device, so no data is used to train Microsoft's AI models. I reported earlier today in Notepad that Recall was originally created before Microsoft's Big SFI over. overhaul began. Recall was developed in secret at Microsoft, and it wasn't even tested publicly with Windows Insiders. Microsoft subsequently identified some of the security issues with recall and started to develop and test changes to the experience in recent months. It clearly now needs more time to make sure those changes stand up to its promise of putting security above AI and everything else, end quote. Meanwhile, Apple really wants you to understand. It does not train its models on users' private data or user interactions and instead relies on licensed materials

and publicly available online data for its forthcoming AI. Greater detail on how this all works, again quoting the verge. Apple uses both licensed materials and publicly available online data that are scraped by the company's Applebot web crawler. Publishers must opt out if they don't want their data ingested by Apple, which sounds similar to policies from Google and OpenAI. Apple also says it omits feeding social security and credit card numbers that are floating online and ignores, quote, profanity and other low-quality content. A big selling point for Apple Intelligence is its deep integration into Apple's operating systems and apps, as well as how the company optimizes its models for power efficiency and size to fit on iPhones. Keeping AI requests local

is key to quelling many privacy concerns, but the trade-off is using smaller and less capable models on device. To make those local models useful, Apple employees fine-tuning, which trains models to make them better at specific tasks like proofreading or summarizing text. The skills are put into the form of adapters, which can be laid onto the foundation model and swapped out for the task at hand, similar to applying power-up attributes for your character in a role-playing game. Similarly, Apple's diffusion model for Image Playground and Gen Moji also uses adapters to get different art styles like illustration or animation, which makes people and pets look like cheap Pixar characters. it has optimized its models to speed up the time between sending a prompt and delivering a response,

and it uses techniques such as speculative decoding, context pruning, and group query attention to take advantage of Apple Silicon Neural Engine. Chipmakers have only recently started adding neural cores or NPUs to the dye, which helps relieve CPU and GPU bandwidth when processing, machine learning, and AI algorithms. It's part of the reason that only Macs and iPads with M-Series chips, and only the iPhone 15 Pro and Pro Max support Apple Intelligence. The approach is similar to what we're seeing in the Windows world. Intel launched its 14th generation Meteor Lake architecture featuring a chip with an NPU, and Qualcomm's new Snapdragon X chips built for Microsoft's Copilot Plus PCs have them too. As a result, many AI features on Windows are gated to new devices

that can perform work locally on these chips. According to Apple's research, out of 750 tested responses for text summarization. Apple's on-device AI with appropriate adapter has more appealing results to humans than Microsoft's Phi-3 mini-model. It sounds like a great achievement, but most chatbot services today use much larger models in the cloud to achieve better results, and that's where Apple is trying to walk a careful line on privacy. For Apple to compete with larger models, it is concocting a seamless process that sends complex requests to cloud servers while also trying to prove to users that their data remains private. If a user request, needs a more capable AI model, Apple sends the request to its private cloud compute servers,

or PCC servers. PCC runs on its own OS based on iOS foundations, and it has its own machine learning stack that powers Apple intelligence. According to Apple, PCC has its own secure boot and secure enclave to hold encryption keys that only work with the requesting device, and trusted execution monitor makes sure only signed and verified code runs. Apple says the user's device creates an end-to-end encrypted connection to a PCC cluster before sending the request. Apple says it cannot access data in the PCC since it's stripped of server management tools, so there's no remote shell. Apple also doesn't give the PCC any persistent storage,

so requests and possible personal context data pulled from Apple's intelligence semantic index, apparently get deleted on the cloud afterward. Each build of PCC will have a virtual build that the public or researchers can inspect, and only signed builds that are logged as inspected will go into production. One of the big open questions is exactly what types of requests will go to the cloud. When processing a request, Apple Intelligence has a step called orchestration, where it decides whether to proceed on device or use PCC. We don't know what exactly constitutes a complex enough request to trigger a cloud process yet,

and we probably won't know until Apple Intelligence becomes available in the fall. There's one other way Apple is dealing with privacy concerns. making it someone else's problem. Apple's revamped Siri can send some queries to ChatGBTGBT in the cloud, but only with permission after you ask some really tough questions. That process shifts the privacy question into the hands of OpenAI, which has its own policies, and the user, who has to agree to offload their query. In an interview with Marquez-Brownly, Apple's CEO, Tim Cook said that Chat-GPT would be called on for requests involving world knowledge that are, quote, out of domain of personal context, end quote.

has debuted Dream Machine, an AI model to generate high-quality videos from text and images, and plans to release APIs and plugins for popular creative software, because this is open source. Quoting Venture Beat. The technology opens a door for a wide range of creators and companies to produce original video content at unprecedented speed and scale. The best part, you can start using Luma Dream Machine today. Dream Machine allows users to enter a descriptive prompt like a corgi puppy chasing a bull, ball on a beach at sunset, and in about two minutes generates a realistic five-second video clip matching that scene. Early beta testers have praised the system's ability to faithfully render

specified objects, characters, actions, and environments while maintaining fluid motion and coherent storytelling. The launch of Dream Machine represents a major milestone in the democratization of AI-powered video generation, while rival systems like OpenAI's SORA and Kiyashu's Kling have showcased impressive capabilities. They remain accessible. only to a select group of partners. In contrast, Luma AI has made Dream Machine available for anyone to experiment with for free on its website with plans to release APIs and plugins for popular creative software. This open approach could give Luma AI a head start in building a vibrant community of creators and developers around its platform by lowering the barriers to entry. Dream Machine has

the potential to spark a wave of innovation and creativity as users explore the possibilities of AI-generated video. However, the field remains in its early stages and even the most advanced systems like Dream Machine, Sora, and Kling can struggle with certain prompts or produce outputs that are nonsensical or problematic. Improving the quality, consistency, and reliability of these models will be crucial for unlocking their full commercial potential, end quote. We've spoken a lot recently about the rising prices in the streaming space, which means we should consider the case of the completely free ad-supported streamer to be, because, apparently amid all these price rises, according to Nielsen, May was Tooby's most watch month ever, with an average of 1 million viewers, up 46% year over year,

quoting the LA Times. The streamer edged out Disney Plus, which averages 969,000 viewers. Tubi also easily beat NBC Universal's Peacock, Warner Bros. Discovery's Max, and Paramount Plus, while also topping free competitors such as the Roku Channel and Pluto TV. YouTube is the only free ad-supported streaming platform, with more viewers than Tooby. Third quarter revenue for Tooby grew 22% year over year at a time when the advertising market was sluggish, according to Fox.

Tooby continues to pull ahead from its ad-supported video-on-demand competition and post faster-than-expected growth, analysts at research firm Moffat Nathanson said in a report for clients. The escalating subscription costs of the competition have certainly helped. With Netflix, Amazon, and others now selling advertising in addition to charging fees, Tooby is looking like a better deal to many budget-conscious consumers. Of course, those are things that are going to positively impact us. Adam Lewinson, chief content officer for Tobe said in a recent interview.

The San Francisco-based company was founded in 2017 and acquired by Fox in 2020 for $440 million. To be currently offers 250,000 TV episodes and movies. Tubi says 63% of its users describe themselves as cordnevers, people who have never subscribed to a pay TV package or cord cutters. About half are what Tooby categorizes as multicultural covering black, Latino, Asian, and LGBTQ plus audiences. Although Tooby has a wide array of live streaming channels that delivers shows live sports and news in real time, 90% of its viewing is on demand. Younger viewers preferred way to watch. The streamer says the median age of its audience is 39, the youngest in television. Tubi has an array of vintage hit network TV series that are discovered and then

devoured by younger viewers not already familiar with them. But it also provides a home for shows that were ignored when they initially ran on traditional TV, even highlighting them in a section called Canceled Too Soon. One such obscurity, the TV show Believe, was gone after 13 episodes on NBC in the 2013-2014 TV season. But the series from Gravity Director Alfonso Curran about a girl with supernatural abilities is now a to-be-hit. If things stay on its current course, we believe it's going to get far more viewers on Tooby than it ever got in its first window on broadcast. Lewinson said of Believe, the next step for Tooby is to expand its production of original programs and movies, which remain an important draw for consumers when they choose a streaming service.

Starting with low-budget 2021 thriller Twisted House Sitter, the company has produced 200 titles, mostly genre movies, documentaries, and animated series. But with success comes the ability to attract higher profile talent. Filming has begun on original To Be film The Thicket, a dark western starring Peter Dinklage and Juliette Lewis, end quote. Finally today, this week of WWDC, I did sort of sub-tweet the fact that nobody seems to be talking about the Apple Vision Pro anymore. Like I said, they have 10,000 apps now. Who knew? No one seems to be blowing trumpets about this. But over at the Intelligencer, Andrew Leland has identified one community who apparently has fully embraced the Vision Pro for daily use. That would be disabled users.

Quote, about 10 years ago, Steve Coulson, a creative director in New York, began losing his hearing, and today he has profound hearing loss. This made his work, constant in-person meetings, often in noisy environments, increasingly difficult. Like many people with disabilities, Coulson found the pandemic isolating, but he embraced certain elements of remote access. He had more control over the audio and virtual meetings, not to mention real-time captions and auto-generated transcripts. Still, he missed the easy, dynamic exchanges of his pre-pandemic brainstorms, and his hearing loss made it difficult to reproduce that feeling in person, even with hearing aids. Now, meeting with his business partner in Spatial FaceTime on Vision Pro, he says the feeling he lost has been restored. It feels like I'm in a room again, he said. We can just sit together in a meeting and I can hear. This technology, Coulson said, is life-changing in a way that a hearing person might not understand, end quote.

Michael Doys, who works as an accessibility specialist and app developer in Austin, has optic nerve hypoplasia. His optic nerves didn't fully develop when he was born. When he's with his family, he rarely sees their facial expressions, since it would be awkward to hold a portable magnifier up to their faces while they hang out. Even on his computer, he has trouble magnifying their images efficiently. But on a group video call wearing his AVP, quote, I could actually see their facial expressions, he said. it's a remarkable feat of engineering for someone who's blind. Are they happy, smiling? Knowing what all that looks like is huge for me, end quote. Neurodiverse users have also found value in the AVP. I generally feel a lot better after having

worn it for a while, a user with autism and ADHD told me, it's like a reset for the brain. When I chatted with them, they'd just drain their AVPs battery by spacing out in the immersive lunar environment. My brain just is hyper-focused on whatever stimulus comes in, so whatever I can do to manually cut those stimuli off helps me tremendously, they said. The Vision Pro is noise-canceling headphones for my eyes. Ryan Hudson Perlotta, who was born with no hands and short legs he's unable to walk on, remembers his first computer in middle school. He would go from class to class with a bulky 90-0 Windows laptop on his wheelchair,

typing notes and using rudimentary dictation software to complete his assignments. But he had to contort himself just to log in. I was literally putting my lip on the control button using my nose or arm to tap the other button, he said. Then someone showed him an Apple computer, which had a function called sticky keys, allowing him to temporarily lock multiple keys on the keyboard, freeing him from gymnastic approaches to corded commands. Today, Hudson-Prolata drives his adaptive SUV to his job in downtown Detroit, putting in long hours as a principal designer at Rocket Mortgage, where he designs the company's apps and websites using a traditional mouse and keyboard, and his Max accessibility features.

One morning this spring, though, his back flared with pain. I was having trouble getting around that day, he said. So he took the day off and did work for his consulting agency Equal Accessibility from Bed, wearing the Vision Pro, surrounded by screens he controlled with his eyes and a series of custom mouth sounds that triggered selections. As I get older and this happens more often for me, he said, I envision myself working virtually with the AVP even more, end quote. Hey, I have a request.

I recently got turned on to the ID. of using generative AI to create VR environments. Can anyone point me to a resource or community where people are creating virtual worlds, even if they're just, you know, just one image, as long as it's, you know, 360 or 180, with AI that I can download and go into on my quest. Like, I almost specifically want to go to one of those worlds that you would see on covers of sci-fi novels from the 70s, but again, as photorealistic and 360 degrees as possible. If anybody knows where I can find that sort of stuff, hit me up online. No bonus episode this weekend. Talk to you on Monday.