Tech Brew Ride Home - Mon. 02/27 – Here Come The Prompt Engineers

Episode Date: February 27, 2023

Snap wants to go the route of the movie Her, with a personified chatbot inside of Snapchat. Interesting layoffs at Twitter. Yes. More. Inside Apple's Exploratory Design Group. The best AR glasses we��ve seen yet. And, right on schedule, here come the prompt engineers. Sponsors: This episode is sponsored by BetterHelp. Give online therapy a try at betterhelp.com/techmeme and get on your way to being your best self. Links: Snapchat is releasing its own AI chatbot powered by ChatGPT (The Verge) Twitter Blue head Esther Crawford is out at Twitter (The Verge) Google announces new features for Android and Wear OS (TechCrunch) Apple’s Secret ‘XDG’ Team Is Working on More Than Just a Glucose Monitor (Bloomberg) Exclusive: These are Xiaomi's new Wireless AR Smart Glasses, and they look like they're from the future (XDA Developers) Tech’s hottest new job: AI whisperer. No coding required. (Washington Post) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Monday, February 27th, 2023. I'm Brian McCullough today. Snap wants to go the route of the movie Her with a personified chatbot inside of Snapchat. Interesting layoffs at Twitter. Yes, more layoffs at Twitter. Inside Apple's exploratory design group, the best AR glasses we've seen yet. And right on schedule, here come the prompt engineers. Here's what you miss today in the world of tech. Friend of the show, Alex Cantorwitz, has been writing about how it's going to be all about the APIs for the next few months.

Can businesses build off the APIs of these large language models? Can existing platforms make use of them? This, then, is that. Snap plans to release a Snapchat chat bot called MyAI, powered by OpenAI's chat GPT, pinned to the app's chat tab, only available initially to plus subscribers. Quoting the Verge. The goal is to eventually make the bot available to all of Snapchat's 750 million monthly users. CEO Evan Spiegel tells the verge. The big idea is that in addition to talking to our friends and family every day, we're going to talk to AI every day, he says. And this is something

we're well positioned to do as a messaging service, end quote. At launch, my AI is essentially just a fast mobile-friendly version of chat GPT inside Snapchat. The main difference is that Snap's version is more restricted in what it can answer. Snap's employees have trained it to adhere to the company's trust and safety guidelines and not give responses that include swearing, violence, sexually explicit content, or opinions about dicey topics like politics. It has also been stripped of functionality that has already gotten chat GPT banned in some schools. I tried getting it to write academic essays about various topics, for example, and it politely declined. Snap plans to keep tuning my AI as more people use it and report inappropriate answers.

I wasn't able to conjure any in my own testing, though I'm sure others will. After trying my AI, it's clear that Snap doesn't feel the need to even explain the phenomenon that is ChatGPT, which is a testament to OpenAI building the fastest growing consumer software product in history. Unlike OpenAI's own ChatGPT interface, I wasn't shown any tips or guardrails for interacting with Snap's My AI. It opens to a blank chat page waiting for a conversation to start. While chat GPT has quickly become a productivity tool, Snap's implementation treats generative AI more like a persona. My AI's profile looks like any other Snapchat user's profile, albeit with its own alien bitmoji. The design suggests that My AI is meant to be another friend inside of Snapchat for you to hang out with, not a search engine.

My AI will likely be a boost to the company's paid subscriber numbers in the short term, and eventually it could open up new ways for the company to make money, though Spiegel is K. about his plans. Snap is one of the first clients of OpenAI's new enterprise tier called Foundry, which lets companies run its latest GPT 3.5 model with dedicated compute designed for large workloads. Spiegel says Snap will likely incorporate LLMs from other vendors besides OpenAI over time and that it will use the data gathered from the chatbot to inform its broader AI efforts. While My AI is basic to start, it's the beginning of what Spiegel sees as a major investment area for Snap. app, and more importantly, a future in which we're all talking to AI like it's a person, end quote.

Another way to look at this is what Benedict Evans has been saying. What if generative AI becomes what's next for social media, or at least what could be next beyond social media? Reports over the weekend that Twitter laid off well above 50 staff members, which is not really a big number in this era of big tech layoffs, but given how stripped down Twitter staff already was, It is notable, and also notable that those leaving include Director of Product Management Esther Crawford, who had made a big play to become Elon's chief lieutenant at the new Twitter and had been the subject of fawning press coverage, also departing review founder Martin de Kuiper, quoting the Verge. Crawford headed up various projects at Twitter, including the company's blue with verification

subscription as well as Twitter's forthcoming payments platform. Alex Heath of the Verge confirmed Crawford and most of the remaining product team were laid off this weekend, leading to speculation that Twitter's owner Elon Musk may be about to install a new regime at the company. In a recent interview, Musk said, I need to stabilize the organization and just make sure it's in a financially healthy place in that the product roadmap is clearly laid out, before guessing that before the end of the year would be a good time to find a replacement for himself as Twitter CEO. During her time at Twitter, Crawford emerged as one of Twitter's most prominent product managers under Elon Musk's leadership,

and notably tweeted a picture of herself on the floor of Twitter's office in a sleeping bag and eye mask. When your team is pushing round the clock to make deadlines, sometimes you hashtag sleep where you work. The tweet reads, Crawford, quote, began angling for a bigger role, end quote, shortly after Musk's takeover, as documented in this inside look at Twitter from Schiffer, Casey Newton, and Alex Heath. She also commented on Musk's massive layoffs that halved the company's workforce last year, and at the time wrote on Slack that, quote, drastic cuts were going to be required to survive no matter who own the company, end quote.

Google announced a bunch of new features for Android, ChromeOS, and WearOS, including fast pair support for Chromebooks, keep shortcuts, and WearOS accessibility modes, quoting TechCrunch. The announcements were time to coincide with Mobile World Conference, which is kicking off in Barcelona today. The company revealed that FastPair will soon be able to connect new Bluetooth headphones to a Chromebook with just a tap. If a user has already set up headphones with their Android phone, their Chromebook will automatically connect to them as well. Google didn't provide specific information about a release window for this update, but noted that it will be

rolling out soon. Another audio enhancement announced today will see Google Meet soon offering noise cancellation on more Android mobile devices to filter out background sounds while talking during a meeting. The company also revealed that users will soon be able to easily increase the size of content on Chrome, including text, images, and videos by up to 300% while still preserving the page layout. Users will be able to set their preferred content size as a default, so they don't have to change it every time they launch the browser. The new update is available starting today in Chrome Beta, and we'll be rolling out soon to all users. The company is also planning to release a Google Keep, single-note widget that will help users quickly manage their

notes and check off to-do lists from their home screen. The widget will display reminders, background colors, and images added to notes from the Keep app while also syncing with your smartwatch. Google Keep will also be rolling out two new shortcuts to create notes and to-do lists with a simple tap on a user's watch face, end quote. Remember last week we heard about that project to bring glucose monitoring to the Apple Watch? This project, apparently lives in a division not reported on before, Apple's exploratory design group. In his weekly newsletter this week, Mark German has some more deets on this division, quoting Bloomberg. The team originated several years ago and was long led by Bill Athos, one of the few people

to have the title of Engineering Fellow at Apple until he passed away unexpectedly at the end of last year. Athos was seen by the late co-founder Steve Jobs and current chief executive officer Tim Cook, as one of the brightest engineering minds at the company. The XDG team sits within Apple's Hardware Technologies Group, led by Senior Vice President Johnny Sruji, and works at a building known as Tantal 9 right outside of the Apple Park Spaceship-shaped ring. The Exploratory Design Group operates as a startup within Apple and is made up of only a few hundred people,

mostly engineers and academic types. That's a far cry from the many hundreds of people in the Special Projects Group, which is focused on Apple's self-driving car, or the more than a thousand engineers in Apple's technology development group, the team building the mixed reality headset. Beyond the glucose work, XDG is working on next generation display technology, artificial intelligence, and features for AR VR headsets that help people with eye diseases. The team originally came together under Athos to work on low power processor technologies and next generation batteries for smartphones, efforts that continue.

Like Alphabet's Moonshot team, and those at other Silicon Valley companies, the XDG staff is given vast financial resources and headroom to explore countless ideas. The members have a different remit than the engineering teams churning out new iPhones, iPads, and Apple Watches annually. Instead, they're instructed to work on projects until they can determine whether or not an idea is feasible. The unit is even more secretive than Alphabet's X, but it's not a pie-in-the-sky operation. It has already had breakthroughs that made their way into Apple products. Many of the chips and battery technologies developed by XDG have been shipping for years in iPhones, iPads, and Macs. While the team operates as a startup, it's still

compartmentalized like any other Apple division. People working on one project within XDG aren't allowed to communicate about their work with other members of XDG that are assigned to different projects, end quote. We mentioned Mobile World Congress is kicking off, and this got XDA developers hands on with the Xiaomi Wireless AR Smart Glass Explorer Edition. Basically, the furthest anyone has gone in terms of actual consumer grade getting of AR glasses into that sweet spot where they're just sort of eyeglasses. In particular, this features two micro-o-led screens, three front-facing cameras weighing just 126 grams, and it's entirely wireless. Quote, the official name of the product is Jaumie Wireless AR Smart Glass Explorer Edition,

and it looks like an oversized pair of Terminator sunglasses with a silver finish instead of being all black. The glasses feature a pair of micro-Oled screens, one for each eye, that pump out full HD visuals at 1,200 nits of brightness. There are three forward-facing cameras on the front of the glasses that are used to map the environment immediately in front of the wearer. Considering its relatively large size and the cutting-edge components inside, the glasses are lightweight at 126 grams. Jemmy says it managed to keep the weight low by using lightweight materials like carbon fiber, magnesium alloy, and a self-developed silicon oxygen anode battery that's smaller than a typical lithium ion battery. The glasses run on Qualcomm's Snapdragon XR2 Gen 1 chip. It has no

onboard storage, so it must be connected to a host device. During our testing, we used a Xiaomi 13 for use. So far, everything I described has already been done before by existing commercial products like the Unreal Light or Roked Air. So what sets Xiaomi's glasses apart? As the name gives away, the glasses are entirely wireless. It connects to a smartphone via the company's proprietary low-latency communication link. An engineer tells me that it's a combination of Wi-Fi and Bluetooth, and Xiaomi advertises a latency of just 50 milliseconds, which is fast enough that you can't quite perceive noticeable lag. I wore the glasses for over 20 minutes scrolling through TikTok, Instagram, and watching YouTube videos, and I couldn't detect lag nor any image-quality drop-off.

This alone blew me away. I have tested several AR glasses, including the aforementioned Enreal Light, Roked Air plus Enreal Air and TCL NextWare S. And I actually use the Nreal Air every day. All of these require a cable connection and the wire can get in the way, especially when I'm using the glasses while lying in bed or in a cramped airplane seat. At its most basic form, Zhaomi's smart glasses can already do the same thing as the Nreal Air, give me a wearable screen that floats in front of my face, without any wires. But Jami's glasses can do more than just play videos. When I reviewed both the Unreal Air and TCL NextWare S for XDA, I concluded that although they were advertised as AR glasses, they were ultimately best used

as just a wearable display because the AR applications just didn't work too well. Even in its prototype stage, Xiaomi's AR applications are already far more advanced. During the demo, Xiaomi reps showed off three AR tests. The first was running an interactive virtual desktop where I could open multiple windows and have them map onto the real-world environment in front of me. I could control the windows, move them around or resize them via using the connected phone screen as a trackpad, or I could stick out my arm and use hand gestures to physically move the windows. When I stuck out my arm with palms facing outward, a laser beam projects from my palm, which I can use as a pointer. I can use a finger-pinch motion to grab a window and move it around. The second demo was an AR

shooting game in which I could plant a virtual machine gun station on a flat surface. For my demo, I used a table, and I had to shoot these weird-looking animals walking towards me. I could control aiming with the phone or my hands. And finally, the third demo had me jumping into a virtual Xiaomi home UI where I could use my hand and finger to flick a virtual switch to turn on and off a real-life Xiaomi lamp that was on site during the demo. So, Geoemy declined to say why this is a prototype right now when the tech seems to work so well. If I have to guess, I'd suspect it could be battery life or cost. During our 30-minute demo session, the glasses ran out of juice and had to be charged.

Maybe there's no way to put a large enough battery without drastically increasing weight or bulk. The other theory could be that Geommy wants to wait until these glasses can be manufactured at a lower cost before releasing. Whatever the case, I am excited for the day when these glasses are mass production ready. I already love the Unreal Air and carry it with me on every trip. These Xiaomi glasses are like a futuristic version of that, end quote. Finally today, you knew it was coming. Some AI companies are now hiring, or at least advertising to hire, prompt engineers who create and refine text prompts for chatbots

to understand the AI system's flaws and coax out optimal results from them. And they make pretty good money, too. Quoting the Washington Post, Unlike traditional coders, prompt engineers program in pros, sending commands written in plain text to the AI systems which then do the actual work. When Google, Microsoft, and the Research Lab Open AI recently opened their AI search and chat tools to the masses, they also upended a decades-old tradition of human-machine interaction. You don't need to write technical code in languages such as Python or SQL to command the computer. You just talk. The hottest new programming language is English.

Andre Carpathie, Tesla's former chief of AI said last month in a tweet, prompt engineers such as Goodside professed to operate at the maximum limits of what these AI tools can do, understanding their flaws, supercharging their strengths, and gaming out complex strategies to turn simple inputs into results that are truly unique. There are people who belittle prompt engineers saying, oh Lord, you can get paid for typing things into a box, Willison added, but these things lie to you. They mislead you. They pull you false paths to waste time on things that don't work. You're casting spells, and like in fictional magic, nobody understands how the spells work, and if you mispronounce them, demons come to eat

you, end quote. Prompt engineers, Carpathie has said, work like, quote, a kind of AI psychologist, and companies have scrambled to hire their own prompt crafters in hopes of uncovering hidden capabilities. With each request, Goodside said, the prompt engineer should be instilling in the AI a kind of persona, a specific character capable of winnowing down hundreds of billions of potential solutions and identifying the right response. Prompt engineering, he said, citing a 2021 research paper, is most importantly about constraining behavior, blocking off options so that the AI pursues only the human operator's desired continuation. It can be a very difficult mental exercise, he said. You're exploring the

multiverse of fictional possibilities, sculpting the space of these possibilities. and eliminating everything except the text you want, end quote. Some creators now sell their prompts on marketplaces such as prompt base, where buyers can see AI-generated art pieces and pay for the list of words that helped create them. Some sellers offer tips on prompt customization and one-on-one chat support. Roughly 700 prompt engineers now use prompt base to sell prompts by commission for buyers who want, say, a custom script for an e-book or a personalized motivational life coach. The freelance site Fiver offers more than nine.

thousand listings for AI artists. One seller offers to draw your dreams into art for $5. But the work is becoming increasingly professionalized. The AI startup Anthropic, founded by former Open AI employees and the maker of a language AI system called Claude, recently listed a job opening for a prompt engineer and librarian in San Francisco with a salary ranging up to $335,000. Must have a creative hacker spirit and love solving puzzles, the listing states end quote. Nothing for you today. Talk to you tomorrow.