Tech Brew Ride Home - Wed. 08/28 – Strawberry Emoji

Episode Date: August 28, 2024

I explain why everyone has been posting strawberries in AI circles. It’s cause of a potential new breakthrough at OpenAI. Cerebras launches the first new AI chip competition to Nvidia. China has reportedly burrowed into US ISPs. And continuing interesting details pouring out of that Pavel Durov situation. Sponsors: Timeline.com/ride Links: OpenAI Shows ‘Strawberry’ AI to the Feds and Uses It to Develop ‘Orion’ (The Information) OpenAI Races to Launch ‘Strawberry’ Reasoning AI to Boost Chatbot Business (The Information) Cerebras Systems throws down gauntlet to Nvidia with launch of ‘world’s fastest’ AI inference service (SiliconAngle) Chinese government hackers penetrate U.S. internet providers to spy (Washington Post) Google Meet’s automatic AI note-taking is here (The Verge) Instagram adds what photos have always needed: words (The Verge) Telegram Founder Was Wooed and Targeted by Governments (WSJ) Can Tech Executives Be Held Responsible for What Happens on Their Platforms? (NYTimes) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Wednesday, August 28th, 2024. I'm Brian McCalla today. I explain why everyone has been posting strawberries in AI circles. It's because of a potential new breakthrough at OpenAI. Cerebrus launches the first new AI chip competition to Nvidia. China has reportedly burrowed into USISPs and continuing interesting details pouring out of that Pavel Durov situation. Here's what you miss today in the world of tech. For months now in certain AI circles, people have been talking about Strawberries. Strawberries in their bios, strawberry emojis, Twitter spaces about strawberries. Some folks thought strawberries indicated the coming of GPT5, but most of the chatter recently made it seem

like it was going to be something along the lines of what is currently being reported by the information. Their source says, OpenAI has demoed a breakthrough called strawberry to U.S. national security officials. One of its top uses is to make training data for the next flagship LLM from OpenAI code named Orion. But there's lots of interesting stuff here. Quote, in case you were wondering why Sam Altman cryptically posted a picture of strawberries earlier this month, the answer almost certainly has to do with Strawberry, a mysterious technical breakthrough that could help open AI's models complete complex tasks such as math problems that conversational AI has traditionally struggled with. If this sounds familiar, it's likely because

Strawberry was previously called QSTAR. This summer, the OpenAI team demonstrated the technology to American national security officials, set a person with direct knowledge of those meetings, which haven't previously been reported. By demonstrating an unreleased technology to government officials, Open AI could be setting a new standard for AI developers, especially as advanced AI increasingly becomes a national security concern. The demonstration could be part of OpenAIs pushed to become more transparent with policymakers who could cause the company problems if they feel threatened by its technology. One of the most important applications of Strawberry is to generate high-quality training data for Orion, OpenAI's next flagship large language model that's in development.

The code name hasn't previously been reported. Using strawberry could help Orion reduce the number of hallucinations or errors it produces researchers tell me. That's because Open AI models learn from their training data, so the more correct examples of complex reasoning they see the better. But there's also a push within OpenAI to simplify and shrink strawberry through a process called distillation, so it can be used in a chat-based product before Orion is released. This shouldn't come as a surprise, given the intensifying competition among the top AI developers, end quote. So Strawberry sounds like it's what we've been waiting for, a breakthrough in reasoning that will, among other things, allow OpenAI to create Orion, which is essentially GPT5,

but also might be released as an upgrade to chat GPT any time now. More on what this breakthrough actually entails, according to a... different information piece. Quote, strawberry can solve math problems it hadn't seen before, something today's chatbots cannot reliably do, and also has been trained to solve problems involving programming. But it's not limited to answering technical questions. When given additional time to, quote, think, the strawberry model can also answer customers' questions about more subjective topics, such as product marketing strategies. To demonstrate strawberry's prowess with language-related tasks, OpenAI employees have shown their coworkers how Strawberry can, for example,

solve New York Times connections, a complex word puzzle. Open AIs prospects rest in part on the eventual launch of a new flagship LLM. It is currently developing codenamed Orion. That model seeks to improve upon its existing flagship LLM, GPT4, which it launched early last year. By now, other rivals have launched LLMs that perform roughly as well as GPT4. It isn't clear whether a chatbot version of Strawberry that can boost the performance of GPT4 and chat GPT will be good enough to launch this year. The chatbot version is a smaller simplified version of the original strawberry model known as a distillation. It seeks to maintain the same level of performance as a bigger model while being easier and less costly to operate. However, OpenAI is also using the bigger version of strawberry to generate data for training Orion,

set a person with knowledge of the situation. That kind of AI generated data is known as synthetic. It means that Strawberry could help OpenAI overcome limitations on obtaining enough high-quality data to train new models from real-world data, such as text or images pulled from the internet. Using Strawberry to generate higher quality training data could help open AI reduce the number of errors its models generate, otherwise known as hallucinations, said Alex Gravely, CEO of Agent Startup Minion AI, and former chief architect of GitHub co-pilot. Imagine, quote, a model without hallucinations, a model where you ask it a logic puzzle and it's

right on the first try, Gravely said. The reason why the model is able to do that is because, quote, there is less ambiguity in the training data, so it's guessing less. AI that solves tough math problems could be a potentially lucrative application given that existing AI isn't great at math-heavy fields such as aerospace and structural engineering. It's a goal that has tripped up AI researchers who have found that conversational AI, chat GPT, and its ilk, is prone to giving wrong answers that would flunk any math student. Improvements in mathematical reasoning could also help AI models reason better about conversational queries such as customer service requests. Strawberry has its roots in research. It was started years ago by

Ilya Suscever, then OpenAI's chief scientist. He recently left to start a competing AI lab. Before he left, Open AI researchers Jacob Pachoki and Simon Sidor built on Susceiver's work by developing a new math-solving model, Q-Star, alarming some researchers focused on AI safety. The breakthrough and safety conflicts at OpenAI came just before OpenAI board directors, led by Suskever, fired a Sam Altman before quickly rehiring him, end quote. More AI horse race stuff, Cerebras has launched what they are calling the world's fastest AI inference service with, quote, GPU impossible performance with costs starting at 10 cents per million tokens. This is the first salvo in the race to rival Nvidia in making better AI-specific chips. Quoting Silicon Angle, Ambitious, Ambitious, Artificial Intelligence Computing Startup, Cerebus Systems is raising the stakes in its battle against Nvidia. Launching what it says is the world's fastest AI. inference service, and it's available now in the cloud. AI inference refers to the process of running

live data through a trained AI model to make a prediction or solve a task. Inference services are the workhorse of the AI industry, and according to Cerebrus, it's the fastest growing segment two, accounting for about 40% of all AI workloads in the cloud today. Cerebrus believes this launch is a watershed moment for the AI industry, saying that a thousand tokens per second speeds that it can deliver is comparable to the introduction of broadband internet, enabling game-changing new opportunities for AI applications. So RebreS is well equipped to offer such a service. The company is a producer of specialized and powerful computer chips for AI and high-performance computing or HPC workloads. It has made a number of headlines over the past year, claiming that its chips are not only more

powerful than NVIDIA's graphics processing units, but also more cost-effective. This is GPU Impossible Performance, declared co-founder and chief technology officer, Sean. lie. Its flagship product is the new WSE3 processor, which was announced in March and builds upon its earlier WSE2 chip set that debuted in 2021. It's built on an advanced 5-anometer process and features 1.4 trillion transistors, more than its predecessor chip, with more than 900,000 compute cores and 44 gigabytes of onboard static random access memory. According to the startup, the WSE3 has 52 times more cores than a single NVIDIA H-H-100 graphics process. unit. The chip is available as part of a data center appliance called the CS3, which is about the same

size as a small refrigerator. The chip itself is about the same size as a pizza and comes with integrated cooling and power delivery modules. In terms of performance, the Cerebrus WSE3 is said to be twice as powerful as the WSE2, capable of hitting a peak speed of 125 petaflops with one petap equal to 1,000 trillion computations per second. The Cerebrus CS3 system is the engine that powers the new Cerebris Inference Service, and it notably features 7,000 times greater memory than the NVIDIA H-100 GPU to solve one of Generative AI's fundamental technical challenges, the need for more memory bandwidth. It solves that challenge in style. The Cerebris Infference Service is said to be lightning quick, up to 20 times faster than comparable cloud-based inference services

that use NVIDIA's most powerful GPUs. According to Cerebris, it delivers 1,800 tokens per second for the open source Lama 3.1.8b model and 450 tokens per second for Lama 3.17B. It's competitively priced, too, with the startup saying that the service starts at just 10 cents per million tokens, equating to 100 times higher price performance for AI inference workloads. The company adds the Cerebrus inference service is especially well suited for agentic AI workloads or AI agents that can perform tasks on behalf of users, as such applications need the ability to constantly prompt their underlying models, end quote. Sources are telling the Washington Post that China-linked hackers have penetrated deep into

two big U.S. Internet service providers and several smaller ones in recent months using a zero-day flaw in Versa Network's software. Quote, the unusually aggressive and sophisticated attacks include access to at least two major U.S. providers with millions of customers as well as to several smaller providers. People familiar with the separate campaigns said, it is business. as usual now for China, but that is dramatically stepped up from where it used to be. It is an order of magnitude worse, said Brandon Wales, who until earlier this month, was Executive Director of the Cybersecurity and Infrastructure Security Agency CISA. The hacks raise concerns because

their targets are believed to include government and military personnel working undercover and groups of strategic interest to China. This is privileged, high-level connectivity to interesting customers, said Mike Horka, a former FBI agent and current researcher at Lumen Technologies, which described one of the campaigns but didn't identify the ISPs it targeted. It was notable, he added, that the groups considered the effort important enough to exploit previously undiscovered software flaws that could have been preserved for later use. Now, there is no evidence that the new inroads are aimed at anything other than gathering intelligence. Some of the techniques and resources employed are associated with those used in the past year

by a China-backed group known as Volt Typhoon. Two of the people said, the U.S. intelligence officials said that group sought access to equipment at Pacific ports and other infrastructure to enable China to so panic and disrupt America's ability to move troops, weaponry, and supplies to Taiwan if armed conflict breaks out. In a blog made public Tuesday, Lumen said that the hackers used a previously unknown vulnerability known as a zero-day flaw in a program made by VERSA networks for managing wide area networks. Vista acknowledged the critical vulnerability late last week, warning only its direct customers. Lumen wrote that it located malware inside ISP routers serving certain groups or individual customers that could intercept passwords

from those customers. Lumen said it believed the malicious software was being used by Volt Typhoon. In a separate report earlier this month, security company Vlexity said it had found another high-end technique in play at a different unnamed ISP. In that case, it said a Chinese state hacking group distinct from Volt Typhoon was able to get far enough inside the service provider to alter the domain name system or DNS web addresses that users were trying to reach and divert them elsewhere, allowing the hackers to insert backdoors for spying. Though they avoided discussing threats to ISPs specifically, some of the top U.S. cybersecurity officials at the recent Black Hat and DefCon hacking conferences said Volt Typhoon remained as active and successful as it was when its operations were

first identified last year, end quote. Quick little omnibus segment here with two feature upgrade headlines. Google has rolled out a Google Meet AI feature that automatically takes notes during meetings conducted in spoken English, available to select Workspace customers. Quoting the Verge. Google Meet's newest AI-powered feature takes notes for me has started rolling out today to Google workspace customers with the Gemini Enterprise, Gemini Education Premium, or AI meetings and messaging add-ons. It's similar to Meets transcription tool, only instead of automatically transcribing what everyone says, it summarizes what everyone talked about. Google first announced this feature at its 2023 Cloud Next conference. Unfortunately, it only supports spoken.

in English right now, but it seems like it could make missing an important meeting less stressful. It automatically takes notes in a Google Doc and will attach that file to the calendar event after the meeting is over so you can reference them later on. It will also send that Google doc to the meeting organizer and anyone else who turned the feature on. Running late to a meeting, Google says its new feature will also give you a summary of what you missed, so when you are able to join, you can quickly catch up and no one should have to worry about repeating themselves, end quote. The second one is this. Instagram. Now lets users add text to photos right from its post editor rather than needing a separate app and add more photos as stickers on top of a photo. Once more from the verge. A bit of text might help your post stand out from the rest of the photos on your feed or help people see information that might otherwise get lost in your caption. From the post editor, you'll also be able to layer another photo on top of a photo as a sticker. When you tap on that layered photo, you can change its shape from a rectangle to a square, circle, heart, or star according to the

Instagram blog post. The feature seems fun to me. I suspect people will find some clever ways to use this in their photos, end quote. Finally, today, a catch-up on the drips and drabs from the Pavel Durov situation. Politico says that alongside Pavel, French authorities have issued an arrest warrant for his telegram co-founder and brother Nikolai. They did this back in March. Sources are telling the journal that Emmanuel Macron invited Pavel Dura to move Telegram to Paris back at a meeting in 2018. Meanwhile, back in 2017, a joint operation between French and UAE spies hacked Duraev's iPhone. What? Quote, the spy operation, which also hasn't been previously reported, was codenamed Purple Music, the people said. French security officials were acutely concerned about

Islamic State's use of telegram to recruit operatives and plan attacks. Governments have targeted Duraov because of the groups that were drawn to his app, which ranged from pro-democracy demonstrators and dissidents, to Islamist, militants. drug traffickers and cybercriminals. For years, the company ignored subpoenas and court orders sent by law enforcement authorities, which piled up in a rarely checked company email address, according to a person close to Derov, end quote. So it's sounding like the various organizations have just been concerned about what goes on on Telegram for a while now. U.S.-based Child Safety Group NCMEC, Canadian-based CCCP, and the UK-based IWF say their outreach to telegram, to flag CSAM on the platform, has largely been ignored. So, is this a telegram-specific situation? Some of the more pearl-clutching tech watchers out there have said this might actually signal a new, more aggressive crackdown on tech. Forget antitrust regulation. might this lead to the criminalization of tech platforms? Should say Mark Zuckerberg be wary of traveling to

Europe anytime soon? The New York Times asked legal experts about this, and they reported back that Pavaldorov's arrest may be an outlier, as the legal bar is high in the U.S. and Europe to prosecute tech executives for content posted on their platforms. Quote, one challenge for prosecutors and law enforcement agencies is proving a tech executive had knowledge of illegal activity. on their platforms and did not try to curb the harms, said Daphne Keller, a professor of internet law at Stanford University Law School. That's difficult to demonstrate since TikTok, YouTube, snap and meta, which owns Facebook and Instagram, have worked to take down and report illegal content to law enforcement officials so their executives can argue they tried to do the right

thing. Knowledge is the key issue here, said Ms. Keller, a former lawyer for Google. It's the usual trigger for anyone losing immunity, end quote. Nothing more for you today. Talk to you tomorrow.