Tech Brew Ride Home - Wed. 04/05 – The Grand Theory Of AI Varietals

Episode Date: April 5, 2023

Sony might be developing a sort of PS Vita mark 2. Sad news of the death of a well-known silicon valley luminary. Meta says its not going to be left behind in the AI race. What happens if the kids aren’t keen on the Metaverse. And I sketch out an example of our grand theory of AI varietals. Sponsors: Bloomberg.com/careers Links: EXCLUSIVE – SONY’S NEXT PLAYSTATION HANDHELD (Insider Gaming) Bob Lee, creator of Cash App and former CTO of Square, stabbed to death (TechCrunch) Google says its AI supercomputer is faster, greener than Nvidia A100 chip (Reuters) Meta to debut ad-creating generative AI this year, CTO says (NikkeiAsia) American teens aren’t excited about virtual reality, with only 4% using it daily (CNBC) What if ChatGPT was trained on decades of financial news and data? BloombergGPT aims to be a domain-specific AI for business news (NeimanLab) Learn more about your ad choices. Visit megaphone.fm/adchoices

Welcome to the TechMeme right home for Wednesday, April 5th, 2023. I'm Brian McCullough. Today, Sony might be developing a sort of PSVita Mark 2. Sad news of the death of a well-known Silicon Valley Luminary. Meta says it's not going to be left behind in the AI race. What happens if the kids aren't keen on the Metaverse? And I sketch out an example of our grand theory of AI varietals. Here's what you missed today in the world of tech. Sources are telling various outlets that Sony is developing a handheld gaming

system codenamed QLight, which will use remote play with the PS5 and feature an 8-inch 1080p touchscreen and adaptive triggers. So if, like me, you were a fan of the PlayStation Vita, your prayers might be answered. Quoting Insider Gaming. Code named the QLite, the next PlayStation handheld, is the next piece of Sony's hardware that aims to be yet another piece of hardware that requires the PlayStation 5. Insider Gaming understands that the QLight is not a cloud streaming device, but instead uses remote play with the PlayStation 5, a feature the console giant has been pushing these past couple of weeks. Sporting adaptive streaming up to 1080p and 60 frames per second, the new device will require constant connectivity to the internet. As for the console's physical features,

early prototypes show the console will look a lot like a PlayStation 5 controller, but with a massive 8-inch LCD touchscreen at the center. The device sports adaptive triggers for haptic feedback and will include what you would come to expect from a handheld, volume buttons, speakers, and audio jack, input, etc. Insider Gaming understands that the Q-Light is in its quality assurance phase and is scheduled to release before the PlayStation 5 Pro and after the detachable disc drive PS5. As previously mentioned by industry insider Jeff Grub, Sony is planning to announce its second. phase of the PS5, which was in reference to its future game slate. Ironically, though, this second phase is very much true for Sony's hardware offerings, with the new detachable disc drive

PlayStation 5, Project Nomad, which are wireless earphones, Project Voyager, wireless headset, and QLight, the handheld, all scheduled to release within a very short period. It's understood that the PlayStation 5 Pro is aiming for a holiday 2024 release. So whenever Sony's next PlayStation showcase will be, it's seemingly going to be a big one, end quote. I woke up to text messages from friends this morning that Bob Lee had died overnight in San Francisco. This seems to have been confirmed subsequently. Lee was most recently chief product officer at Mobile Coin, but was perhaps best known as early on the Android team and the cash app as well as being Square's first CTO.

I want to caution that details around the story are still emerging, but quoting TechCrunch. On Tuesday morning at 2.35 a.m., the San Francisco Police Department responded to a report of a stabbing near the 300 block of Main Street in Soma. He was taken to a hospital but succumbed to his injuries. Shortly after, NBC Bay Area reported that the victim of the stabbing was Bob Lee, 43. Mobile Coin confirmed the information in a statement sent to Bloomberg and ABC 7 News. Before joining Mobile Coin, Bob Lee worked at Google for the first few years of Android focusing on core library development. He then joined Square, the payment company that later became Block, to develop its Android app. He became the company's first CTO and also created Cash App. So sad to

hear of At Crazy Bob's untimely passing. I first met him in summer 2006. He didn't care that I was only 14 and we talked to tech, geeked out about programming. We remained connected over the years and he was an early supporter of Figma. It's so hard to believe he is gone, Figma CEO, Dillon Field wrote on Twitter. What a tragedy. I remember Bob's code that generates certain numbers. Bob's code is in Java, yet uses clever backtracking techniques to achieve the best performance among many solutions and all kinds of languages. Hinty Maud wrote on Hacker News. No arrest has been made in the case, and the San Francisco Police Department hasn't shared any additional details, end quote. Google's stock was up this morning in pre-trading on the simple headline that the company is making claims that its fourth-generation T-P

CPU-based supercomputers used for AI training are up to 1.7x faster and 1.9x more power-efficient than Nvidia's A-100 systems, which, as we've discussed, the A-100s have thus far been the workhours of the AI revolution, quoting Reuters. Google has designed its own custom chip called the tensor processing unit, or TPU. It uses those chips for more than 90% of the company's work on artificial intelligence training, the process of feeding data through models to make them useful at tasks, such as responding to queries with human-like text or generating images. The Google TPU is now in its fourth generation. Google on Tuesday published a scientific paper detailing how it has strung more than

4,000 of the chips together into a supercomputer using its own custom-developed optical switches to help connect individual machines. Improving these connections has become a key point of competition among companies that build AI supercomputers because so-called large language models that power technology like Google's BARD or OpenAI's chat GPT have exploded in size, meaning they are far too large to store on a single chip. The models must instead be split across thousands of chips, which must then work together for weeks or more to train the model. Google's Palm model, its largest publicly disclosed language model to date, was trained by splitting it across two of the 4,000 chip supercomputers over 50 days.

Google said its supercomputers make it easy to reconfigure connections between chips on the fly, helping avoid problems and tweak for performance gains. While Google is only now releasing details about its supercomputer, it has been online inside the company since 2020 in a data center in Mays County, Oklahoma. Google said that startup mid-journey used the system to train its model, which generates fresh images after being fed a few words of text. Google said it did not compare its fourth generation to Nvidia's current flagship H-100 chip because H-100 came to the market after Google's chip and is made with newer technology. Google hinted that it might be working on a new TPU that would compete with the NVIDIA H-100, but provided no details with the company telling Reuters that Google has, quote,

a healthy pipeline of future chips, end quote. Hey, you know who has a huge AI division but hasn't really joined the AI productization race yet? That would be meta. Well, meta-CTO Andrew Bosworth says the company plans to begin commercializing its generative AI this year in 2023 and remains at the very forefront of the LLM field, in his words, quoting Nika Asia. Meta established a research lab for AI in 2013 by inviting Jan LeCoon, a French scientist and leading expert on the technology, to join the effort. It was second only to Google in the number of citations in major studies published on AI in 2022, according to AI research analysis platform Zeta Alpha, based in the Netherlands. We've been investing in artificial

intelligence for over a decade and have one of the leading research institutes in the world. Bosworth, Meta's chief technology officer, told NICA in an exclusive interview on Wednesday in Tokyo, we certainly have a large research organization hundreds of people, and quote, Meta announced in February that it would establish a new organization to develop generative AI, but this is the first time it has indicated a timeline for commercialization. We feel very confident that we are at the very forefront, Bosworth said. Quite a few of the technologies that are in large language model development were pioneered by our teams.

I expect we'll start seeing some of them, commercialization of the tech, this year. We just created a new team, the generative AI team, a couple months ago. They are very busy. It's probably the area that I'm spending the most time in, as well as Mark Zuckerberg and Chief Product Officer Chris Cox, end quote. The technology will also be used in the Metaverse, a highly realistic virtual space. meta is eagerly developing. So previously, Bosworth said, if I wanted to create a 3D world, I needed to learn a lot of computer graphics and programming. In the future, you might be able to

just describe the world you want to create and have the large language model generate that world for you. And so it makes things like content creation much more accessible to more people, end quote. Yes, but on that metaverse front, if meta expects, as tends to happen in technology, or at least has tended to happen over the last 30 years, that it will be the kids who will lead in terms of embracing the Metaverse and taking it mainstream. I've got some bad news. A Piper Sandler survey of 5,690 U.S. teens in February revealed that while 29% of U.S. teens owned a VR device of those, only 14% used it weekly, which would be flat compared to numbers from the fall of 2022, and only 4% of them used it daily, quoting CNBC. In addition, teenagers didn't seem

that interested in buying forthcoming VR headsets. Only 7% said they plan to purchase a headset versus 52% of teens polled who were unsure or uninterested. The survey results suggest that virtual reality hardware and software has yet to catch on with the public, despite billions of dollars in investment in the technology from big tech companies and a number of low-cost headsets on the market. Teenagers are often seen as early adopters of new technology and their preferences can provide a preview of where the industry is going. To us, the lukewarm usage, demonstrates that VR remains early days and that these devices are less important than smartphones, Piper Sandler analysts wrote, end quote. Finally today, for the last few weeks, Chris and I have been

batting around this theory about where the value in the AI revolution might accrue. What is the sort of company you could build on this stuff? You've been hearing me ask that question for months now. I think I've also mentioned this theory before, but we've been gravitating towards a theory of AI varietals. You know, varietals like how in wine, grapes grown in a given valley at a given altitude or latitude in a given soil might develop a different taste signature than grapes somewhere else in the world. Might people develop preferences for the tastes of one AI grape if you were over another? The analogy would be that with AI, if you cultivate the training sets differently, the corpus of data that you use with the model is varied and then you adjust the temperature.

remember, that's a real AI term, you know, tweak the knobs and dials a bit, would users eventually grow to prefer one AI model's output over another? Maybe for greater accuracy in one, or greater creativity in another, would one model provide better outcomes for, say, making breast cancer diagnoses, but not as good for skin cancer? Would an AI varietal end up being like the secret formula for Coca-Cola. Either that, or, you know, it's just going to be the biggest LLM wins and always will, so the theory would be moot. But in case it's not moot, Neiman Lab has a look at Bloomberg GPT, a recently announced large language model from Bloomberg, announced on March 30th, actually, and trained on general purpose datasets and Bloomberg's own archives of news, filings, financial

documents, and more. So, is this an example of an AI-varietal? for finance. I do want to quickly acknowledge that a Bloomberg ad is scheduled to run on today's show, so I want to stress to you, I'm not doing this story because of that ad. This is not a SpanCon situation. It's just coincidental that this story arrived today to give me the chance to go into this whole Varietal's idea. Quote, if you were going to predict which news company would be the first out with its own massive AI model, Bloomberg would have been a good bet. For all its success expanding into consumer-facing news over the past decade, Bloomberg is fundamentally a data company driven by $30,000 a year subscriptions to its terminals. How big is Bloomberg's new

Bloomberg GPT? Well, the company says it was trained on a corpus of more than 700 billion tokens or word fragments. For context, GPT3 release in 2020 was trained on about 500 billion. OpenAI has declined to reveal any equivalent number for GPT4, the successor release last month, citing the competitive landscape. So what's in all that training data in Bloomberg GPT of the 700 billion plus tokens? 363 billion are taken from Bloomberg's own financial data, the sort of information that powers its terminals. The largest domain-specific data set yet constructed, Bloomberg says. Another 345 billion tokens come from general purpose data sets obtained from elsewhere. Here's quoting from Bloomberg's own announcement. Rather than building a general purpose LLM or a small

LLM exclusively on domain-specific data, we take a mixed approach. General models cover many domains, are able to perform at a high level across a wide variety of tasks, and obviate the need for specialization during training time. However, results from existing domain-specific models show that general models cannot replace them. At Bloomberg, we support a very large and diverse set of tasks, well served by a general model, but the vast majority of our applications are within the financial domain, better served by a specific model. For that reason, we support. For that reason, we support, We set out to build a model that achieves best-in-class results on financial benchmarks, while also maintaining competitive performance on general purpose LLM benchmarks, end quote.

The company's specific data named FinPile consists of a, quote, range of English financial documents, including news, filings, press releases, web scraped financial documents, and social media drawn from the Bloomberg Archives, end quote. So if you've read a Bloomberg Businessweek story in the past few years, it's in there. So are SEC filings, Bloomberg TV transcripts, data and quote other data relevant to the financial markets. It's also trained on non-Bloomberg news sources. The non-finance specific data includes a massive corpus poetically known as the pile. It includes everything from YouTube captions to Project Gutenberg to yes, the cache of Enron emails that are

always popping up in AI training. It also has a complete copy of Wikipedia as of last July. Because it shares a training base with other LLMs, Bloomberg GPD can do the sorts of things that we've come to expect from chat GPT and similar models, but it can also perform tasks more tightly connected to Bloomberg's needs. It can translate natural language requests like Apple and IBM Market Cap and EPS into the Bloomberg query language, terminal users love slash hate. It can also suggest Bloomberg style headlines for news stories. It's also better tuned, they say, to answer specific business-related questions, whether they be sentiment, analysis, categorization, data extraction, or something else entirely. For example, it

performs well at identifying the CEO of a company. The paper includes a series of performance comparisons with GPT3 and other LLMs and finds that Bloomberg GPD holds its own on general tasks, at least when facing off against similarly size models, and outperforms on many finance-specific ones. The internal testing battery includes such carnival game-ready terms as penguins in a table, snarks, web of lies, and the dreaded hyperbation. Penguins aside, it's not hard to imagine more specific use cases that go beyond benchmarking, either for Bloomberg's journalists or its terminal customers. The company's announcement didn't specify what it planned to do with what it is built. A corpus of all the world's premium English-language-based business reporting, plus the

universe of financial data, structured and otherwise that underpins it, is just the sort of rich vein of information a generative AI is designed to mine. Its institutional memory in a box, end quote. So let's take this to its logical conclusion. Imagine your company or any company eventually having a company-specific LLM. Then go one step further, all the way down to you having your own personal LLM trained or tweaked only on you and your data. It would almost be as if the AI revolution would mimic the computer revolution, going from huge computers only in offices like that thing in Mad Men, and then going to PCs on every desk in every office, and then PCs in every house in North America, then a smartphone in every person's pocket.

Quoting Van Spina on Twitter, Bloomberg GPD is going to replace the analyst. Analysts are fundamentally chat-based interfaces that senior finance folks use to gather, organize, and output data. Finance workflows are already very iterative, and GPT doesn't care about protected Saturdays, end quote. Here's Ethan Mollick on Twitter. The new Bloomberg ChatGPT AI may be a harbinger of the next wave of corporate AI.

Current AIs are trained on web data, though firms can add their own training. Bloomberg GPD is 52% either proprietary data or cleaned financial data, and it shows signs of being better at financial tasks. And quote, and finally quoting Matt Turk on Twitter. The most interesting AI news of the week for me is Bloomberg's 50 billion parameter model trained on financial data. points to a polyglot future where a number of players can win in AI, as opposed to just big tech and or open AI, end quote. If you do start using that term AI varietals, be sure to credit Chris Messina and I. We can't remember who first came up with it, though I do remember the phone conversation where we stumbled on the analogy. Was at the start of March, maybe? Great minds think alike.

Anyway, talk to you tomorrow.