Tech Brew Ride Home - Fri. 05/05 – Can Open Source Win Out Over Open AI? (Corrected)

Episode Date: May 5, 2023

Apple earnings recap. A new, free code generating AI tool. Which, actually, lots of people are starting to ask the question: will open source AI actually come out on top in the AI revolution? And, of course, the weekend longreads suggestions. Sponsors: Mindbloom.com/techmeme and promocode techmeme OregonState.edu/believe-it Links: Apple CEO Tim Cook says AI is "huge," but care is needed (Axios) Hugging Face and ServiceNow release a free code-generating model (TechCrunch) Google Is Falling Behind in AI Arms Race, Senior Engineer Warns (Bloomberg) OpenAI’s Losses Doubled to $540 Million as It Developed ChatGPT (The Information) Weekend Longreads Suggestions AI Singers Are Unnervingly Good and Already Ubiquitous (Vulture) Is the Federal Government Trying to Kill Off Crypto? (Intelligencer/NYMag) Why Chatbots Are Not the Future (Amelia Wattenberger) ESPN’s Jimmy Pitaro Will Decide the Fate of Cable Television (Bloomberg) Learn more about your ad choices. Visit megaphone.fm/adchoices

Welcome to the Tech meme right home for Friday, May 5th, 2020. I'm Brian McCullough today. Apple earnings recap, a new free code-generating AI tool, which actually lots of people are also starting to ask the question, will open-source AI actually come out on top in the AI revolution? And of course, the weekend long-wring suggestions. Here's what you miss today in the world of tech. Apple earnings, you know, fine. It was all fine. Overall revenue was down 3%. It's second quarter of declining revenue, actually, but Apple did beat on estimates. In terms of the product mix, iPhone revenue was down and expected 2%. Mac revenue was down a whopping 31%. But for the services segment, which includes Apple TV Plus and Apple Music, that grew 5.5% and is now coming in at $20 billion a quarter. Addressing something that I mentioned on this show recently on the earnings call, Tim Cook, said, Apple views AI. as huge. But there are, quote, a number of issues that need to be sorted, and he didn't expand on how

Apple plans to use AI in its products. Quoting Axios. Speaking to analysts after a better than expected earnings report, Cook noted that Apple has used machine learning and other AI approaches to power features such as crash detection and heart rate monitoring. We view AI as huge and will continue weaving it into our products on a very thoughtful basis, Cook said, while noting the company doesn't talk about its future roadmap. The potential is certainly very interesting, he said. Yes, but Cook also sounded a cautious note, quote, I do think it's very important to be deliberate and thoughtful, he said. There's a number of issues that need to be sorted, end quote. Hugging Face and Service Now Research have released StarCoder, a free alternative to code generating AI systems like GitHub's

co-pilot as a part of their big code project, quoting TechCrunch. Code generating systems like DeepMind's Alpha Code, Amazon's Code Whisper, and OpenAI's Codex, which powers copilot, provide a tantalizing glimpse at what's possible with AI within the realm of computer programming. Assuming the ethical, technical, and legal issues are someday ironed out, and AI-powered coding tools don't cause more bugs and security exploits than they solve. They could cut development costs substantially while allowing coders to focus on more creative tasks. According to a study from the University of Cambridge, at least half of developers' efforts are currently spent debugging and not actively programming, which costs the software industry

an estimated $312 billion per year. But so far, only a handful of code-generating AI systems have been made freely available to the public, reflecting the commercial incentives of the organizations building them. StarCoder, which by contrast is licensed to allow for royalty-free use by anyone, including corporations, was trained on over 80 programming languages, as well as text from GitHub repositories, including documentation and programming notebooks. StarCoder integrates with Microsoft's Visual Studio Code, Code Editor, and, like OpenAI's ChatGBTBT, can follow basic instructions, e.g. create an app UI and answer questions about code. Leandro von Wera, a machine learning engineer at Hugging Face and a co-lead on StarCoder, claims that StarCoder matches or

outperforms the AI model from OpenAI that was used to power initial versions of copilot. StarCoder is a part of Huging faces and service now's over 600-person big code project launched late last year, which aims to develop state-of-the-art AI systems for code in an open and responsible way. Hugging Face supplied an in-house compute cluster of 512 Nvidia V100 GPUs to train the StarCoder model, end quote. Free and open source. Overnight, lots of chatter about a leaked April 2023 internal document by Luke Sernau, a senior software engineer at Google, which made the case that open source AI would actually outcompete the

efforts of Google and Open AI in the AI space, that basically, to coin a phrase, Google and Open AI have no moat at the moment because all of these open models are proliferating. Quoting Bloomberg. The engineer Luke Sernau published the document on an internal system at Google in early April. Over the past few weeks, the document was shared thousands of times among Googlers, according to a person familiar with the matter, who asked not to be named, because they were not authorized to discuss internal company matters. On Thursday, the document was published by the consulting firm Semi-analysis and made the rounds in Silicon Valley. In Cernow's analysis, Google's rivalry with startup OpenA.I. had distracted from the rapid developments being made in

open-source technology. We've done a lot of looking over our shoulders at OpenA.I., who will cross the next milestone, what will the next move be, he wrote. But the uncomfortable truth is, we aren't positioned to win this arms race, and neither is Open AI. While we've been squabbling, a third faction has been quietly eating our lunch. I'm talking, of course, about open source, end quote. Sir now asserted that the real threat to Google is coming from open source communities where engineers are speedily advancing models that rival the quality of those at big tech companies and can be made more cheaply. These models, he said, can be faster, more customizable, and more useful than Google's own. We have no secret sauce, Sir now wrote. Our best

hope is to learn from and collaborate with what others are doing outside Google, end quote. He expressed concerns that clients would not be willing to pay for models with such high-quality technology on offer for free. Google would do well to shift its focus to smaller, more nimble models, sir now argued, quote, giant models are slowing us down, the engineer wrote. In the long run, the best models are the ones which can be iterated upon quickly, end quote. quoting a tweet in response to this from Alex Demacchus, quote, 1, open source AI is winning. I agree. That is great for the world and for a competitive ecosystem.

In LLMs, we're not there, but we just got OpenClip to beat OpenAI clip, and Stable Diffusion is better than closed models. Number two, you don't need huge models. High quality data is much more efficient and important. Alpacking models behind APIs further reduces moats. Number three, you can start with a good, foundation model and parameter-efficient fine-tuning, P-E-F-T algorithms like Laura, works super well in a day.

Finally, an opening for algorithmic innovations. Number four, universities and open-source communities should organize more to curate datasets, train foundation models, and build communities with fine-tunings as done with stable diffusion. Good news for AI and the world, in my opinion, end quote. Counterpoint, though, from Raj Singh on Twitter, quote, owning the AI developer platform relationship, which OpenAI is doing, is the moat. It's the same moat Microsoft had with Windows developers.

It's the same moat AWS has with cloud developers, end quote. Interesting to pair that debate with reporting from the information yesterday, suggesting that OpenAI's losses roughly doubled to $540 million in 2022. And that was before, you know, the last six months when things really took off. Also, Sam Altman has apparently privately suggested the company may try to raise as much as $100 billion in the coming years. Quote, even as revenue has picked up reaching an annual pace of hundreds of millions of dollars just weeks after OpenAI launched a paid version of the chatbot in February, those costs

are likely to keep rising as more customers use its artificial intelligence technology and the company trains future versions of the software. Reflecting that capital drain, CEO Sam Altman has privately suggested OpenAI may try to raise as much as $100 billion in the coming years to achieve its aim of developing artificial general intelligence that is advanced enough to improve its own capabilities, his associate said. Seemingly alluding to this possibility, Altman said in a public appearance Wednesday that Open AI is, quote, going to be the most capital-intensive startup in Silicon Valley history, said a person who attended. Besides covering the costs of training at software, it may also need

to pay for access to data sets that aren't on the internet and that it would want to use to develop its AI, end quote. Time for the weekend long read suggestions. First up, Vulture takes a look at that software program that made that AI Drake song that I told you about a while ago, quote, the software that makes this possible is called SoftVC VITS, SVC, singing voice conversion or SoftVITS SVC. It's free, open source, and can run locally on any computer with a decent GPU. When it launched in March, it was buggy and required coding ability to use, but,

but it's been getting easier as updated versions arrive almost daily. If you just want to create a simple cover song, there are now websites that automate most of the process. To train an AI model on the singer of your choice, feed the app 20 to 30 minutes of high-quality, Acapella Audio, and wait a few hours while it works its magic. If you're in a hurry, you can use a model made by someone else. Pop stars such as Bad Bunny and Taylor Swift are available, as you'd expect,

but so are metalheads like James Hetfield and Pantera's Phil and Selmo. A good model can perform any song you want as long as you have it isolated in its vocal and instrumental tracks, and if you don't, there are other programs you can use to separate them. I'm an average consumer with no coding experience whatsoever, says Garrity, but I figured out how to use Sovitz SVC from YouTube, and it trained my Liam Gallagher model in about 12 hours. There are limitations. For best results, it helps if your AI clone has the same vocal range as the singer they're filling in for, which is why Kanye's cover, of Hello lacks some of the Majesty of Adele's version. Sovitz SVC can handle only one voice at a time

so you can forget about having Kanye do the five-part harmonies in Bohemian Rhapsody. Also, your AI model will follow the original vocalist's phrasings and inflections, so singers with accents or other distinctive tics may be harder to replace. See, for instance, Kanye's disastrous rendition of Nina's 99 Luftballoons. Most important, neither Sovitz SVC nor any other software can reliably write good music and lyrics on its own yet. So the best AI-generated songs still require creative input from humans. Of course, at the rate generative AI is advancing, all of those obstacles could be overcome by tomorrow, which is why many who derive their incomes from recorded or published music are panicking right now, end quote. Then New York Magazine takes a look at the question

I'm hearing from a lot of folks in crypto lately. Is the U.S. government actively trying to kill crypto? Will the crypto industry have to go offshore to survive? Quote, in the crypto industry, the experience of Protigo and that of many others like it has led to an almost universal conviction that financial regulators are purposefully trying to put them out of business, not barring them explicitly, but rather through the recent appearance of a web of policies, both written and unwritten, that together make it unfeasible or impossible for crypto to operate in the U.S. It feels coordinated. It feels like a carpet bombing, says Kristen Smith, of the Blockchain Association, and there's a certain realization that we have to fight back.

Some observers from the government and law communities are raising similar complaints. It sure seems like the OCC and specifically acting comptroller of the currency, Mike Sioux, really doesn't want to approve these applicants, says a former regulatory official, pointing out that it's unusual for a firm to receive conditional approval only later to be denied. Generally speaking, once you get your conditional approval to open, it's kind of a glide path. But there's a lot of supervisory discretion inherent in that whole licensing process. where if they're looking for a certain outcome, there are ways to get there, end quote. Former federal prosecutor Katie Juan, who now runs a crypto-focused venture capital firm in Silicon

Valley and others, have compared current government efforts to Operation Choke Point, a secret Obama-era policy that sought to sideline legal but widely reviled industries like payday lending, gun dealing, and porn by cutting off their access to the banking system. Many in the crypto industry, which critics tend to see as neither legitimate nor productive, are now calling what's happening to crypto Operation Chokepoint 2.0. It's different from the original chokepoint in that they are being pretty public about it. Nobody's guessing their views, says the former regulatory official who spoke on condition of anonymity. Another difference is that it's actually broader in scope, end quote. Amelia Wattonberger has an interesting blog post up

arguing that chatbots are not the future of interfaces for large language models. The assertion is that the best prompts are not obvious right now. People will get sick of typing, responses are isolated and basically no one has cracked the UI or UX yet. Quote, when a painter is working, there are two distinct actions up close, smoohing paint around on the canvas and stepping back to evaluate and plan. These two modes implementing and evaluating are present in any craft, programming, writing, you name it. Good tools let the user choose when to switch between implementation and evaluation. When I work with a chatbot, I'm forced to frequently switch between the two modes. I ask a question, implement, and then I

read a response. Evaluate. There is no flow state if I'm stopping every few seconds to read a response. The wait for a response is also a negative factor here. As a developer, when I have a lengthy compile loop, I have to wait long enough to lose the thread of what I was doing. The same is true for chatbots. The way I see it, there is a spectrum of how much human input is required for a task. When a task requires mostly human input, the human is in control. They are the one making the key decisions, and it's clear that they're ultimately responsible for the outcome. But once we offload the majority of the work to a machine, the human is no longer in control. There's a no man's land where the human is still required to make decisions, but they're not in control of the outcome. At the

far end of the spectrum, users feel like machine operators. They're just pressing buttons and the machine is doing the work. There isn't much craft in operating a machine. Automating tasks is going to be amazing for rote, straightforward work that requires no human input, but if those tasks can only be partially automated, the interface is going to be crucial. I want to see more tools and fewer operated machines. We should be embracing our humanity instead of blindly improving efficiency. And that involves using our new AI technology in more deft ways than generating more content for humans to evaluate. I believe the real game changers are going to have little to do with plain content generation. Let's build tools that offer suggestions to help us gain clarity in our

thinking. Let us sculpt pros like clay by manipulating geometry in the latent space and chain models under the hood to let us move objects instead of pixels in a video, end quote. And finally, something we've been banging the drum about for years on this pod, when will ESPN just become an app? Just go online entirely. Basically, what ESPN decides to do and when it decides to do it is going to decide the fate of cable television. Quoting Bloomberg. With ESPN's streaming subscribers growing and its cable subscribers shrinking, the two businesses are starting to converge. Trying to forecast exactly when ESPN's TV slate will go online has become one of the greatest guessing games in the media industry.

Inside ESPN, executives have long wrestled with the math, studying data and graphs, and calculating how many subscribers they'd need to attract and, at what price, for a streaming version of ESPN to replace the money they now get from pay TV distributors. At a recent investor conference, Bob Eiger called the move inevitable but didn't offer a timeline. The eventual repercussions would be widespread. Many industry executives see ESPN as vital to maintaining the already dwindling number of cable TV subscribers and fear that when Disney starts selling the network's whole lineup online, the change could further accelerate cord cutting and the demise of other channels.

It's not a small decision, Wells Fargo analyst Stephen Cahill said. Disney is constantly weighing the risk of not doing it versus the risk of doing it too early, end quote. So one real quick ask, and then I have some big news for you, the ask is this. Anyone out there have a digital marketing agency they can recommend to me? resume writers has been using one for years to manage our Google ad spend, and let's just say their performance has sort of been disappointing of late. Please don't pitch me yourself or your firm.

Well, you can't, I guess, but I'd rather get a referral. Like if someone can be like, oh, we've been using these folks for years, they're great, I'd give more weight to an endorsement than someone pitching themselves. Anyway, if you know of a firm that does, Google Ads especially, please send them my way. at Techmeme.com. Then, the big news, two amazing bonus episodes coming at you this weekend. First of all, yesterday I sat down for a conversation with the great Ben Smith, founder of Semaphore, nay of BuzzFeed News, to talk about his great new book about the last 20 years

of digital media. As you'll hear, this is my dream book. I've been waiting for someone to do this for years. I could not recommend it more highly. Then, as soon as I hit publish on this episode today, I'm going to run down to Industry City down on the Brooklyn Waterfront to record Alex Cantowitz's podcast along with the great Ranjan Roy, barring some unforeseen snafu. The plan is to do this as a podcast crossover episode, so you'll hear it on his feed,

and I'll drop it on our feed on Sunday. Talk to you on Monday.