Tech Brew Ride Home - Mon. 03/13 – Crisis Averted?

Episode Date: March 13, 2023

Unless it affected you, you might have missed a huge tech story this weekend. But now that depositors to Silicon Valley Bank are being made whole, is the whole crisis averted? Tim Cook is pressing ahead with the Apple headset over the objections of Apple’s design team. And the varied and complex dilemmas of this new AI reality in two, somewhat oppositional segments. Sponsors: RelationshipHero.com/techmeme ZocDoc.com/techmeme Links: SVB’s tech failings were a problem long before the bank run that led to its demise, critics say (CNBC) SVB provided for tech when everyone else ignored us (FT) Regulators close crypto-focused Signature Bank, citing systemic risk (CNBC) Tim Cook bets on Apple’s mixed-reality headset to secure his legacy (FT) Microsoft Strung Together Tens of Thousands of Chips in a Pricey Supercomputer for OpenAI (Bloomberg) Large language models are having their Stable Diffusion moment (Simon Willison's Blog) Self Radicalization with open sourced AI-Systems (Good Internet) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the TechMeme right home for Monday, March 13th, 2023. I'm Brian McCullough today. Unless it affected you, you might have missed a huge tech story this weekend. But now that depositors to Silicon Valley Bank are being made whole, is the whole crisis averted? Also, Tim Cook is pressing ahead with Apple's headset over the objections of Apple's design team and the varied and complex dilemmas of this new AI reality in two somewhat oppositional segments. Here's what you missed today in the world of tech. Yeah, unless you were personally involved in all this over the weekend, you missed one of the biggest tech stories in a long time. It all happened over the weekend from roughly Friday afternoon to Sunday afternoon.

Silicon Valley Bank blew up. Everybody freaked out. And then late yesterday, after much gnashing of teeth and renting of garments online, the U.S. Treasury, Federal Reserve, and the FDIC stepped in to say that all Silicon Valley Bank depositors would be made fully whole. and their money would be available today. So far today, I haven't seen anything from anyone to say that this isn't proving to be the case. So it sort of seems what could have been a huge crisis has been averted. No more on that in a second. Let me give you a direct example of what people were dealing with this weekend. One of our ride home fund portfolio companies was able to get its entire $3 million treasury out of Silicon Valley Bank and emailed over the weekend to say they were safe. Another email to tell me that their 13 million pound treasury was all locked up in the UK arm of Silicon Valley Bank,

and they had access to only about 200,000 pounds, which was problematic because they were about to spend $2 million on an acquisition, which might not happen even now. They're one of these roll-up companies. I haven't told you about them on the show yet. But here's another example that is maybe more tangible. Another of our portfolio companies had $600,000 in the bank. They're a team of six. They were just starting to, to raise their A round, figured they had enough runway to hopefully get that done. But they emailed on Saturday to say all of that money was tied up in Silicon Valley Bank, and they had payroll coming on Wednesday. They didn't know what to do. Like, should they lay everyone off? Founders stopped paying themselves. We knew that they would have access to at least $250,000 of that FDIC insured money, but until yesterday afternoon, we didn't know if any more was coming or when. They could maybe make it

two months on $250,000, even cutting every cost under the sun. But what if that was all they would ever see, or almost worse? What if they could only get, I don't know, 70 cents on the dollar of their $600,000? And that would take weeks or months to see even that. That's what was playing out across the entire startup landscape this weekend. And then the Fed stepped in and said, no worries. Every depositor will be made 100% whole. You'll have access to all of your money today. Again, until I hear otherwise, that seems to have happened. and seems to have made this crisis just poof, go away. By the way, HSBC this morning announced plans to acquire Silicon Valley Bank UK for one pound, citing strategic sense for our business. As of March 10th,

Silicon Valley Bank UK had 5.5 billion pounds in loans and 6.7 billion pounds in deposits. So it sounds like HSBC is basically acquiring the British arm and thereby backstopping all deposits there. Hopefully, that's true for that one startup. So what can I tell you? stress, huge story, now it's over. It kind of remains to be seen, whisper it softly, but all banking stocks are getting hit this morning, especially other regional bank stocks. And if that becomes a problem, I don't know, the problems in tech might pale in comparison to the problems all across the economy, which gives me the chance to say, I do regret the segment I did about Silicon Valley Bank being a poorly run regional bank on Friday's episode in my defense. It was what I was

feeling at the time. But over the weekend, when the sky was falling, it did feel a bit churlish on my part, and I considered even taking the episode down. But I felt like that would be dishonest of me, so I kept it up, though I would note others have made the same point I did subsequently, quoting CNBC. Longtime clients and others with intimate knowledge of how Silicon Valley Bank operated say the bank did itself no favors between the bank's refusal to upgrade its technology to meet the demands of modern-day businesses and its treatment of many startup customers, SVB's problems extended beyond its risk profile and a challenging economy. An ex-SvB manager who worked on risk initiatives and asked not to be identified,

said the bank remained technologically stagnant even as it was a haven for startups that had an eye for cutting-edge software and products. As she described it, quote, the back end of the bank is all bubblegum and wires, end quote. But to give both sides of this, here is Mike Moritz in the, financial times talking about what Silicon Valley Bank meant to the startup ecosystem all these decades, quote, before Silicon Valley Bank sprang to life, it was difficult, if not impossible, for a startup to secure a relationship with a large established bank. Small West Coast technology companies

were incomprehensible or insignificant to the large East Coast banks whose customers included international airlines, heavy industry, and nationwide retailers. SVB was like the cherished local market, where people behind the counters know the names of their customers have a ready smile but still charge the going price when they sell a cut of meat. When a small tech business ran into difficulties, we knew we would get a sympathetic hearing, but also that we would have to pay the piper. When a community loses its bank, whether in a Tuscan Hilltown or on the coast of the Pacific, it's like having a death in the family. Once again, the fates of thousands of small technology companies and the vitality of the startup economy will be back in the hands of strangers, and the U.S.

will be all the poorer, end quote. One more thing that needs to be said about all this, and that is, this was a bank run. Silicon Valley Bank was shived by the very investors and startups, as Moritz just pointed out, that it did so much to nurture for so many years. I've heard dozens of stories of folks, founders, even individuals with individual money at the bank, all racing on Thursday night, Friday afternoon to get their money out. Had everyone not done so, the consensus seems to be Silicon Valley Bank could have survived. But such is the nature, the prisoner's dilemma of a bank run. That one portfolio company that told me they got their money out on Thursday night, are they morally superior or inferior to those other startups that did not? At the very least,

they saved themselves and their employees a weekend of stress that everyone else experienced. And here's how Ben Thompson summed it up this morning, quote, When the stakes are so high in one perceived opportunity space increasingly narrowed, every decision becomes a prisoner's dilemma. And in retrospect, what happened to Silicon Valley Bank becomes inevitable. Moreover, it probably won't be the only bad outcome of this new environment. It's hard to understand the value of trust until it's gone, and the full accounting of what has been lost will take years. The irony in this loss of trust is that the ultimate driver is tech itself. What made the Silicon Valley Bank run unique was the ease with which its customers

could execute withdrawals and the speed with which news of Silicon Valley Bank's impending demise spread. Just to put the scale of this collapse in context, a total of $7 billion in depositors' assets was lost in the Great Depression. Seven billion then is $161 billion today. Silicon Valley Bank, meanwhile, processed $42 billion in withdrawals in 24 hours. It was the speed, fueled by zero distribution costs for both rumors and withdrawals that was so destabilizing for an entity predicated on arbitraging time. That destabilization and resultant loss of trust, meanwhile, is everywhere around us, from our politics to business to every aspect of media. This increased uncertainty and destabilization has and will continue to drive demands for more government intervention,

and like this weekend, it may not even be wrong. More government, though, means replacing trust with more rules, regulations, and restrictions, which will have a long-term effect on innovation. This, perhaps, is the inevitable outcome of tech having set disruption as its objective function. The ultimate casualty may be the Silicon Valley that once was, not just its bank, end quote. I do also need to note that signature bank has also been shut down, as U.S. regulators cited a systemic risk exception, just as they did when they announced their SVB decision yesterday, announcing that all of signatures' depositors will be made whole. This is notable, because Signature Bank is another huge bank that banked crypto. Quote,

Signature is one of the main banks to the cryptocurrency industry, the biggest one next to Silvergate, which announced its impending liquidation last week. It had a market value of $4.4 billion as a Friday after a 40% sell-off this year, according to fax set. As of December 31st, Signature had $110.4 billion in total assets and $88.6 billion in total deposits, according to securities filings. To stem the damage and stave off a bigger crisis, the Fed and Treasury created an emergency program to backstop all deposits at both Signature Bank and Silicon Valley Bank using the Fed's emergency lending authority. The FDIC's deposit insurance fund will be used to cover depositors, many of whom were uninsured due to the $250,000 cap on guaranteed deposits. While depositors will

have access to their money, equity and bondholders at both banks are being wiped out, a senior treasury official said, end quote. So we know that Apple is about to release a mixed reality headset. This piece in the Financial Times doesn't have much new to say about that other than this Interesting detail. According to them, Tim Cook has pressed ahead with the mixed reality project over the objections of Apple's designers who wanted to wait until the tech caught up with something that would make this new product more mainstream palatable. Quote, Apple's famed industrial design team had cautioned patience, wanting to delay until a more lightweight version of AR glasses became technically feasible. Most in the tech industry expect that to take several more years. In deciding,

to press ahead with a debut this year, Cook has cited with Operations Chief Jeff Williams, according to two people familiar with Apple's decision-making, and overruled the early objections from Apple's designers to wait for the tech to catch up with their vision. Just a few years ago, going against the wishes of Apple's all-powerful design team would have been unthinkable. But since the departure of its longtime leader, Johnny Ive in 2019, Apple's structure has been reshuffled with design now reporting to Williams, end quote. Two stories now about all this new AI stuff, but specifically, as I think I mentioned previously, some things that everybody needs to keep their eye on. And the first one is this. Can this latest generative AI stuff get cheaper

and more efficient? This matters because if it continues to be so expensive on the compute, on the energy, on the cost side, then that would mean a scenario where any new innovation around this generative AI stuff will be tied to the major cloud and computing platforms just to survive, just to be born. Any startups will have to build on top of the incumbents. The big will get bigger. In other words, any AI revolution might only serve to entrench the already big players. Here's an example of what I mean. After Microsoft invested $1 billion in open AI in 2019, the company scrambled to string together tens of thousands of Nvidia A100 GPUs, costing them several hundred million dollars just to do so. Quoting Bloomberg.

When Microsoft invested $1 billion in OpenAI in 2019, it agreed to build a massive cutting-edge supercomputer for the artificial intelligence research startup. The only problem, Microsoft didn't have anything like what OpenAI needed and wasn't totally sure it could build something that big in its Azure cloud service without it breaking. Open AI was trying to train an increasingly large set of artificial intelligence programs called models, which were ingesting greater volumes of data and learning more and more parameters. the variables the AI system has sussed out through training and retraining. That meant OpenAI needed access to powerful cloud computing services for long periods of time. To meet that challenge, Microsoft had to find ways to string together tens of thousands of Nvidia's A100 graphics chips, the workhorse for training AI models, and change how it positions

servers on racks to prevent power outages. Scott Guthrie, the Microsoft Executive Vice President who oversees Cloud and AI, wouldn't give a specific cost for the project, but said, quote, it's probably larger than several hundred million dollars, end quote. Microsoft uses that same set of resources it built for OpenAI to train and run its own large artificial intelligence models, including the new Bing search bot introduced last month. It also sells the system to other customers. The software giant is already at work on the next generation of the AI supercomputer,

part of an expanded deal with OpenAI in which Microsoft added $10 billion to its initial investment. We didn't build them a custom thing. It started off as a custom thing, but we always built it. in a way to generalize it so that anyone that wants to train a large language model can leverage the same improvements, said Guthrie in an interview. That's really helped us become a better cloud for AI broadly, end quote. Training a massive AI model requires a large pool of connected graphics processing units in one place, like the AI supercomputer Microsoft assembled. Once a model is in use, answering all the queries users pose called inference, requires a slightly different setup.

Microsoft also deploys graphics chips for inference, but those, those, processors, hundreds of thousands of them, are geographically dispersed throughout the company's more than 60 regions of data centers. Now the company is adding the latest Nvidia graphics chip for AI workloads, the H-100, and the newest version of Nvidia's Infiniband networking technology, to share data even faster, Microsoft said Monday in a blog post. Because all these machines fire up at once, Microsoft had to think about where they were placed and where the power supplies were located. Otherwise, you end up with the data center version of what happens when you turn on a microwave, toaster, and vacuum cleaner at the same time in the kitchen, Guthrie said.

The company also had to make sure it could cool off all those machines and chips and uses evaporation, outside air and cooler climates and high-tech swamp coolers in hot ones, said Alistair Spears, director of Azure Global Infrastructure. Microsoft is going to keep working on customized server and chip designs and ways to optimize its supply chain in order to ring any speed gains, efficiency, and cost savings it can. Guthrie said, end quote. Now, obviously what I just described is not something some new AI startup is capable of doing all on its own. What if there is an alternative possibility? What if AI models can somehow rapidly become commoditized? What if, just like everyone has had access to powerful

semiconductor technologies for the last 50 years, getting cheaper and more powerful every year, Moore's Law and all that, blah, blah, blah, which is what has made the whole tech revolution of the last 50 years even possible. What if just like that, we saw something similar in terms of commoditization happened to this new AI tech? Well, finally today, toward that vision, word that a developer has gotten meta's 13B Lama model, considered to be competitive with GPT3, working on his laptop, showing local language models are feasible on consumer hardware. He was only able to do this because Lama leaked on the dark web recently, which is a whole other concern in Kettle of Fish, and also is the fact that he did this even good?

Quoting good internet. Simon Willis wrote a widely shared post about how he ran the leaked version of Lama 7B and 13B on a 64-gabyte M2 MacBook Pro with llama.cpp, following up with how large language models are having their stable diffusion moment. At the same time,

together.xyZ released OpenChat kit, which, quote, provides a powerful open source base to create both specialized and general-purpose chatbots for various applications. These are hardly the first open-source large-language models, with Bloom being the most prominent up to the point of being called the most important AI model of the decade, by Alberto Romero. However, it's not hard to see where this is going. At the end of this year, you will be able to run good chatbots, fine-tuned on whatever tickles your fancy on a laptop,

and their output will match that of models like GPT 3.5, which is the basis for chat GPT, end quote. And here's Willis himself, quote, This technology is clearly too important to be entirely controlled by a small group of companies. There have been dozens of open large language models released over the past few years, but none of them have quite hit the sweet spot for me in terms of the following. Easy to run on my own hardware, large enough to be useful, ideally equivalent in capabilities to GPT3, and open source enough that they can be tinkered with.

This all changed yesterday, thanks to the combination of Facebook's Lama model and Lama.cpp by Georgi Gerganoff. They're in the wild now. You may not be legally able to build a commercial product on them, but the genie is out of the bottle. That furious typing sound you hear is thousands of hackers around the world starting to dig in and figure out what life is like when you can run a GBT-3 class model on your own hardware. I'm not worried about the science fiction scenarios here. The language model running on my laptop is not an AGI that's going to break free and take over the world. But there are a ton of very real ways in which this technology can be used for harm. Just a few. generating spam, automated romance scams, trolling and hate speech, fake news and disinformation, automated radicalization, I worry about this one a lot, not to mention that this technology makes things up exactly as easily as it parrots factual information and provides no way to tell the difference. Prior to this moment, a thin layer of defense existed in terms of companies like OpenAI having a limited ability to control how people interacted with those models. Now that we can run these on our own hardware, even those controls are gone, end quote. Both pieces go on to explore in great detail all of the

potential bad actions that could come with these models now being out in the wild. I present all of this to you without taking a position on it for now. On the one hand, this new technology has the potential to kickstart a Cambrian explosion of new tools, as I said recently, new innovation and new startups, but it might not get a chance if it's tied to the incumbent gatekeepers. And then, on the other other hand, maybe having stuff this powerful at its earliest stages where we don't know all the consequences of it, maybe having it in the hands of gatekeepers isn't the worst thing in the world? Welcome, everybody, to the wild new world of AI. N-F-Y-T, T-T-Y-Y-T-Y, which is short for nothing for you today. Talk to you tomorrow.