Tech Brew Ride Home - Tue. 03/19 – Nvidia Developer Event

Episode Date: March 19, 2024

Nvidia announced… well, a ton of things. Two new AI tools show that video is having an AI moment. An update on how TikTok is faring in the Senate. And an interesting raise from a startup that wants to become a major new platform player in Gaming. Sponsors: FactorMeals.com/ride50 and code ride50 Links: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’ (CNBC) Nvidia launches NIM to make it smoother to deploy AI models into production (TechCrunch) Nvidia enlists humanoid robotics’ biggest names for new AI platform, GR00T (TechCrunch) Nvidia announces Earth-2 digital twin to forecast planet’s climate change (VentureBeat) Google researchers unveil ‘VLOGGER’, an AI that can bring still photos to life (VentureBeat) Stability AI brings a new dimension to video with Stable Video 3D (VentureBeat) DOJ to Push for TikTok Divestiture in Senate Briefings (Bloomberg) Playtron: the startup hoping to Steam Deck-ify the world (The Verge) Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to the Tech meme right home for Tuesday, March 19th, 2024. I'm Brian McCullough today. InVIDIA announced, well, a ton of things. Also, two new AI tools show that video is having an AI moment, an update on how TikTok is faring in the Senate,

and an interesting raise from a startup that wants to become a major new platform in gaming. Here's what you miss today in the world of tech. I know this is sort of a trope of mind to say this, but there was a time when an Nvidia developer event wouldn't necessarily be worth our time. to talk about. But here we are, batten down the hatches for a bunch of Nvidia news. Yesterday, Nvidia announced Blackwell, a new generation of AI chips available later this year, starting with the GB200 super chip, which pairs two B200 GPUs with a Grace CPU. They say this can process a large language model 25 times more cheaply and efficiently than the H-100. Quoting CNBC. Every two years, Nvidia updates its GPU architecture, unlocking a big,

jump in performance. Many of the AI models released over the past year were trained on the company's Hopper architecture used by chips such as the H-100, which was announced in 2022. NVIDIA says Blackwell-based processors like the GB200 offer a huge performance upgrade for AI companies with 20 petaflops in AI performance versus four petaflops for the H-100. The additional processing power will enable AI companies to train bigger and more intricate models, NVIDIA said. The chip includes what NVIDIA calls a transformer engine specifically built to run transformer-based AI, one of the core technologies underpinning ChatGPT. The Blackwell GPU is large and combines two separately manufactured dyes into one chip manufactured by TSM. It will be available as an entire server called the

GB200 NV-Link 2, combining 72 Blackwell GPUs and other NVIDIA parts designed to train AI models. The GB200 pairs two B-200 Blackwell GPUs with one RRGV-B-Won. ARM-based Grace CPU. NVIDIA said Amazon Web Services would build a server cluster with 20,000 GB200 chips. InVedia said that the system can deploy a 27 trillion parameter model. That's much larger than even the biggest model such as GPT4, which reportedly has 1.7 trillion parameters. Many artificial intelligence researchers believe bigger models with more parameters and data could unlock new capabilities. Invita didn't provide a cost for the new GB200 or the systems it's used in, NVIDIA's hopper-based H-100 costs between $25,000 and $40,000 per chip,

with whole systems that cost as much as $200,000, according to analyst's estimates, end quote. NVIDIA also announced NIM, a set of microservices designed to streamline the deployment of custom and pre-trained AI models into production environments. Quoting TechCrunch, NIM takes the software work, NVIDIA has done around inferencing and optimizing models, and makes it easily accessible by combining a given model with an optimized inference engine and then packing this into a container, making that accessible as a microservice. Typically, it would take developers weeks, if not months, to ship similar containers, Nvidia argues, and that is if the company even has any in-house

AI talent. With NIM, NIM, Nvidia clearly aims to create an ecosystem of AI-ready containers that use its hardware as the foundational layer with these curated microservices as the core software layer for companies that want to speed up their AI roadmap. NIM currently includes support for models from Nvidia, A121, Adept, Cohere, Getty Images, and Shutterstock, as well as open models from Google, Hugging Face, Meta, Microsoft, Mistral AI, and Stability AI. NVIDIA is already working with Amazon, Google, and Microsoft to make these NIM microservices available on SageMaker, Kubernetes Engine, and Azure AI, respectively. They'll also be integrated into frameworks like DeepSet, Langchain, and Lama Index.

As for the inference engine, Nvidia will use the Triton interface server, TensorFlow, and TensorFlow RTLM. Some of the Nvidia microservices available through NIM will include RIVA for customizing speech and translation models, co-opt for routing optimizations, and the Earth2 model for weather and climate simulations. The company plans to add additional capabilities over time, including, for example, making the NVIDIA RAGLM operator available as a NIM, which promises to make building generative AI chatbats that can pool in custom data a lot easier, end quote. But wait, there's more. InVIDIA also unveiled Project Groot, a general purpose foundation model for humanoid robots, and Jetson Thor, a new computer design specifically for

humanoid robots. I told you folks are looking at AI unlocking the potential of robotics in a serious way right now, quoting TechCrunch. It's tough to argue with Nvidia CEO Jensen Huang when he notes, quote, building foundation models for general humanoid robots is one of the most exciting problems to solve an AI today. The humanoid form factor is one of the most hotly contested topics in the world of robotics at the moment, raising venture capital by the boatload, while generating massive skepticism along the way. Naturally, Nvidia wants a piece. The chip giant has become arguably the most important hardware company in AI and has more recently been making a compelling case for itself as a driver for robotic innovation through initiatives like Isaac and Jetson.

This week at its annual GTC developer conference, the company is planting its flag in the humanoid race with Project Groot, which may or may not be a nod to Marvel's Talking Space Tree. The chipmaker refers to the new platform as a general-purpose foundation model for humanoid robots. In essence, the company is building an AI platform for the recent spate of entries into the category, including companies like 1X Technologies, Agility Robotics, Apptronic, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitary Robotics, and X-Peng Robotics. That covers nearly every prominent humanoid robot maker at the moment, with a few notable exceptions like Tesla. Agility gets additional FaceTime in the announcement, courtesy of a quote from co-founder and chief robotics officer Jonathan Hurst.

Quote, we are at an inflection point in history with human-centric robots like Digit poised to change labor forever. Modern AI will accelerate development, paving the way for robots like Digit to help people in all aspects of daily life. We're excited to partner with NVIDIA to invest in the computing, simulation tools, machine learning environments, other necessary infrastructure to enable the dream of robots being part of daily life, end quote. Groot will support new hardware from Nvidia as well. Keeping things in the Marvel Cinematic Universe is Jetson Thor, a new computer designed specifically for running simulation workflows, generative AI models, and more for the humanoid form factor. I continue to caution people away from casually tossing out terms like general purpose when describing these machines, but Nvidia's keen

interest is a validation for the category that will almost certainly accelerate development. This week's GTC robotics announcements include two more key programs, Isaac Manipulator and Isaac Perceptor. Manipulation has been a foundation aspect of robotics for decades now, leading the way were the massive industrial robotic arms that have become a fixture of automotive manufacturing. The next generation will be even more dexterous and far more mobile. Naturally, Nvidia wants a piece of the action. Isaac Manipulator offers state-of-the-art dexterity and modular AI capabilities for robotic arms with a robust collection of foundation models and GPU accelerated. libraries, the company writes. It provides up to an 80x speed-up in path planning, and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks,

end quote. InVIDIA already has some big names on board, including Franca Robotics, Picnic Robotics, Ready Robotics, Solomon Universal Robots, and Yaskawa, end quote. And one more thing, as previously mentioned, quoting Venture Beat. Invidia unveiled Earth 2 cloud platform for predicting climate change across the entire planet using simulations by AI supercomputers. In a bid to combat the increasing economic losses due to extreme weather patterns caused by climate change, Nvidia has introduced Earth 2 a revolutionary climate-digital twin cloud platform. Revealed at GTC, Earth 2 offers groundbreaking APIs designed to simulate and visualize weather and climate at an unprecedented scale, paving the way for more accurate forecasts and timely warnings,

end quote. The past few weeks has really been when video generation has risen to the forefront, of the AI moment. Here's another big step. Google researchers have detailed vlogger, an AI model that can generate lifelike videos of people speaking, gesturing, and moving from just a single still photo, quoting Venture Beat. Described in a research paper titled Vlogger, multimodal diffusion for embodied avatar synthesis, the AI model can take a photo of a person and an audio clip as input, and then output a video that matches the audio showing the person speaking the words and making corresponding facial expressions, head movements, and hand gestures. The videos are not perfect with some artifacts,

but represent a significant leap in the ability to animate still images. The researchers, led by Enric Corona at Google Research, leveraged a type of machine learning model called diffusion models to achieve the novel result. Diffusion models have recently shown remarkable performance at generating highly realistic images from text descriptions, but by extending them into the video domain and training on a vast new dataset, the team was able to create an AI system that can bring photos to life in a highly convincing way. A key enabler was the curation of a huge new dataset called Mentor, containing over 80,000 diverse identities and 2,200 hours of video in order of magnitude larger than what was previously available. This allowed vlogger

to learn to generate videos of people with varied ethnicities, ages, clothing, poses, and surroundings without bias, end quote. This could lead to some fascinating applications, including the ability to dub videos into various languages by changing the audio, edit videos to add missing frames smoothly, and generate videos of a person from just one photograph. This innovation paves the way for actors to offer detailed 3D models of themselves for new digital performances. Like, I could create a YouTube video of me speaking these words simply by uploading a photo of myself and this audio. It could also revolutionize the creation of lifelike avatars and virtual reality and gaming, as well as lead to more engaging.

and expressive AI-powered virtual assistance and chatbots. However, vlogger is not without its flaws. The videos it produces as of now are short and set against a static backdrop. The characters do not navigate through three-dimensional spaces, and despite their realistic appearance, their gestures and vocal patterns cannot yet fully mimic those of actual people. More! Stability AI debuted Stable Video 3D,

a generative AI tool built on its stable video diffusion model, letting users create 3D video from a text or image prompt. Quoting Venturebeat. With SV3D, Stability AI is adding new depth to its video generation model with the ability to create and transform multi-view 3D meshes from a single input image. SV3D is now available for commercial use with a Stability AI professional membership, $20 per month for creators and developers with less than $1 million in annual revenue. For non-commercial purposes, users can download the model weights from Hugging Face.

by adapting our stable video diffusion image-to-video diffusion model with the addition of camera path conditioning. Stable Video 3D is able to generate multi-view videos of an object that company wrote in a blog post detailing the new model. Stable Video 3D is a valuable tool for generating 3D assets, especially within the gaming sector. Varroon Jampani, lead researcher at Stability AI, told Venture Beat. Additionally, it enables the production of 360-degree orbital videos, which are useful in e-commerce, providing a more immersive and interactive shopping experience, end quote. Update on the whole TikTok situation sources say senior Department of Justice officials plan to hold closed-door Senate briefings today and tomorrow to advance a TikTok divestiture,

quoting Bloomberg. The department's push, led by Deputy Attorney General Lisa Monaco, is aimed at divestment of the popular video sharing app from its Chinese parent, Bite Dance Limited, as opposed to an outright ban in the U.S., said one of the people who asked not to be identified speaking about confidential matters. Monaco and other national security officials have been working behind the scenes with key lawmakers to pass legislation that would result in TikTok being divested and perhaps bought by U.S. investors or an American entity, the person said. The Justice Department is also developing its first ever regulations to prevent China

and others from buying sensitive personal data about U.S. citizens from commercial brokers as part of a White House executive order issued last month. Monaco said in the interview about the data security regulations that the A. is to close a loophole that allows adversaries to buy certain categories of sensitive U.S. personal and government data, end quote. And allow me to put Playtron on your radar. This startup raised a $10 million seed round to build a Linux-based platform agnostic gaming OS to replace Windows or Steam OS starting with handheld consoles.

Quoting the Verge, what if there were an Android of gaming, an OS you could install on any capable hardware to get a controller-friendly PC experience? Imagine an epic deck that could actually play Fortnite, a 5G deck for cellular carriers, or maybe a PlayStation deck for Sony's growing PC ambitions. If Valve can do this with Steam, we can do this with anything, says Kurt McMaster, CEO of Playtron. It begins with a Linux-based operating system that plays Windows games just like the Steam deck, except this one isn't tied to Steam. Games you've bought from the Epic Game Store are First Class Citizens 2.

Today, in early Alpha form, it doesn't look all that exciting, a light-es. lightweight OS that replaces Windows or Steam OS on your existing handheld gaming PC with fewer features and compatible games than its peers. While it definitely played Death Stranding, Dave the Diver and Hours of Bellatro, just as well as my Steam Deck, not all games that work there necessarily work here yet. But within a year, Playtron believes it'll be competing with Windows as the OS for gaming handhelds. We're in conversation with numerous OEMs and mobile operators to build and deploy PlayToron devices around the world in the 2025 timeframe. McMaster tells me, adding that Aainio plans to ship a native Platron handheld by the end of 2024. Playtron will cost companies like Aionio, a fraction of the price of Windows, he says, around $10 per head, instead of the $80 that OEMs tell him they spend today.

And he says games like Fortnite and Roblox shouldn't have to fear hackers reverse engineering their anti-cheat solutions because its Fedora silver-blue base has an immutable file system. I thought SteamOS was immutable too, but I'll let more knowledgeable Linux users argue that one. Matron isn't just looking at handhelds, by the way. It imagines every laptop, tablet, phone, TV, and car becoming at least a part-time gaming machine. Remember Apple's journey to bring big games to its own arm-based silicon? McMaster says he's hired the developer behind Box 86 to bring Windows games to efficient arm-based silicon, potentially getting us out of the rut where today's portable PCs max out at two to three hours of AAA gaming battery life. If you're wondering how a brand-new startup convinces hardware manufacturers and mobile carriers with what looks like a glorified games launcher and a few.

big ideas, you're not alone. Even Playtron's allies aren't trying to claim this is a sure bet. It's a game of conditional probabilities. Ten different things need to happen, and each of them has a very low probability of success, says Carlos Castanelos, an investor at Samsung Next, who led Pletron's first round of funding. Game publishers, hardware manufacturers, cellular carriers, and the community would all need to be engaged. But Castellanos and former PlayStation CEO Sean Layden, a friend of the project, thinks this team has a chance to win at a time. few others are stepping up to lead. We don't go out into the world awash in handheld gaming platforms, said Layden,

there's room for someone who can make the right proposition, end quote. Finally today, I have a quick update about a portfolio company for you. Stashpad came on the show about a year and a half ago to talk about their lightweight tool aimed at developers looking to organize their thoughts. Now they've added Stashpad Docks, a super lightweight Google Docs competitor. In fact, if you read between the lines here, StashPad is clearly building out a whole super simplified Google. workspace, Google Suite competitor.

I've actually moved to using Stashpad docs as a dead simple way to collaborate on documents myself. No learning curve. No login required. You should check it out. Listen to this quick interview to learn more. Kara Bornstein, thanks for coming on the TechMeme ride home again for your second time to tell us what Stashpad is up to.

Thanks so much for having me. So what you're up to, folks might have seen this on Product Hunt. recently, but you have a full new feature, full new product, stashpad docs. So let's just start right there. What is stashpad docs? Because people, as they're listening to this, can try it out right now as they're listening.

Yeah. So stash pad docs is a minimal Google Docs alternative, and it has markdown support and doesn't require a login. And basically how we got started working on it for anyone who remembers me being on a little while back. We're working on a product stash pad list, which is for personal note taking. And as we were doing discovery calls with folks to understand what they'd want collaboratively, we noticed something really surprising, which is that even in companies where

there officially was confluence or officially was Notion, there were just all these Google Docs all over the place. And Google Docs is lightweight, has a lot of benefits, but it really isn't all that modern of an experience. It doesn't have great markdown support or dark theme. things that developers want. And we found that the, like, authentication handling really gets in the way of just quickly getting in and brainstorming with people. And so Sashpad Docs is our new product. It's that minimal Google Docs alternative.

You can try it right away. You just type stash.new into your browser and would be super excited to hear what you think. One of the things that appeals to me about this, so like I said, you can be listening right now and just go try it out. There's no login, as you say. Like, Chris and I use Notion for, like, preparing for episodes and also doing investment. And so, like, I'll be talking to Chris sometimes and I'll be like, what's this company

all about? He's like, well, it's in Notion. And then I'm like, great, because I know it's going to take me four minutes to find what the hell he wants me to find or whatever. But, like, if I just needed to be like, we're about to do an interview with somebody, let's jam on the questions we're going to ask. we do turn to Google Docs for that, but then that's a whole other thing of like getting,

I can literally be like, Chris, let's go, let's jam on it right now. We can both like the, there's not nine clicks to do something. It's just like, let's go and like let's start collaborating in real time. Exactly, exactly. And that's part of why the no login requirement is so important

because maybe you do have the same company domain, but maybe you don't and you don't want to think about, okay, let's make sure, I have exactly the permissions right because that's just too much fiction. You end up using like some sort of chat instead is what we see. And so instead you just make a link in stashpad docs, share it, and right away you can be in brainstorming. Again, you can just type in right now, dox. Stash.com.

Yeah, or stash.com. Yeah, or stash. New is the shorthand for that. Yep, stash. I'm old, so I like doing it. So again, I'm using the example of like what I would use Docs or Notion for, but this is very developer friendly with Markdown support and stuff like that. So again, if you're working with your co-developer across the country and it's like, no, no,

no, here's the problem. Here, let me just cut and paste this into this and I'll show you. It's like literally that sort of simple stuff. Yep, exactly. And similar to with our first product, Sashpad lists, we've just been obsessed with performance when it comes to this product. So even though it's in the browser, which we think is important for that lightweight collaboration,

we've been really obsessed with making it as fast as possible. So we've had a number of people say, oh, I didn't even think of Google Docs as being slow, but this is just noticeably so much faster. And the reason why is on the back end, we're using an architecture that has sub-50 millisecond, collaborative interactions pretty much no matter where you are in the world. And it also is using a local first architecture. So you're going to have that same like really

snappy feel that you would have if you were just like writing to a plain text file on your computer. And then if I sign up for, you know, like a subscription or whatever, what are the extra bells and whistles I get aside from like you'll priority support you'll pay attention to me that I said it. What else do I get for? signing up for this. So one thing we've seen is that for most people, just doing stash.new, creating a dock and not having additional access control is enough. It's on a nano ID, so it's like virtually unguessable, similar to how Loom works.

But then there are some companies that want to lock things down. So like by default, you have to have the same company email address in order to be able to access the docs. So basically logging in gives you the option to add those additional permissions. It also makes it, I mentioned the local first architecture and anonymous stuff, which basically means that it's tied to your browser. And so if you want to be able to access across different devices, different browsers, then logging in will let you do so.

And that's included in the free plan. And stashpad lists is still available as well. It is. It is. And we have seen an up check and interest with it with the launch of Sashpad Doc. So it's exciting to see both resonating really based on what you're looking for. Well, it is, you know, one of the things that I was sold on when we first started talking was this idea of it being so lightweight and like removing like, you know, it's almost like I think we talked about last time. like sometimes you almost instinctively revert back to like a note file or something like that, something where it's like, you know, I just, all the craft gone, I just need to quickly jam on

something, right? And so it's almost like you guys are continuing to iterate on that same idea for productivity. You're doing the minimum viable Google suite of productivity stuff. Yep, and it really is the same. Like what we had talked about, I think last time, is that the whole need for the notepad is serving your working memory or for the fellow geeks out there. It's like your RAM, like a computer. And so just as folks need that personal working memory,

there's also a need for that shared working memory, shared RAM as you're jamming with another developer on a project. So, yeah, super, super excited to be able to continue with this new product that we have. And since this is your second time on, I wrote it down. You were on in August of 2022, if people want to look up that episode. Just because, you know, we're following along with your progress, how are things at Stashpad? What have you learned over the last year and a half? Yeah.

Struggles you've had to overcome over the last year and a half. How's Stashpad going? Yeah. So it's going well. We've definitely had ups and downs. And so basically something that we had learned with Sashpad List, which is our first product and has that same philosophy, is that the way that we've designed it creates too much of a learning curve for a number of people. So for some people, they love it. They're like keyboard shortcuts.

I'm all in. But for a lot of people, they're like, this is just a little too different from that plain text file. And learning the keyboard shortcuts takes too long. And so through that discovery and also through seeing that this collaborative piece is something that's so core to how people are working. And while it works, it doesn't work in a delightful way, especially not for a technical person. That's kind of what led us to explore sashpad docs. And sashpad docs, although we're still seeing interest in sash pad lists, sashpad docs definitely has had a pickup much faster. Overall, sashpad lists has around 10,000 people who have tried it.

in just the last three days, we've had 10,000 people try stashpad docs and converted a much higher rate. So we're really excited at kind of using the learnings from stashpad lists of like, I personally still love it. And like, I know all the keyboard shortcuts and that's great, but like it is, you have to invest in that if you want to, if you want to benefit from that product. And so with stashpad lists, we're taking just this obsession with simplicity, zero learning curve, just start typing, use markdown, just like you already know. where you can brush up on markdown if you've forgotten some of it. And that's a lot what we're going for, with Sashpad docs.

I think that's a good learning for anyone developing a product, which, a product where you start from the first principles of, I'm doing this because it's something I myself want to use. And the core, the kernel of that idea is often right. But not every user is like you. And so, like, even if the kernel of your original inspiration works, paying attention to how your actual users do it differently than you yourself do. Like, it's not like you have to abandon the kernel of your idea, but if you pay attention to how users are using it, that original kernel can go further. And I'm going to come back to the, you know, Chris loves Notion and all of the 10,000 features that I look at it and my eyes glaze over.

I don't know where to find anything versus, hey, I can just do search in my email and get the answer to the question that I want it, right? So it's like, that's a good learning in terms of not every user is like you, but that doesn't mean that you have to abandon the idea that got you the product in the first place. Definitely. Definitely. And a lot of it was also like, like there are there are folks that love the Sashadless has like this DM style interface. And so there are people that love it, but there are a lot of people that find it to be too big of a change. But one thing that we did see just like really consistently and continues to see and the feedback is like, I love that I could just write and mark down. And that makes me really happy. And so just kind of like taking a pause back and listening to that, we were able to go in this direction of sashpad at docs. And yeah, one other thing I can share that's really fun and kind of a learning for me as a founder is.

we were kind of ending last year and seeing that Sashpad lists was not growing all that much. So it was like growing but slowly and not what you need to see for a startup. And so we had been doing all of these discovery interviews thinking we wanted to add multiplayer to lists and kind of like discussing those as a team. And so we all took off for holiday like Christmas to New Year's and we come back in the office. and two of the four of us had actually created a prototype for what became SashpedD docs independently. And so kind of just sitting with the customer feedback,

what we know how to do, the libraries we know that are available to us, that we know how to build on. That kind of was just where the customer feedback was screaming at us to go. And so it wasn't me that made that prototype. It was my teammate Theo and my teammate Nick independently. and that once we started showing it to people,

we just saw the conversion rate was much better. And we're excited to continue with it. Parallel thinking, great minds think alike. One quick note, I've been slandering notion all this time, but it's air table that Chris uses that I can't, I look at, I'm looking at it right now and I don't know where to go. Simplicity is why I bought into Stashpad and your idea from the beginning. So again, docks.stashpad.com or stash.new. Try it right now. Is there anything else that the listeners, do you have any asked if people are excited about this aside from trying the product, if they want to get involved, anything that you want listeners to know or contribute or anything like that?

So the biggest thing is to try the product. And you can either join our Discord, which you'll see from our website, or send me an email, Kara, at stashpad.com. to share any feedback. And then one thing we have going on right now that we're really excited about is we actually already have an API for Sashpad Docs, and we're starting on building those first integrations. And so if you have ideas of how integrations can work, if you want to even build your own integration,

I'd love to talk with you about that. And yeah, that would be a lot of fun and helpful for me. Yeah, if you try out Dashpad Docs, and it's a great tool, and you'd love to use it with your tool. get in touch with Kara and make it happen. Thanks for coming back on again, Kara, and I can't wait for the next stashpad suite products to come out sometime. Thanks for coming on.

Thanks so much, Brian.