Algorithms + Data Structures = Programs - Episode 122: ChatGPT, GTC 2023 & Stickers on our Foreheads

Episode Date: March 24, 2023

In this episode, Conor and Bryce chat about ChapGPT, the NVIDIA GTC 2023 conference and intelligence augmentation.Link to Episode 122 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)TwitterADSP: The PodcastConor HoekstraBryce Adelstein LelbachShow NotesDate Recorded: 2023-03-21Date Released: 2023-03-24RUN FOR THE FUN OF IT! Running PodcastChatGPTDeep Blue vs KasparovGTC 2023 ConferenceConor & Bryce Chat About C++ Algorithms + Combinators (YouTube Stream)The NVIDIA AI Podcast - Glean Founders Talk AI-Powered Enterprise Search on NVIDIA Podcast - Ep. 190ADSP Episode 97: C++ vs Carbon vs Circle vs CppFront with Sean BaxterClang-TidyGPTDuckOxide & Friends PodcastNVIDIA GTC 2023 - C++ Standard Parallelism - Bryce LelbachNVIDIA GTC 2023 - Defining the Quantum-Accelerated Supercomputer - Timothy CostaAPL Seeds 2023 Conferencecode_report GTC 2023 Trip ReportIntro Song InfoMiss You by Sarah Jansen https://soundcloud.com/sarahjansenmusicCreative Commons — Attribution 3.0 Unported — CC BY 3.0Free Download / Stream: http://bit.ly/l-miss-youMusic promoted by Audio Library https://youtu.be/iYYxnasvfx8

All of that stuff could be like a couple stickers on my forehead and a chip in my wrist. Like, let's go. Come on. I'm so tired. Like, we're going to, by the time we're 60, we're going to look back at the days when you would leave the house and you'd have to bring all this stuff, these headphones and all this stuff. And it's just going to be, like, it's going to be in us or stickers on our foreheads. Welcome to ADSP The Podcast, episode 122, recorded on March 21st, 2023. My name is Connor, and today with my co-host Bryce, we chat about ChatGPT, the NVIDIA GTC Conference, and intelligence augmentation. We're back. We are back. Are we hopping straight into things?

I mean, I got a thing. If you got a thing, we can do your thing. I don't really have any things, but I'll just start off by saying it has been... What do you mean you don't have a thing? I mean, I got tons of stuff, obviously. Some nerve you have. I just meant to to say you said you had a topic to talk about so i did not mentally prepare anything that being said we recorded with zach back on february 16th ages ago yeah that was five episodes the most we've split up a single recording i i got to tell you, I love Zach. Honestly, I was thinking that, I mean, usually we do our Christmas holiday special episode where we just chat. And it's usually just you and me. I feel like Zach would fit in great. Even though it's like, you know, the annual tradition is it's just you and me.

But Zach was just fantastic. I love how hard of a time he gives you. Yeah. From a place of love. But he just had so many zingers. And I just, the cold opens of those. You should just go listen to the first 30 seconds of each of the last five episodes. Because I think at least three or four of the five are him like ripping into you about the, what

are they, the authority filling curve and, you know, Bryce's fancy. We know. Anyways, it was, I, it brings me a lot of joy to have, have someone like Zach, you know, try and bring you down to earth, which of course he failed. He magnificently failed in doing that. But I mean, it's pretty, it's pretty hard to bring me down. It is amusing at least. I'm pretty much helium.

I just rise up. But yeah, we recorded that on February 16th. And then the last time we recorded before that was February 1st. So today is March 21st, which means that we actually, mano a mano, haven't chat in over a month and a half which is probably pushing it in terms of I mean I guess we've had multiple I've been like off in the wilderness you know like I've

been not the literal wilderness although I do that sometimes too but I've just been off in the off exploring some things and and messing around with some new uh some new projects and so we haven't really we haven't even talked really at work yeah no we haven't uh we haven't really

caught up i mean the most we did was although it's funny the person who i've talked with the most at nvidia the past few weeks has actually been your boss. Is it Michael? Yeah. Yeah, that's true. I guess I did see that email thread about sort of the index. Well, that's like public stuff that's going to end up in a public API, right? Well, I mean, it's...

Or is that your topic for today? I wrote a bunch of code and I just put it on GitHub and that's what we're going to talk about today, so... Okay, okay. Well, before we hop into that, is there any update i mean we've been chatting so i mean i think the listener knows this by now if you're a regular listener bryce and i are definitely going to be in europe together in june we've got a road trip i guess we you probably haven't booked that car but we've got a road trip from like roughly the 18th i may have reserved the car have you but yeah i mean it's i

you know we we have the flights figured out. I think the only thing now is we just got to figure out the hotels and then what we're going to do. Yeah, so we'll be doing a little road trip. And because of my other third podcast, which I actually did announce on this podcast back in January, it has launched my running podcast. It's called Run for the Fun of It.

The name came from ChatGPT, of course, because whenever it comes to creative things, I think that's actually what ChatGPT is best at. I use ChatGPT to write a heartfelt message to my girlfriend yesterday. But then I had so much fun doing it that afterwards I told her, because she was traveling yesterday and she was having a rough time I had so much fun doing it that afterwards I told her, Oh, did you get them?

Because she, she was traveling yesterday and she was having a rough time because she, she forgot her credit cards at home. And so she was really stressed. And so I, I had a, I had DoorDash delivered to her from a restaurant that I knew she liked near where she's staying for work. And I had like a little message sent there, but it was like 120 character limit.

And I'm like, you know, I could just come up with something on my own or I could use artificial intelligence. There's literally I'll have to find the Instagram reel, but there's an Instagram reel of a comedian talking about writing a poem for this girl he liked, and he used ChatGPT to do it. And then he admitted afterwards that he had used ChatGPT to do it, and then she was offended. And so then she, whatever, they left part of ways after their date. And then the next day, he sent a heartfelt message apologizing. And then, you know, they ended up dating. And the punchline of the joke is that he got chat GPT to write that heartfelt message, which like, I mean, it's like the thought that counts, right? Ramona wasn't amused, but she's never really amused by me. It's the thought that counts. I think if you're taking the time to get a, I wouldn't call

it sophisticated, but like sophisticated enough AI to assist you with the, you know, creative writing that you may not be the best at, that's totally okay in my books. Well, but the funny thing is I must've spent 40 minutes working with chat GPT to write the message. That's part of the process. Just because like i was having fun with yep prompt engineering and so it probably took longer than it would have for me to write something on my own and like i'm i'm actually a pretty good communicator so i think it probably would have been better too i don't know about that but i was just like i'm a man on a mission i feel like this is the hot thing. I'm going to use it.

I feel like it's kind of like how when, was it Deep Blue beat Kasparov? And then they were like, oh, no, you know, computers are better at chess. But then immediately after that, they had a contest where you could have computer-assisted humans. Yeah, computer-assisted humans. And that was like the best combination. So a human with a computer was still better. Yeah. Computer assisted humans. And that was like the best combination. Like, so a human with a computer was still better. Yeah. So, so, so actually during our, our GTC connect with the expert session, um, typically when we do these connectors, the expert sessions at GTC and, and the C plus plus team at NVIDIA does one every year,

we tend to get a bunch of questions that are not strictly on topic. And one of the ones we got yesterday was about, I thought it was going to be off topic first because it was about chat GPT, but it was actually very interesting. It was about like, hey, can you use chat GPT to auto-paralyze code? And usually if it's a question that I think is going to be off topic, I'm like a little reluctant to go and have us answer it live. But then I actually read it and I was like, ooh, this is actually, this opens up like an interesting topic of chat GPT-assisted or large language model-assisted programming, which is something I've been

thinking about a lot recently for a variety of reasons. One, I am lazy. I really just do not want to spend hours and hours cranking out code myself. That just does not, it's not a good use of my time but two i have a bunch of uh uh ergonomic uh issues that that limit the number of hours i can i can actually code a bunch of like repetitive stress injury things from you got rsi you know when i was what you have rsi yeah yeah i didn't know that i mean it's pretty common um but uh you know. But that's one of the reasons why I don't do it. You consider APL less characters.

What? You consider APL less characters. Yeah. You know, the problem is, though, that you've got to type all those fancy characters. And you've got to use a bunch of key modifiers and layers to do that. And that's actually supposed to make it worse. But anyways, long story short is, I would like to write more code,

but I would not like to write more code, you know? I do know. So I was thinking the other day, until Elon perfects that little brain chip that we're all going to get implanted. Whoa, whoa, whoa, whoa, whoa. Pause, pause, pause. You're actually going to put something that that crazy dude creates in your head? Well, not the one that he makes, but see what I figure.

He's really big on dogfooding, right? I was reading this article the other day about that that elon you know at tesla a lot of the testing got done by like the executives including elon like a lot of the testing for autopilot would be like elon would go and like drive his tesla with autopilot and then like he would send a bunch of bug reports and then then like that was their process like elon would go and drive the car and then like tell the engineers like go fix these things. Instead of, you know, having a more principled process. But, you know, I got a figure that Elon's going to be one of the first people to install this chip.

And, you know, he's just going gonna be brain dead after that and so then and then once once elon is in a cyborg coma um you know maybe somebody sane and reasonable will uh will will take over the company and then they'll make like a safe uh brain chip and then i'll get that one um you know i'll get to generation two after the generation that elon and all the elon stands um install and uh turns them into zombies wait we were talking about rsi and coding less yeah yeah so my point my point was you know until that until i can get that that elon brain chip what that will allow me to just directly interface with the computer and just tell it like, you know, just directly output the code

from my mind to the computer. Until that happens, I've been searching for other ways that I can output code without, you know, doing any work because we've established I'm lazy. And obviously, these large language models, things like ChatGPT and GitHub Copilot, you know, offer opportunities for that. pretty skeptical of github co-pilot when it first came out um and i think that's just my the

curmudgeon-y old man inside of me that is just inherently skeptical at any new thing like listeners this podcast probably know like i do not have a very uh advanced or modern uh development uh setup like i i i open up a shell and like i do all my stuff in v yeah there was on that sometimes on that live stream that you crashed a couple weeks ago there was a number of comments being like why does this guy not have syntax highlighting turned on yeah yeah and i don't believe in syntax highlighting. And, you know, it's just like for me, like I'm just happy with like just a Linux shell. And it's like, you know, half the time it's like I don't even want like some fancy builds. It's like I just want like ChatGPT are so promising and have such potential to transform how we operate that I'm actually interested in improving my development toolbox.

And so I've been thinking a bit about what would I actually want? I think it's pretty clear that the correct interface for pair programming with an AI is not logging into chat GPT and asking it to do things and then copying, pasting output from it into another window and then trying it and then, you know, giving it

feedback. Like that's a lot of copying and pasting. I don't want to do that. I think I'd want something more like the GitHub Copilot, where it's integrated into your IDE, and as you start typing out stuff, it suggests auto-completions. So something that's built into your IDE and that can get feedback from can get feedback from your IDE was like the first thing I was thinking about. And then I was thinking a bit about the problem of, well, what if you've got like a really big code base? You know, like, like what if I, what if what I want to tell the AI to do is like, hey, AI, I have this 1 million line, like, internal proprietary code base that's not part of your training set. Like, and I want you to write me a PR for, that does this thing in this code base. And the problem there is that even GPT-4 has a fairly limited input size.

Like you could maybe feed in like, you know, let's say a hundred pages of text or something, but you probably couldn't feed in a million line code base, right? To it. And so asking it to do, to make some change across a large code base, not clear to me how exactly we'd go about doing that yet. But once I started thinking about that,

it started occurring to me that, you know, what I really just want is I want to be able to go onto GitHub and file an issue against my project that describes the change that I want. And then the AI opens up a PR that, you know, implements that issue. And then I can give it feedback in the PR saying like, okay, well, this is broken here. And also importantly that because it's opening up a PR, it will get feedback from your CI results and whatnot,

so then it can go in. You don't even have to tell it, hey, this code that you wrote for me, it doesn't actually compile because it'll figure it out on its own when it sends the PR in. So that's more or less the interface that I settled up on wanting. Then it doesn't matter what ID. You don't need to use a particular ide or anything it's just you just i just type up a high level description of what i want in an issue and then the ai goes out and does it so uh somebody at microsoft or github or open ai you know get on that get on that, get on that. I mean, I feel like it's incredibly doable for a certain category of issue.

Like the kind of thing, like imagine adding W error, like W all to your CMake file. And then like you have a bunch of errors. And so you just post the output of like, you know, add this command to CMake and then go fix all this stuff. I feel like that's super doable.

But there's like a larger category of things that is going to be – I'm not saying it's not possible, but I think it's going to be a long time before, you know. Well, I mean, obviously, we're going to have a future episode of this podcast where we live code with ChatGPT. But I think the potentially one route to get to what I want is that I suspect we're going to see large companies in the future start building models that are trained on their internal code bases. Oh, yeah, for sure. There's already a couple startups that are specifically, whether it's code bases, I know that there's one that's like SharePoint trained or something. I like literally, I'm, I'm pretty sure I'll find it and put it in the show notes. There was an episode of the NVIDIA AI podcast, which is hosted by Noah Kravitz. I might've made,

I may have made that last name up. Um, But I think like two episodes ago or three episodes ago, they had an episode on this startup that's doing exactly that, like internally trained LLMs that have the goal of providing a search engine that's trained on a cornucopia of proprietary internal stuff. But yeah, I, I actually, I'm pretty interested in, in seeing where, where this, where this space goes of AI assisted development. I think it's, I think it's very promising. Yeah. I mean, I've already seen some cool things with it,

but the cool things that I've seen, they've been impressive in the results they've produced, but not in the interface that one has to them. And a lot of them have been things like, have chat GPT write a chrome extension form for you that does something and and that's pretty easy because writing a chrome consent like you don't need any knowledge of a particular code base for that it's just a knowledge of a very well known publicly documented series of apis it's a little bit harder if if you if you want it to be like hey like i have this existing

code base and i want you to help me like like iterate on it or even just if it's like hey i want to like start a new code base from scratch that you were you were going to help me build yeah well that's when we had sean baxter on that was the thing that i think is just like a game changer like i have three different c++ 20 23 personal project repos or even one of them's for work and like a to-do item on every single one of those is to add clang tidy because it's just a non-trivial thing especially if you have cuda code and not just c++ code and if i could just ask some like i think that i've talked about this on this podcast before is that it's called gpt duck or duck gpt which is like g like GitHub integrated with chat GPT.

And if I can just point it at a repo and say, please go set up clang tidy with like the baseline Google standards and modify the CMake file so that it automatically works on like all headers and these C++ and CUDA files. And it just went and did that. That would be phenomenal. Or like I like spent like six hours trying to set up like a particular thing in CMake last week where I was like I wanted it to like build a bunch of object files with flags that would capture optimization remarks in the compilers.

And it would have been so like I'm sure that chat gpt could have figured out how to do that real quick and that's the thing is these are the kind of tasks that it's literally like i don't want to say plumbing work because i feel like that's like denigrating like a whole you know category of folks's jobs but like we are not build engineers or like i do at least for myself i do not consider myself a build engineer in the slightest well i mean i feel like the percentage of time that i end up spending doing build thing build system related things i mean that's the whole point though is like imagine if you could just say say a question and say this is what i want yeah and that's the you know the i think the more um soft like creative software engineering stuff is like algorithm design or work on thruster cub

or whatever it is you're doing it's not the the plumbing work or the infrastructure work to set up some unit test or get some ci process working on you know a matrix of configurations that stuff is like you know it's not the stuff that i look forward to doing and if i could if i could automate the hell out of that stuff it's on oxide and friends they had an episode too where they were talking about like imagine it's not even like you think oh we're replacing someone's job or whatever it's like think about the apis or code bases that don't really have doc documentation and you just like put chat gpt on it and said hey give me a like a read the doc style uh doxygen generated thing and it just went and did it like you know how how awesome that would be and like it would do more than just like generating

like you could say and do your best to in english document what what each of these apis are doing based on you know your assessment of the semantics of the function like amazing imagine how amazing that would be like it would probably do better than what a human would do that didn't know anything about the code base and that was trying to do it by hand. Like if you got a co-op to do it or something like that. Yeah. It's exciting. I think it is, yeah.

All right. So let's talk about my thing. Wait. We're at the 22-minute mark. Let's talk about a couple of random more things. This will be episode 122 and then episode 123. Actually, give the listener a little taste, a little preview before we jump into episode 123 of what we're going to be talking about. Oh, we are going to be talking about C++ ranges and how to vectorize them. But more generally, we're going to be talking about

optimizing loops in these sort of array language type paradigms for composing different algorithmic primitives. So, yeah, I'm going to go with that. Full stop. All right. Well, the listener has to wait a week in order to hear that topic. That's the way things work. I guess in the last few minutes here, I mean, you referenced a connect with experts with some of the library folks at NVIDIA. That is implicitly a reference to the GTC 2023 virtual conference that's happening right now.

Just in case folks are interested. It started on Monday. People are probably listening to this on Friday or on the weekend. So it is wrapped at this point, but all the talks will be online for free. And I'll have a trip report highlighting some of my favorites. I mean, I've only watched four so far. Did you watch the keynote this morning? No, it was at the same time as the C++ Library Evolution Telecom.

But I couldn't actually really participate in that because I had to overnight FedEx to her hotel. The hotel refused to sign for them. And so then the FedEx people didn't – took the package back. And it was a whole ordeal. And then two or three other things concurrently blew up. Everybody's okay, though? like emergencies that is not like tragic or like no no none of them were emergencies all of them were drama okay all right that's ongoing i mean it's over at this point but if folks want to check things out anything uh i'm not sure if you've taken a

look at the schedule i know you've got a c++ standard parallelism talk is this the same talk you gave last year is it a updated or um or do i have to cut this out no you know i i can be honest with the listener it is an updated talk i had hoped that we were going to make a a whole new uh talk specifically on senders but we just ended up uh making an updated talk okay any any talks that you've seen or not seen? I guess probably you haven't had time to watch any, but have you checked out the schedule yet? Or no, you haven't had time? My colleague Alex is going to give a cool talk about the quantum computing stuff, the quantum C++ stuff that they're working on.

So that's probably going to be pretty interesting. Other than that, no, I got nothing. All right. I'll put a link in the description of that talk. Also in the past, tomorrow on top of GTC Day 3 is the half-day Apple Seeds 2023 conference, which is a sort of beginner online free virtual conference for people that are interested in APL. So I'm going to be... And that's actually...

You know what the best... Here's a question. Here's a trivia question, Bryce. What's the best thing about virtual conferences? There's not a ton of nice things. I mean, it's more accessible stuff. But there is one thing that... I get to wear my pajamas in my bathroom while presenting.

You still probably could do that and probably have in the past but no no but don't give me any ideas that's not it there's something that's way better about virtual conferences than that that is way better way better about virtual conferences compared to in person it is that and if you're a virtual conference that does your talks on zoom like apple seeds will be tomorrow cut that out and do what you should be doing which is having them either streamed on youtube or streamed on some video service that can be sped up because i actually don't really watch any talks live because i don't really have the patience especially when i'm sitting in my chair to watch things at one time speed. So usually I'll just watch talks from the slot before and I'll watch

them at 2x. That's actually one of the hardest things now about... You watch my talks at 2x speed. I talk pretty fast. No. I mean, yes, I watch all talks. The only time I listen to podcasts at 2.3 times speed and i listen to all talks at 2.5 speed the only time i slow there's something wrong with you i mean you're you know you're you're like really you you you do all these things that are very savant like i mean it's not it's not honestly i don't think it's savant like it's kind of like the you know how many hairs makes a beard if 500 hairs is a beard then clearly 499 must be in 498. And then it's recursively goes down to zero.

As soon as like you can't, you can't really tell the difference between 1.0 times speed and 1.1 times speed. And then so you move it up to 1.1 and then you do the same thing with 1.1 to 1.2. And then you just don't really notice until before you know it, you're at 2.3 times speed. And then you go back every once in a while. I listen to my Polish podcast at 1.0 times speed. And then sometimes when I skip to the next podcast, I'll hear it at 1.0 times speed. And like, it gives me anxiety to hear these people talking so slowly. It probably also doesn't help with my own presentation skills because I already was

a fast talker in my presentations. But now I think because I listen to these things so quickly. But the point is, is the only times I slow things down are in certain podcasts. One, if it's in a different language that I don't speak or two, if the audio quality is bad or if there is an act. You're listening to other podcast to podcast in other languages now yeah i mean can you can you leave some room for for the for the the slackers man who i don't know i don't know any of those everyone i i think i got her you're not a slacker my guy i hate to break it to you i am definitely a slacker i am the book definition of a slacker okay mr

mr head of ewg and uh you know a famous person once said and i i don't know who and i don't care a lazy person is a smart person because they're always finding the most efficient way to do things um no uh what they said was um i think it was something along the lines of, smart people think, other people work. I got no intention of doing work. I'm just the ideas guy. I mean, I disagree. A couple of my favorite quotes,

one of my favorite quotes is by Thomas Edison. Genius is 1% inspiration and 99% perspiration. I love that quote. You and I have different philosophies to life. What's another one of my favorite quotes that's in that? It's escaping me right now but uh yeah i just i have uh everyone has like a different style right and my my style is that i want to consume just like an absurdly large amount of information like that's the kind of the problem with reading books

you know most books are like 95 kind of like a waste of time so then you're gonna get the elon chip too oh as soon as they have some kind of intelligence augmentation thing that is like approved absolutely like i think this phone right now like i think we're already like intelligence augmented human beings like how many people can operate normally quote unquote, without a cell phone? Like that's how we spend more time communicating with people through this device that everyone always has on them than we do like face to face. And like there's so many things, you know, like people looking up words in a dictionary, finding directions from point A to point B. It's all completely changed from when we were kids.

And yeah, I think we are already like intelligence augmented human beings and the next step is some chip. Like I think it's absurd that I have to carry around a Jeep. When I go running, I have headphones in my ears that are hardware, a credit card and maybe an ID in my pocket, a cell phone for music, and then a GPS watch, which technically I could use for my music, but I like, you know, messing around with my podcast. So it's like a phone, headphones, a watch, and credit cards. All of that stuff could be like a couple stickers on my forehead and a chip in my wrist. Like, let's go. Come on. I'm so tired. Like we're going to, by the time we're 60, we're going to look back at the days when you would leave the house and you'd have to bring all this stuff, these headphones and all this stuff.

And it's just going to be like, it's going to be in us or stickers on our foreheads. And that is the best. And I can't wait for that. The fact that I'm pulling out a piece of plastic that has some RFID thing so I can pay for stuff. You're going to have to title this episode stickers on our foreheads. Wouldn't that be amazing? And honestly, just put the headphones in my eardrums because I'm probably already slowly destroying my hearing by listening to music too loud.

I'm buying a cabin in the woods and a gun to shoot all the robots. No, you're not. You're going to be right there sitting up in the clouds, above the clouds. Anyways, we're past the 30-minute mark. We're definitely past the 30-minute mark. End episode 122. Be sure to check your podcast app or ADSPthepodcast.com for links to any

of the topics that we discussed today and a link to a GitHub discussion where you can ask any questions or leave any comments that you have. Thanks for listening. We hope you enjoyed and have a great day.