Software Huddle - Tech layoffs, Sora by OpenAI, Gemini 1.5, Apple Vision Pro & more

Episode Date: February 20, 2024

Our special episode is back! Join Sean, Alex & Vino in this fun conversation. 00:00 Introduction 10:08 Sora by OpenAi 16:11 Google Gemini 1.5 22:05 Mixture-of-Experts 38:02 Nvidia’s Valuation 40:19 Apple Vision Pro 49:05 Tech Layoffs

Hello, welcome to Software Huddle. I'm here with my co-host, Alex Debris. Alex, how are you? Sean, I'm doing well. Great to be here. Good to get together, do a banter episode. Excited to have Vino here as well. Yeah, it's been a while since we've actually sort of chatted live, just asynchronous Slack messages to coordinate things. Yeah, and I'm kind of running on fumes a little bit today after a week of our sales kickoff and the prior week running a couple of events. So I'm depending on you and our guests to bring the energy. And actually, our guest, Dino, was at one of the events that I helped host about a week and a half ago at the Snowflake office. So we should probably go ahead and bring her in.

And our last time we did this was with uh merit bear and i think that was such a it's great to have a uh a new energy i guess to these that's not just us talking to each other so we're we're bringing it back we're bringing another guest in and vino's actually been on the podcast before so welcome back you know hey sean excited to be here and i guess today is the day we're going to banter about all things software. I don't know. What are we going to talk about today? What is the banter episode like? Well, mostly we kind of update ourselves on our lives and then get into some of the hot news and some of our hot takes and impressions and so forth.

But maybe you introduced yourself on a prior episode, but maybe so we don't need to go through formal introductions, but we could start perhaps with, what are you currently focused on at Snowflake? What are some of the projects that you have going on there? Well, currently, I mean, I work as a developer advocate at Snowflake, primarily

focused on the data engineering workloads and a bit of LLM stuff, but currently, though, I'm working on a cool demo for the Kafka Summit London next month, and it's about a voice-to-text screaming pipeline that uses Snowpipe Streaming and Dynamic Tables and more. It's just been trying my hand at it, getting things to work. It's been all that this week for me.

Will this be a live demo, like on stage? Oh my God, it will be, which is what kind of scares me because usually live demos go pretty okay. But this one is like like i really plan to make the audience talk about something and use that as a voice input for my demo because otherwise it makes no point but it's also like yeah i think i like problems at this point in my life and i'm like god let's do this yeah that feels fraught with it could be high highs low lows depending on how that goes but you can always have a video backup. I've failed a few times with the live demo. Sometimes for my own reasons, it's been my

fault. But other times it's been actually some sort of AV setup mishap that actually happened to me last September when I was in Croatia. Basically, my mouse was uncontrollable. I don't know what was going on. I think something else was connected to my computer and I essentially couldn't do the live demo. How do you handle that? Do you just skip the demo part and go in and end early? Or do you just like, you know, just talk for an extra like 10 minutes in there that you weren't going to? Or how do you handle that? Ideally, I have a backup where, you know, maybe I have some, depending on what the demo is, I have like still images that I can go through or I have a video backup that I can play or something like that. That's the smart thing in Croatia.

I didn't really have a backup. So I just like, I just was like, imagine that this thing is happening, you know? And the only thing you can do is, you know, make a joke about it or you know, try to describe what it is. But I feel like most people have empathy for those situations because they've also been in uh those situations or could imagine themselves in it and i think as long as you don't like get too embarrassed about it and make a big deal you can you can make it work essentially and it might even feel more authentic and and create more of a bond it's like your trauma bonding

with these people yep yep i i have a related question. Like on this note, I used to do some like dev rel, dev advocacy type stuff. You two are both sort of dev rel or dev rel adjacent in marketing right now. How has that, and I hear about people saying, Hey, the industry has changed,

you know, especially since COVID with, with travel stuff, although now travel is starting to pick back up more, I guess like, how have you seen it change over the last couple of years? Has it changed in terms of travel or tactics or, different things that that you're focusing on well if i'm talking about like my experience per se i think forget about you know the industry and the market and everything else compared to when i started out in devra to compare

comparing it to today i personally would prefer to create an online content that can kind of be there on the internet forever and give me more ROI in terms of how many eyes I get on the content and how many people get their hands on it. This is actually, you know, like take a five hour long flight or even sometimes 10 hour long flight to just go deliver half an hour talk and then come back. And it's just, you know, personally hard for me to justify the travel, although it might have been exciting initial years of, you know, coming out of COVID, everybody was excited about traveling, let's go to this conference and that. But I feel like people are getting kind of done with that and focus more on the online

content creation at this point. At least me, if I'm speaking for myself. What do you think, Sean, what are you up to? Yeah, I mean, I think it depends a little bit on what your focus is and what you're trying to accomplish because ideally the things that you're putting like effort into especially if it's going to be like expensive where you're uh paying for travel and hotels and all these kinds of things then you want it to be like that sort of activity aligned with whatever the business initiative is. And I think where like DevRel functions or practitioners sometimes can get themselves into a bad position with a company is when there's like a misalignment essentially of the activities they're doing or even the activities they like doing and what the business sort of objectives are.

So if you can show essentially a connection between the things that you're putting effort into and you're creating, and that's moving the needle for the business, then I think it could be justified. So it probably depends a little bit on what the business is accomplishing and stuff like that. For me, I travel a fair amount, but it's not all necessarily DevRel related. Since I lead marketing at Skyflow, I oversee a lot of other things like our industry sponsorship events and our dinner programs and stuff. So sometimes I'm there as an executive representation

of the company, or I might be there to just make sure things are going okay. So I have sort of responsibilities outside of just being being there as a speaker um connecting with people yeah yeah it's like a very high touch thing where you're getting a nice relationship with someone that is a heavy user you know will become a heavy user or something like that um how do you you mentioned a little bit about like measurement type stuff how do you how do you think about that with uh like pure conference talks types things you there's are there things you look for there go ahead yeah okay i was gonna say i personally for me the only way of measuring the effectiveness of a conference talk is based on the kind of questions that i get you know like so i used

to work for a you know series a startup we were trying to get a product market fit and everything and at that point our goal was to go to these data conferences and then literally shout from the top of the mountain that you know know, a product like this exists, you should check it out. And, you know, it was a whole different story. But, you know, Snowflake is a whole different deal though. But in that case, for me, I go in with a specific developer messaging in mind.

And at the end of the talk, depending on the kind of questions I get, I'm like, oh my gosh, so this actually resonates with these developers based on those questions, right? Like some people do tend to have vanity metrics. So you know how many people turn up at your talk and like how big is the conference and all of that. But for me, especially for that specific, you know, situation, small startup, figuring out our messaging and PMF, it was mostly the kind of questions and understanding the developer messaging that was very successful or like a fruitful metric for us right but i can't say this is snowflake for sure yeah that sorry sorry to interrupt but like that

kind of reminds me like i remember when i was doing this dev advocacy similar stuff i would hang out like on slack channels or forums and stuff like that and it's like you're you're helping a small number of people there but you're also getting a good feedback loop of like where are people running into problems or like what issues what other things are they using and how does it work or not work with our thing so yeah that's a good point like drives that that whole like cycle there yeah i think that there's i mean there's it's kind of a combination of both qualitative and quantitative metrics that you can look at you know you can generate what's known as a trip report which brings in some of these things of like you know

how many people were in the talk what kind kind of questions did you get? What were things that you learned? I've seen people do things of like, I don't know, like business cards collected and stuff. A lot of these things are just like a superficial metric that you can give to somebody. But I think if you're in a smaller organization, ideally, there's less focus on proving the value and more of a natural understanding that there is value. Because it's usually... If you're a smaller organization, the advantage there is everybody has an opportunity to have company-level impact.

And it's usually fairly transparent if you are. But if you're not, it's also transparent, which might be problematic depending on your position. In a bigger company like a Google or Snowflake, like Dino's experience now or my past experience, you kind of have to put a lot more legwork into like, here's the metrics, here's... And you can also try to do correlations. I used to run events and then I would essentially,

especially if it was like a new market that we were in, I'd look at the activity in that market post the event. So it's not necessarily that I can say definitively that that activity led to people using the product more or making more API calls. But I could essentially tell a story that showed some sort of influence. And a lot of times, I think DevRel activities, also product marketing sometimes falls under this lens. It's like, how are you influencing behaviors? Or maybe if you're a sales organization, led organizations,

how are you influencing pipeline in some meaningful way? So a lot of times it becomes more of this correlation metric or showing that someone touched something that you had your hands on or contributed to. Yep. Yep. Awesome. Should we get into some news? Yeah, let's do it. I will lead this off. And what else can we talk about besides AI, right? Like, just it feels like, you know, every week, there's sort of new stuff going on. I think this week was particularly interesting, right? So we had a few releases, the big one being OpenAI releases

Sora. I don't know if I'm saying that right, Sora, but it's basically like a text to video interesting. Right. So we had a few releases, the big one being open AI releases, Sora. I don't know if I'm saying that right. Sora, but it's basically like a text to video generation type thing. So we've seen text to text to text, we've seen text to image, you know, with Dolly and things like that, but now actually creating 30 second minute long videos from text looks super sharp. Um, you know, not open to everyone yet, but they're at least like starting to show what's possible and things like that. And I don't know, what do you all think about that? Well, I guess the timing of it is super impressive

because only a couple of days ago, if you remember, Jan Lekoon was talking about how difficult it is to predict the next frame. And video generation is one of the biggest problems we've not solved. And then two days later, opening up Sora, and I'm like, we can is one of the biggest problems we've not solved. And then two days later, OpenAI dropped Sora. And I'm like, we can do one minute long videos. And everybody's like, well, who's going to tell Jan now?

Yeah, they should have given him a heads up, not to embarrass him like that, right? How many startups just got like killed overnight because they were working on this problem? Yeah. It's going wild. I remember the Dev Day happening. And then literally just right after the Dev Day event, I go into Reddit and everybody's like, oh my god, this company, like six months of our effort has

gone into like nothing because of announcements. I'm sure there's going to be a similar effect for Sora as well. Yeah, I think you got to be pretty careful if you're making some kind of wrapper around OpenAI's APIs right now and not necessarily doing something that's like, you know, truly like a secret sauce

or something like that, because they keep coming out with these different announcements. I mean, it looks really impressive. And the funny thing is like a week ago, actually leading up to our SKL is putting together some slide and I wanted to have like an animated GIF on it. And I actually tried to use like a free LLM product to create like an animated GIF on it and i actually tried to use uh like a free lm product to create like an animated gif um and it was terrible and i was like and then like literally like a few days later they come out with this like super impressive at least like in the demos and stuff like that super impressive looking video i was like oh wow like this is like completely blows away the the thing that i was trying to do yep yeah it was cool too also that they released

like you know some some really good amazing videos and then also like well it's not always great like here are some bad videos right it was like a guy walking backwards on a treadmill or like trying to dig a plastic chair out of the sand and it like sort of morphed into oh absolutely there's got to be so many like terrifying videos that they created it's just like some of the like images that people make with like Dolly and stuff like that, where it's just, you know, like, I don't know, like a clown face on a, on a baby or something like it's like terrifying. I do like, and I'm not, I'm not surprised that it's not in the public.

Like I think, you know, as impressive as this is and potentially like the types of like cool things that you can do with it. I also think there's like a huge risk involved with how you could abuse it, which is same with any of the like kind of text to voice stuff where you can simulate people's voices. Like we talked about the retool hack a few times ago where they were using deep fake audio through AI, like there's a lot of lot of scary scenarios with this as well. Yeah, absolutely. I'll be curious to see because one thing, I do worry about that as well, but also

if you see an image now, you can sort of tell if it was generated by Dolly. They got the same kind of vibes around what that is. I wonder if that's sort of going to be the same with these videos or if they're going to just keep getting better to the point where it's gonna be like really hard to distinguish that sort of thing no i think it's getting really really hard

like when i was going over the you know sora videos that they released like that's literally all i had in mind right i was just like you know what i'm really gonna like pay attention to every single detail in the video to see if i'm able to tell it apart that it is like originally recorded or it's an AI generated one. It's insanely crazy how hard it is to tell. Even you know that it's AI generated. They say it is AI generated. I'm not able to just not even know minute of details except for the wrong videos that they share. Of course, they were evident. But this one, it's going to make make it extremely hard i don't know the big thing i would say is like there's not a lot of like conversation or interaction or even like depth of sort of emotion type thing like there's a woman like i think she's like making a cake or something like that and she's just like kind of

smiling and looking around but it's like the same smile just kind of like looking around vaguely and it's like once it goes on for like five or six seconds, you're like, like no human would have acted like this, you know? And so like, it's going to be like really short clips or they got to figure out a way to, I don't know, get some like communication interaction between people, things like that. But it is pretty wild.

Yeah. I mean, even if you take the sort of simpler use case of like text generation, a lot of times when you put like a prompt into something like ChatGPT and it comes back with something that's written, like it makes sense in terms of like you can read it, but it might be least in my experiences with ChatGPT, where it's like, they're like signals that ChatGPT, like, generates. And it takes, like, some massaging to kind of, like, get the thing that you want. And you might even need to edit it beyond what it's giving you. But this is, like, taking something that's, like, I feel like a much harder problem of like video generation so there's gonna be i think a while before you get something that could be like you know you're generating essentially like a hollywood level marvel video you know movie or something like that with it i think we're we're a little ways from there yeah or we can like just strap in and you know be like i

want to watch a one-hour video about this And it just like generates the whole thing for you. Like this is not going to happen for a while yet. Hopefully not. Yeah. Right. But I'm switching gears a little bit in terms of like staying on AI, but also like, I think this was also, you know, yesterday, Thursday, February 15th, Google talks about some new models that they have. So they, they talked about some Gemini ultra 1.0 or pro 1.0 i can't remember exactly but they they released a new model this week and also are holding back some things with the big thing being here much larger token context windows like we're talking you know 128 000 tokens that they're that they're making available for regular people they also have

like a million token context window that they're making available for regular people. They also have like a million token context window that they're making available for, you know, select partners and things like that. They said they've tested up to 10 million tokens, which is, that's interesting on one sense. And like,

you know, anthropic has some really large token windows, but I feel like they're not super useful. Like if you put a bunch of stuff in it, it's actually not that good at, at finding it. But then you will have these, these demos of like,

I think they fed like a 45 minute silent video into it and then said hey tell me when this specific thing happens and it was like kind of it was pretty neat she was very like specific and they're like oh yeah it takes a little while to do it took like a minute but then it's like at 1201 this is when it happens and you know it pulls this card out and this is what it says on it and i was like wow that's like jeff dean was showing like some recall on like these enormous context windows and and showing to be really good which is you know that'd be a that'd be a game changer in a lot of ways as well interesting but i feel like before we even talk about the capabilities of the model sean what is up with the naming of these products i'm like i don't know what's going on. Well, as somebody who worked on RCS business messaging, business messages, verified calls, verified SMS, my only experience really working on these products at Google is,

whatever the worst possible name choice is, they're going to go with it. It's like, so it's so hard to track. Like even our group at Google was, I can't even remember what it stood for, but like the sort of org was, the initials was ABC, which is also the initials for Apple Business Chat, which was like a direct competitor to us. Like it's like, what decision-making led to that decision? It was just like, it doesn't make, I don't know.

I have no answer for you is what I'm saying. I was like, there was Bard. It was like, fine. And then they came up with Gemini's family's borders. And I was like, all right, not too hard.

Can keep, you know, keep up with this. And then there is Pro and there is Ultra, there is Advanced and there's 1.0 and there's 1.5. I'm like, oh my God,

just stop already. Like, why? What is this? Yeah. Well, like, yeah, what is going on there? Like, is it not spending enough time on it or is it spending too much time and just like yeah what is going on there like is it not spending enough time on it or is it spending too much time and just like workshopping it to death towards like

the least i don't know maybe least offensive or just most generic or goofy thing out there uh i mean it could be that you know when you get a big company uh you know a lot of these things end up being sort of decisions by committee and they're a lot it's a lot less like google doesn't run like like a company like apple where you kind of have this person at the leadership level that's like really steering the ship i think google is a lot more like a loosely aligned federation of planets or ships that are kind of roughly going in the same direction so i think that's like part of it. There's not necessarily like one visionary that's kind of like pushing these things forward. So you get a lot of like fragmentation. And I also think that's why you end up with sometimes like competing products within

the same organization as well, because these groups are kind of operating a lot of times like independently. And, you know, maybe there's also some bias with just, it's a very engineering-focused organization. So I don't know. They talk about the two problems and the two hardest problems in engineering is cache invalidation, naming things in an off-by-one error. So naming things is one of those problems. And if you have engineers naming all these things, well, this is what you get. Yeah. Yeah. Absolutely. those problems and if you have engineers naming all these things well this is what you get yeah yeah absolutely i speaking of that joke i saw um fly.io this week this is total tangent here but fly.io like launched their own sort of storage like s3 storage and they were saying the three

hardest problems in computer science are you know cache invalidation naming things and trying to serve files better than Amazon. Right. Cause like Amazon's got S3, it's pretty cheap. It's reliable, but like, yeah,

it's got like 12 nines of reliability or something like that. Yeah, exactly. Yep. But they're going after it. They're a cheaper thing. Yeah. Anyway, so total tangent there,

but, but yeah, I think that Gemini is like interesting. They're saying like, you know, it can, I think with that 128,000 context window there, you're talking like 30,000 lines of code, like that's a pretty decent sized code base where it can understand that pretty well. I think it'll be interesting for, you know, just being a coding assistant where I found, you know, these models

most helpful, like how much that's going to improve over the next couple of years. Yeah, I think the bigger context window opens up new types of applications, possibilities than that you weren't able to really do before, especially when you're talking like multimodality, like you talked about this, like silent film, like suddenly, you can feed in like large audio files, large video files and do really like impressive things that, you know, historically probably required a person to go and look through. You think of surveillance tapes and people trying to find some incidents that happened

on multiple surveillance tapes. That's just people going through those videos and fast forward. To suddenly be able to have an AI system to do that is pretty powerful. Wow. Yeah, that's a great use case. I didn't know that. I think it's also, I guess, Chevron Advanced is a mixture of

experts model and not necessarily just that one thing, right? So it's going to be super useful to see the variety of applications it can suddenly power and also of course because of the context windows. I guess multi-modality is probably going to be the word of the year. Forget about ginning IO LLMs.

Yeah. Can you explain mixture of experts for me real quick? I hear it talked about a lot and people like assuming that's what's happening with GPT-4. We're like, yeah, what's going on there? Okay. So I guess mixture of experts, as it says, is you have instructors just having one model under the hood.

You have multiple models that can solve or that will fine tune for like different types of problem solving. And then they kind of like pull together or say some sort of voting is there in the output generation stage that kind of, you know, decides what output actually gets passed on to the user, really. So it's, if you think about it, when you go from one chat GPT to another, to come up with one powerful model that is so much bigger and powerful, then chat GPT is going to be super hard. But then if you can kind of, it's kind of using the power of networks, really putting together multiple networks, which is smarter to get to the higher level of accuracy much faster than just training that one model and fine tuning it all the way. Yeah. And some models outperform others on certain types of tasks.

So depending on the use case, you might want a different model or different measurement approach. And then essentially, if you can bring these things together, then you can sort of get the best of all possibilities. And are we talking like in terms of number of models, like how many are under the hood, like four or 10, or is it like 100, or is it like a million? Like how many models are there? Is that just sort of unclear?

What's going on there? I don't think we will know that. It could be from a few tens to even be few hundreds. I don't think I have an idea. I've not gone over the paper. But the thing is, it's not a new idea per se, right? It's even in the classical machine learning models, they always had the concepts of, you know, stacking and bagging, which a new idea per se, right? Even in the classical machine learning models, we always had the concepts of stacking and bagging, which is literally what you do, right?

You either cascade the models, like feed output of one model to another, where you have a more coarser classification model followed by a more fine-tuned classification model, which is more of a stacking, like cascading effect, or you have multiple models pulled together to solve the problem. It's just going back to the foundations and kind of applying that on top of our existing other models.

Yeah, we do this for entity detection, for basically PII detection within text or audio. And there's lots of different approaches, and it can be a hard problem at scale. But if you combine multiple approaches, you can get really, really high accuracy on it. Okay, interesting. Nice. What about you two individually how are you using ai day-to-day like is it a big part of your workflow is it like you know something you reach for once in a while but but not all the time what does that look like i think for me is definitely a big part of my workflow is that specifically when it comes to text generation right so for blogging it's easy for me to kind of get started

that there's no starting barrier or like a writer's block essentially, because I have just an outline and then it helps me work off of that. And it's like super convenient. And also currently I'm enrolled in an executive MBA program where there's like tons and tons of case studies

to be written, where all you need to do is just take notes and like put together an outline and then the stories are made beautifully with, you know, Chargy PTA. So it's been very helpful for me. And on the other end also, right, when it comes to, I'm almost now starting to use these LLM models as search engines. Like, you know, me going to actually search for documentation and like snippets and

code templates have come down drastically with these you know applications as well so like very much easily 75 80 of my days like going to these motors and working with them yeah when you're doing the writing part are you writing in like vs code like markdown and using copilot or you go into chat gpt directly or like what are you what's your workflow to where it's helping you generate that sort of stuff okay so when i'm writing the actual text stuff it's just directly going to chat gpd and writing through them because you know it's kind of easy to work off my google doc versus the you know chat gpd interface but then it's also interesting one thing that i have noticed now because i use a lot of

chat gpd now you give me a text and i i will be able to differentiate saying oh this is clearly written by chat gpd like i can tell you like a standard you know paragraph structure like like fixed number of sentences in every paragraph and every sentence has like exact same number of word lengths it's like it's too perfect to be written by like someone whose second language is English right I'm not an English speaker so it's like it's super easy to point out maybe for a native speaker you might be able to write as good in English as statuette so it's probably hard but for me I'm like oh my god like it's the more you work with it like you're actually able to see through it like super easily, which I hope, you know,

we're able to do it for the other models though. Yeah, for me, a similar, I use it a lot for sort of either like getting started or enhancing things that I'm working on or even condensing, you know, like sometimes if you're putting together like an abstract for a talk, I'll just start, you know, I brain dump it and then it's like okay well i can only submit 100 words and then it can be sort of your brainstorming buddy for like trimming it down and editing it and i use it for brainstorming i use it for like stuff with my kids like i uh you know uh ideas for

uh show and tell um uh you know you can it's really like my sort of assistant in a lot of ways for both sort of work-related things and then also for like parenting. But I do use Google a lot less these days than ChatGPT because there's certain types of things that are just like way, like the difference is if you need kind of like you're looking for an answer like a definitive answer versus a collection of things then chat

to upt is going to be probably a better resource as long as it also doesn't have to be like real time uh like in terms of up to date to now like i remember i was on a road trip with my wife and we were talking about some mutual friend who was going through like a divorce and they'd been married for 10 years and my wife asked me about like oh wonder what like the alimony rules are or something like that and i said that i i thought california had like a um it like depends on your income and all this sort of stuff and then i started trying to look it up on google and of course you get back like an article and i'm trying to dig through so i was like ah screw this i'm just gonna go and then i go and ask chat gpt that like specific thing and it can actually it's like we're not a lawyer but they'll give you like a specific um the specifics of what

you're looking for rather than having to like come through this stuff and kind of parse it yourself yeah yeah yeah that's interesting man i came across something recent i can't remember if it was like diet or injury or advice or like tax advice something and it like it was like it wouldn't give it to me it was like i'm not an expert in this area you should consult someone else i was like come on just just tell me the details i'll still like check it or something you know i just want to know yeah i get that a lot with like i played around with bard um or gemini or whatever it's called now and um and i get it a lot on there where it'll be like, you know, I'm not trained to do this and stuff.

I'm like, really? Come on. Like, I'm just, you know, asking for, you know, give me 10 jokes about like robots. Like, you know, give me something here. You've read the entire internet. What do you mean you're not trained on this? Yeah, exactly.

Okay. You said you do some like blank page writing or like some writing. Is that in ChatGPT or is that more integrated with some tool that you have, Sean? It's ChatGPT or we actually developed our own internal product called VerbaGPT2 that's trained on our own documentation and stuff. I'm actually demoing it at a a snowflake event next week but the it's um so then and it's templatized too so i can say like i want like a blog post or press release or a piece of documentation based around this idea and it'll create it in the style of skyflow adhering to our you know brand

and language and tone guidelines and so forth so it's it's it's basically like a more tailored LLM specifically for our company. Yeah, very cool. Yeah. One thing I love to use it for is like sometimes I'll be doing some writing and I'll do a lot of like writing on data modeling or something like that. Right. And I need an example. I'm like, you might do it this way, for example, in this situation. And I can never think of an example or I use like the same three examples.

Then go to ChatGPT. I'm like, hey, I need an example of an application that does this. It updates a lot, but it doesn't do this or something like that. Give me 10 examples. And it's just like so good at just, you know, brainstorming like you're saying. I love that.

I've been using it for like random data generation too. Like I had some, I was working on like some Snowflake demo and I needed a bunch of JSON records that represented account data. I started doing this manually, but after three or four, you're like, okay, that's all the fake names I can think of and stuff. Then I was like, take this structure and generate 25 of these. It's awesome for that kind of stuff.

You need to generate synthetic data. I also use it now for if I need to convert the title of something to kebab casing or snake case, I just go convert this to kebab and snake case, and it spits it out for me. Yeah, absolutely. It's pretty amazing. I think the synthetic data use case is super interesting for me because that's been forever a challenge for any demo that you're putting together.

So this is like, okay, I'm going to use it for a lot more than I already do. Yeah, it's really great. So we're talking about how we've seen it now. How do you think it's going to change development in five years like are we mostly still going to be hacking out a lot of our our code or doing a lot of writing ourselves or is it going to be chad gpt is going to do the vast majority of it and we'll just do like higher level sort of maybe structuring or or planning or like how do you how do you sort of see that right does it change i've seen some people speculate like hey we're not going to be needing typed languages anymore because like you know chad gpt doesn't

make isn't going to make those types of errors you know we're a ways off but like imagine something like that so i don't know what do you think about some of those huh okay if you remember dally and how you know chat gpd and dally work together like whatever prompt you give to dally directly there is a chat gpd in between which takes your prompts and writes a detail, you know, a lot more, with a lot more explanation and everything, and then feeds it to Dally to improve the quality of the image it generates. Right. And for me, the first thing that occurred to me when I read about it was, Oh my God,

I can kind of do the same thing with chat GPT on chat GPT. Right. Instead of me having to write a prompt, we can literally just have a template or even a YAML file or whatever that is, where I just given a certain parameters, say, like, what Sean and his team had worked on, right? I just say, press release, and then give ABC keywords, and then just take this, you create a prompt, and then you feed the prompt to yourself, and then give me a lot more detail, you know, because the minute we start working with natural language it purely is like super highly variable right you write a prompt it's going to be different i write a prompt it's going to be like way different i feel like that's going to cut down on the efficiency of the model itself if you're not writing a good prompt but why don't you let the one thing that is super great at generating text write the prompt itself that all you need to do is just given a bunch of keywords so that's will be very interesting but i'm yet to see anything that uses something like that though yeah i think it's hard to predict like you know where we'll be in five years because

partly because like things are moving and changing so quickly but i also think that a lot of times like the last like 20 of like getting there is like is like the hard part like it takes forever like if you look at like autonomous vehicles you know they every lots of people thought we'd have uh you know autonomous vehicles widely in circulation by now but there's a there i mean it's a little bit different because the consequences of mistakes is very very high uh in terms of like human life and stuff like that. Consequences of a bug in software maybe is not as vital depending on what the software system is. But I do think that it could be that last like, you know, 10 or 20% to really get it like human level is very, very difficult to nail. The other thing, at least right now, that is a constraint is you can certainly use it.

It's a super high-value tool. I think 80% of engineers are now relying on some sort of co-pilot assistant. That's super valuable. But doing large-scale stuff is still a really big challenge. Even when I talk to Lee Robertson from Vercel and their V0 launch, you know, as impressive as that is, the more sort of limited the scope of the thing that you ask it to build, the better the generation. Because it's kind of hard to build like, hey, build me, you know, amazon.com e-commerce site. Like that's just going to be really, really difficult to actually build something valuable there. But if you're like, create a checkout form with this button style, like that's a little bit more of a limited scope problem. So I think there's definitely some scoping stuff. But I do think it changes a little

bit in terms of engineering's day to day. But the so things are, I think, moving towards a place where you're working at a higher level of distraction probably earlier in your career. Yeah. Do you think we'll have more or fewer people? Let's say in five years, do you think we'll have more or fewer people than right now who the vast majority of their day is writing code? Which I would say, you know, if you look back every five years for a long time, that has been going up i'm guessing you know so basically it's not necessarily that the role of engineer goes away but are they sort of hands-on keyboard writing code day in day out yeah yeah are they yeah like are they still yeah do you still have that many people who's that's their sort of main main job writing code i don't know i think that's a tough question um yeah like in one sense like

i think of last february because like we're like 15 months from from chad gbt coming out is that right roughly it was like november year and a half ago okay so like three months after that i was like holy smokes no one's gonna be writing code in like a year or two and if i look at that if i look at now it's like that's you know that's obviously not the case, but I also think I use copilot or GPT or things like that a lot more than I would have expected, you know, even last June, like where I was like, you know, crazy expectations sort of crashed back down. And now I'm like, as I look, I'm just like, man, I do use it a ton, you know, just even just like being helpful in the IDE or things like that. So

I don't know, it's hard to it's hard to know when you're sort of on the slope, like where it's going to end up. But yeah, I think there's still a number of like really hard problems. Like I think there's a project called I think it's alpha code, where they're, they're trying to use LLMs to solve the ACM ICPC programming problems, which were competitions I used to compete in in college. And usually there's like a story component to it. And then you have to essentially, the problem is described as a story.

So it's not necessarily like write a, you know, breadth first search algorithm to solve this. It's not like that. It's like the solution is to it. And the algorithm, the problem you're trying to solve is like to figure out the mapping from the story to essentially the algorithm and so forth you might have to design something that new like a dynamic programming algorithm so and like it's not very good at that like that like that's a hard problem because you're sort of having to parse this story and then figure out

the um uh the way to like map it another. Maybe the training session is too small. I don't know. So I definitely think there's a lot of problems that are still going to be difficult. So I think my guess, if I had to throw out a guess right now, do it bold, I think that there's still going to be

a fair amount of hands-on engineering going on day-to-day, even within five years. Yeah. Yeah. Okay. All right. Last question before we leave

ai at some point during this week i'm not sure if it's still true but nvidia was worth more than amazon was worth more than google as a company does that seem right to you that seems like a bubble that seems bubbly i think it totally does it does okay go ahead i mean come on look look at what we've been talking about the last almost year and a half. It's about time. Let's just get it on board. But it's kind of also scary. Is it just NVIDIA really at this point?

Is there no one else who can take advantage of what's going on? But it's good for them, I guess. But totally not surprised. But I'm also excited about what Sam is up to with $7 trillion trying to come up with a phone compute and GPUs and whatnot. So NVIDIA probably. but I'm also excited about what Sam is up to with 7 trillion, trying to come up with a phone compute and GPUs and whatnot. So Nvidia probably has some competition coming up, but like, you know,

way, way too early to say anything about it, I guess. Yeah. I would say it's probably justified, but maybe a little inflated in terms of value because of all the hype. And then also because there's such a resource constraint right now in GPUs and they kind of own the market there. So I guess I don't know that sustainable is sort of what I'm thinking. But the thing is, it's like a lot of these companies that are also high valued in the market depend on NVIDIA.

So their growth essentially leads to more growth for NVIDIA at this moment. So I don't know if it's long-term, uh, maintainable, but like the, I think it's justified in a lot of ways, just because that's what everybody's doing essentially. Yeah. Yeah. Yeah. It's surprising me. I mean, Google just has like such a sweet business model. Right. And then, and then Amazon just has so much built out, both in terms of like the retail warehouses and logistics network and all that stuff,

but then also cloud and the margins there and data centers. Like, man, that just, that seems wild to me for NVIDIA to be worth more than them. But yeah, I think the big thing is, yeah, sorry, go ahead. Yeah, I was just gonna say in terms of Google,

I think because the majority of their revenue comes from ads, the multiple on that is lower than the multiple on other types of products. So you could make an argument that Google is undervalued. But at the same time, like we've been talking about, we're using Google a lot less these days. So their business is potentially under a threat as well. Yeah. Yeah, absolutely. All right. Let's switch gears into another behemoth, Apple Vision Pro. Do either of you have one? Do you want one?

What's the word on Vision Pro? Well, I don't have one. I don't think I would want to have one, at least not yet. Looking at how big and clunky it looks with all the wires going around, I'm like, sorry, keep it. It's fine i'll probably come check back in a couple months later or maybe a couple years later when you have a better lighter sleeker one on the other hand i've been excited to check out the meta ray bands so maybe like something no i don't but that's something of interest to me and i would definitely

like put my money on it if i have to because it's a lot more, I don't know, convenient. It's more like a consumer product than the Apple Vision Pro for me. I don't know. It's probably targeted at a very niche audience, at least at this point. I don't think they've figured out who it's going to be for.

And also it's the first version. You know as a Linux developer, we wait for the stable versions. We never do, you know, the latest releases. So it's great. I've never been really like an early adopter with especially like hardware technology. I was like super late to like, just even like a flip phone.

And then I was really late to smartphones as well. So I'm definitely not a like D1 adopter. And I'm sure like a lot of Apple products, it'll get better as they develop these different versions. Like the first, like a smartwatch was terrible. I think that the first iPhone didn't even have like copy and paste or, and what's other different functionalities is barely a phone essentially.

So, but it, you know, it's hard to argue with the success. I mean, it's kind of like funny, like Apple basically has this reputation that like the V1 is not going to be great,

but people still buy it because it's essentially Apple and they do marketing and product innovation really well. And it has a loyal base. But I was also thinking about this is like the greatest trick the devil ever played,

convincing the world to buy the V1, even though you know that it's not going to be good. It's insane that it's like, oh, I know it's going to be bad, but yeah, here's $5,000 for this device. I mean, honestly, think about it, right? That even as a consumer, I would rather put my money on a first version of a product that Apple builds versus something that Google builds, because I don't know, Google might, you know know if Google might literally scrape the product out of the market six months later, so it's not worth putting the money in their basket. But with Apple,

although I'm not fully convinced of how good the product is, I kind of trust the company that they're probably going to make it better and probably going to get it upgraded to better versions. So there is some sort of customer loyalty there that they've built, and it's kind of hard to you know miss out on yeah that's true yeah in terms of talking about

how bad the v1 is and things like that i had forgotten but i think it was ben thompson linked back to steve jobs demoing the first iphone on stage and you see people just going bananas that like the touch screen worked like you just like hear a gasp in the audience when he's like you just you just click and you just scroll through your, your artists or something like that. And then you pick one and people are like, you can kind of hear it. And it's like,

wow, that's like amazing that that's where things were. But yeah, it was all, you know, Blackberries with the little pixelated screens and you know, the hard, the actual like physical keyboards on them.

And it's wild to see like how big that is. Anyway, like I'm, I'm super interested in the vision pro but i can't justify it's just like too much i i do think if they had like a really good sports tie-in like if they had a a camera that was sitting courtside at an nba game that was like recording all that and you could just put your headset on and watch that like you're sitting courtside and look around and see the whole arena. That would just be so amazing.

I think I would have to get one if they did that. Yeah, well, when we were at reInvent, Alex, we talked to, I think it was NASCAR CISO, and he was talking about, I don't know, it was like a petabyte of data for every car and every race and stuff like that because of all the cameras. But they were super interested in the idea of creating these like immersive experiences for like nascar fans where you could like literally be in the cockpit of a nascar so if and probably like and then i don't know maybe they need like a special chair so you can feel like the the g-force or something yeah but like i mean i'm not a you know big like nascar fan or anything like that but i'm sure if you were that would be like an incredible experience

to really get a sense for what's happening at ground level and stuff. It would be pretty cool. Same with a lot of sports stuff. I could see people... It would be a completely immersive experience. It would change the way that people watch sports. Yeah.

I'm not a NASCAR guy or something like that, or F1, people like that, but I would be tempted to strap in and see that. Although I might get sick just zooming around those corners so quickly. But that would be really... That's just an insane experience that you just could not have any other way. That'd be pretty fun. No, I think just by talking about the different use cases,

be it gaming or even the NASCAR experience, you may have just converted me. Because I'm looking at Apple Vision Pro audios, and I'm like, oh, you know, you can get your whole screen anywhere, and you can work from anywhere. I'm like, sorry, not interested. It's probably the marketing or the messaging. They've still been working around.

It hasn't been there yet. But looking at these exciting ones, you definitely want to be there. Think about sitting in an NFL car. Are you kidding me? Just sign me up. Take my money. Yeah, I was talking to a guy like a week and a half ago at this dinner that we hosted that flew in from like LA. And he said his plane was delayed. He's like, the great thing about that was I had my like Vision Pro and I just like chill it out. It gave me more time to like play with it and stuff so um you know i

think he was pretty pretty happy with experience you know being stuck in an airport and having this completely you know immersive experience to do whatever he wants yeah yeah what do you think like the next like we're talking about it being v1 being kind of beta but like what how much smaller can like i think one of the things people say is it's just so big. Can it be sunglasses based on what they're doing? Where they're projecting stuff onto your eyes and they got cameras facing outward? Or is it just always going to have to be headset-like? And is that going to just naturally limit what it can be?

That's a good question. I think as maybe awkward or strange as it looks, it looks better than Google Glass. But again, I would trust Apple's designers from an aesthetic standpoint over the Google designers. I'm sure if you look at any type of technology, I don't know. I'm certainly not a hardware expert by any means, but we keep shrinking things down more and more. Even just like the... If you think about smartphones,

smartphones, I think, have gotten as big as they can get in terms of physical size, but they've shrunk down the actual CPUs or the transistors so that they're really really powerful so you're like you'll have massive storage massive compute on this thing that you can carry around in your pocket which is pretty crazy yeah i'll be curious how they solve like the solitariness of it as well because like you know i have a wife we have we have kids and it just aside from the price just being like yeah i'm gonna go sit downstairs by myself with the headset on while you guys are

all doing whatever you're thinking like usually my wife wants to hang out at night once the kids are in bed so just like i don't know when i would have time to to really get into it and it that seems like a hard problem to solve as well i mean maybe it's like multiplayer mode or something like that but that even feels weird to be like sitting next to each other on a couch and then being on the moon watching a movie or something so um yeah we'll see yeah Yeah, I think if these, you know, you think about like how much even like cell phones, like smartphones have like, I think, impacted the way that we interact really with other people, like compared to say 20 years ago. Like it's very normal to be in a social situation and someone's like, you know, staring at their phone the whole time so suddenly if you create these other types of environments that are very like immersive and solo activity i'm sure there's going to be some negative effects along with that that's a little bit different to me than hey i have this thing that's like my gaming device and like i go and i put an hour into it as my like entertainment or something that you know my

that's me time but like i wouldn't want to be like hey you want to watch a movie uh all right let's sit beside each other and put on our glasses uh and like you can be just watching two different movies and you're just like in the same space i guess it's just a little bit different it reminds me of like demolition man or something like that uh where they create a create a world where no people no longer have you know intercourse or anything like that and it's like a virtual experience and stuff so yeah oh man yeah we're heading towards dystopia right yeah um speaking of speaking of bad news maybe close it off with with tech layoffs like what's going on i thought we were sort of done with these but but now we've got a bunch of just like really big companies going through layoffs instacart snap, Snap, Google, Amazon, PayPal, Block.

Like you are more plugged into SF and the ecosystem and all that stuff. Like what are you hearing out there or what's it looking like? Well, it's interesting because honestly, at this point, I think I've kind of become numb to it. And I'm like, all right, another layoff. How many people? Which company which company move on kind of feeling because i remember i guess right around the time chachi was announced it was november of 2022 and then meta has announced that you know they're gonna have layoffs for the first time in the history and this and that was like quite the big deal and one of my friends who used to work at meta we used to have him like you know at our place like cheer him up and like have him like you know

give him mental support and everything like the entire week the all the talk and conversations we had was about oh my god what's going on what's going on in your team is everybody okay it was such a big audio and then I guess as the years went on and like the more and more rounds happen at this point it's not even a news item right like when you read a news item you at least give so much of your you put so much of your mind to it and it's not even that anymore it's just like oh my god another company all right whatever and it's funny i was just talking to another friend and they're like it's it's hilarious to even think about it they were like bro they fired sam altman who was the ceo of open am who are you like forget about it they were like bro they fired sam altman who was the ceo of open am who are you like forget about it like like that's a very good you know way to think about it and i'm like sure it's just

kind of you know like could be happening to anybody like it's just seems to be never ending though that's very surprising and i really did think that 2023 was it and then this is going to be a relatively better year in terms of job opportunities but now i'm like don't even know what to hope for anymore like you know what does it look like it's it's kind of hard to say anything at this point what do you think sean you have any insider information so so i do think that sort of the macro is getting better like the the like economy macro economics are getting better but i think the tech sector is the layoffs aren't going to stop i think that this is something that's going to continue because one is like if you look at like like a company like

meta for example has had massive layoffs like their stocks at like an all-time high and a lot of these companies are in that situation where they made these cuts but their their stock is going up so like they probably there's not like a incentive essentially to you know stop cutting like many of them got bloated i think during the sort of windfall when money was easy and and companies were overvalued and stuff and now there's like a natural sort of market contraction that's happening. And these companies have to respond, whether you're a public company, you have to respond to the market. If you're a private company and you've taken venture capital, then you have a board to be responsible to.

And really boards are putting pressure on different companies to make sure that they're running lean and that they're being very burn conscious. And they're, you know, trying to make their dollars last longer, because it's harder to raise capital to like, before, if you raised $100 million, a couple years ago, there was a feeling that, hey, we can just we can burn this try to grow really quickly, but we can go raise more money if we need it. And the dynamics have shifted. So then you need... Suddenly, it's like, well, I'm not going to be able to go out in 6 months and raise another $100 million round. And even if I raise money, I'm going to take a down round. So now how do I make the cash that I have preserve this war chest as long as possible? Well,

then I need to start making cuts, essentially. And even for some companies, I think it's not necessarily they're making, like in the startup realm, they're not making cuts, but they're like sort of reshuffling the deck in some ways where it's like, well, I need to hire these people based on our current strategy or based on like it's better aligned or like our go-to-market or whatever it is.

And I need to make room for those. But then there's other people that are maybe less well- well aligned in the good market. It was great to have them, but they're not as directly impactful to what we're trying to do as a business right now. So then you kind of have to lay them off, bring in other people. So there it's like keeping headcount fixed in some capacity. So I think this is kind of like where we're at.

And I think it's going to continue through this year. And even I heard from friends of mine that are at Google that they set the expectations at the beginning of the year with everybody that these layoffs are going to continue and kind of, you know, get used to it. Yeah, it's a bummer to just see it every week. And yeah, yeah, I wonder what that, yeah, you know, not only for the people directly affected, but then having to do the job search and it's definitely tighter out there and those sorts of things. Especially, you know, it feels weird, I think, where there's, you see all of this stuff that's happening, like where we started with like AI and it's like a really exciting time to be in tech.

And there's all this momentum, but I guess it's kind of similar to if you look back at the dot-com bust that happened. It's not like the internet went away. It was just like it got overvalued. People predicted what was going to happen. They were over-optimistic about the impact it was going to have. And it just took longer to get there. And then there was this contraction that happened. And there was also, in that particular case, so many people didn't really

know really what was going on. So they didn't have any way of even like really valuing the opportunities. I think we're kind of in a similar place with some of the LLM stuff too. Like we don't really know how to value some of the AI technology or startups that are out there. And that's why I think if you're doing something that's like pure AI, you are able to get money easier and you could potentially be overvalued right now because there is a lot more unknowns and it's super hot. But if you're in some other part of the market, you're unfortunately probably undervalued and underappreciated, and you kind of have to run leaner as a response i remember i guess mid last year we were thinking oh my god this year is going brutal but then this ai is gaining a lot of momentum

probably that's going to save us from all that's going on you know but it looks like end of the year and then the new year starts i'm like nothing is saving the market it's just going to continue to go and at this point i'm kind of like, you know, all that you talked about market contraction, folks trying to keep their runway longer and everything. Like, didn't we kind of cut down on, I mean, didn't layoffs happen for the very same reasons last year? And then you think we will continue to do that a lot more too? Yeah. I mean, if you look at, for example, the B2B SaaS space, something like 90% of companies last year either lost revenue. Basically, they started the year with more revenue than they ended because of people churning. Or they stayed flat.

So even if you made cuts, if you were in a situation where you're flat or down, you're probably going to have to continue to make cuts. If you were in a situation where you're flat or down, you're probably going to have to continue to make cuts. Because again, it's pretty hard to raise around the capital if you didn't actually make more money over the year. You have a negative, essentially, growth rate. So you need to look at how do I save my company? Or you need to fold up your business, go into business, and then you're probably churning a bunch of contracts with some of these other companies. So it becomes this flywheel and cycle where some companies are going under and then that impacts other companies, even if those other companies are potentially

doing well, they're going to lose money or lose revenue because those contracts end. Gotcha. It's not just the layoffs. It's all these types of things that are happening. I think that closes off the topics, but I don't want to end on a low note here. Anything exciting coming up for you all,

like either this weekend or, you know, I guess, you know, you got the big talk and you said London for Kafka Summit, is that right? Yeah, yeah. So it's next month, but then because of the complexity of the demo, I really want to be done with the demo this week or next and really have some peace of mind

because I've been in places where I've never been able to wrap up the demo. I can get it to work and then like you know like we can get to the last minute and it's like i'm getting old can't deal with so much of stress at this point like want to you know keep things clearer now but i'm also excited for this weekend i guess it's a long weekend my friends visiting a bunch of hikes planned i'm like it's friday already let's go what are you up to what's exciting um so i'm doing we were speaking live demos i'm doing a five minute live demo uh with uh flipe hoffman from snowflake you're one of your colleagues next week uh so that's going to be intense like uh to be able to pull this off in five minutes and

then the at the first week of march i'm going'm going to Vancouver to speak at a conference there, the Vancouver International Privacy and Security Conference, I think it's called. And I'm doing that new talk there. So I got a lot of work to do between now and then to actually put that together. I know what I want to talk about in my head is very clear, but actually turning that into materials and a talk track and stuff like that is going to take some work. Okay. Quick question, though, talking about, you know, putting together your talk track into a presentation and everything. Did you have had any success with, you know, creating slide decks with any of these AI companies? Yeah, I haven't tried. I haven't

tried that I talked to, or I heard of a company recently, that is doing that where they're essentially generating like a slideshow from a prompt. I haven't experimented with any of them, but I do think it's an interesting idea. I don't know how good they are, though. Gotcha. Just out of curiosity, I'm like. Veena, have you tried any of them? Not really. I've heard of Beautiful AI and I tried it too, but I was like, nah, I make it better. So it's just not up to the mark.

But I was like, would be open to try if there was anything else. It's hard for me. I think it's tough. I like to have so much control over my slides. Like I don't like anyone generating my slides, even though I hate generating them. Sometimes I'm like, I still want them to be like my style or something, you know? No, no. Yeah. That's a hard one to crack, right? That's what I was saying. Like, I was super curious, even if i did a half decent job i would have maybe like went with it but it was just like oh no sorry like it's just like really hard to yeah i think it's i think it's really hard because people have such unique styles of presentation and a lot of times like slides might just be an image right like ideally you don't have

a lot of text on the slides so it's it's more to the more important part to me is like the story that you're telling. And then you have some sort of visuals to like reinforce it or keep it compelling or have like, you know, hit something a little bit harder to stick with the audience and stuff like that. I think where I would be interested in using something like this, and I haven't really experimented with it is if I put together the slides, then on a slide-by-slide basis, be able to feed it to an LLM and say,

how can I make this better? Where you can take a title of a talk or a title of a blog post or something and generate 10 different versions of this. And sometimes that allows you to key into an idea that maybe you hadn't thought of or phrasing that's better. And I think as a brainstorming buddy, that could be interesting if you could do it. Gotcha. All right. Well, that concludes our banter episode. Dino, thanks so much for stopping by. It's great to have you back on the show. Always happy to come in and talk about all things AI, data, industry, SF, and everything. It was wonderful, and thanks for having me. Awesome. And Alex, good to see you live and in person. Well, not quite in person, I guess, but video person.

Good to see the person, yep. Unless you're a Sora-generated animation, I don't know. I'm going to break it to you here at the end here. I'm actually just, yeah, a Sora demo exactly. All right. Well, thanks and cheers. Cool.