Software Huddle - Nile, Racked Hardware, Quantum Computing, Getting Customers Trust, & more

Episode Date: October 31, 2023

Our special episode is back, and it's all about the latest news. Join Sean and Alex for an in-depth discussion. Timestamps: 00:00 Introduction  02:55 Tech Adoption in Japan  06:36 Infobip  09:34 Product Marketing at Rockset 14:38 Trust from your initial customers and early adopters 20:01 Nile - Serverless Postgres for modern SaaS 29:29 AI Models Can Now Selectively Forget 36:46 Oxide’s Racked Hardware 45:03 Quantum Computing Follow Alex: https://twitter.com/alexbdebrie Follow Sean: https://twitter.com/seanfalconer

Hello, and welcome to Software Huddle. I'm here with my co-host, Alex Debris. Alex, how are you? Sean, I'm doing well. Good to see you. Good to catch up. Excited to hear what you've been up to. Yeah, you too. So I guess we haven't really figured out what to call these episodes yet, but this is essentially our special format episode where Alex and I kind of get together, chat about recent shows, what's happening in the world of tech, things that we're interested in. And if you have any suggestions or questions for us, feel free to hit us up on Twitter at Software Huddle or on our individual accounts,

which is just our names, pretty easy to find. But maybe to start, so it's been about a month or so since we connected. At a time, you and I were both kind of setting off on international trips to speak at conferences and so on. So I think you were going to Japan. How was that trip? And how was the conference that you went to? Japan was a lot of fun. Yeah. It was very different from anywhere I've been. I've never

been into Asia or anything like that. So my wife and I went, we had a great time. We did some Osaka, Kyoto, and then Tokyo. Yeah, it was wild. Really great food and just like a fun culture and great to see some interesting culture and great to see some interesting things and and talk to interesting people i we had a great time there yeah i i loved i think i talked about this last time on the comp uh when we we chatted about how i attempted to move there because i enjoyed it so much but um yeah i would it's like i i like really really want to get back there i just need to kind of like figure out either the right work opportunity to go there or maybe, you know, when my kids get a little bit older,

some sort of family trip. Yeah, that's interesting that you wanted to move there. You know, you being from Canada, we met a ton of Canadians that had moved there. I don't know if there's like some sort of special draw there or if we just got the luck of the draw in sort of who we met. But yeah, there were a ton of, you know,

Canadians that had moved over to Japan there there yeah i don't know i mean i think uh a fair amount of people from canada end up like moving abroad at some point in their life like my my sister lived in china for a period of time now lives in ireland so she's lived abroad for a long time i technically live abroad but united states it's a little less different and exotic, but somewhat different than Canada, though. And then how was the conference itself? It was good, yeah. So it was a serverless days, Tokyo there, and it was nice to see that community sort of opening up

and embracing serverless. And I got to meet some or see some people that I worked with very early on serverless framework, and I've seen them over the years. And it was just fun to, you know, I've only met them here in the U.S., and it was fun to get to see them more in their element, their own culture, talking with people they know and just getting to meet them and see that. So that was a lot of fun. It was a great energy. A few other early Serverless folks were there chatting as well.

And then spoke at an AWS loft about dynamo and things like that so just it's fun to see just how they're adopting things over there what they're picking up what they're seeing a lot of cool ai stuff that people were were hacking on as well so it was fun is there like a like your from your experience there are those like companies operating in Japan kind of at a different stage of adoption when it comes to like cloud or AI than what we're seeing in the West? Yeah, I was kind of surprised by that just from talking with different people and sort of culture type stuff. One thing it sounds like is different is a lot of development is done by sort of external agency consultant, like integrators type thing, like Accenture type companies there. So they're saying, I think like someone I talked to said maybe like 30% of software

developers work directly for a company building their stuff, but probably 70% of them are working for basically consultants that get brought in to implement some specific project. He also said um or a lot of people were saying it can be just like a conservative culture on just like hey this has worked we we know this sort of things work one guy was even telling me that over there the government sort of drives innovation or the public sector drives the private sector where once the the you know if the if the public sector adopts cloud, then private sector is more like, okay, it's good enough for the government,

we should start implementing it now, rather than, I'd say we probably think the opposite here in the United States. So a little more just conservative technologically on some stuff and very relationship driven, which I think tends to use a lot of the same material. So I do a lot with Dynamo, NoSQL and stuff, and still relational databases and still like,

you know, relational databases are still very big here and everywhere still, you know, by far the dominant, but I'd say that's even more true in, in Japan. Like the NoSQL community was, was smaller and just earlier stage there.

Yeah. Yeah. I've, I think I've heard that about the Nordics as well, in terms of, you know, a lot of the actual like engineering goes on through like agencies or like partners and so forth um rather than like individual companies i also think

i was talking about this yesterday at a conference as that and i think in some countries i don't know about japan specifically but certainly in certain some company or countries in europe like also like what is considered successful is different different culturally than what you would consider success in the US. I think, especially I feel like in the Bay Area where I live, there's... Success is like some... Unicorn. Unicorn. Just ridiculous bar. It's like, are you Google or you failed, basically. And I think a lot of times success in other countries can be... Well, I see success as like, do I build a multi-generational company that's going to last, you know, hundreds of years and stuff like that.

It's less about like, you know, did I go public in six years and have like a hundred billion dollar market cap or something like that? Yeah. Yeah. I was surprised. Speaking of just like Nordics and the EU, I don't know if you saw that chart that was like around on Twitter recently, but it was showing the size of the US tech sector, the EU tech sector, and Europe without the EU. I was just shocked at how small the EU tech sector was. You can barely see it on this map. Even, I think, Europe outside the EU was bigger than the EU itself.

It really sort of shocked me on how big that difference was. Yeah. No, I haven't seen that, but it is interesting. Yeah. I think even from Canada, it's more... At least when I was living there, I found it's a more conservative sort of entrepreneurship market than the United States. It's a lot about how does your company make money in the first year and stuff. And there, it's, and there's some like, I think good things about that. Some value there certainly, but it also makes it harder to do some of these just like absolute like rocket ship, big bets that like transform an industry.

Yeah. Yeah, absolutely. But anyway, enough about Japan. I think you went to InfoBip in, was it Czech Republic? No, Croatia. Croatia. Sorry. How was that? No, Croatia. Croatia, sorry. How was that? It was awesome. I love that conference. I always highly recommend it.

Croatia is like, you know, like my new like favorite destination. Unfortunately this time, because I had a tight turnaround, I also went to London on that trip to speak at Big Data London. So I didn't get to spend a ton of time in Croatia. I basically landed at like three in the morning, like Sunday, and then had kind of like part of that day to chill out. And then the conference was Monday, Tuesday, and I flew out like Tuesday afternoon to go to London.

So it was kind of a whirlwind trip. It was my first time at Big Data London as well, which I thought was a really good conference. At both conferences, I spoke on generative AI and privacy, but they were both really well attended, got good questions, was engaging. So I really enjoyed it. We'll definitely go back to both conferences probably next year. Nice. Yeah. I saw a ton about that InfoBip one, just like a ton of people there. It looked like some really good talks and things. Are those out that that we can watch those yeah they're now on youtube actually maybe we can get that in the in the show notes but yeah they do i just like of all the conferences especially like developer focused conferences i've been to i think shift is like the best conference that i've

been to in terms of at least as a speaker like they just really take care of you like and they set up dinners um for speakers every night and bless you everywhere like you're just kind of like really taken care of and you uh uh and you really get to know the other speakers because you're all staying at the same hotel and you're having dinner together each night and doing this you know these different things so it becomes like a you know small community essentially and once you've been there a couple times you you know a bunch of people and then you'll see them at other events and stuff like that.

So it's fine. Yeah. I believe, I can't remember if we talked about this on the podcast before, but I believe your wife was going on a big trip and you were going to be alone with the kids for all. Has that happened yet? Like,

did you survive? Yeah. I've recovered. Yeah. My wife was almost three weeks in Africa, in Kenya. Um, for she,

she, uh, has worked with exotic animals for a long time has done a lot of work for rhino conservation who essentially was gifted a trip to africa to go visit one of the places that one of the conservatories that they have raised money for uh so it was like trip of a lifetime and then i pulled down the fort with two toddlers by myself for three weeks yeah both just dealing with wild animals yeah yeah it was a different type of exotic animal care. It was fun.

Like, you know, sleep was certainly impacted and I had to really be on it from a schedule perspective. But she's back now and things are, you know, settling in. Doing some more podcasting, which is good. That really is a little bit of pressure on my schedule. Nice. Yeah, you had some good ones that I'm excited about. I don't want to, like, which is, which is good. That really is a little bit of pressure on my schedule, but nice. Yeah. You've had some good ones that I'm excited about. I don't, I don't want to like, you know, we can eat upcoming guests, but I'm excited about some of the people you've

scheduled, uh, coming forward here. So it's going to be a good, good close of the year here. Yeah, absolutely. Yeah. And speaking of, uh, you know, podcasts, so you talked to your tire beetle rock set, uh, you know, over the last month or so, I really enjoyed, well, both of them, but the Rockset interview, I hadn't, you know, I knew of Rockset, but kind of just knew of it at like a high level. And actually, I really liked some of their like sort of product marketing material that they put together. Like they had this great, I still have it actually in my downloads folder. I pulled up once in a while, this great like PDF that shows all of these different,

like example use cases of using Rockset. And it's just showing like existing architecture, where's Rockset sort of fit into this world. And I actually used that a year ago as an inspiration for something that I did at Skyflow where I did that for Skyflow. And then that became essentially like our like solutions by Skyflow, like kickoff stuff that we so i i borrowed a lot of or inspired by a lot of ideas that they did from like a product marketing perspective but i really enjoyed um sort of how in the weeds that you guys got around like you know data structures uh for you know that like you know querying and

stuff and it's a lot of stuff that you, I hadn't really like thought about or studied, you know, since college. Um, but I, I, you know, I think that was actually like some of my first publications that I did in college was actually in sort of like, um, range queries and high dimensional search and stuff like that. So I ended up moving away from that. Uh, but I really, you know, enjoyed it at the time. And I think I could have gone in that direction at some point in my career. So it was fun to revisit some of that stuff. Yeah, thanks. I'm glad you liked it. I love Druba and just like that whole... Yeah, the product marketing team there. They do a really good job of explaining what's going on. And I think it's... I don't know. Other people, let me know if you like it. But I just like to, with the databases specifically, like dig in and figure out, you know, what's new, what's different between these sorts of things.

Because I think it's really just like about understanding when this can work for your use case. We really are kind of spoiled right now and just like all the different options we have. And it's nice to, you know, really understand like, hey, is this a general purpose thing? Is this more specific? And if it's specific, like what's the specific thing we should use it for? So, um, yeah, I love that one. Tiger beetle. Also, that's what, that was like a, uh, a wild ride. Cause like you're on and some of the things they're, they're doing there is, is pretty fun. So yeah, I love yours as well. Like that, the Twilio one on, on developer education. I mean, they've sort of been

like the model for dev education for a long time. i think probably one of the earliest to do it well um in this space so it was fun to have someone from that team on and just talking about you know what they've learned how they approach this stuff you know the twilio quest game and all those sorts of things yeah yeah and i think like i feel like twilio's in some ways like being forced to maybe i don't think they're abandoning it but they're kind of moving away from some of that stuff to go like a market to enterprise and i think it's like a natural thing that happens with a lot of companies as they get you know to a certain size like stripe's been trying to make that leap as well it's really hard i think to go from sort of like a community-led developer first uh like chronic life growth self-serve, like flow to enterprise.

They're just kind of like two opposing motions. Just different motions, different things. Yeah. Yeah. I mean, even, you know, at Google, like some of the tension that I think around Google cloud is that it feels like a different company because essentially it's like a B2B sales. Yeah. Like motion and partner led motion which is just very different than like where google started it was just like consumer um

you know b2c and i think anybody that was from the b2c world that moves in the cloud they're kind of like ah like this doesn't feel like google anymore um and that's why they ended up bringing in like a lot of microsoft oracle people because i think the people who like grew up in google just didn't even know how to do a b2b um and so it's like a cultural clash and it's it's a hard hard job to make yeah yeah absolutely yeah it's it's fun to it's interesting to see that and just see the different changes we see in in developer marketing education as well yeah but i think like some of the stuff that you're doing on the database stuff like i think i you know the i think especially in like the sort of online tech world we get so wrapped up in like javascript front end web world because they're kind of like easy to demo in a lot of ways

it's like more accessible than doing like a back-end demo like you know you're going to demo like your new database you run a sql query like okay thank you yeah yeah demo. Like, you know, you're going to demo like your new database, you run a SQL query, like, okay, thank you. Why is this, you know, more impressive than what I saw 30 years ago against an Oracle database, right? But people kind of forget sometimes that there's all these other types of like programming where like having a foundational knowledge of data structures and algorithms is like super important.

And it just, I don't feel like it gets enough attention in sort of the zeitgeist a lot of times because maybe it doesn't feel sexy for some reason. Yeah. Yeah. Yeah. I agree. It's like, it's, it's hard to demo, like you have to set up so much context and then just like, again, show the query or what, like, how do you do that? Another thing that's, I think is, is tough. And I've tried to ask a lot of these people about is just like, how do you get the trust from your initial customers and sort of early adopters? Cause it's not like a JavaScript framework or something like that. It's like you're holding their essential data,

which you don't want to lose it. You also want to corrupt it and have this giant mess of stuff. Like how do you get people get trust from that? And tiger beetle especially is interesting cause they're focused on like financial transactions specifically. They have an API built for just financial transactions and it's like wow now it's like extremely high value data you cannot screw up like how can you accelerate that process to where you're starting to make money rather than you know building this thing for four years before you can even get an initial

client so i think that's been like an interesting challenge for a lot of them just talking through that stuff yeah we went through kind of a similar journey at Skyflow because, you know, we're building a data privacy platform. And like, you know, it's kind of hard to do like the MVP of that. Like, it took two years of R&D to build like the initial product before you can even go to market. So it's kind of like a big, you know, vision leap of faith thing. And, you know, I think what we've had to do from like a go-to-market was initially land some like smaller customers um and on like really sort of more of a specific like use case of like point solution and then from there you build up some trust build up some logos some you know uh credibility and that allowed us to move up market into like bigger companies bigger

deals once we've kind of proven ourselves like you have to earn the right essentially to like sell into like the ibms of the world and it takes a little bit of time yep for that initial those initial two years like how big was that team and was there a lot of like uh you know was there funding rounds and a lot of funding money sloshing around or was it pretty tight and like hey hey, we really got to figure this out? Like, what did you know what those first two years looked like? So they did an initial seed round, which, you know, like seed rounds now are like, I don't know, like 17 million. Yeah.

So they so they had they were well funded. And I think a big part of that is like our co-founder and ceo was this is like not the first company he's founded he was also an investor and like you know executive at salesforce and stuff so he's like a well-known quantity in the valley like he knows everybody like his linkedin network is basically linkedin um so yeah um so like it you know that helps a lot right uh with kind of like being able to paint the vision and get people to buy into it and then they did a series a um towards sort of the end of that like two-year cycle but they were able to basically use the seed round and it was all it was almost

it's basically purely product and engineering they had like no like sales and marketing other than like founder-led stuff um yeah and it was like when they raised the a round that was when they started uh like trying to build up sort of like a marketing sales motion and then they raised a b round like later that year as we started to like step on the gas so yeah i think that's so hard that like early stage stuff where how are you sure you're building the right thing especially if it's something that takes a long time to mature and bake like that and then and then at times I've seen like if you pour too much money on early, like now a lot of that urgency is gone,

especially like not by the founding team, but maybe like the sort of other folks there where it's like, hey, we have a bunch of runway. We can sort of get away with this stuff. I think like maintaining that discipline is really hard. I think you just get kind of flabby sometimes. But yeah, I mean, it is like I, you know, I was talking to my old co-founder

for the company that i founded a few days ago and one of the things we were talking about was how you know we did our best work when basically we had like no money or we were running out of money because it's like you know it really forces you to focus and we did like our dumbest decision making whenever we were fresh off like a round of funding and stuff like that, because it's like, Oh, well we have like cash in the bank so we can like try some crazy stuff. And, uh, there is, it does take like,

I think a lot of discipline and you have to have like, like constantly, uh, like a sense of urgency. And actually they, um, so the founder and CEO of nucleus, which I had on a podcast,

uh, a couple of months ago, Elvis Chernova, he actually just came out with this. He posted a long article in a sub stack yesterday about how Nucleus is pivoting and all sort of the decision making that went into that. It's really good. I sent him a message saying that we'll have to have him come back on and talk about startup pivots because it's really interesting.

But one of the things he was talking about was like so they've they've uh actually generated i think they they were at like 150 000 or 250 000 dollars of revenue after six months of like go to market which is not bad for like early stage stuff for sure but yeah i think the really hard decision making around a pivot is when things are kind of working but it's like is it working well enough uh like if it's clearly not working then it's like okay well we need to go back to the drawing board and then if it's clearly working it's like okay well let's just step on the gas but when it's kind of like in that gray area that's like when it's really hard to make a decision about is this the right thing or not um so he goes through his whole thought process of how they came to that decision. And it was like, he basically wanted to share that journey

so that other people, other startup founders kind of going through that can learn from it. That's cool. We'll have to link that in the show notes. That's a tough decision right there. So kudos to him for sharing that. Yeah, it's really hard. So, all right.

So let's kick over to some other news. So I know you wanted to talk about Nile, the service Postgres for the modern SaaS. Yeah. I mean, I think this week was interesting. We were recording this on October 27th in that there were two sort of launch announcements that I call like smart people doing interesting things. So like smart people that you knew about before had done some interesting work before, and now they have like this new sort of ambitious startup. So one is, is Nile, which is a new post-growth startup. This has Gwen Shapira, who's like very big in the Kafka world. It was at, I believe she was at LinkedIn and then,

and then was at Confluent for a long time and did like a ton of education there, as well as like the former VP of Eng at Confluent. So they sort of went off last year and said they were doing something new. It wasn't really clear what it was. But it's basically this managed Postgres offering, which there's a bunch of those in the market. They're claiming serverless.

They haven't really released pricing yet, so I don't know. Connection management, a lot of just management-type issues. But one thing that's interesting about this is there's like application level features that are helpful. And basically it's like a tenant aware database, right? So if you're doing a B2B SaaS and you have all these different tenants, I think one worry is just like, man, are you jamming that all in one database? And if so, how do you make sure you're not sharing one tenant's data with another? Or maybe you have GDPR or other regional requirements. How do you handle that stuff? And what they're claiming is like, hey, we built in tenant awareness at the database level.

So now I believe I'm looking at it super closely in the technical stuff. But it looks like each tenant is in a separate table, probably in a separate Postgres schema altogether. And it's like totally separate. You can have them share tables if you want and things like that. But basically, you know, they're totally separate to where you you're not going to have that issue.

It's sort of like insulating you from that data leakage across tenant issues. So I thought that was like very interesting in that it's not just like a scalability or operational feature, but it's actually like

an application level feature that they're building in to make this easier for you. Yeah, I thought it was pretty interesting. And I think like every time I think like we've done everything there is to do with Postgres, there's like some new company. You're more well-versed in the world than me, but why do you think there's so much continued innovation

like in the space? Like, is there room for so many players to be successful? Or do you think there's so much continued innovation in the space? Is there room for so many players to be successful? Or do you think it's like, we're going to end up with some convergence and this is going to be a couple of power players at some point? That's a good question. I actually want to write up some how I see the post-grad space. I was thinking about this the other day. What are the different axes you can classify companies on? And also when I talked to Sam Lambert at PlanetScale, which is like basically the MySQL version of a lot of these Postgres things,

we talked about how it was interesting. There's like a bajillion Postgres startups and then there's like one MySQL startup and it's PlanetScale. And I was like, why is that? And he said, partly it's the community that is really big around Postgres and has been good. They also have a pretty good extensions system and API under

that to where you can extend Postgres pretty well. So that's why I think Timescale is like a time series database built on Postgres. And it's very easy to just sort of build that as an extension and hook into like most of what Postgres has to offer without having to sort of rebuild all the storage subsystem and all that stuff. But I mean, that's sort of true. But then you also see Postgres databases like YugoByte, which I believe is Postgres compatible, but not using Postgres under the hood. Or Neon, I think it rewrote the Postgres storage APIs and things like that. So I don't know.

It is kind of interesting. It's not just the extensions. It's not just the community. It's, I don't know, it's something of interesting it's not just the extensions it's not just the community it's uh i don't know it's something else to where we'll see that and yeah my assumption is we'll have to see some consolidation there but right now it's interesting to see those differences and you know you have like the the ones that are hyperscalers you know like uh you could buy maybe like amazon aurora postgres compatible, maybe neon, something like that. But then you also have these ones

that are competing on application level features like Nile, like Timescale, Supabase is in there as well. So it's just like a very interesting ecosystem right now, I think. So you mentioned extensions. We actually, like at Skyflow, we built on Postgres as well as like part of our underpinnings of the vault.

And part of that is we can use the extensions to build in. We use it for, you know, our polymorphic data encryption technology, which allows you to run like fully encrypted queries. So that wouldn't be really possible with a lot of other sort of database technologies. So I get that, that makes a lot of sense. But I think it's like hard to,

like if you're a founder and you're trying to figure out what makes sense for me, or even a new project, there's so much stuff to try to navigate. And I guess that's true of everything in tech at this point. But it's like, how do you even make that decision? And are you shooting yourself in the foot at one point? I talked to a founder... Or not founder, a CTO of a company recently that had been at this company for 13 years and had essentially been there from the time that it was a single web instance with the database running on the same server to now this huge modern with where they're doing like a combination of, you know, buying servers from,

uh, from, uh, like, uh, like basically on-prem services to, you know, back up through auto scaling on AWS and all this sort of stuff.

And, um, the, you know, they, they made a choice at one point to move from, you know, like a SQL database to like Mongo.

And then that was like a utter failure for like what they were doing because they were trying to do all this like know, like a SQL database to like Mongo. And then that was like an utter failure for like what they were doing because they were trying to do all this like analytical operation. And it turned out like, oh, we actually need like joins. And yeah. So then they had to migrate back essentially. And they eventually ended up like on RDS. So like you can like really hurt yourself if you make a wrong choice. Yeah. Oh yeah, absolutely. Yeah. I know that's, it is tough in the database world because you do get people that just want to glom onto the hype or the excitement

or what's new. I think in the last five years we've seen database marketing get a little better about this is what we're good at and this is where we fit. We were talking about with Rockset. I think it's getting better at that and hopefully driving better decisions,

but we'll make all kinds of bad decisions along the way as well. That's one of the things I liked about the interview on Roxette as well was he was very clear about like, hey, this is what we're good at. And if you're trying to do this other thing, we're probably not like a fit.

And I think you need to kind of do that. You need to like draw a line in the sand and be like, this is who we are. And that way it's like clear what someone's like signing up for. So one of the things i really liked about the nile article too and actually sent this to our ceo as like an example of like i think well-written thought leadership was they did a really good job of kind of like outlining like why managing and deploying like multi-tenant databases is like a really hard problem to solve and like all the things that

go into that if you're trying to take this on yourself so it's like very clear if you were someone who's like building a b2b sas company and thinking about doing this you're like oh man like maybe this is like a bad idea because like this is gonna be a huge headache for me but you know you know based on like your sort of expertise like can you talk a little bit maybe about the like complexity involved with with like managing and isolating tenants have you had any experience with that i mean yeah so most of the stuff i've done is with dynamo and like dynamo is going to be um you're pretty prescriptive on like how you actually have to query your data it's not going to be very flexible for you so it's kind

of it would be like fairly rare for someone to, you know, a new dev to come in and write a SQL query that doesn't have a where clause with the tenant ID in it. And therefore like grabbing someone else's data, like Dynamo partition keys basically help you a lot. I think people that are doing multi-tenancy on Dynamo use, you know, probably a tenant ID or something like that in their partition key to help

segment their data. But yeah, I just think the worry is like you have this, you know, a filter that basically needs to be applied to every single query to make sure you're only filtering within a tenant and make sure you're not co-mingling that way. And so you have that sort of issue. Also, if you're putting it all in the same database, like are you providing some sort of thing where you want performance isolation from each other, right? If you have one, like, very heavy client, and especially if you allow, like, exports or something like that that can be database intensive, like, is that going to affect the other people in your database?

And then sometimes people do separate database for each tenant, but now you have, like, a huge management issue. You have to have, like, a pretty high dollar value per tenant in order to make something like that feasible and worth it. And you have, like, a you know low utilization databases essentially so there's just like a lot of difficulties on like keeping your data separate performance implications all all sorts of things that make it pretty tricky yeah they talked about they they got into some of the details on some like the performances like basically like you know if one of these, you know, one customer essentially is like exploding and usage, like what does that do to impact like your other people, the other customers and, you know, or like blowing up of users and all this.

And then of course, all the like security things that you have to deal with. So I highly recommend taking a read just to kind of like understand a little bit about like the complexities of why you probably don't want to do this yourself. Yeah. Yeah. Yeah. Cool. So one of the articles I want to talk about was there was this article came out earlier this month from AI business about AI models can now selectively forget data after training and one of the what they're talking about so there's

this big problem around all the stuff that's happening in like the world of lms is that i think we talked a little bit about this last time is essentially like models are designed to learn not unworked so in the world of that we live in now with like gdpr and essentially the right to be forgotten like how if you know if you have user information that you've used to train your model with like how can you essentially delete that information and the challenge is like the models ai models basically aren't designed to delete like it's a little bit like me saying uh like delete the part of your brain that knows what an apple is forget this thing yeah it's like forget this content like head trauma or something like that's hard to do yeah it's a big challenge

around like security privacy copyright issues and so microsoft's been doing this article talks about some of the research from from microsoft and they were able to make llama to forget its knowledge about harry potter uh through this technique which i thought was it kind of reminded me of the movie uh yesterday if you're familiar with that where no i that, where the world forgets who the Beatles are. And it takes you through the implications of what that would mean. And here they're just basically making the world forget, or at least this AI model forget, what Harry Potter is. And the way they do it is through a process of fine-tuning. So they're trying to essentially target the distinct things that the model knows, in this case about Harry Potter, by replacing it with more generic information. So that way, when you do a prompt, rather than get a really specific response, like Dumbledore dies on page X of, you know,

book six, you know, spoiler alert, it'll give you like a really like generic response instead. And there's, there's now like a lot of research in this space, you know, but essentially the, the limitation of the approach is that a lot of these approaches basically

depend on their like internal knowledge of the model like you have to have access to like the parameters and unless you own the model you wouldn't necessarily have that and also fine tuning is like really expensive so it's not something like you could easily do and scale yeah i was kind of curious because i was looking at examples of that and it was good i mean at least theoretically on not being able to answer the har Potter stuff. I wonder if it had any effect on other areas at all or if it really was confined. If you would ask about, I don't know, other fantasy stuff or about England or even if you would ask about J.K. Rowling at all. Right. Like, does it remember anything about her? Does it remember like, you know, she's written a few other books.

Right. Does it know about those or is she just like completely wiped out of the, that AI is universe there. Um, also they don't talk about how much training it to like how expensive that was. And you're talking about knowing the internals, but also like how expensive is that? Um, sort of thing. It's, it's very curious. I'll just be curious how effective that is. Um, yeah, I, I honestly, I don't think, I think the like research has a long way to go

before this become i think there'll probably be like another solution essentially that becomes the more viable solution because it's just not super realistic to me that you'd be able to do this in production like if you had to do this for every user that like requested deletion besides like just the like computation cycles but like all the hand crafting that like really goes into it like it'd be so expensive to actually like do this and then there was a follow-up research from harvard uh university where there's this paper they um published called in context unlearning language models as few shot unlearners and there what they're trying to do is instead of using fine tuning they're using essentially like a bunch of prompt entering

engineering to create what they call a forget point targeting so they're trying to like remove things through uh prompts to try to influence the first cycle but it's not 100 by any means and it's not super easy to set up but i think like essentially the the like only solution viable solution right now is to make sure that you're not leaking you know sensitive data into a model and you have to essentially just like put up your guard rails to prevent that from happening yeah i'll i'll be curious like how much better we or how good we get at just like these incremental changes to the models like the forgetting type stuff but also just like incremental updating, right? Because there is this knowledge cutoff

where if you have a newer API or like, you know, React and Next have like server actions and things like that now, which didn't exist back when chat GPT was, or when GPT-3 and 4 were created. So how are they going to like get those types of examples in there? I'd say GitHub copilot like sort of has it figured out like i'll i'll be typing in github copilot and like sometimes i'll be like

writing some prose and it'll know things that happened in the last year um so it's just like somehow it seems like they're figuring stuff out well i think what i don't know 100 um on how like the underlying structure of GitHub Copilot. JetBrains is coming out with something similar as well. It's going to be integrated into IntelliJ. But a lot of the way that people are approaching this to have essentially more up-to-date information is they're creating a RAG model. So they combine information retrieval with essentially the LLM.

So they're going to go and take the, in the code example, essentially they're going to take your code base. They probably take other examples of code and they create, they basically vectorize it, create vector embedding, store it in a vector database. So then when you're putting in essentially a prompt, they vectorize that,

they run that against the vector database, doing a similarity search to pull back the relevant document materials and then use that as essentially additional context to send to the LLM. And that way, if the answer to your prompt exists within the vector store, then you're getting essentially a better response. It also helps reduce the chance of like hallucinations and uh and you have like more you basically get like more in context uh responses and stuff so that's kind

of like the workaround that people are doing right now and it's also low cost compared to do it trying to do something like fine-tuning yeah sure yep yeah it's interesting i'll be curious to see how this uh how this all works out. There's a lot of unsolved problems here, even though it's so integral to our workflows. Yeah, and I'm doing an upcoming show, not to tease it too much, but with someone from Snowflake that we're just going to be talking a little

bit about how do we productionize this stuff? Because it's really easy to build a demo right now. You can basically tie some stuff together in a few hours or a few days and create something that like looks absolutely magical compared to what you could have did five years ago but actually like moving that to production is not like that thought out there like you know there's like real real challenges to like move that thing to like actually real users and doing it at scale yeah yeah absolutely so all right so why don't we talk

uh the cloud computer okay so similar like on that same theme of of smart people doing interesting things oxide computer company this is um i believe jesse frizell and brian cantrell are the two that that started i can't tell if jesse's still affiliated because he also started a CAD company, AutoCAD, but better, I think, in the last year. But a couple years ago, they were like, hey, we're building a better on-prem rack, basically, type thing. So Brian Cantrell, super famous for my son, Joyant, and Jesse was at Docker and just big in the container

space but they're like hey we're gonna do this they they just came out and released it this week as well and it's like this this fully assembled rack that that comes and it's it's um it's called this fancy stuff i don't know have you ever have you ever racked hardware i've never racked hardware so i'm not that familiar with this stuff no i mean yeah my like closest thing would have been like in early stages my career like when i was in university uh you know back then like your server was in the closet um and you know but like the uh i i didn't i was generally not on the hardware side of that although i could you know see the closet there but and then from there i you know i spent time in university by the time i was at a university doing my company things had moved to the cloud at that point so yeah yeah i basically

started on heroku and aws right away so the closest i've gotten is like a raspberry pi but uh yeah it's like interesting and and it's interesting um you know we've seen a lot from like especially dhh um talking about, the cloud's too expensive. You should be able to run your own stuff. And now, you know, Oxide saying, hey, you should be able to have a cloud like experience to buy, not just to rent is one thing they mentioned in their blog post. And so it's like an integrated hardware and software solution where it's this rack. It's basically fully assembled. You don't see all these cables everywhere. And also there's like a nice UI for managing it, right? So if you

want to be deploying instances or VMs or containers or whatever you want to do, it's not going to be exactly like the AWS experience, but it's some sort of thing like that to help you manage it where you're not just running some know running some raw uh linux instance on it and sort of managing everything on top of it so like i'll be curious to see if it works and like what the market is for this sort of thing i mean i imagine there's still a lot of use cases for that uh sort of thing it's it's you sort of have the database problem of again of like making sure you're building up that trust like how do you get those early customers that can trust this if it's like some very critical stuff that they're not even trusting in the cloud for whatever reason but um yeah it'll be interesting to see

and like i wouldn't bet against brian and jesse so um it's a good team yeah i mean i think that's probably part of the value that they have is like they're like you know somewhat famous like proven people so that helps with like the trust issue but i I think like my big, I don't know, like question I had from like reading the articles, like, like who's this for? Like who's sort of like their target market? Cause like, if you're like a young startup, but you're probably, I don't know.

I like, unless you have a really compelling reason, like it doesn't make a lot of sense to be like, oh, we're going to buy our own hardware and run our own like, you know, CloudRacks and stuff like that. But what they're doing in principle, I think makes a lot of sense to be like, oh, we're going to buy our own hardware and run our own like, you know, CloudRocks and stuff like that. But what they're doing in principle, I think makes a lot of sense. It's like, just like, hey, let's like think about this sort of from first principles and like build the cloud computer as they as they phrase it. It kind of like how we see some of the stuff in the database world of like, hey, let's build a database that That's like cloud first, rather than try to move something that was not designed for the

cloud to the cloud. Um, so I, so I get sort of the like first principles way of thinking. I was just like, I'm not sure like who the market is. I'm sure there is a market like, cause even, um, I mentioned the CTO earlier that had been at the company for like 15 years and started with like a single server. And, you know, one of the things that they did to keep their public cloud costs down was eventually they do a ton of caching

at sort of the edge. So they have like local, they want to do local caching. So if you're in Brazil, you want to hit like the Brazil cache essentially. And then, you know, they might have a centralized database somewhere else. But they knew that sort knew what the baseline amount of scale they needed to serve their baseline use case.

So it was cheaper for them to essentially rent servers from places close to where their users are and then run their cache there. And then they fall back to AWS for like auto scaling when they have spikes. So, you know, if they're doing that and this was less expensive, then maybe it makes sense for them to cut costs further

by essentially like actually buying the hardware and running it directly there. I don't know. Yep. Yeah, that's interesting. And I also like, I'm not familiar with this space at all, but there's still a bunch of data centers

and co-os and things around, and I wonder what that experience is like, especially the user experience on that. And maybe they're selling to those people and doing this, and now it's like, hey, you have this nice hardware system and a nice industry standard

UI on top of that across all these different co-op facilities or something like that i'm not sure um yeah it'll be i'll be curious to see it's also interesting to see like you know we just had this period of it seemed like every startup was was just like a purely software b2b business type play and like i think people are realizing realizing that's a competitive non-defensible space because it's just like hey if you can build it like a lot of people can just build it and it's basically just like um you know sales and marketing and stuff like that at some point whereas like you're seeing a little bit more like looking into hard hard tech again whether that's ai

whether that's i feel like there's been an increase in like defense tech type stuff or like the, who's that airplane company for the guy from Amazon, like Blake, Blake Schnell or somebody, he's like making some super fast air, but he's like the Concorde again, like a fast airplane type, like a passenger airplane that can be very fast. Yeah. Maybe the Concorde was like ahead of his time. Yeah, exactly. Like something sort of like that. And he's saying, Hey, you can go from New York to Tokyo in like two hours or of like that and he's saying hey you can go from uh new

york to tokyo in like two hours or something like that you know well i would think like now with a lot of the like you know um you know platforms as a service infrastructure as a service like all these different things it's like lowered the barrier to entry to build a lot of these like b2b sas products which makes a lot of products that historically maybe were hard to build and they had kind of had a moat around them become more of a commodity thing. So more and more types of software become commoditized. So then it ends up being hard to have a competitive edge, or you need to really go niche, or it's

a race to the bottom in terms of who prices the lowest. So then that's maybe transforming the industry to some degree of people who want to really like push the envelope and be on the edge of tech to go see like what is the next like hard problems to solve like i'm really excited about everything that's happening like biotech right now like i i only spent a year like in bioinformatics formally when i was doing my postdoc and much more on the computer science side of it so i you know i'm like very much stepping outside of my like depth like biology but i just think all the stuff that's going on like the potential of generative ai applied to biotech for like drug discovery and stuff is just a completely transformed like that world and

there's so much like we talk about big data but like big data is like not even a like a reasonable term when you're talking about biological data because like even though like human genome project back in the 90s is like probably like petabytes of data so they there's not even a way in biola as a biologist to like wrap your head around the amount of information there like they basically need they can't like continue to innovate the space i think without leveraging tools like ai so if we can get things that allow them to be able to like really like you know take this state like go from essentially like accidental discovery of drugs to something being more like a design project is um it's going to really change things yeah for sure and and this is a great segue speaking

of like hard tech problems and all those sorts of things your your article you had an article on quantum quantum communications which i'm excited about because i know nothing about quantum computing i didn't need you to explain to me but maybe like tell me about the article first and i'm gonna ask you dumb questions yeah so speaking of dipping inside of our, you know, I technically have, uh, I think three public papers published in the quantum computing world, um, from like 15 years ago, but the, um, the, so there was this article that came out, uh, about the university of York. So they sent quantum information from Ireland to the UK. So across the Irish city, basically.

And it actually in 2021, so that's 224 kilometers in 2021 toshiba sent qubits 660 kilometers but it wasn't across like the sea basically so there's a bunch of stuff around this so i reached out to a friend of mine that is an expert in in quantum computing that i actually co-authored those papers with and um i won't i didn't tell him i was going to talk about this so i won't you know mention his name or anything i'm sure someone could figure it out but anyway so i was like you know like how like legit is this this is like important because i feel like the three areas that get the most like overhyped and like blown up in the news in terms of technology is like um robotics ai and quantum computing because

i think they're just like these like far out, like hard to understand like concepts where you get these like sensational headlines because they lead to people like clicking on the articles. But like, how like real is this? Is this an important thing? And his main feedback was, one is like, you know,

the articles don't talk about like, what's the actual bit rate? So if the bit rate's really low, it's not necessarily very impressive because it doesn't, it's not going to necessarily have impact in terms of like really fast communications. So it's more like they're, you know, showing as possible across these long distances, but it might not actually be like a practical thing for it. But his bigger concern is, uh, they talked about, and I read up on this a little bit too, is that there's not really like a good, that great an application for, uh, essentially sharing quantum information like this. Like the only sort of known practical application for this is, uh, for quantum key distribution distribution which is essentially like if you're doing quantum encryption like how do you do the equivalent of like a public private key uh distribution but the the thing there is that you could also just use like diffy

helman uh key exchange which has been around for forever which we use for like you know tls and ssh and all this sort of stuff like there's basically a purely classical standard hardware way of doing key exchange that would work also for quantum encryption that is like much cheaper and exists today than necessarily doing uh quantum key exchange so that was kind of like the main thing that i was talking about there but i was just interested in talking about it just because i think that there has been a lot of like chatter around quantum computing the last few years like there was um an article a couple years ago from from uh google that talked about sort of uh make like uh uh the first like experiment to show that uh like quantum computing was able to do something that like like a classical computer wasn't able to do and what is that like okay so i am mostly out of the loop on quantum computing other than i hear people talk about

and get excited but like what how will it change things like what's the big deal there basically the idea is that is so you're you're basically have um through quantum entanglement you can essentially make like two uh like basically a particle, two particles that are maybe even separated by distance, be sort of linked to each other. And once you measure the result of one, you essentially get the result of the other. So you can send the information potentially across the asses. The other thing is that the qubit can be in multiple states at the same time. So because it can be in multiple states at the same time, in theory, you can have a lot less qubits to represent a massive amount of information at once.

So that means that if you can do it right, then you can solve some really hard problems like prime factorization, which is a big part of a lot of standard encryption methods for key generation. If you can basically solve those hard problems that using a quantum computer then it would break some things like standard encryption methods so that's like so basically like if you just take encryption as an example they're based on the problem of um like there's this hard problem that a classical computer would take too many compute cycles to ever be able to solve like prime factorization as an example.

And this is the whole notion of what's known as provably secure, which essentially means that it's been mathematically proven to be secure because there's a provably hard problem that would require being solved in order to crack the problem. So anything

that is equivalent to an NP-hard not a non-deterministically polynomial hard problem is a hard problem to solve for a classical computer because we have no way of doing it fast that we know of anyway. But in the belief is that quantum computers are not a standard, essentially Turing machine. They represent a different type of computer that solves a different class of problems than a conventional computer. So you have like, in the compute space, you have, you know, basically polynomial P problems, you have MP problems, non deterministically polynomial problems. And we do. And there's like, most people believe that they're not the same. They're basically a

different set of problems. And there's like a big reward if you could prove that NP equals P or doesn't equal P. But most people believe that NP is different than P. And then in the quantum world, essentially, there's a belief that they can solve problems. Some essentially NP complete problems could be solved by quantum computers. But no one's actually proven that to be the case, but it's kind of generally established that it could be the case. So the class of problems that are easy to solve with a quantum computer are different than the set of problems that can be easily solved with a classical computer.

So what that means is for certain types of problems, suddenly you can solve those problems in polynomial time that you couldn't have done before. Interesting. And then so how far out do you think quantum computers are from being usable? And will they be used

for a sort of narrow subset of problems? Or will they be driving all of our... I guess how do they be like driving all all of our i guess like how do how do they integrate with what we're yeah it'll be i think for a narrow set of problems it'd be like for specific problems one like they're expensive um and like you probably i think even if we had like quantum computers you'd probably use like a classic computer to like interface with it so it's not like you know only the six richest kings in the world will have a quantum computer or something like that. But the other thing is like,

they're designed, I mean, they sort of solve, at least today, in theory, solve like very specific problems. So it's not like you're going to be running like, I don't know, your Unreal Server off of a quantum computer. Like it's just not the same thing, right? So, but I think, you know, according to, you know, my friend, i asked him

about this a while back too i think they're pretty far away from being around there's just so much error involved with it like that's the big challenge is like it's like you're dealing with things on the quantum level so any tiny little mistake is just blown up massively it's like if you draw a line on a sheet of paper it can kind of look straight but if you extrapolated that line to like from the earth to the moon you would see it's like very off in terms of not being straight but that's basically what you're getting at the quantum level is like any kind of mistake is like magnified uh massively like yeah it's it's it's like so if you incorrectly it's like you know having a hand the size of like a

planet and trying to flip a quarter like you could essentially like explode the quarter if you didn't use the right amount of like pressure to do it so that that's basically what you're doing at the quantum level so the fault tolerance is super high and that's where you can't really like no one's been able to use these and like like outside of like sort of like research type problems but there has been companies that um in biotech space back to that have been trying to experiment with using quantum computing in combination with like generative ai and stuff like that to do innovative stuff but i think they still run into this challenge of just like the tech's just not like there.

Like we're probably years away. I think we're closer to having like solve general AI where it's like equivalent to human level intelligence than we probably are to having like a quantum computer that we can use on a, to solve like real concrete problems. Yeah. Well, maybe AGI will create us a quantum computer. Yeah,

there we go. Actually, when we get the super intelligent, super, when we go past AGI, we get to super intelligent, when we go past AGI, we get to super intelligence, then they'll figure it out for us. And they'll just like, you know, bust out the quantum computer.

Nice. Well, I'm glad I don't have to think about it too much. It's not going to change how I write stuff day to day, at least for a little while yet. Yeah, it's probably not going to affect, you know, the next book you write.

All right. So, all right. So that's the news. What's coming up next book you write or yeah all right so um all right so that's the news what's coming up next for you next uh so i've reinvent in a month so that'll be uh focus there you'll be there nice be good to be good to catch up um so yeah i kind of like that it's kind of like a nice bookend to the year i feel like you get back from that and people are are kind of casual into the holiday season and in the next year so yeah um looking forward to that what about you got any any big things coming up uh so i mean i was um at a couple conferences this week spoke on some panels there's actually one panel that i did

for um that was kind of on uh the what does like general ai mean for developers and stuff like that that we might actually be turning into a podcast because there's just like so much to talk about we didn't have time to actually cover that much

in the in the panel so we might go long form version of that at some point so people can you know be on the lookout for that but I have reinvent coming up as well and then I I have

I've been on a it basically no travel through October and then you know travel towards the end of november and then i have a couple weeks of travel we're gonna i think i'm gonna be in paris for a conference in the first of december and then uh new orleans for for something as well and then kind of settle down for the holidays you keep a busy schedule you're all over the place yeah yeah that's good that's what's coming up for me. Nice. Well,

again, I'm looking forward to some of the podcasts you have scheduled as well. I know these guests are going to be great. So, um, yeah, it should be good. Awesome.

All right. Well, until next time, uh, thanks everybody up there that, uh, listening to this again, if you want,

if you have questions, suggestions, anything like that, feel free to reach out to us at software huddle or, or you find either of us on, you know, LinkedIn and Twitter. Uh, we're on, you know, LinkedIn and Twitter. We're there,

you know, by our names. Pretty easy. Yeah. Sounds great. Good to talk to you, Sean. Bye.

Yeah.