Latent Space: The AI Engineer Podcast - Why StackOverflow usage is down 50% — with David Hsu of Retool

Episode Date: February 1, 2024

We are announcing the second edition of our Latent Space demo day event in SF on 2/23: Final Frontiers, a startup and research competition in “The Autonomous Workforce”, ​”Beyond Transformers & GPUs”, and “​Embodied AI”. RSVP here! The first one was aimed for 15-20 people and ended up blowing up to >200 and covered in the Information - let’s see what a year of growth (and competition) does to the local events space in 2024.You can find all Latent Space events here, and of course get in touch with us to host your own AI Engineer meetups like AI Engineering Singapore.In our December 2023 recap we covered the Four Wars of the AI stack. But how do we know when it’s time to crown a winner? As we kick off 2024, we wanted to do a recap of the State of AI in 2023 to set a baseline of adoption for different products. Retool had a great report at the end of last year which covered a lot of it. David Hsu, CEO and co-founder of Retool, joined us to go over it together. We also talked about the history of Retool, why they were too embarrassed to present at YC demo day, and how they got to $1M ARR with 3 employees. If you’re a founder, there are a lot of nuggets of advice in here!Retool AIIn our modeling of the “Software 3.0 Stack”, we have generally left a pretty wide open gap as to the “user interface” equivalent of the AI stack:Retool AI launched 4 months ago with some nifty features for SQL generation, and its own hosted vector storage service (using pgvector). However, as he explains on the pod, the more interesting potential of Retool is in helping developers build AI infused applications quickly, in combination with its Workflows feature. This moves Retool down the stack from just the UI for internal tooling to the business logic “piping” as well. There are a bunch of dedicated tools in this space like Respell, BuildShip, Flowise, and Ironclad Rivet."We think that practically every internal app is going to be AI infused over the next three years." - David on the podRIP StackOverflow?In July 2023 we talked about the impact of ChatGPT and Copilot:This was then disputed by StackOverflow, who pointed out (very fairly so) that there were privacy-related changes in their analytics instrumentation in 2022. StackOverflow no longer reports traffic, but based on StackOverflow’s continuing transparency we can see that organic declines have continued throughout 2023.Retool’s report comes over a year after those changes and has some self reported samples from users:* 57.6% of people said they have used StackOverflow less; almost all of them replaced it with ChatGPT and Copilot.* 10.2% said they no longer use StackOverflow.We also saw a lot more tools being released in the dev tools space such as (one of our oldest pod friends) Codeium (which just raised a $65M Series B), SourceGraph (and their newly released Cody), Codium AI (just released AlphaCodium which was picked up by Karpathy), Phind (which beat GPT-4 with OSS models), and Cursor, one of the most beloved products in the dev community at the moment. Intelligence is getting closer and closer to the IDE, and the trend doesn’t seem to be reverting. We already said that “You are not too old (to pivot into AI)“, and the advice still stands. When asked to rate “Preference for hiring engineers effective at using ChatGPT/Copilot for coding” on a scale of 1 to 10, where 10 is “Much more likely”, ~40% of companies voted 8-10. Having an AI Engineer skillset is extremely important. 45% of companies between 1,000-4,999 employees said that they increased the difficulty of technical interviews to compensate for these new tools, so the gap between users and non-users will keep widening.Crossing the AI in Production ChasmGeoffrey Moore’s “Crossing the Chasm” is one of the most quoted business frameworks. Every market has an initial group of Innovators and Early Adopters, who are willing to suffer through the rough edges of products initially, and eventually crosses into the Early Majority, which expects a full product.In the AI world, ChatGPT and Midjourney / DALL-E have crossed the chasm in the consumer space. Copilot is probably the only tool that did it in the enterprise, having crossed 1M paid users. ~$50B were invested in AI in 2023, and we still only have

Hey everyone. Welcome to the Latinspace podcast. This is Alessio, partner, MCTO and residents at Decibel Partners, and I'm joined by my co-host, Swix, founder of Small A-I-I-I. And today we are in the studio with David Sue from Reto. Welcome. Thanks. Excited to be here. We like to give a little bit of intro from what little we can get about you and then have you talk about something personal. You got your degree in philosophy in CS from Oxford. I wasn't aware that they did double degrees. Is that what you got? It's actually a single degree. which is really cool. So basically, you study context,

you study philosophy, and you studied the intersection. The intersection is basically AI, actually, and sort of computers think, our computers be smart. What does it mean for computer to be smart? As well as logic, it's also another intersection,

which is really fun too. In Stanford, it might be symbolic systems or whatever. And it's always hard to classify these things when we don't really have a word for it. Now, I guess everything's just called AI. Five years ago, you launched a retool. You were in YC, Winter 17,

and just being a straight line up from there, right? I wish. What's something now your LinkedIn that people should know about you, maybe on their personal hobby or, you know, it's just something you're very passionate about. Yeah, sure. I read quite a bit. I probably read like two books a week around about,

so it's a lot of fun. I love biking. It's also quite a bit of fun, so yeah. Do you use Retool to read? Like, what the hell? No, I don't use Retail to read. That'd be funny. What do you read?

How do you choose what you read? Any recommendations? I'm mostly reading fiction nowadays. fiction is a lot of fun. I think it maybe helps to be more empathetic, if you will. I think it's a lot of fun actually to sort of see what it's like to be in someone else's shoes. So that's a lot of fun. I've really good about philosophy as well. I find philosophy just so interesting, especially logic. We can talk more about that for probably hours if you want. Yeah, I have a casual interest in epistemology, and I think that anytime you try to think

about machine learning on a philosophical angle, you have to start wrestling with these very fundamental questions about how do you know what you know? Yeah, totally. What does it mean to know? That's its own podcast. We should do a special edition about it, but that's a, that's fun. Let's just maybe jump through a couple things on Retool that I found out, well, researching your background. You did YC, but you didn't present a demo day initially because you were too embarrassed of what you have built. Can you maybe give any learnings to like founders on like jumping back from that? I've seen a lot of people kind of like give up early on because they were like, oh, this isn't really what I thought it was going to be to be a founder.

They told me I would go to YC and then present and then raise a bunch of money and then everything was going to be easy. So how did that influence us, so how you build retail today, you know, in terms of like picking ideas, deciding when to give up on it. Yeah, let's see. So this is around about 2017 or so. So we were supposed to present at the March demo day. But then we basically felt like we had nothing really going on in no traction, we have no customers. And so we're like, okay, well, why don't we take six months to go find all that before presenting? part of that, to be honest,

was I think there's a lot of doys around Demo Day, around startups in general, especially because there's so many startups nowadays. And I guess for me, I'd always want to sort of underpromise and over-deliver, if you will. And in Demo Day, I mean, maybe you two have seen a lot of the videos, like, it's a lot of

honestly, over-promising and after delivery. Because every startup, you know, says, oh, you know, I'm going to be the next Google or something, and then you keep you're under, and you're like, well, nothing's going on here, basically. So I really didn't want that. And so we chose actually not to present the demo day, mostly because we felt like we didn't have anything substantial underneath. Although, actually, a few other founders in our patch probably would have, you know, chosen to present in a situation. But we were just, you know, kind of embarrassed about it.

And so we basically took six months to just say, okay, well, how do we get customers? And we're not presenting it until we have a product that we're proud of and customers that we're proud of. And fortunately, it worked out. You know, since then later, we did have that. So I don't know if there's much to learn from the situation besides I think social values. was something that I personally had never really been that interested in. And so it was definitely hard because it's almost like you go to college and all your friends are graduating when you failed or something.

You failed to final and you have to redo it here. It's like, well, it kind of sucks that all your friends are up there and on the podium, you know, presenting and they are raising a ton of money and you're kind of being left behind. But in our case, we felt like it was a choice. We couldn't present it if we really wanted to, but, you know, we would not have been proud of the outcome or proud of what we were presenting. And for us, it was more important to be true to ourselves if you will and show something we're actually proud of rather than raise some money and then shut the company down into two years.

Yeah. Yeah. Any Sam Allman stories from the YC days? Could you tell in 2017 that Sam was going to become, like, run the biggest AI company in the world? Wow. No one has asked me that before. Let me think.

Sam was, I think he was, I want to, I forgot, I already know you president of YC, it is, in our bash, we actually weren't in his group, actually, at the very beginning. And then we got moved to a different group. I think Sam was clearly very ambitious when we first met him. I think he was very helpful and sort of wanted to help founders. But besides that, I mean, I think we were so overwhelmed by the fact that we had to go build a startup. We were not, you know, honestly paying much attention to everyone's support or taking notes on them. That makes sense.

Well, and then just to wrap some of the Retool History Nuggets, you raised a series A when you were a $1 million of revenue, with only three or four people? How did you make that happen? Any learnings on keeping team small? I think there's a lot of over hiring we've seen over the last few years. I think a lot of AI startups now are raising very large rounds and maybe don't know what to do with the capitals. So this is kind of similar, actually, from sort of why,

which is not a present a demo day. And the reason was it feels like a lot of people are really playing startup. I think PG has an SAF office, which is like, you're almost like playing house or something like that. It's like, oh, well, I hear that at a startup. We're supposed to raise money and then hire people. And so therefore, you go and do that. and you're supposed to do a lot of PR because that's what, you know, startup founders do it.

And so you could do a lot of PR and stuff like that. And for us, we always thought that the point of starting a startup is basically you have to create value for customers. If you're not creating value for customers, like everything else is going to, you know, nothing's going to work, basically. You can't, you know, continue to raise money or hire people. If you don't have customers, you know, have to lower value. And so for us, we were always very focused on that. And so that's initially where we started. I think it's, again, maybe goes to like the sort of presenting something truthful by yourself or staying true to yourself.

It's something to that effect, which is we didn't want to protect. which is we didn't want to pretend like we had a thriving business we could actually. And so the only way to not pretend was actually to build a thriving business. And so we basically just put our heads down and grind it away for probably a year, a half or so, just writing code, talking to customers. And I think that at that point, we had raised something like maybe a million dollars, maybe a million and a half coming out of YC. So, I mean, to us, you know, that was a huge amount of money.

I was like, wow, like, how are you ever going to spend a million and a half? Our runway was like, you know, five, six years at that point, right? Because we're paying ourselves 30, 40K here. And so then the question was not like, oh, we're going to run on run a runways. The question was like, we better find traction because if we don't find traction, we're to just give up psychologically. Because if you run an idea for four years, nothing happens, you're probably psychologically going to give up.

I think that's actually true in most startups, actually. Most startups die in the early stages, not because you run out of money, but really because you run out of motivation. And for us, had we hired people, I think it would have I spent, harder for us because we want to rent out a motivation faster. Because when you're pretty part of market fit, actually, trying to lead the team with like, 10 people, for example, to Marshall's product market fit, I think it's actually pretty hard. Like, it's, you know, everyday people are asking you, so why are we doing this?

And you're like, I don't know, man. Like, hey, trust this. And that's actually a very tiring environment to be in. Whereas it's just like, you know, the founders figuring out product market fit. I think obviously what sort of safer path, if you will. You're also schooling less with employees. Like, when you're hired employees, you'll have an idea. You have a front of market.

of customers, that's actually, I think, a lot more stable for place for employees to join as thought. Yeah, I find that typically the sort of founder-employee relationship is employee expects the founder just tell them what to do, and you don't really get critical pushback from the employee, even if they're about it and if they like you as an early engineer, it's very much like the role play of like once you have that founder head on, you think differently, you act differently, and you're more scrappy, I guess, in trying to figure out what that product is. Yeah, I really resonate with this because I'm going through this right now.

Awesome. One thing we did actually early on that I think has paid a lot of dividends, especially which was a lot larger now, is we hired a lot of former founders. So I want to say like when we were 20, 30, 40 people who were probably like half former founders at each one of those stages. And that was actually pretty cool because I think you infused sort of a, you know, get things done kind of culture, a outcome-oriented culture of like a very little politics because, you know, no one came from larger companies.

Everyone was just like, this is my own startup, and we go figure out how to shoot the bicycle for the customer. And so I think from cultural perspective, even today, a lot of those cultures totally very soft startgery. I think it's actually because of sort of these early founders that we hired, which was really, really, really, you know, we're really lucky to have them. Yeah.

And then closing off on just a little bit of the fundraising stuff, something notable that you did was when in 2021, when it was the sort of peak ZERP and everyone was raising hundreds and hundreds of millions of dollars, you intentionally raised less money at lower valuations, it's your title. And I think it's a testament to your just overall general philosophy and building retool that you're just very efficient and you do things from first principles. Any updates on like, would you still endorse that? Would you recommend that to everyone else? What are you feelings sort of two years on from that? Yeah. So exactly that is correct. We raised less money

and a lower valuation. And I think the funny thing about this is that when we first announced that, even internally and both externally, I think people were really surprised, actually, because I I think Silicon Valley has been conditioned to think, oh, raising a giant sum of money at a giant valuation is a really good thing. So, like, you should maximize both the numbers, basically. But actually, maximizing both the numbers is actually really bad, actually, for the people that matter the most, you know, i.e., your employees are your team.

And the reason for that is raising more money means more dilutions. If you look at a company like, you know, let's see Uber, for example, if you join Uber at like, I don't know, like a $10 billion valuation, or let's say, joined before their huge route, which I think happened up a few billion dollars in that valuation, they've just got downloaded a ton when Uber fundraises. So if Uber raises, if Uber downloads themselves by 10%, for example, that rates $5.50 to $5 billion, for example,

every employee's stake goes down by 10% in terms of ownership. Same with previous investors, same with the founders, et cetera. And so if you look at actually a lot of founders in sort of, you know, the operation statistics space or those that fundraise like, you know, 2013-2017, a lot of the founders by IPO only have a few percentage points, actually, every company. And if the founders only have a few percentage points, you can imagine how we look at employees happen. So that I think is actually just the really, you know, backing for employees

overall. Secondly, sort of higher valuation, given the same company quality, is always worse. So basically what that means is if you are fundraising as a company, you can command a sort of valuation in the market. Let's say it's, you know, X, for example. Maybe even lucky and you can raise two times X, for example. But if you choose two times X, your company itself has not fundamentally changed. It's just that, you know, for some reason investors want to pay more for it. You know, maybe today you're an AI company, for example. And so investors aren't really signed with AI. the paper for it. However, that might not be true in a year or two years time, actually. And if that's not true in two years' time, then you're in big trouble, actually. And so now I think

you see a lot of companies that raise a really high valuation of 2021. And now they're like, man, we're at like 100x or, you know, we raised 300x multiple, for example. And for our 300 X then, maybe now we're out like 200x. Man, we just can't raise money ever again. Like, you know, we have to grow like 50x to go raise money, reasonable value, I'd say. And so I think that is really challenging and really team motivating for the team. And so I think, I think, lower valuation, actually, it's much better. And so, if we're also retrospect, you'd answer to answer the question,

two years later, we did not predict the crash, if you will, but given it, I think, was done extremely well, mostly because our valuation is not sky high. Because the value we're sky, we're sky high, I think we'd have a lot more problems. We'd probably have recruiting problems, for example, we'd probably have a lot

of internal problems, et cetera. People be like, you know, why is it on which way? We might have to go raise money again, et cetera, but we can't because the relation's too high. So, I would urge, I I think you founders today to quote-unquote, like leave money on the table. Like, there are some things that are not really worth optimizing. I think you should optimize for the quality of the company that you build,

not like the value. You raise that or the amount you raise, et cetera. Highside 2020, but it looks like, you know, you made the right call there anyway. Maybe we should also for people who are not clued into Retool, do a quick, like, what is Retool? You know, I see you as the kings or the inventors of the low-code internal tooling category. Would you agree with that statement? How do you usually explain retail?

I generally say it's like legless for code. We actually hate the low code moniker. In fact, we have docs saying we will never use it internally or even to customers. And the reason for that is I think low code sounds very dot development. And developers, they hear the phrase low code are like, oh, that's not for me. I love writing code. Like, why would I ever want to write less good? And so for us, Retro's actually built for developers.

Like 95% of our customers actually are developers, actually. And so that is a little bit surprising to people. I'll generally explain it is, this is kind of a funny joke, too. I think part of the reason why Reefel has been successful is that developers hate building internal tools. And you can probably see why. I mean, if you're a developer,

you probably build internal tools yourself. It's not a super exciting thing to do. It's like the pie together of QUD UI, probably pieces together many QUD UI's in your life before. And there's a lot of crunch work involved. It's like, hey, state management, it's like data validation. It's like display error messes, like to bounce the buttons.

Like, all these things are not really exciting. But you have to do it. because it's so important for your business to have high quality internal software. And so what Rachel does is basically allows you to sort of piece together an internal app really fast, whether it's a front end, whether it's a back end or whatever else. So yeah, that's what your tool is. Yeah, actually, you started hiring. And so I do a lot of developer relations and community building work.

And you hired Critica, who is now who's not going to open the eye, to start out your sort of dev rel function. And I was like, what is Retool doing according developers? And then she told me about this developer traction. And I think that is the first thing that people should know, which is that the burden and weight of internal tooling often falls to developers, or it's an Excel sheet somewhere, whatever. But yeah, you guys have it basically creates this market. In my mind, I don't know if there was someone clearly before you in this, but you know, you've clearly taken over and dominated every month. There's a new YC startup launching with that it's like, you know, we're the open source retool. We're like the lower code retool, whatever.

And it's pretty, I guess it's endearing. You know, we'll talk about airplane later on. But yeah, I think I've actually used Retool, you know, in my previous startup for this exact purpose. Like we needed a UI for EWS RDS that they can, you know, like the rest of our non-less technical people, like our sales operations people could interact with. And yeah, Retool is perfect for that. Yeah, that's a good example of like that's an application that an engineer probably does not want to build. Like building an Apple top of Salesforce or something that is not exciting.

It's always a fair sucks. It's really limited. It's like not a fun experience at all. But piece you together at Retool. is quite a bit easier. So, yeah, let me know if you had in feedback, but also thanks for using it. Yeah, no, of course. More recently, I think about three, four months ago, you launched Retool AI.

Obviously, AI has been sort of in the air. I'd love for you to tell the journey of AI products ideation within Retool. Given that you have a degree in this thing, I'm sure you're not new to this. But like, when would you consider sort of this, the start of the AI product thinking in Retool? So we actually had a joke internally at Retool. We are part of roadmap for every year. I think it was like 2019 or something. We had this joke, which was like,

what are we going to build this year? We're going to build AI programming. As we always said, this is a joke. But it was funny because we were like, that's never going to happen. I'm like, I don't know if you know, because it's like a buzzwordy thing that enterprises love. So let's look at it.

And so it was almost like a funny thing, basically. But it turns out, you know, we're actually building that now. So it is pretty cool. So I would say maybe AI thinking on a mutual problem with first started, maybe like, I don't say maybe a year and a half ago, something like that. And when we first started thinking about

a sort of in a philosophical way, if you will, it's like, well, what is the purpose of AI? And how can it help, you know, what retail does? And there were two sort of main problems, if you will, value what we thought. One was helping people build apps faster. And so, you know, you've probably seen their copilot, you've seen sort of so many other coding assistants,

piece of order to stuff like that. So that's interesting because, you know, engineers, as we talked about, do some grunt work. And grunt work, you know, maybe could be automated by AI was sort of the idea. And it's interesting, so we actually, I would say, kind of proved or disprove the hub a little bit. If you talk to most engineers today, like a lot of engineers do use copilot, but if you ask

them like how much time is co-pilot save you, it's not like coding this 10x faster than before. You know, coding is maybe like 10% faster, maybe 20% faster, or something like that, basically. And so it's not like a huge step change, actually. And the reason for that is we think is because the sort of fundamental frameworks and languages have not changed. And so if you're building, let's say, you know, like the sales ops, what we're talking about before, for example. Let's say you've got AI to generate, you know, a first version of that, for example. The problem is that it probably generated it for you in, like, JavaScript,

because you're writing for the web browser, for example, right? And then for you to actually go proofread that JavaScript, for you to go read the JavaScript to make sure it's working, you know, to fix the subtle bugs that AI might have caused, hallucinations, stuff like that, actually takes a long time and a lot of work. And so for us, the problem is actually not, like, the process of coding itself. It is more sort of the language or the framework, I think it's like way too low level. It's kind of like anything about like punched cards. Like let's say back in the deprogram,

who designed punched cards, and AI could help you generate punched cards. Okay, you know, I guess that helps me. Punching cards is a little bit faster about because I'm a machine punching them for me. But when there's a bug, I'll have to go read all the punched cards and figure out what's wrong, right?

It's like a lot of work, actually. And so for us, that was the sort of initial idea was, can we help engineers code faster? You know, I think it's somewhat helpful to be clear. Like, again, I think it's 10 or 20%. So we have things like, you know, you can generate school queries by AI

and stuff like that. So that's cool, to be clear. But it's not, I think, the step change. That I think is, you know, the, we're investing someone with that. But the bulk of investment actually is the number two, which is helping developers build AI-enabled applications faster. And the reason why we think this is so exciting is we think that practically every app, every internal app especially, is going to be AI infused over the next, like, three years.

And so every tool you might imagine, so like the tool you were even mentioned, like a sales operations tool, for example. Probably, you know, if you were to build today, you want to incorporate a sub form of AI. And so, you know, we see today, like for us, like a lot of people build, you know, I'll say sales manager tools and retool. An example is there's a fortune,

like five-mage company who's building like sales forecasting tools. So they basically have sales people enter their forecast, you know, for the quarter, at the beginning of a recorder, like, hey, I have these deals and these deals are going to close. These deals are not going to close. You know, I think up upside in these, downside of these, stuff like that.

So you can imagine it's pulling in deals from your sales force database. And so it pulls of the deals, and then actually use AI compute, like, okay, well, you know, given previous deal dynamics, like, these are the deals that are more likely to close this month versus next month versus this quarter, next quarter, et cetera. And so it could actually, you know, pre-write you a draft of, you know, your report, basically.

And so that's an example where I think all apps, whether it's, you know, a sales app, you know, until it looks like a fraud app, a, you know, fintech app, you know, whatever it is, basically, especially internal apps, I think, like you said, a lesseo, in order to make you a whole productive, it's going to incorporate some form of AI. And so the other question is, can we help them incorporate this AI faster? And so that's why we launched like a vector database for example, built directly into Retool. That's why we, you know, launches all these AI actions.

You don't have to, you know, go figure out what the best model is and do testing and stuff like that. Which, you know, give it to you out of the box. So for us, I think that is really the really exciting futures. Can we make every app, let's retool, use AI a little bit and make people a little productive? We talk with Jeffrey Wang, who's the co-founder and chief architect of Amplitude. He mentioned they just use Postgreft's vector. When you were building Retail vectors, how do you think about, yeah, leverage a startup to do it, putting vectors into one of the accessing data stores that you already had.

I think, like, you're really quite large customer scale. So, like, you're maybe not trying to get too cute with it. Any learnings and tips from that? Yeah, I think a general in the philosophical thing I think we believe is we think the open source movement in AI, especially when it comes to all the supporting infrastructure, is going to win. And the reason for that is we look at, like, developer tools in general, especially such a fast-moving space. In the end,

like, there are really smart people in the world that have really good ideas. And they're going to go build companies, and they're going to go build projects, basically, around these ideas. And so, for us, we have always wanted to partner with maybe more open-source providers

or projects, you could say, like PG-Factor, for example. And the reason for that is, it's easy for us to see what's going on under the hood. A lot of this stuff is moving very fast. All times there are bugs, actually, and so we can go look into fix bugs ourselves,

so we can concentrate back for example. But we really think, open source is going to win in the space. It's hard to see about models. I don't know about models necessarily because it starts going to be pretty complicated there. But when it comes to tooling, for sure, I think there's just like so much there's an explosion of

creativity, if you will. And I think betting on any one commercial company is pretty risky, but betting on the open source sort of community and the open source contributors, I think is a pretty good bet. So that's why we have a picture back. Awesome. And we're going to jump into the survey next, but we're going to put a bunch of links in the show notes

about RISO AI and whatnot. Is there any most underrated? feature, like something that customers maybe love that you didn't expect them to really care about. I know you have like text to SQL. You have UI generation. There's like so many things in there. Yeah.

What's surprised you? Yeah. So what's really cool? And this is my sense of the AI space overall. You know, if you're a skin, you take some YouTube as well, is that especially in Silicon Valley where one of the innovation is happening, I think there's actually not that many AI use cases, to be honest. And AI, to me, even as of, what, like January 19th, 19th, the 2024, still feels like in search of truly good use cases.

And what's really interesting, though, about Retool, and I think we're in a really fortunate position is that we have this large base of sort of customers. And a lot of these customers are actually a bunch more legacy, if you will, customers. And a lot of them actually have a lot of use cases for AI. And so to us, I think we're almost in like a really perfect or unique spot, we're able to adopt some. technologies, and provide them to some of these older players. So one example that actually was really shocked and surprised me about AI was, so we have this one clothing manufacturer. I think it's either the first or second one is a clothing manufacturer in the world

who's using retail. And a ginormous company with their three multinational stores on pretty every model in the world. And so they have one problem, which is they need to design styles every year for the next year, basically, for every season. So like, hey, just like summer 2024, for example, and where we're going to design. And so what they used to do before is they were higher designers. And designers would go to study data.

They'd be like, okay, well, it looks like, you know, floral patterns were really hot in like, you know, California, for example, in 2023, and like, do I think it's going to be hot in 2024? Well, let me think about it. I don't know. So let me, maybe, and if so, if I believe it, it's going to be hot, let me go design some floral patterns, actually. And what they ended up doing in Retool, actually,

is they actually automated a lot of this process of weight in Reto. So they actually now built a Retail app that allows actually a non-designer, like an analyst, if you will, to analyze, like, you know, who were the hottest selling patterns, you know, particular geos. Like, this was really hot in Brazil, it was really hot of China. It's really hot, you know, somewhere else, basically. And then they actually feed it into an AI, and the AI, you know, actually generates with Dali and other, you know, in this generation APIs, actually generates patterns with them.

And they put the patterns, which is really cool. And so that's an example of, like, honestly, the use case I would have never thought about. Like, thinking about, like, you know, how clothing manufacturers create their next line of clothing, you know, for the next season. Like, I don't know. I never thought about it to be a big time. And the door than I ever think, you know, how it would actually happen. And the fact that they're able to leverage AI, they actually, you know, leverage multiple things in retail to make that happen.

It's really, really, really cool. And so that's an example where I think if you go deeper into sort of, if you go outside of Silicon Valley, there are actually a lot of use cases for AI. But a lot is not obvious. Like, you have to get into the businesses themselves. And so I think we personally are in a really fortunate place. But if, you know, you're working in the AI space and want to find some use cases, please come talk to us.

We're really excited about marrying technology with use cases, which I think is really hard to do right now. So it's bad. Yeah, I have a bunch of like sort of standing presentations around like how this industry is developing. And like I think the foundation model layer is understood. The lag chain vector DB reg layers understood.

I always have a big question mark and actually have you and VSEL V0 in that box, which is like sort of the UI layer for AI. and you know, you are perfectly placed to expose those functionalities to end users, even if you personally don't really know what they're going to use it for, and sometimes they'll surprise you with their creativity. One segment of this, and I do see some startups springing up to do this, is related to something that you also build, but it's not strictly AI related, which is Retool Workflows, which is the sort of canvassy boxes and arrows,

point and click, do this, then do that type of thing, which every, what are we calling low code? Every internal tooling company eventually builds. I worked at a sort of workflow orchestration company before, and we were also discussing internally how to make that happen. But you are obviously very well positioned to that. Basically, do you think that there is an overlap between retail workflows and AI? I think that, you know, there's a lot of interest in sort of chaining AI steps together.

I couldn't tell if that is already enabled within retail workflows. I don't think so, but you could sort of hook them together and exactly. Like, what's the interest there? You know, is it all of the kind ultimately in your mind? It is 100% on time. And yes, you could actually already, so a lot of people actually are building AI workflows down and retail, which is we can talk about it in a second. But a hot take here is actually, I think a lot of the utility in AI today, I would probably

argue 60, 70% of the utility, like, you know, businesses I found in AI. It's mostly five chat ShpT and across the world too. And the reason for that is, I think, the chat GPT is sort of, of the UI, you could say, or interface, or your user experience is just really quite good. You know, you can sort of converse, you know, with an AI, basically. But that's said, there are downsides to it. If you talk to, like, you know, a giant company,

like a JPMorgan Chase, you know, for example, they may be reticent to have people copy-based data into TechGPT, for example, even on JCP, for example. Some problems are that I think chat is good for one-off texts. So if you're like, hey, I want a first version of representation or something like that, you know, and help me write this version of the version of, Doc or something like that. China is great for that. It's a great, you know, very portable, you know,

if you will, form factors, so you can do that. However, if you think about it, you think about some economic productivity more generally, like, chat, again, will help you like 10 or 20 percent, but it's unlikely that you're going to replace an employee with chat. You know, you're not going to be like, oh, I have a relationship manager at JP Morgan Chase and I've replaced them with a, you know, a chat bot. Like, it's kind of hard to imagine, right? Because, like, the employees actually doing a lot of things besides, you know, generating, you know, maybe the other way of putting it is like, chat is like a reactive interface. Like, it's like when you have an issue, you'll go reach out to chat and chat might solve it.

But like, chat is not to solve 100% your problems. It'll solve like, you know, 25% of your problems, like, pretty quickly, right? And so what we think the next, like, big breakthrough in AI is, is actually like automation. It's not just like, oh, I have a problem when they go to a chat bond and solve it. Because, like, again, like, people don't spend 40 hours a weekend a chat on. They spend maybe like two hours a weekend a chat on, for example. And so what we think can be really big, actually, is you're able to automate an entire processes by because then they're really realizing the potential of AI.

It's not just like, you know, a human copy-pacing data into an AI chat bot, but you know, pasting it back out or copied back out. Instead, it's like the whole process now was actually done in an automated fashion without the human. And that, I think, is what's going to really unlock sort of big kind of productivity or that's what we're really excited about. And I think part of the problem right now is, you know, I'm sure you all found a lot of agents instead.

The agents are actually quite hard because like, you know, the AI's wrong, you know, 2% of the time, but then you like, you know, a few, let's say, you know, raise of the power seven, for example. It's actually wrong, you know, quite often for example. And so what we've actually done with workflows is we prefer, we've learned actually, is that we don't want to generate the whole workflow for you

via AI. Instead, what we want you to do, actually, is what are you to actually sort of drag and drop a workflow yourself, maybe you can get a V-0 or something by AI, but it's coded, basically. You should actually be able to modify the steps yourself, but every step can't use AI. And so what that means is like, it's not the whole workflow is created by AI.

Every step is AI automated. And so as you go back to, for example, like the users are talking about, you know, with a coding manufacturer, that's actually a workflow actually. So basically what they say is, hey, every day, we each see all the data, you know, from our sales systems into our database,

and then we, you know, do some data analysis. And, you know, that's just, you know, raw SQL basically, it's nothing too surprising. And then they use AI to generate into ideas. And then the analysts will look on the new ideas to prove or reject them, basically. And that is like a, you know, that's true automation. You know, it's not just like, you know,

when designer copy, pasting things is that a chat should be like, hey, it can be a design. It's actually, designs are being generated. It's generally 10,000 designs every day, and then you have to go and approve or reject these designs, which I think is a lot, you know, that's a lot more economically productive than just copy-pacing stuff to chat quickly. So we think sort of the AI workflow space is a really exciting space, and I think that is the next step in sort of delivering a lot of business value.

I personally don't think it's, you know, hire chat or, you know, I have had agents quite yet. That's a pretty reasonable take. It's disconcerting because I know a lot of people trying to build what you already have in workflows. You're the incumbent in their minds. I'm sure it doesn't feel that way to you. But I'm sure, you know, you're the incumbent in their minds and they're like, okay, like, how do I, you know, compete with Retool or, you know, differentiate from Retool. As you mentioned, you know, all these connections.

It does remind me that you're running up against Zapier. You're running up against maybe Notion in the distant future. And yeah, I think there will be a lot of different takes at this space and whoever is best positioned to serve their customer in the way that they need to shape is going to win. Do you have a philosophy against around what you won't build? What do you prefer to partner and not build in-house? Because I feel like you build a lot in-house.

Yes, this is probably two philosophical days. So one is that we're developer first. And I think that's actually one big differentiated parent like Austin's accurate. You know, if you know, we're very rare, we'll see them actually. The reason is we're developer first. Because developers, like, if you're like building a, a sales ops tool. You're probably not considering a notion if you're a developer. You're probably like, I'm going to

build this by React, basically, or a user tool. And so, are you we build for developers? It's pretty interesting, actually. I think one huge advantage of some of the developers is that developers don't want to be given an end solution. They want to be given the building blocks to themselves go build the unsolution. And so for us, like,

interesting point that equilibrium we'll get to is basically you can say, hey, retail's a consulting company, and we basically build apps for everybody, for example. And what's interesting is we've actually never gone to that equilibrium, And the reason for that is with some of the developers. Developers don't want, you know, like a consultant coming in and building all the apps for them. Developers are like, hey, I want to do it with myself.

Just give me the building blocks. It's like, can be the best table library. It can be, you know, good state management. It can be easy way to query rest of APIs. And I was doing myself, basically. So we generally end up basically always building building blocks that are reusable by multiple customers. We have, I think, basically never built anything specific for one customer. So that's, you know, one thing is interesting.

A second thing is when it comes to sort of, you know, let's say, like in the AI space, we're going to build and we're not going to build, we basically think about whether it's a core competency or whether there is unique advantages to us building it or not. And so if we think about the workflows product, we think workforce actually is a pretty core competency for us. And I think the idea that we can build a developer first workflows automation engine,

I mean, I think after we released, you know, workflows through workflows, there have been a sort of few copycasts that are, I think, quite far behind, actually, be sort of missing a lot of, I think, more critical features. But, like, if you look at the space, it's like Zapier on one side,

and then maybe like air flow on the other. And so if Rital Workflow is actually is fairly differentiated, and so we're like, okay, we should go build that basically. This is the one else is going to build. So let's go build it. Whereas if you look at vectors, for example, you look at vectors, like, wow, there's a pretty thriving space already, you know, vector databases.

Does it make sense for us to go build our own? Like, what's the benefit? Like, not much. We should go partner with or go find technology off the shelf. Narcist's PGVet. And so for us, I think it's like, how much value is that for customers? Do we have a different take-on space?

Do we not? And every product that we've launched, We've had a different take on the space and the products that we don't have a different take, we just adopt what's off the show. Let's jump into the state of AI survey that you ran and maybe get some live updates. So you surveyed about 1,600 people last August, and AI were this busy like five years ago. And there were kind of like a lot of interesting nuggets and we'll just run through everything. The first one is more than half the people, 52% said that AI is overrated.

Are you seeing sentiment shift in your customers or like, the people that you talk to, like, as the months go by? Or do you still see a lot of people, yeah, that are not in Silicon Valley, maybe say, hey, this is maybe not as worth changing as you all made it sound to be? Yes, well, actually, we're on the survey again, actually, the next few months, so I can let you know, when it changes. It seems to me that it has settled down a bit in terms of one of the, maybe, like, I don't know, a little noise you could say.

Like, it seems like there's a little bit less noise than before. I think people are still trying to look for use cases. I'd say with August of last year, like the United States, yeah, and I think there are slightly more use cases, but still not substantially more. And I think as far as we can tell, one of the engineers surveys,

especially some of the comments that we saw, do feel like the companies are investing quite a bit in AI, and they're not sure where it's going to go yet, but they're like, it could be big, so I think we should keep on investing. I do think that, based on what we are hearing from customers, if we're not seeing what turns out like a year or something, then it will be more skepticism.

So I think there is like a, it is time down, if you will. You finally gave us. some numbers on Stackoverflow usage. I think that's been a Twitter meme for a while, whether or not Chad GBTGPT kills Stackoverflow. In the survey, 58 people said they used it less, and 94% of them said they use it less because of Co-Pallet and ChatGBT,

which, yeah, I think it kind of makes sense. I know Stack Overflow tried to pull a whole thing. It's like, no, the traffic is going down because we change the way we instrument our website, but I don't think anybody bought that. And then you add right after that expectation of job impact by function, And operations people, eight out of ten, basically, they think it's going to, AI is going to really impact their job.

Designers were the lowest one, 6.8 out of 10, but then all the examples you gave were designers of a job being impacted by AI. Do you think there's a bit of a dissonance maybe between the human perception, it's like, oh, my job is like, can possibly be automated. It's funny that the operations people are like, yeah, it makes sense. I wish I could automate myself, you know,

versus the designers or maybe they love their craft more. Yeah, I don't know if you have any thoughts on who will accept the first, you know, that they should just embrace the technology and change the way they work. Yeah, that's interesting. I think it's probably going to be engineering driven. I mean, I think you two are very well,

maybe you two even started some of this wave, sort of the AI engineer wave. I think the companies that adopt AI the best, it is going to be engineering driven, I think, rather than like operations driven or anything else. And the reason for that is, I didn't have a rise in this like profile with an AI engineer. Like, AI is very,

maybe it's kind of philosophical, like, AI is a tool in my head. Like, it is not a, in my head, I think we're actually pretty fondly, I don't see what happens. But AI is not like a, you know, thing that it's not like a black box, but like it does everything you want to do. The models that we have today require very specific prompting, for example, in order to get, like, you know, really good results. And the reason for that is, it's a tool that, you know, you can use it a specific way. It's not a good result for you, actually. It's not like by itself taking Java way.

Right. So I think actually, to adopt AI, it's probably going to be going to have to be engineering first, basically, where engineers are playing around with it, figuring out the limitations of the models, figuring out like, oh, maybe like using vectorized database is a lot better, for example. Maybe like prompting this particular way it's going to be a lot better, et cetera. And that's not the kind of stuff that I think of like an operations team is going to really be like experimenting with necessarily. I think it really has to be an engineering led. And then I think the question is, well, what are they just going to focus on first?

Like I think you're going to focus on design first or like operations first. And that I think is more of this decision. I think it's probably going to be more like, you know, the CEO, for example, says, hey, you know, we're having trouble skilling this one function. So, like, why don't we try using AI for that? And let's see what happens, for example. And so in our case, for example, we are really, we have a lot of support issues. So what I mean by that is we have a really, really high performance support team, but we get a lot of tickets. And the reason for that is, you know, we're a very dynamic product.

You can use it in so many different ways? And so we have a lot of questions for us, basically. And so we were looking at, well, you know, can we, for example, draft some replies and support tickets, you know, by AI, for example. can we allow us for agents to be, you know, hopefully, you know, a double as doubly productive as before, for example. And so I guess I would say it's like business needs driven, but then engineering driven after that. So like, you know, the business decides, okay,

well, this is where AI can be most applied. And then we assign the project to an engineer and the agent goes and figures it out. I honestly am not sure if like the operations, we're going to have much of a, like if they accept and rejected, I don't know if it's going to chase the outcome, if you will. Another interesting part was the importance of a high in hiring. 45% of companies said they made their interviews more difficult. In the engineering side,

made interviews more difficult to compensate for people using co-ballet and chat GPT. As I changed at Retool, like, have you, yeah, have you thought about it? I don't know how much you're still involved with engineering hiring, I get the company, but I'm curious how we're scaling the difficulty of interviews, even though the job is the same, right? So just because you're going to use AI, doesn't mean the interview should be harder, but I guess it makes sense.

Our sense basically of the survey, and this is true for what we believe to is, when we do engineering interviews, we are most assessing like critical thinking or thinking, you know, on the spot. And I guess, you know, when you're hiring the employee, you know, in the end, the job in place to be productive, which they use whatever tools they want to be productive.

So, you know, that's kind of our need to. However, we do think that, you know, if you're thinking about it for the first most of his way, if your only method of my coding is literally copy pasting off of Checkshad BT or like you know it's pressing tab and copilot

I think that would be concerning and so for that reason we still do want to test for like fundamentals understanding of Compsi now that said I think if you're able to use chatshapity or copilot let's say competently we do view that as a plus we don't view it as a minus but if you only use copilot and you

aren't able to reason about like you know how to write a four loop for example or how to write FISB, that would be highly problematic and so for us we do today is we're basically or a race is a hackpad actually. So it's, sorry, I guess there's no copilot there. You just want to see what they're doing or see what they're thinking. And we really want to test for thinking, basically.

But yeah, I mean, we ourselves apparently have embraced our copilot and we would encourage engineers to go over this copilot too. But we do want to test for understanding of what you're doing rather than just copy-based the other one. The other one was AI adoption rate. Only 27% are in production. Of that 27%, 66% are internal use cases. Shout out to Retool.

You know, do you? Do you have a mental model as to how people are going to make the jump from, like, using it internally to externally? Obviously, there's like all these different things like privacy. You know, if an internal tool hallucinates, that's fine because you're paying people to use it basically versus if it elucinates to your customer. There's a different bar. Because for you, if people build an internal tool with retool, there are external customers to you, you know? So I think you're on the flip side of it. Yeah, I think it's hard to say.

maybe a core ritual belief was actually the most software built in the world of the internal facing, actually, which actually may sound kind of surprising. Of course, you're hearing this, but effectively, like, you know, we all work in Silicon Valley, right? We all work at businesses, basically, that's all software as sort of a business. And that's why all the software engines that we hire basically work on external-facing software, which makes sense because we're software companies. But if you look at most companies in the world, most companies in the world are actually not software companies. If you look at the clothing benefactor that I was talking about,

they're not a software company. They don't sell software to make money. They sell clothing to make money. And most companies in the world are no software companies, actually. And so, mostly engineers the world in fact, don't work at Silicon Valley companies. They work outside of something about. They work on these sort of work traditional companies. Like if you look at the Fortune of 100, for example, probably like 20 of them are software

companies. You know, they're 480 of them are not software companies. And actually, they employ those software initiatives. And so most of the software engineers in the world and most of the co-engineering the world actually goes towards these internal physics applications. And so, for all the reasons you said there, like, I think hallucination matters less, for example, because they have someone checking the output and consumer, so hallucination is more okay, it's more acceptable as well. They actually unreliable because it's probabilistic, and that's also okay. So I think it's kind of hard to imagine AI being adopted in a consumer way without the consumer, like, opting in.

Like, chatty is very obviously a consumer, and the consumer is, like, you know, knows that it's chatty pete. it's you're using it. I don't know if it's going to make its weight like you're making at any time soon. Maybe for like, even for support, it's hard because if it hallucinates then, you know, it's actually quite bad for support if you're hallucinating. So it's, yeah, it's hard to say. I'm not sure. Yeah, I think a lot of people, like you said, we all build software.

So we expect that everybody else is building software for other people, but most people just want to use the software that we build on here. I think the last big bugget is like models breakdown. 80% of people you serve it just use open AI. Some might experiment with smaller models. Any insight from your experience at Reefatel, like building some of the AI features? Have you guys thought about using open source models? Have you thought about fine-tuning models for specific use cases?

Or have you just found JV4 to just be great at most tasks? Yes, two things. One is that from a data privacy, perspective. People are getting more, more okay with using a hosted model, like a GPT4, for example, especially because GPT4 or OpenAI often as we have enterprises who went to some companies already, because I think a lot of CIOs are just like, let's give this second house, like, you know, let's do something on Azure, for example, and, you know, let's make it available for a police disparate. So I do think there is more acceptance, if you will, today, feed data into GPT.

That's something of some sensitive data. People might not want to do so, like, you know, feeding in like earnings results data, you know, three days per year and that's earnings. like probably is a bad idea. I probably don't want your earning statement for you. There's still some challenges like that that I think actually with most models could actually help solve

like a lot of three. You don't want to come to... I think that could be excited. So let's maybe just one thought. The second thought is, I think OpenDen has been really quite smart with some of no pricing, and they've been pretty aggressive

of like, let's get, you know, let's create this model and sell it a pretty cheap price and to make it such that there's no reason for you use any other model. Just for like a strategy perspective I don't know if that's going to work. And the reason for that is you have really well-funded players,

like a Google or like a Facebook, for example, that are actually quite interested. Like, I think it's competing in startups, OpenA would win for sure. Like, at this point, Open AIA so far ahead from both a model and a pricing perspective, that like there is no reason for it to go, really, I think, in my opinion, at least, a startup model.

But if like, you know, Facebook is not going to give up on AI. Like, Facebook is investing a lot of the AI, in fact. And so, Capitia against a large thing company, it's making a model open source, I think that is challenging. Now, however, where we are right now is, I think, GPD4 is so far in terms of performance that, and I would say a model performance is so important right now because, like, the average, you can argue, Lama 2 is actually so far behind,

but, like, customers don't want to use Lama 2 because it's so far up right now. And so that item is part of the challenge. As AI progress slows down, so if we get like Lama 4 and 5, for example, maybe it's comparable at that point, like, GDP5 or GPD6, like, and make it to the point of it was like, look, I was going to use Lama. it's safer for me to host it on Prev. It's just as fast, just as cheap. Like, why not, basically?

But I think right now we are in this state. We're opening an XG really well, I think. And right now they're thriving, but let's see what happens in the next year or two. What are you going to ask differently for the next survey? Like, what info do you really actually want to know that's going to change your worldview? I'll also ask you that. But if there are an idea is let me know. For us, actually, we're planning and asking very similar questions because for us,

the value of the survey is mostly seeing changes over time. and understanding like, okay, wow, like, for example, GPD4 turbo MPS has declined. That would be interesting, actually. One thing that was actually pretty shocking to us, was a little bit of the exact number, but like one change that we saw, for example, is like, if you compare GPD3.5 NPS,

I want to say it was like 14 or something, like, not high, actually. And the GPD 4 MPS thing was like 45 or something like that, so it was quite a bit higher. So I think that kind of progress over time is more interested in the same. is our model is getting worse, models getting better, people still love PG vector,

people still love Mongo, stuff like that. That I think is the most interesting. It seems like you're very language model focused. I think that there's an increasing interest in multi-modality in AI. And I don't really know how that is going to manifest. Obviously, GPD4 Vision as well as Gem and I both have multimodial capabilities.

There's a smaller subset of open source models that have multimodal features as well. We just released an episode today talking about IdaFix from Hugging Face. And I would like to understand how people are adopting or adapting to the different modalities that are now coming online for them, what their demand is relative to, for like, let's say, generative images versus just visual comprehension versus audio versus text to speech. Like, what do they want? What do they need?

And what's the sort of forced, like, stack ranked, you know, preference order? It's something that we are trying to actively understand because, you know, there's this sort of multimodality war, but really like multimodality, it's this term that like, an umbrella term for like actually a whole bunch of different things that are quite honestly, like not really that related to each other unless, you know, in the limit. But it tends towards like maybe everything you uses transformers and ultimately everything can be merged together with a text layer because text is the universal interface. But if you're given the choice between, like if I want to implement an audio feature versus

I want to implement an image feature versus video, whatever, what do, what are people needing the most. What should we face to play the most attention to? What is going to be the biggest market for builders to build it? I don't know. I figure we'll just kind of zoom out a little bit to just the general founder questions. You have a lot of fans in the founder community. You know, I think you're just generally well known as a very sort of straightforward, pain spoken person about just business. Something that is the perception from Joseph is that you have been notably sort of sales led in the past. That's his perception. I actually never got that, but I'm not that close to sort of your sales motion.

And it's interesting to understand your market, like the internal tooling market, versus all the competition that's out there, right? There's a bunch of open source retools and there's this bunch of like, you know, I don't know how you sort of categorize that, you know, the various things out there. But effectively what he's seeing and what he's asking is how do you manage between sort of enterprise versus ubiquity or in other words, enterprise versus bottom up, right? I was actually surprised when he told me to ask that question because, I had always assumed that you were a self-serve sign up,

like bottom-up lead. But it seems like you have a counter-concensus view on that. Yeah, so actually, when we talked first started, I started most of them by doing sales, actually. And the reason we started by doing sales was mostly because we weren't sure whether we had product market fit, and sales seemed to be the best way of proving whether the way a part of market fit out. Because I think it's true a lot of AI projects.

You can watch a project and people buy use it a bit, and people might stop using it. And you're like, well, I don't know, is that problem market is it? Is that not? It's hard to say, actually. However, if you work very closely with the customer in a sales-led way, it's easier to understand their sort of requests, understand their needs and stuff,

and then actually go build a product that serves them really well. And so, basically, we viewed sales as like working with customers, basically, which is like, yeah, I think actually quite a, I think it's a better way to describe it what sales is, an own-stage company, and so we did a lot of that. Certainly, when we got started. I think we, over the last maybe five years, maybe like three years ago, four years or something like that,

I think we have invested more on the self-serve ubiquity side. And the reason for that is when we started Retool, we always wanted, actually, some percent of software that can built inside of Retool. Whether AI software or origin software will run, UI isn't whatever, but like software, basically. And for us, we're like, we think that maybe one day, you know, 10% of all the code in the world could be written inside of Retool, actually. Or 10% of the software could be running on Reto, which would be really, really cool.

And for us to achieve that vision, It really does acquire broad basis option of the platform. It can't just be like a, oh, you know, only like a thousand customers, but the largest 1,000 companies the world use it, it has to be like all the developers in the world use it. And for us, you know, there's like, well,

I think 25, 30 million developers in the world. This is the question of how do you get to all the developers? And the only way to get those developers, it's not by sales. You can't have a salesman to talk to 30 million people. You know, it has to be basically in this one of bombs up product land, ubiquity kind of way, basically.

And so for us, we actually changed their focus to be ubiquity actually last year. So, our World Star metric used to always be sort of gravity generated or in a new ARR generated. We actually changed it to be a number of developers building on the platform, actually, last year. And that I think was actually a really clarifying change because obviously revenue was important, you know, with funds, you know, a lot of, you know, our product and funds, you know, the business. But we're going to fail if we aren't able to get something like, you know, 10, 20, 30 million developers one day. If we can't convince all developers and the region was a better way to build a sort of class of software,

let's say the total application for today. And so I think that has been a pretty good outcome. Like, if I think about, you know, the last, like, I don't know, five years of retool, like, I think he's starting off with sales so you can build revenue and then you can actually build traction and you can hire more slowly. I think it was really good. I do think the focus for, it's like, you know, bottoms up ubiquity also is really important because it helps to get to our long-term outcome.

What's interesting, I think, I think, is that long-term ubiquity actually is harder for us to achieve outside of Silicon Valley. Like, to your point, I think it's Silicon Valley. Retool is like reasonable ubiquitous. I think like if you're starting a startup today and you're looking to build a internal UI, you're probably going to consider a retool at least.

Maybe you don't choose it because you're like, I'm not ready for it yet or something. But you're going to consider it at least. And when you want to build it, I think it's actually high probability. Well, actually, I'm not choosing. It's awesome. But it's that if you think about a random developer

working at, let's say, like an Amazon, for example. Today at Amazon, actually we have, I think, 11 separate business units that use Retool at this point, which is really awesome. so Amazon is actually a big Retool customer. But the average year on Amazon probably has never heard of Retool, actually.

And so that is where the challenge really is. How do we get like, you know, I don't know, let's say 10,000 developers at Amazon building by a Retool. And that again, I think is still a bottom of the ubiquity thing. I don't think that's like a, I don't think we're going to like, go to Amazon and knock on every developer's door or send out on email or your developer.

Be like, go use Retool. I don't ignore us, actually. And then it has to be, use the product. And you love it. You tell you a co-worker about it. And so for us, big bottoms up ubiquity by marrying that with, from enterprise or the community

has been something that's really narrated to our hearts. Yeah, just like general market thoughts on AI. Do you think spent a lot of time thinking about like AGI stuff or regulation or safety or like what interests you most, you know, outside of the retail context? Yeah, in my opinion, I mean, I think there's a lot of hype at AI right now

and this again, not too many use cases. So for us, at least from a retail context, it really is how do we bring AI and have it actually meet business problems? and again, it's actually pretty hard. Like, I think most founders that I meet in the AI space will always really be worth use cases. Never have enough use cases.

I sort of real use cases, people will pay money for. So, what's I think really weren't the retail interest comes from? Me personally, I think philosophically, yeah, I'm thinking recently myself a bit about sort of intentionality at AGI and like, you know, what would it take for me to say, yes, you know, GPTX or, you know, any sort of model actually is EGI. I think it's kind of challenging because it's like, I think if you look at like evolution, for example, like humans have to do like three things, if you will.

Like, you know, we are here to survive, you know, we're trying to reproduce and we're here to like, you know, maybe those are just two things, I suppose. So, like, it's basically that wants to survive, if you go eat food, you know, for example. To survive, maybe like having more resources is helpful. You want to go make one, you know, for example. To reproduce me, you should go date, you know, or whatever. You get married and stuff like that, right? So, like, that's, we have a program to do that.

And humans that, you know, are good at that have propagated. And so humans that, you know, we're not just surviving, probably have disappeared. Or just due to the natural selection, humans that were not interested in producing, also disappeared because, or, you know, there are less of them, you could say, because they just, they just don't appear in on, basically. And so it almost feels like humans have sort of naturally self-selected from these, like, two aims. I think the third aim I was thinking about was, like, doesn't matter to be happy. Like, maybe it does. So maybe, like, happier humans, you know, survival, it's hard to say. I'm not sure.

But if you think about that, and the role is like AIs, if you will. Right now we're not really selecting AIs for like, you know, reproduction. Like, it's not like, you know, we're being like, hey, AI, you know, you should go make 30 other AIs. And, you know, those that make you most AIs,

you know, are the ones that survive. We're not saying that. So it's kind of interesting sort of thinking about where intentionality for humans come from. And like, I think it's an argument that's like your human space that comes up with these three things. You know, like, you know, if you want to be happy, you want to survive, you know, your group.

That's like basically your sort of goal, you know, life. Whereas, like, they have that. But maybe you can program it in. Like, if you, you know, prompt inject, for example, like, hey, AI, you can just, you know, go do these things. And, you can even create a simulation, if you will, like all the AI, you know, in a world, for example. And maybe you don't have aGR in that world, which I think is kind of interesting. So that's kind of stuff.

I think that's when I talk about what's on my friends from a sort of philosophical perspective, but that's kind of interesting. Yeah, my quick response to that is we're kind of doing that, maybe not at the sort of trained final model level, But at least at the data sets level, there's a lot of knowledge being transferred from model to model. And if you want to think about that sort of evolutionary selection pressure, it is happening in there. And I guess one of the early concerns about being in Sydney and sort of like bootstrapped, self-boostraping AGI is that it actually is this in if these models are sentient, it actually is in their incentive to get as much of their data out there into our data sets so that they can bootstrap themselves in the next version. that gets trained. There is a scary sobering part that we need to try to be on top of. David, I know we're both fan of Hofstadters, G.B.

And actually, in one of your posts on the Segwai blog, you referred to the Anteater. I don't even know if you called them chapters. And G.B is just kind of like this continuous riff. But basically, like, how ants are like not intelligence, but like an ant colony has signs of intelligence and I think off-satter then used that to say, hey, you know, neurons are kind of like similar and then computers maybe will be the same. I've always been curious if like we're drawing the wrong conclusion for like neural networks where people are like, oh, each way it is like a neuron and then you tie them together should be like a brain, but maybe like the neuron is like

different models that then get tied together to make the brain. You know, we're kind of looking at the wrong level of abstraction. Yeah, I think there's a lot of interesting philosophical discussions to have, Sean and I recorded a monthly recap podcast yesterday, and we had a similar discussion on. What did you say, Sean, on the plane and the bird? I think that was a good analogy. The sour lesson. Are we using the wrong analogies? Because we're trying to be inspired by human evolution and human development,

and we are trying to apply that analogy strictly to machines. But in every example in history, machines have always evolved differently than humans. So why should we expect it to be any different? Yeah. It is interesting because it does feel like, Yeah, if you sort of peer under the hood of HGR, if you insist that HGI, we have always shoot HGR for things like a human. That is the Turing test, I suppose, but whether that is a good point,

like, if it works, no, it's not the Turing, because the Turing test basis is if the output isn't the same as a human that I've had, basically, I don't really care about what's going on inside. And so it feels like carrying about the inside is like a pretty high bar. Like, why do you care? It's kind of like the plane thing, like the flies. It's not a bird. I agree.

It does not fly necessarily the same way as bird, physically it does, I suppose. But you see what I mean. Like, it's not the same one with the hood, but it's okay because it flies. That's what I care about. And I, it does seem to be like, AGI is probably like, does it think and could it achieve, like, you know, outcomes that I can achieve its own outcomes.

And they can do that. Like, I kind of don't care of what it's like in the hood. It may not need to be human life at all. It doesn't matter to be. So I agree. Awesome. No, you kept too long. I actually have G. G.B. right here on my bookshops. Sometimes I pick it up and I'm like, man, I can't believe I got through it once. It's quite.

quite the piece of work. It's a lot of fun though. Yeah. I mean, I started studying physics in undergrad, so, you know, it's one of the edgy things that every physicist starts going through. But thank you so much for your time, David. This was a lot of fun and looking forward to the 24 set of AI results to see how things change. Yeah, we'll let you know. Thanks, love.