Two's Complement - Is Football Possible Anymore?

Episode Date: June 12, 2025

Ben recounts his 2007 startup building a database chatbot (before LLMs were cool). Matt tries their methods live with Claude. Ben foolishly uses his phone as a microphone stand.

I'm Matt Godbolt. And I'm Ben Reidy. And this is Tooth Compliment, a programming podcast. Hey Ben. Hey Matt. This, as always, is our very uninspiring introduction. We say hey to each other, which is I don't think we've knowingly done that in person. I know that like other people will take the Mickey out of us by going, hey, Matt, and

stuff. So, you know, I know that obviously it has become our trademark, but it is not intentional. Yeah. You have to have something to say at the beginning of a podcast. It turns out this was the best that we could do in episode zero and all of them from then onwards, exactly. Right. Right.

So we were talking because before this we arranged, uh, uh, well, before we, we started talking, we, we kind of joined a meeting and you are still doing your day job. And then you started telling a story and I'm like, wait a second, this sounds like a podcast episode. Don't tell me now. Let's tell me with the recording going. So you finished what you were doing. And then you had a name. Do you want to even say the name thing? Yeah. So what is the name of this? The name of this episode needs to be is football possible anymore. And I said, what on earth? Where's that from? What? I've got a story and then we should record this and here we are.

Yeah. Oh, dear listener. Tell me, Ben, what the heck is this all about? So in the in the heady days of the, you know, year or so before the financial crisis in 2008, I was working at a startup. And this startup was it had like chat GPT in its eyes. It like it like it saw it is like, oh, we could do a thing. But we didn't the startup was had nothing we had we had nothing we had no the startup had nothing. We had nothing. We had no... This was 2007. 2007, yeah. Neural net was things that people played around with and then abandoned at this point. Right. And so, like, you know, the glory of the LLM and the transformer and everything. I'm sure there were people that knew about them, but we were not some of those people. But what we did have was a, I'm just going to call it a parlor trick. We had a trick and it was like a patented trick. So we had something.

But it was basically this. We were building, the purpose of the startup was to build a chat bot for a database. And the idea here was that you could ask natural language English questions, just like you would with chat GPT, honestly, these days, but specifically and only about the data in your database. And then the way this worked is it would take your natural language questions and it would map them into SQL. And then it would execute that SQL. And then it would give you a response that it thought was appropriate based on the question

that you asked and the shape of the data that came back. Right? And so it might generate a report for you or it might just tell you an answer. It's like the number of, you know, sales of this unit is 37 or it might do something else.. What the intention was to ask. People would ask would be like yeah give me a summary of the last three months of sales. Right.

That kind of thing. Yeah exactly. Very chat GPT ish you know the kind of thing I see people put into AI all the time. Yeah. Oh yeah yeah yeah yeah it was just very narrow right it was? It was just like only about a database. And the idea was we're gonna build a tool you could point it at any database, it would ingest the schema, it would cache some of the data

and it would sort of build up the semantic model which you could then use to process your questions, right? Got it. So there was like a pre-step where you pointed it at a database and it did some investigations first and foremost and then that's what you use to like build your parlour trick. Yeah. List of whatever snippets of SQL that

you presumably behind the scenes were were part of this. Yeah, yeah. I mean, and it was, you know, that the basic idea behind it was that you'd have like this concept map. And so the concept map just related to databases, tables and columns within the database, right? So it's like, if you're talking about sales, well, then you're talking about this table. And sometimes it was as simple as literally

the name of the table is like yearly sales, but also sometimes there were some things where you had to teach it a little bit and you get a little, maybe a DBA involved and you say like, this is what we keep our customers. And by teach it, you mean hard code, presumably in a little bit and you know, you get a little maybe a DBA involved and you say like, this is And by teach customers mean, hard code, presumably, the big tip statement. And well, in the early days, it was that eventually we got it to

be smarter than that. But this was this was sort of the project. And I worked on this for about a year, I guess it was only remember now. And so because it was so constrained, it, it had a tendency to hallucinate. I mean, we didn't think of it as hallucinating, but sort of retroactively now, you'd be like, that's exactly what you would call it as hallucinating, where it would like reach for information

that there's no way it could possibly even have and try to like solve for it and then give you something, which was like nonsense, right? And so as engineers working on this, to solve for it and give you something, which was nonsense. As engineers working on this, we were concerned with this problem because we didn't want it to purport to be something that it wasn't, to have information that it didn't have. One of our favorite pastimes was to grab a couple of beers and then ask it stupid questions. And the sort of canonical version of this, the very first one that you'd always ask

in one of these sessions is, is football possible anymore? Because that is not a question that even makes sense or is answerable by information in a database or it's like, what do you even mean? And so what we were hoping to see is some sort of informative error message saying like, what do you even mean? And so what we were hoping to see is some sort of informative error message saying like, I don't understand what you just said

because it doesn't make any sense. I'm really sorry that I'm gonna be now typing that directly into Claude 4.0, Opus, the best possible thing. Is football possible anymore? Is that the question? That is it, is football possible anymore? Is that the question? That is it is football possible anymore. Okay. And I think your cell phone is on the desk every time it buzzes. It's the microphone picks it up super loud.

Oh, no, I've been I naively and stupidly used my phone as a microphone rest, which is that's Oh, right. Hence the loud. I don't know why anyone would do that i'm just going to replace that with some oh my god dvds which are sitting on my desk and those are definitely not doing anything is thinking a lot more uh it's gone onto the internet now and it says ah you're asking you football is still possible bit of a loaded question that from what i'm seeing in the search results football's very much alive and kicking in 2025, although it's facing some interesting challenges, depending on which side of the pond you're on. Okay, so apparently, Claude can deliver some kind of question, answer to this question. But but if I if for example, I given it just only the the marketing results for last month's widget sales, right, I might reasonably expect to say that's not a question I

can answer right now. Yes, yes. Yeah. And I mean, you know, the the the interesting thing about that that startup was, there's actually a couple interesting things about this job. One is, is that is maybe sort of like the most kind of like by the book, extreme programming team that I have ever been on, right? Everyone on that team worked, had worked in consultancies. I think there were a lot of thought workers there.

There were some folks from the usual suspects. Oh man, what is the name of that company? I don't know what it doesn't matter, but they, they all, they all kind of knew what they were doing when it came to things like test or development, continuous integration, bunch of other things. That again, this is the same programming, not the one where you climb a mountain and laptop. Okay. There's a lot of Red Bull involved in that one.

And so like we had a group of people that were all very like-minded, that all kind of knew the same practices and knew them very well. And this, you know, that was the core engineering team. I was like the only engineering team in the startup. And it grew to about eight people at the maybe nine even at the biggest. Gosh. And that is like the biggest, that is my example in my career of like, that's as big as I think you can make a team. If you have nine people that all are working in the same code base, solving the same problem, and they all know these practices really, really well.

And they all agree on all the practices are on the way to do. Yes. As well as what you're trying to do. Yes. Yes. Uh, and, and we managed to do it with nine people and even at nine, it was like straining at the edges, right?

Um, but that was, that's, I think that about as big as you can make a team before you start suffering, um, you're just making it worse by adding more people. Right. I'm surprised that it got to nine. Again, you've, you've qualified it with all of these extra like alignment and, you know, skilled people at the top of the, you know, with the same kind of mindset for approaching problems, but just in terms of the, the, the number of pairs of people to talk between, between, you know, that's

what nine times eight, what are eight nines? I don't know over to that. And then plus and then minus one. I well, it's a lot of people. It's a lot of interactions, but you're able to make it work. And I mean, by it work, I mean the the team work, but the team is functional. Yes. You're you are not currently talking to me from your private yacht. The company did not make it. I was the story there.

Oh, yeah. Oh, yeah. No, actually. So this is a somewhat somewhat personal story. I don't know how much of this I want to share on the podcast. Sure. No, that's fair. The punch line of this story was basically that we were my my wife was pregnant with our youngest at the time. And on Friday, March 13th. Of it would have been 2008.

I know I'm going to screw up these dates and it's going to be recorded for all time on this podcast. Friday, March 13, we were finding out whether we were going to get another round of funding. And my wife went into labor. Turned out to be false labor. But after a long day at the hospital and a lot of other things, called my boss that evening and said, Hey, man, do we still have a company? And he said, no,

oh, because our our primary investor had decided to take all the money that he was going to put into the second round and the third round. Or I don't even remember what it was and put it into real estate instead, because it's falling apart. And it was super cheap. And I'm like, you know what? I don't disagree with what you're doing right here. Yeah.

I wish it would be different, but that's not how that turns out. No. Well, that is how these things go sometimes. You know, that's what the, the risk of startups. Yeah. Uh, yeah, that is, but you got a good story out of it. I did. I did.

And it's, and you talk about how it actually worked. I'm intrigued, you know, in this pre pre, you know, LLM pre expert model was it right? Together with regular expressions and text snippets and and hey, we know our three customers are x y and z and so therefore they will talk about this these were people will talk about this other thing and then we, hey, we know our three customers are X, Y, and Z. And so therefore they will talk about this. These were people who would talk about this other thing. And then we just. Yeah.

Well, no, the, the, the, the parlor trick essentially was if you take the sort of English concepts in a typical sentence and you can, you can map those concepts to structures in the database, which is like this, again, this whole concept mapping of it needed to, like the information you needed to seed it with, and it did a decent job of finding this stuff out for itself, I think, but the information you needed to seed it with was

what tables and columns and other things map to what concepts. And if you could, if you did that, then it's actually like a pretty limited search space of like, take this English sentence and map it into this set of concepts. Like you say, it's a trick. So you start from the data, but rather than starting from the human and say, what could they possibly mean? You go like from the database, what things could make sense to ask off this database? And then yeah, like, like all these, as you say, part of trick, but yeah, right. Now that's a neat way of doing it. But yeah, these days, it's just like, I mean, this is why I have I have zero fear that somebody's gonna be like, you gave away our

secrets. I'm like, dude, have you seen? Yeah. Although I will start to do like, obviously, there's a lot of interesting discussion, as we've have you seen? Yep. Have you seen what plot can do? Like, come on man. Obviously there's a lot of, uh, interesting discussion as we've had on this podcast. And I definitely don't want to revisit that just this, this minute, but you know, uh, the power consumption alone of your parlor trick. Well, that's fair. Fits onto, you know, a microprocessor on a cell phone. Uh, it doesn't require a data center's worth of, um, heavy compute equipment.

Um, so there's something to be said for that for sure. But, uh, no, that's fascinating. But like, how can I ask, how did this question come into being? What was the forming for this? Because it, it sat for a minute. When you first, I'm going to be honest with you, when you first said it to me, I thought, Oh, gosh, is this a quote from like Ted lasso that I haven't seen that far in the series? Yeah. And if I just missed it, because I obviously think of football as being the one way you use your foot and the ball. Not hand. Yes, not hand. I mean, I presume it was meant to the I mean, it does. It does. And obviously, Claude picked up on that when we asked earlier, it's like, which side on you're from. But I remember how it came about. How did this come into being? I remember this coming into being

from one of the engineers on our team, Paul, who was an expert Scrabble player and kind of like a he was just really into words like he just he just really and like and I think this startup for him was kind of like really cool and then it was like we're doing a bunch of natural language processing and like all this concept mapping. And he, he just really loved it. And he had a way with words, I guess I would maybe say not the way with words that you would normally associate with that phrase, but like, yeah. And my memory of this, which could be completely false, is that he, he came up with that phrase as a way to kind of poke at this thing that

we were building and we're like, Oh my God, that's drawing on his sort of like deeper understanding of the things that you were doing with the natural language and how it was processed. He was like, well, what would trip this up the bus? This is what, what is the unit test? Yes. This thing exercises the worst possible behavior because there's no concept in this sentence that maps to anything remote right? I hate a base. So yeah, and right now

I think it I think it actually sort of works on two levels too, which is this like it's sort of a nonsensical question Regardless of your data set, right? like What and then if you add in the additional constraint of it's going to have to take this and map it to a schema that's probably about like, you know, employees or sales records or some other sort of mundane thing that is probably not sports. Although I do remember baseball being one of the data. I was going to say there's a danger that with the right kind of database, this might actually make sense. I feel like you could look at MLB database that we would test that thing with sales, sales cost, sorry, incoming ticket revenue versus cost of running a game. And then his football possibly were no, you know, we lose money on every game. Definitely not possible.

No, I actually I hadn't thought about this until just now, but I seem to recall that we had this. It was like player statistics from the MLB, and it was like, you know, it's like the color commentator in a baseball game. It's like, oh, this pitcher has has an ERA of 1.3. I'm just making up numbers. I don't know anything about baseball against, you know, right handed batters in the East Coast over the last four years and it's like, how does anyone know these? Right. And and so we were like putting that guy out of a job by building this thing that could take, you know, the the MLB stats and give you like all

kinds of crazy things based off of it. Right. Like, did you know that this is the highest scoring game on a Thursday where it's raining since, and you're like, well, yeah, when people come up with this nonsense, yeah, you think there is definitely, there's a, it's got to be a market for it. So yeah, you put, well, you were, you were trying to put that person out of a job. Evidently you have, we failed.

So they still have a job and everything's fine. Everything's fine. That's cool. Yeah. Well, now you've explained to me that football may or may not be possible anymore. And yeah, I don't know quite where to take it from here now. Other than maybe that's, that's, is that the end of what we got here I think it might be I think it might be I mean, you know if there's if there's a takeaway from all of this Yeah, it's You're probably not gonna get things right the first time

That's sort of like the way that I that I think about it is, you know back in 2007 we were building this thing. We were like, oh, this is going to be really cool and all that stuff. We also knew there was a startup and this was iteration one. And it's like, okay, the chances of this actually taking off are not high. We didn't really expect the Spanish Inquisition. We didn't really expect the- Nobody does. I mean- Financial, right. We didn't really expect the financial crisis to come and smack us in the face. But even without that, it's sort of like when you're building new technology, you know, I think we've said on this, I guess before, nothing new ever works.

The way to get something working is to get the new off of it as quickly as you possibly can. Right? Yes. And so I think you could take away a lesson of when you're building technology like this, it's speculative. It's probably not going to work. But you will learn things. And when you learn things, then version 2 can be a little bit better. And you repeat that process over and over and over again. And then eventually you get things like Claude answering like,

well, depending on what side of the pond you're on, it may or may not be possible. Gosh. Well, that seems like a decent conclusion to this. I was wondering where we were going to go with it, actually. You know, like, I mean, given our, you know, again, comprehensive planning, this seems like a good place to leave it, so... I would say so.

Until next time! Until next time! You've been listening to Tooth's Compliment, a programming podcast by Ben Reilly and Matt Godbob. Find the show transcripts and notes at www.toothcomplement.org. Contact us on Mastodon, we are at tooscomplement at hackyderm.io Our theme music is by Inverse Phase. Find out more at inversephase.com All right, I just like to give myself work to do in the edit. Right.