Two's Complement - Technical Debts

Episode Date: January 15, 2024

Ben and Matt discuss the original definition of technical debt a metaphor created by Ward Cunningham to explain why software designs that were correct when created now need to be changed. Ben invents a new verb, 'to soapbox' and then demonstrates its practical use. Matt reads timestamps in the future.

I'm Matt Godbolt. And I'm Ben Rady. And this is Two's Compliment, a programming podcast. Hey, Ben. Hey, Matt. We haven't really thought about this, have we? No, I mean, not really. We just said, hey, let's talk about technical debt.

And then I said, hey, Ben, and that's as far as we've got. That's the extent of the planning. That is the extent of the planning. So let's talk about technical debt. Yeah. Well, I mean, I think you may know that this is a little bit, it's not hard to get me on a soapbox, as our listeners are well familiar with at this point. And this is one of those things that I am very happy to soapbox about, which is a new verb that I invented just now. To soapbox?

Yes. I could get behind that. Excuse me while I soapbox. Because I think it is a metaphor, and it is something that has... People have lost what the original intent of this metaphor was, what the original meaning of the metaphor was, and it has become a thing that it was never intended to be. And I think it's an actually... It's a harmful thing in it's current sort of um colloquial understanding of what it means well so why don't why don't you define technical debt

from the point of view of the original idea where the where does the term come from right well the good news is that i don't actually have to define it because ward cunningham is the guy that invented the term and he defined it and we can link a video uh youtube video in the show notes uh to ward's definition right he explains what it is what's the tldr of that then yeah so i'm not gonna i'm not gonna recount the entire video here in the podcast uh for copyright reasons if nothing else but. But the idea behind the technical debt metaphor was Ward was trying to explain to people who were not programmers that their mental model of a software system would change over time. Their understanding of a problem would change over time. And they would design a system for the problem as they understood it,

and then that understanding would change. And then they would have to make changes to the design in order to accommodate that new understanding. And the difference between the current state of the system and the design as it would reflect their current understanding of the problem he referred to as debt as technical debt and he wanted to use that metaphor as a way to right you know tell people it's like the longer that we put this off the more expensive it gets right like we have to bring the state of the design of the system up to our current understanding of the problem otherwise it's going to get more and more expensive. And that is quite different from the way that certainly I colloquially use the term technical debt. Right. And in this video, actually, Ward is very explicit to say,

at no point did I imply or recommend that anyone write bad code and then go fix it later, right? Right. That's not what the metaphor was intended to describe. I don't think you can go faster by write bad code and then go fix it later. Right? That's not what the metaphor was intended to describe. I don't think you can go faster by writing a bunch of bad code and then fixing it later. I think that's a slow way to program. This was about evolving a design as your understanding of the problem evolves, and if you don't put in the time and effort to evolve that design, then the cost to change your system is going to get more and more and more and more like interest payments on a debt. Right.

Right. I think one of the reasons why Ward's original definition got lost is that very few people actually design their software that way. Right. Ward's original definition got lost is that very few people actually design their software that way. Right. Very few people are actually, we were talking in a prior episode about incremental.

I was going to say, it seems to sound very, yeah, on theme for that. Yes, iterative, right? Like, oh, we're going to design it the best way that we know and then we're going to learn more and then we're going to redesign it

and then we're going to evolve the design. We're going to use refactoring and change the design. That's a skill. It's a skill that not a lot of people have and even the ones that have it don't necessarily use it all the time and so like i think the ward's original definition is actually much more narrow in its applicability than the term is used or the term is used all the time for basically as a metaphor. It's not a metaphor. It's just like, that's bad code, right?

When people say technical debt, they just mean bad code. And they're not particularly specific about what they mean when they say that it's bad. And I think some of that is maybe hidden by the metaphor, which we can talk about. But I think that's sort of like what the original definition was. I think that is, Ward's original definition is a very useful definition. I think it's a little bit more narrow in scope.

Very much so, yeah. And it's something that, you know, it sort of like makes me wince every time I see someone use it as basically a synonym for that. Well, that's code is bad and I don't like it. Right. And I think that's the thing that you were saying like prior to us recording that is, as you say, it makes you wince.

It's when people start like building around something and saying this is technical debt and they're sort of poking it with the barge pole further and further away from them and then saying like, well, we don't go there because it's, you know, it's full of technical debt and it either gives you an excuse to ignore problems or it gives you an excuse to write even worse code to keep dogpiling on because you're like well it's just more what's

more debt you know it's like you've already essentially technically bankrupt uh yourself uh and the interest you can't keep up the interest payments so you're like why even bother trying you know we were just just yeah just another hack on a hack that at least from again that's the colloquial yeah understanding of technical that here which is not what you've just described ward's um right right absolutely um and i think another problem with this is that you know the and and i don't know i doubt very seriously that ward anticipated all of the impacts of him just sort of coming up with this idea. But one of the unfortunate negative impacts, I think, of this is that to most people who run businesses, debt is not a bad thing. Debt is a great tool, right?

You take on debt for lots of good reasons. So it's like, yeah, okay, cool. You take on it advisedly. Right. Yeah. Even if it's sometimes kind of high interest like you know if you're if you're starting a new venture and you need to move quickly it's like yeah we'll take on a bunch of debt like that's fine so like you know the the way that people you blend those two ideas together right the the misunderstanding

of technical debt as bad code and the sort of financial application of debt as a useful tool uh you wind up with this weird intersection which is bad code is a good useful tool right yeah right at no point what does anyone who created this term even coming close to implying right right so it sort of like becomes this like received wisdom of technical debt received from who with what purpose right right um it's it's it's the worst sort of argument from authority where you didn't even understand the original argument from right is completely there's not even like a coherent um thought process behind it because it's an it's it's a phantom uh idea no one actually made this point it was right yeah right but i guess it because it's a phantom idea.

No one actually made this point. Right, right. But I guess it's entered the zeitgeist of programmers, right? Yes, it has. So we're kind of stuck with it, whether Ward meant it to mean this or not. When people colloquially, like nine times out of ten, if I'm talking to someone else, I'll be honest with you, I'll be making you wince by saying, you know, hey, we've got that error.

Like, you know, I'm working on something right now. There are like 2000 parallel pieces of work that I'm doing across a multitude of days. There's all sorts of broken bits of data and things that I'm having to deal with. And if one or two of those things fail, I'm like kind of OK with it right now because on aggregate, that's still useful for me. And in my mind's eye, I'm like, I want to chase down every single thing. I need to understand it. For all I know, the reason that day one dies,

you know, crashes out, is actually a systemic problem through the whole code base, right? And then it's just that it doesn't crash on the other days or whatever. But in order to move forward quickly, I am, in my mind, I'm filing that under technical debt incorrectly and saying, I'm going to move on and we'll come back to it a bit later. Because, I mean, maybe you could argue this is part of the, I'm going to try and swing it as part of the progressive. I can't remember what we came up with. Iterative way.

I want to get to the end. I want to get my 2000 days broken or not broken and then say, well, the next stage of the pipeline do this thing and i think it's this but i don't know that it's this as in i don't even know what the design should be or what the facilities i need to produce are so i'd rather get there with some amount of like brokenness to explore the space solution space of the next part and then as actually in two more steps after that i'd rather get all the way and then go back to the beginning and go like okay these things need to be looked at now the these warnings that we're seeing or these crashes we're seeing are a symptom of in my mind the technical debt that i'm paying down but you know you know it's so easy to to go like oh yeah it's just bad code i don't think my code is bad it's just these are these are bugs these are issues that i've got it tracked in github or

whatever and i know i'm going to come back to them they're just not high priority right now and i but they they could could be something that actually is really important to look down like for example i'm going to give you an example that's literally just happened right now yeah yeah um we had a an issue with uh a time stamp that fast forward itself into the future it was like 150 years in the future suddenly we know we're processing data from 2022 and then suddenly it's the year 2187 and you're like oh my gosh what the heck's happened here um i had seen it once and i'd filed a bug and I couldn't reproduce it. And I let it sit. Now, obviously that's kind of like there's something wrong in the system.

Everyone knows there's something wrong in the system. Was it a corrupt file for one day ever? Who knows, right? I certainly didn't at that point in time. And then we let it lie. You know, it happened again one more time. We added something to it, but no one really tracked it down. And then today we had a completely unconnected issue where we ran out of disk space.

And the reason it turns out now we know that we ran out of disk space is that some poor process was trying to fill in all of the gaps between 2018 and 2187 with just empty data. Because it's like, oh, well, you you know i have to do like a forward fill of all this data and it ran out of disk space doing so understandably and of course then we're like oh wait a second where have i seen the year 21 oh i know this is that bug and all this time that has been sat there and actually we've processed other data incorrectly so we found the problem we fixed it and now we're cursed to actually go back and and reconvert a whole bunch of data which is you know expensive so that did not pay off that came back to haunt me um but sometimes it doesn't right and may am i right to you know stop and drop and do everything every time i don't know no no no no and so the here's here's the thing about this, is that the reason I wince is not because I'm trying to advocate that everyone should do everything perfectly all the time for some definition of perfectly.

That's not what I'm saying at all. What I'm saying is, is that why are we using this broken, misaligned metaphor to talk about things that we as programmers can be much more specific about and have much more precise terminology about. Because when I hear you say this is technical debt, that doesn't really align with the other things that I see people call technical debt. Because at least in my mind, those are very different things with very different impacts on the system and the reliability of the system and the long-term viability and everything else. And so the ways that I kind of, the dimensions that I think about when it comes to these sorts of things, and this is something that has definitely been evolving in my mind over the last

few years. You know, it's not, this is not like some, you know, received wisdom is, oh, everybody knows this has been around a long time. I've been trying to say, okay, Ben, well, if technical debt is not the right way to think about it, then what is? And the more modern thinking that I have on this is that you should think of your system in terms of complexity, its capabilities, and its risks. And so what I hear you describing here is actually just a risk. There's a risk in the system that is unaddressed and it can make very good sense not to address all of your risks. I mean, this is true in every endeavor of engineering, right? They don't make bridges that are like impossible to fall over because they would be cost prohibitive, right? There's a that they design to and they say that it's going to fail under these conditions, or it's going to fail after this amount of time if it hasn't been maintained. There are risks that you carry in the operation and the construction of that bridge, and those are deemed to be acceptable for whatever application it is that you're doing. And it's no different in software, right?

So you have risks that are in your system. And it is right and proper for you to make a backlog of those risks and account for them. But it's not necessarily right to just always fix all of them as soon as you possibly can. Right. Right? So that's one dimension, right? And that is a very different thing than technical debt. Than technical debt, right.

That gets lumped in. Oh, we're just talking about this metaphor of technical debt, and it means basically bad code. And what it means is, oh, it's a risk that we have. If the disk fills up, if this problem occurs, if we see this exception we've never seen before, whatever, right? So that's one thing. Another thing is capabilities, just new features, new functionality.

Now, I definitely use the term capabilities more and more working where i do because features means something totally different i mean features means about three different things the three other three people yeah right so yeah capabilities and facilities and whatever we all come up with like alternative functionality yeah right similarly for what it's worth um we always talk about you know if you talk about the uh if you sort an array of things they are ordered but if we talk about you know the ordering or the order or you know if you talk about the uh if you sort an array of things they are ordered but if we talk about you know the ordering or the order or you know what order are they in um again in our world order means something else there's an order yeah right so i always say sequence now are they sequenced in this way are they you know so you kind of learn these ways of avoiding

ambiguity in your day-to-day based on the sort of colloquial expressions that are more common than the perhaps, yeah, more common in general English ways. Yeah, anyway. Right, right. So anyway, the facilities. Yes. No, capabilities. Capabilities, right. The things that your software is capable of doing.

Yeah. Right? And you're going to have gaps in those capabilities. Sometimes those gaps actually represent risks like, oh, if this thing happens, then we won't handle it and it'll crash. Sometimes the gaps in the capability is just like, yeah, we just don't do that yet, right? That's not something that the system does for you, right?

Literally before coming in, I know we said about this bug we found, but I was adding the capability of our cache directory to have a maximum size, which at the moment it doesn't. And we've been working around it by every now and then, you just RM minus RF slash temp slash, you know, my cash, whatever. And we're like, we've kind of been fine with that.

That's a capability we could live without, but it's annoying. And now I'm like, well, we can add that capability. So that's fine. And yeah, all right, okay. Capabilities. And then the third thing is complexity. Now, I think we've talked before in the show about complexity, necessary complexity versus unnecessary complexity.

I think it's good to minimize unnecessary complexity, but I think the thing that you really just sort of want to be looking at is the total complexity. Whether it's necessary, whether it's unnecessary, it's still there. It still makes your code harder to understand, harder to change. It doesn't really matter if it's perfectly designed and is the absolute minimum thing that it could possibly be. It's still complex. Yeah. That still has costs, right? Absolutely, yeah. And so these are the three dimensions of software that I kind of think about. And I try

to, whenever I think about technical debt, I try to think about, okay, is this a risk? Is this missing capabilities? Is this complexity? Like, what is this more specifically? I don't need to use a metaphor, a financial metaphor to reason about this. I'm a programmer. I'm talking to other programmers. I can be more concrete. You can say these things. Yeah. And you're right. A lot of those things. So what I'm hearing from you is that um some of the i mean because everyone we can make up a straw this definition of what other people think technical debt is in a second because i think that'll be useful but right what you're now is saying is like within at least the people we talk to when people say technical debt debt they're probably better served by thinking

of it in one of those three categories because we're assuming they don't mean my code is just terrible and that's okay. Again, now this is the straw man. Well, I would put that in unnecessary complexity. Right? You're right. I could make an argument that just complexity of its

own is not necessary. While bad code can be unnecessarily complicated it can also just be simple and bad and wrong well you know sometimes you have to add complexity to make uh make it better you know like hey i've got this really simple bash script and this bash script is uh you know and you're like but that's why are you doing it in bash you're like well it's because it's three lines of bash and you're like but there's no, why are you doing it in Bash? You're like, well, it's sort of because it's three lines of Bash and you're like, but there's no monitoring. There's no email if it goes down.

I can't parallelize it. All these things. I guess those are capabilities and things like that, but it's not bad. Yeah, I don't know. Yeah, you're right. Now you're, listen, Ben is grinning the I told you so grin at me right now.

Well, I mean, I'm bringing all these things up again. Listen, Ben is grinning the I told you so grin at me right now. Well, I mean, I'm bringing all these things up. Again, this is something that I have been thinking about for a while. And I don't know that I'm not trying to actually say that this is some new comprehensive way to think about it. I'm just I'm trying to figure out a replacement. Like I know the technical deck is wrong. And I keep thinking about like, OK, how can I be more precise about this, right? I mean, first of all, we should note that Perry has just made his first podcast appearance. Oh, has he?

Yeah, he's behind you right now. So Ben's dog came in. I'm sure I saw it. I did not invent that. Maybe he left again. Did he walk back out again? I didn't notice him go out, but I was like, hey, it'sry yeah you know we've had monty making a debut although he's asleep behind me is he not no all right now i've just made it somewhere in here i don't know anyway um i've

now forgot what we were talking about well it's you know can can am i isn't uh i'm trying to find new tests for my hype uh my scientific theory theory here. My hypothesis is that you can categorize, you can take what currently falls into the very vague category of technical debt and actually a little bit broader than that, the aspects of software that you need to care about when you think about technical debt and good code versus bad code, things like that, and fit them into one of three categories. Is it a risk? Is it a capability? Or is it complexity?

And complexity has a necessary and an unnecessary component to it, right? Got it. And I will say that for me, on the projects that I'm working on right now, these things are actually fairly directly measurable, right? If you want to see my risks, go look in the issue backlog. All the risks that we know about are in there right yeah if you want to see the current implemented capabilities look at the tests all the tests describe all the capabilities of the system and you can see what they all are

right if you want to know the missing capabilities well that's probably also in the issue in the issue tracker yeah and if you want to know what the complexity is you can do a lot worse than just counting the number of lines of code like there are other more sophisticated ways to measure complexity in code but it's not a bad proxy it's not a bad measurement of complexity and so that's sort of the way

that I think about it we think about like the okay we could mitigate this risk but is it worth the complexity? Yeah. Right? Maybe it's better just to wear the risk. And like, even if it happens once, it's like, yeah, you know, crashed. We turned it back on again.

Wasn't great. Yeah. This is a bit like that sort of pragmatic thing where if you can't write a test for it, make it fail fast. You know, there's a trade-off there. It's like I could spend a ton of complexity testing every command line parameter to my command line tool, or the first time anyone notices

that dash, dash, whatever doesn't work anymore is like, we go and fix it. I mean, again, you don't want to, yeah, yeah. Right, because I feel like unless you have those ways to discuss the trade-offs, what you wind up getting into is this sort of like semantic argument about like, that's technical debt it's terrible and you're like no that's just a reasonable trade-off and it's like what are you trading off right like if you can't describe that

if you can enumerate that it sort of falls into this vague definition of it's just bad and i don't like it yeah yeah and there's a lot of yeah we've we've talked about fud sort of before about you know there's a lot of fear uncertainty doubt you can use it as kind of like one of those words but once you've painted it with that it's like you you know it's just it's just bad unequivocally bad um but one thing that's i think i think missing and maybe this is maybe you're you've got to i could imagine maybe you're a talk to this but like is there is something really emotive which is perhaps the problem with the term technical debt because i think everybody even when you're talking about engineering stuff everyone can kind of get your head around the fact that debt has a compounding aspect to it and not all of the things

that you said have a compounding aspect to them no that's true missing capabilities don't necessarily have a compounding aspect it's like oh i mean they could do don't get me wrong yeah yeah um the the uh risks could be compounding you could imagine a risk which is like well if that happens then oh now all these other things become really important you know suddenly like you know if you can't if you can't reproducibly recreate all your data from scratch then this this bug that you have that you've seen once that can corrupt a file might be very very a big a big deal because suddenly you might have to go through you know hundreds of thousands of terabytes of you know data again or whatever right that can compound but almost always code complexity has a sort of in-your-face, day-on-day compounding effect where it's like,

yeah, I want to add a new capability. Oh, but the code is too complex. And it's sort of like it gets worse and worse and worse because what you're really talking about is sort of entropy of the code base. And it just gets worse on its own somehow, right? You could go back to a project from two years ago and somehow it's worse than you left it you know bit rot is real yeah um yeah so yeah

it feels like that there's sort of some and again it's emotive rather than like uh you know i can't put my finger on it as you say if we're talking about professionals talking to professionals i think we all understand that that is already present if i just say yeah the code is unnecessarily complicated you can say yeah i guess that makes it a pain to change you're like certainly does and you know that you know right well we don't have to say but it's a debt right but it does yeah but it's so easy it's a trap to fall into i mean i can it is it is a trap to fall into and i think if you go back towards original definition and think about what he was actually talking about, he was talking about unnecessary complexity in my model. Right.

He was talking about like the problem has our understanding of the problem has evolved beyond the current design. There are things in the current design that don't fit that problem. And it could be like, you know, literally just code that isn't used anymore. It could be designs that really aren't applicable for the problem that they're trying to solve. And you sort of like twist them around to try to get it to work. And it's just adding a bunch of things that you just don't need. And the solution to that or a way to address that is maybe a better way to think about it is to refactor the code so that the design better fits the problem that you have and hopefully just sort of shrink the number of lines, but at a minimum, just make them easier to understand.

Right? Yeah. And again, this sort of gets back to my thing of like, that is a very narrow thing. Ward was talking about a very specific type of unnecessary complexity where the design has not evolved along with the understanding of the problem there is an opportunity to remove some unnecessary complexity right uh very narrow but that i mean you're absolutely right to hit on the fact that the complexity period is compounding right here's a thought experiment for you. Let's say that you have a... Well, have we done the Alice and Bob and going to the park?

I'm just going to say this again. Forgive me if you're a dear listener. If this is a repeat, it doesn't ring a bell for me, but that doesn't mean anything. I'm still mildly hungover, as you may have picked up. Yeah, right there with you so you have let's say that you accidentally give you know two programmers on a team the same problem on the same day right Alice and Bob

and you know they're not sending cryptic messages to each other in this particular example and no one is trying to intercept it or whatever but they're just two regular programmers on your team and you say hey and they accidentally one is trying to intercept it or whatever but they're just two regular programmers on your team and you try to solve a problem and they accidentally are both trying to solve the same problem at the same time not knowing that the other person is working on it and so bob sits down and he writes a thousand lines of code to solve this problem and this code

is perfectly designed it is as simple as it could possibly be. It is well-tested. It is well-designed. It is correctly documented. So whatever level of documentation is appropriate for this project, please apply that level of documentation. And it solves the problem perfectly. And he spends all day on working that. At the end of the day, he pushes his changes into a PR and sends the PR to Alice saying like,

Hey, can you please review this? And then meanwhile, meanwhile, while all this is going on, Alice goes to the park and she sits in the park and she feeds the pigeons and she thinks about the problem. And she sits there all day feeding the pigeons and thinking about the

problem. And she comes back to the office at like four 30 and she deletes three lines of code and pushes a pr fixing the same problem that bob fixed which of those two solutions is better i mean the one that involves sitting in the park is better just obviously alice's solution is better right there's no unnecessary complexity in Bob's solution. None. Other than the fact that he just wasn't really solving the right problem. He didn't understand that the way to fix this is to just not do these three things instead of accounting for... And I think we can all think of things that fall into that kind of category of like, you

know, this is a perfect solution to... So, you know, for example, we just talked about this, like the cache that I'm talking about here is a big cache for these giant files that I download from wherever or whatever. And the three line fix, really, if I had access to the code that I needed, would be just read the files directly from the network appliance. Don't download them to the cache and then manage the cache. And so here I am adding complexity here to deal with the the problem that i've created myself whereas if somebody was smarter than me and could work out how to get the the just would yeah exactly you you're with me you're with me on this and yeah it's i like to think of this as the uh the old lady who swallowed a fly you know the children's rhyme yeah and you know

she swallowed a fly perhaps she'll die and then she swallows a spider to catch the fly and we're all we've all written spider level fixes for something right you know like there's this thing i didn't think about it uh but i can solve it by by swallowing a spider and then you know the the the tail continues until she's swallowing a horse and i think at that point you really you should be fired actually if you're considering swallowing a horse but you know it takes it there's a lot of inertia because there's a lot of code that's written in the horse the cow the dog the cat i'm trying to remember now the cat the frog the spider i don't know whatever it was that she swallowed to swallow the fly and then what you really want to do is go back and say back in time as we can as developers say, let's just not swallow the fly in the first place.

Right. And now we're good. Right. And that's Alice there. Or, you know, and getting back to my sort of Alice-Bob hypothetical scenario, one way that that could very practically play out is what Alice was thinking about in the park is, is it really bad if we swallow a fly? Because maybe we could avoid this whole thing by just taking on a very small risk that is a completely acceptable risk and is totally better than another thousand lines of code.

So I'm just going to go delete these three lines and there's a chance we'll swallow a fly and turns out that's just a little bit extra protein and we'll all be fine. Yeah, maybe that's... Yeah, so that's an even better analogy, Dan. Yeah, sometimes it doesn't... Yeah, I'm with you.

Yeah, just deal with it. You swallowed a fly, all right. Yeah. Restart the server once every three months or whatever, rather than the... Yeah, well, we've got this thing that follows this, you know, that you've seen, like, the sipping bird. You know those sipping bird things that,

like, you kind of heat up, and then they kind of, like, keep dipping forward forward there was like if i think it was at google a whole thing with that which was the thing at the very end of the chain that kind of like causes everything to continue running is the sipping bird that's hitting like the space bar on the computer console that's like yeah yeah carry on whatever it doesn't matter just you know somewhere along the line of complexity this rube goldberg machine that you've built you're like something has to be physically at the server pressing the button. And you could do worse than a little sipping bird.

Now, we've gone all over the place with that. But yeah, yeah, that's a really powerful thought is that like the fact that you could be trading a very small risk for a ton of complexity and say i'm taking on this very small risk and i'm fine with it actually yeah it's all right you know what i'm not going to worry about what happens if you run out of disk space um and all the cleanup code that might go with it i'm just like well if it happens we'll get paged that's fine or that particular process will die the machine will get recycled the new one will come up and we're going to look for why that happened but we don't have to have proactive like cash whatever you know just again top of mind for me so i'm gonna keep drawing on what i've been doing the last couple of hours no that's really interesting yeah yeah and i mean

the other thing that you can do with this we're talking about trading off like how do you get rid of complexity right this is the thing that feeds on itself this is the thing that makes it harder and harder to add stuff yes we want to eliminate accidental complexity what a lot of people would say is technical debt what ward was talking about with that very narrow definition there okay yes we want to get rid of that but sometimes the only way to make this better is to get rid of essential complexity and that's by changing the problem everyone has seen like the kanban board or the task board or the Jira, whatever. Here's all the 75

things that we need to do, right? What percentage of those things are remove this feature? Yeah, that's never a... It's never on that board, is it? Very rarely. Very, very rarely. I have one right now, but it's mostly a, we migrated a system and then we should get rid of

the old system. And you know how long that has been kicked down the can? About three months now. I'm like, yeah, it's still not really important enough to get rid of, even though. Yes, yes. Or maybe even more so, here are all the features

that are not carrying their complexity weight. They're useful. They're valuable to somebody. But we've decided that, you know what, the 10,000 lines of code that we have in our system that supports this one feature that is not worth it is going to be removed. And there's been the conversation with the group

or the person, whoever it is that likes this thing. It's like, I'm sorry. I know you like this. Here's this alternative that we wrote for you in 12 Lines of Bash. Right? It's not as good, but it does the job. But it does the job. And more importantly, yeah.

It will save us this giant chunk of complexity. Maintenance headache. That, I mean, you know, a not totally off way to think about these kinds of things in terms of managing complexity is, again, regardless of whether it's accidental or essential or whatever, is imagine if every time you want to add a line of code to your system, you had to read log base 10 of the lines in your system, right? So you can very easily and quickly see

how you're going to get slower and slower and slower and slower the more stuff that you add. And so thinking about how you're going to manage that complexity and thinking about what tools you have in your arsenal for getting rid of obviously getting rid of accidental complexity great if you find it get rid of it you find code that you don't need anymore delete it you find things that are too complicated can be simplified great do that but you have more tools than that. Right. And you need to use them because if you don't, what will eventually happen is your system will turn into a haunted graveyard. It will turn into the thing that everyone is scared to touch.

Like, oh, don't go there. Every time you make a change to that thing, it breaks because nobody understands how it works anymore. Right. Right. Right. And then people are like well we just need to rewrite it right and then you rewrite it and then the cycle begins anew and you have a whole phoenix from the ashes of the previous system with all the same flaws right

how dear listener how many times have you seen in your career the phenomena of the old system that has turned into the haunted graveyard that nobody wants to touch and then they bring in a new person to maintain it or maybe a whole two team and they look at it for a few months and like the solution here is a rewrite we're going to rebuild it yeah and then they rebuild it and then five years later the exact same thing has occurred i can right yeah yeah we we actually we turned that around uh so uh one of the things that i've seen happen a number of times is that if you've got a a a facility that is sort of not exactly required you know maybe it's a visualization tool or a monitoring tool or in in our case a simulator for something then uh that's when the

you know junior folks come in and you say hey we're going to rewrite it again um and you get to do it as a way of learning how we do software development here or we're going to write it in a new language that we're just trying out here and if it you know we know that we're going to rewrite it again in another year's time because we always do so at least in that way you're kind of capitalizing on the idea that this feet this this thing is essentially permanently cursed right yes it is just it's it's a haunted house on purpose right you know yeah it's like a haunted house where you go there it's just a fun time oh it's meant to be like this i see okay right you know we wrote it in you know scala have you got anything else written in scala

no it was just seemed like a fun thing to do and you were going to rewrite it again anyway so gosh that is scary let's not offend anyone who but uh but yeah no it's true that that certainly um larger code bases in particular suffer from this or larger areas especially when you know you mentioned something about like hey this is this code used? Or is it used by one thing once? Or is there a test? You know, we've all seen code bases that grow and grow and grow. And nobody has culled out the stuff that isn't used. And if you're not careful, you find that if you're very lucky, you find the only use is the test that tests the functionality.

And then you ask yourself, well, it apparently is used, but it's used by a test. Does anyone other than the test use it? No. Do we really need it um because it's it is a bit of a millstone now with all the automatic refactoring tools that we've got it's a little bit easier to carry around unused code a little bit easier but in in the old days should we say of like greps and finds and replaces the more chances of hitting things that you don't need um sorry the more um the

larger your code base with unnecessary mentions of things that maybe or may sound like the thing that you're trying to change the more likely you are to get like collateral damage of hitting you know your regex gets renamed the wrong variable or whatever or you forget to update one case of it or any other things like that yeah i mean I think even with the refactoring tools, there's a cost there because the tree of dependencies isn't shaped the way that you want. If you've got some piece of code that's unused, so nothing calls it, right?

It can still call lots of other things. Yeah, that's true. And by doing so, it puts a constraint on what those things can be. Yeah, okay, that's fair. Right? I mean, how many times have you had this thing where you're sort of like,

wow, I really want to change this return type, but if I do, it's going to... So what calls that's fair. Right? I mean, how many times have you had this thing where you're sort of like, wow, I really want to change this return type, but if I do, it's going to... So what calls that? What calls that? What calls... Nothing! Nothing calls this! This whole chain is...

Why have I spent the last half an hour looking through this code? I could have just changed it as soon as I saw it, right? Yeah, that's true. So, yeah, that unused code... And I mean, you know, you say it's sort of like, you know, do we need to keep it? It's like, well, you've already kept it.

If you're looking at it, there's a very good chance it's in your source control. And then you can go and get it. But actually quite difficult to remove from the source control. But then, you know, the counter to that, and this is a weak counter, I think, just thinking out loud, really. The counter to that is that if it is in source control it is stagnant if you come back to it like three months later and go didn't we have a thing that did the other stuff and then you find that it doesn't work anymore but then what you're trading off is the cost of the of the incrementally keeping a piece of code up to date and the transitive closure of things that use it or have been used by it right that maybe is like is like a null set or maybe is a large set of things, but they're just inconsequential and they actually hurt you. Right.

That's one cost against the speculative cost of if we were to remove it and then have to resurrect it. Talking of haunted graveyards and things again. Right. Yeah. If you do need to resurrect this piece this uh piece of code and functionality you might have to do a little bit of surgery there and then that seems often like that might not be as bad as it sounds now obviously if everything's changed under your feet it's no longer a version seven of this library is now version 94 and the whole api is gone and you don't know what you're doing with

it anymore but maybe yeah it's very subjective i guess it depends on the thing but like again this is a risk or a trade-off that we're talking about at the beginning of this like what's the risk if i take this out there is a risk that we may need it in future and you could actually say i'm removing this capability right and trading off for the risk that we might need it again in some indeterminate amount of time and maybe I can wear that risk. And in doing so, I have removed some essential complexity because it's essential because it supports a capability that we say that we needed. Yeah. But I'm trading it off a risk. Yeah. And I think there's another bet that you're making there when you keep that code

because you think you might need it one day, which is your option is delete it and recover it from source control at some point in the future when you decide you need it again, or keep it and pay the cost to maintain it through all the transitions of all the other code between now and then, right? Yep. It is entirely possible that the work to reincorporate it in the future will be smaller than each individual change that you have to make. Than the integral of the area of the graph.

Right, because you might change it one way and then change it back the other way and then change it one way and then change it back the other way. And if you could just hop from the beginning to the end, it would actually be way cheaper, right? Yeah. And it's hard to say.

And so for me personally, if there's code in the code base that isn't used, unless I know, like I'm know, for me personally, if there's code in the code base that isn't used, unless I, like, know, like, I'm working on this next week. I'm going to add this capability in next tomorrow, the day after, right? Like, it's going to be there, like, it's the next thing in my list. You know what I mean? Yeah, right, right. Unless it's something like that, I'm going to delete it every time.

Because I know it's in source control. I'll know I'll bring it back. There's a very good chance that when I bring it back, it'll actually be easier to reintegrate. And if nothing else, I can sort of reevaluate the capability in the context of the new design, right? Like I can be like, okay, this is the capability I want. Here's some code that does it. Actually, now I only want these like eight lines. I'm going to put them over here instead of where they were before.

A step back from it, a little chance to ree-evaluate it in the light of new evidence instead of the incremental, like, well, I'm just keeping the test passing aspect of it that maybe you were doing before, which is not as thoughtful as, sorry, intentional, you know, as the, okay, now I've got it in front of me and I can make it fit the design as it sees as it is now as as it has evolved in one step rather than the thousands of paper cuts along the way yeah yeah

exactly exactly so i don't know but this is this is kind of my my my proposed replacement for technical debt and this is again when you're talking when you're talking to programmers when programmers are talking to programmers they don't need to use metaphors. If you want to keep using technical debt with your boss because he knows what it means and you know what it means and you have a common understanding of what it means, that's fine. I'm not telling you not to do that. But if we're talking about code, we can be more precise. We can talk about risks. We can talk about capabilities, functionality, whatever you want to call it.

We can talk about complexity. I think every programmer suffers at that, right? Suffers from that. Yeah. And we can talk about the trade-offs between those things. Yeah. Right? Because sometimes those trade-offs really make sense to make. Yeah. No, no, it makes a lot of sense to me. I was just about to make an argument for one of the few places where I see complexity budgets being spent in places where

you wouldn't expect them to be but it's still essential complexity is in high performance code but i think that's just a capability you know the capability of like processing something in a certain amount of time you know i i was going to say but performance is another category but no i think we can quite reasonably call it a capability or a sort of constraint, a design constraint on the system. It must be able to do X, and you can say it must be able to do X in Y or with latency lower than whatever. We've all seen those bits of code, right?

The beautiful design where you've got every little object that has its own little responsibility, and then ultimately you go down and say no it just needs to be super quick i'm just going to sit down and write the very complicated thing with all with this more comment than it is code because i'm explaining why we're not calling this whatever and we're resuming it yeah and then you're like this is costly and under the far extreme of that actually i was giving a um a chat at lunchtime a couple of weeks ago, and I was walking through some of my late 90s era Dreamcast code, which includes some Hitachi SH4 assembly code as well. And it's like, you know, you've seen my code. You know that I tend not to comment that much.

I rely on useful variable names and function names and extracting. But when you're writing assembly, you've kind of got no choice. And so it was just a wash with this prose about what I'm doing, how it's doing, why this allocation is like this, why the format is like whatever. And everyone's going like, why is it so well commented? And I'm like, well, because it's so complicated. And then I said, the problem with this, though, is that if someone turns around to me and says,

can you make it do X? I'll be like, no, I can't. This is a huge deal now, and it's hard to change. And that's the complexity. Anyway, that's a complete aside. I guess, looking at the time, we should probably, I think we've reached a natural conclusion here. I think you've said what you needed to say about this, which is that technical debt is probably not what you think it is,

unless you're Ward Cunningham or Ben Rady. And now maybe anyone who's listened to this podcast, maybe it's excusable to colloquially refer to any naff bit of the code base for whatever reason as technical debt to non-technical folks yeah but it's probably no it's not probably it is definitely more useful to describe it in a more specific term and you're you're uh when you're talking to other programmers and it's either a risk a capability or a complexity and those are the things that that you that you are knowingly trading between and the yeah that makes sense i like that this is what i'm aspiring to do i mean

i still use the term technical debt like it's it's it's something where it's like i catch myself and i'm like can you be more precise about this ben can you can describe what you're trying to say that's when you have your one-on-ones with yourself? In the shower? That is the place where most of the real deep thinking happens, isn't it? It's like you're just the right side of wakefulness, but not yet fully booted up. You've not had the coffee or anything like that.

You've got no other distractions. And yeah, it's like a meditation, isn't it? That's when you really, these things happen. Or sitting in the park next to Alice. All right yeah we're feeding the ducks with alice all right that sounds like a perfect place for us to finish cool see you next time bye you've been listening to two's compliment a programming podcast by ben rady and matt find the show transcripts and notes at www.twoscomplement.org. Contact us on Mastodon.

We are at twoscomplement at hackyderm.io. Our theme music is by Inverse Phase. Find out more at inversephase.com.