The Changelog: Software Development, Open Source - You call it tech debt I call it malpractice (Friends)

Episode Date: September 1, 2023

Go Time panelist (and semi-professional unpopular opinion maker) Kris Brandow joins us to discuss his deep-dive on the waterfall paper, his dislike of the "tech debt" analogy, why documentation matters so much & how everything is a distributed system.

Welcome to Changelog and Friends, a weekly talk show about the Dewey Decimal System. Thank you to our partners for helping us bring you the best developer pods each and every week. Fasty.com, Fly2IO, and Typesense.org. Okay, let's talk. Well, we are here with our friend Chris Brando. Chris, thanks for joining us on Chainsawgun Friends. Thanks for having me.

I love this, Jared. Getting to spread the host love even, you know? We've had Matt on, but never Chris. That's right. So Chris is a GoTime regular. Yeah. Been on GoTime for a little while. I remember when I first met you because it was at GopherCon Digital Edition.

I don't know if you were physically there, but I certainly wasn't. Were you there? I think, yeah. I think because that was you were physically there, but I certainly wasn't. Were you there? I think, yeah, because that was in what, this winter of 2020, right? Something like that. Yeah. So I think I was there. I think I gave a talk at that GopherCon. Okay. So we did a show. Matt hosted it. I was just the producer behind, the guy behind the guy. And the show was with you and Angelica Hill and Natalie Pistanovich and Matt hosted. And that was when I first met you and we needed some more panelists at the time. And so I was kind of there, you know, listening, scouting different people, looking at talks and,

and just wondering who could be a good go-time panelist. And literally I just liked that conversation so much. And the three of you that afterwards, I think I just was like, would all three of you like to be GoTime panelists? And they all said yes, and we all lived happily ever after. So that's how Chris came to be a GoTime panelist, at least from my perspective. Does that sound right? Does my memory serve me? Yeah, I think I had actually been on an episode as a guest over the summer. Oh, that's right.

And then, yeah, I got like you invited me to be on the show for that GopherCon. And then you were like, yeah, you all want to be on? We're all like, sure. Yeah, that sounds like fun. Like, let's do it. Yeah. And it has been fun. So one thing I've learned about you, Chris, through the last couple of years is that you really shine in the unpopular opinion segment of the show.

I mean, I might even consider that your wheelhouse. Sometimes I think Chris is there just because he's got a couple queued up and he wants to unload his unpopular opinions. But another thing that's interesting about you, which I'd love to hear more about, is I think that you're from a software engineering background. Your parents or your dad at least, you have some of that in your background, but you didn't want to do that.

You want to be a writer. Is that right? Yeah, exactly. So both my parents, well, they both studied computer science in college. That's actually where they met. And my mom was actually the first one to start professionally working within the CS field. She was a programmer for a decade. And then my dad got his master's and

then he's been a software engineer ever since. He does a very different type of software engineering for me. He does very low level stuff. So he's writing a lot of C, a lot of Ada, that sort of stuff. And my mom, I remember my mom told me she quit programming because she spent some absurd number of hours. I think it was like 16 or 30 hours or something trying to find a problem with the compilation of a program and it turned out to be a missing semicolon somewhere. And she was just like, I'm done. Like, absolutely not. The last straw.

Linters or something. Even with that, she wound up going back and doing like not CS stuff, but where she worked for quite a while was a psychiatric hospital. And she managed their entire phone system that was running on DOS for a very long time. So even as she left programming, she was still within the wheelhouse. But as a result of that and growing up around all of that, I was like, I don't want to do this as a career.

No semicolons in my life, please. Every family gathering would be like, oh, you're going to go into computers like your dad? And I'm like, no. I'm going to be an author. That's what I'm going to do. Building computers is fun. Video games is fun. But I want to go be an author.

Writing is my passion. Writing is what I love. And my dad gave me this wonderful advice when I was getting ready to go to college. He was like, I'm paying for college. So you go to college and you choose a major for something that you love and you will figure it out afterward. My dad has lots of great advice, but that's one of the best pieces of advice he ever gave me. So I was like, okay, I'll go. I went, I got my creative writing degree. After a bit of shuffling, wound up with a second major

of broadcasting and mass communication and really fell in love with the world of not just writing, but also audio and video. The original plan for me going into college was actually to become an audio engineering major, but that major wasn't really ready at the time. So I kind of did this shuffle and wound up getting what was effectively an audio minor. But yeah, so I got my writing degree and I was kind of building websites and things along the way for a while. And then it really ramped up my end of my junior year going into my senior year because I actually got elected as the basically IT person for the television station that I worked at because we had this student run television station that was like completely student run, like no adults in the

room really. So it was all of us maintaining this this multimillion dollar enterprise. It's kind of fun. Yeah. And I was like, we have this like $4 million studio that like, you know, Al Broker was an alumnus of my college. So he had donated a bunch of money with NBC to build us a studio. We have all this nice, shiny digital equipment. But we still managed all of our field equipment checkout with paper forms. And I was like, this is stupid. Like, I don't like this. So I sat down over the summer and I built an equipment reservation system using Drupal, good old software. There was like a module available that basically did exactly what I wanted. So I went in, I configured it, set the whole thing up. I was like, okay, great. Now we can do equipment reservations online. And my college caught wind of this. And they're like like we've been looking for a system like this for years and we cannot find one that's affordable and and you clearly just built one can you build

one for us and i was like yes and my advisor was like for money i was like yes for money so that was my first paid software engineering gig nice wow building and configuring the system for is it still in use uh no it is not they used it for a little bit there was actually a snafu where i wrote a whole user manual but they never gave the user manual to the person that was operating it so he had no idea how the system worked it was just a very miserable experience and he was like i like went back and saw him he's like this thing is not is not working. And I'm like, I'm pretty sure I wrote down how to do this in the manual. He's like, manual? What manual?

I'm like, oh, God. And that's when you learn the acronym, RTFM, for the first time. What manual? That's too bad. I built that. I built a whole bunch of other websites. And then I was like, I kind of like this website thing. I like this Drupal thing.

I think my options are be a writer, which was fun in college, but professionally is a lot more challenging to do. Yeah. Or go be a broadcast engineer, which would have been a lot of fun. A bunch of my friends were broadcast engineers. But I was like, this website thing, this website thing is fun. So I'm going to keep doing this. And then that's when I also realized the irony of me spending so much time being like, I'm not going to go into computers like my dad and then winding up in computers like my dad. That's hilarious. When did you learn the practice? Like, did you learn during that Drupal project or did you kind of pick up things from your parents over the years? I mean, you don't get it through DNA, right? You don't learn to program.

It's my point. You do not. It is through uh a lot of environmental so i think i built my first computer with my dad when i was like eight or nine this was the 90s so it was like i had this computer actually had two computers when all my friends had like one computer for their whole family so it was already in this like ecosystem of being around computers and building them and understanding them and then i i still have the first programming book my dad bought me, but I think I was like in middle school when he bought that for me. So I started programming then, didn't particularly understand it. And I took C and C++ courses in high school, did pretty well in that, but just like it, programming didn't make any sense to me. I was always very confused about like,

what use is this? I remember being like, where's the GUI for this? There's no like user interface. How does any of this work? But it was in college when it actually finally clicked when I was building that equipment reservation system and some of the other websites I built that I was like, oh, this is what you can do with this technology. Now I understand, you know, how to use and how to do it. And then from there, it was just like, I'm just going to dig as deeply into this space as I can. And so really, it wasn't till like after I graduated college that I really started to write software. Like even in that all that time I was building that reservation system, I think I built like

six or seven other websites for various entities and people. I was only doing like Drupal themes. I had never written a module. I didn't write a module until after I graduated. I didn't really learn PHP until after I had graduated. And it was just kind of this mad dash forward from there, which has also caused me to have this really weird career. I've said it to a whole bunch of people

that I've never been a junior software engineer because obviously when I was building those websites, it was just me. Like there was no one else around me. No one had any clue what I was doing. Right. And then over that course of time, I got so much knowledge in Drupal that my first couple of jobs, I was like the Drupal guy. I was the expert in this technology that we were using. And I got hired specifically because I knew so much about Drupal.

So my whole career has been a lot of like architecture and, you know, principle level design and implementation stuff like designing entire systems, fixing entire systems where there's no one to go to, no one to refer to. And I'm having to like mentor people about, you know, this is how you, you know, build a model, a Drupal module. This is how you should structure your code. This is how you build a Drupal module. This is how you should structure your code. This is how you should do these things. Yeah.

Well, that all sounds like it makes sense to me, just knowing you for the brief amount of time that I have. Because another thing, besides the fact that you thrive on unpopular opinions, is that you do seem to like to go very deep on specific subjects, specific technologies, even to specific conversations, like continuing to drill down deeper and deeper, which I appreciate as a guy who's, I'm a generalist always kind of have been, I have like certain areas where I go deeper than others, but I've never like gone all the way down a rabbit hole on a specific thing because I'm always like

trying to get stuff done you know I'm kind of pragmatic in that way but because of that I don't do things like read papers you know that's kind of razzing you because you have this blog post that came out of our unpopular opinion segment about the waterfall paper and I was like wait people read these papers I thought you just write papers and you publish papers and then we reference papers, but do we read them as well? Because Chris does, but I'm looking for the, the CliffsNotes always. Is that, have you always been just like willing to put in the time to just sit there and read like long sessions of reading as well? Or how do you go about acquiring the deep dives? Yeah. I have this deep desire to understand how things work and like my

brain will not rest until i have like a satisfactory understanding of whatever it is i need to like learn and i also have this deep appreciation for nuance so i can't take like simplistic explanation of things and be like oh okay, okay, yeah, that's fine. So like, with the waterfall paper, I was specifically one of my friends actually pushed me to go read the whole paper, because I, for years, I've known like, waterfall is not how we describe it. It's something that's very different. The paper doesn't really say what it said. But that was actually when I first read the paper, it was really just to satisfy your curiosity around like, where did this diagram come from? What is this diagram? And I saw and I saw the other one in the paper. I'm like, okay, that's enough. This is an iterative development model. And I like went on. So it was deep in some other

rabbit hole. And this happened to be a tangent. But I was talking to my friends a couple months ago. And I was talking about, you know, software development and software engineering process. And he was like, Oh, yeah, yeah, you know, this thing is like the program design person in the waterfall paper. And I was like, program design person? I'm like, what are you talking about? I was like, so I went home after that. And I sat down and I read through the paper. And my mind was just like, oh, no, no.

This is one of those things where like everybody has gotten it wrong about what this paper says. Like all of the summaries, all of the stuff, like all of the summaries, all of the stuff, all of the industry knowledge. I'm like, this paper does not say what people claim it says. And that's where it just went from there. Yeah. I wonder how that happens. Besides, I joke that people don't read papers. Obviously, people do. I'm just not usually one of them. But it does seem like as we institutionalize, so to speak, knowledge to be passed down, which is

what books and papers really are for, we've failed seemingly over and over as a software engineering industry to receive the pass down. And that's why we find ourselves reinventing things from the 70s. Good ideas get lost, they get refound again. One of the reasons why I like podcasts, by the way, is because it's kind of a more approachable way to cross pollinate ideas across the industry. Not quite as thorough, of course, but at least to give a seed that somebody can go and, you know, put the time in to dig deeper. But like, why do you think that we all think of waterfall so differently? Like I grew up in my career in the agile era and I learned about waterfall in college. I remember that being like so bored because I didn't have any context

to apply it back then. I'm like, I've never written a program and you're teaching me about waterfall software methodology. You know, I was like, okay, I can see the diagrams, but there's no, I have no frame of reference for how this would fit into a real thing. And so I quickly forgot it. But in the Agile era, it was just kind of like the sense is waterfall bad, Agile good. That's right, right? Waterfall bad, Agile good. Why?

Well, waterfall, you try to design it all up front and you pass it to the next silo and they do their thing and they pass it to the next silo and they do their thing. And then at the end, you built something that you didn't want in the first place. And with Agile, we're recognizing that we have to be able to change things as we go. Like that's the kind of a basic reason why Waterfall, Bad, Agile, Good. But I didn't realize like what you just told me a couple of weeks ago about the waterfall methodology actually being an iterative process. Yeah.

I thought it was a one, two, three, four, five. We're done. Yeah. That's like the biggest misconception about the paper and what, you know, Royce is advocating for is that, yeah, it's, it's iterative. It's not like it's, you know, when I first was thinking about this, I was like, oh, perhaps it's buried somewhere in the paper that it's iterative. And he kind of talks about it as this linear one step to the next step to the next

step thing. And it's on the first page that he says it's iterative, right? It's literally before you even get to the diagram showing that traditional waterfall with no backflow, the page before that says, oh, this is an iterative process. And here's the diagram showing the backward flow of steps. So it's not like it's buried in there. It's just, I don't know how it got missed so much. Well, I guess I have an inkling of how it all got messed up. Because there's a couple other papers that, you know, there's probably tons of papers I could have used, but Wikipedia is like, usually a pretty good source for info. And so there were two other papers that are kind of talked about in the history of waterfall. One of them is where the term waterfall actually got coined from where the

person's like, oh, this is the Royce's waterfall method of development. And the other is just describing a development process ish. That last paper I mentioned that the one that showed me the development process, the author writes an editor's note for that paper in the 1980s and it's like i don't know where it went wrong but people got the idea that software is something you manufacture instead of something you design and build and engineer and that seemed to be where part of the issue came from because he also mentions this divergence of programmers from engineering methodology which is also was happening in like the 70s and 80s and 90s as we kind of get toward more of like the google culture of engineering and he attributes that as part of

the problem as well because it's like oh these people want to be special they don't want to be seen as engineers they want to be seen as something else and that's led to oh, these people want to be special. They don't want to be seen as engineers. They want to be seen as something else. And that's led to all of these problems because now we're trying to manufacture software and you can't really manufacture it. But also the people who are in charge of making sure that we do things well don't want to follow any of this history. They don't want to follow the engineering methodology. They feel – I mean he doesn't say this, but it's kind of implied that they kind of feel like it's too constraining for them. They want to go do – they want to be more free in this way. They build software.

Right. Well, a couple of things. First one is that as a software developer or engineer, whatever you want to call, you know, us today in let's just call it in the 2000s era. I do not have an appetite for rigor. I'll just confess. I really don't. And even looking at this, and I'm like, this looks like not what I thought it was. And when I say looking at this, I'm referring back to Royce's paper and specifically the steps in the process. He does say that there's kind of two different kinds of projects and at a certain scope size,

you don't need any of this stuff. And so I'm like, maybe I'll just like to live right there and let y'all do the rigorous stuff. And that's, you know, there's plenty of work to do in the world. But I don't think I'm all that different than anybody else. When I say I just don't have an appetite for the rigor. I want to just iterate and change and be like you said, like, I want to be more free as an engineer and i feel like that's typical what do you think adam is that a sentiment that you hear people say or am i weird like that i don't believe you jared i think you do have an appetite for rigor i think uh you lead yourself to believe wrong because i think that you are you do make pragmatic choices i believe yes but i believe that you have done it so long that you forget that you have rigor. It's just built into the way you operate.

I believe that you operate with rigor, but you're also very pragmatic when you apply your effort. And so in your effort, you're rigorous and you probably have an appetite, despite what you say. Well, I appreciate that. So that's what I think. And if you represent a population, then I think that others might have the same. But you might also think that I'm pedantic. And Chris would say, I'm nowhere near pedantic enough, because that's another one of his unpopular opinions, is we think we're pedantic,

but we're not as pedantic as we need to be as software engineers. So maybe you and I think I'm rigorous, even though I just said I don't think I am. I think I have certain things in which I apply rigor, but I don't have an appetite. I think I have certain things in which I apply rigor, but I don't have an appetite for it. Like, it's not the kind of thing where I was like, like I believe a lot of what Chris wrote in the piece about the waterfall and your overarching theme is like, we need more documentation, Chris.

And which as a writer, I find that hilarious. But as a non-writer, I'm like, yes, Chris is right. We need more documentation and somebody else should do that. Somebody else should do that. Somebody else should do that. I mean, there is this element of rigor I think being a little bit different than what people think it is.

And I think that actually is what has led to a lot of the problems we have, is that people think that rigor is like we have to write all of these documents and they have to be in this very, I guess, in a way, academic writing and we have to be very formal in how we're describing things and how we're designing our systems.

And there is a place for that level of very precise documentation, which is like what Royce was getting at. Royce was building spaceships and stuff, rockets to get us to space. For sure. And those places need. Yeah. You need to document things very, very well because there's very low margins for error.

And I think a lot of people are not in that space. So the type of documentation you need isn't as much a, it has to be strictly formal so we make the fewest mistakes possible. It needs to be more of, do you actually understand the problem that you're trying to solve? And should you be solving the problem you're trying to solve? And answering those base level questions. And that's kind of fundamentally what the engineering process is about, is like, do we have enough information to actually proceed forward? Which is one of the things that Bennington in his paper highlights is the top down methodology that they're all used to as engineers isn't about you know you write this doc

and then you're with the next doc and then you're with the next doc and then you write the next doc and then you go build the software it's okay we're trying to write this doc do we have enough information to actually write it or do we need to go do a prototype and experiment and acquire more information that's the thing i think that when we say we need rigor that's what we need and i think that's something that you jared do quite often often. I think you do find like, you know, information and you use the information to decide, should I do something? Should I not do something? That's like your whole motto of like, keep the main thing the main thing. Yeah.

And I think a lot of software people wind up trying to build stuff without enough knowledge of what it is they're trying to build. This leads to the estimation problem that we have where it's like people cannot estimate how long something will take. And we say, well, this is because estimation is so hard. It's like estimation is not that hard if you have the proper information. But if you're sitting in a room and one person's like, this is gonna take two days,

the other person's like, this is gonna take two weeks, you need some way of knowing which person is right. Or if they're both right, why they're both right. Or if they're both right, why they're both right. Or if they're both wrong, why they're both wrong. But no matter what, if you have that type of discrepancy, that's definitely a sign that you need to go do some more exploration, some prototyping, some investigation. Something to acquire more information to pull it out of that realm of just optimistic engineering judgment. Because they're probably both wrong and it's probably going to take you two months. So you probably miss a lot of things. And you're thinking,

take your number, multiply by three, now you're good to go. Well, I think in a sense, so one thing that I do a lot is I do nothing. So when I do nothing, I pitch that as me being lazy or procrastinator and stuff. But I think, as I I observe in my experience, what I'm actually doing is information gathering and delaying decision-making until we absolutely need it. Because I've found that when you make a decision too quickly, like you said, Chris, you're probably going to be wrong. And the longer you can put that decision off, now I understand some people have, here's where it hits the road is like you got budgets and timelines and pressures and bosses and, you know, demo days and like all this stuff where it's like, I can't delay any further.

I have to make a decision today. But anytime you can afford yourself more time and just like, let the current thing be what it is, even though, you know, it's not optimal, really feel the pain for as long as you can. I mean, and sometimes I feel for so long. I mean, on our kisels, we laugh. I'm like, nope, still not ready. I'm still not done. I wasn't ready for this.

I'm really just procrastinating, so to speak, but I am information gathering. I want to make that decision with as much information as I can have. And depending on what we're doing, what the feature is, what the project is, I don't have all that much time. I can't say,

well, I'm going to spend 40 hours on this. So I'm just going to slow play it and make that decision with as much information as I have. Now, if I was full-time engineering like you are, Chris, I would obviously be putting the time in to make the decision by the end of the week or by the end of the quarter or whatever it has to be. But I think that in that way, it seems like I'm being lazy, but in a sense, I'm actually just being rigorous. So I like the way that that sounds, because now I sound better. This has definitely been one of my experiences working in software engineering organizations, where there's this sense that you need to move a lot faster than you actually need to move. And it's actually one of the things I've been thinking about a lot lately, because I'm in the process of trying to like build my own thing try a new career adventure oh really yes I've decided that I actually want to be a writer and that writing is

is my true passion I think I can help the world a bit by doing that writing more so than I can with software engineering especially in distributed systems which we can talk about in a little bit is this your resignation from go time or or what is this? This is not a resignation. In a way. So an interesting little side bit, I've recently realized that I don't see writing as strictly only typographic writing, which is writing on the page and publishing and all that. I see writing in the sense of the suffix of graph. So you actually have typographic, but you also have, you know, photographic, audiographic, videographic.

So I consider all of those forms of writing that I want to engage in, which is a long-winded way of saying, I like being on GoTime because it's a form of writing for me. Gotcha. There you go. But as far as like actual

software engineering is concerned, I found myself continually walking into environments where everybody would be like, no, we need to move fast. We need to fast we need to make decisions we need to make decisions and it's just like y'all like this is a multi-billion dollar company we'll be okay if we take an extra three days to figure this out or we'll be okay if we take like a month to get this path right or get these details correct but there's this sense that you always must be moving quickly

and you always must be you know moving forward and i think there's also this sense that you always must be moving quickly and you always must be you know moving forward and i think there's also this sense of uh loss aversion to an extreme degree yeah um like i've been in environments where it's just like nobody likes this process why are we still using it we all know it needs to change we all know it's not giving us what we want people are like well we use this process and now we're successful, so we should keep using this process. And I'm like, that is filled with fallacies and we need to move away from that and away from that thinking. Success hides a lot of problems. You think this process brought me the success, but oftentimes the success came from somewhere completely different and the process just happened to be there along the way whatever it is and it's like it's correlation and not causation yeah i've seen that a lot as well i've done it as

well i'm like is this actually what's producing the goodness are we just doing this thinking that it's producing the goodness you know because actually it's holding us back what if we could ditch it and still have the goodness and have a better process at the end of the day everyone's just chasing things right like these in the case of a multi-billion dollar company, maybe less so chasing, but like they got bitten by that. We should be a startup bug, right? We should have many startups inside of our large Exxon style company, for example. I know people within Exxon and they operate that way.

Like Exxon's big, but their division or their small troop is very startup-esque. And so they feel they have to impress, and there's some sort of urgency that is perceived by their bosses, higher-ups, or whomever that makes them move at a pace that's just not necessary, like you said, Chris. And then literally in startup land, they are in a rush because they haven't built the thing. They're in a rush to reach product market fit in most cases because only then should you apply more financial pressure, i.e. more funding to grow that model, whatever it might be.

Only then should you apply more heat to the fire, fuel to the fire, whatever it might be. But there's this rush to build and i think that's sad that software engineering has to feel the major brunt of that effort really of that that concern because it's software that's really eating the world it also becomes a bit ironic in that like there's not really like the guidance is kind of all over the place it's like well we need to move really fast but if you look at you know all the historical sayings and tropes we have it's like measure twice cut once it's like you know if you have x amount of time to do something spend 90 of it planning and then the last bit of execution it's like lincoln yeah he was successful yeah i mentioned that recently yeah yeah you did and he said if i if i had uh i think it was

six hours five hours some number of hours to cut down a tree. I'll spend the first, you know, 90% serving my axe essentially. Like I want to have a sharp axe so that when I begin to cut, it's a precise and measured cut. Yeah. And this is like kind of the same sort of thing that people like Rich Hickey put out there as well. It was just like, if you have a very small amount of time, you need to be very diligent with your planning because you have very little room to mess up. And I think that's like, I guess this is probably unpopular opinion because I always talk about unpopular opinions. Oh, my gosh. But I think startups actually move far slower than they should be moving because they refuse to admit that planning is required. Documentation is required if you want to actually be able to sustainably move quickly.

Everybody rushes into that first like, we're moving so fast for the first six months. And then, you know, six months later, everybody's like, we don't have any idea how any of this code works. And every time we touch it, we introduce seven bugs. And what are we going to do? And it starts like kind of eating away at their company as they try and push forward. And they just start hitting wall after wall after wall. And either, you know, they wind up getting acquired or they go public or they just disappear

because they just can't maintain the speed that they had in the beginning. And I think a lot of times people just don't understand why that is. They're just confused about why is it so hard to actually do something that's supposed to be simple? Like an example of this is at a previous job, it took so long to spin up a new microservice that people started repurposing microservices that we were decommissioning. So it's like, oh, we have the service,

we don't need it anymore. People would be like, oh, we're just gonna rip the guts out and put the new thing in because it takes so much effort to spin up a new repo and get it all working and all of that. And this was on a code base that was probably, or at a project project like it was greenfield project that was less than two years old and they we could not get a new microservice up and running which is kind of ironic because the point of microservices is that they're supposed to be small and yeah should be able to get lightweight yeah

spun up real quick all the fat but just couldn't do it wow so what So what you're saying, Chris, is that they acquired a whole bunch of technical debt. I would say. I'm pushing his buttons here. Technical debt is our next topic. This is another one. Another thing that Chris likes to do as a writer, as a graph person, he likes to think deeply about words and their meanings and analogies. And so him and I have that in common.

We like to bike shed the meaning of words. And is this a good analogy? Is this a good name, a bad name? And you took issue with the tech debt analogy, which is why I was pushing your button there. I think that the tech debt analogy, we should get rid of it. Because I don't think the thing that we're talking about when we're talking about tech debt is debt. should get rid of it. Because I don't think the thing that we're talking about when we're talking about tech debt is debt. I think it's more akin to malpractice and people are being irresponsible. Because I think most of the time when tech debt gets brought up, it's like, oh, we're just going to skip writing the test or skip writing documentation so that we can get this thing out the door faster. Or we're just going to code this in a really messy

way so it gets out the door faster. And I'm like, that's not debt. That's you not doing your job properly. Please just write the comments and the docs and the tests. It's part of the job. You can't cut out vital things. Or if you do, then you're committing malpractice. And we should call it that. And that's why I think it's going to be unpopular. So it's not tech debt, it's malpractice. Which I think is a great analogy. And so one of the things I like to do is disagree with people. The tech debt analogy is a good analogy

and the non-tech debt, the malpractice analogy is bad. Well, there's some nuance here. Chris, explain maybe your side of this stance because I think it's shifted slightly because we've had some Slack conversations. I agree with you on one aspect, Jared. I think tech debt is a good analogy. I think where I draw issue is that it is just an analogy. It's a thing you use to help people understand something

when they're not in your contextual domain. The problem is we're using that analogy not with people who don't understand our contextual domain, not in your contextual domain. The problem is we're using that analogy, not with people who don't understand our contextual domain, but with people who do. Right. So it's like we use it with each other to try and explain our code bases and their state to each other. And that's where I think it really starts to fall apart. Because if you're not careful with analogy, I think that's something you learn as a writer is you can only stretch an analogy so far before it just becomes ridiculous and it starts falling apart. And I think that's basically where this tech debt analogy is,

where it's like everybody, like people that use it want to use it to explain so much of software, so much of like how you should make decisions about what you, what you, you know, what choices you make and abstractions you use or how you build things. I was listening back to the TechDebt episode that you all did.

Yeah, with John Thornton. Yeah, and one of the things that stood out to me was this whole, oh, well, it's TechDebt to build this abstraction and only build the front end of the email service and then not build the back end for it. And I'm like, is that TechDebt or is that just planning stages? It's like, oh, well, the first thing we want to build is the front end and it. And I'm like, is that tech debt or is that just planning stages, right? It's like, oh, well, the first thing we want to build is the front end and we want to get that right.

And then the next thing we're going to build is the thing that actually can deliver all of the emails. And that just seems like bog standard planning and approach of a problem that gets you, you know, the biggest bang for your buck along the way. And it's kind of weird to, at least to me, to phrase that as debt. Because I think that is a misunderstanding of what debt in the financial sense is. And tech debt derives rather directly from the idea of financial debt. But once again,

an analogy of financial debt, not like a debt itself. Yes. And I do think that the best use of that analogy is exactly what you said, which is to describe a situation to somebody who's not an engineer or on the software team about the situation. I mean, historically, it's been very difficult to get buy-in from people who aren't in the trenches of the code base in order to spend time and money on making the code better. This is a problem for us. How do I convince my boss or my boss's boss

or my boss's boss's boss that it's worth spending two weeks just fixing things? Because they don't understand that, really. They're like, well, we need features. And so you work on features, you don't work on fixing things. Well, tech debt's a great analogy for them

because they do understand debt. They don't understand finances. And they understand that you got to pay that debt down or eventually you go bankrupt, right? Eventually you can't deploy a microservice. You're taking over other people's microservices. And so that's a great use of that phrase or that analogy is to explain, okay, I get it. We've taken on so much debt because we've been moving so fast and making compromises along the way, and now we need to pay down some of that.

And so I can spend some money on this, like actual money that is in my budget as a manager or whatever, and say, yeah, go ahead, take every Friday and have it be refactoring Friday for the next quarter or whatever they decide. But I do think it falls down a little bit as every analogy does eventually. Like you said, the more you apply it,

the more it falls down because financial debt is so measurable. It's so quantifiable. I can tell you my numbers and you can be like, okay, you're in this much debt. How much money are you making? I can tell you those numbers. In fact, we have TV shows and radio shows

where people call in and ask financial advice from people. Dave Ramsey's like, get out of debt. And people call him and they just tell him, he's like, how much are you making? How much is your debt? Et cetera. What can we do about it? You can't do that with technical debt. I can't quantify to you, I'm sure there's startups that are trying, how much debt we have as an organization. Can you? I think the trouble here is that, I guess, like two things. I guess first, this might just be that my experience is like a very unique one. Although talking to friends and other people, I don't think so. But I will say maybe it could just be my experience. But I have had very little trouble getting business buy-in for why we need to

fix things or quote unquote, pay down technical debt. The issue I have run into that I've seen happen quite often is engineers go into the business and saying, well, we need to pay down technical debt. And they ask them, OK, well, what is it you want to do? And the answer to that is pay down technical debt. What do you mean? It's like, no, no. Can you like give me basic parameters? Like once again, can you quantify this as like how much time do you need?

What are you going to do? What's it going to be like in the future after this? And I think because we don't understand ourselves what this debt is, we have a lot of trouble trying to articulate it out. Sure. Because I think that's also where we kind of shoot ourselves in the foot with technical debt. Because the way businesses understand debt is as this very complex tool that they can use to get things done, right? Like whether you ever pay down your debt is an open question. And there's all sorts of like, there's different kinds of debt, there's different qualities of debt, there's different timelines of debt. But also debt is

a thing where it's maybe a liability for you, but it's an asset for somebody else, right? It's like, oh, me having this debt is bad for me as a company. But if I've given somebody else debt, that's good for me, because I'm going to get the benefits of it. And that's where the analogy starts to really fall apart. Because those things aren't really true of the kind of debt that we're talking about. We're kind of just taking bits and pieces of financial information, like interest rates and all of this and trying to apply it to our projects. When what the real problem is, is that we need to just be better at making sure we're being responsible in building out our software, which is why I equated it more to malpractice than to technical debt in that, why do we have all of this debt?

Like, how do we acquire it? Because if the finance department of a company is just like, well, we have all this debt. And the answer is like, well, where'd it come from? Who decided it? Why did we take on this debt? Where was the analysis that went into choosing this debt and they say oh well we don't know we just wanted to move fast like that's that's going to be a problem you're going to get questioned and you're going to start losing trust and that in my experience has been the biggest problem is that the business people the product managers don't trust the

engineering department to be honest about how long things will take or knowledgeable about how long things will take, which is, you know, part of their own optimism. And we say, oh, this will take two hours and it takes two weeks. And they're like, I can't trust anything you're going to say. So you're telling me you need to pay that in this tech debt, but you can't quantify it for me. You can't explain to me what it is. You can barely explain to me how we got it, which is usually is usually just oh the people before us would be like yeah it's just how it all works so it's like we're trying to tell people that we need something that's going to cost a lot of money but we can't actually explain to them in in understandable terms in just like timeline and dollar terms what this like why

we have this what how we will pay it down and how long it will take to do it. And I think there are people that use quote-unquote good tech debt, but I don't think that's debt, once again. I think that is very much, you're just doing planning. You're doing planning, you're doing documentation, you're figuring things out, you're making timelines, and you're making trade-offs and choices. Well, yeah, you could argue. I mean, any good trade-off in software could be aligned with a good loan that you take out in order to accomplish something. So I can see where you could make that line up. But I agree with you that a lot of our struggle is the ability to communicate, period. I was going to qualify that. But then I

was like, let's just stop right there. Because sometimes it's really hard as a software engineer to be able to represent exactly what is the problem, exactly what you need, how long it's going to take. I mean, a lot of times going back to the beginning of the conversation, that's how we got here in the first place is you asked me for a timeline. I had no idea how long it was going to take. I made my best guess. Then I tripled it and I was still under. And now we just shipped a feature that was half baked.

And now you're wanting to build another feature on top of that feature. And I'm sitting here thinking this is not going to end well. And that's just not easy to communicate in a way that doesn't get you fired. It's very difficult to explain that. I think it is hard to quantify, because I could have answers like, well, we're going to go back and we're going to retrofit a test suite around this. And then we're going to rewrite this sub section, right? And then I'm going to maybe swap this dependency for another one. And then what? Well, are we done? Are we paying our debt down? It's like, I doubt it. Like, no, because they can always get better.

How long is that going to take? I don't know. Maybe a week. Okay. Well, at the end of the week, I've only got step one done. Now it's like, hey, I mean, it's just, it's fraught. It's very, very challenging to succeed in this arena. It does all go back to the original point that I had, or the main point I have in the blog post of documentation. We don't know how long things are going to take because how do you actually acquire an understanding of how long things take? Do we sit down and keep all of the documentation for the projects that we've done. So we know like why we made the decisions we made, what the timelines actually look like, you know, recorded things so that we know, okay, well, in the beginning, we said that this was going to take two months.

One month in, we figured out it's actually going to take four extra months. And then we made these decisions to try and get the timeline down. We did all of this, like, if you don't write any of that information down, you have nothing to go back to to refer to, which means that the next time you're in that position, all you have is your intuition of what happened, which will almost always be incorrect. This is why we write things down, because it gives you a tangible thing that you can go back and look at and be like, oh, this is the timeline of events, because everybody's going to have a different perspective on why something happened. And there's a whole bunch of people that have more knowledge than other people and they kind of need to blend that all together and get everybody in a room and be like like if a product manager comes

to you says well last time you told me that this feature that seems like it's going to be similarly sized would take two months and it wound up taking six months like i don't like that and if you can sit down and say well that's because we started that project in September. We had the lull of December and January. And if we look at our attending data, we get less work done in the winter months because of vacations and all of that. And then we had the beginning of the new year and we had to do all this extra budgeting stuff and all this planning. And that's why this project wound up taking six months. But now we're in April and we're planning this project and we're about to hit the boost of summer. So we know that we'll be able to do this work in a focused way and deliver it within this timeline and if you can

say that to people they'll be like oh okay no that makes sense yeah okay let's go through it that way but if all we're saying to them is like well it you know things happened then like that's not gonna be enough they're gonna be like okay it's gonna take six months then right because you don't have that information to be like this is why it went this way you're saying it's called malpractice though right are you advocating that it should not be called tech debt it should be called tech malpractice like what can you label your specific perspective could you document this for us chris put this in writing because i i wanted to hear all of you you know you had to say to before i brought my argument in and it seems like you i couldn't quite tell if you had an issue with the analogy of tech debt being akin to financial tech debt or if it was simply tech debt should be renamed to tech malpractice because you feel like the majority of what we call tech debt is a result of a version of malpractice in some shape or form, not just simply pragmatism or quick analysis,

cutting corners, because we do that in order to get through something, whether it's dishes or not. Did you rinse them before you put them in the dishwasher? Okay, is that malpractice? No, not really. It's just more like, it didn't need rinsing because my dishwasher's powerful enough.

I think there was a period of time, probably a couple decades ago, when it was like, nah, this is just like we haven't learned. But by this point, like enough of us, I'm specifically talking about people that are running projects, people that have been in the industry for quite a while and they're planning projects out. At this point, you should know better, right? Like it's things like the number of code bases I've walked into that lack not just commenting, but any type of coherent documentation about what this thing does. And the people that are like this thing is I don't I don't want to touch it because I feel like I'm going to break it or whatever. And it's like we built this thing 12 months ago. Like there is no excuse for why

something we just built from scratch should be in this state. Yes, I agree with that. We should have hygiene around our code bases, like not washing our hands as a doctor. And if you get someone sick and if you cause lots of problems, that's malpractice, right? If you as a lawyer, like don't keep good notes and don't keep track of things and you lose your case,

like that could be malpractice. And for us, like if we greenfield a project and in less than a year or less than two years, we wind up back where we were with saying, oh, we need to greenfield it again. That is wholly on us. That's wholly our fault and our problem. And when we talk about it as debt,

we kind of get to escape that accountability for ourselves. It's like, oh, well, we just acquired this debt and now we have to pay it off and we don't have to internalize as, we're the ones that acquire this debt irresponsibly and it is our fault and we're not using this tool responsibly. So it's to shift the focus from the tool,

which is a perfectly good tool. Like, should you write comments in your code base? I think that there is no concise answer to that. Sometimes you need comments, sometimes you don't. Sometimes comments can be harmful. There's a whole bunch of nuance that goes into there. But should we wind up with code bases that we don't understand because there's not a single line of commenting in 15,000 lines of code?

No, that's clearly wrong. You've done something wrong. You didn't have a plan of how you were going to document it, of how you were going to make sure that you could maintain it in the long run, continue to add new features to it, onboard new people. And we're at a point in the industry where it's like, this shouldn't be happening anymore, right? It shouldn't be nearly impossible to onboard an engineer onto a project that is less than a year old or really any project at all. Right. I think the key is, though, is as a result of, I think it is tech debt because there is a cost that has to be paid to resolve it,

right? A literal cost in salary, a little cost in time, a little cost in salary a little cost in time a little cost in effort cognitive thinking team effort loss of focus on product etc so there's a cost so debt does make sense but it is a tech debt as a result of what a malpractice scenario or a pragmatic thinking scenario so i think as a result of is more akin to what your argument should stand upon not the not relabeling it yeah and it's less so that i think we shouldn't use i mean i there's a whole other reason why i think we shouldn't use tech debt as an analogy anymore um but it's not because of that part of malpractice is because of that oh i do like your idea bro malpractice and i think you're spot on just calling it that well you can't go to your boss and they'll say why do you need this budget and you say well because of malpract of malpractice. And then they're like, well, who's malpractice?

And you're like, well, it wasn't me. Whose head's rolling here? We have a tech debt to pay and it's a result of these, these, and these. And internally, you may want to deem it as malpractice insofar as that you're trying to institute change not to damn people from in the past for their choices which you may should do by not hiring them ever again or never working with them on a team or calling them out the next time they try to make the decision but it's more so to say okay in the future can we just not do things like not document can we please document our code can we please at least at a low minimum which is like a very low minimum comment our code yeah Can we please at least at a low minimum, which is like a very low minimum, comment our code?

This is not a lot to ask of a team to do. Well, you could institutionalize that thing. You can say, we write, we have to have 95% test coverage. And you could argue on the margins about the value of that. But in terms of

actual concrete steps that we could do, if the question is how do we keep ending up here there are a few mechanical changes you can just dictate top down such as code will have written comments and then of course you have to then bike shed out exactly what that means and we will have test suites that are automated and then you have to bike shed out exactly what that means but like those are things that could help you not end up there again. You'd have to top down do that. Now, you still might.

It could still happen. Another factor that comes into here, and we'll get back to your other reason, Chris, is churn is so high in our industry. You talk about onboarding new people. I mean, so many times, I think when we're coding, and now I'm just speaking personally again, I understand the system and I will be here next year. A lot of us can only say one half of that sentence, right? I understand the system, but I might not be here next year is more often the case.

I mean, people move around a lot. And this goes back to what Jessica Kerr was talking to us about last year, which is like, knowledge transfer is one of the biggest challenges we have in our industry, because people are just hopping around so much that you're constantly onboarding, you're constantly offboarding. It should be illegal. That's my own opinion.

It should be illegal to do it. And you don't need comments. I mean, a lot of code doesn't need to have that manual as you're working on it, and it's living in your head, and it makes total sense. And we think, and this is how it's always going to be on it and it's living in your head and it makes total sense. And we think, and this is how it's always going to be. And then it's not. One of the big reasons why I want to kind of push this idea of calling it malpractice is because I think it also shifts the accountability of the people in the room. Because I've seen it happen

where the acquisition of the tech debt was not something most of the people in the room wanted to do, right? They knew it was bad when they were doing it, but they didn't have language to push back and say, we should really not be doing this. And they didn't have, really, they didn't want to make the political investment in pushing back on a staff or senior or SVP or whatever, or staff or like principal or SVP level person of saying, this is really bad, we shouldn't be doing this. Or no, we can't move this fast because it's going to require this debt.

If we can just say, we can't move this fast because it will be malpractice. That is then something where like, it gives people more tools for the future. So I think the way we wind up in this situation so often is not enough people spoke up. And I'm someone that's pretty outspoken. It tends to get me in a lot of hot water because people don't like it when you push back on

them. And I'll talk to people in private and they'll be like, yeah, I know that's exactly what. And then I'll be in the room and be like, OK. And then no one wants to speak up and everybody wants to be quiet. And then we acquire this tech debt. And then it's like months later and everybody's like, man, I hate that we have to deal with this. And it's like, you should have spoken up in the room.

But I get that we as an industry don't have that ability yet. But it also goes into, we all know that we move around a lot. Yeah. So if you are thinking that you don't need to document this because you will be around, that is also a form of malpractice. It's like you know that you will most likely not be around. The odds are not in your favor that you are going to be here five years from now. So why is it that you think it's okay for you to say, well, I don't need to document this because I'll be here in the future?

It won't be. It's an illogical thing to want to say and do. And so I think reframing in this way can help us as an industry be a lot more responsible with this. And I think there's one other little offshoot of this, of we produce an immense amount of information. And documenting it is hard, but there are some places

where we've documented it very well, but it winds up being a wiki of some sort. And then we wind up in the area of knowledge management, information management. And we as an industry also, I think, kind of refuse to believe that other people are experts in this. Like we should be hiring librarians and archivists, and we just don't. I think that's also in line with this broader problem. Librarians and archivists. Well, I think that I don't disagree with that in principle. I think in practice, no one's going to do that unless they have just huge margins and just are looking for reasons to hire people that they don't think are very valuable, like long-term thinking. But could software help solve some of that? So if I'm thinking about the

difficulties, the rigor of documentation, obviously you can't attest to it, it's not so much, but if we just focus in on docs, right? And manuals and whatnot, we're getting to a point now where we could record all of our meetings, auto-transcribe all of our meetings and make that all searchable. And I feel like a lot of value is right there in terms of like, why did we make this decision? I know there's practices around decision books. What do they call it? Decision. Oh, like architecture decision documents

or things like that. Yeah, exactly. So you can go back eight months from now. Someone's like, why did we pick tail scale? And they're like, well, because I don't know. Bob picked it and he doesn't work here anymore. But now you can go back to this document and say, well, here's why we picked tailscale for X, Y, or Z.

Arbitrary example. Quasi-arbitrary. It's a good example. But even that requires process. It requires the tooling. It requires people to buy in and actually do that thing. And it's rigor. But maybe we'll get to a point where we can just record everything,

transcribe everything, and search everything. And now we have kind of an archivist, right? We have kind of a software librarian. Chris, I know you don't like this, but a little bit? Not quite. Because here's the problem, right? The problem with wikis is that they become these areas of just like, here's some information. What is the accuracy of this information, right?

It's like, I come across a page, it documents something. Is this true to how it works? Is this true? Is this correct? We had a meeting, we talked about some things. I can search, I can bring that up. But like, is that correct context? Do I actually have a good understanding of how all this information fits together? Because like, the work that librarians and archivists do is not so much like, it's more about organizing the information so it is findable and useful. And as we add more information, like I would say, if you're going to record all your meetings and actually start documenting things, I think that increases the need for you to have at

least one of these two people. Because in order to actually utilize the information, you need to have people that can actually present it in a way that you can consume it right like in theory you could go and learn the linux kernel by just reading through all of the great comments that are in there but it's going to be really hard to understand everything if you don't have like a here's where you start and here's like the basic structure of how things work and this is you know all of our you know categorization and all the ways we use taxonomy to find me. This is how we use this word in this specific way. And this is how it's consistent,

right? There's a lot of work you have to do around knowledge if you want to actually be able to use it. And I would actually push back on the idea that like hiring a librarian is something you do when you're big. I think it's far too late to hire these people way down the road. You need to hire them as soon as you start generating a lot of information, which is almost right away. Because once again, if you have organized information, your software engineers will be able to find what they need to find and make decisions much quicker, which will make them more effective. Librarians are force multipliers. Dev tool teams are force multipliers. But I think that librarians and archivists are force multipliers much, much earlier in the process than, say, a dev tools team.

Do you think the average tenure of a librarian would be any longer than the average tenure of a software engineer? Meaning, wouldn't you have the exact same knowledge transfer problem? Who's going to be the archivist of this startup? And they'll know everything inside out and point you in the right direction and organize all the information. They're gone in six months. Well, I mean, that's kind of the nice thing about,

specifically the archivist, because what an archivist does is they just go around and talk to people and extract knowledge and put it someplace and organize it. And this is a science that people get PhDs in, right? So there's a known way to actually organize information very well. And it's not that, you know, it's just all in someone's head. It's like written down.

So even if you do have your archivist leave, you just hire another one and then you continue the work. And it's just continual work of talking to all of the people and making sure that, A, we don't have knowledge that's stuck in people's brains and that it's organized in a way that it is searchable, findable. You can reference it easily, all of that.

So it's really like the thing you get at the end of the day is this comprehensive system that you can use to find information and knowledge about what you have done. We just don't have that. So like I'm sitting here saying we need to write all this documentation. But, you know, even if we did start, even if like a company started today just documenting everything they have, if you don't have a good way of organizing it, you're not going to be able to use it. It's like having a book without an index. And if you want to find a very specific piece of information without an index, you got to go read the whole book. You got to go through the whole

thing. And that's going to be a tremendously difficult thing to do. The more books that you have and the more things you have. So you want to build up that index. You want to build up that way to refer to things. And that's the work that a librarian or archivist does. So in addition to the documentation, you need to have a way of going back and finding the documentation when you need it, which is a very challenging problem. But thankfully, one we've been working on

for thousands of years. So we have some pretty good ideas of how to do that. Yeah, alphabetical order. I mean, easy. In both cases, that makes sense. Dewey Decimal, if you need it. I mean, come on. I wonder if this idea that you have definitely is good, but I wonder if we need a specific

person in the role. If it couldn't just be a function of the team that gets done until it's proven so valuable that we can actually hire a person to dedicate towards it as point one i would say as long as you have the knowledge on the team of like the actual study right it's like you know you're not just going to be like the whole team's going to share accounting responsibilities it's like no you need to have someone that knows accounting well there's usually a pm on the team there's usually somebody to lead or tech there's always somebody that's in charge and responsible but you're saying they must be like actually trained in this area of discipline.

Yeah. If they've like gone and read some books on information and library and information science, then yeah, sure. The PM can do it. Well, I think that we as technologists and those who work around technology tend to gather the skills necessary to do the job, right? So if we bolt on the need for a librarian archivist type skill set to someone in lead or the team in general that we care about these functions then we all will begin to grow that skill set by

no means am i demeaning the phd that some folks get to achieve that level of expertise but we do have to start somewhere and the pragmatic approach would be to as a team care and then as a team collect knowledge and as a team institute that knowledge and do that function until it makes sense to bring on a literal you know because that's how a lot of roles get formed is over time we care enough we do it enough and then we're like well we really need somebody dedicated to this because it's so important yeah i think testing is a similar thing where it's like in the early days, it's just like, or ops, right? It's not like you necessarily hire DevOps people from the beginning. So it's one of the things where it's like, yeah, as long as you, the actual getting the work done is what matters.

Who does it is, as long as they're competent in doing it, it does not matter nearly as much. Well, I agree. And that's why we podcast. We talk about these big points and we figure them out and people listen and they're like, you know what? We're going to do that in our organization. They come back three months later and they're like, we did this idea you guys talked about in happenstance on this one podcast way back when. And they wrote a paper and then you've read it and you've highlighted it and everything. You mentioned the Jared of Tailwind and the why question of why we make a choice is I wonder if we couldn't have the why document. If there's a major decision, the why document essentially says why we made this choice,

the pros and cons will be considered. And the problem I think with documentation we may have is it does institute knowledge. And Chris, you mentioned the point of accuracy is we don't have this lens in the future looking at the past, which is documentation saying who agreed with this document? Is this document accurate? It may have been for a time because I often go back to my documentation. I'm like, well, I'm glad I wrote this and a document where I'm at then, but it's not accurate now because it needs to change. scale or choices like using a certain database or an architecture choice or whatever could be founded in a y document and that y document essentially has a synopsis of why the pros and cons considered and who agreed with you know the ending result you know like it's signed off for lack of better terms by x y and z so in the future you know okay this document is at least accurate then and these four or five people who were in charge agreed with the choices made here. So then it made sense, but now it needs to change

because different circumstances are now in play. I absolutely agree with you. I think we do need to have this document. I just think that it needs to be a lower-level thing because there's always this problem of when does a decision reach the point where you should have this Y document? At deployment.

If you put it in production, it should have a Y document a wide document right i mean how many decisions get deployed i mean that's thousands major i think you can begin with major ones like a css framework or paradigm like tailwind maybe why are we using this database right i think that there's like because there's all there's like yes the decisions you make where you actually do the thing but there's also the decisions you make where you don't do the thing, which I think actually wastes more time. And it's harder to document the things you do. So many places I've been like, why is this thing this way? Couldn't that be done in the why document of what you did, though?

Why not? You need two documents. And why not? If there is a thing you did instead, yes. But sometimes you're just like, we don't want to do this. Or we're not going to do this now. Maybe in the future we can revisit this, right? So there's not always a thing you did instead that you can refer to. So there are these like, and I think those documents are actually more valuable,

because every new person that comes in, will probably have a question of why don't you do it this way? Or why don't you do this thing? There's always some new technology. So why aren't we using Kubernetes? Oh, here's the report that we wrote on the evaluation we did and why we did it and that way too even if the circumstances have changed and that new person comes with a new perspective they can go read consume that history and be like ah this is how things have changed i'm going to make this argument of why we do it you don't have to rehash all of history all over again which is typically what happens now yeah i had a phone call or Zoom call with a listener of Practical AI a couple weeks back. It was an enjoyable call, and he had a lot of good ideas just about the show, about us

as a changelog thing. And he kept saying, have you thought of this? Have you thought of that? Or why don't you do this? Or why don't you do that? And it was like that, where I was like, no offense, because these are good ideas. And he didn't have the domain context that I have. I'm like, I've literally thought deeply about each and every one of these and decided not to.

Not all of them, but most. And had I been on vacation, put me in a larger organization, that guy's on vacation. Somebody else is asking somebody else, I don't know, why didn't we try DuckDB, that brand new database everyone's talking about? Well, you know what? We should try that out. And then the guy who already spent six weeks deciding we shouldn't use it on vacation that day. But decision documents, that's very much a thing, right? In larger orgs, I've always been in tiny orgs. But Chris, in the places you've worked, have there been decision documents like this? There have been, but they tend to be very, like there's a threshold. And what people will tend to do is get around that threshold by getting just up to it and then breaking projects up to smaller pieces so that it doesn't reach that threshold, which is the dance that constantly happens. So I think there's like some rule about American banking where it's like you can only deposit $10,000 in a day before you have to explain it.

So people do like, I'll just do five deposits of $9,999 and that'll be good, right? And that's the type of thing that people start doing if you have that kind of place where it's like, oh, now you have to – because people see it as extra work, right? It's like I have to do this additional thing of documenting why we made this decision. And I don't want to be doing this. I want to be doing other things. So I think an important part of the usefulness of these types of documents is when they're just part of the normal flow of everything. And that's the thing that so many organizations I've been a part of just cannot seem to figure out how to do. It's like, there's I say it to people, and they'll be like, Oh, no no, that makes sense.

But that seems like a lot of extra work. And it's just like, you could just have it as a field in your Jira ticket or anywhere. Lower the bar, lower the barrier to documenting why you made a choice or made a decision. And it makes it so much easier.

That's like, you know, after every meeting, write up an email summary and send it out to everybody. That's some documentation. Or post those notes in a wiki somewhere. It's just like, when you do it in the small, you don't have to necessarily do it in the big because you can pull all of the small pieces together and you'll have that granularity you want. Because there's things you could miss in the big architecture decision document

or the big design document, especially historical things that can be in the small. And I think historical also is very important because Google Docs is terrible at this. And this is why I hate having design docs in Google Docs. So then I'm like, how did you get to this decision? How did you get to this paragraph being written in this way? And you go open and like the history and you're like, I have no idea how to parse any of this.

So you just kind of say, yeah, go talk to people, be like, why did you, and like people are like, oh, I thought we were thinking of this so you just kind of say you gotta go talk to people be like why did you and like people like oh i thought we were thinking of this and then we decided to deal with this other way and blah blah blah i ain't gonna look and you're like that doesn't match with history they're like oh that's something else i was thinking about actually this one so it's a lot easier if you just like in the moment document things so people can go back and go read an email thread or go read the wiki and look at the history of the wiki and be like okay this is this is why this decision was made so going back to estimation uh-oh what happens when the software doesn't ship because like well i was too busy documenting it you don't say that you never say that

well in the world that you're calling for that might become slightly more true right like if you add one more you know critical task let's call it a critical task, like to document and it's not being done and you're calling it for it to be done. I'm just wondering in that world, maybe people will have one more excuse why it doesn't take four months. It takes six months or why it took six months. Yeah.

I think this is also about like kind of the making sure that we have an acceptable bar of what is the minimum that we will do as software people. Because, yeah, if the floor is that it takes six months to build something, then the floor is it'll take six months to build something. And we as an industry need to accept that. Because there's a lot of like, I think we as an industry are still in this kind of, we want to believe that the software we build doesn't have severe impacts on people's lives. And I think a lot of people want to be like, well, I'm not building software for an airplane or for any of these things.

So like my software isn't life or death. And like I worked at a financial institution that was like a literal bank and gave people loans and credit cards and all of this. And I remember one of my coworkers said to me, it's like, well, what we're doing doesn't make that much of a difference. And I'm like, we're literally deciding

whether someone gets a loan to buy something. Like that is a pretty major thing. And if our software messes up, they might not be able to buy something that they could have otherwise bought or they might otherwise need. And so I think like a lot of us are like, oh, no, what we build isn't that important. No, it is. It has very profound implications on society. And we should be very diligent in making sure that we're taking the time necessary to make sure that we're building things correctly and that we're not doing that malpractice, which is a very harsh word,

but I think it's one that we need to start doing because I think, especially if you look at the social media companies and some of the kind of egregious stuff that they've done in the past, it's like, would we maybe have prevented that if we had better documentation, if we had better ways of raising our voices

and things like that? I don't know if the answer to that is yes, but it's certain that the way we're doing things now is not working and we need to change something. And the thing that's been constant since the 50s that has been said is we need to document things more. So I think that's a good place to start, even if it does cost us a little bit more time. But I think also, you know, it's a planning thing, right? When you have more information, sure, this one project,

maybe it takes an extra two months, but you might save six months on the next project because you have all this information to look back on. So it's a thing that also builds over time. Compound interest.

Yeah. Financial tooling. It's a compounding of, it's a compounding of knowledge. Yeah. I love that. Actually, I think there's this interesting thing about, you know, I was thinking about what debt is in the financial sense.

And debt is kind of this weird thing that it's like at the core of how money actually works, right? Like, how do you create money in an economic system? It's like with debt, right? You give your bank some of your money. They give it out to someone else and they get more of it back. And that's what bootstraps the entire system and keeps the whole thing running. So it's like debt is foundational to how money works. And I think in that same sense, like knowledge is foundational to how software works. And we should build things around that. And I

think there probably is some sort of equivalent to debt for software. I don't know if we should call it debt directly, because I feel like it probably operates in a slightly different way. But I think there is something there that we got to get ourselves toward. And I think that's what I'm trying to do here is like, what is the thing? What are the tools that we have

to work with this knowledge? Because I also have another hot take, unpopular opinion. We got time. Let's hear it. So I guess I'll preface this with, in writing, one of the most annoying things to deal with is like pros. So if you're writing a novel,

the words in the novel are very, very expensive because it's very difficult to keep a story coherent and all of that when you've already written a bunch of words on the page. And the thing that I've realized pondering this for a long time now, is that the most valuable words, the most valuable prose, is prose that you consider to be worthless. Because if you consider it to be worthless, you have little problem with throwing it away. But as soon as you put value in those words, it becomes quite expensive, because now you don't want to get rid of them. Now you're falling into the sunken cost fallacy. And there's a whole bunch of sayings around this, but I think the most popular one is kill your darlings, which is basically saying, I know you love that sentence you wrote, but it don't fit anymore.

Just get rid of it. I know you like this character, doesn't fit in the story, get rid of it. And I think the same applies to software and specifically to code. I think code is valuable when we consider it worthless, because when we consider it worthless, we are OK with throwing it away. But as soon as we assess it with value, now we want to try our best to not get rid of it. And where we are now, code is the thought stuff. It's the way we prototype with code. We do designs with code. We do everything with code.

So if every line of code we write we see as having value, we will try and hang on to every line of code that we write. And it will be painful and difficult for us to delete that code. And I think this causes a whole host of problems. This is why our operating systems still have code in them from the 70s for some reason. And it's why we have all these multi-million line code bases that we're like, well, we can't really get rid of this thing because, you know, we have all of this code invested. And so my unpopular opinion there is we should consider code to be worthless and that will

derive value from it in the long run, which is really weird. I don't think that's the best phrasing of it. I think I need a better phrasing. But yeah, it's like, I think we should believe that code is worthless and we should not be so tightly integrated with it, right? We need to let it go much more often than we do. Right. I agree with that.

I think I've heard it said features are assets and code is liability. So in the case of worthless, meaning able to be removed. So the best code is no code and the best feeling as a software engineer is deleting your code. So I'm like, I'm with you on that. It is a weird way of saying it. It's kind of an interesting

way of saying it because if it's worthless, how can it be valuable? It's kind of a contradiction in terms, which maybe makes it sticky. I think we should write our code to be thrown away, for sure. And there's a lot of reasons why we don't. We can get into that, psychoanalyze ourselves. But yeah, I'm with you. I think that the individual lines or characters or words need to be disposable. And when we hold on to them too tightly, we do ourselves a disservice. Yeah. I think it's, because I've been thinking about this a lot. I have this, I have this analogy I've been working on because I think, this kind of goes into like why I'm

Malpractice. Starting my own company and all of this, but malpractice is an analogy I do like, yes. But not in this case. Because I think that we don't understand the scale of how much things have changed in the last 40 or so years. So I've been working on this. I started with me trying to do it based on like square footage, but I figured out that doing it with money probably resonates with people a lot better. So if you take the Apple II that was released in 1977,

and it costs about $1,295, which is about $6,500 in today's money. And you kind of do a nice little conversion with transistors and clock cycles. So you're like, okay, one transistor running at 50 kilohertz equals $1 per year, let's say. The equivalent resources of an Apple II were $60,000 in this conversion, right, of transistors

clock speed. And so you could be like, OK, $60,000 a year, that's your budget for what you want to get done, what you can get done in a single clock cycle with the Apple II. If you take a Mac Studio that was released this year, and it costs roughly the same, if you spec it out with the Max processor, Max RAM, you wind up with a roughly $6,500 machine. The equivalent amount of money in that same transistor slash clock speed is $9 quadrillion, which is 100 times the amount of money that exists in the world. So we went from having the resources of a decent middle-class income to all of the money in the world 100 times over. And I don't think we as an industry understand

that that's how far hardware has moved in the last 46 years. And I think the problem is that we basically just have unlimited money now. And you can't use the same things that you used when you were budget-constra, when you have unlimited money, right? You can use an Excel spreadsheet to manage $60,000 a year. It might be a little tough, but you can do it. You cannot use an Excel spreadsheet to manage $9.4 trillion. You'd have to actually build something very, very customized and very advanced. You're talking about managing world-level economies at that point, or even large government at that point. So the fact that so much of what we have today is rooted in the time when the resources were

that constrained, I think shows that we need to really start giving up things that we've had in the past and really start rethinking the very basics of how we do things. Like I think I very much, I deeply appreciate Unix, but I think it's a model that is kind of too stuck in the past because we're doing, a lot of the hardware that we are using is spent trying to emulate that old world, trying to pretend as if we're still a single processor

running at maybe a megahertz or two megahertz so that, you know, going and fetching ram isn't that problematic but we truly live in a ridiculously different world than even 20 years ago right there's another analogy even i guess comparison i've been working on of 20 roughly 20 years ago we got our first commercial multi-core processor uh it was intel xeon well i think there was a consumer version but the the server version was Intel Xeon. It had two cores, four threads.

You could put two of them in a machine. You get a total of, what, that's like 16 threads? Oh, no, it would be a total of eight threads because it's two cores, four threads each. So you have eight threads and about a megabyte of cache or two megabytes of cache. AMD released a processor this year that has 96 cores, 192 threads per socket. So if you put two of them in, you want to put 384 cores. And each of those processors has a total of a gigabyte of cache. So you have two gigs of cache. That is an extremely different computing device

than the one we had back in 2005. And while we could stretch a lot of the parallelism, we could just utilize four cores or eight cores or whatever. You can't use the same thing. Go routines, baby. Yeah, you can't utilize the same thing to go, you can't use those same resources to go use 384 cores and multiple gigs of RAM or multiple gigs of cache

because it's also cash too, right? Before it was like you had a little tiny bit of cash. So you couldn't really keep a lot in there. Now you have a lot of cash. And if you write your software wrong, you're going to be doing less of cash expiration and your program's going to crawl to a halt. But you want to be able to use all those extra cores

because over that 20 year span, clock speeds have not increased at all. In fact, that AMD processor we're talking about has a slower clock speed than that original Intel Xeon. So we're not getting faster. We're just getting more stuff. Right. Scaling horizontally.

Yeah, which means we're running into physics as well because it takes time to move across the chip. So you got to deal with those latencies and all of that. That's a distributed system almost. Not almost. Not almost, Jared. Not almost. It is a distributed system. So are you here to announce your brand new

operating system? I don't understand. What are you going to work on in this space, Chris? New OS? Throw Unix out and just start from Greenfield? So I don't think, you know, obviously saying we should throw Unix and all the Unix operating systems out is

extremely impractical. But the main thing I've been frustrated with is that I've been a distributed systems engineer for most of my career. I love distributed systems problems. But even I was puzzled when I first started looking into the literature around distributed systems and really reading it. And there were some things that I found very confusing. Like I have this book that I bought called Fault-Tant Message-Passing Distributed Systems. And I'm like, okay, this is an interesting title. Why is message-passing in there? Because aren't all distributed systems message-passing ones? And I learned, no, there's shared memory distributed systems. I'm like, oh, okay. And I'm like, okay, well, what's fault-tolerant in there? Don't all distributed systems need to be fault-

tolerant? And it's like, no, there's distributed systems that aren't fault tolerant. So I went on this journey over the past couple of years of learning a lot about distributed systems. And what I found is that all of our computing devices are distributed systems. And not just like in the sense of like you have multiple cores, but like, you know, you have PCIe. PCIe is a network-switched network system. And that's at the core of literally all of our computers. So yeah, you have a little distributed system that's running your computer right now. Running the computer we're all having this, recording this episode on. Which has some profound implications.

But even your processor, once again, is its own distributed system. Because you have all this cache coherency that you need to deal with, right? It's C. If I wrote 5 to the variable X and then I read it the next clock cycle, I expect that to be 5 because I just wrote 5. But if I got scheduled to a different core that has a different value in this cache, then I might not get 5 and that's going to break my code. So there's a lot of hardware logic that's done to actually synchronize all of that cache coherency that we have. And we call it atomicity when we're talking about the processor. We're talking about atomic memory and all of that and transactional memory, perhaps. But in distributed systems, we call it linearizability, which is also the kind of gold standard for a lot of databases that they want to reach. And it's like that's at the core of every processor that we make that has multiple cores.

Right. So it's like, it's not just that these are technically distributed systems when you look at them. It's like, it's using the same basis of research that we use to go build Kubernetes to go make your processor work, which means that there isn't a difference between when you're programming locally and when you're programming this kind of Kubernetes or what have you. And the reason I have frustration about this is because I've been to too many places where I say, okay, well, we got to go solve this problem. And I hear, well, we want to avoid having distributed systems problems, or we don't want to take

on distributed systems problems right now. And that annoys me because I'm just like, you can't not. It is not just like the basic types of distributed systems either. One of the things I've learned about is Byzantine fault-tolerant systems, which Leslie Lamport, I guess you had previously, he coined that term of the Byzantine generals problem and Byzantine consensus. I don't know if he coined Byzantine consensus, but the whole Byzantine thing is his creation,

his and some other people. And I was like like when am i ever going to need this and like for one thing blockchain is actually a byzantine fault tolerant distributed system which is actually pretty cool that's true but also i realized through thinking about this that writing to disk is a byzantine faulty distributed system because the disk will lie to you about what it's done you might say please write this to disk and then do an you know an f sync and be like no no really write this to disk and it'll be like yep totally i wrote it to disk and then you read it it's like i don't know what you're talking about i i didn't you didn't ask me to write that

to this right it lies to you um so even something as simple as writing to disk is a distributed systems for your hard drives yeah you gotta be like these lying hard drives we uh we need to get better ones right ask him better ask him again so everything's a distributed system is what you're saying and and the thing here is that i think a lot of it is not difficult to understand if we just sat down and taught people but we're still very focused on maintaining the illusion of the old world that we refuse to move into this new world and so that's what i'm trying to do is be like, how can I teach people about distributed systems so that it's actually approachable and you're not going and reading academic textbooks and academic papers trying

to understand what even for someone like me that's been doing distributed systems for 10 years was just like, this is difficult to parse. Gotcha. Okay. So it's not a new OS. It's a distributed systems is like a course or like a book or like, what are you actually going to manifest here? As writing? I mean, right now I want to start with kind of being on the internet, being a person that writes on the internet and doing digital publishing and see where that gets me and kind of like go on this journey. Okay.

Obviously, you know, that includes other things like probably video, podcast, other stuff like that, because, you know, I have skills in that area. And I think like my main goal is just to make it an approachable thing, because I think right now it's kind of the land of experts. And there's a lot of experts that, you know, can only really speak in jargon about distributed systems. Like, I think you kind of ran into this when you were talking to Leslie Lamport when you were trying to explain some things.

And I was like, okay, there's a leveling problem. Leslie Lamport is obviously very well-versed in distributed systems. And Jared, you're knowledgeable, but not in that realm. And so I think there needs to be a way. I was wading in the same pool for the first 45 minutes. And then once we hit minute 46, I was just hanging on for dear life. Yeah. I remember you like brought up like Paxos. Was it Paxos and bakery? And you're like, are these related? And I was just like,

oh, no, I asked at the bakery algorithm had stuttering insensitivity, which was apparently was a very not smart question, but I didn't realize it until he answered that. And I was like, dang it. And I think it's so challenging to understand. I'm a generalist, Chris. I'm a generalist. But it's like, you know, this stuff is all around you. Would it be nice to have a better understanding of how it works?

Or just to get rid of some misconceptions. Like one of the things, I'm working on a series right now about this, about basically helping people understand the misconceptions they have about how computers work. And one of them is about timing models and the distributed systems that we have now or like really the processes we have now. Right. You have no guarantee of when your next clock cycle will be. Like if you're writing a language that runs on a virtual machine or a runtime, you have that scheduler. You have the OS scheduler. If you're running on any type of virtualized environment, you have the hypervisor that's doing scheduling. And none of these things have to give you clock cycles, right? They'll be like, I'll give you clock cycles when I want to give you clock cycles.

And it has really weird implications about the type of code that you can write. For instance, you can't depend on the passage of wall clock time in the code that you write. Because even if you do check and say, oh, did this timer time out? Or yeah, if you check if a timer timed out, then you might assume that the next clock cycle, you're like, oh, I'm operating within some small bounds of when that check was. And it's like, oh, it could be running an hour later. And I had a friend who was working on this type of stuff with sleeps. And they would ask, oh, Linux, sleep this process for 10 minutes. And Linux would get around to waking it up 10 hours later. And it's like, if your code assumed that it was actually going to wake up 10 minutes from now, and it woke up 10 hours from now, is your code actually prepared for that?

Is it ready to handle that type of reality? And the bigger question is, does it need to handle that reality? And my conjecture is that no, it does not. And that we should move away from thinking about things in timeouts and sleeps and wall clock times at all and move to embracing what is asynchrony, which is not having time bounds on when you will execute and things like that. It's tough to explain this to people and difficult for people to wrap their brains around. But it could have some very big implications for

the ability of us to build interesting systems. For example, one of the things I said for years, which I now feel kind of bad about because I was just wrong, was that you can't step-debug a distributed system, right? Because that's one of the things I'd say about, I don't use step-debuggers because I'm a distributed systems engineer. You can't step-debug a distributed system. But you very much could step-debug a distributed system if you don't have timing assumptions, right? You can just pause your process. And if nothing else assumes that your process is going to get back to it in any time bounds, then it doesn't matter how slow it's executing. And you can step-debug it all you want and your whole system will work and it will work correctly.

But if you have a whole bunch of timeouts all over the place and those things expire, now you're in a different path than you would have been otherwise if your system was running at full speed so once you start programming with asynchrony now there's problems you just don't have to worry about i think it's similar to uh the local first movement uh and like what martin kleppman has been working on where it's just like you know we right now think of online and offline. And online is some low set of latencies and offline is you can see it as a high set of latencies. But if we just didn't have that distinction, we're just like, no, there's just latency. And we can handle any amount of latency, we can handle 10 milliseconds of latency, we can handle 10 days of latency, and the whole system just keeps working. That eliminates so many bugs that you might have otherwise. But it's hard to make that shift if there's no buy-in for people,

which is also the thing I'm trying to get at is... Give us good reason to. Yeah, this is valuable for you as well. It makes your life better if you think in this model. It might be tough at first, but your life will be better if you think in this model. All right, sounds cool. It sounds like a challenge. I think we definitely need more documentation in the world of distributed systems so that more of us can

understand. And like you said, it's tough sledding to go down those papers and to understand and not to get lost. And even Leslie Lamport himself said on the show, he's not a very good teacher. He's struggled to actually express his ideas because they're difficult to express. And he's operating at a level that sometimes it's hard to come down and speak to somebody who's new to an idea, to a concept, when you're so ingrained in it or when you invented it 40 years ago. So a valiant endeavor, Chris. I think this is cool. So you're giving yourself full time to this? Like this is what you're doing?

This is my journey now of really just trying to be a writer. And it was me, it was a lot of me sitting down and thinking about who I am. Because I think part of the issue I've had in my career is that I am extremely ambitious in what I want to accomplish. And I would often find myself around people who would say they were ambitious, but were ambitious to a much lesser degree than I was. And I kept butting up against that. And I think I've kind of figured out that I just need to kind of operate my own thing where I can be as ambitious as I'd like to be. I'd actually put in the effort to do that because I found that I was holding myself back quite a bit to kind of be in the environment with everybody else.

Because, you know, there's politics and all that you got to play. And it's like, hey, we should go do this big ambitious thing. And people will be like, oh, no, that sounds like a lot of work. And maybe we shouldn't do it. And I'm like, no, no, no, but we can do it. It's going to be – and, you know, you don't want to annoy your coworkers and all of that. So you kind of give up on it. And I've given up on too many ideas. And I realized maybe if I just do this on my own, I can make it happen because I also write an

absurd amount. I think on one of the GoTime episodes, I mentioned how I, in the past two years with journaling alone, a big thinking on the page type of person. At that point in time, I had written 3.8 million words over the course of two years. So a little under 2 million words a year. I have actually increased it since then. This was my first month and there's three days left. So I think I will meet it. But that old system was me journaling 5,000 words a day. And now I'm up to 10,000 words a day. And I want to do more because it's like, I truly do just think on the page. It's like some people think in their head, some people think with drawings and all of that. I think with words on

the page, which is extremely valuable in this type of space. Because if I think with words on the page and I go out there and I publish, you know, I clean up those words some and I published them out. People can follow that journey. Because I think part of the problem with distributed systems with a lot of teaching is that by the time you become a master, you forget the path you took to get there. So you have a lot of people that are like, but how do I get there? It's like, you just learn,

just read this book. But if you can see the trail of how someone got there and the things they struggled with and the things they thought about as they went along, then it becomes a lot easier for you to be like,

oh, okay. And it becomes easier for you also not feel like you're dumb or something of like, oh, this person that I respect a lot, they struggled with this when they were learning. So I can keep going. And it's not like I'm some fool and they're just a brilliant person. So I think like that's kind of the contribution that I should make to the world is I can write at a volume that is truly absurd. And those people cannot do that. So I should go forth and do that

and do what I said to myself when I was younger and be a writer. Go forth and write at extreme. You might need a personal librarian or archivist just to point out the best of Chris's writing amongst the many, many, many words that he's, the valueless, what did you call them? The worthless words.

It's interesting because I have picked up quite a few books on librarian information science and I'm getting myself there. Because I also have an absurd goal that I do not know if I will meet, but I'm going to certainly try, which is I want to be a truly full stack software engineer and not the stack is in back end and front end, but the stack is in transistors and up. Like I want to know how to design a

processor, design a bootloader, a kernel, drivers, graphics, engines, all of that. Like I want to be someone that knows that entire stack because I want to know a is it possible for one person to know this whole stack and not like i don't think i'll be able to write a production operating system certainly not on my own but can i write a functional operating system that works for me can i implement ethernet that works for me can i implement a network stack can i design a chip can i implement say a risk 5 a RISC-V processor on an FPGA? These are things that I want to know and understand because if I am successful in my endeavors to do writing and be a person that publishes on the internet, I can help be the force that pushes us toward advancing our technology some more. So I think part of the problem that we have is that not enough people

can see the whole stack. And I think right now we problem that we have is that not enough people can see the whole stack. And I think right now we're in this position where we all think that it's impossible for any one person to see the whole stack. And I want that not to be true. So I think there's lots of changes we need to make and that it does require someone that can see or a group of people that can see the whole thing. Well, I would push you towards Andreas Kling and the Serenity OS. You're going to say the same thing, Adam? Yeah, I was going to suggest that too. Yeah, it's funny because I already listened to that episode and I was already a fan of his because I think I saw his talk

at the Web Browser Hack Conference. It just kind of appeared in my YouTube feed a couple months ago. And I was like, actually, he kind of inspired me a bit because he like, and I talked to my friends before, they're like, oh, you can't write an operating system. You can't like don't implement LibC. That's blah, blah, he kind of inspired me a bit because he like, I talked to my friends before. They're like, oh, you can't write an operating system. You can't like don't implement LibC. That's blah, blah, blah. And seeing this one guy that was just like, now I'm going to build my own operating system and my own web browser and I'm just going to do it.

Yeah. That was super inspiring to me. I was like, yo, like someone's actually doing. It's kind of like the first person that does it because, you know, when you're kind of alone in the world and you're like, I'm going to do this thing. And everyone's like, that's insane. Why why are you gonna try and do that takes a lot to push back but when there's someone out there that's done it it's like oh okay well i can follow in their footsteps in a bit or or go on the journey along with them well that is cool chris thanks so much for hanging out with us adam any other thoughts questions unpopular opinions

i've enjoyed the ride, the ride of Chris. You know, it's been fun. This has been a ride, yes. I was super curious about the PCIe distributedness of my own PC sitting here right now. And I explore that, but we can save that for maybe a blog post or a future GoTime rant or something like that. Because I'm just curious about this distributed system we all have in our hands right here, right now. Yeah. When I fell down the PCIe rabbit hole, I was like, this is amazing. And oh, this is awful. Like a little tidbit of it, because of the way PCIe works,

there's a brief moment when any device that uses Thunderbolt, so your monitor or theoretically your charger, really anything, has direct access to all of the physical memory on your machine. And there's things that happen so that that doesn't remain true all the time. But because of the way PCIe works and the way it was designed as a direct memory access system, it can just access raw memory, which is terrifying when you kind of think about it. But it's not something people need to actively worry about, but an interesting quirk of history that, yes, I think would make a good blog post.

That's why I say don't plug your your phone into some weird you know outlet i'm sure that's probably the case where like you're in the airport and you're like oh that's a usb charger place it might be similar to that where on the other end of that thing might be a pcie slot that you're getting funked from yeah usb is a little bit harder to because it's not directly pcie so usb isn't direct memory but pc so thunderbolt certainly is so if you plug your computer in it's but yeah no this is like something else i like i want to talk about so usb c3 plus whatever with the that's not thunderbolt ready or usb4 is basically thunderbolt 3 which so it can put PCIe over it.

So yeah, the USB 4 and all that. This section of things is also like a deep rabbit hole that I also want to talk about more because I think it's fascinating how much we cling to abstractions that don't make sense anymore. Well said. Except for tech debt. That one is just perfect. So we'll just leave it there. I do think you have your work cut out for you renaming tech debt to something else that's well it seemed like a subsection was malpractice he

admitted there's certain proper use of the analogy so we found a middle ground yeah yeah that's why i think you should uh cut tail to as a result of my idea i think you should use my idea as a result of malpractice not and it doesn't have quite the same ring to it as malpractice but yeah malpractice is fighting words you know you say that in a meeting i mean i that's kind of the point of it though right jared it's like it is it is like if you're sitting there and it's like oh are we about to acquire some tech debt it's like one thing it's like are we about to perform malpractice it's like oh this is a different conversation right there's no positive spin i was gonna bring this up during the show but we kind of like went further elsewhere and i was

thinking maybe just a spectrum which is like okay is this decision leaning more towards pragmatism or is this decision leaning more towards malpractice like you need that spectrum essentially like where on this spectrum does this decision fall or this choice fall and in that case you can point fingers. Isn't some malpractice, like there's no decision making. Malpractice, I see it as two things. And maybe I see it differently than you guys. It could be negligence or incompetence.

It's kind of an umbrella term for both of those in my mind, at least. So you're not having a discussion about the decision being made if you're in malpractice mode. You're just scooting on, aren't you? You're doing your thing. I don't know. It depends if you're in malpractice mode you're just like scooting on aren't you you're doing your thing i don't know it depends because there's there's individual malpractice but there's also like the whole of the sometimes whole of the industry malpractice like the whole thing where doctors in the what 15 1600s didn't wash their hands and they like do autopsies and then go deliver babies and they're like why is our maternal mortality rate so high people are like because you aren't washing their hands.

And they were like, no, no, no, our hands. It's like you got formaldehyde on your hands or something. Yeah. And they're like, our hands are golden. Our hands are from God. What are you talking about? We deliver babies. We're amazing.

Everybody's like, no, wash your hands. And it's like, so it's like a collective thing where it's like, individually, you necessarily blame anybody for pushing back because that could have ended your career. Right. But as an industry. So that's the type of malpractice i'm talking about in this sense is like okay so that could be under the category of ignorance right like they were ignorant of what they were doing collective ignorance collective ignorance and probably a bit of hubris in hubris

okay and how you feel about your own skills and what you can do yeah turns out malpractice is a wide umbrella and we can all fit in there if we want to the thing i like about malpractice too as an analogy is that it's not like the thing i like about malpractice that's going to be a title the thing i like as an analogy is uh that it's not a singular decision right you have to actually evaluate the circumstances and what happened to decide like was this just a bad decision was this malpract? Was this, is this not malpractice at all? And it just had a bad outcome, right? You have to actually dig into it a little bit more, which is another feature that I like about it. But it also is incendiary.

And you know me, I love hot takes and unpopular opinions. That's true. All right. To our listeners who made it this far, let us know what you think about malpractice, both as a practice and as an analogy. And do you like it? And do you like it or do you not like it? So, all right. We should end this before we go off on another tangent.

You know me. We did have that title going, Jared, of... Oh, yeah. Sorry, this podcast wasn't any shorter. Or how was it going to go? Yeah, we didn't have time to make a shorter one. Yeah, there you go. I love it. All didn't have time to make a shorter one. Yeah. There you go.

I love it. All right. Bye y'all. Bye friends. See you on the next one. Hopefully the lengthy duration of these change, all good friends conversations doesn't intimidate you from hitting that play button.

We provide chapters, which helps skim and you can always just tune in until you tune out. No problem with that. Of course, if you're listening to this, then that means you made it to the end. That's saying something. Thanks for hanging with us. And thank you to our partners, once again. Fasty.com, Fly.io,

and Typesense.org. To our Beat Freakin' residents, thank you Breakmaster Cylinder. Next week on the Changelog, news on Monday, an interview with Josh Padnick from OpenTF about forking Terraform on Wednesday, and of course,

Changelog and friends on Friday. Have a great weekend. We'll talk to you again next week. Game on!