Future of Coding - No Silver Bullet by Fred Brooks

Episode Date: February 11, 2023

Jimmy and I have each read this paper a handful of times, and each time our impressions have flip-flopped between "hate it so much" and "damn that's good". There really are two sides to this one. Two reads, both fair, both worth discussing: one of them within "the frame", and one of them outside "the frame". So given that larger-than-normal surface for discursive traversal, it's no surprise that this episode is, just, like, intimidatingly long. This one is so, so long, friends. See these withered muscles and pale skin? That's how much time I spent in Ableton Live this month. I just want to see my family. No matter how you feel about Brooks, our thorough deconstruction down to the nuts and bolts of this seminal classic will leave you holding a ziplock bag full of cool air and wondering to yourself, "Wait, this is philosophy? And this is the future we were promised? Well, I guess I'd better go program a computer now before it's too late and I never exist." For the next episode, we're reading a fish wearing a bathrobe. Sorry, it's late and I'm sick, and I have to write something, you know? Links: Fred Brooks also wrote the Mythical Man-Month, which we considered also discussing on this episode but thank goodness we didn't. Also, Fred Brooks passed away recently. We didn't mention it on the show, but it's worth remarking upon. RIP, and thanks for fighting the good fight, Fred. I still think you're wrong about spatial programming, but Jimmy agrees with you, so you can probably rest easy since between the two of us he's definitely the more in touch with the meaning of life. The Oxide and Friends podcast recorded an episode of predictions. Jimmy’s Aphantasia motivates some of his desire for FoC tools. Don’t miss the previous episode on Peter Naur’s Programming as Theory Building, since Ivan references it whilst digging his own grave. Jimmy uses Muse for his notes, so he can highlight important things in two colors — yes, two colors at the same time. Living in the future. For the Shadow of the Colossus link, here’s an incredible speedrun of the game. Skip to 10:20-ish for a great programming is like standing on the shoulders of a trembling giant moment. Mu is a project by Kartik Agaram, in which he strips computing down to the studs and rebuilds it with a more intentional design. “Running the code you want to run, and nothing else.” “Is it a good-bad movie, a bad-bad movie, or a movie you kinda liked?” Ivan did some research. Really wish Marco and Casey didn't let him. Jimmy did an attack action so as to be rid of Brook’s awful invisibility nonsense. Awful. As promised, here’s a link in the show notes to something something Brian Cantrill, Moore’s Law, Bryan Adams, something something. Dynamicland, baby! Here’s just one example of the racist, sexist results that current AI tools produce when you train them on the internet. Garbage in, garbage out — a real tar pit. AI tools aren’t for deciding what to say; at best, they’ll help with how to say it. Gray Crawford is one of the first people I saw posting ML prompts what feels like an eternity ago, back when the results all looked like blurry goop but like… blurry goop with potential. Not sure of a good link for Jimmy’s reference that Age of Empires II used expert systems for the AI, but here’s a video that talks about the AI in the game and even shows some Lisp code. Idris is a language that has a bit of an “automatic programming” feel. The visual programming that shall not be named. When people started putting massive numbers of transistors into a single chip (eg: CPU, RAM, etc) they called that process Very Large Scale Integration (VLSI). Also, remember that scene in the first episode of Halt and Catch Fire when the hunky Steve Jobs-looking guy said "VLSI" to impress the girl from the only good episode of Black Mirror? I'm still cringing. Sally Haslanger is a modern day philosopher and feminist who works with accident and essence despite their problematic past. Music featured in this episode: Never, a song I wrote and recorded on Tuesday after finally cleaning my disgusting wind organ. It was like Hollow Knight in there. Get in touch, ask us questions, send us old family recipes: Ivan: Mastodon • Email Jimmy: Mastodon • Twitter Or just DM us in the FoC Slack. futureofcoding.org/episodes/062Support us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Just to my left over here, I have a drum kit. When I say something really loud, all the cymbals shimmer a little bit. That's just cool. And so I'll sometimes catch myself laughing or something like that, and it's like, oh yeah, it's ringing out over there for a lot longer than it should. Yeah, yeah. Yeah. All right, so do we want to dive in?

Dive in! Which I thought was hilarious, by the way. Your little we want to dive in? Dive in! Which I thought was hilarious, by the way. Your little, let's dive in. I like to have a something, and I like that something to be different every time. Yes, yes, I know. I said I wanted to talk about New Year's resolutions, but really I was thinking more like podcast,

maybe you throw in a personal thing or whatever, but this is our first time hitting a new year together on this podcast. Yeah, I didn't prepare anything, though I spent the last couple of days thinking about it and thinking maybe I should prepare something. And then also thinking, well, if you're going to do resolutions, what's something else I could do? One of the things I like is predictions. But then I couldn't think of anything that would be fun to predict. So yeah, I did enjoy the oxide and friends podcast where they kind of do this like live podcast sort of thing. They did a prediction episode and they did one five and 10

year. And they actually like revisited predictions that they had done like 10 years before. Oh, wow. Which was kind of fun to look at. There was ones right before the iPhone. And at first, before it was announced, it sounded like Brian Cantrell had predicted exactly what Apple was going to do. But then the last sentence in his prediction, and it's going to be a big flop.

Yeah. Yeah, I'm not sure I'm ready to make predictions as much as you didn't prepare for a New Year's resolution I feel like making a prediction is even something more I would want to prepare for but I do think that would be a fun thing to do future of coding related 1, 5, 10 predictions

might be a fun little thing to do. Yeah, well, give us your resolutions, and then I might improvise based on that. Yeah, I mean, okay, so I have some personal things. I posted on a bird website that I know I'm not supposed to post on, or I don't know what I'm supposed to do. I don't know how I feel. I love, like, Maston's been cool.

I don't know, whatever. Not even going to get into Masked On's been cool. I don't know. Whatever. Not even going to get into that. But I did post some. And, like, the one that I think is the most fun for me, for a personal one, was just I want to use, I want to be spending 10% of my time in an editor I built myself. This has really been something that's kind of been, like, on my mind a lot of just, like, using my own tools. And what tool I use the most is definitely my editor. I use VS Code mostly with a smattering of Sublime Text and a smattering of Emacs, but I'm never happy with any of them.

I really want to be able to spend the time, dedicate the time in my own tool and customize it exactly the way I want. What kind of thing do you want this editor to help you do that your current tooling doesn't help you with? There's two things that I don't like about current editors that I want to address, and only if I got both of these would I feel good about it. So the first is just code organization

files are kind of like awkward and like there's either really big files or really small files and jumping between them is really like annoying almost always i'm like working on some small subset of code that i care about and i know what that is but then sometimes i like venture off and go you know read some other code and then come back to that small subset and i just i'm one of those people who have like a hundred tabs open at a time right and i just like i have files open everywhere like i think at the end of a work day i'll often have like uh like three vs code instances and then like 14 sublime text instances that I opened up because they're like so much cheaper to open than a VS code one. And like, they're like scattered around and I have like random temp

files and copy and paste that I put somewhere and log file. Like it's a mess. Right. And like some of this is me and I, I watch people who are, you know, much more organized than myself and they don't get into this mess, but I, I can't not do that. Like I need to have those like stacks of different thoughts going on in places to put them. So like that's criteria number one for me is I want like something better for managing that, that will let me accrue all that mess and maybe sometimes keep it around, but also like tuck it away when I don't want it, right? And then the other thing that I really want is like, I think I've mentioned this on the podcast.

I can't visualize things in my head. I have aphantasia. But like when I'm doing work, I often need to visualize things and I often have to build these like external tools to visualize stuff. And I want that in my editor. I want to be able to take a bunch of data that I have going on, whether it's live code or I've dumped it out to a file or whatever,

and I want to shape it and mold it and visualize it right then and there in my editor. I don't want this to be tied to a programming system. I want to be able to do this in any language or any little bit, ad hoc coding, and have some visualizations going on and so i want my editor to expose that sort of like visual layer um and that those two are kind of the main things that i i want to explore and do and have an editor that does and i just don't see anything that's doing those two things at all those two things

have an interesting intersection because when you think about oh i want to bring in visualizations or snippets of code or little like side bits like when you're cooking right you might be prepping some ingredients long before they're needed and you have them kind of off to the side of the cutting board or you have the little bits that you trim off of you know whatever vegetables you're preparing or whatever there's like lots of little secondary bits around whatever the main thing you're working on at the moment is and in a spatial metaphor you can just kind of put it over there and there's a there you can put it over too but with file centric programming where you have like I've got this file open and that file open, whatever, whenever you want to put one of those little bits somewhere, are they being treated as this is now belonging to a particular source file that I've got open. And so if I save that file, let's save those bits with it. Or is there some other kind of like,

space within which these things can come together? That's organized separately from just which file is open in which buffer. Yeah, I think it's going to definitely have to be both there. I want to be able to open files because I want this to be a practical thing, not a walled garden. I want this to be, I can open up any code base in my editor and productively work on it. And so I'll be able to open up a file and then be like, all I really want is this function out of it and get rid of be able to open up a file and then be like oh all i really want is this function out

of it and like get rid of that file buffer all together like i don't want to see it i just want to see this function and then when i save it it would go back to the the underlying file but i do want like things that persist in my editor that i don't have to think about their location but also that i can like query and bring up later. So I'm definitely going to have to have like a database in my editor, right? Like I think that's going to be a crucial feature if I want all these little bits to be able to be around and persist across sessions. That's the other thing that I hate is like, I have to decide like, like I'll actually like force quit Sublime because sublime is really good at like bringing back up

the old things i had whereas if i like quit it properly it's like where do you want to save this i'm like i don't so i'll just like force quit sublime just so it like persists it but it cleans up my desktop right like which is such a like silly workaround uh that i i just feel like i shouldn't have to do right like just keep that over. And if I want to get rid of it, I'll get rid of it. We have so much disk space and memory and et cetera. So I think I have to have both in order for this to really work with the chaotic flow that I have. I can't help but rabbit hole on this.

It's funny that- No, I'm glad. I want this feedback. That's why I mentioned it. No, no, no, no,'m glad i want to i want this feedback that's why i mentioned no no no no i'm not rabbit holing on that rather than on it's it's like one of those you know evolutionary processes or maybe people would say like oh yeah responding to the needs of the market or whatever uh theory of how things work you want to justify it's funny that going through the 90s and the early 2000s where if a program crashed as programs frequently did that meant you would lose work or if something

bad happened in the middle of a save that would mean you'd lose that file it created this environment where there was a lot of pressure to do atomic saves so your new save can't clobber your old save and you end up with a zero byte file or have um you know an interval where every so often every certain number of edit actions or every certain amount of time you save a backup copy somewhere in some temp directory so that if you do have to quit without saving there's a recent file you can restore to when you load back up and it's funny that like every app that was used in a professional context had to implement that itself so like ableton live does it one way 3d studio max does it one way final cut does it one way and then eventually the operating systems started to add support for that so there's like resume and auto save and versions on mac and then there's whatever

windows and linux do that i don't know because i don't use them um but presumably they do something and so there's like we've gotten to the point now where it's like the forcing the app to force quit like you know kill nine sig term whatever like just go the away used to be a recipe for data loss and is now a recipe for better state restoration than a normal quit i yeah i find that delightful and it's i i see just to go slightly deeper into a side branch of this rabbit hole i see a new thing starting to happen which is programs that don't let you save because their automatic state preservation is so constant it's like every single thing that happens is saved but they have really rich facilities for undo and redo and whatever within the program and i'm thinking like final cut for

instance final cut nowadays you can't save you're working on your video you're doing your edit whatever and when you're done you just quit and it doesn't matter if you quit normally you crash you're never asked do you want to save you're never asked you know this is a new project and you need to pick somewhere where those files are going to live that just never happens um and so it's almost like we're designing and i guess this is maybe a reaction to the google doxification of everything where it's like what are even files anymore let's make the file system less of a thing and so don't make people think about files don't make people think about saving yeah i don't know it's it's cool that your editor is going to be yet another crack at this. Like, hey, files are useful sometimes but annoying other times.

How do we make it so that working with them is nice, which means you still work with them. You're not making files disappear a la Google Docs. But how do you make it so that when you don't want to deal with files you don't have to deal with files yeah and and the thing that i am making as a definitive criteria is it only has to work for me i i like i don't really this isn't me trying to go make a product right and so like i'll probably stumble upon some abstraction between files and not files that it's going to be like a little janky, a little not polished, but like, as long as it fits the way I think about it, I'm going to be

good with it. And like, cool. It would be great if like I built something good enough that someone else wants to use it. But you know, it's an anti-goal for me right like i i want to intentionally like i have to stop myself from thinking all right how can i make this production ready or you know whatever it's like no as long as i am using it and enjoying using it it's good and so like i've done a little explorations like uh some live code uh i've used wasm right now. It's just a fun way of doing extensions that I can do hot reloading for and keep state around even through the hot reload, which has been

nice. And it gives me a way I can interrupt things and make sure I hit frame times. It needs to be the other thing that I care about is just performance here. Because I can do a lot of cool stuff in Emacs with Clojure, but wow, is it just awful performance. It just feels so bad.

If I look at any more than three megabytes of data, it's just at a crawl. It's been really fun. It's very early stages, and I had some exploration on my personal site where I made one before, an editor in Rust and rust and did kind of a canvas thing. And that's, it's going to follow on from that. You know, I do think it'll probably take the form of kind of a canvas thing because that spatial metaphor does just work for like random little bits. Right. And that's how I did math in

school. Uh, they always wanted you to like i had one teacher who finally like in elementary school taught me this way of formulating things which i liked so much better it used to be like you have your sheet of paper and you have to work on your problem and you'd like put one and circle it and then you'd write and then like two and circle it and write the problem there but my paper would just become a mess if i did that and teachers kept like complaining about not being able to find my answers. So instead I put my answers on the left-hand margin and just like labeled one, two, three, four, five, here's all my answers. And then the middle of the page was just like random scratch things that I did anywhere. Right. Uh, and that worked for me so much better. Like trying to keep it organized

just made me not be able to think as well. So that's kind of the same idea here. I want that, but for my editor. Yeah, I think this is, for the editor, it is really fun just to think about these problems and to kind of just have something that you can mold and fit to yourself. And so, yeah, I just, you know, I have been really enjoying it.

I've tried to make my code in general more personal. You know, I've kind of had this like, oh, well, there's all these like style guides and standards. And like, I have to do all this stuff for work and I have to make sure my code's comprehensible for other people. And like, don't put too many personal comments in there, because that's unprofessional or something. Just having very personal code that I can write whatever I want, however I want, and change it, has just been really freeing and fun.

So definitely something I recommend people to do. Make your code, even at work, more personal. I have the luxury of working at a company for a very long time and having been one of a few or the only programmer for much of that time and so i have this whole ecosystem of like oh we have our own way of doing dependency management and and you know code loading and modules and we have our own house style and we have our own way of using you know all of the things where it's like oh there's a standard way that everybody does it just because it's easy to learn and easy to bring new people into it's like no we don't we don't

have to worry about that if there's some pain about using you know javascript modules or require js or whatever don't use it write our own thing that that scratches our own itches and so we are we're a a team of one or two or three full of our own back scratching machines that are very comfortable to sit in if you are shaped like we are every time you tell me about that i have like the dual reaction of like that sounds wonderful and that sounds absolutely awful yeah it would be awful to join and it is scary when when a new person comes in and they say what the hell are you doing um but people who get to be here for a while get to go wait i can make it whatever i want okay yeah um that's actually very nice okay so we've talked about

my personal things yeah yeah yeah we got to talk about something for you and the podcast and the podcast the podcast gets to make resolutions okay yeah the podcast itself is gonna make some resolutions all right that'll be i expect now i've editor ivan to put in a voice of a podcast all right well there goes two hours of my life i'm gonna increase compression uh aka lose some weight no our episodes are actually pretty heavy i don't know if any of the listeners care about this but i ship them in stereo which is rare and i also do not a super high bit rate but a higher bit rate than other people so i'm always self-conscious when i go to do the compression i'm like wow it's a it's a two hour recording two and a half hour recording but it is 300 megabytes that is a lot for putting on somebody's

phone so it's fine yeah i don't. I don't know if it's fine. Anyways, I did think of a resolution, and it's 1920 by 10. No. I have to make that joke every year. It's mandatory as a programmer. I want to learn how to write graphics code on the GPU this year. That's what I want to do. Nice. Because I've done a little tiny bit of that in the past i've written a little bit of webgl um but i've

never gotten good enough at it that i would turn to it to use in anger on a serious kind of project and this year i've got some work stuff where i will need to ship some real-time 3d stuff in the browser for work so i have an excuse to learn it for that nice and then if i want to do more visual programming kind of stuff one of the performance bottlenecks or the performance bottleneck that i ran into with when i was working on that was just rendering and if i ran the programming model without rendering it would do millions of operations per second and if i ran it with rendering it would do tens of thousands of operations per second and part of hest being able to you know run super fast and run in super slow motion jumping way up and down scales of time um means that it needs to be able to go pretty fast when you crank the speed slider all

the way up and the faster it can go the better um and so if if rendering was the big bottleneck there i want to find ways to get rid of that so i think learning how to put code that i write on the gpu for great good is my my i would like to do that this year after many years and you know background backstory i've spent many years telling myself i'm going to learn how to write some code on the gpu and so now i am publicly committing myself to the goal of learning how to write some code on the gpu and and and actually and you have your use it for 10 of your programming kind of benchmark i think my my specific goal will be and i will actually like ship something that other people who don't know me by name will see and use that runs code on their gpus. That'll be my like, I will have succeeded at this goal

if I run some code on other people's GPUs. There's my, you know, if I live in their GPUs rent-free, then I will have achieved my resolution this year. Nice. So do you want to talk about podcast resolutions? Or no, I didn't get that. Oh, I don't have anything.

Do you have anything? I got an easy lowball one resolution that we could talk about if we want to have this. And it shouldn't take too long on conversation. Sure, sure. Cool. Cool. Yeah, so the thing that I've been thinking about

with the podcast is I've really loved the papers that we're doing, but they're all papers that I have personally read before this podcast. I know that maybe you haven't read all of them. About half of them. And so I would love for this year just to kind of explore some weirder papers that maybe aren't kind of in that,

like, you know, the canon, right? Like all of these things have been kind of canonical. They're kind of known. I would love to kind of just explore some weird papers on the edges even if they don't end up being papers that like we personally loved right that's the risk we take right that's why i've chosen like a bunch of papers that are kind of classics right we kind of chose them together but i've kind of you know had a big list of papers to pull from so i really want to like kind of branch out and do some weirder stuff. Yes, we'll keep coming back to the classics. There's still things that we haven't covered that we need to cover,

like Ted Nelson or something. But I want to branch out and just see what it's like to go, still future of coding related. It's not like we're going to change this to a literature podcast or something uh but a little bit on the the fringier side than what we've done i have been threatening jimmy with this for a while and so i'm now going to publicly threaten jimmy with this i want to do some things that aren't papers so i would like to review some things or discuss some things or or reflect on some things that aren't a paper, maybe aren't even written, maybe aren't even the sort of thing that one would normally do an analysis of, but that might get us to think different kinds of thoughts about what programming could be or what we could do as programmers or who the hell knows. Because I think that's my role here is to push in uncomfortable directions.

That's what I was put on this digital earth to do. Yeah, I'm all for it. I think the obvious things are like some videos or something, but I'm down for whatever. We can go read a science fiction book i know that's still a text but that's very far away from this we could go watch a movie i you know i i like the conversations but what i what i like about having a text and for me a text just means is very like broad here like a movie could be a text right a work yeah a work yeah i

want to work just because i think it grounds the conversation and it also forces us not to just fall back on our habits yeah right like we have some other thing that's kind of pushing us along and making us talk about topics we might not naturally talk about and that's what i that's what i really like about it so as long as we have that element and we don't always have to you know we could do a topic a topical show but i like having that the the intrusion of the text that kind of forces us to to wrestle with it rather than just kind of repeat the same things that we might generally talk about if it's just a chit chat show yeah i definitely need something to stop me from being a visual programming

on every episode because that's definitely not what I've been doing with all of our papers so far is repeating my same old tire talking points about visual programming. And this paper will not give you any chance. No, there's not going to be any. I don't think we've mentioned in the audio here what we're reading today, but I think this is a good segue into it. Let's do it. Actually, I've got an even, well, you set it up,

and then I've got a second segue. Okay. We're going to do a double segue today. A two-wheeled segue. Yeah. Yeah, okay. So we're reading No Silver silver bullet here uh essence and accident in software engineering never really think about the subtitle by uh frederick brooks so this is a

pretty classic paper by fred brooks who also wrote the mythical man month uh the book and the shorter chapter inside of the mythical man month uh you know the idea of like adding a person to an already late project adding people to an already late project makes it even later right we kind of all know this uh kind of general wisdom i think no silver bullet is still pretty popular but maybe not as well known uh we can we can give the the summary here but i you know like the little uh abstract i guess but i i you had a segue and now i'm like yeah so here's my here's my second segue right so the so uh-huh no silver bullet essence and accident in software engineering we'll come back to that and it begins with a little kind of italicized kind of pull quote or something where it says, there is no single development in either technology

or management technique, which by itself promises even one order of magnitude improvement within a decade in productivity, in reliability, in simplicity. So to summarize kind of this pull quote, but also the entire point of this paper is that fred in 1986 when this was written is saying there's not going to be any single thing that we're going to come up with that will give us an order of magnitude and so he means times 10 improvement in the way that we make software in terms of our productivity in terms of the quality of that software that there's not going to be any one silver bullet kind of thing that gives us that 10x improvement within a decade so it's saying there's not going to be a 10x improvement within a decade and what i like about that the second segue here is that this is him making a 10-year prediction and throughout this paper he's got some things that are like oh you know

is this a promising thing well when we look back on how it's gone in the past over the last little while it hasn't been very promising but maybe in a longer term it will be promising or here's a thing that hasn't been tested yet that's a new idea that when we think about it it doesn't seem like it's going to have any promise in the near term or any promise no matter how long you give it so he's making lots of predictions about near-term futures far-term futures and i enjoyed that framing and and i will say just right off the bat the first time i read this paper which was a long time ago i liked it the second time i read it i hated it this is now my third time reading it we'll get to the verdict when we get to the verdict but one thing that helped me in thinking

about this this time is and i'm going to challenge us to do this as we go through it is to actually think about this paper with two different perspectives one perspective is going to be taking what he says as he's talking about 10 years and a 10x improvement like using his actual framing to reflect on the things that he said and that one is less interesting because it was written in 86 and so he's making predictions for what's going to come true by 96. So this is in the past already, we can kind of evaluate, did he get it right? Did he get it wrong? Did his predictions come true or not? But the other way that I want to think about it is ignoring those self imposed limits that he put in this paper, ignoring the 10x improvement and ignoring the decade times scale and thinking instead about

which of these things that he's talking about in the time since 1986 how many of them have come true but it took longer than a decade to make a 10x improvement come about which things did make improvements but they weren't 10x which other things did he not think about that brought us 10x improvements which other things did he not think about at all that are we see them now when we don't know how they played out and will they give us 10x improvements or whatever scale of improvement and just sort of taking the spirit of the paper and thinking you know with that spirit in mind where do we see this sort of framework of thinking that he's putting forward here um happening in the world what kind of predictions of our own do we want to make or what kind of reflections on recent trends look like they will pay off and which ones don't and sort of doing no silver bullet again ourselves here now as opposed to Fred 1986. So those are the two frames. Take it literally and then take it spiritually and run with it in our own directions things i hated about it were things where i was ignoring

that framing and when i've read it now with that framing very front of mind of you know he's only talking about a decade he's talking about one idea at a time and he's talking about a 10x improvement so if something takes six months to build he's talking about it in you know uh one-tenth of six months i i should have said 10 months and then reduced to one month that would make math easier uh so if something would take 10 months to do will a single advancement give us one month turnaround on that 10 month project that's a 10x improvement of the sort he's talking about i think the other thing that's kind of, I think never explicitly stated exactly in this, this paper, but that I always had a tension with, he almost seemed to kind of jump between

them is what, what's the scale of improvement we're talking about? Not in terms of the magnitude of the improvement, but at what scale is the improvement happening? So is it the industry as a whole? Is it a particular business? Is it an individual? Is it a team? Like, you know, because I can think of things that might make me 10x more productive, right? A singular thing that could make me 10x more productive. But it doesn't seem like that's the kind of thing he's talking about. He seems mostly, I think, like I said, I don't think he's completely consistent about this.

But to talk about the industry as a whole becoming 10x more productive, right? Or at the very least, like a big business within the industry, like a big software firm, like an IBM or something like that. Would all teams across IBM be 10x more productive if they adopted some given methodology. Yeah, it's got to be kind of this generalized thing that at least could be applied, even if it wasn't applied equally. It could be applied to everyone, and everyone would benefit from it, right? So it can't be like, yeah, if you fed me ice cream

every hour on the hour, I would be 10x more productive, right? Like, that doesn't count doesn't count uh as like as a an improvement here right because that's like a personal sort of thing right so i think those are really important and i you know i i will say the first reading of this where it's like evaluating you know from 86 to 96 or whatever uh fred brooks i think does have a follow-up where he evaluates it that we didn't read i read it okay okay i skimmed it and it seems like he just pretty much agrees with what he said before yeah and he does some some uh takedowns of people who disagreed with him that are yeah that are arguably entertaining to read yeah yeah uh but you know i wasn't born for the first part

of that and then was very young for the second part. I don't really know looking back. I'm just saying, I don't think I can really evaluate were there 10x improvements in productivity from those time periods. I really have no way to do that. So I'm much more interested in your second reading of like, you know, can we get these improvements? Is 10x really, are productivity measurements really measurable like that? I have no idea. But I will also say what I think we should also focus on is 10x improvements in not just the dimensions

he talks about, productivity, reliability, and simplicity, but in any dimension that we're interested in. Could we make programming 10x more accessible, 10x more fun, 10x less boring whatever whatever i think those are two different ones in my mind but yes right like something like that i think those are also improvements that i care about and probably care about more than productivity reliability and simplicity yeah like i i could imagine making programming 10x more uh or 10x less boring by making it like you know some terrorizing like old gods tentacles coming out of the screen like ripping your face apart i'm not bored but i'm not having fun either

so yeah they are they are separate dimensions right right yeah okay okay so yeah let's let's dive in i can't use those words now no you cursed those words they're cursed let's read the paper let's read the paper oh actually oh shoot i have one more uh framing quibble because this he does so much framing uh-huh and when you read this paper without the framing i think it's more enjoyable so, getting away from that framing is good. There's another frame that we have to break out of. And that is the first line of his pull quote. There is no single development.

What is a single development? So we'll get into his examples later on. But it's a really weaselly kind of framing device because it's, it feels like what counts as a single development is, can you come up with a name for this thing? Does this thing have a name? So for instance, one that he doesn't talk about, but like waterfall, right? Waterfall as a, as a software development methodology where it's like, you know, you're going to do your big design up front and then you're going to do, you know, your, after you've got your specifications, you're going to do your big design up front and then you're going to do, you know, your after you've got your specifications, you're going to do your implementation and then you're going to do your testing and then you're done or whatever. You know, the waterfall development process, because that has a name, I think he would count that as a as a single development.

I mean, he does talk about getting rid of waterfall. Yes. Yeah. Okay. So, yes yes i would agree so it the things that are given considered to be a single development are very um loosely treated it's not like you know this exact implementation of classes within this specific programming language with this specific compiler on this machine architecture when you use this methodology with it like it's not very tightly scoped but it is something that he uses as a way of kind of and this comes up in the in the no silver bullet refired uh follow-up essay where

in 95 he reflects back on the nine years since this original paper came out and uses it as an opportunity to dunk on some people who took umbrage with this paper by saying, hey, you can't point to more than one thing and say that those things together led to a 10x improvement because I said only one thing. And so some of the responses where they were taking a couple of things together and saying, look, we got out of that he says no no no no i'm still right so yeah yeah there's there's definitely the the one thing i do agree it kind of feels very it feels very artificial not just kind of it feels very artificial what we're interested in is not is there one thing i get it it's no silver bullet right yeah and and there is something good about that because people do often point to one thing and in some you know defined way there as if that's going to solve all of it and i think it almost always is these combination of factors um but yeah i agree let's take off if we're thinking about improvements in the future let's just take

off the the one thing criteria and say yeah it could be lots of things playing together over a longer time span etc right yeah because we're not looking for trying to you know micro optimize programming we're trying to radically reinvent programming wholesale so we are trying to do that right that's the that's the plan right we're going to tear it all down and rebuild. Welcome to the Radically Recreate Programming Wholesale Podcast. We're the programming separatists. We're going to start our own faction of programming

and we'll sabotage GitHub. We're going to be Luddites, just destroy all the current programming, right? Okay, good. Okay. Yeah. So now I think we get to what might be my favorite part of the paper and maybe the the one you know part that is most resounding to this day and that is most famous from this paper and it's time i think for jimmy's philosophy corner to to cover this first part of the paper which is the essence and accident yes that is

where i was going thank you jimmy all right so so yes this is one of those things that people get you know confused about with this there's these two words essence and accident and accident in our modern parlance means like a mistake right but this actually is like an aristotelian term and the the best way i can try to give here is we think about like what makes something a cat okay so you have catness right the thing that unites all cats is the essence of the cat and then you have the the facts about any particular cat like this cat has orange fur this cat is mean this cat is nice this cat hunts mice right whatever it is that is uh not part of being a cat itself those are the accidents so these are the properties that an object has that it didn't have to have it could have lacked

those properties and still have been the kind of thing it is still could have been a cat so you can have a cat with orange fur and that's a specific cat and the specific cat happens to have orange fur but if that specific cat or even cats in general when you think think about them, if they didn't have orange fur, they would still be cats. Yeah. If this cat didn't have orange fur, it would still be a cat. So you can take orange furnace away and it is still,

we will still treat it as being a cat. But if you took, for example, life, like one of the things, a living, a dead cat is not a cat. A robot cat is not a cat, it might look like a cat but if

you if you opened it up and you discovered oh no this thing is made out of you know metal and silicon and it's you know we would find it's not really a cat right and so there's these things that make something a cat it and it's you know in arist's time, it was kind of admitted that, like, of course, we don't know all of the exact criteria. You know, you couldn't, like, list all of these criteria, but there's something right about that. There is something that takes something from being the kind of thing it is to not being that kind of thing. And these are actually, like, still really important today. Like, if you think about the essence of something, one way you can think about it is its persistent properties. What sorts of changes can it undergo and still continue to exist?

So a piece of paper, right? If I write on the piece of paper, it's still a piece of paper. If I burn the piece of paper, there's no longer a paper there, right? And so these are the kinds of things that we can think about, you know, the undergoing of transformations. And just because I have to be nerdy, I'll mention a fun example that's often used in philosophy, which is a lump of clay, okay? So you can take a lump of clay and I can mold it and shape it into a statue. Do I have one object or two objects i have the statue

and the lump of clay right but those are one in the same except they have different properties the lump of clay can survive me smashing it the statue does not survive me smashing it so we have two objects that have the exact same material but yet they have different properties so they can't be the same object this is the uh i think it's calvin and hobbes the one where they put the bread into the toaster and they take the toast out but where does the bread go that's good i like it yes that is that so yeah so these are these this is what these distinctions are used for right the statue has an essence of being in this certain form the lump of clay does not right these are those this essence and accident and then here the lump of clay has the accidental property

of being in that certain shape of the statue right whereas the statue has that same property essentially and debatably you could say once it has become a statue it is not a lump of clay anymore well yeah but it is still the clay right yes we could say that clay right and this gets into like this is you know an entire field of study uh called philosophy that some you know narcissistic masochistic people get into there's like ship of theseus is you know something everybody's going to be screaming at their at their cassette players listening to us talk about this um the other one is you know is a hot dog a sandwich that's a very popular one on the internet recently you know what makes a sandwich a sandwich

exactly yeah and biology is dealing with these kinds of things as well right and they might get rid of this sort of aristotelian language and that's totally fine i'm not i'm not trying to defend aristotelian concepts here i'm just explaining them but you can imagine you know when do we have another species right speciation seems to be a question about these kinds of of properties right what is it for it to be a member and you might think it's just arbitrary you might think it's just convention but even if it were that doesn't matter for the distinction to hold here right and so how does this apply to to programming here to software well uh fred brooks wants to say that there's certain kinds of problems that we face,

or certain types of tasks that we do. And some of those are essential tasks that they're things we have to do, because they're just part of the problem statement we have. And then some of them are accidental. Some of them are just facts of history, there are ways that we have to do it right now. But if we kind of like take a high minded way of looking at these problems and programming, they wouldn't really have to be there. So if I had a computer that shut itself off every five minutes, and I had to turn it back on and boot it, this would be an accidental property of me programming is that every five minutes, I have to turn my computer back on. But maybe it's kind of harder to say, but maybe there's something like like you know

the concept of data and putting data in a shape right like that's essential to the problem i'm solving i need to have that and so in this like abstract here we can ask the question how much of what software engineers now do is still devoted to the accidental as opposed to the essential. So that's the question. We'll find out that Fred Brooks thinks very little. He thinks most of what we do is essential. Well, I don't know that he actually anywhere in this paper says what he thinks the split is. He does in the follow-ups that I read.

I didn't catch anywhere in here where he says it. But what he does say is unless it is more than nine tenths of all effort so unless the parts that are accidental are more than are consuming more than nine tenths of all of our effort as programming and and thus the essential tasks are consuming less than one tenthtenth. Shrinking all the accidental activities to zero time will not give an order of magnitude improvement. So he's looking for this 10x improvement. And so to get that 10x improvement, if you're only attacking accidental aspects of programming and you want a 10x improvement, then by that means that at least 90 percent of what

constitutes our programming efforts have to be attending to accidental aspects of the programming and not essential aspects and if we've already shrunk the accidental parts down to less than 90 if it's more like you know 70 30 or-30 or 50-50 split between essential and accidental, then any new single technique or single development that just attacks the accidental stuff won't give us a 10x improvement because there's not a way to, if it's 50-50 and you reduce all the accidental stuff to zero, you're still left spending 50 of the effort that you started with and so that is not a 10x improvement that's that's only a 2x improvement and so that is that is the challenge here is is can we figure out if there is still even

90 of our effort going to accidental stuff and thank goodness he actually in this paper gives us some pretty good examples of what things he counts as being essential and what things he counts as being accidental so we don't have to sit here pigeonholing over you know well is this is is a type system essential or is it accidental? He does give us a sentence, maybe two, about what is the essence. But yeah, I kind of took that as like, this feels very reductionistic at the best. So I'll read what I think he's saying,

what you're referring to as the essence, and we'll see, yeah, we can talk about that. So the essence of a software entity is a construct of interlocking concepts, data sets, relationships among data items, algorithms, and invocations of functions. That seems to be what he thinks the essence is. Yeah, and then I also, i had that highlighted as my definition for essence but i also had the second sentence this essence is abstract in that the conceptual construct is the same under many different representations and i think that's important and i'll say why in a second um but just to elaborate on that i believe the hard part of

building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. And this right here, what he's describing is very well aligned with the theory building view. I think that what he's saying is that the the thing that he's calling a conceptual construct is what we given nowers theory building view would call the theory and then the accidental stuff is all of the you know the in performance of the theory um but that the the essential part of the thing you're building is the theory it's the the sort of interlocking concepts that you have to develop as you decide what to program and then actually building it the labor of representing it and

testing the fidelity of the representation that is all the accidental stuff so that's that's how i read it is that if you take a theory building view the essential part is the theory and the theory building and the accidental stuff is all the other stuff. Yeah, I'm going to have to disagree. All right, let's do it. Yeah. So, okay. I see where you're coming from. And I'm happy that we're using Naur as a frame here.

That makes me super excited. But I think Naur's theory, we have to remember, is not fully conceptual. Because it's embodied knowledge right so that's my a quibble but that's fine it's just a quibble but i think the main thing is what is the theory a theory of it's a theory of the concrete program and how it maps to the world it's not a theory of the abstract ideas it's a theory of the real representation, the representation itself. To build the theory is to labor to represent it, and to have the theory is to be able to explain and do certain moves and talk about the representation itself, to make changes to the representation etc so i i do see a mapping here but i think with now we're the representation and the conceptual kind of all come in together as as part of the theory yeah so i see what i i totally agree with that that the theory that you build in the theory

building view includes as part of it some of this accidental stuff and you could cleave that out in theory or replace it with different accidental stuff and still just be speaking of the theory that is built separately from source texts and that sort of thing so i guess it would be a different kind of theory because one of the parts of your theory is knowing how to make changes to the concrete thing yeah right so i think another a better way to to look at this then would be to say all of this stuff in the when you're using the theory building view all of this stuff that is not the theory you built is accidental is that fair huh interesting um source text documentation you know any artifacts that are produced in the course of producing the software that are not the

theory held by the programmers see i i think the theory itself is going to have a bunch of accidental things too yes yep okay But the inverse of that, the non-theory parts in the theory building view, the stuff that are like the source text documentation, stuff that's not the theory, I'm saying all of that is accident. See, okay, so I don't think so. Okay.

All right. And the reason I don't think so is because one of the things we're doing, at least by the frame we have here in this paper, we're trying to build a system to accomplish something in the market, solve a problem, whatever. And we have to have the running artifact in order to meet that goal. And we would not meet our goal

if we didn't have that artifact, so therefore it's part of the essence. And in the theory building view, the running code is not part of the theory but i think here it would have to be essential cool okay yeah i'm i'm convinced of that yep that makes sense so i got one more on this okay good i love it i'm all for it so i have i my highlighting scheme in this pdf and hopefully unlike last time preview.app will do a good job of respecting all of the annotations and highlights and notes and stuff that i put in this pdf and so i won't lose all that work like last time um but one of the things that i do is i

i um i'm very traditional i highlight stuff in green when i think it's good and i highlight stuff in pink when i think it's bad why pink I highlight stuff in pink when I think it's bad. Why pink? Because there's no red. So it's the closest to red. So green means good. And reddish means bad. And I have a giant sea of green for that quote that Jimmy started us off with the essence of a software entity is a construct of interlocking concepts. All of that is a big sea of green with one little pink bit in the middle, which is algorithms. So he says, the essence of a software entity is a construct of interlocking concepts. And then this is a colon. So here are the concepts.

The concepts are data sets, relationships among data items, algorithms, and invocations of functions. And I can see why the choice of data sets, relationships among data items, and invocations of functions can be considered essential. I do not see how algorithms could be considered essential. And I'm wondering if you can convince me of why the choice of algorithms because i'm assuming that's what he means by this not just like the existence of algorithms as part of the program because that seems sort of reductive to the point of uselessness um because the other things are more specific than that right like relationships among data items and data sets

meaning like this particular collection of data, those are more than just like data is going to be a part of it, right? Data exists. I don't think that's what he's saying. He's saying the choice of sets of data relationships among data items, the invocations of functions, those all feel essential, but why algorithms? Why like the choice of algorithm as an essential part? I don't, I don't get that. Yeah, so I will say, I personally read this as just that algorithms are there, but I can defend the other reading. Do you want to defend the other reading?

Yeah, I'll defend the other reading. If we take algorithms to not be so narrow, as I think you're thinking of them, and instead be a series of steps that need to be performed, I think the series of steps could be essential. Maybe there's multiple of them that could be essential, but some set of them have to be for business processes, right? I need a series of steps that the user is going to go through to buy an item, right? And if we're talking about

at that sort of level of distraction like we need to have certain things have to happen after another we need to have these things follow like you can't buy the item and then check it out and you know like you you check it out and then you buy the item you can't pay before that you can't pay after the item shipped whatever right like these sorts of like constraints that we have, I think algorithms here is cast more broadly than just like which version of pathfinding that I use, right? But I think he just means that there has to be some set of algorithms that we chose.

I don't think he means that there has to be a unique set of algorithms if we do it more narrowly. So that's like saying the essence of a software entity is that it is runnable as software i mean invocations of functions also says that yes but that's why i think it says invocations of functions i feel like algorithms is saying something different i think it's saying that there's going to be this like that you are going to arrange a sequence of actions and invocations of functions is saying there will be actions. Okay, I'll try one more time to defend it.

I do think he just means existence, but I'll try one more time to defend it. All right, so if we don't think that he means the particular algorithm, like you had to use A star here, you can't use, yeah. But what he could say is the kinds of algorithms, right? The class of algorithms that we have to use. So for example,

if I need to sort something, that is not required. I can't just use data sets, relationships among data items. Well, relationships among data items covers sort. No, because there's, no, no, there's multiple relationships that I could have.

And one of them is sequence. Yeah, but just because they have that relationship doesn't mean I've put them in the order to access them that way. And so you could say, oh, but any algorithm that does sorting would work. So that's why algorithms shouldn't be in here, because any algorithm would work. But no, BOGO sort would not work for almost any application that you care about. The time complexity of that algorithm does matter

for being able to practically solve the problem and Fred Brooks here is very practically minded. Yeah, he's definitely not. I'm definitely coming at this from a perspective of like saying that the essence of software is that it's runnable doesn't sit well with like the halting problem like those kind of fight each other a little bit right like it's sure for all practical intents and purposes the software has to terminate but we know that that's technically impossible that it's technically impossible for any software to terminate ever.

So I think that statement, I have to disagree with. Technically impossible for any software to terminate? No, software never terminates. You can't quit. Me? Okay, okay. But yeah, so I think that that's why. We have to have, there's a class of algorithms we're going to need to include,

and to make any non-trivial application that doesn't invoke some of these algorithms is just not going to happen. Sure, I'll grant that that's what he meant, though I feel like you could take it out and cover the meaning of that with data sets, relationships among data items, and invocations of functions, and you could probably simplify that even further to just functions and data yeah yeah uh i still think

this essence is awkward and wrong so all right uh it's time for you jimmy to share what is what is your essence of software uh so i'm not going to give an essence because yeah i don't i don't think i can do that just like i this is why i said aristotle didn't think we could you know give a whole criteria here but the problem for me is why he says this essence is he's now i think mixing these levels that i was talking about is this the essence of programming like me sitting at a computer programming or is this the essence of industrial programming building software systems right and he seems here to be narrowly focused on what the essence of a software entity is and then he wants to say the hard part is of building software is the essence basically but like he's talking about productivity reliability simplicity he's

talking about like industrial strength things and the amount of time it takes and cost and all of these things that just don't seem to play into like i feel like even if this was a good definition of essence in abstract it's a very bad definition for the rest of the paper we're going to talk about where he keeps coming back and saying, but that doesn't tackle the essence. As if this is the essence of the whole problem of software engineering at an industrial scale. And that seems very weird to me.

Yeah, totally agree. That makes sense. It could be maybe a definition for programming in the small but once we're like past that it feels like a very weird definition and i would love to get down to like where this really is what i spend all my time with but spoiler it's not and like most of what i spend my time myth is not this hard stuff and even if it's hard it's not hard because it's the most timely part. Like, you know, it just, the whole thing feels a little confused for that. Even though I agree with, I think the general thrust of the paper, I feel like some of these details, he kind of like confuses some levels. Yeah.

And he does, like, we're not just trying to project this sort of, you know, lack of clarity around what level he's talking about onto this paper. Like, he invites that. He says that there is no single development in either technology or management technique. does want us to read all of these things as um ideas that are applying at that industrial scale you know across the whole industry large companies big teams huge projects huge budgets like he is thinking about this as a as a suit and tie you know working industrial programmer in the 1980s so yeah i think it is fair to kind of say this reduction of the notion of essence to be something that's just about like these particular technical aspects of a particular running program is a bit of a weird reduction given that right, he's like justification for why we're talking about a silver bullet, right, is because we see these missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet, something to make software costs drop as rapidly as computer hardware costs do. So, like, to him, how have we gotten the silver bullet?

If software costs drop 10x would be another way of phrasing it, right? Which feels like... That's easy. Yeah, that's unfair. Open source, right? You don't have to pay anybody anything. Well, yeah, we'll get there, right?

We'll get there. But that, which I do think is probably one of the answers for a single thing, if this is the criteria. But now we're talking about costs, and before we were talking about the essence of data sets. But data sets don't cost anything of themselves. So if you tackled the essence of the problem, you wouldn't at all get to the cost part,

because all that's free yeah right or like just pay engineers the way less and then you drop the cost even though you didn't fix the time like none of none of this like it just feels very confused in the different levels that we're at and like i said i don't want this to be a criticism of the paper i just want to be like you've tried to get rid of some of the framing. I think this is also stuff that we have to just like, let's think at these different levels and talk about them there. And to give him credit, the no silver bullet refired follow up, like, yes, it is him responding to a bunch of criticisms that raised many of these points that we're raising and saying,

well, here's why that criticism doesn't count why you know no no you you took away the wrong thing but at least he is giving a platform to those criticisms that i otherwise like if he hadn't published all those things and responded to them i wouldn't have seen those responses and some of them do and he agrees much to his credit some of them do point out flaws in the framing of this particular you know set of arguments and he in responding to them he does say yes you're right if that was the framing this would be a silver bullet but that wasn't the framing i meant so it's not a silver bullet in that definition but it is giving us an order of magnitude improvement

to have open source or whatever it is. That does improve things. It's just not the kind of improvement I was looking for or concerned about. And I think it's also worth saying once again that he is subtweeting somebody with this paper. He's specifically subtweeting somebody with this paper he's specifically subtweeting like that there are a lot of people coming forward and saying adopt my methodology that has never been tested and uh

you know we've done a toy project with it but we didn't have to ship anything to any customers and we saw you know a 10x or 20x or 50x improvement in this metric there's a lot of what he thinks of as like charlatanism in methodology and in technology at the time and this is meant to be a sort of a countering force to that it's meant to equip people who are being subject to the pressures to adopt these new things coming out with some some counter arguments to say hey don't just go adopt the you know the new process du jour and think that you're going to get a 10x improvement because if it's not addressing essential complexity there's no way it can give you a 10x improvement just definitionally so that's that's that's that is the framing that i think we should keep that in mind because um well it is fun to strip away

his framings and play in the space it's worth um honoring the fact that like he is he has a certain goal in mind for this and it's not just to write a really kicking paper that everybody thumbs up to but it's to stop people from being swindled a little bit yeah and i think with his framing you know my my assessment is i think he's right not just for his time but kind of now too like i do think and maybe it doesn't make the point that some people think it makes yeah uh like they think oh well that therefore we can't have a 10x improvement right like that's not the Yeah. like you said he's focusing on people claiming their one thing will do it and i think you know we have to have a holistic view of this but let's let's let's uh continue on oh my gosh i can't say let's dive in anymore next episode i'm just gonna say um and it's gonna be repeated a million times or right which i know i say all the time uh so uh the next section here we kind of get into some of

these uh inherent properties of essence he has a list of these and their complexity conformity changeability and invisibility so i figure we want to you know dive into what he means here i don't have a whole lot of of highlights i do yeah i got tons of stuff i can i can get into about this i found this interesting i will just give you my system for highlights since you gave yours three colors i have blue green and yellow and i change between them randomly when i feel like nice yeah but if something's really important i'll use two colors whoa at the same time uh-huh just like overlay them whoa i don't know you must be using a different app than me because i don't think i can do that uh yeah i'm using muse for oh okay yeah

right i just got got a pen highlighter. Yeah. And I only recently, I usually only had one color, but then I decided I was going to do a color scheme for good or bad, and Ivan. Those are going to be my three. Good, bad, and Ivan, the three genders. But I realized halfway through that I had not done that at all and i had just used one color the whole time and so then i just decided to keep randomly changing colors this was a few readings

back okay cool so i'll i'll tour us through this part and if if we hit something that you have stuff to to say uh about um go for it um so the first one of these so there's the yeah four inherent properties of this irreducible essence of modern software complexity conformity changeability and invisibility first up complexity and this one is interesting to me especially also what we were just talking about with like essence versus accident um because there's a certain other paper called out of the tar pit that references no silver bullet and talks a little bit about essential complexity and accidental complexity and how important it is to reduce accidental complexity and maybe someday we'll do an episode about that next time oh we'll see i think i think at the very least we should wait until april 1st to do

that to do that episode um so but the definition of complexity in this paper is kind of interesting because sort of like essence versus accident in software it's not quite clear to me exactly what he means, but the definition that he gives is that, and I'll read a quote here, software entities are more complex for their size than perhaps any other human construct because no two parts are alike, at least above the statement level. If they are, we make the two similar parts into one, a subroutine, open or closed. And that's an interesting thing to think about, that there's like the size of a program could be a lot bigger if you had tons of duplication, but because we're so concerned with removing

duplication and promoting reuse and making reusable abstractions, that we tend to shrink the size of our software while preserving the complexity. And that the complexity can be measured by things like the different numbers of states that a program can be in, or like maybe the cyclomatic complexity, right? How many paths through the program can you take if you look at all the branching statements and calls into subroutines and that sort of thing so there's this interesting idea that i i kind of enjoyed here that like that software is is large but it's this large condensate of complexity that we've shrunk software down in size while preserving some absolute amount of complexity. And that, to me, is just something interesting to mentally picture. I don't know how useful it is in thinking about, you know, hey, will this technique allow us to attack complexity?

Because that's what he's saying here, right? We've been poisoned by the out-of-the-tarpet paper. Oh, shit, I'm getting into it now we've been poisoned by the out of the tar pit paper into thinking that essential complexity is unavoidable and irreducible um but the essential stuff that brooks is talking about is reducible and is avoidable and he's saying the silver bullet isn't one that attacks just the accidental aspects, but it has to attack both the essential and the accidental if it's to give us a 10x improvement if the split is anything less than 90-10. So he is saying it is possible to reduce essential complexity here. He's not stating that directly, but that's the implication. And so some of these ways of thinking about complexity as being things that could be reduced, don't, as we're talking about complexity, and as we're talking about these other things that are essential, don't think that

essential means unavoidable. Don't think that essential means irreducible. That's tar pit poisoning. That's not what Brooks is presenting here. So that's the start of the section about complexity. I have a couple of other things, but i wanted to give you a chance to jump in jimmy if you've got anything yeah it is it is kind of hard to make sense of what he means by a complexity here being essential because like it's just like the algorithm question right does? Does he mean this exact amount? And I think no, he doesn't mean this exact amount of complexity is essential. I think he just means software will always be complex, right? And he even says here, like,

hence descriptions of a software entity that abstract away its complexity often abstract away its essence. And I think that's right. I think that there's no getting around the fact that software has many states, which is what he kind of equates with complexity, that there's lots of different ways for things to combine together, etc. But I think he jumps from that to, like, and complexity necessarily leads to all of these these cost

problems uh and time constraints and etc that feels like a a weird leap it feels unfounded he doesn't justify it very well i don't think yeah i'll read here uh what he says he says many of the classical problems of developing software products derive from this essential complexity and it's nonlinear increased with size. Oh, you've also got the paper that's full of OCR issues that I was reading. It's nonlinear increased with size. I think it's supposed to be increase with size. That's how I read it at least. Yeah, yeah. It was, I was like, what? This doesn't make, yeah, there are a bunch of typos here. And I highlighted them. I highlighted them in blue. And so as we read, I have decided we're going to include the OCR errors.

Maybe call them out, maybe not. But if you hear us read something and it sounds like that was bad grammar, blame the OCR that produced this password-protected PDF that wound up on Brett Victor's website that I cracked the password for so that I could add my notes to it without having to know what the password was. Blame the complexity of software.

From the complexity comes the difficulty of communications among team members, which leads to product flaws, cost overruns, schedule delays. That feels like a big leap. Yes, it does. If we're

trying to be essence here, why are there even schedules, right? There's nothing like schedule delays that assumes there's a schedule. Like it's just very confusing to me, this kind of, why is there a team, right? Like communication among team members,

why isn't it one person doing it, right? And it's because he wants to operate at this big scale. And I just think that that's really important to have in mind. And so what he's really important to have in mind. And so what he's really asking here about the complexity is not like, is complexity essential in the little artifact? But given the amount of complexity we know software has to have, you know, roughly,

can we have a human organization that can deal with it effectively? Right? And so it is kind of this, like when he talks about problems that will solve the complexity issue, he doesn't mean, can you reduce complexity? He means, can you reduce complexity enough that it solves these human level factors as well?

And that, for me, that just seems obviously no, but also I realize so many people pretend that they can do that right they do have their thing that's going to solve all of your problems even management techniques i mean i think agile here right you just adopt agile your software will be less complex because you'll be building it in this certain way and you'll have rituals for how you do these things and cross-team communication will be fine because you'll be building it in this certain way. And you'll have rituals for how you do these things. And cross-team communication will be fine because you properly divide up things

and blah, blah, blah, blah, blah. And I mean, we've seen how that doesn't work. You do extreme programming. You do two programmers, one keyboard. That's the way to solve this complexity that is inherent in software. It is the essential part of software. And like, I take this whole paragraph that Jimmy's quoting from, like, it gets, for my

money, it gets worse. From the complexity of the functions comes the difficulty of invoking those functions, which makes programs hard to use. From complexity of structure comes the difficulty of extending programs to new functions without creating side effects. Like these are really specific concerns. These are really specific worries about particular kinds of complexity. From the complexity comes the difficulty of enumerating, much less understanding all the possible states of the program. And from that comes the unreliability. Like I'm with you,

complexity as measured by the number of states leads to the difficulty of enumerating and understanding those states. But I don't think it directly follows that from that comes the unreliability. That feels like a leap where it's like, yeah, you can get unreliability of that, but it doesn't follow

in a like an if and only if sort of way so the one thing that he does say that i think is right he says it in a weird way so i'm not going to quote it but that it brings security trap doors yeah yeah that one and that one i do think is very valid i think you know the complexity in terms of number of states almost does necessarily bring the ability to have weird security issues right yeah yeah and that like that's that's if not true in in a like you know galaxy brain what is software kind of sense it's definitely true in an industrial programming we are ibm in the 1980s you know shipping dos kind of sense i mean and just think about like you know the speculative execution that oh yeah these these cpus will do and how that opens up a crazy number of states

and just because it added this new way you can now detect it right like to me it does seem like as we get more and more complex the surface area of those security vulnerabilities just increases and there's always going to be ways to to fit yourself into that fabric that wouldn't have been there if you had a simpler system so i think that's one that he does get right here yeah so this this business of complexity being essential is debatable at best um but that idea that complexity is like as as we get better and better at building software and and saying more with less there is a certain dynamic where the complexity is preserved as the size of the software shrinks

and that makes software a very interesting kind of material or a very interesting kind of you know as a thing that we build it has a curious kind of quality that isn't the same as other things that humans build like buildings automobiles whatever where the amount of complexity does not so it varies more directly with the change of scale like the bigger you make a bridge the more complex that bridge generally needs to be where in software that's not quite the case yeah i think this next thing so we move from complexity to conformity and i think this actually helped me understand the complexity better because it's very clear here what he doesn't have in mind is the with complexity is like the essential complexity

of the tarpet paper yeah um but the like practically minded complexity right because what he mentions here about conformity could go away if we're talking about like the high-minded essence right so i'll just i'll he says you know conformity really he means like you have to conform to something your software has to conform to some other software yes and he wants to say that that's a bad state of affairs it's going to be complicated for this reason. I do love this little quote. So he talks about, he's kind of contrasting physics and software. And how physics can

deal with complexity in a way that software can't. And he says, here's the justification. Einstein repeatedly argued that there must be simplified explanations of nature because God is not capricious or arbitrary. No such faith comforts the software engineer oh how that that to me that was the best that was the best little quote here um i mean so

what he wants to say is like all the complexity we have to deal with is like human-made complexity it's going to be arbitrary it's going to be annoying it's going to be arbitrary. It's going to be annoying. It's going to feel wrong. And it doesn't matter anyways. If we want to have a business, if we want to have a productive product that we can sell on the market, we're going to have to conform.

And the sort of the definitional quote for like what exactly does conformity mean is this one to me. Much of the complexity that a software engineer must master is arbitrary complexity forced without rhyme or reason by the many human institutions and systems to which their interfaces must conform. So you have to write your software

on top of some existing operating system or some existing browser, or you have to use some existing module ecosystem, or you have to write your extension to this existing line of business application. You are standing on the shoulders of giants, and those shoulders are very bumpy and unstable. And so you have to do what you can to balance on top of this trembling giant and not fall off uh anybody who's played shadow of the colossus there's your there's your mental image for what it's like to be a programmer in the 21st century um so yeah so conforming is is the dealing with other people it's the hell is other people of being a software engineer.

And this is where I get confused even more about his talk of these are the essential things. But he says here that they're not essential, right? And the way you can see that is the word essence and the word necessary are very interconnected in philosophical terms i'm not gonna be too nerdy on that but he says these things are not because of necessity but only because they were designed by different people rather than by god yeah okay if god made the interfaces i mean mean, I don't know. Yeah. Would they be good? Biology is kind of a mess, right?

Yeah. But yeah, so I think that this is really weird because he wants to say this is essential, but he even admits that it's not. It's just an accident of history. It's not necessary. We could do away with this. And in fact, there's even ideas about technology the communicating with aliens problem right that could solve this problem that

you don't have to conform because your software itself could inspect and investigate and figure out how to establish communication in a way where you don't have to do this you need to use protocol buffers open shot done easy got it no problem uh but you get my point right like this feels this feels very counter to what he's talking about being essence when he brings something that he very explicitly thinks is accident he just thinks it's we can't fix it yeah he thinks it's beyond repair yeah i'm gonna save editor ivan some work and not force him to interject here to remind you the listener that we are doing a little bit of playing with brooks's frame right brooks's frame is are these things essential within the next 10 years, between 1986 and 1996, at the industrial scale of computing stuff. Between 1986 and 1996 at industrial scale computing, yes, there is no avoiding the technical debt that working software engineers are going to have to stand on top of.

It is an essential quality of what makes software at that scale at that time no it is not an essential quality of software if we get to be you know get to have our fun and play in the space and say oh but what about my own little visual programming editor that i'm writing for myself in my own little walled garden i don't have to stand on top of anybody's shoulders um that's cartex whole thing right it's like how do i like uh moo how do i get away from having to have any you know dependency on the existing tech stack i just want to do everything directly on top x86 and and not even that if possible yeah that's like it maybe you could argue it's essential within his framing. But if you take his framing away, then no. Yeah.

Conformity is not not an essential quality. Yeah. So that I think, you know, this I agree with you. This is his framing and we can see what things he's considering essential. And they're definitely business practical things because the reality is, yes, you're not going to get rid of this. And and I think if you ignore that fact, you can believe that you can make achievements in productivity that you just can't. So I do think he's right here if we take this practical essential.

Well, it's funny you say that because here's where I've got my first instance of a, is he wrong was there a single thing between 1986 and 1996 that produced a 10x improvement in one of these dimensions that he cares about like cost like reliability like whatever whatever i've got a couple of things here that might count and jimmy i'm gonna get you to give a uh oh yeah that did actually do it and oh no no that didn't do it or a yeah i don't know i just want to predict one all right go for it http oh very close yes okay okay yep very close uh so uh actually split http in half tcp yes tcp i don't know if it predates 1986 but i think it had a heyday in the early 90s um and i think that it may may have been responsible for a dramatic change in the way that software is made such that it would reduce costs or increase productivity or allow people to do more with less um is it a good bad movie a bad bad movie or a movie you kind of liked

uh i mean i just i don't want to be all meta but the it let you build more complex software oh no oh no it made it worse right it let you build more complex software did it reduce the cost no because now the complex software you're building you're building more complex software yeah and all sorts of new companies started building software and spending money building software that they weren't spending before meaning the amount of cost spent building software went way up because there was so many more pieces of software being built was their software see this is the problem with evaluating this are we asking the question did it reduce in costs did it get more do we get more productive building the new kinds of software we're now building,

given our new technologies? Or are you asking the counterfactual? If we had had TCP back in this day, could we have built this software project in a 10x faster way? And I think the answer to that one is definitely yes. There were projects that had to recreate their own network protocols that couldn't rely on blah blah blah blah that could have been made in a 10x faster way had they just had tcp already implemented and done for them etc but that's that's it's which question

we're asking yeah i think brooks really wants to ask the not the counterfactual one of what could have happened but are we actually now at a point where we build software that's 10x cheaper because this is a practical question based on people being frustrated from budget overruns etc yeah like like he's he's trying to equip middle managers at you know software uh companies around the world with some arguments for when somebody comes to their business saying hey you know use tcp instead of building your own uh retry logic on top of udp or whatever it was at the time i was like one year old so um i don't know i wasn't programming much at the time yeah um i think that's one of the the like competitors i think that was a protocol

i have no idea i was not born v92 k flex uh d base if i start naming something maybe you know a stopped clock comes up with the right networking protocol twice a year. Yeah, so like if somebody had come to one of these companies and said, hey, use TCP instead of whatever you're doing instead, would that have led them to a 10x improvement? Well, maybe, maybe not. Would it have led to a 2x improvement? Probably. 10x improvement well maybe maybe not would it have led to a 2x improvement probably um so that that one that one jumped out at me here is like conformity tcp solved some of the conformity difficulty i think um because at the very least it was standard and it took care of

headaches that people would have had http is is another great one, but I think by 96, it might have been a bit too early for HTTP to really have had its heyday. Other ones that I had in this to consider are REST, but then that didn't start until the year 2000, so framing is wrong. And then SQL, which I think came

from the 70s, and gradually... Was SQL from the 70s? SQL was from the 70s, right? I's was sql from the 70s sql is from the 70s right i don't think i don't know when sql itself started relational databases are i think god in 70s yeah so that was the other thought that i had that like like sql in particular but i guess the relation was 70s 74 cool nice good um that may have been a if you can count that as a single thing in brooks's definition of a single you know innovation or a single development um that may have helped a certain software teams achieve a 10x improvement um on the development of things that have to do like crud style stuff

so and then and then i will we'll have more of these later when we get to other aspects i've got other like hey what about this what about this um but the next one are you ready to go to the next one yeah yeah i was gonna say i think it's i think this one's the best uh the best in what sense? I think he's right. Changeability. So this is changeability. Okay, so the next one's the best.

Yeah. Yeah, yeah. I mean, I think if we're keeping this practical frame, it is changeability is the essence of software. That's the whole point of why people want to codify something in software is it's much easier to change, relatively speaking, than a physical process or whatever. You can reshape it, recast it, keep up with the market, etc.

That's why we're doing this. And it is just really hard. And it really does contribute to so much of our time, right? Is keeping software changeable, making changes to it. I really think he gets this one right. Even if the words and exact ways he says it aren't the way I would say it, I think he's right about this property.

And so much so that I have nothing highlighted from this section, because reading through it, I was just like, yeah, checks out. Yep. I just have the first sentence. That's it. Yeah. It's just like, yep, we think it's more easily changed than it really is.

But also it is easier to change than almost everything. Yeah. Right. There's no waste that you get from it that you have to like now dispose of or whatever. Like that's why I, I'm fine with like fixing physical objects as long as I don't have to put a hole in something or like, you know, cut something.

Cause like trying to repair that is such a chore, uh, with software. I just undo. It's fine. Yup. Yup. Yup.

Yup. And then, uh, here is where Ivan will go on a 30-minute rant. No. So if this episode feels 30 minutes longer than normal, which will be impressive. Longer than it should be.

30 minutes longer than it ought to be. Here is why. Yeah. So the next and fourth and final quality that Brooks suggests is essential to software that comprises the essence of software, at least as far as he cares about it, is invisibility. And I'll quote, software is invisible and unvisualizable. And I have that sentence highlighted in pink because I disagree with it. But then he – so this is i'm gonna

give a bit of backstory this is the section of this paper that on my first read-through i didn't really care about on my second read-through i hated um i hated this idea that you know software is invisible and unvisualizable like that you know i was big into my visualize everything, visual programming, yabba-dabba-dabba-dabba. I was in that phase, which I'm not in anymore, surprise. And then on my third read-through for today's podcast, I actually enjoyed this section, and I've highlighted pretty much all of it, and pretty much all of it is in green, except for that first sentence, software is invisible and unvisualizable. And the whole rest of the paragraph I have highlighted in green, and I'll get into that in a second. But the reason why, before I get into the what, the reason why it is highlighted in green and why I actually have come around and quite like this section is because I took it with the theory building frame. So

reading through this with the theory building idea that uh you know i brought up earlier this actually resonated with me because the theory of a program is invisible and unvisualizable that's what is a theory a theory is not expressible you can create an expression of a theory but the theory itself is something that is just thought stuff that is not you know it's in your head it's not a thing that you can quantify or or represent itself and so of course jimmy at the start of this episode when i brought up that idea said actually no this essential accidental stuff is is orthogonal to the theory building view so now i think i'm probably actually going to have to flip around and say you know this whole section

this is this is bad no no i think you got this section right well you yeah so do you think in this section what he's talking about is the theory uh so i i don't think it has to only include the theory but that's not required if the theory is a part of this and the theory itself is invisible then the whole thing can't be completely visible right yeah which is what he means by invisible he doesn't mean an unvisualizable he means the totality of it couldn't be visualized because there's all sorts of different ways you could visualize it not that you couldn't visualize some part of it right so i think for him the theory here if we took the theory that is a part of what he's talking about and that part is invisible and unvisualizable yeah so i think you're absolutely right here

i want to so i highlighted this whole paragraph so i'm going to read this paragraph and i'm going to go bit by bit and we'll dig into it but i actually want to read all of this because this is there's there's a lot of stuff i want to jump off of from this so software is invisible and unvisualizable that um no sorry it's just my visual programming instincts are kicking in um but yes, in theory, building view, sure, maybe I'll grant that. Geometric abstractions are powerful tools. No argument. The floor plan of a building

helps both architect and client evaluate spaces, traffic flows, and views. Okay, so yeah, a geometric representation of something that you're physically dealing with, like a building, it's a powerful tool when you're working with that floor plan and i'll quote contradictions become obvious omissions can be caught scale drawings of mechanical parts and stick figure models of molecules although abstractions serve the same purpose a geometric reality is captured in a geometric abstraction. So he's setting up the idea that engineering disciplines like architecture and, you know, mechanical engineering and those sorts of things love to create geometric abstractions, some kind of representation of the thing that you're going to make.

That is not the thing itself. It's some working model. And you can make that as a physical thing in space. And then he goes on and this is where my coloring flips from green back to pink. The reality of software is not inherently embedded in space.

I'm going to come back to that in a sec. Hence, it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics. Debatable. As soon as we attempt to diagram software structure, we find it to constitute not one, but several general directed graphs superimposed upon one another. Yeah, that's true. Yeah, there's several different general directed graphs you can use to diagram software.

That's not debatable. The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, namespace relationships. Yeah, those are all different maps that you can make of software. You can visualize all that stuff. That's all graphicalizable. These are usually not even plainer, much less hierarchical. Indeed, one of the ways of establishing conceptual control over such structure is to enforce link cutting until one or more of the graphs becomes hierarchical. Yeah, yeah, who cares? In spite of progress in restricting and simplifying the structures of software, they remain inherently unvisualizable, thus depriving the mind of some of its most powerful conceptual tools. This lack not only impedes the process of design within one mind, it severely hinders communication among minds.

All right, that's the end of the paragraph. To start pulling that apart, summary is basically, we can make visual models of things in some fields, but not in software. Why not in software? Well, because when you try to make visualizations of software, when you try to make models, what model do you make? Do you model the flow control, the flow of data, the patterns of dependency, time sequence? You could make so many different models of software and visualize it so many different ways. And those

models are complex. They're not even planar. You can't put it down on paper you'd need like 3d space or or some higher dimensional way of arranging these models and so that means it's inherently unvisualizable you can't make visualizations of software because there's like multiple different visualizations you can make of software what the hell um but then it ends with a banger this lack not only impedes the process of design within one mind it severely hinders communication among minds theory building view right the if you want to read this as the theory that the people who make the software have embodied within them and think about it as what is within the mind versus what is coming out then yeah like the fact that software is so hard to represent in other forms other than either in the mind or in

the working runnable model the fact that it's so hard to come up with like you know a popsicle stick and marble or like you know pipe cleaner and and clothespin model of software like the fact that it's so hard to make some other representation of it sure does hinder communication um like whiteboarding is you know it's one of those great things but it's kind of the best we've got for communicating what software is what it does what it should be how it works that sort of thing but like right in the middle here he lists off like here's a bunch of different things that we can visualize and it's it's not a very complete list but in it are the makings of so many different

kinds of visualizations i just like i don't know what to make of this paragraph i love it and i hate it at the same time it's so good if you look at it in some ways and it's so bad if you look at it in some other ways it's almost as though like you know this paragraph is is not written and inherently unwritable like so in the same way that conformity wasn't really the high-minded essence invisibility here i feel like is not the high-minded essence either right this is a practical essence and the argument here is clearly bad let's just like put that out there like it's like oh land has maps yeah what kind of maps what things are you showing on that map is it a topological map is it a population map is it a temperature like you know that would

be like and yes you know you could overlay data on the same map right but we have all these different projections on how we do things then that always leads out some detail that you would care about it's the same argument for programs right so like the argument here is clearly bad but i think the idea here is solid right programs are not inherently embedded in space i i strongly disagree with that claim i meant to come back to that but anyway sorry yes yeah okay i i think i think that's i think that's obvious that they aren't. I think it's obvious they are. Damn it, no. Because the same program

can have multiple different representations that are, and we would still call them the identical same program, okay? Whereas the chair I have here here you can't have multiple different representations of that identical chair why not you could have a schematic diagram you could have a photograph of it you can like that is into the chair so this is a map territory kind of thing right like well i'm saying no that's that's my point the chair exists in space uh-huh and the program

does not why not it exists as bits on your in your ram and if i copy it to another location if i copy those bits to another location is it the same program and the answer is yes it's identical despite having two different spatial locations where if i have two chairs with two different spatial locations they're not identical but this is the like if i dip my hand into the river and take it out and dip it in again is it the same river right like if i copied the bits of the program from my machine to your machine is it the same program no there are two copies of a program same program well define same that's the thing define same does same mean these bits on this machine, or do they mean this pattern of bits on any machine? And I think the more you go towards this pattern of bits on any machine, the more you invite the interpretation of software as being a thing that the spatial character of it is something that you can separate from any particular instance of that program yeah so i think you're you're being a little more high-minded than i think we need to

be here and yet less high-minded than you need to be to get the distinction so all right okay so if i take a program right we have it on one machine it's a bunch of text files let's keep everything simple sure right but they're ascii encoded and then i re-encode them latin one utf-8 whatever right i re-encode the same files and i ask you and we'll assume that the runtime this language it doesn't matter it doesn't do different things right it would be interpreted the same way i I ask you, did I give you a new program or do we still have the same one? You would say, that's the same program. Even though the pattern of bits has changed. Or let's say this, I keep the program,

I make no changes to it, and my hard drive does defragmentation and moves where it is on the hard drive. You would say that's now a new program? I would only care to distinguish whether the program is the same or not insofar as it enables me to win the argument about whether software is visualizable or not.

I'm really goal-oriented about this. Okay, so I think the inherently embedded in space or not is not a good argument for unvisualizable okay i i think it's a bad argument against software being visualizable yeah yeah i'm just saying i think he's right that it's not embedded in space we can agree to disagree on that and and we could we could argue interpretations aha we would end up agreeing i think ultimately uh i i do i do think you don't hold it such a weird identity uh criteria it depends how how much of a contrarian i'm feeling

like but but deep down in your soul i will grant you that you have more powerful philosophical weapons than i do so you would probably win the debate because you know the names for things like the story of when you dip your hand in the river two times, is it the same river? You know the name of that thing. I don't know the name of that thing, and so I would have to bow to your expertise. I mean, that's a Parmenidean idea.

There we go. Yes, Parmenides, I'm pretty sure is the one there. It's one of the Presocratics. Totally not editor Jimmy here knows that this was actually Heraclitus and not Parmenides. Parmenides believed that the world never

changed, and Heraclitus believed that the world was in a state of constant flux. Hence, you can't step into the same river twice. Have to admit, pretty cool to put in one of these meta-statements here for the first time. So, software is not embedded in space because

you can have, the identity is not about where it's located spatially, it's about the program, right? And if I change the text, I could still have the same program. I think there's a notion of program identity that we could get behind, right? And it doesn't take account of space. But the whole inherently not visualizable, I think, is totally beside the point on the space bit. I actually think that a lot of why he wants to say it's not visualizable is about this,

well, I think why he should say, is about this time element to it, right? This chair, it does persist through time time and it might make changes over time but i can make a diagram of it that's just it in space ignoring time and software we always have this element of time and so we have to map space and time to some dimension this is why he says like we have to have you know a higher dimensional thing or whatever unvisualizable here i just think he's using it in a very simple way of practically speaking the software we want to build is so complex that the kinds of visualizations we're capable of making are just underpowered for what we want to do and one answer for for that, that I think is kind of obvious

in his quotes here is just to change the way we make software. So he talks about like, you know, you could make a make a thing talking about namespace relationships, like, that is clearly not s essential, right? We don't need namespace relationships. So I think that this is really him being practically minded, saying in the next decade, you're not going to see people visualize large software systems comprehensively. These things are not going to really change in the next decade. I think he's kind of right about that. Just wrong about the bigger picture,

unless his argument could also be used for land and maps, which he doesn't think it can. Right. So this is why at the beginning of our discussion, I brought up that like, he's sort of inconsistent throughout this paper. Some sections he's good at maintaining that framing that he sets up for us. The 10 years, industrial scale, factor of 10, single development.

Sometimes he's really good at expressing his arguments in terms of those, that intersection of framing devices. And sometimes he makes claims about things that transcend those framing devices. Like he says, inherently unvisualizable and and i think like that first sentence software is invisible and unvisualizable is so definitive and so and then like every sentence of this paragraph or at least the first two sentences or the first two sentence the first sentences of the first two paragraphs, God, it'd be so much easier if this was spatial. Um, and I didn't have to like,

you know, write a indexing system to stuff. Uh, the reality of software is not inherently embedded in space. Like these things feel like universals that he believes independently of these framing devices. That's what sets me off so much about this paragraph is that it does it it feels like he's not just speaking within the bounds of the framing devices that he set up but that he wants to say you know and and and in the uh in either later sections of

this or in the refired i can't, he does specifically dunk on visual programming as a form of snake oil that people are peddling in the mid-80s that would be a silver bullet. And he's setting this up as a like, no, it's not just that your particular visual programming ideas are not going to cut it he's saying like visual programming as a thing is a bad idea and here's why in a way that's not bounded to 10 years not bounded to 10x but just like if you think visual programming is going to be a thing you're out to lunch like i really think he's trying to make a much stronger claim than just the framing devices would permit. And I'm basing that not just on this paragraph. I'm basing that on other things that he said elsewhere, too.

Yeah, but in some ways, he predicts dynamic land. Yeah, right? Like, that's the thing. I think this is cutting off his nose to spite his face. I think that he's wrong about software being invisible and unvisualizable, unless you caveat it a whole bunch. Yes. I think he's wrong about software not being embedded in space,

but I'm saying Jimmy would win that argument because reasons. So this is what I actually kind of, this is why I think I had a similar history of reading this paper as you did right but i started with tar pit and learned about this paper after reading tar pit yeah same yeah okay um uh and this is out of the tar bit by like ben mosley and somebody just because we've mentioned it a few times we'll do it next time um ivan does not want to do it next time we'll see i i i don't know i i i loved that paper when i read it i have changed in my feelings and this is kind of the opposite like i think if we keep his implicit frame in mind about practicality right um i I think this is really getting at some things that instead of thinking of them as essence here,

he really just means these are the things that are too hard to tackle. These are the things that are going to take longer than a decade to really have a research program that really makes this practical and beneficial for us to get rid of. If we take that here and we don't take his language so strongly because he does make a lot of strong language and he does that in contradictory ways

right he says conformity is essential but it's also not necessary which like there's such a thing as accidental necessity which is what he means here but like whatever uh sorry i've been trying to resist that term but i had to say it uh so accidental necessity really easy the past is not necessary but given the present it is accidentally necessary right we can't now go change it is this meant to get around the um the opposition to the idea of essential and accident that comes from needing things to exist before you can talk about them having essential and accidental properties?

Is existence an accidental necessity? Wow, see, you're a much better philosopher than you think. I read plato.stanford.edu. This is a, I mean, so this is a big debate on like, does anything have existence as its essence right and that's that's the claim about god paragraph one when you google essence versus

accident go do it go read yeah god has uh an existence as his essence was the argument right and only god has existence as his essence yes Yes, and Kant comes along and says that existence is not a property, so they can't have it, essentially. But then we're getting into how do properties work. Anyways, I'm not going to get into all of that. Though you would love to. But yes, there's some things about that.

But accidental necessity is really more about uh free will okay and it's it's like weird you know people will say well i can't change the past so how could i have free will because in order to have free will i'd have to have decided differently and if we look at any moment i can't decide differently at that point because the past deter you know etc so that's accidental necessity and that's really what he means. That's how I took conformity. It's not essential in a high-minded way. It's accidentally necessary.

Given where we are in history, given how computers are developed, given the market, there's no changing it now. That's kind of the idea. So I think we should move on past invisibility though and we'll probably come back to it because and do an attack action so that our invisibility is gone ah boo you can't firewall cloaked jimmy you as the world's biggest star trek fan should know

you can't firewall cloaked yeah sorry i was making a pathfinder dnd reference uh all right so we have past breakthroughs solved accidental difficulties and i'll list them here and if we want to talk about them we can but i think he's kind of right here uh that there were a bunch of accidental things that we didn't need to be doing that weren't the essence and there were things that helped us solve them so first is high level languages yeah time sharing yeah unified programming environments these are all things that didn't necessarily attack the essence they focused on all the accidental properties all the things we shouldn't have to do and give us a better way of dealing with them. Now he makes claims in here about how much they got rid of and how much they can continue to get rid of. I'm bracketing off that

for a second. Just the general claim. High level languages are great, but they're not really changing the fundamentals of what we have to do. We still have to do those things. We just have a better way of doing them. Time sharing uh you know uh he doesn't even have in mind here i think he means like actual like time sharing on a mainframe yes not like you know programs switching on your own personal computer because he mentions later like personal computers workstations i think he calls this is 86 right so we're really early in the pc revolution at this point yeah he's definitely talking about time sharing on Mayframes. Which makes perfect sense. Being able to have time sharing and you don't have one dedicated

computer, that was definitely not an essential thing. It's nice to have now and it definitely makes you more productive. And then unified programming environments was a little weird for me. I don't know exactly what they mean in 86, but it seems like your compiler and standard

library all come together and uh oh yeah he mentions unix and interlisp so that sort of thing that makes sense yeah so that thing i it's just weird for me to call unified programming environments unix and interlisp like those feel like the opposites of integrated to me but i get it they are enemies but they are addressing the same issue i'll read a quick quote here they attack the accidental difficulties of using programs together by providing integrated libraries unified file formats and piles and filters i'm pretty sure piles is an ocr files and filters maybe files and folders i don don't know what it is. Yeah, I wonder if files and folders? Yeah, piles and filters.

Piles and filters. So Unix and Interlisp as pile-oriented programming environments did help. I love that idea. Pile-oriented, like instead of folders and files, we just have piles of garbage. I love it. That's my way of programming but i think it's the idea of like having a nice like dedicated editor with facilities to assist you in the act of programming that are like in software right like um like for instance having like an ide right like he says this breakthrough in turn stimulated the development of whole tool benches

since each new tool could be applied to any programs by using the standard formats so it's like instead of purpose building your your software directly against the machine you now have other software that exists that helps you build your new software and that other software its whole purpose is to help you with the act of building software that is a a thing that we had to invent at one point we had to come up with with that and so now that we have that that has helped us solve accidental difficulties which i can't remember if he says it in this paper or if it's in one of the other things I read, but he did say that like at the advent of programming as we know it, accidental aspects of programming dominated the essential vastly. And so we did see 10x

improvements several times over. And it's by the 1980s, the-1980s that he feels like the rate of progress is slowing and so that's why he's worried now is because taking the long view in the past we had rapid improvement now we have improvement slowing in the future it looks like it's going to slow even more and in fact that's what he did think in in years since he did think that progress did slow even further and that despite all that moore's law held and that gets us back to earlier this is you know a software uh engineer feeling envy of hardware engineers and their you know clockwork progress in in doubling every 18 months yeah not anymore though they're in the same boat as us well it depends i'm pretty sure mo Moore's Law holds if you

just count transistor density. Not clock speeds, not MIPS. Brian Cantrell has a great talk on Moore's Law here where he goes through the different meanings you could have, and I don't know, I buy his analysis that it was not just that. It was also

about cost. It was an economic thing, you know, et cetera. Yeah, as long as you keep moving the goal posts, you can maintain Morse law forever. Exactly, right? Like, it's like, it feels a little... Yeah, I would definitely recommend that talk. It's really good

at breaking down kind of the history there. It'll be in the show notes. Message to future Jimmy and Ivan, make sure we put it in the show notes. I mean, Brian Kendrell talks are pretty easy to find. But No More to Give, I feel like is the title. Something like that. I don't know.

Cool. Some pun. Anyways. So yeah. So we got these three here. And he kind of just gives us a general lay of the land here. And then kind of talks about how well they they're gonna do in the future what are the hopes for the future or do you not see it

as that way i so what i what i would say is we have a section where we've got these three examples of things in the past that have been like silver bullets high level languages time sharing and unified programming environments were silver bullets in his view that's the implication next he's going to list a bunch of things that seem like or some people would argue that they are silver bullets that have yet to be fired or are in the process of flying out of the barrel of the gun in super slow motion or whatever you want to use for this metaphor um functional programming he he says you know here are some things that are currently being suggested as likely to cause a 10x improvement let's look at them and see whether or not that seems true let's look at these and see are these silver bullets yes or no so the first is

ada and other high level language advancements which just like already i mean it being ada as the example does kind of uh you know feel funny because yeah it definitely was not the thing that like ushered in a silver bullet he was right about that one uh yeah we have the benefit of history here this is our ability to jump in and out of framing is going to make this this part of the discussion very entertaining um also kind of um speaking of moving the goalposts right one of the framing devices is there is no single development so in the previous section he says high level languages and then compares that against ada and other high level language advances this feels unfair this feels like high level languages

were a you know a silver bullet but incremental improvements to existing high-level languages nah that's not gonna it's like those are two different scales of thing saying yeah high-level language advances nah is like not fair to compare that against high-level languages as an entire concept and all the you know the the benefit of specific high level languages that we had up until this point it just feels like like a not a category error but like an error of scale so that bugs me a little bit but yeah so let's let's get into what he has to say about ada and other high level language advances i think i'll say this I think the reason he's doing it is because he's considering high-level languages of the same

level that Ada, that they're currently at, and he thinks Ada is of that same level. It just might be better. And he later considers things that you could call higher-level languages, like object-oriented programming and expert systems.

That might be these like next generation or automatic programming so i think he here he is using high level language in a like a more specialized way whereas we just mean like the whole hierarchy of them oh but that's that's what i mean is the unfairness that's this is a that's fine okay um so i mean he's yeah i think maybe it's a little unfair, but he just says, like, you know, it's just another high-level language, and the biggest benefit came from a transition from low-level to high-level, and we're not going to see a big, huge payoff

from just adding better high-level language constructs. And I think he's right. Yeah, and that's why I think it's unfair, is because the previous section, he says the transition from low-level to high-level language constructs. And I think he's right. Yeah, and that's why I think it's unfair, is because the previous section, he says, the transition from low-level to high-level is a silver bullet, and it's like... But there are people who still believe just that next increment on high-level

makes such a huge difference. I think we can all agree those people are wrong. I mean, I can see when you're... If we look at this not at the industrial scale, we look at it personally, and we try to abstract out, I am 10x faster at coding in Clojure than I am in Java. I can get things done,

the same problem done, 10x faster. And if you try to extrapolate from that, and I spent a good chunk of time in both of those languages, just to be clear, but I consider them at different levels of abstraction here. So it's not familiarity. It's just the expressiveness I have in both. And maybe Java 8 made it better, blah, blah, blah. Who cares?

None of the details here. But you could try to extrapolate out from that to the industrial scale and say, if I'm 10x more productive, given these two levels of languages, why wouldn't the whole system, if everyone did this, be 10x more productive? And I think that's actually where it fails, right? Is this, it doesn't matter your individual productivity. Not everyone's going to be like that but also even if everyone were like that that wouldn't result in the whole system being 10x

better yeah yeah i've worked at closure shops they were not 10x more productive than than equivalent java shops yes yes yeah and and just to drill into why a little bit do you think that's because when you're on a team and thinking about it at a larger industrial scale, the cost of development, not financial cost, but just costs broadly, are dominated by factors that aren't attacked by Clojure versus Java? Yeah, I mean, to show my hand a little bit here, I think he's very wrong about at least the implicit assumption about how much time is spent in accidental complexity. I think it's most of it, but most of that accidental complexity is not coding. It's management processes, it's human factors,

it's not about the essence of the problem and it's all about these other factors. And that's why you'll see a single programmer who can go off and build an entire language way faster than this company did or whatever. It's because you got rid of all those factors. And I think he suggests some of this in mythical man month which we considered doing today but didn't get into because this yeah

which is good because we already have a lot to say here yeah yeah so the the last thing about this uh ada and other high-level language advances the the last sentence of that section i just want to read it and say something about it ada's greatest contribution will be that switching to it occasioned training programmers in modern software design techniques. And I like that because I don't personally know anything about Ada. I mean, I'm familiar with it as a historical, you know, it's a name that comes up when you look at Wikipedia programming languages. You know, they all say influenced by Ada. So I'm sure it did some cool shit um but i love the idea that he is allowing for some of these advances that he's considering to not have the benefits that they purport to have but to have

benefits that are kind of consequential so like one thing that that made me think of is, you know, closure and Haskell in the last decade and a half have had a heyday. And I think that their impact has been positive, but I don't think it's been positive because more people are using closure and Haskell. I think it's been positive because more people are aware of like the great things that we can learn from thinking about functional programming and that that way of conceiving

of software whether you use closure haskell functional programming you know any of it just going through that exercise of thinking about software in that different way um makes you a better programmer and i i like that he sort of at least touched on that um and he does it as kind of a diss but whatever i think the sentiment is still good and it it reinforces the fact that he's not just even though this whole section is just talking about like technical aspects of adopting ada and whether you know advances in hardware will allow us the cheap mips to pay for the compiling costs you know he gets really into the into the nitty-gritty on that but at the end he zooms back out and says hey i'm willing to look at this very holistically and see is this thing

gonna make us 10x better for any reason not just because it's a better programming language but because it might help us be better thinkers about software so i liked that i like that yeah so object-oriented programming i don't i don't think there's a lot to say here well uh did it bring about a 10x improvement between 1986 and 1996 uh i was too young to say for sure but my bet would be no i think it brought about 10x improvements in some ways that maybe brooks wouldn't care about for instance the the standardization on java as like the industrial programming language in the early 90s as like the way that if you were at a software shop with 500 employees it would be a java shop java was 95 oh god damn it all right late 90s no but okay so so ignore the ignore the 10-year limit, but I think Java, like Java had a period where it became the lingua franca.

It became the language you would have to learn if you were going to go do programming at an industrial scale. And I think that that enabled a whole bunch of big software projects to be written that may otherwise not have been written or not written in the same way. Like I feel like it's sort of like C++, right yeah i could say the same about c++ c++ enabled so much software to be written that if you took c++ away um sure maybe something else would have filled the same vacuum but we needed something to fill a vacuum there and i think like o o did that could have been fp right it could have been you know probably not prologue or logic programming or concatenative programming but we need something to facilitate the industrial revolution of the early internet era and i think o o was that so really though you would say that the thing the

singular thing would have been a management technique of standardization yeah sure yep and and it happened and and standardization on a particular programming practice facilitated by particular languages but really a particular programming language well yes a practice facilitated by the language so like oh oh i consider a practice i don't consider it like you can say it's a language feature or their classes is their sub classing whatever but but do you think the the conformity would have been just as good in c o o as in java o o and brought about the same i think i think c o o is a is a contradiction of terms but you can do o o and c oh i know but like uh i don't know that anybody was doing o o and c in the 90s i think that's yeah yeah i'm just saying counterfactually you think that it

could have been the same i'm just trying to get what you think is the benefit here of the oo i guess what i mean is like he in 86 is saying there are a lot of people coming out with these things and saying do this and it will enable you to be so much more productive and oo i think got uptake in the way that some of these other things didn't and so whether or not oo led to a specific 10x improvement as he was you know claiming it wouldn't is irrelevant to me i think it did though um it did serve as like the vehicle upon which a whole bunch of early internet stuff was conveyed. Yeah, I wonder one thing he doesn't mention in here that might be something some people would claim kind of related to the Java aspect here

would be garbage collection. Yeah, yeah, totally, yep. Right, and not just like the first order effects of garbage collection, but like the second order of making it easier to have interoperable libraries because you don't have to worry about i mean that's why c doesn't have that sort of like package ecosystem yeah right is because you can't you can't have exact clear instructions on who owns what right and why rust has had such a a big thing despite not having garbage collection it has clarity on on ownership yes right yeah um

so that would be an interesting one uh but yeah i i think i mean oh i'm of lots of opinions on it um you know i think he's right in trying to say that the the benefits people think you're going to get out of it probably won't be an order of magnitude in the next decade in his framing right his narrow framing the industrial complex will not get an order of magnitude more reliable and productive and simple by oh yeah artificial intelligence the one that 10 years ago we would have been like yep he was totally right. Even ignoring his decade limit, and today might be a little fuzzier on. Yeah, like he's definitely wrong timeline-wise, but in terms of is AI as, you know,

not a GI, but just AI, something that will enable, at some point, a 10x improvement within a decade? Maybe. Like, yeah yeah yeah um i loved this sentence from this section i have a hard time seeing how image recognition for example will make any appreciable difference in programming practice dynamic land baby we have we have a an exciting cool interesting programming model that will make an appreciable difference in programming practice, at least for the people who use it and the ways that they use it. That depends on image recognition, which at the time in the mid 80s was what you thought of when you thought of AI, right?

It was image recognition. It was like speech generation. It was like those kind of things where it's just fancy algorithms, right? Because that's the constant critique of AI is it's, oh, once you know how it works, it's not magic. It's just, you know, science quote the hard thing about building software is deciding what to say not saying it no facilitation of expression can give more than marginal gains so he's saying like you know having an ai to do the programming for you won't help make building software easier because you still have to decide what to say what to tell that AI to do. And it's funny because at the time they weren't really doing it, but right now we're like testing that idea. Can you get better at building software

by having an AI that can do all the mechanical work of writing the code and knowing what functions to put where and what to name variables and all that kind of stuff. And all you have to do is say like write me a program that given a json array of genders and ethnicities will decide whether somebody's a good scientist or not that that right maybe you don't know this one jimmy that's a that's a no i saw that i mean the bias in these systems is pretty awful. Yeah. Yeah. So, but the hard thing about building software is deciding what to say, not saying it. I think that example, like I bring that up as a kind of a like shot in the ribs of current AI,

but it is illustrative of the point that like you discover that systematic bias by thinking long and hard about what to say and what to do to kind of probe the AI system into giving you a satisfactory result, like prompt engineering and prompt design and prompt artistry. Like that is a thing now that wasn't a thing that pretty much anybody other than like gray crawford and a couple of other people i know of were doing you know two years ago and now everybody's thinking about that and it's it's very interesting to me that this is something brooks was thinking about and saying and pasha and now it's like well i'm pasha so

i will go ahead and make a prediction here, a 10-year prediction. And if we get our prediction episode, I will stick by this one. If we take the narrow frame of this essay, singular thing, next decade, 10x cost reduction on an industrial scale, I will say AI won't accomplish that oh all right uh and and

the reason i'm saying this is not because i don't think again that individually because i've done some things that were a hundred thousand x faster than i could have done them without chat gpt i translated some code you know between languages that I would have had to, it would have taken me way longer than just telling chat GPT to do it. I just, again, I think it's those human factors that dominate. I think it's the market factors that dominate. It's all of those other things that are really the dominant costs. And the programming stuff here is not really the complicated bit. Well, it might be the complicated bit, but it's not the costing bit. So that's why I bit so that's what i why i'm

saying that now does that mean i don't think there's gonna be anyone who ever does 10x more because of these things definitely not there's gonna be someone running a small business that does ai uses ai for it and makes tons of money doing it no question in my mind it's just is this gonna happen at the the industry? And I say, no, because I think it's managerial things that we have to solve in order to have that effect. Yeah, I would agree with you just because I think a 10x improvement in the amount of software that we can produce,

the amount of quality software we can produce for a given level of budget. I don't know that we need 10 X more software to exist than what we currently have. We already have so much software. And I think the thing we're struggling with is, is that software any good? And I don't know how AI would help us get a 10 X improvement in the quality of

software that we're writing. Yeah. Um, so yeah, I'm definitely with you on the, I'm going to predict 10 years, get a 10x improvement in the quality of software that we're writing. I'm definitely with you on the I'm going to predict 10 years AI software generation. I think it's going to get a whole lot better. It's going to be an awesome tool and I'm really excited for it personally. I'm excited to

use it because there's all sorts of things that it can do that would be great for me not to have to do and it helped me build some software that i want to build uh but yeah i just think it this is a the bigger question of like the industry and cost coming down overall that's the thing i just am don't think and just to put some ice water on everybody siri is 10 years old right and how much better is siri today than than Siri was 10 years ago? And that was one of those cases of, you know, it's an AI.

It's, you know, it's used to be what we would think of as like talking computer. Wow, it's an intelligent living artificial entity that you can converse with and ask it what temperature your biscuits are at. Hard, flat, and unleavened. But it's not that much better now than it was 10 years ago, so we may not see much more improvement on code generation beyond what we currently have. I think we will.

My one-year prediction would be that these large language models get pretty darn good at pumping out some code translations. All right, I think we should go a little faster through these next, even though there's a lot to say. Yeah, they're not as interesting. Yeah, so expert systems, which I have actually worked on some stuff

that's like expert system things. They're cool. They have their niche use cases he imagined it as a way to like an advice giver for programmers which is kind of what these chat gpt things can kind of be now uh which was cool uh but he was right they didn't make an order of magnitude difference and we kind of ignore expert systems now if you don't know what an expert system is it's a bunch of rules put all together and they can fire in any order. And if you look at Age of Empires II AI,

it was like an expert system. The whole way it worked was exactly these sorts of expert system setups. So if you hear rules engine, it's kind of like an expert system sort of thing. Following one, automatic programming, where automatic is in scare quotes um nice curly well typeset scare quotes but scare quotes all the same you give it a specification

it gives you the program which again chat gbt is probably the best thing we have for doing that right now like there are like some systems that will try to do it or whatever i again i don't think this i think he was right but in the time frame, definitely didn't. And I think probably today, if you don't include AI, large language models, probably true, that it's not going to be that much different. But also, again, I just don't think it's going to make a big difference at scale. I gave a local talk on this idea before of programs that write themselves.

And Idris has some really cool capabilities where you can give it some types and it will find programs that fit the types. And it's actually pretty darn cool. I was able to write monadic parser just by following Idris's explanations, even though I did not understand what I was doing. It was just like, oh, pattern match here match here pattern match there automatically fill in that clause and like i only had to fill in like a few things and my choices were really constrained so it's there's some cool stuff there but again we're acting for silver bullets not cool things that fun are fun to explore we're going to skip the

next one for a second for a second just so we can finish out the details all right program verification uh you know proving our programs i think he was right it didn't make a big difference and today everyone agrees like it's there's well they do that it's complicated to verify programs and even if it's useful it's not something that everyone's going to do on every single program and so you know it's going to be on these like these very important programs that are going to get this verification done. It's for something that is so useless at the industrial scale and has has been so, you know, ignored, maybe maligned. If you ask me what I think about it, i would malign it for something that is so uh you know so useless in practice this one in particular gets so much attention and activity

and i just have highlighted in green i do not believe we will find the magic here that especially after the previous section which we'll come back to it makes me so happy it's like yeah i agree with you know i like the things that reinforce my priors yeah and environments and tools i think he gives less of uh i i think this is actually more important than he says like he even mentions here perhaps the biggest gain yet to be realized in the programming environment is the use of integrated database systems. I think that's great. Yeah, but I actually think this is not explicitly mentioning, but really you could think about this like Git, right? While it's not all of the details, right?

Like source control here might be a silver bullet at some point or other. And I will say, I think it was, I don't know when that time period is, but my first job, we had a really terrible source control system. Uh, so bad that we basically didn't have one at times for certain projects.

And then like our sequel wasn't source controlled and like having proper source control on real projects when i got to switch to get there it was a huge difference i mean just massive the amount of time we wasted coordinating these sorts of things i do think actually for that group was an order of magnitude difference in productivity so um workstations by that basically personal computers i think that probably actually was an order of magnitude but i wasn't there so i don't know yeah like there's a there's a great quip in the middle of this well how many mPS can one use fruitfully? And I have the comment, all of them. I, yeah, like if you look at the last 40 years, right? Computers are unbelievably better than they used to be.

And are programmers using that power? Absolutely, yes. Are we using it in service of writing more reliable, dependable programs, writing them faster, doing that kind of industrial scale stuff? Arguably, yes. Like you just said, Git, right? Like that kind of source control is enabled by ubiquitous networking and plentiful storage and, you know, fast disks and all that sort of stuff, right?

Like faster, better computers. Do we use every last flop on our machines in service of writing great programs? No, a lot of the computing power that we use is needed because the problems we're being asked to solve are much bigger than they used to be in terms of their demand on computing resources, right? Like of you know game developers use a lot of their computing power because they're making a game and games demand a lot of computing power but i do see programming

benefiting from improvements in hardware and if for nothing else right like our screens are bigger and have more pixels and can display colors and you know if if we get to talk about visual programming at some point um yes advent in computing power is a really important enabling factor in that in that uh in that particular area so so yeah we can go back to the section i skipped hooray which is graphical programming i mean i'll just i'll start yes please since i have the most opinions on this topic of between the two of us i have almost nothing highlighted so you might uh so um i think the most interesting thing here actually is that this is not an argument against it in general. It's an argument against it at the time.

Yeah, oh, of course. Given the technological limitations that existed. So, for example, we see the screens of today are too small in pixels to show both the scope and the resolution of any serious detailed software diagram so first off like it's the assumption that if you're doing visual things you're having to show a diagram right yeah uh which is like not a good assumption um uh why not um dynamic land yeah okay sure yeah but also that you have to be on screens which like at the time of course makes sense but if we're talking about removing the frame for a second yeah yes we know that that's not necessarily the case right um but i yeah i think there's nothing essential in the the big way about saying that visual programming has to be a diagram right yeah of these detailed aspects i think that you can do things that are

visual that are not diagrams that's all i'm saying in fact i think one way of reading this paper that i actually think is really interesting is all the things he points out of like problems you can't solve or whatever if you just take those as like that is the limiting factor and if we just attack that problem we could get 10x is really interesting because like conformity right if we go back to that where he says we have to conform to these existing interfaces well if we just say no that's not essential we don't have to conform to them let me go make you know the chatting the talking with aliens like i can you know negotiate we could get a 10x improvement i think for sure actually in these sorts of things

right because so many of these human factors are around negotiations of interface etc and and here you could see dynamic land right oh the screens are too small well that's still true today for the kinds of things that we want to do in dynamic land. So we get rid of the screens, right? Like if you take all of these constraints that he puts out and just say, yeah, that's the thing I'm going to get rid of. I think you actually get a plan for how to have a 10X improvement

that's not actually inaccurate. Or if not a 10X improvement, a cool new way of programming. Yeah, yeah. I think- 10X cooler. Yeah, yes yes which yeah all i care about yeah 10x more beautiful programs right yeah where beauty doesn't mean some narrow thing but you know whatever we want it to mean yeah and just to finish on this graphical programming thing where

i mean for the most part i agree with what he says given the framing there's a sentence where he says nothing even convincing green much less exciting pink has emerged from such efforts yellow meaning yeah we can talk about it and then pink i am persuaded that nothing will oh you you unimaginative spoil sport yeah i am persuaded that nothing will emerge from graphical programming this this is where i get more of that idea that brooks just really doesn't think graphical programming is ever going to be a thing you're right he does think that none of the other sections he so definitively says this is not it this is the one area where he's like it's not gonna be visual it's not gonna be graphical and he's like he he says frame get out of the room i need to to say visual programming is never going anywhere

um i could make a a claim about graphical programming that i think you would agree with that's a negative one so i think is what he actually thinks but doesn't say no singular static representation of a program is ever going to be the answer for graphical programming i can agree with that but i also could interpret that in ways that i wouldn't agree with that's not interesting so but really he's just looking at the static nature of all of these things i think that's his biggest problem and when that's why i say when he refers to space he really means and not time right because like if you think about all the interesting systems the one that if i keep if i bring up on the podcast here you end up doing weird things. I edited it out.

That one. The project that shall not be named thinks about time in an interesting way. Dynamic land, I think also, while it abstracts away time in the programming system, it allows time in the real world because it lets the real world go into the computation. I think that these are the kind of interesting ideas time in the real world because it lets the real world go into the computation right and so i think

that these are the kind of interesting ideas that need to be done is they have to go beyond just a diagram a static diagram of something um and what he might have is that sort of like bias that one gets when they've been at war with a country for their entire childhood and their whole life and you know he's been beating back charlatan snake oil peddlers who are claiming visual programming is going to be the revolution for his entire career and so he's so resoundingly convinced not only that it is not a 10-year possibility, but that it's not a possibility at all. And maybe if he were, you know, a young person today looking at what we've done with computers in the time since, it may be something where he wouldn't have developed that particular perspective and that uh with any luck you know all of this arguing over whether programming is going to be visual or not is going to seem really silly in a decade so yeah i don't know if i want to say decade on that that sounds like a prediction um the last sentence of this

section i i have highlighted just because it's interesting to think about he says because he's talking about you know some things can be visualized some things can't be visualized and he's comparing with vlsi a very large system integration is that what that stands for uh but anyways he's like thinking about chip design right and he says a chip design is a layered two-dimensional object whose geometry reflects its essence a software system is not. Here he's just trying to repeat the same thing, that software is not essentially space-based.

Yeah, which I essentially disagree with. We could get into that, but I don't think we need to. No, yes. All right. So, graphical programming. These are the things he thinks are not going to make make a big difference and then he gives us a list of things he thinks might uh promising attacks on the conceptual essence so these are the things he thinks get beyond the accidental properties and really get at the essence and this is why i found like once i finally multiple readings of this paper made this so clear

because I had the high-minded idea of essence. And then you see these and you're like, oh, no, he means very practical essence here because he's got buy versus build, right? Here's one of the things. We'll get into the topics, but I just want to read the headings. And then we got requirements, refinement, and rapid prototyping. And then we got incremental development, grow, not build software.

And then we got great designers. And these are all the things that could potentially really attack the essence of the problem and get us a 10x improvement. Are there any, I think buy versus build, it's kind of obvious, but really what happened here was open source software. By 95 or 96 or whatever, don't know the state of that. But somewhere between 86 and now, buy versus build did bring about, I would say, at least 10x. When you think about things like NPM, right? Yeah. And like Rails, right? What Rails and the ecosystem of off-the-shelf modules

that you can just grab to do any particular kind of job, like that had a huge impact on conformity, right? Mm-hmm. Like I need to use AWS. I don't need to learn the AWS APIs. I just go grab their their ruby gem for aws and it's it's written to fit the kind of software that you're going to write from ruby to use aws um like this one yeah 10 years 10x maybe not but huge uh change to the way that software is written and and yet

the only way to evaluate that is on the counterfactual thing of, had I written the software without NPM, would it have cost me 10x more? Not, did costs actually get reduced 10x, and did we stop having budget problems, and did we stop having schedule overruns? I think this is the problem with the original framing, is we always change what kind of software we build in response to these 10x improvements if they are there right so uh but i think yeah i mean he's he's absolutely right here that interchangeable packages off the shelf

packages which you thought you'd have to buy which you didn't uh they're just free make a huge difference right like i mean i think this is absolutely spot on. The details are a little of the time, but in terms of overall, if we ignore the 10-year frame, he's right. It does make a huge difference. Requirements for refinement and rapid prototyping, I don't have anything highlighted here.

I've got a bunch of green stuff just because i think this is speaking once again to stuff that pertains if not as identical to the theory building uh thoughts and some of the stuff we talked about earlier when talking about ai like it is there is he's feeling a need for some kind of solution to the problem of just saying what do you want to do right the hardest single part of building a software system is deciding precisely what to build so if there is something that comes along that makes that easier to do or makes that faster or cheaper or you know blue sky you know pipe dream lets clients do it for themselves instead of needing to work with a software engineer for as much as was needed at the time like that would have been a 10x they

could have been transformative and i think some of the pain that he felt in this section was addressed by other things like he says a prototype software system is one that simulates the important interfaces and performs the main functions of the intended system while not being necessarily bound by the same hardware speed size or cost constraints and so like that's one way of solving like oh you need real software engineers because they can they can build what you need within known speed size and cost limits and as time went on budgets got bigger computers got more powerful and and you know more widely distributed and so it's sort of like that problem was solved not by making better programming systems but by just the things we're programming in the context of got much more

permissive yeah i think you can kind of recast this for today and just talk about scale yeah right yeah right here's the prototype system that does everything you wanted but not at the scale you expected right i've built lots of systems like that where it's like yeah this operates on a hundred thousand rows of data rather than billions yeah right that sort of thing um but yeah i i think i mean i think these things are accurate like prototypes are really cool i don't know if they're 10x improvement and at an industrial scale blah blah but like prototyping definitely is a practice that we should do and is good to do and is helpful and the kind of things that he

meant by prototype at the time are things that we like he says prototypes typically perform the mainline tasks of the application but make no attempt to handle the exceptions, respond correctly to invalid inputs, abort cleanly, etc. Well we have Rails. Rails handles our exceptions,

responds correctly to invalid inputs, aborts cleanly, it handles our security concerns for the most part, makes sure you don't get you know sql injected because you passed unsanitized input into your query right like a lot of the things that you would have to attend to as a working programmer in the mid 80s if you're building a similar kind of thing like a crud app these days you'll use a rails or a django or one of those kind of things to take care of a lot of those concerns for you that like put you on rails and he didn't have that at the time and so i think it's interesting that now it's like just build what he would have thought of as the prototype

and you can ship that so yeah not within his, but definitely something that we've seen change. Yeah. Incremental development versus like a big waterfall model. This is a hard thing for me to say if it really made a 10x improvement, because it also, that model also came with a bunch of other baggage that like stopped you from having the improvement you could have had. In and of itself maybe but as a factual historical matter it also brought on a bunch of like ceremony and annoying

things that happened with it uh but again we're i really think we're trying to be practical here and sadly it it came i we wanted the banana and it came with a gorilla right like that's that's what i think happened here i read this not as just being about methodology and about like agile and that sort of thing i read this as like in the 80s if you were ibm and you were hired by you know let's say uh let's say what's it what's a 1980s company mit mit okay let's say they had all sorts of things that they hired ibm for i'm thinking like a mit accounting system or okay sure yeah let's say let's say you were hired by some some big accounting firm to develop some software for them right like you are hired to build uh a system that runs on a mainframe and the mainframe gets installed at their office and there's terminals put around the building and whatever, whatever,

whatever. Compared to mid 90s, maybe late 90s, maybe early 2000s, where it's quite a bit different now, where by the early 90s, it's not you're putting a big mainframe in but rather you're packaging up a bunch of cds and mailing or delivering those and people are installing that on their pc stations at their desks and then by late 90s early 2000s it's no longer you're packaging up a bunch of cds necessarily oh maybe mid 2000s but you are now sending that software over the internet and you know mid 2000s late 2000s you no longer have to just ship that software and be done with it and then updates are okay we got to ship out a whole new round of cds it's to update it

just download this update over the internet. And then nowadays, you know, like, you know, we CI and we ship to prod five times a day or whatever. It's now the software runs on somebody else's machine, you access it through a browser, and it's not like you've installed anything. And there's no single version. It's this living, constantly churning thing and so we've seen this transformation from the software comes bundled with hardware that is delivered and installed once and is effectively like you got to call it done when that happens to this world where the software is never done and you're no longer buying or like commissioning software you're not commissioning a custom installation and a custom machine with custom programming you're not buying off the shelf software but you

are like leasing or renting or borrowing software that comes through the pipes and runs on machines you never see and i think that gave us this grow not build transformation it just took a little longer than he thought it would yeah i think you have put into the text something that's not in it uh but i like your your interpretation way better i think he definitely meant like what we would call agile today he He cites a spiral model of software development and enhancement, which I have read. So maybe that colors my opinion of what the scope of this is. But I do agree, if he meant as a service sort of stuff

and shipping changes more regularly and small things, etc., and not just the development process before the big ship at the end, if he meant all of that, then I totally agree with you. It makes a big difference. I just think he had a more narrow vision of like, this is like inside until we get to the ship,

and then ship is different. Then the final section of this, I think this paper ends very strangely, in my opinion, is great designers. It is the final thing he thinks can make a big difference for 10x improvement and it's really weird that we don't get a concluding paragraph section anything here this whole like what will make a difference is kind of the conclusion yeah We don't get a big picture other than he thinks that great designers are actually the thing that's going to make

such a big difference. He says here, great designs come from great designers and software construction is a creative process. And so really, I thought it was kind of remarkable because my stereotype of Brooks is kind of Mr because my my my stereotype of brooks is kind of mr corporate right like very informed very like reflective but like he is caring about the corporate environment

and yet here you know he's talking about like these kind of more solo not designed by committee projects uh are way better and more beautiful and better designed. And he thinks that if we could get more great designers, we could actually have a huge improvement in the industry in general. And that's not going to come from putting down a training program with some methodology, but actually finding these people and helping bring them up and give them the support they need to make these great designs. Yeah. How do you, how do you feel about this one? Um, I think, I think he is misplacing where the importance is. Uh, like, yes, I do think people

can become better at software design and write great software, but they almost always do it absent certain business constraints and team setups and all of that. And I think you could take the same great designer and put all the constraints on them that you have in a work life that makes bad software, and you wouldn't end up getting better software. Like, think Smalltalk. that makes bad software and you wouldn't end up getting better software like think small talk that's one of his examples of of a great piece of well-designed software the constraints for xerox park making small talk are so different from the constraints of software products i've worked on that have been very poorly designed but the the people there were

were good people who knew what they were doing. It's just the constraints were very different. Yeah. This section especially bothers me because it's sort of like the thing that we need to do is take a few special people and make sure that we recognize their specialness and like really, you know, treasure those few special people. And I really don't like this way of thinking about some people are special, and we need to identify those top designers as early as possible, and then pull them out and put them on a different track from the cattle.

I just, yeah. Well, I mean, this actually comes back to the framing so to get into the bad parts of aristotle's essence and accident right aristotle thought some people were essentially slaves was one of his examples right he thought that's just their essence they are meant to be that right and so this accident and essence has been used in such terrible ways throughout history for like really racist ideas. And these ideas of like, well, this person is just

essentially great at being a designer, right? Kind of leak into this a little bit, right? I think that there's these ideas that like, you couldn't change that. They're just essentially that way, and we have to identify them, because it's not an accidental property of somebody. You couldn't teach somebody to be a great designer.

They just have to be that. And I do think some of this thinking leaks in to this thing. I'm not claiming Brooks thinks those really strong things. I'm just saying I think you see some of that still essence, which has been kind of, uh, you know, it's a trope in our society, right? In general, not just in Brooks thought where we think of certain characteristics as essential. And there's a lot of modern day philosophers who want to get rid of that. Not

all of them. Some people think, no, we can still use accident and essence without all these problematic notions like sally haslinger's a feminist who who has that opinion but you can see why people might be opposed to some of these ideas yeah like it's definitely uh an embodiment of certain ways of thinking that i think dominated business of the era so like just just to read i'll read what he's written and then i'll offer a counter um so he's got four bullets that are sort of here's the some just some quick hits because he says space does not permit a lengthy discussion but some steps are obvious for how to grow great designers first bullet systematically identify top designers as early as possible the best are often not the most experienced.

So don't let experience kind of, you know, guide you to miss out on some diamonds in the rough. Then second, assign a career mentor to be responsible for the development of the prospect and keep a careful career file. Then third, devise and maintain a career development plan for each prospect, including carefully selected apprenticeships with top designers, episodes of advanced formal education, and short courses, all interspersed with solo design and technical leadership assignments. Then finally, provide opportunities for growing designers to interact with and stimulate each other. And so it's really like find deserving people and pour resources onto those individuals as opposed to find people who show a lot of promise and figure out what it was that got them to the point where they're showing so much promise and like reinforce that right like if there's like 10 of people who are coming into your organization and are showing you know remarkable talent at a certain thing that does not uh you know align with the amount of experience they have right they're coming in green but they're doing really

well why not go back upstream and look for how where did those people come from and what got them there rather than taking all the people coming into your organization as just like an unchangeable fact of the world, right? There's going to be 10% of the people who come in are great, so let's make them greater rather than let's raise the number of people coming into our organization who are at that level of greatness i don't know i i really don't like this this particular idea he's identifying something legitimate which is some people are better at doing this work than others but it's not because of who they are as people in some unchangeable way exactly it's not their essence yeah grosses me out okay so we're at the end of the paper you know yeah i'm curious about you know concluding thoughts i'll give mine like what i yeah okay so like i have found the more we read these papers the papers i enjoy the most i also find very flawed like theory building i think is a great paper that's also

very flawed in its actual writing right the artifact itself had a lot of weird little turns of phrases some bad arguments i feel the same about this paper we've kind of picked apart a lot of things that that brooks has said here but honestly i actually really had fun reading this paper i really enjoyed it i think it kind of gives me some, some hope for my own things. Cause I see the flaws in my own writings. I see the flaws in my own thoughts and I think, Oh, it can't be good because it needs to be flawless. And these papers that have all these flaws are actually really fun to read. And I think that the thrust of the point can get across even when the details are kind of wrong

so i came into this kind of being pessimistic and thinking like i'm just going to disagree with like everything he says and while we can pick apart all these details i think he's right like if we confine ourselves to one single development we're gonna do the wrong thing and think we can like solve the whole world and instead we need to think holistically and while that's not explicit here i think it is implicit in a good way where he wants to push us to look beyond just these basic things i think even this bad designer thing is kind of getting at it the designer can tackle a bunch of things all at once, not just have one technique to put forward.

And so I really liked that and I really enjoyed this paper. And I definitely think if you've already read it, I think it is worth a reread and like a charitable reread kind of coming into it and looking for the good in it. The first time I read it, I liked it. The second time I read it, I hate read it.

And now reading it again liked it the second time i read it i hate read it and now reading it again it's like depending on how much charity i want to give or afford uh it's a really enjoyable source of history and perspective and little tidbits of things that failed to pan out that might have failed to pan out just because they were put in the wrong crucible like there's a lot of ideas in here for things that totally failed right up until they didn't and it's you know they failed within his framing at his scale within his time frame within his weird limitations but you know things like open source and things like you know the way crud apps have taken off and the internet becoming a thing like there's been so many major developments in the world that have changed what kind of software we're writing and why we're

writing it that it's wild that so many of these things that he looked at like played a part had a role in some way or another and so that kind of gives me optimism that if you don't look at any particular future of coding project you don't look at any particular idea that somebody's suggesting for a way we could program better but you instead take like one step back and just think about like what is the delta that's that's setting up between the way we currently do things and what we could do instead in the grand fullness of time we will probably see that delta realized in some way in some context and to some extent like it's it's funny that there's so many different things we could do and i like feeling like that we're gonna get to do all of them at some point so that that makes

me really happy reading this and reflecting on it and thinking about you know all the things that he thought we wouldn't get and they you know or that even if we did get them they wouldn't help and it turns out so many of them did help and have made things better or at least more different and interesting with the exception of visual programming because it's never gonna happen he said it was never gonna happen and it hasn't happened and that means it's never gonna happen it's never gonna happen It's never gonna happen It's never gonna happen

It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen

It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen

It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen It's never gonna happen Never gonna happen Never gonna happen

Never gonna happen Never gonna happen Never gonna happen Never gonna happen Never gonna happen Never gonna happen Never gonna happen Never gonna happen

Never gonna happen Never gonna happen never gonna happen It's never gonna happen It's never gonna happen Never gonna happen Thank you.