CoRecursive: Coding Stories - Tech Talk: Jimmy Koppel on Advanced Software Design

Episode Date: August 1, 2019

Tech Talks are in-depth technical discussions. How do we create great software? What are the important skills need to properly review a PR? How do you identify assumptions of a code base and the stable contracts of a software module? Jimmy Koppel is working on his Ph.D. in the field of program synthesis at MIT.  He was previously paid 100 thousand dollars to drop out of university by Peter Thiel, but yet still graduated with two degrees.   The most interesting, however, about Jimmy is he is working hard to teach the world how to design better software. Due to his time working on program synthesis, he developed some unique insights into what makes software good, and what makes it bad, and he spends time teaching people his insights. Three Level of Software Peter Thiel 20 under 20 Benjamin Franklin Method  You are a Program Synthesizer Jimmy's Coaching Program Episode Page: https://corecursive.com/036-jimmy-koppel-advanced-software-design/

I'd say that learning to read this logical level, it's kind of like being able to detect pitch music and that everything of software design requires seeing it. Hello, this is Adam Gordon-Bell. Join me as I learn about building software. This is Code Recursive. That was Jimmy Couple. He's a super interesting person.

He's working on his PhD in the field of program synthesis at MIT. He was previously paid $100,000 to drop out of university by Peter Thiel, but he still graduated somehow. The most interesting thing to me is Jimmy's working hard to teach the world how to design better software. And I think he has like a really interesting perspective on this. I guess my summary is after spending a lot of time writing programs that write programs, which is what program synthesis is about, he developed some unique insights into what makes software good, what makes it bad. And he spends his time trying to teach people these insights. So Jimmy, thanks for joining me on the podcast.

Yeah, it's good to be here. So, I feel like you have a unique perspective on software design, and I'd like to ask you some questions about that. Okay. So, why is designing software hard? Why is designing software hard? Why is doing anything hard? So, rather than jump to that, I'll tell you, there are certain tasks in software that are automatic versus certain ones that require creativity. So imagine you're writing a system that inputs from one thing and outputs to another. Very often, the nuts and bolts of that thing can be done automatically.

So you get the data in some format, and there's only so many things you can do with it, so it's kind of automatic. Then you have the raw data elements. You need data in some format and there's only so many things you can do with it. So it's kind of automatic. Then you have the raw data elements. You need to write some format. That's kind of automatic. The creativity enters when you have things going on in between. So the creative parts of software design are typically in building the intermediate representations. So the creativity happens in building representations, designing your data abstractions.

And usually the in-between parts become automatic. So the in-between parts are sort of the serialization and deserialization? Yeah, things that break the data down, do something down, and then build it up again. Those are the glue, and they tend to follow very automatically from the actual shape of the data. That makes sense. So do you think that this middle part, like the software design aspect of doing the middle,

that that is like a well-understood field? I think it's more understood than most people realize. So we still are very far from being able to automate it well. There's a lot of things that we do know how to do and that I do teach. But part of what makes it hard is that so much of what goes into that process, and this is something you'll hear from me again and again today, is really about the intention of how that thing is going to be used, which shapes all the ways it can evolve. And that is something which is not directly apparent from the code. Is intention like the intent behind building this piece of software?

Is that the intention? So let's say I work on some software that takes in a Docker image. So binary snapshot of a Linux environment. And then it outputs a list of security problems that it found. So part of the intention comes from how you define a security problem. And then from your definition of what a security problem that determines what it's going to look for. So if you intend there to be some kind of filtering

where it's, say, going to not report certain boring kinds of things or things you're not sure are a problem, then that's going to imply totally different sets of filters and actions and extra checks that you can put in versus if it's a purely mechanical process. So the work of software design is in translating this high-level goal of reporting security problems down to low-level details like point of scan for this, check that. And your high-level description of this piece of

software, like it takes some Docker chain to report security this, check that. And your high-level description of this piece of software, like it takes some Docker chain to report security problems, that might never change. And then for each component, you might have intermediate-level descriptions. And it's possible those intermediate descriptions will change as well. But the way you turn those into the actual details will change all the time. So how would I use my knowledge of software design to build it in a way in which it was not likely to break in the future? I know it's a very open-ended question, but... because the actual elements are ratchets, openness, and assumptions.

So that's actually what I came up with. It was reduce assumptions, add openness, diminish complexity ratchets. Reduce assumptions? So the thing we'll talk about right now is assumptions. Okay. Something I teach a lot is about, so software has to be modular

in order for it to be feasible to write. And that means that you can look at it as a single piece of code out of context and talk about what it means and what must be true about the rest of the code in the system for this to work. So for instance, say you have a point class and you say point.x, that is already placing an assumption on the system, namely that this is a type that has a field called x, which is an integer. It's a very weak assumption, but those are the kind of assumptions you have to learn to identify. So say you're writing your Docker container scanner, and you can look at it as a random 10-line snippet and try to state in

plain English what this is doing, because that forces you to say something which is closer to the intention than to the details. And then you have to think about how can you look at this 10 lines of code and through whatever kind of manipulation to convince yourself this does what you want it to do. Those are all the assumptions that it's placing on the rest of the system. And obviously, some of those assumptions have to be there. Like if I didn't want to do anything with the point, I need to know it has an X and a Y. But I don't need to know that it's stored as a tuple with X first and Y second. That's an incidental assumption. And so those are things that you want to identify and eliminate.

So you look for assumptions and then some of them will be not important to the intention of the software? Yeah. And those are the design mistakes, would you say? Not necessarily mistakes. Then this is a process you can take arbitrarily far. But those are the things that you want to keep localized. So I'm going to run with a point example again, because it's easy to talk about for everyone,

even though it's kind of stupid, which is, say you have an application which is passing around XY points everywhere. One day, you realize that you want your system to support both 2D and 3D. So sometimes these points will be x, y, and sometimes they'll be x, y, z. The parts that actually need 2D stuff, it's fine if they're taking an x, y pair. But then you'll have a thousand other parts of your system, like things are just saving stuff for passing around.

And if you're actually taking them in this x, y pair, like when you're taking a parameter, you write parentheses X comma Y, that is spreading this assumption unnecessarily. And so now when you go to XYZ, that's where you have to change 10,000 places because this assumption leaked to all those places where it didn't have to. So a more realistic example of this kind of phenomenon of an assumption leaking. One of my students works at a large public company, which I won't name, other than to say that I've used their software and a lot of people listening to this have as well. So they have a C++ application and they're just trying to change

the representation of strings. Long ago, they could have put the string behind some abstract interface and it's C++, so there'd be zero cost to doing so in runtime, but they didn't. So now his team is putting in 100 man-hours into refactoring the recent strings, and there are several other 100 teams like his. Yeah, I like this point example, because the point could be in 2D or 3D space, and you're saying we can write the software

so that we only in the places where we actually need to reference these coordinates of the points know about the internals of it. I watched this talk that you did at Strange Loop about, it was called You Are a Program Synthesizer. And it did talk a little bit about this abstraction barrier. So I was wondering, what is program synthesis? And how does it relate? Yeah, so program synthesis, broadly defined, is programs that write programs. So fun anecdotes. Alan Perlis, computing pioneer, he had the quote that anytime a programmer says, I should just be able to say what it wants, and the computer should give me the program, you should give that guy a lollipop.

So in the intro to program synthesis class, on the first day, we give the students lollipops. Most of program synthesis, I mean, writing programs in general is hard, so most of it is about specific kinds of programs. Like every PL conference these days, you'll see a synthesis track, papers like synthesizing a web crawler from a demonstration, synthesizing SQL queries from natural language, synthesizing high-performance container libraries from a normal container library, plus a lot of annotations. It seems like writing programs is difficult. Writing programs that write programs

must be quite challenging. Yeah, it's its own specialty. So the way that I see that that's an assist in influence software design is when you're writing a program, you want to make it easy for humans to read. If you're designing API, you want it to be easy for humans to use it to write new programs. How do you determine that? Can you peer into people's head? So yeah, you can go down the path of psychology of programming. It's very hard. I don't know how far anyone's gone. But there are certain things about writing programs and with looking at code, understanding what it means, which are not only about psychology, but are more fundamental to the code itself. And if there's only one, you can prove there's only one algorithm for determining whether a piece of code can ever return a number greater than 100.

You can show there's only one algorithm for that, which is not the case, but suppose you could. Then you know that a human reading that would have to, in their head, be running something similar to that algorithm. And when you can't say, like, this is what a program means, and these are the only way to communicate that meaning, to compose them and get what these two things together mean. So we can learn about algorithms for writing code and reasoning about code and use that as a proxy for how humans write code and reason about code. And that gives us a lot of insights into the nature of programming and software design. What type of insights does it give us? That makes sense to me. At a high level, you're saying, oh, if we're working on writing programs that write programs, we have to come up with some understanding of program generation. So what insights have you found?

So some of the ones have to do with the search space. So you do things this way, and you're stretching the amount of information available. Now there are like only three operations that you can do in order to put these together and get results. Whereas if you don't have this abstraction barrier and you're at any point have the freedom to go down to the raw bits, the raw character, the string,

and do all this stuff, then you have a lot more possibilities. So that's something that helps both with verification and with generation. In my mind, it seems related to just extensive use of types. Like when people say, make invalid states unrepresentable, is that a similar concept?

So every type system designer, at least the kind of type system designed from the software engineering side as opposed to the pure type theory side. So like you read a paper called, like a type system for concurrent blah, blah, blah, type system, you know, like a type system for concurrent blah, blah, blah type system, you know, a type system for guaranteeing database accesses have correct permissions. We let any of those sort of papers, people who write them have come up with them by studying the domain, studying the kinds of programs people want to write, and usually determining something fundamental about the nature of the problem. Let's say that I want to get better at building software.

How do I even know if I'm getting better? Okay. You say you're learning instruments. How do you know if you're getting better at the instruments? You have to really cultivate heat and ear. So you have to be able to listen to yourself in slow motion and notice the little things, listen to masterful recordings. First, you'll have a teacher correct you, and eventually you learn to correct yourself. So the same thing goes for any skill you're trying to cultivate to a high level. And in software design, I teach people all about... So earlier we talked about the process of turning your intentions, your high-level intentions into immediate intentions, and then it's against code.

And then there's also the other direction of looking at code, discovering what facts it implies about a certain point in the program, and then what those facts imply about a higher-level piece of state, and eventually all the way up to abstract statements about what the program is doing and what you want to happen in the world. So all those extra layers between these high-level intentions and the raw code, it's what I call the logical layer. I used to use the term the third level, but I decided it wasn't descriptive enough.

So I'd say that learning to read this logical level, it's kind of like being able to detect pitch music. And that everything of software design requires seeing it. So every programmer beyond intro can look at some code and then tell you in plain English what it's doing, at least they wrote the code. But there's all these extra details about being able to say what has to be true at this point, what assumptions it's making about the rest of the code. Those are the extra, you know, what else can change about the system that would make this not work. Those are all the extra details that let you see the complexity of your code. So what is the logical level of your code or of your design? How would you phrase it? Is the logical level part of your code?

No, and that's very much not because for a given piece of code, you can give it many meanings. And they might be very different meanings. Like if you say, if x greater it makes coffee and the other arm it opens my mailbox. Well, it does one of those half second before the other. Is that part of the behavior or is it just coincidence? So whether you say it's part of behavior or not, those are two similar but different meanings given to the same concrete behavior. So because there's this many-to-many relation between implementation and interface, You know, a single interface can have many implementations that people get, but people are less keenly aware that single implementation may correspond to many interfaces. Because of that, it is, in general, impossible

to determine the interface by looking at the code. And when I say the interface, I don't mean like in Java, you say, oh, I have a list, and it has interface is these five methods. I mean, actual facts, actual logical facts of the behavior. Like if I put something in and then get something in the index, I get the thing I just put in. That's an actual fact about all possible sequences of calls. So the logical level, it's above the code. It's interesting. It makes me think of sometimes if you work on an old software system

and you need to make changes to it, sometimes you have to do something like software anthropology where you have to interview the people who worked on it. Like software anthropology, software archaeology. Yeah.

You can Google software archaeology. You'll get a ton of hits. And it's because this process about how the high-low intentions were turned into high-low design and then to low-low design and then to the code, those processes are either lost

or at best living in their head or in a place you can't find in documentation, and that's what you're extracting. So when I talk about the logical level, there's a couple of literary places this concept has appeared. So say you flip a bit in something uh what is flipping the bit

doing the most obvious thing most concrete appearance say i changed a one to a zero or zero to one but then then that has a meaning it's like oh i changed the permissions of this file or i said a fact that i've already visited this square. I've already done this thing. So that's an extra correspondence, which is puts on top of that raw physical action. So this is discussed in Gerlach-Rebach by Douglas Hofstadter as you have these numbers. How is changing these numbers do anything? It's because there's an isomorphism between these numbers and something more meaningful. And it's also talked about in the art of motorcycle maintenance, where you have an action like I'm turning this screw,

and that's what he calls a romantic view. In the classical view, it's that the screw is part of this engine, which is part of this drivetrain, which blah, blah, blah. And you have these extra layers built in your head of how these parts of the motorcycle fit together. And when you're turning a screw, you're really acting on those mental layers. And so the same goes for when you write your everyday level of operations and code. So is the logical layer synonymous with your earlier comments about intention? Like is the logical layer the intention of the piece of code?

So the intention is part of the logical layer. Really, when I talk about logical layer, it's kind of a catch-all term for a bunch of related concepts about all these things that code is doing, all these ways to describe its meaning, which are not directly apparent in the code itself. So when I talk about the logical layer versus the code layer, it's really more of a rhetorical device that you can tell people from talking past each other. So one person says, I'm looking at your code, this variable is global, make it not global, that's bad. The person says,

well, it will do the same thing whether it's global or not, the way it's written. What can go wrong by leaving it global? And then person A might maybe, they'll just say, oh, well, it's global state, it's bad. Or they'll concoct some scenario in which something accidentally bad happens because of that. Person A says, well, I'm not going to do that.

The user concocts another scenario and says, I'm not going to do that, and so on. And it's because one person is talking about the code as it is now in concrete implementations, and the other is talking about all possible modifications to the code. And that's they're quantifying over all changes to the code, and that's a higher-level logical statement.

Not very high-level, but still. What all of the things in the logical layer have in common is that they involve quantifying over all possible changes to code so a lot like a logical statement you know i run x equals one and then x equals one at this point or x is equivalent to the id of this one of standard in or standard out whichever one or x or X is a positive number. Those are all logical statements, but they all correspond to infinitely many pieces of code. And those encapsulate the facts that need to be true at that point for the code after that to run. So even very low-level things like I said equals to one and then X equals to one,

that is talking about all possible changes to code, all possible changes that preserve this fact. Interesting. So do you think that as a profession, software developers, like that we put too much focus on reviewing the code and not enough on looking at what these intentions are at the logical level? Something I write about in my booklet, The 7 Mistakes That Cause Fragile Code, is that more junior developers

will tend to review things like whitespace and variable names, or even a little bit better, maybe they'll say you should use this function instead of that function. Whereas more senior developers are more likely to tolerate small variation of that, they're more likely to look much harder at what assumptions are being made by a piece of code. Like, this thing only works if you call this thing at the exact point in the program. That is not a stable guarantee. Stable guarantee, that's a word when I hear it, I'm talking to someone senior or becoming senior.

Or they might say, like, this function takes five parameters, and chances are we're going to add this thing, and then it'll take take seven parameters and it'll take eight parameters. You need to generalize it somehow. Those are the issues more focused on by a senior engineer doing code review. And you note that all of those are not about the current code, but about possible changes.

It's not a stable guarantee that A happened before B. You need to make it so that B still works, even if A is called after B. That is about a possible change to the program. So we should focus on the interfaces. Is that a way you might summarize that? Yeah, I would say like good code review focuses very hard on the interfaces. Yeah. So I read your pamphlet, seven mistakes that make fragile code. I think this one you're discussing you called don't overthink the interior.

Yeah. So I guess the idea is that maybe the method body is less important than actual inputs and outputs. Yes. And you also had this concept called refactoring is not boxing. What does that mean? So one of my friends puts it nicely. Refactoring means changing the factor graph of your code. So, what is not refactoring?

If you have a mess on your floor, then there's throwing stuff into a box, wherever it stands, or there's actually figuring out what should go where. And some refactoring that I've seen is very much this line of boxing. So, I was actually discussing this the other week on Twitter, just like yesterday on Twitter. A common thing that people do, which is not refactoring, but call refactoring, is goes by the name anti-unification. Not many people know the word anti-unification, but a lot of people do without knowing that term. Anti-unification means, say, you have two pieces of code that's different in only one place or two places. So you wrap those into a function and you make this two-place difference into a parameter. That might be a very good operation, but what does that

function mean? Can you give it a clear description independent other than I just found a common pattern between these two things? If you can't, then there really is no way to understand what the thing does other than when I fill in these two parameters it does this when i fill in those two parameters it does that so that is boxing you made your your mess smaller but the complexity is still there so some people would say don't repeat yourself it's like the golden rule of software creation so if i have two two things that are repeated except they vary on this one parameter that actually i'm improving the code by boxing those up. Do you? Yeah. And that's a very shallow metric because it's very easy to write a program that does exactly that. And I've been talking about how much software design about these higher level

facts about all change the code and depends on the entire global intentions, depend on the high level intentions and how all of that stuff is not directly derivable from the code because there's many to many relationships between design and implementation. And so if you think you can improve the design of code by doing something that's trivial to automate, then that's a red flag. So don't repeat yourself. This is really like a simplified version of the real lesson, which is to not repeat concepts. And that will cause you to not repeat code. But if you have two different concepts or one concept that appears in two ways, but you very cludgily smush these two ways together,

then you still need to understand them independently. And you haven't really improved anything. And you've quite possibly made things worse because now if you have more direction in your code, you'll need to actually jump around with the method definitions to figure out what's going on. If you can't actually have a clean description to what this does, it doesn't require you to look at these methods. So would I be able to tell that just by

language usage? Like, is there a word for these two things put together? Is that a valid case? Or is that, that sounds shallow to me. If I've created a function for these two things, can you tell if it's a good thing if there's a simple way to describe it? It's not a full preventative indicator, but I think it's a very strong one in that English has already evolved to try to hide details from things. And we already have a strong intuition about it.

So why not reuse it? And I think eventually you should learn how to talk about logically about, can I give a compact but precise formal description of what this thing does, but using some high-level concepts, each of which have their own formal definitions but i don't have to care about them right now like that's the real thing that goes on when you try to formally reason about software and i think we should turn it to our intuitions as well but when you're bootstrapping that process you can approximate it just by using the machinery

you already have in your brain for using English to abstract every details. Interesting. So actually, whether there's a good name for it can be a great guide. Yeah. In my review of Philosophy of Software Design by John Astorhowitz, I think I've highlighted something quite close to that as the best line of the book. It was a line like, if you're having trouble naming as a concept, then maybe the concept's not well defined. Yeah, that's interesting. I mean, the very short version of some domain-driven design course I took a long, long time ago was like, find all your nouns, and those are like the important things. And I guess it seemed trivial, but I guess there's some deep truth there. Yeah. So, like, when I took my first computer science class in high school, the teacher had us do something like that, and I thought it was silly, and I've ignored it for

many years. And now I'm coming back to something close to that, except you're trying to think a little bit harder about what is your concepts rather than just what are the nouns. So some of your concepts might be verbs and some of them might be groups and nouns and some of the nouns might be mere details, but has a lot in common with that very simple, trivial process. Interesting. Yeah. I mean, I guess ultimately there is some intention, as you're saying, some specification that we're working towards, and that is going to be expressed in English, I guess, or some natural language. So there's some correspondence there. Well, I like to think of them as being expressed in formal logic,

but you should be able to translate between English and logic. Yeah, actually, I want to get back to that topic at some point to ask you about formal proofs. But, you know, we were talking about refactoring, boxing things, unboxing things. I think it's a great vein. You also had this point about getting stuck in an old design, and it seems somewhat related. Yeah. Could you expand on that point or explain it a very common failure mode of junior engineers is you're given this one piece of the system and told to implement it but then the piece that they're using don't really support what they're doing very well so they have to hack around it and in a coaching call just yesterday someone was talking about how he's writing a trading system and using some library

that wanted information about a trade in a string and it's like oh i guess you do everything in strings now and you know i rolled with it but for a more experienced person i would tell them is you should go to everything in the library and tell them it's stupid and give them to change it that's funny yeah so being stuck in the rut of here's the APIs I'm using, here's libraries I'm given. I'm just going to do things the way they do to do my problem, even if it's very awkward, as opposed to I'm going to understand how the parts around me work. And then wherever it's not supporting what I want to do, see if I can find a better way that both supports what already does and what I want to do better so that my code is cleaner,

so the existing code is cleaner or not more ugly. and then go out into other parts of the system and change that, talk to people working with those, get their dependencies changed. It's usually not so easy. But part of going from being a junior or senior is getting more comfortable doing that. Your responsibility goes from your piece to also understanding the piece around it and giving pushback on designs adjacent part of the system. Another statement that I have written down here from you is code quality is not about code. And so what does that mean? I mean, it's definitely catchy.

So it's what I've been saying all along about the logical layer, and that's a common thing that kind of goes along with things like the don't repeat yourself principle is to attach yourself to very superficial measures of code complexity. And that can lead you to all kinds of things that don't change a lot of the program, but seem to make the code shorter. And usually that happens by taking a lot of things that used to be explicit and making them implicit. So one of the worst cases I've seen of this was actually, I think was on the official MongoDB blog, where they showed one example using some kind of schema database.

Like, look, you have this code that defines the schema, then this code that's maybe generated still this code that's like access it and writes to it, versus like you have this MongoDB thing, and it just puts stuff into our giant hash table. Therefore, it's shorter and has less technical debts and blah, blah, blah. And no, it's simpler. And no, it's not simpler. It doesn't have less technical debts. You've just taken this explicit schema and made it implicit in the way that you're using this giant MongoDB hash table thing. So I totally agree, by the way. Yeah, like we've thrown out our schema. Now it's just implied knowledge that has to travel around. And it's not enforced. But by a very superficial read,

the code has become simpler, but the logic has not changed. So we have to understand the code that's not there. That sounds very difficult. I actually like the way you're putting it. And in fact, there is a concept in verification verification which I think I want to talk more about in a future blog post called ghost code, which is kind of like what I was saying earlier, where you're turning a screw on the motorcycle but really you're

modifying this abstract concept of the drivetrain. Ghost code is basically you have some extra variables that exist in your head or in your proof system, but not in the actual executable code, which is like a variable for the drive train. And then when you turn the screw, you'll find the code where it says turn the screw, and you put extra code next to it saying, and of course, bodily modify the drive train in this way.

So you say the code that's not there, but that's actually a very serious concept studied in programming language theory. Yeah. So in the Mongo case, there's all this schema stuff that's not there, but that's actually a very serious concept studied in programming language theory. Yeah. So in the Mongo case, there's all this schema stuff that's just implied. I can think of other cases like type system. So like I was working on some code and it determined if, you know, an element is greater or less than another value. And it does this with like an integer. So it returns like if they're equal, it returns zero. If the first one's like greater than the other one, it returns one. And then if it's the reverse, it's negative like, if they're equal, it returns zero. If the first one's, like, greater than the other one, it returns one.

And then if it's the reverse, it's negative one, right? And so there's something, implied schema there, right? Where, like, although it's an int, the only valid states are these three. Probably a problem, to be honest, that it's represented using an integer. Yes, and I've never been able to remember which is which. Yeah, totally, right? There's a ghost schema there as well, I guess, right? Yeah. So there's an extra constraint in the output data.

From a verification perspective, you'd say it's not only returning a one, it's also returning a proof that if this thing returns one, then the left thing is greater than the right thing. And so in your code, when you write, when you're checking it for the one, you have to make this corresponding logical transformation that because you get into the branch where this return value is one, to make the extra logical inference that because we returned one, therefore, X is greater than Y. Yeah. And it sounds to me like this logical level is actually like the proof level that we are writing. Is that true? Yeah, I'd say so. And in fact, a lot of stuff I'm saying, like you go from this returns a one to this extra step of because it returns a one, I apply some fact about this function,

therefore x greater than y. If you were to do a formal proof and a proof system like Cook or Isabel or Lean, that would be an explicit step that you'd have to type in. Something I've been interested in a while is trying to see if there's a way where I can teach ordinary programmers to do this kind of thing without having to learn Cook. Because if you learn Cook or Isabel, I actually haven't used Isabel. It might be better. I think there's some reason to believe it. If you learn Cook, then you have to do all this very persnickety stuff that has absolutely no relevance to normal programming.

And for all these hours and hours and hours, and if you've ever gotten really sucked into a programming problem and then find yourself addicted and then it's 6 a.m., Cook is like three times better at doing that to you. It just makes hours disappear because you get this constant feedback from the system. But it is a time sink. So you're doing all these things that are no relevance to normal programming, but then also you get this intuition of like, oh, no, from the fact that it's returned one, there's an extra step to make the inference that x greater than y. That's an insight that I want to be able to give to people without having them have to spend all these hours on call.

And I haven't figured out how to do that yet, other than by having these conversations and giving a lot of examples. I found this quote on your website. It says, since the age of 18, I've been obsessed with the problem of automating software maintenance. Yeah. So why is software maintenance important to you? Software maintenance is important because the world runs on software, and changing the world means changing the software. So do you think that maintenance is overly hard at this juncture yeah and most obvious place this happens is in the ytk problem so you have who knows how many hundreds of billions being

spent to change two digit thing to four digit thing or some generalized version and that's a time when all companies in the world have the same problem at once. But things like that happen all the time. Like earlier, I mentioned the string thing with that public company. There's hundreds of hours per team to change their string representation. That kind of thing happens all the time, but it's usually not a case of the entire world has the same problem at the same time. So how can we improve software maintenance? So my life goal is to improve software maintenance by a factor of 100. And I think we can get the first 10x just by exactly what we're doing right now, telling developers to think about the logic of the code, to track what assumptionsstring representation, they could have very simply done well-known C++ things to wrap the string in an abstract interface. And this problem, when they want to change the representation, would have taken five minutes instead of 10,000 hours.

So averaging that with cases where it's not so dramatic, maybe you can get 10x. The other 10x is going to have to come from tools. So if we do have the ability to try to look at a code and infer what its design is, even though there can be more than one answer, then we have to be able to make tools to do the same thing. And then from there, be able to make tools that modify the design. So do these tools exist?

Or I guess your life's goal is to build them? To a limited extent, they do. One very exciting project that I've been a little privy to is from Semantic Design, which is a company in Texas that I spent a few months at a few years ago. They've been doing some work with a large chemical company to try to do design recovery on chemical plants. So they have about 1,000 chemical plants around the world.

They're running this very obsolete language, a hunt. So they have all these controllers, like, you know, the sensor reads this, then flip the switch, blah, blah, blah. So they had a high-level macro language, which changed over time, which they wrote these in. Then they generated some low-level code. Then they changed the low-level code, which means the high-level code is no longer relevant. Now it's 50 years later, and the hardware is failing at an increasing rate. So we've got to switch these over before that happens. And now you have this giant gob of goo, and you need to figure out what it does. So because you know that this low-level code has been generated from these high-level libraries,

even though they change over time and then modified, you can do smart things to try to recover, to try to do this clever decompilation, discover what high-level operation things correspond to, and then translate it into high-level form, and then from there, translate to Python. And they've been doing that, and they're already ported. I don't know what the number is, but now it's probably over 100 chemical plants.

So you have a tool that... I remember this specific line that said, programs that write programs that write programs. It took me a little bit to understand this, actually. You've built a tool that allows for cross-language refactorings. Is that... Okay. So you're mixing up several different things there.

So my research interests... So I was talking about tools like the thingies or diacomical plants. Yeah. And my goal is to make software maintenance better. Part of that is making tools like that better and more common. Tools like the diacomical plant thingy. But there already are a lot of people in software engineering research who do know how to build those tools to give or take sense. But a reason why those aren't very common is that they're all very specific and hard to build. So for instance, if you had a program

that did all its calculations in Fahrenheit, and you wanted to change the code to use Celsius internally rather than Fahrenheit, that might be a very tough change. And even if you could pay a million dollars to an expert for like $100,000 to an expert and give it the right tool, it doesn't. You probably wouldn't. Even if there's 100 people or 1,000 people else in the world doing the same thing, you couldn't find them. And their tool would probably

only be a formal language. And it probably has to rewrite it even from like Scala to Java, even for a similar language. So I think that by reducing the cost of making these tools, we can see a much larger number in an advanced whole

program refactoring things built. So that is my goal in research. And so I'm interested in finding better ways to build these kinds of tools to make them more general and to automate parts of building them. And since I'm interested in automating parts of building these tools that has the cheeky thing, program that's the right programs are right programs. And this is cubics is your framework yeah so cubics it's a stretch but cubics and literally under the tagline programs that

right programs are right programs but the paper i just submitted to popple is on system called mandates which is much more fitting with that moniker goal so what is the end goal here in terms of software maintenance the big dream is that if you wanted to write your thing, which takes a program that does stuff with Fahrenheit and changes its internal code to 7 Celsius, then I should be able to describe it at a pretty high level and then get that tool. And then I should be able to only make it change a few lines

and then get that tool for Java and then get that tool for C and then get that tool for Rust. That makes sense. So the reason it's programs that write programs that write programs is the top level program that you're working on could be fed a program about how to change program in a loose sense, change Fahrenheit to Celsius, and then it will generate programs for a myriad of languages that can do that source transformation to those languages. So I understand you're doing a training course on some of the principles we've gone over today?

Yes. My passion and livelihood is on teaching a lot of the things I talked about with Adam earlier about how to see the logic of your programs and write things in ways that are easy to read and will also need less time debugging and will be easier to add features to, except in a much more concrete way with a lot more examples, a lot of practice, with very good feedback on the practice. That's my Advanced Software Design Webforce. The next run is starting on August 23rd. We'll be taking 15 people. And as a special bonus for all of Adam's listeners. There's a freebie of extra one-to-one time with me. And to get that, just go to www.jamescouplecoaching.com slash Adam.

I even got my name in there. That's awesome. One other question I wanted to ask you. I'm looking into you preparing for this interview. So you currently are at MIT working on your PhD, to my understanding. Yes. But according to the internet, Peter Thiel paid you to not go to university? Yep. So could you describe the program you were a part of and how it was? Yeah. So I was part of the Thiel 20-year-old Intern 20-year-old Fellowship.

I entered in 2012 and got $100,000 over two years. So at the time, I had been a junior at Cargamelon, but I did not have a senior year. I just left. Fortunately, I still graduated with two degrees because I had worked really hard in those three years. I was planning to do a master's, but I dropped out. So for two years, I worked. Now, for one year and a half, I worked on a company trying to write a program that fixed bugs in other programs. And I've talked more about that elsewhere, such as in my interview with Future of Coding. But that wound up not going so well, both because the technology was still in its infancy, because I was not an expert in the technology at the time.

I was learning as I went. And also because I learned the hard way that a good product idea is not a good business idea. Whoever tells you that good business ideas are a dime a dozen, it's totally wrong. Oh, interesting. How so? Good product ideas are a dime a dozen. Good product ideas, that's where you can find the audience, where you can convince the buyer the value that it's protected against competition, where you can market it, where you can hire

people to work on it, where you can sell add-ons and grow. When you add in all those other factors, you just turn a product into a business idea. They become very few and far between. That's interesting because you've kind of carved out a distinction for me that I haven't thought of, which is that people's business ideas are often just, hey, whatever, I built a better mousetrap or something, but that's not really a business idea. That's just a product idea. So did you find the program, your two years, useful?

Was it a good experience? Oh yeah, very. So it certainly laid the foundations for what I'm doing now and that it was from the process building this company that I gained the insight into realizing I need to go one meta level up into making these kinds of tools easier to build, but also managed to become very connected to the industrial side of things, which is why I'm able to do what I do now in my teaching about software

design are very much bridging practice and theory. Yeah. You don't see many people who are working in academia also doing kind of a nuts and bolts advanced software design course. You don't see many people anywhere doing advanced software design courses. In fact, there are zero books written today that I've found that I've looked that I would call an advanced software design book. There's a large number of beginner books, software design, and a small handful of intermediate level ones, but none that i'd call advanced that's

because no one knows how to teach it and i'm working to change that that's interesting you had this post called on your blog called um it was about benjamin franklin method for programming books yes could you explain that yeah so the whole genre of programming books as distinguished from computer science books, usually they're like Postgres 7 or writing games on iOS or intro to Rails. And a core feature of them is they have a lot of code snippets. They're like walking you through these examples. And then what do you do with these code snippets? Do you type them in? Do you download them from the book's website? How do you learn from these code snippets? Do you type them in? Do you download them from the book's website? How do you learn from these code snippets? So from multiple sources, I learned about the Benjamin Franklin method,

which is he wrote his autobiography, this method he had for getting better at writing without a teacher. He found these articles that he liked that were totally better than his, and he didn't understand how we could produce articles like them. So he would take notes in these articles, compress them into a few sentences. A few days later, he'd come back, look at his descriptions, and try to reproduce an article like it from the descriptions. And then that would give him a laser eye to focus on all the little tricks, the rhetorical flares that the professional writers did that made their articles better than his. And so he learned. And that's what a lot of artists in other fields do. So like visual artists might try to copy a painting, and in doing so will discover all the techniques used by the original painter. And so I realized that a modification of that idea can be applied very easily to reading programming books. The most faithful version would be to try to write a description of what this program does that you have a snippet for, come back a few days later, and then try to reproduce it. And you can do that. The feedback cycle is kind of slow, and you might not actually want to care about reproducing the exact same

program. It's not as important to wait a few days to create everything you knew before. You can get a very cheap version of this, which is very effective just by looking at the page, flipping over, looking away, and then trying to write the program. And even though that might only add a few seconds versus copying from the page, in those few seconds, you'll forget so much about the syntax unless you can compress it to a better form. And that gives you more learning. Yeah, I like this method. I haven't tried it, but I think that we've read the same book because i saw reference to uh andrews erickson i think the deliberate practice guy yeah so my summer in texas when i was at semantic designs i had a car for the first time since i'm west california so

listening to a lot of audiobooks and so within a short period of time i read both andrews erickson peak and also josh waiskin the art Ericsson, Peak, and also Josh Waiskin, The Art of Learning, both incredible books. And also Mindsight by Daniel Ecker. And I think all three of those, or at least two of the three, all mentioned this technique. Yeah, I love it. So, I haven't tried it, but when I saw you reference it, I was like, that totally makes sense. So, Anders Ericsson, right, is the deliberate practice guy. I think we touched on this a little before. He spent a lot of time studying how people become experts. So, he has a number of techniques in his book for working in a field that's less defined, right? Because like he studied chess, right? And in chess, there's like

very defined progression. I think violin as well. But as you were saying, like, okay, here we are, software development, there's no advanced books, right? Everything's surface level, how to learn X. So another technique that he had that I liked was he called it like the Top Gun method from like the Top Gun school. Do you remember this technique? I remember him talking about Top Gun Academy.

I don't remember the generalized version. So the idea was like, I think it was the Navy pilots were getting basically schooled by the Russians at some point. And if I murder this story, you know, write me a letter, listeners. But this is my approximation. So they knew they were failing at whatever dogfighting, right? But they didn't know how to get better. So they started this Top Gun school where they just fought against each other. So they didn't have a teacher, but they managed to, as a group, teach themselves how to be better by competing against each other. I've often thought about this in some ways to do with how a team could get better at software design by just critiquing each other.

That was basically their approach, except much more adversarial, like they're pretending to shoot each other down. But they managed to learn by feedback. And I think it kind of boils down to a bigger thing, which is very often the way to get better is to recognize the difference between practice and performance, between just doing something versus probably trying to get better. And so all too often in these kinds of fields, the way to get ahead is simply to try to get ahead. Like other people are just programming, but not really deliberately focusing on what they did and what they can be doing better. Then that's all you need.

I want to share one story that's similar to the Black Academy, but more programming related. An old story of programming folklore, but not super well known. And that is a story of the IBM Black team. I think this is 60s, 70s. IBM had many teams with their testers. So they decided just to take the top 10% of testers and put them on their own team. And they thought, okay, so maybe this team will be like 25% better

than the other testing teams. And that's not what happened. They actually became 10x better than the other testing teams. And that's now you have a whole group of people who are more motivated, and they form their personalities around being good at breaking other people's software, and they fed off each other. So they would learn all these things and share them with each other at lunch. And pretty soon they took to dressing all black and then wearing mustaches that they would twirl as they tried to break someone's piece of software. That's awesome. So you mentioned that you think that practice is different than performance. How so? There's something which is talked about by Kay and Roserickson a lot.

So you use something that you're good at, like maybe running or sailing or playing instruments or certain kinds of programming. And you get into the zone, it feels pleasurable. It's really comfortable. You just please how much stuff you can do. That is performance. That is not practice. You know you're much stuff you can do. That is performance. That is not practice. You know you're growing because you're pushing yourself to your limits because you're stretching your brain because it's uncomfortable.

When we talk about software design, I think a standard idea is you just have to build a bunch of software, right? If you get enough experience under your belt, then you'll be good at software design. Yeah. What do you think of that? If you're writing software, you can't help but occasionally doing things you've never done before. under your belt, then you'll be good at software design. Yeah. What do you think of that? If you're writing software, you can't help but occasionally doing things you've never done before. But there's nothing deliberate about that. You've heard the line forever, practice makes perfect.

Maybe you've heard the more modern line, perfect practice makes perfect. Practice does not make perfect. Perfect practice makes perfect. You can find issue with that line as well, but it's closer to the truth. And so similar in software, just doing it does not necessarily make you better for doing the same things over and over. You need to do it better, do things you haven't done before. What does practice look like? If I want to know how to be better at designing software systems, how do I practice? So let's do a few kind of intuitive things. One is I teach people about ways that they can lock down their design,

not only make invalid states unrepresentable, but also make it so the valid states can only be represented in one way. No junk, no fluff. So you learn all these techniques, you know, it becomes much, it becomes much harder to write the wrong program, easier to write the right program. And then they go and take it too far and try to develop all these really elaborate type stems that force you to do things a very certain way or otherwise everything will break. They write 20,000 different types.

And they're taking it too far? Yeah, totally. And I say, go for it. Everyone needs to go through that phase of taking it too far before they learn how to do it naturally. Interesting. So when I learn something, I should take that thing and really see how far I can stretch that piece of that concept. Not necessarily, but you have to run towards things that are hard and difficult when you do W4,

not run away from them. That doesn't always mean you check that code into production, but you have to experience it. Yeah, I think that I could learn a lot about how databases work by building my own toy database system, but I should never actually use it. It's more of a learning experience.

And that's the case. Even if there are specific tasks where you think you can write something that's better in Postgres for a very specific purpose, maybe your program can be better if you do that, it's very unlikely you'll actually encounter a case where doing that is actually a good trade-off.

So you might never learn, you might never get a chance to exercise at Skull in the Wild. That's what I'm saying, the practice, not the performance. So you can't always lean on saying, if I do this enough, I'll get better at everything I want to get better at.

You have to set a time, side time saying, I'm going to do this thing. This is practice. This is not going to produce an artifact I can use. It's just to help me get better. And I'm okay with that. So would you advocate building a web scraper over and over and over again, like different ways, different languages, abusing type systems, having no type systems, doing it lazily, doing it like, I don't know, is that a valid way to practice? You will certainly learn more doing that than if you just wrote different web scrapers the same way over and over. I'm just, I'm grasping at how we can

practice. I feel like, okay, if you're a baseball player, it's pretty clear the difference between practice and a game, but I don't know how clear it is as a software developer. Yeah, I don't either. But the beauty is that because there's such a weak concept of practice, that means you do anything, you find any of these things are questionable practice, even if it's questionable how good they are, and you're still doing better than everyone else. Because nobody else practices. They're all just playing the game. So any practice... Yes. And this is why I say that there are a lot of good programmers alive today, a lot of master programmers, but I don't think anyone alive today deserves a title Grandmaster Programmer. Interesting. Yeah, I wish there was, right? If you want to become a chess prodigy, I think there's a certain ladder that you need to follow, right? At some point, it involves working with a Grandmaster and spending all of your waking hours doing things. But they know what those things are.

I don't feel we do know what those things are. That's not a question. I'm just thinking out loud here. You got my gears turning. Well, I think that I went through basically every question I had for you at this juncture. So this has been a lot of fun. Yeah. So thank you so much for joining me, Jimmy. It's been great. You too. That was the interview. I think Jimmy's concept of the logical level in software is one I'm going to keep thinking about for quite a while. Programs, the right programs, the right programs. Jimmy is coming at things from a whole different level.

If you're listening to this now, say hello. I'd love to hear what you thought of the episode. I know I could have asked more questions about program synthesis, but I didn't wanna miss Jimmy's insights on software design. Thanks to everyone in the Slack channel as always. And thanks to those leaving comments on the website eli and chris bunda in particular the website comments go into a moderation queue if

they have a link but should otherwise appear right away if i missed some link that should have been in the show notes for an episode dropping it via a blog comment is a great idea tim heaney did this for the cory doctorow episode and that was great. Tim has been requesting some Closure episodes, so I think I should probably get on that. Until next time, thank you so much for listening.