Programming Throwdown - 179: Project Planning

Episode Date: February 3, 2025

Intro topic: Lego event space & retail store: https://www.instagram.com/bambeecave News/Links:StackOverflow Question Count Going Down https://gist.github.com/hopeseekr/f522e380e35745bd5bdc3269a9f0b132DeepSeek claims its ‘reasoning’ model beats OpenAI’s o1 on certain benchmarkshttps://techcrunch.com/2025/01/20/deepseek-claims-its-reasoning-model-beats-openais-o1-on-certain-benchmarks/ Computer Science Papers Every Developer Should Readhttps://newsletter.techworld-with-milan.com/p/computer-science-papers-every-developerNvidia Cosmos - an AI platform to change the future of robots and cars - wins Best of CES 2025https://www.zdnet.com/article/nvidia-signs-largest-car-maker-toyota-to-use-its-self-driving-tech/ Book of the ShowPatrick: Alice’s Adventures in a differentiable wonderlandhttps://www.sscardapane.it/alice-book/Jason: A Beautiful Day in the Neighborhood (Hulu/Netflix/etc)Patreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the ShowPatrick: Digseumhttps://store.steampowered.com/app/3361470/Digseum/Jason: Sqlitedict - Python dictionaries saved to diskTopic: Project Planning and ManagementWhy?Gathering feedbackIdentifying risksDeciding future headcountDocumenting / discovering dependenciesCritical pathScheduleReduce the bullwhip effectHow it worksSMART goalsspecific, measurable, achievable, relevant, and time-boundMT is most importantGantt ChartsScrumAgileKanbanToolsWhiteboard (the generic IRL one)Post it notesJIRAAsanaOpenProjectDealing with uncertaintyBufferingIssues with recursive paddingProject planning Post-Mortems ★ Support this podcast on Patreon ★

programming throwdown episode 179 project planning take it away jason hey everybody um it is so fun to to actually uh do these duo shows i have to say i think the interviews were super lucky that we could get such a like multi-disciplinary set of talented people to do interviews and i appreciate everybody you know reaching out we get many many requests to interview yeah we do we get a lot yeah we get we get i get a ton in my personal email too that you don't even see oh you get a lot in your personal email nope yeah see mine's too available um so we really appreciate it i mean we're honored that so many people want to be on the show but but uh it's just really fun just the two of us talking about tech stuff so we're gonna keep it like this for a little while um and i will kick us off

with a store that we opened so or i say by we now i'm talking about my wife and i and really by we i mean my wife open i i did a lot of the manual labor but um um but we opened a, it's called Bricks and Minifigs. It's a franchise and it's basically an event space and a retail space. So on the retail side, they sell Lego, they sell like bricks and minifigs as suggested, like sets and all that they also have like a thing where you can build your own minifigure and then it's also kind of like a clearinghouse for lego i didn't realize this was a thing but but people will want to sell their lego right which makes sense after they built it and played with it the thing that surprised me was there's a huge audience of people who want to buy already built lego whoa yeah that i did not know that not expecting and and you know now the store is up you uh see it every day like i'll be in the store building some furniture or something and someone will uh say like oh that would look really good on our coffee table and they just buy it as if they're at like an art gallery or something um and and a lot of these people

actually don't want to build it they actually just just want to have it. So I'm really surprised at that. Now, the other part of the business, which is honestly the part that really I like the most is the event space. So we do birthday parties, all different other corporate events and stuff, and people build Lego. We have actually this cool racetrack where there's like a metal car, like a chassis, a metal chassis of a car that has the Lego studs on it. And so you build a car and try and race it down the track, uh,

kind of Pinewood Derby style. Nice. Um, and, uh, I guess, you know, I mean,

beyond just, uh, shilling the store for, for the, for the 1% of listeners who happen to be in texas um you know i want to talk a little bit about career breaks so you know i um i basically took a career break and um i actually you know did a lot of side projects and things like that but i also helped uh my wife kind of get the store off the ground and um it was

awesome i mean i know that this is like a uh um thing that might not be accessible to everybody but you know if you could take you know a short break and especially if you've been in the industry you know 10 15 20 years um you know it's uh it's really nice to just you know instead of i was always a type of person where if i decided i wanted to change jobs you know i would still keep like working you know really hard at my current job until i found the next thing and then i would leave like a one day gap or something between the first job and the second job um and you know this time like having a longer gap is uh really really nice especially if you've been in the industry for a long time i highly recommend it people talk about uh i guess maybe it used to be more common maybe it was never more

common i don't really know but sabbaticals um yeah that's i don't really know like what what goes into a sabbatical but But first of all, congratulations. Tough thing to do. So good luck. Thanks. I won't quote all the statistics about new businesses to you, but I'm just teasing. It helps that there's not like a default. Like, what is it?

Who's the person who... Paul Graham. He says, says you know startups are default dead until they're profitable but in our case like we don't really have to be profitable um it's just really a fun thing for the community so i think that helps our survival rate there you go uh non unintentional non-profit um no i'm sorry i just teased no no best wishes best wishes um but yeah i mean i think like taking a career break change of things is is actually like a good advice and there's a lot of new ones not to derail like into this for for very long but just if you're planning to leave

a position just look at like what the rules are around most places your insurance coverage is a concern so people don't want gaps because of insurance but uh in a lot of places once the first of the month hits your insurance is basically paid for the month by your company and your premium so if you stop on the second of the month or whatever you're actually covered for the rest of the month and so you could take you know a four-week break basically and you know have insurance coverage without any any gap now again you should please like read that that's not something you really ask your manager right because then you're kind of admitting you're yeah uh leaving but for something that if it's helpful to plan for and plan around it can ease some of the stuff that that make it a little more accessible i would say to take a little bit of a gap. And then you can also,

in many positions in tech, you can negotiate a signing bonus to help cover that. And that's not necessarily always obvious or offered, but you can always ask for, hey, I'm going to take this, I'm convincing my spouse or whatever, I need a little bit of help to take a vacation in between. Say whatever excuse helps it in your head if negotiating is hard for you to try to ask for a little bit more of a signing bonus and your negotiating power starting the new job. But I do agree taking time to do something off. People talk about online. I've never done it. I kind of envy it, but it's like an early retirement, like a a mini retirement we kind of wait till the very end of our careers and then we retire and never work again that's at least the their sort of stereotypical thing there's no reason

why you couldn't save money take a year then return to work and for many people um returning isn't that hard after a year you It is very situational and person dependent. And compounding of money does make there a slight problem to the hand wavy. It doesn't make a difference. But if it allows you to come back to work with new freshness, new eyes and new vigor and energy, I would argue there's compounding returns from that as well in your job performance and how people are happy so yeah i think taking a break is is great advice yeah totally i would say a couple of things based on what you said one is um there's this thing called cobra which is probably just an american thing but there's probably equivalents in other parts of the world where you get your employer's insurance for an extra year after you've left. Now you have to

pay for it, but at least you don't have to change all your doctors and all of that stuff. So that is an option. It's also kind of interesting. You have the option of retroactively getting COBRA. So for example, you could say like oh you know i'm just going to go without insurance for six months and then oh you break your arm you can actually sign up for cobra like after they've fixed up your arm and everything and they'll retroactively like recalculate all of your medical expenses which is pretty wild um i mean it's actually kind of remarkable just from a tech standpoint that they can do that um and then the other thing i was going to say um oh yeah you know i think it's important to have some kind of a goal uh you know i actually told people when i was leaving uh my last job

they said oh what are you doing next and i just i had it ready to go i was like here's what i'm doing i'm gonna help my wife build a retail store and uh and so it's like i think having a plan um you know helped a lot i i feel like uh um if you if you just uh you know veg out for a while that's fine that's a different kind of career break than saying i'm going to like hike a mountain or travel the world or whatever and if you're going to have some kind of focus then then you should plan it all beforehand because you don't want to waste your time not getting paid planning you know that's that's time where that you could you could be doing that on the nights and weekends while you work your job you know you're not working the nights and weekends you know do you i i almost never work weekends unless things are really wrong i will work a lot

during the week but i will not work a weekend unless i absolutely have to yeah unless it's just like a really quick statement or whatever by default if someone is saying i anticipating that i need to do it then they've they've needed incorrectly i guess is yeah i try and i try to do that for you know the people that i help on the teams and stuff as well it's like we shouldn't plan that the work needs you know more than the normal working hours yeah i don't know if you feel this way but i get more and more energy as i work so like by 5 p.m like i'm really really excited about work and And then by, let's say, 8 a.m. the next day, it's like I have to start the train again. And I've been this way my whole life. It's not an age thing. Even when I was 20 years old working, I still had this mentality.

And so it's like, I would rather work a ton of hours on Friday than come in on Saturday and work for four hours and get pretty much nothing done. So, you know, it's like, it's like, you know, that, that, uh, um, I feel like that works a lot better. No, I don't share that trait. Um, yeah, but I, but I know, I think everyone's different and I think, yeah, I, we can get into that when we talk about planning for projects, but I think different people have different ways of into that when we talk about planning for projects. That's true. People have different ways of reacting to things. All right. Well, I think it's time for the news.

Oops, sorry. All right. What is your news, Patrick? My first topic was an interesting, I don't even know, these show up on social media. Sometimes you see them on X or on Reddit or wherever, and someone just posts this and a conversation spins up and everyone's talking about it. And anyways, this one was, someone had posted a GitHub gist

where they put some numbers down and I can't speak to the veracity, the sort of correctness of these numbers, but they were saying that they were a big contributor on Stack Overflow and that they were posting these numbers to say that you know stack overflow number of new questions has been going down it of course uh you know sparked an interesting

online discussion around it and so i just bring it up here because i think it is interesting on the one hand i don't know have you ever jason posted a question on Stack Overflow? Oh man, that is itself a great question. I've literally never done that. I've answered questions, but I'm definitely not like a top answer or anything. I've literally never posted a question. I haven't either, but I have read enough stories. Like one of these things, I guess,

like in my head, I've built it up to be a thing where I don't think there's a lot of low effort questions. They get closed. And I think over time, Stack Overflow has gone through curation to try to, you know, get good quality questions asked. And so I think part of it, and you hear the same thing about Wikipedia. I've never edited a Wikipedia article, largely for what I would say is kind of the same reason you read stories about people who talk about the, you know, over process and the context

around what's expected of you if you're going to and i just yeah uh quick quick side note i tried to create a wikipedia page for the show and uh it got rejected this was like 12 years ago or something because we weren't noteworthy or what was yeah they didn't at the time we didn't have enough news stories about us at the time we had zero i probably could make one now but to your point it's like once you get rejected like i put in a lot of time into that yeah and then it's just gone they i think uh i think they literally delete it to where like you just can't even get it anymore and so it's like who wants to work on wikipedia after that uh yeah a bit of an

aside and i'm thankful that there are people that do and they become committed to the idea of i'm going to do this it's like an iterative thing right you get shut down they submit again they try something different they're persistent and once you get over that hurdle i do believe like it can be very valuable for you to continue sort of writing articles submitting things but if you're just seeing it as like a very transactional i want to do this one time i agree i think the bar in my head at least gets to where i i don't attempt it um but i will but but that's not where the conversation went although i i think we should be aware of that the conversation immediately jumped on not not

surprisingly to everyone's just asking their questions to ai instead um and so there were some points and counterpoints which is a very interesting discussion about stack overflow uh profit has actually stayed up in part because um they've been able to license all of these questions and answers to ai training um and so in some ways the sort of questions have gone on to those platforms because more than just a search, they've sort of been able to aggregate and combine all of the information there into the LLMs that you can then interact with and sort of be more responsive and ask your questions. And then not to get sort of conspiracy,

but you get kind of the dead internet theory. Then the problem is, okay, but all of the questions about new languages and this, how do we get that information out on to, to Stack Overflow? And so just a very interesting discussion. I don't have a sort of strong held opinion or belief here, but just as the internet sort of rolls forward, we, I, I've benefited many times from going on Stack Overflow and sort of seeing questions and response style. it fits very naturally with the kind of work that I do. And so it's very useful.

I personally haven't shifted a ton of that over to sort of artificial intelligence, but maybe I will in the future. But then what about the questions about the next libraries and all of that that never get posted on Stack Overflow? You know, yeah, over time, does it naturally sort of fade into the sunset and we

you know just go other places i i mean probably it's a natural course but an interesting discussion nonetheless yeah i mean i think to your dead internet theory point it's like when when you when you ask an ai a question it might be a question that the ai's never thought of before seen before but it can piece together an answer from you know all the other questions it's seen the problem is really that that answer is not in stack overflow and so like you're not growing the collective consciousness if the ai just like tells you the answer and then it's deleted so maybe the ai's need a blog where like they when they have these novel thoughts they're sort of aggregated collected and then published and then the ais can help the other

ais grow and it can be no that's not gonna happen yeah no you know what's what's funny is actually when you were saying that i was like yes yes that's right i think that actually is where it'll go i think i think the ai will uh you could do some kind of like novelty detection it's like oh this is a novel question then we'll post the answer the question and the answer maybe on stack overflow so stack overflow have a bunch of questions and answers from ai and then people can go and correct it and all that yeah it'll be i use ai a ton actually the other day um i wanted to do this is a little weird well hard to explain but i wanted to do like 2000 gaussian mixture models in parallel and so i wanted to vectorize it so that i would without like a for loop because one thing about machine learning stuff or just you know in general these kind of

things like for loops kill your performance right so so if you do like 4x from zero to a thousand you know a plus equals x or a plus equals i, right? So you do that to sum an array, or you just use like some sum operator on the whole array. The second one's going to be like hundreds of times faster, right? And so I needed a totally vectorized way to do a bunch of Gaussian mixture models. And I asked AI, and I'm fairly confident nobody's asked ai to do this and it produced an answer and it was actually totally correct and so that's the kind of thing that like probably should have somehow made its way into stack overflow yeah i had an opposite experience i asked a question that it gave me an answer which was obviously wrong wrote some sample code wrote all this stuff and

then i was just like and i'd even just told i was just i don't know whatever like that's not right and it's like you're correct i did get it wrong actually and then it proceeded to write something else so it's actually still incorrect but at least like closer and less obviously wrong and i found this like you know sort of you know diverging off your thing you know i i it was a very weird mental thing for me to be like by simply telling it it was wrong it's like if you had thought longer would you have given me the second response like couldn't you just tell yourself like are you sure it just is very strange so it's sort of very different than a you know going on to google and searching for a blog post or a stack overflowflow. And so, yeah, I still not

long all of the, you know, asking AI the questions, but I do think it's getting better. Well, that is a great segue to my news story, which is DeepSeek claims its reasoning model beats OpenAI's O1 on certain benchmarks benchmarks and so i have an article linked to tech crunch here but you know what you said is exactly right and so you can imagine like imagine a system where every time you gave it a question it gave you an answer and then you said are you sure that answer looks kind of wrong and it went through and gives another answer um now let's assume that you know if you kind of like tell it okay so let's assume that creating an answer out of the ether versus creating an answer based off a past answer let's assume that the latter is better than the former on

aggregate. So if you have the benefit of this earlier answer and all the computation that went into it, let's assume that you could write a better answer the second time, right? So now that's really interesting because you can actually create training data for yourself

as an AI. So an AI could spend twice as long answering a question and then go back and say hey that that answer that took twice as long like i want that answer right away and so it can fix itself like self-healing kind of stuff and and so this is not a new idea like you see this with games so So for example, look at AlphaGo, right? So AlphaGo will use tree search and like simulate future boards, right? And so it might think, oh, this move is good and this move is good. But then it does a bunch of simulations and it finds, oh, only that first move is good.

The other one's trash. Well, like that's feedback to that first system that has to make that snap judgment, right? And so this is taking that idea and bringing it into the large language model space. And so a really interesting idea. Now, the challenge becomes credit assignment. So there are times where you say like hey think about this more and it does and the answer is worse or it's like more it's like hallucinates even more and stuff so you

have to craft uh you still need to guide the system um as long you know as long as you're making this assumption that just like every second answer is better you know if you want to relax that or push back on that now you have to guide the system using reinforcement learning as other tricks but um but so this so opening eyes been working on this for a while um the thing that makes this really interesting is the the deep seek model comes from a quant finance research lab in china that i think they were i get this story correctly you know they were like doing quant stuff and i'm not a finance guy patrick you're probably gonna like rip on me if this but i guess it's like you're buying and selling things at certain times or i don't know what they're doing so so then all of a sudden they like had to take

a break from that. There was some issue that there was some crackdown or something. And so they started working on some of these other models. And they produced this amazing reasoning model that's completely free, like source code available. The only thing that's not available is the data they use to train the model, is which is not uncommon i think most of these models have that uh closed data um but the model itself the weights are free the source code's free you can fine tune it you can do all these things and so uh it's really kind of revolutionary we'll have to see where it goes

this speaking of credit i've lost uh whose idea this was, not mine, but someone was pointing this out and I've been intrigued. So you mentioned sort of like AlphaGo. And I think one of the things that was really interesting is initially AlphaGo was trained with sort of, you know, all of the Go games that this is the guiding you're talking about there's a framework for winning and losing better and worse um which is sort of different from all human knowledge i guess but the and they kind of later said oh actually we've removed all of those games started again and bootstrapped with just playing each other and just like figuring it all out from scratch without without that anchoring and the the, I don't know, thought experiment, the idea people were talking about as like, I think we've sort of moved the goalposts, given up the sort of like, how do we determine if an AI is sort of like AGI or if it's able to fool a human,

like everything sort of shifts because things kind of became in a way that is different than we kind of thought they would. And so this discussion has been lost. But one of the things someone was saying as like a sort of marker would be right now we feed tons and tons of data, all of Wikipedia, all of Stack Overflow, all of the internet in as this training data. If instead you sort of went back and said, let's take all of the books and newspapers and research up to, you know, sort of 1900 and then sort of ask questions like

the phenomena were often known for sort of quantum physics or for general relativity like everything was written where you could via sort of thought experiment come up with a lot of these ideas and theories and show that the math worked and later we sort of back explained things that were happening that either were misascribed or that we didn't know. So could you give one of these models like everything up to end the date, you know, we kind of argue it, but where you have some of these experimental results, you know, kind of lock it and ask it to thought experiment forward sort of physics frameworks. And would you get quantum mechanics? Would you get Einstein's theory of general relativity? Would they be able to sort of self reinforce, right?

And you kind of need almost like what you're talking about, where they would write their own papers, then they would sort of incorporate those learnings into other learnings, maybe an agentic AI where people are taking different things, so they can kind of bounce off each other, whatever the solution is. But I don't think we're there today, for two reasons, like it seems still all of these models require just gobs and gobs of training data not very small corpuses all of the written recorded data up to 1900 is a very tiny fraction of what goes into these today um and then second of all i don't think there's this ability and i maybe we're getting there with what you're saying but to kind

of roll forward to let these things sort of have their own simulated society and like roll it forward and discover these things and learn them and understand them um and so of course it's even further to say they could run their own experiments i you know maybe that's that's a lot harder and out of reach today but what i'm describing just sort of via thought experiments and sort of deep introspective thinking you know and then later it gets proven out by experiment right but it feels like you could do that if you had the right architecture system training but we're not there today yeah yeah i think you're right um yeah i i i think that with a lot of these things it'll end up like it really multiplies the hallucination problem. It's like, oh, I see there's a kernel of truth here that one plus one equals and so um what we're missing i think the big foundational thing we're missing is

like we have kind of really strong models in our brain of physics that's how people can like hit a baseball out flying 100 miles an hour right it's like we have these like at a really low level it's like primary visual cortex we have something similar i'm sure for audio and for physics all these things and they are evolved so the training data goes back millions of years right and so that's i think those kind of simulators you know we don't really know how to plug that into an AI. We're just starting with very basic things. The AI will decide, oh, what's left here is an algebra problem. I can use Wolfram Alpha. But that's very early days in using tools and having simulators that aren't... Here's a very simple

experiment you can run to prove this to you. If you take a picture of of uh here's a very simple experiment you can run to prove this to you if you take a picture of a maze even a very simple maze that like any child could solve and you tell an llm to draw like an ascii art version of the maze like it won't be able to do that it will completely fail um and and i think the reason for that is um is that you need this kind of simulator where you kind of as you're drawing you're then kind of going back and like you have this like concept of spatial that's kind of independent of the language model um now these are all things that are coming but they're not there yet oh yeah okay that's really interesting i have a yeah a couple follow up thoughts but we we won't get stuck here forever or we'll never cover anything else. But the first one is, I think, and again, not original to me, I think Jan Lekun talks about this in some of his interviews, but the sort generation stuff rather than the llm stuff they end up needing an understanding like you know i have a picture of a guy holding a ball

and i ask you to generate a video of him dropping it right well how far should the ball drop each frame requires some understanding about physics about world money what if i throw it up right like hey the a pitcher is throwing a baseball are you going to draw in a sort of parabola right like this up and down that physics is well described but it doesn't actually it has gobs of data about people throwing baseballs but it's never been given the equation for throwing a baseball you know that it needs to follow a parabola half spin you know this kind of stuff is really interesting um but then more directly into to the deep seek thing i guess is i did see people posting some of the you know interactions with

deep seek and what you're talking about the infamous how many r's are in strawberry so somebody asked deep seek you know how many how many r's are in strawberry and it gets stuck in this arguing with itself loop like it's like well let's list out the letters s t r oh there's one a w it gets to the second r r gets to the third r it's like uh-oh there's a mistake i know they're supposed to be two r's but i just counted three uh wait a minute let's try it again and so like it goes again and it tries to go again and again so like it keeps getting to this like you said a kernel like it's doing the correct thing it knows how to count knows how is arguable but it like it's doing counting which is correct but then it also somehow like has this

belief that the answer is two which you know several llms seem to have gotten stuck with this for whatever reason source grounded that strawberry has two r's and because of that like it sort of goes wait a minute i must have miscounted i must have done an algorithm i wasn't supposed to do so then it tries you know variance on counting basically keeps coming up with the answer three and going but that's wrong and so it's just this very fascinating like clearly the answer is three you could ask anyone but like it it kind of can't do it it's just it's just it's like oh it just killed over and died yeah yeah totally yeah i mean this is like it needs like a roving eye that can like rove like over all the different letters um and and it needs to be able to do that kind of explicitly instead of implicitly and we're just not there yet

all right well my my next uh news article is not directly in this uh but but it is a bit on topic i guess and that is a list of computer science papers every developer should read i'm apologizing i won't attempt to pronounce the author's name dr well i guess i'll try to say the last name do you are you able to say this jason or you want me to go? Milan Milanovic. Okay. Yeah, I was going to say the same thing. Dr. Milanovic. And he wrote this blog post. The link's in the show notes. It's a lengthy link.

And the papers here listed, I don't have strong opinions about these specific papers. But he kind of goes into a defense of why developers should read academic papers now i guess this is going to expose a bit of a difference between jason's practice of computer science and my practice of computer science but in the kind of work that i do with largely embedded folks you know application programming reading academic papers is a it doesn't happen um most folks probably didn't do it in university they don't do it now they it's just a foreign thing and we've talked about this on on the show from time to time but i guess i will say that i think it can be a bit people overuse it but a bit of a superpower if you're in a role where no one reads academic

papers and you're willing to at least try to read academic papers when when the need arises going around and saying oh there's this new research paper that says you know this and we're going to do it probably not the right answer but saying oh hey people must have had this problem before people must have you know worked on this issue and just doing a literature search is kind of what i would call it, often turns up via academic papers or whatever. And you would be floored at the number of people who won't just search for something. Hey, I'm stuck on this hard problem. I need an optimization of this nature. And they just won't sort of, well, we're just talking about AI, maybe that'll be the future, but they won't just shove it into Google and try to like spend a few minutes sifting through this.

It's not going to give you a Stack Over stack overflow answer but you might find a paper where someone has looked at that hey optimizations uh cash efficiency of various hash map styles whatever it might be and so i will agree with with the doctor here say oh i guess we don't say it that way because that makes it sound like a medical doctor anyways but the author of this blog post uh and sort of say that i think you should try to read a few papers um i will be honest whenever i read a paper 90 of it is is it just like one year out the other like i either can't decipher what they're saying or can't understand it but even just reading a few of it looking at the pictures start to pick it up if you do one or two of those you will start to glean the gist and then either

going and giving you the next thing to go search or writing it it really is for certain problems just such a like 10x ability to to kind of like execute on it and i'm not talking about papers that were written this year or last year in this journal i'm talking about like 1980s you know the you know 2000s that there were these papers written and you'll find them and it's just like wow that's yeah this is this is this is great this is exactly what i needed or gives somebody a you know like hey there's backup that i'm not crazy doing this like this is a well thought through thing and in some corners of the universe yeah totally um so yeah i mean i've read probably two to three papers a week for 25 years so i've

read a lot of papers um i've actually read most of the papers you posted which i feel like is hey good for you considering like it's not these aren't really my fields per se uh actually this out of the tar pit sounds really interesting um uh folks can check all the show notes on the whatever app you're using to listen to the show the notes should be there or you can go to programming throwdown.com but yeah these papers are amazing like mapperdu's big table kafka um i will say to read a paper well typically it's good to you know you read the abstract because it's the first thing um and then i go straight to um uh well i'll read the introduction if i'm not already motivated you know the introduction is there to kind of motivate the idea um depending on kind of like the first few lines of introduction i may or may not skip it but i'll go to the background um and see like okay

are is this standing on a foundation that i know like if i start reading the background um and see like okay are is this standing on a foundation that i know like if i start reading the background it's like we all know about uyosaka method and i'm like uh-oh like okay i guess i have to go read that one um if it seems like i can like more or less understand the background i'll actually skip the main like contribution of the paper and go straight to the results and typically in the results there'll be some other method that i know and that will really help even just knowing what they compared with like i'll go straight to the table of the results it's like okay you know they compared with like several different actor critic policies so this must be another actor-critic policy,

these kind of things. And then I'll look at the results. And they'll also give me an idea like, if this is like a 16-page paper and it's like 0.02% better than some other method, well, it kind of tempers my expectations, right? And then I'll go back and read the method so you kind of have to like go in that order i think it is a learned skill i mean i think like what jason's

saying i mean to be fair i've read maybe three to five papers a year um that's i guess maybe i'm gonna shame myself but whatever uh it is it does take time and and it is not uh at least for me not just like i'll pick it up and do it there's a there's a technique to it and you do learn that they're written in a certain way and you know it sometimes does take two or three in the same sort of corner um to to kind of get get and then you go back and i think that's the other thing is it's not like a blog post you can kind of read it once and get it i think in my in my thing at least you kind of skim it look at the pictures kind of get where you're going and you might go back and then as you sort of figured out you might go back again um and as silly as it sounds even in this day like printing it out and for whatever reason like actually having it in

front of me this is literally the only thing i ever print out anymore but i will print out a paper if i'm really trying to get through it and like holding it and sort of like being able to see two pages at once or, you know, or mark on something or like, you know, circle something I want to come back to. I never do that for anything else,

but when I do papers, it, it, you know, it helps me. Yeah. I used to print it out and now I use my, the ink tablet.

Um, and then you could just draw with the with the pencil um man i mean this is okay we're going into into a bunch of side talks here but it'd be amazing if someone made why hasn't someone just made a ink tablet that's the size of a u.s sheet of paper like why is that so hard money i i will pay for that uh you have an you have one person who will buy that um maybe there's some technical thing too i don't know like maybe like making an ink tablet that's that big is probably yeah i would also i mean i guess like once you add in the other stuff maybe it's like difficult to fit into your backpack or something like, you know, and so therefore people,

I would assume most people buying them are reading fiction, you know, novels, and those fit really well on a relatively small screen and just flow, data just flow, the words just flow around, right? I would assume that's, whatever, 95% of the hours spent on eating tablets are spent doing that and so i think today they sort of cluster around the size of a paperback yeah i think that makes a lot of sense there's an amazing thing uh if folks i don't know if this is on iphone it probably is but if you have adobe acrobat on android they have this thing called liquid mode where it takes like the paper and turns it into kind of like that format where you can scale the text and read as you know what I mean.

It basically takes the PDF and turns it into a format that's awesome for mobile and it usually doesn't mess up the diagrams and all that. That feels like a low hanging AI target right there, which is like take this PDF and and make it like you know whatever i don't know what they call that yeah flowable liquid like yeah yeah yeah but like good i've seen so many bad where people it just like it's broken and doesn't work right yeah yeah same um all right my article is my second is nvidia cosmos an ai platform to change the future of robots and cars, wins best of CES. So that's a pretty bold subtitle there. But what is that? Anyways, the thing where you kind of put a title in the middle of the title.

But Cosmos is really interesting, and it ties into what Patrick was saying. It is a world foundation model and so the idea is people have uh people at nvidia have um taken you know like videos and tried to predict the future of the video and they've done other things like that what you end up with is something that the world foundation model is trying to be kind of like a physics simulator um to try to give you like an idea of what's going to happen in the future and then what you can do is take embeddings which are like you know compressed vector representations of images you can take like the embeddings of the future and feed them into a model and so the idea is if you have a robot the robot might know that like hey uh you know this future looks pretty weird like this future looks like it's the kind of future of a robot that's

falling down the stairs and so i'm just gonna stop because i don't like where this is going right um and so the idea is you can combine these world models with like some like uh inertial data and all of that to like make robotics a lot easier um you know nothing practical has come out of it um but i think that i think it's it's something really interesting and i think that it's it's it's going it's skating where the puck is going i just don't know uh what's going to happen there there were a lot of really interesting we didn't put them in here but nvidia announcements so i i mean i think for sake of time we'll just skip them but if you haven't gone and looked at

the various things that nvidia is doing and introduced and you're vaguely interested in ai and and how the the underlying hardware is, is, you know, shifting, I would definitely go. There's some like, what do they call those? Like super cut, like where they cut up the sort of CEO speech and sort of like show you the highlights. Yeah.

I've never heard of that. I think so. Instead of, you know, watching an hour keynote or whatever you watch, you know, five minutes, 10 minutes.

I did that on the one from Google last year, and it was just Sundar Pichai saying generative ai over and over again but then you realized it was a generative ai video it wasn't actually him all right time for book of the show book of the show patrick what's your book okay completely out of character for me but whatever and it ties back a bit to what i was saying trying to to kind of read sort of more academic approach and broad broadened my horizons i'll call it but this is alice's adventures in a differentiable wonderland which i haven't made it all the way through yet it's a little slow going for me um but it is i

think so far it has been like a little bit of an in-between machine learning book so you can get the sort of like elementary takes which i feel like i i kind of get i think a lot of people are out there sort of understand that you know high conceptual 30 000 foot level what's going on with you know the machine learning and more than just you know oh it's an llm like okay well there's you know convolutional layers and there's you know there's this this you maybe watched uh you know the three blue one brown always mess up his name video about lms and transformers and how they work right like you sort of understand some of the the nuts and bolts from i'll call it like the top down to about the middle um and you know you may even could run

python scripts to train your own little little model stuff stuff i've done but then if you flip it around and go sort of like bottom up you know jason was describing this earlier like how do you vectorize this thing i want to do and build models like how do you start to piece together these things why do they work what is the expectation um and then the thing that for a long time has eluded me and is maybe clued in by the title here is these you know ways of combining the math operation so that they're differentiable and what does that mean and that's basically how you get the training how you take an effectively random set of numbers and guide them into weights that actually evoke a certain you know output i'm probably using the wrong words here but that's because i'm not trained and so this book seems to be a pretty middle of the road just what i'm trying

to get through it it doesn't not use any formulas but it's also trying to explain what they're doing and where they're going if you just jump into one of the the papers you know transformers are all you need or whatever oh no that's not the name anyways um the you know you get this sort of like immediate dumping into like way too much context that i don't have um and so this this book it's a book but it's free it's you know published in a pdf i think you can go buy a physical copy as well but i have a link in the show notes to where you can find it sort of kind of takes you through the various things i'll say sort of bottoms up in this sort of middle of the road probably someone who has a math background who knows computer science but isn't a machine learning person and is trying to

learn more than just here's the PyTorch commands you run, but here's what the PyTorch commands are doing or why they're doing it. What is the motivation here, which is a big gap in where I am. So I'll take that maybe right or wrong assumption that if if i'm struggling with that maybe other people are as well and since someone wrote a book clearly i guess yeah this is awesome motivation there so yeah this is so cool i mean this is a what a 300 page book totally for free yeah yeah yeah this is amazing yeah highly recommend it um i'm very cool okay you got me hooked on this i'm gonna have to read this now he's gonna ask the ai can you read it to me please yeah i'm gonna have an ai read this to me now um okay my

book of the show is a movie um this is also totally out of left field i mean you thought yours was out of left field wait till you hear this it's a beautiful Day in the Neighborhood, the movie about Mr. Rogers. And yeah, it's a really deep movie. I actually thought it was going to be a biography, but it's not. It's a fiction movie that's based on a true story of a reporter. The reporter was basically trying to write a smear piece about mr rogers you know one of these like muckraker kind of reporters and he follows mr rogers around mr rogers like actually like uh um gave him permission to like you know follow him around and interview him and all this and uh and he ends up like uh you know like like changing his life

and everything it's a really really touching movie um and uh yeah i highly recommend it i think is is pretty pretty wild actually totally blew my mind i'm generally more into like uh you know documentaries like i saw the um the one Queen, the documentary about the band Queen. I saw another one. Oh, the one about Theranos. I think it's called The Dropout. I saw that. I went to this one

just expecting another documentary, but my mind was completely blown by this movie. It's on Hulu, Netflix, all that good stuff um highly recommend you check it out i never watched documentary i guess i should if it says documentary i'm like i'm basically like instant out like i just for whatever reason i don't know i should probably watch them more instead of just like whatever i i don't know it's just like for me

i always feel like i'm gonna have to do a lot of follow-up work after this documentary to like, like understand the other side or whatever, because it always is sort of a hot take. Right. And so, um, yeah, I mean, you know, I think the, the ones that I've watched have been maybe not too controversial. I mean, like the queen one, for example, it's just really interesting, like how they came up with the songs they did and so it's really like i think it's a way for you to have been somewhere that you weren't you know like i wasn't in the studio when they were recording their songs but somebody was and they they provided insight and you get you get that kind of experience i guess like uh through through that so i will say the most impressive thing is that you said the title without any hint of

sing song in your voice. Oh man, it's definitely a throwback to watch because did you watch Mr. Rogers growing up? Yeah, me too. It was great. Okay. Yeah. All right. What is your tool of the show? Tool of show it's not a tool it is a game

so it will waste your time i would not recommend you to play this game um you wasted so much of my time on bellatro by the way i finally i i beat uh i think all the chips or i'm on the last chip oh you're better than me i got the red and black so good and then i the red and black deck is so stacked i mean you just have flushes whenever you want yeah okay people are like what um don't download that definitely don't yeah save your time yeah i have i have i have made it so that i'm only allowed to play that game when i'm on an airplane that's smart so otherwise it's it's bad for me uh okay this one's not quite so bad this is actually relatively really short game i guess they call these like incremental games um but someone threw

this out the other one that really killed a ton of my time was vampire survivors there's like it must be a whole thing i don't want to know please don't send me emails please um but like three dollar steam games or whatever so this is like a $3 steam game supposed to be short, just goofy. Um, but it's, it's digsium and, um,

on a computer, you know, use a mouse and keyboard on my steam deck. It actually works really great. A lot of the, you know, sort of like clicker games kind of don't work great on your, on your,

you know, steam deck if, if you have one. Um, but this one I did this, I've been playing. It actually works really great. Do you have a touchscreen on the steam deck?

Yes. Okay. And that's, that's what makes this game particularly so uh problematic for me which is uh the idea is you're kind of like I don't even know the story it's like you're running a museum and you're going out to these various fields and digging up relics to put in your museum and you get money for the visitors who come to museum it's all in not even 8-bit graphics like commodore 64 kind of like like graphics um but you have the various places you go to harvest relics uh increase in size and increase in the difficulty of of mining up the dirt and you'll have only so much energy to to kind of mine it up so if you're not able to sort of mine up all the blocks you could not get all the relics that were buried in that particular round. But everything is like very quick, you know, it's maybe five seconds to sort of tap, tap, tap, tap, or click, click, click, you know, clear the field,

whatever run out of energy, you go again, you go again. And then you're also building up money in the background. And the classic, I guess, guess like incremental thing you can buy upgrades and then you can reboot you know yourself but some of the upgrades are permanent and you get for how high you get you get sort of permanent upgrade points that you can use before you start again and it's just like i know it's such a distillation of some of these incremental games either for me they just like have this plateau you get to and oh that's the that's the grind and it's amazing yeah i hate the grind yeah i got that at work i just wanted to be like like uh i was gonna say something it's not i guess i won't say what i was gonna it's just very like i just wanted

to be light and very you know just a soda junk food kind of like get me through it i don't need it to be long lasting i don't need it to be thoughtful i just want to go full steam ahead and feel like i'm powerful yeah totally yeah totally. Yeah. This sounds awesome. This is right up my alley. So yeah, dig Xeum. I don't know if we said it, but yeah, it's available on steam and I don't know where else I tried looking. It didn't look like there was other places, but definitely it's available there. It's like three bucks. Maybe it goes on sale sometimes. Don't know. I just paid $3 and don't do that. It's just as real. You'll regret it. sometimes. Don't know. I just paid $3 and don't do that. It's just real.

You'll regret it. You'll spend, it's actually not that long, but you know, you'll just burn a bunch of time and you'll be like, what is it? There is an ending, right? I believe so. I, I think I'm pretty close. People told me how many hours they typically spent on it. Like when I, that's how I found it, you know, via some random link I saw somewhere. And I think I've about played that hours and so

i do get the feeling that i'm towards the end but i don't actually know so got it cool um my tool of the show is a python library called sequel light dict uh which is a weird name it's you know based on the dictionary uh object in python so so everyone doing Python has thought about this, where you have Python, you know, everything's a dictionary. Classes are dictionaries. All the objects are dictionaries. Data classes are dictionaries. So dictionary is like a really foundational object, right?

It's equivalent to like object in JavaScript, right? And so you're constantly passing these around you're setting keys you're sorting them etc etc um and then you always ask yourself well i have this dictionary and maybe it takes like three hours to compute all the keys for this dictionary and and when i'm done i want it like right? Now, if it's the kind of thing where it's a batch job, like you process this dictionary, and then now you just need it on disk, you can turn any dictionary into a JSON object,

a JSON string, and then write that string to disk. Pretty easy. And there's a similar function for going the other direction. There's a library called Pickle where you can pickle any Python object to disk and then unpickle it later. That's great. The challenge becomes

what happens when you need a backup of a dictionary on disk but it's like incremental. It's not practical. Anytime this dictionary changes, you'd have to save the whole thing to disk, right? So what you really need is you start getting into the realm

of needing a database, right? At some point, you want to use Redis for this or something, right? But that's pretty heavyweight. You need to stand up a database, it needs to live on a port, use TCP to connect to it. It's just like a lot of work, right? You could do it with Docker and stuff to it it's just like a lot of work right you know you could do it with docker and stuff but it's time consuming so ideally you'd have just like a file

on your computer that has the dictionary on it and you could just access that file the same way you access a regular python dictionary like even it would be even better if like you didn't have to change your code you could just or change a lot of your code you could just replace like if i have a dictionary and i'm stuffing a bunch of data into it i could just replace that dictionary object with some other object and get a dictionary that's persisted on disk right and so there's something built into Python called Shelve, which does this. But Shelve sucks. So there's a bunch of reasons why Shelve is terrible.

The biggest reason is if you're on Mac or Windows, actually it might just be Windows now, but there's definitely a time where if you're on Mac or Windows, you got one database format called dumb. It's literally called dumb. And if you're on Linux, you get this other database format called DBM. So you can be on Linux using Shelve, having a great time. Things are super fast.

You run the same code on Mac or Windows, and it's like 300 times slower. And then you're like, well, what happened here? If you could take the time to like dive into it, you find out, oh, it's because instead of using DBM, I'm using dumb, right? So shelf is a mess, right? There's also data corruption. So it's not that often, but sometimes if you use shelf,

when you go to open or when you go to access a key, you'll get this error. And I don't remember what it is off the top of my head, but effectively the database is corrupt and your data is gone, right? So not a big fan of shelf. Now, SQLite is amazing. SQLite is freaking awesome.

Everyone uses it. Your iPhone probably has like 30 SQLite databases running on it um uh you know that's at the os level everyone's using sqlite so sqlite dict basically gives you the api of shelf but under the hood is sqlite and uh you know it's not not going to be... It's one of these things where, like, it's probably a little slower than Shelve. Maybe it's even faster than Shelve. I don't know.

Probably a little slower than Shelve on Unix. But on Mac and Windows, it's the same speed as the Unix Shelve. So you don't have to worry about that. You can access the database with SQLite in another language or stuff. It's going to be whatever you put in. If you're pickling things and putting them in, then you're going to get just random binary garbage. But if you put in, let's say, a number

into SQLite dict, then you'll see a number in the database, which means you can access it from other languages and stuff. So it's an awesome library. If you're using Shelve, stop using it, use this thing instead. If you're not using any of these, it's another sort of tool in the toolbox that i go to pretty often yeah i i've heard so many issues with pickle so yeah i i think putting it even if it's tedious to get the stuff back out i guess like having something that's sqlite that other you know programming languages whatever could access other programs like feel feels like a win yeah i mean by default shelve and sqlite will both what they'll do is they'll look at the data type of what you're trying to put into the dictionary and if it's a primitive data type it'll just store it if you try to put like a python class

into the dictionary both of them will pickle it and i totally agree i think instead of trying to throw python classes into either of these you should like convert to json or something like that pickle is good if you're in a hurry but yeah it's i think there's security issues you really have to trust the person who pickled the object and then also there's weirdness around like if you change the class and all of that. Very cool. Well, guess what time it is? It is project planning and management time. I was going to say, thank the Patreon times, but yes, it is. Oh, thank you, Patreon.

That's awesome. All right. Thank you to all our Patreons. But yes, it is actually... Yeah, big shout out. It is actually time to talk about project planning. And then we realized maybe a little bit of management. I'll give the disclosure. This is not advice.

This is just entertainment. No, I'm just kidding. We are software engineers, both. We are not product managers. We are not project managers. We are not like this is just from a sweet perspective it's a very casual conversation so um trying to give people this sort of vibe i guess this is what the cool kids seem to say the vibe of what what this project

planning is all about and there are things to think about and the you know stuff to go on and uh yeah we're gonna jump right in to to start motivating the reason why. Although, you know, maybe if you took a second, you'd probably get most of these. But the reason to plan your projects, I guess, is to think about the alternative if you don't have a plan. But no, if you work in a company or on a medium or large size project, anything that's, you know, going to take more than just a couple hours, really having a plan is, is important. And, you know, I think there's a bunch of reasons why, but some of it is that having a explicit process where you kind of go and plan what you're doing allows other people

to give feedback on the plan, right? Make sure your coworkers are aware, make sure everybody's sort of gathered together, um, and to figure out, Hey, what parts of the plan are going to be risky? You know, is this, are we, is there,

you know, Hey, we're going to do this by this date, but there's a holiday break here. So if everybody goes on holiday break, then, you know, we're not going to get our plan done or we're going to have to try this new technology or this new algorithm.

We don't, you know, yet know how to do and making sure that sometimes you have blind spots and that people identify, hey, what are you sort of not counting seriously enough? Yeah, yeah, totally agree. I think another really important part of project planning is headcount planning. And this is something where I've seen this go horribly wrong, even at big companies. I definitely am not going to give any personal stories,

although I'm thinking a bunch in my head. But I've seen situations where people who asked for, managers who asked for more headcount just got it, just for asking. And it's like, if you didn't ask, you didn't get it. Or I've seen cases where um it was pretty subjective it's like oh your project is kind of cool let's like put more people on it right but but ultimately like the principled objective way to do this is you should plan like what is the expected amount of work that our team is going to accomplish and when you see team

this could be thousands of people it could be four people that still works um and then come up with like a distribution um and maybe like you know this is like science brain of me but come up with like a distribution where you know if you're like one sigma below the distribution, then like that was slightly below expectations. You know, if I'm one sigma above the distribution, that's above expectations. And then that's really important for so many reasons. One is at the end of the year, you need to figure out, did my team perform at my expectations or not? And you can't do that if you don't set expectations. Right. at my expectations or not. And you can't do that if you don't set expectations, right?

But the second thing is, you get to see what you didn't do. At the end of the day, adding more people allows you to do things you couldn't do. And you have to know what that is before you can make a case for more people. So at the end of the year, we could say, here's a list of ideas we had. Look, this idea, we didn't do it and someone else did it and they published a paper and that paper was amazing. So that's a miss. Or we had a list of ideas, we didn't get to any of them, but it turns out they were all kind of like maybe not that good.

It was all below the fold. And we're actually not really missing out on not hiring people. So I think headcount planning dovetails nicely into this. Thinking about expanding from headcount, not just new hires or you know allocation of people but also making sure if there's other teams upstream or downstream of your work and that there's a dependency either like hey we are going to use this thing that isn't being used today or we're going to assume this thing another team is creating is is going to be available is very important to

communicate so much i mean it's so obvious but and just in life how many things could be solved by just you know communication and so making sure that you communicate expectations around headcount expectations around we're going to talk about schedule um but also you know thinking through what are the dependencies between modules um and sort of of the APIs that I want to call or depend on downstream, upstream. If you are going to use a third party service pricing, right? Like, am I going to be sensitive to if the prices go up on on these function calls down for me or API calls? You know, is that going to be a problem and sort of making sure you have that. And then when we start to talk about schedule and thinking about, you know, depending on stuff that that's not yet available, or how much it's going to take for you to get your sort of modules or work, you know, implemented work items

implemented, there's also identifying the sort of critical path, what both from terms of the data flow, but also in terms of from your schedule standpoint, what is the sort of thing that if you walk through the schedule, that is, these are the items and some items are happening in parallel, but these are items we expect to take longest. And if we sort of line them all up, this is the sort of length and it's the set of things that are, we are most sensitive to in the final schedule, right? These are the set of things where if we get behind or we aren't able to in the final schedule, right? These are the set of things where if we get behind or we aren't able to handle the risks or, you know, something slips, then our, you know, total schedule slip. And that's very important to, and maybe everything is, is on that path.

That's entirely possible. But if there are things off of that path, I would argue actually knowing that they're off of the path is as important as knowing what's on the path, because when it's off the path, you have a little bit more flexibility, you have a little bit more sort of dynamicism, you know, that Jason was mentioning, thinking about things as a distribution, if you start to get involved, and you find out you're needing more resources from, you know, then you the sort of median was then, you know, you, you can kind of pull them from things that were less important, maybe, right. And if you sort of, you know, kind you, the sort of median was, then, you know, you, you can kind of pull them from things that were less important maybe. Right. And if you sort of, you know, kind of understand which things have which party, there's that classic, I guess, kind of joke. Everything is the most

important, but if everything's the most important, nothing's important. Um, and every company seems to fail at this, but this is fine. It's just like a difficult thing, but you can't say everything is equally important because you really do need to know if something's going to slip where's it going to come from where are we going to pull things and having that thought through and communicated is is very vital yeah yeah totally one thing that i you know if i could impart any wisdom at all it would be i always underestimate the ripple effects from decisions that I make in the workplace and maybe just universally I think it's just really hard to to measure multi-order effects because usually what you're trying to do is you're you have your own plan like I want to try and

build this and then I want to try and write this paper and then that paper is like a stepping stone to this other project and then I have this idea maybe it works maybe it doesn't work and so you're like you're doing what deep seek is doing right and it's hard to do that and think about like all the ripple effects of like okay if we're late on this project that causes other team to do this which causes other team to do that um Or even like an even better example is like, given that we're late on this project, if you communicated that now versus next week, that actually can result in totally different outcomes. Even though it was the same, you know, the same effect, the same initial effect could have two totally different outcomes,

depending on when it was communicated. And these are really difficult things to measure. Yeah, I think the interconnectivity, we can talk about the mythical man month, we can talk about the classic, you know, hey, if I just give you another person, can you go twice as fast? But I mean, a lot of that comes down to what you're saying is like, it is very difficult for human software engineers to understand just how much impact one decision, one choice, one slip, one whatever can have and the sort of compounding you get from that. like a bullwhip effect where you know especially if you're at like the if you're doing research you're kind of like at the beginning of the bullwhip and so if you kind of um make a bad

decision or you don't try to advance something or whatever then that causes like this massive swings downstream so that's actually great uh use of of that term can can you maybe or again but can you kind of explain what the bullwhip effect is? Yeah, so I believe this came out of like supply chain management, but the idea is, you know, let's say you end up with slightly less grain this season. Well, then people will start,

you know, people downstream of that will start to experience shortages and they'll start to you know buy up volume and do different things to like handle those shortages and so then some of those processes that they start implementing like buying extra surplus doing all these extra things they'll continue next year even when you don't have a shortage so they're kind of like because they're downstream of you they're reacting to you and their reactions are very quick and so they end up like buying more than they need and the next year you have a surplus and they're still buying more than they

need and so now they have to like double correct um maybe you know yeah and so and so so it has this compounding effect right and that's just one step so imagine if there's a supply chain of like 20 steps the person 20 steps down just experiencing like wild volatility because of all of these kind of like mispredictions did i get that right yeah yeah yeah so if you think about like software that is finally delivered or a product that ends up on a shelf on a store and you think about all the unique steps that it takes to get there and especially when they're sort of like a continuous continuous process which we could kind of talk about software being viewed as a sort of one shot or continuous but yeah it's just sort of like as you add these dependency things small

perturbations in the beginning if you you sort of like, if everybody sort of stacks it up, they just, they get compounding. And so, yeah, I think that can, can definitely be there. And when you talk about, and I think we were going to maybe talk about that, but I think it makes sense now is if everybody sort of tries to, and you were sort of saying like overbuy or have buffer or allocate a little extra, right? And so if you were sort of saying like over buy or have buffer or allocate a little extra right and so if you take all of the teams and all of the steps and everybody sort of buffers up so that you know they're not the ones called out you end up with something that's really like four or five x longer or more expensive than it needs to be because you end up with all this hidden sort of

like padding that occurs all over the place. And so you'll hear that, that like upper management will sometimes say, you know, like I, whatever they tell me, I cut it in half or I cut it in 20%. And this is what they're talking about. And it happens with budgets. It happens with headcount. It happens with schedule where everybody sort of pads and then the people over them pad and then the people over them pad. And you end up with like ridiculous amounts of padding. And so there is this element when you're doing project planning of trust, right, that you need to be actually honest. And there's already difficult enough to even estimate how long somebody is going to take. also playing this game of politicizing it now does that actually happen no i think everybody politicizes it to some extent sure but you know the less of it that you can have the better which is hey you're brutally honest with your manager they're brutally honest with their manager and

you have an actual view up and down the chain of what's happening and just like we talked about the bullwhip effect unfortunately it only takes one of those links or one of those layers to be i don't want to say like a bad actor but to just try to make sure like i'm definitely not going to be the one who is called out and and they sort of add something how at the top are you supposed to know you know how honest all of the estimates all of the the work is and so yeah the bullwhip effect can be unintentional but i will say it also works kind of in estimates all of the the work is and so yeah the bullwhip effect can be unintentional but i will say it also works kind of in this sort of like intentional safety that people add sort of aggravates the effect even more yeah yeah totally right i think that you know that's a good point i

mean sort of a meta point here is like you know i notice i see people who transition from an individual contributor to a manager and they start off with like okay how can everything be perfect like how do i make everybody's tasks done what do i have to do to kind of like rewrite history so that like everything we set out to do in march we now have accomplished you know next year and and uh everyone on my team is perfect they all need to get promoted all the time and then over time you start you start like calibrating and you say okay like the goal actually isn't to like play the organization system as if it's monopoly or something right the goal is to like have the most accurate assessment of the situation and um and so so i mean you know the earlier example of like put things on your

plan that you feel unlikely that you're going to accomplish um and then when you don't accomplish those that's that's normal like you should you's normal. You should set your team up so that there's an upper and a lower bound that they can achieve. And statistically, you will eventually have a quarter where your team massively underperforms. That's how it's going to work statistically. You're going to draw that short straw you know so many times in your life right and so and so uh i think project planning also helps keep people from panicking it's like you know yeah we missed this quarter but you know we didn't miss the past three quarters and so so this is natural. And so we pick it up next quarter. Yeah, I mean, I think that's a great transition

to goal setting, but I think it becomes almost a parody, a joke when you see people who set goals and then they always get exactly the goal or like exactly the goal, plus some like little fudge factor. And it's always slightly more, like I always accomplished 1.03 of whatever my metric was

and it's like if you never come under your goal to your point you actually don't have the distribution set correctly like it should be 50 50 sometimes you're over and sometimes you're under and that's life and that's actually means you have the the sort of estimates well set and if you don't you're just sort of like goofing around gaming the system under whatever, you know, like, at least that's, that's sort of my opinion. I mean, maybe you're in some, you know, hockey stick growth thing. And it actually is just really, really difficult to predict that can happen. But setting goals is, is very critical. And I think also being flexible and not saying which we've can sometimes happen. The plan isn't there to say,

oh, we've ruined the beautiful plan, you know, plans never encounter the first, you know, never survived the first encounter with, you know, real life or the enemy or, you know, use whatever your favorite, you know, quote there, I'm butchering, but I think the plan is there to help give people the feeling that like, no, it's okay. We have these things written down. We have them documented. We know what to go adjust, who to go tell, what is going to be impacted, not, oh my gosh, you've ruined all of our work. And I very dislike working in situations where people are like, we made the plan and now the plan is ruined. And it's like, well, then there was probably not a good plan. Like the purpose of the plan is to help us not hurt us but on setting goals you know there's a mnemonic that that you'll hear which is set your goals to be

smart yeah totally um my favorite um my favorite my favorite quote is the mike tyson one where he says everyone has a plan until they get punched in the face oh there you go um so smart goals stands for that the the acronym smart is specific measurable achievable relevant and time bound and so if you've ever taken any kind of project management course in any of your work, you'll hear this come up. Yeah, I think it's one of those things where, again, maybe it feels a bit obvious. It's actually really useful if you have set goals and you actually ask yourself personal goals, project plan goals. But, you know, have you done all of these things? I often find that, oh, yeah, despite the fact that when you say it this way, it's pretty obvious. Yeah, I often screw up at least two of them, I think everyone's like, well, when's task X going to be done? But the measurable one,

that is the one where I often catch people and have to iterate. So someone will say, for example, like build the foo widget, right? And so we all have to agree that just building the foo widget is a good thing you know and that happens sometimes right but generally i'll push back and say okay like well what what is the point of the foo widget you know like here's a company you know kpis here's a company

you know metrics right um do we need to create a new metric like is this a new dimension that we need to be looking at um and and you know it might be that you know we're going to create the foo thing uh the foo widget um and then we're going to have like a list of things we're going to measure on the foo widget um and so and so then you can like distill down to the goal so so if the if you're creating the foo widget because you have a new sort of segment of customers right what are those customers going to do well they're going to buy the foo widget how much does a foo widget cost it costs twenty dollars okay so so we have a goal that you know we're going to we're going to make like a hundred thousand dollars in the first three months selling this

food that we haven't built yet like like there should be some way to to get it down to a measurable thing where you're not measuring like yeah the thing is built that's that's that's one or the thing is not built that's zero you know like getting a a proper measurement that like connects to the company's objective and and all that i think is is really important um um so you know sometimes you might build things you know you see this a lot in research like you build things that um that are trying to kind of push the industry in a new direction and and even in these cases there should be some way to measure even if it's an internal benchmark or something like that uh maybe instead of building the foo widget what you need to build is the benchmark that you're expecting the foo widget

to outperform in and we need to actually build that thing because otherwise someone will build the foo widget they'll be super proud of it and then they're wondering like why their performance review wasn't good you know so it's like well you have to build the thing sometimes you have to build the thing that motivates or or explains the efficacy of the actual thing um and so so the measurable one is super key you know the other ones i feel are you know like specific it's kind of like okay everyone should always be specific achievable it's like how could you have a non-achievable goal but but the measurable and the time bound seem the ones that really stick with me. You've never had your manager set a non-achievable goal?

Oh, man. If so, you've been incredibly lucky. So I feel like I have to be diplomatic. I've definitely had managers set non-achievable goals. But not once have I ever said, this goal is unachievable and they've said okay we'll change it so so so it seems almost like uh i do think that achievable goals are more achievable but i don't think that knowing that is going to help you get a impossible goal um and make it achievable like i've seen um so one time they wanted this metric to go up by some ungodly amount and i i basically was saying like you know

it's not it's not really going to happen and what ended up happening was the metric went up by a nominal amount and then everyone was kind of meh about it. So the fact that the goal is unachievable, I don't know if it really changed anything. Well, then I think that's the point of the sort of letters working together. Like if it being unachievable didn't matter, I mean, I think that's because it wasn't a good goal. Like you could have had a better goal that would have helped people rally around it or would have done something slightly different. And maybe it could have been at least 2x of whatever the nominal increase is, maybe not 1000x. Sure. But like, it being more achievable, could have been

potentially more motivating to people because there could have been a path to it. Yeah, I mean, this is tough, right? So if someone sets a really ambitious goal, that gets people excited. It gets their managers excited. It gets your team really excited. And then if you are directionally accurate and you do make an improvement, but it wasn't the enormous outstanding outstanding improvement it's really tough i mean like your instinct is to say well i expected let's say a two to four x improvement my director promised

a 400 x improvement we got a 3x improvement like your instincts is to say like yeah that was like a giant mess up like the the the head honcho should have just predicted a forex improvement but then it's like when i look back on it it's like that that like fake number caused so much excitement it's like it's kind of like a counter to it if they get actually worked out um i don't know this gets really complicated, but I guess if it's measurable, then maybe that just makes it at least somewhat really solid. I think people can often, and as we're kind of saying here, I think people can take things for unintended reasons or take them too far. But one of the things is Gantt charts. So if you sort of represent each of the tasks as a sort of, I guess, like a rectangle, a

bubble, and you sort of schedule them over time and you sort of draw interdependencies, it kind of sounds stupid, but you often will discover like we have work scheduled to begin before the thing that it is needing at input finishes right oh well that that is not going to work and if you just sort of say hey you know to do b we need a a is going to finish in q1 we're going to start b in you know february it's like well wait a minute like A is going to finish in Q1. We're going to start B in, you know, February. It's like, well, wait a minute. Like, are you going to be able to do work? And then you would say, well, you know, we can start it. Well, then it's really probably two tasks, like getting ready and then executing, right? And so it helps you to decompose them, which again, then helps with what we talked

about earlier, you know, headcount, slack buffering, understanding what's critical and not critical. And so putting up a Gantt chart is a way to sort of say, and in some ways, you know, not the right use, but work backwards and say, hey, we really need this to be done by this date, if we really put all the processes in place and say, you know, when do we think we need them? And sometimes you realize, oh, actually, we need to start this a lot sooner than we thought. And then you say, oh, we don't have the right personnel. And that's always a very delicate thing to balance, which is, hey, you may need 10 people in the beginning and only two people at the end. Is there a way for us to be thoughtful and make it so that we have a more level head

count throughout? Because it's very difficult to need 10 people and then just get rid of them. Ramping up is generally easy. Ramping down can be a bit more difficult. And so is there tasks that aren't in this dependency chain that we can reschedule or reallocate and sort of make it a little bit more palatable?

And so Gantt charts are a very specific way of sort of identifying the interconnectivity between the tasks and the schedule. Yep. Yep. That makes sense. A couple of like, you'll hear these methodologies get thrown around like Scrum, Agile, Waterfall. And I have to confess, like, they all occupy some like really nebulous part of my brain

where it all just kind of flows together like one of these really weird text-to-image things. If you put in a theory of mind into a text-to-image generator, you just get garbage. So I don't quite understand the different nuances and differences there. I'll just say my piece on it. And then Patrick, you probably have a lot more experience. I generally feel like you should just consistently be replanning. So in my case, like right now, for example, I have a weekly planning

meeting at my current job. And so we look at the last week's plan and the quarterly plan and how are things trending. And I just think it's really important to stay kind of on top of your plan and just discuss it every week. And then also have like a very quick touch base every day of like,

hey, how are things going? What's the roadblocks? What are the barriers? What are the accomplishments? And just kind of touch base. And beyond that, I haven't really found any of these, like, acronyms or any of these words

like Scrum or Agile to be particularly useful. I think it's one of those, like, people are surprised when I talk about it. In general, it has been my experience in talking to a lot of other people that I work with and around that the sort of Silicon Valley tech companies are a bit unique in a couple of ways where they try to hire people that are very talented. They try to hire people who aren't hyper specialized by default. They generally prefer people who are flexible more than specialized. And their customers are like end shipping products and not, you know, pay for, you know, accomplishments and milestones.

So you'll hear about even internally, we'll talk about like setting milestones. If you worked where you were contracting for another company to deliver something and they weren't going to pay you all the money up front without knowing that it was done and you need to pay your engineers. So you can't take all the money at the end when you finished. So you need milestones. Those milestones need to be evaluated. I think you get a lot more process heavy, you know culture which is completely makes sense silicon

valley is a little different and that people can kind of tackle many many of the different tasks that are within a project that may be better or worse but they try to be generally less specialized than than maybe in other uh industries and the fact that most of the stuff is all what is internally funded they're not doing it against a contract they're just internally trying to do research and try to make the product better and ship metrics so all the goal setting is internally motivated rather than tied to dollar amounts that occur for incremental progress it's sort of the end outcomes that matter and nothing along the way it's just sort of like get it done leads to a more uh flexible dynamic less strict stricture around sort of following this methodology but if

you look at like defense contractors or people doing contract software for other companies you end up with a lot more of these and there are good ways of doing them in bad ways and pros and cons of each of them but in the end a lot of it is dictated by how the money is going to flow from the person paying for the work to the people doing the work. And so I also have very little experience. I can't talk about pros and cons of various ways of doing these things because in general, I would say, like Jason is sort of saying, it's a sort of continuous process where we're doing this but it's not very formalized it's a very informal process and you rely heavily on everyone generally

being aligned towards the same goal and acutely we'll see when people on the team aren't aligned to the same goal it's actually problematic that there's none of these processes in place when everyone is aligned and has the right incentives it's actually amazing because you don't end up with any of the overhead that these processes uh would have and so it's sort of a i don't pick your poison kind of thing i'm not saying it's better i'm not saying it's worse it's just very different yeah that makes sense and and um i guess okay maybe we could distill the few sort of like key components to all of these methods. Generally, you know, when you're doing any type of project planning, you have a backlog.

So the backlog are your ideas that you've thought up, but you're not ready to work on. Typically, you'll get a lot of tasks assigned to you from other teams or other members of your team and and then you have your you know current things that you're working on and and the gantt chart that patrick mentioned and all of that associated with that and then you have things that are you've basically committed to like these are things that we're definitely going to do and maybe you even have dates on those but they're not in progress um and so effectively what you're doing on a weekly basis is you're you're looking at your backlog you're saying like are there things that we should add to this backlog like have people thought of new things there's derivative work is there a task that needs to be split and that

second part needs to go in the backlog um and then you're also assigning sort of like a rough amount of work to the things that are in the backlog like is this a a really large task a tiny task and then what's the priority so you're trying to prioritize that backlog even before you start pulling things up um and then you're depending on the job you have worked different jobs where the sometimes like like in research things are really staggered right because you have all these different conferences and they all have different deadlines so like so the person who wants to submit to icml they're on a totally different schedule than a person who wants to submit to ICML, they're on a totally different schedule than a person who wants to submit to NeurIPS because those conferences have different deadlines. If you're not dealing with that kind of world, then you can maybe have something more regular where at the end of the quarter, the entire team does a retrospective of the whole quarter you know that's something that can work if you're in a less heterogeneous environment um but i think you know

so the two big pieces of advice for project planning and this is important whether you are a project manager an individual contributor a manager etc um you know is look backwards just as often as you look forward so you can kind of calibrate yourself you know, look backwards just as often as you look forward. So you can kind of calibrate yourself, you know, have a weekly meeting where you kind of go through those different states of your tasks. And then also make sure that you're putting new ideas down. I've seen a trap where people will just get into bug fixing mode and kind of forget to innovate. And so sometimes you have to put a bug on hold so you can innovate. And again, if people are upset about that, that's when you go back and say,

well, this gets back to headcount planning, right? But I've seen it a lot in research where you know research idea becomes production server and then all of a sudden like it becomes total project planning chaos um and uh and these are all steps that will help mitigate that um quickly just cover some of the the tools that you might hear that are associated with this. My favorite is whiteboard. And then I had to clarify because it sounded like it was a product name, but just like an actual whiteboard and people in a room standing around and quickly like it, you know,

and I'm we talked about work from home from last time. But I will say one of the things that most I think can be difficult without just being there and like on a piece of paper or on a whiteboard is just that real iterative dynamicism that happens very early in the process specifically, or, you know, when you're really under a lot of change, the digital tools, for me, at least are just, they're not as easy to kind of just like put up something, draw it, redraw the arrow, move it here, move it there. You know, you're trying to the tool forces a certain behavior, versus to kind of just like put up something draw it redraw the arrow move it here move it there you know you're trying to the tool forces a certain behavior versus you kind of just want it to be at least for stuff i do a little more free free flowing in that early project planning stage if only someone made a really really large ink tablet like four by six feet like the size of a whiteboard but it's e-ink yeah this is all i

want please some some factory and and what you need is like modular e-ink panels like with magnets on the side you can just like build your own put a bezel on it stitch them together whatever that would be amazing um yeah whiteboarding really important uh really useful do you how do you use a whiteboard since you're remote patrick yeah i don't have a good answer for it occasional like on-site visits is is a good is a good way um to try to accomplish that and planning when done right i mean doesn't that kind of stuff doesn't need to be done super super frequently i mean updates to a plan sure but um you know that that sort of initial stuff doesn't happen quite as often um but i mean there are you know you know digital whiteboards that you can erase and draw

on and scribble for some reason they're just never quite as clean as uh whiteboard oh here's the bonus in real life one post-it notes oh yeah post-it notes plus whiteboard yep okay that's my two okay you can move on to the i guess more serious i mean i've used all of these they're okay you know jira asana open project these are all names that you'll hear i'm sure there are other ones too i can't remember but um you know all of them more or less do the same thing. I mean, if I had to recommend one, I would recommend Jira because it's free. You can have, I think, one or two projects for free. So if you're a student out there,

I know the majority of our listeners are students, just use Jira. It's free. But the interface is a little clunky. Asana, I would say, is a slightly better interface. But is it worth the price? If you're a student, probably not. If you're a professional, then it matters less the cost of it.

They're all great projects. They'll all help you. And they all have all sorts of integrations. So you can get a Slack message or an email whenever there's a new task and all that good stuff so we talked a little bit about the dealing with uncertainty oh uh there was one more tool i don't know anything about that open project do you know is that that's uh yeah open project is an open source kind of like jira equivalent and so if you you know there's times where where going open source just feels kind of cool or you might just be feeling like, I really don't want to depend

on any service or anything. I did run open projects sometime in the past and it was pretty good. It had a lot of the same features and your code, you can totally edit it and do whatever you want with it. If you are really into project planning to where you want with it if you are like really into project planning to where you want to sort of like hack on the project planner itself then you could use open project oh we also didn't cover but one you may hear a term is can ban and it's really sort of like in

all of this stuff but but you know there are many many tools for doing for doing can ban boards but those are more or less having tasks on little boxes or Post-it notes and columns and sort of moving them for just trying to get into that low overhead, understanding that situations are dynamic and a sort of visual representation of it. But the final thing to just talk about is, you know, I guess to continue Jason's quote,

what happens when you get punched in the face, somebody gets sick, the goalposts get moved, whatever, you know, sort of happens, and your plan is no longer a sort of working. And we talked about making sure that you do have some padding, some buffering, specifically around things that are harder to schedule. Like Jason keeps talking about research and I think that's very difficult because it's sometimes very hard to put a specific time on when you're going to have that innovation or when it's going to occur. And so making sure you have buffering in the schedule

can prevent it. But again, once you get punched in the face, it isn't really there, but making sure that everyone's very communicative and on the same space you know space and that you know being really honest with management that there needs to be slack in the uh in the schedule to say hey we're going to have issues we don't want to you know have everything booked you know end to end to end no no downtime no slack like that's just a way to have the you know expectations

get missed and then once they are missed, being honest, like for me, not everyone does it, but I try to be very clear when we're no longer tracking to schedule. A lot of people try to hide it and we'll make it up, we'll make it up. I'm a big believer of let's move the schedule and if we make it up, we'll move it back.

And that doesn't make me always popular with people, but stuff generally gets done. And as you get closer and closer, the certainty of that final date is is more nailed down and you're not always all situations aren't flexible like that but for me that's a huge thing is just saying listen that end date is going to move it will move less the closer we get but we should move it to be accurately reflecting of what's actually happening yeah i totally agree i think that we've covered this already,

but there's a short-term gain. I won't lie. I think there's literally, there is a short-term gain in fudging the numbers, making your plan in hindsight look incredibly prophetic and trying to sell some Cinderella story

about your team. You could probably do that especially if like your manager is also pretty junior like you could get away with that um but it will fall apart it is not a long-term strategy that is going to work um eventually the what is it the chickens will come home to roost uh after you get punched in the face and all the other analogies. But, you know, just being upfront and you could end up at an impasse where, you know, you feel like this set of goals is nominal, is expected. And your team disagrees or maybe a single person on your team says, you know, this is way more than I can accomplish. And those are very difficult conversations, which makes the project planning even more important.

So that at the end, you know, and this is not something that people like to talk about, but, you know, if you do have a person who's just not performing and you have to have some really hard conversations, you can say, look, like, here's the plan we set out. There was disagreement about the plan, but, you know, the rest of the team, you know, feels like this was a very reasonable plan that somebody could do in a reasonable amount of time. And that person didn't do it. And so we have to, you know,

have a performance discussion. So being able to do all of that planning up front and, and have those, you know, hard conversations up front is super important. And, and it will help give you sort of like a long-term sustainable team. All right. I think that about wraps it up. do you have any parting thoughts before uh

or we hit the road patrick uh just reiterating my disclaimer uh nothing we said no i'm just kidding uh yeah i mean everybody does it whether you want to or not informal formal like somewhere on the on the sort of gradient from nothing to overdoing it but you know project planning it happened i mean if you you know are gonna have a have a kid if you're going to get married if you're going to like projects and planning happens and um you know you can either sort of i don't sound bad blunderunder through it, or you can, you know, try to pick the best pieces that work for your situation and really try to have it be something that, you know, maybe it's not your expertise, but isn't a weakness. And so I think there's a lot of self-learning that can

occur by planning things and then looking back and saying like, hey, they didn't go according to plan. That's okay. But like, you know, this was my plan. And sometimes a plan can be a little bit more audacious and, you know, be inspirational. But, you know, maybe you gotta be real that it's that. But, you know, sometimes you're right. Like I think, you know, in my personal life, I just try to say like, hey,

I'm gonna have a plan for the year and at the end of the year, look back and say, hey, did I hit or miss, right? And so not to get uh deeply philosophical i guess but you know i i think planning and goal setting and projects is something that's that that really goes across sort of all the things we do yeah and you know the other thing is there's a other side of this which is celebrate when your team does awesome you know i seen a lot of managers forget to do that

um they're sort of like almost afraid of the first thing i said they're afraid of like people thinking oh we're sandbagging or whatever but like look back and say like wow have we set out to do five things we did six things they're all awesome and and we should throw a party and get sushi for everyone or something oh man i man, now I'm hungry. Oh, man, now I want sushi. All right, we will let everyone go. Super shout out. Thank you to our patrons.

We really appreciate your support. And now we should all go get some sushi. See you next time. Catch you later you and share alike in kind