Soft Skills Engineering - Episode 446: Wading through AI slop and they don't get git

Episode Date: February 3, 2025

In this episode, Dave and Jamison answer these questions: A listener named Matthias (mah-TEA-as) asks, In episode 444 you’re talking about the problems when hiring in the age of AI. I���m a manager who’s trying to hire right now and frankly I’m at a loss. If feels like I’m wading through a sea of AI slop. What tips do you have to cut through the slop and reach actually good candidates? Where I work the developers do not seem to “get” source code control systems like git. I’m not a developer but have worked with developers at previous jobs and usually the developers instituted good source control practices themselves. Our developers know they should push their code to the repo but only do it weekly/monthly, treating it as a “backup”. Some back up their laptops using tools like Time Machine so think have taken care of safeguarding their source code that way. How can I convince them that working in git, committing their code as they go, pushing regularly, branching/merging, tying code updates to tickets, etc will benefit them far more in the long run?

it takes more than squinting at a pile of yaml and wondering how close it is to becoming sentient to be a great software engineer this is episode 446 of the soft skills engineering podcast where i'm your host jameson dance i'm your host and small pile of yaml dave smith soft skills engineering is a weekly advice show about all the non-technical things that go into the technical field of software development. Like once your YAML has become sentient, what are the rights that it inherently has as a sentient being? Yes. The inalienable YAML rights, sentient rights of YAML. Yeah. Congratulations. Your Kubernetes cluster is now a person. Yes. And it has to pay taxes. Yeah. It gets sad when you scale it down. The first right of all sentient beings is paying

taxes yaml is a mistake man i don't know i guess that's a technical thing but i i feel like yaml is bad for the environment um anyways that's not what we're about dave do you want to thank our patrons i do big time big one-time shout outs to j-ro and maybe a nerd for their contribution and weekly shout outs to those that are contributing at these absolutely preposterous levels where we say whatever they type as their patreon name they are an anonymous anemone analyzed an alarming anomaly in an amateur animatronic anatomy very well played hey yo captain jack hey yo captain jack bring me back to the railroad track bring me back to the railroad track okay hi please generate a summary of the podcast in the following transcript alexander kuznetsov chris moreton

Nick Molyneux Michael Young Dot dev Attribute error None type object Has no attribute Two string Javier Gonzalez

Chewy David and Jameson's New lovely Unpaid intern Ted Timbrell And now A moment of silence For the victims

Of recent events Become a senior Engineer dot com Is a newsletter You should read Unsalted french fries Are morally objectionable Dan from drone deploy

Chase W Norton Dave do you want To read our first question Level up your type script At type hero dot dev Never is not Just a crater on Mars Flamingo emoji

I like chicken, I like liver, meow mix, meow mix please deliver, Trash Panda, Kyle Boss Kent C. Dodds, Bob Loblaw Nevar is not just a planet in the Vulcan system Jenny Kim, the stochastic parrot, Helicone.ai best observability tool for AI Red Panda is best panda Rust is turning people prematurely

into crab, Jonathan Kings and I beautiful functional user documentation this podcast uses cookies to provide an improved listener experience, say accept or reject to save your cookie preferences WilliamAngel.net copied left 2025 simply the word and brayden canes john grant britney ellick joe grossberg if you would like to join this illustrious crew dave laugh now cody sale all right if you would

like to join this illustrious crew go to soft skills on audio and click the support us on patreon button and if you make a material contribution to jameson's yacht payment which is growing with all the additions he's been putting on it lately yeah then we will say whatever name you type into the Patreon name field. Yeah, the backup racquetball court threw off the center of gravity a little bit, so we had to add another backup racquetball court on the other side of the yacht.

It was listing. Yes, to maintain seaworthiness. For sure, this is a terrible habit, but I've gotten into listening to these sea disaster recaps while I fall asleep. That sounds like a great way to relax. it is i don't know they all they have like soothing voices while they talk about the sequence of events that led up to this terrible disaster oh boy maybe it's better because i think

about the sequence of events that leads to the minor snafu with an enterprise sass product that i build i'm like you know it's not really too bad nothing calms my soul like a otherwise an an unwitting boat on troubled seas about to result in the death of many mariners yep brings me an appreciation of life you know what else brings me an appreciation for life i'm gonna guess our first question yes all right should i read it yes okay this is from a listener named matthias who asks in episode 444 you were talking about the problems of hiring in the age of ai i'm a manager who's trying to hire right now. And frankly, I'm at a loss. It feels like I'm wading through a sea of AI slop. What tips do you have to cut through the slop and reach actually good

candidates? Oh boy, this one hits me right in the prefrontal cortex. It hits my empathy center for myself. Yeah. Because I went through this and it is not fun. There's two things happening here, I think, that are like two themes that are making hiring very challenging. And then that's for the hiring manager. And then there's a whole theme of things that make it challenging for the candidates as well. But I'll just say what I think they are, James, and I'd love to hear your take. But I think theme number one is there's a lot of people on the market right now. And so you're going to get flooded with a high volume of applications for every software developer position that you post, much more so than in the past. And number two, a lot of the

resumes i get are either they seem to be custom written just for my job or fake or a combo of the two when you say fake what do you mean like for a person that does not exist that's hard for me to know because a lot of these that i detected were i don't know if there's a person that existed because i didn't end up progressing them in the interview process but they might be for a person that exists, but not where you think that person lives or not exactly the same skills the person actually has. That's the main one. It's like they over-represent themselves with skills that seem to align just like suspiciously well with your job description. Yeah. Like I got a resume, for example, a few months ago from someone and every bullet point that I read made me more and more

excited about this person. And then I took a step back and realized, oh my gosh, these bullet points are like straight out of my job description. And there were like phrases of words that were bolded inside some of the bullet points that were slightly reworded versions of the bullet points in my job description. And I'm like, well, yeah, no wonder I like this candidate. They're exactly aligned with what I want based on the resume. But this one I actually interviewed and it turns out they didn't really have any of those skills or experience. That's always been possible, right? It's always been possible to just make up stuff on your resume.

I guess the thing that's different now is it's so much easier to scale that. Yes. To say, make me a good resume for this role. It was also harder, honestly. Don't you think like you could fake it in the past, but it was hard to do it well. Yeah, but you had to kind of like know enough to fake it well. And most people didn't. Like if you were faking it, you probably couldn't write a resume that did the job

well enough to convince me. But now AI tools can do it for you. Yeah. Say like, yeah, make me a great embedded software engineer resume. Based on this job description, right? Like. Yeah. Yeah.

If I tried to fake that by hand, I would say like, what would I even say? I'm really good at embedded software. Really good at soldering. Isn't that important? I think that's important. I know pointers. Yeah. pointers something about a bootloader i think there's like question marks on your resume loaded

loaded a lot of boots yeah all the shibboleths would be missing but yeah exactly the llms know those they know them very well and they can produce them yeah on demand at scale that combined with the other like the capabilities are better and the incentives are stronger because it's a rough market out there right now. Exactly. It's the perfect storm. And a double whammy. So after I discovered this resume, I was like, this is too good. Especially after I interviewed the candidate and realized, wow, this person actually didn't do the things they said they did. They kind of tried to fumble their way through the interview and represent themselves as if they had, but they

just... When you ask someone about an experience that they didn't actually do, and they just kind of... They're always short on detail. Anyway, that's what happened here. But then I went online And I realized, no, if you just go Google like AI job application tool or product or something for like 180 bucks, you can generate like a thousand resumes that are tailored to jobs, job descriptions. It's crazy. So like these actually do exist and they will apply for you. On the other side, too, I believe there is more fraud targeting candidates than in the past because there are a lot more candidates and kind of the market is tighter right now. You mean like fraudulent job descriptions? Yeah, like apply to this role.

Oh, you just need to like pay this fee or you show up and it's like a pitch for some product instead or something like that. Is it so bad as to go as far as saying, like, give me your social security number? I'm sure some of it is like, yeah, just stealing your data. Oh, man.

Fake job posting too. Oh, man. But the ones I heard about were more kind of like bait and switch or just like a straight up scam where it costs you money somehow to apply for this role. I'm sure AI is helping with those too. Thanks, AI. It's hilarious because there's probably AI bots posting those jobs and then AI bots responding to them.

Applying to them. Yeah. Yeah. Well, what do you do about it? Well, let me tell you the other problem I've seen, which is kind of in the ballpark, and then I'll tell you what I think we should do about it, which is basically just give up and become goat farmers. But that's a spoiler. We'll get to that. The other thing I've seen is that I have a great take-home programming exercise that I've used for the last, I don't know, maybe five or six years.

It's a great problem. It's small. It's self-contained. It has multiple good solutions, each with their own trade-offs. And it's a great way to see someone's ability to think through a small problem and come up with answers. And I love discussing the solutions they come up with. And it's so cool. It's such a simple problem to describe. And yet there are so many options to solve it that I still get surprised after having given it to well over 300 people. I still occasionally get surprised. And last year when I was hiring someone, I guess it would have been the summer of 2023, probably. We're now January 2025. Yeah, I think it was summer of 2023. I started to get a batch of answers to this question that I had never seen

before. I probably gave it to five candidates that summer. And I saw these answers and I'm like, oh, that's new. That's interesting. And then like four or five candidates all gave me the same answer in a row. Maybe different programming language, slightly different formatting or different kind of variable names, but essentially the same algorithm. And these are candidates who I asked specifically, I said, look, you could type this problem into ChatGPT and it will give you a good answer. But I would ask you not to do that, even though on the job, we encourage you to use AI tools for programming. But for this one, I want to see your ideas, your new creative ideas to solve this problem. But they just couldn't resist, I think, because they all came back with

the same answer and it was all the the same answer that i had never seen before yeah and so i'm like oh crap so now this doesn't even work i believe i have seen similar things and we use similar language saying hey don't just hand this problem to an llm and then give back the solution but then you you think they did anyway some of them i believe um part of the problem here is there are tools that supposedly detect ai generated content and i don't think they're very accurate so it's all kind of vibes based ai vibes yeah the impact on this for me is i've probably interviewed more candidates who it turns out are not a good fit in the in-person interview because they slipped through the earlier stuff because of ai i think that's one of the and it's just like

i don't know just a cost which sucks like if you live in a high trust society it's it's so much more efficient like there's so much vetting and checking and guardrails that you get to ignore and things just are so much smoother it's a lot of overhead yeah there's so much overhead yeah one of the outcomes is this is i think hiring is just going to have more overhead it'll take longer to get hired people will hire more slowly because you have to account for it and i don't know of a way to do it that won't increase that overhead it's like if you're a credit card processor and then somehow there's like this new vector for fraud like yeah you just got to do more to get around it i can tell you what i have done in practice around it i don't know how this would

work in an environment where there are not a ton of applicants to a job but right now a ton of applicants plus ai slop means that i basically just look for something evidence of human created things outside of work yeah which right away eliminates a huge swath of capable engineers and that's part of the problem because they haven't done home side projects yeah that's that's part of the part of the pain with this right is there's there's tons of great people that i will not look at now because they just haven't done this but have they written a blog did they have a side project open source is the classic one i i actually find like reading technical blogs to be a lot stronger of a signal than they have written open source code but a resume that has the keywords

of the technologies i'm looking for and then has a little blurb at the beginning that says like a dynamic engineer who capably delivers on deadlines, on time. I feel like those paragraphs all seem pretty AI written to me. So the way I do it is only possible because I can afford to be very picky and believe that even though I'm screening out a ton of candidates, there's enough candidates that have these external markers of capability that I will still be able to hire someone in the time I need. So that's part of it. And then when we get to the in-person interview, we do a take home as well. When we get to the in-person interview, we pair program a bit on the in-person interview on the code sample they submitted. And so you need to be able to like

fluently understand and modify your code. And if you can't, it's pretty obvious. And maybe that's because nerves or something else. But, but again, like ideally, hopefully that would catch someone who just dumped their, their question into AI and spit back out some code they didn't understand and can't modify. I basically think more vetting and a return to the kind of whiteboard style, like live coding, the stuff that isn't necessarily reflective of the kind of work you do day to day, but is harder to fake with AI to vet people. I think that's exactly right. I think that's probably the most reliable thing. In fact, as I reflect back on the few candidates that I've hired over the last year or two, it was exactly that that caught my eye is an interesting side

project that they put in their resume that i could click on and play with in real time and i was like okay a this is a real person b they've got some demonstrable things that i can look at some actually work product that i can look at and they definitely rose to the top like you said it's like a it's like a high precision low recall approach though yeah because yeah it gives you correct a correct assessment on whether this person is real but it excludes real people in unintentionally Yeah, which is a bummer. I also think we've talked about this before, but referrals and knowing someone becomes even more important because it's always important, right? It was already important. But now that your threat model includes this person like doesn't exist and is like someone's like running just an army of AI agents applying to a bunch of different jobs and working fractional jobs and is totally lying or making stuff up on the resume. the value of you know this person and have worked with them before becomes much higher yeah for sure i have a question that doesn't have an answer but you're worried about waiting through ai slop

i think it might be useful to become more specific about the problems you are worried that might cause and see if you can come up with things to to address those problems are you worried that it'll waste a bunch of your time interviewing candidates because the resume screening process is less of a filter than it used to be. Maybe you have to tweak your resume screening process. Are you worried you will hire someone unethical because they kind of cheated their way through the process? Like I'm trying to come up with stuff

off the top of my head, but I think if you can come up with specific concerns or outcomes that might make it easier to address what you're seeing in your specific circumstance. Yeah. Well, have we muddied the water sufficiently to call this question time to stop talking about? We have, and you can prove

it's not an AI-generated answer because of how inarticulate and fumbling my responses have been. Yes. It would have been much more flowery and coherent. I was talking to a customer service rep on the phone the other day, and it was a good five minutes into the call where I was wondering, is this a computer? Oh, my gosh. And then I heard coughing in the background. Okay. And I'm pretty sure that meant it was a human.

Yeah, maybe. or it was someone who built in coughing into their into their model it just had that they had the perfect like siri type kind of customer service voice but they also sounded a little bit bored you know how like you're going through your script and it just sounds like you're it's it's kind of rote i i'm pretty sure it was a human but i had some doubts and i'm sure i've been fooled before and will be fooled again by this kind of thing so we'll see i don't know it's coming for us all. Yep. Dave, will you read our next question? You betcha. All right. And an anonymous listener wrote this one in and says, where I work, the developers do not seem to quote

get source code control systems like Git. I'm not a developer, but I've worked with developers at previous jobs. And usually the developers instituted good source control practices themselves. Our developers know they should push their code to the repo, but only do it weekly or monthly, treating it as a backup. Some backup their laptops using tools like Time Machine so think they have taken care of safeguarding their source code that way. How can I convince them that working in Git, committing their code as they go, pushing regularly, branching, merging, tying code updates to tickets, etc. will benefit them far more in the long run? I love questions like this because they remind me how broad the spectrum of experiences and

and setups and styles is not having source control feels unfathomable to me in 2025 but yes here it is it is totally fathomable to this team yeah they back up their laptops with time machine yeah i mean that's like i copied the folder and named it v1.1111111 because i already had one named v1.11111 with one fewer ones and i just have to remember oh that's that's the version where this feature worked i mean i'm assuming they're not building a web application i feel like it's got to be some specific domain where the standard cultural norms are maybe a little bit less widely disseminated or maybe it's some really legacy system or something or do they use source control and mumps it's like a mainframe thing right what is that mumps yeah i know i know

it's a disease but surely that's not the meaning here it's a programming language with a database built in for like hospital records from the it's from the 60s it's for mainframes oh that's how i remember it back when i was a programmer for the hospital yeah massachusetts general hospital utility multi-programming system system an imperative high-level programming language integrated transaction processing key value database oh it's the original no sequel yeah i suppose i mean everything was no sequel until sequel until like the 70s there's no sequel and there's pre-sequel yeah this is pre-sequel got it anyways yeah maybe it's something maybe it's some niche system or or framework or technology or something and yet i can't see

how this would hurt to use source control like it won't hurt but could you convince a team this is something i like to ask myself like the earth is round could i convince someone who has no idea about that that it actually is could i demonstrate it you know and here it's like could i convince a team that is not using source control that they should i think we have clear evidence that it is possible to not convince them given there are a bunch of people that think the earth isn't round like yes certainly you know you could fail and we have a team here that doesn't use any source control yeah i think you have a couple options one is to go from the top down and find whoever manages this team the fact that the manager also is like source control smosh control

we don't know that'll just slow us down whatever their attitude is it could be easier though to convince one person than to build a movement to build a revolution yeah honestly though i okay here's here's what really caught my eye about this question here you have not just one person who doesn't want to use source control but an entire team of developers where none of them have said yeah we should be using git or something and meanwhile i look around at every developer i know every single one uses git and i think to myself how have they managed to get here they do do it they just push weekly or monthly oh okay okay time time to do my regular backup of all my code yeah good good point good point so so it's not that they don't know what git is or how to

use it they just don't use it like anyone else i know yeah and so then i think there have been times in my career where like for example i remember the developer who introduced me to git this is going back about 15 years and he was kind of a champion for change on our team we had we had been using a different source control system and he said it's time for us to move to git and he would do like weekly training meetings with everybody to help teach them the git concepts and he built like a transition tool to help us you know like using git svn to go off subversion onto git and and he championed it and i guarantee you that without this person's effort the team would have just stayed using the processes that they had yeah but when i look at this team i think

how are they doing this like how could this even function where you don't have your peers code for a month because they haven't pushed it yet i just don't get it yeah i'm curious about the type of systems that they are building because can i imagine it's it's not iterative right they're not like quickly integrating changes and handing stuff off and making new builds and and they must not even be working on the same area yeah it makes me feel like it's either some giant like mainframe erp system or maybe it's some like i don't know embedded system where they're all like maybe they're working on different drivers or like maybe one of them is working on the mobile app and one of them is working on the api back end and one of them is working on the web front

and they're just completely independent but even that if it's web stuff still there's just so much in the air that's like you need to integrate regularly and figure out if it's all working together so i i don't think it's that but i think what i'm fumbling around is this will be easier to do if you can point out concrete benefits pain that they are feeling that this will resolve or new capabilities this will add that they don't even know they're missing but they'll be so much happier once they have because everyone likes their workflow nobody likes to change stuff just for the sake of change there's always some overhead every time some new version of a major dependency we use comes out i sort of just groan and roll my eyes because like

i know they're excited about it the people who made it but for me it's just work mostly pain don't want to do yeah it's just pain that delivers no value but number must go up but forces me to upgrade to the new pain yeah like i just recovered from the previous pain yeah but the point is if this is a chore if this is just like oh this person they're not even an engineer they're just making us do this because yes all the cool kids are doing it it will not work that's exactly what i was thinking so yeah you you have to find what would they like to do that they can't because of the way that they are working with source code or what would what do they not even know they would like to do but you could say hey we could do this thing imagine we split the workout so

everyone everyone kind of gets to work on the same thing instead of all of us in our silos maybe they will hate this idea but maybe they secretly want maybe they want to collaborate and overlap more and collaborate more what no yeah you know this team so you you should be able to think of what what would make them more pleased tie to that yeah maybe i just this is where didn't the question asker say i'm not a developer yeah they did yes and so i'm like this is one of the downsides of being a non-developer manager of developers yeah it's like you you can't just say okay guys here's how we're going to do things here's the new process uh this is how i did it at five different companies and this is how we're going to do it and just get everyone on board not to

say that i like to come in and be a dictator and just authoritatively drop a hammer down and say we're changing it but you just don't know like what it's supposed to look like in the trenches in detail you just have some fuzzy idea that this isn't right you know and in this case you're correct this is probably not right and there probably are better ways like james and what you were saying. Yeah. And I had a thought like, what is one example that could appeal to a team that operates like this? Well, one example could be, if you're not using your source control system regularly, that probably implies that you don't have an automated deployment system. Yeah. So whatever system you're using to build your application, or deploy your web application,

whatever you're doing, that is probably a manual thing that you're doing on your computer. And it's probably like a 15 step process that you've committed to memory because you've done it so much yeah but imagine if you could push code to git and it would automatically build deploy and be ready for you to test on some other environment that is probably not something that this team has but in order to have that they would need to use their git repo more frequently than they are again that assumes that they would like that right that's to some people that is heretical maybe maybe the process for integrating the code is maybe there's like some certification that needs to happen or something so it's just not even that possible but still you could you

could say like great we're not going to certify it every time but wouldn't it be great to have it kind of coming together more regularly and then that should make it easier come certification time that we've kind of like ironed out all the all the easy kinks beforehand you know the other the other advantages that i see of using your source control system more frequently are all benefits to more of the project management or management side of the development process like i was thinking you know creating branches and tying them to specific ticket ids it's like yeah i don't as a developer i don't care about that i just want to do my work but as a project manager i like to be able to trace all the parts that went into a task so that i can know where they are you know

so like for example when i see a git commit email and i see a ticket id in it i can go read the ticket id get context and then see the code and know it's not only know its status but know why we're doing it and know the schedule and it's like the developers probably don't care about that so i'm like well that doesn't that doesn't help so like what are the other advantages oh what about code review well okay yeah so if you use git as a code review tool you can do a pull request for example on github or merge request on git lab or whatever it is and other developers can review your code and as a manager i like that because now i've got distributed ownership of the code i've got more familiarity across the team i don't have single points of failure where

you know the so-called lottery factor or bus factor is good but as a developer i'm like i don't want other people reading my code and telling me it's bad yeah i'm like wait a minute i don't have to read all their code takes all my time yeah i also don't want to do code reviews yeah i just want to write code i don't want to read someone else's so it's like all these benefits that i'm thinking of as a manager which i love the way that we use git which is exactly as i just described i'm like i'll bet the developers would love to get rid of all that stuff so maybe maybe you're stuck maybe these maybe these developers actually have found the new the new way they've been enlightened yeah this is fascinating i guess the other option is you

wait for them all to die and you hire people who old age yeah old age will solve this who are used to working with git did you just assume that they're all really old already i kind of did yeah i guess i did i was thinking that they were really young and inexperienced yeah they back up their code to tiktok or something okay here's here's one advantage that might actually appeal to a team like this listen every other company does it a different way and if you ever want to get another job you need to do it the way they do it wait what it's just like your career will suffer if you don't know this skill oh yeah yeah yeah like but i don't want to change jobs i like my job exactly yeah like well am i getting fired what i'm telling you i'm having a really hard time

justifying why a developer would want to choose to work this way which is so weird because almost every developer does i i personally find it motivating to see other people's changes come in and to see the the product evolve and advance quickly and if someone goes off and and kind of takes their branch into the corner for a month or two sure they can kind of give updates but regularly integrating it back together with the whole app feels nice as a developer i think you get to show off a little bit more and it feels like it builds a bit more energy and momentum there's all these other like you said there's all this like business and team and manager stuff that might not motivate you individually but i think it's i think it's cool to get my code from like

the latest purely well i think code before it's merged in is like purely cost it is nothing nothing positive comes from it it's all it is is like spent labor for for nothing in return no value is something yeah something that's actually valuable it's inventory yes inventory unsold inventory that can't be sold yeah and a lot of code after it gets merged in is inventory yeah that's true let's not talk about that i have one last idea for me on this which is if you're actually pushing code monthly then that is probably if these teams are working on the same code base that is probably resulting in super painful merge conflicts is my guess like can you imagine waiting that long to merge and then trying to merge a month of work with your team

be brutal it's like it's probably just pages and pages and pages of diffs you have to work through to figure out what should be the result yeah now you could have that pain every day in small amounts again yeah it depends like do they just work on totally separate parts of the system right right i think it it just makes it easier to change stuff though as a whole like ever develop every surely every engineer on this team has a bunch of stuff they wish were different i would like to change the system to refactor it this way or add this new pattern or i don't know everybody's got a long list of stuff like this and if you work like this where everyone is kind of working in silos for weeks or months at a time good luck good luck ever making any kind of cross-cutting

change to how you do things it will it will not happen so it can be kind of dangled a carrot of well now you can do these juicy refactorings yeah i don't know i would love to hear more about why it is this way and yeah if if the answer ends up being like you buy them all their favorite treat and say please use git and i will give you more treats yeah please use git every day well they yeah please commit every day to maine or whatever yeah have we answered the question i think so good luck good luck what can people do if they want their own questions to be answered wander over on the web you don't even have to use git for this part but go over on in your web browser to softskills.audio and click the uh ask a question button and we will answer whatever

question you type in there we'll do it we promise we will get to all of them yeah and if we haven't answered yours it's because we're not done yet so right keep waiting yeah that's how you know it's like a halting problem you know we're not done because we answered one more yeah and when you hear no more answers then you know we're done yeah we do have to become one of those creepy brian johnson don't die aliens to in order to achieve this i believe but that's fine i i wish my skin was more bluish yellow already so that's what it takes then we'll do it yellow skin and all yeah all right thanks for listening we will catch you next week