The Startup Ideas Podcast - Building a Software Factory that actually works (Full Course)

Starting point is 00:00:00 What are software factories and why is it going viral? I mean, it's basically this concept that allows you to use AI agents to actually ship software that isn't sloppy at all that is more like a factory. More like think about an assembly line and you're just instead of building physical products, you're building software. And that's kind of the dream. I mean, if you're able to just create this factory that builds software and it's valuable software and you can create multiple apps that generate. revenue and add value to people's lives, that sounds pretty good to me. So in today's episode, I brought on Ross Mike, and he clearly explains the entire process. By the end of this episode, you're going to understand how to set your own software factory up yourself. So enjoy the episode.

Starting point is 00:00:45 I can't wait to see what you build. There's a reason why this concept is going viral, and I'll see you at the end. Today's episode is brought to you by Brex. My company's bid on Brex for a year and a half, and I started because I kept hearing companies like Versel, OpenAI, Anthropic, were using Brex, and I figured if they're using it, why shouldn't die? It's been a game changer. The thing that got me is how

Starting point is 00:01:07 smooth it is. It's got high limit cards. It's got banking. It's got AI that handles the back office busy work like expense reports, which I don't want to do on its own. It's really just built for this agentic world. If you're building something new, it's time to get Brex. Check it out at brex.com

Starting point is 00:01:23 slash solutions slash startups link in the description Mickey welcome back to the pod by the end of this episode what are people going to learn we're going to understand what this bizarre phrase software factory means I'm actually going to show you how I run mine it's a lot easier than you think and it's definitely model and harness agnostic so you don't have to purchase some different product to have a software factory it's going to be fun and it's going to be simple okay so you're going to explain what it is why it matters how it works how to think about it by the end of the

Starting point is 00:02:01 this, people are just going to be able to boot up their own software factory if they want, or if they think, you know what, this Ross Mike guy, I don't, I don't like software factories. I don't like what he's saying. They can pass. 100%. 100%. With the term software factory, there's been a lot of like startups who've started. And, you know, I'm not here to knock anyone's hustle.

Starting point is 00:02:24 But a software factory is completely harness and model agnostic, meaning it doesn't matter what model you use, it doesn't matter what harness you use. It should work, right? Because the software factory is more about someone's workflow, skills, and domain knowledge. And it's packed up in specific skills that they use in their development process. Now, I want everyone to think like the last act that they built. You probably went on codex, Claudec, code, cursor, whatever it is. And you just typed, right? You said, I want to build this. And it built it out for you. You saw it. And you didn't like it. And then you made some changes, right? And you saw the changes.

Starting point is 00:03:02 Maybe you liked it. You deployed it to Verselor to production and you're good to go. Or you kept on iterating. That's the process. The whole point of a software factory is in each step of the development process, how can I best maximize the models capability to get the greatest output, right? And also, how can I move fast? Right. Because if we're using the term factory, there's some sort of structure and speed and conveyor-built nature that allows me to shift.

Starting point is 00:03:30 as quick as possible without losing quality. That's the long bloated Michael Shimulus definition of software factory. I haven't lost anyone, hopefully, Greg. That was perfect. Okay, so understood, but like why does that matter? The reason why it matters is intelligence is continuing to increase. We have amazing models like GPT6 Astra.

Starting point is 00:03:54 A good software factory allows you to systemize and use these models and events. very efficient way versus just typing in and continuing to go back and forth. And I think it's better I just show you how mine works to give people an idea. Now, I'll give my skills are available for free, no charge, nothing like that. But I don't want you to blindly copy me. I would like for you to think about it, understand the process, and then apply it yourself. That being said, I have about five or six files that make my software factory. I have an agents.md file. And for every, who might not be aware with what an agents dot MD file is, it's basically this one document,

Starting point is 00:04:36 this one markdown file that's injected into the system prompt, that's inject, sorry, not in the system prompt, that's injected into the agent chat every time you communicate with an agent. So every time I say hi, if there's an agents.md file, before the high is sent, the agents. dot MD file is sent. And what's cool about this file is I can sort of dictate how I want the agent to act. And we actually did a video not too long ago, Greg, where I talked about most people's agent.md file is useless because they were telling the agent empty file what the code look like and already information that's in the code base that the agent can already know about. But if you look at mine, there's a simple workflow. And this workflow is something that's not native to the agent. So it explains clearly how to do it.

Starting point is 00:05:21 And I'll walk you through every single step. The first step is isolating. There's a skill called new feature. And this is what it does. It says every new feature starts in the fresh get work tree branched from origin main. So agents can work in parallel without conflicts. Never build on me. And if you know when Michael is on Greg's channel, there's always diagrams.

Starting point is 00:05:43 We're about to draw. So with the first step, when I'm working on a project and I tell it to work on a feature, the first thing it's going to do is isolate. And what this basically means is you can think of, Greg and Mike's app. You can think of Greg's and Mike app, like the journey being, you know, we started here, we started prompting here. And this is when we launched a production. We've gone thousands of customers. The way most people work with their agents is at every single step of the journey, they're building a feature, they're building a feature, they're building a feature. And it's a very linear process, right? One feature out of time. One feature out of time. One feature out of time. if you ever try to work on multiple features the same time, sometimes there's conflicts, there's issues, people have, you know, oh, my agent deleted this file, it overwrote this file. What this first step isolate does is the following.

Starting point is 00:06:41 I'm going to show via diagram. So let's say Greg was like, I really don't like our landing page. And I'm telling Greg, you know, our API calls can be faster. When Greg works on this feature, what the agent does using my agent's is it's going to create a new branch. And you can think of a branch as a copy paste of the exact moment where the app is. So Greg is going to work on a new branch, a new work tree of the app. A work tree is basically a copy of the app.

Starting point is 00:07:13 Like you can think of someone copying a block of text, pasting it in a new document, working on that document, and at some point going to merge it back in the original document. So that's what Greg is doing. He has a new document working on that. that feature. And guess what? Mike has another feature has to work on. So I can do it at the same time or I can do it a different time. I now have my own branch. Here's what's cool. Me and Greg and another person or you yourself with 50 different agents can work on multiple features and there's not going to be an issue of one agent overstepping on another agent's work. This happens a lot to people. If you ever

Starting point is 00:07:52 see tweets Greg of people saying, man, like I was working on this. And the agent deleted a bunch of stuff almost always. It's because people have their agents working on different features on the same branch. And the agent will do what you tell it to do. So you told it to update the landing page, but then you also told it to make the API calls faster. And now it's noticing, oh, these pages, the way they're calling the API suck. Let me delete these and write these again. But then it was working on a design on that page.

Starting point is 00:08:22 So issues start to arise. Isolate allows me to work on multiple features at the same time, parallelly, without having an agent overstep on an agent. This is the first step of my workflow. This isn't native to the agent. So this makes sense to exist in the agents. And when Greg is done or when I'm done, and I'll explain like the workflow for this,

Starting point is 00:08:50 but just to close the loop, when Mike's done, all I do is merge these changes back. And then what Greg's done, he can either do it before me or he can do it after me. But we're merging into the main document. And there's not going to be conflicts because we worked on a copy at first. So this allows me to ship fast with multiple agents. If I show you, and I can maybe show here, if I show you my terminal, if you notice there's four tabs of Bezillov, three of them finished.

Starting point is 00:09:22 but I literally have four different features on the same exact app. One's working on an email client. The other one is working on a computer environment, a Linux environment. The other one is doing a landing page update. So I can work on these confidently while still having isolation. It's not native to the agent. Thisagent's dot empty plus the new feature skill allows me to do that. I'll pause here.

Starting point is 00:09:47 Any questions, Greg, so far? Yeah, I mean, it's like crystal clear. I think it's, the way I think about it is if you actually had a team of engineers and you are trying to build an app, you obviously wouldn't be building it all on Maine and having everyone just being pushing to Maine and stuff like that. That just doesn't make sense. I think, you know, I'm non-technical. And I think you have all of a sudden all these non-technical people starting to build apps and they're running into this. You know, you basically made mention. you're kind of like, oh, hey, man, like sometimes it happens that agents overwrite stuff. I mean, the truth is 95% of the time if you're going to have agents kind of mess up and an override thing. So this is like, to me, this makes a lot of sense. And it's sort of a bigger idea, which is how do you structure your, you know, the way you

Starting point is 00:10:46 work with agents to be more like a team versus, you know, you know, know, yeah, no, just more like a team. Yeah, I mean, and I mean, it's a better name. Team is better than isolate. I might take the team. But that's basically. That's why I'm the marketer and you're the engineer. But yeah, that's basically what it is, right?

Starting point is 00:11:07 Isolate sounds like some sort of like way protein that I'm going to enjoy. So if you guys check the link, there will be a name update very soon. But that's basically what this does. It allows for me to have multiple agents working on different tasks, all at the same time without overstepping each other. And there's also a cleanup process I won't get into. Like once the branch, like once the work is emerged in, the work tree gets deleted. All that type of stuff happens.

Starting point is 00:11:33 So that's step number one. Number two is the actual building. And this I find has, this was a skill that I shared originally way back, but now it's part of my workflow and I shared with everyone is code structure. We are going to talk about models soon. but one thing about the models is they are great at doing the work. It just might not be done the best way, right, especially when writing code. There's, for example, Fable is, I would say, one of the first models to really write code that I would say, huh, like, this is better than some of the best engineers I've seen, right?

Starting point is 00:12:13 Like, Fable runs, writes really good code. But even Astro, which is workhorse, most powerful model, my favorite model, some of the code quality decisions it makes is, eh. And it's not that the model's not capable. The models is just getting it done. And if you could get it done in a sloppy way, it'll get it done in a sloppy way. What the code structure skill does, it writes it in what's called a service layer architecture. Won't bore everyone. But basically, it's written in a way where, let's say you needed human intervention, you hired a developer.

Starting point is 00:12:42 Very easy for the developer to catch on. or you yourself are a developer, very easy for you to catch on. But even for your agents, for them to come back to the code and read the code, they'll be like, oh, like, yeah, like it makes sense. Like I've had like, I've had GPD 5.6-Soul write code and it works. It does what it's supposed to do. But then I'll have Fable review the code and Fable will be like, this is disgusting. Like there's duplications, there's functions all over the place. There's dead code.

Starting point is 00:13:10 Right. So just because it works doesn't mean it's written well. this skill gives the agent a guideline on how to write code. So going back to this diagram, again, I'm working on a new feature, right? The first thing that happens is isolate. The second thing that happens is, and I'm going to, I hope people are seeing the factory nature of it. The second thing that happens is the building. And in building, we're using a skill called code structure.

Starting point is 00:13:37 And basically in this process, when I'm telling the agent, oh, build me, this ending page, build me this feature. For example, I've been working on this app. And the one feature that I wanted build is I wanted a skills like repository where I can have different agents of mine connect to this app and store the skills. And I basically told the agent, build me that. And as it was writing the code, it kept referencing the code structure skill. So it started to write code in a way that me as a developer, I can review.

Starting point is 00:14:10 But, you know, God forbid I need to hire somebody. To review the code, it's not going to be the slop canon where they're confused and they don't know what to do or they overcharge you. It's written in a way for a developer to understand. And I find that if you use another agent that doesn't have context on your code base, it will understand it very well. So first we isolate, excuse the bad name. Second, we build. And all of these are done in an opinionated way that allow the agent to move fast and to do the thing as best as possible. can be.

Starting point is 00:14:44 I think this makes sense, right? Great. Yep. Keep going. So step number three is prove. Here's the thing. You know, agents can't pinky promise, right? So, like, if you push an agent enough, well, GVT6 Astra is the reason why it's a big leap in model

Starting point is 00:15:03 capability is like it, like the, like, it is the least hallucinating model, which is fantastic. But like, that model aside, most models you can kind of push to, like, it's a little bit. like lie or to believe something or sometimes it'll straight up be like oh I actually realize I didn't do this work for you so one thing that I have the agent prove the agent do is to prove the work that it did and this is probably my favorite skill there's two skills involved in this evidence driven testing and basically what evidence driven testing does if your machine has the capability to do it It will literally record the before state, meaning before the feature or let's say you're trying to fix a bug. It will record the bug in action.

Starting point is 00:15:47 And what it will do after is after it's done fixing, it will record a working version after. Right. So this is what evidence-driven testing is. It proves that the fix actually was made because you'll be surprised sometimes the agent will write the code and it'll think it worked, but it didn't test it or it didn't prove that it worked. And it just told you it worked. So I need to make sure that the agent actually prove this work. But let's say, let's say your machine doesn't have the capability to do that. There's another skill called before and after.

Starting point is 00:16:20 And I can actually, let me pull up a PR to show this, Greg, because I think it is better show you than I tell you. This is a PR my agent made. I didn't make this. The agent made this. and I wanted it to create an admin email page and connect an email service that I made. Look at what it did. It showed me a before state where the page didn't exist.

Starting point is 00:16:46 It just, it didn't. And it shows me the after state, right? So for every feature that I'm building, every PR that I'm doing, I am getting a before state and after state. And the reason why this helps, this helps me like, I'll be honest, I'm not reading all my code nowadays. Like, I might like, okay, did this. Like, let's see. Like, barely. It's a skim.

Starting point is 00:17:10 It's a skim situation. I'll be honest, the skims have even become less and less now. But a lot of the things that I'm doing now is I'm working on the front end where I make sure that it's following my code structure skill and I make sure I'm getting before and after screenshots, right? I can even pull up, let me pull up another example. Here's a more prominent example where I was working. on a specific computer feature where I wanted to give my agent a computer and it just didn't work, right?

Starting point is 00:17:40 The first run didn't work. I didn't even have like my software factory set up. I was working on a different machine and it pushed the code and it didn't work. As you can see, nothing's going on. I told my agent it didn't work. Use the skills. Use the factory. And this is the after screenshot with it actually using the app.

Starting point is 00:17:59 Right. So these skills, the prove and the before and after, force the agent to give me factual, like a before and after proof, whether it's a video or its screenshots. And there are times where it'll do the before, but then it'll do the after and it'll be like, oh, I just looked at the after screenshot or the after video. And I didn't really finish the feature. So I'll go back to what? It'll go back to building. Right. This is the factory nature of it.

Starting point is 00:18:29 I didn't have to tell it, oh, yeah, you failed your before and after. Go finish. The skills are written in a way where the agent knows, okay. The before and after criteria hasn't been met. I have to go continue on building. Right. So this gives me a visual representation of the work that's been done. It makes reviewing, especially if you're a non-technical person, it makes reviewing easier

Starting point is 00:18:51 because, you know, I just look at some screenshots or video. Right. So in the building process, at some point, I'll have a PR where I get to see before and after. And even though I might not understand all this mumbo-jumbo, it will explain, it will show me visual proof that the work's been done. Now, I already hear somebody asking, what if the proof is not visual? And there's actually a PR. I'll show someone performance. I think it's this one. Okay, so I wanted to do another example here.

Starting point is 00:19:27 I wanted to do a performance update, meaning for one of the apps I was building, the clicks weren't snappy enough. Like you can see now everything is snappy, it's loading fast. That wasn't the case. So I told the agent, fix it. And it did.

Starting point is 00:19:42 And it did end up giving me screenshots, but let's see this was like something that it couldn't give me screenshots for. It will write test, and then it'll give me the results. In this case, it checked the speed at which the page was loading before. In this case, one of the pages 815 milliseconds.

Starting point is 00:19:58 This is a sin in web development. This cannot happen. And mind you, this was written by GPD 5.6 sole, right? Great model. But it got it down to 60. 817, 61. So I have, again, actual proof by the agent that it's done what it said it did in the review process. I'll pause right there.

Starting point is 00:20:20 Anything I've missed so far, Greg? Or does it? So, you know, if you're trying to build a software factory, trust obviously is going to be a big part of that. And I think what you're saying is, hey, if you know, you're going to, if you're going to have all these agents, you know, building features, building apps, we need to be able to trust the things that it's going to create. So what you're saying is, here are a couple skills that allow you as like the agent manager, where in this case, Mickey, you're like, you are the agent manager, right? You're not deep in the code anymore. You're kind of just looking at what's happening.

Starting point is 00:20:56 And what's cool about the before and after visual stuff is it's kind of good for, you know, millennials and Gen Z or people on, you know, Instagram stories or snap stories, stuff like that. It's almost like you're just like clicking through story to story. Yes, yes, no. Like it's bite size. Basically, that's literally it right there, right? It's allowing me to build trust with the agent. Because, and what's funny is, like, you said it earlier, and it kind of clicked in my head. This is what, like, normal organizations used to do with their engineers, right?

Starting point is 00:21:29 It's like you build a feature, and then there was someone whose job, like a senior engineer, whose job was to review your work, and you would have this PR with this description. And it would show, okay, basically this is the work that I've done and this is the test that I've written, right? So it's basically the same thing, except now we're doing it with machines. Like, that's essentially an example of the video. this is a this is me using cursor cloud agents it says proof of improvement and like this is a video of the agent at work using the app right so this allows me to your point like instagram tictock i can watch this and see all right okay the agent actually built this and it works this makes it easy for me

Starting point is 00:22:13 to not have to read code and i can just merge away and live my best life and go outside and touch grass So there's that. Now, we have one final step, which is the ship step. Now, the ship step I mentioned before and after, but there's this skill called GREP loop, which uses a third-party service named GREPTal, which is a code review agent. Now, you don't need to use a code review agent, but if you're really serious about building software and it's going to be used by users, I highly suggest using some code review agent. Gretel is my favorite, code rabbit, macroscope. There's tons of good ones out there. But me using greptile, they have this skill called grep loop.

Starting point is 00:22:54 And basically what this does, and I think I'll show it with this PR, Greptile leaves these summaries and then it gives feedback. For example, this was the initial feedback it gave on the PR. There was some issue with, you know, pagination right here. Some menu space wasn't preserved. So it gave this feedback, meaning the agent that wrote the code missed these things. And that's fine. It happens, right? Even humans miss these things, right? But what happens is greptal not only gives feedback, it gives a confidence score. Now, this is a five out of five because after the feedback was given, if you see my name and then this line over here, the feedback was addressed. My agent addressed, my agent addressed. My agent addressed, this score was a three out of five. What that tells my agent is that there are things that it missed and it needs to look at it. What the grep loop skill does, and And by the way, this happens automatically.

Starting point is 00:23:50 Someone doesn't have to write grep loop. The agent will do it automatically. What it does is as follows. It says it opens the PR with the before and after proof embedded in the description. Whenever the change has a visible surface, measured numbers or output periods when it doesn't, it'll give you numbers, right? Or screenshot. And then look what the agent does.

Starting point is 00:24:08 It runs GREP loop or GREP loop apps. The difference is GREP loop apps. If your file change was like 10,000 lines plus, that skill activates. again, you don't have to worry if the agent does it itself. But look what it does. It says, Grebtal reports five out of five until resolved comments, finished by presenting PRURO. Basically what this means is it will, the agent will take the feedback it got. It will go back to build.

Starting point is 00:24:35 So check this out. Let's say we're at the point, let me write this down. We're at the point. We're at built. We're at ship, right? And it's now running Greep loop. when I get a feedback score, a confidence score, and it's a three out of five, what now happens, remember we were talking about loops, this is actually a good loop. What happens is this goes back to building. Now the agent goes back to step two. It starts to build after it builds what does it do. It proves and then it ships. And here's what happens. Automatically, the agent will wait for a new score. Gretel then gives us a four out of five. We caught some things, but there's one final thing we missed. go back to building, right?

Starting point is 00:25:18 It builds, it proves, and guess what it does, Greg? It ships. And now I have a five out of five. When I have a five out of five, what's left now is for me to merge. And I think maybe I have an open PR right here, so I can show you what that looks like. What's left for me is to just click merge. When I click merge, what happens is this is finally back in the main copy of the app, the main version of the app. And I did this while working on 15 either simultaneous

Starting point is 00:25:52 version, sorry, features, 15 different features with different agents, sub-agents, all that type of stuff. What this allows, and this is the factory nature, it allows for an agent to have an isolated instance where it can work on its own. It has guidelines on how to build. It has a methodology to prove its work, and then it has an external service proving its work, checking its work, and if its work is not up to standard, it has to continue in a loop working until that standard is met. Once I get a five out of five, this is when I enter the picture. This is what a software factory is. Notice we didn't talk about model. We didn't talk about harness. It's all workflows, skills, and a little bit of domain knowledge, right? Not

Starting point is 00:26:38 everybody works the same. This is how I work. But I found great results with this. I'll pause right here, Greg. Let me know what I need to further explain our add-on. I'm like what's clicking in my head is just really this physical factory analogy for a software factory. So just to like summarize, I'm going to tell you how I'm, I'm singing. I want I want your thoughts. So like the isolate piece and the step one, that's like a factory taking a custom order and giving it its own station so it doesn't mess with the rest of production. So you, you know, you called it a work tree, you know, in software. It's a branch, a work tree, an isolated environment. But, you know, that's the basic concept.

Starting point is 00:27:26 Yes. Number two, build, right? Build is the assembly line. So the agent is actually like cutting and welding and assembling and wiring the product. Obviously, in software, you're not doing that. You're writing code. You're changing files. You're adding structure to things.

Starting point is 00:27:44 You're actually creating something that's real in a software sense. Step three is the proving step, which is basically a fancy way of saying quality control. Basically, right? Before anything leaves the factory, someone has to test it. You're not just going to create a product and not have people test it. Does it turn on? Does it fit? Does it break under pressure? Like all the things.

Starting point is 00:28:11 I'm picturing like a car factor or something like that. In software, you know, you can run tests like, well, you can run tests. You can preview it. You can do logs and the screenshots you showed. And then lastly, the shipping piece, it's basically like once it passes quality control, you know, it's going out the door. but there's going to be some things that you have to like let you know you're going to have to merge it you're going to have to deploy the PR you're going to have to do release notes you're going to have to give feedback back to the product team because maybe it doesn't pass quality control and then you have to like do that loop again right? I think I might actually rename everything you like I've written to what you said because like now I'm realizing like oh yeah my names are terrible. Greg says makes sense. That's exactly how this works. Cool. So basically what we're doing

Starting point is 00:29:10 here is we're taking a factory and we're making a software factory. Basically. So and this is why like I've seen it. Not to knock people startups and products and stuff like that. Like a software factory is not a product. It's not a special harness. It's not a like, oh, this company built a software. No, a software factory is literally just a bunch of markdown files. And this is also another insane thing. off topic, maybe a different video, is some startups are now an agent with a couple markdown files, right? We've really entered that time. So I hope this made sense for everyone and it's excites anyone. I'm very excited with the time we're in right now because a lot of things are possible now. 100%. I mean, no, this is, this is insane that you're actually able to do this.

Starting point is 00:29:56 It's cool that you, people like you are like sharing this because I think it's worthwhile. I want to just do one quick note on grep loop or any code review software. I think if you're serious, you know, no affiliation with grep loop or anything like that, but I think if you're serious about creating software, having, you know, some code review software is pretty, like I don't understand why you wouldn't use a grep loop, a code rabbit, one of these tools. Something, right? Like, it makes Like, because, again, if you have, like, you know, we, in business, and I know, because a lot of business people are watching, you know, we take the service we provide and all that stuff seriously, but it seems like with building software, like, we just don't care, right? And if you have people who are going to use your app, like, I don't know, like, there's a level of, like, empathy I have for the user on the other side. And, you know, like, a lot of these, like, startups, because they've raised billions of dollars, like, they have a lot of free tiers.

Starting point is 00:30:57 You can cycle through free tiers and use a bunch of this stuff for free. So I highly encourage if you're building something that you're serious, you're passionate about, I would use a code review agent of any kind. Yeah, cool. Thanks for coming on. Thanks for sharing the sauce. I'll include links for where to follow Mickey on the internet, on YouTube, all those places, his software that he's creating to go give him a follow.

Starting point is 00:31:22 And dude, I'll see you next time. I appreciate you, Greg, as always. Thank you, everyone for showing love and watching. Yep, we'll see you in the next one.

The Startup Ideas Podcast - Building a Software Factory that actually works (Full Course)

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.