PurePerformance - 80% AI-Written Code: How 1KOMMA5 Measured the Real Impact of AI on Engineering

Episode Date: August 3, 2026

AI coding tools are everywhere, but how do you prove they're actually making engineers more productive?In this episode of PurePerformance, hosts Brian Wilson and Andi Grabner welcome Michael Reichenba...ch, Platform Engineer at 1KOMMA5°, to discuss the company's journey toward more than 80% AI-written code. Rather than relying on anecdotes or hype, Michael shares how his team designed a real experiment to measure the impact of AI-assisted development.We explore the metrics they chose, why traditional DORA metrics such as deployment frequency and change failure rate were not the right indicators, and how they instead focused on "Time to Code" from ticket creation and first commit to merged pull request. Michael also explains how AI lowered the barrier for contribution across the organization, enabling even non-engineering teams to prototype and build solutions faster.The conversation also dives into the operational side of AI adoption, including AI observability dashboards, budget controls, Slack alerts, usage monitoring, and the surprising decision to intentionally limit AI spending during their proof of concept.Whether you're evaluating Cursor, GitHub Copilot, or other AI coding tools, this episode offers practical lessons on measuring value, maintaining quality, and scaling AI adoption responsibly.Links we discussedMichael's LinkedIn: https://www.linkedin.com/in/michael-reichenbach/Klaus's LinkedIn: https://www.linkedin.com/in/langenheldt/Talk at Cloud Native Munich: https://www.youtube.com/watch?v=Ntf0h0vFuMQ1Komm5 Website: https://1komma5.com/Kenote from KubeCon: https://youtu.be/P1phxZHJGrA?t=570&is=9DYnbK8VGorMmaXoMichael's YouTube Playlist: https://youtube.com/playlist?list=PLn-u2xOcMlXlVweZ0aB4pu6VM6Xv6895i&si=MZ6jzF-VwIAY3HFF

Transcript
Discussion (0)
Starting point is 00:00:00 It's time for pure performance. Get your stopwatches ready. It's time for Pure Performance with Andy Grabner and Brian Wilson. Hello everybody and welcome to another episode of Pure Performance. My name is Brian Wilson. And as always, I have with me my wonderful, amazing, talented co-host, Andy Gravner. Hello, Andy. How are you doing today? Oh, very good and especially if I get so many nice words coming from your way over to the pond to me.
Starting point is 00:00:45 Hey, why not? sometimes I like to be nice exactly hey you know what this is the first episode after the World Cup oh yeah did we do a prediction who wins I don't remember there was some talk about it yeah I think both
Starting point is 00:01:01 both of them were knocked out but I know that Dan Gomez Blanco he was our last guest he's originally from Spain so he obviously should be happy he should be happy exactly the best team won
Starting point is 00:01:18 unfortunately the teams that we three are representing today and that includes our guest we all didn't make it too far I think actually the US how far did the US make it round of 32? I forget which
Starting point is 00:01:33 round but it was you know the big takeaway from the US playing I took away was that if we don't get our way we get our present involved that's right There was one thing. But I think enough with the World Cup, and maybe it's still a segue,
Starting point is 00:01:50 because I'm obviously representing Austria. We got kicked out by the later world champion. Michael from Germany, our guest today from 1.5, how I think Germany also made it to the round of 32. Yeah, I think a new 16th finale. We got kicked out. Hi. Thanks for having me here.
Starting point is 00:02:13 Yeah, and thanks for listening. for such a long time to Brian and myself and wondering when will they finally stop talking about football because today's topic is definitely something much more important. Michael, I met you and your colleague at Cloud Native Summit Munich just a couple of weeks ago and you were on stage and I just want to be precise. You had a keynote. It was called How We Transform 1.5 to more than 80% AI written code. I was really blown away by your presentation.
Starting point is 00:02:45 It was you and also Klaus Langenheld. It was a great talk about how you adopted coding agents and lessons learned how you measured the impact. And for me, I really wanted to get you on stage here and learn more like your approach, how you can really, how you tackled in showing the value of AI because I think right now everybody is trying to figure this out. and this is definitely my biggest question that I have for you today,
Starting point is 00:03:16 but to kick it off, can you tell us a little bit more about you, who you are, your background, also what 1.5 does before we go into the main topic? Sure, definitely an interesting topic. I mean, AI is everywhere right now. Maybe to myself first. So I'm a platform engineer at 1.5, been one of the first platform engineers there with the company since 2.5 years. So this also tells you it's a pretty new company,
Starting point is 00:03:42 startup. I think we just had our fifth anniversary. And 1.5 has this vision that they say living on wind and sunlight forever for free, bringing electricity to all of the households and dynamically controlling electricity to follow the wind and sun. So not just putting solar panels on roofs and batteries in basements, but also steering this capability dynamically to, yeah, follow the wind and sun. Because in the night it gets dark and the wind dies down, then you still need electricity, more so than in the day maybe even. And this is what 1.5 is creating, not just selling solar panels and batteries to all of the households to install, but also then getting this software layer in it
Starting point is 00:04:33 where we steer dynamically steer these capabilities and create a virtual power plant. My role there is as a platform engineer, building up the platform on which the other engineers built on. So getting the CICD pipelines good and performing, getting the terraform setup correct, but also in the last year introducing AI and the company. And this is what my talk was about, where we really started this journey back mid-last year, where we wanted to introduce AI in the company. Can you, before we go into the I topic, first of all, thank you so much. I really think it's a great goal and ambition that your company has, that you're working for.
Starting point is 00:05:20 And, you know, I think we all know now the next heat wave is coming here across Europe, that this is something we definitely all need to tackle. But I want to have one more question on your tech stack. Can you tell me a little bit about what's the coding languages, the people are using, what's your CICD look like? And if something has changed, since you adopted AI, maybe let's take the state of what it was before you chose AI. Yeah, so our tech stack,
Starting point is 00:05:55 there's one very interesting bit, save this to the last. So our tech stack is pretty, I would say, usual. We are on GCP, we have GitHub actions. We have TypeScript Stack. We have on Versel some apps with NECJS, but we and NESTJS, but we also have some Python apps.
Starting point is 00:06:18 And then we have this whole IoT layers. We actually created our whole own IoT device that we shipped to the households, which runs a set of microservices in that with a written Golang, which then connect and steer each individual household. household asset. And the most interesting part, I would say, is that we have not introduced Kubernetes and are actively trying to avoid introducing Kubernetes. We are purely running on Google Cloud Run, with one very small exception, just to stay
Starting point is 00:06:52 faster and to avoid this overhead in a platform team of having to manage a whole Kubernetes cluster, especially when it starts growing and fast changing. This choice has been really good so far and we've not regretted it because CloudTurn also adding capabilities and growing. And I think this was also one of the arguments. Either this was a question after your keynote or in the hallway when somebody said, so you're speaking at Cloud Native Summit Munich and you are saying, you're not using Kubernetes, but then the point is Kubernetes doesn't mean Cloud Native, right?
Starting point is 00:07:28 You're using all of the Cloud Services from Google to run your... to run your digital services. But I got one question, though. There's also GKE, right? If you want to avoid the overhead of running a Kubernetes cluster, was GKE ever a consideration? We actually have GKE for our endpoint where the devices connect to. So our IoT fleet connects to GKEe,
Starting point is 00:07:54 just because we need MQTT there and you can't have like, you can only have HTTP endpoints in Cloud Run. I think they might have changed it recently, but yeah. So this is our only GKEe. And we actually never had like the need to introduce Kubernetes because we scale so well with Cloudrun so far. But yeah, we looked into it, but thought, okay, the trade-off is there because you still need to manage all of around, right?
Starting point is 00:08:23 So you still need to set up the whole platform stack, I would say. It's yes, the clusters manage, but you still need the whole platform stack. And so far, we've gone really well with CloudTram. See, Andy, that's an example of picking technologies based on real reasons. So Andy and I, or I often bring up the fact that a lot of people will say, oh, I'm going to take and run this in Kubernetes because that's what everybody's doing. And my thought is always, well, there has to be a reason you pick Kubernetes, not just because people do that, you know.
Starting point is 00:08:54 And obviously, this is a great example of, in your case, it was like, yeah, I'm not going to pick Kubernetes because of these specific reasons. You know? So to me, it's music to my ears. I love hearing that. It might change even in the future, but I'm not sure. So because you also ask me what changed with AI, right? And one trend I'm seeing, and this is probably not us alone,
Starting point is 00:09:21 is that I have a feeling or I'm seeing is that we're shifting more towards these complete vertical providers. for a cell, for example, in our case, where we're even moving more away from a self-hosted stack, let's call it, self-hosted platform stack and more towards a complete integrated stack
Starting point is 00:09:41 where you can iterate even faster, right? Because there are always some bottlenecks, be that you run your own Kubernetes, be that you have Terraform or even GitHub Actions and a custom deployment pipeline to Google CloudRun. It's always having some kind of bottlenecks.
Starting point is 00:09:59 you need to work around as platform engineers. And I believe we're shifting more and more towards some kind of vertical slices, hyperscalers, or whatever you want to call it, like Rasell that also tightly integrate in the whole AI loop. That's a trend I'm seeing also. Or maybe call them a platform as a service, right? Because basically that's what they are.
Starting point is 00:10:20 They provide exactly what platform engineers have tried to build over the last years to avoid or reduce the complexity of Kubernetes and other things. Exactly. Hey, so coming to your AI journey, I would be interested in, before we talk about measuring the impact and kind of also justifying the investment, was there a certain moment where you said,
Starting point is 00:10:49 this is now the moment where we need to seriously look into this? Was there any need for it? What was the triggering point to seriously? then look into coding agents, or was it just because everybody at the same time figured out now it's at the time where it's actually really producing good value? We're actually a bit ahead of the curve there. So we started looking into it in the middle of last year. I think November or December, so this Opus 4.5 breaking point where then everyone started realizing, okay, shit, this is something we need to get on. But we actually started earlier. And the trigger point was that we had some really early adopters in the company and just talking to our users and we're creating the platform, but also through various communities.
Starting point is 00:11:40 And just we got this vibe that there are some few people, it was only a handful, that actually started to leverage AI more and more. And we wanted to find out who is that and how are they getting good results out of it. And it was me too, right? So I was one of these people, but also some other. in the different departments. So we actually started a very honest survey where we just asked, hey, what shadow tooling are you using? Because there was like no AI tool, it was approved, no, not compliant or whatsoever.
Starting point is 00:12:14 We just asked him honestly, hey, here, please give us the tools you're using because we want to know and find out what the actual state is. And surprisingly, we found out that quite a few, was it like 30% I believe was not using GitHub copilot. So we had a GitHub co-pited license at that point. But 30% were not using GitHub co-pilot because it wasn't giving them enough value. So we started to look further into this and then founded a chapter where we said, okay, let's bring everyone together in this community of practice and actually get the people
Starting point is 00:12:54 that are at the frontier together to talk about exchange thoughts and ideas. and why they're ahead, what's getting them value. And that's the point where we realized in the chapter, just talking amongst each other, we need to do something. This is having an impact, and we need to get this somehow trialed and approved, and then actually buy-in from the company. And this is where we then started this journey.
Starting point is 00:13:22 It was in, I believe, September or something last year, where we then actually went onto this journey of trying to introduce Cursor. Why Cursor can talk a bit later, but trying to introduce Cursor in a company. And where you are, that means in that stage where you had co-pilot anyway through GitHub, and people still chose something else. That means they just did this on their own. They didn't ask for permission. They just, I don't know, use the personal credit card or they just, like, how did that work?
Starting point is 00:13:55 they actually saw so much value in getting another tool other than other than GitHub copilot that they used their own money to buy it and that was like the strongest signal we got and it was okay, you need to look into this right because people are using their own private money
Starting point is 00:14:13 for the company in order to get something that's unusual I would say. And then we started looking further into this and found out that GitHub co-pilot at this point wasn't sufficient, right? And when you look at it always was a few weeks, month behind, they were slow in adding new models. They were slow in actually getting this agent development in their auto completion wasn't so good at this point. At that point, remember last year,
Starting point is 00:14:43 auto completion still mattered. Very fast didn't matter anymore, but at that point, and we looked then for a tool, we looked around, okay, what should we adopt? We have clot code, we have codex, and we have cursor. And then we some people were pulling towards cloud code. I think GPD Codex wasn't that popular at the time.
Starting point is 00:15:07 And then we had cursor, which was its best in class and auto-complete. But the thing that really got us to implement or add cursor and try with it is that it had all of the models, the latest models, and always they could choose any model.
Starting point is 00:15:23 And for us, it was really clear at that point every week a new model comes out. And we don't want to be bound to just one provider. We want to give the users the free choice to switch the models and actually use the model that's best if you're going to invest in AI. So that's when we started to set up a trial with our early adopters. I also then remember from your talk. I hope I remember this correctly, so correct if I'm wrong, but I remember you then say,
Starting point is 00:15:54 when you did the PUC, the proof of concept, the company, like Kursa, they came to you and said, hey, you can have unlimited tokens, right? Go crazy because this is your chance. But you made a very deliberate decision in not following the lead. So can you tell me what did you decide and why did you decide so? Yeah, and that's a really good point. Really funny story also because we did the people in the chapter actually and there was like this pull.
Starting point is 00:16:23 okay, we want to get this out to the whole company. Let's like flush everything and just give everyone unlimited credits. That's what Curse came along in November. We wanted to do this trial. We did a contract and everything. And then Chris said, hey, here, you can have unlimited credits for the whole company, 30 days and we even have this program with onboarding. Then we thought about it, okay.
Starting point is 00:16:47 But what if it doesn't turn out? What if the metrics don't match? because maybe some background, the company is very cost-abare. And you have to prove with data that you're actually doing an impact and it's worth of an investment. And what if we now release a AI coding tool to the company? And like I said, November, right? So Opus 4 or 5 didn't drop yet. No one was sure this is the future, right?
Starting point is 00:17:15 It was still a leap of faith. And I said, okay, if we do this and the people like it so much, much because they get dopamine hits and suddenly the codes gets written faster, but there isn't a measurable impact. So maybe there is no change in change lead time or any other the other Dorom metrics. Maybe there isn't really more work done, but it just costs us money more than I think it was cursor costs. I think GitHub co-pilot is $20.
Starting point is 00:17:45 And I think a cursor license is 100 or 200. I'm not sure, but one of these, though, was a significant more money investment. So I said, okay, let's restrict the trial against, like, cursory, no, do it with everyone. You get the most benefits.
Starting point is 00:18:04 We said, okay, no, let's only do it with the chapter. Let's take 25 people who are already in our early investment group, right? The people that are already interested and let's do it with them. So that's actually how we started. And we thought about how can we do this, right? So we didn't want to just release it to everyone and then just go at the end, yeah, how did you like it?
Starting point is 00:18:29 Like I already hinted, we wanted to measure things. So we came up with a plan to do an actual almost scientific experiment, I would say, we even called it, I think, experiment where we said, okay, let's do an experiment and do a two ways of measuring this. let's capture all of the objective data we can get. Because we didn't know what data would actually be useful. We don't know what the numbers change, right? There was also very few other companies who actually measure it objectively. There was a lot of talk about doing it subjectively and so on, asking the people how they feel.
Starting point is 00:19:08 But there was a really little about what metrics can you measure, and everyone was unsure. So we just, okay, let's measure everything. and then decide afterwards, like metrics matter. So we really ingested all of the GitHub data, we looked at the linear data, we looked at the deployment, Dorometrics, we had them in place already,
Starting point is 00:19:29 and we looked at, I think we then, merge time, we also looked at and PR size and all of that. And then we also said, okay, but we also need subjective data, and we want to combine those, but we don't want just subjective data at the end, We want it three times, actually. So we constructed three surveys, one for the beginning, one for the middle, and one for the end, to have a trend, but also a starting point.
Starting point is 00:19:58 So we have a start survey where we lay out the baseline where we actually ask, okay, how was your coding experience before, right? Before of the experiment. How is your felt productivity? How's your felt satisfaction? and ask these kind of things, how faster are you and so on? Then we had this in the middle and then at the end. And this turned out to be a really good format for us
Starting point is 00:20:26 because it, and we then extended this service as the first trial went on. So we actually improved the last survey a bit after we did the second one. So that turned out really well. and got us a lot of data. Can you fill us in then? What was the, because I'm curious now.
Starting point is 00:20:48 What are the data points and the metrics that worked and which ones didn't work out for you, didn't tell you anything? So what worked out really well was looking precise. So data points worked out was looking at the zooming in at the time to code. So the time from a linear ticket gets in progress or your first commit until that PR is ready to merge or merged. So this is this coding time window.
Starting point is 00:21:28 And this reduced significantly. And the data actually here, I think we also shared it in the presentation. I think it reduced by 70%. So the coding time went down by 70%. But if looking at the overall change lead time, so it would be the whole thing, right, until deployment, there was only a small improvement. And I think the merge time actually went up.
Starting point is 00:21:59 This was like also countermetric, right? Because the PRs got bigger. You had to put more thoughts into it. is something we also saw. But what definitely saw is like this huge decrease in coding time. And what metrics didn't work out at all were some of the dora metrics, especially to lagging ones. So change failure rate wasn't really telling.
Starting point is 00:22:21 Then deployment frequency also wasn't really telling because you could have projects where you suddenly push and you don't know how skewed the data is. But there was some indications, for example, So this is data from our second global global rollout on February, we did then the whole organization thing, some data here, where the merge pull requests per month actually went up to by 24%. Or the PR, the, what is it here? The commits per month went up by 16%.
Starting point is 00:23:01 And the change lead time went down by 18%. So there were some good numbers in there, but the qualitative number from the DOR metrics didn't tell us much because there was just lagging. I'm curious with the code time, right? Did you have to put in any kind of safeguards to ensure that that time wasn't improving because developers are just taking whatever they're given
Starting point is 00:23:32 and saying, good push it? instead of reviewing it, making sure it's quality code and all that kind of stuff, right, to prevent people from taking shortcuts and suddenly introducing a bunch of, you know, bloated or just inefficient code? During the POCs, we did not, because we're mostly focused on introducing the tool. But we definitely saw this as the big work. So in our survey, we also asked a lot of other questions, like what are your worries about AI? I coding. And the biggest worry was that it produces more slop, or the code base gets harder to
Starting point is 00:24:12 maintain. And this is then something we actually started to heavily look into afterwards, where I'm also focused on right now. I'm in an AI task force or Tiger team, where we look at, okay, how can we retain this high quality and still ship faster and move even further towards this agentic workflow? But I think it comes actually after introducing the tool, because otherwise you will like gate people or have them hold back. I think you first need to speed up before you then can slow down a bit again and learn from the mistakes and then actually iterate and see what kind of controls, verification you can put into the systems.
Starting point is 00:24:54 Makes sense. Yeah. Michael, today, before we kicked off this week, you also sent me an email with a couple of really interesting quotes, right? Because obviously you started a PUC last year, you did the presentation at Cloud Native Summit,
Starting point is 00:25:13 Munich in early July at the time when this airs, we're going to be in August. And you had some really interesting quotes, and I'm not sure if you can just, I don't know, read one or two out loud because I thought there's just really
Starting point is 00:25:28 great with evidence, even though it's anecdotal, but very strong anecdotal evidence on the impact of AI. Yeah, yeah, I can. So that's from our February survey. So in February, we then launched the same experiment, same structure, all objective metrics, and all three surveys to the whole organization.
Starting point is 00:25:51 There we ask for a lot of feedback. And some of the, what we got was, for example, from an engineering manager, were actually several engineering managers. They all go in the same direction. And it was really interesting that one said, it's not only time saved, but actually time created. My all contribution to the code base was higher
Starting point is 00:26:12 because the hurdle to contribute is lower. And this we got from many engineering managers that they, one other said they now have in these 15 minutes between meetings, they now can suddenly contribute to a code base where they couldn't before. And this is a really strong pattern. kept seeing across non-engineering goals, I would say. Another really, really interesting thing was that we, when we did the February experiment, we deliberately wanted to take in product managers, but also design.
Starting point is 00:26:46 So we have a really strong design team who mostly or exclusively worked in Figma at the point of time. And we wanted to take them into the, and give them cursor and the head of design, said, it allows my entire team to close the gap from concept to code. It has totally transformed the way the design team approaches their work. They were so blown away because they suddenly could in the code itself prototype. They directly prototype. They fixed small things instead of doing the Figma thing and then updating Figma,
Starting point is 00:27:21 handing it back to the engineer, having them implemented. They actually implemented, I think 40%, 60% already there. and then the engineer wired it up and it extremely shortened this gap between discussing the design and actually seeing the design in the app. They could spin up their iOS simulators and look at the app themselves
Starting point is 00:27:42 and actually see how their design behaves in the app. This was a really game changer for them. This reminds me of the German word Medianbruch. So they basically eliminated a mediumbruch because you're first creating something in the system A and then you're taking it over to system B. That's quite phenomenal.
Starting point is 00:28:04 Thank you so much for sharing those. And thanks for reading them out loud. Great testimonials for the impact. Now, this was in February, you said. Now, six months later, can you share a little bit about what the state is right now? Do you have more evidence that the money that you put in for using these AI tools? is really paying off? Yes, so we actually have a whole dashboard, which we created,
Starting point is 00:28:37 where we have this call it AI adoption, where we show individual KPIs per team, but also breaking down per users, where we look at cycle time, merge PRs, AI code percentage written. If you also look at special cursor stats, we break up the automations, because in Cursor, I think in every other AI tool, you can create automations that run under service accounts. So we want to know how much are people leveraging automations, which goes into the direction of loops.
Starting point is 00:29:09 We have introduced bugbots since then. So Cursor Buckboard is a review tool, how many people are actually looking at findings, resolving them, how much does it cost? This brings me also to one really strong point I want to maybe highlight is that we approach. cost a bit differently. I don't know if you remember from the talk, but we did something a bit strange, I would say, in the experiment,
Starting point is 00:29:38 counterintuitive, where cursor offered us unlimited tokens, right? I think my spend was like insane, it was like 50K, 60K for a few weeks. But then we said, okay, in the last week, we are going to
Starting point is 00:29:54 deliberately restrict a token usage and budget, per user to $50. We wanted to really cut down, this number was an arbitrary, it was calculated based on the actual budget we know we can get when we introduce cursor later. We knew, okay, we have a rough budget of, I think it was $200 or $300 per user per month. So we roughly calculated, okay, this would be $50 for one week.
Starting point is 00:30:27 they had $200. And we then turned on this budget valve and told the people upfront and then had this really great shift in perception of how people use the tool. They're not just burning on Opus Max High Ultra Fast and burning through the token limit, but really got into this mode of,
Starting point is 00:30:50 hey, I need to be aware of my budget. And we introduced even more controls there. We had a super simple rule was just a one-liner in our global rules where it said, okay, in front of every plan, write the costs. And we gave it the links to the cursor cost table per model. And it knows what model it is, it asks itself. And then it calculated the estimated token costs for this model.
Starting point is 00:31:16 And it calculated the estimated token cost for the cursor auto model, which is always built on the same tokens input output, no matter which model it chooses underneath. So the user immediately, when creating their plan, so hey, okay, when I continue in Opals, 4.6, it was at a point, max fast, I burn $10,000. If I go to the auto model, it's only $10. Right. So we made this really transparent. Then we also introduced Slackbot alerts where for every $50 spend, you get an alert where say, hey, you now spend $50 of your $300 budget.
Starting point is 00:31:55 and we introduced a whole AI tool, AI budgeting tool, where managers could self-service the budget for the team. So every team gets a pool based on the user. So let's say you have five engineers in a team, five times 300, you get the budget for your whole team. And then the manager can bump the individual budget for individual team members. when they hit it. So this made us a really cost-aware and helped us avoid this cost trap, I would say, many other
Starting point is 00:32:35 token maxing companies fell in where was all over the news. I think we shared at the beginning of the talk where Uber, for example, burned a whole year's budget in just a single month. And then the company started to pull the brakes and actually reduce token spend. And we did that from the beginning. And then rather now,
Starting point is 00:32:55 slowly ramping it up and giving people more budget. Yeah, it's easier to allow people to add more use than say, nope, we're going to pull back, you know, because they already agree in those habits. So it's interesting that part of your evaluation was building the habits of conservative token use. Yeah. Right. Hey, in coming to this awareness of costs and one of the questions that I have then
Starting point is 00:33:31 is what happens because obviously your engineers got very used to having these tools available and it changed the way we develop what happens if all of a sudden a model is not available what all of a sudden a complete service is not available may it be for you know Claude has an outage there is a geopolitical situation where for whatever reason we're not allowed to use
Starting point is 00:33:55 certain things. Or they finally raise the prices. Or they raise the prices so that's unsustainable. Do you have a backup plan? Do you have a fallback or is it just like a developer's basically say well we cannot do our work anymore because we don't know how to code anymore without these tools?
Starting point is 00:34:17 So basically we're screwed then, I would say. But we hope that in cursor, we pick cursor for a reason and that status has a lot of models, right? It can even have these self-hosted GMM models. The Chinese models, it has a really broad range of providers. So even if one is down, there might be another. We can switch to. But yeah, if a tool is no longer available for a longer period, I mean, that's with probably any production service. I would say I like GCP.
Starting point is 00:34:50 So if GCP is an outage and all of your services get down, we also have a problem. But I don't think if the thing we're too early to actually start thinking about backup and recover strategies for such a case. And I would hope that engineers then still would be able to code manually, at least at reduced velocity. Yeah. I'm wondering if there's like, you know,
Starting point is 00:35:15 if you think about resilient architectures where you have a failover, strategy from, I'd say, from GCP to AWS, right? You're doing a multi-cloud strategy. Will we see a multi-AI strategy for coding where just as a precautional thing or even a cost thing where you can switch between very easily because this also helps you as an advantage for pricing negotiations, I guess, right? You can say, I've designed my platform and my development process. so any minute I can switch from provider A to provider B.
Starting point is 00:35:54 Yeah, definitely makes sense, I would say. But I think it comes down to the point in time, right? So where are you in the journey? I think for us, it's right now more to focus on how can we push even further into the agendas engineering. So how can we, I would say, get the human at the side of the loop or only at spots into the loop and push more towards self-driving product development with even PRs merging on their own.
Starting point is 00:36:26 So that's rather the point we're like, we're still pushing towards getting more performance out of it and actually increasing the AI adoption in the company. So we now have cursor for everyone. We still want to level up everyone. So it's going to want to get the best performance out of the tool for everyone. And we're still there just starting to really leverage those tools in a way that everyone is on the same level. Because we still see a lot of skill gap.
Starting point is 00:36:57 There are some engineers that perform really, really well with the tools they have. And there are some that still, I would say, perform on the same level as before. Not bad, but they have not completely leveraged AI. And that's we're very currently trying to find a way to upskill everyone and to, get these automation loops in place. Cool. Yeah, I remember, I just, sorry, Brian. I was just saying,
Starting point is 00:37:24 I just had a session with our internal head of AI adoption here at DynamoTrace and he also said that we are also observing our usage of AI and the impact and then we also try to detect patterns between different teams so that we can also use this as a learning opportunity.
Starting point is 00:37:40 And we can, and we see that team A is doing different things and is producing better output or more output than another team, it might be an educational gap that we have and not just a tool gap, right? So I thought that's also really interesting. Yeah, I was also going to say, you know,
Starting point is 00:37:59 one of the things that, you know, Andy and I have seen, it's your, the point you brought up, Andy, about, you know, the fallback plans, but not necessarily the fallback plans, but the idea of how do we switch, how do we do all these things, right? what we've seen over the years in the performance side is that people for way too long said
Starting point is 00:38:23 we don't need to think about that yet we don't need to think about performance isn't that right and of course they get hit by something and like oh we should have thought about performance but I think the bigger issue is when people don't think about all the things they'll have to do eventually let's say when serverless first came out people are like oh let's let's jump on serverless let's jump on using serverless and then it was like okay well you need to monitor their performance of it to understand
Starting point is 00:38:55 how things are going oh well there's you know and especially in its earlier days it's pretty impossible to monitor performance on serverless so as a result with their choice they lock themselves into this box because they're like oh we'll we'll deal with that later right so just or even if you go back to the M plus one problem Andy right All the same patterns we see with every new technology are going to come into whatever else is being used, whether it's now the AI usage or something else. There are certain patterns that are always present in something new. And I do think it's important for people that at least have a mind.
Starting point is 00:39:35 Yes, there's going to be this ramp-up period. Like, okay, is that going to be possible with what we choose? Are there already signs that there are ways to handle this? we don't have to look into exactly what we're going to do, but are these possibilities there? Because otherwise we'll find a lot of people in that same situation. We found them in in the past where it was like, oh, now that it's time to implement these things,
Starting point is 00:39:56 we're too ingrained in what we've done. So now we have to wait for whoever provides the ability to do X, Y, or Z, to catch up and we'll have to fly blind for a bit longer. But that was all just spark, because I understand what you're saying, Michael, and I know I'm ranting here. I understand what you're saying, That's a really good point.
Starting point is 00:40:16 I understand what you're saying about, like, we're not there yet, right? But it's like, yeah, it makes sense you're not there yet, but at least have that end goal in mind for that design. Absolutely, yeah. And I think with, I mean, I see personally use Claude Code, right? And I see it again and again that cloud code suddenly is down. And then I can't work on my side project, right? Because I'm not going to start coding manually from my site project.
Starting point is 00:40:39 I've actually not seen this with cursor or if Claude is down, and then at least in cursor we can switch around. So that was also a choice and we thought about it. What if a model provider is not available? Or with Fable 5, for example, we have not enabled that yet. And we can deliberately disable or enable certain models or whole providers inside cursors that acts as the router. And this gives us the leverage to also look at, okay,
Starting point is 00:41:08 Fable 5, we don't want to share our data and every single transcript with Anthropics. so we are not going to. And people don't even have the choice to do so, right? I don't know if you have this option, if you are on the Cloud Enterprise plan, for example. Yeah, that's a good question. I don't know either. Michael, I have an, to this topic,
Starting point is 00:41:32 I know you are in Europe, right? You're in Germany and also at Cloud Native Summit, the whole topic around Sovereignity. Also obviously heavily discussed, because I also know Max Kjurbecher, who is organizing this conference, is very strong on that topic as well, a big advocate for sovereign cloud and sovereign AI.
Starting point is 00:41:52 Is this something that also comes up in your discussions that you're really using services that are only available in your local region, or is this not a big concern? We're trying to, but I would say, again, it comes back to the point where are we as a company, where are we scaling
Starting point is 00:42:14 and it's coming back to this too early right so we are we are aware we try to pick Europe clouds we try to have these data agreements try to keep the models there but at the same time we are aware and picking US vendors
Starting point is 00:42:31 like cursor or even other vendors GCP I mean there we are in the Europe region but again you still have a very drift towards US or non-sovereign or non-Europe companies and we are aware of this, but it's not a priority right now, to be honest. And it's more of a priority to get this velocity out of the tools we have and get the performance first and then think about, okay, how can we then make it more protected of the data?
Starting point is 00:43:11 Because we're getting almost to the end, I have another question. So, and this must be true, obviously, for every engineering organization. We now have this additional tool that consumes a lot of budget that we didn't factor in before. What does this, how does an organization deal with this? Does this mean you are either saving somewhere else or you're expecting just the increase in output and therefore the output translates into generated revenue. Can you elaborate a little bit on, if you can. I don't know it's a sensitive topic, I assume,
Starting point is 00:43:49 but are you, and to be very blunt, are you reducing human headcount or are you reducing costs somewhere else because you're removing other waste? Or are you just saying this money directly translates in X amount of revenue increase? I would say I'm too far away from this exact reasoning process, but partly I was part of preparing this decision. And one headline first, I'm in the, like I said, in the AI Tiger Team Task Force right
Starting point is 00:44:24 now. And our goal is to decouple headcount growth from organizational growth. So it is not so much reducing, but really unlocking this performance and like the velocity in the organization through AI to be able to scale without scaling headcount growth. So this is the bet you're taking. And when we actually pitched this to leadership and management, we calculated, Klaus did this, calculated a very complicated ROI formula. I know if I can find if you're in my docs, but he did calculate the ROI of actually using it
Starting point is 00:45:06 And the key question we asked in the survey was how much time did you save per project? We had the people list all of their projects that worked on and how much time did they save before or after cursor. And we came up at the table here with a median time save between four to six hours per week. And the top quartile reported savings of one to two full workdays per week. So this was then our basis for the ROI calculation. We said, okay, look, people are saving so much time here, self-reported namely because there are not really objective metrics to measure this. It was backed up by the other metrics.
Starting point is 00:45:49 But this was like the key thing we pitched, right? So the average hour saved was 7.79 hours per day. Across and the participants, I can tell you this, we had 140 participants across four weeks. and we had 245 survey responses with 107 unique people. So it was really not just a small data set, but truly almost everyone in engineering took the survey because we didn't give them cursor if they didn't take the first survey.
Starting point is 00:46:26 And at the end, we said, okay, take the last survey and it will give you back the unlimited tokens for as long as the trial still last, which was about a week, which was a really good incentive for people to actually start taking those surveys. So we had the biggest tip in the like mid survey. It's like if you behave, you get ice cream in the end. Yeah, exactly. There was a month was flying in really fast at the last service.
Starting point is 00:46:53 That was our tokens back. Yeah, it's awesome. Cool. Yeah, I like the, obviously, growing, growing the organization without growing the headcount at the same pace, but basically using these tools to improve the efficiency in the end of the human headcounts.
Starting point is 00:47:12 Yeah, I think it's interesting, too, the idea that not even without a focus of growing headcount, right? Not that you're not going to grow the headcount, but you said decoupling. I think a lot of companies look at company growth and headcount as financial indicators that they're doing well,
Starting point is 00:47:31 and it almost becomes that headcount will become much less relevant in terms of company's success. In fact, you could even say if the head count does not grow at the same time, then a company might be being more successful or at least more efficient. I don't know if that's going to turn into like an efficiency operator. Yeah, definitely if the other metrics grow at the same time. And we do have a really well eye on like this AI adoption, dashboard where we have all of these metrics
Starting point is 00:48:04 okay how's AI adoption doing across the organization hey Michael I have one final question for you looking back at the last month since you started the journey is there one thing that you regret or that you would have done differently
Starting point is 00:48:20 work related at least yeah let me think yeah yeah I would have started measuring this AI adoption after introducing the because we had this gap in February, we introduced cursor. And then we kind of said, okay, we let it sit. And then now in June, we started with this AI task force where actually look,
Starting point is 00:49:03 okay, how can we drive AI adoption further? And we started to look deeper into the metrics. But kind of dropped the ball there after February, and I would have do it differently now and actually continue the journey and be closer to actually then drive AI adoption and this whole automation, performance and velocity further directly after introducing such a tool. And I believe it comes down not to just a few people doing a weird chapter, but there must be some kind of team-creating.
Starting point is 00:49:39 to fully drive and own a high adoption. And I believe platform teams are perfectly suited for that, or at least some people out of the platform team in a combination with business users who are close to the product teams. Because the platform team has a really good eye, bird's eye view would say, on organization and where they have the levers and how they can improve velocity in the teams. I mean, it's our day-to-day business, right?
Starting point is 00:50:05 Reducing hurdles and improving velocity. then paired with the product view of actually how do developers, engineers code every day. I think we could have done better there. And I would say that someone who still is on this journey should really not end the journey on introducing the tool, but it just starts there. It's the beginning of the journey. And it becomes much, much harder afterwards to actually get all of the leverage out of it. It feels like the platform team is going to be the new central ivory tower that has to know everything and controls everything, everything from where you deploy, how you deploy, which tools you're using, which AIs you're using.
Starting point is 00:50:52 But I mean, I guess that's the reality, right? You need to have a central team that then provides all of the tools and processes to the rest of the organization to do their job. yeah yeah cool yeah hey michael thank you so much for being on our podcast we will make sure to add links to the description of the podcast for instance the link to youtube where you everybody can watch the presentation from cloud native summit munich 2026 and if michael if you have any other links to share we'll definitely link obviously to your LinkedIn profile also the one from Klaus, because he was your co-presenter, and then also 1.5, if there's anything else, we just happily added to the description so that people that listen to this have some
Starting point is 00:51:43 additional material to read and go through. Thanks for having me. It was a really fun conversation. Thank you for being on. Really appreciate it. Until the next episode, everyone. Yeah. Bye.
Starting point is 00:51:56 Bye.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.