Technology, Connected - One Wrong Button Can Take Down The World's Most Advanced Data Center

Episode Date: January 22, 2026

Two-thirds of data center outages are caused by someone pressing the wrong switch. Not a hacker, not a hardware failure. A person, in a room with thousands of switches, and their mind elsewhere.We tal...k with Shapol, CEO and co-founder of Entangl, about the engineering layer underneath everything we now call AI. Before Entangl, Shapol led a reusable rocket program and oversaw four launches. He hated his engineering design software so much he built his own, and that software is now keeping AI data centers up.He walks us through why AI data centers are fundamentally different from the ones we've been building for thirty years, why generators have an 18-month lead time and what that does to design, how lights-out autonomous operations are reshaping the industry, and the thesis underneath all of it: the AI revolution is bottlenecked less by compute than by the engineering ability to keep compute running.Enjoy.--Other ways to connect with us:⁠Listen to every podcast⁠Follow us on ⁠Instagram⁠Follow us on ⁠X⁠Follow Mark on ⁠LinkedIn⁠Follow Jeremy on ⁠LinkedIn⁠Read our ⁠Substack⁠Email: hello@thinkingonpaper.xyz--TIMESTAMPS(00:00) Trailer(02:17) From rocket launches to data center automation(06:00) How Entangl integrates with building monitoring systems(08:34) Data Center Design constraints: How AI fixes it(15:37) AI, Dunning Kruger And Hallucinations(21:42) Will humans always have the final say in data centers?(24:53) Space-based data centers and solar power(25:04) Kevin Kelly's question: What should humans become?

Transcript
Discussion (0)
Starting point is 00:00:00 I'm reminded of the episode of The Simpsons where Homer goes to work and he presses the wrong button in the nuclear power plant and Springfield goes into an emergency because there's a nuclear meltdown at the nuclear power station, which was Homer's error, human error. About 2,000 a time whenever there's an outage at a failure center, it's someone planning on the wrong switch. It's as simple as step. You find access, Zoom or EWS or planes don't take off, hospitals don't work. In your systems, do the humans have the final say? Will they always have the final say? Do you envisage your time when they won't? Disruptors and Curious Minds, welcome to another episode of Thinking on Paper where we unpack the future with the people building it. My name's Jeremy.
Starting point is 00:00:56 This is Mark. Today we have a really special guest that is going to help us unpack some things beneath the layers of the way we operate with AI. For instance, users as you're busting around, messing around with nanobanana, or chat GPT or Claude or whatever you're doing, you don't tend to think about the layer beneath that supports the systems, right? Not just the IT systems,
Starting point is 00:01:18 but the systems that support the IT systems. We're talking about mechanical, electrical infrastructure that helps keep all that stuff up and running. So we want to get a couple layers deep, but hang with us. We'll walk with you. We'll help you understand this, because I think it's beneficial to understand
Starting point is 00:01:33 what the heck is going on with AI in general, why people are spending billions upon billions to build these facilities and what it means to the user operating the tools that are supported by them. So, Mark, who are we talking to today? We are talking to Chappell, the CEO of Entangle. And I just want to read a message which I read, which really, when I read this, we have to speak to the people at Entangle.
Starting point is 00:01:55 Quote, hey folks, Chapal and Antanis here, his co-founder. We used to lead a reusable rocket program and oversaw the launch of four missions. We hated engineering design software so much that we created our own, which is why we're here today. And it's why Chappal is here today with us. Welcome to the show, Chapol. Thank you for thinking on paper with us. Thanks for having me. Great to read it. There's a lot of experimentation that happens with trying to build rockets, right? You build rockets, you blow them up initially. They fail. You learn from the failure. There's a lot of experimentation. There's a lot of refining, a lot of iteration. Data centers don't like experimentation. They don't like to
Starting point is 00:02:36 be messed with, right? So how do you do with? that interplay. What we're doing is trying to optimize manufacture, trying to make it as cheap as possible. And that's where a lot of experimentation happens. With data centers, you're obviously trying to make them as cheap as possible. In terms of actual real and data centers, there's not much room for experimentation there. Obviously, you have a lot of experimentation on the chip side, on the actual electrical hardware side of things. And, you know, you see companies like Navidia do a lot of this. And they probably fell all the time in their labs, but you just don't see it as much. But in terms of the actual building of full walls of a data center,
Starting point is 00:03:11 there's not that much room for optimizing manufacturing of or construction of that data center. So that's one key to do. Let's talk about specifically how your company, how your platform helps on those two phases. If we look at design and engineering up front, how these facilities are built, and then we look at operations on the back end, what you guys are doing to automate or create an autonomous means of engineering and operations? I'll start with the latter. And the main problem that we're solving today is, as you rightly pointed out, very start,
Starting point is 00:03:45 one of the biggest risks here in data center operations are the people that operate them. About two dozen a time, whenever there's an outage at a data center, it's someone turning of the wrong switch. It's as simple as step. And the reason why they'd want to turn of the wrong switch is because they don't have that thorough understanding of a data center is like you've got thousands of thousands of switches in a data center. and when you've got to do maintenance or you've got to do some sort of work, let's say in the video releases and your tasks with upgrading and insisted hardware,
Starting point is 00:04:15 you've got to figure out, okay, how do I do this work? Which switches do I press, et cetera, et cetera? And for a normal engineer, that's quite hard to figure out just because of the level of complexity that there is. So what happens is, you know, they press the wrong switch, the site goes down, you can't access, and Zoom or EWS or players don't take hospitals that work, et cetera, et cetera. So it's a very big problem, especially given that our society pretty much runs on data centers. You're not aware of it.
Starting point is 00:04:49 I mean, you'll be aware of it when they stop. But other than that, they're not really interact with on a day-day basis. Now, where do we come in? We can take all the data centers design. So we have a full understanding of how all the circuits work, how, Every server is linked with power plant, et cetera, et cetera. And we essentially direct the engineers to do the right work. So the engineers here is a...
Starting point is 00:05:14 These new racks that I want to install, these new servers that I want to install. AI just breaks that down to them. Okay, you've got to go to this data hole first. You've got to go to this rack. You've got to press this thing. Do that X, Y, Zed. That's what we do today. It's still very much human-driven in terms of what's happening on site.
Starting point is 00:05:34 But a lot of these companies also building it to be Lightsout. And by Lightout, I mean, it's autonomous. You have robots in these data centers that run the show. And we are one of the very few companies that actually have the understanding of, okay, how do you operate at a data center based on the designs. So that's where we come in. But today predominantly people very much human driven. Is there any real-time monitoring?
Starting point is 00:06:04 Do you tie into like the monitoring system and talk me through how that all works. Yeah. On site, you typically have lots of sensors, lots of things that track, how everything iszzling to something called BMS, building monitoring system. And we integrate into that. So we can connect to that, understand how everything's doing. If something goes down or if something's about to go down, we can tell them, hey, this is going to go down.
Starting point is 00:06:28 This is the work that you're going to do in order to keep things running. So that's one part of our integration's onsite. And there's another piece, too, of the operational side that is pretty traditional, right? You have these operational plans that include what are called standard operating procedures, methods of procedures. So if this happens, the human response should be this. Sometimes they're complicated. Sometimes they're easy.
Starting point is 00:06:51 Sometimes the training programs are great. Do you import those into your, is that part of the input side to help them? We actually output that. So let's say you've built inside. you can just upload all your designs and then it protects every single piece of equipment that you have, every single possible
Starting point is 00:07:10 work that you could possibly do it on site and then it creates every single one of those documents you mentioned. Each one of those contains instructions for how to do that work when, let's say, a server goes down or generator goes down. So that's the output they need for us.
Starting point is 00:07:28 What do you do if someone already has that developed? It's a really good question. You still need. to like update your procedures. It's never done. Maintenance is still one of the big costs of running the data centers. It's about 50% of running data center is maintenance.
Starting point is 00:07:46 Ideally, every company wants to have the perfect documentation. In reality of that, you don't really get to that from a slam. You always have things changing. You've got to upgrade yours and hardware. For example, a lot of these AI data centers today, they don't have generators. So you typically have power from your mains that you bring from the power station or substation directly to data center. And then in addition to that you have a redundant power supply in case the mains goes down, which is through generators.
Starting point is 00:08:15 In a lot of these cases, you don't have generators because the lead times are so crazy, like eight months, 19 months in most cases. What they do is they continue without that second power source. They build a data center. But you're still going to need that generator in 18, 90 months down the line. and that means you're going to update your designs, we'll update your procedures. You mentioned something very interesting, that one of the major constraints in the design time is lead time for switch gear, lead time for generators and that sort of things. Is that something you guys help with on the design side?
Starting point is 00:08:45 We help with how you make that design as reliable and redundant as possible. So we take in the learnings from the operations of your existing sites. If a particular incident keeps recurring and that can be preventant on the design side, that's something that we help. Example would be, let's assume a data center has got only one power source for AI data center. People on site keep turning off that main power supply and things go down because, I don't know,
Starting point is 00:09:15 you've got to interact the component in between your racks and your power supply. We would basically take that learning and tell the design team, hey, in order to prevent these same things from recurring, you've got to add a second power supply. So those are the kinds of, things that we help with. We take the learning to an operation site and back to the design. We do help with also taking the designs and cross-checking them against local policies. A lot of climate. Data center is very quickly cutter. You've got your four walls, you've got your
Starting point is 00:09:43 power equipment or cooling equipment and your service. It's pretty quickly cut it. You can go to any place you can cut down, you could cut it and build data center. What really changes is the policies that you're building around. So when you're building a data center in California, It's very different to building a data center, Texas. And a lot of the time you have these, you know, you go through these long approval processes just to prove it to design matches the local regulations. And sometimes it conflicts and you've got to fix that issue.
Starting point is 00:10:15 That's something that we help. So we check your designs and be like, okay, these are all the conflicts that you have. And this is how you solve it or easier regulatory approval. I want to jump in as like, I feel like a GUSB on a date with two people who know everything about the data centers having a conversation about the intricacies of a day center. I'm reminded of the episode of The Simpsons where Homer goes to work and he presses the wrong button in the nuclear power plant and Springfield goes into an emergency because there's a nuclear meltdown at the nuclear power station, which was Homer's error, human error.
Starting point is 00:10:51 And a lot of what you're talking about is human error. For the layman out there, essentially, data centers go down because, people press the wrong button or there isn't a secondary power supply. If those are the two main, would you say those are the two main causes of downtime first? Human error and lack of a power. Is there any technical reasons they would go down? The main one is human error. You obviously have lots of other causes that take up the last third, you know, cyber attacks, flooding, etc., that are very much hedge cases. But about two thirds of time, it's literally, as you described it, someone pressing the wrong thing.
Starting point is 00:11:30 So on thinking on paper, we like to connect the dots of all these technologies. A long time ago now, back in the early days of thinking on paper, we had Don Norman on the show, the godfather of design. And he spoke about the error of design that essentially leads to humans making mistakes and whole systems going down. And he was speaking from very, very simple things like a light switch or the revolving doors in a cinema, like these very simple everyday occurrences. And it's both quite reassuring and also ultimately terrifying that those same problems go into, as you said earlier,
Starting point is 00:12:10 perhaps the most important technological infrastructure propping up humanity right now. Before you guys and Entangle, how were the hyperscalers, how were the metas and the open AIs and the X, working to resolve these problems before you came along. Before what you used to do was you'd write these procedures by hand. So you go into your word or Excel. You would say this is the work that I'm trying to do. I'm not right instructions for how to do this so I don't make a mistake. You bring out your design drawings.
Starting point is 00:12:43 You bring out manuals maybe. So if you have a particular equipment that you're working on, it probably has a manual. You'll check against that. And then you write each step. Okay. Step one based on the design. document, I would go and go to this place.
Starting point is 00:12:58 Then once you dropped that, you would send it for review. And different companies have different ways of reviewing their work. But typically, you would go into a meeting where you have a few people reading that procedure. They'd verify, okay, this makes sense or it doesn't make sense. And then they do the work. And then once they do the work, then it would either be correct or it would be incorrect. It sounds so slow and so outdated when you speak like word documents and instruction manuals. You'd actually do rehearsals.
Starting point is 00:13:25 You'd actually do mop rehearsals where you'd sit around the table and read it like a script and be like, okay, now Shepal is going to turn this breaker off. Mark, your next step is to make sure your team is positioned in XYZ locations to verify this happened. Yeah. Yeah, no, no, it definitely is very slow. I mean, it can take, in some cases where you have those documents months ahead of time, a schedule ahead of time, but okay, this is when I'm going to go and do this in a month. month's time. But the issue still is there. You know, you still make mistakes and that's because you still have that human error in checking those design documents, checking those manuals,
Starting point is 00:14:05 bring out the right items from that data. In an ideal world, they would check all those design documents, but most of the time they don't as well. You can have some capacity, human capacity in there where someone thinks they know what to do, especially as you gain more experience. You're like, oh, I know what, I know my around data center. And then you write. You're right. something and then it's actually the wrong thing. That can happen as well. The other thing is, that design data is not always up to the fact, rarely up to date. You have those design documents submitted once you've built your site and then you've done lots of work over time. This data center that says 10 years old. You've installed new generators. You have new racks and may have a different
Starting point is 00:14:46 structure. You may have different components within that data center. So things change. And those design documents are not updated over time, whereas our software basically tracks every single piece of work with the stand. When someone was a new generator, it detects that. It says new, there's a new asset in the data center. And then it updates your drawings, and that keeps the system update. Whereas with the current process, you have error in understanding those design document. You have error in updated. Those drawings are, and you also have that human complacency in writing and also reviewing that procedure. You got me thinking, the Dunning-Kruger effect, A lot of these errors come from people over evaluating their strength and their knowledge in a particular area.
Starting point is 00:15:29 We're all guilty of that, but sometimes the stakes are a lot higher. Hallucinations, almost like AI is a victim sometimes of the Dunning-Krooga effect. It over-evaluates its capacity to understand what's going on. Some of these designs will be incredibly complex and complicated. They will be bringing sources from all over the place. AI is prone to hallucination, if less, perhaps less than it was. First question, how are you dealing with that? Second question, how do you convince the chiseled old guard that they should be trusting this AI system?
Starting point is 00:16:09 I'm glad you're off that because we, I mean, we get after a lot. When you go to Tadrup Kee, you're also something that hallucinates. That's most people's experience with AI. The difference between our system is that, you know, ours is a hundred. percent deterministic. That means every single time you ask us to do this thing, every single time will output the exact same thing. But what do I mean by that? When you go to chat, Jupy, if you ask it, how are you doing on a Monday, he'll come back when you walk with one answer. If you ask it, the same question immediately after it, it'll come back with another
Starting point is 00:16:41 answer. There's variability in that answer. In ours, there is no variability. And one analogy that can make is an aircraft autopilot. An aircraft autopilot is a hundred percent deterministic. It Pilates the pain itself. It can predict what's going to happen, et cetera, et cetera. It's autonomous, and you may even say it's intelligent to a certain extent,
Starting point is 00:17:00 but it's 100% deterministic. And in those kinds of fields where you really can't afford mistakes, you really need to follow that alternative where you don't have any variability. Our system is based 100% of new designs, and every single time you say, how do I do X work?
Starting point is 00:17:17 It will come back with that same exact answer because that's what the fact So we don't want autopilot getting creative. And is that determinism? Is that it always gives the same response? Why chat GPT and these LLMs are, they want to be creative. They're programmed and designed to be creative and think outside the box. And whereas in these systems, yeah, you don't want any creativity.
Starting point is 00:17:45 Yeah, no, no. And that's a very good way to actually put it. With these LMs, you want them to be general, you want them to have that. creativity. In some ways, hallucinations is actually a feature, not a bug in some of these other lens where I can think outside the books, as you mentioned. But in stuff that you want done 100% accurately, you don't want that. Let's stay on the trust piece of the puzzle, because I know you're having conversations with these large providers and hyperscalers and that sort of thing that have a lot on the line. They have a lot of tribal knowledge about
Starting point is 00:18:17 how to operate these systems. How do you convince them that the model has what it needs? to take them where they want to go. But the process to actually convince them is a very long process. It's never like single new Knail or two meetings that's months, long, years long, where they test it out, they bring all these different people
Starting point is 00:18:34 to review what the tech is and verify that it's earned to actually work. But typically, you know, you would either test this in these lab environments or at one single site. So most of these IPISC goes like labs where they test out with technology.
Starting point is 00:18:48 It's less critical than data centers that actually serve the public. And once you verify that, oh, this actually works as it's supposed to, then you can roll it up. And a lot of, you know, there's various tests that you can do. One test you can do is, okay, you get one of your engineers and you say, okay, write the instructions for this work. Then you get the AI to do it. And then you compare, okay, how long did it take with the AI? How long did it take with the human?
Starting point is 00:19:15 The AI is instant. The human takes a few hours or a day or a week, depending on the work. and then you look at the issues. Well, our system is 100% deterministic. It's based on the designs. It's got 100% accuracy, just like how an autopilot needs to have 100% accuracy,
Starting point is 00:19:29 our system has that. A human has got that very bit. And then the second test you can do is if you've had a particular incident in the past, which you almost certainly have, you can do a blind test where you put in hundreds of procedures of the system and you're like, okay,
Starting point is 00:19:46 find out which one caused the issue. And then the system finds out, oh, it's this one. If you give 100, 100 of these documents to a human and he tell them, okay, figure out which one of these caused an issue, I can guarantee that they were able to find it. Whereas with AI, because it knows as a matter of fact what happened and what will happen, it can very easily figure out, okay, this is the wrong one. And that also proves that the system works as intended. Once you've proven that value, then you can roll it out across its sites and take advantage of it. What are the coolest use cases that you've seen that come back that you never would have thought of? Like your clients, your customers are coming back.
Starting point is 00:20:24 Hey, we use it for this and it helped us do this. There's a lot of different ways I guess people use it. Training new engineers is one thing where we didn't really design the system for that. But we've seen people tell their new engineers, the junior engineers, hey, before you actually touch anything, we want you to just use the system and get your work up to a level of standard. And then we can unleash you into the actual data center. That's something that we never designed it for. We didn't think about it.
Starting point is 00:20:55 But people are using it for that. I want to take it out into space in a minute. But just there's not much documentation out there about you and intangle. But what I did find, one of the leading pictures was the Challenger, Crash 86. And that was caused by the faulty O-rings because it was too cold and then they launched and it was too cold and they were damaged by that. But some people in NASA control were worried and flagged the temperature that day and their decisions were overridden and the launch went ahead. So even though the problem gets flagged sometimes, in your systems,
Starting point is 00:21:37 do the humans have the final say? Will they always have the final say? Do you envisage your time when they won't? When are they do? And And what happens for the future, I mean, many things could happen. It could be that humans always stay in a loop. It could be that we have AI systems that are far better decision-makers than any of me would be. And they'll always say they don't know what the future will look like. But what I will know, or what I do know, for a matter of fact, is that these companies will always want things to be done more lively, and they'll always want things to be done more cheaply.
Starting point is 00:22:11 And I don't know what the future will hold, but it would definitely be constrained by those two things. Market forces will dictate. We can't speak about data centers without two threads which have been coming up more and more and more and more on thinking on paper. One is data centers in space and the other is space-based solar power. How do they link to what you're talking about? At the beginning of the show, you've mentioned secondary power supplies and lead times and how problematic that was. We have on reliable information that with With space-based solar, you can connect, you can have your own sovereign power source stuck to the top of your data center, beam power down from space without connecting to the grid.
Starting point is 00:22:54 What's your thoughts on space-based solar power? If you get to a certain scale, it will be cheaper. To build in space than on Earth. It'll be interesting to see what the regulations are with beaming energy down. I can see a world where people complain about this kind of stuff where, like, do we want energy beam down from space? And then you have 5G towers and forth near homes and people go crazy.
Starting point is 00:23:19 So if you have energy being down from space, I think people will be a bit more. You know, they'll complain a little bit more. But that said, they may not even know that energy is being down from space. And by and they do find out that it is being being down from space, maybe like years down the line. And they're like, oh, it's coming from space.
Starting point is 00:23:35 I didn't know that. We've been fine for years. I mean, space-based power. Will we need it for some applications? I know it may be for on-earth applications. It may be for out-of-space applications. Okay. So you're not totally convinced by space-based solar power.
Starting point is 00:23:56 Okay. StarCloud, data centers in space. Well, data centers in space, I am bullish. So I don't know if you don't know this, but Pidup and I are in the same YC group. So Pid up from StarC cloud. And I actually referred them to YC. So I think it's not a matter of if it's going to happen, just a matter of when, in my mind,
Starting point is 00:24:20 like eventually you are going to have all these heavy industries out in space. People don't want this stuff on Earth. You would much rather have it not on Earth. And that will happen. And I think there's many reasons, many benefits for that, even on the design side. Obviously, it's not an easy thing. People think that these guys that are doing space, space,
Starting point is 00:24:41 they are sense these think it's easy and they're like, you know, an easy ride. It's going to be very, very hard. It's going to be little some design challenges. I think it's going to happen. And I'm very excited for that future. Let's get back to the human side. So Kevin Kelly's question that we like to ask all of our guests is, what do we want
Starting point is 00:25:01 humans to be? And how does technology help us get there if it does? The vast majority of people are doing jobs that they probably don't want to be doing. What they do enjoy about their jobs is socializing with people, seeing other people on a daily basis. But if you actually ask them deep down, that they actually enjoy their work on a date date basis, they probably won't say yes. It would be a very small minority of people that truly do love their job. You know, if you go back 200 years and you were to ask people the same question where they were doing heavy manual labor, where there was an agriculture,
Starting point is 00:25:40 etc. If you truly ask them, do they love with it, they probably have said no. And it's probably, or the better that people are doing those jobs less today than what they were doing previously.
Starting point is 00:25:51 So I'm very pro a world where people are no longer doing these jobs that don't like. And it sounds very scary at first. But I think it'd be better. So really the question is, how do we shape society so that people still get fulfillment.
Starting point is 00:26:13 I only get fulfillment by speaking to others, having a lot of helping out other people, et cetera, et cetera. And that's where I think we should really focus on. And I think the vast majority of work that's being done today will be done by some AI in the future. And I don't necessarily see that as a bad thing. It's scary because it's a lot of things change. The other scary thing is how that technology is controlled.
Starting point is 00:26:35 If it's only 10 companies in the world that control this thing, that would be a very, very good-stopping world. should be avoided at all costs. And that would be the thing that I would fight against the most, not against the tech itself that helps people live their lives, but actually the power being concentrated in the hands of very, very few people. To answer your question, what will humans be? They will no longer be slaves to job that no longer want to do.
Starting point is 00:27:00 That's how it describes it. I love how everybody answers this question differently, Jeremy. It always, everybody takes it in a different direction, but it's so interesting to hear. I think I agree with the most part of the answer. Agreed. Thanks for joining us today, Shapal. Disruptors and Curious Minds.
Starting point is 00:27:16 You can check all of this stuff out, thinking on paper. XYZ. Got a lot of amazing guests coming up. We got a book club, Space to Grow. We just started that this past week. We're in space world right now,
Starting point is 00:27:27 and it's very exciting. We got Philip Messker coming on the show. Speaking of putting things in space and more industries going in space, he had a great paper that we broke down. Yeah, we're excited. Buckle up. It's going to be a heck of a year. Stay curious. Be disruptive. Keep thinking on paper.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.