Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 05x03: Developer-Friendly Edge Application Management with Avassa

Episode Date: May 15, 2023

There is a long-standing gulf between developers and operations, let alone infrastructure, and this is made worse by the scale and limitations of edge computing. This episode of Utilizing Edge features Carl Moberg of Avassa discussing the application-first mindset of developers with Brian Chambers and Stephen Foskett. As we've been discussing, it's critical to standardize infrastructure to make them supportable at the edge, yet we also must make platforms that are attractive to application owners. Hosts: Stephen Foskett: https://www.twitter.com/SFoskett Brian Chambers: https://www.twitter.com/BriChamb Carl Moberg, CTO and co-founder at Avassa https://www.linkedin.com/in/carlmoberg/ Follow Gestalt IT and Utilizing TechWebsite: https://www.UtilizingTech.com/Website: https://www.GestaltIT.com/Twitter: https://www.twitter.com/GestaltITLinkedIn: https://www.linkedin.com/company/gestalt-it/ Tags: #UtilizingEdge, #Edge, #EdgeComputing, #EdgeTechnology, #EdgeApplications, @UtilizingTech, @GestaltIT, @AvassaSystems,

Welcome to Utilizing Tech, the podcast about emerging technology from Gestalt IT. This season of Utilizing Tech focuses on edge computing, which demands a new approach to compute, storage, networking, and more. I'm your host, Stephen Foskett, organizer of Tech Field Day and publisher of Gestalt IT. Joining me today as my co-host is Brian Chambers. Hey, everyone. Brian Chambers. I am the leader of the enterprise architecture practice at Chick-fil-A. I also do a lot of writing about technology

and specifically about Edge on my Substack, Chamber of Tech Secrets, which you can find at brianchambers.substack.com. So, Brian, you and I have been in IT a little bit here,

and I think that both of us have experienced that there's a big gap between developers and infrastructure. And of course, we're trying to close that. This whole DevOps approach in cloud application development is trying to attack that.

But honestly, there's still a gulf. There's still a void between developers and infrastructure, let alone operations. Is it worse in edge infrastructure, let alone operations. Is it worse in edge infrastructure, do you think, than in data center and cloud? Yeah, it's a really good question. I think it is in a lot of ways. I think there is a convergence happening

between the tooling that we're used to seeing in the cloud, where infrastructure has become API enabled, which is a construct that developers are familiar with. But one of the challenges at the edge that I've observed is sometimes the tools aren't really built with the type of scale in terms of the number of footprints that you may have in an edge environment. If you're thinking about a couple of regions in a hyperscale cloud, that's very different than 2,000, 5,000, 10,000 different edge sites. So that can break some of the tooling, whether it's from a

user experience perspective, or another thing I see break a lot is just commercial licensing models where they haven't contemplated that type of scale, and it just is a non-starter for organizations. So I think a lot of the tooling is going to end up converging and being very similar, but there's definitely still some challenges and some friction out there that exists in that world. And I'd say it's still easier in the cloud to find things that make sense than it is at the edge. Yeah. And this is one of the conversations that we had at Edge Field Day in February when we were talking about how applications are deployed in the cloud and versus in the edge. We're talking about Kubernetes specifically and containerization.

And the person we were talking with about this was Carl Moberg from Avasa. So because of that, we decided to invite Carl on the show today to join the conversation. So welcome to Utilizing Edge, Carl. Hey, Stephen. Hi, Brian. Thanks for having me. I'm Carl Moberg. I'm the CTO over here at Avasa. And it's funny that you would invite me just when you're talking about

my favorite topic. So liking the setup here. Lots of thoughts and observations going. So looking forward to this conversation. Yeah, it's funny how that works out, isn't it? Crazy. Crazy. But well, it's good to have you on, isn't it? Crazy. Crazy. But, well, it's good to have you on here, Carl. Now, I've known you for a long time. You're definitely a lot of fun in your presentations. But one of the things really that caught my attention from Edge Field Day with almost every presentation was that it's not about the technology so much as how you're using the technology. And I guess that's how it is everywhere. But it certainly seems to be that way with Edge. We

talked about that last week on utilizing Edge. And here we are again with kind of the same idea. So first off, I guess let's try to kick the conversation off by talking about developers. Who are these people? What do they want? What do they want from me? Who are they? Well, I think, you know, maybe, you know, in the same vein, I mean, I've heard, we're trying to count them at the office. We know we're talking to a developer or actually some sort of application-centric personality when they say something along the lines of, please, I just want to run my applications. And the kind of flip side of that is that

please don't provide too leaky of an abstraction in terms of infrastructure. I don't need to see the entirety of the proverbial sausage mating factory in the infrastructure itself. Please provide me something that has my application workloads in focus. And they come in different shapes and forms, of course, but that's something we always look for.

It's like, if there's that kind of term, like I care mostly about my applications, I trust others to run the infrastructure for me, then that's something I would, you know, someone I would probably put in the application bucket. And again, they don't have to necessarily be, that's why I think maybe the word developer at times is a little sketchy because they don't necessarily have to be living, you know, a part of their working day inside of VS Code or something or Vim or Emacs or something, but they definitely have an application first mindset. So I don't know whether we want to start inventing terminology here, but at times I feel like developers might be a little distractive. I would rather call them something, I don't know.

Man, we have an opening for a new TLA here. Application-centric engineers, ACEs or something. I don't know. But again, applications first and foremost, and they are happy when they see less of the naked infrastructure and the sausage making. Yeah, I think that's really the key. And the idea of application first or application centric is important to me because too often, especially in IT infrastructure communities, application last is the approach that they bring. Essentially, I don't know who's going to use this. I don't know what they're going to use it for. I have no idea how valuable or important it is. All I know is that these are my SLAs.

These are my SLOs for this particular platform that I've created. And I'm going to meet those as well as I can. And whatever happens with it, whoever's using it, if anyone, is completely fine with me. And in a way, I kind of heard that again, I'm not trying to criticize you, Brian, but I kind of hear that sometimes when we're talking with people who are talking about deploying infrastructure at the edge, because it's so important to deploy basically a sort of a blank slate at the edge that you can run different things on, whether it's networking or compute. And so in a way, you kind of have to not care what's being run on it. It has to support whatever it needs to support, whatever comes down the line. But at the same time, once that's running, you need to make sure that it's

really optimized for those users. And you have to make sure that whatever you're building is attractive so that the developers and the application owners are going to want to make their home on that infrastructure, right? Because if it's the wrong thing, then they're just going to buy something else and deploy it out there. And then it breaks the whole goal of standardization, right? So by having this sort of application first mindset, I think that's important, but I don't understand how we're going to get there. So the way I tend to think about it, and I think this has been an interesting exercise over the last couple of years as the edge market has come to happen.

An interesting task when I see new solutions or new software is to try and figure out what is the abstraction here? If I look at the world through this product, what do I see? I don't want to make this too fluffy or too hippie-like, but of course, I'd like to see computer programs or platforms that has the application as the central object. That has that as the anchoring point in terms of abstractions and then have operations on applications. I think there's a lot of systems out there for edge computing, for example, that has the infrastructure as the first class objects at the center of the abstraction. And that's all good and well for the platform and IT teams. But I think what we should be looking for are systems that has applications as the central abstraction.

Because what we'll find and what we find is that if you have that, that will harmonize and it will integrate really easy with the rest of the application team's tooling. Because let me tell you that they have the applications as the central object, the central thing to deploy, the central thing to object, the central thing to deploy, the central thing to update, the central thing to monitor, the central thing to drill into in terms of observability. So I think as vague as it sounds, I think looking at abstractions and really trying to make abstractions about applications and build supporting infrastructure for that is kind

of key to making this something easily digestible or easily consumable by the application side of things. I don't know, Brian, maybe you can make it a little more tangible. That was a little abstract, maybe. Perhaps. Let me ask you a question. So when you think about that person that you have in mind, the software engineer, the developer, or the just application centric person that wants to

get something done, and they're going to use the edge because they have a business reason for it, right? It's not because it's the easiest place or the most fun place. There's some reason they want the latency or the tolerance for disconnect or whatever the case may be. Given that, what are some of the things when we think about being friendly to those people, what do you think is important? What are some of those components of a solution that you're thinking about that would make that experience actually work that would be developer-centric or developer-friendly in nature? Yeah, I think a really simple litmus test is this. So go into whatever UI or command line interface, or maybe a REST API, and you look at the data structures. And hopefully

what you're seeing in systems that are built to be enjoyed by application teams, they start, again, they start with the application. So you can ask the system, show me what applications are actually running on this system right now. And for each of these applications, show me where they are running. And this is in stark contrast to saying, show me all my 2000 locations. And for each location, show me each application that's running. See what I mean? So you start with the application. So here's an application. Where is it running?

Not here's a million locations. And I need to go into each and every one of them to keep track of how my applications are running. So that's kind of the basics, right? That's really one of the basic things. And after that comes, okay, so what is the health status of each of these replicas? Show me that rolling up into an application view, because I want to understand not the fact that 10 out of my 2,000 locations

are out of pocket, but which applications does that actually impact? And of course, I can go on, right? How are they performing? Can I drill into a subset of them? So instead of going the route through, here's my infrastructure, can I see which applications are running on each, have the applications at the central object, because that will fit nicely into your release orchestration. It will fit nicely into your monitoring. It will fit nicely into your, oh my God, I have to actually upgrade constituent parts of this. And it will fit very nicely into when someone calls the application team and says, we're going to take these three locations offline tomorrow.

What's going to be the impact of that? Well, that's easy. I can see here that these are the two applications that I need to do something about, right? So having, again, I started to repeat myself, but I think this is a very, very fundamental issue here is to start with the application as the managed object and as the life-cycled object. Which is a bit of a change from what we sometimes will see happen, which is very infrastructure centric solutions that then layer the applications in.

And I think what I hear you saying is, let's start with the basic construct of the application and then figure out how you understand the infrastructure that it's living on, or maybe to the minimal degree, like maybe you don't need much understanding of the infrastructure. Let's be app first and see infrastructure supporting it, not infrastructure first. And there happen to be apps running on it, which I think is a really important distinction. So I think that's super cool. And I mean, that is also the entry point to such an interesting conversation, which I think we all should be having, and we're kind of having it piecemeal, is what are some of the useful aspects of the infrastructure that could actually enrich the application person's worldview? What is it that we should leak from the infrastructure to the application teams?

Things like, you know, canonical example these days, is there a GPU in this site? Because some of these applications simply won't run without a GPU. Is there a camera attached? You know, is this labeled to be, I don't know, a Chick-fil-A restaurant of size gigantic? And it's got so many, you know, it's so big that it needs an additional size cluster or something. So kind of what you were saying there, that's when it starts to get interesting is what parts of the infrastructure configuration or behavior or whatever you want to call it actually makes sense to the application. So we can make informed placement decisions on it. Right. And that's, that's where they meet. And that, you know, again, I just find it fascinating. What is it that application

teams are interested in knowing about the infrastructure? I know for sure, I'm going to take the first jab here. They're probably not very interested in what type of service mesh is running in each site. But again, they may be interested in the kind of hardware equipment that's attached to the hosts that these applications are running on. Yeah, that's a good point. Because to me, I still feel like there's this fundamental push and pull between standardization and being application centric and application focused. In other words, to play the devil's advocate here, I am developing a standard stack to deploy across my 2000 retail stores. I am not going to have bes, custom offerings for each individual application. That's what's wrong at the edge in decades past. Instead, what I'm going to have is a standard environment that

either runs virtual machines or containers that runs them in a standard way. And as you say, Carl, there are certainly offerings that can be given. So for example, as you said, GPU, yes, no, right? Maybe even sort of not hopefully to the extent that Amazon does it in the cloud, but some sort of offerings in terms of, I want a big one with a GPU, or I want a little one that can, you know, that doesn't need much memory or something like that. I imagine that there could be things like that. But at the end of the day, is it really possible to create application-specific infrastructure and yet have it be supportable across a thousand locations with no IT admin? I'm going to say no, right? It has to be standardized. So I know there's this sort of this push and pull here, but can we overcome that by having

sort of a standard language that we use to describe the infrastructure in a way that's attractive to application owners? Because like you said, yeah, we can't fall back on ITBS and be like, oh, well, this is the Intel XYZ PDQ processor, and it's got the PZY smart flash component. No, no, no, no, no, no. But there's got to be some way that we're describing what the application owner needs in a way that fits with a standard approach to deploying that. Oh, yes. And it's not for the lack of the IT industry trying, to be very honest. I mean, there are things, if you stick to Linux land for now, there are things like SM BIOS and DMI. And there's actually server level things like Redfish. There are, you know, there's something called UDEV in the Linux kernel.

And they're all kind of from different eras with different structures with different original intent. It'll be interesting to see whether we, then the IT industry as a whole, of course, the royal we, will ever get our ducks aligned or in a row enough that we believe that we need a single standard for it. It's rough being a CTO and not having a formed opinion, right? So I'm going to be really burying myself here to say I don't actually have a very strong bet on where this is going. Whether we're just going to build stuff on top of the existing stuff in a very strong bet on where this is going. Whether we're just going to build stuff on top of the existing stuff in a very pragmatic fashion, that's certainly how we do it. We happen to love data models, so we've wrapped a data model around the subsystems that I just told you about,

SMBIOS, DMI, and UDEV. The good thing about that is that it allows you to do tree-based label matching for it. There's a slew of really challenging things. They're overlapping. The semantics are a little tough to understand. I'm sure a lot of application-centric people will be underwhelmed or not impressed at all by the lack of structure and such. But maybe I should try and form an opinion about this, actually. It feels like a weakness that I don't have one.

Maybe, Brian, do you have any ideas of how we, I don't know, should we structure that kind of data? Is that worth doing at all? Yeah, I don't know that I have a strong opinion. I mean, keeping kind of in the same theme of the topic we were talking about with friendliness to developers, and then kind of trying to use cloud here as a parallel in my mind, there are times where some of those details matter when those things matter to a specific application, right?

It's like sometimes you might care what sort of host you're running on because let's just be trendy. You're training the Databricks Dolly LLM or something like that. You need like a really big instance and you care. But in a lot of cases, people just simply are not that concerned about that.

And what they do want to know is, is there some reasonable abstraction I can use that lets me focus on building the app and understanding if it's working well and if it's safe and secure, but then I want to do the handoff via API in the cloud or can I do it via API at the edge? Can I do it through a construct like Kubernetes or HashiCorp Nomad or something different, container runtimes, whatever the case may be. I think that's where the energy is going to go. And there's going to have to be people who care about the infrastructure, right? We've pretended with cloud that nobody cares about infrastructure, that serverless is a thing and that we just run our stuff. But somebody cares

because they have to run the thing behind the scenes. So it still matters. I just think it's going to be a decreasing amount that it matters that it surfaces up to the level of someone who's building an app with probably GPUs for a while being one of the exceptions. And that being the thing that came up in the LLM example I just mentioned as well. So that's kind of my thought. How does that resonate with you? It resonates.

And that's the danger because it's kind of a little all over the place. And maybe there's a position to take where you say, look, we're going to be prudent with the infrastructure. We're not going to need or require that kind of detailed mapping, right? Because we actually know what kind of equipment we have. So maybe there's a, as usual, there's a boring middle ground, Stephen, that makes no one upset in the least, but everybody kind of mildly happy, where, like you said, there's a couple of things that we could probably use some labeling for, which is probably the presence of some external device and the type of that, the presence of some superpowers on the board

level, including things like a GPU, and then the rest, we can probably leave it to humans. But you guys know that there is only three things, or is it two things that's tough in computer science, right? Actually, two, isn't it? It's naming, and that's what we're talking about. And it's the second thing that everybody forgets about. It's cache invalidation, but no one really cares because it's not a very cool subject. But naming is tough. I mean, something I've seen already, and I'm saying already because again, we're not,

you guys are the forerunners here to some extent, Brian. I've seen a couple of other instances of ambitious edge container environments. And one of the things that people lose control over really fast, funny enough, is namespaces and label spaces. Because it is really, really hard to be the librarian of a useful set of taxonomy across a sprawling infrastructure. And that's also meaningful. But maybe that's what we do first. We allow humans to do it, And we're careful with the more automatic or more infrastructure, you know, derived matching patterns. And that's probably I keep, I keep kind of throwing this, throwing this ball in there that, you know, well, it's all well and good to have custom environments, but we just can't do that in a supportable way and a maintainable way.

In a way, the, the fundamental aspect of edge to me is, is the limitations more than anything. And I think that that's one of the, one of the aspects that gets in here, whether it is in terms of customization and supportability, but also just in terms of what kind of hardware can we deploy out here? Because yeah, as you said, if we want to say, yes, we can have an application that needs a tensor processor in order to do some kind of AI app, like an inferencing app, then that means you have to roll that out everywhere.

And it has to be universally available everywhere. And that's a much, much bigger question than it is to say, oh, I have one application that I run on the cloud and I'll run this in Amazon's GPU enabled instance and I'm good. And it's the same with all these things. And one of the things I think that we heard

all throughout the discussion here at Utilizing Edge, but also at Edge Field Day and in other places is this whole, I don't know, sword of Damocles above your head of, yes, but you got to make sure that it's not using too much, too much memory, too much storage, too much CPU, too much special hardware. It's all about the limitations. And in a way, I think that the descriptions that we're kind of getting around here about how to describe the application's needs also kind of work both ways in order to say, look, but application owner, we've got to make sure that it fits within this envelope. So what is that envelope and how do we describe that and how do we work with application owners in order to create an

application-friendly environment within the context of these limitations? Another tiny, tiny topic here. I think maybe one angle to take this on around is that, and I've written a little bit about this actually on our website out of asset.io, something we call the second application problem. I've had a number of conversations with people planning for, let's call it their first kind of modern endeavor into Edge. Maybe they've had Edge kind of computing before, meaning computers in many locations. But now they're thinking about the design of the first, let's say, general application infrastructure.

You know, it's us, So we do containerized applications. So it's generally containerized applications. And to my absolute horror, to my absolute horror, they size this after the first application. So they don't think very much further down the line in terms of spend than their first application, right? Which is frustrating. Let's just say it's just frustrating because there's probably,

and I'm sure Brian, you can speak to this. There's probably nothing more time consuming and generally resource consuming than rolling out new hardware infrastructure to hundreds, if not thousands, in your case, locations, right? So you would think that they would pony up for some future-proof infrastructure

that would cover the first, let's say, 20 applications. But I've seen a horrendous amount of conversations where it's like, you know, we're going to go for the small, we're going to go for the, I'm always getting, I can feel myself getting closer to jumping up and down raspberries here. I'm not going to do that.

We're just going to build the smallest possible infrastructure for our first application, maybe first and a half application. And they're doing themselves such a disservice when they do that. It is, it is crazy. And the way it usually ends up, we usually have maybe half a handful of examples, is that this is the way that some of these application people get to know what the out of memory killer is in Linux. I don't know if you guys have heard that, but Linux has a very interesting behavior when you start to run out of memory. And it's not something you want to get operationally knowledgeable about, right? So I think the first thing we should talk about is that how do you please size your edge infrastructure for at least two, three generations of applications,

for at least two or three handfuls of containers? If you don't do that, there's just no way an application team or application teams in general will be particularly happy for particularly long. And I keep coming back to, I feel like I'm talking about Brian's expertise here. I'm hoping he can. How did you think about this when you sized your install base? Did you have hard and fast thoughts about what is the size of infrastructure that we need for application teams to be able to live on this? Yeah, well, I would say number one is it's unfortunately, at least in my experience, a little bit more of an art than a science. There wasn't a spreadsheet to put a bunch of numbers in and get an answer that told us to get a certain size of Intel NUC.

You have to get an Intel NUC, right? That's the only option that actually works at the edge as far as I understand. But when you think about that, we tried to anticipate and find a sweet spot between what is it going to cost for us to make this initial investment, knowing that some of the use cases that we anticipate having success with are unproven at present, right? We're going to have to prove some things out and we don't want to over-invest, but we also don't want to be in the situation you described where very quickly we're, yeah, there it is. Very quickly we're saying, whoops, we can't run any more workloads and we made this great investment and we built

this platform that enabled us to do a bunch of cool things, but it's all kind of stuck now supporting number one or number two or number three in terms of use cases. So for us, it was a little bit of that, let's be conservative enough so that we don't over invest. We didn't put giant machines in every store. And our constraints forced us to do that to some degree because of physical space. But then also, what do we anticipate seeing over the next three or four years? Because hardware has a life cycle. We can forget that in the cloud, but it's not going to be there forever. And at the edge, it's probably not going to live as long because there's a good chance, at least in my kind of edge, it's not optimal conditions.

So we try to think about what's going to happen in our business and what are our plans over that window of time that we think that hardware is going to live. And then I want to kick a question over to you because I think we're dancing around a word that doesn't get used at the edge very much. And I'm curious your opinion, that word is scale. Is there such a thing as scaling at the edge and what might that look like? What do you think, Carl? I mean, of course, there's, here's what's actually spinning in my mind, Ryan, is that, and let me, I don't want to be too much of a ping pong player here, as much as I love table tennis, but would you say, and this was going to lead back to my observations here, would you say that

over-investing was actually part of your lists on what not what not to do and you now mention i guess two things i i'm sure you you're thinking both financial over investment and also you mentioned physical space and maybe other things i mean were both of those important or was financial more important than floor space or what was the what was the concern here or or was there maybe maybe this is a great you know hard hard shooting question for a for a for a for a program like this was there or is was there let's let's say was there a future where that kind of project would have sunk and you would have declared it failed and you would have to write off the expense was that actually in your in some sort of planning horizon here?

Yeah. So a couple of questions there. I'll do my best. Correct me if I'm missing any. The last one first. So I think the possibility of a failure at the edge was completely possible in the early days that this didn't make sense, that it wasn't actually possible to operate it effectively, at the scale of, you know, 2500, now 2800 restaurants, and on and on. So we certainly contemplated that as a possibility. And then when we thought about that reality, we wanted to make an investment that was big enough, again, that it would enable us to have some room to accommodate more if we succeeded, but that wouldn't be like, you know, a complete and total disaster if it failed. And we actually we made our investment over about a two and a half year period or probably two year period when it came to rolling that solution at the edge out across our restaurants. And I think that's something that people will need to think about. Do they need to be everywhere on day one? Or is there a period of time to learn with a little bit less risk and figure out what works? And then when I think about

what we did, if we had had 10x the demand we expected, the size of the hardware investment that we chose to make was not so big that we couldn't have done one of two things, refreshed it quicker than anticipated, refreshed it like two years instead of three or four years, or plus one. So we started with three nodes. We do have physical space and probably could have found a way to make it work with some dancing to go to four or five if we truly had to. And so to me, when I think about scale, that's kind of what I'm thinking about is there's the pod scaling in Kubernetes for us or the container scaling or the WASM scaling or whatever else you want to say at an application level. But the infrastructure scaling, you know, it's not dynamic and on the fly.

It requires procurement and shipping devices now instead at the edge. But it is still possible given you have the physical space for it, the network capacity for it, et cetera, et cetera. So that's what I would say is the way that we thought about it and sort of the way that I would answer that question about scale. Yeah, no, so thank you. And now you said half of the words that I was thinking about saying. I think there are, again, at least two layers, right?

Like you said, there's the physical layer. So I mean, I focus mostly and I'm thinking about mostly, let's call it a platform style edge as opposed to very heterogeneous, you know, one piece of hardware per application, that kind of vertical, a little more exotic or maybe some old school edge. So, of course, good practice is to build it, I guess, the way that you mentioned, like make it easy to N plus one, make it easy to N plus one physically, make it easy to N plus one in terms of what kind of clustering mechanisms that you have, the scheduling mechanisms that you have. also, and I know you guys are running, I guess, a single administrative domain, but we've seen a lot of users where they want resource, well, they want multi-tenancy on the resource level to make it easy. And of course, that's almost like a next level scaling is how do we actually assign more or less? How do we scale out existing tenants? How do we add new tenants? How do we maybe even contract certain tenants? And I guess the long term planning, and we'd love to hear if you're doing any of that, is like, how do we track

resource consumption so that we can reliably say, well, so you can go back to your team and say, guys, in three and a half months, there's a fourth node coming or fifth node coming. Do you guys do any of that kind of forward planning based on resource consumption? Is that a thing yet? Or is it still a manual enough kind of analysis labor here? Yeah. In our world, the quick answer would be yes.

And we factor in both like, are we getting to a point where we're using all the capacity we have, considering failure conditions too, right? Like we can't go to 100% of all our nodes, because that means if one of them fails, stuff is no longer running. So we factor that in from a user perspective. And then we're thinking about that timing with our refresh cycles for the hardware that are somewhat already planned. And that's a good sizing input for us into how much do we need to go up in our next iteration. So that's the way that we're thinking about it. And I'm cognizant too, one of the things that we've heard from other people in this space is that there are really some hard limits here, especially in terms of things that I guess data center people and cloud people are not used to having to deal with. For example, number of ports, like number of switch ports. You know, adding one more node sounds easy

until you realize that that means that the switch isn't big enough for one more node. And then I need to have a completely different switch at every place, which means we need to replace that everywhere. And there's these weird cascades like that. And maybe it's possible to add a fourth node, but not a fifth node, or something like that. And so there are all sorts of strange constraints and limitations at the edge that people just aren't used to. I just want to say real fast that that may sound silly to some people, but that is like our exact reality in our world is we have limited port capacity on our switches available to us. And we have to factor that in as another constraint

to consider. So that is 100% reality. When you've done this in the real world, like you do have weird constraints that you just do not think about in the cloud anymore. Absolutely. And, and, and of course, you know, obviously there's power and space requirements, but generally you can find another electrical outlet or, you know, stick another thing in there. But, you know, in many cases you can't get much bigger. And also some, some solutions don't, don't lay out nicely and don't scale nicely. And, and, and, and that's another, another constraint. So I think, honestly, for a lot of people, the answer to scaling is going to be deploying a bigger one next time.

So we actually are going a little bit long here, but I hate to cut this off because we're really warming up on this topic. But that being said, given all of this, you know, what's the answer? How do we build a developer friendly and application centric infrastructure when we're subject to all of these constraints and all of this requirement and making it supportable and making it something that can be rolled out everywhere? And Carl, that's kind of for you. You're the guest. Yeah, sure. Again, well, what we tried to do was to really, again, look at a distributed infrastructure with an application-centric worldview. And I think, I mean, what we did, and I know that this is one of your favorite topics, we started from the API down, right? So we looked at what would the abstraction look like? What would an API look like, right? And then we pick the components all the way down to the constituent nodes, right? And one of the many things that we realized

was that there's a number of things that we could just easily reuse, right? There are things like, what is the structure for describing what an application looks like? There's insane amounts of prior art around that in terms of Swarm and Kubernetes charts and all kinds of things that you can look for. Really things that would have been painful and stupid to reinvent, like the fact that applications have names in a version. There's something called a service that is the set of containers that needs to be. Like the fact that applications have names in a version, there's something called a service that is the set of containers

that needs to be scheduled on the same node. All these things, right? They're already there. No need to reinvent that. So, okay. So that holds true then. Let's see if we can reuse that. And here I'm gonna, I like poking on my pet peeves here.

This to me, for example, takes out the whole idea of an edge-specific marketplace. I don't like that. The marketplace for containerized applications in every application-centric person's worldview is called a registry and a repository. There's no need to create another app store or marketplace for that. Actually, that's probably a disservice. So try to build on what's already there and what kind of works.

The second thing then, again, and that's where it starts to get interesting, is if we are to allow application-centric people some sort of self-service experience, then we have to give them the reins in some way, shape, or form to describe to a system under which circumstances or where does this application run. Of course, at the fundamental heart of that, that scope of where applications can run needs to be managed by the platform or IT

team. But given that they have control of that, I think that's the second part. Allow them to define their applications and allow them to describe to a system under which circumstances or where do I need to run this? And that's where we touch on what we touched before. Things like only if there's a GPU or only in Sweden, because we love Sweden so much, or only in Chick-fil-A restaurants that are sized more than 100 chairs or something like that, right? A configured aspect of it. That one is new. And that one, I'm very interested to see where that kind of description will end up. Is that something you should put in your GitOps? Is that something you should manage as a separate application release orchestration manifest? Is

there something you should version? I don't know yet, but I think those two abstractions, what's an application and where should that run, should be at the heart of the conversation. And I truly think that that's a great starting point. And again, that's just for the lifecycle. Then talking about observability and manageability maybe is a different thing because that seems to be even more undecided whether we should eventually look to the application teams to actually monitor the lifecycle or the health and actually be kind of almost like on call for that or whether that

is better left to the IT or operation teams. But again, leaving the description of an application and where to run it and make those beautiful abstractions and easy to use, easy to understand, opinionated abstractions, I think could be a game changer for this industry here. Yeah, I have to agree because in order to make this stuff supportable, it has to be abstracted, it has to be well-described, and it has to be standardized, but yet customized enough to be attractive to the developers. And I think that that's really where it comes down to. So unfortunately, we went a little long,

but I've got to cut it off. Carl, I really appreciate having you joining us here today. If you're interested in this topic, if you want to kind of continue this, I do recommend checking out the Avasa presentations from Edge Field Day. Just Google or Bing or use your favorite search engine to find Avasa and Edge Field Day. You'll hear a lot more discussion about this. But Carl, where can we connect with you? What are you doing lately? Where can people continue this

conversation? Well, if you're on the event circuit, you can find me at the Edge Computing Expo in the coming week, I believe it is, in Santa Clara. I'm going to be all caffeined up in the booth, ready to demo. We're going to have some exciting hardware with us as well to show the power of edge computing in many ways, shapes, or forms. So that's a good one. Otherwise, I'm always writing my thinking at avasa.io under resources. We have a little bit of a blog there. And I'm at Twitter at C. Moberg. And you can always find me alongside yourself, Stephen, at Mastodon these days. Yeah, right on. Go Mast. How about you, Brian? What's going on with you? Yeah, this is a fun conversation. Thanks, Carl. You can find me,

as I mentioned at the beginning, at Chamber of Tech Secrets on Substack. It's bryanchambers.substack.com. I write in once a week on that, and I touch on all kinds of topics that are relevant from an enterprise technology perspective. Cloud and Edge are two of the big ones, so I'd love to have you follow me there. I'm also on the Twitters at B-R-I-C-H-A-M-B, part of my name. So you can connect with me there or LinkedIn. If you just search Brian Chambers, I have cornered that market.

So that's where you can find me. Excellent. We got to get you on over on the Mastodon, Brian. And as for me, and as for me, you'll find me at S Foskett on most social media networks, including yes, Mastodon. You'll also find me here with our weekly Gestalt IT podcast, the on-premise IT podcast, as well as our weekly Gestalt IT news rundown, which you can find on YouTube or your favorite podcast applications.

I do want to call out that during this conversation, those of you listening on audio, I held up this bad boy. I have a great many Intel NUCs, probably not as many as Brian, but I just happened to have one here to wave around and he mentioned that during the show. Let me know when you have 8,000. Yeah, I don't have 8,000. I think that there would be complaints about that. I think I only have 30, but there you go. Well, thanks everybody for joining us. Thank you as well for listening. I hope you enjoyed this. This is Utilizing Edge, which is season five of the Utilizing Tech podcast series. If you enjoyed this discussion, we would love a rating. We would also love a message from you. You can find us just to send us an email host at utilizingtech.com. Of course, you can also connect with us on the socials, utilizing tech on Twitter and Mastodon. This podcast is brought to you by gestaltit.com, your home for it coverage from across the enterprise,

but we have our own special website for this. So go to utilizing tech.com and you'll find all the episodes of this season, as well as our previous seasons, focusing on CXL and artificial intelligence. Thanks for listening. Thanks for being part of this. And we'll see you all next week.