Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 05x08: The Middle Mile is the Heart of the Edge

Episode Date: June 19, 2023

Between the so-called last mile and first mile lies the middle mile, the realm of colocation and network service providers. This episode of Utilizing Tech features Roy Chua and Allyson Klein, discussing the middle mile with Stephen Foskett. This middle area includes content delivery services like Varnish and Akamai, as well as companies like Cloudflare that are delivering content and compute there. The middle network includes providers like Equinix, Digital Realty, and Megaport, which provide connectivity to the cloud and service providers, the hyperscalers themselves, and some interesting networking startups like Packet Fabric and Graphiant. We must also consider observability, with companies like cPacket and Kentik as well as companies like Cisco and Juniper Networks. Host: Stephen Foskett: https://www.twitter.com/SFoskett  Guests: Allyson Klein: https://www.twitter.com/TechAllyson  Roy Chua: https://www.twitter.com/WireRoy  Follow Gestalt IT Website: https://www.GestaltIT.com/  Twitter: https://www.twitter.com/GestaltIT  LinkedIn: https://www.linkedin.com/company/Gestalt-IT Tags: #UtilizingEdge, #EdgeComputing, #Edge

Welcome to Utilizing Tech, the podcast about emerging technology from Gestalt IT. This season of Utilizing Tech focuses on edge computing, which demands a new approach to compute, storage, networking, and more. I'm your host, Stephen Foskett, organizer of Tech Field Day and publisher of Gestalt IT. Joining me today in this co-host-less-less discussion, we have Roy Chua from AvidThink and Allison Klein from the Tech Arena. Roy, welcome to the show. Thanks, Stephen. Glad to be here. And Allison, it's nice to have you back. So good to be here, Stephen. Thank you.

So we are getting ready for our forthcoming Edge Field Day event, And both of you were involved in our previous one. And of course, both of you are also wonderful thought leaders in this space. So I thought it would be nice to bring you in, sort of as a follow-up to the conversation that we had just a few weeks ago with our friend Andrew Green from GigaOM, where we talked about the differentiation between the near edge and the far edge. And Roy, this is one of the things that's kind of interesting, isn't it? Because if you've got near and you've got far, you've also got middle, don't you?

Absolutely. I think, you know, the middle child is always an interesting one, right? That's where things get interesting. And yeah, so I think the reality is when we look at infrastructure and separating the edge for a moment, let's look at sort of the network part of the edge because we're all connected and that's where the edge comes in. Is you got the first mile, you got the last mile, and the last mile is what connects your homes to sort of the aggregation points. And then you've got the middle mile, which historically in networking, that's where a lot of the excitement goes around because the first mile is dominated by hyperscalers and the telcos. Your last mile, which historically in networking, that's where a lot of the excitement goes around because the first mile is dominated by hyperscalers and the telcos.

Your last mile, again, usually you have some service providers and telcos. And in the middle, there's a lot more play in there. There's data center operators, there's your colo operators, alternative access vendors, right? That's where it gets really interesting. And we'll talk a bit about that, I'm sure, as we go along. But certainly that middle place is an interesting place. I think that what's interesting about this, I just completed an e-book on Edge that's

available on my website, thetecharena.net. And one of the things that I talked about in that, after interviewing a number of folks from across the industry is that the edge is really moving us towards distributed computing. And what Roy's really set out is a further distribution of compute capability across different environments for edge so that you can place your workloads exactly where you want them. I think that this concept is interesting, especially when you consider that while Edge has been built out with some of the same orchestration capabilities of cloud, it doesn't have the, you know, the far edge doesn't have the notion of infinite compute. In fact, it could just have a single nook.

And so how do you actually ensure that compute can scale, compute can burst? A middle edge is an interesting alternative. Yeah, absolutely. I would say, you know, you have a lot of cloud players have the regional clouds, right? Those are huge data centers located more centrally in metro areas. And then you have, as you pointed out, Alison, you know, you could have that little computer right on premises. And I think that's an interesting part in the middle. And the middle is obviously a larger slice. And it could range from, I think, in past podcasts we talked about, it could be somewhere in the radio, you know, access network, your RAN on towers, you know, all the way to sort of local data centers, regional, not the regional data centers, but sort of localized data centers

or sort of metro area data centers, if you will. And that spectrum lent itself based on the workload that you have, based on the characteristics of the workload, you know, how much you're willing to pay, what kind of computer you need, you know, what kind of latency are you looking for, and maybe even what's available to you, right? I think that is going to be an interesting orchestration and placement problem as we go forward.

But that middle mile, I view as a very interesting place because, and I call it the Goldilocks region, right? Because it's sort of close enough to you potentially to meet some of the latency constraints. And maybe in Europe, it helps you with data sovereignty as well. But it's far enough upstream potentially to have aggregation and economies of scale.

So you have the balance of both, right? And some people don't like Goldilocks region. You can call it the little bear region because I guess Goldilocks was the aggressor, but regardless, right? it's an interesting region anyway absolutely and there's a ton of things going on there too i mean there's um all sorts of networking products um along with uh content delivery uh sort of pseudo cloud, and even basically function as a service worker kind of things going on there. There's a lot of stuff happening in the middle edge that can be leveraged to improve the user

experience and overcome some of the limitations of the client side in terms of limited resources, limited connectivity, that sort of thing. I think content delivery is such an interesting space, Stephen, because when you look at what was maybe the biggest workload during the pandemic, content delivery networks just took off. And companies like Varnish are doing incredible things to drive performance capability for content delivery networks exactly in this middle edge. And, you know, the use case is so proven, the growth of content delivery network demand, it's not just video anymore. It's, you know, every type of

content that consumers are, you know, basically addicted to are demanding these content delivery networks that are sitting in this middle mile that Roy has described. Yeah, I think, you know, absolutely. I think the one thing about the middle mile, and middle mile has historically been more of a networking term, right, than a compute term. When we talk about edge, we usually focus on compute. But I think the realization increasingly is that the edge is enabled by the network topology and network capabilities that are available. And so we, at least at Everthink, we talked about sort of the new middleman, which is sort of the amalgamation of compute and networking as a joint decision, because it's not just about what compute capabilities are available. It's also what network paths are available, what the capacity of those paths are,

number of hops to get to those things. So in reality, I think, Alison, what you said, you've got to put that placement, that orchestration, that we need to do the placement logic. It's a joint decision between the compute slash storage capabilities, as well as the topology of the network that that connects to that location. And I think the joint decision is what makes it really interesting

as we go forward in terms of that awareness of network by the application or application placement and likewise, the network providing a service or quality of service to the applications on top of that, right? So we're seeing that come together, I think, finally. And this leads to a whole world of products like SASE and observability products that are allowing us to reconfigure the network dynamically, make use of multiple links, make use of multiple resources, connect endpoints, and do all this in a secure and highly available way. It really is a world of products that are happening in there, a world of companies and technologies as well.

Roy, what do you think are the key players there on the network side of this middle edge? Yeah, so in that sort of the new middle model, sort of that middle network piece of things, I mean, some of the clear dominant players in there are going to be the co-location providers, right? So you pick your Equinix's of the world and your digital realties of the world, your megapods of the world that connect, that manage that fabric, if you will, right? Because that's where you sort of get connectivity to a direct path to your Amazon and your Google or whatever it is, or other SaaS services.

It's also the places where you get connected to other hubs or other telcos or other service providers and so on. And so I think those companies certainly are important. There are other classes of companies in that space as well. The CDN providers certainly play, right? So your Akamai, your Cloudflare, I think we've talked about those before, certainly. And then the hyperscalers all have the CDN networks

that play in there. And then beyond that, there are some up and coming players on the startup side in terms of networking, because if you think about that place, you're going to be able to, as you find out, how do you bring up networks quickly? How do you bring on networks and tie them together in a flexible way? And so the historical, historic legacy SDN, software defined networking used to be hard, but it's still there and still

very important. You have the new SDN type players, right? So your packet fabric, for instance, that provide this kind of capability. Or more recently, Graphiant that's providing that flexible capability as well in terms of giving you flexible connectivity and underlay within that region. And I'm seeing some of the fiber providers want to play in that space as well, certainly. But in order to do that, they have to come up with more SDN type capabilities. Telcos also want to play, and that's where they need SDN capabilities. And so what we heard from a company like Graphion is like the carriers are excited about their software

because it enables the carriers to use their middle-ball infrastructure and monetize it because it becomes more flexible. And I think Packet Fabric was seeing the same thing as well. So some of the ideas of the companies, at least on the networking side that we're seeing. I'm glad you brought up observability, Stephen.

This is an area that I know I'm a geek. You know, I started in semiconductors and I love telemetry, but I love observability in terms of what it brings to a distributed computing environment and how I think this technology is going to become so much more important and valued than it has been in a traditional, you know, multi-cloud world as we get more and more distributed and enterprises are relying on more compute locations and much more on the quality of network moving forward. And there's incredible innovation in the observability space. I was just talking to the folks at CPacket. I don't know if you've heard of them, but they are an incredible player in this in terms of delivering near real-time views of network packet transfer and what that can mean in terms of managing a distributed computing environment. And I think that there's a number of players in that space that are doing really interesting things that we'll be talking about more and more as this distributed world comes into play. Well, and I also have to mention Kentik as a key player in that space too. They're doing

some very, very cool stuff in sort of seeing the truth of the internet, because that to me is the reason that this observability space is so important in that it's easy, well, not easy, but at least it's theoretically possible to get your mind around what's going on in a data center or even in a cloud. But the internet is the internet. It's everywhere. It's everything. And it's much harder to really have an understanding of what's happening there. Yeah. And regardless of whether you control the underlay, your network underlay, and you can get quality of service provision to the places that you need to, regardless of that, you're absolutely right. Even if you pay for the SLAs, how do you know you're getting those SLAs, right? I mean,

you can't count on your provider giving it to you accurately. You still have to monitor and observe. So I think some of that, I think we're seeing as well, that observability is important. And I know, obviously, I'm sure Cisco and Juniper and those folks have their own product suites around those, right, to provide visibility and assurance, right? Cisco with 1000 eyes and Juniper with its Paragon assurance suite and some of the companies have acquired recently, I think increasingly, and even even the networking vendors, not so much the application side of things, but even the networking side of it is talking about observability. And historically, that's been

the term more used by application developers than than networking. But I think they have gotten religion around observability as well. So I think the point Alison made is an excellent one. I'm surprised we haven't talked about sovereignty yet in this discussion, because it seems like something that's on the top of most enterprises' minds on where data is lying and where applications are actually functioning. How do you two see sovereignty influencing this middle mile versus far edge versus near edge? We need to come up with, is it medium edge?

I don't know. I think from the edge classification, people will call the middle mile the near edge because that's where the network elements are and then for them the far edge is usually the on-premises so generally speaking you know i mean at least for for for when i work with my clients i just call it on-premises edge versus network edge and that seems to be okay and network edge encompasses the middle the middle mile elements itself because the i think steven said before, near to whom and far from whom, right?

And I think that becomes ambiguous. When you say on-premises, usually enterprises understand on-premises being, oh, it is on-premises. Okay. It's the one place we know for sure. Yeah. Oh, you mean on my premise? Oh, yeah. OK. On my premises. Exactly. So, yeah. are using distributed functions and caching, it really does kind of poke a hole in this notion that we can control, that we can tightly control sovereignty. Because if network resources are used, those network resources may reside in the United States or in Germany or in Saudi Arabia or in China.

And it's very difficult to guarantee that your application doesn't touch someplace in the network. And I mean, I guess it's one thing whether your data is simply passing through, as it were, or actually checking in. But it's quite another if we're talking about having workers that are actively processing data, storing, processing transactions, and then sending things on. Those workers could be in different places. And I think that right now, the idea is, oh, well, just like the cloud, we will deploy regions and we will control which region you're using. I don't know if that's really practical in the long term, especially as this expands. And as we get more and more of these kind of middle of the network services, I wonder if maybe encryption is a solution where maybe we have really zero trust edge compute, you know, that it could be running on resources in the United States or

in China. And whether you want to think about the laws in one of those two places or not, if it's really encapsulated and encrypted, then it kind of maybe doesn't matter which location it's running in. I don't know if that's practical, but it seems to me that we're going to need this sort of approach because data is going to be not just transiting, but being processed in many different places. Yeah, I would say so. Two elements to that. I think number one is that many of the edge providers and orchestration providers for the edge are tagging workloads with metadata tags that indicate where they can be processed and where they cannot. So I think that's one element of it. Secondly, the SDN guys that we talk about in terms of the routing of traffic and the like increasingly are allowing you to dictate constraints on the nature of those routes as well. So they'll say, this cannot exit the United

States, or this cannot exit the United States, or this cannot exit whatever it is, or this cannot get into Eastern Europe or Russia or whatever it is. So I think that's one element of it. And we're seeing that already. So we've seen those capabilities coming up in orchestration for both the compute workloads itself, the storage elements as well, and obviously the networking. So that's one. The second one, with regard to what you're saying, I think that's an interesting proposition. I know that some of the hyperscalers have started thinking about that, the concept of confidential computing, right? So secure enclaves, you know, or some of the CPUs have the ability, right, to do secure computing, right? And in other words, secure enclaves where the compute happens inside the enclave

is not exposed. The key is pulled over the internet in a secure manner. The CPUs are manufactured with an identity. You know it's basically a secure Intel CPU for the most part. So SGX sort of trust type basis systems. I think that certainly is a possibility in terms of you dictate that the workload can run anywhere. The key control is what gives you the sovereignty and therefore you should feel safe. That can happen. I think that the danger is that,

you know, for instance, there was the exploit called SG, you know, Intel's SGX was, there's an exploit called SGXAXE, right, a couple of years ago, that showed how it could actually be overcome. And you could fake being an Intel, trusted Intel CPU. So I think a few more years until people actually trust it. And maybe when people understand the nature of encryption and they're comfortable with it, then I think the vision that Stephen, you laid out, I think that's certainly a possibility, right? But in the meantime, we're trying to do very rudimentary controls

by putting metadata tags in there to say, this can go here, but not here. And please don't violate it. And you hope the software does the right thing, right? Yeah. I think that that's really gotta be the answer right now. I'm kind of inventing that on the fly. Anybody wanna start a comment? and you hope the software does the right thing. I think that that's really got to be the answer right now. I'm kind of inventing that on the fly.

Anybody want to start a company? I think that it's interesting. No, I don't want to start a company. But I do think that what Roy just said is really interesting, and I think that it's true. I'm so glad you brought up my beloved SGX. I worked on that technology, Roy. Oh, you did? You made me proud. You wonder who fixed it.

I wish I did. But I think that what's interesting is I think that this concept of sovereignty has actually accelerated some of the placement of this middle mile compute capability to address customer requirements for specific locations. And so in a way, it's been a driving force to give other customers the opportunity to take advantage of nearer to on-prem, I don't know what you'd want to say, compute gravity, if you want to think about it that way, where there is more compute in a near location and they can start thinking about workload placement in a much more flexible manner. I think Cloudflare's growth over the last couple of years has been driven in part by this, and I'm sure that that's true for some of the other providers as well. What I'm really interested in is IT's sophistication on workload placement and really

having a good view of that full distributed dynamic and where they're going to place workloads for best return on investment for the company. I think that in talking to a number of the edge players, especially those who are working on far edge, on-prem deployments of simple compute and branch environments, I think a lot of companies need to catch up on this. And I think that over the next couple of years, we'll see who really leaps forward in terms of sophistication of having a really great topology of where they want to deploy each and every workload under management, whether it be in a cloud, in the middle location that we're talking about, or on-prem in one of their own facilities. I think that's what's coming next. Yeah, I would second your mention of

Cloudflare there as a company that's doing some very cool things with not just one of these spaces, but all of these things. I mean, basically, they, to me, seem like a company that's really focused on solving these problems. And I would call out as well, I mean, Akamai, they maybe, you know, people think of them as sort of the old company, but they've actually made a lot of investments in this area too. And I know Equinix has as well, as, you know, kind of different, three different companies coming at this problem from three different directions, in my opinion. Yeah, no. And I would say absolutely that the co- providers are adding bad metal capabilities. The CDN providers are going from storage to compute capabilities, functions as a service.

I think given all the placement of compute capabilities, given the assets they have into some network topology and network connectivity, everyone's trying to leverage it. And I think the key is to make it as simple as possible for the developers to figure out how to write applications or how to tag data in the right way so that the orchestrator can do the right thing. Because I don't think it's possible for someone to sort of humanly and say, oh, and this component is going to go into this region and this location and all that. I think that's not feasible. I think what's feasible is to say, here's my application. Here's the kind of quality of service I would like for this application. Optimize for cost, please, or optimize for experience, please. And then it does the right thing.

And I know that the hyperscalers have to be thinking about that. For instance, like, say, don't know, just pick AWS for example. AWS has, they have this continuum of capabilities. You can have your local snow family or outposts, and you have your local zones and your wavelength stuff in the telco, and then you have your regions. And I should be able to say, hey, I'm writing my application. I deploy my application. Here are my constraints. Here are the optimization that I want. Go figure out how to place it for me, right? And don't exceed my budget, please, right?

And then it goes and does the right thing. Or come back and say, Roy, you can't expect to pay 10 cents an hour and expect to get all of these, right? It's going to be at least 30 cents or whatever, right? That probably should be that model in terms of the application. That's what I would like to do and not worry about it and have someone deal with it, including constraints of sovereignty and the like, right? So I think that's hopefully where we're going, but we'll see. That would be the goal. Yeah. I'm glad you brought up the hyperscalers

specifically because they would seem to be the antithesis of the edge. I mean, basically the edge is as everything they're not, but yet it also would seem that they're well positioned to, to play in this middle area, but also as you point out, you know, really at the, at the end points, maybe, maybe, maybe, maybe are they, I mean, I mean, do they get the message? Or is it one of those cases where the paradigm is shifting around the market leaders, and they're not able to jump on a new way of doing what they're doing? I don't think I'd ever bet against the hyperscalers in having a strategy to take advantage of a business opportunity, Stephen. And I think that this opportunity will make some interesting bedfellows, right? I think that you're going to see, and you have seen the hyperscalers look to some of the telco providers for strategic

collaborations. I think that the hyperscalers will use their software stacks as a way to pivot into edge deployments. And you've seen them make early moves. I would expect them to continue down that path. And, you know, I think that they're going to look for the key use cases. You know, we talked about some of them at the top of the podcast around content delivery and, you know, edge analytics. I would expect them to start delivering services at the edge for these types of things. I wouldn't be surprised at all to see them all move in interesting directions

for partnerships with some of the companies that we've talked about. And Roy, I would love to hear your perspective on this too. No, I think you're absolutely right. So we're seeing, so again, never count them out. I think there's a couple of elements. One, the developer relationship is held very strongly

by the hyperscalers. The application developers work with hyperscalers. They're used to that whole, you know, tool chain and pipeline. So that's where they'll go first. And to the extent that allow the edge to just be an option

in terms of the deployment of the application, it's automatic. So, and Alison's right. The telcos and the hyperscalers have all different relationships across the board. On top of that, all the hyperscalers have an on-premises edge option. So Azure Stack Edge, Google distributed cloud edge, AWS with Outpost and the Snow family. So they absolutely are there already with a single pipeline deployment, regardless of, you know, what infrastructure it is. And Azure, you know,

wants, you know, Azure Arc wants to control all your elements, right? Even on your, some third party band metal that you're running some Kubernetes cluster on, they want to control it. You know, Google would like to do the same thing, right? And likewise, AWS would like to do the same thing with EKS anywhere. So they certainly see it and they're partnering to do that. And the other people they're partnering with, I see is some of the application infrastructure

and the databases. A lot of the real-time databases, like say, Couchbase, for instance, or Redis, they recognize the value of the database at the edge. And so what they're doing is working with the hyperscalers and making that happen. Like, you know, Couchbase, Capella has partnerships with, I think it's Verizon and AWS simultaneously.

And so I think they all understand that they're trying to figure out what the route to market is. But I think never forget the developer because that's the very, very critical relationship. And the hyperscalers and developers have a great relationship. Telcos don't always, you know, and the ISVs sometimes do. And I think that's probably the best way to look at it, I think, from my perspective. Yeah, I definitely think you're right.

And I will just put in a little pitch, too, that it seems that the traditional data center ISVs, especially on the networking side, are really, really getting religion on this. If you followed Cisco Live recently, the whole keynote was about modern applications, cloud scale applications, observability, something they call the Panoptica, which is too close to the Panopticon for me. But, you know, they really are embracing the developer in a way that I wouldn't have predicted Cisco would. And you mentioned Juniper. Absolutely. I know that they've always they've also been leaning in toward this as well. And you'd think that that companies like that, you know, the reason I bring up the hyperscalers, it's one of those things where it's very difficult as an incumbent or a market leader to undermine your own business by sort of competing against yourself. But you have to or you lose relevance. And so for me, I absolutely agree with you, Roy, that, you know, a company like Amazon, AWS, they're sharp enough. They know that they need to do this. They know that they have a great lead with the developers that they could squander if they don't.

Similarly, I would look at Microsoft and I would say, you know, here's a company that has tremendous developer connections. And they also would be, I think, well positioned to deliver this sort of on their own terms. It's certainly going to be a different solution than the Amazon solution, but one that would be very embraced by a certain enterprise customer. They do. I mean, I can tell you, I sat in a couple of sort of Fortune 50 conversations around the edge, and Microsoft has very strong presence, for sure. No doubt. Absolutely. Yeah. So I guess to wrap this up, though, my question would be, if this middle

mile, as you say, is dominated by companies like Amazon and Cisco and Microsoft. And is this the edge at all, or is this just an expanding cloud? That's a great question. Yeah, I mean, at the end of the day, it's a distributed cloud, right? And when you say it like that, and like, oh, does that mean that the cloud providers win? Not necessarily. I think the middle mile is a messy place. There's a lot of elements.

And yeah, scale matters, but agility also matters. And I think I will expect a more diverse ecosystem in that middle mile than in the first mile, which is dominated by hyperscalers and the telcos, right? So yeah, that's what I hope anyway. I think that when you were talking, Stephen, one of the things that I was thinking about is that on-prem data centers still exist, and they exist for a reason, because ultimately computing will be driven based on the right thing in terms of TCO for the customer. Some things will be run on-prem, some things will be run at the edge, and there is an opportunity for incredible innovation from a wide variety of companies, including the hyperscalers. I don't think that we should look at it as just an extension of cloud. I think we look at it as a bold step towards distributed computing.

And what that represents is an incredible opportunity for companies across all industries. I'm really looking forward to it. And I am looking forward to seeing that broad industry innovation to make it happen. Yep. I agree. And I think that we're seeing that. And I think that that's what's fun right now about this whole space. Just like in the far edge and on-premises, all the exciting things that have been done around developing small, inexpensive, highly available, zero-touch endpoints, the same thing is happening in the middle of the edge, in the middle of the network that are moving workloads out of the cloud toward the consumer, reducing latency, bringing new flexibility, new capabilities. It really is an exciting time to watch, and it's exciting to see both the incumbents pivoting as well as the smaller, younger, newer companies coming up with new ideas.

So thank you so much for joining us for this discussion, Roy and Allison. And I look forward to seeing you at Edge Field Day and in the coming years as this field emerges. Before we go, though, please let us know where can people connect with you and where can they get access to some of the great resources you've mentioned? Allison, let's start with you this time. Yeah, sure. So I run the Tech Arena. That's at www.thetecharina.net. And I mentioned my Edge 2023 report earlier. I've also interviewed a number of the companies that we've talked about in this conversation today. And you can find podcasts with some of their leading technologists on the platform.

Yep. And for me, you can find me at avidthink, A-V-I-D-T-H-I-N-K dot com. And our reports are actually hosted on a site that we have jointly with one of our media partners, Converge Digest. It's actually at nextgeninfra.io. So that's N-E-X-T-G-E-N-Infra.io. And as for me, you can find me at gestaltit.com, where we do a weekly news rundown where we talk about some of the companies and products here. And you can learn more about Edge Field Day

by going to techfieldday.com. Thank you for listening to Utilizing Edge, part of the Utilizing Tech podcast series. If you enjoyed this discussion, please subscribe. You can find us in all of your favorite podcast applications. You can also find us on YouTube at Gestalt IT video. And you can learn more about what we're doing with Gestalt IT by going to gestaltit.com, your home for IT coverage from across the enterprise.

For show notes and more episodes, head over to utilizingtech.com or find us on Twitter or Mastodon at Utilizing Tech. Thanks for listening and we'll see you next week.