Everyday AI Podcast – An AI and ChatGPT Podcast - Ep 661: Out of the Shadows: How to Manage AI Sprawl

Episode Date: November 25, 2025

Even if you've banned AI, your employees are 100% using it. 🥵To make matters worse? Even if you've approved a certain AI system, your teams are probably using whatever they want. And those choices are likely putting your enterprise data at risk. So, how do you reel in manage the AI sprawl? Kevin Kiley, the CEO of Airia, is laying out the playbook. Out of the Shadows: How to Manage AI Sprawl -- An Everyday AI Chat with Jordan Wilson and Kevin KileyNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion:Thoughts on this? Join the convo and connect with other AI leaders on LinkedIn.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTopics Covered in This Episode:AI Sprawl Challenges for EnterprisesShadow AI and Security RisksCentralized AI Management PlatformsModel Agility and Vendor Lock-In SolutionsSecure Sandbox for Model TestingFinancial Impact of AI SprawlAgentic AI Security ConcernsSteps to Discover and Control Shadow AITimestamps:00:00 Navigating AI Chaos & ROI04:30 AI Tool Chaos and Security Risks09:45 Employee Decisions Risk Data Security12:50 Managing AI Complexity and Security16:31 "Agility and AI Vendor Risks"19:50 Model Performance Testing and Flexibility22:31 "AI Investments Failing Businesses"25:44 "AI Risks: Security and Weaponization"28:39 "AI Awareness and Discovery"Keywords:AI sprawl, shadow AI, managing AI chaos, enterprise AI security, AI orchestration platform, unsanctioned AI tools, AI spaghetti, AI agents, business process automation, secure AI implementation, modular AI setup, model garden, vendor lock-in, multiple AI models, open source AI, GenAI ROI, financial risk of AI, prompt injection attack, indirect prompt injection, agentic AI, token consumption, AI operational complexity, departmental AI usage, security vulnerabilities, data protection, unauthorized AI tools, central inventory for AI tools, scalable AI solutions, cloud AI attacks, domain-specific models, failover models, autonomy in AI agents, model benchmarking, latency testing, budget management for AI, positive ROI on AI, cross-platform AI management, AI system redundancy, natural language security risks, AISend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. Here we are three years past the chat GPT moment that has redefined how the majority of people work.

Yet, it's still kind of AI chaos. And that's coming from the guy that talks about AI every single day. That's because the models are always changing. The players are always changing. The capabilities are over. change are always changing. And it's almost created just this AI sprawl because yeah, some companies have approved AI tools. Others don't. And sometimes you have entire departments using unsanctioned AI, kind of this shadow AI that's casting a huge shadow over,

are you even getting a return? It is extremely difficult for decision makers in organizations to not just know, hey, what's the right platform to use, maybe take advantage of the best models, regardless of the provider, but also how to do it in a secure way that can manage the chaos and actually lead to a positive ROI. So that's exactly what we're going to be tackling on today's episode of Everyday AI. What's going on, y'all? Welcome. If you're new here, my name's Jordan Wilson.

I'm the host. We do this every single day, at least Monday through Friday, bringing you an unedited, unscripted, live stream podcasts and free daily newsletter, helping everyday business leaders like you and me make sense of the AI chaos and leverage the best there is to offer to grow our companies and our careers. If that's what you're trying to do, it starts here, but make sure you go to our website at your everyday AI.com. We're going to be recapping the highlights from today's podcast,

as well as giving you all of the other AI news that you need to know. All right, enough of me. Let's talk to someone that's actually a global leader. in this and I'm excited for today's conversation. I talk about this concept all the time, but we have yet to have a dedicated episode just tackling how to manage AI sprawl and actually show R-O-Y. So I'm excited for today's guest, live stream audience. Please help me welcome to the show.

Kevin Kiley, the CEO of Area. Kevin, thank you so much for joining the Everyday AI show. Hey, Jordan, really excited to be here. All right, let's get into it. Well, actually first. For those that don't know, and you probably should know, because Area has been on the airways for the last week or two here on everyday AI, but explain a little bit about what area is and what you all do, Kevin. Area is an AI orchestration and security platform. We're a global company, about 400 customers now around the world and growing very quickly, all around this idea of how do we help enable organizations to move faster, more securely with AI.

So AI sprawl, it seems like, has kind of gotten out of control, right? If I would rewind to the chat GPT moment and then look where we are today, I never would have imagined, right? There's literally hundreds of models that companies are using. The agented capabilities are extremely impressive, yet there's huge security risks. How are you all tackling this concept of AI sprawl? Yeah, a great question. and so relevant right now, as you say, about three years ago that the Chatschipakee 3-5 came out. And with that, gauntlet thrown down, everyone saw what was possible or saw some version of what could be done.

And in the years since, it's been a race to go employ AI as much as you can, anywhere you can, almost a reckless sort of fomo that happened across the industry, right? where every organization felt like they had to be showing they were doing something with AI. That also meant that your vendors were going to start throwing a lot of AI at you and AI washing some of their existing products, trying to launch new products, maybe turning on capabilities that they didn't really communicate to you that they were enabling. You had departments doing their own thing, even employees doing their own thing. Shadow AI, as you mentioned, where perhaps a well-intended employee wants to be more productive. it. They want to be more impactful. So they're bringing their own chat GPT or perplexity or something

to work with them and using it with their company data. Well, all of that sprawl has created the spaghetti mess that we're in now, where you've got all these different tools, each with their own management needs that may or may not be met. And that's huge security risks. You've got agents that are being created by employees, often being given the employees individual permissions which are very broad versus maybe the agent was supposed to go do this one thing, but instead now it's been given all these capabilities to do a lot of other things that could create risk. And then there is the financial aspect that you sort of touched on as well.

All these different purchases have been made without one sort of unifying direction, a lot of overlap redundancy and capability and neat. And it means that the companies have been drowned in different invoices from different vendors without really any clarity as to whether those projects are performing for them and they're really getting return on it. So that is the mess that we've set out to try to address, to bring some clarity by providing kind of an essential control plane, if you will, across the organization.

Help them understand what AI has been employed, who's using it, how they're using it, create some record or observability of it all. And then once we know where it is to apply some controls, some guardrails, and then ultimately help them build more efficiently. So you can build within our product using any model. We've got thousands of options. Integrate any of your tools or applications all within one.

But if you've already got an existing vendor, you're building on great. Let us just help apply the controls, the guardrails, the visibility, and give you that overarching management across all the AI in your organization. Yeah. And Kevin, I love that analogy. I'm going to just maybe steal it because it's so good, right? It's like AI spaghetti, right? Like everyone's throwing AI at the wall,

except they might not actually go back and see what sticks, right? Maybe they just throw it at the wall and walk away and hope that revenue goes up, productivity goes up, right? But I'm curious, when you're talking enterprise leaders right now, what does AI sprawl actually look like on the ground? I think the most immediate thing that resonates is the security risk of it. And I talked about maybe, you know, that these are overly permissed agents that an employee may stand up. Nobody really knows what it's doing apart from that one individual. And if they leave

the company, who knows? If you speak to a lot of these CIOs and even the CISOs will often acknowledge, there isn't a central inventory of what's happening out there. That's the immediate concern. And that's only going to become more pronounced as we see more and more these attacks happen. This past 90 days, we've seen a number of these in the press where, whether it was prompt injections or indirect prompt injection. There's a couple of big ones that were announced just in the last two weeks. And I think you'll see a lot more of these scary stories starting to show up where companies are finding that they're very vulnerable because AI has been thrown out. So sort of rushed without a lot of control or visibility. So that's the most immediate concern from there.

it is often about how do we help rationalize some of the use, find a way to enable our employees to embrace it, whether you're a citizen developer who's never done any AI engineering, you know, data scientists, et cetera, which you're really good at your one function within the business. Maybe the best at finance or marketing or whatever it might be. Let's give you capabilities to build your own AI using your knowledge and we'll take care of the plumbing bits around it that help make it more productive. All the way up to the most advanced users who really already know AI, but maybe don't want to be bogged down with doing some of that integration work.

We can help with all those components. And so that's usually, I will sort of progress our dialogue with the client is starting with discovery, providing some security, and then enabling them to do more with AI in control way. Yeah, it seems like, yeah, you know, we're kind of referencing the, you know, original chat, HVT moment. It seemed like, you know, back then the model was maybe the most important thing. But fast forward to today, it's probably not. It's the scaffolding. It's the plumbing, you know, like you talked about. That's extremely important. And another thing that's extremely important is to just

call a spade a spade here, because I talk to people all the time. I'm sure you hear it. You know, leaders worried and rightfully so about employees using unauthorized AI tools, right? Regardless of what report you look at, you know, the stats are all over the place, sometimes up into the 90% of people using unauthorized AI tools at work. And the worst part is sometimes they're using the free version, which doesn't product data. And employees aren't always sure of that. So, you know, what are you seeing in terms of shadow AI behavior that maybe worries you the most? it is often than well-intended employee that I mentioned to who, you know, they're really trying to move up. They've got a mandate to go build their piece of the business and drive some new product launch or whatever it might be.

And they are not trying to do anything that would undermine security, but they're, of course, giving the agent permissions or maybe it's uploading sensitive data that in their context doesn't look that dangerous. but if a bad actor were to get to it or your competition were to get to it or someone else that could take advantage, that's where the risks really start to get introduced. And of course, at the same time, we've got this Cambrian explosion of models. You mentioned already, you know, there's so much innovation happening out there and so much of it is very positive. But if you look back of this past year, things like Deep Seat coming out, you don't have to look far to see all the risks that comes with it. They're pretty upfront with their terms and conditions. If you read through the legal agreement, it says you have to look far.

have no guaranteed confidentiality. They're going to send your data to China. And look, again, the employee who's trying to just get stuff done maybe isn't taking the time to read through all that. They just know that this helps them write a better presentation in or improve their analysis. These are the sort of things that you've got to give them tools. It's not enough to just say no. And I think it's almost more dangerous if you try to have some sort of prohibition in place. A lot of companies have started with that. Your employees are trying to move quickly. trying to make it work, they're going to find ways around it. So you have to serve something up, better that you give them a model set that you can trust or at least a canvas they can work on

where you know you've got some protections in place. That's a good point. And I think also, you know, yes, employees are going to go out and find a solution if one isn't provided. But sometimes they might find more problems than solution and maybe they aren't even aware of it. But, you know, one kind of common trend that I saw, especially in 2024, what I referred to as duct tape AI, right? When you have different organizations using different models and, well, if you don't have a platform that brings it all together, that just might create more work in the long run than it's even worth. You know, if you're doing a certain task, you know, 30% faster, but you have to rewrite your business processes. It's not going to lead to a positive ROI. So can you talk a little about a little bit

about because one thing that I love about area is just the platform that allows even people, you know, no code, low code, so kind of regardless of experience level, to use any model, right? So whether you're using an impropic, Google, OpenAI, right, and keep it in a secure environment. So can you kind of talk about that kind of like any model, any skill level approach that you all take? Sure, absolutely. So again, back to all the innovation out there now, two million plus models on Hanging Face, which is crazy to think crazy. of fast that has exploded in the breadth of options that are out there. And again, a lot of that's really positive.

There's innovation. You're going to find models that are really good at certain things that you can use and bring correct benefit to the business. But all of that means more operational complexity. They're all going through a lot of upgrades. You may get onto a model that gets deprecated sometimes a matter of months or even weeks after it's released. Of course, there's security vulnerabilities, which are continually

being discovered and and having a way to see those or at least update and manage those is a really important part of all of this as well. So back to your question again, being able to provide almost like a model garden for your employees is an important part of this where we could serve up a list of models that we consider to be trusted or perhaps we've already done the procurement work behind the scenes. So you don't have to worry about putting a card in and get all the different spend that's kind of gone out of control. Let's have a central view into where we're consuming, set of budgets, reading thresholds, for notifications. These are all capabilities that we can provide within the platform that help bring some insight into, again, how are we using AI? Are we

getting good return on it? But also being able to manage this from a project level, departmental level, which to date has been really difficult to do. And I don't know, I don't want to accuse the model providers of deliberately trying to make it kind of opaque, but it certainly is each of them have their own consumption models around their tokens, and you have to set up each one individually. It just creates a lot of operational overhead for especially a larger company. Yeah, and that's a great point because I think it's something, you know, in the chase for, you know, business leaders to always want to use the latest and greatest model, right?

I see a lot of times sometimes people just boxing themselves in, right? Because you might not think that there's a huge difference between, I don't know, a enchropic Sonnet 4-5 and, you know, Google Gemini 3, right? But there actually is and there can be. You know, can you talk a little bit about the modular kind of setup and why it's important to have a platform that maybe allows you to avoid vendor lock-in, right? because, yeah, like you said, what if a model gets shut down or if all of a sudden a new update causes it to be overly sycophantic or something?

Right, right. Well, the first thing I would point to is, again, that there's this arms race of innovation happening right now where month to month, sometimes week to week, you're seeing a new leader take that top benchmark for performance, accuracy. We just all continue to be blown away by how quickly the space is evolving. and it's performance, it's the fact that they may be shedding that old model that you got hooked on, or it may be the cost implications. You know, if you started on Chad Shp.T. 4-0 last year and somehow tried to stay on that.

Very powerful. And really, again, it was so exciting when it first came out, but it was also quite expensive. And we've seen that every subsequent release, the cost has fallen significantly. Even going from 4 to 4.1, I think, was something like an 800. percent difference. Massive amount of change there between how your token costs were being occurred.

So that continues to happen. In this space, it's sort of war's law on steroids, where the cycle time is getting faster, faster. The models are getting more and more powerful, and often the costs are coming down on a token basis. Now, there's another level of detail behind, again, how the

tokens are consumed, but it's, you can't deny that the models are getting better and by and large, they're getting cheaper for what you need them to do. So being able to stay agile becomes really, really important. I would argue a key part of your organization's AI strategy is making sure that you maintain some free agency, if you will, to be able to swat as these different large companies battle between each other to deliver a better in technology. Keep yourself open so that you can harness the latest and greatest. at the most efficient rates. And then the bigger issue of vendor locking comes into play

where they are going to, of course, be trying to hold you into their ecosystem and gain more or leverage. And for every organization, as you build into greater dependency on AI, you have to look at this as a risk, not just from a financial perspective, but also from business continuity.

There have been a lot of outages over the last year. I won't name names, but you can Google most of this. you'll see that most of the major providers had significant outages, sometimes six, 10, 12 hours long. And if you built an application that is business critical or worse yet, maybe patient critical, and that goes down and now your team is really relying on that for the way that they're working, got a really significant problem. The ability to route between models and providers, I think, will become sort of a standard of care.

You have to have that if you really got it credible program. Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant, now live in the Adobe Firefly app, the All In One Creative AI Studio. Powered by Adobe's creative agent, Firefly AI Assistant lets you start with your vision, just describe what you want, and shape the outcome as it takes form with the Assistant. The assistant orchestrates multi-step workflows, drawing on 60-plus pro-grade tools across Adobe Creative Cloud apps,

including Photoshop, Illustrator, Premiere, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks, like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible, so you can refine, redirect or take over at any time. You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adobie.com.

Yeah. And that ability, right, it requires a secure sandbox for people to go out test, right? Because what a lot of people don't realize is you might look at a model like a GPT51 and assume it's the same until you see a GPT5. or a GPT 55, but that's not the case. There's always kind of updates going on under the hood. So can you talk a little bit about the importance of having that secure environment where you can not only constantly test or prepare, you know, in the case that you just

brought up, prepare if, hey, a model does go down for six hours, 10 hours, 12 hours, have you securely tested the next model up, so to speak? Can you talk a little bit about the importance of that and how you all provide that option? Absolutely. So I think we start by looking at, and as you're building a model, have you tested the different models to see which is going to be the most performative for your use case? And we can measure that on a number of levels. It could be latency. It could be cost, as we talked about. So running the same task or prompt across four or five models to see how they're all going to compare to each other. And then continue to monitor on that basis. The initial exercise will allow you to benchmark them for what you're going to be. trying to do and define what your primary model is and then maybe a secondary or failover, or even a third failover. So that in the event that something goes wrong, not only can we detect

that, let's say, latency is spiking on mistral's model or quad or whatever it is we've used for this and that there's real performance concern, but we know to now route to the next model, which could be from, again, their competition. And so not only is it, important to have, I think, the capability to swap between them that way without having any disruption and service. But the fact that we're not beholden to any one of those companies, I think becomes a really important value for us, that we park our with all of them and a long list of other open source models that our clients can choose from so that you can trust us to move between them. And if you were just all in on Microsoft or anthropic or any one of them,

Could you really ask them or expect them to route you over to their competition when something goes wrong? I think that's why you need an independent third-party company to sit as this sort of friend that they managed that. Yeah, I liked what you said earlier, kind of the free agent provider, right? Like, that's important. You know, but so I'm curious because you also mentioned, you know, two million models on Hugging Face. And I know even, you know, smaller domain-specific models are getting more and more common and popular in the enterprise. So, you know, I assume this two million model, if we talk next year, it's going to be way more. But that also creates more sprawl.

It creates more potential security if you're trying to just do them all individually on your own without using a provider. But maybe if we look at it from a CFO's perspective or a CLO's perspective, or a CLO's perspective, you know, where does AI sprawl kind of quietly start destroying value before anyone notices? Because I think from people I've heard of, once you notice, it's either not too late, but you've already lost a lot of ground. So where does it start to pop up before anyone really notices? Well, the big problem that has been well documented in a lot of different studies. And most recently, there was the MIT Gen A.I. Divide

about the state of AI and business report that was really kind of an earthquake across the industry, and I suspect a lot of your audiences has had a chance to see this, or at least read parts of it, where they saw that 95%

of these pilots never get to production. That's staggering when you think about all the money that's being spent and the good intention behind all the tools that are being purchased, it implemented, consultants. You know, it's a lot of waste, unfortunately, that thus far has just not been able to deliver real value to the business.

And I think the number that they put in the study was they ranged in from $30 to $40 billion worth of investment across the enterprise right now. So, I mean, it really is just wild to think about how much, how much money is being burned on some of this right now. And that's got to be front of mind for the CEO. follow is his understanding all this ike how much of it is really translating into to value are we making good use of the investment and you know we've kind of touched on security a little bit but i want to dive a little bit deeper right i'm i'm one that's always you know testing out the latest models

and seeing how they work and if if i'm being honest over the last i don't know two quarters um seeing what just models are capable of um it's it's it's it's it's it's it's almost getting to the point where it's like, wow, a single model can do a lot more than an entire system could do, you know, two years ago or even 18 months ago. You know, talk about how quickly security changes on the AI side and, you know, what business leaders need to know about, you know, kind of the intersection of agentic AI and security. Well, I think, again, there's so many movement pieces here for security to keep up with, and it really is a new world. The existing stack of security tools and even methodologies that most organizations have in place today, just they weren't built for this.

They never contemplated what an agent might be able to do. And so if you think about that, an agent is deliberately given some sort of goal and some autonomy to go. accomplish it. It's also going to be given permission and access to go touch multiple systems and tools. So it's very, very capable. As we've talked about, you know, already today, sometimes too capable for what it was initially set out to do. It was given permissions of an individual that may have very broad set of capabilities. That's a little scarier because then security doesn't know who really touched what information, what systems. You've got it running around.

There's going to be a lot of ambiguity about how do we lock that down. If the employee leads the organization, do we know to turn that off? What do we want to turn off? So the whole agenic paradigm shift here that's happened is a real issue for security. Beyond that, though, AI itself being so powerful, but only when it's prompted correctly is another risk. You know, as we move from APIs, which were very deterministic, and I could call this system and get this bits for these fields,

to something that is driven by natural language, which is subject to interpretation and can be manipulated. That's a real challenge. You want the AI to be able to understand a command that may be presented in a number of different ways, but that could be deliberately weaponized to try to take advantage of the ambiguity of natural language. And then lastly, if I had to give a third sort of concern around this,

and this is that eye level, just that AI enables a scale and a sophistication that wasn't humanly possible before or wasn't even capable within the boundaries of computing. Now you can have swarms of agents that are going to attack your defenses. And instantaneously, communicate deficiencies, gaps, vulnerabilities that they're finding with each other to further

manipulate those and surge attacks on certain parts of the business or your defenses that never would have been able to be coordinated at that sort of scale or velocity previously. It's very, very scary to think what AI will be capable of doing in the wrong hands. And we're already again starting to see some of this, as much potential as there is for good and productive outcomes, there's at least as much potential for bad with those that want to manipulate it and come in with malicious intentions. Yeah. And great, right, great example of that.

Yeah, there is the recent report, the Chinese backed, you know, using Claude Code. Yeah, scary stuff, but just using off the shelf models, right? It's not like you have to have the world's most sophisticated cyber attacks anymore when you can use natural language and off the shelf models. So, you know, we've covered a ton in today's show, Kevin. Again, I'm stealing the AI spaghetti thing. That's great. But I'm wondering as we wrap today's show, if an enterprise leader right now and they're like, wait, something you said, this hits. Clearly, you know, my organization, we're dealing with AI sprawl or, you know, we need to get shadow

AI under control. What are the things that you would tell them to do immediately, you know, in the next, you know, couple of weeks once they realize that sprawl has become a problem within their organization? Well, again, I think awareness is where it starts. So the fact that they recognize some of it is great, but truly investing in the discovery is going be critical. So whether it's technology like ours or using some other methodology to go really identify where AI has been employed across the organization. And those could be vendors that you know about. You may just not know that they turn on the AI and inside their

tool. That's a very common one. Second, there may be other decisions that have been made, again, in this huge rush to employ AI that a lot of people started pulling out company cards or even expensing work that they were using on other tools. And this is where you start to get into some of the shadow AI. So identifying where the AI is and then applying some protections around it. Again, you may find a lot of this is probably good work that's being done. And you want to continue to support that. And if anything, maybe fortify it.

Make it more durable. But I can guarantee 90% of the situations you're going to find AI that you weren't aware of or you certainly didn't intend for and that there is real risk beyond. Such great insights, especially timely, right, as we're looking at end of year and making everyone's plans into 2026. Kevin, I think you just gave us a great roadmap on where we should start to rain AI sprawl in. So thank you so much for taking time out of your day to join the Everyday AI show.

We really appreciate it. Jordan, my pleasure. Really glad to be here today. And thank you for the opportunity to join. All right. We covered a ton of great content there, y'all. If you miss anything, don't worry. We're going to be recapping everything that Kevin just went over in our newsletter.

So if you haven't already, please go to your EverydayaI.com. Sign up for that free day and newsletter. Thanks for tuning in. We'll see you back tomorrow and every day for more Everyday AI. Thanks, y'all. Meet Firefly AI assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words.

and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premier Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time. See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us.

If you enjoyed this episode, please subscribe and leave us a rating. helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.