Everyday AI Podcast – An AI and ChatGPT Podcast - From Automation to Agents: Why Weak Data Makes AI Guess

Episode Date: December 11, 2025

Algorithms and automations have been buds for a decade plus. 🤝But the old 'smart' automations were rigid. If one thing was wrong, the automation would bust. But with LLM-powered agents? Those automations are different. If something's wrong, the agent might just..... guess. 😳Weak data = weaker outcomes. Here's how to fix it when agents come first and they're gonna finish the job, whether the data is strong or not. From Automation to Agents: Why Weak Data Makes AI Guess -- An Everyday AI chat with Jordan Wilson and Ed MacoskyNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion:Thoughts on this? Join the convo and connect with other AI leaders on LinkedIn.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTopics Covered in This Episode:Automation vs. Agentic Workflows: Key DifferencesAI Agents: Data Quality and Output RisksAgentification Trends in Enterprise AutomationPros and Cons: Converting Automations to AgentsAI Agents Impact on Business ProcessesImportance of Data Governance in AI AgentsAI Agent Control Towers and ObservabilityMeasuring ROI: AI Agents and Data InvestmentsTimestamps:00:00 "Trusting AI for Business Growth"06:31 "Agent-Based Automation: Pros & Cons"08:29 "AI Agents Simplify Workflow Management"10:40 Expense Report Workflow Frustrations15:05 AI Governance and Data Integrity19:04 "Governance and Multi-Agent Data Strategy"22:50 "AI ROI and Data Focus"26:01 AI Studio: Create Apps FasterKeywords:AI agents, Agentification, Agentic workflows, Automation, AI-powered automation, Business process automation, Deterministic workflows, Non-deterministic workflows, Data quality, Data governance, Data management, Integration platforms, Boomi, Chief Product and Technology Officer, Large language models, Generative AI, Rigid workflows, Flexible workflows, Conversational agents, Expense report automation, Policy adaptation, AI decision-making, Human-in-the-loop, AI observability, AI traceability, Multi-agent orchestration, Ecosystem synchronization, Agent control tower, AI governance tools,Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info)

This is the Everyday AI Show, the Everyday Podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. Here's a very obvious trend as we gear up for 2026.

Almost everything is going to get agentified, right? Your standard procedures are going to become agents. Traditional processes, agents, features are now just called agents. And what used to be called automations will be agentified as well. But here's the thing that's important. I think when we think about, old school automations, right, those that have been around for a decade or more, if you didn't have the right data or if it wasn't configured correctly,

that workflow probably would just fail. And the output, well, you wouldn't get one. But when we are starting to make the transition from automation to actual AI agents, that agent, even if something's not right, the data, something else, it's still probably going to give you an output. It might straight up lie or guess. So how do you get around that? And how do you make sure that as you are bringing more and more of your business processes

into agentic workflows or building AI agents, how can you make sure they're not guessing and hopefully giving you outputs that you can trust and actually use to grow your business? All right, we're going to be tackling that and a lot more on today's episode of Everyday AI. What's going on, y'all? If you're new here, welcome. My name's Jordan. I'm the host, and we do this every day. This is your daily, unedited, unscripted, live stream podcast and free daily newsletter

helping everyday business leaders like you and me, not just keep up with all of the AI craziness, but how we can make sense of it to grow our companies and our careers. So if that's you, starts here with the podcast, but make sure you go to our website at your everyday AI.com. Sign up for the free daily newsletter. We're going to be recapping the highlights from today's show and a whole lot more. If you want the AI news from today, make sure you go read it in our newsletter. All right, enough of me chit-chatting, y'all.

I'm excited for today's guests. So please help me welcome to the show, Ed McCoskey, the chief product and technology officer at Boomy. Ed, thank you so much for joining the Everyday AI show. Thanks for having you, Jordan. Excited to be here. All right, me too. Ed, before we get started, tell everyone a little bit about your background and what you all do at Boomy. Happy too.

So I am the chief product and technology officer here at Boomy, which means. I run all of product and engineering for Bumi. And who is Bumi? We are in integration, automation, and AI activation platform. So we spend every day working with our customers on automating their businesses through either traditional workflows or now through AI agents and or activating AI within the business. A little bit more about me. I've been an integrator now for, gosh, 24 years.

So I've spent my entire career. thinking about integrating and integration and automation within a business and with the rise of AI and agent technologies. It's like a whole new, it's a renewed sense of, you know, pride and excitement around how we can apply these within the business. And that's what we do everything. And I love talking to people with a deep background in tech, specifically in automations, right?

Someone myself, I've loved, you know, the marketing automation sector for a long time. I've always experimented with different, you know, workflow. So it's obviously been exciting, you know, over the last two years as we're able to add a little bit of agency to traditional, you know, automations that didn't have those. Can you just maybe catch us up to where is the space at today? You know, our most, you know, enterprise businesses, are they using, you know, still old school automations? Are they having agentic workflows?

Are they having just agents, a little mix? What are you seeing in your role? I think a little bit of both actually. And I think we will be both deterministic and non-deterministic or agentic workflows in the enterprise forever. The reality is there are certain use cases where you're just going to want to have the performance of a fixed workflow in your business where that applies. You want to be structured on the rails. And the way I think about it, you know, agentic capabilities actually helps you augment and expand more and more capabilities. is there some overlap where we do have agentic workloads that are overtaking some of the traditional

workflows to help with maintenance and how we can think about these capabilities and be more robust. I think that's where businesses are today. They turn the corner from AI pilots and yes, individual productivity with AI that is still a thing, but in terms of automating the business, we're starting to see some real results to the tune of saving millions in these, you know, team workflows and whatnot that we're agentifying with our customers. Yeah, I think agentifying and agentification, right, it's almost like the new buzzword, you know, where it was agents maybe a year ago. But can you kind of walk us through some of the

pros and cons, right? So I'm sure that many companies out there, you know, have been piloting different AI solutions over the past year or year and a half. And they maybe have some, you know, automations that have been the backbone of their business for maybe a decade or more. Can you kind of walk us through some of the pros and cons of as companies move from old school automation to newer school agentification or just AI agents out there accomplishing tasks on their own? What are some of the pros and the cons of kind of converting those older processes to agents? Certainly the pros are you can kind of as a human, or your humans within organization

when you convert some of these workflows into agent-based flows, you can take your hand off the wheel a little bit. I like to think about it when you're driving, you know, these self-driving cars where you've got someone sitting there and running, but getting more to the point of, you know, a way mode that is fully self-driving. We see that happening. So that's the pro.

You can get a lot more out of your workforce. You can let the humans do more and provide more value in the business versus these mundane kind of driving things that they click a button every day and what traditional workflows bring. The cons are you have to build these agents and these automations, these new agentic automations on a strong foundation of data or bad data in as worst data out in terms of the activity that you would expect from agents,

right? Whether it be the decisions agents are making or the actual ways in which they're wielding tools and actually activating change or updates within your business, you have to be very, very careful. So that is a big cons side. But we're here to make that better and help you. Yeah, I love that. Bad data in, worse data out.

That's a great takeaway already. You know, one thing that I even remember, you know, personally, you know, way, you know, 10 plus years ago working with whether it's RPA software, you know, marketing automation software, I would spend a lot of time building these automations. And if one thing wasn't right, right, maybe there's an extra space or, you know, you know, a period instead of a comma or something like that. The automation wouldn't run because it was overly rigid and, you know, obviously deterministic.

Can you walk us through maybe for our more non-technical audience, right? What does this mean specifically when agents have agency to maybe make their own decisions and what that does for the importance of our data? Yeah, it makes it super easy. So, you know, the genetic technologies like Bumi, for example, make it, very easy to not have to maintain to find, you know, these rigid if then else rules that if you're missing a semicolon or you've got a space in there or something that they go bad or new rules get introduced and data changes. That happens that can throw off these rigid

workflows. AI agents and using agents to help make these decisions in a workflow, which could be an AI agent wrapped with a traditional workflow if you'd like, but makes things much, much easier and less brittle when implementing them. And then they run and they can make decisions on your behalf. So you can go in terms of you can do things like just defining your policies within the normal documents that you normally do for your employees and let the agents read the policies like your employees would and agents can adapt and make those decisions in a workflow. That is all great.

the downside or what could happen when you do these is now you could take your eye off of managing those policies. You would be forced to in an old fixed workflow because you'd have to go to some human who would write the code to say, I'm going to add five lines of it, then else because we added a new policy. It's just shifted to today. You need to make sure that your policies are well defined. You're maintaining your internal wikis or your documents or your PDFs or the things that you have, which you should be doing for normal good business hygiene anyway. But as long as you live and work in the systems you have to find, and then you have your agents point into those, you should be fine.

Could you maybe give us an example? You know, maybe one of those tried and true automations that may be more deterministic in years past. And now with AI agents, so companies that have maybe converted those older automations into agentified them and, you know, either agentic workflows or AI. agents, do you maybe give us an example of, hey, here's what that process looks like, pre-generative AI powered AI agents, and here's what it looks like now with them. Yeah, there's a lot of them, particularly around the finance area and different things in the business, but my favorite one is expense reports. I think it's one that everybody can kind of relate to that works in a business where you have

to file expenses. Whether you be the person filing the expenses or the manager leader in the business, that is approving the expenses. traditional expense report workflow is I submit some expenses. I have a whole bunch of roles that someone has defined that tells me error or not as I'm entering expenses. And then once I click submit and I finally get through all the errors, which I probably faked half of them to get through the errors. It goes to my manager and then he or she then has to approve and do the same thing from the other side. Now there is frustration on both sides in terms of the people, one who submits and the person who then has to approve.

And then there's a lot of human inter, well, A, there's a lot of humans in the back encoding all of those rules in some way, shape, or form or defining them. Then there's humans on both sides that are doing those activities to either submit or approve them. In terms of an agenic process, you can make life easier on all three parties. The person who's submitting, you can make it, you know, we are doing this. You can have that as a conversational agent to make it easier for the person submitting. an expense report to just really give the information and or set the tone with the humans filing

the expense. Here's what you need. Here's the only stuff I need. Make it really easy expense report taken, created. Then for all of the rules that would be part of that, much less rigid, much more open that these agents can then use large language, there's small language models, go out and read all these rules and dynamically adapt and adjust. And then on the approver side, these agents can also learn and only engage approvers when necessary. So if you are submitting expenses to me, Jordan, and I'm approving the $100 bottle of wine every time, even if it's not part of the policy,

the agent will learn, Ed will approve Jordan's $100 bottle of wine. But if it's a $20,000 bottle of wine, the angel will know, wait, Ed's never approved this. It's outside of the policy. That's when I'm going to engage the human to come in the loop to kind of make that statement. So an approval.

So you can kind of get, you know, best of all worlds by being adaptive and not so rigid in this process, but also the foundation of data underneath, maintaining these roles, understanding them, maintaining the who's Jordan's boss. So I know who to submit this to, who's this and that. Those are all data quality things underneath that people have, you know, unfortunately in the last number of years in enterprise haven't really paid much attention to because humans have been a crutch. Humans have come in and all that data. as bad, but I know I can approve this and, you know, I can't automate that because of X, Y, and Z. That's why data quality is becoming more and more critical for these agentic flows. And I want to dig into that a little bit more. And also, there goes, there goes my tactic of, you know, expensing things that, you know, 10x the price. There, that goes out the window. But, you know, one thing, and I'm not too big of a dork to admit this, I enjoy reading summarized chain of thoughts, right? I love, you know, working with the latest, you know, models, whether that's, you know, Gemini 3 or GPD 51 Pro or the newer Claude 4-5 opus, right? And I love looking to see,

you know, how they take whatever inputs and then what they do and what they think through, right? And one thing I've really noticed is, well, obviously, models can be bold and they can sometimes be a little too confident, right? Especially when you throw a ton of data at them all but the same time. And sometimes even just one wrong assumption, maybe the model isn't even wrong, but maybe they just assumed something and it's not fully accurate. They might take that and run with it and go in a completely different direction than you may want, right?

And unless you're a dork like me looking at the chain of thought or, you know, really paying attention to what your agents are doing, you might miss that. So with that in mind, right, and as these models become more and more agentic by default and more and more capable. Talk a little bit about what the average, even the average everyday business leader needs to do when it comes to working with data inside of large language models.

Well, there are a couple of things because I see this like a sandwich based on what you said. The foundation business needs to continue implementing tools and make sure that the underlying data is as pristine as humanly possible or technically possible. and not overlook that and put in true data governance capabilities within your organization. And then you need to make sure that your AI agents have a true security layer so that agents are accessing the proper data sets and not overstepping bounds there.

That's another way of protecting from an agent, getting some level of data, and then going and stepping over some boundaries. That is at a foundation. So to your point, the littlest data that can be off can have bad results. I said this earlier, bad data and worse data out in terms of the actions. The top layer of the sandwich, as I see it, is investing in AI governance tools that can oversee AI agents and what they're doing. There's tools in market, you know, there's observability and telemetry tools that are in market today that you would think about governing and having visibility into just your business operations or IT having those, et cetera. we have actually brought to market an AI agent control tower.

And what that does is we'll monitor the behavior of agents that are running in your business that you've implemented as another backstop for these types of things. So agents are often running. They don't even have to be boomy agents. They could be agents in agent force agents in Salesforce. They can be AWS Bedrock agents, combination of boomy agents, all orchestrating together. And we will monitor those to look for an obelies. So using another layer of an agent, an agent.

layer to oversee the agents that are running so that if you see some anomaly behavior like, hey, agents have been doing AB&C, but this one looks like it may have stepped out of bounds, it will at least it will alert the business user that, hey, this agent did this, or you might want to check on this, or do you want me to stop this agent from behaving here to help you with that governance? Because as a business user trying to manage these thousands or millions of agents in your organization without using AI to manage them as well, is going to be nearly impossible. So those are the types of solutions of things I think folks should look out for.

You kind of led into the next question that I naturally had is I feel observability, traceability, right, is maybe a little bit easier with earlier scopes when it comes to AI agents, right? When there's maybe one AI agent that you completely understand. and it's a little bit easier to look under the hood, so to speak. But as we start going into these multi-agent workflows and multi-agentic orchestration, how does the kind of compounding impact of good, clean, and accurate data, how does that play out when, yeah, now all of a sudden you might have agents handing things off to each other without a human in between?

Yes. So even worse, data out. So it's important to double down on the governance of your data in that aspect and to double down on having observability in aggregate to these agents working together and what they're doing, what data they're passing, looking for anomalies without just shamelessly plugging the booming technology the whole time. There are other control, there are control tower capabilities out there. The different hyperscalers are providing visibility at a control tower perspective to get. insights into what the agents are doing in their ecosystems and not just hyperscalers, but application providers are providing these. A value that Bumi has is helping to aggregate them all together so that you can see, okay, from an end-to-end series of agents that can be orchestrating across ecosystems, which isn't a big focus right now. A big focus is A-to-A and communication protocols across ecosystems, but not yet. That's where we're going next, around the governance across ecosystems and making sure that

what agents are doing together are, you know, sponsored and what the businesses want to do. The business wants to do. But when you think about there's two levels of your data when you have multi-agent or multi-eco system orchestration, there's the data that resides within the system that the agent is on top of. So agent force on top of sales force data. Then you can have a Service Now agent on top of Service Now data. that if that data is not in sync and of quality on both sides, you can have disaster. Those agents can talk past each other and not be sharing data.

You need to come up with your strategy for your foundation of data that is kept in sync or you're leveraging an agent, multi-agent system that is on top of the same data set. Depending on your organization, I know I'm getting a little technical here, but depending on your organization and your maturity, you're going to want to think about those different things. Most companies are probably working with agents within their silo their silo ecosystems versus on a common data set. But based on the use case, you're going to want to think about how you, you know, you architect that within your business.

Yeah. And I think that it's, you know, well past a foregone conclusion now that you need high quality data in order to fuel agents or agentic workflows. Yet, I mean, I've talked to hundreds of people on this show. and I'd say dozens of Fortune 500 executives that even themselves, whether they admitted it on the air or not, don't always feel the most confident about their data. So how should those types of business leaders be looking at this problem, right? Because data is always evolving.

We're getting more and more of it. It's coming in different formats, right? Especially as some of it's being formatted specifically for large language models. and maybe humans don't jive well with that, right? But how can those organizations that feel maybe iffy on their data game, how can they still use AI agents? Or should they not? Yeah, I'm going to try to give some very, very practical advice here

because this is what we're seeing every day. So we're in this, the fastest innovation lifecycle we've ever seen in IT, frankly. Like every day, I mean, your podcast is showcasing, a lot of this that's happening every single day. And what's happening is we have the new wave of technology that's emerging around AI and a lot of people being very innovative there and so on and so forth. But those folks may or may not have all of the context that an enterprise or a business would have around their data and workflows and all those other things we talk about.

So the very practical thing to do, it sounds dead simple, but I'm not seeing enough of it. Pull your AI innovators together with your business owners, those that own the data and your AI innovators and get them talking and working together. We just, a lot of what we're doing is pulling line of business owners that own these data sets together with the AI innovators and facilitating discussions on what they can accomplish together and focus very practical, practical on not boiling the ocean of all of your data and your whole organization.

Look at the data sets that you need for automating your business or the outcomes that you're looking for and really double down on those. You'll be much more cost effective than some large data. data project that you want to do in your organization. Focus on the data sets that are going to matter for your agenda workflows. Start small and iterate from there. Prove value very quickly and then build on top of the value you've already. You've seen in previous cycles.

All right. That was good. I mean, Ed, you took my last question out of my mouth because I think that's some great kind of parting words for business leaders. But I'll go this here. You know, I think, you know, at this point here, rolling into the end of the fourth quarter, a lot of CEOs, business owners are, you know, looking back on their 2025 AI investments and, you know, trying to look at ROI and plan for

20, 26 and beyond. What would your advice be specifically when it comes to kind of measuring the return on agents? And then also even the return on investing in your data. because at least for me, I've seen a lot of companies, you know, putting way more resources into their data in 2025 than they had in maybe the last half decade combined because of this tight correlation between data quality and agentic output. So kind of two-prong question there, where should they be focusing for showing ROI and then also investing in their data processes? I would, great question.

And I would flip the script entirely in terms of the way you look at defining a project and its outcome. So in the last couple of years, from an AI to data projects, people have focused on AI technologies or data technologies or data and tried solving those problems which have led to or addressing opportunities in the business. They what I always refer to as these science projects. They go off. People get fascinated with the technologies. They get fascinated with the data itself, et cetera. they go and their outcome that there is clean data. Well, the clean data by itself doesn't mean anything or this really cool AI thing doesn't mean anything.

Flip the script. Start with discrete projects where you want to define your ROI in the business process or thing you want to optimize and what is that ROI and work backwards. So from there, you say, okay, what are the outcomes that I want and what are the returns that I expect from those outcomes? Then apply the technology and the data projects to those. then you can't go wrong. You're consistently focused on the outcomes that you're looking to seek

versus kind of working on a solution hunting for a problem. I would just flip the script. I love that. Great way to end today's show by flipping the script and starting with the outcomes, building back from there. Love to hear it. Ed. Thank you so much for taking time out of your day to join the Everyday AI show.

We really appreciate it. Thank you. It's great being here. All right. And if you miss anything that Ed said, don't worry. We're going to be recapping it all in today's newsletter. So if you haven't already, go to your everyday AI.com. Sign up for that free daily newsletter.

If you're listening on the podcast, appreciate your support. As always, make sure that you please subscribe, like, and leave us a rating on Spotify or Apple Podcast. Thanks for tuning in today. Hope to see you back tomorrow and every day for more Everyday AI. Thanks, y'all. Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words and the assistant handles the rest,

orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time. See it today at firefly.adop.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating.

It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.