Experts of Experience - The Art of Conversation Design for AI Agents

Episode Date: April 9, 2025

Agentic AI isn’t coming — it’s here and already changing everything.Irina Gutman, Global Leader of AI Professional Services at Salesforce, breaks down what agentic AI really is and why it’s a huge leap beyond predictive and generative AI. We get into why your first AI agent should be boring (and repeatable), and why building the tech is easy compared to rewiring your people, processes, and leadership models.Irina shares why businesses need strong guardrails, real operating models, and why AI adoption without organizational readiness is a recipe for disaster. We also talk about the skills humans actually need to stay relevant, how to spot hidden risks, and why the future belongs to companies who rethink their structure — not just their tools. Key Moments: 00:00:  Irina Gutman Explains Salesforce’s AI Agents03:03: Predictive, Generative, and Agentic AI — What's the Difference?05:20: How Agentic AI Thinks and Acts08:32: Chatbots vs. AI Agents: Why It Matters14:22: The 5 Building Blocks of an AI Agent18:13: Organizational Readiness: New Skills, New Roles22:26: The Right Way to Start with AI Agents26:27: Future-Proof Your AI Strategy29:53: Rethinking the Operating Model for AI32:45: Upskilling is Non-Negotiable35:14: Vendors Can Help You Be AI-Ready36:25: Rethinking Change Management for Agentic AI42:38: What’s Next: Multi-Agent Collaboration48:09: Building AI Responsibly: Guardrails and Risk51:39: Real-World AI: A Standout Customer Experience  –Are your teams facing growing demands? Join CX leaders transforming their AI strategy with Agentforce. Start achieving your ambitious goals. Visit salesforce.com/agentforce Mission.org is a media studio producing content alongside world-class clients. Learn more at mission.org

Give me an example of the sexiest, the most complex, advanced agent that you've ever built. And my answer is, sure, we can do that. But actually you want the most boring, the most repeatable, the most low-hanging fruit agent to start with. I know people who are hiring AI managers, people whose role is just simply to manage the agents. The thing about agentic AI that is just so incredible, it can think for itself. Nobody wants to be a headline for the wrong reason or have a lawsuit. Designing experience between an agent and a human is another flavor of a skill that is very new.

A critical component of that operating model is that business and IT partnership. Turning on tech is the easiest part. It requires constant monitoring, updating, and kind of having a pulse on where it is and where it needs to go. When I talk to my innovation team, I'm constantly blown away. How they take capability of Salesforce AI product and turn it into the solution that elevates customer experience to a new level. I'm like, what?

We can do that? This is not science fiction. Hello, everyone. Welcome to Experts of Experience. I'm your host, Lauren Wood. Today, we are diving into a very, very big topic, agentic AI, and what it means for businesses, for workforces, for leaders, and really humans at large. And we are joined by Irina Gutman, the global leader of AI professional services at Salesforce,

who is here for the second time because so much has happened since we talked to her six months ago. And we're really going to dive into what is happening in AI when it comes to the agentic AI layer, how our business is implementing it, how our organization's adapting, and really how can we powerfully leverage this technology safely?

So we're going to be talking about all of that today. I am so excited for this episode. I've literally been like giddy for it. Irina, thank you so much for coming back. So great to have you. Thank you, Lauren. Super excited to be here and share with you what's happening in the AI world. And as you mentioned, a lot had happened.

So in preparation for this episode, I was actually reflecting and thinking, we didn't talk like that long ago. And when we first started and talked through different types of AI and evolution of AI, I was mentioning that agentic technology is somewhere there on the horizon. But let's just not talk about it yet.

Literally a few months later, not only we're talking about it, it is the primary focus for Salesforce, as well as many tech companies. So very happy to be with you and unpack this very relevant topic. Amazing. I want to make sure that everyone is caught up to what agentic AI is, because it is different from the generative AI and the predictive AI. So can you just really quickly define those different types of AI and where we are today?

Absolutely, yes. Let's get down to the definition and kind of bring everyone up to speed. Let's start with predictive AI, which probably been around for longer than all the other AIs and the definition is literally in the name for all those technology categories. Predictive AI focuses on making predictions based on data, based on rule, and based on the structure that we provided with. A good example in using Salesforce example

would be lead sales lead clarifications. We'll give all the necessary rules, information, and data to Predictive AI using that information, it would categorize and qualify the leads and put it in the priority category for salesperson to focus on. Or case qualification, when again, using a set of rules, AI would qualify and classify various cases

for the service agent to work on. Now moving on to a generative AI. Again, the definition is in the name. It generates dynamic content. It still uses the data, but it is using natural language models to generate dynamic content based on the request or what we call as prompt

or basically what you put in chat GPT because chat GPT is the most common example of generative AI. I want to stop you right there and just make sure because this is the LLMs. Generative AI is LLMs. So it's when we can ask a question and it can pull from many different sources where predictive was really here's the box we want you to operate in, generative can go further than just a simple set of find me, for example, a sales lead that has this revenue in this industry with this number of employees. Okay, great.

Correct. Correct. You're absolutely correct. So now we're going to go one step further and talk about agentic technology, which is the topic of today's conversation. It is the components of everything that we talked about before, but elevates it to the next level. It still has rules. It follows the rules, but it makes a decision how to act based on those rules. It still uses data, but it uses the data to almost reason on what action to take

and how to interact. And yes, it interacts to us based on the instruction we provide it with, but it uses natural language processing and converses with us in the natural language. So to summarize, we still have rules, we still have data, we still have actions and instructions, but this agent uses this information to reason, to make decision, and to converse with us in natural language. One of the examples of an agent, and we will talk further about different type of agent, but the funnest example that I can give is if you've been to Bay Area, they have VAMOS,

which is the self-driverless car. This is an example of autonomous technology that operates without a human, makes decision of how to switch lanes, it takes the information provided to it, instruction provided to it, as well as the data that it takes from the road and makes decision how to switch lanes, how to accelerate, when to stop, et cetera.

I love that you use Waymo as an example, because if you are in a city that has Waymo and you see them driving around, it's this crazy experience. I mean, I'm still like, I can't believe we are here, but we are, they are driverless cars. They are following the rules of the road. And when you think about it,

there are pretty strict guidelines for how to drive, but then there is also this element of intuition that is required of, oh, I feel like that car is coming at me really fast. Maybe I should slow down and let it pass or anything like that. And the thing about agentic AI that is just so incredible, which we're gonna dive into so much more, but the thing about it that is so

incredible is that it can think for itself. It's kind of creepy, but here we are. And there's so much opportunity. We refer to it as a reasoning engine, which kind of sort of an evolution of things. One step further, it also has memory. So it can tap in into the previous data and information and make decisions not only on the data provided at this time, but also from the learnings of data that it has available to it before. So we refer to it as it has memory in addition to reasoning.

Great, we love this. I wanna make sure that we ultra define this for all the folks in customer experience because I do hear chat bots and AI agents being intermingled. And can you just now apply this definition to the difference between a chatbot and an AI agent? Perfect.

So let's keep in mind what we just learned about the agentic AI, right? Components of reasoning, memory, conversation, being able to infer action based on the information that's available to it and compare it to the chatbot. Chatbot follows a very prescribed process flow. It is pre-programmed how to intake information and how to respond based on that prescribed information.

It cannot deviate, it cannot change a course of action or interaction based on that process flow. And it does not understand natural language. For example, let's say we're using a chatbot. And don't take me wrong, chatbots are super effective and super fast for addressing repeatable processes with very few deviation. And in fact, if you have that instance, that it's literally a repeatable processes with very few deviation. And in fact, if you have that instance

that it's a literally a repeatable process that has prescribed process flow with very few deviation, maybe chatbot is an effective technology. However, if we let's say ask a chatbot, where is my order? You have to ask it in a certain way and provide an order number. Otherwise, poor chatbot might be lost.

If you interact with an agent, you say, hey, the thing that I wanted from you last week, what's up with that? An agent literally should be able to interpret the slang, the various variation of the language and come back to you saying, oh, you're looking to find out the status about your order. Let me get that for you.

Would you mind providing an order number? Agents should be able to infer based on the information provided to it, what the next step should be, and what response should be to a human. Agent could pick up on the tone and change the way if I speak in slang, maybe in an interaction or two,

agent will start responding to me in some slang versus a formal language. Chat cannot deviate from that. Every company using agent, they might want that agent to reflect the tone of this company to begin with. More formal, more casual, cool versus some other type of a flow representing company's brand.

You have that flexibility with an agent, chatbot would not be able to do that. I think from a consumer standpoint, I'm just so excited as a consumer because we've all had that experience where we're on with a chat bot and it is not understanding what we are trying to say. We are, you know, maybe there was a spelling mistake and it's just like taking us a totally different direction and it's just downright frustrating. But with an agent, I feel so much more willing and able to actually solve my issue with that agent.

Like before I felt, oh, chatbot, you're just deflecting me because you don't want to talk to me. You don't want to hear what my problem is. And now with an agent, I feel seen and heard and my problem's getting solved faster. So it's really like this incredible balance between an actual human who can be slower at answering these things at times. And you know, the chatbot that just wasn't solving my problem at all. So yeah, I'm really excited about this. So last time you were on we talked a lot about human in the loop. But this is now advancing.

How are you working with agents now? If you can kind of give us an overview of that. Absolutely. And by the way, yes, you are allowed to make spelling mistakes with agents, which is awesome. Great, because I make them all the time. But going back to your question,

we did refer to a previous iteration of generative AI as a human in the loop, meaning that human makes the checks and final decisions. Now you have a technology that almost operates agentically. And we change our phraseology from human in the loop to human plus AI. We now have AI augmenting and expanding human's capability with an assistant of this digital, let's say assistant called agent. Human still has a decision power and we provide agent with instructions

what to do or not to do. But once those instructions are provided, that agent is able to assist us in a way that literally translates to human plus AI versus human is in the loop or in the hell making decisions for AI. So I think the thing about this is it's much more analogous to an actual human or an employee or a customer experience agent.

If we think about, here's the guardrails, here's the types of things we want you to answer, here's how we want you to answer them. Now, instead of saying, yes, answer it, we're saying, okay, you answered this, I'm going to give you feedback so you can do it better next time, because it's really being like trained through those types of interactions of feedback where it's operating on its own. We don't have to say, yes, go do that. It's doing it. But we now need to work with it in helping it to grow. Absolutely. And before we talk about agent us, potentially, when people refer to it as digital labor,

I think it would be helpful to understand main components of an agent. Because that will help us to have conversation about agent being that digital employee. Agents have five components. We're going to start with a role. Just like a human will have a job description, an agent is going to play a specific role.

For example, let's say this is going to be a customer service agent. And the job of this customer service agent is to answer a specific set of customer questions. So we just define the role of an agent. Next step is action. What is it actually gonna do? Well, we wanted to answer FAQ, frequently asked questions,

and we wanted to do certain things based on the instructions that we're gonna do, but that we're gonna give it, and we also don't want it to do certain things based on the instructions that we're going to do, but that we're going to give it. And we also don't want it to do certain things. So actions are based on instructions that provide the agent. And it always includes things that we wanted to do and absolutely things that we don't want it to do. Which brings me to our next component. It is called guardrails. Guardrails are super, super critical. When you're putting autonomous technology in front of a customer, and by the way,

we'll talk about an agent being assistive internal to the company versus customer facing agent. But let's say in our example, we're talking about a customer facing agent that is playing a role of the customer service representative. We define its role, we know it needs to take specific action, but now we're going to give it

guardrails to tell it what it's not allowed to do. For example, it can only answer questions about the order and whether shipping is free or you have to pay for based on some zip code. But if it goes beyond a certain set of questions that agents knows how to answer, it has to hand off to a human.

That's where human plus AI comes in, is that always, always, always, as part of those instructions or card rails, there is an instruction of when to hand off to a human. The next component that we need to discuss is data. It can take any of the actions or follow any instruction unless it has information, the knowledge, to make those decisions

and take those steps on. So data is the first component. And the last component is called channel. How is it interacting with us? Is it living on the customer's website? Is it an employee agent that lives in Slack? Channel is how this agent interacts with a human being or human beings being a customer facing or internal agent. So these five components of an agent really help

us understand of how we're starting to make this transition from a technology to almost digital assistant or digital employee. Thank you for explaining that. It really helps me to start thinking about how do I start onboarding an agent because I just cannot wait. I cannot wait until I have my own AI agent assistant. I'm like, if you know of any tools, please let me know because I'm so ready. We can build one for you. Okay, perfect. I can't wait. I really cannot wait. So you, as leading the professional services department at Salesforce,

you are helping organizations to implement these AI agents. And I'd love to spend some time talking about how do you really approach that? If you can walk us through a little bit of the practices or the frameworks or the structures that you utilize to help leaders and teams think about where are we bringing these agents in and how are we training them with everything. An AI agent your customers actually enjoy talking to? Salesforce has you covered. Meet Agent Force Service Agent, the AI agent that can resolve cases in conversational language anytime on any channel. To learn more, visit salesforce.com slash agent force you just shared. Absolutely. And when I meet with customers, sometimes customer would tell me,

give me an example of the sexiest, the most complex, advanced agent that you've ever built. And my answer is, sure, we can do that. But actually, you want the most boring, the most repeatable, the most low-hanging fruit agent to start with. Because what you want to start with, with autonomous technology in the area where you want to augment humans. We want to leave creativity to humans. We want humans to address more complex scenario. And we want to augment human's capacity in the areas when there is repeatable, there is low risk. And those are the questions that human may not want to answer. Or if we're

looking at the assistive agent, the agent that lives inside the company and helps its employees, the same the same logic applies, you would want it to answer the question and help with those frequently asked problems or frequently occurring problems that humans are facing. I spoke with the client recently from a regulated industry from financial services and this particular customer was from the bank and she said, well, you know how hard it is to implement agentic technology in the bank. And I said, absolutely.

But if I ask you to make a list of questions that your customers call and ask that always, always, always end up in pretty much the same answer with very minimum variability, what products does this bank offer? How do I open account? I bet you'll be able to provide me a list mile long. She said, absolutely. So that's the list that we give to the agent. Yeah.

So we start with identifying the, it's going back as to any other project, identifying the business objectives, identifying pain points, and saying, can this technology help address those pain points? We also want to start small, incremental, and low risk. In addition to that, we need to identify data that will support that type of a use case. But, and it's becoming more and more relevant with AI and agentic technology, turning on tech is the easiest part. The hardest part to tackle, and we talked about it last time,

and guess what did not change change is organizational readiness. Yeah. I just had an executive actually awesome event. It's called Empowering Women in AI and we had a room full of women executive from all over Tri-State area and I am local to New Jersey. I wanted to say New York but local to New Jersey but we did have an event in New York. And what we talked about is exactly that. How do we get started? What are some of the blockers within organization? And we had absolutely incredible discussion that was focused on organizational readiness,

on the new operating model, way more than about the tech. If I could boil it down to the successful approach, I think we'll talk about five phases. So first, just general readiness assessment. Kind of checking off all the points, do we have minimum required criteria? Check, check, check. Just enough data, identify use cases,

enough people that support this initiative and wouldn't run away. So that is checked. Second, incremental implementation and unlocking of the capability. You don't want it to be a year-long transformation program where you don't see any outcome until we implement all complex agents in the world. Three, foundational technology. We need to have data and integration capabilities in place in order to unlock agenda capabilities and broader AI capabilities. And that we can also

do as an iterative approach, identifying just enough minimum criteria from a data and integration perspective, but also building out that holistic roadmap for the end state. Tell me a little bit more about that foundational tech. What are some of the key components that are needed there? From a Salesforce infrastructure perspective, and I'm going to talk about Salesforce, we have two key elements. We have data cloud, which is critical for agent force, which is the technology component

that Salesforce refer to. That's our version of agentic technology. We call it agent force. Data cloud, which allows for data syncing and consular synchronization across various sources. And MuleSoft is our integration layer that allows to connect any data from anywhere. Combination of data cloud and MuleSoft

unlocks any data needs, any communication needs, being a structured data, unstructured data within Salesforce or outside of Salesforce. Okay, great. So it's really having the technology to store and process the data that is then going to be used by AI, by your agents. Okay, great. Thank you for answering that. What's step four? Absolutely. Absolutely. Next step is what we talked about, organizational readiness. Do we need to build that new operating model? What are the new roles? And by the way, new roles

are required. We're now talking about agent owners and agent monitors. Some people go as far as saying, do we need agent HR? So I don't know how many agents who has, but with one, you probably don't need an HR department for, but. Totally, I know people who are hiring AI managers, people whose role is just simply to manage the agents, which is a, it's a management position, but it's very different. Yeah, exactly.

And there are variation of those new roles. Those new roles need to fit with an organizational operating model. Standing up center of excellence that help manage that, building out new operating model, defining that process of introducing new roles, enabling new roles, upskilling people is the next absolutely critical component. We talked about it before. If some of those things were considered optional

with more traditional technology, AI and especially agentic technology truly changed it and made it, as we said, as foundational as problem as data. Because if your organization is not ready to take responsibility for this technology, no one's gonna use it. Change management is even more and more critical.

Yeah. Last piece is the roadmap. We don't wanna stop with one agent. Understanding what that ultimate north star and how organizations are gonna progress in maturity of introducing AI agents and data is the last piece of the puzzle that I would like to introduce.

Awesome. I want to go deeper into the new operating model. I think a lot of leaders are thinking about, and if they're not, they probably should be thinking about what is my organization going to look like? Like you said, what are the roles, the skills, the governance that we need to implement? And I'm curious to know a little bit of, you know, any examples you've seen of this being done really well. And I know it's a lot of this is future thinking. We don't even fully know what it's going to look like in six months from now.

But yeah, what have you seen? You're right. Absolutely. This is very new and future thinking and a lot of my more mature customers like, do you have an example? And I'm kind of say, well, you're the first one, so let's figure out together. Totally. But more and more, we have instances when some of the leading customers are being trailblazers

in this area. But let's talk about some foundational components of that operating model. It is absolutely critical. So when we're talking about bringing in new operating model, it is probably the easiest way to think in the form of standing up some kind of body, meaning a center of excellence, or practice focusing on that. In fact, Forbes Magazine says that the best way to handle emerging technology is to have some kind of a center of excellence around it. There are debates, oh, center of excellence can be obsolete.

There that ivory tower. Not if you decide it right. If you design it right, there's a way to make center of excellence really foundational and the way to help control and also drive that innovation. Let's discuss some of the key elements. We need to align on strategy and objectives of what it is. We need to get visible and actionable leadership support,

understanding that it's not an extracurricular activity, that it's actually critical for the organization and everyone is on board. What's that common vision that we're working towards and what's going to be the charter of this particular center of excellence? For short, I'm going to refer to it as COE. And let's assume it's an agentic or AI-focused COE. Do you have any examples of what that vision might be?

The vision might be to have agent-first mindset and to prepare organizations to prepare organization to intake agentic technology and to elevate, I don't know, our customer service and to the next level with the help of agents. Granted, I made it up on the fly. So no, all good, all good. I just wanted to understand because it can be so broad and I, but I also, the vision is so important because it at least starts to give us a little bit more focus in what is the, what is this body of work about? It's the headline. We need to set that headline to kind of guide everyone towards something.

What's the headline, but also what is company trying to achieve? What is the company trying to get to? And then we can articulate how's that particular body within organizations is gonna help company to get there. That's where it comes, vision is supported by strategy, by objectives, right? Then we need to define this center of excellence itself. That would entail operating model, meaning how is it gonna be structured? Is there going to

be an overarching leadership committee? How is business and IT are going to working together in that operating model? We talked about it during AI. It is even more critical with agentic technology. IT alone, unfortunately, so sorry everyone who works for IT, but you can make that decision alone. It has to be in super strong partnership with the business because guess what? AI is augmenting most likely a business employee. Therefore, IT becomes innovator, capability provider, but business needs to be in the room saying which capabilities they need,

which business problems need to be addressing. So critical component of that operating model is that business and IT partnership. And then there are kind of discipline going across such as governance, risk mitigation and responsible AI usage. Another critical element that I would say is a fun element is innovation.

You can have, I wouldn't recommend having a genetic COE without explicit focus on innovation to drive that culture of innovation, but also to drive the culture of experimentation, to give a specific focus area where people can freely try new things, try new technology, fail quickly, improve upon, and adjust it per that company's need. Another important area is education. We just talked about all the new skills that we will need to define that will go into that operating model.

You said agent manager, I said agent owner, some people say agent supervisor. I've heard roles that is a content creator and we'll talk about that why it is important. Prom builders, etc., etc. There are roles ranging from the ones that you need to manage agent to the roles that you need to have to build agents, such as I said, content creators or from building, because guess what? Articles that we human consume need to be written differently for consumption of AI. AI consumes information differently from humans. All of those roles need to be up skilled, trained, and then continuously trained as new innovation or new capabilities come about.

And with AI, it is faster than we've ever seen before. So that continuous upskilling needs to be a pretty robust process. And that is why we need to have explicit focus on it as well. Where can leaders go to upskill their teams? Because like you said, it's happening so fast. And if we try to build our own systems of education, I assume we're going to be a little bit slow. Is there any resources or things that you would suggest people go to look at or learn from? No, this is definitely a hard one. And I can tell you where Salesforce

currently addressing that situation as well. As you know, my team is explicitly focused on AI, and we're working very closely with product organization to have us to stay ahead of the curve. But Salesforce professional services, 10,000 people, Salesforce partner ecosystem is multiplied that by a hundred. All of those experts need to stay up to date on the latest capability and technology. Yes, sure. There are a million classes in very respectable institution that will give you foundation and to give you a core knowledge,

but then it goes to the next level. So there are multiple up-skilling, right? Understanding the technology, understanding the essence of this particular, what agents are, how they constructed, how you do prompt engineering, and that skills could be gained from any respectable known institution. There are classes, there are courses there,

but then there's a next level of specific to that product. And that I would suggest getting from partnering, being at a Salesforce, or if you're working with a different technology provider, working with that particular company to know the nuances of that product. And that's what we're doing. We're working closely with product. We're putting new programs in place that combine propagation of knowledge and propagation of

experience across entire professional services. And we came to the realization that there's no longer an option to spend three months putting a training program together. Materials and information needs to be available very fast and we need to build into those enablement program constant updating and changes because sometimes the answer I provide you today might be obsolete in a week or two. It's happening so fast. I mean, the good news is that AI can help us put those training programs together really quickly, which is great. But what's coming to mind for me right now is the

fact that, you know, the vendors, for example, Salesforce, you now have another role of education that goes beyond simply this is how you use our tool, but this is how you actually bring your tool into your organization, which is exactly what you do. And I think it's something I tell my clients all the time that if you want to start using AI, first look at the vendors you're already utilizing and go and see what technology they have rolled out and then ask them to help you learn how to bring it inside. And I just, I think that there's this new role almost for SaaS companies to not only

say here's technology, but here's actually how you can utilize it in the way that's going to be most effective. And it goes beyond just simply like click these buttons, but actually like, you know, here's the roles and the education your team needs and all of that, because that's really essential to using AI. No, you're absolutely right. And that's why it folds all into those component of the organizational readiness workstream, because this is exactly what it will focus on. In addition of what this tool and technology is,

is how it's actually gonna be used and how companies are gonna assume responsibility for it and continuously work with it because AI is ever evolving, it's not once and done. And that's what makes it unique and super uncomfortable for a lot of us. Because if you build a system, unless somebody goes and changes the code, it's pretty much going to be

operate as expected. With AI, with its learning capability, with its reasoning capability, it is going to evolve. The data that it's grounded on is going to change. So there's a lot of variability in this technology, and therefore it requires constant monitoring, updating, and having a pulse on where it is and where it needs to go. Yeah. The other thing I'm really curious about,

and I'd love to even hear this from a personal perspective from you, is what are the skills we need to be learning as we look towards the new AI age? Like what are the human, what's the human advantage that we can really like focus on? And I'm curious even just from a personal perspective, what have you been learning and trying to upskill yourself on? What's foundational to all AI is how we interact with it. Because AI is grounded or is based on natural language models or natural language processing. Yes, when you are a user of AI, it will respond to you.

And as I said, use example of broken English or slang, but in order for it to do it, somebody had to tell it to do that. And that's where we get to prompt engineering. Prompt engineering is absolutely foundational skillset to have. And when your agent is not actually answering the questions the way you want to, one of the first things we look at

is how the prompt is constructed. Sentence structures makes a different. Punctuation makes a different. Word choice makes a different. Like you can be free with the words when you interact with the finished product, but when we construct an agent, all those things are critical. Prompt engineering is a new skill with a new requirement. I also mentioned that AI consumes information differently. When we refer to knowledge articles, like, oh yeah,, we have million knowledge articles, sometimes we see that

those knowledge articles need to be restructured for AI consumption. So there are roles, there are skills to restructure unstructured data, I know it's a mouthful for AI consumption. So AI interprets that and for me, making it a mouthful for AI consumption. So AI interprets that information, making it kind of easier for AI to interpret this information, that it could provide a better answer. And another fun one, and you mentioned in the beginning that sometimes you want to scream at the bot and say,

what are you doing to me? Are you trying to torture or deflect me? That talks to customer experience. And we want agents to provide amazing customer experience. Experience designer is a known role. Now we want them to elevate their skills to the next level, almost like conversation designers. Designing experience between an agent and a human is another flavor of a skill that is very new. And that is from a person who is responsible for the team developing the agent. Those are some of

the technical skills. I don't need to be an expert in those skills, but I do need to have foundational understanding. However, what I was focusing on is what we were talking about is understanding how that organizational ecosystem needs to change. How do we approach change management differently? What are all those nuances of things that we've done before from standing up those organizational operating model from defining all those departments and new roles, from integrating all of that new stuff into organization, what needs to be nuanced for agentic technology and how is that different from more of a traditional change management? A lot of it is the same, but still this nuance is important. Yeah, we're dealing with a very different technology

than other technology rollouts. And it's also the nature of how it changes needs to be built into how we're operating. And the thing I also wonder is like, now that learning how to use AI, learning how to design our organizations around AI, everything we've been talking about is table stakes. But then I think about how are organizations creating a competitive advantage?

And the thing I think about, and I love your thoughts on this, is that it's really like in the connections and the way that we're listening to what it is that the customer really wants and needs. And it's almost like now that AI is dealing, handling a lot of these tasks that we've typically been bogged down with, what's next? I love your thoughts on that. Like now what can we focus our attention on?

Well, we still need to focus our attention on what's happening now. And what's next? It's really, actually, it's funny you asked because it really depends on people's personality and readiness for change and readiness to embrace this technology. When you're talking about, okay, we're good with AI, we're good with agent again, arena, what's next? Some of my customers are like, really? I can turn an email generation and no one's going to come after me. So we have a full spectrum of maturity when it comes to AI. But I think what's next and very near or already here is the agent, multi-agent.

That is the most logical next evolution is that most of the companies even now have multiple agents. When we talk about agentic technology, the natural progression for a lot of companies is internal assistive agent or multiple internal assistive agent, meaning they're not customer facing. They're still autonomous, but they are within the company and they're interacting with employee and they're there to help employees do their job. Then you have customer facing agent

and then you have agent to agent collaboration. When you have what we call the multi-agentic network and then you have agent to agent collaboration. When you have what we call the multi-agentic network, and then you have agents. And it doesn't mean from the same company, it could be Microsoft agent talking to a Salesforce agent. So that is broad spectrum of multi-agentic collaboration, I think is the next wave. And I can't tell you what's gonna happen after it.

It's maybe way too fast. That went for me. I mean, I'm just, I just can't wait until I have my own agent who's been doing all my customer service chats for me being like, Oh, I need to order a new dress size or can you, this thing showed up broken. Can you send me another one? So, so actually it has an agent like that. This thing showed up broken. Can you send me another one? So Saks actually has an agent like that.

Her name is Sophie and she handles this type of questions for Saks. They were one of our first agent customers. And the demo that was shared at Agent Forest, sorry, at Dream Forest last year, all the forces, the demo that was shared at Dream Forest last year, all the forces. Dreamforce. The demo that was shared at Dreamforce last year blew my freaking mind. It's on YouTube. If you want to find it, the sex example, it was great. And I'm really excited for that.

My question for you is, what are you the most excited about? You're in such a unique place where you can see what is happening in this space. What are you the most excited about? So I'm going to use an example of a session, as I said, that we conducted this week for Women in AI. And we started the session with the exercise. We asked participants to go to a work cloud space and put one word that comes to mind when they think about agentic technology and digital labor. And the responses that came back were mixed.

Some were optimistic, word opportunity popped up, but some of the words were job replacement, scary, new. Then we conducted this discussion when we talked about how companies get started. What is the incremental and iterative approach? What are some of the foundational elements that you need to put in place? How we partner with the customers

to take it through the journey. And after only two hours, we did this exercise again. Oh my God, such positive word. I did not see a single negative word on the screen. People were excited. Opportunity still was the hardest word, highest word, but people were excited. They said, they use words as innovation, new roles, new opportunities. So very, very forward looking in just two hours.

And that is what I'm most excited about is the chance to partner with my customers to help them understand and embrace this technology and to help them start taking advantage of this technology. Because if it's done right, and I love your attitude, I love that my company can build agents for you to help you with all of that. But those that are a bit more of a skeptics or have a more of like an enterprise or industry where it's harder to introduce to help them handle those objectives and show them how this technology could be really helpful is very rewarding. The other thing that I'm excited about is when I talk to my innovation team, I'm constantly

blown away. I'm like, what? We can do that? This is not science fiction. You actually build it. And it is amazing how they take capability of Salesforce AI product and turn it into the solution that elevates customer experience to a completely new level. I mean, the opportunity, like you said, that word came up a lot.

The opportunity at Play is immense, but it is almost so big that it is scary. I mean, it's a lot of change, and it's a lot of change happening really quickly. And I could not agree with you more that there's the skept know, the skeptics in the room. We should all be skeptical to some extent. I think it's healthy to have a little bit of that. But, but, but, but this is changing whether we like it or not. So let's jump on.

Let's jump on the bandwagon. Actually, now that you bring up responsible AI, since we do have a couple minutes left, I do want to talk just a touch on that a little bit around what does it mean to be responsible with AI? Because it's a little bit feeling like the Wild West out there. And how can organizations approach this in a responsible way? So, you know, they don't have a lawsuit on their door next week. Oh, that's no thank you for bringing it up. It is a very, very important topic. When we're designing agents or we have a dedicated and explicit exercise called guardrails and risk management definition. What we're advising our customers to do as they define the role of the agent and

what job it needs to do, we asked it to define risk in categories that very familiar to them. People, process, data, and technology. So we work with the customer to list all the possible risks, all the possible violation that this agent could cause in those four categories. And then we take each of those items and we say, okay, how are we going to mitigate it? Are we going to mitigate it with people? Are we going to mitigate it with process, with data and technology. Yes, the risk is inherent. There's some variability, vulnerability inherited in AI but it does, as you said, it's not an option.

This technology is here to stay. So what we're doing is developing framework, just the one that I explained to you of how companies can go systematically from defining an agent to defining risks, to defining risk mitigations and then turning them into what I referred to earlier in our conversation as guardrails, which is instructions that we give to the agent what it's not allowed to do.

The other thing that we do to make sure we do stress testing for bias and toxicity. Salesforce already has a trust layer that is very, very robust, and pretty much nothing can get in past us. Some nothing bad from the outside. However, we put agent through additional stress of bias and toxicity testing. When we overload the system, we talk to the agent in bad language, profanities, using racist slurs.

On purpose, it's not a pretty testing, but it's absolutely necessary because as you said, nobody wants to be a headline for the wrong reason or have a lawsuit. So that type of testing is absolutely critical. And my responsible AI team developed a methodology of how to do that kind of testing. And we set thresholds really low, like 1% or less than 1%. And I'm proud to say that our agents have.

Amazing. I just for anyone who's thinking about utilizing, which we all should be, how are we utilizing agent technology, agentic AI, we have to have those conversations. What are the risks at play? Just put them all out on the table. Let's make sure we're thinking about the edge cases because they are there.

And when we are not hand in hand with our AI the whole way through, when the AI is operating independently, we need to make sure that we've thought about all the risks at play. So, thank you so much for sharing that, Irina. As per usual, we always like to ask this question to end our show. I'd love to hear about a recent experience that you had with a brand that left you impressed. Why was it amazing? So I thought about that.

I think I'm going to go with Starbucks. I know it's simple and I know everyone's familiar with it, but you know what? When I'm getting off New Jersey Transit headed to my way to the office, I click on my Starbucks app. It allows me to set my favorite stores. There are a few. I have one set up for San Francisco because Salesforce headquarters are in San Francisco. There is a Starbucks across the street from our office.

It has my different locations. It has my favorite stores per location. I click, it remembers what I've ordered before. It displays my frequent orders. I usually go with Caramel Macchiato, so it comes up. I click and when I get to the store, I can see on Tableau saying, hey, Irina's order moved from received to in progress. I know next is gonna be on ready. It is simple, it is simple to use, but it has just enough key elements to make my life easy. And also from a customer service perspective,

I went to the store that happened to be backed up. I had an urgent meeting. I came to the assistant and I said, I'm so sorry, but this is the situation. Is there any way you can help me? She looked and said, in fact, you are next in queue. Let me move you up. So they start processing your order. So it's not only visible to me as a customer, there is an option for an assistant to see where I am to do the overrides and to address my needs. And this is how human plus technology working together provides amazing experience.

I love this example. Speed, efficiency, thinking about what you need. You're in the morning getting off the train. You don't want to stand in line. And the key thing that you shared is that it's also enabling the staff to support you, to jump in and intercept like a team. Absolutely. So thank you so much, Irina. This has been such a wonderful episode. I'm sure I'll see you in a couple of months. I would love that. As things keep changing.

And we'll have something new to talk about, I'm sure. I'm sure we will. Well, I wish you a wonderful day and we'll talk soon. Thank you so much. Thank you for having me.