The AI Daily Brief: Artificial Intelligence News and Analysis - The Time Savings Era of AI Is Over

Starting point is 00:00:00 Today we are looking at the results of the AIDB Intel January AI usage pulse survey, and the results are very clear. Everyone is a vibe coder now, and the time savings era of AI is over. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, Insightwise, Super Intelligent, and Blitzy. To get an ad-free version of the show, go to Patreon.com. If you are interested in sponsoring the show, send us a note at sponsors at AIDailybrief.aI.i. And while you are on AIDilybrief.aI, click on that little number six

Starting point is 00:00:44 to check out AIDB Intel. This is the research information and benchmarking platform we are cooking up, to which this new survey that we are exploring the results of today is, of course, connected. Now, one final note before we dive in. With my luck, yesterday Sam Altman turned into a 50-foot robot and became president of the universe thanks to Chatchip-T-6, leaving you to wonder why the heck I'm talking about some survey results when that's going on. Well, as I mentioned earlier in the week, I'm actually in the midst of a very long travel couple of days down to South America with the fam, and so I preloaded a couple of episodes. If you're not, once I'm down there, we will be on our completely normal schedule, but for these couple of days, we had to do things

Starting point is 00:01:21 in advance. Luckily, we were due to talk about the January AI usage pulse survey. This is basically a look at how people used AI in January, where they saw the most value, and how that was changing over time. Now, before we get into the findings, let's talk about the sample. 583 people responded. Obviously, it was all listeners from this show. It was the only place that I was pushing this. I didn't even post about it on social media, so a highly concentrated AIDB audience, which means, of course, that this should not be taken as a representative sample of AI users in general. You got to think that this represents an extremely active, enfranchised subset of AI users. And so I think the best way to think about these survey results

Starting point is 00:02:02 is less about where AI is and more a way to skate where the puck is going. I think the folks who responded to this are out on the vanguard of trends, which will come to all sorts of other types of users a few months down the line. Just to give you some specific numbers around this, 97.6% of the respondents use AI daily, with 43% spending more than 10 hours per week using AI for work. There is a pretty wide distribution of roles represented in this survey, and honestly a pretty decent spread of company sizes as well.

Starting point is 00:02:30 Now, as always, whenever we do surveys, there's going to be some concentration among small companies and solopreneurs, because that's such a big base of AI users right now, people who are venturing out onto their own, crafting new things, figuring out how to make the most of these tools either as individuals or with very small, high-impact teams. 38% of the respondents were in that category of small companies with between zero and 50 people. However, there's a pretty good distribution across the rest of the company sizes as well, with 27% of the respondents coming from large enterprises that have 5,000 or more employees. There is a lot that we're going to get into, but here are five quick critical insights.

Starting point is 00:03:06 Firstly, although chat GPT has broader reach, with 87% of people saying they used chat GPT last month compared to 80% who said they used Claude, Claude is the number one primary model, chosen as primary by 45.8% of respondents. As we will discover, Claude primary users are heavier users, more agentic, and report greater value gains. Next, both usage of and value from AI are increasing. 71% of respondents increase their AI usage month over month, and 83% say that their value increased. Vibe coding has absolutely gone mainstream.

Starting point is 00:03:41 69% used vibe coding tools, and most of them come from outside engineering and IT. Fourth, as we have seen in our broader discussions, we also see in these numbers that some sort of agentic threshold has been crossed, with more than a third, 37.6% report reporting agentic AI use. Finally, we are seeing a shift in benefits. In this survey, time savings was not the number one benefit, and that represents a big change from what we've seen in the past. In fact, when we did our AI-R-OI benchmarking survey at the end of last year,

Starting point is 00:04:13 time savings was absolutely the universal entry point. 76.7% of the survey participants cited time savings as one of the primary benefits that they got from AI. It was the dominant benefit across every single industry role in company size. It was, as we put it then, the low-hanging fruit of the AI era. It was nearly double the next highest level of benefit in terms of its prevalence among these use cases. Interestingly, though, we also found that it was not the most valuable benefit. We actually saw an inverse correlation where respondents who only focused on time savings reported lower overall.

Starting point is 00:04:47 Conversely, the people who deployed use cases that had strategic benefits like improved decision-making, new capabilities, and increased revenue reported significantly higher ROI scores, meaning that the shift away from time savings is something that could be pretty exciting. But with the key findings out of the way, let's start to get into some of the big areas of exploration. Let's talk about the model landscape first. One thing to note is that this set of respondents are very polyamorous when it comes to their models. The average person, in fact, reported using 3.5 models. Only 5% of respondents used a single model.

Starting point is 00:05:20 This is one of the areas that I think might be most out of sync with the average enterprise user who's going to either use A, only what their enterprise gives them access to, or B, that plus whatever model they use at home. Overall, each of Claude, Chatsybt, T, and Gemini saw pretty significant breadth of usage. Clot and Gemini both had 80% of people who had used them in the last month, with Chachybtee, as we said before, seeing the broadest usage at 87%. When it came to which model people used most, Claude spanked it, like I said, at about 46%. Chatshapit was the primary model for 31%, and Gemini was the primary model for 16%. Certainly this suggests that at least among the bleeding edge, the reports of Gemini catching up to chat GPT are at least for now, perhaps a little

Starting point is 00:06:01 overstated. Now, in terms of models that aren't showing up on the summary chart, 39% of respondents said that they had used copilot in the previous month, and 23% said that they had used GROC. In terms of most used model, copilot only had 4% and GROC only had 1%. Now, let's dig into the profile of the Claude Power user. When we compare the profiles of the Claude Primary versus Chat ChbT primary users, a fairly dramatic divergence emerges. First of all, Claude primary users are just heavier users. 53% of them use AI 10 hours or more a week, as opposed to 40% of chat GPT users. They are dramatically more agentic. 52% of Claude Primary users report agentic AI usage, as opposed to just under a quarter 24% for ChatGPT.

Starting point is 00:06:48 87% of Claude users report vibe coding, which makes sense because of Claude code, as opposed to 52% of chat GPT users. And while primary users of both chat GPT and Claude, both report big increases in the value that they saw from AI over the last month, Claude was higher at 88% compared to 74% of chat GPT primary users. The top benefits also look different. In each case, it was increasing output, but that was the top benefit for 48% of Claude users as opposed to 31% of chatchbt users, Number two for Cloud primary users was new capabilities, whereas number two for ChatGPT users was time savings. Probably unsurprising based on broad perception, but Clod has very clearly captured

Starting point is 00:07:28 the builder practitioner segment, the people who are deepest into AI augmented workflows and likely pushing the frontier of what's possible. Now, one quick note about Gemini. Although it only has 16% primary usage, its 80% overall usage, certainly suggests that its workspace integration makes it kind of ubiquitous as a secondary tool. I think if you are Google, there's a lot to build on there, even among this group of highly enfranchised users. Next up, let's talk about momentum.

Starting point is 00:07:55 This is one of the stats that I'm most excited to see how it changes as we move to a more monthly cadence for this, and I wondered to what extent this is a first time responding sort of bump here, but users in general were using AI more and getting more value out of it in January as opposed to December. Like I said, at the top 71% saw increased usage and 83% saw their value increase, leading to what I call a value premium, which is a 12-point gap that's evidence that people are not just doing more with AI, but also getting better at it. Another piece of evidence that supports that thesis is that of the folks who had flat usage, i.e. weren't using AI anymore in January than they did in

Starting point is 00:08:29 December, 63% still reported increasing value. That is some combination of skills development, model improvement, and the learning curve paying off. Sure, there's hype about AI, but KPMG is turning AI potential into business value. They've embedded AI and agents across their entire enterprise to boost efficiency, improve quality, and create better experiences for clients and employees. KPMG has done it themselves. Now they can help you do the same. Discover how their journey can accelerate yours at www.kpmg.us slash agents. That's www.kpmg.org.coms slash agents. As a consultant, responding to proposals can often feel like playing tennis against a wall. You're serving against yourself trying to guess what the client really wants.

Starting point is 00:09:20 That all changes with the Insight Wise Proposals Platform. Now you've got an AI coach that thinks just like your client. It returns to the brief time and time again, identifying opportunities, showcasing your track record, and making recommendations to improve your pitch. Suddenly you're on center court, but this time you've got a secret weapon. Insight Wise does away with all the time-consuming manual work, so you can focus on winning more business more often. Generate reports, pull insights from your own data,

Starting point is 00:09:44 build competitive advantage and go to sleep before 2 a.m. When it comes to proposals, you only get one shot. With Insightwise, make yours an ace. Go to insightwise.a.i to start your free trial today. Today's episode is brought to you by Super Intelligent. Super Intelligent is a platform that very simply put is all about helping your company figure out how to use AI better. We deploy voice agents to interview people across your company, combine that with proprietary intelligence about what's working for other companies, and give you a set of recommendations around use cases, change management initiatives that add up to an AI roadmap that can help you get value out of AI for your company. But now we want to empower the folks inside your team who are responsible for that transformation

Starting point is 00:10:23 with an even more direct platform. Our forthcoming AI Strategy Compass tool is ready to start to be tested. This is a power tool for anyone who is responsible for AI adoption or AI transformation inside their companies. It's going to allow you to do a lot of the things that we do at Super Intelligent, but in a much more automated, self-managed way and with a totally different cost structure. If you are interested in checking it out, go to AIDailybrief.ai slash compass, fill out the form and we will be in touch soon. Blitzy is driving over 5x engineering velocity for large-scale enterprises. A publicly traded insurance provider leveraged Blitzie to build a bespoke payments processing application, an estimated 13-month project, and with Blitzy, the application was completed

Starting point is 00:11:01 in live in production in six weeks. A publicly traded vertical SaaS provider used Blitzy to extract services from a 500,000 line monolith, without disrupting production, 21 times faster than than their pre-Blitzy estimates. These aren't experiments. This is how the world's most innovative enterprises are shipping software in 26. You can hear directly about Blitzy from other Fortune 500 CTOs on the modern CTO or CIO classified podcasts. To learn more about how Blitsey can impact your SDLC, book a meeting with an AI Solutions consultant at blitzy.com. That's BLYTZY.com. Now, let's talk about the benefits beyond time savings. As I mentioned before, this was definitively the top use case in our AI ROI survey last year. However, in this survey, it was down at number three.

Starting point is 00:11:50 Increased output and throughput was the most commonly reported primary benefit at 38%. New capabilities was number two at 22%, and time savings was just behind that at 20%. Improved decision making was next at 11% and improved quality of output was after that at 8%. Now, there's also some interesting differences in terms of how the benefits associate with different patterns of usage. It is absolutely the case that the deeper people go with AI, the less time-saving captures the real benefits they're seeing. Among those who reported using AI for more than 10 hours a week, just 10% said that time savings were their primary benefit, as opposed to 49% who said output and 27% who said new capabilities. And unsurprisingly, among the people who cited new capabilities as

Starting point is 00:12:32 their benefits, the things that they referred to tended to be around coding an agentic use. Speaking of agents, let's talk about the agentic threshold. We ask response. to categorize AI usage from the previous month into one of three categories. Assisted, AI helps me do something better or faster. Automated AI handles a task end-to-end, or agentic, AI figures out steps and executes. People could of course select more than one. Unsurprisingly, assisted was the top with 84% of respondents, having used assisted AI. Automated was at 40%. And Agenic, it jumped all the way up to 37.6%. Now, it's not an exact comparison, but once again, looking at what we saw among the 5,000-plus use cases that were contributed to the AI-R-O-I survey,

Starting point is 00:13:16 the data for which came in last November, 57% of those use cases were assisted, 30% were automated, and 14% were agentic. Unsurprisingly, the folks who reported agentic use tended to be some of the heaviest users in general. 57% of those who said that they had deployed agentic AI last month were in the heavy user 10-plus hours a week category. They also used more models than average at 3.8, and their top benefit overall was new capabilities. People who identified as leaders either in the C-suite or VPs and directors also had higher-agentic usage than other categories. Fifty-seven percent who reported being in the C-suite said that they had done something agentic, as well as 32 percent of VPs and directors.

Starting point is 00:13:54 Next big takeaway is that vibe coding is absolutely no longer just for engineers. And once again, this is another question that shows that this is a very forward-looking group in terms of how they're using AI, but coding was full-stop, the number one use case in this survey. It was the most common at 36% and had the highest cited value at 38%. What's maybe even more interesting is that 49.5% of people who reported coding work outside of engineering and IT. 34% of people in executive and leadership roles were coding, 13% of people in product roles, 11% of people in operations roles, and 8% of people in sales roles.

Starting point is 00:14:33 69% said that they had vibe coded this past month, with another 21% saying that they hadn't tried, but were interested. Now, me personally, I think we are going to see these patterns come to the rest of the AI using market much faster than one might think. While yes, I think this is very out of sync with what the average AI user and the average enterprise is doing, give it six to 12 months, and let's talk again. Now, in terms of what's holding people back, while a lot of these users, especially the solopreneurs among them, were empowered innovators where nothing was holding them back, lots of folks did have issues that impacted just how much they were getting from AI. The number one issue not unexpected at all based on my conversations with so many different people was simply not

Starting point is 00:15:12 having enough time to learn. I've been haranguing everyone recently on this show to go fire up a quad project, create a build partner slash coach and start hacking it open claw, but it is undeniable that that takes hours and hours and hours that many people don't have. There's also a skills gap with 18% saying that they felt like they didn't know how to use AI effectively. I think the demand for training resources is enormous and just going to grow. 17% said that they have policy and approval barriers. There were also 10% who said that they didn't have access to the right tools, with 8% saying they didn't know what use cases to deploy. Now, one interesting thing I found was in and around folks who reported that their organization's stance around AI was restrictive.

Starting point is 00:15:52 They had similar patterns of agentic use, similar patterns of vibe coding use. The biggest difference was that they spent less time doing it. Among people who were at organizations that encouraged AI use, 47% said that they spent 10 hours or more per week using it, whereas among those who were in organizations whose stance was restrictive, only 29% were in that 10 plus hour bracket. And this is really what shows the very real cost to people for being in a restrictive AI organization. They don't have as much time as their peers in other organizations to get up to speed and fully take advantage of these tools. We had one open slot for people to answer the question, what's one thing AI helped you do this month that surprised you? The responses painted

Starting point is 00:16:33 a pretty vivid picture of a workforce in transformation. A lot of a lot of the A lot of the discussion was, of course, non-coders who had become builders. One person said, Claude Code transform me from a non-coder to developer within a week. I've now created websites, dashboards, web apps, and Python code that perform specific tasks in my regular workflow. People are getting more agentic in their work. Someone said, I've created a workflow where I set up my tasks on Asana and Cloud Co-work completes them for me.

Starting point is 00:16:58 Effectively, I'm delegating tasks in a structured way to a general agent. There are also, alongside all of the work-use cases, a lot of surprising and cool personal applications. One person said that AI helped them design a hydraulic water pump system replacement. One cyclist said that it had been great at helping them improve their time at threshold for cycling. One person used it to help set up a generator that was not well documented. One person even used it to set up a grocery optimizer, both for time and cost. So what are some key takeaways? First of all, like I said at the top, the time savings era of AI is very quickly giving way to some higher order benefits. We are no longer just talking about

Starting point is 00:17:35 doing the same things faster. Instead, we're talking increasingly about I produce more and I can do things that I couldn't do before. This has implications, of course, for how organizations should measure AI ROI. I don't think it means they should be only about that as fundamentally limiting. Next, it is very clear that the new capability to write software to solve problems in your work is in and of itself redrawing job roles. We talked about this a couple of days ago in terms of that Berkeley-Hoss study where they found similar things by embedding in a tech organization, Anthropic also called this out in their coding trends. But the implications are immense.

Starting point is 00:18:10 The org chart is up for grabs. There are implications for hiring because what skills even matter now. Organizational design is going to change. Procurement and internal tool selection is changing. And then of course there's the issue of training where the comfortable lines around what people are supposed to learn and know have just been completely obliterated. This is one of the most exciting aspects of how AI is changing and the inflection point we're living through. But also when it comes to the enterprise itself, one of the most.

Starting point is 00:18:35 challenging as well. Number three, I feel very comfortable arguing that the evidence here suggests that agentic adoption will accelerate. Not only are we seeing more than a third of people report agentic usage, the fact that folks in leadership positions are leading adoption suggests that organizational permission structures and just organizational compunction to go get agentic will follow. Now, I think that there are big implications there for AI strategy. Infrastructure, tooling, governance frameworks, data access, all of this should be a

Starting point is 00:19:05 priority for this year and next year's AI strategy if it's not already. The last takeaway is an interesting one. It's very clear that multi-model usage is the norm. Like I said, the average person in the survey was using 3.5 models. That means they're not picking one winner, they're building portfolios. They're using different models for different purposes, which is for sure one of the best ways to get the most value out of AI. The question that I have is how realistic that is scaled across everyone. This may be one area where I think that this early adopter and highly enfranchised power user group does always look a little bit different than the average user. Some people simply won't be willing to spend both the

Starting point is 00:19:43 actual money as well as the time to map out which models are useful for which different use cases and to then pay for that access. So that's one we'll have to see, but it certainly is the case. If you were looking for what some of the most active users of AI are doing, it's a portfolio approach to using models. So that is the AIDB Intel AI usage Pulse Survey for January. Lots and lots of really interesting things there. I have huge thanks to everyone who participated in that survey. I'll open up February's in a couple of weeks, and we will see how things have changed. Like I said at the beginning, apologies if there was some crazy big news that I missed,

Starting point is 00:20:16 but we will be back to it soon. For now, I appreciate you listening or watching as always, and until next time, peace.

The AI Daily Brief: Artificial Intelligence News and Analysis - The Time Savings Era of AI Is Over

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.