The Infra Pod - 1,000 tokens per second: the case against predicting one word at a time (Kumar, VP of Engineering at Inception Labs)

Episode Date: August 24, 2026

What if the entire LLM industry has been solving language generation the slow way — one token at a time?In this episode of The Infra Pod, hosts Tim Chen (GP at Essence VC) and Ian Livingstone (CEO of Keycard) sit down with Kumar, VP of Engineering at Inception Labs, to unpack Mercury 2, the company's diffusion-based language model, and make the case for a fundamentally different way to generate text and code.Kumar breaks down the core mechanical difference: where a GPT-style transformer predicts the next token one pass at a time, a diffusion model predicts a whole batch of upcoming tokens at once and iteratively denoises them in parallel, freezing the easy ones early and spending extra compute only on the hard ones. That approach — borrowed from image generation but re-engineered for text, where output length isn't known in advance and streaming is a hard requirement — yields roughly a 10x speedup and 3-5x cost efficiency by simply doing fewer forward passes. Mercury 2 hits 1,000 tokens/second on commodity NVIDIA hardware and now benchmarks competitively against cost-optimized models like Claude Haiku, Gemini Flash, and GPT-mini, though Kumar is candid that no diffusion model — Inception's included — has yet reached Sonnet or frontier-tier intelligence.The conversation moves from algorithm to product: why Inception keeps its API OpenAI-compatible (Kumar's electric-car analogy — same interface, very different feel under the hood), why most agentic sub-tasks don't need frontier intelligence at all, and why voice and search are the workloads where sub-second latency stops being a nice-to-have and becomes existential. Kumar closes with a genuinely spicy take![00:00] Guest introduction: Kumar, VP of Engineering at Inception Labs[01:31] Diffusion vs. autoregressive LLMs: what's actually different under the hood[07:00] Why isn't diffusion the default already? Trade-offs and diffusion's late start on text[09:05] From Mercury 1 to Mercury 2: the road to enterprise-readiness[11:26] Open source diffusion models, ICML's best paper, and Inception's head start[16:15] Under the hood: how Inception hits 1,000 tokens/sec without sacrificing latency[20:52] Data strategy: what training a diffusion model actually requires[22:51] Does diffusion change how you build agents and products on top of it?[28:00] Mercury 2 benchmarked against cost-optimized frontier models[29:23] The model routing problem — and why it may already be mostly solved[37:47] Spicy Future: AGI

Welcome to the InfraPod. This is Tim from Essence, and Ian, let's go. Hey, this is Ian Livingston. Co-founder and CEO of KeyCard, the one-stop shop to secure your agent and their identities. Could be more excited to be joined by Kumar, VP of Engineering at Inception AI, who are interviewed our audience to you and some of the stuff that Inceptions are up to. I hear you've just released a new diffusion model, Mercury, too, and we have so many questions about what's going on there.

Thanks for having me. Super excited to be here. I lead engineering at Inception, and we actually launched our first Enterprise-ready Mercury model in February. It actually brings to light diffusion-based text LLMs for typically text or coding. The main advantages that it has are around two things. One is being super fast at producing intelligent responses. It is a reasoning model, but for the class of reasoning intelligence that it produces,

because it's very fast. And it does that by leveraging a set of diffusion algorithms, both at training time and in a conference time. It's able to hit 1,000 tokens per second, and it benchmarks really well in terms of intelligence against cost-optimized models, such as the haikus of the world, or the minis of the world, or the flashes of the world.

And help us understand, you know, traditionally the last several generations, you know, since like 2022-inted-tich NEC came out, it's all been focused on these large language models. Help us, the audience, understand, like what's the phone one difference between like an LOM, traditionally what you find out with like a GPT and what's happening here with the diffusion model in Mercury.

You know, traditionally, I would always associate diffusion with image models and not text models. So help us understand what those two things are and like why a diffusion model is so powerful when applied to text. Yeah, definitely. So GPT is a very well-understood architecture. It's a transformer model, but it's really trained and is really good at next token prediction.

So basically when given an input problem, it is good at predicting the next token. What diffusion algorithms do, on the other hand, is they're able to predict multiple tokens in parallel. And rather than predicting them in one pass, they actually generate an approximation of the next K tokens or N tokens, and then they iterately refine them through a process called denoising. And in that process, they're able to be 10 times faster.

So you almost only have to do like a tenth of the forward passes to decode all of the output tokens. that gives you like a 10x speed up. And cost-wise, because we're parallelizing a lot more, you get 3 to 5x cost efficiency. So nowadays with token efficiency being super interesting, diffusion models for text and code tend to be way more cost-efficient than the equivalent auto-regressive models,

which are the GPTKL. If you want to compare them to image diffusion, which everybody knows, almost every image generator there is designed to be a diffusion algorithm, there are significant differences between an image or a video diffusion model and a text model. First and foremost, we don't know the size of the output. So while the input prompt might look like text, the output, in the case of an image diffusion model is pixels. In the case of text or code, the output is code. You don't know what the length is. Text LLMs are expected to be

able to support streaming. You want to be able to see the output as it's coming out, whereas all the image generators do everything in parallel, but they're just blocked. So you have to kind of wait for a very long time. So very hard to support use cases like speech, voice, use cases or search use cases doing that. And so some of the things that we've done at Inception is to bring the transformer architecture that is very similar to all of the GPT models or the equivalent ones from the other major labs and making sure that we can actually bring diffusion algorithms to both train them and also serve them in production while supporting all the requirements that you have for streaming and also

generate efficiencies that you typically get from a diffusion output. Incredible. And like this has been a progression, right? like one of the, it's one of the things you mentioned about is like non-fixed output side. So can you help us understand like some of the work that has to be done to transit the technology, support text specifically like the fact that one, text isn't fixed, whereas image or video is? Yeah. What makes it difficult for text is that you have to optimize for several performance characteristics that are beyond just the quality of the output. Because you want to be streaming,

you have to make decisions around specifically to the hardware. depending on the kind of hardware you have, the amount of parallelism that a diffusion algorithm can leverage is way more than the parallelism that typically a single token predictor does it. So that's one. How do you sort of parallelize for propagating multiple input tokens that have not been finalized yet and predicting multiple output tokens? In that process, you also have to sort of optimize for,

intuitively, just think of it as, they call it the acceptance rate or something. So let's say you will generate 100, tokens, the next 100 tokens in one batch, but not all of them have converged in the sense that they're not ready to be accepted. You have to have designs that allow you to sort of figure out which ones are ready and can be frozen, and then you go back to the next four pass and then freeze the remaining ones. So you're almost like deciding which tokens should I say are generated. And they're not generated in the same order that a typical auto-regressive model would do. Like predicting the next one, next one, next one is pretty easy. Here in the next 100, I need to figure out

which ones are ready. And intuitively, if you think about it, if I gave you 10 sentences that are the output, many words can be predicted pretty easily. And then there are some keywords that have to be iterated a few times. And so if in this process, you have to tune both the parallelism that you can leverage so that you can do forward passes very efficiently, and also tune the algorithm, the denoising algorithm, so that the simplest tokens get, they converge quickly, and then you only spend additional compute on the ones that are a bit difficult to predict. And then, let's say, in 10 forward passes, if you're able to then have all 100 tokens converge

and you're happy with them, then you move on to the next batch. And in this process, you've done only 10 forward passes, but you've kind of converged 100 tokens. So that's kind of, it's interesting. That gives you to get 10x speed up. And one of the things I really like about diffusion algorithms is that the way they generate the speed up

is by avoiding computation. It's so expensive. These models are huge, right? Hundreds of billions of parameters to trillions of parameters nowadays. just to compete in the arena. And imagine if you didn't have to do as many forward passes to predict all the tokens that are coming out.

So to me, the fact that diffusion avoids computation and that's how you get to speed up also immediately translates to being a cheaper model to operate. Since you're not doing as much compute, we can take that cost savings, time savings, and return it back to the customer or the user so they can spend it on additional tokens. And so for all the normal developers,

like less that really don't really have a huge nuanced understanding about this. Why isn't just everybody using diffusion text models? Why is it just not everywhere? Why can we just use it by because it's not like there's so much benefits, right? It can, you know, have a lot less footprint, can generate long sequences

much more faster. There must be trade-offs, right? Or maybe something we haven't figured out yet, right? What are the fundamental limitations or things that we're still figuring out that makes diffusion model not the default choices for everybody out there? Very big question. Let me maybe talk about the journey we've had in bringing diffusion models to the market.

Both of them are transform our architecture, basically, is for text. So the original seminal paper of attention and laws you need is still relevant, right? But how you train and how you do inference is more complicated. So predicting the next token is relatively easy, and then just doing a simple auto-regressive system is easy. And I think the beauty of that was discovered first, and then they discovered scaling laws. Once you had those two in hand, you had a flywheel where the more data you could bring, the larger model you could train, and you could efficiently train larger and larger models, which then became more and more intelligent, right, on a log scale, effectively.

Diffusion came to images, but because people didn't figure out how to stream or, you know, in some sense, it sort of stayed in the image land. And only recently have people started to publish and actually figure out or have the breakthrough that's needed to bring it to text. So text diffusionette models are like the newest innovation. So that's one of the things. You just had a late start. That's number one.

But that's 2024. So by 2024, the research publications were in a state where you could go tinker with it and start building text LLMs that were powered by diffusion, right? 2025, even at inception, released Mercury 1, Q1 of 2025. It was fast. We could get a thousand tokens per second, but it was a small model. It was not very smart. we were still figuring out how to get it to work really well on production use cases.

It also was an early version of the model. We're proud of it, but looking back, Mercury 2 is way stronger, and we have a few others that are coming this year that are going to be even better. And so in my mind, 2025 was we were showing that we could actually train these models to actually be intelligent enough and be useful enough for production use cases. And then with Mercury 2 is the first one where we actually went. It's a larger model. It's also significantly optimized for Blackwells and so on. So the speeds are taking advantage of the latest hardware. I feel like 2026 is the year where it is enterprise ready.

We're seeing a ton of traction, especially from large-scale voice search, and even general purpose, people who want sub-agents that are very fast, and the intelligence is good enough for doing a lot of agentic work. And that's sort of telling us that, hey, it's actually finding that enterprise readiness. Now, if I fast for two years, and this is me dreaming now, I do work at Inception and we're very excited about this technology. I would be surprised if most of the models haven't really moved on to using some form of speculative decoding or diffusion

or things that allow us to break out of this, just the next token, like a typewriter, type the next character, type the next character, but you could just literally blast through in parallel multiple tokens that you predict. Okay, so there are two trajectories that are still ongoing. We don't have a frontier diffusion LLM yet. We have something in the works. Like I mentioned earlier,

we are able to benchmark quite well against the cost-optimized versions, and then you get to like the sonnets and the pros, which is around the corner, and then you want to go to Methos level or the 5.6 or 6.5.6 point and whatever is going to do the next big thing. So to me, the scaling laws, there's nothing intrinsically limiting in the architecture, but then it takes time.

Everybody knows these things have to be built, training very, very large models using diffusion. That'll take some infrastructure investment. They'll take some algorithmic investment. And so that's sort of like what's remaining. And I'm optimistic that two, three years from now, everybody will be using diffusion in some way, shape, or form to sort of get out of this, just predict the next token limitation. And given this is all so new, and you talk about like it's not just new on the architecture. When you have a different architecture, even though we've seen diffusion for image in a. I assume like even the text site, that side is so new, you probably can just slap in like an open source, you know, LLM inference engine and just hope it works.

There's probably a lot of nuances when it comes to like both training and serving and even the training side. I don't think it's like a very widely public knowledge how to do it. And I feel like in this space of models, we're kind of just assuming open source will always keep up or catch up whenever. where there is a new model, just the base assumption we all have. But diffusion, I don't think we have a very good understanding. Is it possible to create a very good open source version of something? And so I know you guys are pursuing, creating like this mythos level and frontier level diffusion text model.

Do you think this, I guess the system know-how, the training, all this stuff is hard enough. Their open source is going to be even harder to pursue in this path. And that's kind of give you guys, this really advantage in this space. or how should folks even think about open source will come but in a much more even smaller version given how hard it is? Just trying to understand

what is possible to approach from the outside world to get even closer to an open source alternative. Yeah, so right now we've spent the necessary time and the founders are pretty much the folks who invented many of the early diffusion technology, the algorithms. And so we have been able to bring

that to product. The field itself has gotten a lot more attention. Overall, if you look at 2026 versus 2024, when inception first started, I already see many more people looking into diffusion algorithms. Even for traditional auto-regressive models, if you don't retrain, but only at inference time, you want to get some of the benefits from speed or lower latency, there have been technologies that are under speculative decoding, which are like somewhat simpler but weaker versions of like a full-blown diffusion. The architecture space for diffusion is very rich, and so I'm optimistic a lot of innovations will come. With open source, I see two things. They're very good at sharing what they know, and as a community, they produce amazing progress.

And I've been at both companies that are, you know, avid users and proponents of open source and also close source, large companies that actually truly want to create a durable moat that keeps their technology ahead of open source, right? In my sense, there's more open source in 2026. I mean, you saw announcements around diffusion Gemma, there's also other companies that are thinking about doing more diffusion. This is all good. The fact that more people are working on it will help bring that technology

from early stages of research to something that's ready, has been tested, has been engineered, and can be used safely and is useful for customers. we are a little bit ahead, so we do have that advantage, but I think the whole field will accelerate, which means that we will also get benefit, right? The nice thing with open source is if you do it well, you take advantage of open source,

but you also give it back. And if enough people do that, then there's a flag wheel that benefits everybody. And I think it's just a staging thing, not to talk about research, but ICML just ended last week, and one of the best paper awards went to a diffusion-based paper.

So there's a lot more graduating students who are now very well-wurst in text diffusion that will bring about a lot of the innovation. On the traditional diffusion algorithm itself, I don't think it's that unknown. It's just that for images, people have understood it really well. The basic denoising algorithms,

the basic tenets are the same. So in theory, they have it. It's just crossing that over to make it work for text. So Inception has a certain amount of momentum. The other thing also is, it's one thing to know, how to train the model. It's another thing to build a product that can bring that technology safely and in an enterprise ready way for a lot of users. So being able to bring diffusion models to

actual use cases, that ticks another cycle of product and engineering innovation. So we have, I think, some of that stuff, we've been able to both benefit from the improvements in the AR technology itself, the auto-regressive models. One of the things we strive to do is run diffusion models on commodity hardware, right? Some of the properties we're talking about can easily come from custom chips and also, you know, everybody wants to build an inference chip now. There's nothing that says you couldn't marry that with a diffusion algorithm to just get amazing speeds, right? While at the same time, the availability of those custom hardware is limiting, whereas

general purpose custom hardware is much more ubiquitous. So being able to get speeds like hardware using just software and commodity and VD hardware, it will also help people leverage Inception models much better. I'm curious, like, shifting sort of like the inversing side. You know, one of the things I think that's relatively opaque to the broader world is just how hard it is to actually like deploy scale and serve these models and be so fast. Like one of the things, you know, I was listening, I went to the homepage. They have this very cool like demo on Inception's AIS, the homepage where you can use their model.

And it's just so fast. Like, it's so fast. So can you help us understand how do you get it that quick? what type of infrastructure you have to build on the hood to actually enable serving of that. Like, is there something differentiated with a diffusion model and how you deploy it and run it? Like, help us understand how you reduce latency and generally just create the experience, the end user experience of using the model. This is almost a combination of what the algorithm can provide for you and how do you take advantage of that to build a right kind of product, right?

I mean, there are many ways to get latency. I think I talked about hardware. If you don't go into hardware, you think of the algorithm itself. Two things that we learned early on is that you have to make latency and interactive speed. It almost should feel like it's ready before you are to consume the output. And you do that by focusing on latency. There are two or three parameters that define how quickly the answer comes. There's a time to first token.

We have a reasoning model. So typically the model actually generates a bunch of tokens. There are reasoning tokens, which is like thinking time. And then it emits the first answer token. the time to getting the first answer token has to be short. And the second thing is, if you do want to generate a large output, being able to generate it very quickly allows you to sort of have that instant feel for the answer. Because reasoning token complexity, the way you generate reason token is exactly the same way you generate the answer tokens.

Even though it's like thinking time, you're just like talking to yourself. And then once you're done, you're like then speak out loud, right? So if you can make token generation very fast, our Mercury 2 models can actually generate over 1,000 tokens per second on commodity hardware, right? That allows you to then think at 1,000 tokens per second. Most models out there barely get 100 tokens per second. And frontier intelligent models are like even slower. And there are innovations that are coming now that are trying to speed that up.

So you can actually think 10 times faster, which means now you can respond 10 times faster. So latency comes down. That's an obsession with that. The other thing we also do is when, if you just leave alone the algorithm and just a single instance, right, you want to concurrently serve hundreds of not thousands of customers at the same time, and you want to be able to scale horizontally. So we make a lot of architectural choices that are obsessed with making sure that no request

sort of takes too long. What a lot of systems do today is under duress, they'll queue. And that allows them to get, actually, if you queue, it's actually good. You should be queuing with traditional auto-regressive models. A little bit of queuing is always good because it maximizes GPU utilization because you get throughput.

And throughput always improves efficiency. Versus we try to tune our systems to try to not queue as much. So there's other ways in which we can change the diffusion algorithm to be optimized for higher loads that don't require QM. And so that's sort of like another dimension. And because of those two,

we've also targeted things like real-time voice applications. SLAs that are grueling. You want time to first token that's in the 100 to 150 milliseconds. And time to the first sentence being completely responded to is like 300 to 500 milliseconds, just super demanding. And we run a reasoning model. And we also, this model is like hundreds of billions of parameters. It's not like a tiny model like a 1B or 4B that you can optimize.

And so that is almost like an engineering feat in itself to sort of show off how quickly these diffusion algorithms can. can generate useful out there. I think there's so much work that you guys done, right? And obviously you have the funding, you're just getting started. I feel like building any model today, it feels like such a huge effort required from all fronts, system level, you know, model, training, all kind of things. But yet I feel like what we are still amazed by is how much improvement it can still make.

And I'm very curious, like to build a reasoning model and diffusion, like you said, you have the system level work need required. There's the inference level, infrastructure level required. But I assume like just any model, you still need data, right? And they had to figure out how to get the data to train these, right? And we're just seeing like the big frontier labs. They're so hungry for data. They're buying data everywhere.

So maybe just because I don't think we have a very nuanced or understanding of this, I'm sure you still need data to train diffusion models. Are you trying to pre-trained from scratch from these and you have maybe some other unique type of things you want to keep capturing the data? Do you need much less data to train your models? What is your data strategy when it comes to train these models? Is it exactly the same as frontier labs,

or do you have some different mindset that comes down to it? Yeah, so the data story is likely more consistent across even traditional machine learning, LLMs, transformer-based, auto-aggressive, and diffusion models. We have the advantage of coming a little bit later so we can be more data efficient than in 2020 or three when really early models used mostly brute force to train their models. We do all of the pieces that you mentioned, pre-training, mid-training.

A lot of the effort goes into post-training and RL-based, especially for agentic workflows. And if you want to truly optimize the outcome of a task versus just what is the next response, and the data story is not that different. Maybe one component that we do see is if you train a diffusion algorithm, because it has benefits for propagation, training times are also shorter. Now, we're not training for internal models yet,

but that's one advantage we have. So being five to ten times faster makes your training runs go five to ten times faster, and R you can just use lesser hardware. But other than that, the data story, unfortunately, there's nothing about the algorithm that changes the intrinsic need to represent your problem, capture the right dataset.

And everything that we know about data for these LLM training, whether you're training a front-term model or a useful model for your special purpose task, kind of still remains high-quality data. And for reasoning models or even for agentic workload-capable models, you need RL environments, you need traces. So you just inputs, outputs, and the higher the quality, the better the model tends to do. And so let's maybe talk about the consumption side, like as a customer or a user of division models.

Obviously, you go to a website, it's so fast, right? But I'm sure it's probably not exactly the same because even on our website, you saw how the iteration looks like, right? It's no longer token by token. It's actually making much more major changes. Their tradeoffs, I assume, on both sides by able to actually make much more broader changes when it comes to outputs. So as a user or a customer of diffusion language models, is there any behavior change when it

comes to integrating insure agents, like I'm building a multi-agents product? or I'm using coding with a bunch of factory things. Do I just slop in diffusion LMs in exactly the same way, exactly the same assumption, just faster? Or there's actually some other way I should need to understand the nuances when it comes to using these models at scale. Interestingly, at least the way we're bringing that technology through the Mercury APIs, we're trying to keep them as compatible.

AI itself is hard for people to ramp up. I mean, it used to be prompt engineering. Then from there, it went to, you know, overall, model selection, fine-tuning, and then now people have to do harness optimization, picking the right provider, picking the right architecture. The complexity has gone up. So at least in terms of product and engineering design, we try to make it, I mean, our API is open AI- API compatible.

If you want a server, you pick an MCP standard. So we've tried to kind of keep everything the same. I give people the example of an electric car versus a traditional ice car, internal combustion engine car, right? If you're driving it and you want people to adopt it en masse, they may today know it's an electric car under the hood, but they design everything to be as much the same as it used to be with some tweaks, right?

But the car field is different. It's much more responsive. Traditionally, mileage used to be an issue. Today it's not. But the car is heavy, right? And the car has a lot. I mean, it has what maximum RPM at zero torque, sort of maximum torque at zero RPM, which is like a, yeah.

So when I think about that in product design, you want to translate as much of the users know-how to Mercury and diffusion models as you can. So we've kind of hidden some of those complexities. Now, while it may not appear that different from the API standpoint because we've worked to make it look similar, when you design the product, you think differently. And so there we are finding that certain things that were not possible before, because you're generating a reasoning model's output in 200 milliseconds or 150, milliseconds just unheard of.

If I told you it's 1,000 milliseconds, voice is just off the table. You do custom models. And so being able to unlock these experiences that are sort of not possible before without going really low on the intelligence scale, really small in the model size, is one thing we see.

And so in product design, people can think more aggressively about using that. On the agentic side, what we've also found is, and you probably heard this from other areas where while Fable, mythos, 5.6, these are like amazing models. They're frontier. They're pushing. But most agents, you know, three out of four tasks that they have to do along the way

to accomplishing their goal, doesn't need frontier intelligence. And while token efficiency has now become the main reason why people pursue, like if you want a simple router, you want a simple planner, or you want, you know, you want a compact and construct your context along the way as your agent is executing, you likely don't need Frontier model. And the tokens are expensive. They are limited in capacity. And you're like, oh, you know, you also want them to be fast. If an agent is taking 30, 40 minutes to run, and 80% of the time it's doing these non-frontier tasks, if you could make that go from, you know, 30, 40 minutes to five minutes. That's phenomenal. Even the agent becomes much more

nimble. So, and then the last one, I would also say search. Search is an interesting intermediary where behind the scenes, everybody wants to put their knowledge in rag, not everything is in the model, not everything can be pumped into the prompt. When you create these search systems, you have to sort of process quite a bit of input and also generate a summary of some sort

or context of some sort that goes into an agentic workflow or into your LLM as a tool call. And there we're finding that even that's unforgiving. If you look at the largest search providers that are tools in these LLM, either chatbots, or your agents, they promise several hundred milliseconds turnaround. I think on one second would be considered

acceptable but slow, whereas voice, one second is you're not eligible. So you can design these things that can dramatically change the ability to deliver AI to a lot of users. I'm curious where Mercury sits against the general benchmarking. So we're trying to understand the pro versus con. I mean, everyone produces muddle cards. I've had a chance to look. But generally speaking, like how does it perform against

you know, the generally accepted benchmarks that you find for most of these cases. Yeah. Yeah. So Mercury 2, which is, it's already, I guess, four or five months old, Mercury 2 benchmarks well against cost-optimized models. So those are your Claude Haiku. I think at that point it was 4.5. You have Gemini Flash models and the GPT mini versions.

And so for the size of the model and the amount of training investments we made on the model, that's what, and so it proves that you can build diffusion models that can reach that level of intelligence. It is the most intelligent diffusion model out there, including the other ones released both open source and others we know of. We're in the process of bringing something that's in the sonnet and pro use case. So I think that's another one in terms of what you can consume today versus what to look forward to in the coming quarters. And, you know, there is still no model that's a diffusion powered that can give you pro or summit level capability or go to those are the frontier really frontier model capability. Awesome. And one of the key questions

I have going back, you know, finally what I'm hearing from you is like we produce this very fast, this very cheap model that's like generally very good at specifically a task where you don't need like, you know, a whole fact of the ultra think like capacity. You know, one of the discussions in industry right now is broadly, you know, broadly. There's an open source model movement. There's movement towards cost efficiency. There's these discussions around data residency and IP. I'm curious, how do you think about the model routing problem?

How do you think about what we need to do to actually build the appropriate task to model switching layer that can entourage a route and switch the appropriate model based on the actual task? Do you think we've seen large progress there or do we need to see large progress there? Obviously, from this mini model you have today, like you're missing some sort of primary piece, Whereas if, you know, where you're trying to have agents like new dynamic tasks, where they can scale up different types of models based on the tasks we're taking on. I'm very curious to get your insight or how you think where we are today and enable, like, intelligent model routing,

and then where you think we have to go long term. And if we're missing a piece of infrastructure, or there's something that's something like an inception is going to deliver or where you see this going. It's such a fast-moving field. It's hard to say anything that'll last for too long. My current feeling is that a lot of the tasks have been, become possible because people have started to use AI.

Or traditionally, they would have to write code or they would have to invest quite a bit of development before they could bring something that was of reasonable quality. Now, with this AI under the hood, they're able to quickly develop. One of the things people have not spent a lot of time on, I mean, I'm going to pull like a computer science 101 where there's no complexity chart for these tasks. If you told me that, you know, this is an order N complexity problem, this is an order n-squared complexity problems. This is it an exponential or n-p-hard problem. You can say, oh, for the N-p-hard problem, bring me AGI. Oh, but for an order-end problem, does it really warrant running an N-cube

algorithm to solve like an order-end algorithm? And in computer science, you are always taught to, like, think more, bring it down to what's the most efficient version of this algorithm you can build. So I think routing is one of those where we already have more AI than we need to solve routing really well. But how do you quantify that? If you ask me to take a developer task off of somebody's, I don't know, their bug queue or feature queue and say, you're going to implement the sprint, it's going to take your 60 hours to implement it. That's probably today at the frontier level of producing human level quality at that description that is very, very abstract and vague. Right. And so my take there would be we need to somewhat think a bit more.

in terms of classifying the complexity of a task. We know intelligence levels for all the models that have ever been released. We have so many evils. Evils is way far ahead for the model, but the benchmarks, it's not clear. We're always creating harder and harder benchmarks. So we have a good sense of where the waveform is in terms of what can be done, what cannot be done, what should be done next in terms of cracking the wavefront. But the problems that are there today that are already, in some sense,

accessible to existing AI, people haven't thought through, like, you know, do we already have, is the routing problem pretty much soda and it'll be solved well enough using an intelligence score of X? And that's a good question. So on the routing side, I actually think it's one of those things that's already, quote-unquote,

you should be able to build a pretty good router with even the medium-tier intelligence models that are available today. And you already see that. You already see that routing is one of the most efficient things. It also improves token efficiency. There's no reason to burn a digital. trillion flop to forward prop and generate the next token if the input is a very simple question.

And so just looking at OpenRouter and you're seeing how Mercury's current usage and when they announced and stuff like that, it sounds like there is definitely good amount of people trying it. You know, I see you have a coder model and reasoning model and stuff like that. And, you know, I remember seeing one of our VC friends, Tomas, talking about like the model lifetime, how often people jump on a model and then some generally jump on another one. Because I feel like today, I think most people, especially when it comes

of coding, I guess most models are pretty slow anyways. They're looking for the most accurate. Like the highest quality comes to code outputs, the highest quality in comes the text output. Speed, I guess it's not, haven't been like the top of mind thing yet for everyone. And so I'm just trying to understand if

thinking about this correctly, you know, diffusion models are great for all the deficiencies and the speed but when it comes to quality accuracy, it still comes down to the fundamentals of the model training of the data and all the kind of things still involved, right? And so given how much harder it is to do this,

I guess for everyone to want to use this model and get the most benefits, I was just wondering, how are you able to chase out the highest quality outputs while building diffusion models, architecture, and the same time, do you think there are certain type of benchmarks and accuracies?

you guys are more focused on to give you a much better shot at, like, okay, we will be at almost like the frontier level, like the Mythos level you just mentioned, and able to continue be on par so that, you know, the user and truly all of everyone want to jump onto it. Or do you believe that the speed itself, such a big difference, will become much more of a motivation because people are building different kind of applications that requires latency? Like, I'm trying to understand, like, what is sort of the way we should think about the difference when it comes to how people will shift over to behaviors. Yeah, so there's maybe two answers here, right?

There's nothing intrinsically limiting in the technology to one day produce a frontier model, right? Like I said in the next, let's say if this was to come true, which I believe and hope it comes true, is that two years from now, everybody's using diffusion and for every model, right? They figure that out, and it's no longer like a one AI startup having the know-how, right?

That'll benefit everybody. That should bring you frontier intelligence combined with frontier speed. I can think of it that way. And I think our journey, we are still early in that journey. The second thing I would say is intelligence with a compute bound is what latency truly represents. I mean, all three of us are AGI. And while I am speaking, I see heads nodding, I see smiles, I see you interrupt me or I'll interrupt you. Allums can't do that today.

So how do you bring frontier intelligence in an interactive mode that can become part of us? Why can't we have another agent that is just like another AI? It might be mythos, but mythos today just takes too long. It's like I'm going to go off like an intern, come back two weeks later and show me something that I would take any three months to do. And if you look at even human intelligence, when you see these spelling bees or competitions where people have to like under stress or under pressure, solve something, it takes a lot of effort.

And your intelligence is no longer frontier. when you put a latency or a compute bound. I'll give you a perfect example, voice applications, right? If you pause five seconds before responding every time, it doesn't matter how intelligent you are. They're not going to want to talk to you, right? Socially, super awkward, right? Not to mention an AI, you'll just come switch over to something else

and then you just won't benefit from it. And so in voice, it's very clear. If you look at the frontier models in terms of adoption for voice, their intelligence is not frontier. To prove it, GPD voice that came out just last week, if you benchmark the intelligence of that, it is not that good, compared to what you would expect from a mid-tier LLM. But it is amazing to talk to because it's that experience.

And it's not dumb. It does really well in Q&A, it's factual, ritual. But when you ask it to reason or count how many letters of a certain vowel are in a word or so on, it may or may not always get you the right answer. And of course, I'm super confident. There's other problems with LLMs that are ubiquitous. So to me, those are the two answers, I'd say. One is, I think, eventually, almost every model lineage will reach frontier intelligence.

If anything, I see compression in 2026. There are more models that are sort of crunching up in the front line versus in 20203 or 2024, where, I mean, GPD3.5, I was blown away. Before, I think, I think, for the time, my most favorite model is still a lot of the non-reasoning models. And then when you put a latency budget, which is required, then you can say Mercury probably has the frontier intelligence under our fixed very tight latency budget. So here we want to jump into our most favorite section of this podcast called The Spicy Future. So, Kumar, tell us your spicy hot take about anything. What is what you believe that most people don't believe around you? Yes.

Yeah. So maybe I'll give you guys two. I don't know. One seems sort of, I'll repeat what I mentioned earlier. I actually feel like AI capabilities are way more advanced than the usefulness that people are getting out of it. I mean, me included. I'm so used to my old ways of doing things. And it's not that AI is not ready or you need something better than Frontier, Frontier Plus, or Super Intelligence or AGI for you to be benefiting from it.

And to me, that I think at the model, especially the reasoning models are like really good. People should use it all the time. So I kind of, my hot take would be that AGI is probably distracting people from actually getting more adoption of AI today and getting more value, even as society. And it'll figure, if it'll figure that out. That's sort of like one claim I'd make. I think people are just fixated on what is the next best model. It's like a race, right? To lead a healthy life, you need to do, out of it.

I don't know, 10-minute mile, maybe 12-minute mile. But you're so obsessed with like a four-minute mile, a three-minute mile, that it's really not the right thing to get the most value. I think adoption-wise, people don't realize how much AI is already at their fingertips. So that obsession, that's one. I don't think it's necessary. You can still move the way front while bringing useful technology. I usually point to the iPhone.

It doesn't have all the advanced features, but it's so useful. You just can't live without it. The other dimension I would say is that in spite of the intelligence being ready, I don't think it's that useful. I find that kind of hard. We think a lot at inception about how do you build an AI that's useful, right? It still makes stupid mistakes. I routinely corrected it.

It's, oh, yeah, you're right. You're right. This is not true. I mean, I ask it some, I mean, it gets basic stuff wrong. For those of you who are LLMs, and I would ask when a new model comes out, or if I didn't remember what the architecture for an old model was, was this model, a dense model? are a mixture of experts model. And it would tell me, oh, it's a dense model.

And I'm like, okay, and then I ask you a few more questions. And I'm like, hey, this doesn't sound right. It's too big for a dense model. And then when I pointed out, it's like, oh, no, no, you're right, Kumar. I made a mistake. It is a mixture of expert's model. And then I would ask how many active parameters are there? And I'm like, my God, how can you be like so flippant and so confident and so

flippant, right? So to me, that's a usefulness block. It still hallucinates. It never says I don't know. I mean, this is 2026. guys, not 2023 or 24 when we let it run. It never says, I don't know. I mean, there's no person that I know that if I asked them a question, even directions

to take it. I do know how to get to the movie theater from here. I don't know. Let me pull up my phone. Right. AI never says that. I don't think there are things that AI doesn't know. The AI doesn't know a lot of things.

We didn't know. People don't know a lot of stuff, right? So to me, that is sort of like, I think it's a usefulness blocker. So even though intelligence is way better than what you need to be, it's not as useful. I think a lot more people should spend time on making it useful. That's sort of like my hot take. And then the other thing I'd like to maybe encourage people to think of is there should be something like intelligence complete class of problems.

People should be like, this is intelligence complete with an intelligence score of X. This task is intelligence complete with an intelligence score of Y, right? If they brought that in, then it'll bring a lot more rigor to people can then have wavelengths or rings of solved problems like in traditional algorithms where they're like, okay, Now AI is good enough to solve X. They kind of sort of happen in the folklore, but people don't use that. They just like benchmark models, not really tasks. I love dive into your statement that it's not that useful,

because it's clearly got some utility in coding, it's got utility in summarization. Help us understand what you think that Delta is for one and what we actually have when you say it's not that useful. Oh, maybe I didn't mean it not that useful. I think it could be more useful. The way people think they'll make it more useful is by increasing the intelligence score, or building the next big model that can now reason for six hours of human tasks and finish it,

versus in terms of the current experience, coding is the best example, by the way. Because I know a lot of engineers love and think about these things. If anything, the verifiability of the coding task has just made it. So coding is the most useful. I mean, at the inception, too, we cannot live without the, latest coding suites that are AI powered, right? I love those. But I think for, like, if you want to ask it questions, it just hallucinates way too confidently. And if you don't know about something, you just kind of take it and go. As soon as you know something, you almost know like, okay,

I need to double check or maybe ask you a few more questions to validate. That's what I mean. I think there's like a 10, 20 percent usefulness gap that's still there, whereas for coding is probably the lowest. amazing. There's so much questions we can want to ask you, but just based on time, I know, I want to hang you out all the time. If our listeners want to try out the models, want to learn more about diffusion, LMs, want to learn more about Inception. Where can I find the company and more information? Yeah, I think the best place is go to Inceptionelabs.AI, and that's our homepage. Actually, one of the things I like about it is there's a, on the homepage, it's similar to, like,

Google search, there's a prompt box. Just type in your typical favorite five queries. Just take Mercury 2 for a spin. We also have a set of blog posts. If you're working on voice, if you're working on search, if you're working on subagents, you'll find interesting stuff there. We're just getting started. So, and if you have any questions, you know, just reach out. Amazing. Well, thanks, Kumar. This is super great. That was so interesting. Thank you so much. It's great to hear all about diffusion models and all the of infrastructure after building. Thanks for having you, Tim, and Ian.