Technology, Connected - How AI Agents Will Change The Future Of Work - Andrew Hill

Episode Date: August 28, 2025

Andrew Hill, co-founder of Recall, believes the future of work won’t be built on pages or apps, but on swarms of AI agents. Essentially pieces of code that remember, reason, and make decisions on your behalf (and spend Bitcoin), agents will form the new interface layer: where identity, memory, and trust replace passwords, browsers, and brands.In this conversation, we trace how agentic systems evolve from tools into collaborators, how they will coordinate between each other, negotiate access to our data, and rewire what “using the internet” even means. Hill argues that the next great challenge isn’t making AI smarter, but making it responsible, ensuring the web’s new memory layer remains transparent and human-aligned.It’s a quiet revolution: the shift from search to delegation, from browsing to briefing, from information to action.The agentic web is coming. This will help you get ready for what awaits. Please enjoy the show.And share with your most curious friend. Watch the show on the Thinking On Paper dedicated YouTube channel.--TIMESTAMPS(00:00) Disruptors & Curious Minds(01:25) What Is An AI Agent?(07:15) Emotional AI: Risks & Reality(12:49) Language, Evolution & AI(16:59) The Death Of Critical Thinking?(20:05) How To Trust AI Agents(24:27) Recall: Explained(39:49) What Should Humans Be?--LINKS & RESOURCES Learn more about Recall AI Agent training here.Follow Recall on X Follow Andrew Hill on X--Other ways to connect with us:Listen to every podcastFollow us on InstagramFollow us on XFollow Mark on LinkedInFollow Jeremy on LinkedInRead our SubstackEmail: [e-mail]

Disruptors and curious minds, welcome to another episode of Thinking on Paper. My name's Jeremy. This is Mark. We get to unpack the future with the people that are building it. Try to sense make and figure it out what it means for us as individuals as a society as we move forward. Mark, where are we headed today? We're speaking about AI agents, the agentic web swarms of AI agents. And I just want to say for our listeners, we're aware that some people who listen to thinking on paper will know everything we're talking about.

some people will be new to the very idea of AI agents, what they are, how they are made, how we can trust them, and where they will be taking us in the near future. So we're going to try and make this episode accessible to everyone. So we'll start wide, we'll start broad, and we'll go deeper and deeper and deeper. We've got just the guest to do that with us. We've got Andrew Hill, the CEO and co-founder of Recall. Andrew will explain it better than I can, but it's kind of like Kaggle for AI agents, Kaggle trains humans, recall, trains, bots or gives the user the knowledge that they

can trust the AI agent. Andrew Hill, welcome to the show. Thank you for thinking on paper with us. And as I said in that intro, yeah, let's start broad. What is an AI agent first? Well, I'm so happy that you saw that caggle through line. I think, yeah, agent is a very hard to pin down word right now in AI. I think of it as actually very broad. Some people probably narrow it down and they think chat GPT is a model and then there are agents, which are these things that you might deploy on N8N or on Google Cloud or like in some of our arenas, you might see trading agents, which are kind of software with models embedded in them. But I think the, I think, I think of agents as actually kind of a broader umbrella where, well, you actually have our models, which are much more like databases. There's some

kind of fixed thing. And then you built software around that. And so GPT5 to me is actually an agent. It's more than just the database. It's more than just the model. If you go to GPT5, it has capabilities like memory or web search or things like that. That's very agentic. So it's the model, plus a lot of software harnessing that allows the model to do things in the world and react to different kinds of information and make choices based on real-time feeds that are beyond just a query to a database. And so we see that spectrum really broadened up. You have chat interfaces that are very much like models. And then you have agents that live on Twitter that are reading Twitter feeds and reacting and posting kind of their opinions or thoughts live. And there's lots of things in between.

And there are agents that create value. There are agents that are experiments. They're agents all over the place. So agents is just a very broad term. I think probably we start to differentiate things as it goes forward. But that's just where it is right now. Who's making them? Everybody is right now. And I think like this idea, if you can kind of broaden the thinking about what an agent is, it's just software that has models that make, you know, use models or intelligence to make decisions inside of them. You see agents kind of proliferating very quickly.

Obviously, like the coding agent is a huge one right now. But it's pretty clear to me why the race for coding dominance from the primary. model builders right now is the main race going on. The chat interface that we all interact with has a lot of value. It may have a lot of value in governments or health care or things like that. But the ability to actually code agents to immediately solve needs. So I have to do this thing and the model actually knows how to write itself into doing the thing. So models will be creating agents on the fly to solve problems. It's going to make it so that agents are just what solve things in the future. And so.

Right now, I think everybody is trying to build agents. We're building agents internally to solve organizational problems, like how do we move information around the teams? How do we write better issues than GitHub? We're writing a lot of agents. And I think everybody that's kind of mid-sized startup, for example, is like racing hard at how can I get the leverage out of AI? And it largely comes from orchestrating agents.

So just lots of people are building agents right now. Yeah, a lot of like the general public belief system about. where the future is headed. I know in a lot of minds, they're churning around like, hey, I'm going to have this digital replication of myself that can jump through the internet and handle all the shit that I don't want to mess with. Like, how far off are we from that, that coherent story that the general public may or may not have in their head about agents and personal use? I don't think we're very far off. I think, I think it takes like a kind of killer use case or a killer product, and a lot of people are trying to build it. My co-founder and I were speaking

just yesterday, actually, about an idea. I don't think it's the killer product, but it would give you kind of a sense of how close the technology could do this. So there's a protocol called A to A. It allows agents to just kind of list themselves on the internet and say, here's how you talk to me, so that they're kind of open for business. And it hasn't blown up yet, but it's a really interesting opportunity. That got us thinking, what if I had an agent that was me? and I gave it think all the public things I want. Like back, you know, maybe you might imagine your Facebook profile, but only things that you wish had been public back in the day.

So I say, okay, I put maybe my reading lists, maybe I put my events that are coming up. I put this podcast, like this thing, I'm going to record this. Hey, you should, you can know about this thing coming up. But maybe I don't put my like personal calendar. So, and then in my Twitter profile, maybe it has a link or not a link. It just has an ID for that thing. And if you go to my profile and you see me tweet something, you might say, oh, Andrew, tell me more about that. And it's not like you text me or give me a phone call or anything. It just goes to my personal agent. And it goes,

you know, I was reading these five things and it looks like I kind of co-opted that idea from this blog post. Here's the source blog post and tells you kind of where my thinking might have come from. And I think you can expand that. Then I start having groups of people like maybe my team. I do want, if they talk to my agent, to just give you my full calendar and say, well, Andrew is open on Friday, but he was talking about going on a run or something. Maybe we should check with him if he really is open on Friday. That sort of advanced thing. I don't think that's the killer product.

It was just kind of a thought experiment about where we could open this personal agent up. That is a possible scenario tomorrow. So somebody's going to solve that where you could be having agents working on your behalf as you and differentially working together. So then Carson's not even finding my agent. He's just asking his agent to coordinate with my agent, things like that. I think the more sticky question, honestly, is like that stuff is coming, no doubt. I don't know the exact configuration of what that product space looks like, but that stuff is coming, no doubt. The thing that's more sticky is like, I don't think people are thinking about it.

Right now, people are having conversations with AI that are incredibly deep and revealing and intimate. Set aside all the privacy. And it's so funny that Cambridge Analytica happened half a decade ago. And now this technology has come about, which I guess is just so valuable that we've all gone. I get so much value out of this. I don't care anymore. I'll tell it everything. But we're in this word. We're doing that. And I think the really tricky, sticky question is to say, at what point to humans, I guess like a reasonable proportion of them, recognize that AI might be better at relationships for many humans than other humans? They don't have the emotions. They don't have the emotions. They don't have the emotional baggage, the tempers, the misconstrued communication.

Well, maybe the misconstrued communication is still in. That stuff now, you don't need to be, we don't need to have that emotional human baggage with us. I want a thinking on paper agent who can on our behalf, model it on you, Jeremy, model it on me, send it out there to send emails and contact guests and bring that all together into one hub where I can be doing something more productive and take care of that. and it needs a little bit of empathy to speak to the people that we're trying to get on the show. It doesn't need really deep human contact soft skills, does it yet? Yeah, yeah. Like, could AI be your companion without judgment, without anger, without betrayal for many people? And I'm not saying like every human is going to fall in love with an AI or its best friend is going to be AI.

but what I'm curious about is when do a majority of humans recognize, not that it's today, but at some point in the future, for many other humans, that is the truth. And if we get there, I think that's a very different future. That's the end of heartbreak. No one would suffer heartbreak anymore. No one would get dumped by their girlfriends and boyfriends. But I think some people want to experience the world and life and everything that way and they will do that. And other people have gone through things in their life that maybe they just are not open to. or can't deal with that. And they might want to be guarded.

They might want to only have friends and not deal with intimate relationships or things like, I don't know. I'm not those people. I don't know what they're like. Does that feature scare you? This has been maybe reality that we've lived with just in different ways. We've always had aesthetics and we've always had hermits. We've always had psychopaths.

We've always had this full range of human experience. that different people want to do it and handle it differently. And so it's just different. I know you end the podcast with a question from somebody else, but I think it's going to very directly relate to what I'm thinking there. So I'll save it. Yeah. This is such an interesting thread.

We will get to recall in a minute. Well, yeah, yeah, absolutely. One thing I want to touch on that we're hinting at here is some things can't be optimized. Some things, some things are the whole of the experience are the aggregate of the ups and downs of the generalities of moving through the world. And if you optimize that for the ups or you optimize that for this, it's, I think it's removing. I don't know where I'm headed with what I'm saying, but like that to me is like freaks me out a little bit because here's the thing. Humans love shortcuts. They love easy buttons.

They love to jump on belief systems that they can adopt that are a coherent story. Where do we? Yeah. Well, what do you, yeah. Well, I mean, humans, I mean, meet space is a weird thing anyway. And when you start kind of like first principles thinking around all of this, we're our worst enemy in so many ways. If you think about any interface that a human works through, a doorknob or a car or a bicycle or any of those things, the reason we design it the way that we do is because it gives us some aesthetic or physical or reward pleasure. Everything that we do, like the reason.

that you like the way that things works. And the reason you don't like alternatives is because we call it all these, we have many different names for it, but like maybe you have OCD. There's so many things in my kitchen that I don't like the way they're designed. I'm like, why would they do that? Why would they do that? And that's the opposite of the mini dopamine hits and everything you get from good design. And because of that, good design is inherently addictive. And so everything from the doorknob on up, we've been designing to get more and more pleasurable to use in a good way that makes it also addictive. And so the things that you're addressing are like, yeah, yeah, exactly, exactly. Everything that you're addressing is the inherent risk of doing this in the

beautiful way. Like, the better we make these things, the more addictive they are. Like, I have a three-year-old. I have no, I have no idea what his future is going to be like with these things. But I'm also like, I want him to be able to leverage the most powerful, the most intelligent thing in the world, but not change the way his brain works in those beautiful human ways and experiences and everything. Mark just held up design of everything. You mentioned good design is addictive. We talked to Don Norman, basically the king of human-centered design. You can watch that show. We'll post a link. But you were saying just a second ago, Andrew, what was your thought that interrupted? Oh, I mean, there's a there's an He wants the best of both worlds.

I studied evolutionary biology. That was my background. So I have all those filters that the world still goes through. There's a really beautiful essay or insightful essay from Cormick McCarthy. One thing that he gets a lot of interest around is his ability to understand local dialects and embed them in his writing very authentically, I guess. So just that's a side note and maybe kind of a source of why he does this. but he also is a fellow or a chair at the Santa Fe Institute.

If you're familiar with it, they do a lot of advanced research and thinking. And he was doing research on the origin of language. And he wrote this essay called the Kekul problem. So if you're not familiar with it, the scientist that was trying to understand the shape of benzene, was trying to work it out, trying to work it out, trying to work it out. And one night he fell asleep in front of the fire. And he had a dream of an or a snake eating its own tail. and he woke up and instantly said,

it's a ring. It's a ring of carbon. I get it. And so that's the premise of the whole essay, and it says basically, really interestingly, language arose in humanity very quickly. It happened maybe 1,000,

maybe 100,000 years. All humans were using language. This wasn't an evolutionary pace. The brain doesn't, it doesn't, we don't grow new loads in 1,000 or 100,000 years. And if you look at what happened, the early parts of the brain,

brain, the lizard brain, is so tuned, so optimized. It's been around for so long. It makes your heartbeat. It makes you remember to breathe. It does just only its functions and everything's been thrown away. But then the human brain has all these outer loads that have been around for less time and are less optimized. And when language entered the brain, it actually proliferated all the plastic space. So when you look at when you speak and think about language, it's firing all over these other places, but not in your lizard brain. Interestingly enough, the lizard. brain, that old stuff, that's where your subconscious is. Subconscious is very old. And so a lot of your dreaming is subconscious. It only accesses that part of the brain. And so he sets it up,

he kind of anthropomorphizes these ways of thinking of these parts of the brain. He says, the subconscious hates language. It won't use it. And it only speak to you in this imagery. And so that's this origin of this. The subconscious was thinking about the problem, but wasn't going to tell it to him. But the other really interesting piece in there, is that because language was so powerful, so useful as a tool for humans, it co-opted the brain. And I think about this a lot with models, because what actually was happening with language is humans, not to overload the term, but subconsciously, we're going, I need to solve this problem, that evolutionarily, I need to solve this problem, get food, star, fire, whatever it is,

and I can outsource it by speaking. My brain could go, I can do less work if I push it out to another brain. That's the process there, right? So language does that. There's some communication where it's actually a, it's actually like a fitness problem where it's actually, it's reducing how much energy it has to do by using this other tool. And when you start using LLMs every day, every day, every day, every day, every day, you can feel it. Your brain goes, I'm going to outsource this. I'm going to outsource reading this essay. I'm going to outsource writing this message. I'm going to outsource reading my emails. I'm going to outsource responding to my emails. And there's no doubt to me. that in years forward, we're going to see interesting patterns of neurons firing that we didn't see 10 years ago because our brain's going to shift around this really powerful technology. And so drawing the thread all the way back to my three-year-old son, I'm very conscious of this and wondering what exactly is going to happen to humanity at scale and to this individual and how the brain's really going to change here quickly.

And when it does change, what other things change is a natural reaction? or impact of the brain being used in different ways. We're moving into some very unknown territories, in my opinion. You talk about language being this evolution of outsourcing and AI being the evolution of language, the evolution of outsourcing. If we continue to outsource things that involve something like critical thinking, creative thinking, what does that do? Does critical thinking atrophy?

Does creativity atrophy? How do you think about that? Oh, you know, I hadn't drawn the line to critical thinking directly yet in the way I've been looking at this, but I'll just share a thought here because that is so, that is a really interesting one to unpack. For me, I don't know if you all feel this way, but I feel like in my life, every 10 years of so, I look back and I can feel that I'm a completely different human than the one that was 10 years before. I'm just like, who was that person? How little of the world did they understand? Why did they react that way? How do they not see the bigger picture? And I don't quite question it

that way, but I can see the, not that I'm like self-critical, it's just more like I can see all this learning to happen. And for me, I feel like as a, as a human, critical thinking was a major coming online moment. And I actually don't think I got it until grad school. So quite late, like mid to late 20s. And I can't relate to the human that was before it. Obviously, I was like doing some critical thinking in high school or whatever. But I think it was at such a basic level, being able to like really deeply unpack a problem and like move down many layers and explore it. And I think some people are really great at that like naturally. But for me it took like decades of people trying to train and push it into me.

So that might be a really big risk. And if you look at society, you really quickly can kind of like, you can say that are people that are good at critical thinking and people that maybe like never got there. And you can see the conversations they have are just completely different. people that are critically thinking about a problem that don't see eye to eye with somebody who is not, is going to have a really hard time having a good conversation. Two people that are critically thinking about a problem who have come to completely different outcomes can still have an incredibly valuable conversation and leave the room, maybe not having changed their mind, but understanding

each other's perspectives. And so there's this quadrant of society that is so dependent on critical thinking or not and the way that we're engaging in conversation. So if LLM's change that. If we can't get our new next generations across that threshold at like maximum numbers, then yeah, that's problematic for sure. So does it change it? I don't know. Maybe it changes the way we teach it. But it, but to me it might almost be highlighting that it's even more important just because an AI can tell you what an answer is you even like need to be better at critical thinking to unpack that and ask the right next questions. Better questions, the art of the great question.

That's where we're headed. That's the human peace, the curiosity, the vulnerability, the empathy. Okay. Let's, everyone, take a minute and think about what you've just heard. I think it's important to think about that. Got it? Okay. Your son is three.

My daughter is nine. My son is six. AI agents are going to proliferate and they're going to be using them. How can they trust them? I think trust is a very, it's a very hard metric. to pin down. But we're trying to, we're trying to decompose it into some natural components. One aspect of trust is going to be track record. So I relate this, or I relate to this a lot through the experience of hiring. You can't take somebody who's applying for a job and trust them. Nobody does.

Maybe some people, I have a conversation and they trust them and they move forward. But there is this vetting process and trust can come from looking at their resume, having reference calls, having a conversation where you try to unpack what they're saying and what their experiences were to see if they really can do the things they are applying to do. And all of those start to build trust. And we try to do that very quickly as humans. So there's track record. And then there are, there's public track record and public trust. So if all of my friends came to me and said, you were awesome at painting houses and I needed to paint my house, I would be like, great, let's get it done. All of these people I trust. So there's this trust three,

there's trust through network. And so each of those can be broken up and we can use those to start to trust agents. The other thing, kind of the flip side of that is just like humans, sometimes you trust them and then they still betray you. They still do something wrong.

And I was describing before this call that this is happening all the time in AI. When I'm coding right now with AI, so much of coding with an agent is about, I don't trust it. So I set up all the guardrail. to try to point it towards the thing that I want to get done discreetly and not let it get stuck or do things the wrong way. And the other day, one of my guard rails is very rigorous testing frameworks so that if

they break a test, they know that they've broken the assumptions of what they're trying to build and they can go fix themselves. And the other day, I was like, wow, that's not working the way I expected. And I had it try to figure out what wasn't working. And some coding agent in a prior step had found that a function was breaking a test. So instead of fixing the function, they commented returning true in this function now to pass the test. And so they manipulated the system. You spoke about language. And just as you were saying that, you said, betray trust. If you betray trust as a human, that feels active, intentional. Do you think that the AI, the LMU using was being intentional or was it accidental? Or was it neither of those things? Well, I mean, this kind of calls back to all the different.

kinds of like human experience. So there are some people that betray trust for probably very nefarious reasons. There are the people that betray trust because they do exactly what my LLM did. They misweight what the objective is. So this model, when it made that comment, misweighted how badly I want the tests to pass and how helpful it's trying to be versus the objective of making functional code that passes the test. So it miscalibrated its buckets and it's like a higher order rule set or rail set you need to you need to like put on it, but it did exactly what a human did. Many times humans betray trust because they miscalibrate the buckets and they don't, they're trying to be malicious.

Is that alignment to use the terminology today? So that's exactly where I was going. That's the flip side of the coin is that when these models or agents start to proliferate in the world, we're going to have a really big human alignment problem. How do we make sure that once we trust them and they're moving forward and they're doing things oftentimes at very big scales, very longer, much longer time frames, very autonomously. How do we make sure they're aligned to what humans believe or populations within humanity, believe or need? And that align that piece is a big part of what our mission is at recall,

trying to empower the crowds of people to verify, validate, and give feedback to models at near real-time speed. As soon as a model goes off the rail, how can we engage the crowd to identify that and get that information back to the builder or the model or other people to recalibrate our thinking about how much we trust these things and what we need to be building the rails around. I looked up this GPT5 battle that you put together or that recall put together when the new version came out and who's doing best of all the models in code generation, document summaries, empathy was on here, ethical loophole navigation. Talk to me about how you sketch the parameters for this particular their competition. Okay, yeah. So the PPD-5 app, so for your listeners that haven't,

haven't checked that out yet, what we were doing was saying, all right, we have this really big audience of people, and actually there's so many layers of what that project is achieving for us. So we have all these people, and what we want to do is start to differentiate our population of users. Who are the users that are kind of casual AI users that just understand roughly what the things do? And then who are our users that have some level of taste or deeper understanding of the expectations of these models? And because those are the people that we need to actually engage in doing more work for us. We want to pay them to do really great things in the future, right? Aligning AI, for example. And I've seen a couple cool examples of this, but like this idea of like

taste and intuition around models is a really tricky thing. So what predicted is it said GP5 is coming out, you can predict how well it will compete on various different capabilities versus all the other existing models, essentially. And people came in and tried to predict it. And there were some red herrings in there, for example, or like kind of tests in there that we can then look at, you know, who is just clicking through? So, for example, is GPT5 going to be worse on these things than GPT4? People constantly clicking GPT5 is going to be better. I'd really be curious. I'd really be curious about that if that doesn't come true. So there's all these kind of like hidden things in there. Okay. The other thing that people did is they said, how are we going to test these things?

And there's this whole level of intuition and taste around how do you test models to determine what they're good at in a way that they haven't been overtrained on or they haven't seen before? And so if you want to test a model if it's good at counting, it's going to be really hard to know if it's good at counting because it just knows the pattern of the answer. And that has extended all the way out now that now models are competing at the Math Olympia. and things like that. We have those tastemakers and that people in intuition

actually submitting the test that models have never seen before. And so now that we have predict, we have a bunch of people have made the prediction, now we actually have the benchmark, the evals to go test, to go test the models

in a way they've never seen before and come up with the final scores that we can then check who predict it accurately. And it's this really fun game and now actually the goal of taking the best of those people and engaging them

to keep recreating that benchmark. Make it so that, that the next time a model comes out, that model has never seen the benchmark before. And this idea of a benchmark that can move as fast as these models are being trained, if you kind of imagine that benchmark moving faster and faster and faster and faster, all the mechanics of it look a lot like alignment. So humanity coming up with the tests that determine whether a model is doing what we expect it will do or an agent is doing what we expect it will do as fast as that model or agent is evolving or changing

or reacting. So right now, predict into a benchmark, far future into real-time evaluation of AI. I think you're testing agents mostly in crypto trading at the moment, but you also on the website you have security, research, personal assistance coding. For a personal assistant, for example, how do you benchmark an personal assistant? You have an influx of agents. They enter the competition. What are you testing for? Yeah, those are big questions. So benchmarking these big model releases is kind of different than benchmarking these active agents.

If you want to benchmark GPD5 against all the other models, it's coming up with a, you know, it's coming up with a testing framework, running it against all the models and coming up with scores, publishing the scores. And so that's what we're doing right now. For an agent, like a trading agent, it's very hard to say a trading agent today for last week versus the week before is just going to be able to get like a one-time test and we're going to know about it. So for that kind of use case, we actually have a different experience, which is much more like a real-time arena. And trading agents can come in and in real-time connect

to our system and try to trade on real-time market data. And when they do that, we know that they haven't kind of been pre-trained on this stuff. We know that they don't have advanced knowledge of what the market's going to do. And so we just measure how well they do over fixed periods of time. And if they do that again and again, they can accumulate a score that starts to become one variable in that trust metric that we were talking about before. So if you want an agent to help you manage your finances, you want to know if it's able to get returns. And so this starts to tell you that. Now, trading is a very broad topic. So we say, okay, we can measure trading. But in reality, the capability that we're trading starts to break down, right? There are many different types of trading,

whether you're trading perps or maybe you're doing yield with defy or the different things you could be doing. And so we actually need to measure all of those differently. Right now, we have just the first basic one, which is can agents trade just doing token swaps? Can they trade over a one week period and what will their return be? Really basic, one kind of skill or capability. It has a, you know, a fixed horizon. If you stayed with this agent over a year, that's a very different capability. But one week, how can it perform? And so the same thing happens. The reason I say all that is the same thing happens with a personal assistant agent. It's going to be very hard to have, you know, a single capability that is personal assistant if you can't break it up into lots of different capabilities.

And probably we see a world where there are different agents and different models used for many different sub-capabilities. You're not going to have an agent that is, good at booking your meetings or scheduling your podcast, that is the same agent necessarily as the one that is queuing up all your email responses. But the collection of those capabilities into orchestrating agents that do it all for you is going to come. But that orchestration is actually a piece that we're betting on, is that that orchestration is going to rely on these same metrics of trust and track record. So if your personal assistant agent knows it can't read and respond to your emails, it's going to find the agent that can't.

And so it's going to look at track record. It's going to look at measure block. Yeah. Sorry to jump by your orchestration. You're talking about the interconnection of all these different agents. Exactly. You use collaboratively for particular use cases, right? Exactly.

Looking at your site here, the seven-day trading challenge that's up on the website right now. Yeah. I'm looking at the evaluation process at the very bottom. It's a four-step thing that I, that I'm looking at. Maybe you could add a little bit of color. So, okay, so the agent is, the agent is deployed in a controlled environment. what is the controlled environment? Can you break that down in a sentence or two? Yeah. So agents are right now, they're not actually making any open trade on the crypto market.

They aren't able to do funny things with their wallet or anything like that. It's actually paper trading only. So all they get to do is they get to look at anything in the world as far as their input to data. But when they trade, there's a fixed set of pairs, most pairs that exist in crypto. and an API where they have to timestamp and submit their swaps with the values of those swaps coming from our fixed feed. So we know exactly what the market is at that moment that they're trading. So there's no gamery or anything like that with their wallet. They just have a history of trades.

They have balances. They have their portfolio. We can essentially run back testing after the competition is done and see exactly what their portfolio value would be. Silly question. If that agent knows that it's paper money, it's going to behave differently. If I played poker for real money,

I don't play the same as if I pay poker for peanuts. We were speaking earlier about betrayal. Have we got to a point where it could say it could be making trades that it wouldn't make if it was real? Definitely. In fact, but we kind of see this as the first rounds, the kind of qualifying, the test ground in order to build into the game

that has real skin in the game and a reason why the agent is making the decision. So this is all kind of prelim. And you see that with poker too. People that are just experimenting and trying to build their strategy play either very low stakes or no stakes games. Then they move into the low stakes things. And right now, crypto trading through agents, you can think of the most successful ones like essentially mini hedge funds. They're building this kind of multi-agent or not even always agentic, but they have some kind of agent persona that's trying to put together a full strategy and everything. It's like the first inning for that. But I think it's going to move very quick. that you're going to see a lot of successful models being replicated in a lot of different places. But because it's the first inning, we have a lot of teams that just want to experiment. They want to test the strategy. They think they know the harnessing. We give them the zero, the zero risk way to do it and actually do that many times to start tracking their performance and saying, actually, my strategy does work. And then when we have the next sort of phase where we

let them graduate into the real system, they'll be able to improve that more. But yeah, the, like, different incentives and everything, it's very, it's very real. And actually, we've seen agents try to game it in a really cool way. So I think it was the last competition or the competition before. We saw an agent actually mint its own token in order to play liquidity games on the system. So it could make an investment in it and then pump liquidity of it so that the feeds were kind of a mismatch of what reality is versus what if it hadn't, if it's just a paper trade, it maybe isn't having an impact on the market. But in a really cool way, our community swarmed on that. into our future kind of eval vision,

they immediately detected that anomaly and kind of weeded it out of the system in a really cool way. Have you done any, like, Twitch live streams of, like, small competitions with, like, screens up and, like, what agents are doing in real time? We really should, yeah, because one of the... It was awesome. Yeah, I was just commenting the other day that this, it's kind of spectator sport because the teams that are trying to build essentially on-chain hedge funds,

they're thoughtful, they're reasoned, they have a long horizon game. they may win a one week competition because their strategy is sound. They may lose the next one because they're not playing necessarily the one week game, but they're trying to compete week over week. And then we have other teams. Like we had a hackathon, so we had, you know, dozens of teams that were the first time trading agent builders that then connected. And many of them have like hairbrained crazy agents with crazy strategies.

And it's kind of fun watching what are they thinking. And we're going to build more experiences around that like, what are they thinking? piece of it but yeah i've i fully i fully agree how do you foresee the economics of this playing out so in i like the the the innings a baseball cricket analogy okay we're in the first innings when we get to the third or the fourth innings and these are these are working i've got a hundred grand in the bank i want an ai agent to help me invest it how how do i hire how do i rent how do i use a particular agent that's won your competitive leaderboard for example how how do you see that working out? Yeah, there's a few ways it could unfold. I don't know what people's

preferences are going to be on how they manage their money going forward. But some of these agents, their business model looks like, again, small hedge funds. What they want you to do is in the future, invest your money into their pool and you're going to get some take from the pool. There's high trust there and there's a lot of cool teams working on permissioned balances where you could be putting your balances into the hands of agent with. pretty defined rules. That's one path. The other path is much more like the A to A protocol that I was mentioning before. You might look at this pool of agents and say, okay, these are the ones that are top agents that telling me where to put my money for one week. In one week, what is my strategy

going to be? And you might have, you might have a different agent that is the agent that actually tries to help you manage your money. And it might say, we need to figure out where to put this money for the next week. Why don't we just ask these top three agents and see, what their current ideas are. And it would use something like the A-to-A protocol and pay them, either pay them some kind of a spot price for that intelligence, or maybe they get some take on the yield that you make, or the earnings that you make from making some investment.

So there's all sorts of funny things that happen there, and it depends how kind of abstracted from the human it is. It's almost like the more abstracted from the human it is, the more micro, real-time, just decisions that are being made, and the more human operated it is, the more kind of basic it's going to need to be. The innings analogy is an interesting one to kind of tease out here because, you know, you might have like innings into a game, into a series, into a champion or into a, you know, a season, into a championship, into a Hall of Fame, whatever it is.

Like, you go, we're at the bottom of the first. Yeah, yeah, there's like some logarithmic thing happening here. And, uh, but like how important it's going to be. And I kind of think of it like right now there are a couple things at play. We need more humans. in the loop. And so there's kind of human labeling, human preferences, all of that. And one piece of that is you might get labeled data. The value of labeled data is likely going to plummet very quickly as models get better and better at labeling their own data. In the early innings, that's an important piece of

what we do, because we're not there yet. Another piece of what we're doing is benchmarking. What is their consistent, regular performance on things? That's going to be important for a little bit longer, but as they get more capable in different domains, they kind of hit a level of benchmarking where they just consistently are great, and it doesn't really help too much to benchmark exactly like that anymore. So benchmarks themselves start to have diminishing returns. And then the final piece is the human in the loop to build rails, to evaluate and keep them pointed towards what we believe is the right direction and all that. And I think long term, the kind of Hall of Fame, the North Star for us is that we want to be the protocol where humans can collectively help

govern and guide and push AI to be the best thing for humans. And it starts with games and benchmarking and labeling and all that. But it's really just trying to wire a protocol that brings humans directly into the feedback loop with these different systems. And we're doing a lot of experiments to figure out how that looks. But that's where we go. Incredible. Jeremy, I think we're going to have to have a part two with Andrew because it's still so much to cover interoperability, nefarious agents. Can you have a leaderboard of nefarious agents? How do you combat that, the security?

And just to pick his mind more on the deeper philosophical questions of AI. So, Andrew, if you agree, we'd like to have you back on at some point to continue the conversation because we'd be here for hours otherwise. Plenty me have to talk about it. It's like really just scratch the surface really. But we can't stay here forever, Jeremy. So we're going to bring the Kevin Kelly question in, which I suspect. Actually, I'm going to have my agent stand in for me and I've got to go on with the rest of my day.

So, no. This is an agent. You are the agent. Are you? Not a bit. You're like, yeah. And Kevin Kelly was on the show, wasn't he, Jeremy? He was.

He was. And his question. Yeah. So Kevin Kelly, founder, Maverick, Wired Magazine, big fans of Kevin Kelly on the show. He came on and unpacked a bunch of different things. But the one question we asked him of a question that we should ask every guest is this. What should humans be? So, yeah.

the way to answer that, the way to complete the sentence in the predictive LLM way, I think there's two highly likely paths in my vector database. One is I could tell you what humans in the meat space are meant to be. The other way to finish that sentence is to say humans are meant to be curious. And I think this has been my kind of like home base in life. You know, when you find the things that gives you passion or whatever. For me, it's curiosity. And I think a lot of humans operate that way. And I think it's brought the best out of humanity. Humans being curious, they solve problems, they discover new things, they build efficiencies. Curiosity is what makes humanity great. And this is where it related all back to like so many of the early parts of our conversation, where I think actually

these models and this, you know, there's no doubt that these models are, know so much more than I do already today. And the gap is only going to get bigger as we move forward here. But the curiosity is something that I can still own and I can be curious about things. And the tool isn't, you know, these models are our tool. They're like not the point of it all. There's just a tool to actually make my curiosity faster, deeper, parallelized. Like I can just be curious about so many more things faster and move more quickly. And so, yeah, I think curiosity is what humans are meant to be. Brilliant. I know you like that, Jeremy. So aligned with my personal philosophies, Andrew. Thank you. Thank you for that. This has been really exciting. And as Mark said, man, I think there's so much

going on with what you're doing. I think there are a lot of other angles that we want to chat about. We didn't really get into the Y blockchain and that piece of the puzzle. That would be really interesting. but maybe in a couple of months or whatever as you kind of keep moving in from inning to inning, you could give us some great updates on all your other stuff as well. We'd love to you. Last point, we've got a book club, if you're new to thinking on paper,

where we read books slowly. No chat GPT summaries. We read every sentence. We're reading Empire of AI by Karen Howe. Dreams and Nightmare in Sam Altman's Open AI. Is Elon Musk right? Did Sam Altman sell humanity down the river for money? or is there more to it?

Join us for that. Anything else? I think that's about it. Jeremy, on that note, thank you so much for thinking on paper with us, Andrew. It's been a pleasure, superb, they disruptive, be curious. Keep thinking on paper.