CoRecursive: Coding Stories - Story: Ethereum Rescue

Episode Date: May 2, 2021

Today I talk to Dan Robinson about trying to get someone their money back on Ethereum. He's going to be battling this murky world of blockchain high-frequency bots. Along the way, we'll learn how trades are executed on Ethereum and a bit of game theory and political philosophy. It's an entertaining peek into a world that seems like pure science fiction to me, a world where nobody's in charge, where there's no regulation, and where these forces of greed and idealism are in direct conflict with each other Episode Page / Transcript

Hello and welcome to Co-Recursive. I'm Adam Gordon-Bell. Sometime around 2012, one of my co-workers read this article in Wired about this thing called Bitcoin, and he wanted to get his hands on some. And the only way he could find to do this was sending cash in an envelope off to some address he found on the internet, and the person on the other end would supposedly transfer Bitcoin back to him in return when they got his money. I was pretty sure this was a scam, but he was certain that the Bitcoin people weren't just in it to scam people. They wanted to bring forth this new world with this new type of currency. They were idealists and

not about the greed. So he sent his money off and it worked out. He actually eventually got his Bitcoin. Today, cryptocurrencies are bigger than ever, and it still seems like a strange mix of idealism and greed that are driving it forward. Today, I brought in a guest who has a great story about the collision of these two forces. My name is Dan Robinson. I am a research partner at Paradigm, which is a crypto asset investment firm. And I should mention that here I'm speaking on my own behalf and not on behalf of the firm, and nothing that I say during this call is going to be investment advice. That's the only thing I have to say.

Dan is going to share the story of trying to get somebody their money back on Ethereum. He's going to be battling this murky world of blockchain high-frequency bots. Along the way, we'll learn how trades are executed on Ethereum and a little bit about game theory and political philosophy. It's an interesting peek into a world that seems like pure science fiction to me.

A world where nobody's in charge, where there's no regulation, and where these forces of greed and idealism are in direct conflict with each other. It all started out when Dan got a support request on Discord. There's an application on the blockchain called Ethereum. This application is called Uniswap. And full disclosure, my employer Paradigm, we're invested in Uniswap and we're at the time of this. And so I was trying to basically help out with explaining how it would work to people who were trying to use it. So Discord is where a lot of the customer support for crypto protocols happens. And it's actually mostly not people at the companies who are providing the support.

It's mostly actually the community helping each other. And so I was just kind of like trying to pitch in in this way. And in this case, someone had done something that was, it was a fat finger error, basically. They'd accidentally sent tokens to this contract that they shouldn't have sent tokens to. That's not sort of part of the way that you're supposed to use it. So this person, I feel like they need a name. Call them like Susan or Daryl or... Sure. Yeah. Yeah. Daryl. Okay. So what did Daryl do wrong?

Daryl was providing liquidity on Uniswap. And I'm not sure what he meant to do. But what he did was, well, when you provide liquidity on Uniswap, you get liquidity tokens. And those are a newly created token that represents just like your stake, your share of the liquidity that's in the pool. And in this case, what the user had done was sent tokens directly to this contract, which isn't really supposed to receive them. And so my reaction was, you know, like, I'm sorry for your loss. I think these are stuck forever. And, you know, that's a, this is a common thing that happens in crypto. People lose money, fat finger errors all the time.

You know, they'd sent $12,000 worth of tokens accidentally to this contract that they didn't need to. I felt bad for them that they'd done this. But again, this happens in crypto all the time. And so I didn't really give another thought to it until just like late that night. I was thinking about how Uniswap worked, as I often do. And I'm sort of obsessed with this product. And I'm annoyed that I hadn't thought of this sooner. But I was thinking about how it worked. And I just thought,

oh my God, they can actually get the money back out. There is a way. There's an escape hatch. And it's part of how the contract is supposed to work. But this unintended consequence of this feature that the contract has is that someone who did this thing they weren't supposed to do might be able to recover it. So I was super excited about this. Bitcoin. NFTs, non-fungible tokens, a lot of those are on Ethereum, as well as ICOs, initial coin offerings. A lot of those take place on the Ethereum blockchain and give birth to new crypto tokens, of which there's at least 900 types that run on the Ethereum

blockchains, besides ETH, which is the original Ethereum coin. So Daryl was reaching out about Uniswap, which is actually just an exchange for exchanging these tokens. And it works like this. The high level is that anyone can provide liquidity between a pair of assets on Uniswap.

So take like Ethereum, which is the native currency of the Ethereum blockchain and USDC, which is a dollar stable coin that's on Ethereum. And what someone can do is provide equal amounts of these two assets. And it gets pooled with a lot of other people who've done the exact same thing, equal value, like a 50-50 portfolio of these. They all pool it into this on-chain agent called a smart contract.

And a smart contract is just a program on the blockchain. And it's something that can own and control tokens and distribute them and send them around and call other contracts according to its own internal logic. But Uniswap, it can't be upgraded. The contract's on-chain. There's no way to fix a bug once it's in there, which is why we put a lot of effort into trying to write bug-free code on it. When people put in all these tokens, then they get used by other people who want to trade them. So if somebody wants to sell ETH for USDC, they will go to this pool, they'll sell some ETH to it, and the pool will automatically make a market between these two tokens. So there's over $4 billion of tokens right now on Ethereum and Uniswap v2, which is the version that this happened to. And you can

see how actually the trustlessness of it is actually a pretty important feature. Like there's nobody, assuming there's no bug in the contract, there's nobody who can actually just steal that $4 billion. The tokens are there, they're the property of the contract itself, of the code. And so that's simultaneously a very powerful thing. It's also extremely scary because A, if there is a bug and somebody exploits it, there's no way to undo it. And if somebody makes a mistake, it's possible that they've just sort of lost it.

So it's a very harsh and unforgiving environment. So it's like I can lend various currencies to this airport exchange booth. That's right. And they'll do it and they'll market make with your currency on your behalf. And since they're bound in what they can do, they can only follow these rules that they've promised that they can do. It's like they've signed an unbreakable contract with you, but this is all we will do with your money. We're going to do this with your money. That's it. And that again is very powerful. If you wanted to trust, as people have,

over $4 billion of assets to these contracts, you'd better hope that nobody's able to betray that trust. And in this case, there just is nobody who actually can. So Daryl, Dan wants to keep the actual person anonymous, but he or she or they, they had money on Uniswap and they made an error. They had taken these tokens and they'd sent it to the contract. And normally when you send tokens to a smart contract that isn't expecting them, there's just no way to get them out. There's literally hundreds of millions of dollars, yes, stuck in contracts because they just don't have a way to send them. I thought this was one such case.

And what I'd forgotten was, this is ironic because I helped design this feature, but the way that Uniswap actually works, it's somewhat peculiar, but I think it's a single-threaded environment. You're saying all of Ethereum is just lockstep one thing at a time that's right it's a single state machine and each smart contract on it you can think of as effectively an object with a particular interface and its own internal logic and these objects interact with each other just by just by message passing but ultimately all all these calls all these contract calls and everything all happen synchronously. And this has been a big limitation for Ethereum scaling because there's only so much you can do with a single-threaded virtual machine like Ethereum has.

But it's very powerful. It means, effectively, you can, for example, do an arbitrage between two different decentralized exchanges. And Uniswap is the biggest decentralized exchange. It's not the only one. And if there are different prices on different decentralized exchanges, then somebody can come in and do an arbitrage that is actually guaranteed to either work and be profitable or to revert. You can do this atomically. And that's somewhat cool. So you can send some sort of message that is the equivalent of doing two things and they will happen atomically together?

That's right. Well, you can send one message, you get a response from that, and then you can use that response in sending a message to a completely different contract, get a response from that. And then at any point in this process, you can just hit a, hit a button and revert and the whole transaction state goes back. You can always do that. And so like, that's, there's, there's a very like try catch kind of, uh, uh, uh, style that's common where you'll just, you'll just optimistically try to try to do everything. And then, um, I'm forgetting what that's called in Python, but you'll, yeah, you'll, you'll, you'll try to do stuff and then, um, revert if it doesn't work out the way that you wanted. Yeah.

So when a contract calls another contract, it has this synchronous lock on the entire Ethereum state. And so you can do these funny things. And one of the funny ways Uniswap was designed to take advantage of this is that in order to burn liquidity on Uniswap B2, what you do is you just send tokens to the pool. You do exactly what this person did by accident. And then you call the contract and tell it, hey, I sent you these tokens. Check that you got them and send me the money that I got. And because nobody's able to, because this is done as an atomic transaction and done synchronously, this works.

It's secure. Because you send the tokens and then you tell the contract like do this thing with those tokens that i just sent you and it checks how many tokens it has and sends you the sends you the amount of liquidity you deserve back out and so because you have this this lock on it you can do it in this in this kind of weird two-step way and what this person had done was take the first step they sent these tokens in but they hadn't taken the second step and so what should have happened after they sent this in was that anybody could go and call that function on the contract and just say, hey, I was the one who sent that one in. Give it back or give me the liquidity for it.

It's sort of like an ATM. You withdraw your money, but if you don't take the cash out of the dispenser, it's just sitting there for anybody else to take. And it's worse than that because on Ethereum, there's all these bots who are looking around for free money that they can grab. There was nothing at this point tying him to it at all. Anyone could call this contract. And that's what led to this crazy situation that we found ourselves in. The money was put in the change thing and then the person left. And it's like, if anybody else comes to get money, they're just going to take it because it's going to be in the change thing with their money. That's exactly it. It's like, yeah,

you just got the free change in the vending machine, right? Yeah. Because someone accidentally left it there. What do you think the odds are? If there's money sitting somewhere, what's the probability

you can get it? Yeah. There's a joke about the economists, right, who are passing a $50 bill on the sidewalk and one of them points it out and the other says, oh, no, it can't be a $50 bill. If it was, someone would have picked it up already.

And so, yeah, that was part of my thought. And, you know, I think I, as I looked for it, I was fully expecting it not to be there anymore. But nobody had picked it up immediately after this person had sent it. Otherwise, they wouldn't have been asking about it. So it seemed like it hadn't been caught automatically. So there was at least some chance that it had gone through a blind spot of all these bots who were looking for this kind of opportunity. But in fact, I went and looked at the contract. This was past midnight. I sort of woke up in a cold sweat and went and checked this and looked for the contract and found it. And these tokens were still there and

had been there for the past eight or nine hours. So when you thought like, hey, maybe the money's still there, were you excited or scared? Yeah, a bit of urgency, certainly. And this is not my normal state. I spend a lot of my time basically as a researcher, but mostly I sort of have the luxury of sitting around and thinking through problems and trying to sort of imagine how they could play out. And that happens often before this code is launched at all, and then it's somebody else's problem. But the thought process that went through my head first was, so first, okay, great. I can actually call this contract. I can call burn on this. I'll get the money out, and

then I can send it back to the person. The first thought was like, this is great. I can help. And then immediately after that, my thought was, I better do that very fast. Because if anybody else withdraws liquidity, not even intending potentially to get this money from this pool, maybe just accidentally, they'd get the money out of it. So I had to try to do that before someone else did. And then my almost immediately after that thought was, I actually, I can't do that. I can't just call this contract. I'm glad that I'd heard a story about this a while before that made me realize that I couldn't do this because it's this sort of crazy sci-fi concept. But I realized it

was going to be a lot more complicated than I had initially thought. So what's the problem with grabbing the money? So I'd heard this story from this kind of horror story from someone who's probably the top researcher in this particular subfield of cryptocurrency called minor extractable value. And this is a term that he had coined. And so minor extractable value is the idea that there's a lot of opportunities on this blockchain to make money, right? Like one such way is by arbitraging one of these on-chain decentralized exchanges. There's some like more sinister ones, like front running users and sandwiching their trades and pushing their trades and, you know, like basically doing what maybe a less reputable like centralized financial institution might do to its users by essentially front running them. So front running, I understand. So front running happens in the stock market. centralized financial institution might do to its users by essentially front-running them. So front-running, I understand. So front-running happens in the stock market.

People have access to the market at a faster pace than you do. And so they see your order driving to the market in a slow car and they speed ahead and just make the same order. Then you end up buying it from them at an increased price and then they sell it back, right? Yeah, and that's unfortunately pretty common. And pretty common and that's and like decentralized exchanges have to kind of grapple with that fact a block of transactions is mined every 12 seconds and that's when the transactions are executed and that sounds fast but it's an eternity to a fast running bot transactions within each block are executed in priority order based on who paid the highest fee

so if you put it in order to buy something, while it's sitting there waiting to be executed, other things can see it. And if they think that your transaction might move the price of something, they can just put in a transaction ahead of yours by paying a higher fee. So that's classic front running. But in a world of smart contracts, a bot can do something a little bit more strange. What it does is it looks at pending transactions and it tries for every one of those transactions. It says, what if I sent this instead of the person that sent it? There are all these other more complicated ways to make money from calling contracts on Ethereum. And what they're doing is they're piggybacking on other bots or on ordinary users who've spotted some kind of opportunity. They're saying like, if somebody is doing this, maybe there's a way,

maybe they're making money on it. And maybe they're making money on it in a way that I could actually make money on it. And so they just do a copycat transaction. And they can do this, they can just simulate this. They just think, what if I ran that transaction? And if it would work, what they do is they submit their own transaction with a slightly higher fee. So it appears ahead of the one that they're copycatting, and they take that money. And so that's this generalized front-running bot, where instead of caring at all about what it's actually doing, it's just blindly copying anything it sees. And this can be quite sophisticated. It isn't just the transaction itself, but within the transaction, one of the contracts of the transaction calls might make a

call to another contract. These bots can actually introspect deep into the call stack and see, oh, this call is happening. What if I made this call, not just the whole transaction? And so it's actually quite hard to obfuscate your transaction in order to avoid this. The nature of the problem is that everything is transparent, if I'm understanding. Because all of these contracts, you can see what they do, right? And all of these requests, you can see what they will do. So you're saying there could be something that just simulates what's going to happen when you do something and then just does it itself instead? Yeah. And it's amazing because it's a deterministic environment on-chain. But yeah, anyone can just simulate

the effect of a transaction. And obviously, this is way too fast for any human to make decisions based on it. But the program can just shoot off a transaction as a result of seeing something like that. So what's cool is that on-chain, there's this very efficient market for taking money, free money, that's there.

And money won't last in a smart contract for very long if a smart contract can be hacked. But this money had been there for eight hours and nobody picked it up. But my worry was as soon as I tried to pick it up, that just the act of doing that would cause it to be taken. And so that's an even more hostile environment than the blockchain, which is already ridiculously hostile. This was all a theoretical problem to Dan. But if these monsters did exist, there would be no better way to attract them than to try to grab Daryl's free money back.

Maybe I'm like a predator in this environment where, in this case, I was a white hat hacker, but maybe I'm a black hat hacker. Maybe I'm trying to take money from a smart contract. And I think I'm so smart. I put a bunch of effort into figuring out this way to steal this money from it. And as soon as I go to do that, just this whale comes up from the deep and just devours me whole. I think I'm actually at the top of the food chain and I'm not even close. And so this, this like, that's the metaphor that just immediately came to mind when I heard about these, these things. And I'd never seen one. I never, I hadn't seen like a, like, that's the metaphor that just immediately came to mind when I heard about these things. And I'd never seen one. I hadn't seen, like, an example of it actually happening.

But I'd heard about them, and it sort of made this deep impression on me that this was something to be feared. So it's like, even if you're able to hack something, like, you probably won't get the proceeds. It's a kind of, like, very uneasy mutually assured destruction, where maybe nobody can take this money because they know that it'll alert these bots. But ultimately, someone finds it and then it just becomes just pure war. So there's this sort of peacefulness to the blockchain state until suddenly just an outbreak of war and everybody's trying to bid to get their transaction in first. To build one of these bots, Dan says you'd have to run a lot of Ethereum nodes. You'd have them sort of all over the place, and they're directly connected to all the miners, and they're listening for all transactions across the entire network. And then whenever they see a

transaction that they want to front run, they'll spam everybody with transactions that go ahead of it. And they'll participate in this auction where if then the other person tries to go ahead of them, they'll keep raising the bid and the amount of the price that they're willing to pay for gas in order to be first. And so this is, you know, that's sort of a complicated process. And they're fighting something that, yeah, I'm like, I would have no chance going toe to toe with. This was after midnight in Dan's time zone. And he hadn't told anyone, this wasn't really the type of thing you want to advertise, but he knew he was in over his head. So he put together a team. The first person that I called in was my paradigm colleague, Georg advertise. But he knew he was in over his head. So he put together a team.

The first person that I called in was my paradigm colleague, Georgios. And he's a much better smart contract engineer and general developer than I am. Like I'm fully aware now that my strengths lie elsewhere. And he's sort of a fantastic programmer. And he fortunately lives in Greece. And so he was on a time zone

that he could actually help with. He was awake and could help with this. And then another European developer, Alberto, who works at another portfolio company of ours, also pitched in to help write the contracts that we were going to write for this. And then there were a few Ethereum smart contract security engineers that I called in. This was on Telegram. It was an Ethereum smart contract security group. That's like 500 people.

I just basically asked anyone the question, like, should I worried about these these generalized smart contract front runners and these are the two people who responded yes so some people some people didn't believe it but uh they were the ones that i that i pulled in um to help but uh but yeah so so sort of put out the bat signal for these for these uh and uh security researchers and they helped me come up with this plan. Was it because you thought this was like ludicrous? Like, did you feel like, Hey, this, okay. I heard that there's these monsters under the ocean, but do they really exist? I definitely felt a little silly even asking about it. And then we go on to basically do this like really convoluted scheme to try to hide what we're doing. And the whole time I'm like, and like, you know, taking, taking a few hours to do this. And the whole time I was just like, am I,

am I crazy? Like, does, does this make any sense? This is sort of, it's sort of like, like being, being weirdly paranoid. Cause I had never seen any, anything like this happen. Cause it doesn't, it does not sort of like an everyday Ethereum thing where there's just sort of free money sitting somewhere to pick it up. And so I just sort of just had... I knew I'd heard about it and also just sort of had this general game theory sense that like this... If someone can build this bot and if they can, then they would. And so the bot is out there.

Yeah, it's the $50 on the ground. Well, in this case, it's me thinking like, well, would it be safe to leave my $50 out in this case it's me thinking like well is it would it be safe to leave my fifty dollars out here on the street and thinking like no no i shouldn't well and and like also um isn't the clock kind of ticking because the money's also just sitting there so we were under this time pressure because a somebody could have spotted it b somebody could have just taken their own liquidity out and accidentally gotten it and either way we would have been out of luck. And so there was this time pressure.

And while we were doing all this, we had to write custom smart contracts for this crazy obfuscation plan. And the whole time I was thinking, I'm going to feel really stupid if we waste all this time doing it and then someone just goes and picks it up. And we could have done that the whole time. So there was definitely part of me that was thinking, I'm glad we're being careful, but I feel a little paranoid during it. Rescuing Daryl's money makes for an exciting story. And the ending is wild, so make sure you stick around to the end. But there's a much bigger theme here than just somebody's misplaced money. It's this tension between coordinating and competing.

On Ethereum, even miners who execute transactions could lose out by trying to help somebody get their money back. When you just think about the game theory of it, even if there is a benevolent miner, they're going to get out-competed by the malicious one. Because the malicious miner can make more money from the blocks that they mine. And that means that they can spend more money on electricity, they have a lower break-even. And so they can just out-compete these miners. Ultimately, mining is a very competitive market. Unless you're honest and you have some other kind of sustainable advantage and a proof- improve a work network like Ethereum,

ultimately any bit of advantage that you can get, other people can get and they'll out-compete you if you don't. Honestly, competition is the worst. Like it's a scary world. Honestly, I say that like Ethereum is super competitive. It's nowhere close to how like traditional finance has just been like very ground down.

There's still a lot of low hanging fruit. We're in this grace period where there's still time for experimentation. There's still sort of space for relatively mediocre people like me to sort of have an impact before everything gets just ruthlessly optimized. Do you know this Meditations on Moloch?

Exactly. Yes, this is exactly Meditations on Moloch. Moloch is this fantastic blog post written by SlateStarCodex. And I recommend people check it out when they're thinking about this general category of just sort of game theory actually being a very scary thing. And the idea is that the condition of perfect competition, and this could describe like an efficient market in the economy, or it can describe as a situation of total war. It's just a terrible situation to be in from the inside. It can produce great things like the competition does, like it's fantastic for consumers

when it happens in the economy, but as an experience, it really sucks. And it doesn't leave you a lot of slack. Like it doesn't leave you room to sort of, for art and other things, if you sort of always competing to survive. And the issue is that just like ultimately any entity that doesn't just maximize survival and getting as much as possible

in a world of scarce resources could just get like, could end up just being out-competed by those that do. And this is actually sort of like also what Thomas Malthus thought. And one of our goals, and like this is something that we at Paradigm care a lot about, is trying to defeat Malik, is trying to prevent that kind of race to the bottom in whatever way possible. Yeah, but the thing I got from the essay

is that maybe it's impossible, right? In a system with people competing against each other, being nice has a cost, and you can be as nice as you want, but you will lose. I think the point of it was that regulation is important and needed

because without it, everyone just claws each other into smithereens, right? Yes. So what do you think of that? Are you concerned that you're building something that, well, interesting is just like a vehicle for, I don't know, sidestepping things that might be important, like rules about money laundering or... Yeah. And that is a great question. Which side ultimately of the Moloch problem is crypto on?

I think... So starting with the point about regulation as a solution to this, I think that's definitely correct. And in fact, it's not exactly historical, but philosophically, a lot of regulation is rooted in this concept. And so Hobbes, before I even went to law school, I was a government major in college. Hobbes had this concept of the state of nature being a war of all against all. Nasty, brutish, and short. Exactly. Life is nasty, brutish, and short. And that's sort of a Moloch-type situation, is when everyone is just fighting everyone else for every scrap. Hobbes' solution was we raise

some Leviathan above everybody else. And for Hobbes, it was just like the monarchy, hereditary monarchy specifically, right? Because then you just don't have a struggle over who the next king is. We know, oh, it's that guy, right? And the idea is by doing that, you've now removed the entire, the problem of like, oh, we're going to struggle about which one to use, who to listen to, who has power. And we just say like, all right, that one guy has power and everybody else has descendants. And it's unambiguous. And the advantage of that certainty and the advantage that this person can like just say, oh no actually, you guys don't fight in that way or don't fight about this particular thing.

This solves some of the problems with competition. And I think there's really something to that. Civilization sort of arose because some people seized massive amounts of power and used it to stop everybody else from competing on stuff, right? I think he's getting at this idea that like even a horrible dictator is is better than having nobody in charge that's right so that

that's that's the basic philosophy is anything is better than than the massive root of short life and so like here we are and this is the only solution and i think i yeah i in college honestly i thought that was i was i was sort of a hardcore libertarian like i thought that was that was i hated that concept i now think it's at least directionally correct that like, actually that is a role that the state plays is in preventing this, you know, not, not just like, yeah, I don't know, not even just in the philosophical sense, but like really like, uh, and then this core way preventing everyone from just being competing with each other all the time for as much power as they can grab. And that there's a benefit to not having conflict, even if it involves that sort of world.

But still as something of a libertarian, I hope there's a better way. And so the alternative possible Leviathan is just, well, democratic government, right? And I think I happen to believe that democratic government is a big improvement on total dictatorship, even though it has these failure modes. And one of the objections to democratic government was that you're actually, you're leading to just populism. You're leading to another kind of despotism. You're leading to just people are going to achieve power through some other means. And then there's now there's more competition for it. And now we have competition again, and that's bad. So democracy is still an experiment.

Hopefully, I think it has been working. We'll touch and go for a bit sometimes. But I think it seems to be working. And then the next, this is, I don't know, this is quite grandiose to put this in that progression. But the idea, I think, of smart contracts, hopefully, is this is another way for us to come together and coordinate our actions that doesn't require us vesting infinite power in a single fallible human. And that's that you can just say like, we're all going to sign sort of a constitution. We're all going to put our money in a smart contract. I feel incredibly cheesy putting it this way. And like my law school professors

would probably smack me down, but we're going to, we're going to invest all of our, all of our power into this, into this code. And then just, it's. And then it's going to keep us all honest, right? In other words, there's no struggle over who's in charge and who can undo a mistake because everybody knows that no one can. People can lose money with fat finger errors, but also people can use smart contracts to build cooperative market-making services.

It's a really wild idea. And when people do make mistakes, Dan and others like him can try to help out and try to rescue the money. In Daryl's case, the key to the rescue plan is obfuscation. Because ultimately, you know, this is all deterministic and everything we were doing was going to be public because we just didn't, we didn't know a miner to send stuff to. There was no way to sort of like solve this like cryptographically. We knew that basically by whatever we did, it was going to like, that would, that would get this money out. It would be possible

for somebody to, to spot it and then intercept it. And we hope we would just make that as difficult as possible. So what we did was we split it into multiple steps. So for example, creating a contract that would do this call and then calling your own contract, even if your own contract, like would only, could only be called by you, wouldn't work. Because if this contract made the call, somebody can run it, see the execution trace, see that internal call, copycat it, and make the internal call. So that wouldn't be enough by itself. So you're saying something could see you create the contract

and call it, and then they would automatically create the exact same contract, but that pays it to them? No, no, no. So every Ethereum transaction is sort of just a call stack of contracts calling other contracts and pouring some logic. And that call is just like a message

passed from one contract to another. And so somewhere in this transaction in which we got the money, some contract, no matter how much we put in there, some contract was going to make a call to the Uniswap contract that would get this money out. And we just couldn't avoid

that. And so if somebody simulated, if our transaction was going to work, then it was going to have to basically reveal that this call was in there. And someone could look and pluck this particular call right out of the call stack, simulate it, copycat it, mutate it, and run it themselves. So again, this is why it's not a perfect solution, but we were hoping they're just not sophisticated enough to spot it. So just doing our own contract wasn't enough. We split this contract into two contracts. There's a setter and a getter contract, and we split it into two transactions,

a setter and a getter transaction. And so there was a lot of just indirection here. Basically, the getter contract was not activated. It just couldn't be called until someone had called the right function on the setter contract was not activated. It just couldn't be called until someone had called the right function on the setter contract, which could then call the getter contract and activate it. Once the getter contract was activated, we could send a separate transaction to the getter contract that would then go make this call to Uniswap. So what we did is we submitted the

getter transaction in the same block. So they would look to anyone just running these transactions independently. There'd be nothing to actually link them. They'd be two transactions from two different addresses to two different contracts. And if you ran them separately, it wouldn't actually do anything. It wouldn't touch the Uniswap contract. It's only if you run first the setter transaction and then the getter transaction that the getter transaction would then call the Uniswap contract. So we were hoping that what these bots were doing was they'd simulate

these transactions independently. They would just run them against the current state before this block, rather than, say, trying to construct a candidate block out of the transactions of the mempool and running them all. Because if they did that, then we'd be toast. But if they were just looking at these transactions one by one, then this could obfuscate it. So we tried this. And I don't know, at least for me, the time pressure was getting to me at this point. And we were just having trouble basically submitting these transactions, in part because the second one looks like it was going to fail until the first one was included.

And just like Ethereum infrastructure is designed to kind of protect you from submitting transactions that are going to fail. And so we actually just had trouble getting these into the same block. And we had to like try it and then reset it and try again. And ultimately I was like, forget it. We're just submitting them. Let's do it in the same, in different blocks. We'll do it like, we'll do the setter transaction and then the getter transaction in the next block. And we'll just hope it works. And that was what turned out to be this critical mistake. The second transaction finally succeeded. So we'd done a few tries and it's like, oh, thank God, it actually worked. And we looked at the address that we were going to send the money to and it hadn't gotten anything. So we looked at the transaction that had succeeded

and it said there's no money there, which means someone had already gotten it. We looked and there was a transaction just like a few seconds after ours had been submitted. Someone had created transaction that that would take it and what happened was we got front run the monsters are real before this happened you probably thought well there is a story here the story is going to be how i like wrote in with my white hat and save the day right yeah yeah afterward i was just like i'm too embarrassed looking back on it and and this feels very insensitive about the person whose money was lost. But in retrospect, it would have been a much worse story if we had been able to get the money out. Like, to some extent, it was slightly vindicating. Like, I'd been feeling paranoid this whole time that there were these crazy monsters. And if the story ended with, and then we got the money out and it was fine, like, we could act act victorious, but really it's just like, was there ever any conflict? Was there ever really any drama in this? Dan thinks of these battles in terms of game theory.

In an iterated prisoner's dilemma, people can learn to collaborate as long as the group stays kind of small. If everyone's playing tit for tat, then as soon as you start getting some defectors, yeah, I think I previously thought, oh, minor extractor of value is like an interesting problem in theory. Like a lot of these game theory games just aren't actually going to be played for a while. And I think we're starting

to see the incentives are getting such that actually like there's real money at stake. The Ethereum community talks about solving these problems using shelling fences. Shelling fences are sort of a fence you put in front of a slippery slope to prevent everybody from sliding down it. But in the early days, enforcing norms with shelling fences was a lot easier. We were kind of in the Garden of Eden early on. And this is true of Bitcoin mining too, where you could mine Bitcoin on your computer. And there was this nice spontaneous order where people just wouldn't exploit every possible way that they could exploit things. And we're seeing one by one

these, this is a bummer of an answer here, but we're seeing these kind of like shelling fences that would prevent us from collapsing into total war of all against all kind of collapsing faster maybe than I would have expected. And that's where things stand right now. Dan and his colleagues are actually looking for ways to prevent a further race to the bottom towards outright bot warfare. His company invested in Flashbots, which is a pirate hacker collective focused on this very issue. Yeah, that's right. They're a pirate hacker collective. Nobody can claim that the cryptocurrency space is not very interesting. So that was the show. I'd love to hear what you thought about the episode. If you aren't subscribed, now is a great time to do that. If you really liked the episode,

maybe think about sharing it in your favorite Slack or Discord or Telegram channel. Until next time, and I mean this sincerely, thank you so much for listening.