Epicenter - Learn about Crypto, Blockchain, Ethereum, Bitcoin and Distributed Technologies - Dan Guido: Trail of Bits – The Evolution of Smart Contract Security

Episode Date: June 30, 2020

Just like all software, smart contracts on the blockchain are subject to serious security vulnerabilities and coding errors. The fact however that smart contracts are often directly in charge of assets and cannot be changed once they are on the blockchain, makes secure development and running essential. Some smart contract platforms have their own languages, for example Solidity in Ethereum. Bugs and vulnerabilities in the source code, and errors in the virtual machines used by the network, are the main reasons behind security issues in smart contracts.Projects using blockchain applications should expect constant changes in the security landscape. New bugs, security risks, and best practices will continue to emerge over time. Trail of Bits is a software security firm who advise in a range of industries for some top companies, including in the blockchain space. They are experts at identifying top-level risks and implementation vulnerabilities, and providing essential recommendations on best practices. Dan Guido, the CEO and Co-founder, explains all things software security in a really detailed and technical, yet easy to digest way. We also recommend you check out their exceptional blog packed with invaluable resources.Topics covered in this episode:Dan’s background and how he came to create Trail of BitsWhat led Dan into the blockchain fieldHow security software has changed over the last 20 yearsThe unique challenges for security on blockchain and smart contract protocolsSmart contract languages and securitySlither - Trail of Bits’s suite of Ethereum based security toolsDan’s opinion on Solidity’s future and Vyper as an alternativeFormally Verified LanguagesA use case on how Trail of Bits worksWorking with upgradeable contractsComposability and securityAre compilers trustworthy?Other security issues in the blockchain space as DeFi growsThe future of software security and the role of AIEpisode links: Trail of Bits WebsiteAnatomy of an Unsafe Smart Contract Programming LanguageSlither, GithubSlither: The Leading Static Analyzer for Smart Contracts246 Findings From our Smart Contract Audits: An Executive SummaryRapid Risk Assessment (RRA)Our Full Report on the Voatz Mobile Voting PlatformA Guide to Post-Quantum CryptographyBSides Lisbon 2016 - Keynote - The Smart Fuzzer Revolution by Dan GuidoThe Smart Fuzzer RevolutionTrail of Bits TwitterDan Guido TwitterThis episode is hosted by Sebastien Couture & Friederike Ernst. Show notes and listening options: epicenter.tv/346

This is Epicenter. Episode 346 with guest, Dan Greedo. Hi, welcome to Epicenter. My name is Sibbisankwuchuo. Today our guest is Dan Grito. He's the CEO of Trail of Bits. They're a software security firm that has been around since 2012, and they work quite a bit in the blockchain space, but not only. They also do work in the sort of broader software security space. They work for government institutions as well in the U.S. They're very good at, I mean, one of the things things that I love about Trail of Bits is that they're great communicators. So they have an amazing blog that just captures a lot of the work that they do. And so if you want to learn about

software security, I think like their blog is a great place to start. They also have a publications repository on their GitHub where they publish all of their security audits and also all the research that they're doing. So they're really sort of a great thought leader in the space of blockchain security research. And I've been one to get Dan, on the podcast since about a year or now. I mean, they were previously a sponsor of the show, and we're supposed to do this interview at ECC in Paris. Obviously, that didn't happen because Dan didn't come over. So it was glad to finally get him on the podcast. And it was really great conversation. Frederica and I did this one together. And Dan is just so articulate and explains things

with lots of clarity. Also, you know, in a way that for us, software security laymen are able to understand. So after the interview, Frederica and I went on for an extra 10 minutes talking about our thoughts and what we learned from the interview, also describing her experience with security audits, obviously, you know, working agnosis. She's had to deal with that quite a bit. And we also discussed the potential for more gatekeepers in the DAP ecosystem in order to create a safer space for people to invest their money and use DAPs that have been vetted by gatekeepers. So if you want to hear of that conversation, conversation, please subscribe to Epicenter Premium. You can do so at premium.com.com.

That gives you access to host debriefs, where we go on for an extra 10 to 20 minutes, share our own thoughts about the interview, also roundtable discussions, and bonus content that we'll put out from time at time. So with that, here is our interview with Dan Guido. We're here with Dan Guido. Then, thanks for joining us today. Thanks for having me. Actually, we've been wanting to get you on the podcast for some time. Actually, we were supposed to talk at ECC and you didn't make it. I think that was probably a wise decision of Vute not to fly over. Yeah.

Turns out the professional risk manager knows how to avoid certain downsides. Thanks for joining us. I mean, you know, we've been talking for quite some time also because our listeners remember that Trillipitz was a sponsor of the podcast about a year ago. And so it was great to get to talk to you then also and kind of get into this topic of security. And so this is an episode that I've been wanting to do for a long time because we've obviously talked about security a lot on the podcast, but never really done any episodes that

focus on security. And there's a lot of things here. I think that we'll learn and also maybe be surprised about like some misconceptions and things like that that I'd like to get into. So yeah, tell us a bit about your background and how you came about founding Trill Fits. So I've been doing security stuff for the better part of my adult life. Started probably when I was about 13, 14, 14, breaking into my school computers like one does, as one does. Luckily escaped being severely punished for that, but I ended up going to college for a concentration program in cybersecurity. It was called Polytechnic University when I went there, but now it's the NYU Tanton School of Engineering. And they have one of these NSA Center of Excellence programs

that teach kids a formalized education in cybersecurity. I think the people that are a little bit younger than me have a lot more formalized education and people that are a little bit older than me don't. They learned from their peers. They learned in kind of like a master or apprentice kind of set up. So I'm right on the cusp of that. So I have a formal background in computer science and computer security. And this is the only field that I've ever been interested in working in. So I've worked at the Fed Reserve doing incident response, helping prevent people hacking into the currency reserve for the United States. I've been a consultant at ISIC partners, now NCC group. So I was at ISIC partners before they were required, helped start their office on the East Coast, worked with dozens of technology

companies across the globe. But I was pretty frustrated that it seemed like an unending treadmill, that you kind of go back to clients year after year, and there's always the same bugs. And they don't really internalize the information that you give them. I thought that there was some improvement that we could make. And I wanted to make fundamental improvements to the whole field. So I found a trail of bits with two friends of mine back in 2012 to fundamentally advance the science of computer science and computer security. I think we've by and large succeeded at doing that in very small ways. You know, the company started as a DARPA contractor. We worked on four-year-long research programs in automated program analysis and advanced cryptography. And then from there,

we've branched out to help provide those advances to commercial firms and now to blockchain firms. So that's the, I guess, the medium-length overview of where I came from and what we're doing now. Tell us about how you got interested in blockchain as a cryptographic field. Because, I mean, basically, you found a trade of bits in 2012, and obviously then it was still pretty new. So what exactly spoke to you about it? Yeah, a couple of things. I think it was really driven by employee interests. There were about two or three people in the company that were just really enamored with blockchain as technology,

because it was a green field, not necessarily because it was anything that you could do with blockchain, but because the field was in its infancy. It was a chance to start over. It was, you know, there were no security tools. There was no security knowledge. People were building their own programming languages, building their own compilers. The execution environment looked a little bit different. So there was this huge gap of knowledge that we could rush in to fill and create things that were correct from the first step. So back about three to four years ago, we had a couple people dabbling in that area of technology. and what we contributed was a symbolic verifier. That was our very first thing. We didn't raise our hands and say, hey, we'll audit your code for you. We're engineers. So we set up a little

unit of people that wrote symbolic capable Ethereum virtual machine into a tool that we have called Mantecore. And then once we were able to do that, we realized that, hey, this is actually kind of valuable and people would love to work with us to improve their own security. So because we'd already mastered the field through that activity, that research activity, that's how we started offering services for blockchain. How much of your client base today is blockchain base? Because you told us just before we started the call that we should update our versions of Zoom because you just released a fix.

So how much is basically in the web two word and how much is in the Web 3 word of your company clients? Trailbits as a company is split into a bunch of different pieces. We have a research team that continues to work with DARPA, the National Science Foundation, Department of Defense, like various other agencies of the U.S. government to fund long-term research. That's probably about 10 to 15 people, depending on how you count. We have an engineering team. So because we have these expert software developers that can apply research and make real software with that research,

we also have an extraordinarily capable team of people that you can hire to build software for you. It doesn't need to include research. It just needs to be something that you need. Usually it's security software. Sometimes it's high assurance software, things like cryptographic libraries. So our engineering team is about another, you know, 10 or 15 people. And then we have an assurance team. And our assurance team is who the blockchain industry typically collaborates with.

There are people that are specialized at taking owner or taking a look at other people's code and helping identify hotspots, weaknesses, and specific security risks inside them. And that team works across the defense tech and finance space. So we've reviewed things like autonomous drones, satellites, cryptographic protocols for satellites. We've looked at embedded firmwares, self-encrypting hard drives. And most famously right now, we're working with Zoom. When people have a need to really rapidly get to the root of difficult problems, especially ones that involve compiled code and cryptography,

Trail of Bits is among the top choices for that assistance. So blockchain right now, I'd say, takes up about like a third of the company's business, maybe a little bit less, maybe 20%, but we have a really healthy, diversified business. and we work across a lot of areas of technology. And I try to shuffle kind of things that we've learned and the tools that we've built and the talent that we've acquired across all those different industries, which is what makes us kind of a unique company in the blockchain security space. So it's interesting that you work with Zoom because they just bought Keybase and brought on that entire team of cryptographers.

So you're saying that they bought Keybase, they got all these cryptographers, but they also need to get extra cryptography experts in addition to that team? Oh, yeah. I mean, look, their code base has been evolving over the better part of 10 years. It's got hundreds of thousands, millions of lines of code that's cross-platform with Linux, Mac, Windows. They even have a Blackberry client. There's server infrastructure out there. It's a custom network protocol, custom cryptographic protocol. There is just a voluminous amount of stuff there. And when you have that level of scrutiny on you, as a firm like Zoom in today's environment, you need to really apply a lot of effort to uncover flaws before other people find them

and to keep your clients safe as best as possible. So we've done a really detailed review of all the client software. Other firms were brought in to look at all the server software. But a really key part of our responsibility was to look at the existing implementation of end-to-end cryptography. We've helped them kind of analyze the current state, figure out, where it's hooked into the client, how it exactly works, what the kind of cryptographic properties of it are today. And then the Keybase folks are working a lot more on what it should be tomorrow. So a lot of the initial value that the Keybase folks provided is a white paper that's now published

up on GitHub that I think a lot of people have read. And that's great. And that proves kind of like where things should go, but it's going to take weeks, months, you know, possibly even longer to get there. And in the meantime, you need to make sure that the software you've got today is safe, which is what I'm working on. So in your career, you've been working on on security related stuff since you were in your early teens. How has software security changed the most in your view? What's been the most fundamental shift between, say, how software security was looked upon and conducted like maybe like the early 2000s to now? I'm kind of jaded. I don't think it's changed really at all. There's been a lot of movement between programming languages. Like everyone jumped on the No.js

bandwagon at some point, and then everybody jumps on the Rust bandwagon, and everybody jumps on the swift bandwagon. And it's history repeating itself to a great extent. Like, we saw this with the iPhone and Android, right? In the first versions that came out, they didn't really apply any of the best practices or lessons learned of the previous generation of software. They weren't sandboxed. They weren't hardened with compiler protections. They didn't have a great user experience. Like, there just wasn't a lot there. And some of those early mobile phones were some of the easiest software to hack on the planet. But now, you know, they took all their punches and they had to learn from failing, which is kind of unfortunate. And now an iPhone is kind of the most secure consumer device

that you can buy. There's not a single device that I could go to Best Buy and say is more secure than an iPhone. And I think that's kind of the process that a lot of fields of technology go through for better or for worse. So really what it means as a security professional is that you've got to learn this kind of fundamental underpinning. There's a lot of like foundational security knowledge that you need. But the technology that you work with from day to day over the course of your career is going to shift pretty rapidly as things get to be less mature and more mature. Right. So it's more about technology changes so much as it is about changes to like how you secure a software or hardware. The risks are the same or fundamentally the same, but the tools that people are building to use the infrastructure are changing. I mean, on any given day, our assurance team is working with people that write code in, you know, Java,

P-HP, Ruby, Rust, Rails, Django, various C-Frameworks or C++ frameworks, and blockchain software and like everything under the sun. And everybody just kind of expects you to know that stuff. And really like learning how to code in C or learning how to code in Rust is not the hard part. People challenge us to do that all the time, which is why the whole blockchain thing also comes kind of naturally to us. Like if you're a good security expert, then you're going to be able to figure out exactly what's going wrong in a smart contract too. The most important part is the security expertise, which is something that you have to learn and grow over, you know, a decade or a lifetime or a full career.

If you look at blockchain technology and smart contract technology, are there unique design and security challenges in these protocols? The unique security challenge is really that the tools that people are using to build blockchain software are awful. What changes between different languages and frameworks is that sometimes certain tools have a lot more footguns in them than others. So it's a lot easier to write security vulnerabilities in PHP than it is to write security vulnerabilities in Rust because there's a lot of safety to fall back on in terms of the

language and the compilers and the development frameworks. But with solidity especially, not just solidity, but many other blockchain languages, they've managed to reinvent the entire history of software security flaws and make it the developer's responsibility to be aware of all of them. So really the key difference here is that developers are not getting any help at all from the tools and the language that they're using to develop blockchain software, which makes the after-the-fact assurance that's provided by companies like mine critical. Now, on the other hand, what blockchain has that's going for it that other languages don't

is that smart contracts tend to be a lot more testable code than typical, traditional code. It's really easy for me to throw it into an emulator and for me to send inputs to any given function and to evaluate what the impact is on a really simplistic state machine. But on regular software, like if I want to set up a fuzz test client for Google Chrome or something like that, there's a lot of global mutable state. There's a lot of libraries that change their behavior depending on what operating system you're running on or what specific configuration of your operating system you have. That's why, like, you know, when your parents call you and some software on their computer crashed, you have no idea how to debug it because you have to be aware of every single setting on their entire computer, all the other software. that's installed alongside it, all the shared libraries that are there. But on the blockchain, everything runs on the same blockchain.

So if I have a copy of the blockchain and you have a copy of the blockchain, then it means that we're testing and we're running the same code. So that's actually made things a lot easier for blockchain software, despite the fact that there's incredible insecurity in the tools that you're using to build it. Can you give us an idea of what a mature tooling kit should look like? So basically, if you look at languages that have been around for a while, like Rust, for instance, what ideally would we have in terms of tools in order to be able to have the same trust in the tooling as you do in traditional systems? Yeah, so it's not a question of third-party tools.

It's more a question of the language and the compiler itself, right? Like everybody's familiar with the fact that solidity can't seem to make it more than one release without a critical flaw. Everybody's also familiar with the fact that tools like Slither, you know, one of my own static analyzers from Trail of Bits, have to watch out for 90 specific flaws or that the use of integers is inherently unsafe inside solidity. And you need a third party library like Safe Math in order to catch you when you fall. So the kinds of things that make a language safe are the avoidance of those issues that, you know, I shouldn't have to worry about the order of operations because the order of operation should be the same as every other language that I use. they shouldn't be different like they are in solidity. That's a big part of the language design. There just should not be so many foot guns.

There should be, right now, I think that there's kind of a pressure to make solidity more featureful to add additional kind of things to it that help you develop code. And as a security engineer, I'd pull back. I would want to make something that looks like solidity minus minus, not solidity plus plus. I want something that's a little bit more simplistic that fits 99% to people's use cases, but that 100% of those potential uses are safe and that it's much, much harder to screw up. That's why you see, you know, Rust, things like memory safety, type safety. It takes the universe of things that could go wrong and limits it, where now I just have to worry about logical flaws.

That's much better. Other things that, you know, tools like Rust have that are really nice is that it specifically declares when you're doing something that's unsafe. It makes it easy. So like I, as a security engineer, the most precious resource I have is time. When I'm looking at a piece of code, it might be larger than the amount of time that I have to fully review all of it end to end. If I could specifically identify like, hey, this part of the codebase is insecure because the developer told me so because the compiler forced them to, then I know where to spend my review effort to get the most bang for my buck. So Rust again is really good there. It helps identify ahead of time what you're doing is unsafe. All those are features and capabilities that solidity in particular,

lacks, but that also aren't present in most other smart contract languages. To kind of summarize here for the laymen's in the audience, what you're saying is that solidity has sort of inherent issues that make it unsafe because as a developer, you're able to do things in solidity that in other languages would just be, you know, impossible to get wrong or the compiler or your tooling would alert you that what you're doing is unsafe. Yeah, absolutely. We actually have a whole talk about this. We gave a conference presentation called Anatomy of an unsafe smart contract programming language. It's up on the trail of its YouTube. It's really entertaining. It's got tons of great graphics and a little bit of some movies and a jingle in the middle of it. So if you're interested in looking at it further, that dives into the specifics of why we think solidity is a poor choice that we're kind of stuck with now for developing smart contracts. But again, luckily, what we've proven is that over time it is possible to build. secure solidity smart contracts because they're more testable. Because it's easier to do after the fact

verification of them means that you can still produce secure code and possibly even more secure code than if the compiler tools helped you. But it's not a question of either or. Like obviously, why not both, right? You shouldn't have one set of tools undercut you and then your testing tools lift you back up. And that's kind of the struggle that I always have in the blockchain in communities, I would love to work on the compiler tools. I would love to improve them. And the things that we've done with Slither specifically to formally kind of model things like the Yule parser and the AST for solidity are things that should be adopted upstream. We have all of these testing tools and everything that allow us to, in the end, essentially write

secure smart contracts, right, because we're able to catch all these bugs and we're able to fix all of these little inherent issues with solidity. Does this introduce other challenges or other security vulnerabilities that, you know, perhaps another language like Rust or a more matured language like C++ wouldn't encounter? Like, what are some of the negative effects, perhaps, of like having a language that is inherently insecure by its construction, even if we can, you know, go in and fix all the bugs after the fact? I mean, it just puts a lot more cost into the process of delivering good software. So, you know, a lot of blockchain firms don't go through a $50 million ICO. A lot of blockchain firms have difficulty hiring a security engineer to work full time

on their team. And then my resources are always stretched thin. People are complaining all the time about how, oh, I try to hire trail bits and they told me that they couldn't start until a month from now or two months from now or whatever it is. And I need them tomorrow. And that's because there's only like, you know, two dozen people on my entire team that are trying to hold up the entire industry. what it means to have good compiler tools is it means that security is easier. It means that more people can produce more secure code without expert help. So that entire gap of the poor tools and poor language that the Ethereum community currently and others are suffering under slows down the whole field. It has a systemic effect on the ability of teams to ship software.

It's really important to solve, and I'm surprised that more people haven't been angry about that than they are about angry about my project. project lead time or something. It's more systemic risk. It's ecosystem risk rather than individual project risk, you'd say. Yeah, I mean, the specific design of solidity is what handed teams like mine, kind of the power and position that we have in the community. I'm self-interested, of course. Like, it's great that people turn to me for help and everything,

but I would love it if people didn't. Like, ultimately, I like to try and put myself out of a job. That's why I produce tools like Slither and why I release them open source and let people use them without talking to me. is I would like to have to try harder from day to day. And I want people to give me new challenges to solve. But the position that we're in is that there's just this influx of people that continue to try and write new smart contracts every day. And they fall into the exact same traps as the last person.

And that's kind of a poor state of affairs. Can you just briefly explain what it's Slyther and what it does and how it helps people save time, I guess? Trellibits has a suite of Ethereum security tools. Slyther is kind of the first layer of them. that's what we use where it's the lowest effort, highest value at the other end. You can just press a big red button and it tells you bugs that are in your code. The way that it works is when you compile solidity code with the solidity compiler, it produces an AST, an abstract syntax tree, an intermediate representation of some sort prior to creating bytecode that goes on the blockchain.

So we take that AST, parse it, and we parse it into something called static single assignment form, SSA form. And then using SSA form, we use a whole bunch of stuff from compiler theory to kind of process it and analyze it as well as traverse it for the security issues that we know how to find. So every single project that we do where we review a client's code, we have a process afterward where we collect what we learned. If we found a new bug, like a class of bug or some issue that's prevalent within the client's code base, we try to encode that programmatically into what's called a detector inside Slyther.

And then in the future, Slither should be capable of finding bugs of that type. The goal for Trail of Bits is I never want to find the same bug twice. I only ever want to find a bug once. And Slyther's the primary tool that lets us do that. In very layman's terms, it's like a spell checker for your code sort of thing. Kind of, but way more advanced than that, because this thing can parse, like, it can understand English grammar. Yeah, it's grammarly for your code.

Yeah, maybe. Do you think solidity is beyond fixing? Should we just scrap it and try a new language? Or do you think we can turn this around? And what are you thoughts about Viper? So I think Viper is a great improvement. I do think it's possible to fix solidity. It needs somebody to come in with a really strong vision of here's where we need to be and here's why. And right now, the kind of iterative, you know, baby steps kind of approach that I see of each incremental release kind of marching in the same direction isn't really trending. the direction that I'd like to see it go. I think it needs fundamentally a really strong leader to come in and say, this is hurting our entire community. It's hurting our ability to ship software. It's making people lose money. And it makes us too dependent on security engineers. We have to fix this problem. Do you think it would be wise for Ethereum just to say like adopt rust or something like that? I mean, or would that put too much of a learning tax, let's say, on developers to have to learn

this sort of complex language? There's pros and cons, right? Like, as a security engineer, I know that I'm not the most important person in the room. Like, the most important person in the room is usually the product manager, because you don't have a product unless you have users, unless you have a market, unless you're providing value to those users and you actually do something that care about. A perfectly secure product no one uses is valueless. Trying to say that everybody should stop develop and pick up rust over the next six months

is, you know, it would hurt a lot of people. It would, you wouldn't be able to ship software. You'd lose all your users. There'd be a lot of developers to fall off, a lot of companies that fall over. So the kind of hard break, I don't think, is really in the realm of possibility. But I do think it's possible to get incrementally better. And it's possible to have other alternatives like Viper. So, you know, a lot of the criticism lately that I've seen a Viper, I don't think is on

target. The Viper compiler, you know, while it's suffered a couple of setbacks now and then, people find bugs in it. We found bugs in it, right? Somebody paid us to do a Viper security review. We worked for computable. They had their entire code base written in Viper, and we found a compiler bug during the project. But just because we found one issue doesn't mean the whole thing is fundamentally unsafe.

I think from an architectural, like, high-level perspective, there's a lot of things about Viper that I like. We've talked a lot about formally verified languages on this podcast, and lots of blockchains have implemented this programming paradigm. I think I can kind of explain what a formally verified language is. languages, but I'd like for you to perhaps try to do a better explanation. And one of the things I think people think when they think formally verified language is like no bugs or it's basically free of being able to be hacked, right? And I think that's like a misconception, obviously. Can you explain what a formerly verified language is and what people are getting wrong about it when it when it comes to security? Yeah, sure. So you formally verify code that you wrote, right? And the way that

you do that is you express some security properties about it and then you prove them. You use a tool to prove them. And there's two main components here. There's what properties you express, and then there's how you prove them, which creates this widespread difference in what it means to be formally verified. Like, if I write a smart contract and it's 2,000 lines of code and I write a single security property, is it formally verified? Yes or no. So like trying to say that code is formally verified or not as a binary tells me that you don't know what you're talking about. what's really at issue here is what the properties are, the fidelity of them, like how accurately they explain the critical functionality of the code, and then the method

that you used to prove them. Because you could prove a security property with a unit test, but a unit test might only test one positive interaction in the contract. It might not test all the potential negative interactions, all the things that you didn't think about. So then you start getting into things like fuzz testing, where that's a test case generator. So you can use a test case generator to probabilistically generate a wide variety of potential inputs that might break a security property. And that's good, but then is that the same thing as a symbolically verified security property or an abstract interpreted security property? So all these things kind of change the fidelity of the security properties that you've defined, where at a very

low level, you know, maybe unit test might be really good place to start, but then over time, you prove that security property with stronger and stronger methods that are harder to kind of escape gaps in coverage. That's one take on formally verified software. Another is we published an academic study. Trail of Bits went back and looked through two years of Ethereum security reviews that we had performed, and we split all the bugs we found into one of two categories, bugs that you can find automatically with a verifier or other tool, and bugs you can't. bugs that require a human brain. And we didn't talk about the state of current tools when we did that. We actually just talked theoretically, like, is it possible to build a tool that finds this bug?

And what we found is that 50% of the bugs that we find could never be found by an automated tool. Now, there's some gaps in that. There's some really nice things that came out of that paper. One thing that came out is that for the highest risk bugs, for the bugs that were the highest severity and the easiest to exploit, 80% of them can be found by automated tools. So if you're effectively using automated tools, including verification tools, then you are wiping away 80% of the highest risk issues that could show up in your project, which is a great result. There's a lot here about formally verified software that I think people don't understand. So when you say you split these bugs up into two categories,

the ones that can be automatically found and the ones that couldn't. So is this just by like some sort of program that kind of checks against known bugs? Is there like some machine learning or artificial intelligence involved in? We hand-reviewed every single bug. We had a team of smart people sit down and think about every single bug we found for two years and tried to imagine in their head if a supercomputer, you know, alien came down from the future and provided us with the perfect tools in the universe to find,

software security bugs, would it be possible to find this bug with that tool? And for 50%, the answer was no. So you still have to have a smart human. How do you determine that, though? Yeah, exactly. I have exactly the same question. If you kind of have a computer and you don't know that it's a human inside, right? So basically, how is that different from a just a super smart computer that you can say, look, you really need a human to identify this bug. No computer could ever do it on its own. Well, so a lot of it means that you have to think of a about, okay, so here's how you talk about it. There's a lot of bugs that are generic to the programming language, like access control issues. It's a problem if you forget to control the

access to some kind of function that lets you upgrade the contract, right, or kill the contract. That is a fundamental invariant that you can express for the entire programming language and all programs written in it. On the other hand, there are security properties that you might need to define that are specific to your contract. There's something that you are trying to do. And the only way for me to tell that this is a bug or this is not a bug is if I have a smart developer tell me that that's what their intention was. But if you have a specification of some sort that specifies that, then you could presumably also have a computer. I mean, I guess that's maybe what formally verified languages allow you do is you create kind of this like this framework

or this spec. in the traditional software sense. Like, let's say, so the reason why fuzzers are really prevalent inside of like compiled code, C&C++ is that I know that when I get a crash, that that's going to be a bug, right? Software should never crash. So therefore, if I find a crash, it is a bug. On the other hand, if I have a smart contract and I manage to, you know, transfer some money to somebody out on the blockchain, is that a bug or not a bug?

I don't know, because that might have been my intention to do. it. So you really need some element of guidance that you typically don't need for traditional software. The other thing here is that a lot of these vulnerabilities, there's just like high level logic involved. There's high level thinking involved that a machine is not going to discover on its own and that it would be incapable of kind of creating on its own. But there's a blog post that summarizes it called 246 findings from our smart contract audits and executive summary. And we will link to that in the show notes. great. We've kind of jumped in the deep end here. So maybe let's go back to square one. So basically

say, I've built a project on Ethereum, say some type of decks. And there's a team of developers on the projects and we're making the leap to main net. But of course, we think we should get this audited. And this is probably one of the times when people call you and say, oh, why can't you start tomorrow? But, you know, like only two months from now. So we call you And what do you guys do? So basically, what does the journey look like? First off, you know, I think there's a misconception that people should hire security firms the second that they finish their software. Usually that's way too late for exactly the reasons that we just got done discussing.

A lot of your security depends on the ability of the developers to express what their intentions are. What are they trying to do with the software? Did you write a spec for it? And then did you write the minimum amount of code necessary to actually implement that spec? because complexity breeds insecurity, and the more unnecessary parts of your code that you write, the more bugs that you're going to have. So engaging with a security professional early is really critical. We really like it when people engage with us at the design stage. We can talk through different implementation strategies that might reduce the footguns that they leave in the code base,

the areas where they're likely to have bugs. That ends up making the after-the-fact review, the final stamp, you know, the final test before it goes into the Thunderdome on the blockchain. it makes that process a lot easier. So when we do actually get hired for a security review, a lot of what we end up trying to do is reverse engineer the intentions of the development team because the developers don't always know what they are. So it involves a lot of communication, a lot of back and forth conversation about, hey, what were you trying to do here?

What is the intention of the system? And then we'll encode that into a spec for the team that we work with. We'll write out the security properties ourselves. We'll prove them with property testing fuzzers. like Echidna, or we'll prove them with a symbolic verifier like Manticore. And in addition to that, we'll find those generic language-level invariance, things like a lack of access control or integer overflows or re-interency and that kind of stuff that might sink your ship in different ways.

That process continues for as many weeks of the review that we have. And then what we do at the end, we sort of doing this a couple months ago, about two months ago, is now we also include a code maturity evaluation. because security isn't really like, okay, great, you're done now. Like somebody looked at your code. All the bugs are gone. That's not how it works because you created that code. You could have hidden any number of potential issues in there.

And really what we're doing is assisting you. That's why I call it a security review, not a security audit, right? This isn't like accounting where I can tell you if everything is like, you know, all the dollars are properly accounted for and therefore you're not committing fraud. This is more of a probabilistic check where, hey, you know, I noticed you're doing some things that aren't so great. It's like, you know, when I used to play baseball as a kid, I videotape myself swinging the bat and then watch it, and I'd have somebody else that knew a lot better how to play baseball, take a look at it and say,

oh, you need to step like this, and then you'll connect with the ball better. That's kind of what a security professional is doing. They're trying to tune you up so you produce the most secure code possible. So the code maturity evaluations give you a long-term view as to where you are in the process. Like, what is the state, what is the maturity of your testing and verification? What are the maturity of your access controls? What is the maturity of your use of assembly? And how is that going to cause or not cause issues in the future? Because this doesn't really end after you release. Usually, you've got a version two that's cooking, or there might just be advances in the field, or the blockchain itself could change underneath you. You know, who knows what the Ethereum

foundation is going to push in their, you know, next biannual hard fork that might, eliminate all the kinds of security protections that you put in place, or the next person to invent flash loans that all of a sudden takes the security margin that you had for interacting with all your oracles and drops into the floor. For exactly these reasons, basically what we see more and more upgradable contracts, right? So upgradeable smart contracts, but they come with their own pitfalls. Can you talk about that? Yeah.

So upgradable contracts are a really interesting case study because they're another example where the solidity language shoots itself from the foot. In order to do upgradable contracts, you have to use a substantial amount of low-level solidity to get them done. And the fact of the matter is that we've identified 17 unique ways that upgradable contracts can fail and either lock your contract or lose everyone's data or be exploitable after you put them on the blockchain. And it's really not as simple as people think. Not only that, but it creates risk because now you have to handle private keys on your corporate IT. Like, you know, you've got a network somewhere, you've got a computer somewhere that's got some keys that let you manipulate the functionality

of a smart contract on the blockchain, which is massively fraught with risk for a number of reasons. Doesn't that defy the whole point of a blockchain? It does. It really does. So at Trillibitz, we're not too big fans of upgradable contracts. We understand the need for them. Like, it's critical that people have, you know, contracts they can pause when they're having a security. issue. It's critical that people can recover from an incident when they experience one, but the methods that people use for upgradable contracts today tend to be inherently unsafe. Again, Slyther is a tool that enables us to only find bugs once, or only have to find bugs once, and we've encoded all those strategies for accidentally failing to write an upgradable

contract correctly into Slyther. So it checks for all those 17 cases of failure. That doesn't mean that's exhaustive. There's probably more. And there's a lot of other ways that it can go wrong. Like, you can just get hacked and lose your keys, which is another thing that we consult on, like how to use an HSM to store them like a professional. But upgradable contracts tend to carry a lot of risk with them. So instead, we try to promote the use of contract migration, which is just burning the whole house down.

When you've got a contract or token or whatever out there on the blockchain, you can save all the data and roll it over to a brand new contract. That presents a lot of issues. since clearly you have to notify people that might be using your old code to use your new code, also has implications in the defy space, since there might be some, you know, defy app that's sitting up there on the blockchain that's hard-coded to think that you're at a certain address, but now you're not, how do you fix that? So, you know, there's not challenges to it, but it's inherently safer since there's a lot less moving parts. It's cleaner. I also think it's

more honest. I totally see that. I mean, there's pros and pros. So I totally see that it's probably the most secure way or it's definitely the most secure way. Then the question is, do you kind of leave it up there and just tell people to migrate? And then basically you have the problem of split liquidity and different versions and you can't do that too often. And it's kind of fraud both ways. So when we think about security and blockchain technology, we almost invariably go straight to smart contract failures. But there's also plenty of, you know, like front end and back end vulnerability. that we pay not as close of a mine to.

And so they're often overlooked. So for instance, with what happened with loop ring a couple of months ago, I don't know whether you remember that. I don't. It was about the way that a hash was stored somewhere. And there was like a limited number of hashes that they were actually using. They were generating hashes in 32-bit space. Oh, well, that sounds bad.

Do you think we pay too little attention to the things that happen around the smart contract? So basically the front end and back end infrastructure rather than because everyone goes straight to the smart contract, right? And people do smart contract audits. But I know way fewer companies who actually do complete front and back end audits just because it's an expensive thing to do, right? Yeah. I mean, I think there's some properties about smart contracts that make them a little bit more like, I think it's correct that you focus on them for the first line of your security review. They're fully exposed to the public.

Anybody can go read and run the code, which is really unique, right? If you have like server infrastructure up in the cloud somewhere, I can't easily grab the source code to that and start messing with it. I have to just, I have to just poke a black box, which is a fundamentally different kind of problem to solve. On the other hand, the information, you know, quote unquote information that you steal from a smart contract is inherently financially valuable. So when you think about this from an attacker's perspective, what they ultimately usually want to do is make money. Sometimes they want to get fame. Sometimes they just want to harass people and troll. But for the most part, they want to make money.

It's much easier to make money hacking a smart contract than it is to hack a server somewhere on the internet. Because once you hack the server, usually that's a pivot point to access something else that does have financial values. There's more steps involved in cashing out, right? So for those reasons, it's definitely appropriate to focus on the smart contract part first. But yeah, by no means, don't stop there. You absolutely need to think through the entire perspective. Because again, how secure is your smart contract if you publish the keys for upgrading it on the internet, right? If you're just storing them on your personal laptop and that personal laptop attacked, then so does the smart contract and every single piece of data on it.

So, yeah, depending on what you trust and the way that you designed your system, the security of your oracles, the security of cloud infrastructure, the security of your mobile applications, the security of your authentication system can all play a role in ultimately the security of your product, which is what you need to think about. You need to think about is this product doing the things that my clients expected to, or am I going to unintentionally lose all their money or unintentionally do something that they don't expect? Right. And that's a larger problem than simply is your smart contract safe? For a lot of clients we've actually started doing threat models. People in the blockchain space haven't been too up on it,

because they're more like interested in the really quick kind of like, you know, rush out to production. I know what I need and, you know, I'm really, you know, kind of ego-driven, secure tech master of the universe. And not a lot of people stop and think like, okay, well, what don't I know? And what should the interactions be between the systems that I've designed? And what are the risks that I need to mitigate against? So for a lot of people and more, mature fields, we've been doing threat models. For instance, election systems, voting systems, closely related to governance systems, but not quite. For those, you know, you could throw security at the wall and see what sticks. You could just go and say, hey, I've got 10 code bases. I'm going to have somebody do an application security review. I'm going to make sure that I can't crash.

But really what you want to understand is how easy is it for someone to manipulate this election? How do users interact with the voting system that I've designed? Is it possible? Is it possible? possible to hack the end user and somehow convince them to use the system in a way that they're not supposed to. Like, you know, can I tell people to download the wrong mobile app? Maybe it's not even a security issue in my system, but it's the way that I communicate things about it. So we've done a lot more threat models where we've tried to figure out, like, what's actually important? For the blockchain, this is the same thing. Like, if you want to sit down and start developing security properties for a piece of blockchain software, what properties are you going to

right. What are the interesting properties to express? And in order to answer that question, you need an accurate understanding of your threat model. We use a couple tactics for building threat models. We use Mozilla's rapid risk assessments most frequently. They're a really great 30 to 60 minute scripted interview that you have with a developer to help them understand what's actually at risk in the software they've built. But then other times, there's NIST standards, there's all this Adam Shostack kind of literature that came out of Microsoft that are a lot more heavyweight. But the rapid risk assessment stuff fits really well into a one or two week consulting engagement. So that's what we end up preferring.

If you want to see an example of that, we have a great example of a really well-developed threat model up on our publications repository that we did for a vote, a blockchain voting system. That's the area where all this stuff overlapped. It was quite entertaining to us to get a client that had both. But you can see what it looks like. And we'll be publishing more of those as time goes on. So hopefully there's stuff that people can learn from. Cool. We'll link to that as well.

We've talked about the security vulnerabilities of individual smart contracts, but now it gets even trickier, right? So basically in the defy space and in the open finance space, composability is key. And we have seen several attacks and vulnerabilities over the last past months whose gateway was leveraging some weakness in composed protocols. So what do you think we should be doing about that?

So who should be responsible for checking that things actually fit together and that basically interfaces work? And is it the people who actually build the blocks and thereby also the interfaces? Or is it the people who kind of use these money Legos to build like more complex protocols or should the users check or should we have like an external body that kind of certifies these kind of composed protocols? What's your take on this? Yeah.

So, I mean, for better or for worse, there's no gatekeeping. Like, I can't, you can't tell me that I can't put code on the blockchain. So ultimately, this, you know, creates a whole bunch of surprises that people have to deal with. The defy space, I think, is really immature. Like, it's a beta use case for Ethereum and for blockchain technology in general. And people need to understand that. People are throwing a lot of money into defy. applications right now and our kind of holistic end-to-end understanding of exactly how the whole

field works is very limited. So I expect that there's going to be failures and that there's going to be these new use cases that people come up with, whether it's a flash loan or a deflationary token or like whatever people end up designing that flips all the security properties I thought I had on its head. So the defy space, I think users really need to understand. It is a work in progress. This is not like, just because I've got a couple of Legos doesn't mean I can just start using them today and not suffer any potential consequences. Like there's always going to be some downside risk. And the downside risk in the DFI space right now is really high because that level of maturity just is not there yet. I think there's a lot of other things than DFI space that that kind of caused me concern.

And for people listening to this, they can't see my body language right now as I'm kind of like nervously rubbing myself. because all the defy space stuff stresses me out. You know, there's assets with really low liquidity. Even ignoring flash loans, there's things that people with a very limited amount of money can interact with and cause failures for other people. There's all this reliance on external oracles where not everybody that's deploying a defy application has evaluated how secure their system is against that Oracle being manipulated or how many oracles being manipulated or what the cost is to manipulate.

manipulate an oracle, like, just that kind of full system thinking really isn't there. How can you have full system thinking when if the number of oracles you can have in a system is basically unlimited? And then on top of that, like the number of bugs you can have per oracle is unlimited. You're basically just reducing the security to the lowest common denominator, and you don't know what that is. So it's like, you know, it's virtually impossible then in that case, if you have external inputs to a system to ensure full security. Yeah, I mean, it gets even worse when you start thinking about external contract interactions, right? Like, there's a lot of players out there, you know, the compound folks, I think,

are one of the best where they've whitelisted a lot of the interactions they have with external contracts because you just don't know exactly what everyone's going to do and you need to consider all the interactions between yourself and a counterparty before you start engaging with them. There's a lot of contracts out there that say that they're ERC, 20, but don't actually implement the spec is the easiest example. There's a lot of contracts out there that, you know, if you don't flow through all the second, third, fourth order effects of what it is to interact with a given contract, then there's the potential that somebody finds one of those composability issues and gives you a really bad day.

So I think it's really incumbent on defy applications, at least right now, to take these baby steps towards full decentralization and not exactly go for, hey, I'm going to interact with the whole world, and everybody on the blockchain could call every single. function of mine and I'm going to call every single function of theirs and it's going to be great because it ends up taking on too much risk too quickly. So I think a lot of people in the defy space need to take some baby steps and be sure of themselves when they take those steps through either security review, really effective and thorough modeling, work with economists to figure out incentives, and actually understand what kind of risk they're inheriting, maybe do a threat model,

right, understand what kind of risk that they're inheriting by interacting with those systems or creating certain features. Cool, thank you. That's super interesting. So there's a different kind of bug that we don't think about often enough. So basically composability is in everyone's, at the forefront of everyone's mind. But there's one thing that a lot of people who've been in the ecosystem for a very long time are deathly afraid about, but that we rarely talk about.

so bugs in the compiler, just because if there's a bug in the compiler that kind of causes a certain error, and basically it's kind of people don't generally check bytecode in their audits. We kind of rely on the code itself and the AST and so on. We don't really check the bytecode. So do you have thoughts on this? So are we too reliant on individual compilers? A little bit. You know, like I said earlier, the quality of the compiler that most of us are using the solidity compiler is extremely low.

So the testing just really isn't there. And there's an ongoing discussion right now about improving some of that testing by actually importing some of the techniques from Slyther and offering more standardized interfaces for software like IDEs and testing tools so that we don't accidentally break the entire ecosystem of tools that people rely on to test the software that they create by updating the compiler. But it is a little bit of a misconception that we're not testing bytecode. Most of the tools out there that are doing property testing or are doing symbolic verification or are doing abstract interpretation are actually working on the bytecode. So when you get a manual review, normally that manual review is of the solidity that you wrote.

And then if you have a static analyzer, that static analyzer is usually, again, going to be of the solidity that you wrote. So you inherit some potential bugs from the compiler. But when you're writing properties, writing security properties, in general, the approach that engineers have taken towards building those tools has been to, evaluate those security properties on the bytecode that gets put into the blockchain. The risk is that the emulation that you're doing for the Ethereum virtual machine to evaluate that bytecode might not be identical to the Ethereum virtual machine that powers the blockchain. I have a Python Ethereum virtual machine that's built inside Manticore. Is that identical to the execution of the Ethereum virtual machine on the real blockchain?

I try to get it to be as close as possible, but there's always the potential that there's divergences there as well. So, you know, this is a little bit of a, like, you know, turtles all the way down, but every single one of these things that you do ends up reducing the scope of what can go wrong. Like, it doesn't hurt you to do a manual code review and find bugs in solidity. That only helps. It doesn't hurt you to depend on a, from scratch, Python implementation of the Ethereum virtual machine. It generally only helps you find bugs. So these things over time, like you get this preponderance of evidence, like, okay, I've reviewed the source code. I've statically checked it with a linter. I have property tests that run on the

Ethereum virtual machine, and then I'm only going to compile it with a safe version of the compiler. So, for instance, a recommendation that Chale Abits makes in our reports is we don't want you to use the latest version of the solidity compiler. We actually roll back to versions that we think are safe. I don't know the exact version that we recommend now, but generally we look at the maturity of new features they're developing and take our own view as to how safe we think the long-term will be for each of those features. And then we recommend something before those features were introduced. So things like the ABI encoder version 2, the Yule parsing, that kind of stuff, I would absolutely avoid and use a version of the Solidity compiler that's older,

since it's less likely to have bugs introduced into your code. Sure. So are there other dark courses in the blockchain? security space that should keep us up at night. I mean, how do you feel about non-quant resistant code? Is that something we should worry about? What should we worry about that we're currently not worrying about? So give us some nightmares. Yeah, sure. I mean, the post-quantum stuff, I'm not super worried. Post-quantum stuff does not keep me up at night. I think it's still an open question if a quantum computer can even exist. A lot of people talk about the number of qubits

and whatever that you can get and how that's growing over time and there's a larger number of qubit quantum computers. But that's not actually the metric that people use to evaluate if quantum computer works or not, because you have to deal with noise. The larger your quantum system is, the more noise that you have to look through in order to get results out the other end. And what we've seen is that as these quantum computers scale up, so does the noise, and they end up kind of equaling each other out. So I don't think that quantum computing is a five-year problem. It's definitely not a 10-year problem either. I think it's more of like a 30-year problem. And when you look at it on that time horizon, there are some really good research results that are

out for developing post-quantum cryptography. Trailibits has a great guide on our blog called a guide to post-quantum cryptography, where we reviewed a lot of the NIST candidates for quantum cartography and broke down the different categories of lattices, isogenes, codes, hash functions, and all the other kind of potential ways to get there. So I think that on that, kind of time horizon, cryptography can kind of mose you along and end up giving us a good result and prepare us for the future if quantum computers were effort to exist. And that's,

you know, I'm not discounting. That's a possibility, but I'm just saying that it's hard. So that's definitely not what keeps me up at night. Honestly, like I think the outlook for smart contracts is pretty good. Like,

I think that in 2020, we figured out a lot of the things that we needed to know to build smart contracts safely. Like there are tools. that are available for people to use. There's a process that they can follow. There's a lot of lessons learned from other failures. And we generally know the footguns that exist inside solidity, even though we'd prefer they weren't there. And I think that for teams now that are developing software, they have the tools, they have the knowledge, and it's within the realm of capability

for them to produce secure software. It's just a matter of whether they do it or not. So generally, what keeps me up at night is that there are people to sling in code, that there are people that that just ignore all these lessons that don't do their research, that don't use the best tools available to them to produce software. The adherence that we have to people that use Slyther to check code before they put it on the blockchain is still pretty low. I don't know exactly what the numbers are, but I'm sure it's not 100%. And that's a huge problem, because there's 90 flaws that Slyther finds that the Solidity compiler can't. That's kind of what keeps me up at night. There's all kinds of new stuff that gets developed, like the DFI space. I'm kind of like, I've learned to love the bomb.

right? Like, I know that's an immature space. I know there's going to be hacks and failures. And that's what I expect to have happened. So when I see it happen, I'm not surprised. So I'm comfortable with the fact that we're going to discover more potential things that cause issues like flash loans or deflationary tokens or people that just don't implement the ERC20 spec properly and people inadvertently trust them that they do. There's a lot of stuff there. I don't know. I'm a little bit jaded. So I think I've seen all this stuff. before, so it doesn't really scare me. I have to be able to sleep easy. I wouldn't be able to work in this field if I went to bed every night. I was worried about the world being there when I

woke up in the morning. Well, I mean, as the space continues to grow and more, say, like, real world assets or like institutional finance comes into this space, like, do you think at some point there will be a tradeoff where we either have to go towards more centralization or more whitelisting or like kind of pull back because if just composability just grows exponentially, that means also like exponentially more security issues and security vulnerabilities. And institutional finance is not going to want to deal with that. So what's the future look like? Yeah.

So the defy space, I think, is a really special case, right? Where if you want to have some defensible unit of code out there that's capable of remaining safe despite changes to the environment around it forever, then you need a really solid understanding of all the economics of that system and all the incentives that people have to interact with and all the safety margins for all the bad things that could potentially happen. We recognize that. So we partnered with Prison Group, a small team of extremely talented economists that, like us, used to work professionally with large financial institutions and central banks and that sort of

thing, to offer a combined service offering called Mainnet 360, where we look at things from a combined economic and security perspective. What was good in 2019, what used to pass the bar in 2019 of like, okay, I don't have reentrancy in my contracts. I can put this thing up on the blockchain. That is not the bar in 2020. So, like, there's a higher standard of proof that I think people are going to need as time goes on. So we're certainly not done. Just because I wrote Echidna and Manticor and slither, it doesn't mean like, oh, security solved. And if everybody runs this stuff before they put it on the blockchain, they're totally safe. Like, there's certainly much farther that we have to go. And I think having a really solid, formal understanding of the economic security of your smart contracts is probably

the next level to that. And that's where Trail of Bits is investing a lot of our tool development and a lot of our partnerships and a lot of our own team's kind of internal knowledge transfer and learning. So what's the future of software security? And what role is AI play in terms of being able to detect bugs? I mean, like right now, there's like human auditors, but you're also relying on a whole lot of tools. Like we kind of gave this analogy to like a spell checker or something like that, right? Like grammarly. But the next logical step would be, in my mind, be like, let's now build like some artificial intelligence that goes in and tries to figure things out, perhaps based on in some input or something like that.

But, like, what's the future look like in your opinion? Yeah, I don't know if AI plays a huge role. So AI has been really cool. There's a company, I think they're called Tab 9, that created an AI-driven software development tool, like an IDE. They took the entire universe of source code on GitHub, and they trained a model around it. And then when you're inside of your IDE, you can just type out one word and then press

tab a lot of times. And when you press tab a lot of times, it just fills in the code that it thinks you're writing, based on the model that it trained from GitHub, which is super cool. But for security, you know, a lot of what makes a program secure is an understanding of your security properties and the ability of you to test them. And the tools that you use to do that aren't probabilistic. They're deterministic. That's what you want.

I don't want like a model that's kind of fuzzy that like, you know, these guys out on the internet said it was safe, therefore it is. Like that's not the standard of proof that I want. I don't want to train a model to tell me that my code is safe. knowing that it might be wrong or that there might be, you know, things that I trained it on that are incorrect. Instead, what I want is I want like symbolic execution to fork off every potential configuration of every single function in my entire smart contract and provide a deterministic result back to me that it's impossible for me to reach a state that I don't want. That's kind of

what security looks like in the future. You know, we participated in this DARPA program that tried to examine the exact same thing. Back in 2014, 15 and 16, it was called the DARPA Cybergrine Challenge. The way it worked was, I don't know if you know, the development of computers like Deep Blue, but back in the 60s and 70s, that was the first time that people created robots that would play chess, not robots, computer programs that would play chess. And the computer programs were just so awful at their jobs that they would get creamed with even a amateur chess player. And there was no way that they would ever win a game. So they ended up making a league of only chess playing robots, where the chess robots would play against each other,

where they were all just as dumb as the other one. And then eventually they learned from each other and they figured out strategies that worked and they build up a huge repertoire of potential attacks and defenses. And then after years of training against each other, they were able to play against and beat human participants. So the DARPA CyberGrant Challenge was supposed to be the same thing for computers. It was a league where computers could play, capture the flag, a simulated hacking exercise, against each other. And then the winner of that competition would play a Capture the Flag in

DefCon CTF, which is the largest Capture the Flag Challenge, or most prestigious, let's say, Capture the Flag Challenge in the world. We put forth a machine to do that. But the kinds of advancements that you need to improve software security tools are things like automated crash triage, better program analysis tools that help you understand and model what a program is doing and where data is going. Those kinds of things are really critical to the future of software security, not so much the AI portion. It's not a clean kind of relationship between software security and chess, if any of that makes sense. I actually have a great talk, the SmartFuzzor Revolution, where I try to review all the research that's been made and the progress that's been

made in the last 10 years or so at automated program analysis and software security. Generally, I think the future for this kind of stuff is that a lot of the tools are going to get embedded into IDEs. Like right now, it's a specialized task for me to use a program analysis tool like a symbolic executor or some other kind of program verifier. You need a security expert to do it. And it's a third-party tool. You have to go download it separately. But I think the availability of a lot of these tools inside standard development tool kits will increase over time. So hopefully the bar of knowledge required to use them goes down. More people take advantage to them, and then we end up finding a lot more bugs. There's also the potential of like using

them as gatekeepers. In the talk, I talked about things like, you know, a lot of people that are developing apps today, they deploy them to an app store to go out to the iOS app store, the Android app store. And there's a really nice point of control where you can use these program analysis techniques to ensure that every app that everyone has access to meets a minimum bar of safety. And I think that the blockchain kind of has a lot of the similar characteristics as an app store where there's probably some upfront verification that could happen. prior to entry, like, are you this tall to board the ride kind of deal that might end up making things safer? So that's where I see kind of, you know, maybe the two-year future for a lot of

this stuff. In two years, there might be a test to see if you're tall enough to board the ride to get on maybe the Ethereum blockchain. But that might be an anathema to a lot of the like cypherpunks out there that really don't want to do gatekeeping. Yeah, they'll go fork it and take their toys somewhere else. And then they'll go build their own blockchain and we'll start with the same problems all over again. Dan, thanks so much for coming on. Where can people find you and read? I mean, we'll link to all of these great posts you mentioned in the show notes,

but where would you like to send people? Sure, yeah. You can follow me personally on Deke Weido on Twitter. You can also get to Trail of Bits at Trail Bits. Our blog is exceptional. I highly recommend reading it. We try to have fun with it. Yeah, blog.trailobits.com.

And then if you're in the market for building smart contracts, you should go look up our repository called Building Secure, contracts, which includes the accumulated lessons learned from all of our security reviews and how we recommend clients build secure contracts without us. That's going to walk you through all of our tools, a lot of our guidance, checklists, tips and tricks, all that good stuff is right there. Awesome. Thanks. Thanks a lot. Thanks for having me. It doesn't end here. There's much more of this conversation, and you can hear it by signing up for Epicenter Premium. As a premium subscriber, you'll get access to a private RSS feed where you can

hear the interview debrief, which goes on for an extra 20 minutes. You'll also get exclusive access to roundtable conversations with Epicenter hosts and bonus content you won't hear anywhere else. Go to epicenter.orgs slash premium to join the community and support the podcast.