The Breakdown - Crypto's No-Fun Hack Week

Episode Date: August 6, 2022

This episode is sponsored by Nexo.io, Chainalysis and FTX US.   One of the biggest challenges to the mainstreaming of crypto is the propensity for novel protocols to be exploited. This week, the Nomad bridge was drained of some $200 million while another vulnerability led to more than 10,000 wallets (largely but not exclusively holding Solana's SOL) being attacked. NLW looks at what happens and what the industry thinks the lessons are.  - Nexo is a security-first platform where you can buy, exchange and borrow against your crypto. The company safeguards your crypto by relying on five key fundamentals including real-time auditing and insurance on custodial assets. Learn more at nexo.io. - Chainalysis is the blockchain data platform. We provide data, software, services and research to government agencies, exchanges, financial institutions and insurance and cybersecurity companies. Our data powers investigation, compliance and market intelligence software that has been used to solve some of the world’s most high-profile criminal cases. For more information, visit www.chainalysis.com. - FTX US is the safe, regulated way to buy Bitcoin, ETH, SOL and other digital assets. Trade crypto with up to 85% lower fees than top competitors and trade ETH and SOL NFTs with no gas fees and subsidized gas on withdrawals. Sign up at FTX.US today. - “The Breakdown” is written, produced by and features Nathaniel Whittemore aka NLW, with editing by Rob Mitchell and research by Scott Hill. Jared Schwartz is our executive producer and our theme music is “Countdown” by Neon Beach. The music you heard today behind our sponsors is “The Now” by Aaron Sprinkle. Image credit: Boris Zhitkov/Getty Images, modified by CoinDesk. Join the discussion at discord.gg/VrKRrfKCz8.

Welcome back to The Breakdown with me, NLW. It's a daily podcast on macro, Bitcoin, and the big picture power shifts remaking our world. The breakdown is sponsored by nexus.com, and FtX, and produced and distributed by CoinDesk. What's going on, guys? It is Friday, August 5th, and today we are talking about crypto's No Fun Hack Week. Before we get into that, however, if you are enjoying the breakdown, please go subscribe to it, give it a rating, it a review or if you want to dig deeper into the conversation, come join us at the Breakers Discord. You can find a link in the show notes or go to bit.ly slash breakdown pod. Also, a disclosure as always, in addition to them being a sponsor of the show, I also work with FTX.

One weird summer tradition in America is the Discovery Channel's Shark Week programming. This has been going on since I was a kid and has just exactly as it sounds a ton of programming about sharks. At the beginning of this week, as we seemed to be beset by hack after hack, I had Shark Week, which just happened, I think, a couple weeks ago on my mind, and all I could think about all week was that this was Hack Week. Today, we're going to look at what happened and how the crypto industry is interpreting what needs to happen next, and how the crypto industry is thinking about what the lessons might be.

Let's start at the beginning of the week with Nomad. The cross-chain bridge protocol Nomad was exploited on Monday. attackers drain the bridge of essentially all of its funds, approximately $200 million worth of tokens. The Nomad team acknowledged the exploit in a statement. An investigation is ongoing and leading firms for blockchain intelligence and forensics have been retained. We have notified law enforcement and are working around the clock to address the situation and provide timely updates. Our goal is to identify the accounts involved and to trace and recover the funds. Sam CZ Sun, a researcher at Paradigm, called the attack, quote, one of the most chaotic hacks that Web3

has ever seen. Now, the reason the attack was so chaotic was that there wasn't anything particularly clever done by the attackers. No complicated smart contract exploit or anything like that. The issue was that the bridge's smart contract had been upgraded in a faulty manner. It approved incorrect transactions as valid and allowed the withdrawals of funds improperly. Again, Sam CZ Sun. You didn't need to know about solidity or merkle trees or anything like that. All you had to do is find a transaction that worked, find and replace the other person's address with yours, and then re-broadcast it. In other words, for a little while on Monday, anyone who could copy and paste a transaction from a block explorer was able to extract

tokens from the bridge. Although the nomad hack was the biggest in terms of total losses, it wasn't the scariest. That belongs to what people thought was at the attack on Solana. Around 9 p.m. on Tuesday evening, that's East Coast time, Solana Auditor Ottersack, tweeted that around 5,000 Solana wallets had been drained over the previous few hours. The tally at that time was around $2 million worth of sole tokens had been taken. Within a few hours, that had climbed to 7,000 wallets being drained for over $6 million worth of tokens. The hack seemed at first to be affecting mostly hot wallets that had not interacted with

the blockchain for more than six months. It wasn't at all clear what the attack vector was or what vulnerability was being exploited. Throughout Tuesday night, many accused it of being a problem with the Solana blockchain itself, while others said that it seemed like it was more about a wallet. Early in the attack, Phantom Wallet, which is one of the most prominent for Solana users, tweeted, we are working closely with other teams to get to the bottom of a reported vulnerability in the Salana ecosystem. At this time, the team does not believe that this is a phantom-specific issue. As soon as we gather more information, we will issue an update.

Unlike previous hacks, this one didn't appear to be caused by permission. granted to a malicious smart contract. The funds were being drained using correctly signed transactions, indicating that private keys must have been compromised in some way. Tuesday night was just super, super confused. One Solani user told CoinDesk, quote, I was getting my sunglasses refit when I got a push notification from my mobile wallet that I had sent all the soul from my wallet. It was my main hot wallet, so I had it connected to lots of different mobile and web extension wallet providers as well as a lot of DAPS. Not clear to me what could have been the root cause. Glad to have most of my funds on a cold wallet. Remember, as I said, over the course of Tuesday night, there was a huge amount of

back and forth about whether this was a specific wallet issue, whether this was a Solana issue, responses raised from confused to scared to completely shot in Freud. At 10.40 p.m., Solana status tweeted, engineers from multiple ecosystems with the help of several security firms are investigating drained wallets on Solana. There is no ever, evidence hardware wallets are impacted. An exploit allowed a malicious actor to drain funds from a number of wallets on Solana. As of 5 a.m. UTC, approximately 7,776 wallets have been affected. At 2 a.m. they updated. This exploit has affected several wallets, including Slope and Phantom. This appears to have affected both mobile and extension. Engineers are currently working with

multiple security researchers and ecosystem teams to identify the root cause of the exploit, which is unknown at this time. There's no evidence, hardware wallets, have been impacted and users are strongly encouraged to use hardware wallets. Do not reuse your seed phrase on a hardware wallet, create a new seed phrase. While its drain should be treated as compromised, then abandoned. So already you're getting a feel for the vibe going on. Confusion, some information about what it likely wasn't, but not a lot of information about what had been the cause. Slightly frantic pleas for people to get their assets out of their hot wallets. Zero X Fubar tweeted attacker is stealing both native tokens like Seoul and SPL,

tokens like USDC, affecting wallets that have been inactive for over six months, both phantom and slope wallets reportedly drained. Exploit cause unknown might be an upstream dependency supply chain attack. Revoking approvals will probably not help, only transferring to an offline hardware wallet. Why doesn't revoking approvals help? Because these sole and SPL transfers are signed by the users themselves, not transferred away by a third-party using approvals. So while you can revoke, it's likely something has caused widespread private key compromise. Zerox Fubar used a key phrase that started to come up more and more. In a thread, Kiomi wallet also used that same phrase, quote, it may be a supply chain issue, meaning existing software dependencies in browser extensions may have been exploited. Amir from Avalanche

wrote a thread about the whole thing, and also made this suggestion. Because the transactions are signed properly, it's likely that the attacker has acquired access to private keys, but how? One possible route is a supply chain attack where a JS library is hacked and it exfiltrates or steals users' private keys. Now, even though by the end of the evening, at least on the East Coast, focus had started to point to some sort of private key, compromise, and some sort of issue with wallets versus an issue with Solana that didn't change the fact that a huge amount of the discussion that night and the next morning was focused on a tweet from Kyle Simani from Multicoin. Multi-coin had been a huge investor in Solana and so is widely associated with the project.

The tweet effectively says, why do security have to be the most important thing I prefer speed? And that thing decontextualized from the conversation it was in was splashed everywhere by everyone from Bitcoiners to ETH Maxis to you name it. Basically anyone trying to suggest that Solana was somehow at fault. Adriano Ferrea, a software engineer, was not having it, saying, for f' sake, I don't care what you think of Solana, but their network was not hacked. This exploit was related to a couple of third-party hot wallets that were primarily used for Solana. This sucks. People are losing money, and it's another event that damages crypto's reputation. In times like these, security of your assets should be your number one priority.

If you want to offset risk as much as possible and still stay in crypto, you need a trusted partner by your side. Nexo is a security-first company that manages risk by relying on money. mechanisms such as over-collateralization, real-time auditing, and insurance on custodial assets. Learn more about Nexo's reliable business model and start your crypto journey at nexo.io. That's nexo.io. Eager to make more informed decisions around crypto, chainelysis is here to help. Chainalysis demystifies cryptocurrency by providing industry-leading compliance, market intelligence, and investigations support. for all crypto assets.

For organizations like Gemini, crypto.com, and BlockFi. Gain unparalleled visibility and maximize your potential with the leading blockchain data platform by visiting us now at Chainalysis.com slash CoinDesk. The breakdown is sponsored by FTXUS.

FTXUS is the safe, regulated way to buy and sell Bitcoin and other digital assets with up to 85% lower fees than competitors. There are no fixed minimum fees no-ACH transaction fees and no withdrawal fees. One of the largest exchanges in the U.S. FDXUS is also the only leading exchange that supports both Ethereum and Solana NFTs.

When you trade NFTs on FTCS, you pay no gas fees. Download the FTCS app today and use Referral Code Breakdown to support the show. Another tone of the conversation was just why people were keeping anything on hot wallets. DGEN Spartan wrote, apparently people keep significant amounts of money on mobile hot wallets. If you're too lazy to set up and use hardware wallet, unfortunately, you are beyond salvation. Thanks for playing, though. By the afternoon, things were starting to get a little bit more clear. Remember, the key difference between this and

previous defy hacks, for example, is that the transactions were actually being signed by private keys, rather than being approved by an exploited smart contract which victims had granted permission to. This meant that it was likely that private keys for some group of wallets had been compromised. That next morning, Solana status wrote, this does not appear to be a bug with Salana Corcor code, but in software used by several software wallets popular among users of the network. Anatoly Yakovenko, the CEO of Salana Labs, wrote, Android seems to be affected as well. All the confirmed stories so far have had the key imported or generated on mobile. Most of the reports are slope, but a few phantom users as well.

So far seems like phantom users also use slope. So seems more likely that this is a slope-specific bug. So what then is a supply chain attack? I'm going to try to do a quick version of this, using some excerpts from a thread by Joe McCann. He writes, Today is the day Normies learn what the meaning of a software supply chain attack is. This is a major issue for all developers, not just Solana. Web 2 has had massive security breaches and exploits due to supply chain attacks. So what is a supply chain attack? Attackers hunt for unsecured network protocols, unprotected server infrastructures, and unsafe

coding practices. They break in, change source code, and hide malware and software build and update processes. Because software is built and released by trusted vendors, these apps and updates are signed and certified. In software supply chain attacks, vendors are likely unaware that their apps or updates are infected with malicious code when they're released to the public. The malicious code then runs with the same trust and permissions as the app. End users are the victims as they trust the app they use daily to function normally and are unaware of the tampered code, as they should be. This is why supply chain attacks are so lethal. The publishers of the applications rarely know they are shipping infected software. The current explanation for the ongoing Solana exploit is a supply

chain attack which explains why some wallets that have been dormant for months are getting drained. This investigation is still ongoing, so the exploit may not be supply chain related, but it appears likely. On Wednesday afternoon, Slope released a statement. Here's what we know at this juncture regarding the breaches to our user base. A cohort of Slope wallets were compromised in the breach. We have some hypotheses to the nature of the breach, but nothing is firm yet. We feel the community's pain and we are not immune. Many of our own staff and founders' wallets were drained. Where we are now is that we still don't know how these private keys were breached, although there's some speculation that they might have been contained in some plain text file on a centralized server

somewhere. Slope said that they were still conducting investigations and audits as well as working with security experts to identify and rectify the problem. The total amount stolen isn't perfectly known at this stage. Current estimates are all below $10 million worth of liquid tokens, and the largest individual loss is reported around $500,000, but most are significantly less. Overall, Solana's blockchain integrity does not appear to be in question anymore, and this seems to be an isolated problem with wallet addresses that were generated with or migrated to slope. The biggest problem is that this is a massive private key disclosure, not a problem with a smart contract or a blockchain that can be patched or fixed. Those private keys are permanently vulnerable now.

And the only solution is to encourage users to transfer out their assets into a new uncompromised wallet. Essentially, there is no fix that can be performed by Salana Labs or Slope to uncompromise these wallets. Anatoly again tweets, if you have ever imported a seed phrase into slope, considerate compromised. Move all the assets to a new non-slope wallet. Even if the attack didn't claim those tokens yet, the phrase is leaked. Just a matter of time. So what are some takeaways from the crypto industry for this? Well, the first is that it's a real thing.

really good thing that this wasn't an actual layer one issue. It's a lot easier for people to understand and get over some smaller wallet having an issue, even a pretty catastrophic one, as compared to a large L1 having a damaging and materially compromising bug. Second, people are definitely articulating the value of open source here. Left Source Carapetus wrote, so I heard a few things about slope wallet, so I tried to investigate. Within five minutes, I noticed, one, not open source, no GitHub link. Two, their privacy policy is an unedited copy paste from one of consensus's privacy policies. Three, imagine the code. What's worse is the uneducated end user would have no idea. We have to be calling out bad behavior like this which is going to hurt end users and through

them our field as a whole much earlier in the process. In this case, the argument of open source versus closed source is that people can't figure out what's going on behind the scenes and make good decisions about whether to trust an application or not. Third conclusion, there is some serious debate to be had around the user experience of key management. Udi Wertheimer tweeted, Seed phrases are terrible for many reasons. One of them is because they encourage sharing key material between wallets, which is how so many people lost money here. The bug happened because the wallet devs messed up, not because of seed phrases. But the effects of the bug were severe because many people imported seed phrases from their safe wallet apps to the compromised app months ago,

then deleted the compromised app and forgot about it. They didn't understand the implication of importing seed phrases into another wallet and that they remain insecure even after deleting the compromised wallet because they're not experts and seed phrases are designed for experts. Now this kicked up a huge amount of debate in terms of what should the expectation be for normal average users as relates to key management. And I don't think there are good answers or a clear consensus on this, but it definitely seems like a conversation that we need to have and bare markets are the right time to have it. Fourth, and maybe most importantly, crypto just has to have a different mentality. Imran Khan from Alliance Dow says it should be clear to founders and investors that the MoveFast

break things mentality from Web2 will not work in crypto. By overlooking security and prioritizing growth, you are effectively telling your users to go to your competitors or worse, leave crypto permanently. What's the rush anyways? Metamask has 10 million monthly active users, peanuts in the Web 2 space. We have some time before we get to billions of users. Focus on security and slow growth. Even if your competitor moves faster than you, they will eventually succumb to negligence. Bordilon Musk put it even more succinctly. Crypto is like rocket science. You miss a bolt and the entire ship implodes. So there we go. That was the story of this hack week. I hope that none of you guys were affected. And if you are, my heart goes out to you. I want to say thanks again to my sponsors, nexo.io,

and FTX, and thanks to you guys for listening. Until tomorrow, be safe and take care of each other. Peace.