The Breakdown - Security After the Bybit Hack with Ledger CTO Charles Guillemet
Episode Date: February 27, 2025The Bybit hack was the largest exchange hack in history. NLW is joined by Ledger CTO Charles Guillemet to break down what happened and what the implications are for crypto security. Sponsored by: ...Ledger Ledger, the world leader in digital asset security, proudly sponsors The Breakdown podcast. Celebrating 10 years of protecting over 20% of the world’s crypto, Ledger ensures the security of your assets. For the best self-custody solution in the space, buy a LEDGER™ device and secure your crypto today.Buy now on Ledger.com. Enjoying this content? SUBSCRIBE to the Podcast: https://pod.link/1438693620 Watch on YouTube: https://www.youtube.com/nathanielwhittemorecrypto Subscribe to the newsletter: https://breakdown.beehiiv.com/ Join the discussion: https://discord.gg/VrKRrfKCz8 Follow on Twitter: NLW: https://twitter.com/nlw Breakdown: https://twitter.com/BreakdownNLW
Transcript
Discussion (0)
Welcome back to The Breakdown with me, NLW.
It's a daily podcast on macro, Bitcoin, and the big picture power shifts remaking our world.
What's going on, guys? It is Thursday, February 27th, and today we are digging into the ByBet hack.
Before we get into that, however, if you are enjoying the breakdown, please go subscribe to it,
give it a rating, give it a review, or if you want to dive deeper into the conversation, come join us on the Breakers Discord.
You can find a link in the show notes or get to bit.ly slash breakdown pod.
All right, friends, as promised, we are back today with an episode all about this big
by-bit hack, the biggest exchange hack in crypto history, and we're going to be joined by Charles
Giam A, the CTO of Ledger.
Now, Ledger is, of course, the sponsor of this show right now.
And for those of you who think that that's too big a conflict of interest, I'll see you
tomorrow.
Totally appreciate it.
But to be clear, this is not a sponsored episode.
It came out of the fact that in the wake of By-Bit, I saw a thread from Charles that
actually dug deep into some of the technical specifics of this hack, which I think is really
pertinent and interesting. So, as you know, I was traveling this week and I sent a set of questions
to Charles and the Ledger team, and what you're going to hear is a self-recording of the answers to
those questions. First, though, for those of you who haven't been paying close attention, which I can't
imagine what percentage of you that is, let me do a little bit of background. The TLDR is that at the end
of last week, By-Bit was hit by the biggest hack in crypto history, and likely the largest theft of any kind.
On Friday, North Korean hackers, the Lazarus Group stole over $1.4 billion from Bybit. Most troublingly,
the funds were stolen from the Bybit cold wallet, which is supposed to be firewalled from making
external transactions. Lazarus Group managed to drain the entire ETH cold wallet, including
various stable coins and ecosystem tokens. Everything was quickly converted into ETH tokens and is
currently sitting in a network of the most watched wallets on the planet. Now, if you're not familiar
with Bybit, the exchange is based in Dubai with close to $20 billion in overall customer assets on the
platform. They have around 60 million users worldwide and are extremely popular across Asia and the
Middle East. Bybit is now roughly the same size as Crypto.com or OKX and somewhere in the top 10
exchanges in the world. In other words, this is by no means a smaller fly-by-net operation. Within half an hour
of the newsbreaking, Bybit's CEO Ben Zhao took to Twitter, declaring,
Bybit is solvent, even if this hack loss is not recovered, all of client assets are one-to-one-backed,
we can cover the loss. He followed up with the Twitter spaces to explain exactly what happened.
It appears the hackers infiltrated the devices used to sign cold wallet transactions.
Zhao commented, By-Bet-Eth-Eth multi-sig cold wallets just made a transfer to our warm
wallet about an hour ago. It appears that this specific transaction was musted. All of the signers
saw the must-U-I, which showed the correct addresses in the URL was from Safe, our infrastructure
provider. However, the signing message was to change the smart contract logic of our ETH-Cold
wallet. This resulted in the hacker taking control of the specific ETH-Cold wallet we signed
and transferred all ETH in the cold wallet to this unidentified address. Please rest assured
that all other cold wallets are secure, all withdrawals are normal. Rather than shutting down withdrawals
and causing a few sleepless nights over the weekend, Bybit kept them open throughout. The exchange
managed to service around 4 billion of withdrawals which have now slowed to a trickle. Most of the
assets have already been redeposited. Zhao has also stated that the ethel hole has been completely
filled using a combination of bridging loans and conversion of corporate assets. Bybit has now
published an updated proof of reserve showing that all assets are backed one-to-one, and it appears
that a bigger crisis has been averted. Many commented on Zhao's conduct, with Zero X-Geft,
posting, this is a master class in crisis management and communication. Others noted that this
would have played out very differently had a bank suffered such a large hack. Kashif Raza, the founder
a bit inning wrote, If a bank were to be hacked for $1.4 billion, freeze withdrawals, informed government,
set up an inquiry commission, years of investigation, customer standing in queue, branches closed,
competitors getting an advantage. Bybit exchange hacked for $1.4 billion. CEO came live,
industry united, competitors lend support, one hour and everything back to normal. Industry itself
is tracking and freezing. Zero withdrawal pending. This is what they don't like about crypto,
Unity. Binance CEO, CZ added, don't worry. Most banks don't have $1.4 billion in cash for you to
steal. Fractional reserves. So, with the crisis, Cee,
seemingly over, the thing left to discuss is what this means for the industry moving forward.
Although customer funds were backfilled, there's still the fact that Lazarus has an additional
$1.4 billion at their disposal. One of the silver linings might be that this is an absurdly large
pile of crypto to try to launder, and every on-chain analyst is keeping a close eye on where it goes.
This is far larger than even the biggest mixers can handle. The failure and desertion
of Thorchain has also closed off one of the few decentralized paths to convert Ethan to Bitcoin.
In other words, there's just not as many ways for Lazarus to launder money as there used to be,
especially at anywhere close to this size.
The other part of the story that will be ongoing
is a discussion about the need to update security.
And that is the part where we are going to dig into
with Ledger CTO Charles GiamA right now.
All right, so as I said,
this interview was done asynchronously, not altogether.
So I did record some introductions to questions,
but this did not happen as a conversation,
which is why if it sounds a little less fluid, that's why.
That said, I am going to ask these questions
like I was talking directly to Charles.
So with all that out of the way, let's dive in.
Charles, welcome to the show.
Could you start by giving us a brief overview of what happened,
what the attack vector was and how ByBits cold wallets were accessed?
So this was an attack from a well-organized group.
The attackers could not directly steal the fans,
as they were secured by secure hardware and also the smart contract.
But instead, they tricked the By-Bit organization
into signing a malicious transaction
that sent all the funds to the attacker.
This is a nation-state hacking group.
They have a lot of resources and a lot of knowledge.
Compromising computer networks is really part of their daily activities.
They have found a new lucrative attack vector.
They are draining companies that don't use like enterprise-grade solutions,
including clear signing and secure display for their custody,
and they won't stop here.
Now the details are a little bit more clear,
It mirrors the Radiant and Wazir-X bridges.
What Lazars did for the first two hacks,
they compromised the computers of the transaction authorizers.
Victims believed they were approving a legitimate transaction,
but in reality, they signed a malicious one.
The malware simply swapped the transaction to be signed
right before it's sent to the signer.
In ByBits case, they didn't compromise the endpoint,
but they compromised part of the safe wallet infrastructure,
and it simply swapped the transaction wide before signing it.
The ByBit organization was thinking they were about to transfer funds
from their cold wallet to their hot wallet.
However, they unknowingly signed a transaction that swapped safe implementation for a backdoor version.
So Lazarus, like, deployed a comprehensive,
compromise contract and they manipulated the cold data, redirecting control to the attackers' malicious contract.
And then it's game over.
And for me, the real question is, how do we prevent this type of attacks that could stop new entrants coming into the space and protect customers at the retail and institutional level from these hacks?
That's why at Leger, we have Legend of Prize with clear signing.
And this product ensures transactions are displayed in format that,
you can understand and verify.
And this completely solved this kind of threat.
Legend Enterprise Solutions do not rely on smart contracts,
and it allows organization to implement a flexible governance layer
and also to provide clear signing on dedicated security device.
And I really think the ecosystem needs to evolve
using smart contract and retail devices
that don't support the smart contract to secure $1.4 billion,
is not an acceptable situation.
We really need to up the game as a community.
Could you explain a little bit more
about how enterprise wallet systems typically work?
For example, what are the main vulnerabilities
and areas for improvement?
Enterprise cryptocurrency security
typically involves multi-authorization wallet,
hardware security module and governance frameworks
requiring multiple approvals for transaction.
Attacks like the By-Bit incident
succeed when the system fails through human error, social engineering, or inadequate security infrastructure.
The core vulnerability often lies in blind signing practices, meaning that authorizer simply authorize a transaction that they don't understand.
So I think this is the biggest issue there.
Solutions like enterprise with car signing prevent such attacks by showing users exactly
what they are authorizing before approval.
And it also provides institutional grade hardware security and governance.
There is no smart contract involved,
so the risk associated to bad smart contract call simply doesn't exist.
In a setup called Hot Wallet,
organization will define different governance rule,
preventing to move more than a certain amount per day,
requiring at least like, let's say, five authorization to move funds,
also limiting asset transfer only to a specific address,
the hot wallet, for instance.
And this governance is implemented in secure hardware.
It's set up using the security device,
and any action requires human approvals on these devices
where all the information is displayed on the screen.
As cryptocurrency adoption goes,
we really need a robust self-custody solution that eliminate both clear signing and counterparty
risk. This will be essential for protecting assets at both retail and institutional level.
Hello, friends. I am thrilled to share that Ledger is once again partnering and sponsoring with
the breakdown. Many of you know, but for those of you who don't,
Ledger is the most secure hardware wallet for your crypto and logins. It's trusted by 7 million
users and secures 20% of the world's digital assets.
What's more, Ledger is a lot more than wallets.
Over the recent years, they've built a comprehensive ecosystem of products and services,
all of which are designed to make digital ownership more secure and accessible.
You can buy your Bitcoin with Ledger and Ledger Live and so much more.
Basically, not only did they want to keep your assets secure,
they want you to be able to do more with them.
Ledger's newest devices, the Ledger Stacks and Ledger Flex,
introduced the world's first secure touchscreens,
making it easier and safer to manage your transactions and assets.
Alongside Ledger Stacks and Ledger Flex,
the company also launched the ledger security key app, offering a safer alternative to traditional
passwords and enhancing your digital security. If you are in this space, you owe it to yourself to at least
check out Ledger and their ecosystem what they have available to you. So thanks, once again,
to Ledger for sponsoring the show. One of the biggest criticisms that's floating around is
the idea that ByBit shouldn't have left so many assets exposed to a single point of failure.
Do you think the industry should move to using multiple cold wallets, maybe even diversifying
their security procedures between them.
Leaving million in crypto assets vulnerable to a single point of failure
contradicts the fundamental security principle.
Like Bibis reliance on consumer-grade product with blind signing functionality,
essentially authorizing transactions without understanding their content.
This created a catastrophic vulnerability.
Segregation and diversification can be good practices.
However, the most important part is really to use like enterprise-grade.
solution with its associated governance.
This includes clear signing, as I said, at the protocol level,
where transaction detail are fully visible before approval.
We really need to eliminate this blank check problem that enabled this hack.
A company, handling the significant crypto holdings,
must adopt multi-layered security strategies with hardware-based protection
and distributed approval mechanism to prevent
any single compromise from causing devastating losses.
Following from a discussion that was happening a lot on Twitter,
ByBit changed their APIs to provide real-time liquidation data just prior to the hack.
Do you think that change could have contributed or was it just a coincidence?
It's difficult to know for sure.
ByBit is now on the spotlight.
They went through significant results that can create liquidity instability.
Additionally, if Arbitriders moved from ByBit,
clients could get liquidated while other markets reflect different prices.
It's difficult to know now for sure, and buy bits for being as transparent as they can is
very important now for them, but also for the ecosystem.
This attack has now been linked to the Femex and Wazir X hacks, and they all seem to have
exploited the same vulnerability in cold wallet systems.
Why do you think these attacks are becoming more common than the defy hacks that were
prevalent in the last cycle?
For attackers, it's a matter of opportunity cost, like low-level attackers spend little resources to do like basic fishing on retail,
high-level attackers target like higher-values target.
Unfortunately, some of them don't have proper security setups, and this is what enables their attacks.
It's also important to mention that now, Lazarus have stolen more than 3 billion USD in total.
That means that they now have significantly more.
resources to conduct even more sophisticated attacks.
So unfortunately, we can expect this kind of attack to become even more sophisticated.
I really want to say it, like actors with a significant wallet value that don't take security seriously will get hacked at some point.
It's not a matter of if, it's just a matter of when.
Cold wallets are becoming more common due to perfect storm of factors growing,
a cryptocurrency adoption that has created like higher value targets,
and also at the same time you have like sophisticated threat actors like the Lazarus group
that continue to evolve their techniques.
As cryptocurrency holdings increase in value,
attackers naturally raise their operational sophistication to match the potential reward.
Many institutions still rely on consumer-grade solution with fundamental security flows
like blind signing, essentially authorizing transaction
without seeing their content.
This practice is equivalent to signing blank check.
So maybe you are signing the transaction you wanted to sign,
maybe you are signing away all your assets.
Traditional security measures are no longer sufficient
against these advanced threats.
Enterprise-grade solution with clear-signing technology,
which shows users exactly what they are authorizing,
we present the necessary solution in security to counter there's increasingly sophisticated attacks.
Does this mean that exchange wallets are now the weakest link in the system rather than defy protocols?
Or is it more like they're the best risk reward given how large the top exchanges are becoming?
Exchange wallets have indeed become a prime target for sophisticated attackers,
representing a concerning weak link in the cryptocurrency ecosystem.
While G5 protocols have hardened their security through audits and formal verification,
many exchanges continue to rely on inadequate security practices like blind signing,
essentially authorizing like transaction without seeing their content.
The bi-bit hack demonstrates how even cold wallet systems can be compromised when using like consumer-grade product instead of institutional security solution with proper governance and clear signing capabilities.
As threat actors like Lazare's group continue evolving their techniques,
exchanges managing billions in assets must implement
enterprise-grade-custody solution
that eliminate a single point of failure and provide complete transaction visibility.
Without this improvement, exchange wallets will likely remain the path of least resistant
for high-value attacks.
Attackers drained several defy protocol in the past,
The previous biggest hack was the running protocol.
And as I often say, like, security is a chain.
It's as strong as its weakest link.
So new Defy protocol relies on battle-tested smart contract.
The security of this protocol has significantly improved over the years,
but it doesn't mean that they are secure forever.
Like new attack techniques, new vulnerability will occur.
So as of now, the attacker's trade-off is that attacking an excellent,
that is not using enterprise-grade security
is the best opportunity cost ratio.
But this is the state today.
It might change in the future.
Is there anything that retail users should take away from this hack?
Is it time to get more serious about moving away from exchange custody and onto hardware
wallets?
Retail users should always use secure hardware to safeguard their digital assets,
like ledger stacks or ledger flex.
With a hardware wallet, even if an attacker succeeds in getting control of your
computer, they will not be able to steal your private keys and access your crypto assets because they are in a segregated and secure environment.
Your private key is kept offline and that limits a lot the risk of hacking.
And the challenge of using an exchange wallet is ownership. On this type of wallet, you will never be in control of the private key for your asset and exchanges when the risk of being hacked.
And most importantly, the ethos of blockchain technology is its permissionless aspect.
And it's possible thanks to decentralized consensus of the blockchain, but also thanks to self-custody.
With self-custody, you don't have to have the permission to anyone to own and use your value.
You've mentioned that one of the big issues with Bybit was that they were using consumer-grade signing devices that require blind signing.
Could you talk a little bit more about this and maybe share how Ledger Enterprise
specifically differs from consumer-grade devices.
Ledger Enterprise uses secure hardware with a personal security signing device that allow for
clear signing and a full governance layer that is implemented at the HSM level.
It allows like institution, corporate organization to self-custody or keep their asset with
regulated exchanges.
And it also allows to trade off exchange through ledger enterprise.
trading, and this also removes the counterparty risk and lower the fees.
The governance itself is implemented within the secure hardware, and clients have end-to-end
security from their personal security devices to the H-System.
We've been working since the end of 2023 on defining standards for an implementation for
clear signing. We have created an ERC 7730, and many of our partners and leaders in the space
have jumped on the board, but not everyone has jumped yet.
Everyone of us has a responsibility to protect ourselves and also to protect users.
Overall, what are your big takeaways from this incident?
What do you think needs to change about security best practices moving forward?
If we want to have the freedom that crypto provides, we need self-custody and the best security
in the world. The real question is, how do we prevent these types of attacks that could stop
new entrants coming into the space
and protect customers at the retail
and institutional level from this hacks.
Ledger Enterprise with ClearSigning,
which shows you what you will sign,
solve this.
ClearSigning needs to become the industry standard.
So Leisure is working on this for quite some time now.
And we have a growing number of networks
and partners to properly integrate clear signing.
So you shouldn't sign a,
blank check. It only takes one signature to lose everything. And in the coming years, we are
aiming to evolve our clear-signing technology beyond Ethereum, including more blockchain networks,
Solana and other chains.
All right, friends, and that is the conclusion of this interview. Thanks again to Charles
from Ledger for joining the show and providing that insight into the inner workings of exchange
security. Obviously, every time we have a hack like this, it creates a new context to have
discussions about how we move forward and how we improve. And even now, I'm seeing so much
interesting stuff happening, much of which I'm sure we will talk about over the coming weeks.
Good on Bybit for taking a situation that could have been terrible and doing the best they can with
it. Good on the industry for really digging into these challenges. For now, that's going to do
it for today's breakdown. Appreciate you listening, as always. And until next time, be safe and take care
of each other. Peace.
