TFTC: A Bitcoin Podcast - #197: Joost Jager

Episode Date: October 2, 2020

Join Marty and Matt as they sit down with Joost Jager, independent Lightning Network developer, to discuss: - Griefing Attacks - Circuit Breaker - Routing on Lightning - Multi-Path Payments + Wumbo - Public v. Private channel capacity - Whatsat - Multiple implementations and how they interact - much more Follow Joost on Twitter Shoutout to this week's sponsors. Cash App. Start #stackingsats today. Use the promo code: "stackingsats" to receive $10 and contribute $10 to OWLS Lacrosse when you download the app.

What is up, freaks? Welcome back to Tales from the Crypt. It's your boy Marty Bent here to introduce this episode. Very dense episode topic being the Lightning Network. Matt O'Dell and I sat down with Joost Jager, independent Lightning Network developer, focused on security now, building out some security tools for the Lightning Network. beyond security aspect Yoast has done a bunch of work on the Lightning Network so we're very happy to have the ability to speak with him for an hour

about his experience working on the Lightning Network how he views the future of the Lightning Network and the hurdles it needs to overcome as we head into that future so I think you guys are really going to like this episode this episode is brought to you by our good friends at the motherfucking Cash App Cash App, Cash App, Cash App

get your money in the trap dog get the cash out stack those sats send those sats receive those sats sell those sats if you so please we're saying sats at sats because you can make sats the standard in the app i was trying to think of a rhyming rap there but your boy marty uh you can write uh pretty poorly you can speak into the mic cannot rap at all but i can talk to you about stacking sats on the sats receiving sats selling sats if you so please sat sat sat sat sat so the standard in the app if you want to make it the standard you can also make whole bitcoins the standard in the app if you want to i prefer sats stacking whole sats over fractions of bitcoins just feels more powerful uh when you're doing your your daily

purchases oh did i mention you could dca via the cash app you can set daily buys weekly buys bi-weekly buys uh if you so please you can set it and forget it have the peace of mind knowing that you are constantly stacking sats on a set cadence so you don't have to FOMO ever. You can DCA in. On top of that, you can stack slivers of stonks if you want to. I know you guys don't like the stonk market. I know the stonk market puked a bit last night

when we found out Trump had COVID. I know Bitcoin did as well. But in the Cash app, you can stack sats or slivers of stonks. What is a sliver of a stonk? It's a term that I came up with to help conceptualize the fact that you can buy as little as $1 of your favorite stonk via the Cash App. If it's too expensive, out of your price range,

you can buy a sliver as little as $1. Cash App Investing is subsidiary of Square and member SIPC. As always, make sure you use the code STACKINGSATS when you download the Cash App. That's S-T-A-C-K-I-N-G-S-A-T-S. You're going to get $10, and $10 is going to go to our good friends at Al's Lacrosse. That's Al's Lacrosse.

here's a good stacking stats download the cash app enjoy this episode with yos yager if you're enjoying the podcast in general this is our fifth episode this week freaks don't say we don't put up for you all right we're putting out content hopefully it's quality content let us know give us feedback if you're liking the content and you feel so compelled to give us a review particularly on apple podcast that goes a long way helps get uh get the podcast more visible on the platform and gets that quality information out to uh pre-freaks pre-freaks we have pre-coiners we call them pre-freaks that they're not freaks yet what do we call them let's find a name for these people uh thank you guys for listening appreciate your continued support uh we're hitting some crazy

numbers and downloads so i just want to thank you guys for consistently coming back and listening to my dumb thoughts on Bitcoin. Enjoy. currencies bitcoin is the victor i mean that's part of the bull case for bitcoin if you're not paying attention you probably should be what is up freaks welcome back to tales from the crypt it's your boy marty bent here for another interview joined with matt odell this time around matt how are we yo yo yo happy to be here we are we are sitting down with uh independent lightning network developer yos yeager who wrote a uh enlightening piece last week i reached out to him and uh figured we'd have a discussion about

what's going on with the lightning network yoast welcome to the podcast how are you hello yeah i'm good thank you um thank you thank you for agreeing to do this and like i said you wrote a post last week uh you had a um a twitter thread and then a post describing an attack that could be levied against channel operators on the lightning network pretty easily and the other theme of that post is that there are a lot of security flaws that need to be tightened up on the lightning network and that is the effort you're putting forth with your circuit breaker project but yeah so Matt and I I wanted to get you on to just talk about the Lightning Network in general, where it may be lacking as it pertains to security and then the attack that you outlined specifically. So maybe we can start with the attack as it's a timely subject, considering your post was dropped last week, and then get into broader security flaws in your view of the Lightning Network at large.

Yeah, sure. um so let me say first like this is a tech and attack sounds pretty grave but with the thing that i described it's not possible to steal funds from someone else so it's more like they always also call this a griefing attack so you can cause grief but you're not actually losing money and it it is you could consider it as the equivalent of a ddos attack on a website uh basically that but then on a lightning channel so with relatively little effort and also cost it's possible to disable a channel by spamming it with payments and and those payments are not normal payments but these are payments that take a very long time to complete so on purpose you set up a system that

allows you to make payments that take a very long time to complete and all that time those payments will sit on the channel and block the channel from from being used for something else and and so you described it in your uh your post the the amount of money needed to levy this this griefing attack is is not a lot you can do it with very few sets yeah because the main reason that you can do this for very few sets is that in lightning it doesn't cost you anything to send the payment you only pay when the payment is settled so payment attempts are free and that's also a good thing because you need to find routes you need to try routes and if those fail and you would have to pay for all of those it wouldn't be as like compelling as it is now because you

can try and if you have a lot of time you can try a lot of routes try and try to find the absolutely cheapest route that's there and once the payment reaches the final destination it is settled and only then you're paying paying all the hops along the route but this very same property like that allows you to make these payment attempts for free also allows you to um to block those channels if you are malicious and and your focus was particularly on wombo channels that are that are public so without a major project like circuit breaker how would you attempt to mitigate this this attack to private channels help anywhere here yeah so it's not particular to wombo channels you can also execute this but this is also something that has been known for a very long time i haven't

tweeted anything that wasn't known already i just try to emphasize like the the significance of it um but on a regular channel you could do the same thing the only issue is with a wombo channel because you have a lot more capital committed to a wombo channel um the the grief is is more basically so if you have a 5 btc channel and then someone sends those 483 sets in 483 uh distinct payments and you can't use your 5 btc for two weeks that's more painful than this would only be a 0.1 btc it's it's way more significant of attack in that situation because i mean right now correct me if i'm wrong um i mean the the lightning network is still very immature it's very young right so i'm i would imagine i haven't done any like intense analysis but i would imagine that

uh an overwhelming majority of the routing capacity is is in these very large channels between a very small set of nodes so you can disrupt you could disrupt the payments across the whole network if you if you hit a couple of the big channels right well you're you don't really know because if you've got two nodes and they may have like one very obvious wombo channel between them it could also be that they have like 20 other even larger channels that are private so if you target this wombo channel and you think you can get it down with 483 hlcs it may not work out because if they have these shadow channels alongside the Wombo channel, you need a lot more HLCs to actually block that pathway.

And it could even be that one thing to explain is if you create a route, you create a route based on nodes. So you don't select the actual channels, you select the nodes that you want to go through. So if you go from node A to B, node A is allowed to choose whatever channel they like to forward your HLC over. So if you have a 5 BTC Wombo channel

and you have a 0.1 BTC channel as a shadow alongside it could be that your first 4083 HLCs will go through the shadow channel and leave the one more channel unaffected so it's more subtle than that and and this is also something that you can't observe really so the public part of the graph is observable but you don't really know what else is there yeah the estimates have been made based on channel force closures to try to figure out how many private channels there but it's still a pretty indirect measure so in the end we don't really know how big the lightning network is at the moment and what we also don't really know is what the actual

velocity is on the network how many transactions are flowing from to those flowing through those channels and something to add to that if you're a node operator maybe you don't want to advertise your one more channels like you can advertise them if you want to show your like if it's in in your your interest to show that you've committed a lot of money to it but a smarter thing to do could be to just advertise a single channel that's sufficiently large to route the biggest payment that you expect let's say I advertise 0.5 BTC and then I put like 10 BOMBO channels alongside it just to create more lanes on that payment highway without revealing myself

as a potential channel jamming target and I know it's hard to calculate and estimates have been made but just as somebody who's been around the protocol and experimenting with it for some time now it do you have a feeling of the the size of the lightning network that is unknowable the private with private channels do you think it's larger the most suspect smaller the most suspect um no comment i think yeah i don't know really i don't still don't have a feeling but i do know node operators that intentionally create those shadow channels just to keep um say invisible basically so that does happen i don't know to what extent and i also expect that over time more node operators

will do this like they will realize that it's not really great to advertise that you have so much capacity so they might start converting channels into private channels and if you look at a graph explorer it looks like the lightning network is actually shrinking and maybe people would say the lightning network isn't it isn't growing it's shrinking it's a bad thing but it could also be a sign of maturity that people become aware of the risks involved with publishing their the the existence of a channel um adjusting their their policy there and it could be a bigger success if the network is smaller would would you say in your opinion that for the average user the end user someone who's not running a routing node um they should never have any need for a

public channel themselves they should only have you know maybe a couple private channels to some well-connected nodes would you agree with that statement uh yeah yeah i would agree if you're not intending to forward any traffic it's only a disadvantage to to publish your your channels if you make a payment yourself it doesn't matter because you find your own route and if you're receiving money then you pass on your private channel identifiers to the to the sender the only limitation that currently exists is if you make a key send payment so a spontaneous payment at the moment there is no route hints supported there so if you want to do that you need to have a public channel but this is something that will also be improved over time

so in the meantime how let's jump into circuit breaker in your efforts there how does it mitigate these attacks uh the way i understand it you're simply allowing a node operator to decide and limit the amount of inbound htlc's there are at any given one at any given point in time yeah that's basically the idea so your own outbound site you are controlling this yourself so the idea is by throttling your your inflow of htlcs you can mitigate this risk or at least make the attack more expensive because even if you allow only a single htlc to exist on every channel and you have 483 channels created by the attacker they can still send 483 htlcs but i think security it's not a it's not a binary thing so it's all about like making it more

costly to to execute this but at the moment if you just accept channels from anyone uh with the maximum number of HLCs allowed, then if the conditions become more adversarial on the network, you might become a victim of this or you might facilitate this attack towards bigger nodes because that's also the thing, this channel jamming thing, you can execute it remotely. So if you are connected to insignificant nodes, you can still use that to forward your HLCs to the actual targets. And because of the lighting properties, it will also be harder for them to figure out

where the attacks coming from because they need to contact the previous hops find out where the packet came from because because everything is onion onion encrypted so by limiting the number of incoming HLCs that's indeed like a mitigation and the simplest way to do this at the moment at least with L&D is to don't to not install any external software at all but just set the policy policy on um new channel opens so there is a configuration flag where you can say okay every new channel that's open to me i only allow five htlcs in parallel to exist there um that's a quick mitigation but the downside there is it doesn't work for existing channels and also once

a channel is opened with a low maximum in-flight htlc parameter it's not possible to change it anymore it's an immutable setting so if you want to change that you need to close the channel and open a new one and and then you're basically you're denial the attacker can still denial a service a particular route they just can't denial of service the channel right um how do you mean denial serves a route because it's like a because you're limiting a route to let's say five htlcs right well you only take five hlcs from a particular source meaning that they can only um um block so many slots on all uh subsequent channels um and they need more many more channels to actually do this so they're ddosing a slot not not a route well you're ddosing the

route indeed but the number of routes okay maybe this is like a language thing but indeed the details in the route and it can only ddos five routes instead of 483 uh that makes sense so So this is like kind of a similar situation as what the Tor network is having with their DDoS attacks, right? Because of the onion routing, because you can't really detect what the source is, right? Because historically, the way you stop these denial of services is you figure out if there's a malicious source and you just block that source. Yeah, IP tables and block the traffic indeed. So that's not possible. and the interesting thing here is also that um if you set up these limits you probably won't

want to set them up for all nodes because there can be like trusted peers if you are two big routing nodes and you have a few big channels between each other you don't want to apply this limit but if you don't apply the limit you are vulnerable to the attack so you really want to know that your channel pair is applying the same limit to their um to their like untrusted peers so the thing is that this in the end i'm not completely sure how this is going to work out over time but it seems that some kind of like web of trust is is forming in that way of nodes that trust each other that allow high limits and everybody outside of that will get lower limits and that's quite significant because um these are restrictions that currently don't exist on

the lightning network so do you think that's a centralization concern like are are we going to end up in a situation here where we have you know six big routing nodes that know each other and and agree to this type of green zone demilitarized zone type of situation i don't think it necessarily needs to work out like that but um i'm not sure what's what's going to happen once this is uh this is being implemented yeah it seems that if that is implemented there would be a a social incentive to respect that web of trust and so that's what i'm just thinking about in my mind right now does the incentive to protect the web of trust exist and is it large enough to mitigate any attacks between the trusted nodes in that web of trust i'm thinking out loud

here yeah the the mechanics will definitely change and like another potential um solution would be to to really to charge money for this so you're just saying okay i trust nobody if you want to deliver an h you want to have me to forward an hclc you just have to pay for it and you're going to pay exactly what it cost me so if you send an hclc and it remains blocked for one day you're going to pay the missed the missed interest that i could have on my bitcoin if this wouldn't have happened um and maybe that's another way to do it and because it's lightning it's also easy to to create uh and accept these these payments so maybe that's a more elegant way to to solve the issue yeah financial incentives uh this is fascinating stuff and i'm just why i'm

very happy i have you on yeah uh we have you on the podcast this week is because your piece was followed up by uh another piece on the lightning network this week by antoine riard i just thought the two complemented each other very well and i think what yourself and antoine are attempting to do is just to set expectations uh straight uh be as upfront as possible with the limitations that currently exist on the lightning network and then from there attempt to uh paint a picture of a path forward in order of operations of how this build-out may need to happen going forward because Antoine referenced in his article this week the sort of push and pull between the conservatism at the protocol level and the, he used an analogy, Silicon Valley-like

startup mentality brought to the lightning network which may not be ready for that type of development at this point in time yeah well the same thing also is you could also recognize the same thing on a feature level if you're developing for lightning you can choose to add more features like i also worked or contributed to new features for example multi-path payments is a new feature key centers new feature you've got this whole tov and tov shop thing is also a features all new features uh which is which is cool because it also makes people more enthusiastic it also shows unique properties of the system things that regular payment system couldn't do it's also important to do this but on the other hand you have got maybe the not as spectacular

projects that further strengthen the foundation of of the lightning network so the things that antoine is talking about like he's mostly focused on like on on-chain security when transactions go to change can we get to convert in time um this is this is also something that needs to be looked at at some point at some point like if lightning becomes the payment system of the world uh the environment will be much more adversarial than it currently is at the moment it's not much is happening like given that these attacks have been known for such a long time uh and are not being executed it just means that the lightning network is not it's not a target so there's like there's the decision where do you want to work on do you want to add more functionality to make it more

usable or do you want to make it more more secure even though we don't have a security issue at the moment practically speaking so this this is a hard decision to make like where do you focus your your your time on but i think at this stage also with the bigger channels and more users more users committing money to it maybe it wouldn't be bad to to shift over a bit of the effort to the to the foundation site from the from the new feature site so that's basically also what i'm try to try to do by writing those tweets and making it more digestible because not not every user is reading those mailing list posts and not every user is reading all those pdf white papers so i try to take like the most important parts of it and maybe make a small demo or show some

transactions in the blockchain explorer to create more awareness of that and hopefully also create a demand for um fixing those security issues to stay ahead of potential um bad actors really like we want first i'd rather attack ourselves first before someone else does it yeah it seems most advantageous to me as well and so do you think people are too optimistic about lightning right now do you think people are moving too fast uh or do you simply think just attention needs to be diverted to the security side of things i don't think people are too optimistic about it because it is like an amazing invention really i like i've dedicated already two years of my life on working on this and i wouldn't have

done this if i wouldn't believe in this and during all those three two years i see no fundamental issue why this couldn't work like there are a lot of issues like plenty of them many also unsolved there's still a lot of work to do but i totally believe in the whole concept just thinking about this if i would be sitting on a cafe with someone else and i can make an off-chain payment with pen and paper basically that's what it is and it would be enforceable on chain it's just magically like some now and then i think about this again and i think is it really possible that math can do this and apparently math can do this so it's it's just amazing and yeah so that's very optimistic and also the fact that there isn't really like another second contender or another project that

is as far as lightning it's also has proven itself over over time yeah it was uh it was funny people took your tweet thread particularly people uh working on projects outside of bitcoin and claimed it as a death knell the lightning network i saw that yeah i saw that it's probably hard to communicate the right thing to everyone right well they obviously didn't read the whole thread where you presented a problem and then uh but at the end of the thread presented a solution in the form of circuit breaker it highlights the uh the inability of of uh some some reading attention spans that we have in this space yeah i also wouldn't want to say that circuit breaker is the solution to everything also if you look at it it's just a tiny tiny project at the moment

it's just it's basically a front end to lnd's hc interception api so it's nothing it's nothing really special but i just wanted to get the thinking going around this like a thing like circuit breaker do we need it do we want to install it already now or do we want to wait until until something happens i think it's just important to to get the thinking started yeah matt do you want to step in there well i mean from my perspective i mean i think i mean correct me if i'm wrong uh you know way more than us that it seems like the single best improvement routing nodes can make are these shadow private channels right oh i'm not sure if it's the single most important thing they can do but it's like it's an easy thing to uh to make it

better there's not there's no downsides to this the only reason why you want to publish this one channel of five btc is if you actually want to accept five btc sized payments but that seems yeah and that won't even be like when we get multi-part payments or more throughout the network then then that won't even be as big of an issue right that's also the case you could even make a decoy channel just make a 0.16 channel always just to show that you that the connection is open and leave all the rest hidden like i'm sure if that if everybody would do this like the routing capabilities would hardly be um uh would be hardly any effect on the routing capabilities of the network but the total locked value would shrink dramatically

yeah so we we've already come to the conclusion that for end users uh except for key send they should basically only be using private channels uh if they're not routing and then for routing nodes uh dedicated routing nodes the majority of their capacity should probably be in private channels right so so a successful lightning network in your opinion is going to have it's going to be like an iceberg the overwhelming majority of capacity is going to be in private channels not public yeah definitely and there's another reason why you want to have private channels it is because it makes pathfinding much easier because now you've got all these public channels that don't really serve a purpose for for routing but they are still explored when you send a

payment. So another thing that I've been working on last year is to try to penalize nodes that advertise public channels that are not functional. And the idea was if we penalize those nodes and we skip them for the next payment, then they will see their routing revenue go down. And if that's the case, they might choose to close or convert those channels into private. So create pressure from the senders on routing nodes to only advertise the channels that are actually usable. And if this would happen, this is another reason why the visible graph would shrink.

And this speeds up finding the path. It's also better for mobile clients, for example, that have limited resources. Yeah, but it's pretty weird because people always look at these 1ML stats and they see, ah, there's 1,000 BTC committed now. And last year, it was also 1,000, so nothing's happening. It's just a bad metric.

Yeah. Yeah, it's a stat that is used to throw a lot of fun at the Lightning Network. There's more RAP BTC on Ethereum than the Lightning Network. Yeah. It's hard to get through to people that are only looking at public channels. Bringing this back to multi-path payments and Wumbo, So focusing on the benefits of the combination of those two upgrades, particularly, I believe Breeze Wallet wrote a post about a month ago at this point describing how the combination of MPPs and Wombo channels will create a better UX while increasing liquidity.

I believe they described it as MPPs have an additive property to liquidity and flow on the network that has really made the user experience significantly better or will make the user experience significantly better when it's more widespread. Yeah, multipath payments are definitely a very good user experience improvement because it's always difficult to explain on Lightning what's your balance. like you can sum up all the channel balance that you have but this is not really the size of payment that you can can make like you can only make a payment uh smaller than the size of your largest channel and it's just hard to explain that and with mpp with an ideal mpp we're not there yet because the routing algorithm isn't that sophisticated but with an ideal mpp you would be able to combine all the balance that you have in those channels and then use it to settle a single payment uh at the at the recipient and yeah sorry go ahead oh if you're going to continue yeah

so and in a way it also there's less need for one more channels because if you can combine channels you don't need a one which on it anymore to make a large payment like there are other reasons why you want to have one more channels but mpp in a way also reduces the need for one more channels yeah and the that's interesting going back to the the routing algorithm that's another thing that uh the breeze team mentioned that actually it's just very interesting to me there's people are talking about applying something like a fibonacci sequence to to channel routes or so what are the options right now for the routing algorithm one is just to try to send uh actually i'm not even describe it you know it better than me yeah well routing our algorithm is very very difficult for

uh multi-paths because you have so little information you don't know you know the capacities of the channel but you don't know what the balances are so what you're doing is you're trying to complete the payment and while you're doing this you're learning something about channel balances because if a particular uh part doesn't reach the destination you get better failure you know okay one million through that channel that's not possible but there might be some of those charts that have already arrived from which you're not getting feedback because that's the property of lighting at the moment if your chart arrives and the recipient is still waiting for the other charts to arrive there's no feedback so it could also be that this chart is stuck so with that very

limited information you need to decide okay what am i going to try next and also you can't try too many things at the same time because then those hfc's will interfere with each other and they might make each other fail while in reality if you would have sequenced do them sequentially there would be enough balance on those routes so there's a lot of factors and also timing factors involved to actually make this make this optimal so I can only speak for what's happening L&D in L&D at the moment and there we've implemented something that's relatively relatively simple just to get it started to show that multi path payments can happen but that doesn't

mean that if you have three channels with a total capacity of one BTC that you can also make a payment of exactly one BTC because the algorithm needs to try out options and at some point you also run out of time like you can't try all options so this is this is an area that still needs like a lot of research to figure out what's the best what's the best way to do this multi-path route planning that's fascinating is it fun working on this stuff as a intellectual like trying to figure out these hard problems. Oh, definitely. Lightning is the most challenging thing that I've worked on.

What were you doing before lightning? I was doing also routing, but then for ships. So ships, their routes need to be optimized according to their weather predictions. Heavily influences like the safety of the crew and also the fuel consumption. So it was, in a way, a related optimization problem. Yeah, it's fascinating.

So are you able to apply anything that you used in that lifetime to the Lightning Network? Not really, not really. The Lightning Network is pretty special. Like basic pathfinding is the same, but the constraints are very, very different. Figuring out logistics for a while now. Is that what you draw you to the Lightning Network?

It's like, hey, this is a harder... No, no, no. What drew me to the Lightning Network is Bitcoin. And I wanted to, after like working all different kinds of projects at the site, I'm only watching Twitter and reading about Bitcoin. I'm not doing anything myself. I need to, I need to make a decision now. Am I going to contribute seriously? Why do I keep watching all the time? So then I, the contract that I was on at that moment, I didn't extend that. And then I took sort of a crypto sabbatical let's call it like that so i i used three months just to become an open source developer and i had my eye on lightning already um on lnd specifically and then within

lnd i was quite overwhelmed with like the number of lines of code it's it's very complex like lightning in general and also to understand the code base it's just hard like that even after two years there's like like whole parts of the code base that i never touched and don't still don't really know how it works so i was looking okay where can it start to contribute and there were a couple of issues around routing then so then i thought okay maybe maybe i should look at the routing part because at least there's some familiar familiarity in that for me um so yeah that's why i started to uh to work on routing in lightning and you contributed to open source before lightning or is this your first foray no never before how's the uh how's that been

comparing it to your prior life where i assume you had a product manager and a team uh dictating what to work on like how is this this broad canvas differ to that the interaction with the open source community any different i think it's very good like there you get so much direct feedback from users like users commenting on on pull requests even submitting pull requests creating issues that you see um like in the companies that i worked before there was like all like different departments in between like filtering stuff and it was much more indirect like you didn't really know who you were doing all these things for because you didn't know those people and this is this is quite different and also as a developer it's nice that your work is public so you can always show

what you've done you can show um the things that you found in reviews like your communication style because it's also pretty important in software development like the way you communicate with colleagues with users yeah so a lot of pluses yeah well we're happy to have you here you're getting some user feedback right now we're uh again we're big we're big fans of the lightning network and uh we were actually huge fans of uh another project you work on i forget what it's called whatsapp was it whatsapp yeah whatsapp yeah yeah whatsapp so sending messages over light network this is infinitely fascinating to me but you were describing to us before we hit record that it actually sort of contradicts the the solution you uh you present a lot uh in

circuit breaker yeah yeah it's indeed interesting because um so with whatsapp what you're doing is sending these tiny tiny payments and the message is attached to them and you can go do this back and forth um but the thing is with these tiny tiny payments is that uh if you are routing node and you have these tiny payments on your channel and you need to force close the channel it can get very expensive i've published another tweet uh i think was it last week or maybe monday um i think monday where i show what the transaction looks like so if you have a channel and you have 400 ATC 483 small ish HDLCs on them not I don't say one set but let's say 10,000 sets to this this transaction it becomes huge and also the fee on the transaction

becomes huge so it could be the case that over time it turns out that for lightning to work there is a minimum payment size that we want to that we want to have so that's the contradictory part to what's that yeah push pull what's that seems very like to me what's that uh just having a distributed messaging protocol uh on lightning it theoretically cannot be censored was uh just an incredible idea and people have put it into practice like i have the sphinx app on my uh on my phone and a couple others uh what's the one that uh john control me juggernaut juggernaut yeah i've played around with that too so it's been cool to see people run with the idea that you put out there and actually

build products that work yeah that's pretty nice and also i think um there's also other ways to create messaging on lightning uh without using one hclc per message because another model could be that we say okay the message relaying we just we don't do this with lighting but we just take our own like our own um out of band channel for that but those notes still need to pay each other so it's more like prepaid so one note pays the other note uh 10k sets to relay 10k messages and then a counter starts ticking and maybe if messages blow uh flow in both ways the the balance will go back to the other side again so you make the um the billing a bit more loose from the message passing itself and that could also be a way to get around this whole

thing that every message is an HLC and maybe the HLC shows up on chain and then it becomes expensive lighting it so like you can make these payments between machines so frictionless like it really opens up a lot of possibilities also for this so perhaps like a second version of what's that would be like with this style of building model yeah and you could also use the it's still in that model you could easily just use the pub key for encryption web of trust you can do that you can still do the exact same things but you just don't put it in a bitcoin transaction that's the that's the only thing you don't do i mean that makes more sense to me just on an external it just

seems cleaner just in general yeah yeah i think so too yeah and again we are very bullish on the lightning network and what excites me something uh that you just touched on as things like lsat and ln url off and the the fixing of the 402 error on the web like the thought that bitcoin via the lightning network can actually uh fill in a puzzle piece that's been missing on the internet at for decades is incredible. And again, it seems like a very worthwhile project to attempt to tackle. Even if it may be unsuccessful in the long run,

I think trying to solve these hard problems is fun. Yeah, it definitely is. Yeah, the 402 payment requirement, it's amazing that they figured out that code already back then and never used it. Yeah, it's pretty cool. It's almost like it was just waiting for Bitcoin to come along. Yeah.

Yeah. Somebody new. All right. And so let's envision a future where a lot of these security flaws that currently exist are figured out. Like, what do you think the overall potential for the Lightning Network is in the long run? how how will it change our lives even beyond a a simple payments infrastructure very very difficult question like what's the future going to look like

yeah i well there are some some clues like this it's like full integration in your browser like everything connected together it's totally frictionless pay buttons like for every webshop that you that you visit yeah that sounds nice to me but there's also like the interaction with uh bitcoin as a savings technology um they could also uh strengthen the bond between spectrum bond between on-chain and off-chain like if everybody owns on-chain bitcoins then it also becomes more logical to to actually use lightning to to spend them so yeah perhaps the gaming the gaming use case also seems to to become more important like i wouldn't really be able to set out the future like how this is exactly going to look

yeah what am i being quite go ahead matt go ahead well i kind of wanted to flip it on its head like what are your biggest concerns about lightning besides um this htlc dos you know denial of service issue yeah so in general i i don't have any fundamental concerns like there's all these security things that needs to be fixed like also in the ux in the end you don't really want to know that the channel exists you want everything to be transparent just use it as a regular wallet i can send i can receive everything happens under the hood and it's really a lot that needs to happen under the hood and it all needs to be automated like in a way that also makes it work uh as the user expects so work work work uh but yeah fundamentally the only thing i i would think

like if everybody needs to have a channel on the earth how much change space is going to cost so is that going to work in the end like this is a stage we're very far away from and i also don't want to dismiss any idea uh now because it might because there is a chance that it wouldn't be able to serve everyone on the planet uh like lighting is already like an order of magnitude difference in scalability than layer one. But I do wonder sometimes what's it going to look like. But probably there will be like some custodial things happening again, like people keeping small money

on a service that maintains the channels. It's already happening now. So that might also be a way out for that. But yeah, indeed, if opening a channel costs you $1,000, then it's not for everyone anymore. Yeah, I mean, we're already seeing, I mean, a lot of usage of lightning right now is custodial and i mean as far as ux and feature wise you can do a lot more yeah if you go that custodial route so i kind of wonder if there will be a balance

there i mean i know historically as bitcoiners and and me in particular like not your keys not your coins 100 but at the same time i start to see myself wander uh you know today i someone showed me this project coin os um dot io and it's this combined wallet that is on chain lightning and liquid all together like you can receive on any of those and you can send on any of those but it's custodial it's a web wallet too right yeah but it could be run through tor and like anybody can run it so like there's some interesting models there where like you're kind of distributing trust even though it's custodial the ux is completely different so i i'm kind of curious how that all plays out i thought you could i thought you could point

it at your own server yeah you could run your own server you could use it non-custodial you could use it custodial you could run it custodial for other people right there's like all these different and people have talked about stuff like chami and ecash and stuff where it's it's you know it's theoretically perfectly private but custodial there's like all these different little custodial things you could do and i don't know if that's a trust model we want to explore but the nice thing about bitcoin is it is permissionless so people can if they want yeah i'd say that if you manage to create a new financial system that has a backbone that's non-custodial and decentralized i think that's already quite an accomplishment and then if there are like numerous like parties

that offer custodial services is probably going to happen anyway but to decentralize the backbone That would already be amazing. Yeah, this may be a little controversial, but you also have to take into consideration too the incentive for people who want to make transactions that other people don't want them to make may be high enough that they're willing to pay higher fees on-chain to open up channels

and facilitate what others would deem as nefarious activities. So that could actually protect the lightning in the long run as we see it in the traditional financial system the drug cartels and kleptocrats and corrupt people around the world are able to pay to get their transactions pushed through the banking system which is supposed to be preventing those um so you mean that can happen or that can't happen in lightning no it can happen i think it will happen um yeah yeah it becomes very expensive to run a note then

you will get those issues probably but for me these are all worries like for later like if we get to that stage we're so successful already it's uh yeah it's a good problem to have um so i actually have two more topics that i know i want to cover uh while we have you here uh the first one is watchtowers uh eye of satoshi the first uh proper release came out today the first stable release came out today which i think is the first like proper watchtower um they is this is big news right like i we've been talking about watchtowers for years now yeah i i'm not not very familiar with watchtowers of course i know what they are roughly how they work um there's watchtower in lnd as well but it is it's probably

not the same as i as i as satoshi i don't really know what they what they have so i wouldn't be able to comment much on that matt's got uh got dogs he's shutting out of the room right now uh yeah watchtowers i mean it's something people have been talking about for a while well they are important they are definitely important to to have and also it's the interesting thing about it is that you don't really know if someone is using a watchtower so just the fact that there is a watchtower implementation available already has a deterrent effect probably on anyone trying to breach your channel because you never know if somebody's running those watchtowers so even without nobody running the watchtowers there still is a fact you just have

to code out there yeah i mean that's what i'm most excited about is just having them out there like i just want you know the the the capability being there so so the malicious actors are worried about it and maybe they won't make their move yeah it also doesn't seem to happen that much that these channel breaches is pretty rare yeah why i'm surprised you know i mean i've been uh my favorite way of of learning and understanding things is just dive in head first um and i'm just surprised that my channels haven't been getting attacked i mean i'm not a knock on wood you know but i i yeah but i've been using watchtowers where's the game theory isn't it like they don't know that you don't use watchtowers well now now everybody knows well now i started

using watchtowers right before this episode i'm just talking about before that but yeah i guess the case is like if you breach and you lose you you lose the full channel balance so there's a penalty yeah that is true that is a pretty big penalty um given that attacks aren't happening either um it's probably like the whole friendly thing still uh being the case in lightning yeah that it kind of bothers me it's like the calm before the storm that there's just no there's like no real attacks that have been happening that have been reported yeah i've also been like advocating a little bit for like the chaos monkey idea so having some kind of a note that randomly randomly does things that it can do just to keep ourselves tuned to this those possibilities

yeah it's like a white hat hacker like i'd like to see some yeah like a white penetration if you have like a regular company and you hire a penetration tester it's all right because they're going to attack you but on the lightning network you can say okay i'm going to pay someone to attack me but you are might be at the same time be attacking other nodes as well like because you use the network maybe you jam the network so it's it's a bit harder like you can't just decide this on your own sometimes and ask someone to hack you so and how effective is attempting to to test out these attacks on a test net uh how effective is that if at all yeah you can get it done on the test net like the problem with test net is always uh the the fee estimation like the blocks

are never really full and a lot of these attacks have to do with fees and congestions so you would need to figure out a way to um to simulate to to simulate that but i think if you do it on test net it doesn't really contribute to um making it important because only when it happens on main that it it hurts and then uh something is might be happening like that these attacks exist and that they work that that's that that is clear like it's not nonsense what's written in those papers it's just it doesn't really happen so you need to decide prepare for something that doesn't happen or do something else that i need today yeah no because there's been attacks that have been able to drain channels correct um that have been made public within the last year

well there are there are still a lot of things that you can do like the whole reason that you have anchor channels is also uh so this new commitment format with anchors the whole reason that you have this is to to patch a hole that was there like if you're stuck with a commitment transaction that is pre-signed and the fee is not high enough and you publish it to chain if you you need to and the fee market went up uh since it was signed it's not getting confirmed and then you might reach the deadline of your hclc and once you pass the deadline uh both parties can claim the hclc if the race exists so that might make you lose money so but not that nothing exists at the moment yeah but for all these things so there's also trade-off with all these things like

some of these attacks also require significant investment for example in setting up channels and also it may be dependent on the fee market conditions which you don't really know so it's also not easy to to do those attacks you'll need to write custom software to do it and then you need to calculate what your odds are of actually succeeding so it's a lot of work there it's as someone on twitter said as well it's probably easier just to scam somebody out of their seats and yeah i mean i like that we saw some guy he lost 1400 bitcoin in his electrum hot wallet right that's larger than the public capacity of lightning yeah it's a lot easier just fish them with a link so this is another reason to keep the public capacity down fair

good that's a good point make it less of a honeypot yeah lightning is not a success just stay away from it right i like that line of reasoning um my so my other question is also probably something that so you said you weren't like that in tune with watchtowers um i was just wondering if you have any thoughts on liquid oh no not at all liquids i know nothing about that really fair enough it's it's really impossible there are so many things going on and even within the lightning domain there's a lot i can't keep up with so it's yeah you just need to choose what you want to work on yeah very fair well i have another question particularly around the multiple implementations you've mentioned you've been focused on l and d specifically and this is a

question i like to pose to lightning developers whenever i have them on the podcast is is what are your thoughts on the different implementations how they interact and how they may evolve going forward together yeah that's a that's a good question it's definitely good that we have multiple implementations like for example with the anchor with new anchor commitment format other implementation also start implementing this we find inconsistencies and and work those out just the fact that we all try to remain interoperable it just uh makes the whole thing more more robust like there's also timing timing issues that might not show up if you have two of the same implementations communicating and with two different implementations they do show up

so it's very valuable in that sense and also it seems that the different implementations have different uh things that i find important to work on something like anchor channels is like universally uh supported but um yeah as an as a project team you can also decide on what your own priorities are within that and influence each other like with mpp for example i believe all implementations have a different algorithm to do mpp so it's very interesting to to look at each other's approach and see if you can can come up with something better so there are more than three implementations but i'm talking about like the three uh ones that are mostly in production the moment yeah it's been it's been fun watching them grow alongside each other and i guess a

follow-on question on a follow-on question here would be like how hard is it to maintain interoperability how how much communication is going on between the different implementation teams yeah it's there's quite a bit of communication like we've got this two week bi-weekly uh why weekly is that every two weeks no that's twice a week right no it's both it's confusing part of english every two weeks we have this irc meeting and we chat about uh issues that that exist and they are often about interoperability like some things don't need to be discussed because every implementation can decide for that on their own and then we've got these prs with spec changes the spec repository itself is pretty cool like there's one repository

that describes the lightning spec um and there's a lot of like very lengthy discussions um you need to have a lot of patience uh in that process but in the end a lot of good things come up as well like things that you haven't thought about um i think in general like humans are just not smart enough for software development it's always a fight against the complexity so you need to do whatever you can to make sure you minimize the number of books that you have and getting a lot eyes on something it's just very very valuable so what i've experienced from this whole cooperation process i find it pretty good nice and looping back to your comment earlier that when you first approached lightning lnd specifically it had so many lines of code is the protocol

in that implementation working to uh reduce the overall amount of codes or amount of codes lines of code uh is it is that not a focus is it is it more refining right now or add additive uh it's still it's still additive because there's still a lot of things that need to be that are missing especially the automation part making sure that you don't need to worry about anything you just start up your note maybe you can even like stake some bitcoins i just throw some bitcoins at it and it will open channels for me and earn routing revenues everything transparently like we are far from that stage but it could happen at some point and i think similar discussions uh happen in bitcoin core is that a split will be made between like the

more core functionality like just maintaining the security of your channels uh and uh functionality that's more in the periphery and the split that make two projects out of it but like the total code base will still be growing but at least it becomes more more manageable but i don't think shrinking is really an option like if you're lucky you may be able to delete some code perhaps we with a new commitment format or different layer one possibilities there might be something that can be deleted but in general i don't think it will be it won't be won't decrease much interesting thank you for these insights just uh and thank you for your hard work and uh vocalization of everything that's going on the lightning network again

i want the lightning network to succeed and knowing that people as smart as you are working on it gives me uh some comfort at the end of the day matt's got a question here yeah um so a lot of us are kind of hoping that lightning uh becomes or like plays a major part in being a user-friendly option for bitcoiners to to have you know relatively decent transactional privacy with uh with their usage of bitcoin uh do you think do you think that these um beliefs are founded like in reality do you think that this this could be a a a way for us to to achieve some level of privacy for the end user where they don't really have to to know like all the specifics yeah well there there is a level of privacy by default but you

still need to know like it's limitations like if i make a payment to you uh through marty and you ask marty where did the hlc come from and he tells you it came from came from me then at least you know something like i i could not be like the the sender but just relay but um it's not that it's impossible to figure out where it came from so i guess i guess to re reword my question uh because we've talked about that a lot on the podcast and i absolutely agree uh i think part of the reason is that improves privacy so much for bitcoiners is just because the privacy guarantees of on-chain bitcoin is just they're so low to begin with um i guess what my question really is is you've been in the weeds in lightning development do you think that improving

like accessible privacy for bitcoiners is a goal that is shared among a lot of lightning developers i i think so this is a theme that comes up quite often and if you talk about like new changes to the spec the whole privacy thing uh it yeah it comes up uh often like uh for example there we talked about amount randomization to uh like never send round amounts but add a bit of noise with mpp don't split in one plus one btc but make it 0.3525 and the remainder so yeah but awesome maybe in the future like suppose uh routing nodes need need to go and do kyc let's suppose that's the future then like all those writing nodes will keep logs of hlcs where they came from where they went through and then yeah you don't have the privacy anymore so it can get pretty messy pretty

quickly yeah yeah so it's not like with the uh with the um the coin joints where there's like mathematical properties that make it impossible to figure out where it's uh what input went to what output like with lightning it's it's different but it's also a matter of like how hard is it and how much privacy do you need for whatever you are buying like there's probably also limits to what information someone can obtain so yeah and how hard would it be to enforce that regulation how expensive would it be and like where the priorities are right like i'm thinking of things like with multi-part payments and like enforcing more hops right even something as simple as enforcing more hops for or like looking for routes that have more hops rather

than less hops could be like little things like that little priority things that might result in higher transaction fees but they will result in better privacy guarantees yeah yeah that's the level that we currently really don't have we can't set like a minimum hop length a minimum route length but from a privacy perspective that would be that would be good it's a good point matthew i never thought of that makes a lot of sense though yeah it's a rabbit hole uh well just uh again i appreciate your time do you have any any parting notes anything you want to get off your chest before we wrap up here that you think the freaks should know?

I'm just interested to hear any feedback like on the security themes that I tried to shine a light on and also looking for feedback on this whole circuit breaker idea because circuit breaker is one approach to tackle this, but there's also other ways to tackle this. So in general, any feedback about security is very welcome.

awesome where uh where can we find you uh whether on twitter uh github uh anywhere that yeah i'm on twitter and github that's it indeed so it's a yoast jgr on uh on twitter and uh yoast is probably very hard to to spell this maybe you have show notes or something that you can put it in yes we'll put it in the show notes definitely uh but i'll spell the twitter for the freaks it's j j o o s t j g r on twitter right awesome well yes i hope you have uh an incredible rest of your evening i know it's uh getting late where you are uh matt do you have anything to wrap up with no just i i truly appreciate you and thank you for all the work you're doing to improve lightning yeah thank you for having me a very

enjoyable conversation yes i uh i'm very excited to get this out to the freaks i think this is going to be very educational for everybody who's interested in lightning that's all we got this week freaks peace and love