LINUX Unplugged - 498: Rolling Papercuts

Episode Date: February 20, 2023

Sometimes running the latest and greatest means you have to pave your own path. This week two examples from living on the edge.

we have two big milestones coming up for the show. And this is kind of exciting. First of all, episode 500 is precariously close. Oh my. Yeah. And just doing the math, we're almost a 10 years of the unplug program.

Oh my goodness. Yeah. We're sitting at 9.5 years right now in August ish, 10 years. Holy what? Isn't that something? To me, this is still like the new show. I feel old.

Yeah, well, you feel old. To me, this is the show I started to read email from Linux Action Show, and I just needed a place. Well, come on, dude. Like, I started my professional career at the same time that LUP started. Whoa. Well, now I feel old. talk show. My name is Chris. My name is Wes. And my name is Brent. Hello, gentlemen. Coming up on the show today is high performance remote desktop like an oxymoron? Is it even possible? And is it doable on Wayland? Well, today we'll talk about Moonlight and Sunshine and share some tips that

you might use for years where it's worked for us and where it's fallen down. And up first, we'll talk a little backups with brent get an update on what he's been working on getting all that kind of going and then we'll round it out with some great boosts and picks and a lot more so before we go any further let's say time appropriate greetings to our virtual lug hello mumble room hello hello everybody great to have you there and those of you up there in the quiet listening. And this is pretty exciting. We have converted a virtual lug to an IRL lug.

Casey's here. Hey, Casey. Hello. Hey, welcome in studio. Thanks for joining us in the red seat. It's actually Levi's seat you're in right now. I'm sure he can fight me for it. He tried when I walked in the door and he'll probably try again in the future. He'll just turn those big eyes up, you know?

He does that. Also, I want to say good morning to our friends over at Tailscale. Tailscale.com. Tailscale is a mesh VPN protected by Wirecard. It's so slick. All your machines talking directly to each other. Go try it out. We love it.

It's going to change your networking game. VPNs? No, I use Tailscale. Go try it out. We love it. It's going to change your networking game. VPNs? No, I use Tailscale. Go say good morning. Try it out for free for up to 20 devices at tailscale.com. Tell them we sent you. Tailscale. So episode 500 is just around the corner. It sort of boggles the mind and we had to do a little get together to celebrate. So we're going to have a little micro-meetup Saturday, March 4th at 4 p.m., and we have the meetup page up if you go to meetup.com slash jupiterbroadcasting. It's going to be at the 192 Brewing Company in Mount Vernon, Washington, and I expect it to be kind of an intimate gathering. I hope everyone in the area is able to make it, kind of an intimate gathering. I hope everyone is in the

area is able to make it. But of course, I understand if you can't. Brent's going to be there. Wes is going to be there. Casey, I hope you're going to be there. Of course I will. And we'll be having some brewskis, some snacks and some drinks. It should be a lot of fun. I think we got about five or six people so far. It's not going to be a huge thing. Yeah, I'm just realizing I hadn't joined the meetup yet either. So I fixing that right now well there you go get signed up and it's just kind of a nice way to kind of reflect on the last nearly 10 years of the show hang out with the listeners if i could you know i'd be doing some ginormous huge meetup somewhere in the center of the country or something like that if i if we could pull that off. But this is what's

within our means right now. I thought we were going to do that all expenses paid Hawaii. For episode 500, right? You could justify it for 500 episodes. How rare is that? I do really feel like 500 episodes is pretty special though. You know, in that sense, because not only is doing

something for 10 years every single week kind of rare uh but having a any kind of media content last for this long feels super super rare so uh i'm going to mention that uh i'm going to start collecting uh thank you or congratulation messages if you do want to send them in for episode 500 and i do have a special boost amount that i came up with for doing the show for 9.5 years if you want to have a message that i that I came up with for doing the show for 9.5 years. If you want to have a message that I'll collect and set aside for episode 500, boost in with the amount 9,529. That's how old our podcast is, 9.5 years.

And you can also come up with your own amount. Whatever numerology works for you, just make it clear that it's for the 500th episode. Can I get that amount one more time? Yeah, sure. It is 9,529 sats. And I'll tag all of those for reading in episode 500. I'm curious about the 29 portion of it. Can you give us some insight there?

I think it works out to the, when Hadee and I were doing the math, that worked out to like the days. 9,529 days or something like that. I forget now i was trying to come up with something that i could clearly mark as for episode 500 i guess you could do 500 000 sats i'd memorize that too that's pretty uh you know it could be also a great opportunity for a behind the scenes type question if you've been listening to the show and you've wondered

about how something works or uh anything wants to follow up on or anything kind of like a production question what is brent's real name right right never know yeah how did we meet brent how does the why does jb have a studio i mean i don't know what your questions might be but you could send them in does the studio exist is there a studio is wes a robot is well yeah mean, do we want to reveal that secret? That's one of our trade secrets. Maybe at 1,000. Are you saying you won't answer the question's truth, really?

I mean, that question? That's like the one question we can't answer, for some reason. As an AI chatbot, it'd be inappropriate for me to... Yeah, there's certain things that we don't want to touch on. But yeah, if we make it to 1,000, we could talk about the Westbot AI. So you can send those in now if you'd like for the 500th episode.

We'll be collecting those. You know what? The other thing I thought about is we have tens of thousands of new listeners over the last few months. And there's got to be questions they have about the show. Because there's stuff we don't even talk about anymore that's just sort of baked into the show. So if you're a new listener too and you have some questions that we haven't touched on that's a great opportunity to send them in all right so meetup.com slash jupiter broadcasting for the meetup and uh 9529 sets if you wanted to automatically

get categorized as a 500th episode boost but brentley why don't we shift gears and talk about backups here on the show because this is a topic that we keep bringing up and we've started behind the scenes kind of doing some of the work and you have been deep diving into Borg, Vorta, that whole ecosystem and have raised some great questions. Yes. Thanks, Chris. I realized during our discussions these last few months about backups that my situation was actually worse than I thought. So I figured, OK, well, I should probably heed the advice of listeners. And so I chose Borg as a more modern way of moving towards doing backups for a Brent. And actually, it's been pretty great.

It has been with Vorta's GUI. Really great to use. It was a little bit of work to get it set up, but not anything daunting. Chris, I think you had hesitations when I chose Borg, but actually it's been fairly good. And I know you can use something like Rustic as a common Borg alternative. So if you're thinking about creating your backups, I would definitely suggest looking at it. But some things that started coming to mind as I was setting this up was, which I often suffer from, is like implementing solutions that actually are quite outdated. Like, is this so 10 years ago?

And I'm coming new to a backup solution that's maybe 10 years old. And I got thinking that because we've been dabbling with more modern file systems, you know, things like butterFS and ZFS, and they have send functions that do a lot of what Borg and Restic, for instance, do in the backend, which is some deduplication, some compression of your data. And that got me thinking, well, some compression of your data. And that got me thinking, well, is it time to move to using these file systems as the backup solution, especially locally versus using something purpose-built like Borg or Restic? And I don't really know if I have that answer. So I wanted to bring that conversation here and see what we can come up with. I kind of feel like this is a deep question. Are we using backup tooling

that was created before file systems solve some of these problems? And so they are doing some of the same functions that your file system could do. And I think the implication there, and I think this is where maybe it might resonate with you, Wes, the implication there is you could just build simple tooling on top of the file system and avoid a lot of this complexity if you just use modern file systems and modern tooling where do you land on this well i suppose it depends on you know one of the advantages of a sort of third-party system is bring whatever stuff you have right so compatibility then across different functionalities yeah exactly and fewer upfront sort of assumptions about how you're going to set things up.

And I think maybe Brent had run into some potential issues like you just need more sort of upfront architecture to get it right. You need to make sure that you're able to use those file systems in all the places that you want or come up with a method for sort of retrieving and ingesting those files into the file system from other third-party sources that can't use it or something. Or what about for the sake of argument, though? Let's say it's like Brent, me, or you, where we could, in theory, use the same file system everywhere. So you could assume either ZFS or ButterFS is omnipresent

in all these systems. Is it worth building this entire orchestration of backup management software when you could just maybe do a zfs send and send the data somewhere else and back it up that way like you get where what i'm worried about i think what what brent touched on that really made me kind of think about this was is oh oh yeah am i am i just duplicating effort here yeah i suppose so but um i think it depends on what kind of flexibility you want do you have a backup system when you're running a big old array? Or are you, you know, as Tiny points out here in the Matrix room,

you know, file system backups are something like ZFS or better FS. You got to have like a, you know, a NAS or a cloud system or something that's running that versus a lot of these other systems can back up to stuff like object storage. Yeah, that's true. Or Backblaze or TarSnap or something like that. Now, there are stuff, right, especially for ZFS,

there are hosts out there where you can just send them, you know, you can ZFS send to them. So, there's middle grounds here, and are you comfortable with those tools? Do you like that? Does it meet your needs? I know Butterback seems like it's got a lot of this stuff sort of figured out, so that's pretty tempting. I think Brian should try it, personally. Butterback? Well it's got a lot of this stuff sort of figured out. So that's pretty tempting.

I think Brian should try it personally. Butterback? Well, just, you know, going down the file system route. Just to see like what are the ergonomics? How do you like it? How well does it work when you are in a similar like remote situation? Because that's the only way you're going to really know. Well, I will admit to having tried to solve this question for myself this last week or two. And especially with

Butterfest, I'm not that well versed in ZFS yet. I will get there. I will get there. But I've been trying Butterfest and trying to get that to work in the similar ways that we've been mentioning, using even, Wes, what you mentioned, Butterback, which seems like the suggested standard for doing automating backups with butterfs send and receive and i've been struggling i you know i'm just trying to do this on my local network first to try to get it to work and then i'll test the remote stuff later but i've i've really been struggling to even get it to work once um a lot of the tutorials that I've found for ButterFS send and receive especially have been just to a local mounted drive. And so not very much stuff over the network, which seemed kind of strange to me. I have looked and looked and looked and the best documentation

I could find for accomplishing such a thing was actually the Butterback documentation, which is excellent and very in-depth. And even following that, I just haven't quite gotten to a successful path. And so that's why I thought to bring that conversation here is I'd like to solve this. It seems pretty cool to use your file system to accomplish a lot of the things that these backup systems have been doing because like it's there it's gonna do it anyways whether you want to or not if you have it set up so but i have had a hard time implementing it and i thought geez isn't that a shame there's there are all these features that it seems like should be fairly straightforward and so i feel like maybe i'm just missing something obvious or maybe someone

has some hints for me if if if you do please send them in linuxunplugged.com slash contact well maybe we need to know like what what went wrong what have you tried where did you and where did you where did it not connect yeah it's a really good question i think many of my issues have come from two places one of them is with using SSH and requiring some super user permissions on either end. And so there's some advice to enable things like root logins. And thankfully, they say, you probably shouldn't do this. But if you just want to get up and going really quickly, you could do that. So there is some advice out there on how to solve the ssh problem and then the other problems i'm seeing seem to be having is mismatches in

butter fs structures like open suza has their kind of way that they like to organize snapshots and things on the file system and i'm trying to back up to you know an ubuntu system that has butterfs and it seems like there's a small disconnect there at least in butterback i found some evidence that they haven't quite implemented the ability to do root level file system backups the way that opensusa. And so there's just a lot of little weird gotchas that I've been running into that I, to be honest, didn't really expect. And I was curious for those who have successfully run this kind of stuff, like, am I holding up to my title of just running into all these little strange edge cases? Well, no, I think you're running into a lot of the pain points

when you have mixed systems. I want to, you know, so what I see here, Brent, I think you're running into a lot of the pain points when you have mixed systems. I want to, you know, so what I see here, Brent, are really two questions. Is it a backup system that has reporting and error management? And I guess what I'm trying to say is logging. Is that what maybe is best for you? Or is it using the primitives and building your own tooling around that, which would be like the file system, snaps be it butterfest send or cfs send i think that's your core question here and i think for me it feels like i would be worried about becoming a slave to the tooling i i would be worried that to get to my data i first have to go set up a borg backup system and then i can get to my data

and when i need my data i want it right then i don And then I can get to my data. And when I need my data, I want it right then. I don't want anything between me and my data. And that's honestly, what's just great about tar and things of like that nature is it's really just kind of built into the system and kind of you build the tooling on top of that, that you need.

So we're shipping printed tape drive. Well, I honestly, I wish tapes were more accessible and more reasonable because I actually think it's still a very good backup medium. I mean, you kind of joke, but it's a ton of storage. There's a lot of software that still works great with it. I love tape backups. I just don't have it as like an accessible option because whenever you get to the kind of storage I'm looking at, which would be 30 plus terabytes, it gets really expensive. Well, Chris, I will say that my hesitation to

adopt any of these like more modern backup systems has been exactly what you mentioned, which is, you know, I've been on the path of learning more Linux-y stuff and I'm having a ton of fun with it. But in the past, I always wanted to make sure that no matter which system, you know, if I was just booting up a live environment, I could just access that backup data because maybe I'm traveling and I have this backup drive with me. And it's that point where everything's going wrong and you just need to get the one thing done. Right. And so there's been something really attractive about that. and so that's what kept me on on solutions like using rsnapshot for instance which just uses rsync on the back end and helps you automate some rsync snapshotting which does use hard links to do deduplication it's not perfect because there's a lot of things wrong with it but it's been good

enough up to this point but the audience has convinced me that i can do better and so i thought well i gotta look at some of these more modern ways of doing things. Because, come on, why am I doing it like a caveman over here? I mean, it is embarrassing. We don't want to say anything. I would say consistency is nice, though. So one plus to being, if you say, a 10-year-old backup system, that's a good track record. And when you're talking your data.

Because I think it's easy to over optimize sometimes you know and like but like if it works if it meets your needs it's whatever it can duplicate up to the amount that matters for you or it you know it has the set of features that for your particular case you need that might be worth it and you successfully can and have recovered data right yeah you know how it works you? It's easy to return to and you're happy to know. It's easy to understand six months down the road. There's that too. So Brent, you're overthinking the whole thing.

But I'm having fun. Right. No, no, no. That's all fine and good. So your underlying problem is that you can't figure out how to reliably move Butterfist snapshots from one system to the other and then look at them later for recovery purposes, right?

So all you have to do is just put it in a, like when you do a Butterfest send and receive over the wire, you can just, on the receiving side, just put it in its own folder and the entire hierarchy will just be isolated that way. That's how I actually do my backups. I have an archive drive, which is a little portable SSD, and it has a

folder representing each computer name. And I just put the whole hierarchy inside of each folder in there. And that makes it so whenever I want to go look at it, I can just go into the hierarchy, take a look, and I can see the whole setup as it was.

If I want to bring it back onto the system, I can reverse migrate it by doing a send from there back to the root hierarchy of the computer I want to move it to at any point in time. Or even into a subdirectory so I can just pull some of the data out or just look at it and copy files or whatever. In this case, simpler is better. And I have way too many systems with different configurations to worry about any of that stuff. Well, I think that's pretty good advice, Neil. And I did think of you often as I was playing with this. It's like, what would Neil do? Neil avoids all the weird tools because it's just not worth it. Well, but I think one aspect that maybe you missed in my description was that it sounds like, from what I gather from your

system, that you're using a USB drive and you just kind of plug it in and then you ZFS send to it and then you unplug it and that's your backup. And that's fairly straightforward, I think. What I'm trying to accomplish is doing that over the network, which I've been struggling with and haven't seen very... It's same principle. Yeah. I just can't get it to work. It's a funny thing. The thing, the reason I don't do it over network is at home. I don't actually have a desktop computer worth backing up. All of my computers are mobile. And so it like network-based backup falls apart really quickly in that scenario. Now, when I was at, at, at my previous job,

I had a lot of stationary computers with good network access that I needed to back up. And in that scenario, I just made a disk that was iSCSI mountable and then did that. Because if you mount a disk over the network and that disk is formatted ButterFS, you're done. Also, even if you don't want to do this, another pro tip, if all you have is NFS, you can also just send and receive the other side as a file. You can send a Butterfuss stream and capture it as a file. And that file can then be catted and replayed back anywhere to restore it as a Butterfuss subvolume. Yeah, it makes it quite flexible as we're seeing, right? You've got to just shuffle those bytes across systems

some way or another. It doesn't matter how, it sounds like. You know, like our friend Netcat. Netcat. If you're doing it over the network, there's a couple of ways to do it. The Netcat way would also work, but that's just too much Unix for my blood.

Oh, man, Wes. But when you're sending a ButterFS send stream also work but that's just too much that's just too much unix for my blood oh man wes but like when you're sending a butterfs send stream over the network you can do it with a tcp connection through netcat or whatever um or an http stream some other wrapper terminal whatever on the other end you can either receive it as a file or if the other end has a butterfs disk you can receive it that way the cheap way which is the way i it, was I just mounted the disk over the network directly as a network block device. And then I formatted the network block device as Butterfest, mounted it on every system, and then just did the normal Butterfest send and receive. There's a lot of different ways to do this. ways to do this probably the simplest way for you to do it is to uh if you don't want to do the whole trick of like tunneling over a network to another computer is set up a network block device

and mount it remotely and make sure that block device has butter fs on it and then you can just do sentence tree as if it was local even though it's over the network that's the easiest cheese way to do it well maybe the easier way would just be to not make any data that I need to back up. Brantley, Brantley, stop it. Delete your photos. Delete them. There is no such thing as data you don't want to have backed up. I feel like I'm in a room with friends.

I have worked at a backup company for eight years. That makes you biased. I'm in a room with friends. I have worked at a backup company for eight years. That makes you biased. Yeah. Trust me when I say there is no such thing as data you don't want to have backed up because you're going to regret it no matter what if you don't.

Linode.com slash unplugged. Go there to get $100 in 60-day credit on a new account. And it's a great way to show your support for the show while you're checking out the Linux Geeks Cloud. They got 11 data centers today and they're expanding to a whole bunch more, another dozen this year, plus some other really cool plans.

They got it figured out. Like they got a massive, massive rollout coming up this year. It's really exciting. I'll talk more about that as it gets closer. They've been really doing this for a long time. And so if you like to build it yourself or if you want one-click type deployments, they can accommodate whatever level you're

comfortable at. They have excellent options. And at every level, the performance is incredible. You guys know me. You know me. I wouldn't be hosting all of Jupyter Broadcasting stuff at Linode unless it was super fast. And they've been rolling out screaming fast upgrades the entire time I've been a customer. They don't just just sit still and i really think that after you use it for a minute you'll see why we love it so much when we use it for projects just when we want to try something out if it's something for like a personal thing i'll you know i'll spin up like a kid's minecraft server or if it's something that we want to withstand thousands of people downloading files simultaneously we choose linode every single time.

And after you've been using it for a little while, you'll also like the features, like the API, things like their S3 compatible object storage cloud, firewall, super fast networking, just all kinds of things. And every now and then, you'll just be working along inside Linode and that Linux culture.

You get a feel for it. You see it, it's baked into the product. It's in the DNA of the product. I think it puts them just above everybody else, especially the hyperscalers that want to lock into their crazy platforms. Go try a better way to do things, either a DIY way or a one-click deployment type way. Whatever you're comfortable with, try it out at Linode and see what I've been talking about. So let's put it over the top. Let's go get that a hundred bucks, support the show, and then you can really kick the tires. Linode.com slash unplugged.

That's where you go. You try it out, you get the $100, and you support the show. It's Linode.com slash unplugged. All right, so when I decided here in the Studes that we were going to deploy Wayland on our new systems. You would accept nothing less. It's at a high bar. The bar was so high, Wes. It was so high. You know the

one thing I never considered? I thought about all these other aspects like OBS and all that, but I never considered remote desktop control. Sometimes you want to not be in the studio. Take yourself a little break. I like to go out into the woods from times to times. And I kind of had this problem solved for a hot minute on Gnome using Gnome's new built-in RDP server.

Right. Weren't you using that with Dylan's laptop? And that worked with Wayland. However, here in the Studes, we went with Plasma. And there's no RDP server. Well, we'll get to that. we went with Plasma and there's no RDP server. Well, we'll get to that. And we got a note from several of you out there that said essentially, hey guys, you got to try out Moonlight. Moonlight is really slick. And Moonlight, you can think of it as

basically, it lets you build your own Google Stadia. It's really focused on gaming. The idea being that it is an open source implementation of NVIDIA's GameStream protocol. And the Moonlight project implemented that protocol that's used by the NVIDIA Shield, and they have wrote a set of third-party clients for it. So you have Moonlight, which is the streaming client. And then you need a streaming server. And that's the system whose desktop or application you would be controlling remotely. Yeah, your host. Yeah. And so Wes and I found Sunshine.

It was also sent into the show a couple of times. And Sunshine is a game stream host for Moonlight. So you've got Moonlight, which is the client, and Sunshine, which is the server. Well, isn't that cute? Yeah. And Sunshine is a self-hosted game stream for Moonlight, and it focuses on low latency, it focuses on hardware encoding, software is available though, and those things all check the box with me. And of course, it's Wayland compatible. It provides a web UI once you get the service up and running, a web UI that you can configure a few things.

It's available as a native package for a lot of popular distributions and it's available as a flat pack. Oh, whoa. And an app image, actually. So it's easy to get, which is, I mean, kind of something you want in this kind of technology. Yeah, for the most part.

There are a variety of Moonlight clients that are available, including mobile, Pixel, iOS, you know. And, of course, there's a Moonlight client for the Linux desktop. I was looking for something that we could use in an X11 and Wayland world that we live in. And I'm looking for something that could work over a pretty intermittent connection when I'm remote. And so my setup was a couple of machines. All of them have tail scale on them. So I didn't need something that would do any kind of Internet proxy or routing. Basically, yeah, your flat mesh network.

Yeah. And I think it was, what, after last week's episode, you and I were like, hey, let's try to get, let's see if we can get it working on the OBS machine. The OBS machine that we have here in studio, we decided to do it in hard mode. So it's NixOS with Plasma Desktop with Wayland and Pipewire. So it's like...

Making it really easy on ourselves, aren't we? We're just basically running all the latest and greatest Linux technologies, so that way we can sit here and tell you about it and you don't have to. But man, does that make it hard to just snap into any kind of management system? Like RustDesk, for example. I love RustDesk. Which we previously had running on all the machines. Yeah, but it only works with X11.

And I don't think they have any intention of supporting Wayland. So we set it all up and we gave it a week to bake to see what it's like. And so I've been trying it for the last week, and I wanted to tell the audience my impressions so far. I'll run you down the setup. So you have all the machines that are all connected on Tailscale. They're running Moonlight on the clients, Sunshine on the clients sunshine on the servers when you first connect to a sunshine server you have to exchange pin codes between the moonlight client and the sunshine server you need to have access to that web ui to enter the pin code this is already an immediate challenge because oftentimes

i'm remote on a blank new system and i need to without planning ahead of time it's sort of an unplanned thing comes up i need to connect into the studio system right you got to do this pin exchange so that meant i need to build infrastructure that allows me to remotely connect to the systems and exchange a pin code before i can even establish a remote desktop connection so you got to configure this entire thing to talk over tail scale so that way you can remote in add your pin to your local host to that you're going to be remoting into and then you can finally connect you do that over your first connection but once you've exchanged pin codes you can then just click on connecting and and the sunshine server can give you a full desktop it can give you a low-res desktop and it can run applications directly as well.

It'll also forward sound if you have all the dependencies. So when you log in, you get remote sound when you, and it is performant enough that I was at home on Starlink connecting into the studio systems, and I could watch a YouTube video. I could play and pause it. I could watch the video. I had audio. The audio was in sync with the lips. Very impressed by that.

That aspect of it worked really well. And the client will actually tell you, hey, your bandwidth's a little rough here. And there's a good amount of options. You can go in there and dial it down. By default, it's using 60 frames per second like that's a bit ridiculous right for games that's great right but for desktop so i turned that down to 30 frames i i dropped it down to like four megabits or three megabits and you can kind of fine tune that and get it right where you want it is impressive the performance uh it's almost real time enough that you could use one of these remote clients as just your mouse, and it would capture it.

Here, I'll play this for you guys. Okay, so here, let's look at the performance as I switch desktops. You can see it's right on the money, and this is running on the iPad Pro. We have Moonlight here on the iPad Pro, and it just looks like a Linux box. Look at that. It looks like I'm just running Linux here. And I would say the performance is good enough that you could use this all day. It's almost right on the money so much that you could just use it as a mouse. And that's on Wi-Fi, you know, completely wireless. On Wi-Fi, on the iPad, over tail scale.

That's a pretty good little test. Yeah, and it just is so smooth that you could almost use it as the mouse if you needed to. And the fact that you could bring full applications through means you could bring, like, in our case, our recording application through, and I could check on that. And it's just so, so smooth. The problem is there are so many little gotchas. Number one, you got to have your video acceleration dialed in. So I tried this, Wes. On the OBS machine, we don't have it all dialed in and I was having issues. So I, you know, I love NixOS. I love it. So I broke the Intel video acceleration on my laptop and verified that the Sunshine server was not working. And then I fixed it by re-enabling video acceleration via the Nix config, reboot, and Sunshine works great. So it does need on Wayland hardware acceleration. If it's Intel, AMD, or NVIDIA, you can pick one, but you need to make sure it's all working because it will use hardware acceleration to render the desktop back to the client and it just doesn't work without that

does not work without that so there are those types of things you have to sort out it's not perfect and there were scenarios where it was a little crashy depending on what i was doing the reliability seems to be really pretty solid when you're game streaming like if you're just telling it to like pull a game from steam and just stream that to you directly, or you want big picture mode, that seems really solid. When you're doing the whole desktop, there's just a lot of little edge cases that can cause the session to crash. It's usable and it feels like it's a, it feels like I'm putting together a bunch of different pieces that i can build something with and that it's worth spending more time figuring out but i can't say it's a rust desk replacement at this point i just can't say that it it needs more

work than rust desk like rust desk if you have all x11 systems which i i think most people probably would you just install it and it's done you know with wayland and plasma and using sunshine they're all layers every piece of that stack has to be working right for all of this to work and that's tricky sometimes so it's not just like install a deb get a get a service running in the background enter in a pin and you're done it's more like make sure you have full video acceleration working. Make sure you've got all this stuff figured out. It's going to behave a little bit differently on GNOME or Plasma, depending on what your desktop environment is. And if you're on XFCE, there's probably some dependencies you're going to have to install before you'll even get mouse control. Right? Like all these edge cases. You kind of got to plan around

design around making this work on your system. Yeah. But man, is it slick when it works, dude. It's so good. It's so fast. It's totally like you could work all day in it. Here's what I think we should do for the studio. Here's my moment of truth. I think we use Moonlight on a couple of the systems and Sunshine.

But I think for the OBS rig rig where it's like it has to work every single time i think we build a pi kvm we set up pi kvm connected to the obs machine and for this rig until all of this stuff is a little bit further down the road i think we go with something like pi kvm and i think that would be a lot more reliable plus we have a pi 4 that i took out of lady jupes that we could use for this and then we would get full console access regardless of what wayland is doing or anything like that that does sound like more fun and then we could get the grub menu and we could actually roll back if something were to go wrong from a remote session does feel like we're maybe you know hunting on the yeah i think so well i feel like it works for the reaper computer and the airmaster

machine but for the obs machine well job one is something's wrong with our video acceleration because remember we tried the arc for a bit then we went back to the amd something isn't right there and i gotta sort that out i'm about to put the ARC in AirMaster though, so I'm going to have to figure it out there as well. I support, let's do both. Why not both, right? Why not? Let's get the PyKVM going because A, like why not? That's just a nice thing to have around the studio for sure.

And then, I don't know, maybe we can keep using the OBS machine as a test case to nail down our Wayland setup because we're going to need to solve that one way or another, right? How long are we going to stay on XFCE over here? Not much longer. Basically, as soon as we have to reinstall. And PyKVM, you know, I think it is a good backup, right?

Because you take a Raspberry Pi, ideally a Pi 4, and you hook it up, and then you have console access to that machine. And when we're remote, that could be really, really, really, really valuable. And, you know, in the meantime, we keep figuring out the rest of this stuff. And then maybe one day, we just obsolete the need for a PyKVM. But it's something I've wanted to try anyways. You were digging around, though. You did find some other threads that could

potentially work, especially for open SUSE folks and whatnot. Yeah, it seems, I don't know. There's little threads around. Maybe KRFB support is slowly getting better. I see there was just something about hooking up Wayland key events just merged within the last week or so, so hopefully out in a release sometime soon.

So we may see kind of a built-in remote desktop server into the Plasma desktop, kind of like GNOME is doing. And it looks like there's some other options. There's WayVNC, which is a VNC server if you're using a WL roots-based Wayland compositor. So, like a lot of things in Wayland, it's, at the moment, a little fractured. Yeah, and you could

pick your own adventure at this point. There's WayPipe, too, which is a proxy for Wayland clients, and it forwards Wayland messages, easy for me to say, and serializes those changes into a shared memory buffer and then sends it over a socket. So Waypipe is something else, you know, like Wes is saying, there's some stuff here.

Maybe one of these is the direction we'll go. But right now, the thing is, is PyKVM just takes me completely off the box. So even if we need to get to the BIOS, we could do that with PyKVM. And for the broadcast machine, that seems really useful. I sure hope we don't have to get to the BIOS. Yeah. But you're right.

If we're trying to get live and we need to monkey around the BIOS, something else has gone wrong. Bitwarden.com slash Linux. Head over there right now and try it for free as an individual or get started with a trial for a team. It's really powerful in both settings. Bitwarden, straight up, it's the easiest way for yourself or a business to store, share, and sync sensitive data. Bitwarden has all kinds of

customized options, which you know I love, but enterprises as well, including things that work with your enterprise policies to adapt to your business needs. And Bitwarden vaults are end-to-end encrypted with zero-knowledge encryption. Z-K-E, my friend. Zero-knowledge encryption means there's no metadata that leaks. The URLs aren't plain text, right? It's not like little encryption with caveats. It's good stuff. In fact, they've recently just upgraded their encryption as well. They're always kind of iterating. Now, I say that in a layman's term. You know how this stuff works, right? They're always making this better. They're always making it more secure.

And just in February, they've been putting work into that to make sure that they have the strongest possible security for you. And you combine that with a strong, long, complicated master password, and you are cooking with gasoline, as they say. Also, another update that landed in February, you can now log in with more devices. So you can quickly log into Bitward now without having to type in that long, complicated master password. If you want to log into like the browser extension or the mobile app or the desktop app, you can use an already logged in authenticated mobile app or desktop app to do that authentication now.

I think that's really nice. I mean, I have my master password memorized, but I think this would make it more approachable to rotate my master password. And I think that is a great feature. Go try out Bitwarden, bitwarden.com slash Linux. See why Wes and I use it to manage our passwords, our two-factor codes, and other sensitive data like recovery keys and passphrases for certain applications that require that. And the Bitwarden account switching makes it easy to move between a personal account and a work account or a personal account and an open source project. I really like that. That means I can keep work and play separated. Bitwarden just keeps getting better. Go try it out and support the show. Bitwarden.com slash Linux. Head there right now. Try it out and support the show bitwarden.com slash linux head there right now try it out for yourself or for a team at bitwarden.com slash linux we got some great feedback again this week thank you everyone who wrote in linux unplugged.com

slash contact if you want to send us a note we We did get a note from Mark Pearson, who's a technical lead for the Lenovo Linux PC program, and he was trying to help Chris with all of his Wi-Fi issues. Chris, do you have any more details on this? Yeah, Mark wrote into the show and was asking just some clarifying questions about the Wi-Fi woes that I talked about last week. You know, I never even thought about it as a ThinkPad issue. I don't think it is a ThinkPad issue, but you got to give credit to Mark and the team over there. They want to look into this

and figure out what was going on there. And also, I believe Bark was one of the folks responsible for some of the plumbing with some of the power management stuff. So I'm planning to chat with him more. Mark writes, Mark, I love you. Thank you for writing in.

But man, do you write some long emails. And a guy is has a super hard time getting through his inbox but you're special and i'm gonna try to get back to you he wants some more information about uh it's it's cute he asked what kernel i'm running like i'm not running the most recent kernel whatever the release kernel is mark that's what i'm running but um you know I understand most people aren't like that. Like the moment 6.2 comes out, I'm installing that,

right? Like the moment. So anyways, I will get back to you, Mark, and try to follow up on that because why not have some more eyes on it? And maybe you and I can have a conversation in the future, Mark, and we'll kind of,

uh, bring some broader understanding about how some of this power management stuff works because it sounds like it's, uh, it's more complicated than I realized it sucks is what it is i talked to mark on a somewhat regular basis because he uh we worked together in the fedora project on the thinkpad enablement stuff and like yeah he's fantastic to work with and i really do appreciate him coming in and and and putting himself you know in front of everyone for all this kind

of stuff, because it really does help make Linux on the desktop even better. And I know I don't know if Mark is going to hear this, but thank you so much for doing this. And we love having you in Fedora to help us make this better. Here, here. I was really blown away is a bit of an exaggeration, but I was really impressed that Mark was listening and reached out and he wants to replicate this and test these Wi-Fi things. And I never even thought of it as something that should concern the ThinkPad team. But the fact that they want to take this on and try to get ahead of it is a great sign. We got some more feedback in, some from Kevin saying, I caught the mention of the Linux Unplugged episode 500 mini meetup that's going to happen on March 4th, and I'm excited to head down to Mount Vernon.

I moved to Bellingham recently from Bozeman last October after spending the summer in our van full-time visiting family on the East Coast. Wait, wait, what? I know. Hold on. So this guy's a hashtag van lifer. Yep. And you moved from Bozeman to Bellingham? Don't you know most people are going the... Doing it backwards, perhaps.

Yeah. Boy, Kevin, you got to flag me at the overlaps with Bozeman in particular. During the build-up phase of my van was also when you began discussing AM Solar, and I ordered associated equipment, except panels and batteries, from them exclusively, so thanks for the tip, Chris. I'll be headed down solo to the meetup, and this will be my first ever Linux-related meetup or event. It feels weird to be in the position of being so familiar with you all, but not vice versa. Regardless, I'm looking forward to exploring the area that hasn't been at the top of my to-do list. Well, this area is fantastic.

If you're in Bellingham, though, you're already experiencing some great Pacific Northwest. Just wait. Wait till the spring, my friend. It's going to really blow your socks off. Kevin, we have so much to chat about. This is really exciting. Looking forward to seeing you. Yeah.

Kevin also offered to help with any in-person projects as they're really close now. So that's a little bit of value for value offering. I honestly think we have a core group of listeners because, you know, the thing about this show is we have listeners everywhere now, but because you know the thing about this show is we have listeners everywhere now but it's spread out around the entire world and so when you think about people in our area it's like a solid dozen in our really core area that are within a driving distance for the day maybe maybe maybe it's 50 that are within maybe it's more but from what i've perceived it's about 50 maybe it's maybe maybe it's 100 i don't know but it's somewhere it's it's a small number that are actually within a day's travel from the studio so kevin i would love to see you at the meetup chris in my area i

have a single listener who's within a day's drive and it was great meeting up and it's uphill both ways. Booster Gray. We did get some boosts into the show. And Dean L70 comes in with 60,000 sats. I hoard that which your kind covet. Thank you, sir. He says, hello, team. Dean here from Geelong, Australia. I've been listening to Jupiter Broadcasting since it went independent about three years ago. I'd love to know how we showed up on your radar at that moment Dean He says he goes on to say I just wanted to let you know how much I've enjoyed your content

Both on the unplugged program and on self-hosted and office hours Please keep up the good work and know you are all appreciated That's so sweet Well thank you Dean It's pretty great to hear a check-in from Australia. I bet I got the pronunciation wrong of where you're from. So if you want to correct me and also tell us how you heard about us three years ago when we went independent again.

Again, I always have to caveat, we were independent for a decade before that. But that's a long story. Perhaps something people would like clarification on could be a good episode 500 boost uh um how do you guys think we say this next one holland hum all right all right 50 000 sats i mean that's what matters to me right keep the change you filthy animal i just joined pod the podcasting 2.0 world. I switched from AntennaPod to Podverse, which I find to be about halfway towards perfect podcast listening solution. Yeah, that's true. I mean, it's a big job.

A podcast app is actually more work than I think I fully appreciated. He knows he does have high standards. The road ahead for podcasting 2.0 seems long and uncertain, but I'm doing my part. You know, I think it depends on what slice of podcasting 2.0 you look at, because I think things like transcripts

and chapters and cast information, those are slam ducks. And I think the reality is that hosting platforms are going to want to add those features regardless and instead of having to invent the thing from scratch they're going to just adopt whatever the open source standard is so there are certain things about podcasting 2.0 that to me feel like foregone conclusions like we see a lot of passion in our community around transcripts yeah

i think you could argue the boost stuff um is probably the one that's the biggest long shot. However, I'll save this for a future episode, but I was just reading a really hard-to-read blog post from an open-source developer who's creating this library that's used by like 300 million people a month. It's got billions of downloads, and they cannot afford to eat breakfast

and i think it's so silly that over here in podcasting we've solved splits i'm gonna get a split west is gonna get a split brent drew the network we have splits built in every time i mean it's just like we've solved this in podcasting. I wish we could figure out how to solve this in free software, but we'll get there. But I think first we have to pave the way in podcasting. I think it is the boost that have the longest shot, but I think it's also the most important thing that's going to have the biggest ramifications for free software and for content. So thank you very much for boosting in and doing your part we really appreciate it and then where's my cat comes in with some grandpa ducks 22,222 sats this old duck still got it uh they're right hey guys greetings from a dutch guy living in japan well that's an interesting arrangement i'm answering the request of boosting in our whereabouts i I'm using Linux as a daily driver at home

and my work as a neuroscientist. It's great for efficient workflow and data analysis. Thanks for all the steady stream of amazing content you've been motivating me to try new things. I am loving my Odroid server. That's great.

This strange booster named Drew of Doom sent in 5,000 sats. How spooky. Since Chris has so much trouble pronouncing the open source NVIDIA driver name, I propose we pronounce it like the generation ship in the expanse. Chris, would you give us the pronunciation? The Nauvoo. The Nauvoo. You know, I like Drew's idea here, right? Like, we just apply expanse lingo to everything. You know, like belterlota. You know, we just belter terms for everything.

I think that's pretty great. Now, to give a little context, Drew of Doom is actually Drew, our audio editor, and sent in another 5,000 sats saying, Lol, I just sent myself three cents with that boost. Coming in hot with the boost! I like that he's having fun with it. Circular boosting. Deckbot boosted in with 2,000 sats. Yes, I appreciate the Steam Deck coverage.

My colleagues game, but I don't. Their knowledge of Linux comes from the glorious reporting from LTT. So at times they ask me Linux questions. Having some awareness of the Steam Deck Beyond at Runth Arch, by the way, makes me engaged and relevant. Thank you. Also, for the Mbox question, what about running Dovecot and Roundcube to interface with the archive? I think they're referring to a question we had about Mbox and trying to move that to an archive that was easily accessible over HTTP, if I remember. Yeah, that might just work. Worth a shot anyway. And what do you think? MBKulik?

Kulik? Kulik comes in with 3,167 stats. Hello from New Hampshire, longtime listener from the Lunduk days. Here's all the stats I've gathered while using Fountain FM. I recently switched to Podverse. I find it suits me better. Now I just got to figure out this Albie thing. Yes.

Yes, indeed. Although Albie is fantastic. Albie is worth your time. Gene Bean comes in. It wouldn't be an episode of the Unplugged program without hearing from Gene Bean. 4,096 sets. To build on Brent's passing reference to a Zoom call, a key aspect of a distro being suited for new users, especially less technical ones, is that it can use the installers provided by a vendor

from the vendor's website. For example, on Windows and Mac OS, I go to zoom.us and I follow the download link. Newer Linux users would tend to do the same. This is the main reason I suggest Ubuntu LTS-based distros for new people. It's a very real factor. Yeah, I've helped a bunch of new listeners move to a variety of Linux systems. And I've noticed that actually some Ubuntu suggestions, like Signal is a good example. You go to their website and they suggest you install Signal via a PPA. And I wouldn't call that especially user-friendly. More so what I'm seeing these days is something like

Flatpaks is just so much easier because it tends to also solve many of these little gotcha issues, you know? And so that's what I've been suggesting. And it's been pretty good for new years, new users. This feels like the one thing where app image does kind of have some logic is the idea that you just download something and run it. Because it's funny that Gene Bean mentioned Zoom. How many times, I mean, I don't know about you guys, but I have seen new users that literally do go to Zoom.us and they download like an EXE or a DEB and it's not...

What do I do with this? Yeah. You can double click it, your system doesn't know, it doesn't do anything. And it's not just Zoom. It's kind of anything. They don't even think about like software centers or package managers. That is a really tricky thing. Well, though, for what it's worth, right? Like, it depends on where the user is coming from. centers or package managers. That is a really tricky thing.

Although for what it's worth, right? Like it depends on where the user is coming from. If they're coming from the Mac world, their first instinct is to go to the software center on the system. If they're coming from the Windows world, then their first instinct is to go to the vendor website and go find it out. Very true. Even in the Windows world, people are increasingly using the software center that's built into Windows, the Microsoft Store. So, like, over the, like, the most recent Windows users I've brought over to Linux, they're like, well, okay, so where's the software store? And it's like, okay, GNOME Software or Plasma Discover and call it a day. Maybe that's better as Windows users get more familiar with an app store. Maybe that'll.

Well, they're already familiar with it because that's how it works on their phone. So, like, just telling them, oh, yeah a we have an app store just go here it's like the phone that short circuits everything okay i like that idea it's like your phone it's got an app store yeah you know how that works if you want software go here yeah yeah and then uh call me if it doesn't happen gene bean comes in again with 30 180 sats boost hello from villarica georgia uh this boost amount is my zip code i kind of want to look that up real quick did you already look it up no but now we can spy on gene bean gene bean you are the best doxing yourself like that just to tell us where you're at you sir are a sweetheart can you just put in zip codes to the old Google Maps or what? We're about to find out, I think.

Yeah, you can, dude. You totally can. Look at that. So he's just about, you know, east of Atlanta. God, Gene, that's great to know, actually. I legitimately do like to know that. I've always wanted to be down there. I've always wanted to go down there. Also, he sends

in a row of ducks to just do a plus one to the livestream Gentoo party. I've been thinking about that. i've been thinking about that i've been thinking about that we did promise such things i've also got a pc that is impossible to install linux on from a gui and i think i think i want to challenge you west to try to get nixos booting on it and because i know you can do it i think all right i'm in because i think you and I have done it once before years ago. But like Fedora, SUSE, Ubuntu, and NixOS with the graphical installer

all fail to install on this system. Wow. It's a copy of the old OBS system, but with like way more random disks in it that completely confuse

all the installers. I think it'd be a lot of fun to see if we could get it going. You could also just do a kickstart if you wanted to bypass the GUI for at least Fedora. I mean, I know that's how he's going to, he's going to, he's going to do it by doing a manual install of the command line. That's what's going to be so much fun about it. I mean, that's the fun way to do it. The reliable, easy way is to just use a kickstart or an auto yast.

Nobody wants that. Oh my gosh, you, you old man hee-haw me on this that's how people do this not old man hee-haw just that oh we don't want the easy option no we don't want that no but i do want to see him like do it the hard way like go into a rescue environment like actually format the disc do the dnf install route to set it all up and then just do the manual command way of doing it, I want to see someone do that. That would be great. That would be great. I mean, can we combine the two challenges

and just do Gentoo on this thing? Oh, no. We can chew our bubble gum and walk at the same time. Don't you worry about that. That is definitely... We're going to have time. We're going to have time. It'll take forever, but it'll be fun.

Yeah. The immunologist, the immunologist, comes in with a row of ducks. And you know what? Let's go for it. That's right. And they say, I tribute my humble sats from, I assume, listening to the show. Keep up the great work. And then speaking of a row of ducks, the originator of the row of ducks, the golden dragon comes in with a row of ducks.

Plus one for XFce lightweight it's user-friendly and it's vm swappable quite the tasty linux experience yeah there is some there is some nice things about xfce however when i was trying to do all the sunshine stuff and do everything in wayland and whatnot xfce was like the one odd duck out the entire time. Because, like, I would go to remote in and, oh, I got to install this to make the mouse work. And then I would go try to use the keyboard. I got to install this to make the keyboard work. Like, XFCE just didn't have that stuff ready to go. Well, we've got some more Where Ya At Booster boosts.

First up, from RGHVDberg, a row of ducks. Here's a row of ducks to help you guys get on the way to Cape Town. Cheers from the Netherlands. We gotta do this Cape Town thing. This is sounding too good. Another row of ducks, and this is a special request here for the

sound clip. I'm a duck. D-U-K, duck. Loaded with talent. That's from patrick and patrick writes listening to 497 right now boosting a row of ducks from lebanon tennessee hey oh thanks for the great show and you get a duck sound clip sir and 2 000 sets from an anonymous booster who's listening from toronto canada hey Coming in hot with the boost! One of your countrymen coming in with a little bit of support there. Yeah.

Thank you, everybody who boosted into the show this week. If you'd like to boost in, we would appreciate it. Of course, you can boost from a new podcast app at newpodcastapps.com or don't switch apps, just go get Albie, get albie.com and then go over to the podcast index and send your boost into the show. And of course, don't forget, there is the LUP500 boost amount, 9,529 sets for 9.5 years of the Unplugged program. You can send that in, and we'll collect all of those for episode 500. Thank you, everybody who does boost in.

Newpodcastapps.com. You want to get yours in Alright, I'm searching my hard drive for some picks this week And Wes Payne comes in with a couple of specials Oh, it's just one, it's just two links for it Oh, two links for one pick? Aren't you special? I just wanted to use one of the links

To point out that this pick is packaged in Nick's package. That's all. Okay, alright, good. But our pick this week is written in Rust. It's known as the Static File Server or SFZ.

And nothing fancy. It's just a simple command line tool if you just need to serve some static files. You know, maybe you're trying to share some things, you need to get something off a server real quick. You just want to say, like, make this whole directory that I'm in available over HTTP with a friendly little GUI. So it just basically, instantly takes a directory

and makes a web page out of it. Yeah, exactly. But it supports directory listing, partial responses, cross-origin resource sharing, which means it works great with Chromecast, automatic compression built in. All right. It respects gitignore files

if you don't want to share everything in the directory. So you could potentially use this as a way to expose a directory that has some video files over HTTP and then Chromecast it. I was just doing that this weekend. Brent and I were watching the original series episode Balance of Terror and

needed to both watch it. We found it, just someone had stashed it on one of those video sites online. It was quick and easy. It was kind of off the cuff. We had a little extra time. We were both willing to keep watching another episode. I thought this would be fun. But it was kind of unreliable. You'd watch it there and if you paused for too long, then the playback kind of stopped. I thought, this would be fun. But it was kind of unreliable. You'd watch it there, and if you paused for too

long, then the playback kind of stopped. I thought, this is too fuzzy. Let me just YTDLP it really quick, and then we can watch it that way. So this was able to easily share that file with Brent so he could stream it from a better source. And then, at the same time, it meant that I could have my Chromecast just play from a reliable source and not have to worry about some sketchy

CDN stream. I always find it entertaining how West solves these problems. You know, these are always, the Wes solutions are always the best solutions. Yeah, I agree completely. And I will say watching, you know, esoteric videos on last minute with Wes is very easy from my end and very entertaining in the process. I love it. I bet.

Yeah. You just get these weird links from Wes and you're like, all right, I'll click this, right? You know, I was reflecting on how there are like three people in the world where I'd click on any link they send me. I think it's Wes, you, Chris, and Alex, although he likes to throw me with a few Rickrolls from time to time. But you're lucky, Wes, you're in that category because most other people I just wouldn't click. Yeah. I promise to respect that trust. You're right.

That's the thing. Don't break it. Don't break it. That's a special, special trust. You're right. That's the thing. Don't break it. Don't break it. That's a special, special trust. Casey, you're right. Python, Ruby's also got one.

There's handy sort of standard lib HTTP servers as well that you can run if you don't want to run some Rust binary on that system or if it's not available for your

architecture. Yeah. Casey, thanks for joining us for the show, man. Yeah, glad to be here. Sorry I can't see you over there.

It's okay. There's like three monitors between me and him. I will verify he is over there. He is over there, though? Mm-hmm, confirmed. That's nice.

I like the idea of having an in-studio audience, at least. I should just be holding a mirror. Yeah. All right, well, that does wrap us up for this week's episode. Just one more until 500. Why not join us live as they come and go? Pop in over at jupiter.tube on a Sunday. We do it at noon Pacific, 3 p.m. Eastern.

See you next week. Same bad time, same bad station. And of course, if you're feeling like you need more show, linuxactionnews.com. This week, we took a look back at 10 years of Steam and went back to the old Linux Action Show coverage of the Steam announcement. That was quite a trip down memory lane, Plus covered a whole bunch of new stuff. LinuxActionNews.com. If you're not catching it every week, you're missing what's going on

in the world of Linux and open source. As for us, links to what we talked about today will be at LinuxUnplugged.com slash 498, as well as a bunch of other good stuff over. It's a website where we got links. It's so useful. You want to try that pic? There it is. It's right there, linked.

And also, just for Palmer Duckworth, we'll see you right back here next Sunday! Thank you. wish you hadn't told me i was no longer allowed to bring the hammer into the studio after i had one microphone wes wes i don't care if the if the if the hammer gets you in the zone i don't care it's too expensive to in the zone. I don't care. It's too expensive to replace these microphones every week. We can't smash them at the end of the show. Yeah, I have a little note here from the wife.

Do you see this? This little sticky note I have here? Palmer Duckworth ordered some swag, like the coat of robe and I think a journal from jupitergarage.com. And he had a note in his checkout he's like uh i love the show except for i really miss the cnx sunday thing could you please bring back cnx sunday which was funny right but even funnier that my wife then wrote it down on a sticky note and then stuck it on the mixer in the studio to make sure that i would say cnx sunday you know what i'm gonna keep it there for as long as we can just for you duck, Duckworth. You know, my brain

still thought you were going to say Tuesday. I know. He's like, he did right in there. He's like, well, Tuesday was great, but I'd take a Sunday.