LINUX Unplugged - 644: The People's Filesystem

Episode Date: December 8, 2025

Kent Overstreet joins us for a full update on bcachefs. What's new, what's next, and the surprising upside of getting kicked out of the kernel.Sponsored By:Managed Nebula: Meet Managed Nebula from Defined Networking. A decentralized VPN built on the open-source Nebula platform that we love. 1Password Extended Access Management: 1Password Extended Access Management is a device trust solution for companies with Okta, and they ensure that if a device isn't trusted and secure, it can't log into your cloud apps. CrowdHealth: Discover a Better Way to Pay for Healthcare with Crowdfunded Memberships. Join CrowdHealth to get started today for $99 for your first three months using UNPLUGGED.Unraid: A powerful, easy operating system for servers and storage. Maximize your hardware with unmatched flexibility. Support LINUX UnpluggedLinks:💥 Gets Sats Quick and Easy with Strike📻 LINUX Unplugged on Fountain.FMLUP's Great Holiday Homelab FormLUP's Great Holiday Homelab Old Fart Form (markdown)bcachefs 1.33.0Bcachefs Ready With Its Reconcile Feature As Biggest Change In Two Years — "Biggest new feature in the past ~2 years, I believe. The user facing stuff may be short and sweet - but so much going on under the hood to make all this smooth and polished."Kent Overstreet - Patreonbcachefs — A copy-on-write (COW) file system for Linux first announced by Kent Overstreet in 2015. It is intended to compete with the modern features of ZFS or Btrfs.LINUX Unplugged 545: 3,062 Days Later — Kent Overstreet, the creator of bcachefs, helps us understand where his new filesystem fits, what it's like to upstream a new filesystem, and how they've solved the RAID write hole.Git or not a Git man page?proxmox-nixos: The Proxmox Hypervisor, on NixOSPick: Dotfiles Installer — Install dotfiles easy, fast and secure. Or provide an installation application for your dotfiles.ML4W Dotfiles Installer Wikidotfiles-installer: GitHubPick: SSH-Studio — Easy, GUI SSH config editor and validator built with Python, GTK 4 and libadwaita.LubeLogger, Vehicle Maintenance and Fuel Mileage Tracker, slips into December with some banging new features

Hello, friends, and welcome back to your weekly Linux talk show. My name is Chris. My name is Wes. And my name is Brent. Well, hello, gentlemen, coming up on the show today. It's a big one. Kent Overstreet, the creator of BcashFS, joins us to give us a full update on what's going on. What's new?

What's next? and a surprising upside of getting kicked out of the kernel. Then we're going to round it all out with some great picks, some boosts, actually like a really loaded pick section, and a lot more. So before we get to all of that, let's give the old time-appropriate greetings to our virtual lug. Hello, Mumbleroom. Hey, Chris, how we're saying.

Hello, Bert. Hello. Good afternoon. Hello, just a couple of you in the on-air, a group of you up there in the quiet listening as well. Mumb Room always cooking. We're always happy to have everybody in there. And a big good morning to our friends over at Defined Networking.

Defined.net slash unplug. Go check out Managed Nebula. It's built on top of the open source Nebula project. They handle the tricky bits that you might not want to think about. 100 devices totally for free. No credit card required. Defined.net slash unplugged. This is a MASH VPN optimized for speed, simplicity, and really great security.

But ultimately, it kind of goes a little bit beyond. When you're picking your network infrastructure, If you've been in tech for a minute, you know that there's also the company behind the product and what they're all about and what their goals are. And I think the way they're configured, the way they're designed, all of that stuff ultimately ends up getting represented long term in the product. And that's where I think to find has been really, really dougious in how they have built and designed themselves to really align with how I would like a company that's building this type of tool. And I think you would also probably align with that if you're listening to this show. So it goes just beyond the software and the fact this is a truly free project that's totally self-hostable, if you'd like. It's optimized for speed, simplicity.

It uses less battery on your devices, less network resources. You can host it completely or you can use their managed solution. But beyond just that, it's about the company architecture. It's about the funding model. It's about all these things that go into a long-term sustainable product that you can build a network on top of. And they have been solid since 2017. They had to be. They had to go in production day one for Slack in their global infrastructure.

Now, now it's incredible what they power. Rivian cars on the road, all the metrics they send back to Rivian HQ, that's secured by Nebula. Think about all of those roads, all those machines on the road, communicating over Nebula. They have to keep that secure. That would be such a scandal if anything ever happened for Rivian. It's really impressive. It has an incredible level of resilience, speed, and scalability that you can tune to your needs from a home lab to an enterprise. So go get started, 100 hosts, absolutely free, support the show, no credit card required.

go to defined.net slash unplugged. Well, you know, it is actually very scary close. So we wanted to give everybody a heads up that your time to submit to the Holiday Home Lab is running out. We want your holiday home lab submission. Home Lab for Linux Unplugged holiday special coming up. We're looking for your creative or efficient or chaotic overbuilt, just beginning setups. We want everything from the community and we'll feature some of the winners on our show. We'll have probably a batch that make it on

the show. We'll have a batch that we'll maybe do in special editions and all kinds of stuff. We have a form over at LinuxUnplugged.com slash holiday. Your time is almost out because we have picked our live record date, which we would invite you to join us. It'll be on Friday, December 2025 when we record the holiday homelab special. So if you'd like to join us for the live event, especially if you've submitted your home lab, Friday the 19th, we'll have the exact details and time

at jupiterbroadcasting.com slash calendars. We get closer, but more importantly, because I sometimes forget that bit, we always put it in the podcasting 2.0 feed. So if you have a podcasting, 2.0 app, you'll see it in there when we're going to be live. It'll be, I think, our last live episode, maybe not.

But one of our last live episodes of the year. I don't know if we have after Christmas quite dialed in yet, but it's getting pretty close. And so you have, well, you've got to get in before the 19th because we need days. So I guess we should have probably told you a deadline, not just when we're going to do it live. So I'm going to say the deadline is the 17th. That gives us at least two days to go through all the submissions, right? Yeah, that seems reasonable. So December 17th, we're going to cut off after that.

No guarantees. Yeah. So there's your cutoff. And then the live show will be on the 19th. And we'll hopefully have more details on exactly when. Probably, we could probably just safely assume our regular time. Seems like it. Sure. Why not? Yeah, there you go. Works for me.

It's going to be fun. It's going to be a lot of fun. And we've had some great submissions. You can't have a holiday without a home lab. No, you've got to play your Christmas mixes and stuff. Well, last week, we played some clips to just get you up to date on the B-Cash-FS current status and the kernel. and honestly, our real-time reactions because a lot of that developed while we imposed both times there were major developments

while we were traveling. Yeah. So we were like processing all of that. Attempted to travel or attend a conference or not think about file systems. So if you missed that last week, you might want to go grab that context. So that way you have like sort of the full context

for today's conversation when Kent joins us. And as the year rounds out, we thought this is a topic we should talk about because there's been a lot of changes. Bcash FS was kicked out of the limit. Linux kernel. We talked about that last week. But I think it's best to get the information straight from the horse's mouth. And in this case, that's Kent Overstreet. So we're going to get into this in the way that only Linux Unplug does with the details and the technical stuff. And

the best source, of course, is Kent. He's the creator of BcashFS. And he's a rare, rare breed of developer. And I think as you listen to our chat with him, this is going to become evident to you. I wanted to mention something that maybe isn't obvious, though. Kent is creating what I consider to be maybe the most modern file system, period right now. And he's making it GPL and available for Linux. And he's doing it from what he calls a glamping tent attached to the back of a truck in South America via a Starlink connection, living off of the support of his patrons. And he is developing every single day. He committed, this is a Sunday morning and he committed three hours ago.

He sure did. And then he had released a major version, which we will get into, with major new features, a few hours before our chat that day. Because we talked to Kent on Thursday. Yeah, yeah. And he had just released a massive, massive update. And he's just constantly working. He's constantly communicating. And he's doing this while creating this project while being nomadic and keeping his cost of living as low as possible.

So I just think he's an incredibly interesting developer from that standpoint. We're very lucky to have him. And just the insights here are great. So if you're not familiar with BcashFS, buckle up. And if you are, there's a lot to love in here, too. Ken Oversry, welcome back to the show. It's great to have you, sir. Great to be here.

And also, congrats. Bcash FS 1.33 came out this morning, I think. Metadata version, reconcile. Huge, right? Tined it perfectly. I think we should start right there. And then we'll kind of get into other stuff. But let's talk about the new reconcile system, because reading through the show notes,

It looks like it was, it looks like one of those things on the surface seems like a small amount of work. But then when you dig into the details, it was like almost a total refactoring of type. Not total, but it touched all kinds of stuff. Yeah, like had to get metadata plugged in, all kinds of just little bits here and there. Because whereas before, this grew out of rebalance, which just did background compression and background target. Now we're aiming for strict correctist so that we can actually check the validity of, all store data against the Ayurpath options. So if you're doing correctness checking,

that tends to be a different level. So a big project. And it's actually done now. So it's congratulations. Just reading through that, I was just like, this is a big piece of work. And it's pretty cool that it landed right before you able to join us.

Now, as somebody who's using B-Cash-F-S now, will I need to prepare or make any changes before I update? The only thing to be aware of is there will be an upgrade process when you mount on the new version. One of the things that should be coming soon but didn't land for this release is we should be able to do these upgrades in the background. And we're still doing it at mount time. And we still haven't got this communication with System D going to let System D know that we're actually doing stuff at Mount, and it just thinks that we've timed out. So that's a little bit of an annoyance.

but just let it go until the kernel log says it's finished or the D message, whatever you're watching, and I'll reboot it again if you need to because of the system D thing, and it'll be fine. We had a lot of that back in, like, the 614 upgrade for back pointers, or I had to redo something, some minor detail of the back pointers. Actually had to rewrite half of everyone's metadata.

I think I remember upgrading from that one, yeah. Yeah, yeah. No one noticed anything besides. It's going, hey, why were my mouths taking so long? People would kick it a couple times and they were just finally finished. So that was fun. This is not as big an upgrade. Okay, okay.

That puts it in perspective. All right, so let's walk it back a little bit. It's been just about a year or more since we talked. I can't quite recall, to be honest. But it feels like it's been five years. I mean, time flies. It's just shy, 693 days. Oh, okay.

Oh, so, you know. Yeah, it's been a bit. So things have changed a lot. Let's talk a little bit about user base, who's using it these days, maybe how just off the top of your head. I know this is kind of a wild one, but how you say things have changed since 2024, just kind of the best recap that you can recall. I think the biggest, biggest change is that things have really stabilized. Even like six months ago, even probably three months ago, like back in July, I was still in the midst of, you know, So the bugs are coming in about as fast as I could handle them and things that...

I could see the trend back in like 6.14, but there was a noticeable inflection point, like 6.16, 6.17, and now it's like, things are starting to feel calm. Past couple months where a giant slog with getting reconcile out the door, but I'm feeling pretty good about where we're at. And we're getting really close to taking the experimental label off. I pegged that for Christmas. What a lovely present, huh? Yeah, yeah. There's like almost nothing left. Yeah, I was going to ask, you mentioned sort of, you know, assessing stability and here, getting ready to take off that experimental label, what actually are you looking for?

Because I know you're helping people one-on-one. You've also got, like, actual, you know, bugs getting filed upstream. Probably you have, like, automated tests running. Lots and lots of automated testing, yeah. There's a whole cluster of machines off in Finland, big 80-core R machines with a quarter-chreibite RM each and I run those machines hard. So I'm constantly watching the test dashboard, which is never as clean as I would like it, but I'm always watching like triaging and, okay, that one can wait.

No, that's actually something serious. I need to jump on now. And most of it is just how are, user feedback. Am I, are the bug reports coming in? How severe are those bug reports? How quickly are we dealing with them? Are we finding stuff that it's like, oh, that report? Parapath needs a ton of work, or is it like, oh, that's a, that's like a trivial 10 or 20 line fix.

So it's a lot of things. I think in back, it was really still pretty experimental at the time. It feels like now we're at a point where if I deploy it, I'm probably not going to see any big format changes or anything that's really going to be disruptive to my workflow. And that's huge, right? Yeah. One of the things with format changes is I've actually gotten less conservative with doing format changes since we're no longer upstream, we're DKMS.

Interesting, okay. Because, yeah, we no longer have users running a whole bunch of different versions and maybe rebooting into older kernels. It's very easy now for users to just stay on the latest version. And as long as I'm not pushing out regressions, and we've been doing great on regressions, then there's no reason for users to have to roll back to older versions of BcashFest, which we can do.

But it was an expensive downgrade and then re-upgrade. in the kernel. And now it's like, if we have to change something, and there's a legitimate reason, like Reconcile, we've got another one coming for per device fragmentation LRUs. That'll fix some issues with copy GC and tiering. Then it's just not as big a deal,

especially once in the next couple weeks we can do those upgrades in the background. Then it'll be completely smooth. Yeah, then we really probably won't notice, huh? Yeah. So, Kent, when all of this was going down with, you know, the discussion, will they or won't they extract B-Cash-FS? Did this benefit even occur to you during all of that?

No, that one actually did not. The whole D-KMS transition went much, much smoother than I expected. I had a lot of concerns and fears of the time how things are going to turn out, and it turned out so better than I could have expected. How so? I mean, besides obviously what we just talked about, but in other ways as well? Yeah, I didn't seem to lose users like I expected. The people who actually care, stuck with it,

and we're able to get the DKMS packages done, all the automation, everything we needed, that came together really fast because a lot of people jumped in and helped out. It's shocker, in hindsight, not so obvious going in, but it's a lot easier for people to help out with a non-core kernel file system stuff. And there's a lot of people who are interested in want to help out, but they're understandably intimidated by diving into 100,000 line secrets, code-based, but like packaging stuff, yeah, that was a lot easier.

I hadn't even thought about that, Kent, so I actually appreciate you bringing that up. I guess what I have been stuck on is, I guess it seems like it's a good thing for B-Cash in the short term, but it's not a great thing for Linux in the long-term, right? I have really expressed on the show, I really think Linux needs a sophisticated, modern file system that's GPL and built in. I think it's imperative. Yeah, I don't think the Linux file system world is. is in great shape, and I feel for those guys.

But, you know, a lot of the stuff, it's, there's fairly deep-seated problems. And I, I wasn't able to make any headway. And I can't do anything if, if no one wants to, everyone is stuck in their ways and doesn't want to change. Is it momentum of just history that, you know, and people that have their fiefdoms? Is it just that? It's built fighting through that. That's some of it.

Yeah. There's a lot of reasons. You know, the kernel is a big old project, and, you know, basic processes, how we do things, tend to, you know, in any big institution, tend to ossify and become harder to change. And the Linux kernel came of age, you know, really back in the 90s, early 2000s, with everything being done on the mailing list, patch-based workflow, didn't use source control for years and years famously. And the one place where it really lags today is like automated testing, QA. So one of the pain points for me of being in the kernel is I do rely very heavily on automated testing. And if you're in the kernel, you're expected to stay on mainline, rebase onto RC1s every time RC1 comes out. And that's a good thing.

You want people to be dogfooding and catching issues like that. But if the file system people are the only people doing really comprehensive stress and torture testing, and there's other subsystems in the kernel that aren't testing the code at all, that's really painful, especially if you're like me where you've got, I'm just one guy with 100,000 lines of code. And if, you know, if things aren't working right, it'd be like every other release. my test dashboard would blow up and it'd be a couple weeks where it'd be tracking people down and I can't get work done if my test dashboard is a whole bunch of red and I can't tell

if my own patches are buggy or not. And, you know, think about that for the kernel as a whole. It boggles my mind trying to get work done in that environment without automated testing and not being able to get good feedback on what code is working or not. Yeah, they're spending way too much time chasing regressions and regressions in other people's code, and it's not fun. It does make me reflect just on, yeah, all the pros and cons to the coupling to the kernel. You get, you know, if you are coupled, you get the advantage that is there by default. But if you're uncoupled, you know, there's a lot of pros.

Although also, of course, some cons, I was just curious how you thought it was going being out of the kernel. Besides DKMS, like I know there were some symbols that were no longer exported that had to get vendored, like that. kind of thing. Do you anticipate being able to keep the current model working without too much of us? I'm curious how that's going to go. The core kernel interfaces that file systems depend on and B-Cash-of-S, they've gotten a lot less churny than they used to be. So I don't expect things to be too painful, but the kernel is not very friendly to out-of-tree projects and code, so it's a question mark. It's fair. That's fair. It doesn't make me feel much better, but it's a fair answer.

I'm sure you probably feel very much the same. Yeah. Yeah. All right. Well, I really appreciate you addressing that. That actually gives me, I think, a better understanding overall of the situation. I mean, as us as users, I've been really impressed how some of my favorite distributions have just made it. They've really stepped up to make it pretty simple for me to just keep using B-Cash-FS. Yeah, the Nix OS and Arch people have been awesome to work with.

I haven't had to do anything for those distributors. besides keeping contact and make sure we're on the same page. And those guys hang out in the B-Cash-FS IRC channel, and we give each other a shot whenever something comes up. And generally, things are just chill and awesome. Okay, so you made a hint about a lab. And I've heard you kind of made reference to some pretty significant large B-Cash-FS deployments. Just, I don't know, off the top of your head,

what are some of the larger that you're aware of the people have been experimenting with or using in production? Oh, actually, I'm not sure what references I've made, because I honestly do not know. I hear from people when stuff breaks, and if things are working great, then, like, there have been quite a few times when, you know, the rare user impacting regression comes through, and I find out that there's actually a lot more users than I thought there were. like the Debbie and Brew Haha, where the maintainer shipped a package and broke the build and then sat on updates, and there was a bug and mount where users couldn't pass the degraded option. And I start getting all these bug reports from users I've never heard of that I just had

a drive die and I can't mount my file system. So, yeah, I have no kind of telemetry or statistics, and I would really love to get telemetry and just see all the data I possibly can when something breaks. But I don't have that yet. Okay. Okay. Have you run into any interesting systems, though, or heard from folks reporting successful or otherwise

deployments? Oh, there's definitely some big phyllisisms out there. There's been, for quite a while now, file systems north of 100 terabytes. Like back, I mentioned the 6.14 backpointers upgrade. That was specifically, but because people are running into

scalability issues and had 100 terabytes fs checks were taking more than a day and now we can scale up we tested scalability up to like 10 or 12 petabytes and

no one's hitting scalability issues anymore okay well I'm nowhere near those numbers but I've kind of this year started building my home lab rebuilding my home lab it's really going to be a project for next year

and I think I want to do B-Cash FS and I just sort of wanted to pick your brain where you think B-Cash-F-S is for home labors, if that's maybe a good use case, some things I should maybe try or know about, anything like that, you know? Like, if I got you on the line, I got to be selfish and I got to ask. Yeah, yeah. Oh, I love working with the home labors. Those are the guys that hang out on the RSC channel and that we have the most, like,

productive working relationships because those are the guys that, like, like, like to tinker and investigate and discover how things break. And that's where we find a lot of the weird. corner cases and that's that's where the polish comes from is working with those guys okay so if uh if i'm going down the route go for it yeah i mean just any like any tips anything i need to worry about is it just deploy any mount options i want like any any pro tips you got for me one angle might be like uh raid five six is that available yet like you know how should he structure is array maybe if you got any tips yeah uh aside from erasure carding

we'll get to that okay everything should be pretty fire and forget if you've seen like the documentation on like how you set iopath options, you know, all that stuff can be set per directory, application level, target options. You can set up whatever kind of tiering or caching setup or pin different directories to different devices. There really shouldn't be much to configure. Actually, the one thing that I should mention is there is an outstanding bug on the bug tracker that looks like it might be a bug in Z standard. The title of the bug is, is data corruption after writing hundreds of gigabytes, and it's only used with C-standard.

So, yeah. Maybe choose a different compression. Not the first C-standard bug that we've tripped across. Right, right. And one of the hardening projects that's like next up on the to-do list will enable us to, like, pinpoint and flake up those guys like a test case or something. And just harden us against compression library bugs or compression algorithm bugs and a couple of other things in the data update path.

So, yeah, aside from that, I mean, you should be good to go. Okay. You mentioned erasure coding, though. You mentioned something about erasure. Yeah, that's coming, hopefully, like, first half of next year. There's not much left. People have been running erasure coding and not much to say about it besides the fact that it's fast and seems to work well. The only thing that's missing there, which the reason you shouldn't use it yet, unless you really know the gotcha, is we're missing, like, re-silvers.

or like the Eracial coding version of re-replicate. But drive dies, and you've got degraded stripes, you can still read your data, but we don't have any kind of path to resilber those stripes. Does your recent reconcile work impact the implementation of that? It does, actually. That kind of gave us the hooks and the engine that's going to drive, all that resilvering stuff.

And I reserve space and some metadata data on this 14. format stuff for exactly where erasure coding is going to plug into that when I got the rest of the design sketched out. So it's looking like it's not going to be that much work. But I got I got two more hardening projects to do before I can think about that stuff. So when you say hardening projects, I think of security hardening, but do you mean a different type of hardening? Yes. I mean the file system, like this is file system land. The file system jobs is to never lose your data, I always tell people, I don't care what happens to your file system. I don't care whose fault it is, if it's like user error or hardware error. As long as there's data on your

file system, it's the file system's job to get back anything that can be gotten back and repair it and get it working again fully rewrite with no user intervention. That's the goal here. And we're, I mean, judging from the log reports that I'm saying, we're basically there. I just got a bug report, like last week, two-device-replicated file system. Tons of B-tree node errors only on one device. Like, you glance at the log at first, and you think that must be a B-Cash-F-S bug, but then you look further, and all the errors are like got wrong B-Tree node, but on one device. The other device is good.

And the amount is like, there's no way a B-Cash-F-S bug could, like swap out all your B-tree nodes with the wrong B-tree node only on one device. That's just not possible. So my suspicion is that device mapper, like a configuration, got screwed up or something, and shifted all the offset of every data on that device. So that's the only thing I can imagine that could do that. Oh, man. Okay.

But if those kinds of situations, you want things to just work from. You want to be able to get your data back from that situation. Yes. Yeah, that makes sense. I am very insistent and very hardline, even more hardline than the ZFS folks on this. Like if you compare B-Cash-Feds and ZFS, ZFS was designed for enterprise deployments where you can depend on replication, you can depend on hardware that doesn't screw up flush and fua. And if you're running it in those kind of situations, it should always work.

But if you're running at single-device mode on garbage hardware, there are failure, that ZFS explicitly does not protect against, and you'll be sending your drive to a driver service. B-Cash-FS handles those. Like, I don't care if you metaphorically, let your file system on fire. We're going to get your data back. We love that because we love garbage hardware.

I was going to say, this is where it sounds like it fits our home webs anyway. We're all about garbage hardware. Make it last as long as possible, right? So it sounds like, I mean, I shouldn't even be asking this since you just released 1.33 this morning. but it sounds like the next bit of focus is some of this hardening work.

That's kind of where you're going next. Yeah. The other one is kind of cool, too, because it's, again, a garbage hardware thing. We want to be able to run on devices where maybe Flushing Fua is just broken and we can't do anything about it, like cheap SD cards or USB controllers. USB controllers often have buggy flush Fua. I've seen the bug reports. This idea came out of meeting with Valve a couple of weeks.

ago, what I'm going to be doing is we've got scrub now, and then in recovery, we've got a journal, so we know what the most recently written data was, why not just run a targeted scrub on only the last, like, second of data we wrote? And then we can detect if we can detect specifically and prove that it was broken flushed through a in that case. And even better, we've also got whole file system rollback from the journal rewind thing. I remember that whole thing. So if we detect that, we could just roll your file system back to the last consistent point.

And users don't have to care. Then we can run on garbage hardware, and it does not matter. That sounds great. I love that. Okay. So am I to get the impression, perhaps, that Valve is interested in a more robust file system for future use? They might be. Yeah, that's interesting.

I mean, if they're paying attention, they probably should be. So that's good. That's good to know. That was going to make me... More people should be paying attention to file systems, in my opinion. Yeah, I agree with you. I'm glad you are paying so much attention. And I'm glad it sounds like maybe Valve is consulting with somebody that knows their stuff.

Makes me curious. How's the support been like, both in terms of... You mentioned some of the volunteers for DKMS, but also just, you know, monetary support as well. You mentioned during some of the turbulent times in the kernel that some of that back and forth was generating bad press that was maybe making it harder to find consistent support. Has anything changed there? Well, things have gotten stable since Valve has jumped in and sporting. Yeah, it still really should be funded better. If you have any listeners with big pockets, this is a project that really should have

like three or five or six engineers working on it. And so to me, working ridiculous hours, seven days a week. That would be awesome. But it's stable and good, and I'm not sweating it like I was anymore. Oh, good. Glad to hear that. That's good. And that's mostly, is that due to the single, to Valve helping there? Is that stabilized things a bit? Yes. Yes.

That's good to hear. And before Valve, a European funding agency and LNET, they were, they jumped in and helped out. And that was a big help. Yeah, really would be nice if a few more folks would step up. A few companies are going to probably no doubt one day make a lot of money off of B Cash FS should step in now, right? No doubt. Yeah. No doubt.

Hey, does that cross your mind? You are, you are, you know, you're probably, you know, I'm sure you're not living, you know, luxurious. You're just, you know, you're getting by on this. And someday some companies are going to probably sell network-attached devices that sell for $30,000. Yeah, but I'm also thinking long-term here. One of the things about the whole criminal community is they've gotten too far down the route of just chasing where the money is going. and things in file system land got very project manager focused without enough, you know, engineering focus on long-term plans. Like, how are we going to architect things so we can do what we want, not just a year from now, but 10 years from now.

And right now, Linux kernel funding is very much dominated by, you know, the big cloud providers. And the big cloud providers don't care about reliability the same way the rest of us do. Yeah, that's kind of the whole cloud thing, right? They're like, well, you'll just spin up more of these throwaway instances or whatever. Yeah, if a node dies, it's no big deal if you've got your everything replicated across the cluster. But a lot of us actually do care about reliability of the single node. So I honestly don't care that much about the big cloud providers or, you know, the big money part of the tech industry, because they're not pushing technology forward and the field that I care about.

Yeah, they have a totally different approach to storage and how they manage this stuff that just doesn't really make sense for those of us trying to run regular sized arrays. But there are people out there who do need a file system that will never fail, who do need the core part of the system to be as reliable as humanly possible. So I'm not thinking about the big plot providers. I'm thinking about like, I didn't even realize this until years ago I saw it on my GitHub page. There was a little sticker. Something about, I had code running on Mars. Someone had noticed that everyone who had got code into the Linux kernel and probably some time frame or whatever,

well, NASA's running Linux on the Mars rovers, which apparently they are, that means I've got code on Mars. That's so cool. So, yeah. But they don't just need the kernel. They actually need a file system that's reliable enough that it's not going to take down the rover if it crashes, even if you're running in a high radiation environment where bidders are just common and expected. That really can recover from anything. Yeah, hopefully you can

save that data before you transmit it and have a chance of sending it out. Yeah. Unraid.net slash unplugged. You want to build your own dream server while Unraid 7.2 makes it easier than ever. Go unleash your hardware. Now with a fully responsive web gooey, Unraid now works beautifully across all your devices. You can set there and build your favorite application stack from your couch if you want. I think what you're really going to like, too, if you do the ZFS thing, is it also adds ZFSRade Z expansion support.

That means you can now grow your ZFS without having to start over. Man, that's great to see. And for those that maybe have a spare USB or external hard drive, unraid 7.2 introduces support for extended 234 and NTFS as well as extensive. extended fat. And so if you've got Grandpa's photos like I do on an old NTFS drive somebody gave you, you can just now, instead of having to build that Windows box

or try to load that NTFS driver on your Linux desktop, just plug it into your own right. There's also a new API. It's officially here, it's real, it's beautiful, people are building stuff on top of it, and it's open source. Fully integrated, it gives you secure programmable access to system

data for building dashboards, automations, or, you know, your own external apps. It even supports external authentication via Odick, or OIDC, as you called. It's massive. I mean, 7.2's already had well over 25,000 downloads, lots of applications coming out. You can get a free 30-day trial and support the show when you go to unrayed.net slash unplugged. It's the OS that grows with your skills, and 7.2 introduces the new unraid API, and it's chef's kiss. Check it out, support the show, and get a free 30-day trial,

unraid.net slash unplugged. Okay, so speaking to architecture stuff, you seem pretty open to the inclusion of Rust code in BcashFS in the future. Can you touch on that? I'm pushing hard for that. Yeah. Tell me about it. Well, I mean, Rust is a sea change moment for systems programming. For decades, how we wrote systems code didn't change much, and you just had to deal with.

You had to know that your code is. is perfect and be able to prove to yourself that, you know, your code is perfect. But the dream for a long time has been code programs with embedded correctness proofs. Things like Compert, you know, the first C compiler that is written in a language clock, where you can prove that it always generates, you know, code that matches the specification. It wouldn't it be great if all code is like that? Especially file system code? Right.

Yeah. I mean, I'd love that because I hate debugging. Yeah. So the question has been, is that stuff ever going to be practical? And Rust actually solves a huge chunk of the problem for, you know, in a language that we can actually use in the system space. That's huge. And there's deep reasons why Rust is designed the way it is and why it's well designed. Rust is about as close as you can get to a pure functional language

in an imperative language as you can get and you could argue that it's just as good as a pure functional language for all intents and purposes and not just that what's also coming on the horizon there's been a big development in how you actually make embedded correctness proofs practical and not something that you

that make you spend 10 times as long to write something like Comcert, which is dependent types. And Idris is the language that's been exploring this. It's a Haskell offshoot. And it's one of those things where in hindsight, if you're a kind of programming language nerd or a computer science nerd, you go, oh, that was obvious. Because there's this thing called Rice's theorem,

which is an offshoot of, or kind of a corollary to the halting problem. The halting problem says, in a trying-complete language, you can't prove or know ahead of time if a program completes or not. You just have to run it and see what happens. Rice's theorem extends that to,

in a turn-complete language, any property of the program, anything that program does is either a direct consequence of the syntax or it's undecidable. You just can't. know. So the direct consequence of the syntax thing is interesting because type systems are syntactical. You can reason about them syntactically. So what that kind of says is any practical

in proving properties of programs has to be a type system thing. And that's what they address independent types are doing. And it actually makes this stuff ergonomic. And I think one way or the other, that's going to make it into Rust someday. I mean, we could even do it because Rust has really, really good macros. We could probably add this stuff to Rust as procedural macros. And so my game plan is, you know, this might be like five, ten years out, five years before the Rust rewrite is mostly done. That won't be the big advance for B Cash Fest. The big advance will be dependent types.

Can you explain more? So there's File systems are giant state machines There's tons of things that just have to be consistent Sure Like in the B tree code Like all the locking stuff of the B tree Like

iterators Lots of higher level stuff And you can't model that stuff With the type systems of today So I have to write These very long And very heavy assertions that in debug mode that just methodically check,

is the entire state of this object correct? So you have a giant state space to explore that you have to explore manually because the type system, the compiler can't because you can't fully describe the complexity to the type system as is. Is that right? Yeah, yeah. So anytime we mutate like the state of the objects

that describe what locks we hold on the B-Tree, we have to call these verify functions the search that check, is this a valid state? but all that stuff with dependent types that becomes stuff that the compiler can check for us yeah okay that's great well is there anything you have to do you're trying to do now obviously dependent types takes getting to rust and rust

getting that functionality but is there are there things you're doing now practically today to try to prepare the code base in your approach basically yeah leading into something that I mentioned in the chain blog yeah so maybe I don't know a year ago? Well, rewind. C finally got the cleanup attribute, which lets you do, which lets you run a destructor for an object when it falls out of scope. Now, this is basically

pseudo-R-A-I-I, like C++ and Rustav. And then the kernel wrapped this up into a nicer interface, the class thing. And I found that, I started refactoring code and realized, as I was doing it, you kind of have to get rid of go-toes because go-toes, because go-toes don't play nicely with the R-A-I stuff. Sure, yeah. So doing the whole conversion going all the way has real benefits. You kind of have to. You don't want to be in this weird intermediate state

where you're trying to make sure the two don't conflict. I got kind of far along and I realized, okay, this is actually making the code fairly close to what the Rust version would be. Like, it's not going to be a full rewrite for a lot of the code. It's going to be mostly syntactic. So I've been pushing on this stuff pretty hard for the past six months

and we're pretty far along like six months ago when I started I grew up word counted and BcashFS had 2,600 go-toes in the codebase which was about the same number as XFS so pretty standard for a C codebase that size. Sure. And now we're down to about 600.

Wow. A lot of files don't have any at all. So the next step is going to be And we've already got some rest code and some B-Tree iterator, B-Tri API and transaction interface bindings. Those need to be improved and fleshed out. And then we can start thinking about just taking a file and just doing not a rewrite, but just a translation. If we've got the bindings, then it should start to become straightforward. Start to become straightforward with giant air quotes because there's always things that comes up and there will be a lot of interface work.

But it's actually looking tractable to convert 100,000 line code base to rust or at least large swaths of it. And that's, that's freaking awesome. Kent, I think it always impresses me. And you just demonstrated it here. It's like you have a grasp of like the really small nuance details that need to be addressed. And at the same time, and you're holding space in your head for like five year changes and really big macro stuff. Like that's a pretty rare talent. Well, been doing this for a long.

time at this point. Well, it's really impressive. Okay, so I think the listeners are probably wondering, what is your Linux setup like? Are you constantly breaking your system with new builds? Do you kind of have a standard box? Do you have a specific distro you go to? You know, the kind of things they might be interested in. Oh, I don't faff with that stuff much anymore.

I got too much work to do to screw around like I used to. So the big workstation is, which is still up in Canada, who's the, who's the? by a friend in his colo facility. That's a Debian installation that I haven't reinstalled in like probably 15 years. A proper depred installation.

Yes, that's a proper deviant. Yeah, yeah. So that's running not B-Cash-FS, yeah, that's running E-X4 on top of B-Cash on top of MD-Rade 6. Yeah.

And that setup hasn't changed in, yeah, forever. I've swapped out every drive individually to like, grow the array, but that's literally the same array that I've had for like 15 years. So that's like a home-based box that you SSH into or whatever, and it's kind of like your master machine? Yeah, yeah. Okay.

Yeah. And then do you have like a laptop or something with you that's sort of your on-the-go machine? Yep. And the laptop is running B-Cash-F-S and NixOS. Excellent. Ah-ha. Hey, very nice. Yeah, I mean, NixOS has made it very easy to run B-Cash-F-S.

It's really great. so we're all three of us to them elvis jericho as well right like from before it was in the colonel to in colonel to after colonel like all of it has been seamless which is great yeah elvish jericho hangs out in the channel and he's been interested in the system d stuff and we've been talking about ideas about how to sort that stuff out and get proper keep lives what i want to do is uh there's plymouth i think has some kind of mechanism for giving you information about your boot up I can't remember off the top of my head what exactly is there. But what I want to do is plumb proper progress indicators up to Plymouth. In the not too distant future, we shouldn't need it for upgrades. But if like there's major file system damage and we have to fall back to like Btree node scan, we don't want to tell the user like what we're doing and how long it's going to take. Yeah, it's great.

Or if a drive is going bad and we know that, we want to be able to alert the user. So, yeah. That would be really nice. Okay, now, no judgment zone. Do you have a desktop environment that's your go-to desktop environment on that laptop? Sway. Oh, fun. All right.

Why Sway? Well, before Sway, I was using Awesome for years and years. Nice. Yep. And I love that, but it was X-11 only, and it was never going to come to Wayland, and I finally did the Wayland switch for a little reason. I don't have strong opinions on it. Wayland's been fine. But, yeah, Sway was the wheyland option, and it's actually been really clean.

Yeah. I like how it does configuration, and yeah, it's pretty cool. So your NixOS configuration, is it like a configuration you've brought with you from machine to machine, or do you start fresh when you got a new laptop? Like the first time you set it up, did you start fresh, or is this something you drag along with you? Yeah, I try to keep my configs in a Git repository, which I am very bad about keeping up today. So I generally just moved the whole SSD to the new mobile. machine.

That's such a fun system guy thing to do. That's great. How could I be laziest? Hey, as long as the disc still good, right? It's solid. That's what matters. It makes it easier.

Well, gentlemen, do you have any other questions for Ken? Oh, did you want to talk about maybe BcashFS top? That was maybe something. Oh, yeah, yeah, new tool. Or improved tool? Oh, yeah, yeah. Actually, that's been out there for a while, but some improvements landed.

So BcashFs Top is, this little thing that just shows you counters for your file system. It's the simplest, dumbest thing ever. But, you know, let's see what your file system is doing. It's like a new thing in the process. We love it. I like all this. I mean, there's a lot of stuff in here.

Copy G.C. weight, a whole bunch of error information. I.O. Read. I. I.O. Right. Oh. Yeah. Some of those are old and probably don't make much sense. It should be swapped out for better ones. But I've been doing a lot of work on, like, the reconcile and data update path, events and trace points. So the big thing that I just did was I was already moving in this direction, trying to hold to it, but I finally did a giant code-based refactoring so that now it's done and things will never get out of sync.

Every counter has a corresponding trace point. So the idea is that you see what your file system is doing, you look at FS top. That will show you like what events, what code paths, like what is it doing? And then if you want to drill down and find out anything more, the name of the counter is the name of the trace point and the trace point gives you information about that specific events. Wow. Oh, that's so great.

Wow. Smooth. That's simple enough we could actually do it. Maybe with a little LLM help, but we can get there. And if any of those trace points isn't showing you the information that you need in order to understand what's going on, it's dead easy now to just, you know, everything is pretty picture-based. We can just dump a couple fields in or dump.

but another variable in there, and it's like a five-line patch. So you can just hop on the IRC channel and say, hey, I'm trying to figure out what this is doing. And it could be like, here's a five-line patch for you and it'll be in the next release. That is. I love how much, just from the developer perspective,

how much is sort of the modern development affordances and workflow you are trying to impose on this otherwise very austere and, like, you know, hard-to-use environment. And it seems like it's really paying dividends. I mean, from the file system as a database to insisting on automated testing, to all the stuff you're just talking about.

Yeah, and a lot of the recent stuff, as the project has grown, a lot of the real polish and the stuff that I really like to see has come from just working with users. A lot of developers think of support as a chore. But what I've realized is that, you know, actually, basically any support issue that comes up, there's something that we as engineers can do to just make the system easier to use

or tell you exactly what's going on or make the issue go away entirely. So doing support, it motivates you to just improve that kind of polish. And you get lots of really fruitful interactions about, hey, the system is doing something monkey. And he's got to be talking to people. Yeah. I feel like, you know, when I think about BeCash's, B Cash Fest's contribution to Linux over the next 20 years, you are in a very unique period of time right now where it would be great.

to have a lot of engineers at the same time, you're really, you're very hands-on. I mean, Kent, you're not only like doing the day-to-day work, but you are communicating with users. You're writing comprehensive release notes that I think are better than most projects release, and then you're communicating with folks at us at the same time. And it's really, it's a special thing that's happening right now

that 20 years from now is going to people be using a file system, they may not even know that's had this kind of intention and energy put into it. But it's really great. It's a lot to juggle, but it's a lot of fun, too. And let me flip some of those questions around, too, because you've been playing with B CashFS a bit. What do you like about it?

What do you find modern and interesting about it? I think I'm really kind of excited just having a file system that I can trust at the caliber that I would trust ZFS, but with features that feel more approachable, it seems more reasonable on my type of systems, and is built in to the kernel or is easy enough with, like, Nix OS, to run as a DKMS module. And, like, you add all that stuff up,

and it's essentially, it feels like the true Linux competitor to ZFS that we've been waiting for since Sun Microsystems was a company. That's been the dream. That's what I and like funders and everyone have been talking about for years and years. And not to take away from ZFS. Like I want people, they can have it. It's great.

Oh, totally. Totally. Like ZFS deserves so much accolades and respect for showing us what a modern file system should be and demonstrating that it was even possible. Right, right. And it's made a lot of people happy. It's great. It's just, we need something in Linux, too, right?

We need something that's GPL and something that's created kind of in the Linux ethos inside the Linux community for Linux, which is exactly what you're doing. And there's also legitimate technical advancements to be done on beyond what ZFS did. Like, probably not a lot of non-file system nerds know about the inter workings of the stuff, but ZFS was actually a very conservative design in a lot of ways. technically for very smart reasons. If you're trying to do a very ambitious feature set, you don't want to reimagine everything all at once.

But the core design very much is in line with the original Unix file systems of like radix trees, of blocks hanging off the I node and like all the on-dust data structures. It's block-based. It's not extant-based. And we knew like 20 years ago, 30 years ago, like XFS did X-Sense. The extents are the way to go. The trouble was that extents and snapshots are a very difficult combination, and one that took me years to figure out.

And there's lots of other stuff, like the whole file system as a database thing. I could talk for a while about that one, but that was BcashFs' biggest contribution that's really simplified the design and made a lot of future work just dead easy compared to what it would have been in previous systems. I'll give you one example. So towards the end of Reconcile, I was debugging with a user who is, there's like some stuff with like Betri nodes, not going on the pending list or coming off the pending list or something. But he mentioned a slow evacuate, and he's running on rotating disks. And I went, hang on, the main reconcile implementation scans the key space and logical key order.

Like I know a number offset within the file. But for rotating disks, you really want to be doing it in like physical LBA order. The disk layout, data is laid out in the disks. And no fits are off in the channel pointed out that, hang on, the old style of evacuate, actually, you mentioned this one, I was already doing the fix, but you mentioned that the old style of evacuate code is already walking back pointers to do it in physical albiator. This is going to be a real performance regression. And, you know, evacuate is something that actually you do want to run at a reasonable pace. Yes. So I had to add extra indexes for indexing pending reconcile work by physical LBA order,

plumb it all into the triggers, add the FS check passes to make sure that that stuff is correct, hook it into the back pointers code. I was able to get all that done in a day. Start to finish. When I was thinking about the design to passing tests and working, that's like not everything is like that but that's what you can sometimes do when you've got a good solid foundation and you know like of course the database engineers are going that's just adding a table like we could do that and like yeah yeah we follow that yeah wow that's awesome you know you said that i think that like

a good a good solid foundation that's kind of what bcashf s feels like i'm using here with compression and the built-in encryption and we we use and like Butterfess on the show, but it just feels like BcashfS is primed to be sort of like a less sharp version that you could just sort of always trust. Something that you can debug.

Yeah, yeah. Yeah, that tells you where it's wrong, that you know you can at least recover data off of, and where like the features that are in, you know you can rely on them. Yeah. Yeah, that's kind of what I'm going for. I think it's one of the projects that we follow with the most interest and

excitement. So thank you for just all love the work you've done, especially recently. It just seems like it's really been getting, going from strength to strength. Yeah, the community feedback. Doing it for the normal people, that's, that's what I'm in this for. Like, I don't care about, like, what Redad or Facebook are doing. I'm just, you know, writing code that's solid and reliable and makes people happy. That's, that's fun. Yeah, it is. Yeah. And, you know, there'll be just every day more and more people using it every day. And it's going to solve more problems as people learn about it. And we're going to keep spreading the word too. If we do have

users who give it a go after this and they do run into issues, what are the best ways for folks to actually interface with you and either get support or file bugs? Jump on the IRC channel. You can file a bug, but it's generally a lot quicker to get stuff resolved. If you can join the IRC channel,

then we could just talk people through gathering what information we need and having a normal conversation makes things a lot quicker. I don't know we can bring them into the community and get them to do more testing and file I file more bugs. Excellent.

Thank you, Kent. Pleasure. OnePassword.com slash unplug. That's the number one password and then unplugged, and that's all lowercase. Go take the first steps to better security for your team by securing credentials and protecting every application, even the unmanaged one. So go learn more. You need to go to one password.com slash unplugged. This is a real challenge.

There's a lot of assets to manage. I mean, you have hardware, all the different devices, from mobile to desktop to laptops and more. But there's also identities, of course, and there's applications. And there's more and more of those all the time that just spin up that you might not even know about. It's a lot. It creates a mountain of security risk. And you can conquer that mountain of security risk with one password extended access management. It's a big problem.

Lots of people report this is their biggest challenge in IT, just a sound. Sprawl, you could say, which creates shadow IT, account services you might not even know your users are using. And it's not hard to see why the users get more work done. I remember this when both Dropbox and Slack rolled out. It was a big user up adoption. And it created friction between IT and the users. That's something that one password extended access management really smooth out. And one of the ways you are empowered is with Trellica. Trellica by one password can discover and secure all your apps, managed or not. That means you're going to get an inventory of every app in use at your company.

Trellica has pre-populated app profiles, like SaaS. So you can get idea of the SaaS risks. You can get an idea of who has access to what, if there's redundancies. You can really optimize your spend with tools like that. But probably most importantly, you can enforce best security practices across every app your employees use. You can actually manage the shadow IT. You can securely on-board and off-board employees, and you can make sure you're meeting compliance goals. Trellica by OnePassword provides a complete solution for SaaS access governance.

It's just one of the ways that extended access management helps team strengthen compliance and security. So go check it out and support the show. You go to OnePassword.com slash unplug. You know about their award-winning password manager. It's trusted by millions of users. This goes way beyond just passwords. One-Password extended access management. It's powerful and it gives you a complete oversight of the sprawling landscape of SaaS apps,

whether your company knows about them or not. That's a big deal. Go learn more, support the show. Go to the number one password.com slash unplugged. That's one password. com slash unplugged. You're going to change the way you look at managing IT. And it's a great way to support the show.

Go take the first steps to better security for your team by securing credentials and protecting every application. even the unmanaged shadow IT. OnePassword.com slash unplugged. That's one password.com slash unplugged. Join crowdhealth.com and use the promo code unplugged. It is open enrollment time, the season where the health insurance companies are going to hope you'll just sign up again for overpriced premiums and confusing fine print that never seems to work in your favor. And the prices just seems to get worse all the time. It's horrible.

I had a tough call to make, especially as a small business owner. or what direction I was going to go. My wife also owns her own small business. And we looked at all the options. We tried the traditional routes. It wasn't working for us. I was so grateful over three years ago when I found Crowd Health. I think it's time to stop playing the health insurance game. You can join Crowd Health. It's a community of people funding each other's medical bills directly. No middleman, no networks, no nonsense. But don't just take my word for it. Go trust yourself and go take control of your future with CrowdHealth. It's a health care alternative for people who make their own decisions. Health insurance is, it's really stressful. It's confusing, and you never really get what you want. And when you see the prices, you wonder what's ever going to stop this train. Well, this is how we take the power back. And it's incredible how well it works. And they have a beautiful app that makes it really simple, straightforward.

Of course, they have a website too, chat, customer supports, all right there, really great leadership team. I've talked to the CEO. I've talked to people around there. Just casually and, of course, through the course of doing business with them now for the ads. and I'm really impressed. And not only do I like what I've seen over the last three years, but I like the momentum.

I feel like I've picked a winner here. I think it's something you should really check out because it works well for me. It works well for my wife and I, and you can get health care for under $100. You get access to a team of health bill negotiators, low-cost prescriptions, lab testing tools, as well as a database of low-cost, high-quality doctors

that get vetted by crowd health. It's an insurance alternative. And if something major happens, you pay the first $500, then the crowd steps in to fund the rest. It's really the way it should work now. And, of course, you'll join the crowd. It's a group of members just like you who want to help pay for each other's unexpected medical events. You'll be impressed of how well it works, too. The system is betting out there that you're just going to keep buying the same overpriced insurance.

And man, are they just making a boatload of money? And it gets so complicated now. And if these subsidies expire, prices are going to go sky high. If you join CrowdHealth and use our promo code unplugged, you can get started for $99 for your first three months. That's fantastic. So far, CrowdHealth members have saved over $40 million in health care expenses because they just refuse to overpay for health care.

Now, crowd health is not insurance. You should opt out and take your power back. This is how we win. This is how we change it. Joincrowhealth.com promo code unplug. Take your power back and come join the crowd with me. I think you're going to be really impressed. Joincrowdealth.com and use the promo code, unplug.

And you will get your first three months for $99, which is incredible. Use the promo code unplugged at join crowdhealth.com. While we have a special baller booster today, it is our dear PJ. Hey, Rich Lobster! Jeff Boosin, a row of Mick Ducks, 22,22. Things are looking up for old McDuck. Jeff says, okay, Brent. Okay, call me out for not boosting while I'm out running errands and can't defend myself.

A guy only has so many sats to boost. That's true. Always down to boost a good soapbox episode, though. I should make some soapbox art for merch or something. Oh, great idea. That'd be good. Could you get me, if I'm on a soapbox, could it be me in a soapbox with a dress shirt and shorts, my traditional style for the summer, and then like maybe holding like the, like the, like

the, a flag for Linux unplugged, it's like, rah, you know, and I'm like crying to the sky with my my rant, I guess. I don't know. I wanted to, you know, invoke a real image. Your quest for your desktop freedom. Yeah. I think I just came up with a prompt. Yes, you did. Can someone just run that for an animated Jeff? Thank you, PJ for being our baller booster, sir, and glad you're listening. Appreciate that support. You know, also just a quick note, P.J. minds those sats most, I don't know if these ones in particular, but often would he boosts in, he's mines with solar energy. He uses a clean solar energy to mine his sats and then he boosts in the show.

How about that? How cool is that? For Jeff, those sats are as local as possible. They're made on his roof. Yep. And, you know, Jeff sends us all kinds of value and all kinds of ways. Magnolia Mayhem comes in with 2,123 sets. That's 2.1, 2.3. Make it show.

I'm back at it, like an old pair of jeans. That's the last time I leave my filthy clanker, let my filthy clanker touch my Nix config. Uh-oh. Yeah, I, every now and then it goes bad, man. It can go real bad. Wipe out a user account or two on you. How would you know anything about that?

Wouldn't, wouldn't know anything about that. Thank you for the boost. Thank you, Magnolia. Appreciate you. Southern Fred Sassa comes in with a row of ducks. Oh, it's for me, for West. Oh. If the family wants to rid themselves of that radioactive artifact.

Oh, yeah. I don't know if the list was from the members feed. Yeah, the members feed. You told us about something you discovered over the holiday. Yeah, a radioactive personnel identification marker from the World War II era. Southern Sasa goes on to say, most gun and knife shows will have at least one guy who peddles in World War II memorabilia. That's true. No shipping required.

Also, I had to tune into the video stream to see the infamous kitties. Brent, your co-stars are the true stars. That's true. Sorry. Hey, I completely agree. And good idea there, thank you. During our chat with Kent, for most of it, Brent's cat, Cosmo was completely wrapped around him like a scarf.

It's a live scarf. Stylish. Yeah. Yeah. It even purrs. Thank you, Southern Frad. Appreciate you. Well, so I am sent in 2,000 sets. B-O-O-O-S-T.

Just a clarification on Git here. I don't mean the people shouldn't learn or be taught kit. Rather, that it's basically impossible to teach people get because of its incredible complexity. and just downright weird UI and U.X, for example, just take a look at, and they link us to a blog here, it looks like, maybe a man page, and tell me if that page is real or fake. Yeah, you should check this out because it's a great.

It's a great site. Okay, all right. What do you mean if it's real or fake? What do you mean? What are you talking about? Well, take a peek. It's going to show you a man page. Okay.

Oh, get Saw Command. Is this real? Get exhibits, which blob? Okay, I would have no idea. I'm going to guess. this is not a real command. No, it's a generator of fake commands, but the point is they look shockingly like actual Git.

That's a pretty great way to draw that point. That's pretty clever. I appreciate that. You know, I was just, while Brent was reading that boost, I was just browsing GitHub.com. It's just fascinating how nowhere on there does it say Git. It says GitHub. Very little, but nothing about Git.

Nothing. It's just. No, not until you kind of get into the details. Yeah. If you use their, like, you know, GH tool, even less. Yeah. It's interesting. You actually, with the web interface, you almost don't need to know Git if you're just, like, managing a project or something like that.

Yeah. Yeah. Hey, there he is. It's Gene Bean, guys. He came in with a row of ducks. It's as though I use iOS as my daily driver. I always keep some kind of phone around running lineage OS or something similar. Given a reasonable price, I'd absolutely love.

for that device to be one that shipped with graphing OS. Oh, yeah. Mm-hmm. So that'd be fun. The kicker for me is that I need not be as pricey as a flagship phone so that I can justify it as a secondary device. Fair enough. Fair enough.

You know, as crazy as it sounds, there's a market for a secondary phone device. I mean, I carry two phones. And I agree you don't need the second phone to be the most cutting-edge phone. There's small computers. And how many computers do we have already? Maybe that's where the new Yala phone will fill a gap. Thank you, Gene. It's always great to hear from you.

Appreciate you very much. Hybrid sarcasm comes in with 10,000 sets. Put some macaroni and cheese on there, too. Just completed my holiday homelab submission. Well, thank you. That's great. All right. Be gentle, he implores us.

Also, who's in the lead for the Boosties 2025 prize? Oh, okay. All right. And remember, there's going to be a giveaway, right? Hybrids mentioned this, a giveaway for a Jupiter Party membership. Now, we don't want to give away the winners or the totals. So we looked it up. We got the current numbers, and then Wes put the names through a randomizer.

So this is who is currently in the top four for the most boosts to this show in 2025. Black host, our podcast, Adversary 17, and the dude abides. All in the range of top four. That list was in no particular order. It was randomized. And we are, I guess, in line with. the holiday home lab going to, I think the boosies will coincide, right? I think that's the way we'll do it. We'll do it together. So it'll be one big

spectacular episode for the holidays. Which would mean that you have until, say, maybe the 18th or 19 to get any boosts in if you do want to. Be at the top. Yeah. Reclaim your standing. Mm-hmm. Mm-hmm. Thank you. Hybrid. Well, Deoma. Deoma boosts in 2000 sets. This is the way. Lately, I've been able to leverage more open-source tools,

namely Ansible and Open Tofu at work as a SIS engineer. I find myself thinking about it constantly and working much later hours while still feeling that I have a good work-life balance, even though I'm working for free, basically? Should I find other work that would better utilize this passion? Is it possible to just pick up an IT cis gig on the side, for instance? I'm not a dev, so I struggle to understand how I can contribute beyond submitting a PR on GitHub for a typo.

Well, infrastructure is a great skill set. You don't necessarily have to be a developer, and of course they do blur more and more. Side-gid question, yes. I think small businesses are always looking for help. They need somebody who can do part-time work because they can't afford a full-time IT person often. So if you know a small business in your area,

could be somebody that you patron and you've noticed they have computer problems. That has literally worked for me in the past. And then going back to essentially working for free, I often find myself, when I'm in the learning phase, I've got a lot of energy to just really commit to this and maybe sometimes an employer benefits from that, but I'm also benefiting. Yeah, I think you can probably frame it as like an investment in yourself and there's a certain value at least I have. I imagine you do too on, you know, curiosity, learning and the sort of the fun play discovery aspects of making the stuff work. What's that Dunder Mifflin curve too about learning stuff, you know, I'm talking about where.

at the beginning you don't know what you don't know and you start to think you're an expert and then as you learn more you realize you have so much more to learn and then you kind of go down that curve and become an expert again and I think too like you could burn through a bit of that phase during this kind of learn what you don't know

have a better understanding of a wider context and then absolutely make a career out of it for me is also something to be said for having a way to apply the things you're learning so sometimes the work environment is that place and if you're going to be learning it on your own, maybe you don't have the same opportunity to solve a problem or implement a new solution.

I do think you're right. Infra is often something open source projects need. I think it's probably a little tricky to get involved because you're probably going to have to hang around a little while to figure that out. It's not always advertised. But I'm just thinking, you know, the Podverse folks are hard at work on their latest versions,

and I know there's some folks who are part of our community who help out on their Infra side. And that's... Podverse is a great shoutout. It's a great idea. So I don't know about the particular communities that you're involved with necessarily, but there's probably something out there somewhere that needs some love. Retro gear boosts in with 2,998 cents.

Oh my God, this drawer is filled with broolopes. No, using Albi. Oh, cool. Oh, yeah, connected to Fountain now. That's a beta feature. Neat. Yep, yeah, Albi Hub, Nosterwall, connect. Okay, retro says, Jens, Proxmox, NixOS actually works pretty well.

the sort of hybrid, right? It's like Proxmox, but it's running on NixOS. I've been seeing a bit more buzz around this, so I'm interested to hear what he has to say. I'm running a few VMs on there that can't be done with modules or Docker containers.

Cheers. Wonder what those would be. Well, so basically I think just like stuff you might want to run into VM. So as an alternative, right, instead of doing the setup we were doing with Home Assistant, you could write Home Assistant on ProxMox.

Right. Still running on NixOS. Right, right. Interesting. That's a lot of layers, but it could solve some problems. Yeah. I mean, you do it. You probably want to have all the ProxMocs stuff, right?

Like the API and all the... Yeah, and if you're, like, baked into the ProxMox backup system and, you know, you know that system, that makes a ton of sense. Appreciate the experience report. That's pretty useful. I'd love to hear more of them. Well, Terd Ferguson here sends us a boost, 14,44 cents. Turned Ferguson!

Hey, boys, did you see that Bazite pushed a petabyte of iron? so as in the last 30 days. It looks like they also just crossed 40,000 active weekly users. Do you think everyone is loading these on Steam decks or something? Hmm.

If they're hitting 40,000 active users, there must be some Bazite, active Bazite users out in the audience. I also would love some feedback on if you put it on a Steam deck. I have my Steam deck. I'm not, I've not been using as much the last few weeks because we went back to the switch for a smoother, rather the wild playback.

Yeah. So, I mean, that would be a good time to reload that thing. I'd love some field experience out there. I kind of imagine it's people that are trying to do a Steam-like experience on non-deck hardware. That would be my gut. That makes sense to me as well. Although, it would work just fine if you just game a fair amount and also need a workstation. Or if you've got, like, sort of like Jeff does, if you have a living room computer that you want.

to have a similar experience. That might be a nice option. True. I think that makes Bazai clearly the winner of the UBlue projects. That's great for them and probably great for all the other projects, too. Well, there you go. Thank you, everybody that boosted this show. We really do appreciate it and all of you who stream sats.

22 of you stream those sats and you collectively stacked 37,000, 664 sats. That would be one of our largest boost for this episode. That would be the largest boost if you collected it altogether. So thank you very much. When you combine that with our messengers, we stacked 98,395 sats just under that 100K sats line for this episode. Not our best week, but we do appreciate everybody who contributes to the show with their time, their talent, or their treasure, be it through a membership or through a boost. We are here every week for you doing the show that only Linux Unplug can do, and that's because we're focused on our audience and not clickbait or things that might play well on shorts or YouTube or things like that. We're just trying to make great quality content for our audience.

And thank you everybody who boosts with something like Fountain FM or AlbiHub or one of those. We really do appreciate you. Fountain does just keep getting easier with lots of features, fountain.fm, for all of the hosted infrastructure, podcast apps.com for those new apps. And of course, if you'd like to support with a membership, that's LinuxUMPLug.com slash membership and jupiter.com for all the shows. I gave you a lot of URLs, but I trust you. You're pretty clever.

You're pretty clever. All right, we've got an overloaded pick section. We should probably move through this pretty quick. So let's start with the one you probably shouldn't use. Dot files installer. Dot files installer, everybody. What is going on here? Is this your way of not using Home Manager?

No, no, no, no, no, no. I'm not using this. Although I did use it to back up my dot files. That was great. No. So it's really for people that publish, different communities publish these curated dot files. So you can pull down one file that has all these configurations for popular apps and desktop environments. And it's really, it's a way, it's way beyond theming.

It's a whole system vibe. And you can have lots of different sources of these. You can have the problem where you're overriding things. What this application does is it lets you manage the various sources, turn things on and off, backup your own things, and essentially try to prevent any kind of loss that would mess your system up entirely while you're yoloing everybody's different configurations down. And so it does try to prioritize data safety, offering a robust backup functionality before you start just yolowing different configs down. And then you can also publish your own ones if you want to help other people just yolo into their config. And it is a nice G2K application. It makes very clear how you could possibly be messing your system up.

You're making me like it. I know, right? I know. I mean, I don't think anybody should do this. Don't do this, blah, blah, blah. Safety first, all that. Don't do as we do. But if you just had a system where you didn't carry a whole bunch or it was easy to roll back, this does make it kind of easy to roll back.

Or do you intentionally want an experiment? Yeah. Okay. Okay, I'll try it. Yeah. Say you wanted to get yourself on Hyperlinder or sway or one of these and you don't really know how the hell to set up anything that looks nice. Sometimes these can be a nice starting place. That's a good point.

So the stop files installers, GPL3. I believe it's on FlatHub as well. Okay, now you're ready for this next one? Yeah, this one, I kind of like. This one for me, when MacOS started getting good, now it's gone the other direction, but, you know, when the first releases, it was pretty rough,

but they started getting to a point where a lot of independent developers started getting on board and creating these purpose-built applications that just make life a little bit simpler and easier for a lot of users on MacOS. Right, they could just focus on this, like one little task and do that well

and so it was pretty polished and it had all the like features that would come with working on it for a little while. And it kind of become like the go to application for doing this. Well I hope that's SSH Studio. It's a GUI that's a GTK4 Libidwadia GUI that is an easy to use config editor

for your SSH setup and it gives you what they call a visual host editor to edit common fields like the host name, the host, the users, the identity files, any forwarding agent setup. It does inline validation. Of course, that helps you search across your machines. But I think the bigger feature in here is really, probably for folks that would be using a tool like this is maybe the SSH key management stuff. And it's if you connect to a handful of hosts, if this is an area that has sort of been troubling for you, like this exact kind of example of these purpose build applications that as a platform grows in user base, we just start to see a demand for it. And so in a way, I wanted to note it because I feel like it's important in that way, but B, it's actually kind of a useful. app it's pretty lean pretty mean yeah i mean it just kind of shows you what's going on it could

be a nice way to investigate a system um it could also be a good way to discover some of the options that do exist for your ssh config and connections if you haven't taken advantage of those and it's easy to install and it's gpL3 nice last but not least is what i'm calling the honorable mensch section of the picks because it's something we talk about too much but and you couldn't resist talking about it some more. They just came out with a great release. I found it personally very useful for tracking my total cost of ownership of my vehicles. We've talked about Lou Blogger before, but a big version just dropped, and I want to let you know about it because I think these are great. First up, a new feature, inspections, pretty much what you'd expect. It's custom forms where you can

go inspect your own vehicle and create reusable inspection forms, which is awesome for me. That's great. And create your own inspection checklist and then log that in Lou Blocker. Huge. Huge. Also now support for households. So this is one of the most requested features of Lou Blogger. It's the ability to allow users to inherit vehicles in a garage and limit what actions that can perform on those vehicles. It's kind of like a permission structure. And so you no longer have to add a user and each vehicle.

Instead, you can add them to a household. And then you just add that car, you add them to the household. They automatically get access to the vehicles in your garage. And they also created a viewer only, so a read-only user. They are handling, I think, the integration of AI in a pretty reasonable way. they say we know it's controversial. So what we're doing is instead of adding AI directly into Lublogger, we're asking for an API key.

And then we've decided to create an MCP server for Lou Blogger that you spin up separately, probably like a container or something like that. Right. And then you would use that as a bridge between AI agents. So instead of having it built directly into main Lou Blogger, it's like a sidecar. Yeah, I mean, it seems reasonable enough, right?

Basically, it's just repackaging probably the existing API into the MCP standard. Now, there are some features in there that are kind of neat. It allows you to add fuel records from receipts, odometer records from a picture. Oh, that's nice. Yeah, so you just take a picture of your dashboard,

update the odometer reading. Yeah. You might actually do it. Or like when we take our trips to any event that we drive to, we have to track all the receipts. And so, like, when we're on a road trip, we will sometimes just have, like, just a massive bundle of receipts when we're driving across the continent.

And this time around, I was using, I think I don't quick scan and paperless, you know, to try to capture it. A little more real time. A little more, but something like this for my maintenance records and stuff like that, where I could just bring it right in. Just anything that makes that smoother is really, really, really great. And also invoices and those types of things to like for repairs. It's still an experimental stage, but that's, I think, an idea where it's implemented not,

so they're not slamming it in your face. You can run this sidecar and you have opt in. And then, by the way, once you do opt in, here's an actual couple. practical, useful things. Not bad. I think they nailed that. So I think that deserved an honorable mention, huge release for Lou Blogger,

and it has been, it's really been a game changer for me because it helped me understand that, like, this vehicle's getting, actually two of my vehicles are now beginning to cost more than I want to spend on them in maintenance,

and you can track that kind of stuff in a way that isn't tedious. So I have not been historically great at that, and Lou Blogger has made that very easy. So thank you to them. And congratulations. Also thank you to Kent for taking the time to join us. He's a busy guy. He's got a lot of demands on his time,

and we really appreciate him taking the time to chat with us. And we'll put some links to the new release in the show notes. And don't forget, he's also on Patreon. So if you want to support his work directly, that's how he's getting it done, we'll try to put a link to that in the show notes, too, so people can do that. And Wes, besides those show notes, being over at Linuxonplug.com slash 644, if they're in a mobile client that has a few fancy features, they might get some bonus stuff, some stuff that makes the show even better for them. Yeah, cloud chapters for one. That's right.

JSON delivered live right to your podcatcher. And don't forget about those beautiful transcripts that are derived from individual tracks of each host. So it is the most pristine transcription we can possibly generate for you. And for the clients that support it, we are also doing attribution. So it tells you which speaker is speaking. That isn't easy, but we do it for you. And it's out there. And hopefully, you know, as we just do more episodes, we're going to create a huge dataset that we'll use for

something one day. I don't know what. And then last but not least, as we round out the year, see you next week. Same bad time, same bad station. We sure would love to have you join us live. JBLive.tv on a Sunday. Make it a Tuesday on a Sunday, 10 a.m. Pacific, 1 p.m. Eastern. Hey, that is also technically a magical feed feature. That's true. It's in those feeds. And then, don't forget, December 19th live, it is the Home Lab holiday special, and you need to get your submissions in by that Wednesday. So that would be the 17th. Get your submissions in at Linuxunplug.com

slash holiday. Thank you so much for joining us on this week's episode. We sure appreciate you listening. And we'll see you right back here next Tuesday. As in Sunday. You know, Thank you.