Tech Over Tea - Reverse Engineering the Apple M1 GPU | Asahi Lina

Episode Date: December 14, 2022

Asahi Lina has been at the forefront of reverse engineering the M1 GPU driver and today she's here to talk about that process, the state of the project and much more. ==========Guest Links========== Asahi Lina Twitter: https://twitter.com/LinaAsahi Asahi Lina YouTube: https://www.youtube.com/AsahiLina Asahi Lina Mastodon: https://vt.social/@lina Asahi Linux Website: https://asahilinux.org/ ==========Support The Show========== ► Patreon: https://www.patreon.com/brodierobertson ► Paypal: https://www.paypal.me/BrodieRobertsonVideo ► Amazon USA: https://amzn.to/3d5gykF ► Other Methods: https://cointr.ee/brodierobertson =========Video Platforms========== 🎥 YouTube: https://www.youtube.com/channel/UCBq5p-xOla8xhnrbhu8AIAg =========Audio Release========= 🎵 RSS: https://anchor.fm/s/149fd51c/podcast/rss 🎵 Apple Podcast:https://podcasts.apple.com/us/podcast/tech-over-tea/id1501727953 🎵 Spotify: https://open.spotify.com/show/3IfFpfzlLo7OPsEnl4gbdM 🎵 Google Podcast: https://www.google.com/podcasts?feed=aHR0cHM6Ly9hbmNob3IuZm0vcy8xNDlmZDUxYy9wb2RjYXN0L3Jzcw== 🎵 Anchor: https://anchor.fm/tech-over-tea ==========Social Media========== 🎤 Discord:https://discord.gg/PkMRVn9 🐦 Twitter: https://twitter.com/TechOverTeaShow 📷 Instagram: https://www.instagram.com/techovertea/ 🌐 Mastodon:https://mastodon.social/web/accounts/1093345 ==========Credits========== 🎨 Channel Art: All my art has was created by Supercozman https://twitter.com/Supercozman https://www.instagram.com/supercozman_draws/ DISCLOSURE: Wherever possible I use referral links, which means if you click one of the links in this video or description and make a purchase we may receive a small commission or other compensation.

Good morning, good day, and good evening. Welcome back to Tech of a T. I am, as always, your host, Brodie Robertson, and today, I believe, is episode 146. Yes. And today we have a returning guest. Last time she was on with Luna the Fox Girl, but I decided to do this time separate episodes so we can go more deep into their individual topics. Welcome back to the show, us, Helena. How's it going? Hello! It's going pretty good! For anyone who doesn't know, like, what you do, I'm sure most people have probably heard of you on the Foss Space at this point, at least to a little bit, or at least heard of the project you're working on. Give a brief overview of sort of what you do.

So I've been working on the Asahidinx project, which is a project to bring Linux to M1 Macs and the Apple Silicon family. And so I started working on the GPU kernel driver after Alyssa had been working on the GPU user space driver because GPU drivers have two parts. So we had a user space driver in the works, but there was nothing for the kernel. So I started working on that this year. And that's what I've been doing pretty much this whole time.

And the last time you were on, it was definitely not in the state that it's in now. We were talking about this just before, but you were just getting like... So you had a Python user space driver, is that correct? Yeah, so that was a Python user space driver that does what a kernel driver would do as an experiment because we have to reverse engineer the GPU because we don't know how it works. Because Apple won't help you.

Yeah, so as part of that, initially to get a feel for, you know, everything works first, we use a hypervisor to like snoop on what macOS does with the GPU. And that's all based on Python stuff. And so then I use the same code, but sort of backwards as a demo to sort of prove the concept that it could work. And so this is basically a Python code doing what a kernel driver would do in user space on a development machine that is not the m1 but it's connected

to the m1 via usb so it was this crazy thing where like all the video frames and all the geometry and all the gpu stuff was happening over usb so it's basically kind of the worst case scenario for getting anything working well but enough to at least work out if it can be done in the first place yeah the idea is um that um like to get the design of the kernel driver right i need to understand how the gpu works to a certain point and it's much easier to iterate in python on a host machine um you know where i can just run a script and keep rebooting the m1 or whatever. And I can have an interactive shell to debug things and pretty color printing of everything

and all that stuff. So it's a lot more convenient to initially figure out how it works in Python and then build this proof of concept. And then once you have a proof of concept that is like, okay, I can actually run some real apps with this and it renders multiple frames, and it basically works, and it's doing what it's supposed to do. And you figure things out, like how to allocate on free memory properly and how to get the notifications that a frame has finished rendering

and what the concurrency issues are and how can you do multiple things at once. I can answer all these questions with experiments in Python, and part of it was running real apps, part of it was actually just dumping single frames from an app and rendering them manually in Python, but then I can do more complicated things, like doing multiple things in parallel. And so by doing all these experiments, I can figure out how it works,

and figure out a design that makes sense. And then after that is when i started on the kernel driver so what would you so the earliest thing that was like basically rendering something when you consider it doing at least that basic of rendering a single frame was that the was that like the triangle was there something a little bit earlier than that uh that was the triangle in python yeah so that was the triangle that was, it wasn't running a real app. It was a frame, as in the geometry and the commands dumped from the triangle running on macOS, as the macOS Mesa driver would send to the macOS kernel driver,

taking that data and putting it in Python and doing what the kernel driver would do in a Python script and getting the GPU to render a triangle. That was the very first thing that worked. That was what? So last time we recorded, it was five months ago and you were a bit further ahead at that point. So somewhere between five to six months ago, you would say you were sort of in that state? Yeah, something like that. That was, I think it was five to six months ago, you would say? Sort of in that state? Yeah, something like that. That was, I think it was five to six months ago, yeah.

Because, yeah, when you were last... Because I know you definitely had a Notch2D... Okay, I wouldn't say working well, but functioning to some extent. Because you had the... I think one of the thumbnails, I used the glitchy render that you worked out of force. But things have come along a long way since then i've got a i think a video coming out tomorrow doing sort of

an update on where things are at right now but where where are you at right now because if someone hasn't heard uh considering like the jump that there been, I think they might be a little bit impressed where it's at now. So we have a real driver now and it's not like a full driver that can run the latest Falcon, OpenGL everything, fancy pants

but it's a real driver that can run OpenGL ES2 and ES3 and Mesa is also improved to pretty much that point. So ES3 is not 100% yet, but it's getting there. And basically, we can run desktops. We can run games. We can be playing Zoonotic and Quake and Super Ducks Card

on accelerated wheel and XOR desktops with KDE and GNOME. And yeah, it all works. I think Alyssa was working on getting Firefox Web Render to go smoothly. I think it's already there with OpenGL ES3. So yeah, it's getting to the point where, you know, this is something you want to use as a desktop driver for GPU acceleration. Is there a release out of RC Linux that has this ready, or is that still sort

of in the works? Um, so we want to release before the end of the year, uh, because we think it's pretty much there. Um, so right now we're kind of just tying up loose ends and, uh, yeah, hopefully, uh, it'll be out soon. I honestly think it's like, it's really impressive how far along this has come along. Like, as I've said multiple times, I have no idea what's going on with reverse engineering a GPU driver. Writing a GPU driver with documentation, I wouldn't have any idea what I'm doing. I'm sure many other people

are sort of in this state as well. Seeing how far it's come along, going from, you know, you have the hardware here, the hardware has no documentation, to getting where you're at now. I think anyone who's seen that has to be at least a little bit impressed.

I don't know what sort of feedback you've had about it. Like, yeah, what is the feedback you've sort of received about where everything's at right now? I don't know. I mean, a lot of people are saying that that it works uh like just the other day i got someone saying that it works better than mac os for one game which is pretty hilarious um to be clear that's not the case for most uh workloads right now for a bunch of reasons um like there's there's some stuff that is like 10 times slower than on mac os you still don't know why uh but

you know where there's a lot of low-hanging fruit for performance to be figured out so right now benchmarks don't really mean anything other than it's getting better um but uh but yeah i mean like if you just run a desktop it's it's just smooth you know it's it's it feels like mac os that's really really cool we have seen the um the geomark 2 benchmark you posted where i think it it puts it in line with like an RX 550, something like that. Which I'm sure you can get more out of that chip. But just the fact that it's where it's at now, I think is just really cool. Yeah, like it's kind of funny because I always get people talking about benchmarks. Like is it faster on Mac OS? How much is it on macOS?

That stuff doesn't really matter right now But what matters is that we're improving all of these things about the whole stack So the fact is Right now it's at a point where It completely changes the desktop experience Versus not having a driver Because not having a driver It's actually surprisingly smooth with the CPU.

But, you know, as soon as you do something like use the full screen window on one of the machines with the higher resolution screens, it's really obvious that it's dropping frames. And then with the GPU, it's just like, no, you can do 4K at 60. It just works. It's just smooth.

And that makes a huge difference. And also tear-free video and all that. Well, one thing I did want to ask about was specifically Weyland. Because I know you've said that X11 can be done. But the focus has clearly been on getting the Weyland version of GNOME, the Weyland version of KDE, things like that working. Why is Weyland... I know there's definitely this idea that X11 is like,

you know, this giant spaghetti code base, but from your perspective of working with the... working with getting these desktops working, working with the drivers, why is it that Wayland is the focus right now? So X11 does work, by the way. Like, right now you can run an X11 desktop, and it works fine. You get tearing, which is expected with the way. Like right now, you can run an X11 desktop, and it works fine.

You get tearing, which is expected with the mod setting driver right now, but that has nothing to do with our part. There's a patch set for the mod setting driver. There's a PR that was just opened to add tear-free to it. So it's kind of a hack for Xorg, but it makes it tear-free with some limitations. So this is less about the GPU and more about how Wayland and XOR handle the entire concept of getting frames from apps into a composited desktop

out into a display. Not some other GPU, but the display device itself. Because XOR is basically this ancient, it's based on this ancient model where, you know, you have a framebuffer and the display is just scanning out of the framebuffer and you draw into the framebuffer directly, right? And that's where you get tearing. Or you like try to copy a framebuffer in just in time when the frame is changing so it doesn't tear. But you know, like there's always this race, right? And you're trying to track the frame as it changes on the display device. So your XOR expects to know, like get an interrupt on every frame, like

the 60 FPS or something like that. So the M1 display controller doesn't have that. The M1 display controller is based on a model where you give it a frame and it will switch to that frame instantly on the next frame on the display and tell you that it finished. And so that's exactly the model that Wayland uses. And it's the model that macOS uses and macOS kind of pioneered this and Wayland was kind of the, I think, I don't know if it was explicitly the response to that, but it was definitely the answer to the, okay, XORX model doesn't really fit how

we want to do composited desktops these days. We're like, the idea is that every frame should be perfect, right? So yeah, like Wayland naturally maps and works perfectly on the display controller on one devices. XR by default doesn't even use the page flipping, so it just writes to the frame before you get tearing. With the tear-free option, it does use that, but there's still one option that Vsync, as in the frame limit doesn't work because there's no wait for the next frame operation on the display controller.

You only get that when you actually change the display, but Xorg kind of wants that separately. So if you run, like, a game VSYNC'd on Xorg, even though it won't tear, it also won't limit the FPS to the proper display FPS. Okay. So VSYNC basically doesn't work on Xorg right now. Right, right. And I'm not sure if we can fix that. It's kind of an imitation of how the whole stack works. So Wayland doesn't have any XR right now. Right, right. And I'm not sure if we can fix that. It's kind of an imitation

of how the whole stack works. So A-Lang doesn't have any issues like that. Well, I'm not too surprised that X- like, things have changed with the way that graphics and display tech is handled, considering that X11 was... I think the spec was originally defined in... 82?

72? It, like, a very long time ago. Yeah. I can't remember the exact uh yeah i've done a full video on x11 nonsense but a very long time ago i'm not not surprised that things have changed a little bit since then yeah it's uh i mean we it feels like we've added a lot of patches on top and you know like workarounds and bypasses for the old stuff to make things like fullscreen games work properly. But at the end of the day, the model is you still have the old stuff under the hood, right? So you can't get rid of it. And that's kind of the problem that XOR has, right?

That you can't really get rid of the craft. It's always going to be there. So Wayland just says, takes that, puts it in a box, calls next wayland and say okay yeah here's your uh here's your xorg and by the way v-sync works fine on next wayland because that's presenting through wayland so that's no problem yeah yeah um yeah so i was gonna say something that completely slipped my mind um i well actually you know wayland um i know there's a lot of people that will definitely complain about sort of the state that the general Wayland experience kind of is right now, because there are, you know, there are things that are not supported in the Wayland desktop available

right now that have been available in X11 for a long time, and it doesn't surprise me that there is this sort of I guess pushback against the use of Weyland on like a general experience but from What I've seen is it's definitely getting a lot better now for most people most of the time Weyland's in basically a good enough state that you could reasonably use it. Now, when Fedora and stuff swapped a couple of years, like way too long ago, that's another story entirely.

But I think right now, especially with the recent changes where you don't necessarily have to have something B-frame perfect, where there's the new protocol to do that, and then the new protocol to handle that, and then the new protocol to handle fractional scaling, like, passing fractional scaling data properly. I

wouldn't be surprised within, like, a couple of years if Wayland is basically at the state where, I don't know, a good, like, 99% of people on Linux are perfectly happy using it. There's always going to be those edge cases because, you know, it's not a one-to-one drop-in for what X11 is. But, yeah, I don't know where I was going with this. I'm just talking about

Wayland now. I tend to do this. Yeah, I don't know where I was going with that, actually. Yeah, yeah i don't know where i was going with that actually um yeah uh unless you have something to add to that you will just move to somewhere else where like i think wayland and x are right now kind of at that point where they both have a similar amount of problems yeah so it's actually reasonable for most people to uh to try out both both and see what they like I think at the beginning of the year we were still seeing some issues with Wayland Like the fractional scaling stuff I remember I tried it on Asahi Linux

And the first thing I did was open up Firefox and it was blurry And I was like, wait So that was like xWayland on KDE And it was trying to do that So KDE fixed that with one stuff and then with, like, asking X to do native fractional rendering and now we have the native weight and rendering, which I think KDE and Qt have always supported anyway,

but, like, in their own way. Because Qt always had fractional rendering. I believe so, yes. Yeah. But probably not standardized in a way that would work with other compositors or something like that. Yeah, yeah. Well, there was no like

general standardized protocol yet for it so KD was sort of like doing their own thing and then there's Gnome where it's like I know fractional scaling worked but like it was not there was something around there then there's Gnome where it's like we could implement fractional scaling but then there's GTK which doesn't support it

in the first place so I know that a couple of people very involved in the GTK, which doesn't support it in the first place, so... I know that a couple of people very involved in the GTK project have said it's probably not going to support fractional scaling until GTK 5. So, it's going to be a while.

And then you've got to wait a couple of years after that for the desktop to actually be in GTK 5. Yeah. You know the funny thing is that apple just cheats and doesn't do any of this yeah how does that yeah um they only do 1x and 2x and then they do full screen uh like at the display controller level they just do virtual resolutions that are not the real physical resolution and then just scale and that's. So you're under your desktop at a different resolution than your physical resolution to handle

fractional scaling. And you use more GPU power and it looks a little bit softer but macOS has always had this sort of, you know, we don't do font hinting, we do smooth vector graphics so we don't care about pixel perfect you know, like their entire UI

is designed not to expect pixel perfect graphics. There's always been kind of smooth and dirty stuff that doesn't really matter if you scale it. And yeah, I guess they just have a good scaling algorithm. They just make a controller and they just go with that. Well, Apple also has sort of the same advantage that Android has, where a lot of stuff is just built in the one toolkit so obviously there is

the electron stuff um but actually that's fractional though android does actually do like for um like configurable size dpi all that but what i mean is you don't have like you know 10 different toolkits doing all of their own different things um well you obviously you can have that on mac os but most of... This problem falls apart when you give people a choice, but a lot of stuff is built with the main toolkit that is expected. When you go with that,

there's a lot more... You have a lot more control over what is done. Yep, yep. That's why if you stay... If you stay just in KDE, everything that's going to be using the KDE scaling is going to work nicely. The second you step outside of that...

For the most part. Yeah, okay. You know, I found a really funny race condition. I don't know whose fault this is or if it's supposed to work in some other way, but when I start KDE sometimes at 1.5 scaling, sometimes the taskbar is blurry and apparently what happens is that like it applies the directional scaling while it's starting up and so if plasma starts before that setting gets changed it starts at

the wrong scaling and because it doesn't support changing scaling for existing apps on the fly what akiria seems to do on wayland is that they just scale the output so you get blurry surfaces until you restart the app, which is better than nothing. So I agree with that. I think that's a good idea. But then you better set the fractional scaling globally before starting Plasma. So something is backwards in the startup sequence there. So yeah, those kinds of things obviously need to be fixed.

No software's going to be perfect. Definitely not. But, you know. No, it could be worse. There could just be no fractional scaling in the toolkit. Like, it's a GTK. Yeah.

I've heard multiple people tell me that it wouldn't be that bad to refactor it. Then I've heard other people saying that it would be an absolute nightmare. I have no idea who's actually telling the truth. All I know is it's probably not going to happen. So, I don't know. Virtual scanning is pretty hard for things that assume integer pixels. Because, like, there's, especially, I don't know if even websites have this issue.

Because, like, there's especially, I don't know if even websites have this issue. Like, I use 1.5x and sometimes I scroll Twitter and the DM window starts scrolling on its own. And I am convinced that's a fractional scaling issue where they, like, set the position of the window and the scroll and then read it back and set it again. And do the rounding issues. It just crawls. It's so annoying. So you get things like that right like where like you're rounding down to pixels and back and converting and that just makes everything such a pain yeah well one of the main folks that people talk about with fractional

scaling is like blurriness but i'm i'm yeah it doesn't surprise me there are other things that are a mess like blurriness is sort of the least of your issues because a lot of the elements that you do have aren't that big of a deal. Like, text scales properly. Any UI elements you have that are drawn in code, so, you know, like your buttons that you're probably drawing code, that will scale fine.

It's sort of only the rasterized images that are a problem. But I'm not surprised that other issues tend to crop up when you don't fully test your fraction of scaling implementation. Yeah, I think the sort of code issues like that are kind of the worst part, because the UI elements, I mean, if you have 2x elements and you scale them down to 1.5, it's fine.

I mean, that looks fine. It's only people who are like, I must have exactly square pixels and, like, you know, all my designs should be super sharp and mapped one-to-one to my display. You know what? That stops mattering with high DPI displays because we can't see the pixels anymore. So that only really matters for low DPI. And we're kind of moving away from that. Yeah, from...

But... Sorry, go on okay but um but yeah it's it's more about like retrofitting fractional scaling into a system that doesn't already um you know natively base itself on floating point coordinates and things like that can get really messy yeah it doesn't surprise me that it's a uh it's a mess um one thing you made sort of you you mentioned uh i think you did a couple of streams on it was moving up to another version of opengl so you're working on your opengl 3 stuff um for anyone who has and me as well who has no idea like why going up to a new version of OpenGL matters, what's the big deal with that?

It's just more features. Applications, for example, Firefox will not enable web render on OpenGL 2, so it will work, but it will use software rendering for the actual website. Though you do get WebGL. So it's things like that, right? So the apps can require certain versions or have more features in certain versions. Super Dux card has, I think, three or four renders, depending on which version you have. And as you go up, it gets fancier and the graphics get fancier.

Yep. So it's things like that, right? Mm-hmm. And where are you at with the OpenGL 3 right now? Because last time I saw you posted about it, it was like 90% test coverage or something. I think OpenGL ES 3, which is different from MobileGL 3, is at like 96%, 97% or something like that. And I think most of what was missing was,

I think it was transform feedback and some issues with multi-sampling and stuff like that. Yeah. But we have most of it. So, for example, we have multiple render targets, which are required for the modern versions of NFT2D to work properly. So that's good. And, yeah, a lot of apps are really starting to come together on OpenGL ES3. That's different to OpenGL 3.

It also OpenGL 2, which we also don't fully support yet. And that's just about like really old clunky legacy GL features. Because like OpenGL ES is like the cleaned up version of OpenGL. And then OpenGL is like XOR. It's like so old. And it has all kinds of crazy things that make no sense on modern GPUs. But you have to emulate them anyway. all kinds of crazy things that make no sense on modern GPUs, but you have to emulate them anyway. And the thing is, there are no official OpenGL 2 or 3 conformance tests. Right. There are only official OpenGL ES 2 and 3 conformance tests, and then there's like an OpenGL 4 conformance test, like 4.something. So we can't do that yet. So we've been focusing on ES 2

and 3 because that's both the most important part and that's we have tests for but uh there is a separate open to your test um called piglet and so alissa has been fixing stuff there and i reviewed some of the merge requests um to fix that and it's really just random stuff it's it's it's like you know telling mesa to emulate uh some things that should be emulated and fixing you know uh minor bugs in places and just a lot of very random things um and then i think there's a couple um more major uh things actually need development for open gl but it's mostly yeah just just a lot of really old crusty features that we need to either support or just decide nobody cares about and just not support um which we're also doing

because there's literally stuff in there that's just like, why would anyone ever do this, right? There's at least one code path in Mesa that just has a warning message printed that is like, the app is doing this thing, we don't think anyone cares about this thing, it might render wrong. Because there's no point, right?

Like, in spending time on stuff, unless we find an app that actually cares about that. So OpenGL has a lot of that. So we're not trying to do 100% OpenGL compliance because it's a waste of time. It makes more sense to work on newer versions and implement the stuff that apps actually use. And so that's why the ES conformances are more important initially. And honestly, by the time we get to Open open gl4 that might be the time to switch

the vulcan with zinc on top um so we'll see how that goes um but right now you know what's useful to people is open gl es2 and 3 and you know the open gl parts that work easily on top of that and so that's what we're doing we i'm sure if later down the line you're like hey you know what we actually do want 100 compliance why not we're bored that can come later if you really wanted to it's sort of i would imagine it's a lot more important just to make sure that things that matter mad like are working and anything else like optimization for example like this is a whole different thing but optimization can be dealt with later as long as

things are working you can deal with the rest of it at some point else there isn't even like 100% OpenGL compliance like as far as I understand it basically NVIDIA is sort of the standard for how OpenGL works because it's so complicated

and it has so much craft that there's actually no consistent specification that tells you how all of those crazy features interact. So, you know, you have all these separate things and then if you combine enough of them, the spec doesn't actually tell you what's supposed to happen. So there is no 100% OpenGL compliance other than copying what NVIDIA does.

Right. But it doesn't really matter because unless an app is actually doing any of this, nobody cares. Yeah, yeah. So what matters is supporting OpenGL ES and getting compliance there because that's a good test suite and it's stuff that people actually use. And then making sure that the things that people actually do in OpenGL 2 and 3 actually work. making sure that the things that people actually do in OpenGL 2 and 3 actually work. And then, you know, focusing, like, it's more important to focus on adding new GL extensions and incrementing the version support to add things that people actually use in newer versions, as opposed to, like, having all this legacy stuff that nobody uses, right?

Yep, yep. One thing you mentioned in there was the Vulkan drivers. That's not something that's a thing right now, but what is that Zinc thing you mentioned? I've heard other people mention Zinc in the context of Vulcan. I never bothered to probably look into it, though. So Zinc is just OpenGL on Vulcan. So it's a translation layer,

just like all the other Vulcan and Metal-type stuff that there is. Right. But it's built into Mesa. And so the idea is that when you get to that level of underlying feature support, there is one implementation of OpenGL that handles all this craziness once. And then if you have a Vulkan driver for your GPU, you can just run it on that and not have to worry about all the corner cases. So that's the idea there.

And it supports like modern OpenGL 4 or something like that. So when we have a Vulkan driver that can handle that, then it might make sense to effectively deprecate the current OpenGL 2.3 driver and just switch the whole sale to Zinc on Vulkan. And it will still use, by the way, at least the shader compiler that Alyssa wrote and also all the

knowledge we've gotten writing the OpenGL driver. But it's entirely possible that, yeah, in the future, what we're doing now will be deprecated. But the whole point of this is the learning experience of understanding how the hardware works and how to implement all these things correctly on the hardware. Yeah. And also the shader stuff, which is going to be shared anyway, which is a big part of it. So it's not wasted work.

A lot of people ask that, it's not wasted work. A lot of people ask that. It's like, why are you doing OpenGL? We can just do Vulkan and then do OpenGL on top of Zinc. Well, because doing Vulkan to the point where Zinc is going to work is going to take a much longer time than OpenGL, and it's a lot less incremental than OpenGL. Right. So it makes a lot more sense to figure all this out with OpenGL

and go, you know go step by step. And then in the end, if we have to throw it away, that's fine because we got there through a path that makes sense and that we could follow to get to that destination. While if you say, make a Vulcan driver now from the beginning, it's like, where do you even start? Too much stuff has to work at the same time for Vulkan to work. Right, you wouldn't build a bullet train before building a steam train. You want to have that fundamental, we understand how a train is supposed to function,

then we can build the more complex stuff on top of that. Yeah, exactly. And since Vulkan is going to take longer, this means that today, hopefully this month, we can release something that people can actually use and get, you know, an accelerated desktop. And so that's really important, right? And so, yeah, that's the answer to why not Vulkan. But Ella is working on a Vulkan driver.

And that's already, you know, she's working on Mesa. So it's on the same code base as what we're working on. And I have fans for the kernel site to support all the features that Vulkan needs eventually. And so the idea is that my kernel driver is not going to be upstreamed until we've settled on an interface that should support all the future Vulkan features. So that's one reason that people ask me, it's like, when is your driver going to be upstreamed? Linux has a policy that the interface between user space and kernel space can never be broken

in terms of backwards compatibility. And so that makes it very hard to fix that graphics driver design. But making the good design from the get-go is also hard. So that's also a different road that I'm walking, right? So the UAPI that I have now started off as a prototype, and I've been cleaning it up into something good enough to ship to people. It has a version check so if you run a driver that is mismatched with the kernel, it'll just fail. Same as like with Nvidia and stuff like that. Which is not upstreamable, but it's what we want for development because it means we can fix things, right? So whenever this gets released, we have to tell people, make sure you upgrade all the

packages at the same time, because if you upgrade Mesa and don't upgrade the kernel, it's going to stop working. And also when you upgrade, probably new apps will stop launching until you reboot. That's just how it is. That's part of being on the bleeding edge of a non-upstream kernel driver. But this time, I have a version check, so it's not going to corrupt or something. It'll just stop working. That's the work-in-progress

state, right? And then I'm going to be modifying that UAPI to support the features that Vulkan needs. What is a UAPI? Yeah, UAPI. It's the API between the user space and the current space. Ah, yeah, okay. Because you were just throwing that term out there without saying what the U was.

Sorry, sorry. So yeah, it's called a UAPI. And so that has to be stable when it's upstream. So I'm going to be working on improving that to add things like multiple command queues and things like there's a thing called sync objects, which is where you can set certain things to be dependencies for certain other things.

So you can queue work on the GPU and say, you know, when, but even across devices, you can say, you know, okay, when a frame comes in from a webcam, then run the GPU render job that'll render this Discord window, which will then enable the compositor to render the whole desktop, which will then enable the display controller to present the frame. And in theory, if all this magically worked as it was supposed to, you could have the CPU asleep the whole time. In practice, that doesn't work unless you have a lot of very overcomplicated hardware support for all these things.

But at least you can have the user space apps asleep and have the kernel do all this chaining and dependency management. And so that's the idea that the API is going to support these sync objects that are called that let you chain things like that and have dependencies. And the kernel will manage triggering the GPU driver when the webcam driver has a frame and that kind of thing. And so that has to be added for Vulkan because Vulkan needs these for...

It also exposes this for the user to use, like in games, like render this, render this, and once both of those internal frames are ready, then combine them with a post-processor shader, all that kind of stuff. The games these days need all the same stuff. And also compute is another thing we're going to have to add, both because a lot of people want it and because we're going to need compute

to emulate some GL features that the hardware doesn't support. And then another cool thing is that, I don't know if you heard about this, Carl has been working on Rusticle. It's already released in Mesa. Yes, yeah. Yeah, the Rust new OpenCL stack in Mesa. So that should also pretty much...

Very proud of that. Yeah, that should also pretty much just work on our driver once we have the basic compute support because it's all based on the same shared shader infrastructure that Mesa already uses. So I'm also really looking forward to that. And yeah, that also needs new API changes to support creating compute command queues in the kernel and submitting jobs and all that. And then also memory management, Vulkan needs more advanced memory management, which is going to need a bigger revamp of how the kernel

does all of that. So it's all these changes that we're planning because we need them for future stuff like Vulkan. And they don't necessarily have to wait for the Vulkan driver to be to that point, but I do want to at least have the drivers, you know, using the features so we can prove that they work. And yeah, so that's all the work that's going to be happening before upstreaming

because that's sort of the process that you have to follow. Yeah. Well, you sort of gave like a thousand different directions to go there um sorry no it's fine the more that you talk the less that i have to talk um um so you've said a lot in there about vulcan drivers one day vulcan drivers eventually if you you probably don't want to even remotely give a time frame because then someone's going to hold you to that

if you had to imagine, let's imagine a theoretical way you can actually work this out, if you had to imagine when Vulcan drivers could possibly be a thing how long do you see that being out for the project? I have no idea

how to answer that question. But what I will say is that my hope is that by around the middle of next year, we have at least enough of a proof of concept to prove that these UAPI features work and function properly. So not necessarily a Vulkan driver that supports anywhere near compliance,

but at least that runs can run demos that use these features and prove that all of this works as intended so that we can stabilize the UAPI and submit it upstream. Yep. That's my hope. I have no idea if it's going to happen. I think it's plausible though. You know, just need to work together to make sure, like with Ella, to make sure that we

get all this backend stuff hooked up. And I i mean she's already running vk cubes i mean vulcan you know it's at the cube stage at least so it's not like there's no vulcan driver um but right of course yeah yeah i mean we have the cube in vulcan um so it's not zero um but um obviously there's a lot of work to be done to support all the fancy stuff so I hope we can at least have the proof of concept level for the UAPI features by around the middle of next year, just so that it doesn't block starting to upstream the driver. But yeah, as far as Vulkan that you would actually use with games and with Zinc and all that, and they would work at least as well as the current OpenGL does. I really have no idea how to answer that question. It could be sooner, it could be later. It also depends on, you know, on how much time people have to work on it and all that. And the thing about this kind of project is that it depends a lot on, you know, people, like, volunteering their time and, you know and people becoming interested.

Ella just showed up and started working on Vulcan, right? It's not like we asked her. So things like that. So I can't answer that question, unfortunately, but that's kind of my hope. Well, you mean your games are more complex than a blank cube in an empty space? One thing I did want to ask you about,

because you get a lot of attention you know being a vtuber doing all this doing your what 10 12 hour streams um obviously you get a lot of attention but you mentioned there is a lot of other people involved in doing this as well obviously there is also a list and elissa does get a lot of attention for a lot of the other stuff that she's involved with but who else do you think sort of deserves more attention for what they're doing that no one really hears that much about? I think there's a lot of people in the Asahi team that, like, others don't hear much about. Like, Jan Grunow talked Alyssa's DCP driver, which is the display controller driver,

which has nothing to do with the GPU. And that's what we need. That's required for the GPU to actually run a desktop. So we can't run a desktop with a dumb display driver we've been using until now. So Jen has been working on that. I helped out a little bit recently, mostly by deleting code. And it had emulation for this vSync stuff for XOR, and it was broken. It was actually breaking it. And I took it off, and it fixed it. But then there's no vSync, which is, I think, what you're actually supposed to do.

But, yeah, there's, you know, he's been working on that. Sven's been working on USB 3.0, on a lot of other random stuff. Also, DisplayPort output, which is not shipping yet, but it's in the works. And then it says that Ella's working on Vulkan, and she almost never gets mentioned, but that's uh it's in the works um and this is that ella's working on vulcan and she almost never gets mentioned but that's going to be a very big thing pretty soon um and um there's uh i mean of course there's mark and that kind of you know keeps everyone together um and then um who else um i'm sure you just keep going all day with this. Sorry, yeah, I mean, there's like, working on the speakers with Povic,

doing the whole audio system stuff. Um, and then someone, what was the name? There was, there's someone doing, I think it was Alan or something like that, working on the, um, neural engine, which is really surprising, because I didn't know if anyone was going to sign up for that. Um, no, like no one in the main team kind of had had an interest in that and then someone just showed up and said,

I'm going to figure out the Neural Engine, which is really cool. Also, apparently it can be used for image processing too. So I wonder if there's more than just neural network things that it can do. So that can be kind of fun. And yeah, the thing is, anyone is like welcome to say i want to do this right um and and so there's that there's a whole team of people like that um it's just that everyone wants to hear about the dbo tribe well yeah the point i was getting at was the work that you're doing is like it's it's impressive to see it's it's easy for someone who has no idea what it

what goes into this project to see hey look kde works now hey look i can play zanotic at ultra settings hey look you know um i don't know let's say one day you have opencl working and then i don't know some compute stuff's happening what you're doing looks really impressive but there are less visually impressive things that are going into this project as well that are just as important to getting a a good experience with this hardware yeah like um someone uh recently uh uh like submitted the keyboard backlight driver and it's like one of the simplest drivers right um it's also their first time uh reverse engineering

uh that kind of stuff and writing i honestly had no idea the current macs had backlights so many people were asking for that right and you think it's just the keyboard backlight yeah but people really appreciate like having a keyboard lit up in the dark right yeah it's even stuff like that yeah that's totally fair um so one thing I wanted to ask is sort of... I don't know how much attention you pay about this, but what sort of reaction have you gotten from doing this work while being a VTuber?

Because there is a lot of people, I've certainly noticed, at least in my comment section, who have no understanding at all what this is. Someone asked me if Luna was an AI the other day. So, yeah. What sort of reaction have you gotten from, yeah, just from generally generally obviously your streams are very positive like i'm sure you've had the occasional troll shop from time to time but um yeah just how's that sort of gone for you so the actual streams um i actually don't really get that many trolls um like

at the beginning i got a few i don't think i've had to ban more than like four people or something like that wow not counting not counting the porn bots oh yeah yeah yeah yeah um i mean that's just youtube but um yeah i mean pretty much uh yeah people are very very nice and i really really appreciate that um and also very helpful like under my streams especially when i started working on Rust. I should mention that because, like, until the Rust part, I was working with things that I was familiar with, Python and C. I'm a Rust newbie, or I was. And then you became a Rust show.

And, like, I got so much help from the Rust community, both offline and in this and the like Zulip and like directly on my streams from viewers you know developing this whole like object model for the GPU to make it all work reliably and have the memory safety and all that I had no idea how to like architect that in Rust I thought it might be possible but like I couldn't have done it myself thought it might be possible um but like i couldn't have done it myself not with my trust knowledge at the time and so you know that was really really cool um that i could get all this help from so many people and even today you know people are like oh you can do this in rust oh thank you yeah um but um yeah as far as the comments um i get a lot of really cool comments uh directly like over dms on twitter on my youtube channel um there's a lot of really cool comments directly, like over DMs, on Twitter, on my YouTube channel. There's a lot of people that, from the very beginning, you know, send me messages like, you know, thank you so much for doing this.

You're an inspiration to me and you inspired me to go into current programming or computer science or something like that. And that just makes me really, really happy. So I really, really appreciate that kind of stuff. And I'm really happy to be getting those kinds of comments. And then I guess there's Hacker News. Yeah. Yeah. Yeah.

Yeah. So... Hacker News is a fun place. I think it's kind of interesting that it seems to be very split between people that really don't get it And people that think it's really cool And sort of they always fight it out in the comments every single time And I don't know, I mean

Another thing is that a lot of people seem to like Confident confidently answer questions, like wrongly confidently answer things. Because the whole VTubing thing is very different for different people. We each have our own reasons to do it. And it's just a form of self-expression. I think one comment i like this uh i don't remember if this was a hacker news or uh reddit but someone mentioned that

hacker news is just orange reddit depends on which subreddit the rest of reddit is usually pretty nice okay yeah that's fair um go to r slash linux though when you change your mind but anyway i think that they said it's like this isn't really any different from someone on a forum with an anime avatar it's just you know the equivalent in uh in 2022 and I think that's a good way to put it right that it's just the image you choose to represent yourself online and the name you choose to represent yourself online and um and the way you choose to express yourself and you, you know, the reasons for that is, you know, might be different for each person, but this isn't new, right?

Like people have been adopting, you know, different virtual appearances in different ways since the dawn of the internet pretty much, at least since we had like images. So a lot of people like seem to have this sort of visceral reaction that this is weird and new and i think that's a good place to to think about it for us it's like no we've had this all the time you know since the dawn of the of forums and and you know all that stuff uh in the late 90s and 2000s it's just that now it's animated and on YouTube and we make videos about it. That's it.

It's not that different. The only difference is that we've got this tech to do the moving 2D images. But it'd be no different. What you do on your streams would be absolutely no different if you just did the exact

same thing without a face cam. Yeah. The actual work would be the same. thing without a face cam like yeah i mean the actual work would be the same i think a lot of people really appreciate the uh like having the a view of the person doing things um because it makes it more um you know engaging and friendlier yeah and so i credit a lot of that to you know those nice messages i get so i'm really happy doing this um and i think a lot of people also don't understand like why it makes that kind of difference because they're just like you could just not show your face yeah but it's not the same there is a lot of there is a we could

go into a lot of the um the research to be done over the years over, um, why having some sort of, some sort of avatar, some sort of face cam, something there is just better for making content. Like this is something that is well established at this point. There's no point even having the discussion about whether having just no face cam is better. It's not. Like there's a reason why,

there's a reason why every single streamer now has a face cam or does VTubing. It makes for better content. Yeah, and it's not just like, because people think about it from the, oh, you get more engagement and therefore more money or more ad views or whatever. People are more interested in the stuff I do because I'm doing it like this, right? It's getting people interested in computer science. It's getting people interested in computer science. It's getting people interested in computer science and programming and kernel programming that wouldn't otherwise and that wouldn't have gotten interested if I were just to face

the streamer because it wouldn't have been as engaging, right? So the engagement stuff isn't just about, you know, the numbers under the video. It's actually getting people interested in the subject. And so that's why I keep doing it, right? Like, I don't actually care about engagement. I don't even know how I ended up with like 17,000 Twitter followers. I was not expecting that. I was not expecting that. And like, that's not my goal. But the only thing I can say is that like, as that number goes up, so does the number of people leaving comments like that and saying that I helped them get interested in this kind of stuff.

If that's what this is resulting to, then I'm going to keep doing it because that's what I want. Yeah, just working on reverse engineering GPU drives is definitely weird enough to get a bit of attention, but it also being done by a cat girl is going to get a bit of attention, but it also being done by a cat girl is going to get a bit more attention. No. No.

Yeah, I mean, the cat ear is going to come later, but... Well, yeah, that's a new thing. I did notice that you rebranded your Twitter. Actually, wait, that reminds me of a hilarious... Oh, that reminds me of a hilarious thread I saw on Twitter. Hilarious thread I saw on Twitter. A hilarious thread I saw on Twitter. So, I think it was about the recent stuff about KD running and things like that. Someone said, they're like, oh, it's so impressive how much work RCLina has done on this.

So, no, they said, they used the new name you got on Twitter they said Lena and someone someone corrected it under it with like asterisk Lena and there was like this giant thread just discussing like just the merit of VTubing I mean that's part of the fun right

and it's also you know sort of it helps break this idea that you know if you are doing this kind of work you need to do it in a certain way and be a certain kind of person

because you don't because that has nothing to do with the work right so that's also why I do this because it's like well i'm doing this because i enjoy doing it like this and if you want to do something like that too you can do it too right um i've seen comments like well no one's going to take you seriously you know in technical circles if you are a vtuber um you know and doing it this way and whatever

well um i mean i gave a presentation at the exorc developers conference and the people there took me seriously yeah um and i've been on video calls with like mesa developers didn't you have the witch hat on as well during that one was that a different that was that was the first one yeah that was uh that was the um for the talk. I rigged it the day before. I'm like a member of Sony Stream and spent like nine hours on it or something. Including fixing TTT bugs. And yeah, that was the sort of, I mean, it's a joke, but I kept it on the whole month.

Because, you know, let's Alyssa have the whole sorcery thing going on with the GPU reverse engineering, so we just made the talk theme like that, and I got a hat! It's pretty cute! Oh, it moved! I actually haven't watched the talk yet. I didn't realize the thing moved. Oh, yeah, yeah. I added the physics.

Like, the ribbon on the side there has like six different chained pendulum nodes, it's pretty cool I was just paying attention I wrote the physics engine for InnoCGD Oh I was just paying attention to the bat I didn't even know if the ribbon was moving Yeah, the bat like jumps around a little over the place

That's awesome The only reason I little over the place? That's awesome. The only reason I asked about the reception you've had, I don't know, maybe they just like to bother me about, whenever I do a video about RC Linux, maybe they just like to bother me

about, like, who you are. Because, I'm not going to name the person, but i have had a certain disgruntled gpu developer on a certain project uh who was really bothered by the fact that you'd done all this work and they couldn't find any sort of discernible previous work that you had done and they're like how is how is this a thing that exists? This person has no history. And somehow trying to use that to

discredit the work that you've already got? Like, I don't understand. I mean, I don't know, like, I hadn't done any DP work before this. It's, you know, just kind of picked it up as I went. I mean, at least it helped me out a lot. I mean, obviously, like, I mean, I have her on single and everything. I mean, we talk a lot.

We have calls and stuff. So a lot of people were confused when I started working on this. I'm like, what? Isn't Alisa working on that? We're working together on the same stuff. I took it on my, like, intro stream that, like, you know, it was all reading me and not her. I mean, obviously, that was a joke. But we've been working together since the beginning. And yeah, like, but I mean, as far as like, personally, I didn't have any GPU experience before this. I just kind of picked it up as I went. And the thing is, like, it doesn't really matter, right? Yeah, ultimately, what really matters is the work that you've got now.

Yeah. And I was going to say something else. Oh, yeah, I remember another comment on Reddit. Someone said that I must be like a secret Apple employee working undercover or something like that. It's like, I wish. Yeah, no, the things people come up with are kind of funny sometimes. Mm-hmm. uh yeah no the things people come up with are kind of funny sometimes yeah it it it's kind of like people just want to know who is behind they don't just want to

accept that hey this is how you're portraying yourself online they are they're just obsessed with finding out who like who is behind the cat girl. We must find out. We cannot just let the work exist as it is. We must find out more. Yeah, I mean, I don't even know how to answer that question because this is who I am. This is how people know me and Maisa and the Colonel. It's not like I'm putting on a show for YouTube

and then there's another secret Mesa developer actually doing the work, right? I was on a video call with Jason Extra and then Melissa. The same as I'm doing right now, right? It's not acting for YouTube. It's just who I am, right? If you act for hundreds of hours while developing gpu drivers that's actually really impressive and you should

probably not be a developer you should probably just go into acting i'm actually terrible at acting like i that's by the way that like the the cat ears thing was more of like a you know joke on purpose and i'm really bad at like consistently like playing that role on my stream. That's how you can know that I'm not an actor, right? Like I'll say the nya at the end and stuff. And like the first day I kind of got into it. And then like, you know, a week later, like completely forgot. I don't think there's any V vtuber out there that keeps their character

up like i don't even the ones who are dedicated their character eventually it slips you know what this is too hard i'm just gonna go back to talking normally all right that's the thing right i mean like i i'm not it's it's not like i'm playing a party or anything it's just like i'm just doing party or anything. It's just like, I'm just doing my development work. I just chose to do it like this because I enjoy it. And because, I mean, the feedback loop of what I'm getting from people is really positive. And I'm just going to keep doing it. But yeah, I mean, there's no like deception or big secret or whatever.

It's just me. It's just you and It's just you and the work that you're doing. Pretty much. And that's the other thing is that people say, how is it possible that you do this? Watch my streams, right? It's all on there. Yeah, yeah, yeah.

You can see how I started from nothing and got to where I am. I mean, it might take a while to watch them all, but it is documented. A while's one way to put it. Wait, if I go to Social Blade, it should tell me how many streams you have and then I can

work out how many hours you've streamed because it's too many. Actually, one of your streams is more than I stream in a week. Social Blade. Let me see. 58 uploads. Oh my god, there's 58 uploads.

Let's just say at least 300-400 hours. Yeah, probably. Might be 500 depending on when i really got into the 10 plus hour streams yeah that's that like just just the streams alone is kind of crazy to me like how do you how do you like manage your stream i'm not actually sat down and watched entire one of your streams i know i'm sorry about that but how do you you know handle your streams like do you take a break at

some point during it do you get some food do you are you a crazy person do you just go 12 hours straight how do you do it no and so i i i take um brief breaks um i do eat on stream i people seem to enjoy it when i do like eat the invisible food thing so i do that um i so i usually have a snack um towards the maybe two-thirds uh portion of the stream and i i have lunch before my streams i always start at 3 p.m um and i do have water of course and uh you know tea and that kind of stuff um and i take very short breaks sometimes you know to go to the toilet or just take a very quick walk or just if i need to like refill my water and all that kind of stuff um but um i don't take any uh long breaks except that one time there was like the epic cube on rust on the next stream

which is like i think it was like 15 or even 19 hours in the end and that got broken first when youtube crashed because youtube like streaming crashed that day first when YouTube crashed because YouTube streaming crashed that day. And when it crashed, I told people, okay, I'm going to go to a convenience store, buy dinner. I'm going to eat dinner and see if it's fixed. And then I'll redirect or something and continue. And I split the stream up. That was the only time I've taken a proper dinner break. But that's fine because I have enough snacks to last.

And then I usually have a bit more dinner after the uh the stream and it's fine so that's usually what i do is i have lunch before the stream i have a snack about two thirds in and then i have you know a lighter dinner after the stream and that's uh that's the pattern um and a lot of people ask me like how can you do this for so long so streaming actually helps me concentrate somehow i've heard other people say that as well yeah like because i can't go you know goofing off or you know starting to chat to people or yeah watching youtube videos or something like that um but also just the feedback from people in the chat also helps and depending on how concentrated i am i'm better or worse at um you know following following the chat.

But I've also tried, you know, over time to make it more engaging just by talking more about what I'm doing, which also helps me sometimes. So, you know, that's not that particularly difficult thing, but it does take some time to get used to doing that. So that's something that I try to keep up more these days. It's just kind of keep people a bit more in the loop because I know it's very hard to follow what I'm doing already if you're not already experienced with this. So anything I can do to help people find it more interesting, I will try. But that's pretty much it. I'm really just working, trying to concentrate on what I do and I and trying to talk my way through it, that's pretty much it I definitely have a bad habit of

goofing off when I'm supposed to be working it takes me a little bit longer to plan out my videos than I probably should yeah you know a couple of YouTube videos here, a couple more YouTube videos,

a couple more YouTube videos. Yeah, I'm just going to watch this one and this other one. Oh, look, another one popped up in the recommended feed. Yeah, I know the feeling, yeah. So streaming helps a lot with that. And, like, you know, some people ask me, like, do you do, like like part of your work streaming? I stream almost all of my work. I sometimes do like, you know, Git management or fixing some particular bug or some random thing on stream.

And you can probably find the, you know, the Git commits with the times to tell you that. But most of what I do, almost all the important stuff that I do is on stream. And part of it is just that, is that I concentrate on stream better. There have been at least a few, I think it was like last week on Thursday, Alyssa nerd sniped me with a problem and I spent like all day on that off stream. So I basically did a stream without streaming. That's rare. I almost never do that so like people are getting pretty much um all my work on the stream which i think is important

because there's no like secret you know like i'm i'm really preparing to get this all like um because you could do that right you could you could like pre-prep for stream and kind of know ahead of time what you're gonna find and and turn it into a show which might be um might be more engaging and might get um you know people more interested but it's also kind of lying right uh because then you are presenting that you are doing all this stuff um you know in a short time when you actually prepared for all of it that um so yeah but you get like i you know my stream prep is like 15 minutes before turn on the computer turning everything on and if i'm going to be working on a particular thing i might like clone the git repo or install the os like for the m2 i was halfway through a reinstall last minute reinstall because i kind of messed up and stuff

like that but i'm not actually doing any of the coding or reverse engineering um before the stream other than as i said, the few cases where I did a bit of work off stream, but not in preparation, just because it happened to be interesting at that point. Or because, you know, maybe something that Alyssa asked me to do, and it was like on a weekend, and I'll just do it tomorrow.

I mean, that does happen, but the vast majority of what I'm doing, I'm doing it on the stream. Yep, yep. Well, I think it's important, it sort of depends on what you're trying to achieve with the stream. Like, your stream seemed to just be, you know, as you said, you're just working. But if your goal is more to do like a, you know, like with the XDC talk, it's a demonstration of what's going on. Like, that's something that makes a lot more sense to go and prepare then.

of what's going on like that's something that makes a lot more sense to go and prepare then yeah so i did um i do i did do like a presentation and q a stream where i had an actual slides uh like the like the xcc talk um a few months ago um and so like that's different of course i mean if you're going to be talking about how something already um done works that's pretty cool and i do want to do more of that um just so people can pick up on when we're at right now and, you know, how we got here and where things are going. That's also why I just wrote that Asahi Linux article that kind of covers the, I think we already saw it, that kind of covers the whole story from April this year to now. So I do want to make more sort of produced content like that, that is like explicitly produced to explain to people what's been you know what's been going on or teach something because i think

that's pretty cool and i should find more time for that but what i meant is like you know like it's not like i am um saying that i'm reverse engineering something and then i actually already figured it out before right yeah yeah that's yeah yeah well look at this magical thing i found crazy yeah yeah no that that absolutely makes sense yeah i've not read this article actually i should actually go and do that well actually it does go over everything doesn't it so it's goes from all the way from when alissa joined from you starting on nothing, getting the triangles done. Huh, okay. Yeah, I crossed over Alyssa's, like, first year

because she already documented all of that on her blog. So people should go read that on her blog if they want to get, like, the 2021 story. So I kind of started from where, you know, when I started working on stuff this year and then including what Aly Alice has been doing this year until now. Well, you got interested...

Jump back into some of the development stuff. You got interested in Rust. And as I said earlier, I don't mean this in a mean way, but you are kind of a bit of a Rust shill at this point. You definitely like to proclaim the benefits of Rust. I like to mention Rust fairly often in my videos just because it tends to annoy people. There's a lot of people who get just very angry about Rust existing.

Like, oh just do it in C! See, just everything should be in C. Why Rust? What is the benefit of Rust for the work that you're doing? I mean, I think the easiest answer for that is the example that when I got the Linux driver rendering a cube after that crazy YouTube crashing stream, the next week on the next stream, I did one stream working on stuff. And then I had an idea like the next morning to fix a problem that had nothing to do with Rust. And then I got a full non-desktop running.

So I went from the cube to a full non-desktop in about two actual days of work. Because all the concurrency and memory management and leaks and overflows and race conditions and all those things that R would make, going from demo renders a cube to full desktop with a bunch of apps running at once you know so difficult just didn't happen with Rust and like this is a story that I just keep hearing from people it it's really it you can't really explain it until you try it because it's just that feeling that you're like, the code compiled and it works, right? You get C code to compile and then you're going to spend the next day debugging it. And with the first, it's like, it's going to take longer to get it to compile because the compiler is going to tell you how dumb you are all the time. But then once it compiles, it works.

I mean, of course there's bugs right i mean we're always um you know sort of um uh like uh i mean i'm enthusiastic about rust because i can really see the benefits it doesn't mean it's going to magically make all your code perfect obviously there's you know there's logic bugs and things that are going to be wrong and obviously i'm still debugging things in my driver um but it's mostly at this point it's mostly about my understanding of the underlying hardware and firmware and the reverse engineering and less so about like coding bugs because I messed up.

So the, it's a very different feeling from writing C in terms of like how much time you spend before the compiler and then how much time you spend debugging. And the most important thing is that when you write code in C, there's this like, this is the thing where like you kind of have to keep in your head all these rules about how everything works together in the entire program. these rules about how everything works together in the entire program. Because if any two parts of the program disagree about those rules, then that's an avenue for like, you know, something to go really wrong if that triggers, right?

And because C doesn't make you enforce those rules, doesn't give you any tools to enforce those rules, and doesn't encode or even force you to think about those rules, then you don't, right? So you end up with this vague idea in your head of how it's supposed to work, but because it's a vague idea and the computer isn't helping you enforce any of it, you are going to make mistakes, right? And so Rust kind of turned that on its head by saying, okay, you actually have to think about how this is going to work, but because you think about it from module to module, you write these rules and it's built into the Rust syntax. It's just built into the type system. It's built into how Rust works.

Then the compiler guarantees that those rules are upheld by everything else. And so when you write Rust, you think about one box and as long as that box is doing the right thing, which might include even unsafe code, which bypasses the Rust safety, but that's fine because like that unsafe code only has to work together with the safe code around it and in that one part of the program to make sure that it's not actually breaking. And then anything that uses that box doesn't have to care about any of that because the language guarantees that as long as just the unsafe parts are doing the right thing, then

nothing else is going to break in a way that's going to crash or overbuffer or something like that. And so a lot of people say, it's like, well, if you're writing Rust, if you add unsafe code, you're just throwing away everything that Rust does. No. Like, my driver has, I think I'd add, like, maybe, how many unsafe blocks? And let me just prep for unsafe real quick. Drivers, GPU, DRM, Asahi. There's about 100 unsafe blocks in my driver.

And it sounds like a lot, but, like, they're but they're all doing one thing in one part of a module. Some of them are duplicated and it's just a pattern that is the same everywhere. And so each one of those only has to care about one particular part of the code. And I actually need to write comments. Not all of them have the comments yet, but I actually need to document all of those unsafe codes before I'm streaming because that's a Rust for Linux rule and it helps other people understand why that unsafe code is actually safe. But that's the idea, right? Is that you go there and you think, okay, I can mentally prove that this is safe because A, B, and C, and those A, B, and C are in this one file that I'm looking at right now and I can prove that it's like that. And that is something that someone can review and then come to the same conclusion.

Right. And, and then, you know, you say, okay, this box is fine. Move on to the next box. And then all the interactions between those boxes, all those like, you know, exponential ways that all of that can go wrong, don't happen anymore. And that's where you get the, you know, the magic of Rust. That's why that driver went from cube to full desktop in a couple of days because that crazy web of things to go wrong

when you have six apps using the GPU at once, there's nothing for it to go wrong there in Rust. So what you're saying is that you just need to be a better programmer and see what works. That's what all the people say. That's what all the people say. That is what all the people say. I've yet to meet a good C programmer that can actually do what you can do with Rust. No, I don't think there's anyone who's actually like a really, maybe there is, but I think

most people make the argument that, you know, you don't need Rust to do this, you just do it in C I have a feeling a lot of those people don't have that kind of experience working in that sort of code where it is really really important these people may have experience with C but may

not fully understand the problem space that they're commenting about. Yeah, like, the thing is, we're human, right? We're not perfect, and so we rely on tools to help us get our job done. C just doesn't give you those tools, right? So you can tell yourself, well, yes,

you can do the same thing in Rust as you can do in C, because by definition you can. You can also do it in Assembler. You can also just write the machine code by yourself, right? Like why not use the tools that make your life easier to make you more productive and make you make fewer mistakes, right? And so like of course you can write perfect C code. You can even take a Rust program that already follows all these rules

and like convert it to C. And as long as you don't miss anything, it's going to be as long as you don't miss anything, it's going to be as reliable as the Rust program because like the Rust compiler already forced you to design it in a way that solves these problems, right? So then it just becomes an issue of like translating every single thing that the Rust compiler does behind your back to C, which is possible, but then you're still taking advantage of the Rust design, right?

So you've just re-implemented Rust. Yeah, like design, right? You've just re-implemented Rust. Yeah, like manually, right? You just turn yourself into a Rust compiler. So it's because Rust forces you to have that design, right? It's very difficult to have the self-discipline to come up with a

design that doesn't have any weird corner cases in C. You might think you can do a better job, and there's people who come up with paradigms that in theory improve things, and you can do better in C, you might think you can do a better job, and there's people who come up with paradigms that in theory improve things, and you can do better in C++ with smart pointers and things like that. But nothing, you know, it's like a proof checker.

When the compiler is telling you it can prove that your code doesn't have any of these problems, that's not something that you as a human can do without spending an inordinate amount of time on it yeah so just let's just let's do it right yeah there's no point just wasting your time like we improve tooling for a good reason like there's no point just wasting your time just

you know making things harder for yourself just for the sake of it you know, making things harder for yourself just for the sake of it. Like, there obviously are merits to doing things in C, but in the same vein, there are merits to the advanced, like, there are merits to the more, how would you say it, like, complete, not complete, the better checking that the Rust tooling offers you, if that makes sense.

Yeah, I think really at this point, the only reasons that you see are because it's already a C code base or because Rust isn't supported on the target you want to support. You see because you have C developers. Yeah, pretty much. And I mean, Rust doesn't run everywhere that C does, so it's obviously not going to replace C for everything right now. But like, there's relatively little

reason to start a new project in C versus in Rust on a platform that has both. If it's, I mean, if it's, of course, there's the argument that if it's something simple, you can just do it in C and you don't have to care. Sure. But like, that's just inertia, right? You can still do it in Rust just by learning Rust. But there's, you know, Rust can do almost everything C can do. I ran into one, there's one thing, I have one complaint about Rust that, well, there's one thing that Rust right now makes very annoying. And I've talked with the Rust for Linux people about this, and I'm using a workaround now. But basically, Rust's idea of creating objects in memory always puts them on the stack. And then you can move them, because it works by moving into some other memory location. But there's no ergonomic way to create an object already at a memory location that you chose, like with a custom unsafe allocator or something.

And so the problem is that in user space, it's fine. And the kernel stacks are like 16k. FRANCESC CAMPOY FAULKNERY. Right. LUCAS FRIEDMAN. Some objects for the GPU are 64k. FRANCESC CAMPOY FAULKNERY. OK, that's a problem. LUCAS FRIEDMAN. Yeah. So that's a very specific to Rust for Linux problem, although it can make optimization a bit harder on user space too. And the compiler can easily optimize these things, but optimization

is not something you can rely on. So it helps if you can tell the compiler to do this directly. There's no good way of doing this in Rust right now. I have a ridiculous, horrific macro that kind of works around it by piecewise constructing the object field by field directly in the target memory. And that's what I'm using and it works. And, you know, that's fine. We work around language limitations there. We work around, you know, problems with the tools we have, you know, see what I've had its own giant set of problems if I try to do what I'm trying to do in Rust here. So every language has its pros and cons.

And I'm not talking about safety. I'm talking about just general programming stuff. I would have had to do some horrible things in C to get this driver done in C, which Rust makes a lot easier. So it's not perfect. But I mean, there is a discussion with the wider Rust community about this. It's been an open issue for a long time. We'll figure something out eventually, because Rust is only getting better. But yeah, I mean, it's not a perfect language.

Of course, there's things that are still, you know, a bit annoying. But, you know, I spent like the first two or three days when I started writing the kernel Rust driver, chasing these sort of Rust pain points on Linux and discovering more of them. Because I'm also kind of the first person to make a big driver in Rust for Linux, right? I mean, nobody had needed big structures in a Rust for Linux driver until now, because the drivers that people have written until now don't need that. So that's to be expected, because I'm kind of an early adopter here. But once I got past that, right, and I got into like writing the actual driver itself and using all this tooling and these workarounds that I had built in the first couple of days,

that was really like a joy. And it's not just the safety stuff. Like even just error handling in Rust is like one character, you put a question mark on and it just like does the error return and uh because it does memory cleanup for you you don't have to worry about cleanup so regular linux c code is full of these patterns where like every function is like do something do something do something do something and after everything it's like if it failed go to error if it failed go to error and then because you have to do cleanup it's like if it failed go to error clean up this if it failed do the error clean up that and then at the

end of the function you have returned something and then error clean up this clean up this error clean up that and you have this like pile of you know error clean up like manual error clean up you have to do and that right there is one of the obvious places you can make a mistake. Yes, and people find those all the time. That just goes away in Rust. And it also manages all the, like, you know, when you get complicated functions, the possible go-to paths get insane. Nobody gets that right in C, sorry.

So with Rust, that was another thing that happened, that Xenotic was running out of memory and it turned out it was just legitimately running out of memory. It wasn't a memory leak. It's just that Zonotic on Ultra without texture compression, which it didn't have yet, does not fit in 8 gigabytes of RAM. So that was just legitimately running out of memory.

And, you know, the app crashed and the kernel ump killer went crazy because the priorities were wrong and like it killed the whole desktop or something. The driver returned errors and the basic driver complained about out of memory errors. Nothing went wrong on the kernel side, right?

Like the out of memory paths on the kernel driver just did the right thing. And normally that's another thing. It's like the first time you test error paths, something is wrong. Something doesn't get cleaned up. You end up with another, you just have to freeze something. No, all's another thing. It's like the first time you test error paths, something is wrong. Something doesn't get cleaned up. You end up with another or you just have to freeze something. No, all that just worked.

So that's the other thing with Rust is the error handling together with the, you know, implied cleanup of resources because it's all, you know, does that for you. Makes it so much easier than, you know, the whole C thing where you have to return an error code from everything and convert it and forward the error codes and check for it it's literally in Rust, it's just put a question mark after the function and it does it for you and if you don't put a question mark it complains that you didn't do that so yeah

one of the things that I think some people tend people will use this as like a disadvantage of Rust, but also will forget at the same time. Rust is only like... 0.1 came out in 2012. It's still a really young language. And it's not a young language that nobody's using.

There are engineers at Microsoft. There are engineers at Google. There are engineers at all of these massive companies that are using Rust That, you know, to different extents, but this tooling is only going to get better and better and better Like C is how old at this point? I don't know, too- very old. C language- I think the first NC version was 79?

And then yeah, the actual like like ancient sea goes far back yeah yeah but yeah rust is 10 years old there's and as these like as projects like rust like there haven't been these really big ambitious rust projects before what you're doing before what rust philindex is doing it like this tool needs to be tested on this large scale to work out where it you know where it has these um inadequacies and what what needs to be addressed to bring it to this state where it's you know going to be properly usable for this and it needs these people who are i guess who are trailblazing who are trying to like find out where it's going to be inadequate where it needs to be improved

and sort of what can really be done with it yeah like um and a lot of it is just specific to like specific environments, right? So what I found with the stack stuff on Rust for Linux, I know there's like whole OSs written Rust now. And they probably either don't have to deal with this because they have dynamically growable stacks or whatever, you know. It depends on the design of the OS. So you're always going to find new interesting issues when you start using a tool for the first time in a different environment. And the Rust project has made changes in response to Rust for Linux and what Linux needs from Rust. And Rust for Linux themselves, they've done a lot of work into

how to use Rust in Linux and what the right way of modeling Linux concepts is, how to handle Linux errors, how to make it convenient for the code and what the right balance is between it looks like regular user space Rust code and it looks like kernel C code. You need to pick exactly where that line lies and there's an art to that. So that's something I experienced when I wrote the... Because I had to write not just the driver, I had to write the general Rust support for graphics drivers for Rust for Linux. And so that was the first thing I had to do. And so, I mean, I haven't even got that reviewed yet, but it took a couple of iterations to figure out a reasonable way of doing that.

So there's a lot of work that goes into that. And yeah, of course you're going to run into things and we're still going to keep running into things and and uh and we're still gonna keep running into things uh but that's that's the case with any tool and it's the case with c i mean people run into weirdness with c all the time too so um it's not unique or anything um but i think lena starburst merged it into a rust um precisely because like it's good enough that people can actually use it and it's only gonna get better and actually use it. And it's only going to get better. And one thing that I noticed is that even though there's a lot of random media attention, the rust for Linux community is actually tiny.

There's a relatively small amount of people, even if you're actually doing stuff. And so it needs more attention from actual developers who might do stuff with this. But the intersection of Rust programmers and kernel programmers is very small right now because like everyone's doing C in the kernel and everyone's doing Rust in user space. So I think what Zitas wants to do and and also what I'm doing with this driver is sort of trying to get those two groups of people together more to to get this out there and more people using it. And some of that is just having long discussions.

We had a long talk on the Rust for Linux Zulip about how safety works in Rust drivers. And there was a lot of nuance because, like I mentioned about this box thing with safety and safe box inside and stuff, right? When you're writing a library in Rust, the box is the library, or the module inside it usually. And so if you have multiple modules, you usually want every individual module to be safe by itself. Like, you can't do anything unsafe from the outside.

Yep. But with the driver, the safety, the required safety boundary is actually the user interface because the user from user space can't do anything evil. But because it's hardware, when you get to complicated hardware like my driver, I can make every individual box as safe as is possible for that box to be. But I can't make it 100% safe because at the end of the day, the whole driver is driving the hardware and there are complexities to like, to make it safe. I need to understand how the firmware works, the firmware's written in C,

and it's not nice little boxes like that. And, you know, in the end, the whole driver has to work together properly. So there are some properties like, for example, when I submit a command to the GPU and there's a bunch of data structures in memory, those can only be freed when the command has completed. And that logic spans multiple modules, right? Because like the command completion comes in through one

and then ends up in another and all that. So all that, obviously you could, like I could write code into the command completion receiver to say that you're completing a command before it really completed and that would crash the GPU, of course. And Rust can help me with that, but that's a very simple process for someone to understand who's looking at the driver. That's not a process where that thing is going to magically say a command completed when it didn't, because we got the flag and you can look at every part of that chain and say this is right um and so that means that from if you're if you think of safety as you know

including the gpu in this case but also it can include the cpu in more complicated cases um with other hardware um it's not as nice as user space rust because you can't say every individual block is memory safe but you're still like the number of things that are related to safety that span multiple of those blocks is much smaller and you can you can think about everything individually and then very easily convince yourself that it's safe and the whole driver is safe overall so that's a discussion we had to add with the rust for for Linux folks because it's like, how do we define safety for a Rust driver?

It's not like a user space app. And I had to both teach some of the Rust folks how this whole hardware thing works and why it's not like user space. But yeah, that's why we have these conversations, right? Because we need to all be on the same page as to how these things work. And at the end of the day, the thing that's why we have these conversations, right? Because we need to all be on the same page as to how these things work. And at the end of the day, the thing that matters is that Rust is making this so much easier, right?

Yeah. So there's nothing to be lost by saying, well, it's not as nice as user space Rust. It's still way nicer than Currency, right? One of the things that I've seen people who are very, you know, against Rust bring up, I don't know if I can find... It's one of the early... From one of the early emails where Rust for Linux was going to be... I think it was one of their first requests for comments,

where Linus was saying something about you can't just crash the uh i think when like uh the code ran out of memory it was just going to just uh instantly crash or something like that i see when you find the email you probably probably know what i'm talking about um i can understand like that's a big thing about kernel rust is that a lot of the regular rust api's are disabled for that reason? Yeah. Yeah, I wish I could find the email Right now, but I can't find it But yeah, you do you do know the one I'm talking about that I was just trying to find about everyone else remember the email, but I know the issue you're talking about. Yeah. Yeah Yeah, and I can understand like

Why there there's like you have these people who are really interested in rust and then This section of like these developers who are very interested in keeping things you know stable and making sure everything they know that everything works and if these people are coming from more of a user space background i can see why there'd'd be a bit of a disconnect there initially, but that doesn't mean that it always has to be like that. You can share that knowledge and sort of bring this bring these Rust developers in line with what

the kernel needs for its requirements. One of the things that I can understand as a concern from people who are maybe a bit wary about the introduction of Rust is whether Rust is going to be a language that's around for a long time. Because it is still a really young language. Because you've got Rust coming into Mesa with Rusticle. You've got the Rust for Linux project.

There is obviously right now a lot of hype, a lot of interest around Rust, but you know C's been around for all this time, we know C is going to be here forever. Where do you sort of like, do you see Rust being this thing that becomes that like, maybe not completely replaces C but becomes a language alongside C or do you see it sort of fading into the background like many languages of the past like Fortran and Kobol

they still exist to some extent but no one really cares about them unless they need to be touching them so the thing is like Rust is bringing in something very useful to the table, right? So the reason people are using it is because it does something that no other language has really gotten right before. Right.

And I don't want to dig on any languages, but this is a random example. Go is sort of supposed to be another modern C replacement thing. But it doesn't have the guarantees that Rust does. So you can do lots of evil things in Go to crash your computer. So Rust got a lot of these things right. Got a lot of the can actually be used in kernels and embedded systems and low level systems programming, right? And so we are using it, right?

And since it's an open source project, I don't think there's really a risk that it's going to just disappear because that's only going to happen if people don't use it, right? So if people use it, it's going to be around because there's always going to be someone to help out with the compiler, even if the current team disappears or something, because there's enough people using it that it matters, right? So I don't think there's really any risk that it's gonna fade away unless something even better replaces it, at which point it's fair. Because, I mean, these open source projects

are sort of maintained in a feedback loop, right? And especially something as big as Rust, it's not something that's just gonna go undetected and end up unmaintained and nobody notices or something like that um so i i think it's you know we don't really have to worry too much about that well that's sort of replacing another tooling kind of did happen for like not to rust but rust doing that to something else with the, what was it? The Clover implementation, the OpenSea implementation, Clover, where no one cares about the C++ implementation anymore.

Everyone's focused on Rusticle. It's like, well, someone could come along and deal with this, but no one did for, like, two years. Now there's a new one. I guess this is what we're doing now. I guess it's Rust. Rust is the future. Who cares about that old...

And the person who wrote the C++ one like showed up and tried to argue against the Rust version of the MR. And the thing is like, this is just like pragmatic, right? Like everyone involved agreed that they didn't want to touch the C++ one, and that they'd much rather use the new Rust one, and, like, maintain it, and it meant that it was

a better idea. And, like, it's just about what people think on the project, right? Like, this is, um, if nobody wants to touch the C++ codebase, and everyone is happier, um, trying out a new programming language, maybe there's a good reason for that i can't recall who said it on that git lab but someone might have been dave yeah i think it was dave who said even if this like this rust implementation doesn't end up working on the future and you know the same thing happens that happened with clover everyone moves on to something else it doesn't really matter it's not wasted effort because now we know that it can be done with rust now we have a better understanding of how to properly implement opencl and we can do it better into the future and you know hopefully keep something around that is going to be this long-term implementation.

Yeah, that's also something that I talked about with Alyssa when I started working on this in Rust. Because it wasn't merged into Linux yet. There was no guarantee that it was going to be merged. But if it never is, and sort of the thinking at the time was like, well, even if Rust never makes it into Linux, the thinking at the time was like, well, even if Rust never makes it into Linux, doing this in Rust first and then rewriting it in C is probably going to end up being a much higher quality driver, as I explained before, the same kind of process, right?

Than if I just did it in C from the get-go. So, yeah, even that has merits. Well, it sort of takes you back to the thing from before, where it's... The C implementation is going to be a a harder thing to do whereas if you do it with this what's the reason why you started with Python didn't start with doing it in in like C or whatever you start with the easier tooling and then if you need to go to something that is a bit more complex then you can do that later down the

line but it's not like that previous knowledge you had is just wasted because you know you don't just throw away your in this case your reverse engineering knowledge of the driver you still have that knowledge so doing that next implementation is going to be easier yeah and it doesn't even have to be all in rust right um so rusticle is a rust layer but it still uses all of the Mesa C nerd tooling. It bands it, you know, and it integrates with the rest of Mesa. I'm sure a lot of work

and sweat and tears went into that because it's not that nice to make Rust and C work together. I mean, it's a lot nicer than C and Go at least, but it's not, you know, you have to think a lot about those interfaces. Sorry, I was going to say, also to hook in with the um the sorry i was gonna say also to hook in with like what's happening with the um the intel drivers and novo and radion si drivers yeah exactly so it all has to work together right so when i started the rust driver uh for the gpu on linux i was actually

using the mmu code from pan frost um modified that didn't last too long uh because i realized i had to kind of redo it for other reasons. And at that point, it didn't make any sense to keep it around in C. But had that not been the case, that might have been in C for a long time. That's fine, right? I mean, it's still just one part of the driver. It's based on existing code that has been tested. And no one's forcing you to do the whole thing in Rust.

And there's still some random parts of the kernel API that I use that are not properly wrapped in Rust, and that needs to happen eventually. But again, that's something that you can do incrementally, right? So none of this forces you to say, throw away all the C and do everything in Rust, right? I think I had something to say on that. One thing I can...

I don't know how i skipped over this earlier but we're gonna jump all the way back to the um the current drivers you have and one other thing that wasn't directly related to the drivers but it was the battery life um the eight hour battery life that was estimated from the uh the battery controller was that so that was just the estimation you hadn't you hadn't done a like a proper eight hour benchmark to see if it was properly gonna do that or yeah it was the estimate but i mean i think those estimates are accurate because they're they come from the battery controller and it's based on like power consumption over time and how much capacity the battery has so it's not one of those random like um you know

based on battery voltage things or something like that or based on like you know um software it's literally measuring the current so i mean i could do the test i just haven't bothered because i don't think it's necessary i don't think there's a reason not to trust that i mean maybe it's seven hours instead of eight but you know whatever but it's not gonna be like two right do a development two hours yeah yeah do a development stream where you just run it on battery see when it dies maybe I should do it on Twitch because I already did a stream on Twitch where I was like

collecting data what was it the power levels and I just had like a for loop trying every possible input just to log the results to try to figure that out and I think I left it running on Twitch for like 30 hours or something like that. Jeez. It's like, watch Lena's computer reboot the M1 Ultra every 26 seconds for 30 hours.

It's not the worst story I've seen on Twitch. There was someone who, ages back, I think they were installing... I think they were compiling Gen 2 I think it was on like a PS2 or something. Oh my, that sounds terrible. They just had the compilation just sitting there. They weren't

talking, no facecam, they just had the stream going. You know what's funny is that sometimes when I have to do like a Mesa recompile or a kernel recompile, especially kernel recompiles, and I do that while I have a snack on my streams, and the kernel recompiles, like,

rebuilds on my Intel, old Intel desktop, like, forever. But lately I've been doing more Mesa recompiles And those happen on the M1 And on the M1 they're fast enough that I usually can't finish a single snack Before they end And then there's the M1 Ultra

And that's just like Don't even bother It's gotta be done before you take the wrapper off The machines are very nice I think there's a lot of people that The machines are very nice. Mm-hmm. I think there's a lot of people that have a hard time separating the hardware that Apple has made from the general tech practices Apple has.

I think we can all agree that the way that Apple handles right to repair is pretty terrible, but we have to admit that what they've done with with their new silicon is very impressive like this hardware is kind of crazy It really is and like in our battery life a lot of that is just down to their power management. And that happens in hardware or it's just built into the system. Yeah. Because people say, like, this has no power management.

Well, it has some power management and the hardware does the rest of it. So that's why it works, right? And, yeah, it's really impressive. And I think a lot of it comes down to both having Silicon engineers that really care about the project, but also a smaller team that somehow ends up working together. Because, I mean, there are some interesting horror stories of Intel and how it works internally. And it sort of feels like every big company kind of becomes more inefficient over time. I've also heard horror stories from Apple, you know, sort of having people crunch

like it's a startup, which also isn't good. But for us users, at least, the end result has been really nice. And the, like, the silicon is one part, and the other part is just having the whole machine being made by one company means you don't have, you know, 37 OEMs fighting each other for how to make this all work together. I can understand how it happens with Apple, but it's very difficult to compete with that.

Yeah, yeah. I think one of the important things about this whole Asahi Linux project is even if it never gets to that performance parity with the macOS system, it's going to... You know it already runs faster than macOS some things, right?

Yes, sure. Someone said a game ran better than macOS, some compute workloads run better than macOS. Linux is a different OS from macOS, so some things it does better, and those other things run better than mac os some compute workloads run better than mac os like linux is a different os from mac os so some things it does better and those are the different but the the point i was getting out here was there's going to be a point where apple no longer is providing software support for these original m1 systems and at that point basically the system becomes e-waste but when you have something like asahi linux and hopefully by

that time you know a lot of the stuff that you're doing a lot of the stuff the rest of the project is doing is being upstreamed is out on other distros and it's not like hey you have to go through this like weird extra step to get everything working it's more like just grab a thing and you're good to go basically it gives this hardware another life and it's like I can't remember how long Apple generally does their hardware support cycle for but

by the time they stop supporting it it's still going to be really fast hardware and it's still going to have a use case yeah and it's also going to be very power efficient hardware so still very nice for you know like travel or uh like you know simpler use cases and stuff like that um and yeah that's also very important right there's that um linux has a support cycle that is kind of insane right i think they removed 386 support recently. 486.

386 was dropped in 2012. Which, even so... We're talking like 20 plus years, right? Nobody else does that. So that's a very good opportunity for the hardware to have a longer life. And like in my experience, it's not as useful for like, you know, the typical, you know, well-off tech person who's going to be having the latest shiny anyway. But it's a lot more useful to avoid e-waste by just making it useful for either, you know, for example, as a server you have in the corner to replace a Raspberry Pi. Sounds ridiculous to say that today, using anyone to replace a Raspberry Pi, but, you know, 10 years down the line, maybe, right? And then also, you know, I have friends who do things like

take older secondhand computers and refurbish them a bit, put Linux on them and take them to like, you know, less developed countries and run run coding workshops there and things like that. And that's really cool stuff, right? And you can really do that if you have an OS that supports the other hardware and runs well enough to make use of it. Yep. So what is the next big goal for what you're doing?

Like, where do you go now? So the next big goal is releasing it. Okay, besides releasing it. And then after that, so it's going to be working on the UAPS stuff to get it into a sort of, you know, should run Vulkan in theory with all the features. And that's going to be also working with Mesa

to get these things tested and adapted. But also Alyssa is doing a lot of Mesa work and I've been getting more familiar with Mesa's side. So I'm actually going to be doing more C code on that front because I can help out a little bit too and and then yeah we I mean it's just kind of a long road right but at this point I don't think there are really any big major milestones like the release right now because from this point over and word it's a lot more incremental so I think the next big milestone would be upstreaming the driver or starting to upstream the driver.

Until then, it's going to be a lot of, you know, chipping at things, individual things. And there's probably going to be a, you know, okay, we are like OpenGL 3 compliant point and things like that. And this will probably disagree with her. And for her, the milestones are, you know, OpenGL compliance versions. But like in the end, all of that is, you know, like incremental work, right? You chip in the number until you pass the test. And so it's the same with the UAPI, you chip in it until it finally gets to the point where you can upstream it and then we'll have a party about that um but yeah there's not gonna be any um like planned you know major thing after this i think other than that it's just working on things um picking up performance issues improving the uh the design of things uh

refactoring stuff like we got uh compressed texture support in, we got frame buffer compression support in now this is working on like multi-sample anti-aliasing and there's going to be, you know, there was a bunch of performance stuff around caching buffers and batch tracking and a bunch of other amazing things and I'm not going to explain what all those are now but

you know it's a lot of stuff, right, and she's been doing some crazy stuff recently and all of that is going to know it's a lot of stuff right and she's uh she's been doing some crazy stuff recently and all of that is going to improve the the driver a lot um but there isn't like any one individual thing i didn't think you would have like one specific goal because you seem like you jump around with the different things you're working like i'm gonna focus on this thing today and well that's done let's go to this next one and this next one. There's clearly a lot of stuff that still needs to be done

a lot of stuff to keep you entertained. Yeah and like you can't predict what's going to take longer because this is always an unknown with reverse engineering so like on yesterday's stream I was planning to work on Mesa and get some test cases

for tiling. I ended up spending the and get some test cases for tiling. I ended up spending the whole day getting test cases for the kernel site, getting clustering working, because that was broken and it was a lot more, I don't want a line x so a multiple of two a line times y to a multiple of four divided like plus something divided by something like that one line of code took eight hours to figure out to make it do the same thing mac os was doing right because i could just see the input on the outputs um and the alignment really threw me off um but the different x and y elements like why is X different from Y? What? I don't actually know how critical like getting that 100% perfect is, but I do know that when it was wrong, it was crashing with larger render sizes on the Amon Ultra.

So it's all these little things, right? And you never know like how long the mystery is going to take. The power calculations took a couple of streams to figure out and a lot of data gathering. So now that I know how they work, and the M2 was very similar to the M1 Ultra, I got the M2 done the same day I wrapped up the M1 Ultra. So that's another good thing, right? Which I guess we should also talk about

about how much easier it's been to get the M2 working once all the other stuff was already working. Because a lot of people said, oh, when the M2 comes out, you're going to have to start from scratch. Yeah, I think we mentioned this the last time you were on. And correct me if I'm wrong, but what you'd said at the time, and you can expand upon this,

Apple doesn't just draw where they're silicon each generation. It's just an incremental improvement over what they were previously doing. That seems to be a pattern that they've been having for a long time now. On pretty much all fronts. And so, for example, I found an old debug log from iOS on some A10 or something device. And sometimes you find interesting things on the internet. And it had verbose GPU debugging,

and it had this verbose GPU debugging. And it had this giant dump of all the GPU states. And everything undermatched what I know about the firmware on the M1. So clearly they've been doing this. I'm sure the actual data was a little bit different and stuff. Like the concepts were the same. They haven't rewritten this. So that's what we've been seeing with the new devices.

And we knew M1 Pro Max Ultra were going to be pretty similar to the M1 because it's the same generation, just a different variant of the GPU. It turned out that there was actually more work to get those working than to get the M2 working. And the reason was that those... It turns out that Apple implements scaling GPUs in an interesting way. We knew the M1 Ultra, people say the M1 Ultra has two GPUs because it has two M1 Max chips.

It's actually eight GPUs. So it turns out that the M1 Pro is two GPUs, the Max is four GPUs, and then the Ultra is just eight. So they've been doing this sort of scalability since the Pro, and that required new support in the kernel to make it work,

and that was what I was fixing yesterday. It might still be broken. We'll see about that. But PM2 turned out to be the m1 with slightly different structures and a bunch of different power management data which is exactly what i knew was gonna happen for sure and zero backwards compatible breaking changes in user space wow the same mesa build Wow. The same Mesa build ran on the M2 and the M1. Well, that gives you hope for the M3 then. Yeah. So I'll also take a bit of credit for preparing the UABI before we got to that point

so that it could fetch the GPU generation from the kernel and it calculates the marketing name. So it literally said M2 and the right version and everything with zero code changes. So I'm going to take credit a little bit that um because it's nice when all that works but yeah there were no actual um like logic changes needed and the kernel needed a bunch of um not logic changes it just needed some structures um which are different and then you that was going to happen because it happens with different firmware versions too on the m1 so that was totally

supported i just had to figure out the differences and put them in. And the power management stuff, yeah, the power management stuff is just crazy. It's just like every time there's a new chip there's like new numbers and new structures and new things where those numbers go and we have no idea what most of those numbers are. At this point my job is just

to like organize those numbers in a way that looks vaguely reasonable and put them into a file and mostly make it look like it's not a firmware blob. Because if you literally take a memory dump, then you're going to get shouted at by people saying that this is a proprietary firmware blob. So my job is to at least break the binary blob into a bunch of numbers that look pretty on the screen so that you can convince people that it's not a firmware blob.

And it's just a bunch of power management curves or something, which is not copyrightable data. This is Harvard descriptions. So as long as I can do that and get it into a form that looks like, okay, we don't know what these numbers are, but they're clearly something that's not code, then that's

fine. They're clearly something. No, I mean, it's like that like the M2 had a bunch of new curves and a bunch of the memory structures that we hadn't seen before and it's like okay minus one

some incrementing numbers the count is the number of power states the GPU has and then padding with minus ones I have no idea what those numbers mean. But I need to give them to the firmware so I can do its job. So yeah, I mean, I just put them into a table

and call it unknown one, unknown two, unknown three and what am I going to do, right? But this is what happens with reverse engineering, especially low level power management because it's literally magic. It's just like magic constants. And unless the person who wrote the code tells you, or you go reverse engineering the driver

or the firmware to the point where you can understand it, there's no way to figure it out. Like, I'm never going to find out what that curve does unless, like, I could spend the whole week changing it and trying to find what it does. It supposedly does nothing, and maybe it only triggers when the GPU goes over temperature or something. I don't know, right? Yeah.

So as far as all that power management stuff, what I always do is I clean it up. I try to calculate everything that seems to be related to something else to avoid duplicating data. I put it into at least some vague form of a structure that looks reasonable with the data and the right types and stuff so it doesn't look like a hex blob. And then I implement it in the driver and send it to the GPU.

And I take then a dump of the output and compare it with the same dump from macOS and just make sure it's the same or the same within running error. And as long as it's the same, I'm like, is good and that's it so only the parts that change like from chip to chip uh are things really have to work out that's the parts are just static it's just like it's just data in a in a you know in a file and the driver yeah well the last thing i wanted to ask you about is let's say you get to the point where at least you're happy with it and you're ready to get things upstreamed.

You haven't, like, you know, sensed it up yet. What is the process of actually getting something, you know, merged mainline? mainline. So it's not a very complicated process, but it's a little bit baroque, to put it mildly. So you basically just have to submit it to the

kernel mailing list, and that is appropriate, and send an email to the right people. And in this case it's going to be more involved because, of course, it's not just a driver, it's also the Rust it's also the rust abstractions for the drm subsystem in linux yeah and it's also i depend on the rust for linux patches so if there's stuff there that isn't upstream yet that i still depend on by the time

i'm upstreaming i need to block on that or uh help push that forward um because what's in rust and the kernel right now is you know just, just like hello world level. Yeah, yeah. It doesn't even support ARM64. I hear there's new patches queued for the next round. So, you know, they're clearly not leaving it there. It's going to keep moving forward. But we'll see where we are when I get to that point. So I pull in downstream Rust for Linux right now.

And so it's going to be a long conversation. And I'll probably have to start with some of those abstractions and pointing people at my driver and how it uses them so they can review that. And so in this case, this would go through the DRM subsystem maintainers and maintenance. And then you pretty much go from there, right? So there's going to be iteration and cleanup. Even before going through that, I am probably going to send it to the Rust for Linux folks to get some reviews from them first and figure all of that out. Because they also have to review it in terms of Rust, but the DRM people have to review

it in terms of graphics. And so yeah, it's going to be a pretty complicated thing for this driver for a bunch of reasons. But yeah, the process is fairly spelled out and fairly intuitive if you know the people you're dealing with. And that's important that I've been talking with the rest of our Linux folks and DRM folks and all that. So it's not like I'm showing up out of nowhere with a giant patch. They know I'm doing this and we all have been talking about how to do this and I got advice from DRM folks about how to design the UAPI. I got advice from the rest of the next folks on how to do some abstractions. I gave them some advice about how to think about some things

because I learned stuff while writing the driver, right? So I'm already talking with people. This isn't some I'm alone on YouTube doing my own driver thing. So, yeah, I mean, it's pretty much just getting the conversation started at that point. I think the talking to the right people is certainly a good call. I've seen, there have been a lot of cases I've seen in the past where someone shows up with, you know a 50,000 line patch set they've not talked to anyone yet it's not even set up I've seen some where it's like

a 50,000 line patch not a patch set they submit it and they're just like what is this? why are you here? should we mention power vr oh god so it's a whole story right with the the apple gpu being based on power vr and

to what extent it's similar and to what extent it isn't um and power vr started upstreaming their open source graphics driver for some PowerVR GPUs in about the middle of this year. So after we got most of the work done, we did learn a bunch of stuff from that driver. It also kind of scares me. And I do remember the Linux kernel upstreaming post at one point being a single patch for the whole driver. So, yeah.

So, basically, even once you have everything ready, it's going to be sort of a long process to get all the pieces merged in that you need to make sure everything's going to

be able to work pretty much yeah and you want to make it easy for people to review yeah i'm not actually sure how to do this with rust um because um i mean you get this problem with any driver that is big and large um but i think rust Rust exacerbates the issue a bit more because you tend to have more interdependencies between modules as far as getting it to compile than with C because the whole type system kind of, you know, spans everything. So it's, I don't know how I would like, obviously I can't submit my current Git history, that has to be a question re rewritten and i don't know how i would break the driver into pieces that can be added and actually compiled um independently until the whole driver is complete so my feeling is that i'm just gonna say like one file per patch and make a

nice long explanation of what that one file does and then like add the make file at the end and say okay now the whole driver builds because i don't think I can do any better for this one. And that, yeah, that just depends on the driver and sort of how complex it is. Sometimes you can build it up in steps or in logical parts of the functionality. But I think in this case, it's pretty much going to be just say, okay, please review each file individually. It's already modular and split up in a way that is reasonable. And we're just going to add the files one per patch, and

at least that way we can have email threads per module. And because the makefile comes in at the end, nobody cares. It's just going to compile when you get to that point, which is fine. As long as you don't break the build, it's fine. So that's my idea right now. I don't know if anyone else will have any better ideas. I'm open to that. But yeah, in in this case i think that's pretty much the best i

can do i'm sure if you submit it like that and someone has a better idea you'll definitely hear about it i mean i'm gonna ask about this before submitting it sure sure sure yeah that makes sense well it seems like everything is sort of coming along pretty well and this has been you know a pretty informative episode I hope that I hope that everybody listening has sort of maybe learnt something about what's going on with

the work that you're doing with the work that other people involved in the project are doing and yeah I think we'll sort of end it off there if people want to get involved with the the work that you're doing or just generally the work that asahi linux is uh involved in uh where would they where would be the best place to go um so the main community is on irc on oftc um so the gpu stuff specifically is an asahi gpu um asahi dash gpu um but there's other channels for the other stuff um they are all

listed on the uh asahi linux website so you can find them there and like i'm there this is there um it's usually there and uh you know a bunch of other people working on different things related to the GPU are there. If you're interested in Rust for Linux specifically, you should probably ask to join their Zulip instance, which is currently invite only, but they will give you invites for that if you ask them via email. And then of course there's DRI Devil and OFTC, same network for the general Linux GPU stuff. So if you want to get involved with what I'm working on specifically, then yeah, definitely join SahiDBU and talk to us there.

Like we're happy to get more developers and more people working on the code and fixing features and finding regressions. If you just want to try the driver and see if it works, please wait a little bit until release, because we have had cases of people showing up and it's like, oh, I tried this. It doesn't work. It's like, well, yeah, we know.

We're working on it. But if you're going to be developing it and hacking on Mesa or hacking on the kernel website, by all means, just join in and get okay you started um and i do think release is coming pretty soon so hopefully that's going to make things a lot easier for everyone um because right now you have to compile like three different things and and uh get them all together to get it to run at all and there's like an xr conf and um you know until two days ago there was a kernel command line argument on the m2 and there's there's always you know the

the difference between something developers are working on and that works on their machine with their setup and a difference you know and something that everyone could run on their machine so we're getting there on that um but um but yeah it's to the point where developers can definitely help out um so if you're interested uh drop by uh as for yourself where can people find you and uh what you're doing so uh do i get to to plug bd social here you can plug anything you want to plug so um i'm on twitter at lina asahi uh but uh with all the uh recent uh chaos around twitter um i thought that it would be nice to have an alternate place for um vtubers to hang out and meet creators and other people uh involved with youtubers so lula and i started a vt social which is a master instance for vtubers um and over the next

few weeks we're gonna be working on the infrastructure to make it scale and all that. We got some new hardware already. But, yeah, so we have that. And I am lina at vdsocial. Or if you want to go to the URL, vd.social slash at lina. And, yeah, so I'm on there. I usually post to both places. But sometimes I post to Masternode only. So you might get a bit more stuff there.

And there's my email. um actually you can just go on my website as a helena.net and i have like all my socials and the links there uh so that that'll get you everywhere and yeah i hang out on uh irc and also on the no pointer live discord uh which you can also get an invite on my website and so thatointerLive. It's just a group of friends, VTubers. We do development from games to hardware. We have people across the entire spectrum. And it's just a group of friends. There's no agency behind it or anything.

But we have a Discord, so if you want to talk to us there, you can do that. I have a channel for me. And I think that's pretty much it. But yeah, if you go to us at Hidina.net, that has all the links to everywhere so you can pick and choose i will uh chuck those in the description down below if there's anything that you remembered you wanted to mention just send it to me i'll add it in there

cool thank you so much as long as i don't forget like i forgot half luna's links last time but it's fine if you put the website link on that has like all the other links in it, so then you're good. Yeah, I'll definitely put the other website there in case I forget other stuff. Is that everything you wanted to mention? I think so, yeah. Thank you so much for the interview. That was a lot of fun! Well, as for me, you can go find my stuff over on my main channel, Brodie Robertson. I do Linux videos six-ish days a week, usually. I've got the gaming channel,

which is in the process of being rebranded at some point. Brody Robertson plays right now, playing through Devil May Cry 2, and also... Wait, what am I playing? Kingdom Hearts 3502 Days. Yeah, that one. I'm soon going to be doing a collab stream with Rogue Ren, we're going to be playing through Pokemon

Sapphire and Ruby doing a Soul Link, so come hang out for that one it's going to be chaos, neither of us can remember Gen 3 Pokemon at all so, it's also going to be a randomizer, so it's going to be a mess, as for this, if you're listening to the audio version, the video version

is available over on YouTube at Tech Over Tea, if you're watching the audio version, the video version is available over on YouTube at Tech Over Tea. If you're watching the video version, the audio version you can find on any podcast platform, Tech Over Tea, iTunes, there's RSS feed, chuck it in your favorite app. I like to use AntennaPod. It's a great app.

It's open source and yeah, it's pretty good. Yeah, I'll give you the last word, Lena. What do you want to say? I need to get into more game collabs and stuff like that It sounds like so much fun I know Luna's trying to get you to play FFXIV I know, and it's just an environment thing I don't have a good setup for that right now

And I just need to figure out how to do that But we do play I do have Mario Kart, we played that recently And that was a lot of fun So I need to do more of that And more collabs and get more people involved. And then I can sneak them into getting interested into drivers. Well, yeah, then I guess that's going to be everything.

So I'll catch you guys later. Yeah, thank you so much for having me. No worries. You're always welcome.