LINUX Unplugged - 364: Linux Arm Wrestling

Episode Date: July 29, 2020

The past, present and future of Linux on Arm. The major challenges still facing full Linux support, and why ServerReady might be a solution to unify Arm systems. Plus we chat with the Manjaro team about recent changes. Chapters: 0:00 Pre-Show 0:58 Intro 2:01 Terminal 2.0 in ChromeOS 4:41 Manjaro's Process Problems 13:49 Manjaro Sneak Peaks 15:41 Weekend Manjaro Journey 21:02 Housekeeping 22:09 ARM on Linux 24:01 The History of ARM 28:16 Single Board Computing Revolution 31:47 ARM Reaching into the Present 33:17 The Future of ARM 36:42 Not Everyone Loves ARM 43:01 Wants and What Ifs 48:30 App Pick: tuptime 49:48 App Pick: s-tui 50:21 Outro 51:36 Post-Show Special Guests: Brent Gervais, Dalton Durst, Drew DeVore, Jeremy Soller, Marius Gripsgard, Neal Gompa, and Philip Muller.

Dang it, my Rona hair is getting ridiculous. God, it's just in my face constantly. You just need to glue it down. But then it gets all gnarly and it gets all hard and greasy. Crystal ass dreads, I think, is what's about to happen. So then I go for the nothing in the hair thing so that way it doesn't get greasy, but then it's just all over the place. And when it's like poking me in my eyes and my forehead, I can't focus.

And everybody says you'll get used to it. Well, guess what? It's been like a solid month and I'm not used to it, so I don't think I'm says you'll get used to it. Well, guess what? It's been like a solid month and I'm not used to it. So I don't think I'm ever going to get used to it. My damn hair needs to get cut. But I just, I haven't got it. You know, there's like, there's better reasons to go out when during a pandemic. And my own vanity doesn't seem like a good enough reason.

So I just haven't done it. But now it is driving me crazy. Well, if it starts to affect your productivity. That's usually the way I talk myself into everything. Right. That's why you need ice cream today. Hello, friends, and welcome in to episode 364 of your weekly Linux talk show. My name is Chris. My name is Wes.

Hey, Wes. Hello. Nice to see you there. Nice to is Chris. My name is Wes. Hey, Wes. Hello. Nice to see you there. Nice to see you there. Hello, Cheese. Hello, Drew. Hello, Internets. Hello.

Nice to see you there. Everybody really dressed down from last week. Like last week, everybody was excited and they dressed up. Y'all just in your shorts. It's hot this week. That's true. That's true. Well, we have a big episode.

We're going to get into some community news. It's hot this week. That's true. That's true. Well, we have a big episode. We're going to get into some community news, and then we're going to discuss the past, present, and future of Linux on the ARM platform, which has become of a little more interest recently with Apple's news. But additionally, there's just some significant things that are in the works, even within

the last couple of days as we record this episode, that we thought the timing was right to do that. So we'll get to that we thought the timing was right to do that. So we'll get to that in just a moment. But before we do that, time appropriate greetings, Mumble Room. Hello. Hello. 31 people hanging out in that mumble today.

That is pretty great. Hello, everybody. Good to have you there. And say hello to the new terminal on Chrome OS, which is shipping in Chrome OS 84. It's been completely revamped as a system web app and it adds a new settings menu and a bunch of customization. I think Chrome OS just got really serious about the Linux terminal. Yeah, right. It's even got tabs built right in. Well, now, you know, they're serious, Wes. Tabs, you say? That's how power users get work done, Chris. Oh, man. Could you imagine the meeting where they're discussing the new power users they're

going to add to Chrome OS and you know like tabs was on the whiteboard. I will say, you know, this continues to make me more and more curious about Chrome OS in that having a usable, configurable terminal is probably one of the things I would want if I was going to actually be using it day to day. Yeah, and they invested pretty heavily in the Crostini environment and have added significant functionality to it over time. And now they're improving the user interface to that. It seems to be a fairly significant through line of continued investment in the Linux environment for Chrome OS. And I think the message here is this could be your simple, Chrome OS. And I think the message here is this could be your simple, no hassle, plug right into your corporate environment development machine. And that sounds ludicrous unless your corporation

uses Google apps. And all of a sudden, this thing's plugging right in. It's got your calendar, it's got your inbox, it's got your cloud storage, it does your Google meetings, and it's got a terminal. Right. You can go, you know, hop onto your Google Cloud VPS and start getting work done. Oh, what a Google Cloud future it is, Wes. And it's just, I think, remarkable because it's a clear strategy that has been adopted by Microsoft. Their manifestation of it is WSL, which is a much more sophisticated implementation. But Google was able to take a few shortcuts here since they're using the Linux kernel. And so they're able to sort of respond

competitively very quickly. It's interesting to see them converge on similar ideas though. Yeah, and I wonder, just because we are going to be talking about ARM today, I wonder how this impacts the Mac platform. They seem to be getting further away from making it possible to run Linux applications

outside of virtualization. Right. And there does seem to be in the latest builds of Big Sur, there does seem to be specific OS improvements to enable Linux virtualization. Did you see that? Yeah. Isn't that interesting? But I mean, it does make sense.

You got to have Docker. You got to have things like, you know, developer productivity tools if you're going to have developers use your platform. How much do you think it's about Docker? Right, right. Yeah, yeah. It's probably mostly about being able to run Docker containers really easily. But then you may see other lightweight virtualization suites that sit on top of a set of APIs that Apple's creating

to make Linux virtualization easier. If it works smooth enough, I suppose you might not even notice. Yeah, yeah. All right, well, let's shift gears to Manjaro. This week, there was an announcement in the change of the team composition and it happened in a fairly public way and it appears to involve the use of finances and community contributed funds. And so that always gets everybody's attention. And Philip is joining us from the Manjaro project to walk us through his side of what happened and kind of get a better understanding of the greater context here. So Phil, I don't know if my characterization

is correct, but it seems to me that there was a disagreement in the use of funds to buy a fairly high-end laptop for a maintainer on the project, not yourself. And it seems that there was a process that was followed. That process didn't work out. And so then the decision was made by yourself to proceed anyways with the purchase of the laptop, which led to a disagreement that fairly rapidly went public, maybe before the conversation was even complete. And now people don't really know what to believe. Well, the laptop is for a community developer who is mostly based in Germany. And he is presenting Manjaro at several trade shows.

So everybody in Germany at least know him. And the laptop was actually about 1K. And we added two hard drives because he has a special backup system and he extended the RAMs because he is simply doing all the testing and installs several Mancharos on his machine and he needs that. And it looked like you guys took a look at doing like an 18-month contract with a cloud provider to do the builds and you look at the cost and it sort of just works out to just end up buying the laptop. Plus then the developer has a machine to actually use as well. So I think

that totally makes sense and seems like a pretty reasonable laptop for a development workstation, especially one that's showcasing the distribution at events once they resume. I guess what a lot of people seem to be thinking something foul happened is with the stepping outside the process for the purchase. So I'll tell you my interpretation of it from an outsider and then you tell me where I land. But I looked at this situation and I said, well, it seems maybe there's some personal things going on between some team members. So that could have some influence here. But is it possible that the project created a very rather convoluted process to handle expenses and then that process failed?

And then as a project leader, you decided, well, this is a necessary expense for the good of the project. Is that essentially the events that happened? Essentially, kind of. But Matt is also here. He is one of the community developers and he can also show you his side. It was accepted by the whole team.

We talked about it and even the guy who didn't approve it also agreed on the laptop itself, but slightly disagreed with the additional configuration. but slightly disagreed with the additional configuration. So there was a process, but not like everybody on the team can discuss with him. So we said it doesn't work like that. And since I started the project, it's really important also to me

to have the developers give their goods so they can develop. Matty, for example, has a laptop not working and he's still in the team. And we are now currently looking for a working laptop also for him. So a lot of purgages will happen and the process is currently due. And Mati can tell you about what the community board will do for the next thing. I mean, that actually tracks pretty closely with what I would think would happen is sometimes a process is created for the project. And that process doesn't necessarily serve the project's end goals.

And so it needs to be modified. Tell me if I'm wrong here, but I think maybe where the breakdown was is instead of modifying the process to make it possible for the purchase to happen via a process, the process was just sort of bypassed, which is sometimes necessary when you don't have a lot of time and a lot of energy to mess around. But that seems to be sort of like the core problem there is the process wasn't really serving the project as you saw it. the project as you saw it. Is there, and it sounds like what you're sort of saying there, and maybe I'm misunderstanding, but it sounds like you're saying there's maybe a new process that's being developed now? Yes, since I'm more part of the company, I give Matty more or less the freedom to decide

what he thinks for the community project. And yes, we will see how that works out. The fonts are secured now, everything is safe and sound, and we can only move forward from now on. And it seems like this probably means long term, you'll have a better representation of what it actually costs to run a team of developers. Exactly. We will be more visible. And also the team members who are not known yet might be up here in podcast shows youtube channels whatever it depends also what to do yeah mati is joining us as well yes i'm here hello sir so uh do you want to chat at all about what you kind of have in mind i know

it's super early days and still but it sounds like maybe a new process will be put in place any insights you can share uh one thing I would like to say about what happened, I think it's a bit misleading or a misconception to say that the process was bypassed or that the process was convoluted because I think the problem with the previous way the things were processed wasn't so much that it was convoluted,

but it was kind of unclear for the team. And it was mostly run by the developer who left our team. And now we're coming together as a team to make a new process. So we're making sure that everybody's on board with how the things work and everybody knows what to do. Which should be a lot more sustainable, I would imagine. I think this in long term going to, or actually not very long, it's going to lead to a better process about this.

I think that's probably what Jonathan would have wanted anyway. Fair point. I think that's probably what Jonathan would have wanted anyway. Fair point. Is there a plan to communicate the expectations to the Manjaro user base? Because I have to imagine part of this is people are getting a snapshot of a conversation, essentially one side of a conversation that is being responded to. And I wonder now you must be thinking it'd be good to get ahead of this for the next time. And I wonder, now you must be thinking it'd be good to get ahead of this for the next time.

Yeah, it's going to be better when we have finalized the process and decided how we're going to do it. Then we can make it public and show how the process actually works. I think there's going to be much more transparency with the project and the use of the donation funds. Yeah, I'm glad to see that. You know, that's whenever we can get to a sustainable place that the project is healthy, continues and can actually use the resources that it has. That's what I think everyone in the community wants to see. So we'll have links in the show notes to the announcement and both sides of the discussion. Also, the item itself, you talk about transparency to your credit is on open collective. The expense is documented there. The process has been, I would say, somewhat transparent just in that regard.

somebody comes out and starts talking before something's supposed to be public and they end up setting the narrative and the language that they use sets the tone of the conversation that happens on reddit and in telegram groups and on irc rooms and wherever else and it then becomes the onus of the other party to either defend themselves or just remain silent, which is usually my trick. And that's a difficult position to respond from. So I really appreciate you guys willing to just come on the show and just answer any question I throw in your face and respond to it genuinely, because I know you've probably been getting a lot of negative criticism and I appreciate your transparency and just doing this part of, of this process. And when I said convoluted, I didn't mean it as an insult. I just meant when you're beginning and you're starting up as a project, you have an

idea of how things will work and you create a process and chains of command and whatnot that maybe a year or two into it or whatever it might be, you realize with hindsight, well, that structure doesn't really make sense. That's not really how this works. And so at some point there has to be a change. And generally it's some kind of event like this that forces that change. And sometimes people do get their feelings hurt and it really sucks to be on that side of it.

So I appreciate that position and I appreciate you coming on the show, guys, and we'll keep an eye on the situation and best of luck, Philip. Is there anything else you want to chat, like any teases of future developments that you want to get out there or hardware sneak peeks? Well, we're working on several projects. So currently on the Pine Phone in winter, we might have our own community edition then a new mini pc with an ryzen cpu will come and a floor of course more ryzen cpus based laptops from tuxedo and under and other partners so how much is known about this mini pc how much can we share that you and i've talked about this

mini pc because that seems pretty exciting is it Is it pretty locked in as likely to happen? Well, we have a new partner in China and it's one of the biggest manufacturers of mini PCs. We will sell it in Amazon US and EU and have it there when it's ready. So we will see. To recap, I'm really excited because this is a ryzen based system that's smaller than a mac mini that has some really nice performance characteristics and it's going to be available to buy on amazon pre-loaded with manchero that's so cool that's that's a pretty big deal wow and here in the states that's like well okay that's a possibility all of a sudden

it's like hmm i mean i look at we a couple of machines in the studio here that are getting seriously old. Like the recording machine actually had a hard drive problem over the weekend that I was able to recover and mark the blocks bad, but that's the beginning of the end, you know? And I think, okay, it's time to replace this machine. Something like that, that I could order on Amazon when, when the computer dies and I can have in a couple of days, game changer. And it would fit nicely in the studio. We don't need a giant tower in here. No, and something low profile we could have under the table. Anyways, again, thank you guys for joining us and keep us posted on future developments because it seems like there's a lot going on and some cool hardware in the works. So thank you very much. And I just, by the way, recently reloaded my workstation upstairs

with Manjaro GNOME Edition. It had started as Manjaro XFCE, and I switched over to Plasma. Of course you did. And then I switched over to GNOME Shell. Even. And then,

and then over the weekend, because I was in here just like doing computer stuff, I was doing an update on my workstation, and I was doing it in retro CRT because it's awesome. You just feel like a badass admin.

And watching your packages go by on a cool retro CRT screen is great. During the actual installation of the packages, Gnome Shell crashed on me. Dropped back to the login screen, killed my terminal session that was running the update. So I see I'm not even going to mess with it.

I drop to the shell, you know, do like the control alt F2 or whatever. I get on just the full screen shell. I log in, Pac-Man SYU, all packages up to date. No, they're not, because I was watching them install, right? So I do a pac-man clean cache you know thinking like you know the just clean everything out re-download all the packages let's just start over again we're just gonna hit go again does the whole operation takes a minute cleaning stuff up do the pac-man syu no updates available so now what do I do? Right? Because I know it's not done,

but Pac-Man's pretty convinced it's done. Switch to Fedora and DNF. It's funny you say that because that legit, when I reached for the thumb drive and I had the thumb drive in my hand, I'm like, what do I flash this with?

Right. And I thought, you know, DNF, this wouldn't have happened. But this is what I get for doing updates in a cool retro GPU accelerated terminal. Let's be honest. I should know better. So I can't 100% blame Pac-Man in this case.

So I decided I got no other option. I've got to reboot. And sure enough, the system just does not come up. And I'm thinking, like, do I try to recover here? Do I just switch to fedora and you know i just decided i don't want to do upgrades um like of distros i just want to keep my packages up to date i just want fresh packages so i decided to stick with manjaro again for a bit oh and then honestly this sounds really lazy of me because flat packs and snaps are not hard but I just kind of like having one package manager for everything.

It is nice. I mean, it's just slightly simpler when you have such wide support in your package management system. And there's a bunch of crap I got to install that's like Slack and Telegram. And now I even have a reason why I have to have Teams and Outlook. Well, I use the web app, but it's bad. It's getting to the point now where I have so many weird apps. Big old mess of software. I think, you know, it's more stuff that's outside the repo that's in the repo these days

that I use on a regular basis. And so it's just nice to have one package. So I just went to GNOME Shell, Manjaro edition, and loaded it up there, and it's great. It's nice. Yeah, isn't that nice? I mean, if you had a traditional Archbox,

you'd have to reinstall, which might take you a bit, but with Manjaro. Yeah, I still have to load a bunch of stuff, but I really don't mind the default configuration.

I, of course, turn it into dark mode. Naturally. Although I'm starting to get a little done with dark mode. That's a topic for a future show.

Well, I know, but you know, when you go all in on dark mode for so long, it starts to... That's like your personal brand. Well, I know. I know. But, you know, when you go all in on dark mode for so long, it starts like your personal brand. Well, I'm still using it, as you can clearly see. Everything in front of us is currently in dark mode. I thought that was going to be a new handle, Dark Mode Chris.

The dark tipper. Anyways. So, yeah, it's back up. It's a fresh install. It feels good. I do like doing that from time to time anyways but it was funny to sit there with

literally the thumb drive in my hand and go hmm it's a big moment of decision all the power right there and then I run through this like I would totally go to Fedora

for DNF and then I'm like but then I have to install a dozen flat packs and all that kind of stuff and it's just not that it's impossible it's just

it's a lot sounds like you need ButterFS underneath there for your snapshots. I did actually choose ButterFS this time through for my main partitions. I've never really done that with like an Arch install. So yeah, it's all ButterFS. It's all of the disks. And there's a lot of disks in that box. It's got like six attached disks.

I'm looking forward to whatever updates we get down the road here in a couple months. It's got like six attached disks. I'm looking forward to whatever updates we get down the road here in a couple of months. It's pretty great so far. I actually like it a lot. I like that you've built this strange machine

that seems to just constantly be stress testing Linux. I know. It's a weird configuration and you keep finding these fun little bugs. And then like a jerk, I just insist on having

two vertical monitors, which I don't think any developer of Linux desktop has because it is always so wonky. Anyways, that's my, that's my update. It was one of those Sundays where I came in for the LEP plug predominantly

because it happens every Sunday. Hey, so I came in a couple hours early and it was like, well, I'll just bang out a couple of things real quick. And literally up until like two minutes after the Leplug started, I was still fixing various computer stuff around the studio. And then I hung out in the Leplug for like, I don't know, two hours, hour 45.

Once you start, it's hard to start. It is. You get sucked in. All right, well, let's do a little housekeeping real quick and then we'll get the show back in gear. Because we do have a lot to get to still. Big show. So really, really quick, I do just want to mention the Love Blog is on Sundays at noon Pacific.

It's on our calendar at jupiterbroadcasting.com slash calendar. So you can just kind of get our live times. And you never know, there may be new live shows popping up on that calendar soon. So do check it from time to time. Also, you are welcome to join this show live. Please do. We do it Tuesdays at noon Pacific, which is 3 p.m. Eastern. You got any other time zones you just want to randomly throw out there?

We should just pick. 3 a.m. if you're in China. I don't know how you knew that. It's just so funny. And we'd love to have you join us if you can. And you can participate in the mumble as well or the chat room or just watch however you like. And you can get all of that at jblive.tv. And we also have that full screen IRC.

If you just want to jump in our chat room really easy, lazy style, go to bit.ly slash Jupiter chat. And that is all the housekeeping I have. Nice and tidy today. Quick and easy. All right, well, let's talk about ARM. Let's specifically talk about the past, the present, and the future of Linux on ARM. I think this is a little more convoluted than I expected. As we record this episode, convoluted is my word today.

Did you notice that? Yeah, it is. As we record this episode, Intel has announced a major restructuring of the corporation. And they've also announced that they will be delaying,

again, their seven nanometer process. Which is kind of embarrassing because they couldn't really get 10 nanometer down for more than a couple of lines. And so they were essentially punting that problem by going straight to 7. We'll have 7. We'll have 7.

And now they can't hit 7. And meanwhile, TMC is ramping up for 5 nanometer production for the new Apple chips and some other ARM CPUs. Right. And of course, there's all the intricacies around what are those sizes really mean and the differences in the processes. But just continued failure on the Intel side is just a trend that they don't want to see. And a lot of people aren't happy with. And the massive restructuring they've announced has people leaving the company. It has new people in charge of different operations. Intel is in a vulnerable position right now because while we won't talk about it a lot, there's also the amd platform now

and how competitive that has become and we will talk about risk 5 in a in a little bit but there's also that in the background as well as the open power pc architecture which is actually picked up so there's a lot of downward pressure on intel combined with their own failures and of course this is all in the shadow of Apple announcing the switch of their entire Mac line over to ARM CPUs, which early benchmarks would seem to indicate are actually fairly impressive.

And that's just their early dev kit stuff. So where does this leave Linux? And I think to appreciate it, you have to understand the history of ARM, which was founded in Cambridge, UK, and was sold to SoftBank many years later, but we'll get to all that in a moment. The early days of ARM are actually kind of quaint. It started as the official Acorn RISC machine in October of 1983. That is a long time ago, especially in computer time.

You think of the x86 platform as the old legacy platform because it's from the 80s. Right. But yeah, it started as the Acorn RISC machine in 83. But the first samples of ARM silicon that worked properly when received by a customer. Who wants that? I know. April 26th, 1985. And there's a post from 1988 where some people are talking about getting the machine working.

It's such a trip to go back to 1988 and read what I think was probably an old Usenet post that has been saved by Google groups. The first ARM application was really like a second tier processor inside of BBC Micro where it helped with simulation software to finish development of other chips and also sped up CAD designs. That's a trend you see to this day, you know, just little helpful ARM CPUs on board. And an interesting little historical anecdote here is in the late 80s, Apple and VLSI technology started working with Acorn on a newer version of what became the ArmCore. And in 1990, Acorn spun off that design team into a company named Advanced Risk Machines Limited, which became obviously the Arm Holdings parent company eventually, which then eventually sold to SoftBank. The new Apple Arm work eventually evolved into the Arm 6 platform, which was released

in 1992. And Apple used the Arm 6 based Arm 610 CPU and system on a chip as a basis for the Newton. Wow. It's like a weird parallel history here. Yeah. The first time around. If the Newton had been successful, they may have stuck with it. But what I think most people probably are aware of, but maybe don't fully appreciate about ARM,

is it's really a set of specs and licenses that ARM Holdings and now SoftBank, and potentially maybe one day NVIDIA, who's looking to buy them now, sell. And that really became the business for them in 1994, when they had that proven CPU that worked in the Newton at 233 megahertz. They could sell a slightly faster one to customers as a license. They do actually sell chips too. That actually is something that they do. But licensing is really what picked up for them as a business in 1994. And now you see all these different vendors out there that have an implementation of ARM. As far as Linux is concerned, really ARM came to the masses via Debian 2.2, which was released on August 15th,

2000. ARM for Arch or Arch Linux on ARM came out in March 11th, 2002. Wow. That is way earlier than I would have expected. Yeah, and then you see just a lot of development across a lot of different Linux distros from there. That isn't necessarily the first time Linux developers had their hands on ARM boxes. I can find interviews with Linux developers going back to 1994 where they had custom builds of early Linux kernels

that were working on ARM. But as far as a mainstream distro, it appears to be Debian 2.2, which is pretty cool. That is really neat. It's been a long time now since 2000. Here we are, and we're still kind of working it out. Right, I mean, we've covered three decades.

Now there's two more since. Wow. Yeah, yeah, how about that? By 2005, 98% of all mobile phones had at least one type of ARM processor in them, 98%. By 2010, producers of chips based on ARM architectures reported shipments of 6.1 billion ARM-based processors, which is 95% of all smartphones, 35% of all digital televisions, and 10% of all mobile computers. How about that?

Yeah, and I mean, since then, they've kind of gone everywhere, right? What device sitting around you doesn't have an ARM chip in it? And of course, one of our favorite little projects, the Raspberry Pi Model B, was released in February 2012, followed by a simpler and cheaper Model A later on. So that's when the Raspberry Pi came to the masses. And then things kind of took a whole new development track. At this point in 2020, we've had so many different vendors of ARM systems that you can't just say ARM and mean like you just take this image and run it on anything. Right.

That's been one of the big problems. But the Pi really did sort of pave the way for like the concept of a super cheap board that students, enthusiasts, integrators could buy. Right. It was actually an end user platform instead of something that you would take as an integrator to build into your own product. Here I am now running it as even a server, you know, for home use. That was it. It's a general use platform or almost. A lot like Linux. And it really just took things to the next level. And Cheesy and I were chatting about this last night and about just sort of how now we've sort of set the stage

for a whole new generation of devices that really span the range of different capabilities. My only other forte really into Linux and ARM, aside from the Raspberry Pi, were like the WRT54G routers back in the day, flashing those with DDWRT or Tomato or Letty or, you know, one of those others. But one thing that I find really cool specifically about the Raspberry Pi and how it enabled a lot of us, you know, Linux users to get something in an inexpensive board that we could tinker with. you know linux users to get something an inexpensive board that we could tinker with i think what's what that's doing now is it's opening doors for a lot of us to really become more maker oriented as well uh because along with the raspberry pi you got gpio dsi now

spi i square c so you've got a way to really attach a bunch of different elements onto this machine and really do something I think that we haven't been able to do with Linux and our general desktops for quite a while. There's just a ton of boards out there now, right? So, I mean, since the Raspberry Pi, you've had everybody kind of come on board. Obviously, one of my favorites is Pine64. They tend to be mainly rock chip. They were founded by TL Lim, the same guy that did the Popcorn Hour series of media players,

which back in the day, again, like Chris had mentioned, a lot of your set-top boxes and stuff like that have ARM chips in them, and still to this day, they do. BeagleBone, I think, needs a mention. The Asus TinkerBord, the Onion Omega 2, the Libre computer. I mean, there's just tons. The market has really exploded. So there's no reason that you can't get into one of these ARM boards and work on it, develop for it, tinker with it. And I suspect that now we're going to, we already have, but even more so we're going to venture into ARM CPUs in data centers, you know, like AWS's new Graviton.

I think you're going to see more complete products too. Like look at the Pinebook Pro. That is sort of bringing together so much stuff that we have been seeing develop over the last 20 30 years and it's a genuine product that by version two or three i'm going to be recommending to just about everybody i know it's great because i think in a way it showed me that this really there really is something to it i don't know if it's for all tasks but there really is something to linux on don't know if it's for all tasks, but there really is something to Linux on ARM. And I know you've had a chance to kind of like actually open the thing up and take a look at it from more of like modifying the hardware standpoint.

Did it hold up to that process? I purchased the original Pinebook that had the ISO keyboard. So I switched it over to ANSI. I bought the top case, which was super inexpensive. I think it was like 15 bucks or something. But doing so meant that essentially you had to disassemble the entire laptop aside from the screen. Oh, go figure. So once I disassembled the laptop, you know, it was super easy getting into it. I would say maybe 25 screws, maybe 25 screws completely. And as you open it up, I mean, really the largest portion of the inside of that laptop is the battery, like a lot of laptops these days.

But the motherboard, for lack of better terms, is a quarter or a third the size. Is there room for more battery in there, do you think? I would say no. I would say there's no room for an extra battery. I did add the NVMe adapter. I haven't popped in an NVMe drive yet because I still think there's some issues with U-boot and booting directly to the NVMe drive.

You could leave the eMMC in there to boot off of, right? Right. And so that's the idea, right, is you could boot off of that and then throw it to the NVMe. That's totally worth it, dude. That would haul. That'd be awesome.

Yeah, I mean, I think that's when you would start to see the limitations of the hardware itself, right, as compared to the current eMMC that it uses. Yeah. Why don't we talk about the future a little bit? Stay a while and listen. And Dalton from the Ubiports Project

is joining us to chat a little bit about. I wanted to really kind of get a handle, Dalton, on what some of the challenges are for the ARM platform kind of going forward, what some of the significant barriers still are. I alluded to like all the different hardware versions and whatnot, but I know you've kind of probably gotten a pretty close look at this being part of the UbiPorts project. I would imagine a bit. Yeah, for sure. So one of the things about ARM and what makes it complicated to develop on is that ARM products are designed to be this vertically integrated full stack thing. You know, most phones that are out today, you have Samsung who buys their chips from Qualcomm. Qualcomm gives them this blessed version of Linux that works on that Qualcomm chip. And then Samsung takes it and makes their product out of it. And it's not like the mainline Linux, and it's not like any Linux distribution.

It's Samsung's, well, in most cases, Android that's going to run on that hardware device eventually. And that makes things more difficult when we are a third-party software vendor. What we expect and what a lot of our users expect is the horizontal platform that x86 has become, where you take Ubuntu, period, and you boot that on whatever computer you want to. You can boot on my Dell XPS, my Ryzen 7 workstation downstairs, all the way back to some Core 2 Duo machine from 2008. Where we're going now is people are seeing, especially server vendors, don't want to be that vertical integrator. You know, Dell doesn't want to make, well, maybe they do, but Dell probably doesn't want to make

Dell Linux for the Dell EMC server platform, right? Right. And they're selling to customers who sort of expect this plug and play, just they're going to put on their own operating system and configure it afterwards. They're going to want to put Ubuntu on the server server for lack of a better option. And I think that's kind of where we come into ARM server ready, which is a set of technologies, a specification and a certification by ARM Holdings. So this is really cool. I didn't know about this Dalton until you educated me on it. And as luck would have it, on July 20th of this year, a VMware developer who works in the bootloader area for the ARM platform held a video stream about ServerReady. We'll have a link to that in the show notes.

But you guys know me. I grabbed you a couple of essential clips. And here's his definition of ServerReady. ServerReady is about making boring infrastructure. of server-ready? Server-ready is about making boring infrastructure. Boring in a sense where I can buy an HPE or Dell server today, and I don't have to ask myself whether it will run an operating system that I can also run on my laptop. So that's kind of the definition of boring. Boring means that I don't have to retrain my IT staff. Boring means that you don't need any new skill sets in OEMs or ODMs to design platform solutions around chips. Boring means that software developers can continue doing what it is

they do best, and they don't have to completely kind of change the worldview. So server-ready seems like it could have some potential if it's implemented correctly. Neil, I wanted to give you a chance to jump in on that particular topic, because I know you have thoughts. This isn't worth the paper it was printed on. Server-ready certification boils down into two parts. The first part is implementing an ARM-based system that follows the server-based system architecture, SBSA. And this essentially mandates that it should behave somewhat similarly to how modern x86 machines do with UEFI, exposing an ACPI-like interface, all these sorts of things like that, like PCIe support is actually specified in there.

That is fine and dandy. support is actually like specified in there. That is fine and dandy. You're not really going to see anything that's in an approachable price range supporting SPSA. So where you're typically going to see this is in rack mount servers and stuff like that. So the developer from VMware, he addresses because he agrees the issue right now is that it's for very high cost systems, server grade systems. But he makes the case, and I'll play his clip, that for the developers that are going to be writing applications for these future high end ARM servers, they need something within their comfort level to develop on. But I'll play it and then Neil, I'll let you respond. Play it and then Neil, I'll let you respond. There are folks in the audience who don't care about IoT,

don't care about small board computers, and might actually not really like the Raspberry Pi. Why would any of you care? Well, because there are no good client platforms today to build a good mass of developers. If I want folks to build for my Xeon E7, they can do it on a $200 laptop. If I want somebody to build for my Xeon E7, they can do it on a $200 laptop. If I want somebody to build for a ThunderX 3 whenever that comes out, what am I going to use? And I think some of the pressure here will be that cloud providers will offer a discounted rate,

well, like Amazon already does, if you run on their ARM system. The problem with this is, quite frankly, you're not going to see anything that isn't vertically integrated for this space either. So let's talk about, for example, Apple, which is the silver elephant in the room. They will follow SPSA because that is how they're going to design it. However, it doesn't matter. You are not going to be able to take advantage of it. They do SPSA not because it makes things easier for you as a developer or consumer, but it makes it easier for them to transition the Mac OS platform from x86 to ARM without having to retool the plumbing layers because the plumbing layers will behave the same. For some context, Jeremy from System76 is here. layers will behave the same. For some context, Jeremy from System76 is here. And I also emailed Carl, the CEO of System76. And I said, just your personal thoughts, not like a company position, but your personal thoughts on ARM, because Carl thinks a lot about this stuff. And by the way,

great brunch with Brent. So go check out brunch with Brent. That's well worth a listen. Carl got me thinking about how I view choice in Linux and how it's the true competitive strength. It's a good chat. But I asked him about ARM and he said that he doesn't really find it very exciting. He said device-specific OS images are pain, hard, and limit hackability, especially without good documentation.

The architecture, he thinks, makes sense for a lock-in device maker like Apple. I like that term, lock-in device maker. Remember that one for me, okay, Wes? I'm outsourcing my memory to you on that. He says, so for like a company like Apple or perhaps an entry-level performance machine, but System76 cares about performance hackability and users easily running whatever they want. So that's kind of how I think about it. Now, of course, that's a great position for him to take, but that's as an end user, how I think about it. I mean, it's your device. You want to do stuff with it. That's it. He says, I am excited to see where RISC-V goes, though. We'll talk about that in a moment, but Jeremy, I saw you tweeting recently about moving

some fancy stuff over to ARM machines possibly, and I'm just curious about your thoughts on this. Yeah. So I agree with Carl and I think what Neil has said is completely correct. What we want to see as Linux users and Linux developers, we really want to see the high performance chips run mainline Linux. We want to see something like the Huawei trips or the Qualcomm chips that are being used in top of the line mobile phones that are able to do 4K video from YouTube, which the Pinebook Pro can't right now. And we're simply not seeing that. Apple and what they're going to have, that is not going to run desktop Linux. So for us who are outside of the server space and the developers

who really want to have a platform like you get from an x86 laptop, there was some hope. There was some hope when Windows was pushing Windows on ARM because things would have been more like PC on those ARM platforms. Now, unfortunately, they had mandatory Secure Boot, but that's something that can be worked around. Microsoft does sign other distributions. But I think we're kind of doomed with the way ARM is right now because ARM Holdings prefers that they have value add being done by every single SOC vendor. So they will produce

the minimal, absolute minimal instruction set. And then the SOC vendor will add a whole bunch of memory addresses and memory map devices to do things like acceleration. And those accelerations are in many cases important to running a desktop operating system. So having support for that means often you have to run a specific kernel for that specific device, which is what we see in the Android space. So my hope is that RISC-V doesn't run down the same path. same path. And I'm really hoping that there is a general purpose, very well-defined thing, such as what we have in the x86 world, which is we have ACPI, we have UEFI, we have PCI, and these standards allow you to do hardware discovery in a very generic way.

Yeah, that really speaks true to me. The fundamental, like just nature of the way these CPUs are designed and the value add that's created by the parties involved, it fundamentally means they're always going to have unique properties. So I kind of look at this, though, acknowledging all of that in a perspective of things do change. And imagine a world, Dalton, in a perspective of things do change. And imagine a world, Dalton, where NVIDIA buys the ARM holdings from SoftBank, which is the current rumor that's being reported on, and structures change a little bit. Apple releases extremely successful ARM-based MacBooks, at least from a performance standpoint. And Windows OEMs, like they have historically now for 30 years follow suit and release high-end HP and Dell laptops that are based on ARM running Windows. In this scenario with server-ready,

you could see just that market pressure creating more adoption combined with the fact that it's kind of close on the Raspberry Pi 4 right now. What are your thoughts? Yeah, that's absolutely correct. And as the talk that you'll link to kind of goes into, there have been Qualcomm ARM laptops and mobile processors or mobile devices that use a variation of ACPI or something similar to it. But at the moment, they only run custom Windows on Snapdragon images. But I see that going in the direction that we want,

where devices that ship with Windows on ARM can boot enough just out of the box to say, I have all of this hardware, Linux, please look for your drivers for all of this hardware. I mean, we want the ARM vendors to be putting all those drivers in the kernel too, but it'd also just be nice if we could know that we need those drivers to be available in order to use the hardware that we're currently booting Linux on. No kidding. So I see a future where these fundamental problems that Neil and Jeremy have touched on are not solved because it's sort of now 30 years in, it's what the platform is. Right. And someone has to pay for it. Yeah, exactly. As Neil said. And I think ultimately, you could see a moderation of it if there is enough competitive pressure brought by risk or power PC. You know, I mean, a man can dream. But I don't think it's ever going to be fully solved in the ARM ecosystem, which is why I kind of fall down with Jeremy and Carl and Neil and sort of saying we really kind of need a different solution.

Carl and Neil, and sort of saying we really kind of need a different solution. But what I like about what Dalton is touching on here is that doesn't mean the ARM ecosystem goes away. That's never going to happen now at this point. And so if there is something we can do to apply some kind of pressure, some sort of at least standards that we can expect, it would make life so much easier for Linux on the ARM platform and for Windows and for server vendors and for developers. I'm just tired of downloading the wrong ARM build or something. Yeah, I think that's really kind of has gotten all of us a little bitter about it. It's like, let's just solve this now.

And, you know, like I would love to be able to just, when the Raspberry Pi 5 comes out, I'd love to be able to just assume I can put the current build of CentOS on it. Like I don't have to wait for somebody in the community to create some special fancy boot system. It'll just work. Same with Ubuntu and Fedora. And I mentioned CEDOS because that's really what I tried last go around. And we're just not there. But maybe in a future we could. sedos because that's that's really what i tried last go around and we're just not there but maybe in a in a future we could i'll play the last clip from that presentation that that touches on this and then we'll link for the rest of the show notes far as os support for those you know for

example in linux it's non-existent nobody has done that work um within netbsd and openbsd today are probably the the best operating systems in terms of support for the platform using ACPI bindings. So some of the biggest challenges is in the way how USB is wired up. Due to the silicon bug, the PCI Express root complex can only address 3 gigabytes, while the Raspberry Pi comes in a four gigabyte flavor. Because of that, there needs to be a way by which the firmware can communicate to the operating system that it shouldn't use the full four gigabytes for DMA. In other words, it's still a ways off before it's going to be there, really what's the reality of it. It's a fascinating talk to watch though. I mean, just some of the technology

and the clever things going on to make this work as much as it is so far. And I think maybe ultimately the other thing that's really nice about it is just to see people addressing this and starting this conversation. It really will kind of, we'll kind of see the results will be driven by what the predominant uses for Linux on ARM end up being, which is probably going to be data center and IoT. Something tells me if I were just to guess. Thank you, guys. It doesn't seem like the silver bullet we were hoping, but you could see a Raspberry

Pi 4 or 5 one day getting support for it because it's just small things. Just small things now at this point. But small things that persist. Hey, Wes Payne. What do you say we do an app pick or two before we get out of here? Oh, an app pick you say? I do say, Wes. I do say. I do say it, Wes.

I do. I want to talk about TU uptime? T-uptime? Tup time? I like Tup time. No, it's got to be T-up time, rightime? T-Uptime? Tuptime? I like Tuptime. No, it's got to be T-Uptime, right? It's got to be. I think Cheese decides he found it. It doesn't matter, Wes. Who cares? It's a cool utility.

I'm going with Tuptime. Tuptime? You guys are crazy. It just sounds good. Tuptime. It just don't matter. It don't matter. It's a tool to report the historical and statistical real time of your system between restarts. Yeah, you know, we're all familiar with uptime and just find out how long the system has been online. But what if you're interested in a longer term behavior, right? How long was it online? When was it last restarted? And history about previous runs.

And what was the load during that time? It looks like as well, which is great. Want that. Reminds me of a classic command that doesn't give you nearly as much information but this is great because you get all the load info too and it's a super easy command because once you get it installed it's just t uptime which has got to be I can't believe I'm the one advocating

for that you guys are trolling me right now aren't you that's what's happening you all know how to say it well don't worry Chris taptime is available in the AUR taptime even with you right now know how to say it. Well, don't worry, Chris. TupTime is available in the AUR. TupTime. Even with you right now. And then we have one more bonus pick, which I think we've

mentioned before, but recently came handy, so I want to mention it again, is Satui. S-T-U-I, which is a terminal-based CPU stress and monitoring utility, or as Wes Payne calls it, Satui. Well, now you're just putting words in my mouth.

I'm pretty sure I've heard you call it that, though. I thought that one was S2E. Yeah, that's how I would do it, I think. No, I'm pretty sure you said Satui. Stewie? It's time and S2E. All right, well, that brings us to the end of this week's broadcast. Thank you so much for tuning in. Don't forget, we'd love your feedback at linuxunplugged.com contact.

You can also join the conversation in our Telegram group. That's at jupiterbroadcasting.com telegram. Links and more information like that talk are all available at linuxunplugged.com 364. He's at Wes Payne. I'm at Chris LAS. The show is at Linux Unplugged. And that's probably all I have to relay to you. When are we going to be back here?

Oh, you want me to? You want me to say when we're going to be back? You want me to say it? I do. We'll see you right back here next Tuesday. ស្រូវនប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ប់ Dalton, we didn't get a chance to talk about it in the main show, but I'm just curious if you have any project updates while you're here. Oh, well, everything's going pretty decently on the Pine phone. We got all the, fingers crossed,

all of the Pine phone UbiPorts edition devices have shipped out and are arriving in people's houses now. Hey, congratulations. That's a big moment. That's a bit of a wait off. Yeah. Big thanks to Pine64 for that. On the device, there's still some of the basics to get going, and it's been pretty exciting. And it's made me learn a lot about what most integrators do on the ARM platform,

and really let me bring the perspective that I had today. Because before this, I only knew the Android perspective of the world, which simplifies a lot of that out for you. That's an interesting insight, and I can totally see that. That's a pretty solved problem at this point on the Android side. To a point, yeah. Oh, yeah? Only to a point, huh? Yeah. Okay.

Why am I not surprised?