Podcast Archive - StorageReview.com - Podcast #125: 45Drives and the Creation of the HL15 Homelab Storage Server

Episode Date: December 4, 2023

45Drives has been a cloud-scale staple for some time, delivering some of the first… The post Podcast #125: 45Drives and the Creation of the HL15 Homelab Storage Server appeared first on StorageReview.com.

Hey everyone, Brian Beeler with Storage Review. On this podcast, yes you can see we're doing it in the lab, we've tried to make it quiet, but being here felt right today because we're talking about this, the Homelab 15 and other things from 45 Drives. I've got Doug Milburn from 45 Drives who's joining us and Doug will get us up to speed on everything about 45 Drives, everything about the HL15 Homelab server. And as usual, we're streaming this live on our Discord.

So if you want to participate in this podcast and put your questions in and have me ask those of Doug or any of our guests, be sure you're in our Discord. Discord.gg slash storage review is how you do that. Get in there. We've got about 3,000 strong of fellow geeks and nerds that are talking 24-7, mostly about technology, but all sorts of

other things. Lest I blabber on for more. Doug, thanks for joining me. How are you doing today? Awesome. Thanks, Brian. I am doing well here in the late afternoon in Nova Scotia in November. You must have snow in Nova Scotia? Nah, you know, we're a pretty mild climate. Our snow is probably not going to come. I'm lucky to have it at Christmas. Oh, really? That's not so bad then.

I think everyone, no, not everyone. I think me, as an American in Cincinnati, just pictures anything north of about Columbus as snow territory. We don't get much in the Ohio River Valley. Yeah, you know, we're actually way east is what we are. People, you know, you look at the maps, they've squeezed east coast, north, south. We're way east.

So yeah, we get a lot of precipitation in the winter and sometimes they're buried in snow and then the next, you know, next week, we had a big rainstorm and it'll all melt. Well, it seems an unlikely place to build a storage company. How did that come about? Actually, give us a little bit of perspective on who you are and how you got into this in the first place as well. Sure. You know, why are we here? I was born and grew up here. That's the roots. I'm a geek by birth and uh liked all things electronic computer

mechanical whatever else uh uh went off and studied physics for a couple degrees uh then did a phd in mechanical engineering uh very experimental uh learned electronics digital electronics analog electronics uh played with computers uh loved tying computers to things. You know, along the way, a bunch of other things, you know, I ended up being an entrepreneur. I set up all our networks and everything else as part of entrepreneuring. So that's kind of it.

Got out of grad school, started my first company, and decided to move back to this part of the world. And, you know, it's one of the really interesting, it's what I love about technology is, you know, and with Internet, you know, we're talking in video here and, you know, you're in Ohio Valley, I'm in Nova Scotia, we can live where we want to live, right? And it's really, really cool and do what we want to do. Now more than ever. Yeah, I mean, when you made that decision, though,

you were a bit of an early pioneer on not exactly work from home, but make everyone work from where you wanted to be. I mean, that's... You know what? Nova Scotia is an interesting place. Nova Scotia had nine universities, a couple of them amalgamated, and there's about a million people in Nova Scotia. Really, really highly educated population here. And so uh so yeah we have all kinds of people and in fact the you know i'm

an old steel and coal country and uh and you know it's all coal mine steel plant here when i was growing up so you know people with technology either you know got to work on that or you you know work the university or you moved right in the old days uh so it was actually it's good it's uh you know a lot of pent-up demand for people who want to live in this part of the world and people like the outdoors uh you know but sea coast sailing uh we got a couple ski hills around here in the winter time and all the hiking and you know outdoor stuff you ever wanted surfing though if you like that kind of stuff, great place to live. All right.

If you like Huffles Towers, not so much. Well, we'll have to come visit you. That seems, it seems like a little bit more outdoorsy version of Cincinnati, so we're not so different in that regard. So are you, I mean, you're talking about the technology, but you're also a metal bender, right? I mean, to put it in the simplest terms possible.

I mean, how did you figure that part of the business out, too? Sure. Let me. So, yeah, PhD research, you know, postdoc. I went to a place called University of Waterloo in Ontario, my grad school. And entrepreneurial, I started a company that's still, actually, it's doing great. It's its leader. It's Field Natural Lighting, Lake Confusing Glass.

Another long story. In trying to do all my research, I was just stuck. You know, when you're doing things, it's one thing if you're just in the electronic world. But as soon as you step even seriously into that world, you move into where I was, materials research, building manufacturing equipment, stuff like that, you need metal things. Yeah, they're kind of just dumb pieces of sheet metal or machine parts or whatever else. I'm being facetious about that.

There's a lot to it. And it struggled in the work we were doing. So we started a company called protocase uh my business partner and i who went through you know similar challenges and uh it's become you know protocase become the name if you want electronic enclosure sheet metal parts if you're a scientist or engineer and now machine parts and we built that it just kept on growing. And, you know, we anybody is anybody in technology who builds electronics or builds technology things. It's so broad aerospace, private space, you know, every level of innovation. We become the name for it in building these companies.

We had this real strong research background uh you know you think about bending metal and it's like to do what we do small jobs for science and engineering it's about moving information that like the metal bending straightforward you know you got to invest lots of money buy fancy machinery but it's straightforward but to do small jobs it's about being able to take in designs from people and we reach back we actually developed our own 3d cad software it's called protocase designer built from the ground up whole server background uh fully electronic will go from a scientist's brain in through our cad system and all the way up to our machinery and FedEx completely electronically. That's all built from the ground up over the last

20 years and gives us an enormous competitive advantage. So we built up this. We're really hidden inside that metal company as a technology powerhouse. Absolutely leading in it. And along the way the story of this company called Back the way, you know, the story, there's a company called Backblaze, which, you know, lots of the audience might know them as a pioneer backup.

And now the word now that people recognize what cloud storage provider is, one of the leaders in that. And we started, we got involved with Backblaze as they tried to reinvent and rethink storage servers to take them down from, take them down from enterprise. When I say down from enterprise, they wanted up the capacity. And for their thing, you know, it's a cold storage business largely. And so speed wasn't an issue.

And they were, you know, way ahead in cl way ahead in clustering to do what they do. And anyway, we got involved with them and started, you know, worked through the whole form factor that they built their business on. Along the way, some other companies came to us. One of them was Netflix. And we had a chat with Netflix about building computers for them. Never went there. But they shared their electrical architecture with us.

And they're different from Backblaze because speed became very important to them. Streaming. If I remember correctly, at the time they're talking about one to two thousand video streams coming out, coming out of one box. And it's a lot of data. And and so we're all ears and very open about their architecture. We had a couple other communications company to work with. We talked about that electrical architecture.

We ended up, you know, so the whole basis is Thornator. And, you know, now it's, you know, Homelabs optimized offspring, the HL15. It was all about taking that form factor, evolving it, making it really convenient for server room people, you know, tool-less, simple, robust, built to maintain and live forever, along with an electrical architecture that, you know, our philosophy is everything we design has a straight lane from each storage

device through to the PCI bus and into the CPU and then some good wide lanes coming out of there. So that basically the machines are designed to be able to operate all the storage devices in parallel to get some incredible data rates. And with that comes reliability in a strange way that the rest of the high-density server world has been built around multiplexing to save costs. And the minute you multiplex,

two things happen when you multiplex. Number one one you slow your speed down which is not necessarily for a lot of things you still big boxes with enough parallelism still pretty fast but you end up there's another driver layer you have to go through which was when we uh you know dealt with early on with people dealing with backblaze servers trying to adapt them to multiple purpose, just tripping over the drivers and tripping over the drivers. So, you know, we developed our, you know, evolved that direct wired architecture that we call it. Now, everything we do is based on that ultra simple, ultra fast build, you know, big, strong, fast. So that's who we are. That's

the basis of who we are technically anyway. All right. Well, Doug, you covered about 37 points I want to get back into and touch on. But for anyone that wants to get up on the HL15, or maybe if you're listening to the audio only on your favorite podcast player, do go check out our video on YouTube. We've done an initial unboxing and a rig up of this thing. of course um the fun fun part depending on your perspective of fun is that my guys jordan and kevin turned this into something i don't think it was necessarily intended to be so check out that video it's it's fun and you'll see uh my exasperation at why they've put three ssds a gpu and then swapped out the CPU in this thing. Oh, and then got this rig from Noctua to put in there. I don't know what they're doing. Sometimes I just quit asking questions

and just let it wash over me. So that's one thing. Check out the YouTube for a video on the HL15. Tells you everything you need to know to get started there. Second thing, when you go back to Backblaze as where, I guess, this concept around big storage servers started, you sort of casually said they were a cloud provider or an archive off-site data services provider or however they pitched it back then. But at that time, the cloud world was still pretty nascent, right? And you had the big guys, AWS was out there

with their cheap and deep and slow sort of Iron Mountain alternative for getting offsite copies out there. And there were other things there. But if you look at the traditional infrastructure space, hard drive density wasn't necessarily a big focus. Now there for sure were boxes from Supermicro and a bunch of the other OEMs and ODMs, but maybe not, as you said, with the simplicity and cost profile with also an eye towards performance that you went after with that first Storinator. Is that a fair characterization? Oh yeah, absolutely. You know, what's been happening since we've started, I mean, since eternity, the tech world evolves, right? And, you know, what we're seeing is just, you know, a massive growth in the amount of data that our world produces now. And, you know, we're seeing people evolve, you know, companies evolve,

we see more AI, for example. And they talk about AI, you want to learn on a mega data set. Well, now you got to be able to read in, you know, read out of it quickly, you got to be able to, you know, fill the darn thing, and you got to be able to read out of it at a speed that works. So there's, you know, I think if you go back 15 years and you look at it you know data sets were very much smaller so the demand for you know for very large storage was significantly smaller and i think you know what people did i mean you just look at video now and you know we just see we do a lot of work with people who do video, various stuff with video post-production or everything, not to surveillance to whatever.

And you look at that, and it's like 4K is just ordinary now. We've got lots of people, anybody professional shooting 8K. Man, that is a lot of data to lift around. And you want to move it from your server to somewhere else, you better have a lot of cups of coffee around because it'll take you a long time unless your infrastructure has some you know you know has some horsepower that you can you know push that data up to speed right now that that's what's changed on it and that that's you know we've been very much in the forefront of that so when you think about the the core 45

drives business then where where you are today, what's today's goal? How is it different than back in the early days of figuring out these storage solutions for cloud providers? What's the focus today for that business? Yeah, so I have to say focuses. I wish we had one, life would be easier, but we follow where our customers are. And I'll tell you our main, you know, and we'll leave aside our home labs world that we're in.

We earn our living largely off of, you know, vastly off of enterprise. And that word enterprise is broad generalization some of the areas that we work in government strangely governments produce a lot of data for some reason I don't know for better or for worse but we especially a federal government level we deal with federal government, everything from law enforcement to scientific agencies to, you know, military services, etc. We do a lot of work with them. And a lot of that, and it ranges from our high density standalone servers and a lot of clustering on that, a lot of data sets to go larger than that and the whole thing of clustering, getting into high availability, being able to survive a server outage and

still keep running. So that's an important area of ours. Another one, research. Universities and research lab, well there's a crossover in the federal government there in research. We do a lot of research stuff. Really, really interesting projects. I'm just thinking every seat of California, San Diego wildfire project is one that just popped out the other day. We were talking about them. I don't think they mind us talking about

them. Yeah, exactly. No, I know I don't think there's any big secrets there. They got a wildfire project where they use basically video cameras, a huge array of video cameras to do fire observation and be able to spot wildfires and artificial intelligence and

algorithms and just massive data sets, like huge flow of video and then processing it. And it's tens and tens and tens of petabytes. So we do that high-end clustering thing. It's a real area of expertise for us, doing open-source clustering. We have another area that's really interesting to me that's really developed, and it's small and medium municipalities.

They, interestingly, have a huge data problem. They have all their general data, which bloats everywhere. They've got taxpayer databases and GIS data, et cetera, et cetera. But it also takes in education, which is having data bloat. And the other one, police departments, interestingly. So forensic data, you go to crime scene, there is ultra high res video everywhere, large photos and forensic data off computers. And on top of that, they've got body cams and car dash cams. And they have a massive data problem. And they developed something we call ServerZoo,

which so many organizations have. You bought a good old Dell back in 2015, and it's still chugging away, but it's a little small. You bought another one, and it's an HP Enterprise. You but it's a little small you bought another one and it's an hp enterprise you bought another one from somebody else bought another one somebody else and your data is here and there and everywhere and if any one of those standalone servers goes down and in that municipality case the da's got to go to court to prosecute serious crime and they can't get their brief printed out off the storage server, you got an emergency. So what we do with them, they are adopting storage clustering, which is really fascinating because, you know, I mean, before it was a little too exotic for them. We do,

we internally call a niche clustering for everyone. And we create clusters that really work in every which way for them. And all of a sudden, they got one piece storage infrastructure that's infinitely elastic, software spaces designed, or shares designed, set up in software. So that, you know, you just want to expand, good, plug in more hard drives, add another server. Server goes down, no big deal. I'm going fishing in the weekend. I'll fix it on Monday. So those are some of our areas that we do. There's a number of other ones, but that's probably 60% of our business right there. Well, when you're thinking about the underlying hardware, obviously the company's name is 45

drives. Everyone knows that, or at least our audience knows that, initial Storinator product. But are you looking at, I mean obviously you've got Flash solutions now that are a little bit different. You're looking at higher capacity servers. Do you care or do you need some 100 plus bay, super dense thing? And then on the other end you were talking about

some of the edge work, be it around forest fires or any of the hundreds of other use cases. Are you looking at anything in the enterprise portfolio that would be smaller and rugged, ruggeder for these edge use cases? Are those two sides of the 45 middle interesting to you? Well, you know what? We really cover, well, 15 drives up to 60. We've looked at higher density. You know, get into the whole JBOD world. If you want slow, ultra large, JB you look at, we don't have a hard number for it. I've always wanted us to do a little bit of math on it. And around, you know, CPU power per terabyte or, you know, per storage device or per capacity. There's a whole issue of density, of storage versus density of CPU and PCI lane. And we think we're in 60.

We think we're somewhere in the top of that. When you get past that, you get into cold storage. And because of where our pricing is, we have a lot of enterprise. They like the reliability. They like our stuff for cold storage for that. But there's other stuff in cold storage you can get to. So right now, we're not going there low end getting down to four drives okay and on on performance uh yeah moving to flash uh we have a stornator flash 32 drive unit which is out now

and doing very very well for us it's a SATA SATA SAS machine and we have actually the our last our last component we need to manufacture manufactured we sort of pre-released it a few months back but it's a 32 drive NVMe go. It's actually the first tri-mode machine out there with the new UBM standard. And we built a card and worked on that tri-mode card for quite a while. So you can plug in SAS, hot swap SAS, SAS, and NVMe's in it. And it will do 16 million IOPS and 32 gigabytes per second transfer speed. So it's a beast though. It's going to be cool.

When you have that, you need to get one over here. We've got all those big fat Soladyne QLC drives. We'd love to throw in there and fill that thing with the 30 terabyte or maybe we can get the 60s by then in big quantities. It's absolutely amazing what the SSD guys are doing in uh in in flash capacity right now really impressive and it's not all that expensive obviously it's more than a 20 or 25 terabyte hard drive but you were talking a little bit about software if we get into

some more advanced software that can do compression deduation, and if you start putting enough gear in here in terms of CPU and DRAM, it gets pretty interesting in terms of what you can do to extend the life or the effective capacity of those drives. Now, it's not for every workload. Obviously, backups, probably not the right use case, media entertainment, anything uncompressible, but there's plenty of other use cases where that capacity is really, really great and you can stretch it. Okay, so I know we've talked a lot about 45 drives and how we got here, and I think that's sort of foundational for how we got here, and for anyone on the audio, I'm gesturing at our HL-15 sitting on the desk next to me. You know, I don't know if you saw our video, but our team was pretty enamored by it

and they don't get excited about much really from a hardware perspective. And I don't mean to be disrespectful in any way, but we see, I've got hundreds of servers in this lab and downstairs. We've seen it all, touched it all. They really love the build quality of this system all the way down to just little things on soft edges and powder-coated

interior and and lots of little things but before we get into all of that how did the how did you conceive of the HL 15 and what sort of conversations or Market research did you do to determine that for your first launch anyway that this was the config you wanted to go with? Can I duck your question for one second? Sure. And we talked about your video, and I loved it, it was incredibly flattering, so obviously I loved it for that reason.

But first thing I did when I heard it, so we manufacture in North America, right? And not that common, and all the physical side of it's done in North America and factor bring our new NVMe boards North American made and whatever else so big believer in that anyway our people actually make the metal and our people who assemble and we saw that and and love the fact you know you pick things up edges you know but buy anything today go buy a new barbecue at Home Depot or wherever you get it.

And, you know, you risk cutting your hands. You know, it's funny. We are our crew, the manufacturers. We spend a lot of time with them. You know, industrialism, incredibly efficient way to make things. In the old craft days days people knew their customer and they got this satisfaction they did their work and their customers they'd be in touch with them and somebody say thank you you work the restaurant industry somebody says thank you that's great you

know thanks for the coffee you poured me at the at the coffee shop or whatever else they don't get that right and and these people we've been working we always work and we say to everybody you know if you're working at the deburr station and parts come through, you know, you're not there to shove it in a machine, take it out, and just take it out. We get robots to do that. We get humans to do it because they use their brain and go up a level and say, no, I'm here to make that part, that rough piece of metal into something good that's going to go to somebody like yourselves at the end of the day. You guys pointed that out. We took that. We cut up a little bit of your video took it over to all our production

ships and showed them that and said guys am i ever proud of you guys you know what this is what you guys do for other human beings so uh and and they appreciated it so thank you very much for that well it's it's like a legit it's a legit thing though and especially when you choose white which is a bold conversation we'll talk about paint schemes later if you bleed in this it's going to be really obvious and uh i uh i think it was dylan our intern i hope osha's not listening just last week or maybe two weeks ago cut him cut his hand a little bit oh jordan did it too uh and kevin's not everyone's cut themselves on a server or a piece of equipment in here for some reason or another that's that really shouldn't be sharp but like especially some of

these uh these edges like to hold your fans in and the the drives on many other servers those things are quick cut and they're not they're not smooth and uh you know what i recommend someone buy a server only because it wouldn't make them bleed yeah I mean maybe it's not a top eight feature but it's a huge differentiator when especially with something like this where people are going to tinker a lot and play around a lot and be inside this thing a lot that's why it stood out as such a an important thing to us anyway. But all right. So I appreciate you pausing and calling that out after we noted it. It's a great thing. So, you know, the the conversation right before that was like, how did you get to the HL-15? So maybe you can take that on and and talk to us about why we're here with this? Sure, Homelabs and geeks,

I say it me as a proudly being a geek. Yeah. I wish I had time at my age 59 now and three companies that rock and lots of stuff going on in my life. I wish I had more time to play with technology and I don't, I gotta do it vicariously unfortunately, but we got a whole

bunch of people in our company that you know when they come to work they do technology and when they go home they do it for recreation and uh you know we we early on with when we first started in the you know first started 45 drives you know as you know a lot of video a lot of youtube video and your audience would know uh and we started to do that and and uh and and out of that we got a lot of people who they liked our open approach you know our new enterprise let's get rid of this snobby crap let's be real about technology let's make great stuff and and look after people so uh doing that so we crossed over into that audience and we had people so many people over time said will you sell us one of your enterprise chassis uh will you uh you know

and and can i buy one of them and you go you know anyway so and why do they want it it's the big strong fast thing it's you know i don't know what do you want i know my son is a little bit of a got a little touch the redneck i joke with him uh and a half joke with him and uh and and he loves the outdoors he snowmobiles and stuff like that we live in a you know in a relatively small low population density and uh and so he drives a large diesel pickup truck okay and and i think that you know that's part of the attraction. We did, when we first started talking about the product, we said, you know, we used the analogy. We said, what we build in Enterprise are the equivalent of 18-wheelers.

And there's lots of people who are fans of machinery and look at an 18-wheeler. I got a little bit of that in me. I look at them go by and I got 600 horsepower turbo diesel and lots of chrome on it and all that horsepower and capacity. And they go, hey, that'd be cool. I like doing that. And then you go, yeah, but when I bring it home and the wife sees my visa bill and when I turn it on and that beautiful redundant, you know, 7,800 watt power supply starts blasting, it's just not working.

And we said, you know, how do we boil this down and how do we go people are asking us for this and and both you know for full build stuff and also because we know because it's us too that there's people want to tinker they want to build their own stuff they got their own visions in their head and and you go yeah what can we do for them and we've been talking about that for ages and, you know, turning away this, you know, steady trickle of requests on it. We said, we'll put one more data point on this. We are excellent low volume, you know, this high end enterprise, you know, very large data storage thing. It's not a high volume business.

Right. Right. And we looked at that. We know how to run a business and build products at a reasonable price you can't build a budget price but a reasonable price at very low volume so you said let's put something out let's build that product you know in automobiles you know you can get to work in a kia and it's probably much more sensible my son wanted to buy his gmc pickup truck with 450 horsepower turbo diesel and thousand foot000 foot-pounds of torque.

And does he really need that? No, but he likes it. So, you know, do you really need, you know, 1,180 megabytes a second, you know, as one half of your HL15 in your home lab? No. But, you know, lots of people would like it. So we said, can we do a product? We said, so we said, yeah, well, let's put some energy into it. Let's see if we can do it. So that's

where we went. Well, you must have, I mean, I know the pre-order was wildly successful. I don't know what that means in terms of units built, but I mean, you must have built quite a few of these. Do you have any, is there any perspective there on this versus your enterprise business in terms of units? Is this a material thing for you? Is this a fun project for you guys? How do you characterize this?

You know, when we preordered this thing, we looked at this and we said it's boutique. I look at the comments, a lot of videos, and people say it's out of my affordability range. And I get it you know we we get who the home labs world is and uh a lot of young people in the home labs world certainly not all but it's young people and you know you're early in your life your financial means aren't necessarily there so said you know it's you know at a you know five thousand dollar machine is just not people aren't putting their home labs. We get it down to $2,000, bring it down to where we can bring it down.

And we know it's a MacBook Pro, just for perspective. We look around at other products and MacBook Pro. A lot of people buy MacBook Pros, but they buy it for style. I've got one on the desk. Yeah. You know, good machines.

They do great stuff. Well engineered. They're beautiful. I would argue that, actually, you know, machines they do great stuff well engineered they're beautiful uh i would argue that actually you know my own choices uh i don't use them i'm much more attracted to a little bit more utilitarian uh you know sort of kia type laptop uh if i could say that i get it you're more of a slide rule guy i hear you yeah i'm at more of a slide rule but he said can we get in that range let's see where we go and you know our aspirations if we had when we did this and you know we're entrepreneurial and said we'll take a risk on this we'll invest in it uh lots of energy into it uh and uh he said you know if we sold 250 of these in year one we we'd call it, say, that's okay. And, you know, we would have,

you know, I don't have the numbers around, whatever, but we were way in excess of that in our pre-sale period. So I'm going, wow, this is, so we hit something, you know, we're not intending to, you know, we came into this, we said, you know, this is not a product for everybody. It's a niche product. And, you know, and what we want to do is fill a gap that, you know, take that enterprise, take that power, simplicity, robustness. And along with the physical side of it, put that together, boil it down in a home lab and build something that's big, strong, fast that you do crazy data rates out of, you know, you know, load it up with all kinds of things from virtual machines to Plex servers to, you know, you know, whatever else you want to, you want to put on that. Yeah, gaming automation.

And yeah, and, and, and I think we hit it. I think we're there for first product. We had lots more products coming out in this, given that we've got our uptake in that. So we're very, very happy with the reception we got. Great. Just a reminder to our Discord guys listening live, you can put your questions in the chat

if you want to ask Doug anything, or ask me anything too. I don't care. We'll happily put those questions up. I did pre-solicit some questions yesterday that came in. One of the first ones, Doug, was why bother offering a white and a black chassis? The effort to paint these things, wouldn't it have been easier just to make it black and move on? Why did you even waste time injecting style into this thing? So I'm thinking Henry Ford, you can have it any color you want as long as it's black right this is black you know what we asked around

and you know we did a lot we went outside we know we asked reddit and we asked people you know like you know people in the in in our youtube influencer world whatever else what they thought about it and uh just people said you know would it be nice what would you think and and we could so uh you know we can customize because of our you know you know look if we're doing this way everybody else does and go off to a go to an offshore shop and get somebody to make 10 000 of them bring over a container load of them or a couple container loads of them uh you know it's an issue uh our production techniques our mass custom production techniques uh it lends itself to doing that and you know if anything if people like the color is good we'll come up with more so we're able to do

it we thought we'd throw it out there and it's been really interesting because there's been a pretty equal mix on them so good well does it uh actually while you were describing that we got a question about specifically on the cost does the adding color and graphics to the front does it make any much different i mean it obviously to your point if you outsourced them to asia and bought a thousand brought them over they would all be black and they'd all be cheaper but uh i i guess there's a question of is what's the cost impact of the high quality paint and graphics here? Is it significant, not significant? I don't really have visibility into your production methods.

So if you manufacture in North America, if you're a first world manufacturer, your big cost is, you know, what we bear is our labor costs are way higher. We're a big believer in paying fair wage to our people who work here. And they're very much in with variable pay, too, and they share in how we do as a company. And that's your biggest cost is getting somebody just moving a part through a manufacturing station has a way higher cost. The impact of what you put on it, whether you put on garbage, cheapo powder coat, or whether you put on, you know, high quality powder coat, is secondary, you know, in terms of a percentage. And you do the equation and say, we can't compete in cheap. You can't do it in North America. That's why everybody does it

offshore. It's not possible to do it. So we have to make it up by being good by putting quality in it i i i don't know another formula to really do it in north america other than get rid of all the people put robots in there and uh it's not our game we're not into it we're actually very much a people organization so uh that's what we can do so that's what we do do and uh i'm a big believer in free market and when i say free market i think it from the customer's choice first that you know people we're just all better off we have more choices in front of us and you know and and people are free to buy this or they're free to buy inexpensive and you know what we want to do is something that fills a niche for certain people and that is part of it.

So that's very much part of it. So you talked about cost and this has been sort of a hot button, I'm sure you're well aware of this and you addressed some of it in your prior conversation around, what is it? It's 2K for the chassis, the board, the power supply, fully burned in, all you'd need to bring are drives. Is that the, the board the power supply fully burned in all you need to bring are drives is that the that's the highest end right it's a ballpark yeah that's a full building yeah

whatever whatever it is but you've got two at least two steps down from that right where you can get it uh just the chassis or just the chassis and the board you've got some other options to provide some flexibility for people that want to get into this but might want to tweak it one way or another yep oh exactly you know we look at that market you know again we don't have to look very far we just look inside our walls got a bunch of people or their hand their hand goes up or their mouth opens every time we talk about this they got their own view yeah and they uh you know home labs cost conscious world okay number one it just is and and i think it's in in this you know the marketing world says you know they analyze groups of people they say the tech enthusiast the very intelligent uh technology oriented human being they tend to think that you know things should all cost what the lowest mass-produced item costs and that's a

feature of the marketplace right that's sort of a center of that and and and and so it's yeah that's in there so you know you kind of yeah put me put a premium option in there is putting a premium option in there. And it's just, you know, let's get that choice out there to people. And it's got to be real. So we put out a choice. The other feature, and sorry, the place is going. The other feature of that group is they like technical control.

I'm watching. I really enjoyed you guys. You know, just, you know, good, you get it, take her apart, reconfigure it, right? That thing, that's what we like to do as technical people. We like to take control and do what we want. And, you know, for some people it's full build because I want to take control of how I use it and go up a layer into software and function. Other people want to get down that hardware layer. So we looked at that and said, you know, we've had so many requests for chassis and that people have very much have their own idea of what they want to do. You know, some people and we're good. So we set that up as an option for them should they want to build direct wired high speed you know uh you know the elegant simplicity architecture

server uh that's what it is that's what it's for you know there uh and we put that out and it's been good we um i think we're probably you know full builds of our orders you know 45 percent full builds okay 55 one of the one of the other options so which is pretty cool i i love it i i you know that this is people that both have fun with technology also drive the world forward man there's a lot of smart people and a lot of these smart people go to work and move the technology world forward there so they want to get out and tinker and get some of their creative energy out on it love i love it. I'm proud to be part of it. Yeah. I mean, we have this debate every time a QNAP or Synology, whatever comes into the lab.

The the the first the first response is normally can it play crisis? And then after that, the response is that's expensive. I can build it on my own. And in some cases with those prepackaged in all cases with those prepackaged NAS is what you're really paying for is simplicity right and maybe feature set with some of the applications and stuff they have built into the OS but buyers of those devices want to slot drives in hit power and go they don't necessarily want to tinker if they do it's more on the app side. And there's plenty of reasons why getting one of those is a great idea.

This is different than that. This is, if you buy the whole system, this is drop your drives in and do whatever you want with it. And the roadmap is wide open. I assumed incorrectly that we'd be putting TrueNAS on this and using it pretty much as a straight NAS box. But the guys here, as you said, did something entirely different. That's also an option. The Homelab guys will also go buy a $300 server off eBay with a lot of age and wear and

tear on it and then go make that work and tinker and do things that way. There's a wide spectrum I guess is the point. And, uh, and I hear you that, that you're trying to address a certain piece of the market with this solution while also offering other iterations without the board or whatever that people can pick up and go run with. Um, and we actually did get some questions on the board. This is a, it's a Supermicro board X11, I think, on this thing. How did you end up on that as the right choice? Was it purely cost-based? Did you think about going with something more modern?

Is it just a balance of cost and cores and clock, that kind of thing for this audience. Tell me about that. Yeah, cost would be a big driver. We're looking at two grand and, you know, and the volumes that these things are going to be produced at. And, you know, and we put that together with the fact that, you know, our brand big, strong Strong Fast, and we got to keep to that. So you need to get enough horsepower in there to, number one, I mean, number one thing it's got to be able to do. And, you know, one of the things we did, we put a little video, just, you know, hooking it up to one machine, little video editing and do a drag and drop. Good to see 11180 megabytes per second on a drag-and-drop transfer in Windows. Need to get enough board to be able to do that.

And actually, Mitch Hall just did, actually just told me today, he's playing around with multi-channel SMB, and so he's got two lines and two lines in and a simple drag-and-drop in Windows, and he's at 1,800 megabytes per second. So So yeah, you need enough board to do that, right? And and on top of that we wanted to get enough horsepower in here such that somebody wants to put Proxmox on there, you know, and then wants to run a Plex

server at the same time and whatever else. We wanted to put enough in there so they could run all that stuff simultaneously. And we thought this was a compromise and holding to that, you know, our whole line inquiry, you know, where's the price point? Good, we can put whatever you want on there. It's just, you know, you start moving up into server boards and the price goes up.

And people can do that on their own if they wanna do that. They can build the architecture. They can figure out how to get the you know 15 drives you know plugged into it and and and get through the bus and they can put in whatever they want and and yeah it's all there yeah it's just it was a yet at a compromise you don't have the price point thing sitting around and we don't enterprise I mean we sell some really cool stuff in enterprise people get us to beef up things and uh you know if you're a if you're a hot rodder uh you know it's pretty cool but uh we and right now we'll we'll leave people to do that on their own if

they wish to uh put higher horsepower and you know what's in there yeah it'll move oh there's no doubt yeah But it sounds like one of your guiding principles in developing the HL15 was really holding the line at 2K on price point and not drifting above that. Is that accurate? Well, that's

you know, look, this first product out of the gig, we got a roadmap going forward. And we said, let's stay there that's what people told us especially we did our little run at uh at reddit you know just uh you know gingerly stepped into the potential lions den and uh it was actually good we uh you know we very very respectful uh reception that we got in. We got a lot of very good feedback from people. And that $2,000 price point kept on coming up. So we said, yep, we hear you.

We'll do our best on this product number one in here. Well, I mean, you guys know in product development, there's no pleasing everyone. And in this market specifically, it's even harder. So look, I mean, like we said, eBay is there. If you are on a strict budget and 250 bucks for an old PowerEdge that you want to work on and hack around on, there's nothing wrong with that. And in fact, that's the beauty of Homelab. Every time we're doing a live or something and people are like, I get started my advice is always to go on Newegg closeouts or whatever the the big retailer is in your region find one of those small form factor uh Optiplexes or whatever the HP version is those little workstations that often come with a

Windows license and a little SSD boot drive for like 130 bucks, 110 bucks. Buy one of those and start and just do something. Install Proxmox, install Hyper-V, install ESXi, go put a nick in it, just go do something. But Homelab doesn't have to be expensive. I mean, this is just like your car analogy. Driving doesn't have to be expensive. Do you want something a little more fun for autocross

yeah i mean that kia may not be a blast but you know they do have some other models that are quicker but maybe you want a little gr86 from toyota still affordable and more fun i mean i in my mind that's really where i put this this is a premium home lab offering in my opinion and uh but it's it's at its core it's a server and it's really flexible and you're giving your customers the ability to pick this up without having to to get out the screwdriver much uh and wrench on it if they don't want to and i think that's that's pretty fun so uh you know i think there's a fair distinction there there's plenty of swim lanes in in the home lab space and it sounds like I'm sure you're not ready to divulge your roadmap now but you're working on some other stuff that might be pretty

cool one of our favorite things here I'll give you my pitch one of our favorite things a couple years ago for for the home lab audiences HPE's got this little proliant micro server it's a little four bay sata three and a half inch dude um gen 10 plus or gen 10 anyway real little form factor it's uh i don't even know what it goes for these days four or five hundred bucks but again a really nice progression up to not everyone needs this or wants this or can fit it under their their bed in their dorm room or whatever. So I do think a little tiny one that's even more aggressive on price point

would be pretty fun. I'm sure you're considering something like that. Well, you know what, look, I'll tell you where we're going. Real rough, I'll sketch it out. Okay. You're gonna see a thin client come out soon. And there's gonna be an interesting form factor on it on it all uh and uh actually some clients coming out it's not really any pull there it's one of these

things going to go up and i don't know if anybody's going to want to buy one or not uh you know it'll be a a very nice single board linux computer uh i have one in my living room that we built because I wanted something for my network that sits underneath my TV and and and so I got that you know ultra dead quiet fanless anyway that's just a little aside so for fun you just went and build a mini home theater PC terminal basically that's basically it and you know because I looked around and I thought the, you know, there's stuff like that out there, there's plenty of them, but you know,

something that comes with, you know, I got Ubuntu on it and, you know, just, you know, there's GUI and, yeah, and it's just, it's there and it's dead. I wanted something dead quiet and that looks good. Yeah. That sits in there. So we'll be following along that path on there.

And the storage side of it, as I said, that's a little aside. And it's a very small project for us. You're going to see a four-drive and eight-drive spinner machine. And you're going to see a couple of see a couple of of ssd machines in the same size nice and uh yeah yeah that that's the uh that's the majority of what you're going to see over the next little while well and yeah price point we're going to try to keep in there the price point we have right uh you know one of the things uh you know, we look at, you know, the price that we see for, say, a Synology, and they're, you know, they're sort of, you find your discount end of it that'll go lower than that.

But they're pushing a couple hundred dollars per drive bay on it. The machine we have out right now is $133, if I do my math right, $140 per drive bay. That's because you saved all the money on caddies you the the direct drop in you got a big caddy money you save there and actually that was a design question I don't mean to cut you off but I I guess I did you your design is is three fans in the front for the hard drives three fans behind the hard drives and it's uh it's a it's a drop in from the top loading mechanism caddy lists it actually has a little friction uh kind of springy dude on there so you can't drop the And it's a drop-in from the top loading mechanism, caddy-less. It actually has a little friction kind of springy dude on there,

so you can't drop the drive onto the backplane and break it, which is another pretty big deal from a design perspective. But why did you guys opt to go with that versus the more traditional front slot load caddy design for something like this? Yeah, it's our server room heritage. Okay. You put in a drive, a 60 drive machine. That's got four screws per caddy, 240 screws.

And then the, your employer's got repetitive stress injury issues. That's why, that's why when Kevin mounts hard drives and caddies, he usually uses one screw and then he gets like the front right and then just leaves it oh it's all you need uh we we hate that idea it is a big waste of valuable human time so that's our biggest reason for not doing that uh and the other part caddies accomplish this that spring is there yes you don't it. It also controls vibration when it's in place,

self-induced vibration. And that whole issue, if you want to engineer a really, really robust storage server, get the power right and get vibration looked after and get thermal looked after. We very careful, a lot of engineering time into those. We actually got a video our Brett Kelly did a little while ago. We got another one coming out about how to handle hard drives. And there's a couple of motions where if you just pick two hard drives up and click them together, you can damage that. There's a certain axis that they're extremely sensitive in and won't necessarily kill the thing but what it'll do is it'll shorten its lifetime dramatically so we set them up that that's why because you don't want to drop it in and you don't want it vibrating and uh yeah handle them carefully and they're the most

incredible things don't handle them carefully you get problems so anyway that we've seen yeah we've seen plenty of those that'll come in with a little a little dent in the case and you don't think it's much but but it doesn't take much. They're precision instruments. I mean, inside there, you've got 10 platters spinning at 7,200 RPM, right, and little swing arms that have got to go out and all the heads and everything. I mean, they're really remarkable inventions. And actually, the podcast that's out now, the one right before yours, was with Seagate talking about hammer and getting 50 terabyte hard drives again similar design but using the uh the the heat assisted to fire that little track up right the data and then instantly

cool it off I mean that the the run you know I posted a tape video on social yesterday tapes dead tapes dead hard drives dead yeah Yeah, yeah, yeah, yeah. I mean, this stuff's going to be around for a very long time. There's nothing. Flash is great, and we talked about that, but there's nothing on a cost-per-terabyte basis that can beat this stuff for hundreds of use cases. Yeah, and cost-per-terabyte,

the other thing that when we look at Flash, and we're fans, I mean, you know, the right that uh you know when we look at flash and and we're fans i mean you know you know the right tool for the right job and flash has its place there's some workloads that flash is awesome for uh but there's other ones on a on a cost per terabyte but also in lifetime we're doing a little project that we'll share with you at some time uh where we're just we're torture testing drives we're just writing them fully and then erase them writing them fully erasing them and uh you know it's a lifetime thing and we said let's just do consumer and enterprise then we threw flash in there just

for fun knowing who's going to die first the flash is is is not going to last and in fact they're they're now gone they're they're essentially dead they're like a drunken man wobbling all over the place right now barely barely standing and both the consumer and the enterprise are going like like full blast nothing so if you have a right you know it's got nothing to do with home usage because you just don't write that much data right but when we get into our enterprise you know any any you know a lot of professional workloads that are right heavy. Yeah, you got to watch because you're going to be replacing your flash

on a regular basis. And the worst part of replacing it is you kind of get to a period where it's starting to get fuzzy on you and then it just bogs everything down and you have to make that decision to pull them out and the time to let everything,

you know, readjust and rewrite your redundant data and everything else. You speak like a man who's had eye surgery before. I had laser. That was about what it was. It slowly gets foggy until you don't realize it. But then when you fix it, you're like, oh my gosh, I had no idea my vision was so poor. You talked about wobbly and you talked a little bit about taking care of hard drives. We got a couple of questions here about taking care of the system itself. How do you characterize support and warranty for something like this? And I guess specifically, there's a couple of questions about if something happens, do I ship it back to you?

Because that's, I mean, I guess that's a plan, but it's sort of a drag when you get into heavier things like this that get expensive to move around the planet. But can you talk at all about warranty and support and what you guys are doing for that for the HL15? So we know our warranty, we have an expected defect rate. And it's really low, our experience in enterprise. I don't think home labs will be any different. People don't know what they're doing.

I don't think there will be any big abuse-induced problem or anything like that. So here's how it will work, and it's very much different from enterprise business. Our promise to you, if you buy one of these things, you're going to get a working machine. It's going to be built to a very high standard, and the components will all work for you for the warranty period. If they don't, you've got a hold of us at our email, our 45-HomeLab support address, and somebody will get in touch with you from our service team. We will diagnose it together. We'll ask you to do a few things to figure out what's wrong and we shall get new parts replaced. Sometimes we'll ask for them back.

Sorry, if you wish to replace them yourself. If you want to get the parts, and most people prefer that because it's a quick FedEx run and you can get up and going in very short time. Most people in home labs, most of our enterprise customers do that, too. You know, something goes and they go, yeah, send us a new one and we'll plug it in. If you don't want to do that, good, you get it back and you get it here and back and it'll go FedEx. It's on our nickel. So we owe you a working machine. We do not support application level in Homelab. It's built into the price side.

If you're enterprise, you pay for the support. They come pre-configured, but we don't directly support that. But we've set up our forums. And our forums, what's been happening is we have an active community on there already. And there is mutual support in the community. Plus, we will be regularly, like daily, monitoring it from personnel in here and adding what we can to it. So, you know, one of the things that we can't support within the price point, not possible. Somebody does a home build and comes and says, I can't get my hard drive to show up in Linux on my home build, obviously. But come on to our forum and ask in our forum because there's a community there. And that's what's lovely about open source and Home Lab and whatever else.

We love that. So we set that forum up for all that application level stuff. Come on in there, post your stuff. We will be doing passes in there. We're not going to answer every question or anything like that. It's not possible. But where we can add or unstick something, we'll be doing passes in there we're not going to answer every question or anything like that uh it's not possible but where we can add or unstick something we'll be in there well i would i would say on that note that as i'm sitting here looking at ours the component count here is is not high i mean you guys have put together a pretty simple board and if i've got to get at these fans

or need to remove i mean it's a couple screws i mean nothing nothing in here isn't is what i would call complicated i've seen some of our pc builds that look um much less tidy than this if you want to troubleshoot something so yeah i mean i i i get your point and uh and and wanting to self-service it if you can i i think that that's a an interesting angle and probably will be a popular one and of course people are going to tinker with this on their own they're going to take the fans out they're going to put in whatever they want they're going to we had a guy uh in the chat while we've been talking that wants to put a liquid loop in here so they can make this into a true gaming mass. I don't subscribe to that idea.

I think it's a terrible idea. Just because our guys did it doesn't mean it should be done. But we've got guys in here that want to run liquid loops and put a full GPU in here and do some crazy stuff. So, you know, they're saying it's a great idea. Damn it. This is the chat also in the chat just so you know you're getting a lot of support for the drop-in uh caddyless design the number of uh of screws we've dropped around in in here and uh that reside under the racks behind me there must be thousands um a little screw army underneath underneath the the racks waiting to attack but

you know I think it's pretty neat and truth of the matter is is even home labbers aren't swapping drives around all that much it's so I don't know that it's a big ask to to take the lid off and one other thing that I liked about it and I this is so basic but the little captive screws two thumb captive screws on the front it's just another thing where you know until you do it a lot and and think about the value of some of these components that this probably costs i don't know six cents or something but a regular screw is half a penny and so when you look at volume servers a lot of them don't do that but we've had problems where you drop a screw on the board and you don't know it's there. And you go to boot and now all of a sudden you've got a fun electricity lesson.

The more pieces that you can make captive or remove from the system, I'm all in favor of. Look, I've spent thousands and thousands of hours building electronics building boards from scratch screwing stuff together working on machinery working on computers and uh i've got the cuts from the jagged edges i've got the drop screws and the little funeral pyre white smoke resulted and oh yeah yeah yeah been there done that well that's that's why they're part of why they're built that way. Myself and a bunch of other people, my business partner, that's our background.

And yeah, try to keep, inflict as little pain as possible. Yeah, yes. Once you've been there, done that, you realize the pain and watching your investment go up in smoke is not rewarding in any way. But yet it happens all the time. Well, I know we're a little over on time. We've done an hour and I feel like you and I could probably just keep chatting about

technology and hard drives and servers for another hour without much trouble. I don't know if anyone wants to listen to that, but if they do, we'll come back and do a follow-up one. For now, the website for people that want to learn more about the HL15, that's hlhomelab.com. Did I get that right? Sorry, the HL15, 45homelab.com. All right, 45homelab.com. We'll link to that in the description here. Any last words of advice for anyone thinking about this? What's your last sales pitch?

Well, you know, my sales pitch, I'm a believer in choice. You know, want to buy one of these things, premium hardware, buy it. We look after our customers, and, you know, we to buy one of these things, premium hardware, buy it. We look after our customers and, you know, we've done our darn just to build something that's worthwhile, not for everybody. If it's not right for you, good. We understand. If you want to buy it, come on in, join the community and we'd love to have you. That's all. And yeah, that's it. All right. Awesome. I appreciate your time. These guys want us to do a giveaway giveaway so we'll have to think about seeing if we can enable that uh on on discord so think about that while while you go back and report back to the crew how awesome this podcast was but again we thank you for your

time i really appreciate the insight and uh learning more about the background of 45 drives everything that you've been involved in and uh yeah this thing's awesome we're we're gonna keep cranking on it and finish off our our review. We've got actually a couple other ideas for this thing. So you tease your roadmap, I'll tease mine. And we've got some more plans for you coming up. But again, thanks again, Doug. Really appreciate it. Thank you. Can't wait to see what you guys do with it. Looking forward to the next episode and come visit us sometime. Got it. All right. Thank you. Cool, man. Thanks, Brian. looking forward to the next episode and uh come visit us sometime got it all right thank you cool man thanks brian