Podcast Archive - StorageReview.com - Podcast #128 – The Best Storage for Veeam with Object First

Episode Date: June 1, 2024

For this show, Brian sits down with Anthony Cusimano, Director of Technical Marketing, at… The post Podcast #128 – The Best Storage for Veeam with Object First appeared first on StorageReview.com.

We are live on the podcast. I've got a very good conversation, I hope, today about the state of backup and recovery. Some of the new stuff, some of the tie-ins to what Veeam's doing with their Object Direct in V12, and these guys, this bezel has garnered more attention than any other bezel we've ever had in the lab before. Every time we make a video, someone wants to know what's the orange thing over Brian's shoulder. Anthony, you're here to explain it to us. You're the evangelist over there at Object First, but you've got maybe a more diverse role. Why don't you tell us a little bit about yourself? Yeah. So, you know, Object First started as a startup. I mean, it still is, but we have grown tremendously over the last few years. I think I was employee or U.S. employee

number four when we got started. So that was even before we were public. And, you know, when you're at a startup, you wear many hats. Sometimes all of them. Yes, yeah. When I started, it was almost all of the hats. And I've since taken a few off, but I put a few more on. So I do a lot of everything here. And it's been a great time. I've been at the company over two years now, and I absolutely love it. It's just a fun place to be. Well, I mean, this is funny, though. though i mean you've only been commercially shipping for correct me but call it a little bit less than a year maybe is that that's right okay so we we launched in february 14th of 2023 that's the first day you could buy a new fee okay and the funny thing though is you say startup but i've known you guys for so if you've been shipping a little less than a year at least three years because you guys have been showing up at at Veeamon early days from a

concept where you were showing an earlier version of this hardware early software I mean this has been you know a quick overnight three-year success or something right I? I mean, it's been a bit of a burn to get here. My first week on the job, which was in March of 2022, I started and my boss said, we're going to Veeamon in a month. I need you to make all the technical messaging of what we're going to say and do in the booth. I was like, okay, let's go. Let's have fun.

And at that time, it was an opportunity to come out of stealth. That's better than most people that don't know VeeamOn or how to say VM or whatever it is in the first place. They still get people that can't pronounce Veeam. So you were a step ahead of the game already. Yes. I did not come from Veeam. I actually worked for a Veeam competitor previously, but intimately familiar with the technology and I was ready to go. Well, good. So VeeamOn, as we're doing this live on YouTube, is next week in Fort Lauderdale. This should be your biggest event ever, right?

I mean, I know you were at the last one, but each one subsequently is probably bigger than the last, I would guess. That is correct. Every year, we call it our Super Bowl internally. It is the biggest event we can plan for because it's our customer base. I'm sure, as we'll discuss, we only sell for Veeam customers today. And this is our opportunity to get to spend as much time humanly possible with as many of them as we can in a single room. So we're going bigger than ever. Every year we go bigger. And this year, I think we've got three different speaking panels. We're giving away more prizes in the booth than ever. Every year we go bigger and this year, I think we've got three different speaking panels. We've got, we're giving away more prizes in the booth than ever before. We have trivia. We're also going to do a big launch at the event.

So it's a lot that we're planning for and I'm excited for it to happen. I'm also excited for it to be over because I need a little bit of a break. So you said launch and I really hope I don't screw this up because we're recording this, but we're also live. So I'm going to be really careful to not bust your embargo because sometimes I know things, and when we're having conversations, it's okay. When they're public conversations, then I'm going to try really hard. I can't promise it,

but they shouldn't have maybe briefed me before we did the live anyway that's very exciting the events down in in fort lauderdale and anyone that's uh in our audience that's going to be down there checking out vm on definitely check these guys out at the expo and the uh the multiple panels um you said your vim backup which is you know true you guys really came out as V12, rolled out the direct-to-object. Let's talk about that piece of it a little bit, because I think that part's important, the object-native nature of what Ootbee does and Veeam enabling that direct-to-object technology. Absolutely. So it's no coincidence we launched on February 14th of 2023. That was the same exact day Veeam came out with version 12, which had the direct-to-object support. So a little behind-the-scenes baseball. You know, we had talked to Veeam as technical alliance partners even before we were selling OOPI.

And we wanted to be part of that launch specifically of that API. You know, OOPI is an object storage device. You can write to it via S3, but that protocol specifically interests us. And we built OOPI around that, knowing about it in technical preview, because we knew that leveraging that could mean better management and orchestration of data,

as well as more speed on our box. So we didn't want to ship the product until Veeam had version 12 out, because if we did, it would just be traditional S3 calls. And that would be slower than what we wanted to put out in the market. So we had to wait for them to launch. And I think it worked out all for the better because they had a fantastic launch and it allowed us to show our best selves at the gate. Yeah. And I guess there are some others that had some S3 integration, but this is much deeper than that.

And actually, we've got a video we'll be dropping hopefully next week. We've already published a detailed, gosh, it's a deep dive how-to and what we did with this box. I'll make sure that we've got that link in the description. If you want to learn more, read more about Ootbee, we've got a deep piece on it. We've got a video coming. But your fundamental architecture there also hits on a lot of security topics, which is why you went with this architecture. Maybe talk a little bit about Zero Trust,

some of the other security topics that are inherent to what you do with Ootbee. Absolutely. So, you know, to kind of understand why security is so important for us, you know, to kind of understand why security is so important for us, you almost have to go even further back than when I joined the company. You know, Ratmir and Andre, our founders, who also founded Veeam, let's say five years ago, were noticing just how pertinent ransomware was when it came to backup attacks specifically. It was all these, the latest

cyber attacks were going after backup vendors and backup storage now because they knew if you can't recover, you're far more likely to pay the ransom because that was sort of the scapegoat at the time. So they started ideating. And the funny joke we all tell is Ratmir and Andre said they would never, ever get into hardware, but then they see the problem lies with the fact that storage hardware was just not secure enough or up for the task of ransomware. So what do they do? They start a hardware company. Moving on from that thought, everything they did when they started designing and building this device, even the concept stage, was focused on security. They knew they wanted to sell to Veeam specifically, but they also knew they wanted to deliver an experience that really doesn't exist out there today

when it comes to a secure backup storage target. And it's interesting we put those three words together because I think a lot of storage options out there, and you know, you're the storage whiz, I'm just the fun guy on the other end of the call. They try to be everything for everyone. And what Ratmir and Andre wanted to do was build something that was

very specifically designed, very narrow in scope for the VM use case. The first thing they did was say, we're going to do our own proprietary object storage codebase. We're going to lock out any administrative privilege to that operating system, codebase, hardware itself. There's no admin access. We say zero access to root, but there's no operating system, code base, hardware itself. So there's no admin access. We say zero access to root,

but there's no operating system back in anything. And then they also said, we're gonna stay up to date when it comes to, ensuring the penetration testing is done. Third party security testing and validation. Every little aspect of this product has to be secure from the second it comes out of the box to, day 40 of holding your Veeam backups.

If the immutability was ever compromised, then the entire purpose of what they were trying to do was lost. So it was an engineering effort. And then from then on out, it was basically just our motto is to build something that's secure, simple, and powerful and continue to deliver on that. Well, I've got two anecdotes on that. So as you know,'ve got I've got two anecdotes you know on on that so as you know we're hands-on we have we've had this thing in since before commercial launch I think so it's been in our lab for for a very long time and when we first set it up I everyone says you're the evangelist guy everyone says oh it'll be done 10-15 minutes no big deal everybody says that and then we go

and do it and it's like the networking or the thing or it's got an update or it's you know the wrong key or it won't load or whatever and now three days later you're still screwing around with it so Kevin comes in here to the lab has a an hour scheduled with one of your your tech guys he's got it racked it's powered it's networked and they start going I said okay well you're starting now that's fantastic I'm gonna go eat lunch and I'm just gonna go downstairs hit the break room grab lunch and then I'll come back up catch the the back end of this thing so I come back up here I walk in the lab and Kevin's not in here and which is normally a bad sign I go down to his office he's like oh yeah we're done like what do you mean you're done he's like yeah it was like

nine minutes or something and and the thing was operational and uh yeah so when you say 15 minutes you might be overplaying it in terms of the length of time required to get this thing operational but it is real and we're we're not backup and recovery experts. And this next story will reaffirm that. But we play with all of this stuff all the time. And so we're very much tech generalists in terms of what we cover. This leads to the second point, though. When you talk about immutability, that's the I in OOPI. I think it's a powerful word that not everyone understands. And I'm going to be honest, when we set it up the first time, I'm not sure we really understood it either because Kevin's like immutability. Well, that sounds good. Backups,

that sounds good. Sure. Let's make it all immutable. And he threw it all into this bucket and we were doing full backups to try to put some load onto the system and three days later i think we had filled the entire system which isn't great when you've got a 28 day mutability flag on all of that data yes but but to your point there's no root access to log into this thing, easily blow it away, refoundation and start over. The best thing that we could do, that we had two choices really. One, get a new system or two, wait 28 days.

And we just waited the 28 days for that flag to expire and uh it's not i can't quite tell if if my story is an endorsement for immutability or or a cautious tale or both maybe at the same time but yeah i mean you talk about security the system really is secure uh in terms of access and and the immutability flag really works can confirm yes yes it you know it's funny. Your first story, I got to experience that a lot in early days because we didn't have any sales engineers or anything. So I'd get on these hour-long demo calls that were scheduled and I'd demo the entire product in under 15 minutes. And I'm like, so what do you want to do the rest of your time? I don't know how to fill this time any more than with what I've shown you right now now." I got really good at just making up words and

just rambling on because it's too simple. Part of that is because when you secure something and you purpose build it, you want to remove all the optimization and options that a curious user might go down and start toggling switches and flipping things on and off. It made it very easy to make a incredibly simple procedure. But on the security end, it is one of those things that we caution people about now because you're not the only ones who've ran into that. And it's

a blessing and a curse, right? Immutability in compliance mode means that it's not in governance mode. There is no governor. And if you write too much, you will sit with that data and there it will be until the flag is out. Now we've realized we need to put more education out there, have more conversations with our customers when they go through the initial buying process just to understand that. Do a little math on their backup chains and sizes. But it is one of those things where you'd much rather have it be immutable and unchangeable than to find out that someone's factory reset your box and you've lost everything and are now in a negotiating situation. Yeah. And yeah, that's an important point. And I think when we went through it, to be fair, we were trying to fill it as fast as possible. So we were

doing some abnormal behavior, so you know nobody does full full full full you know right in in a quick succession on their their vim backups um but we did we filled it and uh and that was that uh and then we extended the uh the loan on it by 28 days to continue to wait for the flags to expire. But you were talking about something in terms of the setup, the switches and the simplicity of the decision tree to deploy it. But that carries through to the GUI too. I'm not a storage practitioner, but I can log into this and administer just about everything with very little risk of me breaking anything for one, but for backup admins to see this,

this is going to look really simple. And for IT generalists that might be tasked with backup and recovery duties, I think it also should be a pretty seamless operation. What are you hearing from your customers in terms of the ongoing day-to-day usage, GUI, that kind of thing? Well, once they get their hands on it

and actually run through the experience and realize like, I mean, usually they've seen a demo, they've either done a beta or tried it themselves and they're like, this is really cool. The, we try to delight every customer and give them the least amount of heartache and pain possible. Just how do we make your life easier than it is today? And I think we've excelled at that.

But it is funny. I went to Wiemann in Munich, Germany last year. And I'll tell you what. I have never been asked more difficult questions in my life than what the Germans brought to me as far as like technology and orchestration and all of that is concerned. I just, I wasn't ready for how intense and dedicated the Germans are to backup and recovery. They would almost get a little upset with me when I tell them, no, you can't do that. You can't change that. That is baked into our product. No, you don't have the administrative privilege. No, you can't go and change the way we ingest data. It's always going to be S3 over direct object via the SOS API. And you'd see the gears turning

because it's a sacrifice of control at the end of the day. These admins, their entire working lives have basically had to be the decision maker for every technical question and problem that's ever come their way. They expect root access. They expect super user privilege. And we take that away, we take away all of the customization and optimization because we've done it ourselves and we know the operating environment it's going into. And once the gears click and they realize they're not trying to insult us, we're not

trying to make you, we're not saying you're dumb or you're not an expert. We're saying, buying back your time. We're trying to make your life easier and you can just have this experience and it will just work for you. Then they get excited and they see exactly why we're doing what it is we're doing. It's twofold because no one wants to administer another or administrate another box. And no one wants to go take another security course and learn how to make these things more secure than they already are it's a nightmare in both situations so we just said well we'll take the easier path of just make it work for them and make it worth explicitly for this uh this specific end user

and give them no problems so talk a little bit more about security. We're getting a question on the YouTube live. And for any of you that are catching this after we've already live streamed this, we're doing the pods live on YouTube. We're taking your questions. So if you want to interact with our guests, that's the best way to do it. And if you know we've got a pod coming up

and you can't make it, send in the question on Discord or email and we'll take it there. This question, or a little bit more of a statement it would appear about concerns over security bloating data and storage. This user is suggesting that security should be peripheral and light to the system and not add a bunch of bloat and complexity to the system. When we talk about security in the context of Ootbee, when I'm using the system, I don't feel it, but maybe you can talk about the intent

of the way you've implemented security and secure policies on this system. Yeah, it's completely by design. We say no security expertise required, and we mean that. We don't want you to understand or have to go learn something new. We've been putting out this whole motion in the last few months, basically since November. It's a research paper we worked on collaboratively with Veeam. It's called Zero Trust Data Resilience.

And if you go read the research paper, you're going to see a lot of security talk and specifically towards backup and backup storage and what that means. Because frankly, I don't think the security situation is great when it comes to backup and backup storage. It's you hear zero trust and you immediately think production apps and infrastructure, because it's kind of what the framework was designed around. At least privilege access, policy endpoint protection, policy decision,

policy detection points, policy, like it's all about, I don't trust you are who you say you are. And I don't trust your app is what your app says it is. And every step of this conversation with other apps and services, we're going to verify explicitly, we're going to gate everything, we're going to limit you from being able to do anything. And it's a great principle, but it was not ever broadly applied to backup. So what we've built with BOOTB follows very closely with the Zero Trust Data Resilience Framework, meaning it's separated from the backup storage or the backup software, I should say, and the backup infrastructure, which if Veeam should ever be compromised, if there was an attack or there was an infiltration and it got into your Veeam server, which is on Windows,

we would not be impacted by that because just the natural separation segmentation. Now, from a technology side, there is the additional, you know, what our developers are doing on the backend, right? We third-party test and validate this box as often as we can. We're trying to, we're not trying to, we are staying up to date with zero-day exploits.

We're ensuring that even though we don't give access to anything, if you try to brute force through the text user interface or you're trying to get through SSH, all of those methods and routes of communication and potential devastation are not just minimized, but eliminated. We're constantly thinking about what does the end user actually need to use our product? And in truth, it's two things. You need an S3 key and an S3 bucket. And that's truly all you need to connect to Veeam. So that's where we focused on the user experience. You can type in a name, click go, and you have an S3 key. You can type in a name, click go, and you have an S3 key. You can type in a name, click go, and you have an S3 bucket.

And you copy paste those to Veeam, you're done. The additional securing, if we ask the end user to do it, then we'd be reliant on their security expertise. And we just know there's not enough time for that, especially for the backup admin with as crucial as that role has become. So it's all abstracted. It's all done behind the scenes on our end. And I'm very proud of what we built. I can't say everything because there's obviously secret sauce and mix here, but it's not something that's one and done. I would say cybersecurity is just a continuous practice forever and ever. Even the biggest tech companies fall victim to day zeros. And no one is an exception to that. But it's the mentality of the practice, right? Like accepting

that you have to stay vigilant. Things will go wrong and you need to be able to quickly address them and minimize the damage as much as you can. That's how we implement the Zero Trust Data Resilience Framework into what we're building. Well, we are lucky or not, depending on your perspective, to be less than a two-hour drive for Rick Vanover to get to our lab. And dude, there's no more vigilant guy out there pitching ZTDR than Rick Vanover. And he was down a couple of weeks ago talking about it. He came down to help us upgrade our Veeam software. I don't think we really needed help. I think he just wanted to come down and beat on this drum of zero trust and try to get us marching along with him on that.

But as much as I give him a hard time, he's absolutely correct. And having some gaps between infrastructure certainly mitigates a good deal of the ransomware risk. The thing about the Zero Trust Data Resilience Framework that I love, as someone who's a little bit more sort of activist-minded, it's free, right? We're not trying to peddle or sell anything. At the end of the day, you could go with Veeam, you could go with any other vendor, you could go with Oopie, you could go with any other vendor. The principles that we're trying to get across are universally applicable regardless of your software or your storage. If everyone goes and reads the research paper and just applies a little bit of it to their data center setup, they will find themselves in a far more secure posture than they would had they not done it. And a lot of it feels super obvious after you read it. It's

just, oh yeah, that makes sense. But I think it was the fact that no one ever wrote this down. No one ever thought, hey, I'm going to actually take the principles of zero trust and extrapolate them into backup software and backup storage. And just through doing that, I think we've done a lot of potential net good for the industry because it is something that I think everyone needs to be cognizant of, think about, and go educate themselves on. Yeah, I mean, the 3-2-1 concept has been around for many, many years at this point in time. And this is an extension, right, of some of those initial concepts of how best practices to manage your data. So that's good.

You talked about updates a couple of times, and I want to talk about this in relationship to OOPI as well. That's one of the most interesting things to me is that with a backup appliance, I think the industry is pretty used to quarterly updates, maybe every six months, sort of depending on the vendor. Obviously faster if there's some sort of issue that's immediate. But you guys have cranked out updates on a pretty progressive rolling cadence, which is interesting. It's a little bit, I don't want to say counterintuitive, but it's a little different than the traditional purpose-built backup appliance industry. Can you talk a little bit about the update cycle and what you guys are trying to achieve there? Absolutely. And I think next week we'll be revisiting this thought a little bit. If I can tease some of the things that we'll be launching at Veeamon.

What? A possible update at Veeamon? A possible update. You're absolutely right. And I think part of it comes from the fact that as such a new product, and I mean, this product is new, right? We literally designed this within the last five years. So it's all from scratch put together. I think that's true for a lot of newer products in general.

If you developed with like a lean startup or an agile mindset, there is the idea of the sprint cadence and delivery that comes along with that. But more importantly, we're responding to customer feedback, right? Like if we shipped out four OOPIs to a customer and found out that when they hit 98% utilization, something went wrong, which this is just a hypothetical, this didn't actually happen. We try and respond to that as quickly as humanly possible and work that up our stack of importance. We just brought on our CPO, Chief Product Officer, Eric Schott, recently, and he has been helping us just wrangle in and deliver absolute value when it comes to the product. And the one thing I'll say is we get a lot of feature asks.

A lot of different people want a lot of different things. Which is the hard part, right? Because you've got to curate it is. Yes, you can never you can never please everybody. So there's always going to be a sacrifice. But I think the really smart thing we've done is focus on one, ensuring that, you know, our three pillars, which is secure, simple, powerful, never take a hit because of some bigger idea, right? Like there are things that we're cooking that you're not going to see next week. And it's because we're still cooking them. It's because they are not ready for prime

time. But the things that we put out, it's always focused on ensuring that there's no security compromise for our customers. It's ensuring that if they need an extra feature, that it's applicable to the use case. And it's not sacrificing something on our side of simplicity. We're not creating more complexity for them to manage. It's always, it's an enhancement. It's applicable to the use case, and it's not sacrificing something on our side of simplicity. We're not creating more complexity for them to manage. It's always, it's an enhancement. It's not just a feature request, right? So it's unique because I think in specifically hardware, you don't get a lot of interim updates.

It is quarterly, yearly, more likely yearly. And we're gonna continue this cadence, because again, we're a startup, we are fast, and our product is so new, we haven more likely yearly. And we're going to continue this cadence because again, we're a startup. We are fast and our product is so new. We haven't reached that. It's spaghetti code yet. And you fix one thing, you break 20 others. Fortunately, we're still in the young phase where we have control of everything. And I think another advantage we have working for us is that narrow scope again, right? Like we're not testing against all the other backup vendors out there. As long as it works with Veeam, and that's the only thing it needs to work with,

then we're good to go with it. So it makes testing in QA a lot simpler too. That's a fair point. If you've got a supported dozen backup software vendors, then your quals are not just for one guy, they're for 12 and it, you know, 12X is the effort. The one thing I will say about the updates that's been really interesting to us is that we just did one today. We left it un-updated for a while to make sure that we had one for the video we were shooting. And we were at 148 something and it was up to 150

something uh it hit go and it spins grabs the downloads installs that process took four minutes maybe and then it's like bloop you're done yeah what no reboot, no, no, you know, lack of availability. I know not every update, you know, is like that. And sometimes you'll have to reboot the system. But for a single controller system to not have to kick over and reboot every time you run an update. That's kind of cool. Yeah, and it fortunately, this one was a friendly one obviously some you might have to you know in the past we've had to rebuild the the raid which did

take some time and it meant some downtime for the device so they won't always be so pretty i'm not i'm not so uh naive to think that it will always be a nice and easy process, but that's the goal, right? We don't want to make more work. Truly, the thought process is deliver value, deliver it simply, and ensure that if you're spending more time with Ootbee than you are administering Veeam, or just going off and doing everything else in your job, we've failed, and we need to improve our situation.

And thus far, we've not found ourselves in that situation and i think having that in the back of our minds ensuring that our customers stay delighted that's that's really what keeps us going and keeps us focused on doing those kind of updates yeah well that's so that's the other thing that's interesting is i don't know what period of time that was six or eight weeks maybe that we didn't update but it's also because we didn't have any challenges like it was just sitting there doing its thing and we happened to log in and be like oh it's got an update let's let's get that and then capture it for the video so it's been you know in terms of delighting customers just it's just been in the rack chilling doing its thing for the last you

know nine months or so and you know relatively unremarkable, which in backup and recovery is a wonderful compliment, right? I mean, just not doing anything wrong. But that's been a great experience. We had another question come in about target audience. Let's talk about that a little bit. Where Ootbee is targeted in relationship to where Veeam is targeted, what does that overlap look like? Are you trying to be everything to every Veeam customer? Can you dive into that a little bit in terms of market segmentation and how you guys are thinking about where Ootbee is a good fit? Absolutely. So today we're chasing what we call the mid-enterprise.

It's not the large end. Well, firstly, we can't because we do have some limitations, which I think is worth talking about. Right now, our limitation is half a petabyte of usable storage. That might change in the near distant future, but right now it's half a petabyte.

So if you have a cluster of four 128 OOPs, you got half a petabyte you can use. You can have multiple clusters, but that's independently managed. So we say mid-enterprise because we know they have a little bit less than that amount of backup data. There might be some that have way more and there might be some that have way less and that is why we have the 64 terabyte option as well and you don't have to build a four node

cluster. You can just do 128 or 164 or however many you want. And you don't have to build a four node cluster. You can just do one, 128 or 164, however many you want. And, you know, in all honesty, like right now we're chasing the sort of typical IT infrastructure backup admin. There are, you know, SLED when it comes to education, they love what we've done because of how seriously we've taken the immutability and how uncompromising it is. You know, when you look at other entities out there, let's just throw, you know, finance or fintech or, you know, big government, healthcare. If they need special certifications or if they need to know that we have special compliance features built in today, it might not be a fit. I shouldn't like I would never say let's not have a conversation. We should always have a conversation. But if a certain certification is going to hold you back from buying our box, then it might not be the right fit or right time for you. Right now,

we're trying to get that mid-enterprise user that needs the amount of backup storage we're offering. It needs it simply and needs it quickly and wants a little bit of power behind it. But we're not so naive to think that we can just solve everyone's problems with a box. There's always more work to be done. And it's one of those things where, again, I'm glad we have our new CPO and he's doing these evaluations and we don't have to, but where we go next

and the users we try to wrap into our fold, that's going to be based on the kind of conversations we have with the folks who are watching this podcast, right? Like we want to hear from you. What do you want? Are you able to use this today? And if not, why? Like we want to have those conversations and understand more of the use case where we don't fit and see if that's worth pursuing or just follow the course we're on. Yeah and you know it's interesting because to this point you know Veeam does a lot but now as you're well aware the upheaval in the hypervisor space there are a lot of questions about you know where I mean there's no sugar coating it the renewals on

vSphere are very, very high. And a lot of organizations are considering other alternatives, whether Hyper-V or now with Veeam announcing future support for Proxmox. That's really interesting. And that might be a little more SMB. But with a 64-terabyte appliance, I mean, you can still hit some relatively smaller size mid-market customers with that if they do go to Proxmox or some other hypervisor. Absolutely. And that's kind of the funny thing too, is I was talking to the head of our aces,

which is our community, Jeff Burke, and he loves just dabbling in everything. And he was dabbling with Proxmox just a few weeks ago, just to kind of get his own feel for it and how it works with Veeam. And, you know, I was asking him, like, what does this mean for us, right? Like, you know, is Veeam going to shift to this or, you know, what? And, you know, he has his opinions. But I think what's important to note is there is no shortage of data. And I think the funny thing is at the end of the day, 64 terabytes might seem like it's our smallest box today and it might seem too big for even some of the smaller organizations out there. But just look at how big files are getting, right?

Like the growth of data has never slowed. What is it? Moore's law with CPUs. We are reaching that threshold of, I don't know if I can actually overclock my gaming PC any more than it has been for like the last five years. And I don't expect to get that many more hertz in the next 10. But with storage, Moore's law just doesn't apply. The amount of storage or data that just gets generated by individual users,

let alone businesses, is astronomical and we're not slowing down. And 64 might be too big today, but it's definitely not too big five years from now. So I'm kind of curious to see just where the industry goes and where we go as users. I was shocked this morning. I pulled up my MacBook and looked at just how much storage I was using. I was almost at a full 800 out of a terabyte used which it felt weird to me because i don't i don't feel like i have that much data on my laptop but if i don't feel that way but i do yeah you know it doesn't change the fact that it's true and i think a lot of folks will find themselves that

way well it's funny i was at a uh semantec event back i know exactly where it was it was the mgm hotel i can't remember if it was 2013 anyway it was a long time ago and that that was uh an interesting one because that was one of the first times i started to hear a conversation about organizations we should be thinking about what data you have and what data we can delete and And the concern back then was a lot about compliance and holding data too long from a legal perspective. And what are we ever going to do with it? You know, why would we want it? That kind of stuff. Now, fast forward to, you know, this year, last year, even the year before, you know, business analytics, business informatics, before it was called AI. All of these things depend on having all of the data ever, because especially if you had retail,

I'm here in Cincinnati, I got Kroger, Macy's headquartered here. All of that data is so valuable if you've got the right algorithms or AI models to take advantage of it. And so now organizations, as they embrace this, if they haven't already gone all the way in on AI, they're never going to want to delete a file ever again. Never again will anyone want to delete anything. And that's going to put a lot more stress on on the backup and recovery part of the house don't you think i completely agree i'm also just i want to applaud you we got 30 minutes in before we even said ai which that's a victory that's a victory for anything well we've got minor burns from it though because every every pitch we hear leads

with ai so i like to stuff it in the back you know i'm good we can talk about it in the middle but you know just we lead with that i'm out of here so i'm glad it took this long and to your point it's it's absolutely true the the unfortunate truth i i for some reason i dabbled with jim and and I a couple of weeks ago and built my own little AI just because I wanted to just mess with it and see if I could. And the more I was reading about it and the more I was learning about it, the more I realized it's not just lots of data, it's lots of good data. And how do you know what good data is?

Because as a business, businesses are going to find themselves biased to what they think good data is. And they're going to use third party analysts to look at their data and say, oh, no, your actual good data you want to train your machine-learned AIs with is this. And it's just like you said, you can't throw anything away because it might actually be useful. So you have to keep all of it and you have to continuously evaluate it, churn through it, reuse it, retrain on it. It's all potentially good data and it's all potentially garbage. And unfortunately, that means that there's just a lot of it. And I love the point you made because not only is the size of things just getting bigger. I mean, just look at videos on YouTube, right? Like

people upload 4K videos now and those are not small files. 4K five minutes is what, like an hour of 720p. So you're just, we're in a different world entirely when it comes to information and spacing. And it's only going to get worse because now we can't get rid of anything. So it's going to be a real problem when we start to hit just absolute stupid amounts of backup data. And how do we adequately protect that? And the better part of the question is, if something happens to it, how do you recover it? Because when it gets so big, it becomes unwieldy.

Cumbersome, for sure. You can't just put it all back in a bucket and say, job well done. just put it all back in a bucket and say, job well done. It has to go back in various places and be used in various ways. It can be a nightmare, and it's one that I'm not excited to think about. I mean, you guys talk about performance a little bit. Yeah, I mean, everyone worries about the backup ingest rate, but that's not the problem. The problem is the other way when you got to get it back out, right? Because in that scenario, there's a literal fire that some IT admin's trying to put

out or backup admin, and it's someone's deleted something, something's been compromised, and now we need this data set quite urgently, please. So that's a big challenge, the egress, right? No one's lost business over a slow backup. People lose business over slow recovery, right? That's just every minute of downtime is potential dollars lost. So, you know, we've always, we focus on the backup speeds because it's the thing that the admin interfaces with most. 99.9% of the time, it's fast. Good. All right.

I don't have to worry about it. Oh, it's slow. What's wrong? Now I have to diagnose a problem. But testing recovery and ensuring that you have a plan for recovery because you can recover fast and still recover incredibly slow if you don't know where it's going, what you're doing with it. I think that's one of the biggest pitfalls of ransomware in general is people are so focused on the attack and the outcome, but they don't think about the three weeks to three months. I think three weeks is the average recovery rate. Three months is like

a bad one. That's brutal. It's brutal, right. And people just don't think about what that means and and how much time and energy and life it's going to take from them to get it back and it's not just ensuring you can recover quickly it's testing it and running through an actual realistic scenario and saying where is this going to go what are we going to do with this if we don't have access to our network switch or our you know know, our storage is completely hosed, it got fried in the process. Like these attacks are clever and they work because they are so devastating and it gets into so much more than just the data. It's, it's an infrastructure wide

attack. So, oh, I mean, yeah, absolutely. Absolutely. And, and we're seeing this recently. There was a health care system up north that just got absolutely destroyed because, well, we all wanted great. Until that gets ransomwared, the whole system's down, and now you can't book a procedure because no one can find your information. These are real problems. And if you talk about recovery playbooks, it's not, I mean, you know this, but it's not just, here's our plan if something goes wrong and it sits on the shelf and waits for something to go wrong. It's like, just like the football team they practice the plays you gotta go and do the work and a lot of organizations don't see the value in that until they get burned i mean it's uh i i know i'm not telling you anything new or even our audience i'm sure there's a lot of it admins you know swearing at the screen like that happened to me. We don't get budget to run the playbook. And then management screams at

us when some data store goes away and it takes us a little while to put it back. I mean, these are real problems that need to be addressed with practice and repetition. Yeah, you know, it's the story that always comes to mind is when I heard in regards to the Kronos attack against the UK healthcare system, there was a dude who was literally getting heart surgery. They had opened him up and it was right when the attack happened, you know, getting ready to replace his heart

with a donor heart. And they couldn't pull up his records because the attack happened they had to sew him back up you know not do the surgery send him home and then like you know he didn't know how long he had to live and he had to basically wait months later to come back and have the procedure done which is horrifying right yeah that's lives on the line at that point uh because data got you know data got locked down due to an attack. And unfortunately, even in the best case scenario, what are they looking at? Like a three-week recovery time?

You can shave minutes, maybe hours off the day, but these processes are lengthy. And they're frankly terrifying because you never know if you're going to get everything back where it needs to be and you never know what's still hidden in there that you're you're restoring and i think that's that's part of the goodness that does come with you know potential utilization of ai and things like beams yara rules and being able to inspect data more closely upon recovery having that tested run book and actually going through the process, finding the budget to take time out of your day and pretend like it's real. Make it as real as humanly possible because when it does go wrong, and it will go wrong, it goes wrong for everybody, some knowledge is better than none. Yeah, I know. I haven't seen the whole

schedule book for Veeamon yet, but I'm quite certain that they'll be talking about that issue specifically. And you're getting some echoes in the comments from Chrome Donkey and Beard of Knowledge, both agreeing that both backups are one thing, recoveries are everything, and recovery is a pain but necessary. So like I said, the people that have been there know, and hopefully they have management that understands and agrees with that. Immutable backups with zero access saves lives. They're offering taglines at this point for Ootbee, I think, which is pretty great.

I'll take some notes. Yeah, good. One of the things we haven't talked about, and we're hardware nerds and I've got this appliance sitting on the table here, is the actual hardware components and construction i think it's kind of interesting and maybe worth highlighting if if you're up to it on on yeah so this is a 2u server standard server nothing real special about that it's got 12 bays in the front and correct me if i screw this up i think 10 are in a raid six one is a hot spare and one is an nvme cache is that is that right 100 correct okay so so far so good and in the back which yeah which you can't see or or just imagine if you're listening to the audio or two two and a half inch uh that just hold the boot image but nothing real

special there either in terms of if i swiped any of these drives or components, I couldn't really do anything with them. That's right. Yeah. They are useless outside of the box itself, which is a good thing. It's better they don't get your data than they get it. And it's better they can't access your data than they get access to the hardware and can do nothing with it. So we've got a hot spare, which is great the way this is set up. Tell me about the NVMe cache. Yes. So that's a little bit of the secret sauce and part of the reason why we had to launch with Veeam version 12. The way that we ingest data over the SOS API allowed the management communication part of that API to basically optimize how we pull data. In the back, there are two 10 gig network switches,

which just copper. We also offer the option for SFP plus, so you swap, you pick which one you want. The SOS API afforded us the ability to dual stream that data into those two 10 gig NICs. So one, we're parallelizing the data we receive. Now, it does send as per VM. So if you're only backing up one mega, you know, terabyte VM, unfortunately, you're only going to get, you know, one 10 gig NIC worth of throughput. But if you have multiple things coming across that's, you're doubling your throughput, right?

So the NVMe comes in as the landing zone for that data. It comes through the network, it lands on the NVMe, and from there, our own software is the one that goes and disseminates it and puts it in the object storage array with the, well, I should say the combination of the 10, if you have the 128, it's 16 terabyte spinning disks as part of our RAID 6 array. And all of that happens simultaneously, which is nice because the network throughput is not fast enough to max out the NVMe, which is not fast enough,

which is too fast, I guess, to max out the throughput to the RAID 6 array. So we've eliminated the bottlenecks on our end to just make optimized data throughput from Veeam to us. And we could not do that without the SOS API. If you just used S3, you'd be writing directly to the RAID 6, which would take a hit on performance. So these things all work in conjunction together from our design phase to ensure the fastest backup we can get. Yeah, that's interesting. I know you can't disclose any secrets, and I know you're happy with the box that you've got now. You must be always considering, though, when do we offer, say, 25 gig NICs? Or do we put more flash in the box?

Or do we make a smaller one? Do we make a bigger one? Do we do more other things or accelerators in the box? How do you guys, what guides your hardware decisions in terms, and I know it's only been nine months of shipping, but you've always gotta be evaluating these technologies and thinking about what makes sense, what doesn't, what adds too much cost, what adds great value, that kind of thing.

So it's pretty simple. When y'all, the storage review, betaed our box, we actually sent it out to, I think it was 14 other reviewers to kind of give a look at what we had. And at the time, we only had the two 10 gig nicks we received so much feedback from our beta about the importance of sfp and our sfp plus and why we needed that that we actually changed our our final shipment to include that card optionally to pop in and utilize for launch so we made a last minute decision on that and it played out in our favor because a lot of people use SFP Plus. So it truly comes from what do you need, what are you using today and where can we improve? And if we get enough feedback from our customers, you know, next week, it's not just our Super

Bowl because we can talk about all the cool things we do. We can talk to a lot of customers or potential customers who don't have us yet. So it's a mission for us to gather feedback and hear, why aren't you using BootBee today? And just tell us the truth. I want to know the whole truth, all the ugly bits and everything is part of it. Because those conversations are really what help us grow to make those next decisions about what to use and how to build the next box. So this is a... And I'll also say, you know, as part of the thing that we'll be talking about next week that we'll be launching, that's direct customer feedback in action. That is why we made the decision we made for that launch. Good. So talking about customers a little

bit more, another question that we got as we were going through the process, I know you're targeting a little higher up the stack, but if you can imagine, you know, a smaller customer for a moment, we saw a lot of Veeam customers using a NAS of some kind as a backup target. And there may not be anything inherently wrong with that, but I think they thought they were getting some features with nas including potentially mutability that maybe may not mean the same thing if they're using you know some off-the-shelf small package nas as what you guys are talking about with utb i'm not really asking you to dive into really small competition for this, but if you're talking to, say, a VAR or some sort of other channel provider that's working with a lot of smaller customers, what's the case that you make for Ootbee versus some small NAS for your backup target? Well, you know, you can never argue with,

hey, we have this thing that's just sitting on the shelf. We could use that as backup storage, right? Like what beats free? That's fair. And my argument is always, right, right? What beats free? And my argument to that is if you are victim of an attack, have you done your due diligence to secure that thing?

Whether it's free or it's cheap or you've had it for years. You know, Veeam offers a hardened repository, which is great. Like, it comes with the product. If you have a little bit of Linux know-how, you can go set this thing up and have really secure storage tied to whatever hardware you attach it to. It's a perfectly acceptable option, except if you are the victim of an insider breach or someone uses your credentials to get access to that thing. As long as there is an admin, as long as there is a user with power, you don't have true immutability. And this is not a knock on any competition or any

other storage vendor, because there are cases where you want to have administrative privilege and ability. I would argue that becomes more negotiable when it's backup storage. If it is for the purpose of backup, if it is a vault, if it is the Fort Knox for your data, where you just save a copy of that stuff to ensure its recoverability at all costs. I would always gesture towards ensuring true immutability in compliance mode, making sure that when it comes to least privilege access that is considered and maybe no admin privileges or very reduced admin privileges, at least not delete or factory refresh or factory reset. All of those things come into play because if you're just taking a gamble on free,

I guarantee you the amount of time you're going to spend updating, ensuring that your security policies are always up to date, and accounting for day zeros and all of the individual parts of the kernel and the software that gets tied to it, that becomes a management nightmare if you want to stay truly secure. And if you don't, that's exactly what the bad actors want to hear because that's how they find their targets. They're already in there and they're looking for holes. So if you create the hole, then you've just made their day. And I'm happy to say that we've offered a solution with no holes in it. And that's the idea. And obviously it's going to cost a little bit more because

security expertise doesn't come cheap, right? It takes time, effort, and energy. And that's exactly what we've built and are delivering. We talked about backups before, or I'm sorry, updates before to OOPI, and it does go to the internet to get that data. For people that are really concerned about this kind of thing, is there a vision that at one point where there could be an offline version so you could take the platform entirely away from any external access?

That is a huge piece of feedback that we have received numerous times and we've heard you and stay tuned because your questions will be answered very short. So you were talking about internal bad actors and this is something that's come up a ton in the last month or so with us. We've been talking a bunch about liquid cooling and I know this is going to be a weird parallel to backup, but I'll get there, I promise. So we've got a liquid loop in here now with cool IT.

We retrofitted a server and it's got physical tubes that go to a manifold, that go to a coolant distribution unit. And it's a real world loop. And if something happens to that, you could have a problem, right? If you have a leak, then you've got to do something and shut down the server. So now, every time we're posting about whether it's cool IT or Chilidine or Zootacord or anyone that's got a tube coming out of a server, there's this whole brand new paranoia that some internal bad actor is going around slicing these lines and then, I don't know, running away laughing maniacally like Sideshow

Bob or something. And maybe this is real. And I guess it is real if people are really concerned about that. But I don't know if this internal bad actor fear is increasing or what's going on, but we're hearing it a ton more lately. I just bring that up because you were talking about that a little bit. Are you hearing more concern or an increased awareness over someone in, you know, the call coming from inside the building? I love that you use that analogy. I say that so much and I feel like no one ever gets it, but thank you. Also, I want to say that video you posted of snipping the thing, I'm not one to watch videos on repeat.

I think I watched that 10 times. Are we sure there's no drops coming out of that thing? I was so impressed by that technology. It just blew my mind. It's amazing. I sent it to so many people. It was so cool. The vacuum and everything. If you haven't seen that video, you have to go watch it. It's the coolest thing I've seen in a long time. But to your point, unfortunately, the whole reason zero trust even became a thing was because of insider breach. The number one statement made with zero trust is assume breach, verify integrity, right? Like don't trust your

own people. And it's not because inside of every organization, there's one guy who's secretly bad and you've got to play like a game of werewolf to figure out who he is. It's actually more likely that it's one guy that accidentally, you know, used the same password in two places twice or was part of a breach at, you know, some large credit union and now his information is being used to manipulate himself. Insider breaches don't just mean that there's a bad actor. It means that someone can be posing as you inside your organization to get access to them.

Or accidental too, to be fair, right? Right. Because if you scroll through red reddit sysadmin and i saw one today actually where there's a dude like i really messed up i think he might have used an explicative there and i could see i did some robocopy thing i thought the stuff was there i did a slash you know delete by accident and now it's all gone and we didn't have backups running because we didn't want to spend the money on consuming backups until everything was migrated.

But that was a whoopsie, which is troubling too, right? It is troubling. And we're all human. I've made my fair share of IT mistakes in my life and I'm not proud of them, but I've made them. We all do. The story, and apologies if there's any cross-following here, I brought this up on our weekly podcast multiple times now, is there was a bank heist in China about a month or two ago. And the way that

it was initiated was the guy who was in charge of wiring money got an urgent Zoom invite from the CEO and the CFO of the company to send $25 million to this end user or this other firm for some important reason. But they were on the call with him, just like you and I are, their face, their background, their voice, their everything. And it was puppetry. It was a combination of AI, deep fake, and human beings pretending to be someone else. And they got him to wire the money. And he legit didn't know. He said there was nothing suspicious about this phone call at all. And I don't think there's anything more truly terrifying than not knowing the person you're talking to at the other end of the Zoom is who they say they are. But you could find yourself in a situation where you are doing bad actor actions under the sort of, you know,

false guise of someone who is your boss or your boss's boss or your CEO. It's scary times and we are fallible enough as it is. Now there are other people trying to pull the strings. I think that just makes it so much worse. So why even put yourself in that situation where you could be the reason your company loses 25 million yeah that's that's that's pretty terrifying i also like your work your use of the word heist there that's a very uh 70s 80s kind of thing in speaking in speaking of 80s, and I promise I will tie this one back to Veeamon, or at least adjacent to Veeamon, your banner pick on your LinkedIn profile page is the Luck Dragon from Neverending Story. Is that correct? That is correct.

And you have an appreciation for that movie that extends beyond many others but I think that one has a if you're of a certain age in IT I think almost everyone's probably seen Neverending Story that was around as a child in the 80s. I know I saw it on loop on HBO you know for for a good part of of my youth. What is it that attracts you so much to that character? Firstly, he's just a cool looking design. I'm a person who loves the weird and I especially love the weirdly practical. I was born at the end of the 80s.

So I missed what I consider to be the best decade. All of my favorite movies and music came from that time period. And I just love a good practical effect. And Falkor is just, he's so big and goofy. He's got fur. Little wings. And I don't know. When I wanted to figure out my LinkedIn background, if you spend any time on my LinkedIn at all, you know I'm not a very serious LinkedIn user.

Most of what I put out there is either meant to be humorous or some kind of fun take. I just wanted something cool, and I think Falcor is still really cool. Never Ending Story, great movie, great story, great soundtrack. Perfect 80s movie, in my opinion.

Why not give Falcororstin the ball? They had lasers at the eyes of those statue things. I was actually, so I'm older than you are, and I saw it in relative real time. What year would that have been? 86 or something, maybe? Got me, yeah. When that came out, and I was a kid and that that movie was intense scary but not not over the top not horror scary it was uh intense scary anyway so i'm at i'm at uh veams analyst summit two two years ago i missed the last one and one of the things that they do

is uh they put on an awesome event for analysts to go and talk about backup and Veeam and all that kind of stuff. But they have little at the dinner, they'll have little artists or neat things, crafts and that kind of thing. Well, they had a wire bending artist there and people were getting their kids names done or the face of their dog or whatever and I'm watching this guy all night bend the artwork was amazing just bending bending bending dog cat lizard you know name heart whatever and I went up to him at the end and I said you know I've got this golden doodle but you've made so many dogs. I said, she's a white golden doodle. And he was a little bit older. And I said, do you ever see the never ending story? You know, that luck dragon, she kind of looks like that, but a little furrier. How about you make me

a golden doodle luck dragon with wings? And he's like, done. That dude was so happy to make something other than just a standard issue dog and and now i've got at my office at home a uh a wire art uh golden doodle luck dragon so i'll uh i'll take a picture and send it over to you sometime i want to see that so badly i also have a golden doodle and uh she she is truly golden but she does have the falcor face so there you go yeah they've got that sort of yeah yeah yeah i. Yeah. I'll, I'll be sending you some, some pictures of that. Okay. Before we lose everybody after an hour and us talking about eighties movies and surely being somewhat off track. Uh, this has been great. I appreciate you doing the podcast. Your willingness to do it live

is awesome. The engagement with the community, we really appreciate that. I look forward to seeing you next week. Will you be wearing that shirt next week when I see you? Nice bright colors? I wish. I got to wear the required uniform to quote the Breakfast Club. I will be wearing a purple shirt. It says best storage for Veeam and has object first on it.

But I do have a cool jacket. So a few of us get the cool jackets. And that's because we do our weekly show. So we've got to stand out from the crowd. Okay. Well, I look forward to seeing you there. For everyone else in the audience, if you're going to be, like I said, in Lauderdale for that next week, look us up. I'll bring, I of our lab bunk magnets that you can use to defend your lab from nefarious bad actors since we're on backup and recovery.

And like I said, we've got a full report on Ootbee in the description of this live show. It's on the website, storageview.com. Check that out. We'll be dropping a long-form video with Kevin and I talking about this platform, hopefully next week as well. And the audio for this will be out in the next couple of days on all major podcast platforms. So check that out too. Until then, any parting words before I send you off? No, thank you so much for having me on. It was a great conversation and love to come back.

Awesome. We'll gladly have you. And I can't wait to see what your mysterious announcements are next week that I got dangerously close to blurting out, but did not. So, you know. Okay. I did too. So. Kudos to both of us for not screwing it up and having PR come after us. But thanks again for the time and everyone else online, thanks for tuning in.