Screaming in the Cloud - Forty-Five Years in Tech with Hal Berenson

Episode Date: January 12, 2021

Links Referenced: Gaia PlatformHal’s BlogFollow Hal on Twitter

Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. Veeam. Their AWS backup and recovery solution is made to save you money, not that that's the primary goal, mind you, while also protecting your data properly. They're letting you protect 10 instances for free with no time limits, so test it out now. You can even find them on the AWS marketplace at snark.cloud slash back it up. Wait, did I just endorse something on the AWS marketplace?

Wonder of wonders I did. Look, you don't care about backups, you care about restores. And despite the fact that multi-cloud's a dumb strategy, it's also a realistic reality. So make sure that you're backing up data from everywhere with a single unified point of view. Check them out at snark.cloud slash back it up. When you think about feature flags, and you should, you should also be thinking of LaunchDarkly. LaunchDarkly is a feature management platform that lets all your teams safely deliver and control software through feature flags. By separating code deployments from feature releases at massive scale and small scale too. LaunchDarkly enables you to innovate faster, increase developer happiness,

which is more important than you think, and drive transformation throughout your organization. LaunchDarkly enables teams to modernize faster. Awesome companies have used them large, small, and everything in between. Take a look at launchdarkly.com to learn more and tell them that I sent you. My thanks again for their sponsorship of this episode. Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Hal Berenson, who at the moment is the founder of Gaia Platform, which is an industrial strength, low-code platform for developers.

But you've done a lot more than that. Hal, welcome to the show. Thank you, Corey. It's great to be here. So it's rare that I see a line that jumps out in a biography like the one that you sent over. In your 45 years in the industry is how it starts, at which point that sort of boggles my mind. In the interest of full disclosure, I am not yet 40 years old, at least at the time of this recording. And it's phenomenal to me to imagine the idea of, first, staying in a particular industry that long, but also having a career that transcends my own lifetime. What's that like? You know, it's one of those things that's gone really quickly. And I kind of get shocked when I think about it as being 45 years. And it's really kind of longer

than that. I grew up in a computer family. My father was in IT from the late 40s. And so I've always been around computers and it goes fast and it's largely because I have so much fun at it. And it's fun looking through, I guess, obviously you're cherry picking different bits and pieces of your career when you talk about these things because the full unaccounted list would almost undoubtedly take up most of the show.

But things that you've done, for example, moving backwards before Gaia platform where you founded it, you were the vice president of RDS at AWS, which one of those feels like it should be expanded. You were a distinguished engineer and general manager at Microsoft. You were a senior consulting engineer at DEC or D-E-C, depending on pronunciation. We're not starting that fight again. But you've done an awful lot of stuff. You do technology and management consulting. You're a song and dance man. What is the acting term? Triple threat? Actually, when I started, computers were a hobby, which of course,

today is a pretty common thing. But I started out really in the 70s, and access to computing was relatively difficult. So it was fun for me. And when I went to college, I wasn't thinking computers. My first year, I was thinking microbiology or something, but computing just drew me in. And a lot of the things I'm best known for, like databases, are things I never intended to do. I've tried to get away from multiple times. It really wasn't until I went to AWS where I said, okay, I'm giving in on my desire to get away from database work and let it happen. To be clear, you were at Amazon doing RDS VPN, I guess we're going to call it that,

from the end of 2014 through the end of 2017, which saw a lot of changes. That's when Aurora started coming out. That's when DMS became a thing. And from a personal note, that's when using RDS somehow transitioned so subtly, I didn't even know that it was happening at the time. It went from, that thing's for idiots. If you care about your databases, don't use it, to becoming an extremely viable answer for running large-scale production database workloads. And it was just the reliability story, the maintenance story on it. An awful lot of stuff changed, and it wasn't one feature or one announcement. But backups got better.

Replication stories got better. The encryption stories became radically more mature. And it was just at some point you look up and realize, huh, this service has gone from, I don't know, to something super legitimate. What drove that? I mean, am I imagining this? I don't think you're imagining it. And while I'd love to take credit for some of that,

I can't take credit for a lot of it either. Before I got there, you know, it take credit for a lot of it either. Before I got there, you know, it had gone through a lot of the maturing that everything at AWS did or anywhere in the cloud did. And some hard lessons were learned in terms of things like just how reliable you needed to be. Whereas people knew it theoretically before, by the time I got there, that was what, you know, that was what was back to mind. We can't have a big outage. We can't have a backup not work, et cetera, et cetera. So it was already underway, and then my enterprise background led me to support that and push that direction instead of trying to change it to something else.

It's sort of a microcosm of the entire cloud story. It's the idea that if a cloud provider, any of them, I'm not singling out AWS on this, releases a feature today, it's probably not fully baked for all workloads. Surprise, surprise. Every customer is different in some ways, but they also don't get worse over time. They tend to improve sometimes in leaps, sometimes just by chop wood, carry water. But over time, they wind up becoming much more acceptable for a wider variety of workloads. And it still sort of boggles my mind that the same underlying service is equally suitable for fast-moving startups who are just trying to get something out the door before they run out of runway next week, and these large enterprises whose primary goal is risk mitigation.

There's some amount of dichotomy there in terms of how you can do more at the low end and do more at the high end at the same time. But you don't have to degrade the low-end experience as you do the high end. And a lot of people at the low end can benefit as you do the high-end. And a lot of people, the low-end can benefit as you do this. For example, having more high availability options. Well, if you're a startup that needs super high availability, the fact that what drove somebody to implement all these availability features was a Fortune 500 company screaming for them before they'll adopt their service.

It doesn't matter. You did this great feature and now the low end could use it. One of the things I didn't like about the package software product world is you would take features and you'd reserve them for a high-end, super expensive edition that only your enterprise customers could pay for. Oh, if you want it encrypted, that'll cost extra. If you want SAML federation, that'll cost you. Oh, you want audit logs. Get out the checkbook. Exactly. And those things harm startups. And it goes to one of my frustrations with SQL Server after I left, which was I had envisioned that what we would do is every new version, we would take a few features that we had put in the enterprise version and push them down to lower-end versions, to push it to standard, for example. But people got so excited about the money you got out of enterprise that, for a long time, they let standard stagnate.

It's only three or four years ago that they finally moved a lot of those see in terms of how big SQL Server got, but things like MySQL would never have gotten traction if Microsoft had done just a few small things. Multi-platform certainly could have been one of them, but the other was pushing more functionality further down on the curve. And I'm not the only former SQL Server leader who sees it that way. I've talked to a number of them who are kind of like, yeah, this was a big miss on our part. But the package product world makes that very common. In the cloud, it's a lot easier to find differentiation and ways of justifying doing work without making it inaccessible to even an individual starting a business. Well, you can even see it in the way

that companies talk about themselves. And the things that a startup needs and uses today are what enterprise is going to be using down the road. There are virtually no startups that describe themselves as, we're like an enterprise. But basically every enterprise likes to do the recruiting pitch of, we're like a startup. Just don't examine that statement too

quickly. And if you can get the things that enterprises value, which let's be very clear here are important. I'm not here to talk smack about large enterprises generally, but things like durability, things like being able to trace transactions and who did what, and being able to trace accountability for every step of the, I guess, custodial supply chain of a piece of software is important. And enterprises require this from a risk mitigation perspective, whereas startups don't need it until suddenly they very much do. Yeah, and we saw that a lot. I mean, this is one of the great things of having all these different jobs is, of course, you learn very different things. So in the case of going to AWS, I got to experience a lot of these startups who had become daily things. I mean, one day I did look at my phone and all the apps I was using and realized, oh, my God, they all run on AWS and their household names.

And as you just indicated, they had the same availability requirements. Look, when you can't get your shared car ride, it's as big a problem as when you can't get $10 out of an ATM. And this goes through everything. People notice it. It makes the front page of the Wall Street Journal when these services are down. So the startup world and seeing these people become big and go from, oh, there's one that didn't have a DBA until they were way into being a household name. And then, of course, they start having the exact same requirements as the Fortune 500 companies around their mission-critical apps. So it's fun doing it in that world and seeing startups grow to need the same things as enterprises. And it's fun in the cloud that you can deliver that. One of the things that I find gets missed so much is this idea that

companies are able to speak to each other and they're using the same language. I mean, one of the biggest pains that I have to deal with as a small business dealing with AWS bills on a consultancy basis is the onboarding vendor procurement forms, where we wind up having to go through the enterprise-grade processes. What is our data protection strategy? That's super important. We have to get that right, and our answers there are pretty solid. Then they ask things like, do you have at least $10 million in commercial vehicle insurance? For what? We don't have cars or forklifts or anything handled by the company on that. So it's just a big, no, not so much. It's this idea that the

same intake process that enterprises use that they would use for their caterer or the bus company they're having do a company-wide event don't necessarily apply to pure services-based, high-level expertise-oriented consulting work. So it's this going back and forth and having those conversations. In my previous job, I wound up feeling this very acutely. I was employee number 41 at Future Advisor, a fintech startup that handled basically a robo-advisor. And three months in, we got acquired by BlackRock, a company that at the time managed something like $4 trillion in assets. There was a bit of a culture gap between those two experiences. And watching the transformation that hit us at basically warp speed as a direct result of this was eye-opening for me and forever

changed how I view large-scale enterprises and the things that they care about. Yeah, I have similar experiences from the independent consulting standpoint, which is every time I deal with a large company, I go through those same things. Somebody will want me to have $25 million in IP liability protection in case I somehow transfer somebody else's IP to them.

And, hey, I'm a single proprietor thing. And first we have to find that insurance. And then it's really expensive. And I'm probably not going to do enough work for you to justify it. Whereas with a startup, it's like you talk to the CEO and the CEO goes,

yep, can you start yesterday? And don't go through all this vendor craziness and everything. And so I have a view of different large companies based on exactly that problem. And when I first did some consulting back to Microsoft, I was talking to my friend and now co-founder, David Vascovich, who was Microsoft CTO at the time. And I said, oh, there's really a pain becoming a vendor and whatever. And he goes, yeah, you're in this interesting position. You're not quite at the point where Steve Ballmer, who was CEO at the time, will just tell them, make it happen and kind of ignore all the policies. You're just below that level.

But of course, you know everybody and you know everything. And everybody's trying to make it happen fast, but they're not the person who can say, you know, blow out the policies. But doing the work for Microsoft, dealing with their vendor policies was eye-opening, seeing it, having seen it from both sides. And I've done that with others. There's another large computer company I did some work for where their vendor policy was insane. And it needed approvals from people in multiple countries is where the chain went. And I was supposed to fly to China to do the consulting for them.

And I didn't have the approvals. And I just decided to go and hope everything got approved. Because if not, it was going to be like tens of thousands of dollars out of pocket when you threw in airfare and hotels for weeks. And fortunately, the day I got there, I walked into the customer and the approval just came through. But I mean, it took weeks to get through this vendor process. So it does tend to cloud your view of big companies versus small ones. It consistently surprises me that a lot of these process-driven companies, that employees who need to write a justification document to spend $50 on a book, would be able to either spin up hundreds of thousands of dollars worth of infrastructure, or more commonly, call massive meetings where the actual cost of salaries for the duration of that meeting are astronomical.

And that's never questioned. It's a really weird perspective, and I used to be very cynical toward that. Then I start looking at how these things play out and what drives these. And the more I look into things like this, the more sympathetic I become. Yeah, the problem is they never die. The policies never die or get revised away. That's mostly the case. And so if you look at Microsoft's continued thing with vendors,

that all goes back to this 1990s lawsuit by contract test people in which Microsoft lost the suit that had them declared to be employees. And so ever since then, it adds layer and layer on making sure that anybody there on a contract basis will never be considered an employee. And so it gets pretty messy for people. They've been working there for, I forget what it is now, but like nine months, and then they have to not work there again for another six months or nine months or something.

Oh, and I understand that. It's no joke. When you misclassify employees, I hear you on that. And even if they had work elsewhere, which would kind of automatically say you're a contractor because you're doing work for more than one person, you still can run into those things. But the processes that get thrown on there and the processes that get thrown on, yeah, for acquiring things like books because you go and discover what were the past abuses in that, for example. But those policies never kind of go away. At Amazon, you could make the policy go away. You could go and write a narrative. You could go justify it. You could say, look, this is violating the leadership principle because you're actually wasting more money you probably have an incredibly complicated architecture, which means that monitoring it is going to take a dozen different tools,

and then we get into the advanced stuff. We all have been there and know that pain, or will learn it shortly, and New Relic wants to change that. They've designed everything you need in one platform, with pricing that's simple and straightforward, and that means no more counting hosts. You also can get one user and 100 gigabytes per month totally free. To learn more, visit newrelic.com. Observability made simple. Something I want to talk to you about is a form of diversity and inclusion, because it's fun. Two white guys talking about this is

all well and good. But what I do want to talk about here is the idea that, first, we talk about women. We talk about people of color. We talk about the very real struggles that they wind up facing. But an area that we don't talk about is age, which, barring tragedy, is going to affect each and every one of us as we move through our career. Very often, I talk to people on this show who are at the beginning or toward the middle of their career. It's uncommon that I speak to someone who's been doing this for 45 years in the industry. What have you seen from that perspective as you've gotten inevitably older as time goes on, but do you find the way that people respond to you or the way that you're treated has shifted? Interestingly, I haven't seen it as much as typically gets reported.

And I experienced it more on the other end of the scale, actually. I quit college after one year and went into the industry. So I didn't have a degree and I was super young. And I very quickly was ending up in positions where people would have typically been as much as 10 years older than me. And so very early on, I experienced a little bit of it. It was countered by the fact that there was such a demand for expertise that wasn't there and I had the expertise. So people would kind of give me a chance. And as time went on, I never really noticed it. When I went to talk to Amazon, and I was already in my late 50s when I went to do that, I didn't think about, oh, I'm in my late 50s. I thought about, I've mostly been retired

for four years. What's Amazon going to think about that? But I didn't really think too much about my age. And it was kind of funny because I learned during the interviews that I would end up being, I think, the oldest VP at AWS when I joined, if not the oldest, pretty much tied for it. But I never really thought about it. And on the other side, I don't have a problem with millennials. I think millennials are great. I don't get this generational thing. In fact, I never have. From when I was young and others were old and on, I've never gotten it. So millennials, actually, that worked for me. We were shocked when I told them my age. I've never found it to be a real issue in

my career. I have found other things more important to focus on, you know, experience, expertise, willingness to listen, ability to deal with people in various ways and at various levels. Those to me all were more important in every aspect of my career than this calendar age thing. And the thing that hits me now is, yeah, I keep retiring. I've retired three times. I don't know if I'll ever claim to retire again. I may just claim to take a break because I come back into it. And it occurred to me relatively late in that something my friend Jim Gray had said years ago.

On my first retirement, he was like, you're crazy. How could you ever retire? You should work basically until the day you die, which sadly Jim did, having passed prematurely. And I realized if you're enjoying it, it's true. Why would you ever retire? And people don't really want you to most of the time. Well, something I do want to highlight is you started your career as an engineer. And now that you're founding a company, everything gets weird and screwy. But again, your last quote-unquote real job or capital J job or corporate entity job was as a VP.

I've talked to people who've been in the industry for 30 years who are still engineers, and oh my god, are they great engineers. But they seem that they have to deal with a headwind of, if you didn't transition to management and executive management at that, then somewhere along the way, you must have gotten stuck. Is that something that you've seen? And is it something you agree with? I've seen people get stuck, but not because they didn't transition to management. And I've seen people not get stuck at all. There's an awful lot of people my age who are fellows and distinguished engineers and the like who've maintained largely individual contributor jobs.

I started out actually as, my first jobs were actually more technical support jobs, which I actually highly recommend because it gives you a lot of customer focus, but people get a little scared of it because sometimes you'll find some resistance when you try to transition into engineering. Depends on what you've shown in terms of skills.

But I kind of went through these phases. I really wasn't interested in management. I did switch back and forth between what I'll consider operational line roles and staff consulting kind of roles. I liked having the influence over the direction of things and how organizations were built and so on. But I didn't really want to manage people. And I did a short stint at management at DEC, which was positive, but not a plan when my boss had a heart attack. And so I filled in for him for a few months. And then I went back to individual contributor. And when I got to Microsoft, Microsoft was not as good a culture of having influencers.

You needed to own things. DEC had been, the ladders were really peered. And I have to tell you the truth, a consulting engineer was considered more influential. In fact, it's a funny thing. I discovered that in the consulting engineer job description, that it basically said I could go stick my nose into business anywhere in the company. I've never had a job description like that and I'd do it anyway. I have problems. I'm a terrible employee. There was one time I really exercised it specifically as a consulting engineer versus just me being nosy or whatever. I actually did go to

somebody at one point and say, I'm sticking my nose into your business because I think you're screwing up. I mean, I've organization way far away. And they were like, fine, come on in. And so there were these peer level kind of roles. And then at Microsoft, Peter Spiro and I, Peter had been my partner in crime in the database world at DEC. And we get to Microsoft and we realized one day that we were having trouble influencing things,

which is that the development managers were themselves ignoring us, even when our senior management was supporting us. And so I was standing in Peter's office one day, and he looks at me and goes, I think we're going to have to become managers to make this happen. I was like, yeah, I think you're right. And so we took over as development managers initially and product unit managers pretty quickly. You hoisted the veritable black flag, as it were. Yeah. And it turned out for both of us, we liked it. It was a weird thing. It's not that neither of us had it as a career goal. Both of us loved mentoring people. We loved helping with the direction of things.

And both of us liked still being engineers. I mean, this is the thing. We had the opportunities. That's why Peter never was listed as a, he never became a corporate vice president at Microsoft. He became a technical fellow. He was one of that first wave of technical fellows. And the thing is that we could keep being technical and become managers. Now, over time, it gets harder and harder because of time, but we're able to make that transition.

And even with that, I went back to IC roles. This is why I list the Microsoft time as distinguished engineer and general manager, because sometimes I played general manager and sometimes I really played distinguished engineer. And my last management role at Microsoft was we did a reorg in the security products division. And we ended up about a half general manager short. That is, we couldn't find anybody to give some stuff to without overloading them. And my boss, knowing I had the management

experience, looked at me and said, would you mind being the GM and mentoring the guy who was the product unit manager for this big piece of it until he was ready to be promoted? And I went, no, I'd be happy doing that. Suddenly, I'm back into management. But I go back and forth. To your specific point, I think it's fine. And I know a lot of senior people who never make that management leap, but they either continue to grow in technical ability and most importantly, judgment so that they have a broader span or they get very happy with the niche they're in. There's not a lot of people that like what they're doing. They actually don't want to be a deck. They didn't want to be a consulting engineer. At Microsoft, there were people who

didn't want to be a partner or didn't want to be a distinguished engineer because it meant that they would have to spend a very large percentage of their time not working on their own technical problem, but helping others. And they were very heads down kind of people. Oh, I hate helping others. I kid, I kid. But I'm with you on that. It feels like in many companies industry-wide, there's this perception that if you're a great engineer, cool, then you should get promoted to management, which is not a promotion. It's an orthogonal skill, but it's not compensated that way. And it's not respected that way in most companies hierarchies. So it's terrific. So, and very often what you'll see is, wow, we're losing a great engineer and gaining an absolutely terrible manager. Yay us. And the way that it's

done is you can't ever go back and have someone go back to being an IT role in most cultures because it would widely be perceived as a demotion and people won't rightly stand for that. So it's a really hard problem that I think very few companies seem to get right. Now, I think that from the outside, Amazon and Microsoft and Google have done a very good job of this. But as far as smaller companies go, I question. It's hard to see at small scale. Yeah, I think you're right. It varies a lot in company culture. One of the weaknesses of its culture was it would push people into management who would not be good managers. It didn't focus on their management skills. And so that frequently happened. Amazon is kind of

at the extreme other end of it, which is that they're kind of skeptics about people wanting to move into management. They wait until they're higher level than most companies would before they'll let them move over. In AWS terms, they mostly wanted managers to be level six or higher. And occasionally, you'd let somebody in L5 be a manager because you were about ready to promote them to L6. You just want to give a taste of management. But it was pretty rare. You wanted to focus on their management skills. Microsoft's kind of in between those two scales. But yeah, there are a lot of companies which are like,

okay, we need a manager. You're the best we got. You're the manager. And that can be a disservice, a big disservice to the employees. That's the manager there to serve employees, basically, and help make them more productive. The idea of servant leadership is great.

The problem is, look at managers you've had over the course of your career. How many lived it? Yeah, I think I've been pretty lucky, but I've been, most of the time, able to select my manager. So if I avoid the ones I don't think will be good, that I won't learn from or won't be supportive, I had one at DEC who I thought was not only a terrible people manager, he was a terrible large organization manager. And I remember he came to me when I was at Microsoft looking for a job.

It was very polite, but I was thinking in the back of my head, no way will I ever let this person get hired by Microsoft. And he's like the only case that really falls into that. I've been pretty good at finding managers who've been very supportive of my career. Even some managers who others didn't care for. So I had one manager who's actually very well known for another company, becoming one of the longest serving top leaders of another company, who I was like, you don't understand this. When people would complain to me about him, I'd say, he's fantastic with managing up. And so you just have to think

about using him the right way. He's not going to be there for you every day, but if you understand that he's making sure nobody interferes with what you're doing and what you want to do, and that's where he's putting his energy and use them the right way. Then he's great. And that worked for me because I'm pretty self-sufficient. And for others, though, they needed more direction from him and more time with him. And it didn't work for them. So even some people who get perceived one way are actually pretty good as long as you can figure out how to interact with them the right

way, how to take advantage of the things they're good at. And so I say, I think overall, there's only been the one person who I really thought was damaging to myself and to other employees and to the company. Everyone else has been good to very good or even excellent. Yeah. There's a very different way of framing these things, I suspect. When I hear managing up, I think of something radically different than I think most people intended. What I hear is telling your boss to go to hell in such a way that they look forward to the trip and begin eagerly packing. I don't think that that's exactly

how people mean that phrase, but it's what I hear. And in my experience, it seems to be just about right. Well, I'm sure in some cases that that is true, but a lot of cases, who says that you're doing work that three or four or five levels up the management structure understands, believes in, is willing to fund appropriately? When times get tough,

they're not going to disproportionately attack it from a budgetary or layoff standpoint. How do you keep something strategic? Again, this is even more important in a big company. How do you keep something strategic to the company when it has a hundred things and it's going, I can't do a hundred things or a thousand things. I need to whittle this down. How do you stay up there? How do you find opportunities for your technology or products to be used more broadly in the organization? And what happens when you have senior leaders who are enigmatic and strongly opinionated and how do you keep them supportive of the team overall?

And there are people who are very good at that. You can look at some big companies, you see executives disappearing all the time because basically they finally had it out with the CEO or something. And so somebody who can manage one of these very opinionated CEOs is a very important skill and organizational structures and all that. But they may not be that great at one-on-one management of people below them, especially as you get more senior.

If you're expected to be able to operate independently and you think your manager's going to be there constantly to support you, well, it could be a problem. Now, if you're junior, you expect it. I mean, you expect first and second level managers to be able to be very supportive of the people below them because they're just not that senior themselves and they are going to need the guidance. I think that's very fair

and also probably a good place to leave this episode. If people want to hear more about what you have to say, where can they find you? So I blog at hal2020.com, H-A-L 2020.com. I have not been super active this last year. I go through periods where I kind of get a writer's block thing going. Other times I've been super active, but there's a lot of material there. There's a lot of historically interesting material there, I think.

I've had ex-Microsoft people or current Microsoft people. There were people trying to understand what was going on in Microsoft that told me they'd read my blog to find that out, even though I'd been out of Microsoft for years. That's the best place to go. I'm also on Twitter. I don't even remember my Twitter handle. I think it's at Hal Berenson. And I'm on there a fair amount. Again, not as active lately, but I go in and out of being very active.

Excellent. We'll, of course, include links to those in the show notes. Hal, thank you so much for taking the time to speak with me today. I really appreciate it. You're welcome. It's been fun. It really has. I'm delightful.

And so are you. Hal Berenson, founder at Gaia Platform, LLC, as well as many other things, as we've just discussed. I'm cloud economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice. Whereas if you've hated this podcast, please leave a five-star review on a podcast platform of choice, once again, and a comment telling me that I don't understand it yet, but I will when I'm older and been in this industry just a little bit longer. This has been this week's episode of Screaming in the Cloud. You can also find

more Corey at screaminginthecloud.com or wherever Fine Snark is sold. This has been a HumblePod production. Stay humble.