Screaming in the Cloud - Resigning from AWS on Ethical Grounds with Tim Bray

Episode Date: August 4, 2020

About Tim Bray:Tim is a general-purpose Internet-software geek. He specializes in Web, search, writing, speaking, business.  He founded Textuality in 1996. He is available for consulting on issues of technology leadership, software construction, and distributed systems. You can follow Tim's musings on his blog ongoing and on Twitter.Links Referenced: “Working Effectively with Legacy Code”: https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052Twitter: https://twitter.com/timbrayBlog: https://www.tbray.org/ongoing/Article Tim wrote about his PR FAQ: https://www.tbray.org/ongoing/When/202x/2020/06/21/A-Cloud-PR-FAQTim Bray’s “Split AWS from Amazon” PR FAQ he wrote: https://github.com/timbray/a-cloud-prfaqTim Bray’s “Break up Google” article: https://www.tbray.org/ongoing/When/202x/2020/06/25/Break-Up-GoogleCorey’s Fake PR FAQ: https://www.lastweekinaws.com/blog/introducing-aws-elastic-beanstalker/

Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. Learn about the dancing flames of EKS cluster security, evade the toxic dumpster of the standard controls, and tame the wild beast of best practices for minimizing the risk around cluster workloads. Become renowned for your feats of daring as you learn the specific requirements for securing an EKS cluster and its associated infrastructure. To learn more, visit snark.cloud slash stackrocks. That's snark.cloud slash stackrox. This episode is brought to you by Trend Micro Cloud One,

a security services platform for organizations building in the cloud. I know you're thinking that that's a mouthful because it is, but what's easier to say? I'm glad we have Trend Micro Cloud One, a security services platform for organizations building in the cloud, or, hey, bad news, it's gonna be a few more weeks, I kinda forgot about that security thing. I thought so. Trend Micro Cloud One is an automated, flexible, all-in-one solution that protects your workflows and containers with cloud-native security. Identify and resolve security issues earlier in the pipeline and access your cloud environment sooner with full visibility so you can get back to what you do best, which is generally building great applications. Discover Trend Micro Cloud One, a security services platform for organizations building in the cloud at trendmicro.com slash screaming.

Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Tim Bray, formerly a VP and Distinguished Engineer at AWS, and now an unemployed bum. Welcome to the show, Tim. Delighted to be here. So no matter what side of the world you live on and what your perspective is, it's pretty clear there's one thing everyone can agree on. And that is that you are basically just this side of a war criminal. You were one of the original voices behind the XML spec. And for those rare few of us who really like XML, you also participated in the JSON atrocity. Well, so what's plan C if neither XML nor JSON are going to work for you? YAML. It's how we build our safest, whitest spaces. Yeah, you know, YAML is so convenient and pretty looking,

and most senior engineers hate it for the simple reason that in YAML, when you have a message, there's no thing that marks the end of the message. So if you screw up and drop some off the end or the network breaks at the wrong time, well, you just go ahead and process the YAML as if you got it all, which you didn't. And that can cause some really horrible things. It's worth talking about XML and JSON a little bit, though. So we did XML in the late 90s. It was completed in 98.

So the really astonishing thing was before that, there just weren't any data formats that were vendor independent, whether it's computer vendor or database vendor or programming language independent, which is shocking because we'd been doing computing and networking for 20 years at that point. And so since XML was the first one, it got picked up and used for everything, which didn't work out that well. XML was dreamed up by a cabal of publishing technology bigots who really cared about deeply nested document structures and tables of contents and index cross-referencing and things like that, which turns out not to be the greatest choice for using in REST APIs. Having said that, there's still a thriving XML community doing things like legislation, large-scale technical documentation,

like for airplanes, humanities computing, so on and so forth. So if you're in the publishing space, hey, that's what you want. As for JSON, you know, JSON is irritating in some respects, but we've made it work. It does most of what you need. We all know what the next few things we would change in it, if we could, to make it better are. But we're not going to because it's too late. You know, all the software is written and you can't be changed anymore. The question of what the best way to package up data is to ship it back and forth between heterogeneous systems is interesting. You can have a lot of fun. And, you know, these days there's things like Thrift and Avro and Protobuf.

And people, for very poorly worked out and verified reasons, seem to think those are the universal choice in the future. Eh, I'm unconvinced. Zero MQs, you can go schemaless with it and have all kinds of fun ways of packaging. I mean, SaltStack back in the early days was Python pickles on top of that. Yeah, that's absolutely true. And actually, that was one of the most interesting things I was working on there at AWS is we did a schema repository last year. And that means that from the AWS point of view, you now have data types with ARNs.

And I think that there's way, way too many AWS services that interchange... You can stop the sentence there if you'd like. Way too many AWS services. Full stop. That's another interesting discussion you could have. You know, Andy loves to get on stage and talk about how many new things they shipped every year. And everyone else gets this sinking sense of dread in the pit of their stomach when they see it.

Oh, dear Lord, at least 20 of those would have helped solve problems that I have, but I'm too busy doing my job that isn't keeping up with Amazon services. You know, having said that, are you actually going to say it's wrong? Because, you know, it seems to be going pretty well. Yeah, yeah, there's more there than any one human being can stay on top of. And in a lot of cases, people pick the technology based on what they already know, as opposed to what might be optimal. I suspect that's the case on almost every scenario. People talk on stage about the exciting, far-future stuff they're doing. But if you look at actual cloud bills and the environments that the serious, huge reference customers have,

it's all a bunch of, to be frank, the boring stuff. And we pick that boring stuff for the simple reason of we know how it breaks. If you don't know how something breaks, it's hard to trust it with critical workloads. Sure, fair enough. But, you know, in 1982, you could have said, well, you know, COBOL's what everything runs in. So, you know, why would we be poking around looking at other stuff? And, you know, you need to throw a bunch of new stuff at the wall to see what sticks. And some of it is sticking. I mean, if you look at the numbers for things like Lambda and EventBridge and so on, a lot of people are using that stuff these days.

And it is non-traditional. And the way things are done in 15 years is going to be quite different from the way things are done today. And the only way we're going to get there is by introducing new things and finding out which ones actually meet a demand. So there's a lot of things we can talk about in the technical space, but first I want to get to the, well, I wouldn't even call it an elephant in the room because everyone acknowledges it, talks to it, it's wearing a name tag, and often gets its own introduction and slide deck on stage at the podium. You were a VP slash distinguished engineer at AWS. There are less than 20 of them at all of Amazon. It is the

highest pinnacle of technical achievement. At other companies, they would be called fellows in many cases. You resigned on the basis of ethical issues earlier this year. Talk to me about that, since that is virtually unheard of at any company for someone at that tier of technical achievement and that level of seniority within an organization to leave publicly citing anything. Well, on the other hand, I didn't see really any other realistic choices. You know, when you achieve VP rank, you know, you're expected to be on the side of the company consistently and be willing to get behind what the company is doing and, you know, not be a shithead in public. And I respected that.

And when I found the company was doing something that I just couldn't live with, I made that clear, going through proper channels internally. And having done that, what other choices were there? It was not a thing that one can just get on stage and say, well, firing whistleblowers is okay because, because there's nothing that comes after the because. So resigning seemed like the only possible thing to do at that point. And it didn't make me happy. You know, AWS is a fantastic organization. I loved working there. The people are great.

The customers are greater. The work is fantastic. But, you know, it just didn't seem like there were any other options. I hear you. Specifically, for those who have not been following that particular saga, there were a number of high-profile firings of warehouse workers who just so happened to be involved in speaking out specifically around aspects inextricably linked to labor organizing. Which, heaven forbid, we wouldn't ever want the people who work in our warehouses to be able to bargain collectively. That could end badly for us.

Well, except for that's empirically false. I mean, there was a really great case study that just happened in May, I believe, where the Amazon warehouse workers in France were concerned about the handling of the COVID and didn't think the company was doing enough to protect them. And so they started raising their voices. But, you know, Amazon doesn't talk to unions. Only in various jurisdictions in Europe, Amazon doesn't talk to unions. Only in various jurisdictions in Europe, the law says you have to. So the union took them to court and won a judgment

that Amazon had to talk to the union and ship only essentials. Amazon reacted by shutting down all the warehouses, which doesn't seem like the smartest thing to me, and appealing the court judgment, which they lost on appeal. And then, having been backed into a corner, they went and talked to the union in a matter of a few weeks, worked out a deal, and got all the warehouses reopened and working. And, you know, Amazon's doing okay in Europe. So it is empirically true that Amazon can, in fact, work in a unionized environment and still be successful. To be clear, my statement earlier was dripping in sarcasm, which doesn't always come through, especially if you're reading the transcript of this episode rather than listening to it so you can pick out the sarcastic overtones of my voice. But yeah, it absolutely makes sense to me that if you are working in virtually any employee role, that organizing

makes an awful lot of sense. And I say this as a business owner. I don't view the fact that employees can come and talk to you on a collective basis to in any way be a negative direction for society to evolve in. Now, I understand that Amazon has its own position on this, and that's fine, to be very blunt. I don't know what it takes to run an 800,000-employee company. I don't ever expect or hope to find out. I feel that if I've

gotten to that point, something has gone egregiously wrong. Well, it sounds trite to say it's just too big, but I think it's just too big. And it's not just an Amazon problem. Amazon is a symptom of the problem. I think that it's not controversial to say that there is an overly high concentration of wealth and power in the economy. And that's specifically egregious and visible in big tech, where you have, you know, Google, Microsoft, Facebook, Apple. Who am I missing? The big five that dominate their sectors and behave in classically monopolistic ways. And I don't think it's controversial to say that that is dysfunctional, regardless of your ideology. Even if you are a firm believer in capitalism first, last, and always, capitalism doesn't work that well when you get overly high degrees of

monopolization and concentration. And I'm a little bit even more fundamentalist than that. I think that it just doesn't work that well when the companies get just too big. And there's some line in the sand when you get bigger than that. It just becomes hard to exhibit any aspects of humanity in dealing with either your employees or your customers. And I think objectively, you can look back at the last couple of decades and say, no, a lot of things about the economy just aren't working that well. The experience of dealing with businesses as a customer or as an employee is not getting better. It's getting worse. And we need to explore some, I think, fairly radical measures to sort that out. Again, I run a 10-person company. And believe it or not, we do have an ethics policy of who we will and will not do business with. And to be direct,

the way that I've structured that has been that in 15 years, when my now three-year-old winds up going to college, when I tell her where her college fund came from, I don't want to be ashamed by the answer. And that means different things to different people, and it's super squishy. But because I'm small, I have the privilege of being able to say that. I don't want there to be a logo on the Duckbill Group's website for customer wall that is the same logo you're going to see on bomb parts scattered around a village somewhere. It's just a baseline level of who I will and will not do business with. I think you don't get to be a certain size of company and still hang on to that.

I don't think there's any large-scale cloud provider in the trillion-dollar range of valuations where they've kept their souls. I don't think you can and still get to that level. Do you agree with that? Do you think that there's a way around that where there was a better path forward? Or is this just the trade-off you have to make for growing to that scale? Well, the world is complicated and nothing is white or black. I will say that in my view, AWS as an organization is run pretty ethically. You know, there were very few things we did that gave me personally heartburn. And I noticed that, you know, things that are stinky do get addressed, like there isn't change of policy around recognition. I would be much happier if the company walked away from a

whole bunch of policing and security sector businesses. But, you know, would just making it smaller address that? I don't know. I think that, in general, if you dislike the way that large organizations are behaving, the only effective solution to that is to change the rules. Because Amazon and its competitors, AWS and its competitors, by and large, sort of kind of do play by the rules. And as John Adams says, we still are mostly an empire of laws rather than of men. Although if he'd said that these days, it would be people. And so change the laws. I mean, politics is boring and icky and slow and tedious, but it's really, I think, the only hammer we have

to drive this nail. So, you know, if you don't like the fact that the company is selling to businesses that you think are awful, well, maybe we should change the rules to make those businesses not possible anymore. You know, if you don't like the way that ICE behaves, well, change the way that the legislative framework so that ICE is simply not allowed to do all the egregiously brutal things they're doing these days. I hate to say it. I mean, I would like to be able to talk to, say, Facebook the way I talk to my kids and say, now you play nice or no dessert. But, you know, that doesn't work. It has to be done, I think, through traditional legislative frameworks. No, and it comes down to a lot of power imbalance, too.

We saw recently AWS filed yet another non-compete lawsuit against a former employee, in this case, Brian Hall, who went from AWS to GCP. And on some level, it's, yeah, okay, it's a contract that you signed, and you should live by that contract. I'm sympathetic to that argument. The counterpoint is, though, is the way it is scoped is so hilariously overbroad. It's scoped to all of Amazon, which first, can you name a single industry that you could safely guarantee by the time you leave that company, Amazon will not be doing anything with, and it covers anything you may have had access to

by a strict reading, which is really the only way to read something like that before you sign it. And it's the only company I've seen that enforces these things this aggressively and at this scale. And I've got to say, more than almost anything else, things like that, where Amazon is, I guess, unkind and punching down, are the few points where I lose respect for them. Sure, they can ship a bad technical product and I can make fun of it, but we're all still friends at the end of the day.

You can't start punching down at your own employees, which is really what ties this all together, and still be someone I look at with an, oh, you look on my face. Well, yeah. And obviously the thing that drove me from the company, the firing of the activists, is you can describe it using similar terminology, punching down. Having said that, it isn't just Amazon. Microsoft has a history of doing this too, you know, bringing out the non-compete canon. And what do Amazon and Microsoft have in common? Well, they're both Washington state companies. Non-competes are non-enforceable in California.

And that's one of the big reasons why a large part of the technology industry is still in California. I mean, the birth of Silicon Valley was Intel. And Intel was an act of treachery when a bunch of people from Fairchild Semiconductor went off and decided to, you know, take what they'd learned and build a new company around it. And that's fine. You know, I've got no problem with that. And I think Washington State would do its citizens and its employers and its employees a service by striking down the ability to enforce non-compete clauses. I would absolutely hope so. I think that there is so much that goes wrong and could be better than it is. It's frustrating. I think that non-competes are one of the most aggravating things in the world because when it comes just so you're aware, there's a non-compete issue here. It's very easy for me to say, oh, I don't want to go into a legal battle with Amazon,

so I'm just going to go with my second choice candidate instead. It provides an incredible chilling effect. Now, personally, because I am who I am, my response is bring it. I will burn this place to the ground fighting you if I need to, just because I don't know when to pick and choose my battles. But it is the common case that it is incredibly chilling. And the cavalier attitude and the leaked memos that come out of Amazon about these firings

has just been bizarre to me. It's, what are they so afraid of? Well, yeah, at the end of the day, it was the ethical dimension of the whistleblower firings that drove me out. But you also have to point out that that was like really stupid, egregiously stupid. I thought maybe I'm missing something. As you say, I don't run an 800,000 person company either, but firing whistleblowers feels like I never heard of the Streisand effect. I'm putting up letters of fire

50 meters high in the sky saying, hey, there's something here we really don't want you to look at. I don't see how anybody with any maturity could think that doing that would have a good result, creating a climate of fear in response to activism. And the people who are activists had no thought of gain for themselves. They weren't trying to make money or advance their careers or anything like that. They were doing something that I thought was, you know, wholly admirable. I mean, they could have done something like say, okay, you guys are upset about that. Great. You got a new job serving on the task force to make the warehouses safe and prove we've done it. There was a certain amount of loss of respect in my mind for leadership around that. I would agree. I think that there's

so much that Amazon could do in this space and demonstrate real leadership. And for whatever reason, it's choosing not to. I mean, I consider myself an Amazon fan. I think that they do a lot of good things. And I think a lot of what they do is admirable. But I'm believed when I say those things, because I also call them out when they do things that are awful. And this is one of those awful things. In terms of all the places I've worked, and I'm an old guy, I've been doing this for 40 plus years. AWS was by far the best managed place I ever worked. And also the household density was really low compared to anywhere I've worked. And, you know, there would be whole weeks at a time when I never got mad at anybody, which, you know, as you know, in most jobs is not the common state

of affairs. So yeah, they're doing a lot of things right. And that's why, you know, this kind of stuff stands out in such stark contrast. One thing that, especially in the wake of the non-compete lawsuit, which I've been getting more traffic on than I have the warehouse firings, I've gotten a bunch of outreach both from people considering working at Amazon and people who do work at Amazon expressing concerns about their non-competes and what should they do. And I have my own thoughts on it, but where do you stand? So I got the same thing. You know, when I quit, I got a lot of outreach from Amazon employees saying, oh, gosh, now you quitting makes me feel bad. Can I go on working here?

And to be fair, I had to point out that I am elderly, nearing retirement age, and financially secure. So this was a lot easier for me than it would be for a lot of other people. Oh, we are both dripping in privilege in this context. Absolutely. And I think that at the end of the day, the problem isn't really Amazon. The problem is the Reagan-Thatcher consensus, 21st century capitalism, and the imbalance of power and wealth in modern society. And, you know, if you're going to boycott anything remotely related to that, you're going to have a hard time making a living. And there are lots of worse places to work. Amazon is by a wide margin not the worst operator out there. So I've actually been advising people, no, don't do what I did, unless you're

absolutely unable to sleep at night and got a financial plan. But that's all I could say. In what you might be forgiven for mistaking for a blast from the past, today I want to talk about New Relic. They seem to be a relatively legacy monitoring company, and I would have agreed with that assessment up until relatively recently. But they did something a little out there. They reworked everything. They went open source, they made it so you can monitor your whole stack in one place, and most notably from my perspective, they simplified their pricing into something that is much more affordable for almost everyone. There's even a free tier, with one user and 100 gigs per month, totally free.

Check it out at newrelic.com. So, let's pivot a little bit towards the thing that, again, you were one of the best in the world at, which is sort of funny, because originally you were one of less than 20 people who had reached the pinnacle of the individual contributor engineering ladder at AWS. Then you resigned in protest, and suddenly, oh, that guy had terrible judgment. We never liked him anyway. It was probably an aberration, was the entire thrust of the messaging around you. That's some impressive level backpedaling. But let's not kid ourselves here. You can be as modest as you want, but I'm not going to be. You're one of the best engineers in the world. That is why you were in the role you were in. It's why you've done the things that

you've done. And I want to talk to you a little bit about technology. So from a high level, let's talk about cloud stuff, for example. I think most notably in a combination technical slash political venue to transition us, let's talk about fear of lock-in. A lot of people are scared of being all-in on a provider that reduces their choices in the future. Where do you stand on it? I think it's a really important issue and not one that's simple or straightforward. You know, AWS is a $40 billion business these days, which is to say bigger than a lot of the customers. And in the IT world, historically, there has been a trend to platform lock-in. For example, you know, if you are a large company, you no longer get to decide what your budget for desktop technology is.

You pay whatever Microsoft says you're going to pay. You have no bargaining power. Similarly, on your database spend, you pay whatever Oracle says you're going to pay. Once again, no bargaining power. Similarly, on your database spend, you pay whatever Oracle says you're going to pay. Once again, no bargaining power. And you just know that in a lot of industries that have some decades of experience in IT, the leadership, the CEO, the board even is going to the CIO and saying, now, don't you come back here in a couple of years and tell me that you've lost control of our cloud spend. And okay, that's a very reasonable concern to have. And I can't see

how you could argue against that point of view. Well, you can because in practice, it turns out to be really difficult to be cloud agnostic. I mean, if all you're going to do is rent CPUs and storage and operate databases, fine, you can do that. But even at that level, you find that there's a bunch of really annoying semantic gaps between the way the major cloud vendors think about what you mean by instance and what you mean by object and so on and so forth. So you can, in fact, build out your technology stack in a cloud-independent way, but you're going to pay a lot more and you're going to go a lot slower. So I think the choice for a startup is a no-brainer. A startup should pick one of the cloud vendors, jump into bed, aggressively make use of all the hot stuff they're shipping in that particular year. And that way, they'll get the most velocity. They'll spend less.

They'll move faster. They'll have a better experience. You know, startups are short of time and short of money. So that's what they should do. Now, if you're a bank or a pump manufacturer or a book distributor or something like that. Can I honestly say that, ah, blow off that stupid lock-in concern, just go jump into bed with a cloud vendor? I don't know. I think that is a more nuanced discussion and one that you have to have. You know, how much are you willing to pay? Now, a lot of people say, well, you know, the cloud lock-in

isn't really at the level of the APIs. It's at the level of data gravity. Once you've got several petabytes of data sitting in Google Cloud or AWS, well, you're just not going to move out because it's too hard. I'm not sure that's true. I think you can, in fact, do that. And in fact, these days, you can go on a case-by-case basis. I mean, are you willing to use a proprietary database like Dynamo or Bigtable or something like that?

Or Route 53? You know, in the knowledge that then you really pretty well are going to have a hard time getting off that vendor. Maybe not. On the other hand, if you are going to decide I'm going to standardize on Postgres, am I willing to use hosted Postgres and use somebody else's control plane to create databases? Well, that's a much less severe degree

of lock-in. And this is a very complicated discussion. And, you know, I would be willing to have an opinion after a discussion with any individual business that is getting into the cloud. But would I lay down a dictum saying, oh, fear of lock-in is dumb or always resist lock-in? I would not say either of those things. It really is a case-by-case situation. Very much so. My railing about multi-cloud is always from a best practices position.

I think that as a general best practice, pick a provider, I don't care which one, go all in. Now, there's a litany of exceptions to that where it does not make sense, where you cannot do that specific thing, and that's fine. My argument has been against hearing this from a conference stage from some second- or third-rate vendor saying, oh, multi-cloud is absolutely what you want to be doing

and accepting it blindly. That has been my issue. Fair enough. And I think this is a super hot area of technology, and I think there's lots of opportunity for vendors to facilitate the process of being multi-cloud where necessary. I mean, HashiCorp is now a unicorn, right? And we should all be paying pretty close attention to what they're doing because I think their existence shows that there's a hot button out there that they've pressed. And a lot of people care about it a lot.

You know, at the moment, I can only speak to AWS because that's where I worked. But, you know, AWS is a really pretty safe place to jump into bed with at the moment because people like Andy Jassy and Charlie Bell and so on are just really customer obsessed. They spend a huge proportion of their time trying to figure out what's making customers unhappy and just fixing it. And they're super aggressive about cutting prices and things like that. And that's great. So that's the kind of vendor I want to have as a customer. But 10 years from now when I'm retired and Charlie's retired and so on and so forth,

is it still going to be the same? Or is it going to be a bunch of ex-private equity MBAs running the thing and trying to figure out how they can extract the maximum rent? It's a thing that's reasonable to worry about. Always a disturbing concept. One thing you said a minute ago about HashiCorp being aimed at the multi-cloud story, when I had Mitchell Hashimoto on this show previously, we talked about this.

His idea was never that Terraform should be used in this multi-cloud way. It's not about workload portability, it's about workflow portability. And having seen that play out at multiple customers in the past, it's always seemed to align in that particular direction, where, okay, you're always going to have to do a lot of rework to get something that works on one provider in Terraform to work somewhere else. But the workflow, how you interact with your code base, how you get things from your head into production, that's what you can wind up porting between providers. So I want to be very clear on that,

oh my stars, will I get letters? But let's talk about something else that I think has areas of lock-in commonality. Let's talk a bit about, for example, the idea of event-driven architectures. It feels to me like the event model is one of the least portable things you could have, depending upon what it is you're doing. Am I wrong on that? It depends. I mean, each of the cloud vendors has their own event, pub-sub, routing, invocation frameworks. And it's shocking how similar what they do is. If you care about all the different semantics that you can have around messaging and eventing services, there's not that many. And you can make up a laundry list of things that you care about. And you say, my requirements are A, B, and C. And yeah, I can do that at Google Cloud or AWS or Azure.

And given that, it's kind of annoying that the things are, relatively speaking, incompatible. You know, there's a lot of people I talk to who are using things like Apache Camel, which is this big, hairy, complicated API. But what it does is provides an abstracted send message, receive message thing. And I'd see people using that and then, you know, wiring it up to SQS or RabbitMQ or something like

that. And then the person who's writing the Java production code didn't have to care about their messaging framework. So there are some tools for making it less proprietary. But having said that, that's what you kind of expect because the event-driven stuff is sort of the new shiny at the moment with Lambda and EventBridge and that kind of stuff. That is where we're pushing back the boundaries and figuring out new ways to build applications. So you go explore the new territory first. Then after you figure out what you use, you write the rules for it. After you figure out what works, you write the rules for it. And we're still in the stage of figuring out what works. And the initial signs are that, by and large, event-driven software works pretty well and is highly applicable in a lot of situations.

But do we have a set of commonly agreed-on best practices and standards yet? No. That's part of the issue, is that as standards fail to emerge and they start turning into these, I guess, de facto standards rather than the ones that are imposed, there's an awful lot of companies jockeying, for whatever it's worth, to start being the voice that determines what those standards are going to look like. And increasingly, it seems that those standards are being driven less by the community and more by whoever has market share and is the first mover in the space. Does that align with your experience? Well, you know, this is the classic thing is you get a market leader who becomes de facto incumbent and then all the market trailers get together and form a standards organization.

And whereas they would never come out and say that, one of the goals of the standards organization is to slow down the incumbent to give the up-and-coming smaller parties a chance to compete on a slightly different playing field. And that's okay. That's fine. I've been in both roles now as the market incumbent and as the upstart trying to get a standard put in place. It's part of the organic flow of how we do things in this technology. In the space at one point, SQL was seen as a highly proprietary technology, and these days it's regarded as a pretty level playing field. I'm not smart enough to predict how this is going to shake out, but I think I put my stake in what I said a couple minutes ago, which is that you can either use the cutting-edge, sharp new stuff that does magic,

or you can play on a level well understood standards driven playing field, but you really can't do both. Yeah, it definitely seems that for better or worse, there's going to be a shakedown at some point. It feels to me, this is aligned in a lot of ways, with the idea of complexity gets built on top of complexity, and it almost looks like a sawtooth graph, because eventually people look at it and say, this is nuts, and it collapses down into something that a human being is a hope of understanding. Kubernetes right now feels like it is incredibly complicated and overwrought. In a few years, it won't be, because something's going to come along with, huh, okay, maybe a team of engineers who cost a quarter million bucks apiece. It is not something every company is

going to want to run its application architecture, and there's going to be some evolution there. I guess cynically, I tend to view things like that, let companies cosplay as cloud service providers themselves when they don't work for one. Yeah, I don't want to diss Kubernetes too much because it does some- Oh, I do. I don't actually use it either,

but I don't want to diss it too much. It does some remarkably clever things that do make operators' lives easier once you've got everything set up. But it fails one important test in my mind. I think that technology should make easy things easy and difficult things possible. And Kubernetes clearly fails the first half of that. It's too hard to understand and bootstrap. And it costs too much, as you just said, in terms of engineering to keep it going. You know, we've seen this movie lots of times before. There was ISO networking and

seven-level stack and so on. And then the internet geeks came along with TCP IP, which only did 20% as much, but it turned out to be the right 20%. And it swept the rest of that stuff away. Same thing happened with the web. You know, there were increasingly elaborate, complicated application frameworks, starting with Visual Basic and Adobe's offering and Novell's offering and so on and so forth. And then the web came along with a vastly simpler and impoverished user interface paradigm and swept all that crap away because the 15 things it did really well turned out to be the ones that matter. And I would be deeply unsurprised if that didn't happen to Kubernetes. Kubernetes does a whole bunch of stuff

and all of it's not equally important. What I frankly expect to see is something come along that does the things you actually really care about and you can read about it Monday morning and have an app up Tuesday afternoon, which is incredibly important. And if you look at the adoption of each real game-changing

technology, it's almost always had the characteristic that it's easier than what came before. And this profession is hard enough as it is without making things unnecessarily complicated. So at a high level, now we've spent some time looking back. As you said, you've been doing this for 40 years. What technology trends are we seeing that interest you the most? What's the future look like? At the moment? Well, it's one that we already talked about a bit, which is, are there interesting tools that are going to be truly useful

in enabling you to avoid vendor lock-in without grossly increasing the cost and decreasing the velocity. You know, we can all be a little bit cynical about that now, but these things have happened. And the most classic example of that was the World Wide Web when it came along. Before the web came along,

there were all these competing technology stacks and you had to figure out whether you were going to be on the Microsoft stack or the Sun stack or the IBM stack. And then the web came along and it was a platform without a vendor. That was a profoundly important thing. It was the first time we'd had a platform without a vendor. And is there going to be a public cloud platform without a vendor out there?

Any steps towards that would be super interesting in my mind. So that's one. Another one, moving in a completely, totally different direction, is I'm all excited about augmented reality, AR. How many years has it been since a new mobile device changed anybody's life? Come on, they're just not interesting anymore. Or how long has it been since a new programming framework has really rocked the industry? Well, serverless in 2014, maybe, but they're few and far between. So AR, I think, has the ability to really dramatically enrich the life of humanity at large. And right at the moment, the technology isn't good enough. What you would like to have is technology so you can just hold your mobile phone up and look

through it and see a decorated version of the world. You know, you're walking in a park at night and you can see, you know, fiery dragons climbing up the trees. You walk into a superstore and say, where's the deodorant? You know, a big arrow appears in front of you pointing you towards it. There's just so many applications. Technology isn't good enough yet, but when the technology does become good enough, it's going to be a huge new area of human creativity and innovation and new business sectors that we can't even begin to think about. There's little birds here and there telling me that Apple's got some great stuff coming along. I'll believe it when I see it. But the problems we're trying to solve are basically

computer vision problems and ML problems and so on, and they'll get solved all right. So that one's got me all excited. Another area that I'm super interested in these days is public sector procurement. In my career, I got twice mixed up in large government procurements of technology, and oh my god, what a clusterfuck. It's done very, very badly, and I think this is sort of generally universally true across the governments of the world. And the blue suit consulting companies that specialize in government technology procurements are just not working properly. And we have endless disasters. So I'm aware of some efforts to try

and bring sanity to that process. And that's super interesting to me as well. So we could dive down any of those rabbit holes. I think that the problem with rabbit holes is that in some cases, there's a rabbit at the bottom. And be careful, that rabbit's dynamite, pointy teeth and whatnot. One of the problems that I see is, as I look across the landscape, it seems that a tremendous constituency is treating the cloud as someone else's data center, where they run VMs, steady state, there's no auto-scaling,

and it seems like the primary problem that they're trying to solve is that they suck at running data centers themselves. Some people give up halfway and call it hybrid, and it feels on some level to me like they are improving their data center environment at the cost of leveraging the cloud for what it's really capable of doing. And I find that depressing on the one hand, but on the other, I understand that they have what we'd like to disparagingly call legacy workloads, which means they make money.

How do you feel about that? Well, to a certain extent, that's simply an inevitable result of arithmetic. So, you know, AWS is what, $40 billion a year now, and maybe the whole rest of the cloud industry together is that again? I don't know, something like that. And so $80 billion sounds like a lot of money. Last time I checked, the enterprise IT spend globally is like $2.7 trillion per annum. So the whole public cloud infrastructure is a small sliver of the business. Oh yeah, everyone talks about AWS being a $40 billion annual run rate, but Dell's revenue was $60 billion in 2012. There's a tremendous,

and that's one company that basically was selling hardware and some services back then. We've only seen the tip of the iceberg. And for that very reason, as you say, running your own data center sucks. The cost, admin, velocity, security advantages of moving to the public cloud are, I think everybody agrees now, are pretty high. And so I think that whether we like it or not, the revenue growth and core business of the public cloud is basically going to be doing exactly what you said, getting the people the hell out of their data centers and improving their security posture, including their availability posture, that kind of stuff, just by moving existing stuff to the public cloud, your classic lift and shift. So,

okay, that's fine. There's nothing wrong at all with that. It's a little bit boring, but it's an important big money, big deal. So let's look at what comes next. If you look at your typical large enterprise, they'll have an inventory of apps. A couple of hundred is typical apps that their business depends on. In any given year, they maybe do work on a single digit number of apps, you know, five or 10 kind of thing. And the rest of them just tick along. And so what's actually interesting is the new stuff that gets built. And so for new apps that are getting built, are they being just done on classic monolithic vert architectures, you know, web server in front and database engines behind?

Well, yeah, some, and there's nothing wrong with that, especially for smaller scale departmental apps. But anything that's being built that's new and ambitious in scale, I think the technology uptake of the new stuff that I was working on is pretty good, actually. And incrementally, as we do these five apps this year and another five apps next year, eventually the present and future start to outweigh the past. But at the moment, they don't. The past is winning. We're moving the past from data centers into the cloud. And realistically, that's where the revenue is coming from right now in its largest bulk. And that's okay. I don't see if there's anything wrong with that. But obviously,

those of us with an eye to the future are really, really worried about making event-driven and serverless and container and all that stuff technology irresistibly attractive for the construction of the new apps. The idea of throwing away the old in favor of the new is one that feels pervasive and is something that everyone loves to talk about, but it feeds into a common deception that I'm seeing, where everyone seems to believe that just after this next sprint is over, then, then we're suddenly going to start making good decisions instead of the bad ones we've made to date. And, oh, I'm as guilty of this as anyone. This is not me casting shade.

I wish that, oh, as soon as I get a little chance to breathe, I'm going to go back and fix all of my crappy, broken code. Spoiler, he would not. But it's the hope that springs eternal. And you see that companies never seem to quite outgrow this. Well, as a former principal engineer and distinction engineer at AWS, one of the things that principal engineers spend a lot of their time doing is stopping people from doing that.

You know, respect the past is a core engineering principle. And you may hate the existing code base that's running your business-critical applications or your business-critical AWS service that was launched prior to 2010, but it works. And part of the problem is that a lot of developers hate reading other people's code and don't want to learn how it actually works. They just want to rewrite it themselves. And once you get to be in a position where you've done this for 20 or 30 years, you realize that, you know, that isn't as easy as

you think it is. And embodied in that crufty old code base is a huge inventory of decisions that were made to meet particular weird situations and corner cases and achieve non-obvious behaviors that shouldn't have to be correct. And there's no way to know that by looking at it. Now, things are getting better. There's this great book called Dealing with Legacy Code, and it defines legacy code, interestingly, nothing to do with age or anything like that, as code that lacks unit tests. And since unit tests became pervasive, you know, sometime between 2010 and 2020, things have gotten better because in many cases, the unit tests realistically represent the contract between, you know, the code base and the outside world.

And they make it much more thinkable to replace the code base with something that's more modern, runs faster, runs cheaper, runs cleaner, emits less carbon. And if it still passes the unit test, hey, it's probably going to work. So yeah, respect the past. Don't flippantly decide that you're just going to rewrite the system

because you're smarter than the people who wrote it because you're not. But on the other hand, we should be super dogmatic and make sure that we never do anything that isn't fully and completely unit tested to enable our successors to be a little bit more courageous in replacing the past where that's the right thing to do. I think that's a really good takeaway here.

If people want to learn more about what you're up to these days, how you're thinking about things, where can they find you? Now that you're an unemployed bum, there's no corporate blog you can drive people to. Where do they find Tim Bray? Just type my name into Google. There's lots of stuff that comes up. I talk to the world on Twitter and on my blog, and I enjoy doing that. And I'm not going to stop doing that as long as I can lift my hands up to the keyboard. I published a PR FAQ for how to spin off AWS from Amazon, which is something that should obviously be done. I also published an article about how Google should be broken up, which is another thing that should obviously be done. And I'm not as funny as you, Corey, but I do try to be kind

of entertaining. Oh, I was a big fan. I took inspiration from that to publish my own fake PR FAQ about Elastic Beanstalker. I love the format. I think it's an underappreciated comedic medium. It's definitely an underappreciated medium. You know, I've had a few advisory consulting gigs since I left, and a lot of people want to learn about what's this, you know, six-pager thing that Amazon talks about, and how does that really work? And it does really work, and, you know, I think I've given them value by talking to them about that. But, you know, I think the comedic six-pager is an underappreciated opportunity, so you keep running with that. That's a good way, I think, of framing it and a decent enough place

to leave it. Tim, thank you so much for taking the time to speak with me. I know that your days are full of unemployment these days. That's right. But again, your time is always appreciated. Oh, well, thanks. It's been delightful and entertaining. And I love talking about cloud stuff. So give me a call anytime. Don't offer if you're not serious. Tim Bray, former VP and Distinguished Engineer at AWS, now unemployed bum. I am cloud economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on Apple Podcasts, whereas if you've hated this podcast, please leave a five-star review on Apple Podcasts. Whereas if you've hated this podcast, please leave a five-star review on Apple Podcasts and a comment telling me what's going to get you to leave your employer on ethical grounds. This has been this week's episode of Screaming in the Cloud. You can also

find more Corey at screaminginthecloud.com or wherever Fine Snark is sold.